- Email: [email protected]
A Focused Breast Skills Workshop Improves the Clinical Skills of Medical Students
Journal of Surgical Research 106, 303–307 (2002) doi:10.1006/jsre.2002.6472
A Focused Breast Skills Workshop Improves the Clinical Skills of Medical Students Glenn T. Ault, M.D., Maura Sullivan, R.N., Jeanine Chalabian, M.D., and Kristin A. Skinner, M.D. Department of Surgery, Keck School of Medicine, University of Southern California, Los Angeles, California 90089-9034 Submitted for publication November 20, 2001; published online July 24, 2002
Purpose. The aim of this study was to determine the effectiveness of a focused breast skills workshop for teaching clinical skills to third-year medical students. Methodology. One hundred twenty-four third-year medical students involved in the surgical core clerkship were randomly assigned to two groups. Group 1 (n ⴝ 67) participated in a 2-h focused breast skills workshop. Group 2 (n ⴝ 57) received “traditional” ambulatory teaching for a period of 4 h in the breast clinic. The focused workshop consisted of a series of encounters concentrating on mammogram and ultrasound interpretation, physical examination skills, lump detection, and workup of a breast mass. Both groups received a didactic core curriculum lecture from surgical faculty. All students completed a satisfaction rating scale and a subset of students completed a pre- and postencounter self-efficacy rating scale on several aspects of breast skills. Student’s t test was used to compare the groups in the areas of clinical skills as evidenced by performance on the breastspecific items on the end of the clerkship Objective Structure Clinical Examination and student satisfaction as evidenced by their response on a satisfaction rating scale. ANCOVA (controlling for preencounter self-efficacy rating) was used to compare the change scores between pre- and postencounter self-efficacy ratings. Results. Students in Group 1 performed significantly higher than the students in Group 2 in the areas of clinical examination skills (t ⴝ ⴚ2.99, P < 0.05); in sensitivity (t ⴝ ⴚ5.82, P < 0.05) and specificity (t ⴝ ⴚ7.27, P < 0.05) in the examination of breast models; and with their satisfaction with the encounter (t ⴝ 10.72, P < 0.05). Students in Group 1 also demonstrated a higher level of confidence in their breast skills at the
Presented at the Annual Meeting of the Association for Academic Surgery, Milwaukee, Wisconsin, November 15–17, 2001.
end of the clerkship than students in Group 2 (F ⴝ 6.22, P < 0.05). Conclusions. The focused breast skills workshop is more effective than the traditional ambulatory setting for teaching clinical breast examination skills. This setting also demonstrated the development of higher confidence in breast skills than the traditional ambulatory setting. © 2002 Elsevier Science (USA) Key Words: physical examination skills; surgical education; medical students; breast disease.
INTRODUCTION
Clinical education in medicine has continued to shift out of the hospital ward and into the ambulatory care clinic, where it has been estimated that 95% of doctor– patient relationships occur [1]. Champions of education in the ambulatory setting state several arguments for the important and unique learning opportunities offered in outpatient clinics including but not limited to continuity of care, practice of health promotion and disease prevention, and development of physician/ patient communication skills [2–7]. A recent review of the literature revealed that education in ambulatory care clinics is characterized by variability, unpredictability, immediacy, and lack of continuity [8]. Furthermore, the review of the studies found that learners often see a narrow range of patient problems in a single clinic and experience limited continuity of care. Few cases are discussed with attending physicians and little observation of the student occurs during their encounter. When case discussions occur, they are generally short in duration, involve little teaching, and provide virtually no feedback [8]. Increased costs and decreased efficiency of the faculty and clinic time are also concerns regarding the logistics of teaching in the ambulatory setting [9]. Despite these imperfections,
303
0022-4804/02 $35.00 © 2002 Elsevier Science (USA) All rights reserved.
304
JOURNAL OF SURGICAL RESEARCH: VOL. 106, NO. 2, AUGUST 2002
the push to provide experience for medical students in the ambulatory setting continues. The Panel on the General Professional Education of the Physician reported that fewer than 5% of current patient encounters result in hospitalization [10], which places significant emphasis on the ambulatory setting to teach medical students essential physical examination skills. The increasing use of performance-based tests such as the Objective Structured Clinical Examination (OSCE) has shown that both medical students and surgical residents possess disturbing deficiencies in clinical skills [11–16]. Breast disease and breast cancer are areas in which the performance of students and residents has proved to be deficient, as measured during interactions with standardized patients [11, 12, 17, 18]. In response to the documented deficiencies of clinical examination skills in breast disease as well as the irregularities of education found in the ambulatory setting, we inaugurated a focused breast skills workshop during the required surgical clerkship at the University of Southern California. Building on the success of structured clinical models previously reported in the literature [19, 20], our multidimensional focused clinical workshop concentrated on physical examination skills, mammogram and ultrasound interpretation, lump detection, and the workup of a breast mass. The intent of our study was to determine the effectiveness of this breast skills workshop in teaching medical students physical examination and diagnostic skills for breast disorders in comparison to that of the traditional ambulatory setting. METHODS One hundred thirty medical students completing their required 6-week third-year surgical clerkship during the academic year 1997– 1998 at the University of Southern California participated in this study. This prospective randomized study assigned students to either the intervention group (N ⫽ 62) or the control group (N ⫽ 68) according to a table of randomized numbers. A power of analysis was performed to a power of 90 and ␣ of 0.5 which defined an appropriate sample size of 56 students. Students in the intervention group participated in the 2 1/2 h breast skills workshop while the students in the control group participated in a 4-h traditional ambulatory experience in the breast clinic of our teaching hospital. The focused breast skills workshop consisted of three separate learning modules. Prior to rotating through the modules, the students received a brief orientation to the workshop as well as some fundamentals in radiographic diagnosis of breast disease. They also observed a faculty member model the breast examination emphasizing key examination maneuvers. The first module required students to review actual screening mammograms from patients with both benign and malignant breast disorders and to work through the diagnostic workup of those patients. This included interpretation of specialized mammographic views as well as ultrasound. During this module, students interacted with an expert faculty member during case discussion and received feedback regarding their radiological interpretations. The second module was composed of a standardized patient (SP) station in which the students were given the opportunity to practice their examina-
tion skills and to receive feedback from the SP. The student was afforded the opportunity to ask specific questions at the end of the encounter and receive instruction in deficient skill areas. The surgical nurse educator was available at this module to provide assistance and to answer any questions that arose. The last module consisted of improving lump detection skills utilizing latex breast models (Mammacare). This module, led by the surgical education fellow, had specific instruction regarding the workup of both a palpable and a nonpalpable breast mass. Attention was given to diagnostic techniques, surgical biopsy options, and definitive surgical care. In all modules, there was ample opportunity for students to ask questions and receive feedback regarding their examination or diagnostic skills. Students in the control group participated in the outpatient breast clinic for a period of 4 h. The outpatient clinic in our county hospital setting is primarily a diagnostic clinic in which the referred patients either are symptomatic or present with abnormal physical examination or mammogram findings. A surgical oncology faculty member or the residents rotating on the oncology service supervised them during their clinic experience. In addition to whichever arm they were randomized to, all students received the same didactic core curriculum lecture on breast disease. All students also had access to a breast disease self-study module consisting of 23 slides with comments and explanations, which was available for their use in the student learning center. After the completion of either the clinic or the workshop, the participants completed a one-item satisfaction rating scale (scale: 1, strongly disagree, to 4, strongly agree) evaluating the educational value of their experience. In order to determine any perceived change in self-efficacy, a subset of students (N ⫽ 110) completed a five-item pre- and postclerkship self-efficacy rating scale (␣ ⫽ 0.797). The rating scale asked the students to self-rate their skills in the areas of physical examination, mammogram and ultrasound interpretation, lump detection, and the ability to teach breast self-examination, as well as problem-solving and management issues (scale of 1, poor, to 5, outstanding). At the end of the clerkship, each student participated in the surgery OSCE that included one breast standardized patient station and one breast paper case (␣ ⫽ 0.707). The standardized patient station tested both clinical skills and problem-solving abilities, while each paper case tested only problem-solving abilities. To assess clinical skills, only the physical examination portion of the checklist completed by the SP was utilized. The SP was blinded to which arm the students had participated in. To ascertain the student’s problem-solving skills, a combined score from both the SP station and the paper case was determined. There was also a lump detection station in the OSCE where the students were asked to palpate seven latex breast models (Mammacare) and document their findings. Sensitivity and specificity of their lump detection was determined from this OSCE station. The lump detection models used for the OSCE were different from the ones utilized in the workshop. In addition, each student also took a 75-item multiple-choice examination (MCQ) at the completion of the clerkship. Six items on this examination (␣ ⫽ 0.732) tested cognitive knowledge of breast disease. These examinations (OSCE and MCQ) occurred at a period of 1 to 5 weeks after either the clinic or the workshop experience. The scores obtained from the problem-solving portion of the OSCE and the MCQ were combined into one score. In order to assess retention of clinical skills, we looked at the student’s performance on the Clinical Performance Examination (CPX) which is taken at the completion of the third year of the curriculum. The CPX can occur anywhere from 1 to 7 months after the completion of the clerkship. We assessed the performance of the students on the one breast case included in the CPX. In order to assess the significance of differences between the groups in the areas of clinical skills; problem solving and knowledge acquistion; retention of clinical skills; sensitivity and specificity of lump detection; and student satisfaction with the encounter, the independent samples t test was utilized to compare the means. To
305
AULT ET AL.: FOCUSED BREAST SKILLS WORKSHOP
TABLE 1 Mean Assessment and Standard Deviation of Performances on End of Clerkship Measures
Physical examination skills (scale 0–10) Sensitivity of lump detection (scale 0–1) Specificity of lump detection (scale 0–1) Knowledge acquisition (scale 0–10) Student satisfaction (scale 0–4) Self-efficacy composite score change (scale 0–20)
Focused workshop
Ambulatory clinic
8.33 ⫾ 1.66 0.77 ⫾ 0.12 0.79 ⫾ 0.12 7.74 ⫾ 2.7 3.89 ⫾ 0.32 6.55 ⫾ 5.52
7.52 ⫾ 1.8 0.64 ⫾ 0.15 0.62 ⫾ 0.15 7.06 ⫾ 2.3 2.44 ⫾ 1.0 4.48 ⫾ 2.89
determine the difference between the pre- and post-self-efficacy ratings, we calculated a composite change score (subtracting pre from post ratings) on all students. ANCOVA, controlling for preencounter self-efficacy ratings, was used to compare the difference in change scores between groups.
RESULTS
Students who participated in the workshop performed significantly higher than students who attended the traditional ambulatory experience in areas of clinical examination skills (t ⫽ ⫺2.832, P ⫽ 0.005) as well as in sensitivity (t ⫽ ⫺5.714, P ⬍ 0.001) and specificity (t ⫽ ⫺7.78, P ⬍ 0.001) of lump detection. Over 91% of the students who participated in the workshop rated it as an outstanding educational experience which led to a significant difference with student satisfaction with the workshop as an educational experience (t ⫽ 10.718, P ⬍ 0.001). There was found to be no statistical significance in comparing the experimental group and control group outcome on the surgery clerkship final examination (t ⫽ 1.06, P ⫽ 0.290). The experimental group also performed higher than students in the control group in the area of knowledge acquisition (as measured by problem solving on the OSCE and answers breast questions on MCQ) and in retention of skills (as measured on the CPX), but the difference between the groups was found not to be significant (t ⫽ 1.50, P ⫽ 0.136 and t ⫽ ⫺1.49, P ⫽ 0.137, respectively) (Table 1). The students who partook in the workshop demonstrated significantly higher self-efficacy rating change in their breast examination skills at the end of the clerkship compared to the control group as determined by the ANCOVA analysis (F ⫽ 8.92, P ⫽ 0.003). Specifically, we found significant differences in examination skills (F ⫽ 4.00, P ⬍ 0.05), mammogram and ultrasound interpretation skills (F ⫽ 37.81, P ⬍ 0.001), and knowledge base (F ⫽ 4.23, P ⬍ 0.05). All reported P values are two-tailed. The cost of the standardized clinic was approximately $39.00 per student. DISCUSSION
As students enter their years of clinical training, they begin to truly develop and master the examina-
P P P P P P P
⫽ ⬍ ⬍ ⫽ ⬍ ⫽
0.005 0.001 0.001 0.136 0.001 0.003
tion skills essential to the practice of medicine. During these years they interact with a variety of faculty and residents to learn more focused examination techniques as well as refine basic physical examination skills. A majority of this interaction occurs in the ambulatory setting, which can be hampered by many barriers [3]. This, in combination with several reports regarding deficiencies of physical examination skills of both medical students and residents, is a cause of concern [11–18]. The focused breast skills workshop was created to provide students with a structured learning experience for clinical competence specifically tailored to breast examination and diagnostic skills. Our study demonstrated that this format is more effective in teaching clinical breast skills than the traditional ambulatory setting specifically in the areas of clinical examination skills, sensitivity and specificity of lump detection, and overall improvement of student’s self-efficacy. Students also demonstrated significantly higher satisfaction with this format as an educational experience. One could argue that the students who participated in the workshop did not demonstrate a significant difference in the acquisition of cognitive knowledge, but this was not expected as the workshop itself only focused on clinical skill acquisition and the fact that all students had received the same didactic core curriculum lecture and had access to the self-study module. One could also argue that the students who participated in the workshop performed better in lump detection because they were exposed to the models prior to the examination whereas the control group was not. Granted, breast models are an imperfect surrogate for lump detection, but it is the only methodology we have to reliably assess lump detection skills. We also did not demonstrate significance between the two groups in the area of retention of clinical skills as determined by the breast case on the CPX examination. Students at the University of Southern California are required to pass this OSCE-style examination at the completion of their third year. Successful completion is required for graduation. Perhaps one of the reasons that no significant difference was observed was because of the way that the CPX is scored. We were
306
JOURNAL OF SURGICAL RESEARCH: VOL. 106, NO. 2, AUGUST 2002
unable to tease out clinical examination performance from the entire case score. Therefore, the CPX score included history taking and patient–physician interaction scores as well as the clinical examination score. Since we were only specifically interested in clinical examination scores, our inability to exclude the other criteria may have clouded our results. The variability in the length of time between the clerkship and the administration of the CPX may have confounded the retention results as well. We believe that the workshop was more effective in teaching clinical skills because it provided students with a practical level of exposure to the three modules and sufficient time was spent in practice and feedback. One of the problems with teaching in the ambulatory care setting is that teaching is often limited and sometimes rushed secondary to the high volume of patients that must be seen in a short period of time. Furthermore, the workshop provided all participants with a consistent exposure to the areas of radiological diagnosis, examination skills, and lump detection. In this study, students who participated in the workshop demonstrated higher self-efficacy ratings in their breast skills than students who did not. Self-efficacy is an individual’s evaluation that he or she has specific performance capabilities on a particular type of task [21]. We felt that this was an important finding because perceived efficacy can influence motivation. Students with a high sense of efficacy are more likely to choose difficult tasks, expend greater effort, persist longer, apply appropriate problem-solving strategies on tasks, and have less fear and anxiety regarding tasks than students with a low sense of efficacy for a task [22]. One could argue that perceived efficacy does not translate into better clinical skills; however, the students in our study did perform significantly better in the area of clinical skills. We believe that the workshop helped students achieve a higher level of selfefficacy because of the focused instruction; the abundant immediate feedback provided to students on their clinical skills; and the provision of an environment that was conducive to learning. The workshop also has an advantage in that it specifically focused instruction on the clinical examination aspects of breast disease. It could offer the opportunity for multidisciplinary instruction. Another advantage of the workshop is its cost. Instruction in the ambulatory setting is usually one on one, translating into mentors either seeing fewer patients or working a longer day [23], which could be one of the explanations for the higher cost of educating students in the ambulatory setting. Considering the costs of educating medical students in the traditional ambulatory setting, which one study concluded could range from $100 to $327 per student per day [23], the cost of our intervention was much less. The workshop
utilized faculty time efficiently and optimized the student’s exposure to them as well. The Focused Breast Skills Workshop demonstrated several advantages over the traditional ambulatory setting in the areas of clinical skill acquisition and higher ratings of self-efficacy by the participants. This intervention standardized learning and was found to be an effective educational experience. It optimized exposure to expert faculty without excessive demands on their time. This workshop also overcame some of the barriers cited in the literature as detrimental to learning in the traditional ambulatory setting. While we are not advocating that the workshop take the place of the clinic experience, it is a powerful tool to be used in conjunction with traditional ambulatory experiences to assure uniform mastery of clinical examination skills. REFERENCES 1.
2.
3.
4.
5. 6. 7. 8. 9. 10.
11.
12.
13.
14.
15.
Buffered, J. I. Changing paths and places for training tomorrow’s generalists. In B. Gaste and E. E. Rogers (Eds.), Clinical Education and the Doctor of Tomorrow, New York: New York Academy of Medicine, 1989. Pp. 67– 80. Barker, L. R. Curriculum for ambulatory care training in medical residency: Rationale, attitudes, and generic proficiencies. J. Gen. Intern. Med. Suppl. 5: S3, 1990. Feltovich, J., Maswt, T. A., and Soler, N. G. Teaching medical students in ambulatory setting in departments of internal medicine. Acad. Med. 64: 36, 1989. Gruppen, L. D., Wisdom, K., Anderson, D. S., and Woolliscroft, J. O. Assessing the consistency and educational benefits of students’ clinical experiences during an ambulatory care internal medicine rotation. Acad. Med. 68: 674, 1993. Lawrence, R. S. The goals for medical education in the ambulatory setting. J. Gen. Intern. Med. Suppl. 3: S5, 1998. Ruane, T. J. A year-long clerkship in ambulatory care. J. Med. Educ. 63: 699, 1998. Wooliscroft, J. O., and Schewnk, T. L. Teaching and learning in the ambulatory setting. Acad. Med. 64: 644, 1989. Irby, D. Teaching and learning in ambulatory care settings: A thematic review of the literature. Acad. Med. 70: 898, 1995. Abbott, A. V., and Lee, P. V. Medical student education in ambulatory care. Acad. Med. 64(Suppl. 2): S9, 1989. Muller, S. Physicians for the twenty-first century: Report of the project panel on the general professional education of the physician and college preparation for medicine. J. Med. Educ. 59(Part 2): 1984. Cahlabian, J., Formenti, S., Russell, C., Pearce, J., and Dunnington, G. Comprehensive needs assessment of clinical breast evaluation skills of primary care residents. Ann. Surg. Oncol. 5(2): 166, 1998. Chalabian, J., Garman, K., Wallace, P., and Dunnington, G. Clinical breast evaluation skills of house officers and students. Am. Surg. 62(10): 840, 1996. Sloan, D. A., Donnelly, M. B., Johnson, S. B., Schwartz, R. W., and Strodel, W. E. Use of an objective structured clinical examination (OSCE) to measure improvement in clinical competence during the surgical internship. Surgery 1144: 343, 1993. Endean, E., Sloan, D., Veldenz, H., Donnelly, M., and Schwarcz, T. Performance of the vascular physical examination by residents and medical students. J. Vasc. Surg. 19: 149, 1994. Sloan, D. A., Donnelly, M. B., Schwartz, R. W., Felts, J. L., Blue,
AULT ET AL.: FOCUSED BREAST SKILLS WORKSHOP
16.
A. V., and Strodel, W. E. The use of the objective structured clinical examination for evaluation and instruction in graduate medical education. J. Surg. Res. 63: 225, 1996.
20.
Cohen, R., Reznick, R. K., Taylor, B. R., Provan, J., and Rothman, A. Reliability and validity of the objective structured clinical examination in assessing surgical residents. Am. J. Surg. 160: 302, 1990.
21.
17.
Sloan, D. A., Donnelly, M. B., Schwartz, R. W., Much, L. L., Wells, M. D., Johnson S., et al. Assessing the clinical competence of medical students and surgery residents in surgical oncology problem solving. Ann. Surg. Oncol. 1: 204, 1994.
18.
Sloan, D. A., Donnelly, M. B., Schwartz, R. W., McGrath, P., Kenady, D. E., Wood, D., et al. Measuring the ability of residents to manage oncologic problems. J. Surg. Oncol. 64: 135, 1997.
19.
Blue, A. V., Stratton, T. D., Plymale, M., et al. The effectiveness
22.
23.
24.
307
of the structured clinical instruction model. Am. J. Surg. 176: 67, 1998. Sloan, D. A., Donnelly, M. B., Plymale, M. A., et al. Improving residents’ clinical skills with the structured clinical instruction model for breast cancer: Results of a multiinstitutional study. Surgery 122: 324, 1997. Bandura, A. Self-efficacy mechanism in human agency. Am. Psychol. 37: 122, 1982. Schunk, D. H. Self-efficacy and cognitive skill learning. In C. Ames and R. Ames (Eds.), Research in Motivation in Education: Volume 3. Goals and Cognitions. San Diego: Academic Press. Pp. 13– 44, 1997. Bentley, J. D., Knapp, R. M., and Petersdorf, R. G. Education in ambulatory care—Financing is one piece of the puzzle. N. Engl. J. Med. 320(23): 1531, 1989. Adams, M., and Eisenber, J. M. What is the cost of ambulatory education? J. Gen. Intern. Med. 12: SS104, 1997.
A Focused Breast Skills Workshop Improves the Clinical Skills of Medical Students Glenn T. Ault, M.D., Maura Sullivan, R.N., Jeanine Chalabian, M.D., and Kristin A. Skinner, M.D. Department of Surgery, Keck School of Medicine, University of Southern California, Los Angeles, California 90089-9034 Submitted for publication November 20, 2001; published online July 24, 2002
Purpose. The aim of this study was to determine the effectiveness of a focused breast skills workshop for teaching clinical skills to third-year medical students. Methodology. One hundred twenty-four third-year medical students involved in the surgical core clerkship were randomly assigned to two groups. Group 1 (n ⴝ 67) participated in a 2-h focused breast skills workshop. Group 2 (n ⴝ 57) received “traditional” ambulatory teaching for a period of 4 h in the breast clinic. The focused workshop consisted of a series of encounters concentrating on mammogram and ultrasound interpretation, physical examination skills, lump detection, and workup of a breast mass. Both groups received a didactic core curriculum lecture from surgical faculty. All students completed a satisfaction rating scale and a subset of students completed a pre- and postencounter self-efficacy rating scale on several aspects of breast skills. Student’s t test was used to compare the groups in the areas of clinical skills as evidenced by performance on the breastspecific items on the end of the clerkship Objective Structure Clinical Examination and student satisfaction as evidenced by their response on a satisfaction rating scale. ANCOVA (controlling for preencounter self-efficacy rating) was used to compare the change scores between pre- and postencounter self-efficacy ratings. Results. Students in Group 1 performed significantly higher than the students in Group 2 in the areas of clinical examination skills (t ⴝ ⴚ2.99, P < 0.05); in sensitivity (t ⴝ ⴚ5.82, P < 0.05) and specificity (t ⴝ ⴚ7.27, P < 0.05) in the examination of breast models; and with their satisfaction with the encounter (t ⴝ 10.72, P < 0.05). Students in Group 1 also demonstrated a higher level of confidence in their breast skills at the
Presented at the Annual Meeting of the Association for Academic Surgery, Milwaukee, Wisconsin, November 15–17, 2001.
end of the clerkship than students in Group 2 (F ⴝ 6.22, P < 0.05). Conclusions. The focused breast skills workshop is more effective than the traditional ambulatory setting for teaching clinical breast examination skills. This setting also demonstrated the development of higher confidence in breast skills than the traditional ambulatory setting. © 2002 Elsevier Science (USA) Key Words: physical examination skills; surgical education; medical students; breast disease.
INTRODUCTION
Clinical education in medicine has continued to shift out of the hospital ward and into the ambulatory care clinic, where it has been estimated that 95% of doctor– patient relationships occur [1]. Champions of education in the ambulatory setting state several arguments for the important and unique learning opportunities offered in outpatient clinics including but not limited to continuity of care, practice of health promotion and disease prevention, and development of physician/ patient communication skills [2–7]. A recent review of the literature revealed that education in ambulatory care clinics is characterized by variability, unpredictability, immediacy, and lack of continuity [8]. Furthermore, the review of the studies found that learners often see a narrow range of patient problems in a single clinic and experience limited continuity of care. Few cases are discussed with attending physicians and little observation of the student occurs during their encounter. When case discussions occur, they are generally short in duration, involve little teaching, and provide virtually no feedback [8]. Increased costs and decreased efficiency of the faculty and clinic time are also concerns regarding the logistics of teaching in the ambulatory setting [9]. Despite these imperfections,
303
0022-4804/02 $35.00 © 2002 Elsevier Science (USA) All rights reserved.
304
JOURNAL OF SURGICAL RESEARCH: VOL. 106, NO. 2, AUGUST 2002
the push to provide experience for medical students in the ambulatory setting continues. The Panel on the General Professional Education of the Physician reported that fewer than 5% of current patient encounters result in hospitalization [10], which places significant emphasis on the ambulatory setting to teach medical students essential physical examination skills. The increasing use of performance-based tests such as the Objective Structured Clinical Examination (OSCE) has shown that both medical students and surgical residents possess disturbing deficiencies in clinical skills [11–16]. Breast disease and breast cancer are areas in which the performance of students and residents has proved to be deficient, as measured during interactions with standardized patients [11, 12, 17, 18]. In response to the documented deficiencies of clinical examination skills in breast disease as well as the irregularities of education found in the ambulatory setting, we inaugurated a focused breast skills workshop during the required surgical clerkship at the University of Southern California. Building on the success of structured clinical models previously reported in the literature [19, 20], our multidimensional focused clinical workshop concentrated on physical examination skills, mammogram and ultrasound interpretation, lump detection, and the workup of a breast mass. The intent of our study was to determine the effectiveness of this breast skills workshop in teaching medical students physical examination and diagnostic skills for breast disorders in comparison to that of the traditional ambulatory setting. METHODS One hundred thirty medical students completing their required 6-week third-year surgical clerkship during the academic year 1997– 1998 at the University of Southern California participated in this study. This prospective randomized study assigned students to either the intervention group (N ⫽ 62) or the control group (N ⫽ 68) according to a table of randomized numbers. A power of analysis was performed to a power of 90 and ␣ of 0.5 which defined an appropriate sample size of 56 students. Students in the intervention group participated in the 2 1/2 h breast skills workshop while the students in the control group participated in a 4-h traditional ambulatory experience in the breast clinic of our teaching hospital. The focused breast skills workshop consisted of three separate learning modules. Prior to rotating through the modules, the students received a brief orientation to the workshop as well as some fundamentals in radiographic diagnosis of breast disease. They also observed a faculty member model the breast examination emphasizing key examination maneuvers. The first module required students to review actual screening mammograms from patients with both benign and malignant breast disorders and to work through the diagnostic workup of those patients. This included interpretation of specialized mammographic views as well as ultrasound. During this module, students interacted with an expert faculty member during case discussion and received feedback regarding their radiological interpretations. The second module was composed of a standardized patient (SP) station in which the students were given the opportunity to practice their examina-
tion skills and to receive feedback from the SP. The student was afforded the opportunity to ask specific questions at the end of the encounter and receive instruction in deficient skill areas. The surgical nurse educator was available at this module to provide assistance and to answer any questions that arose. The last module consisted of improving lump detection skills utilizing latex breast models (Mammacare). This module, led by the surgical education fellow, had specific instruction regarding the workup of both a palpable and a nonpalpable breast mass. Attention was given to diagnostic techniques, surgical biopsy options, and definitive surgical care. In all modules, there was ample opportunity for students to ask questions and receive feedback regarding their examination or diagnostic skills. Students in the control group participated in the outpatient breast clinic for a period of 4 h. The outpatient clinic in our county hospital setting is primarily a diagnostic clinic in which the referred patients either are symptomatic or present with abnormal physical examination or mammogram findings. A surgical oncology faculty member or the residents rotating on the oncology service supervised them during their clinic experience. In addition to whichever arm they were randomized to, all students received the same didactic core curriculum lecture on breast disease. All students also had access to a breast disease self-study module consisting of 23 slides with comments and explanations, which was available for their use in the student learning center. After the completion of either the clinic or the workshop, the participants completed a one-item satisfaction rating scale (scale: 1, strongly disagree, to 4, strongly agree) evaluating the educational value of their experience. In order to determine any perceived change in self-efficacy, a subset of students (N ⫽ 110) completed a five-item pre- and postclerkship self-efficacy rating scale (␣ ⫽ 0.797). The rating scale asked the students to self-rate their skills in the areas of physical examination, mammogram and ultrasound interpretation, lump detection, and the ability to teach breast self-examination, as well as problem-solving and management issues (scale of 1, poor, to 5, outstanding). At the end of the clerkship, each student participated in the surgery OSCE that included one breast standardized patient station and one breast paper case (␣ ⫽ 0.707). The standardized patient station tested both clinical skills and problem-solving abilities, while each paper case tested only problem-solving abilities. To assess clinical skills, only the physical examination portion of the checklist completed by the SP was utilized. The SP was blinded to which arm the students had participated in. To ascertain the student’s problem-solving skills, a combined score from both the SP station and the paper case was determined. There was also a lump detection station in the OSCE where the students were asked to palpate seven latex breast models (Mammacare) and document their findings. Sensitivity and specificity of their lump detection was determined from this OSCE station. The lump detection models used for the OSCE were different from the ones utilized in the workshop. In addition, each student also took a 75-item multiple-choice examination (MCQ) at the completion of the clerkship. Six items on this examination (␣ ⫽ 0.732) tested cognitive knowledge of breast disease. These examinations (OSCE and MCQ) occurred at a period of 1 to 5 weeks after either the clinic or the workshop experience. The scores obtained from the problem-solving portion of the OSCE and the MCQ were combined into one score. In order to assess retention of clinical skills, we looked at the student’s performance on the Clinical Performance Examination (CPX) which is taken at the completion of the third year of the curriculum. The CPX can occur anywhere from 1 to 7 months after the completion of the clerkship. We assessed the performance of the students on the one breast case included in the CPX. In order to assess the significance of differences between the groups in the areas of clinical skills; problem solving and knowledge acquistion; retention of clinical skills; sensitivity and specificity of lump detection; and student satisfaction with the encounter, the independent samples t test was utilized to compare the means. To
305
AULT ET AL.: FOCUSED BREAST SKILLS WORKSHOP
TABLE 1 Mean Assessment and Standard Deviation of Performances on End of Clerkship Measures
Physical examination skills (scale 0–10) Sensitivity of lump detection (scale 0–1) Specificity of lump detection (scale 0–1) Knowledge acquisition (scale 0–10) Student satisfaction (scale 0–4) Self-efficacy composite score change (scale 0–20)
Focused workshop
Ambulatory clinic
8.33 ⫾ 1.66 0.77 ⫾ 0.12 0.79 ⫾ 0.12 7.74 ⫾ 2.7 3.89 ⫾ 0.32 6.55 ⫾ 5.52
7.52 ⫾ 1.8 0.64 ⫾ 0.15 0.62 ⫾ 0.15 7.06 ⫾ 2.3 2.44 ⫾ 1.0 4.48 ⫾ 2.89
determine the difference between the pre- and post-self-efficacy ratings, we calculated a composite change score (subtracting pre from post ratings) on all students. ANCOVA, controlling for preencounter self-efficacy ratings, was used to compare the difference in change scores between groups.
RESULTS
Students who participated in the workshop performed significantly higher than students who attended the traditional ambulatory experience in areas of clinical examination skills (t ⫽ ⫺2.832, P ⫽ 0.005) as well as in sensitivity (t ⫽ ⫺5.714, P ⬍ 0.001) and specificity (t ⫽ ⫺7.78, P ⬍ 0.001) of lump detection. Over 91% of the students who participated in the workshop rated it as an outstanding educational experience which led to a significant difference with student satisfaction with the workshop as an educational experience (t ⫽ 10.718, P ⬍ 0.001). There was found to be no statistical significance in comparing the experimental group and control group outcome on the surgery clerkship final examination (t ⫽ 1.06, P ⫽ 0.290). The experimental group also performed higher than students in the control group in the area of knowledge acquisition (as measured by problem solving on the OSCE and answers breast questions on MCQ) and in retention of skills (as measured on the CPX), but the difference between the groups was found not to be significant (t ⫽ 1.50, P ⫽ 0.136 and t ⫽ ⫺1.49, P ⫽ 0.137, respectively) (Table 1). The students who partook in the workshop demonstrated significantly higher self-efficacy rating change in their breast examination skills at the end of the clerkship compared to the control group as determined by the ANCOVA analysis (F ⫽ 8.92, P ⫽ 0.003). Specifically, we found significant differences in examination skills (F ⫽ 4.00, P ⬍ 0.05), mammogram and ultrasound interpretation skills (F ⫽ 37.81, P ⬍ 0.001), and knowledge base (F ⫽ 4.23, P ⬍ 0.05). All reported P values are two-tailed. The cost of the standardized clinic was approximately $39.00 per student. DISCUSSION
As students enter their years of clinical training, they begin to truly develop and master the examina-
P P P P P P P
⫽ ⬍ ⬍ ⫽ ⬍ ⫽
0.005 0.001 0.001 0.136 0.001 0.003
tion skills essential to the practice of medicine. During these years they interact with a variety of faculty and residents to learn more focused examination techniques as well as refine basic physical examination skills. A majority of this interaction occurs in the ambulatory setting, which can be hampered by many barriers [3]. This, in combination with several reports regarding deficiencies of physical examination skills of both medical students and residents, is a cause of concern [11–18]. The focused breast skills workshop was created to provide students with a structured learning experience for clinical competence specifically tailored to breast examination and diagnostic skills. Our study demonstrated that this format is more effective in teaching clinical breast skills than the traditional ambulatory setting specifically in the areas of clinical examination skills, sensitivity and specificity of lump detection, and overall improvement of student’s self-efficacy. Students also demonstrated significantly higher satisfaction with this format as an educational experience. One could argue that the students who participated in the workshop did not demonstrate a significant difference in the acquisition of cognitive knowledge, but this was not expected as the workshop itself only focused on clinical skill acquisition and the fact that all students had received the same didactic core curriculum lecture and had access to the self-study module. One could also argue that the students who participated in the workshop performed better in lump detection because they were exposed to the models prior to the examination whereas the control group was not. Granted, breast models are an imperfect surrogate for lump detection, but it is the only methodology we have to reliably assess lump detection skills. We also did not demonstrate significance between the two groups in the area of retention of clinical skills as determined by the breast case on the CPX examination. Students at the University of Southern California are required to pass this OSCE-style examination at the completion of their third year. Successful completion is required for graduation. Perhaps one of the reasons that no significant difference was observed was because of the way that the CPX is scored. We were
306
JOURNAL OF SURGICAL RESEARCH: VOL. 106, NO. 2, AUGUST 2002
unable to tease out clinical examination performance from the entire case score. Therefore, the CPX score included history taking and patient–physician interaction scores as well as the clinical examination score. Since we were only specifically interested in clinical examination scores, our inability to exclude the other criteria may have clouded our results. The variability in the length of time between the clerkship and the administration of the CPX may have confounded the retention results as well. We believe that the workshop was more effective in teaching clinical skills because it provided students with a practical level of exposure to the three modules and sufficient time was spent in practice and feedback. One of the problems with teaching in the ambulatory care setting is that teaching is often limited and sometimes rushed secondary to the high volume of patients that must be seen in a short period of time. Furthermore, the workshop provided all participants with a consistent exposure to the areas of radiological diagnosis, examination skills, and lump detection. In this study, students who participated in the workshop demonstrated higher self-efficacy ratings in their breast skills than students who did not. Self-efficacy is an individual’s evaluation that he or she has specific performance capabilities on a particular type of task [21]. We felt that this was an important finding because perceived efficacy can influence motivation. Students with a high sense of efficacy are more likely to choose difficult tasks, expend greater effort, persist longer, apply appropriate problem-solving strategies on tasks, and have less fear and anxiety regarding tasks than students with a low sense of efficacy for a task [22]. One could argue that perceived efficacy does not translate into better clinical skills; however, the students in our study did perform significantly better in the area of clinical skills. We believe that the workshop helped students achieve a higher level of selfefficacy because of the focused instruction; the abundant immediate feedback provided to students on their clinical skills; and the provision of an environment that was conducive to learning. The workshop also has an advantage in that it specifically focused instruction on the clinical examination aspects of breast disease. It could offer the opportunity for multidisciplinary instruction. Another advantage of the workshop is its cost. Instruction in the ambulatory setting is usually one on one, translating into mentors either seeing fewer patients or working a longer day [23], which could be one of the explanations for the higher cost of educating students in the ambulatory setting. Considering the costs of educating medical students in the traditional ambulatory setting, which one study concluded could range from $100 to $327 per student per day [23], the cost of our intervention was much less. The workshop
utilized faculty time efficiently and optimized the student’s exposure to them as well. The Focused Breast Skills Workshop demonstrated several advantages over the traditional ambulatory setting in the areas of clinical skill acquisition and higher ratings of self-efficacy by the participants. This intervention standardized learning and was found to be an effective educational experience. It optimized exposure to expert faculty without excessive demands on their time. This workshop also overcame some of the barriers cited in the literature as detrimental to learning in the traditional ambulatory setting. While we are not advocating that the workshop take the place of the clinic experience, it is a powerful tool to be used in conjunction with traditional ambulatory experiences to assure uniform mastery of clinical examination skills. REFERENCES 1.
2.
3.
4.
5. 6. 7. 8. 9. 10.
11.
12.
13.
14.
15.
Buffered, J. I. Changing paths and places for training tomorrow’s generalists. In B. Gaste and E. E. Rogers (Eds.), Clinical Education and the Doctor of Tomorrow, New York: New York Academy of Medicine, 1989. Pp. 67– 80. Barker, L. R. Curriculum for ambulatory care training in medical residency: Rationale, attitudes, and generic proficiencies. J. Gen. Intern. Med. Suppl. 5: S3, 1990. Feltovich, J., Maswt, T. A., and Soler, N. G. Teaching medical students in ambulatory setting in departments of internal medicine. Acad. Med. 64: 36, 1989. Gruppen, L. D., Wisdom, K., Anderson, D. S., and Woolliscroft, J. O. Assessing the consistency and educational benefits of students’ clinical experiences during an ambulatory care internal medicine rotation. Acad. Med. 68: 674, 1993. Lawrence, R. S. The goals for medical education in the ambulatory setting. J. Gen. Intern. Med. Suppl. 3: S5, 1998. Ruane, T. J. A year-long clerkship in ambulatory care. J. Med. Educ. 63: 699, 1998. Wooliscroft, J. O., and Schewnk, T. L. Teaching and learning in the ambulatory setting. Acad. Med. 64: 644, 1989. Irby, D. Teaching and learning in ambulatory care settings: A thematic review of the literature. Acad. Med. 70: 898, 1995. Abbott, A. V., and Lee, P. V. Medical student education in ambulatory care. Acad. Med. 64(Suppl. 2): S9, 1989. Muller, S. Physicians for the twenty-first century: Report of the project panel on the general professional education of the physician and college preparation for medicine. J. Med. Educ. 59(Part 2): 1984. Cahlabian, J., Formenti, S., Russell, C., Pearce, J., and Dunnington, G. Comprehensive needs assessment of clinical breast evaluation skills of primary care residents. Ann. Surg. Oncol. 5(2): 166, 1998. Chalabian, J., Garman, K., Wallace, P., and Dunnington, G. Clinical breast evaluation skills of house officers and students. Am. Surg. 62(10): 840, 1996. Sloan, D. A., Donnelly, M. B., Johnson, S. B., Schwartz, R. W., and Strodel, W. E. Use of an objective structured clinical examination (OSCE) to measure improvement in clinical competence during the surgical internship. Surgery 1144: 343, 1993. Endean, E., Sloan, D., Veldenz, H., Donnelly, M., and Schwarcz, T. Performance of the vascular physical examination by residents and medical students. J. Vasc. Surg. 19: 149, 1994. Sloan, D. A., Donnelly, M. B., Schwartz, R. W., Felts, J. L., Blue,
AULT ET AL.: FOCUSED BREAST SKILLS WORKSHOP
16.
A. V., and Strodel, W. E. The use of the objective structured clinical examination for evaluation and instruction in graduate medical education. J. Surg. Res. 63: 225, 1996.
20.
Cohen, R., Reznick, R. K., Taylor, B. R., Provan, J., and Rothman, A. Reliability and validity of the objective structured clinical examination in assessing surgical residents. Am. J. Surg. 160: 302, 1990.
21.
17.
Sloan, D. A., Donnelly, M. B., Schwartz, R. W., Much, L. L., Wells, M. D., Johnson S., et al. Assessing the clinical competence of medical students and surgery residents in surgical oncology problem solving. Ann. Surg. Oncol. 1: 204, 1994.
18.
Sloan, D. A., Donnelly, M. B., Schwartz, R. W., McGrath, P., Kenady, D. E., Wood, D., et al. Measuring the ability of residents to manage oncologic problems. J. Surg. Oncol. 64: 135, 1997.
19.
Blue, A. V., Stratton, T. D., Plymale, M., et al. The effectiveness
22.
23.
24.
307
of the structured clinical instruction model. Am. J. Surg. 176: 67, 1998. Sloan, D. A., Donnelly, M. B., Plymale, M. A., et al. Improving residents’ clinical skills with the structured clinical instruction model for breast cancer: Results of a multiinstitutional study. Surgery 122: 324, 1997. Bandura, A. Self-efficacy mechanism in human agency. Am. Psychol. 37: 122, 1982. Schunk, D. H. Self-efficacy and cognitive skill learning. In C. Ames and R. Ames (Eds.), Research in Motivation in Education: Volume 3. Goals and Cognitions. San Diego: Academic Press. Pp. 13– 44, 1997. Bentley, J. D., Knapp, R. M., and Petersdorf, R. G. Education in ambulatory care—Financing is one piece of the puzzle. N. Engl. J. Med. 320(23): 1531, 1989. Adams, M., and Eisenber, J. M. What is the cost of ambulatory education? J. Gen. Intern. Med. 12: SS104, 1997.