The American Journal of Surgery 193 (2007) 771–773
Association for Surgical Education
A CRASH course in procedural skills improves medical students’ self-assessment of proficiency, confidence, and anxiety Robyn A. Stewart, M.D.a,*, Linnea S. Hauge, Ph.D.a, Robert D. Stewart, M.D., M.P.H.b, Robert L. Rosen, M.D.a, Angella Charnot-Katsikas, M.D.a, Richard A. Prinz, M.D.a a
Department of General Surgery and Office of Medical Student Programs, Rush University Medical Center, 1653 W. Congress Parkway, Chicago, IL 60612, USA b Children’s Memorial Hospital, 2300 Children’s Plaza, Chicago, IL 60614, USA Manuscript received April 11, 2006; revised manuscript January 3, 2007
Abstract Background: Medical students enter the clinical education years with minimal exposure to procedural skills. This study aims to determine the effect of a pre-clinical skills course on students’ ratings of proficiency, confidence, and anxiety levels. Methods: An elective pre-clinical course in procedural skills was given to medical students prior to clinical rotations. Self-assessment of proficiency, confidence, and anxiety in performing these skills was made prior to and on completion of the course. Results: Course participants reported low proficiency and confidence, and high anxiety regarding performing procedural skills. Following the course, there was significant improvement in self-assessed proficiency, confidence, and anxiety. There were no gender differences in responses. Conclusions: Medical students entering clinical rotations have poor self-assessment of proficiency, low confidence, and high anxiety regarding procedural skills. A concentrated course in procedural skills significantly improved students’ assessment of their proficiency, confidence, and anxiety levels. © 2007 Excerpta Medica Inc. All rights reserved. Keywords: Procedural skills; Clinical skills; Medical students; Confidence; Proficiency; Anxiety
The first 2 years of medical school are comprised predominately of didactics and rarely include procedural experience. Moss et al showed that medical students experience anxiety at the prospect of entering the clinical education years. When specifically questioned, some of the things they were most anxious about included situations where they would be required to perform tasks on patients such as suturing or taking blood [1]. Several studies have shown that instruction in performing specific skills improves individuals’ perception of confidence, as well as evaluated competence in performing that skill [2– 6]. Some of these studies required courses that took several days to several weeks. Due to the increased demands on the medical school curriculum, it is difficult to accommodate such time-consuming “mini-courses.” Furthermore, the effect of such courses on students’ anxiety levels is unclear. We hypothesized that even a very brief course would
* Corresponding author. 1725 W. Harrison St., Suite 818, Chicago, IL 60612. Tel.: ⫹1-312-942-3074; fax: ⫹1-312-942-6520. E-mail address:
[email protected]
help alleviate the anxiety and improve confidence of medical students leading into their third-year clerkships. The purpose of this study was to determine the effect of this course on students’ perception of anxiety, confidence, and proficiency. Methods In June 2005 at Rush Medical College, a 1-week elective course was launched with instruction in various clinically pertinent topics including reading electrocardiographs and abdominal and chest x-rays, obtaining and interpreting bedside chart data, and other clinically pertinent topics. The course was titled a Clinical Resources And Skills for the Hospital or “CRASH” course. The CRASH course was administered in the week prior to the first clinical rotation. As this was the first time the course was given, it was offered as an elective. Within this course was a specific section concentrating on procedural skills. The procedural skills portion of the course included knot-tying, suturing, nasogastric tube place-
0002-9610/07/$ – see front matter © 2007 Excerpta Medica Inc. All rights reserved. doi:10.1016/j.amjsurg.2007.01.019
772
R.A. Stewart et al. / The American Journal of Surgery 193 (2007) 771–773
Table 1 Schedule of procedural skills sessions Students in each session are divided into 4 groups and rotate through the 4 stations: Session I Suture and knot tying 30 minutes IV placement 30 minutes Foley placement 30 minutes Informed consent 30 minutes Session II Suture and knot tying* 30 minutes IV placement* 30 minutes NG placement 30 minutes Sterile technique 30 minutes * Second session consists of observed practice and feedback by faculty.
ment, intravenous catheterization, and bladder catheterization and was completed in 4 hours (2-hour sessions on 2 consecutive days). The schedule for each session is demonstrated in Table 1. Each skill was taught in a 30-minute segment by attending and senior resident surgeons on inanimate models or mannequins. For the more complex skills of suturing, knot-tying, and intravenous (IV) line placement, the course included a second 30-minute session of supervised practice time with instructor feedback. Student groups were kept small, allowing for individual attention to each student. This study included 118 Rush Medical College students who had completed their second year of training and were preparing to enter their first clinical year (M3). There were 47 men and 71 women. One hundred of the 118 M3 students elected to participate in the CRASH course. This study examines the effects of the 4-hour procedural skills portion of the 1-week CRASH course on students’ perceptions of proficiency, confidence, and anxiety in performing the procedural skills of IV placement, suturing and knot-tying, nasogastric tube placement, and Foley catheter placement. All 118 students completed a self-assessment of their proficiency, confidence, and level of anxiety in performing these procedural skills prior to the CRASH course regardless of participation. The 100 CRASH participants completed a second self-assessment on completion of the course. Students’ self-assessments were made using a numeric rating scale. An example of the survey questions and the rating scale are included in Table 2. Data were analyzed using paired Student t test when comparing pre- and post-procedural skills course results. Unpaired Student t test was used when comparing results based on differences in gender and course participation. The analyses were accomplished using Statview statistical software (SAS Corp, Cary, NC) and a 2-tailed P value ⬍.05 was considered statistically significant. Results Of the students who took the CRASH course, the mean proficiency score for each of the clinical skills fell between “no proficiency” and “low proficiency.” Upon completion of the course, their self-assessed proficiency was just below “somewhat proficient.” Similarly, students rated their confidence levels for all skills between “no confidence” and
“low confidence” prior to the course, and on completion their levels had increased to just below “somewhat confident.” Students initially documented their anxiety levels between “high anxiety” and “some anxiety.” After the course, they reported their anxiety levels for all skills as between “some anxiety” and “low anxiety.” Students who did not take the course assessed themselves as significantly more proficient in all skills areas (P ⬍0.04); however, there was no difference in their confidence or anxiety levels (Table 3). There were no gender differences of perceived proficiency, confidence, or anxiety levels prior to CRASH and both male and female students had similar improvement in the outcomes following CRASH. Comments This study showed that medical students entering their clinical years had low levels of confidence and high anxiety regarding the performance of common procedural skills such as knot-tying, suturing, nasogastric tube placement, IV catheterization, and bladder catheterization. However, a brief preclinical training course significantly improved self-assessment of confidence and proficiency and lowered levels of anxiety, supporting our initial hypothesis. Previous studies have documented increased confidence and competence following instruction; however, they have not explored the effect on anxiety levels. A brief 4-hour course was sufficient to produce a significant effect in all areas of perceived confidence, proficiency, and anxiety. The significance of this finding is demonstrated by the study of Fincher et al. They examined the association between medical students’ self-assessment of clinical competence following a 2-day skills training session and the number of procedures performed at the completion of the third year. For most procedures, self-assessment of competency correlated with the frequency of performance of the procedure [2]. Students who did not take the CRASH course perceived themselves as more proficient in performing these skills than their colleagues who opted to participate in the course. This is despite sharing low self-assessment of confidence and high assessment of anxiety. It is possible that these students felt they were competent enough to function in the clinical setting, and that they did not consider that a course of instruction could improve their confidence and relieve anxiety. Another explanation could be that these students’ self-assessment of proficiency does not represent an accurate depiction of their actual competence. Kruger and Dunning have shown that people who Table 2 Survey questions and rating scale 1. Rate your current proficiency in placing an IV line. 1 2 3 4 no proficiency low proficiency somewhat proficient very proficient 2. Rate your confidence about being able to independently place an IV line. 1 2 3 4 no confidence low confidence some confidence great confidence 3. Rate your anxiety when you think about the results of your independent performance of placing an IV line. 1 2 3 4 high anxiety some anxiety low anxiety no anxiety
R.A. Stewart et al. / The American Journal of Surgery 193 (2007) 771–773
773
Table 3 Students’ ratings of proficiency, confidence, and anxiety Procedure
Suture IV NG Foley
Participation
Yes No Yes No Yes No Yes No
Proficiency
Confidence
Anxiety
Pre
Post
Pre
Post
Pre
Post
1.34 ⫾ .65 1.89 ⫾ .87§ 1.16 ⫾ .42 1.42 ⫾ .70储 1.05 ⫾ .21 1.32 ⫾ .58¶ 1.12 ⫾ .42 1.58 ⫾ .84**
2.72 ⫾ .60*
1.53 ⫾ .84 1.84 ⫾ .83 1.41 ⫾ .76 1.42 ⫾ .69 1.36 ⫾ .73 1.47 ⫾ .61 1.35 ⫾ .72 1.63 ⫾ .83
2.69 ⫾ .72*
1.93 ⫾ .90 1.95 ⫾ .93 1.72 ⫾ .82 1.58 ⫾ .84 1.81 ⫾ .85 1.58 ⫾ .69 1.78 ⫾ .84 1.74 ⫾ .93
2.45 ⫾ .80†
2.78 ⫾ .64* 2.82 ⫾ .54* 2.76 ⫾ .61*
2.64 ⫾ .73* 2.71 ⫾ .60* 2.80 ⫾ .67*
2.25 ⫾ .76‡ 2.52 ⫾ .72* 2.50 ⫾ .73*
IV ⫽ intravenous; NG ⫽ nasogastric. * P ⬍.0001 comparing pre- and post-procedural skills course by paired Student t test. † P ⬍.006 comparing pre- and post-procedural skills course by paired Student t test. ‡ P ⬍.005 comparing pre- and post-procedural skills course by paired Student t test. § P ⬍.03 comparing participants and nonparticipants in pre skills course evaluation by unpaired Student t test. 储 P ⬍.05 comparing participants and nonparticipants in pre skills course evaluation by unpaired Student t test. ¶ P ⬍.02 comparing participants and nonparticipants in pre skills course evaluation by unpaired Student t test. ** P ⬍.0009 comparing participants and nonparticipants in pre skills course evaluation by unpaired Student t test.
are unskilled often do not recognize their incompetence and therefore overestimate their abilities [7]. In the clinical setting, previous studies have shown that pre-instruction confidence is not correlated with actual competence in performance of a skill [4,8]. Leopold et al actually showed an inverse relationship between pre-instruction assessment of confidence and evaluated competence [3]. This study revealed no difference in self-assessment of confidence, proficiency or anxiety based on gender. This differs from the findings of Leopold et al, who showed that men have higher pre-instruction levels of confidence regarding the performance of clinical skills than women [3]. The finding of higher self-assessment of proficiency in the group who did not take the CRASH course cannot be explained by an increased number of men in that group, as there were 10 women and 8 men who did not participate in the CRASH course. One limitation of this study is involvement of a single institution. Additionally, there may be selection bias due to the elective nature of the course as well as the fact that students were not asked about previous experience in performing clinical skills. It is unclear if the students who did not take the course declined it because of perceived higher confidence, or if they had conflicting responsibilities during that time. Finally, all study results rely on the self-assessment of the students who were queried, which, as previously mentioned, is not always an accurate indicator of actual competence. A comparison of the competence, as measured by skill performance, of students who participated in the
CRASH course as compared to those who did not take the course would potentially legitimize students’ assessments of increased confidence and proficiency and decreased anxiety, and presents a promising direction for further study. Additionally, it should be explored whether improving students’ self-assessment of confidence and proficiency and decreasing their anxiety prior to starting their clinical education years will result in their performing procedures more skillfully and more frequently during those years. References [1] Moss F, McManus IC. The anxieties of new clinical students. Med Educ 1992;26:17–20. [2] Fincher RM, Lewis LA. Learning, experience, and self-assessment of competence of third-year medical students in performing bedside procedures. Acad Med 1994;69:291–5. [3] Leopold SS, Morgan HD, Kadel NJ, et al. Impact of educational intervention on confidence and competence in the performance of a simple surgical task. J Bone Joint Surg Am 2005;87:1031–7. [4] Barrett SV, Zapka JG, Mazor KM, et al. Assessing third-year medical students’ breast cancer screening skills. Acad Med 2002;77:905–10. [5] Ault GT, Sullivan M, Chalabian J, et al. A focused breast skills workshop improves the clinical skills of medical students. J Surg Res 2002;106:303–7. [6] Peyre SE, Peyre CG, Sullivan ME, et al. A surgical skills elective can improve student confidence prior to internship. J Surg Res 2006;133: 11–5. [7] Kruger J, Dunning D. Unskilled and unaware of it: how difficulties in recognizing one’s own incompetence lead to inflated self-assessments. J Pers Soc Psychol 1999;77:1121–34. [8] Barnsley L, Lyon PM, Ralston SJ, et al. Clinical skills in junior medical officers: a comparison of self-reported confidence and observed competence. Med Educ 2004;38:358 – 67.