- Email: [email protected]
Breadth versus volume: Neurology outpatient clinic cases in medical education
Journal of Clinical Neuroscience xxx (2016) xxx–xxx
Contents lists available at ScienceDirect
Journal of Clinical Neuroscience journal homepage: www.elsevier.com/locate/jocn
Education
Breadth versus volume: Neurology outpatient clinic cases in medical education Dara V. Albert a,⇑, Angela D. Blood b, Yoon Soo Park c, James R. Brorson d, Rimas V. Lukas d a
Nationwide Children’s Hospital, Ohio State University, Division of Neurology, 700 Children’s Drive, Columbus, OH 43205, USA Rush Medical College, Rush University, IL, USA c University of Illinois at Chicago, Department of Medical Education, IL, USA d University of Chicago, Department of Neurology, IL, USA b
a r t i c l e
i n f o
Article history: Received 11 December 2015 Accepted 27 December 2015 Available online xxxx Keywords: Case logs Medical education Neurology clerkship Neurology education
a b s t r a c t This study examined how volume in certain patient case types and breadth across patient case types in the outpatient clinic setting are related to Neurology Clerkship student performance. Case logs from the outpatient clinic experience of 486 students from The University of Chicago Pritzker School of Medicine, USA, participating in the 4 week Neurology Clerkship from July 2008 to June 2013 were reviewed. A total of 12,381 patient encounters were logged and then classified into 13 diagnostic categories. How volume of cases within categories and the breadth of cases across categories relate to the National Board of Medical Examiners Clinical Subject Examination for Neurology and a Neurology Clerkship Objective Structured Clinical Examination was analyzed. Volume of cases was significantly correlated with the National Board of Medical Examiners Clinical Subject Examination for Neurology (r = .290, p < .001), the Objective Structured Clinical Examination physical examination (r = .236, p = .011), and the Objective Structured Clinical Examination patient note (r = .238, p = .010). Breadth of cases was significantly correlated with the National Board of Medical Examiners Clinical Subject Examination for Neurology (r = .231, p = .017), however was not significantly correlated with any component of the Objective Structured Clinical Examination. Volume of cases correlated with higher performance on measures of specialty knowledge and clinical skill. Fewer relationships emerged correlating breadth of cases and performance on the same measures. This study provides guidance to educators who must decide how much emphasis to place on volume versus breadth of cases in outpatient clinic learning experiences. Ó 2016 Elsevier Ltd. All rights reserved.
1. Introduction The USA Liaison Committee on Medical Education requires students to document exposure to a range of clinical cases during clerkships [1]. While the number of cases, or volume, logged during an outpatient clinical experience correlates with performance on objective measures of knowledge and clinical skill [2], studies have not examined the relationship between breadth of cases and performance. Prior studies within specific medical specialties examine the case-mix encountered on clerkships, however they focus primarily on evaluating differences in types of cases seen and not their contribution to medical acumen [3,4]. Studies across specialties demonstrate that domain-specific knowledge and ⇑ Corresponding author. Tel.: +1 614 722 4916; fax: +1 614 722 4633. E-mail address: [email protected] (D.V. Albert).
general problem solving skills are necessary for clinical competence [5], and across specialties variety of cases and quality of supervision may influence acumen [6]. Although case logs requiring a broad clinical exposure are de riguer for surgical residency training [7], little has been published on the role of breadth of exposure for non-surgical specialties, and its impact on clinical acumen. This study hypothesized that the volume and breadth of cases encountered by students within the outpatient clinic component of a Neurology Clerkship would correlate positively with performance on measures of knowledge and clinical skill. The performance measures for this study include the National Board of Medical Examiners (NBME) Clinical Subject Examination for Neurology as an indicator of knowledge, and the Objective Structured Clinical Examination (OSCE) as an indicator of clinical skill. The role of the OSCE in assessment of students in a Neurology Clerkship has been previously reported [8–11].
http://dx.doi.org/10.1016/j.jocn.2015.12.020 0967-5868/Ó 2016 Elsevier Ltd. All rights reserved.
Please cite this article in press as: Albert DV et al. Breadth versus volume: Neurology outpatient clinic cases in medical education. J Clin Neurosci (2016), http://dx.doi.org/10.1016/j.jocn.2015.12.020
2
D.V. Albert et al. / Journal of Clinical Neuroscience xxx (2016) xxx–xxx
2. Methods The University of Chicago Institutional Review Board deemed this study exempt from review. Case logs are from the outpatient component of the required 4 week Neurology Clerkship for 486 students from July 2008 through June 2013, totaling 12,381 patient encounters at The University of Chicago Pritzker School of Medicine. The majority of students are third years (n = 452, 93%), with one second year student (n = 1, 0.2%), and the remainder being fourth years (n = 33, 7%). The outpatient clinic experience occurs at two sites, The University of Chicago Medicine and Northshore University Health System a community-based teaching affiliate. Students’ preferences are solicited when assigning students to sites, either University of Chicago Medicine (n = 366, 75%) or Northshore University Health System (n = 120, 25%). The outpatient clinic component of the clerkship is approximately 1 week in length. It encompasses adult, pediatric, general and subspecialty clinics. Degree of independence in patient evaluation varied from shadowing of faculty to independently taking the history, examining the patient, and presenting the assessment and management plan. This variability was not static and was influenced by a number of factors including the pace of the outpatient clinic, time available for teaching and motivation of the student. Students log each outpatient clinic encounter with the primary diagnosis or chief complaint. Students who did not log any cases (n = 9, 1.8%) or did not include diagnoses/complaints (n = 1, 0.2%) are not included in the analyses. The cases were retrospectively classified into 13 categories: seizure, cerebrovascular disease, peripheral/neuromuscular, spinal disease/neck/back pain, central nervous system demyelinating disease/neuro-immunology, headaches, sleep disorders, neuro-oncology, movement disorders, dementia, other neurodegenerative disorders, other, and not listed (Table 1). When more than one diagnosis or chief complaint is listed, the primary one is used in the analysis. Some diagnoses, such as cerebrovascular, which form a substantial percentage of the cases encountered in the inpatient setting, compose a smaller number of the cases logged in the outpatient clinic. While students are provided with a schedule of suggested faculty with whom to work each half-day, their schedules are flexible to allow for the maximum amount of clinical exposure. In turn, the distribution of cases logged is influenced by numerous factors including patient availability, faculty availability, and student interest. Students are assessed on the final day of their clerkship rotation via the NBME examination and the OSCE. In the OSCE, students collect a patient history, perform a complete neurologic physical examination, and complete a patient note to document their
Table 1 Cases per diagnostic category logged as part of the outpatient component of the 4 week Neurology Clerkship for 486 students Category
Number of cases
Cerebrovascular disease Seizure Dementia Peripheral/neuromuscular Spinal disease, neck/back pain Central nervous system demyelinating disease/neuro-immunology Neuro-oncology Movement disorders Sleep disorders Headaches Other neurodegenerative disorders Other Not listed Total cases across all categories
446 (3.6%) 1305 (10.5%) 217 (1.8%) 1487 (12.0%) 409 (3.3%) 1545 (12.5%) 1901 (15.4%) 932 (7.5%) 489 (3.9%) 1031 (8.3%) 357 (2.9%) 2031 (16.4%) 231 (1.9%) 12,381
clinical formulation for two standardized patients (SP) cases. The OSCE score is made up of history and physical examination checklists completed by trained SP, and the patient note clinical formulation checklist scored by trained faculty. Analysis includes pairwise Pearson correlations for each dependent variable (NBME, OSCE: patient history, OSCE: physical examination, OSCE: patient note) and the two independent variables (volume of each of the 13 diagnostic categories as well as breadth of each of the diagnostic categories). Volume is calculated based on number of patient encounters logged per category; breadth is coded as either one (student logged at least one case in the given category) or zero (student logged no cases in a given category) per category. Multiple linear regression compares the volume and breadth of cases overall with each dependent variable. Students are divided into quartiles of the academic year to evaluate for effects of clinical experience on outcomes. 3. Results There are similar numbers of students present in each quartile of the academic year, and calendar year, p = 1.000 (Table 2). Based on a chi-squared test, there are no significant differences in student logs across quartiles or calendars years (v2 [12] = 1.40, p = 1.000). 3.1. Volume and breadth considering each diagnostic category When considering each of the 13 diagnostic categories, the volume of cases seen per student correlates with performance on the NBME examination (r = .290, p < .001) and the OSCE patient note clinical formulation (r = .238, p = .010). Correlation is also present between volume of cases and the OSCE physical examination (r = .236, p = .011), a finding which was not noted in our original study [1] perhaps because a smaller data set was used. There is no significant correlation between volume of cases and the OSCE patient history component. Again when considering each of the 13 diagnostic categories, breadth of cases is significantly correlated with the NBME examination (r = .231, p = .017), however breadth is not significantly correlated with any component of the OSCE (history, physical examination, patient note) (Table 3). These results indicate that volume (rather than breadth) of cases has a stronger relationship to student performance in objectives measures of performance (NBME, OSCE). 3.2. Volume and breadth overall The total number of cases logged by students, regardless of category, is significantly correlated with the NBME score (r = 0.118, p < .001), the OSCE physical examination (r = 0.97, p = .032), and the OSCE patient note clinical formulation (r = .101, p = .026). The total number of case categories logged by students, regardless of which ones, is also significantly correlated with the NBME score (r = .114, p = .012), however is not significantly correlated with any component of the OSCE. These results indicate that sheer volume of cases regardless of category has a strong relationship to student performance, while breadth of cases helps to some extent with NBME score. Students are therefore benefited by a large volume of cases in total, and even one case per category (rather than none) is helpful (Supp. Table 1, 2). 3.3. Specific case types Interestingly, specific types of cases are significantly related to students’ performance on other measures. For example, the NBME score is significantly related to the volume of cases for spine/neck/ back pain (p = 0.002) and headaches (p = 0.006). This means that
Please cite this article in press as: Albert DV et al. Breadth versus volume: Neurology outpatient clinic cases in medical education. J Clin Neurosci (2016), http://dx.doi.org/10.1016/j.jocn.2015.12.020
3
D.V. Albert et al. / Journal of Clinical Neuroscience xxx (2016) xxx–xxx Table 2 Frequencies – demographic characteristics of students undertaking Neurology Clerkship
Total
Education year
Calendar year
2nd
n=1
3rd
n = 452
4th n = 486
n = 33
2008–2009 2009–2010 2010–2011 2011–2012 2012–2013
Site 90 109 96 99 92
Rotation quartile
UCM
n = 366
NUHS
n = 120
July, August, September October, November, December January, February, March April, May, June
n = 120 n = 126 n = 115 n = 125
NUHS = Northshore University Health System, UCM = University of Chicago Medicine.
Table 3 Regression analysis of volume and breadth of cases, and NBME and OSCE performance, of student undertaking Neurology Clerkship R-squared
F
p value
Volume of cases NBME score OSCE: Patient history OSCE: Physical examination OSCE: Patient note
.084 .027 .056 .057
3.330 .998 2.144 2.179
<.001 .452 .011 .010
Breadth of cases NBME score OSCE: Patient history OSCE: Physical examination OSCE: Patient note
.053 .027 .034 .029
2.039 1.000 1.297 1.094
.017 .450 .211 .362
NBME = National Board of Medical Examiners, OSCE = Objective Structured Clinical Examination.
having more cases in these categories may be related to better NBME performance above and beyond larger volumes in other categories. Students’ OSCE scores on the patient history are associated with volume of cases in cerebrovascular (p = 0.041) and breadth of cases in seizure (p = 0.031). This means that having many cases in the cerebrovascular category is helpful, but that experience with even one seizure case is also related to students’ ability to conduct a patient history. The students’ OSCE scores on the physical examination are related to volume in seizure (p = 0.039) and peripheral/ neuromuscular (p = 0.003) cases, while none of the case types in terms of breadth are significantly helpful for physical examination skills. Students’ OSCE scores on the patient note are related to volume in seizure (p = 0.011) and peripheral/neuromuscular (p < 0.001) cases, while none of the case types in terms of breadth are significantly helpful for the patient note clinical formulation skills. 4. Discussion This study affirms that the volume of cases seen by medical students in the outpatient neurology clinic correlates with performance on objective measures of specialty knowledge and clinical skill, as previously suggested [2,10]. It is possible that our earlier study was underpowered to detect a relationship between the volume of cases and performance on the physical examination component of the OSCE. These findings reinforce the concept that directly observing and interacting with expert clinicians in a patient care setting fosters the development of more universally applicable skills essential to clinical evaluation and management of patients. It may be that top performing students are driven to see more patients, and also study harder and work harder in other ways that improve their shelf examination and OSCE performances. This may contribute to the significant relationship between volume of cases and NBME examination and OSCE performance. Interestingly, fewer significant relationships emerge correlating the breadth of clinical case types seen by individual students and their performance on objective measures. These findings are of
great practical significance to clinical educators who are designing or adjusting Neurology Clerkship activities, as educators may need to make tough choices in how to allocate student learning time. With relatively little evidence that the breadth of cases seen by individual students is significantly influential, Neurology Clerkship programs can consider focusing efforts on developing a wellrounded clinical outpatient learning experience without need of achieving a specific threshold of case types. Allocating student time to work with clinical faculty most interested and/or available to teach the universally applicable clinical skills or with faculty whose clinics have the most appropriate patient volume may be a reasonable expenditure of effort. The relationship between student exposure to specific types of cases and performance on specific objective evaluations may reflect the case types more frequently encountered on the NBME or OSCE. This may also emphasize the benefit from the outpatient exposure complementing the inpatient experience where some specific case types are less likely to be encountered. As specific categories in the outpatient clinic appear to be especially high yield for students, it would be reasonable for them to be routinely included in any outpatient clinic educational plan (spine/neck/back pain, headaches, cerebrovascular, seizure, and peripheral/neuromuscular). The OSCE cases are designed by the two Neurology Clerkship Directors (J.B., R.L.); cases are selected based on systematically sampling of the Clerkship curricula, and ability for SP to simulate positive and negative findings. The case details are not discussed here as these materials are in active use for assessment, however all materials are available but password protected through MedEdPORTAL [8]. Without giving specifics, the authors can communicate that the case categories which appear to be most influential for student performance in this study are not all directly related to OSCE cases, indicating that exposure to particular case types in the outpatient setting provides skills and knowledge which translate to cases in the objective measures, even if those cases are not within the same diagnostic category. While exposure to some subtypes of clinical cases is associated with higher scores on components of the assessments at the completion of the clerkship, we would argue against ‘‘teaching only to the test.” 4.1. Limitations An earlier study evaluating the effect of patient volume in both the inpatient and outpatient setting in a smaller number of students did not find a correlation with performance on internally developed measures of knowledge or faculty evaluations [12]. The difference in methodology may be the likely factor underlying discordance with our results. In our study, while the number of students and logged cases included is large, it is only from one university. Future research in this area conducted at multiple sites would add validity to this study’s findings. In addition, it may be that student exposure to specific case types is linked to cases included in the NBME examination or the OSCE. Finally, it may be that more motivated students log more cases in the outpatient clinic and devote greater effort in preparation for assessments of clinical knowledge and skill.
Please cite this article in press as: Albert DV et al. Breadth versus volume: Neurology outpatient clinic cases in medical education. J Clin Neurosci (2016), http://dx.doi.org/10.1016/j.jocn.2015.12.020
4
D.V. Albert et al. / Journal of Clinical Neuroscience xxx (2016) xxx–xxx
Conflicts of Interest/Disclosures The authors declare that they have no financial or other conflicts of interest in relation to this research and its publication.
[5] [6] [7]
Appendix A. Supplementary material Supplementary data associated with this article can be found, in the online version, at http://dx.doi.org/10.1016/j.jocn.2015.12.020. References
[8]
[9] [10]
[1] Denton GD, Durning SJ. Internal medicine core clerkships experience with core problem lists: results from a national survey of clerkship directors in internal medicine. Teach Learn Med 2009;21:281–3. [2] Albert DV, Brorson JR, Amidei C, et al. Education research: case logs in the assessment of medical students in the neurology outpatient clinic. Neurology 2014;82:e138–41. [3] Rattner SL, Louis DZ, Rabinowitz C, et al. Documenting and comparing medical students’ clinical experiences. JAMA 2001;286:1035–40. [4] Markham FW, Rattner S, Hojat M, et al. Evaluations of medical students’ clinical experiences in a family medicine clerkship: differences in patient
[11]
[12]
encounters by disease severity in different clerkship sites. Fam Med 2002;34:451–4. Wimmers PF, Splinter TA, Hancock GR, et al. Clinical competence: general ability or case-specific? Adv Health Sci Educ Theory Pract 2007;12:299–314. Wimmers PF, Schmidt HG, Splinter TA. Influence of clerkship experiences on clinical competence. Med Educ 2006;40:450–8. Richards MK, McAteer JP, Drake FT, et al. A national review of the frequency of minimally invasive surgery among general surgery residents: assessment of ACGME case logs during 2 decades of general surgery resident training. JAMA Surg 2015;150:169–72. Blood A, Brorson JR, Lukas RV. The Neurology Clerkship Objective Structured Clinical Examination (OSCE). MedEdPORTAL. 2013 [available from: www. mededportal.org/publication/9368]. Lukas RV, Adesoye T, Smith S, et al. Student assessment by objective structured examination in a neurology clerkship. Neurology 2012;79:681–5. Lukas RV, Blood A, Park YS, et al. Assessment of neurological clinical management reasoning in medical students. J Clin Neurosci 2014;2:919–22. Blood AD, Park YS, Lukas RV, et al. Neurology objective structured clinical examination reliability using generalizability theory. Neurology 2015;85:1623–9. http://dx.doi.org/10.1212/WNL.0000000000002053. Poisson SN, Gelb DJ, Oh MS, et al. Experience may not be the best teacher: patient logs do not correlate with clerkship performance. Neurology 2009;72:699–704.
Please cite this article in press as: Albert DV et al. Breadth versus volume: Neurology outpatient clinic cases in medical education. J Clin Neurosci (2016), http://dx.doi.org/10.1016/j.jocn.2015.12.020
Contents lists available at ScienceDirect
Journal of Clinical Neuroscience journal homepage: www.elsevier.com/locate/jocn
Education
Breadth versus volume: Neurology outpatient clinic cases in medical education Dara V. Albert a,⇑, Angela D. Blood b, Yoon Soo Park c, James R. Brorson d, Rimas V. Lukas d a
Nationwide Children’s Hospital, Ohio State University, Division of Neurology, 700 Children’s Drive, Columbus, OH 43205, USA Rush Medical College, Rush University, IL, USA c University of Illinois at Chicago, Department of Medical Education, IL, USA d University of Chicago, Department of Neurology, IL, USA b
a r t i c l e
i n f o
Article history: Received 11 December 2015 Accepted 27 December 2015 Available online xxxx Keywords: Case logs Medical education Neurology clerkship Neurology education
a b s t r a c t This study examined how volume in certain patient case types and breadth across patient case types in the outpatient clinic setting are related to Neurology Clerkship student performance. Case logs from the outpatient clinic experience of 486 students from The University of Chicago Pritzker School of Medicine, USA, participating in the 4 week Neurology Clerkship from July 2008 to June 2013 were reviewed. A total of 12,381 patient encounters were logged and then classified into 13 diagnostic categories. How volume of cases within categories and the breadth of cases across categories relate to the National Board of Medical Examiners Clinical Subject Examination for Neurology and a Neurology Clerkship Objective Structured Clinical Examination was analyzed. Volume of cases was significantly correlated with the National Board of Medical Examiners Clinical Subject Examination for Neurology (r = .290, p < .001), the Objective Structured Clinical Examination physical examination (r = .236, p = .011), and the Objective Structured Clinical Examination patient note (r = .238, p = .010). Breadth of cases was significantly correlated with the National Board of Medical Examiners Clinical Subject Examination for Neurology (r = .231, p = .017), however was not significantly correlated with any component of the Objective Structured Clinical Examination. Volume of cases correlated with higher performance on measures of specialty knowledge and clinical skill. Fewer relationships emerged correlating breadth of cases and performance on the same measures. This study provides guidance to educators who must decide how much emphasis to place on volume versus breadth of cases in outpatient clinic learning experiences. Ó 2016 Elsevier Ltd. All rights reserved.
1. Introduction The USA Liaison Committee on Medical Education requires students to document exposure to a range of clinical cases during clerkships [1]. While the number of cases, or volume, logged during an outpatient clinical experience correlates with performance on objective measures of knowledge and clinical skill [2], studies have not examined the relationship between breadth of cases and performance. Prior studies within specific medical specialties examine the case-mix encountered on clerkships, however they focus primarily on evaluating differences in types of cases seen and not their contribution to medical acumen [3,4]. Studies across specialties demonstrate that domain-specific knowledge and ⇑ Corresponding author. Tel.: +1 614 722 4916; fax: +1 614 722 4633. E-mail address: [email protected] (D.V. Albert).
general problem solving skills are necessary for clinical competence [5], and across specialties variety of cases and quality of supervision may influence acumen [6]. Although case logs requiring a broad clinical exposure are de riguer for surgical residency training [7], little has been published on the role of breadth of exposure for non-surgical specialties, and its impact on clinical acumen. This study hypothesized that the volume and breadth of cases encountered by students within the outpatient clinic component of a Neurology Clerkship would correlate positively with performance on measures of knowledge and clinical skill. The performance measures for this study include the National Board of Medical Examiners (NBME) Clinical Subject Examination for Neurology as an indicator of knowledge, and the Objective Structured Clinical Examination (OSCE) as an indicator of clinical skill. The role of the OSCE in assessment of students in a Neurology Clerkship has been previously reported [8–11].
http://dx.doi.org/10.1016/j.jocn.2015.12.020 0967-5868/Ó 2016 Elsevier Ltd. All rights reserved.
Please cite this article in press as: Albert DV et al. Breadth versus volume: Neurology outpatient clinic cases in medical education. J Clin Neurosci (2016), http://dx.doi.org/10.1016/j.jocn.2015.12.020
2
D.V. Albert et al. / Journal of Clinical Neuroscience xxx (2016) xxx–xxx
2. Methods The University of Chicago Institutional Review Board deemed this study exempt from review. Case logs are from the outpatient component of the required 4 week Neurology Clerkship for 486 students from July 2008 through June 2013, totaling 12,381 patient encounters at The University of Chicago Pritzker School of Medicine. The majority of students are third years (n = 452, 93%), with one second year student (n = 1, 0.2%), and the remainder being fourth years (n = 33, 7%). The outpatient clinic experience occurs at two sites, The University of Chicago Medicine and Northshore University Health System a community-based teaching affiliate. Students’ preferences are solicited when assigning students to sites, either University of Chicago Medicine (n = 366, 75%) or Northshore University Health System (n = 120, 25%). The outpatient clinic component of the clerkship is approximately 1 week in length. It encompasses adult, pediatric, general and subspecialty clinics. Degree of independence in patient evaluation varied from shadowing of faculty to independently taking the history, examining the patient, and presenting the assessment and management plan. This variability was not static and was influenced by a number of factors including the pace of the outpatient clinic, time available for teaching and motivation of the student. Students log each outpatient clinic encounter with the primary diagnosis or chief complaint. Students who did not log any cases (n = 9, 1.8%) or did not include diagnoses/complaints (n = 1, 0.2%) are not included in the analyses. The cases were retrospectively classified into 13 categories: seizure, cerebrovascular disease, peripheral/neuromuscular, spinal disease/neck/back pain, central nervous system demyelinating disease/neuro-immunology, headaches, sleep disorders, neuro-oncology, movement disorders, dementia, other neurodegenerative disorders, other, and not listed (Table 1). When more than one diagnosis or chief complaint is listed, the primary one is used in the analysis. Some diagnoses, such as cerebrovascular, which form a substantial percentage of the cases encountered in the inpatient setting, compose a smaller number of the cases logged in the outpatient clinic. While students are provided with a schedule of suggested faculty with whom to work each half-day, their schedules are flexible to allow for the maximum amount of clinical exposure. In turn, the distribution of cases logged is influenced by numerous factors including patient availability, faculty availability, and student interest. Students are assessed on the final day of their clerkship rotation via the NBME examination and the OSCE. In the OSCE, students collect a patient history, perform a complete neurologic physical examination, and complete a patient note to document their
Table 1 Cases per diagnostic category logged as part of the outpatient component of the 4 week Neurology Clerkship for 486 students Category
Number of cases
Cerebrovascular disease Seizure Dementia Peripheral/neuromuscular Spinal disease, neck/back pain Central nervous system demyelinating disease/neuro-immunology Neuro-oncology Movement disorders Sleep disorders Headaches Other neurodegenerative disorders Other Not listed Total cases across all categories
446 (3.6%) 1305 (10.5%) 217 (1.8%) 1487 (12.0%) 409 (3.3%) 1545 (12.5%) 1901 (15.4%) 932 (7.5%) 489 (3.9%) 1031 (8.3%) 357 (2.9%) 2031 (16.4%) 231 (1.9%) 12,381
clinical formulation for two standardized patients (SP) cases. The OSCE score is made up of history and physical examination checklists completed by trained SP, and the patient note clinical formulation checklist scored by trained faculty. Analysis includes pairwise Pearson correlations for each dependent variable (NBME, OSCE: patient history, OSCE: physical examination, OSCE: patient note) and the two independent variables (volume of each of the 13 diagnostic categories as well as breadth of each of the diagnostic categories). Volume is calculated based on number of patient encounters logged per category; breadth is coded as either one (student logged at least one case in the given category) or zero (student logged no cases in a given category) per category. Multiple linear regression compares the volume and breadth of cases overall with each dependent variable. Students are divided into quartiles of the academic year to evaluate for effects of clinical experience on outcomes. 3. Results There are similar numbers of students present in each quartile of the academic year, and calendar year, p = 1.000 (Table 2). Based on a chi-squared test, there are no significant differences in student logs across quartiles or calendars years (v2 [12] = 1.40, p = 1.000). 3.1. Volume and breadth considering each diagnostic category When considering each of the 13 diagnostic categories, the volume of cases seen per student correlates with performance on the NBME examination (r = .290, p < .001) and the OSCE patient note clinical formulation (r = .238, p = .010). Correlation is also present between volume of cases and the OSCE physical examination (r = .236, p = .011), a finding which was not noted in our original study [1] perhaps because a smaller data set was used. There is no significant correlation between volume of cases and the OSCE patient history component. Again when considering each of the 13 diagnostic categories, breadth of cases is significantly correlated with the NBME examination (r = .231, p = .017), however breadth is not significantly correlated with any component of the OSCE (history, physical examination, patient note) (Table 3). These results indicate that volume (rather than breadth) of cases has a stronger relationship to student performance in objectives measures of performance (NBME, OSCE). 3.2. Volume and breadth overall The total number of cases logged by students, regardless of category, is significantly correlated with the NBME score (r = 0.118, p < .001), the OSCE physical examination (r = 0.97, p = .032), and the OSCE patient note clinical formulation (r = .101, p = .026). The total number of case categories logged by students, regardless of which ones, is also significantly correlated with the NBME score (r = .114, p = .012), however is not significantly correlated with any component of the OSCE. These results indicate that sheer volume of cases regardless of category has a strong relationship to student performance, while breadth of cases helps to some extent with NBME score. Students are therefore benefited by a large volume of cases in total, and even one case per category (rather than none) is helpful (Supp. Table 1, 2). 3.3. Specific case types Interestingly, specific types of cases are significantly related to students’ performance on other measures. For example, the NBME score is significantly related to the volume of cases for spine/neck/ back pain (p = 0.002) and headaches (p = 0.006). This means that
Please cite this article in press as: Albert DV et al. Breadth versus volume: Neurology outpatient clinic cases in medical education. J Clin Neurosci (2016), http://dx.doi.org/10.1016/j.jocn.2015.12.020
3
D.V. Albert et al. / Journal of Clinical Neuroscience xxx (2016) xxx–xxx Table 2 Frequencies – demographic characteristics of students undertaking Neurology Clerkship
Total
Education year
Calendar year
2nd
n=1
3rd
n = 452
4th n = 486
n = 33
2008–2009 2009–2010 2010–2011 2011–2012 2012–2013
Site 90 109 96 99 92
Rotation quartile
UCM
n = 366
NUHS
n = 120
July, August, September October, November, December January, February, March April, May, June
n = 120 n = 126 n = 115 n = 125
NUHS = Northshore University Health System, UCM = University of Chicago Medicine.
Table 3 Regression analysis of volume and breadth of cases, and NBME and OSCE performance, of student undertaking Neurology Clerkship R-squared
F
p value
Volume of cases NBME score OSCE: Patient history OSCE: Physical examination OSCE: Patient note
.084 .027 .056 .057
3.330 .998 2.144 2.179
<.001 .452 .011 .010
Breadth of cases NBME score OSCE: Patient history OSCE: Physical examination OSCE: Patient note
.053 .027 .034 .029
2.039 1.000 1.297 1.094
.017 .450 .211 .362
NBME = National Board of Medical Examiners, OSCE = Objective Structured Clinical Examination.
having more cases in these categories may be related to better NBME performance above and beyond larger volumes in other categories. Students’ OSCE scores on the patient history are associated with volume of cases in cerebrovascular (p = 0.041) and breadth of cases in seizure (p = 0.031). This means that having many cases in the cerebrovascular category is helpful, but that experience with even one seizure case is also related to students’ ability to conduct a patient history. The students’ OSCE scores on the physical examination are related to volume in seizure (p = 0.039) and peripheral/ neuromuscular (p = 0.003) cases, while none of the case types in terms of breadth are significantly helpful for physical examination skills. Students’ OSCE scores on the patient note are related to volume in seizure (p = 0.011) and peripheral/neuromuscular (p < 0.001) cases, while none of the case types in terms of breadth are significantly helpful for the patient note clinical formulation skills. 4. Discussion This study affirms that the volume of cases seen by medical students in the outpatient neurology clinic correlates with performance on objective measures of specialty knowledge and clinical skill, as previously suggested [2,10]. It is possible that our earlier study was underpowered to detect a relationship between the volume of cases and performance on the physical examination component of the OSCE. These findings reinforce the concept that directly observing and interacting with expert clinicians in a patient care setting fosters the development of more universally applicable skills essential to clinical evaluation and management of patients. It may be that top performing students are driven to see more patients, and also study harder and work harder in other ways that improve their shelf examination and OSCE performances. This may contribute to the significant relationship between volume of cases and NBME examination and OSCE performance. Interestingly, fewer significant relationships emerge correlating the breadth of clinical case types seen by individual students and their performance on objective measures. These findings are of
great practical significance to clinical educators who are designing or adjusting Neurology Clerkship activities, as educators may need to make tough choices in how to allocate student learning time. With relatively little evidence that the breadth of cases seen by individual students is significantly influential, Neurology Clerkship programs can consider focusing efforts on developing a wellrounded clinical outpatient learning experience without need of achieving a specific threshold of case types. Allocating student time to work with clinical faculty most interested and/or available to teach the universally applicable clinical skills or with faculty whose clinics have the most appropriate patient volume may be a reasonable expenditure of effort. The relationship between student exposure to specific types of cases and performance on specific objective evaluations may reflect the case types more frequently encountered on the NBME or OSCE. This may also emphasize the benefit from the outpatient exposure complementing the inpatient experience where some specific case types are less likely to be encountered. As specific categories in the outpatient clinic appear to be especially high yield for students, it would be reasonable for them to be routinely included in any outpatient clinic educational plan (spine/neck/back pain, headaches, cerebrovascular, seizure, and peripheral/neuromuscular). The OSCE cases are designed by the two Neurology Clerkship Directors (J.B., R.L.); cases are selected based on systematically sampling of the Clerkship curricula, and ability for SP to simulate positive and negative findings. The case details are not discussed here as these materials are in active use for assessment, however all materials are available but password protected through MedEdPORTAL [8]. Without giving specifics, the authors can communicate that the case categories which appear to be most influential for student performance in this study are not all directly related to OSCE cases, indicating that exposure to particular case types in the outpatient setting provides skills and knowledge which translate to cases in the objective measures, even if those cases are not within the same diagnostic category. While exposure to some subtypes of clinical cases is associated with higher scores on components of the assessments at the completion of the clerkship, we would argue against ‘‘teaching only to the test.” 4.1. Limitations An earlier study evaluating the effect of patient volume in both the inpatient and outpatient setting in a smaller number of students did not find a correlation with performance on internally developed measures of knowledge or faculty evaluations [12]. The difference in methodology may be the likely factor underlying discordance with our results. In our study, while the number of students and logged cases included is large, it is only from one university. Future research in this area conducted at multiple sites would add validity to this study’s findings. In addition, it may be that student exposure to specific case types is linked to cases included in the NBME examination or the OSCE. Finally, it may be that more motivated students log more cases in the outpatient clinic and devote greater effort in preparation for assessments of clinical knowledge and skill.
Please cite this article in press as: Albert DV et al. Breadth versus volume: Neurology outpatient clinic cases in medical education. J Clin Neurosci (2016), http://dx.doi.org/10.1016/j.jocn.2015.12.020
4
D.V. Albert et al. / Journal of Clinical Neuroscience xxx (2016) xxx–xxx
Conflicts of Interest/Disclosures The authors declare that they have no financial or other conflicts of interest in relation to this research and its publication.
[5] [6] [7]
Appendix A. Supplementary material Supplementary data associated with this article can be found, in the online version, at http://dx.doi.org/10.1016/j.jocn.2015.12.020. References
[8]
[9] [10]
[1] Denton GD, Durning SJ. Internal medicine core clerkships experience with core problem lists: results from a national survey of clerkship directors in internal medicine. Teach Learn Med 2009;21:281–3. [2] Albert DV, Brorson JR, Amidei C, et al. Education research: case logs in the assessment of medical students in the neurology outpatient clinic. Neurology 2014;82:e138–41. [3] Rattner SL, Louis DZ, Rabinowitz C, et al. Documenting and comparing medical students’ clinical experiences. JAMA 2001;286:1035–40. [4] Markham FW, Rattner S, Hojat M, et al. Evaluations of medical students’ clinical experiences in a family medicine clerkship: differences in patient
[11]
[12]
encounters by disease severity in different clerkship sites. Fam Med 2002;34:451–4. Wimmers PF, Splinter TA, Hancock GR, et al. Clinical competence: general ability or case-specific? Adv Health Sci Educ Theory Pract 2007;12:299–314. Wimmers PF, Schmidt HG, Splinter TA. Influence of clerkship experiences on clinical competence. Med Educ 2006;40:450–8. Richards MK, McAteer JP, Drake FT, et al. A national review of the frequency of minimally invasive surgery among general surgery residents: assessment of ACGME case logs during 2 decades of general surgery resident training. JAMA Surg 2015;150:169–72. Blood A, Brorson JR, Lukas RV. The Neurology Clerkship Objective Structured Clinical Examination (OSCE). MedEdPORTAL. 2013 [available from: www. mededportal.org/publication/9368]. Lukas RV, Adesoye T, Smith S, et al. Student assessment by objective structured examination in a neurology clerkship. Neurology 2012;79:681–5. Lukas RV, Blood A, Park YS, et al. Assessment of neurological clinical management reasoning in medical students. J Clin Neurosci 2014;2:919–22. Blood AD, Park YS, Lukas RV, et al. Neurology objective structured clinical examination reliability using generalizability theory. Neurology 2015;85:1623–9. http://dx.doi.org/10.1212/WNL.0000000000002053. Poisson SN, Gelb DJ, Oh MS, et al. Experience may not be the best teacher: patient logs do not correlate with clerkship performance. Neurology 2009;72:699–704.
Please cite this article in press as: Albert DV et al. Breadth versus volume: Neurology outpatient clinic cases in medical education. J Clin Neurosci (2016), http://dx.doi.org/10.1016/j.jocn.2015.12.020