- Email: [email protected]
Associations of medical student empathy with clinical competence
Accepted Manuscript Title: Associations of medical student empathy with clinical competence Author: Rachel S. Casas Ziming Xuan Angela H. Jackson Lorraine E. Stanfield Nanette C. Harvey Daniel C. Chen PII: DOI: Reference:
S0738-3991(16)30511-0 http://dx.doi.org/doi:10.1016/j.pec.2016.11.006 PEC 5502
To appear in:
Patient Education and Counseling
Received date: Revised date: Accepted date:
21-6-2016 23-9-2016 9-11-2016
Please cite this article as: Casas Rachel S, Xuan Ziming, Jackson Angela H, Stanfield Lorraine E, Harvey Nanette C, Chen Daniel C.Associations of medical student empathy with clinical competence.Patient Education and Counseling http://dx.doi.org/10.1016/j.pec.2016.11.006 This is a PDF file of an unedited manuscript that has been accepted for publication. As a service to our customers we are providing this early version of the manuscript. The manuscript will undergo copyediting, typesetting, and review of the resulting proof before it is published in its final form. Please note that during the production process errors may be discovered which could affect the content, and all legal disclaimers that apply to the journal pertain.
Title Page
Title: Associations of Medical Student Empathy with Clinical Competence
Authors: Rachel S. Casas, MD, EdM1, Ziming Xuan ScD, SM, MA2, Angela H. Jackson, MD1, Lorraine E. Stanfield, MD1, Nanette C. Harvey, MD3, Daniel C. Chen, MD, MSc1
Affiliations: 1Evans Department of Medicine, Boston University School of Medicine, Boston, Massachusetts, USA; 2Department of Community Health Sciences, Boston University, Boston, Massachusetts, USA; 3Department of Family Medicine, Boston University School of Medicine, Boston, Massachusetts, USA
Corresponding Author: Rachel S. Casas, MD, EdM 801 Massachusetts Avenue, Crosstown 2, Boston, MA 02118, USA Email: [email protected] Phone: (617) 414-5951 Fax: (617) 638-7472 Author emails: Ziming Xuan: [email protected] Angela H. Jackson: [email protected] Lorraine E. Stanfield: [email protected] Nanette C. Harvey: [email protected] Daniel C. Chen: [email protected]
1
Highlights
Empathy predicted communication scores on Objective Structured Clinical Examinations (OSCEs)
Associations between empathy and overall OSCE scores were confounded by gender and age
Empathy was positively associated with the Pediatrics clerkship score, but not other clerkships
Self-reported empathy did not predict United States Medical Licensing Exam scores
Abstract Objective: Empathy is a crucial skill for medical students that can be difficult to evaluate. We examined if self-reported empathy in medical students was associated with clinical competence. Methods: This study combined cross-sectional data from four consecutive years of medical students (N=590) from the Boston University School of Medicine. We used regression analysis to evaluate if self-reported empathy (Jefferson Scale of Physician Empathy (JSPE)) predicted scores in clinical clerkships, United States Medical Licensing Examinations, and Objective Structured Clinical Examinations (OSCEs). We separately analyzed overall and OSCE communication scores based on interpersonal skills reported by standardized patients. We controlled for age, gender, debt, and specialty affinity. 2
Results: JSPE scores of medical students were positively associated with OSCE communication scores, and remained significant when controlling for demographics. We found that JSPE score was also predictive of overall OSCE scores, but this relationship was confounded by gender and age. JSPE scores were associated with performance in the Pediatrics clerkship, but not other clerkships or standardized tests. Conclusion: JSPE scores were positively associated with OSCE communication scores in medical students. Practice Implications: This study supports that self-reported empathy may predict OSCE performance, but further research is needed to examine differences by gender and age. Key words: medical education-undergraduate, medical educationassessment/evaluation, clinical skills assessment, communication skills
1. Introduction Empathy is a crucial component of physician-patient interactions and has been associated with improved patient compliance, satisfaction, and clinical outcomes and with decreased provider burn-out and litigation [1-7]. Consequently, the Association of American Medical Colleges (AAMC) has outlined interpersonal and communication skills as core competencies for medical students [8, 9]. Studies suggest that education interventions may be effective in teaching empathy to medical students, but empathy remains difficult to evaluate [10]. 3
Self-reported empathy tools have been developed for education research, but the predictive value of these tools for future empathic behavior remains controversial. This study utilizes one of the most widely used self-reported empathy tools, the Jefferson Scale of Physician Empathy – Student Version (JSPE), which was designed to address patient care situations and has established validity and reliability in medical students [7, 11-14]. The JSPE was designed to measure “cognitive” empathy (the ability to understand another’s experience without a personal emotional response) as opposed to “affective” empathy (the passive emotional response to another’s emotions) [15, 16]. While self-reported empathy measures, including the JSPE, have sufficient evidence to support their use in education research, they may lack validity for use in medical student admissions or evaluation [17]. Consequently, it is important to clarify the relationship between self-, standardized patient-, and faculty-reported empathy to determine if tools like the JSPE can accurately predict student behavior in clinical settings [18]. Current measures of clinical competence in medical students vary in their coverage of empathy. In the United States, these evaluations include Objective Structured Clinical Examinations (OSCEs), clinical clerkships, and the United States Medical Licensing Examinations (USMLEs). Core clerkship grading measures vary by institution, but are generally composed of direct observation of clinical performance by supervising faculty and a multiple-choice knowledge based subject examination from the National Board of Medical Examiners (NBME). OSCEs also vary by institution, and consist of multiple simulated patient scenarios where clinical
4
skills are evaluated by standardized patients. The USMLEs are standardized examinations which include three components during medical school: Step 1, Step 2 Clinical Knowledge (CK), and Step 2 Clinical Skills (CS). Step 1 and Step 2 CK are multiple choice exams which assess mainly medical knowledge while Step 2 CS uses standardized patients to assess patient-centered skills in clinical scenarios. Existing studies show mixed associations between self-reported empathy and measures of clinical competence. For example, while self-reported empathy has been associated with communication scores on OSCEs, associations with the overall OSCE score and clinical clerkship grades are conflicting [19-23]. Studies examining multiple measures of clinical competence in the same cohort of students are limited. One study demonstrated a positive association between self-reported empathy and clinical competence in core clinical clerkships but not standardized tests (USMLE Step 1 and Step 2 CK) [24]. To our knowledge, no existing studies have examined self-reported empathy and USMLE Step 2 CS. Because of the heterogeneity of local OSCE evaluations at institutions throughout the United States, comparisons between empathy measures and Step 2 CS will be valuable for clarifying the relationship between self-reported and observed empathy in medical students. There are limited studies examining self-reported empathy and clinical competence in students of differing genders. Prior analyses have shown consistently higher selfreported empathy scores in women relative to men, and mixed results by age [11, 12, 22, 23, 25-29]. One existing study found that women had higher self-reported empathy scores compared to men but found no association between gender and 5
core clerkships scores, suggesting that gender did not confound the relationship between these variables [24]. This study, however, did not include OSCE scores. Empathy scores of women by standardized patients may be higher than men, and OSCE physical exam scores may differ by student gender [30-32]. Another study of JSPE, core clerkship grades, and gender found differing trajectories of empathy scores by gender, but did not report associations between gender and core clerkship scores [22]. Potential confounders are important to verify, as the predictive use of self-reported empathy for clinical competence may differ by student population. Based upon these studies, we expect gender to potentially confound the associations between empathy and OSCE scores, but not clinical clerkship or standardized test scores. Our study clarifies existing research by examining the associations between selfreported empathy and all currently available institutional and national measures of clinical competence in the same cohort of medical students. Because empathy is an important component of interpersonal interaction, we hypothesized that examinations which assess these skills (clinical clerkships, OSCEs, and USMLE Step 2 CS) would be positively associated with self-reported empathy scores. We hypothesized that standardized examinations which do not assess interpersonal skills (USMLE Step 1 and 2CK) would not be associated with self-reported empathy scores.
2. Methods
6
This is a study of cross-sectional populations of third year medical students from four consecutive years enrolled at the Boston University School of Medicine (BUSM) between 2007 and 2010. The protocol for this study was reviewed by the Boston University Medical Center Institutional Review Board and determined exempt.
2.1 Study Population All medical students enrolled at BUSM (N=1,162) between 2006 and 2010 were eligible to participate in empathy surveys through previously outlined methods [21, 23, 28]. While students from all years of medical school were surveyed, we limited this analysis to third year students because the core clerkships, OSCEs, and USMLE Step 2 exams were completed in this year. The BUSM includes two years of preclinical study with sixteen weeks of interviewing and physical diagnosis workshops, followed by two years of clinical clerkships.
2.2 Data collection We recruited participants to complete self-administered, anonymous surveys at the end of the third year of medical school from 2006 to 2010 [21, 23, 28]. Surveys prior to 2007 were excluded because de-identified empathy data could not be linked to individual measures of clinical competence. We surveyed students about demographic information (gender, age, anticipated financial debt, and future specialty affinity) and self-reported empathy using the JSPE - Student Version. An anonymous dataset of clinical competence examination scores linked to empathy and demographic data was 7
provided to the study authors by the Boston University Information and Technology Database.
2.3 Measurement Variables 2.3.1 Demographics: Likelihood of choosing a specialty as a future career was ranked on a five point Likert scale (very unlikely = 1… neutral = 3… very likely =5). Specialties were grouped as “people-oriented” (Internal Medicine, Family Medicine, Obstetrics/Gynecology, Psychiatry, Pediatrics, Neurology, Rehabilitation Medicine, Emergency Medicine, Ophthalmology, Dermatology) or “technology-oriented” (Radiology, Pathology, Surgery and Surgical Subspecialties, Radiation Oncology, Anesthesiology) based upon prior groupings in medical education research and evidence of higher empathy in students with “people-oriented” compared to “technology-oriented” specialty affinities using the JSPE [11, 33]. Two dummy variables were created for regression analyses, with each student assigned a value of 1 for “people-oriented” if they indicated they were “very likely” to choose a career in a people-oriented specialty, 0 otherwise. Similarly, each student was assigned a value of 1 for “technology-oriented” if they indicated that they were “very likely to choose a career in a technology-oriented specialty, 0 otherwise. Projected debt by the end of medical school included seven ordinal categories (0$25,000 to >$200,000) and was dichotomized into higher (> $150,000) and lower (≤ $150,000) debt to approximate the median debt ($160,000) of medical school graduates in the United States in 2010 [34].
8
2.3.2 Self-reported empathy: The JSPE is an internationally utilized, reliable, and validated survey containing 20 questions on a seven point Likert scale with higher scores (range 20 to 140) corresponding to higher empathy [7, 11-14]. In prior studies, mean JSPE scores of medical students ranged from 80 to 122 (standard deviation 9.9 to 11.2) [19, 22, 24, 28].
2.3.3 Clinical competence: Clinical competence measures included scores from third year OSCEs, clinical clerkships, and standardized tests (USMLE Step 1, 2 CK, and 2 CS). During the OSCE at the end of third year at BUSM, students complete simulated clinical encounters which are evaluated by trained standardized patients. The OSCE consisted of six clinical cases which were scored using checklists in four content areas: communication, history taking, physical exam skills, and documentation. The communication score was based on 14 items about rapport building, questioning technique, and interview organization. Each content area was graded as “met standard” for scores higher than two standard deviations below the mean, otherwise the student “did not meet standard”. For the overall OSCE score, students “marginally met standard” if they failed one content area and “did not meet standard” if they failed two content areas. Given the small number of students with marginal and failing performance, these two categories were combined for analysis. We compared JSPE scores to both the overall OSCE score and the communication score sub-component.
9
We included final grades from the eight core clerkships (Internal Medicine, Family Medicine, Pediatrics, Obstetrics/Gynecology, Surgery, Psychiatry, Radiology, and Neurology) at BUSM in this analysis. Clerkships grades (honors, high pass, pass, marginal pass, fail, incomplete) were based upon faculty clinical evaluations and the National Board of Medical Examiners (NBME) Subject Exams. We divided students into categories of achieving “honors” compared to all other combined grades as the scores overall were skewed with few students receiving less than passing grades. For the standardized tests, the USMLE Step 1 and Step 2 CK were continuous scores (potential ranges from 1 to 300). USMLE STEP 2 CS was graded Pass/Fail.
2.4 Data Analysis Demographics were descriptively summarized. We used logistic and linear regression analyses to evaluate JSPE score as a predictor for binary outcomes (OSCEs, core clerkships, and USMLE Step 2 CS) and continuous outcomes (USMLE Step 1 and Step 2 CK) respectively. We also completed bivariate regression analyses with demographics (age, gender, specialty affinity, and debt) as predictors for JSPE scores and clinical competence scores. To evaluate for confounding, we conducted multivariate regression analyses which included demographic variables (age, gender, specialty affinity, and debt). Statistical significance was considered at p-value < 0.05. Data analysis was completed using SAS 9.3 (SAS Institute, Cary NC).
10
3. Results The average survey response rate was 81%. JSPE data was available for 603 third year students. We excluded 13 students due to missing JSPE data, leaving 590 students for analysis.
3.1 Demographics: The majority of students were female (54%) and reported high debt (63%) (Table 1). [Insert Table 1] Age, gender, and having a “technology-oriented” specialty affinity were predictive of overall OSCE scores in bivariate analyses. Students who were women, younger, or had a “technology-oriented” specialty affinity were more likely to pass the overall OSCE (Beta=1.2, p=0.0005 for gender; Beta=-0.10, p=0.02 for age; Beta= -1.4 p=0.0003 for “technology-oriented” affinity). No demographic variables were predictive of OSCE communication, clinical clerkship, or standardized test scores.
3.2 JSPE scores: Median JSPE score (118, IQR=7) corresponded with the expected range from prior literature [10, 13, 17, 22]. In bivariate regression analyses, gender and age were predictive of JSPE scores (Beta=7.2, p<0.0001 for gender; Beta=0.5, p=0.015 for age). Median (IQR) JSPE scores were higher in women (121 (14)) compared to men (113 (19)) and in older (119 (16)) compared to younger (116 (18)) students (t=-6.8,
11
p <0.0001 and t= 2.0, p=0.04 respectively). Debt and specialty affinity did not predict JSPE score.
3.3 OSCE Communication Scores and JSPE Five hundred and eighty students met the standard and 10 did not meet the standard for the OSCE communication score. Median (IQR) JSPE scores were 118 (18) for students who met the standard and 97 (18) for students who did not meet the standard (t=-3.5, p = 0.0005). JSPE scores were significantly predictive of the OSCE communication score in simple and multiple regression analyses (Table 2). In the adjusted model, every one point increase in JSPE score was associated with a 6.3% increase in the odds of meeting the standard for the OSCE communication score (AOR=1.06 95% CI: 1.02, 1.10). For example, a student with a JSPE score of 130 (about one standard deviation above the mean) would have an approximately 189% increased odds of passing the overall OSCE compared to a student with a JSPE score of 100 (about one standard deviation below the mean). [Insert Table 2]
3.4 OSCE Overall Score and JSPE Five hundred and fifty-two students met the standard and 38 did not meet the standard for the OSCE overall score. Of these 38 students, 10 failed both the communication 12
section and overall OSCE. Students who met the standard had median (IQR) JSPE score 118 (18) and students who did not meet the standard had median (IQR) JSPE score 114 (21) (t=-2.5; p = 0.015). JSPE score was significantly predictive of OSCE overall scores in bivariate but not multivariate regression analyses controlling for all demographics (Table 2). In multivariate analyses modeling empathy and overall OSCE with the adjustment of gender, age, and “technology-oriented” affinity separately, the beta coefficient for empathy changed by 10% or greater for gender and age only.
3.5 Clinical Clerkships and JSPE The number of students receiving honors ranged from 145 (Obstetrics/Gynecology) to 384 (Psychiatry). Figure 1 shows the adjusted odds ratios for JSPE scores and honoring each of the eight core clerkships, of which Pediatrics only was significant. [Insert Figure 1] A one point increase in JSPE score was associated with a 1.7% increased odds of honoring Pediatrics when adjusting for demographics (Figure 1) (AOR 1.02 95% CI: 1.00, 1.03). For example, a student with a JSPE score of 130 would have an approximately 51% increased odds of honoring Pediatrics compared to a student with a JSPE score of 100.
3.6 Standardized Tests and JSPE 13
Students had median (IQR) USMLE Step 1 score 228 (29) and Step 2 CK score of 236 (29). For every 1 point increase in empathy score, there was a 0.2 point decrease in USMLE 1 score (p=0.011) in simple but not multiple regression analysis (data not shown). There were no significant relationships between JSPE and Step 2 CK scores or JSPE and Step 2 CS scores (N=6 students failing, N=584 students passing).
3.7 Sensitivity Analysis Graphed JSPE data showed a bimodal peak at a score of 80, which corresponded to choosing the neutral Likert category for all questions and may reflect response bias. We conducted sensitivity analyses by excluding students with JSPE scores of 80 (N=10) and noted no change in reported relationships.
4. Discussion and Conclusion 4.1 Discussion We demonstrated that self-reported empathy scores were associated with OSCE communication scores, OSCE overall scores, and the Pediatrics clerkship, but not standardized examinations. The associations between empathy and overall OSCE scores, however, were confounded by age and gender. This study adds to existing literature by examining associations of self-reported empathy with all currently available measures of clinical competence in the same group of students. Self-reported empathy may predict observed performance on OSCEs in medical students, but
14
these associations may be limited to communication scores and differ in students by age and gender. Empathy scores were predictive of OSCE communication scores, but not overall OSCE scores, when accounting for demographic variables. Similarly, existing literature supports that self-reported empathy may be more highly predictive of the communication component of the OSCE, but not the physical exam component or overall scores [19, 20]. For example, empathic concern measured through the Davis’ Interpersonal Reactivity Index (IRI) was positively correlated with communication skills during OSCEs in 166 third year students from the United States, but not with physical exam skills [20]. While this study used a different self-reported empathy tool, the IRI has been shown to have a statistically significant correlation of moderate magnitude with the JSPE [35]. In another study, JSPE scores were not associated with overall OSCE scores in 57 Australian third year students [19]. Other studies have also found that observed but not self-reported empathy scores were associated with overall OSCE performance, with mixed trends in empathy scores as student progress through medical school [19, 21, 29, 36]. In these comparisons, self-reported and observed empathy scores may reflect differing aspects of affective and cognitive empathy. While our study showed that the JSPE was associated with higher communication scores on OSCEs, it is unclear if this relationship predicts more empathic behavior with real patients. In one previous study, empathy ratings of clinical performance with real patients related to empathy ratings in standardized patients [37]. Selfreported empathy, however, may not meaningfully reflect interactions between 15
providers and patients and it is uncertain if behaving empathically without shared emotion impacts patient care [10, 14, 30]. Educational interventions can enhance empathic behavior in medical students, but these students may develop a protective distancing of their own emotional responses with clinical experience [10]. For example, in a qualitative study from Japan, medical students viewed sharing patients’ feelings as an important component of empathy (affective empathy), but could decide to act empathically during OSCEs regardless of shared emotion (cognitive empathy) [38]. Further comparisons of self-, faculty-, standardized patient-, and real patient-reported empathy measures are needed to determine if we are accurately predicting empathic behavior in medical students and if the potential incongruity between observed and self-reported empathy impacts patient care. Our results suggest that the relationship between self-reported empathy and overall OSCE scores were confounded by gender and age. Women had higher self-reported empathy scores and were more likely to pass the overall OSCE compared to men. Interestingly, older students had higher empathy scores but were less likely to pass the overall OSCE. Technology-oriented specialty affinity was also predictive of passing the OSCE, but did not relate to empathy scores. In regards to gender, confounding is supported by a prior study showing higher selfreported and observed empathy by standardized patients in women compared to men [39]. We expected both the communication component and overall OSCE to be confounded by gender based upon prior studies showing that both observed empathy and physical exam scores during OSCEs differ by student gender [30-32]. To our knowledge, existing studies of self-reported empathy and OSCE scores did not report
16
comparisons by age. We would expect older students, with more life experience, to be better equipped to self-assess and display empathic concern. The reasons for gender confounding and lower pass rates of older students for the overall OSCE in our study will require further qualitative exploration. We did not detect an association between empathy and clinical clerkship grades, with the exception of Pediatrics. One possible explanation for this lack of association was that 25 to 35% of each core clerkship grade at BUSM consisted of NBME Subject Exam scores, which could not be separated from the faculty clinical evaluations for analysis. At many medical schools, including BUSM, faculty receive limited training in clerkship evaluation with locally developed measures, leading to potentially limited reliability and validity. In comparison, at Jefferson Medical College in the United States, higher JSPE scores from 371 third year medical students were associated with more honors grades in core clerkships [11]. In another study from the United Kingdom, JSPE scores were not associated with measures of overall academic performance [22]. Differing grading and training systems may contribute to the variability in relationships between empathy and core clerkships. At BUSM, we are in the process of revising tools and educating faculty to increase standardization of evaluations. We found that self-reported empathy was not associated with standardized test scores. These results are in agreement with one existing study from 2002 which found no associations between JSPE and the USMLE Step 1 or Step 2 CK exams [24]. USMLE Step 2 CS was implemented following this study in 2004. Given that Step 2 CS is designed to assess communication skills, we expected to find a significant relationship 17
between empathy and Step 2 CS, but did not. Conclusions regarding Step 2 CS from this analysis are limited, however, because failure of this exam was an uncommon outcome. Repeat analyses using a larger data set are warranted to evaluate these relationships. A strength of this study was that it included data from all major, current assessments of clinical competence of medical students in the United States. The data were from a large population over four consecutive years with a high response rate. A limitation of this study is that it was from a single institution and may have limited generalizability to other programs, especially with differing local assessments and national exams. The JSPE is a widely used tool in empathy research with published reliability and validity, with reviews showing similar quality compared to other self-reported empathy tools [17, 40]. This tool, however, was designed to evaluate “cognitive” aspects of empathy, and may not reflect the true complexity of empathic interpersonal interactions [41]. The JSPE, as a fixed, self-reported scale, has also been criticized to have limited predictive value for future empathy in educational settings, and may not be meaningful to clinical practice or patient satisfaction [15, 17, 18]. Self-reported surveys of empathy in medical students are also subject to reporting and desirability bias, as empathy is a known desirable characteristic of physicians.
4.2 Conclusion
18
Self-reported empathy scores in third year medical students were positively associated with performance on the communication component of OSCEs. JSPE scores did not predict overall OSCE scores in multivariate analyses, with age and gender as potential confounders. We found that self-reported empathy scores were also positively associated with the Pediatrics clerkship score, but not other clinical clerkship or standardized test scores.
4.3 Practice Implications Ideally, a brief and easy-to-implement instrument like the JSPE could be used to predict medical students’ future empathic behavior during patient encounters. There remains insufficient evidence that tools like the JSPE can fulfill this role, and further research is needed to evaluate the relationships between student-, standardized patient-, real patient-, and faculty-reported empathy scores. Research using standardized evaluations of OSCEs and clinical clerkships is also needed to accurately understand the relationship between self-reported empathy and these measures. For example, empathy ratings of medical students by evaluators of real patient interactions using a well-established penalty point system (Objective Structured Long Examination Record) correlated with empathy ratings by simulated patients on OSCE exams [37]. We would advocate for increased use and sharing of evaluations between institutions to improve standardization and future research. This field of education research may benefit from additional methods, especially qualitative and potentially from the social neurosciences, to gain a more comprehensive understanding of empathic 19
behavior and to determine if self-reported empathy can predict future empathic behavior [15, 41]. Our findings suggest that the associations between the JSPE and OSCE scores may differ in men and women, and this relationship warrants further exploration.
Conflicts of Interest The study authors have no conflicts of interest to report.
Acknowledgements Contributors: We thank Gerald Coffman and Michael Winters from the Boston University Information Services and Technology Department for compiling the project dataset. Funders: This research did not receive any specific grant from funding agencies in public, commercial, or not-for-profit sectors.
References [1] H.B. Beckman, K.M. Markakis, A.L. Suchman, R.M. Frankel, The doctor-patient relationship and malpractice: Lessons from plaintiff depositions, Arch. Intern. Med. 154 (1994) 1365-1370.
20
[2] S. Del Canale, D.Z. Louis, V. Maio, X. Wang, G. Rossi, M. Hojat, J.S. Gonella,The relationship between physician empathy and disease complications: an empirical study of primary care physicians and their diabetic patients in Parma, Italy, Acad. Med. 87 (2012) 1243-1249. [3] D. Rakel, B. Barrett, Z. Zhang, T. Hoeft, B. Chewning, L. Marchand, J. Scheder, Perception of empathy in the therapeutic encounter: effects on the common cold, Patient. Educ. Couns. 85 (2011) 390-397. [4] S.S. Kim, Kaplowitz, S., Johnston, M.V., The effects of physician empathy on patient satisfaction and compliance, Eval Health Prof. 27 (2004) 237-251. [5] S.W. Mercer, B.D. Jani, M. Maxwell, S.Y. Wong, G.C. Watt, Patient enablement requires physician empathy: a cross-sectional study of general practice consultations in areas of high and low socioeconomic deprivation in Scotland, BMC Fam. Pract.13 (2012). [6] R. Zachariae, C.G. Pedersen, A.B. Jensen, E. Ehrnrooth, P.B. Rossen, H. von der Maase, Association of perceived physician communication style with patient satisfaction, distress, cancer-related self-efficacy, and perceived control over the disease, Br. J. Cancer. 88 (2003) 658-665. [7] M. Hojat, M. Vergare, G. Isenberg, M. Cohen, J. Spandorfer, Underlying construct of empathy, optimism, and burnout in medical students, Int. J. of Med. Educ. 6 (2015). [8] R. Englander, T. Cameron, A.J. Ballard, J. Dodge, J. Bull, C.A. Aschenbrener, Toward a common taxonomy of competency domains for the health professions and competencies for physicians, Acad. Med. 88 (2013) 1088-1094. 21
[9] Association of American Medical Colleges: Learning objectives for medical student education. https://members.aamc.org/eweb/upload/Learning Objectives for Medical Student Educ Report I.pdf, 1998 (accessed 30.08.2014). [10] S.A. Batt-Rawden, M.S. Chisolm, B. Anton, T.E. Flickinger, Teaching empathy to medical students: An updated, systematic review, Acad. Med. 88 (2013) 11711177. [11] M. Hojat, J.S. Gonnella, T.J. Nasca, S. Mangione, J.J. Veloksi, M. Magee, The Jefferson Scale of Physician Empathy: further psychometric data and differences by gender and specialty at item level, Acad. Med. 77 (2002) S58-60. [12] M. Hojat, J.S. Gonella, T.J. Nasca, M. Salvatore, M. Vergare, M. Magee, Physician empathy: Definition, components, measurement, and relationship to gender and specialty, Am. J. Psychiatry. 159 (2002) 1563-1569. [13] S.K. Fields, P. Mahan, P. Tillman, J. Harris, K. Maxwell, M. Hojat, Measuring empathy in healthcare profession students using the Jefferson Scale of Physician Empathy: health provider-student version, J. Interprof. Care. 25 (2011) 287-293. [14] K.M. Glaser, F.W. Markham, H.M. Adler, P.R. McManus, M. Hojat, Relationships between scores on the Jefferson Scale of physician empathy, patient perceptions of physician empathy, and humanistic approaches to patient care: a validity study, Med. Sci. Monit. 13 (2007) 291-294. [15] R. Pederson, Empirical research on empathy in medicine – a critical review. Patient Educ. Couns. 76 (2009) 307-232.
22
[16] M. Neumann, J. Bensing, S. Mercer, N. Ernstmann, O. Ommen, H. Pfaff, Analyzing the ‘nature’ and ‘specific effectiveness’ of clinical empathy: a theoretical overview and contribution towards a theory-based research agenda, Patient Educ. Couns. 74 (2009) 339-346. [17] J.M. Hemmerdinger, S.D. Stoddart, R.J. Lilford, A systematic review of tests of empathy in medicine, BMC Med. Educ. 7 (2007). [18] S.H. Sulzer, N.W. Feinstein, C.L. Wendland, Assessing empathy development in medical education: a systematic review, Med. Educ. 50 (2016) 300-310. [19] J. Ogle, J.A. Bushnell, P. Caputi, Empathy is related to clinical competence in medical care, Med. Educ. 47 (2013) 824-831. [20] T.D. Stratton, C.L. Elam, A.E. Murphy-Spencer, S.L. Quinlivan, Emotional intelligence and clinical skills: preliminary results from a comprehensive clinical performance examination, Acad. Med. 80 (2005) S34-37. [21] D.C. Chen, M.E. Pahilan, J.D. Orlander, Comparing a self-administered measure of empathy with observed behavior among medical students, J. Gen. Intern. Med. 25 (2010) 200-202. [22] E.J. Austin, P. Evans, B. Magnus, K. O'Hanlon, A preliminary study of empathy, emotional intelligence and examination performance in MBChB students, Med. Educ. 41 (2007) 684-689.
23
[23] D.C Chen, D.S. Kirshenbaum, J. Yan, E. Kirshenbaum, R.H. Aseltine, Characterizing changes in student empathy throughout medical school, Med. Teach. 34 (2012) 305-311. [24] M. Hojat, J.S. Gonnella, S. Mangione, T.J. Nasca, J.J. Veloski, J.B. Erdmann, C.A. Callahan, M. Magee, Empathy in medical students as related to academic performance, clinical competence and gender, Med. Educ. 36 (2002) 522-7. [25] M. Schweller, F.O. Costa, M.A. Antonio, E.M. Amaral, M.A. de Carvalho-Filho, M.A., The impact of simulated medical consultations on the empathy levels of students at one medical school, Acad. Med. 89 (2014) 632-637. [26] M. Hojat, S. Mangione, T.J. Nasca, S. Rattner, J.B. Erdmann, J.S. Gonnella, M. Magee, An empirical study of decline in empathy in medical school, Med. Educ. 38 (2004) 934-941. [27] L.F. DiLalla, S.K. Hull, J.K Dorsey. Effect of gender, age, and relevant course work on attitudes toward empathy, patient spirituality and physician wellness, Teach. Learn. Med. 16 (2004) 165-170. [28] D. Chen, R. Lew, W. Hershman, J. Orlander, A cross-sectional measurement of medical student empathy, J. Gen. Intern. Med. 22 (2007) 1434-1438. [29] A. Ferreira-Valente, J.S. Monteiro, R.M. Barbosa, A. Salgueira, P. Costa, M.J. Costa, Clarifying changes in student empathy throughout medical school: a scoping review, Adv. in Health. Sci. Educ. (2016).
24
[30] K. Berg, Majdan, J.F., Berg, D., Veloski, J., Hojat, M., A comparison of medical students' self-reported empathy with simulated patients' assessments of the students' empathy: an analysis by gender and ethnicity, Acad Med. 86 (2011) 984-988. [31] S. Humphrey-Murto, C. Touchie, T.J. Wood, S. Smee S. Does the gender of the standardized patient influence candidate performance in an objective structured clinical examination? Med. Educ. 43 (2009) 521-525. [32] J.A. Carson, A. Peets, V. Grant, K. McLaughlin, The effect of gender interactions on students' physical examination ratings in objective structured clinical examination stations, Acad. Med. 85 (2010) 1772-1776. [33] T.A. Lieu, S.A. Schroeder, D.F. Altman, Specialty choice at one medical school: recent trends and analysis of predictive factors, Acad. Med. 64 (1989) 622–629. [34] Association of American Medical Colleges: Physician education debt and the cost to attend medical school: 2012 Update, https://www.aamc.org/download/328322/data/statedebtreport.pdf, 2013 (accessed 30.08.2014). [35] M. Hojat, S. Mangione, C.K. Kane, J.S. Gonnella, Relationships between scores of the Jefferson Scale of Physician Empathy (JSPE) and the Interpersonal Reactivity Index (IRI), Med. Teach. 27 (2005) 625-628. [36] S. Grosseman, Novack, D.H., Duke, P., Mennin, S., Rosenzweig, S., Davis, T.J., Hojat, M., Residents' and standardized patients' perspectives on empathy: issues of agreement, Patient Educ. Couns. 96 (2014) 22-28. 25
[37] B. Wright, J. McKendree, L. Morgan, V.L. Allgar, A. Brown, Examiner and simulated patient ratings of empathy in medical student final year clinical examination: are they useful? BMC Med. Educ. 14 (2014). [38] M. Aomatsu, T. Otani, A. Tanaka, N. Ban, J. van Dalen, Medical students' and residents' conceptual structure of empathy: a qualitative study, Educ. Health (Abingdon). 26 (2013) 4-8. [39] K. Berg, B. Blatt, J. Lopreiato, J. Jung, A. Schaeffer, D. Heil, T. Owens, P.L. CarterNolan, D. Berg, J. Veloski, E. Darby, M. Hojat, Standardized patient assessment of medical student empathy: Ethnicity and gender effects in a multi-institutional study, Acad. Med. 90 (2015) 105-111. [40] J. Yu, M. Kirk, Evaluation of empathy measurement tools in nursing: systematic review, J. Adv. Nurs. 65 (2009) 1790-1806. [41] I. Preusche, C. Lamm, Reflections on empathy in medical education: what can we learn from social neurosciences? Adv. in Health Sci. Educ. 21 (2016) 235249.
Legends 26
Figure 1: JSPE scores and honoring core clerkships in medical students
Table 1: Student Demographics (N=590) Age (median (IQR)) Female (N (%)) High Debt (> $150,000) (N (%)) Specialty Affinity (N (%)) Neither Technology oriented* People oriented† Both
26 (3) 319 (54) 372 (63) 406 (69) 37 (6) 144 (24) 3 (1)
*Internal Medicine, Family Medicine, Obstetrics/Gynecology, Psychiatry, Pediatrics, Neurology, Rehabilitation Medicine, Emergency Medicine, Ophthalmology, Dermatology †Radiology, Pathology, Surgery and Surgical Subspecialties, Radiation Oncology, Anesthesiology
27
Table 2: Regression analyses of JSPE and OSCE scores Outcome = OSCE communication score Bivariate Models
JSPE Age Gender Debt People affinity* Tech affinity†
Beta coefficient 0.065 -0.11 1.0 -0.017 -0.29 -1.36
Standard error 0.020 0.076 0.70 0.14 0.70 0.81
Multivariate model
P value 0.0012 0.17 0.15 0.90 0.68 0.093
Beta coefficient 0.061 -0.11 0.49 -0.055 -1.3 -0.5
Standard error 0.021 0.78 0.73 0.15 0.90 0.75
P value 0.0033 0.16 0.51 0.71 0.16 0.50
Outcome = overall OSCE score Bivariate Models
JSPE Age Gender Debt People affinity* Tech affinity†
Beta coefficient 0.027 -0.10 1.2 -0.13 0.20 -1.4
Standard error 0.011 0.045 0.38 0.090 0.41 0.46
Multivariate model
P value 0.016 0.028 0.001 0.15 0.62 0.0028
Beta coefficient 0.022 -0.086 1.1 -0.13 0.053 -1.3
Standard error 1.7 0.012 0.047 0.39 0.090 0.43
P value
*People-oriented specialty affinity: Internal Medicine, Family Medicine, Obstetrics/Gynecology, Psychiatry, Pediatrics, Neurology, Rehabilitation Medicine, Emergency Medicine, Ophthalmology, Dermatology †Technology-oriented specialty affinity: Radiology, Pathology, Surgery and Surgical Subspecialties, Radiation Oncology, Anesthesiology
28
0.067 0.067 0.0078 0.13 0.90 0.0082
S0738-3991(16)30511-0 http://dx.doi.org/doi:10.1016/j.pec.2016.11.006 PEC 5502
To appear in:
Patient Education and Counseling
Received date: Revised date: Accepted date:
21-6-2016 23-9-2016 9-11-2016
Please cite this article as: Casas Rachel S, Xuan Ziming, Jackson Angela H, Stanfield Lorraine E, Harvey Nanette C, Chen Daniel C.Associations of medical student empathy with clinical competence.Patient Education and Counseling http://dx.doi.org/10.1016/j.pec.2016.11.006 This is a PDF file of an unedited manuscript that has been accepted for publication. As a service to our customers we are providing this early version of the manuscript. The manuscript will undergo copyediting, typesetting, and review of the resulting proof before it is published in its final form. Please note that during the production process errors may be discovered which could affect the content, and all legal disclaimers that apply to the journal pertain.
Title Page
Title: Associations of Medical Student Empathy with Clinical Competence
Authors: Rachel S. Casas, MD, EdM1, Ziming Xuan ScD, SM, MA2, Angela H. Jackson, MD1, Lorraine E. Stanfield, MD1, Nanette C. Harvey, MD3, Daniel C. Chen, MD, MSc1
Affiliations: 1Evans Department of Medicine, Boston University School of Medicine, Boston, Massachusetts, USA; 2Department of Community Health Sciences, Boston University, Boston, Massachusetts, USA; 3Department of Family Medicine, Boston University School of Medicine, Boston, Massachusetts, USA
Corresponding Author: Rachel S. Casas, MD, EdM 801 Massachusetts Avenue, Crosstown 2, Boston, MA 02118, USA Email: [email protected] Phone: (617) 414-5951 Fax: (617) 638-7472 Author emails: Ziming Xuan: [email protected] Angela H. Jackson: [email protected] Lorraine E. Stanfield: [email protected] Nanette C. Harvey: [email protected] Daniel C. Chen: [email protected]
1
Highlights
Empathy predicted communication scores on Objective Structured Clinical Examinations (OSCEs)
Associations between empathy and overall OSCE scores were confounded by gender and age
Empathy was positively associated with the Pediatrics clerkship score, but not other clerkships
Self-reported empathy did not predict United States Medical Licensing Exam scores
Abstract Objective: Empathy is a crucial skill for medical students that can be difficult to evaluate. We examined if self-reported empathy in medical students was associated with clinical competence. Methods: This study combined cross-sectional data from four consecutive years of medical students (N=590) from the Boston University School of Medicine. We used regression analysis to evaluate if self-reported empathy (Jefferson Scale of Physician Empathy (JSPE)) predicted scores in clinical clerkships, United States Medical Licensing Examinations, and Objective Structured Clinical Examinations (OSCEs). We separately analyzed overall and OSCE communication scores based on interpersonal skills reported by standardized patients. We controlled for age, gender, debt, and specialty affinity. 2
Results: JSPE scores of medical students were positively associated with OSCE communication scores, and remained significant when controlling for demographics. We found that JSPE score was also predictive of overall OSCE scores, but this relationship was confounded by gender and age. JSPE scores were associated with performance in the Pediatrics clerkship, but not other clerkships or standardized tests. Conclusion: JSPE scores were positively associated with OSCE communication scores in medical students. Practice Implications: This study supports that self-reported empathy may predict OSCE performance, but further research is needed to examine differences by gender and age. Key words: medical education-undergraduate, medical educationassessment/evaluation, clinical skills assessment, communication skills
1. Introduction Empathy is a crucial component of physician-patient interactions and has been associated with improved patient compliance, satisfaction, and clinical outcomes and with decreased provider burn-out and litigation [1-7]. Consequently, the Association of American Medical Colleges (AAMC) has outlined interpersonal and communication skills as core competencies for medical students [8, 9]. Studies suggest that education interventions may be effective in teaching empathy to medical students, but empathy remains difficult to evaluate [10]. 3
Self-reported empathy tools have been developed for education research, but the predictive value of these tools for future empathic behavior remains controversial. This study utilizes one of the most widely used self-reported empathy tools, the Jefferson Scale of Physician Empathy – Student Version (JSPE), which was designed to address patient care situations and has established validity and reliability in medical students [7, 11-14]. The JSPE was designed to measure “cognitive” empathy (the ability to understand another’s experience without a personal emotional response) as opposed to “affective” empathy (the passive emotional response to another’s emotions) [15, 16]. While self-reported empathy measures, including the JSPE, have sufficient evidence to support their use in education research, they may lack validity for use in medical student admissions or evaluation [17]. Consequently, it is important to clarify the relationship between self-, standardized patient-, and faculty-reported empathy to determine if tools like the JSPE can accurately predict student behavior in clinical settings [18]. Current measures of clinical competence in medical students vary in their coverage of empathy. In the United States, these evaluations include Objective Structured Clinical Examinations (OSCEs), clinical clerkships, and the United States Medical Licensing Examinations (USMLEs). Core clerkship grading measures vary by institution, but are generally composed of direct observation of clinical performance by supervising faculty and a multiple-choice knowledge based subject examination from the National Board of Medical Examiners (NBME). OSCEs also vary by institution, and consist of multiple simulated patient scenarios where clinical
4
skills are evaluated by standardized patients. The USMLEs are standardized examinations which include three components during medical school: Step 1, Step 2 Clinical Knowledge (CK), and Step 2 Clinical Skills (CS). Step 1 and Step 2 CK are multiple choice exams which assess mainly medical knowledge while Step 2 CS uses standardized patients to assess patient-centered skills in clinical scenarios. Existing studies show mixed associations between self-reported empathy and measures of clinical competence. For example, while self-reported empathy has been associated with communication scores on OSCEs, associations with the overall OSCE score and clinical clerkship grades are conflicting [19-23]. Studies examining multiple measures of clinical competence in the same cohort of students are limited. One study demonstrated a positive association between self-reported empathy and clinical competence in core clinical clerkships but not standardized tests (USMLE Step 1 and Step 2 CK) [24]. To our knowledge, no existing studies have examined self-reported empathy and USMLE Step 2 CS. Because of the heterogeneity of local OSCE evaluations at institutions throughout the United States, comparisons between empathy measures and Step 2 CS will be valuable for clarifying the relationship between self-reported and observed empathy in medical students. There are limited studies examining self-reported empathy and clinical competence in students of differing genders. Prior analyses have shown consistently higher selfreported empathy scores in women relative to men, and mixed results by age [11, 12, 22, 23, 25-29]. One existing study found that women had higher self-reported empathy scores compared to men but found no association between gender and 5
core clerkships scores, suggesting that gender did not confound the relationship between these variables [24]. This study, however, did not include OSCE scores. Empathy scores of women by standardized patients may be higher than men, and OSCE physical exam scores may differ by student gender [30-32]. Another study of JSPE, core clerkship grades, and gender found differing trajectories of empathy scores by gender, but did not report associations between gender and core clerkship scores [22]. Potential confounders are important to verify, as the predictive use of self-reported empathy for clinical competence may differ by student population. Based upon these studies, we expect gender to potentially confound the associations between empathy and OSCE scores, but not clinical clerkship or standardized test scores. Our study clarifies existing research by examining the associations between selfreported empathy and all currently available institutional and national measures of clinical competence in the same cohort of medical students. Because empathy is an important component of interpersonal interaction, we hypothesized that examinations which assess these skills (clinical clerkships, OSCEs, and USMLE Step 2 CS) would be positively associated with self-reported empathy scores. We hypothesized that standardized examinations which do not assess interpersonal skills (USMLE Step 1 and 2CK) would not be associated with self-reported empathy scores.
2. Methods
6
This is a study of cross-sectional populations of third year medical students from four consecutive years enrolled at the Boston University School of Medicine (BUSM) between 2007 and 2010. The protocol for this study was reviewed by the Boston University Medical Center Institutional Review Board and determined exempt.
2.1 Study Population All medical students enrolled at BUSM (N=1,162) between 2006 and 2010 were eligible to participate in empathy surveys through previously outlined methods [21, 23, 28]. While students from all years of medical school were surveyed, we limited this analysis to third year students because the core clerkships, OSCEs, and USMLE Step 2 exams were completed in this year. The BUSM includes two years of preclinical study with sixteen weeks of interviewing and physical diagnosis workshops, followed by two years of clinical clerkships.
2.2 Data collection We recruited participants to complete self-administered, anonymous surveys at the end of the third year of medical school from 2006 to 2010 [21, 23, 28]. Surveys prior to 2007 were excluded because de-identified empathy data could not be linked to individual measures of clinical competence. We surveyed students about demographic information (gender, age, anticipated financial debt, and future specialty affinity) and self-reported empathy using the JSPE - Student Version. An anonymous dataset of clinical competence examination scores linked to empathy and demographic data was 7
provided to the study authors by the Boston University Information and Technology Database.
2.3 Measurement Variables 2.3.1 Demographics: Likelihood of choosing a specialty as a future career was ranked on a five point Likert scale (very unlikely = 1… neutral = 3… very likely =5). Specialties were grouped as “people-oriented” (Internal Medicine, Family Medicine, Obstetrics/Gynecology, Psychiatry, Pediatrics, Neurology, Rehabilitation Medicine, Emergency Medicine, Ophthalmology, Dermatology) or “technology-oriented” (Radiology, Pathology, Surgery and Surgical Subspecialties, Radiation Oncology, Anesthesiology) based upon prior groupings in medical education research and evidence of higher empathy in students with “people-oriented” compared to “technology-oriented” specialty affinities using the JSPE [11, 33]. Two dummy variables were created for regression analyses, with each student assigned a value of 1 for “people-oriented” if they indicated they were “very likely” to choose a career in a people-oriented specialty, 0 otherwise. Similarly, each student was assigned a value of 1 for “technology-oriented” if they indicated that they were “very likely to choose a career in a technology-oriented specialty, 0 otherwise. Projected debt by the end of medical school included seven ordinal categories (0$25,000 to >$200,000) and was dichotomized into higher (> $150,000) and lower (≤ $150,000) debt to approximate the median debt ($160,000) of medical school graduates in the United States in 2010 [34].
8
2.3.2 Self-reported empathy: The JSPE is an internationally utilized, reliable, and validated survey containing 20 questions on a seven point Likert scale with higher scores (range 20 to 140) corresponding to higher empathy [7, 11-14]. In prior studies, mean JSPE scores of medical students ranged from 80 to 122 (standard deviation 9.9 to 11.2) [19, 22, 24, 28].
2.3.3 Clinical competence: Clinical competence measures included scores from third year OSCEs, clinical clerkships, and standardized tests (USMLE Step 1, 2 CK, and 2 CS). During the OSCE at the end of third year at BUSM, students complete simulated clinical encounters which are evaluated by trained standardized patients. The OSCE consisted of six clinical cases which were scored using checklists in four content areas: communication, history taking, physical exam skills, and documentation. The communication score was based on 14 items about rapport building, questioning technique, and interview organization. Each content area was graded as “met standard” for scores higher than two standard deviations below the mean, otherwise the student “did not meet standard”. For the overall OSCE score, students “marginally met standard” if they failed one content area and “did not meet standard” if they failed two content areas. Given the small number of students with marginal and failing performance, these two categories were combined for analysis. We compared JSPE scores to both the overall OSCE score and the communication score sub-component.
9
We included final grades from the eight core clerkships (Internal Medicine, Family Medicine, Pediatrics, Obstetrics/Gynecology, Surgery, Psychiatry, Radiology, and Neurology) at BUSM in this analysis. Clerkships grades (honors, high pass, pass, marginal pass, fail, incomplete) were based upon faculty clinical evaluations and the National Board of Medical Examiners (NBME) Subject Exams. We divided students into categories of achieving “honors” compared to all other combined grades as the scores overall were skewed with few students receiving less than passing grades. For the standardized tests, the USMLE Step 1 and Step 2 CK were continuous scores (potential ranges from 1 to 300). USMLE STEP 2 CS was graded Pass/Fail.
2.4 Data Analysis Demographics were descriptively summarized. We used logistic and linear regression analyses to evaluate JSPE score as a predictor for binary outcomes (OSCEs, core clerkships, and USMLE Step 2 CS) and continuous outcomes (USMLE Step 1 and Step 2 CK) respectively. We also completed bivariate regression analyses with demographics (age, gender, specialty affinity, and debt) as predictors for JSPE scores and clinical competence scores. To evaluate for confounding, we conducted multivariate regression analyses which included demographic variables (age, gender, specialty affinity, and debt). Statistical significance was considered at p-value < 0.05. Data analysis was completed using SAS 9.3 (SAS Institute, Cary NC).
10
3. Results The average survey response rate was 81%. JSPE data was available for 603 third year students. We excluded 13 students due to missing JSPE data, leaving 590 students for analysis.
3.1 Demographics: The majority of students were female (54%) and reported high debt (63%) (Table 1). [Insert Table 1] Age, gender, and having a “technology-oriented” specialty affinity were predictive of overall OSCE scores in bivariate analyses. Students who were women, younger, or had a “technology-oriented” specialty affinity were more likely to pass the overall OSCE (Beta=1.2, p=0.0005 for gender; Beta=-0.10, p=0.02 for age; Beta= -1.4 p=0.0003 for “technology-oriented” affinity). No demographic variables were predictive of OSCE communication, clinical clerkship, or standardized test scores.
3.2 JSPE scores: Median JSPE score (118, IQR=7) corresponded with the expected range from prior literature [10, 13, 17, 22]. In bivariate regression analyses, gender and age were predictive of JSPE scores (Beta=7.2, p<0.0001 for gender; Beta=0.5, p=0.015 for age). Median (IQR) JSPE scores were higher in women (121 (14)) compared to men (113 (19)) and in older (119 (16)) compared to younger (116 (18)) students (t=-6.8,
11
p <0.0001 and t= 2.0, p=0.04 respectively). Debt and specialty affinity did not predict JSPE score.
3.3 OSCE Communication Scores and JSPE Five hundred and eighty students met the standard and 10 did not meet the standard for the OSCE communication score. Median (IQR) JSPE scores were 118 (18) for students who met the standard and 97 (18) for students who did not meet the standard (t=-3.5, p = 0.0005). JSPE scores were significantly predictive of the OSCE communication score in simple and multiple regression analyses (Table 2). In the adjusted model, every one point increase in JSPE score was associated with a 6.3% increase in the odds of meeting the standard for the OSCE communication score (AOR=1.06 95% CI: 1.02, 1.10). For example, a student with a JSPE score of 130 (about one standard deviation above the mean) would have an approximately 189% increased odds of passing the overall OSCE compared to a student with a JSPE score of 100 (about one standard deviation below the mean). [Insert Table 2]
3.4 OSCE Overall Score and JSPE Five hundred and fifty-two students met the standard and 38 did not meet the standard for the OSCE overall score. Of these 38 students, 10 failed both the communication 12
section and overall OSCE. Students who met the standard had median (IQR) JSPE score 118 (18) and students who did not meet the standard had median (IQR) JSPE score 114 (21) (t=-2.5; p = 0.015). JSPE score was significantly predictive of OSCE overall scores in bivariate but not multivariate regression analyses controlling for all demographics (Table 2). In multivariate analyses modeling empathy and overall OSCE with the adjustment of gender, age, and “technology-oriented” affinity separately, the beta coefficient for empathy changed by 10% or greater for gender and age only.
3.5 Clinical Clerkships and JSPE The number of students receiving honors ranged from 145 (Obstetrics/Gynecology) to 384 (Psychiatry). Figure 1 shows the adjusted odds ratios for JSPE scores and honoring each of the eight core clerkships, of which Pediatrics only was significant. [Insert Figure 1] A one point increase in JSPE score was associated with a 1.7% increased odds of honoring Pediatrics when adjusting for demographics (Figure 1) (AOR 1.02 95% CI: 1.00, 1.03). For example, a student with a JSPE score of 130 would have an approximately 51% increased odds of honoring Pediatrics compared to a student with a JSPE score of 100.
3.6 Standardized Tests and JSPE 13
Students had median (IQR) USMLE Step 1 score 228 (29) and Step 2 CK score of 236 (29). For every 1 point increase in empathy score, there was a 0.2 point decrease in USMLE 1 score (p=0.011) in simple but not multiple regression analysis (data not shown). There were no significant relationships between JSPE and Step 2 CK scores or JSPE and Step 2 CS scores (N=6 students failing, N=584 students passing).
3.7 Sensitivity Analysis Graphed JSPE data showed a bimodal peak at a score of 80, which corresponded to choosing the neutral Likert category for all questions and may reflect response bias. We conducted sensitivity analyses by excluding students with JSPE scores of 80 (N=10) and noted no change in reported relationships.
4. Discussion and Conclusion 4.1 Discussion We demonstrated that self-reported empathy scores were associated with OSCE communication scores, OSCE overall scores, and the Pediatrics clerkship, but not standardized examinations. The associations between empathy and overall OSCE scores, however, were confounded by age and gender. This study adds to existing literature by examining associations of self-reported empathy with all currently available measures of clinical competence in the same group of students. Self-reported empathy may predict observed performance on OSCEs in medical students, but
14
these associations may be limited to communication scores and differ in students by age and gender. Empathy scores were predictive of OSCE communication scores, but not overall OSCE scores, when accounting for demographic variables. Similarly, existing literature supports that self-reported empathy may be more highly predictive of the communication component of the OSCE, but not the physical exam component or overall scores [19, 20]. For example, empathic concern measured through the Davis’ Interpersonal Reactivity Index (IRI) was positively correlated with communication skills during OSCEs in 166 third year students from the United States, but not with physical exam skills [20]. While this study used a different self-reported empathy tool, the IRI has been shown to have a statistically significant correlation of moderate magnitude with the JSPE [35]. In another study, JSPE scores were not associated with overall OSCE scores in 57 Australian third year students [19]. Other studies have also found that observed but not self-reported empathy scores were associated with overall OSCE performance, with mixed trends in empathy scores as student progress through medical school [19, 21, 29, 36]. In these comparisons, self-reported and observed empathy scores may reflect differing aspects of affective and cognitive empathy. While our study showed that the JSPE was associated with higher communication scores on OSCEs, it is unclear if this relationship predicts more empathic behavior with real patients. In one previous study, empathy ratings of clinical performance with real patients related to empathy ratings in standardized patients [37]. Selfreported empathy, however, may not meaningfully reflect interactions between 15
providers and patients and it is uncertain if behaving empathically without shared emotion impacts patient care [10, 14, 30]. Educational interventions can enhance empathic behavior in medical students, but these students may develop a protective distancing of their own emotional responses with clinical experience [10]. For example, in a qualitative study from Japan, medical students viewed sharing patients’ feelings as an important component of empathy (affective empathy), but could decide to act empathically during OSCEs regardless of shared emotion (cognitive empathy) [38]. Further comparisons of self-, faculty-, standardized patient-, and real patient-reported empathy measures are needed to determine if we are accurately predicting empathic behavior in medical students and if the potential incongruity between observed and self-reported empathy impacts patient care. Our results suggest that the relationship between self-reported empathy and overall OSCE scores were confounded by gender and age. Women had higher self-reported empathy scores and were more likely to pass the overall OSCE compared to men. Interestingly, older students had higher empathy scores but were less likely to pass the overall OSCE. Technology-oriented specialty affinity was also predictive of passing the OSCE, but did not relate to empathy scores. In regards to gender, confounding is supported by a prior study showing higher selfreported and observed empathy by standardized patients in women compared to men [39]. We expected both the communication component and overall OSCE to be confounded by gender based upon prior studies showing that both observed empathy and physical exam scores during OSCEs differ by student gender [30-32]. To our knowledge, existing studies of self-reported empathy and OSCE scores did not report
16
comparisons by age. We would expect older students, with more life experience, to be better equipped to self-assess and display empathic concern. The reasons for gender confounding and lower pass rates of older students for the overall OSCE in our study will require further qualitative exploration. We did not detect an association between empathy and clinical clerkship grades, with the exception of Pediatrics. One possible explanation for this lack of association was that 25 to 35% of each core clerkship grade at BUSM consisted of NBME Subject Exam scores, which could not be separated from the faculty clinical evaluations for analysis. At many medical schools, including BUSM, faculty receive limited training in clerkship evaluation with locally developed measures, leading to potentially limited reliability and validity. In comparison, at Jefferson Medical College in the United States, higher JSPE scores from 371 third year medical students were associated with more honors grades in core clerkships [11]. In another study from the United Kingdom, JSPE scores were not associated with measures of overall academic performance [22]. Differing grading and training systems may contribute to the variability in relationships between empathy and core clerkships. At BUSM, we are in the process of revising tools and educating faculty to increase standardization of evaluations. We found that self-reported empathy was not associated with standardized test scores. These results are in agreement with one existing study from 2002 which found no associations between JSPE and the USMLE Step 1 or Step 2 CK exams [24]. USMLE Step 2 CS was implemented following this study in 2004. Given that Step 2 CS is designed to assess communication skills, we expected to find a significant relationship 17
between empathy and Step 2 CS, but did not. Conclusions regarding Step 2 CS from this analysis are limited, however, because failure of this exam was an uncommon outcome. Repeat analyses using a larger data set are warranted to evaluate these relationships. A strength of this study was that it included data from all major, current assessments of clinical competence of medical students in the United States. The data were from a large population over four consecutive years with a high response rate. A limitation of this study is that it was from a single institution and may have limited generalizability to other programs, especially with differing local assessments and national exams. The JSPE is a widely used tool in empathy research with published reliability and validity, with reviews showing similar quality compared to other self-reported empathy tools [17, 40]. This tool, however, was designed to evaluate “cognitive” aspects of empathy, and may not reflect the true complexity of empathic interpersonal interactions [41]. The JSPE, as a fixed, self-reported scale, has also been criticized to have limited predictive value for future empathy in educational settings, and may not be meaningful to clinical practice or patient satisfaction [15, 17, 18]. Self-reported surveys of empathy in medical students are also subject to reporting and desirability bias, as empathy is a known desirable characteristic of physicians.
4.2 Conclusion
18
Self-reported empathy scores in third year medical students were positively associated with performance on the communication component of OSCEs. JSPE scores did not predict overall OSCE scores in multivariate analyses, with age and gender as potential confounders. We found that self-reported empathy scores were also positively associated with the Pediatrics clerkship score, but not other clinical clerkship or standardized test scores.
4.3 Practice Implications Ideally, a brief and easy-to-implement instrument like the JSPE could be used to predict medical students’ future empathic behavior during patient encounters. There remains insufficient evidence that tools like the JSPE can fulfill this role, and further research is needed to evaluate the relationships between student-, standardized patient-, real patient-, and faculty-reported empathy scores. Research using standardized evaluations of OSCEs and clinical clerkships is also needed to accurately understand the relationship between self-reported empathy and these measures. For example, empathy ratings of medical students by evaluators of real patient interactions using a well-established penalty point system (Objective Structured Long Examination Record) correlated with empathy ratings by simulated patients on OSCE exams [37]. We would advocate for increased use and sharing of evaluations between institutions to improve standardization and future research. This field of education research may benefit from additional methods, especially qualitative and potentially from the social neurosciences, to gain a more comprehensive understanding of empathic 19
behavior and to determine if self-reported empathy can predict future empathic behavior [15, 41]. Our findings suggest that the associations between the JSPE and OSCE scores may differ in men and women, and this relationship warrants further exploration.
Conflicts of Interest The study authors have no conflicts of interest to report.
Acknowledgements Contributors: We thank Gerald Coffman and Michael Winters from the Boston University Information Services and Technology Department for compiling the project dataset. Funders: This research did not receive any specific grant from funding agencies in public, commercial, or not-for-profit sectors.
References [1] H.B. Beckman, K.M. Markakis, A.L. Suchman, R.M. Frankel, The doctor-patient relationship and malpractice: Lessons from plaintiff depositions, Arch. Intern. Med. 154 (1994) 1365-1370.
20
[2] S. Del Canale, D.Z. Louis, V. Maio, X. Wang, G. Rossi, M. Hojat, J.S. Gonella,The relationship between physician empathy and disease complications: an empirical study of primary care physicians and their diabetic patients in Parma, Italy, Acad. Med. 87 (2012) 1243-1249. [3] D. Rakel, B. Barrett, Z. Zhang, T. Hoeft, B. Chewning, L. Marchand, J. Scheder, Perception of empathy in the therapeutic encounter: effects on the common cold, Patient. Educ. Couns. 85 (2011) 390-397. [4] S.S. Kim, Kaplowitz, S., Johnston, M.V., The effects of physician empathy on patient satisfaction and compliance, Eval Health Prof. 27 (2004) 237-251. [5] S.W. Mercer, B.D. Jani, M. Maxwell, S.Y. Wong, G.C. Watt, Patient enablement requires physician empathy: a cross-sectional study of general practice consultations in areas of high and low socioeconomic deprivation in Scotland, BMC Fam. Pract.13 (2012). [6] R. Zachariae, C.G. Pedersen, A.B. Jensen, E. Ehrnrooth, P.B. Rossen, H. von der Maase, Association of perceived physician communication style with patient satisfaction, distress, cancer-related self-efficacy, and perceived control over the disease, Br. J. Cancer. 88 (2003) 658-665. [7] M. Hojat, M. Vergare, G. Isenberg, M. Cohen, J. Spandorfer, Underlying construct of empathy, optimism, and burnout in medical students, Int. J. of Med. Educ. 6 (2015). [8] R. Englander, T. Cameron, A.J. Ballard, J. Dodge, J. Bull, C.A. Aschenbrener, Toward a common taxonomy of competency domains for the health professions and competencies for physicians, Acad. Med. 88 (2013) 1088-1094. 21
[9] Association of American Medical Colleges: Learning objectives for medical student education. https://members.aamc.org/eweb/upload/Learning Objectives for Medical Student Educ Report I.pdf, 1998 (accessed 30.08.2014). [10] S.A. Batt-Rawden, M.S. Chisolm, B. Anton, T.E. Flickinger, Teaching empathy to medical students: An updated, systematic review, Acad. Med. 88 (2013) 11711177. [11] M. Hojat, J.S. Gonnella, T.J. Nasca, S. Mangione, J.J. Veloksi, M. Magee, The Jefferson Scale of Physician Empathy: further psychometric data and differences by gender and specialty at item level, Acad. Med. 77 (2002) S58-60. [12] M. Hojat, J.S. Gonella, T.J. Nasca, M. Salvatore, M. Vergare, M. Magee, Physician empathy: Definition, components, measurement, and relationship to gender and specialty, Am. J. Psychiatry. 159 (2002) 1563-1569. [13] S.K. Fields, P. Mahan, P. Tillman, J. Harris, K. Maxwell, M. Hojat, Measuring empathy in healthcare profession students using the Jefferson Scale of Physician Empathy: health provider-student version, J. Interprof. Care. 25 (2011) 287-293. [14] K.M. Glaser, F.W. Markham, H.M. Adler, P.R. McManus, M. Hojat, Relationships between scores on the Jefferson Scale of physician empathy, patient perceptions of physician empathy, and humanistic approaches to patient care: a validity study, Med. Sci. Monit. 13 (2007) 291-294. [15] R. Pederson, Empirical research on empathy in medicine – a critical review. Patient Educ. Couns. 76 (2009) 307-232.
22
[16] M. Neumann, J. Bensing, S. Mercer, N. Ernstmann, O. Ommen, H. Pfaff, Analyzing the ‘nature’ and ‘specific effectiveness’ of clinical empathy: a theoretical overview and contribution towards a theory-based research agenda, Patient Educ. Couns. 74 (2009) 339-346. [17] J.M. Hemmerdinger, S.D. Stoddart, R.J. Lilford, A systematic review of tests of empathy in medicine, BMC Med. Educ. 7 (2007). [18] S.H. Sulzer, N.W. Feinstein, C.L. Wendland, Assessing empathy development in medical education: a systematic review, Med. Educ. 50 (2016) 300-310. [19] J. Ogle, J.A. Bushnell, P. Caputi, Empathy is related to clinical competence in medical care, Med. Educ. 47 (2013) 824-831. [20] T.D. Stratton, C.L. Elam, A.E. Murphy-Spencer, S.L. Quinlivan, Emotional intelligence and clinical skills: preliminary results from a comprehensive clinical performance examination, Acad. Med. 80 (2005) S34-37. [21] D.C. Chen, M.E. Pahilan, J.D. Orlander, Comparing a self-administered measure of empathy with observed behavior among medical students, J. Gen. Intern. Med. 25 (2010) 200-202. [22] E.J. Austin, P. Evans, B. Magnus, K. O'Hanlon, A preliminary study of empathy, emotional intelligence and examination performance in MBChB students, Med. Educ. 41 (2007) 684-689.
23
[23] D.C Chen, D.S. Kirshenbaum, J. Yan, E. Kirshenbaum, R.H. Aseltine, Characterizing changes in student empathy throughout medical school, Med. Teach. 34 (2012) 305-311. [24] M. Hojat, J.S. Gonnella, S. Mangione, T.J. Nasca, J.J. Veloski, J.B. Erdmann, C.A. Callahan, M. Magee, Empathy in medical students as related to academic performance, clinical competence and gender, Med. Educ. 36 (2002) 522-7. [25] M. Schweller, F.O. Costa, M.A. Antonio, E.M. Amaral, M.A. de Carvalho-Filho, M.A., The impact of simulated medical consultations on the empathy levels of students at one medical school, Acad. Med. 89 (2014) 632-637. [26] M. Hojat, S. Mangione, T.J. Nasca, S. Rattner, J.B. Erdmann, J.S. Gonnella, M. Magee, An empirical study of decline in empathy in medical school, Med. Educ. 38 (2004) 934-941. [27] L.F. DiLalla, S.K. Hull, J.K Dorsey. Effect of gender, age, and relevant course work on attitudes toward empathy, patient spirituality and physician wellness, Teach. Learn. Med. 16 (2004) 165-170. [28] D. Chen, R. Lew, W. Hershman, J. Orlander, A cross-sectional measurement of medical student empathy, J. Gen. Intern. Med. 22 (2007) 1434-1438. [29] A. Ferreira-Valente, J.S. Monteiro, R.M. Barbosa, A. Salgueira, P. Costa, M.J. Costa, Clarifying changes in student empathy throughout medical school: a scoping review, Adv. in Health. Sci. Educ. (2016).
24
[30] K. Berg, Majdan, J.F., Berg, D., Veloski, J., Hojat, M., A comparison of medical students' self-reported empathy with simulated patients' assessments of the students' empathy: an analysis by gender and ethnicity, Acad Med. 86 (2011) 984-988. [31] S. Humphrey-Murto, C. Touchie, T.J. Wood, S. Smee S. Does the gender of the standardized patient influence candidate performance in an objective structured clinical examination? Med. Educ. 43 (2009) 521-525. [32] J.A. Carson, A. Peets, V. Grant, K. McLaughlin, The effect of gender interactions on students' physical examination ratings in objective structured clinical examination stations, Acad. Med. 85 (2010) 1772-1776. [33] T.A. Lieu, S.A. Schroeder, D.F. Altman, Specialty choice at one medical school: recent trends and analysis of predictive factors, Acad. Med. 64 (1989) 622–629. [34] Association of American Medical Colleges: Physician education debt and the cost to attend medical school: 2012 Update, https://www.aamc.org/download/328322/data/statedebtreport.pdf, 2013 (accessed 30.08.2014). [35] M. Hojat, S. Mangione, C.K. Kane, J.S. Gonnella, Relationships between scores of the Jefferson Scale of Physician Empathy (JSPE) and the Interpersonal Reactivity Index (IRI), Med. Teach. 27 (2005) 625-628. [36] S. Grosseman, Novack, D.H., Duke, P., Mennin, S., Rosenzweig, S., Davis, T.J., Hojat, M., Residents' and standardized patients' perspectives on empathy: issues of agreement, Patient Educ. Couns. 96 (2014) 22-28. 25
[37] B. Wright, J. McKendree, L. Morgan, V.L. Allgar, A. Brown, Examiner and simulated patient ratings of empathy in medical student final year clinical examination: are they useful? BMC Med. Educ. 14 (2014). [38] M. Aomatsu, T. Otani, A. Tanaka, N. Ban, J. van Dalen, Medical students' and residents' conceptual structure of empathy: a qualitative study, Educ. Health (Abingdon). 26 (2013) 4-8. [39] K. Berg, B. Blatt, J. Lopreiato, J. Jung, A. Schaeffer, D. Heil, T. Owens, P.L. CarterNolan, D. Berg, J. Veloski, E. Darby, M. Hojat, Standardized patient assessment of medical student empathy: Ethnicity and gender effects in a multi-institutional study, Acad. Med. 90 (2015) 105-111. [40] J. Yu, M. Kirk, Evaluation of empathy measurement tools in nursing: systematic review, J. Adv. Nurs. 65 (2009) 1790-1806. [41] I. Preusche, C. Lamm, Reflections on empathy in medical education: what can we learn from social neurosciences? Adv. in Health Sci. Educ. 21 (2016) 235249.
Legends 26
Figure 1: JSPE scores and honoring core clerkships in medical students
Table 1: Student Demographics (N=590) Age (median (IQR)) Female (N (%)) High Debt (> $150,000) (N (%)) Specialty Affinity (N (%)) Neither Technology oriented* People oriented† Both
26 (3) 319 (54) 372 (63) 406 (69) 37 (6) 144 (24) 3 (1)
*Internal Medicine, Family Medicine, Obstetrics/Gynecology, Psychiatry, Pediatrics, Neurology, Rehabilitation Medicine, Emergency Medicine, Ophthalmology, Dermatology †Radiology, Pathology, Surgery and Surgical Subspecialties, Radiation Oncology, Anesthesiology
27
Table 2: Regression analyses of JSPE and OSCE scores Outcome = OSCE communication score Bivariate Models
JSPE Age Gender Debt People affinity* Tech affinity†
Beta coefficient 0.065 -0.11 1.0 -0.017 -0.29 -1.36
Standard error 0.020 0.076 0.70 0.14 0.70 0.81
Multivariate model
P value 0.0012 0.17 0.15 0.90 0.68 0.093
Beta coefficient 0.061 -0.11 0.49 -0.055 -1.3 -0.5
Standard error 0.021 0.78 0.73 0.15 0.90 0.75
P value 0.0033 0.16 0.51 0.71 0.16 0.50
Outcome = overall OSCE score Bivariate Models
JSPE Age Gender Debt People affinity* Tech affinity†
Beta coefficient 0.027 -0.10 1.2 -0.13 0.20 -1.4
Standard error 0.011 0.045 0.38 0.090 0.41 0.46
Multivariate model
P value 0.016 0.028 0.001 0.15 0.62 0.0028
Beta coefficient 0.022 -0.086 1.1 -0.13 0.053 -1.3
Standard error 1.7 0.012 0.047 0.39 0.090 0.43
P value
*People-oriented specialty affinity: Internal Medicine, Family Medicine, Obstetrics/Gynecology, Psychiatry, Pediatrics, Neurology, Rehabilitation Medicine, Emergency Medicine, Ophthalmology, Dermatology †Technology-oriented specialty affinity: Radiology, Pathology, Surgery and Surgical Subspecialties, Radiation Oncology, Anesthesiology
28
0.067 0.067 0.0078 0.13 0.90 0.0082