- Email: [email protected]
Emotional intelligence and its correlation to performance as a resident: a preliminary study
Journal of Clinical Anesthesia (2008) 20, 84–89
Original contribution
Emotional intelligence and its correlation to performance as a resident: a preliminary study☆ Joseph F. Talarico DO (Assistant Professor)a,⁎, David G. Metro MD (Associate Professor)a , Rita M. Patel MD (Associate Professor)a , Patricia Carney PhD (Professor)b , Amy L. Wetmore BA (Residency Coordinator)a a
Department of Anesthesiology, University of Pittsburgh School of Medicine, Pittsburgh, PA Oregon Health & Science University, Portland, OR
b
Received 6 November 2006; revised 4 December 2007; accepted 4 December 2007
Keywords: Anesthesiologists; Emotional intelligence; Residency program
Abstract Study Objective: To test the hypothesis that emotional intelligence, as measured by the Bar-On Emotional Quotient Inventory (EQ-I) 125 (Multi Health Systems, Toronto, Ontario, Canada) personal inventory, would correlate with resident performance. Design: Prospective survey. Setting: University-affiliated, multiinstitutional anesthesiology residency program. Participants: Current clinical anesthesiology years one to three (PGY 2-4) anesthesiology residents enrolled in the University of Pittsburgh Anesthesiology Residency Program. Measurements: Participants confidentially completed the Bar-On EQ-I 125 survey. Results of the individual EQ-I 125 and daily evaluations by the faculty of the residency program were compiled and analyzed. Main Results: There was no positive correlation between any facet of emotional intelligence and resident performance. There was statistically significant negative correlation (−0.40; P b 0.05) between assertiveness and the “American Board of Anesthesiology essential attributes” component of the resident evaluation. Conclusions: Emotional intelligence, as measured by the Bar-On EQ-I personal inventory, does not strongly correlate to resident performance as defined at the University of Pittsburgh. © 2008 Elsevier Inc. All rights reserved.
1. Introduction Parameters commonly used in resident selection have shown little correlation to subsequent performance. Wood et al [1], George et al [2], Crane and Ferraro [3], Clark et al ☆
This study was supported by the University of Pittsburgh Department of Anesthesiology. ⁎ Corresponding author. UPMC Montefiore, Pittsburgh, PA 15213, USA. Tel.: +1 412 648 6946; fax: +1 412 648 6014. E-mail address: [email protected] (J.F. Talarico). 0952-8180/$ – see front matter © 2008 Elsevier Inc. All rights reserved. doi:10.1016/j.jclinane.2007.12.008
[4], Warrick and Crumrine [5], and Bell et al [6] found that performance on standardized examinations was not an indicator of clinical performance. Metro et al [7] found that resident selection score, as determined by the residency selection committee, correlated with the score on the first year in-training examination but did not correlate with parameters of clinical performance. Considering the questionable correlation between measures of pure intelligence and resident performance, this study assessed whether another potential indicator of resident performance—emotional intelligence—correlates with resident performance.
Prediction of resident performance Table 1 1-2-3
85
Residency evaluation, University of Pittsburgh's anesthesiology residency program 4-5-6
Unsatisfactory Satisfactory X = Unable to evaluate 1
ABA essential attributes • Ethical, moral, reliable, conscientious, responsible, honest • Learns from experience, knows limits • Reacts appropriately to stressful situations • No known substance abuse or other impairment that precludes responsibility for any aspect of patient care. • Respects patient dignity
2
Patient Care “Residents must be able to provide patient care that is compassionate, appropriate, and effective for the treatment of health problems and the promotion of health” • Preoperative evaluation, development of plan, room preparation • Skill in conduct of regional anesthesia, general anesthesia, monitoring and perioperative care, special procedures appropriate for level of experience • Work habits: organized, neat, efficient • Vigilant • Uses judgement: data collection, evaluation, problem solving, decision-making
3
Medical knowledge “Residents must demonstrate knowledge about established and evolving biomedical, clinical, and cognate sciences and the application of this knowledge to patient care.” • Demonstrates knowledge of general medicine appropriate for anesthesiology practice • Demonstrates knowledge pertaining specifically to anesthesiology • Able to answer faculty questions during preoperative case discussion • Possesses sufficient knowledge to discuss case intelligently; uses knowledge to organize facts • Able to apply knowledge to develop plan, care for patient
4
Professionalism “Residents must demonstrate a commitment to carrying out professional responsibilities, adherence to ethical principles, and sensitivity to a diverse patient population" • Motivated, punctual, industrious, takes initiative • Appropriately concerned for patients • Committed to learning, interested and curious • Adaptable and flexible • Careful and thorough
5
Communication and interpersonal skills “Residents must be able to demonstrate interpersonal and communication skills that result in effective information exchange and teaming with patient, their patients' families, and professional associates” • Courteous, cooperative, good rapport with other professionals and patients • Treats nonphysician professionals with respect • Able to communicate effectively with physicians and other professionals • Able to present and discuss cases in an organized, concise fashion • Accepts criticism • Keeps accurate and thorough anesthesia record, good charting out of OR
6
Practice-based learning and improvement “Residents must be able to investigate and evaluate their patient care practices, appraise and assimilate scientific evidence, and improve their patient care practices." • Recognizes gaps in knowledge and expertise • Learns from experience, does not repeat mistakes • Recognizes when to call for faculty assistance • Shows evidence of effort to continuously improve practice: discusses reading, asks questions, seeks and accepts faculty suggestions
7-8-9 Superior
86
J.F. Talarico et al.
Table 1 (continued) 1-2-3
4-5-6
7-8-9
Unsatisfactory Satisfactory
Superior
X = Unable to evaluate 7
Systems-based practice “Residents must demonstrate an awareness of and responsiveness to the larger context and system of health care, and the ability to effectively call on system resources to provide care that is of optimal value.” • Is an advocate for quality care Utilizes information technology and system resources to optimize patient care • Recognizes contributions of non-physician professionals to patient care and utilizes their expertise appropriately • Contacts and communicates with consultants and other physicians involved in the care of a patient • Acts as a consultant in anesthesiology and is involved in the perioperative care of patients • Demonstrates cognizance of the business and legal considerations relating to patient care
8 Overall Impression Comments It is essential to comment on any unsatisfactory elements of performance. Please illustrate with specific examples. ABA = American Board of Anesthesiology; OR = operating room.
Emotional intelligence is defined by Martinez [8] as an array of noncognitive skills, capabilities, and competencies that influence a person's ability to cope with environmental demands and pressures. Because this array of skills has been shown to correlate with success in the workplace, it would be reasonable to hypothesize that those who possess these skills would excel in a residency program. The concept is further defined by Duliewicz and Higgs [9], who focused on the subject of “emotional intelligence” and attempted to develop an emotional intelligence scale based on 16 relative competencies. The instrument that they developed showed promising reliability and validity. The validity of this measure was enhanced when considered in concert with intellectual intelligence and managerial intelligence. In spite of the difficulty in constructing a universally accepted definition of emotional intelligence, managers outside the realm of medicine have considered emotional intelligence a potential indicator of future performance, and many have come to seriously consider this indicator in selection of applicants for managerial positions. Goleman [10], in research performed at nearly 200 large global companies, found that effective leaders have a high degree of emotional intelligence. In the realm of medicine, Stratton et al [11] concluded that aspects of emotional intelligence are modestly implicated in students' clinical skills as assessed by standardized patients in an objective structured clinical examination. Wagner et al [12] suggested a limited relationship between physician emotional intelligence and patient satisfaction. Because it is intuitive that superior leadership and management skills are invaluable characteristics of the competent physician, it was hypothesized that exceptional emotional intelligence would correlate with perceived competence in residents. The purpose of this study was to conduct a preliminary test of the hypothesis that emotional intelligence, as
measured by the Bar-On Emotional Quotient Inventory (EQ-I 125; Multi-Health Systems, Toronto, Ontario, Canada) [13], is a predictor of resident performance and determine if a multiple year analysis with a large sample size would merit the considerable resources necessary to conduct such a study.
2. Materials and methods After receiving University of Pittsburgh School of Medicine institutional review board approval, our CA1-3 (clinical anesthesiology years 1-3, which correspond to PGY 2-4) residents were asked voluntarily to submit to the Bar-On EQ-I 125 emotional quotient inventory in March and April. Requirement for written, informed consent was waived by the institutional review board. The EQ-I 125 is a personal inventory survey that consists of 125 questions concerning attributes that encompass emotional intelligence. The residents who consented to take part in the study completed the survey by secure access to the Web site of Multi-Health Systems, the organization that owns the rights to the EQ-I inventory. Privacy was maintained as follows: each resident was given a confidential identifier and a password for access by our residency coordinator. The principal investigator obtained the results with numeric identification from the same Web site. No participants or other investigators were given access to results. For the participant's own edification, individual results were placed in a sealed, numbered envelope by a study-blinded individual and distributed by the residency coordinator, who had access to the residents' names and corresponding number. The residency coordinator also supplied the primary investigator with a
Correlational analysis of emotional intelligence a and residents' performance on ACGME and other competencies (n = 26)
Emotional intelligence
ABA essential attributes
Patient care
Medical knowledge
Professionalism
Communication and interpersonal Skills
Practice-based learning and improvement
Systems-based practice
Total performance score
Interpersonal #1 Self-regard Emotional self-awareness Assertiveness Independence Self-actualization Interpersonal #2 Empathy Social responsibility Interpersonal relationships Stress management Stress tolerance Impulse control Adaptability Reality testing Flexibility Problem solving General mood Optimism Happiness Total emotional intelligence score
−0.11 0.03 −0.11 −0.40 ⁎ −0.22 0.16 0.12 0.05 −0.07 0.04 0.00 −0.20 0.21 −0.22 −0.32 −0.10 −0.22 0.08 −0.01 0.25 −0.05
0.01 0.07 0.17 −0.27 −0.16 0.19 0.06 0.18 −0.09 0.10 0.06 −0.04 0.13 −0.06 −0.13 0.01 −0.11 0.13 0.04 0.21 0.04
−0.04 0.03 0.07 −0.33 −0.12 0.16 −0.04 0.10 −0.14 0.00 0.04 −0.02 0.09 −0.07 −0.15 0.00 −0.11 0.13 0.02 0.17 −0.01
0.04 0.06 0.22 −0.23 −0.13 0.22 0.17 0.25 0.02 0.18 0.17 −0.03 0.27 0.03 −0.09 0.13 −0.04 0.23 0.11 0.32 0.13
0.01 0.04 0.22 −0.27 −0.15 0.17 0.18 0.27 0.04 0.18 0.13 −0.07 0.25 −0.02 −0.12 0.07 −0.06 0.17 0.04 0.29 0.10
−0.09 −0.02 0.10 −0.36 −0.22 0.12 0.00 0.15 −0.13 0.04 −0.01 −0.14 0.13 −0.13 −0.21 −0.01 −0.22 0.04 −0.06 0.15 −0.04
−0.02 0.01 0.18 −0.32 −0.15 0.18 0.08 0.20 −0.04 0.10 0.12 −0.04 0.22 −0.03 −0.10 0.05 −0.09 0.15 0.05 0.23 0.06
0.04 0.09 0.21 −0.25 −0.12 0.23 0.11 0.20 −0.04 0.14 0.12 −0.02 0.19 −0.02 −0.10 0.07 −0.09 0.19 0.07 0.28 0.10
Prediction of resident performance
Table 2
ACGME = Accreditation Council for Graduate Medical Education. a As measured using the Bar-On Emotional Quotient Inventory (Eq-125). ⁎ P ≤ 0.05.
87
88 summary of the results of each participant's daily evaluations from July through December of the previous calendar year (Table 1). The summary results, identified by number only, were compiled, as well as a global evaluation of each participant by the faculty members of the evaluation and competence committee. Evaluations used for this study were completed before the respondents' completion of the EQ-I 125. Although emotional intelligence may change over time, generally any short-term change would be minimal and would be unlikely to affect the results.
2.1. Data analysis Results of the EQ-I in each of 15 categories and an overall EQ-I score were correlated with results in each of 7 categories of resident performance and an overall resident performance score. Overall scores in the EQ-I and daily resident evaluations were not an average of the individual categories but were determined independently. Summary data on a mean of 30 daily evaluations per resident completed by 122 faculty members were used for correlation to emotional intelligence data. Evaluations were based on the individual faculty members' impression of clinical performance. The components of this evaluation included American Board of Anesthesiology (ABA) attributes, the 6 core competencies established by the Accreditation Council for Graduate Medical Education, and an overall evaluation of resident performance. Participating in the study were 26 of 46 residents who were enrolled in the residency program at the time of the survey, for a participation rate of approximately 56.5%. Response rates by year were 14 (74%) of 19 CA-3 residents, 8 (67%) of 12 CA-2 residents, and 5 (36%) of 14 CA-1 residents. Because different scale values were used, data were normalized by converting mean scores on all measures to percentiles. We then used Pearson correlation coefficients to examine the relationships between emotional intelligence and resident performance. All tests were 2-tailed, and α was set at P ≤ 0.05 to test for statistical significance.
3. Results Correlation coefficients among variables within the EQ-I instrument were consistently significant, as published previously [14]. Correlations between the variables on each instrument with the other ranged from −0.40 to 0.33 and yielded one statistically significantly negative correlation (Table 2), which was found between assertiveness and ABA essential attributes. Two other variables, assertiveness and problem-based learning and improvement, were moderately negatively correlated but did not achieve statistical significance (Pearson r = −0.36; P = 0.07).
J.F. Talarico et al.
4. Discussion Considering the difficulty encountered by the previously cited authors in selecting residents who excelled in the clinical setting, it is important to investigate untried indicators of resident performance with the objective of uncovering new parameters that may improve the resident selection process. Studies conducted outside the realm of medicine have suggested that emotional intelligence is a strong indicator of success in managerial positions. Studies conducted in medical students and physicians showed correlation between emotional intelligence and performance on an objective structured clinical examination and patient satisfaction, respectively. It is reasonable to think that there are a number of attributes that would be equally predictive of success in managers and physicians (eg, self-confidence, self-awareness, stress management, empathy). It was therefore hypothesized that a high degree of emotional intelligence would have the same predictive value in resident performance as in management performance. Statistical analysis of the collected data, with one notable exception, showed no correlation between emotional intelligence, as measured by the Bar-On EQ-I, and resident performance, as measured by daily faculty evaluation of residents. The exception was the finding of a negative correlation between degree of assertiveness, as determined by the EQ-I 125, and the ABA essential attributes component of our resident evaluation (Table 2). The fact that the ABA essential attributes component encompasses traits that may be perceived by some evaluators to be counter to assertiveness (ie, knows limits, reacts appropriately to stressful situations) may explain the negative correlation. The determination that only one indicator correlated (negatively) with one aspect of our evaluation suggests that a comprehensive study using a large sample size would be necessary to show utility in resident selection. Conducting a study of this size would involve a multiyear, multiinstitutional, or multispecialty sampling. This study, like the preponderance of previous studies on resident selection, failed to extract a reliable indicator that could be of assistance to those who are involved in the resident selection process. In addition to the small sample size, other factors may have contributed to our inability to show any significant correlation between emotional intelligence and performance. As previously stated in the study of Metro et al [7], it is possible that the selection process is successful in eliminating residents who will not perform well, thereby eliminating the potential control group. The percentage of residents who chose to participate in this study was highest in the CA-3 year (74%), followed by the CA-2 year (67%) and the CA-1 year (36%). The impact of this bias is not clear, as the determination of whether emotional intelligence varies as a resident progresses through the program is beyond the scope of this study. Finally, the resident evaluation system may not effectively
Prediction of resident performance differentiate between competence and excellence. Like the evaluations used by virtually all residency programs, the evaluation on which these conclusions has been made has not been validated independently. Potential weaknesses in the evaluation system include the large number of evaluators who may have had limited exposure to any individual resident, a lack of grading consistency among evaluators using a 9-digit Likert scale, and the relatively tight grouping of scores (mean, 7.4; range, 5.4-8.6). More research is needed in this area. The rationale for the hypothesis that emotional intelligence would correlate with resident performance was the assumption that characteristics of a good manager would correspond with those of a good resident. It is commonly presumed that a successful resident is one who works independently, functions well under stress, and exercises sound judgment. These qualities are also essential to success in management. The same characteristics, however, may not be perceived by some faculty to be desirable in a resident. Anecdotal evidence suggests that the management style of our peers varies from autocratic to democratic; the more autocratic among us tend to value flexibility and subservience to the attending physician over assertiveness, stress management, and other traits that have proven to be characteristic of a successful manager. Our finding that assertiveness correlates negatively with ABA essential attributes suggests that this may well be the case. Although the primary hypothesis clearly was not supported by this study, a number of questions that have not been addressed may warrant further study. As previously mentioned, the resident evaluation process has not been validated independently. Potential weaknesses of the process include the lack of consistency among evaluators, as well as the large number of evaluators. Normalization of evaluations based on evaluator inconsistency is not current practice. For example, a good resident possibly may have interacted with an inordinate number of faculty who tend to give lower grades, thereby resulting in a lower average evaluation than that of a weaker resident. In a large program residents interact with many faculty members, thereby resulting in greater potential for inconsistency in evaluations, especially when considering a small sample. In conclusion, this study found no significant correlation between emotional intelligence, as measured by the Bar-ON
89 EQ-I 125, and resident performance as measured by the evaluation process used by the anesthesiology residency program at the University of Pittsburgh. Although study size certainly limits the utility of this particular study, the primary purpose of this study was to determine the strength of correlation between emotional intelligence and performance and the potential design of further studies in this realm. The lack of a strong correlation between emotional intelligence and performance indicates that a large study, whether multiyear, multiinstitution, or multispecialty, would be required to determine the utility of emotional intelligence in resident selection.
References [1] Wood PS, Smith WL, Altmaier EM, Tarico VS, Franken EA Jr. A prospective study of cognitive and noncognitive selection criteria as predictors of resident performance. Invest Radiol 1990;25:855-9. [2] George JM, Young D, Metz EN. Evaluating selected internship candidates and their subsequent performances. Acad Med 1989;64:480-2. [3] Crane JT, Ferraro CM. Selection criteria for emergency medicine residency applicants. Acad Emerg Med 2000;7:54-60. [4] Clark R, Evans EB, Ivey FM, Calhoun JH, Hokanson JA. Characteristics of successful and unsuccessful applicants to orthopedic residency programs. Clin Orthop Relat Res 1989;241:257-64. [5] Warrick SS, Crumrine RS. Predictors of success in an anesthesiology residency. J Med Educ 1986;61:591-5. [6] Bell JG, Kanellitsas I, Shaffer L. Selection of obstetrics and gynecology residents on the basis of medical school performance. Am J Obstet Gynecol 2002;186:1091-4. [7] Metro DG, Talarico JF, Patel RM, Wetmore AL. The resident application process and its correlation to future performance as a resident. Anesth Analg 2005;100:502-5. [8] Martinez MN. The smarts that count. HR Magazine 1997;42:72-8. [9] Dulewicz V, Higgs M. Emotional intelligence, a review and evaluation study. J Managerial Psych 2000;15:341-72. [10] Goleman D. What makes a leader? Harv Bus Rev 1998;76:93-102. [11] Stratton TD, Elam CL, Murphy-Spencer AE, Quinlivan SL. Emotional intelligence and clinical skills: preliminary results from a comprehensive clinical performance evaluation. Acad Med 2005;80(10 suppl): S34-7. [12] Wagner PJ, Moseley GC, Grant MM, Gore JR, Owens C. Physicians' emotional intelligence and patient satisfaction. Fam Med 2002;34: 750-4. [13] Bar-On R. The Emotional Quotient Inventory (EQ-I): a test of emotional intelligence. Toronto: Multi-Health Systems; 1996. [14] Bar-On R. Bar-on emotional quotient inventory technical manual. Toronto: Multi Health Systems; 2004.
Original contribution
Emotional intelligence and its correlation to performance as a resident: a preliminary study☆ Joseph F. Talarico DO (Assistant Professor)a,⁎, David G. Metro MD (Associate Professor)a , Rita M. Patel MD (Associate Professor)a , Patricia Carney PhD (Professor)b , Amy L. Wetmore BA (Residency Coordinator)a a
Department of Anesthesiology, University of Pittsburgh School of Medicine, Pittsburgh, PA Oregon Health & Science University, Portland, OR
b
Received 6 November 2006; revised 4 December 2007; accepted 4 December 2007
Keywords: Anesthesiologists; Emotional intelligence; Residency program
Abstract Study Objective: To test the hypothesis that emotional intelligence, as measured by the Bar-On Emotional Quotient Inventory (EQ-I) 125 (Multi Health Systems, Toronto, Ontario, Canada) personal inventory, would correlate with resident performance. Design: Prospective survey. Setting: University-affiliated, multiinstitutional anesthesiology residency program. Participants: Current clinical anesthesiology years one to three (PGY 2-4) anesthesiology residents enrolled in the University of Pittsburgh Anesthesiology Residency Program. Measurements: Participants confidentially completed the Bar-On EQ-I 125 survey. Results of the individual EQ-I 125 and daily evaluations by the faculty of the residency program were compiled and analyzed. Main Results: There was no positive correlation between any facet of emotional intelligence and resident performance. There was statistically significant negative correlation (−0.40; P b 0.05) between assertiveness and the “American Board of Anesthesiology essential attributes” component of the resident evaluation. Conclusions: Emotional intelligence, as measured by the Bar-On EQ-I personal inventory, does not strongly correlate to resident performance as defined at the University of Pittsburgh. © 2008 Elsevier Inc. All rights reserved.
1. Introduction Parameters commonly used in resident selection have shown little correlation to subsequent performance. Wood et al [1], George et al [2], Crane and Ferraro [3], Clark et al ☆
This study was supported by the University of Pittsburgh Department of Anesthesiology. ⁎ Corresponding author. UPMC Montefiore, Pittsburgh, PA 15213, USA. Tel.: +1 412 648 6946; fax: +1 412 648 6014. E-mail address: [email protected] (J.F. Talarico). 0952-8180/$ – see front matter © 2008 Elsevier Inc. All rights reserved. doi:10.1016/j.jclinane.2007.12.008
[4], Warrick and Crumrine [5], and Bell et al [6] found that performance on standardized examinations was not an indicator of clinical performance. Metro et al [7] found that resident selection score, as determined by the residency selection committee, correlated with the score on the first year in-training examination but did not correlate with parameters of clinical performance. Considering the questionable correlation between measures of pure intelligence and resident performance, this study assessed whether another potential indicator of resident performance—emotional intelligence—correlates with resident performance.
Prediction of resident performance Table 1 1-2-3
85
Residency evaluation, University of Pittsburgh's anesthesiology residency program 4-5-6
Unsatisfactory Satisfactory X = Unable to evaluate 1
ABA essential attributes • Ethical, moral, reliable, conscientious, responsible, honest • Learns from experience, knows limits • Reacts appropriately to stressful situations • No known substance abuse or other impairment that precludes responsibility for any aspect of patient care. • Respects patient dignity
2
Patient Care “Residents must be able to provide patient care that is compassionate, appropriate, and effective for the treatment of health problems and the promotion of health” • Preoperative evaluation, development of plan, room preparation • Skill in conduct of regional anesthesia, general anesthesia, monitoring and perioperative care, special procedures appropriate for level of experience • Work habits: organized, neat, efficient • Vigilant • Uses judgement: data collection, evaluation, problem solving, decision-making
3
Medical knowledge “Residents must demonstrate knowledge about established and evolving biomedical, clinical, and cognate sciences and the application of this knowledge to patient care.” • Demonstrates knowledge of general medicine appropriate for anesthesiology practice • Demonstrates knowledge pertaining specifically to anesthesiology • Able to answer faculty questions during preoperative case discussion • Possesses sufficient knowledge to discuss case intelligently; uses knowledge to organize facts • Able to apply knowledge to develop plan, care for patient
4
Professionalism “Residents must demonstrate a commitment to carrying out professional responsibilities, adherence to ethical principles, and sensitivity to a diverse patient population" • Motivated, punctual, industrious, takes initiative • Appropriately concerned for patients • Committed to learning, interested and curious • Adaptable and flexible • Careful and thorough
5
Communication and interpersonal skills “Residents must be able to demonstrate interpersonal and communication skills that result in effective information exchange and teaming with patient, their patients' families, and professional associates” • Courteous, cooperative, good rapport with other professionals and patients • Treats nonphysician professionals with respect • Able to communicate effectively with physicians and other professionals • Able to present and discuss cases in an organized, concise fashion • Accepts criticism • Keeps accurate and thorough anesthesia record, good charting out of OR
6
Practice-based learning and improvement “Residents must be able to investigate and evaluate their patient care practices, appraise and assimilate scientific evidence, and improve their patient care practices." • Recognizes gaps in knowledge and expertise • Learns from experience, does not repeat mistakes • Recognizes when to call for faculty assistance • Shows evidence of effort to continuously improve practice: discusses reading, asks questions, seeks and accepts faculty suggestions
7-8-9 Superior
86
J.F. Talarico et al.
Table 1 (continued) 1-2-3
4-5-6
7-8-9
Unsatisfactory Satisfactory
Superior
X = Unable to evaluate 7
Systems-based practice “Residents must demonstrate an awareness of and responsiveness to the larger context and system of health care, and the ability to effectively call on system resources to provide care that is of optimal value.” • Is an advocate for quality care Utilizes information technology and system resources to optimize patient care • Recognizes contributions of non-physician professionals to patient care and utilizes their expertise appropriately • Contacts and communicates with consultants and other physicians involved in the care of a patient • Acts as a consultant in anesthesiology and is involved in the perioperative care of patients • Demonstrates cognizance of the business and legal considerations relating to patient care
8 Overall Impression Comments It is essential to comment on any unsatisfactory elements of performance. Please illustrate with specific examples. ABA = American Board of Anesthesiology; OR = operating room.
Emotional intelligence is defined by Martinez [8] as an array of noncognitive skills, capabilities, and competencies that influence a person's ability to cope with environmental demands and pressures. Because this array of skills has been shown to correlate with success in the workplace, it would be reasonable to hypothesize that those who possess these skills would excel in a residency program. The concept is further defined by Duliewicz and Higgs [9], who focused on the subject of “emotional intelligence” and attempted to develop an emotional intelligence scale based on 16 relative competencies. The instrument that they developed showed promising reliability and validity. The validity of this measure was enhanced when considered in concert with intellectual intelligence and managerial intelligence. In spite of the difficulty in constructing a universally accepted definition of emotional intelligence, managers outside the realm of medicine have considered emotional intelligence a potential indicator of future performance, and many have come to seriously consider this indicator in selection of applicants for managerial positions. Goleman [10], in research performed at nearly 200 large global companies, found that effective leaders have a high degree of emotional intelligence. In the realm of medicine, Stratton et al [11] concluded that aspects of emotional intelligence are modestly implicated in students' clinical skills as assessed by standardized patients in an objective structured clinical examination. Wagner et al [12] suggested a limited relationship between physician emotional intelligence and patient satisfaction. Because it is intuitive that superior leadership and management skills are invaluable characteristics of the competent physician, it was hypothesized that exceptional emotional intelligence would correlate with perceived competence in residents. The purpose of this study was to conduct a preliminary test of the hypothesis that emotional intelligence, as
measured by the Bar-On Emotional Quotient Inventory (EQ-I 125; Multi-Health Systems, Toronto, Ontario, Canada) [13], is a predictor of resident performance and determine if a multiple year analysis with a large sample size would merit the considerable resources necessary to conduct such a study.
2. Materials and methods After receiving University of Pittsburgh School of Medicine institutional review board approval, our CA1-3 (clinical anesthesiology years 1-3, which correspond to PGY 2-4) residents were asked voluntarily to submit to the Bar-On EQ-I 125 emotional quotient inventory in March and April. Requirement for written, informed consent was waived by the institutional review board. The EQ-I 125 is a personal inventory survey that consists of 125 questions concerning attributes that encompass emotional intelligence. The residents who consented to take part in the study completed the survey by secure access to the Web site of Multi-Health Systems, the organization that owns the rights to the EQ-I inventory. Privacy was maintained as follows: each resident was given a confidential identifier and a password for access by our residency coordinator. The principal investigator obtained the results with numeric identification from the same Web site. No participants or other investigators were given access to results. For the participant's own edification, individual results were placed in a sealed, numbered envelope by a study-blinded individual and distributed by the residency coordinator, who had access to the residents' names and corresponding number. The residency coordinator also supplied the primary investigator with a
Correlational analysis of emotional intelligence a and residents' performance on ACGME and other competencies (n = 26)
Emotional intelligence
ABA essential attributes
Patient care
Medical knowledge
Professionalism
Communication and interpersonal Skills
Practice-based learning and improvement
Systems-based practice
Total performance score
Interpersonal #1 Self-regard Emotional self-awareness Assertiveness Independence Self-actualization Interpersonal #2 Empathy Social responsibility Interpersonal relationships Stress management Stress tolerance Impulse control Adaptability Reality testing Flexibility Problem solving General mood Optimism Happiness Total emotional intelligence score
−0.11 0.03 −0.11 −0.40 ⁎ −0.22 0.16 0.12 0.05 −0.07 0.04 0.00 −0.20 0.21 −0.22 −0.32 −0.10 −0.22 0.08 −0.01 0.25 −0.05
0.01 0.07 0.17 −0.27 −0.16 0.19 0.06 0.18 −0.09 0.10 0.06 −0.04 0.13 −0.06 −0.13 0.01 −0.11 0.13 0.04 0.21 0.04
−0.04 0.03 0.07 −0.33 −0.12 0.16 −0.04 0.10 −0.14 0.00 0.04 −0.02 0.09 −0.07 −0.15 0.00 −0.11 0.13 0.02 0.17 −0.01
0.04 0.06 0.22 −0.23 −0.13 0.22 0.17 0.25 0.02 0.18 0.17 −0.03 0.27 0.03 −0.09 0.13 −0.04 0.23 0.11 0.32 0.13
0.01 0.04 0.22 −0.27 −0.15 0.17 0.18 0.27 0.04 0.18 0.13 −0.07 0.25 −0.02 −0.12 0.07 −0.06 0.17 0.04 0.29 0.10
−0.09 −0.02 0.10 −0.36 −0.22 0.12 0.00 0.15 −0.13 0.04 −0.01 −0.14 0.13 −0.13 −0.21 −0.01 −0.22 0.04 −0.06 0.15 −0.04
−0.02 0.01 0.18 −0.32 −0.15 0.18 0.08 0.20 −0.04 0.10 0.12 −0.04 0.22 −0.03 −0.10 0.05 −0.09 0.15 0.05 0.23 0.06
0.04 0.09 0.21 −0.25 −0.12 0.23 0.11 0.20 −0.04 0.14 0.12 −0.02 0.19 −0.02 −0.10 0.07 −0.09 0.19 0.07 0.28 0.10
Prediction of resident performance
Table 2
ACGME = Accreditation Council for Graduate Medical Education. a As measured using the Bar-On Emotional Quotient Inventory (Eq-125). ⁎ P ≤ 0.05.
87
88 summary of the results of each participant's daily evaluations from July through December of the previous calendar year (Table 1). The summary results, identified by number only, were compiled, as well as a global evaluation of each participant by the faculty members of the evaluation and competence committee. Evaluations used for this study were completed before the respondents' completion of the EQ-I 125. Although emotional intelligence may change over time, generally any short-term change would be minimal and would be unlikely to affect the results.
2.1. Data analysis Results of the EQ-I in each of 15 categories and an overall EQ-I score were correlated with results in each of 7 categories of resident performance and an overall resident performance score. Overall scores in the EQ-I and daily resident evaluations were not an average of the individual categories but were determined independently. Summary data on a mean of 30 daily evaluations per resident completed by 122 faculty members were used for correlation to emotional intelligence data. Evaluations were based on the individual faculty members' impression of clinical performance. The components of this evaluation included American Board of Anesthesiology (ABA) attributes, the 6 core competencies established by the Accreditation Council for Graduate Medical Education, and an overall evaluation of resident performance. Participating in the study were 26 of 46 residents who were enrolled in the residency program at the time of the survey, for a participation rate of approximately 56.5%. Response rates by year were 14 (74%) of 19 CA-3 residents, 8 (67%) of 12 CA-2 residents, and 5 (36%) of 14 CA-1 residents. Because different scale values were used, data were normalized by converting mean scores on all measures to percentiles. We then used Pearson correlation coefficients to examine the relationships between emotional intelligence and resident performance. All tests were 2-tailed, and α was set at P ≤ 0.05 to test for statistical significance.
3. Results Correlation coefficients among variables within the EQ-I instrument were consistently significant, as published previously [14]. Correlations between the variables on each instrument with the other ranged from −0.40 to 0.33 and yielded one statistically significantly negative correlation (Table 2), which was found between assertiveness and ABA essential attributes. Two other variables, assertiveness and problem-based learning and improvement, were moderately negatively correlated but did not achieve statistical significance (Pearson r = −0.36; P = 0.07).
J.F. Talarico et al.
4. Discussion Considering the difficulty encountered by the previously cited authors in selecting residents who excelled in the clinical setting, it is important to investigate untried indicators of resident performance with the objective of uncovering new parameters that may improve the resident selection process. Studies conducted outside the realm of medicine have suggested that emotional intelligence is a strong indicator of success in managerial positions. Studies conducted in medical students and physicians showed correlation between emotional intelligence and performance on an objective structured clinical examination and patient satisfaction, respectively. It is reasonable to think that there are a number of attributes that would be equally predictive of success in managers and physicians (eg, self-confidence, self-awareness, stress management, empathy). It was therefore hypothesized that a high degree of emotional intelligence would have the same predictive value in resident performance as in management performance. Statistical analysis of the collected data, with one notable exception, showed no correlation between emotional intelligence, as measured by the Bar-On EQ-I, and resident performance, as measured by daily faculty evaluation of residents. The exception was the finding of a negative correlation between degree of assertiveness, as determined by the EQ-I 125, and the ABA essential attributes component of our resident evaluation (Table 2). The fact that the ABA essential attributes component encompasses traits that may be perceived by some evaluators to be counter to assertiveness (ie, knows limits, reacts appropriately to stressful situations) may explain the negative correlation. The determination that only one indicator correlated (negatively) with one aspect of our evaluation suggests that a comprehensive study using a large sample size would be necessary to show utility in resident selection. Conducting a study of this size would involve a multiyear, multiinstitutional, or multispecialty sampling. This study, like the preponderance of previous studies on resident selection, failed to extract a reliable indicator that could be of assistance to those who are involved in the resident selection process. In addition to the small sample size, other factors may have contributed to our inability to show any significant correlation between emotional intelligence and performance. As previously stated in the study of Metro et al [7], it is possible that the selection process is successful in eliminating residents who will not perform well, thereby eliminating the potential control group. The percentage of residents who chose to participate in this study was highest in the CA-3 year (74%), followed by the CA-2 year (67%) and the CA-1 year (36%). The impact of this bias is not clear, as the determination of whether emotional intelligence varies as a resident progresses through the program is beyond the scope of this study. Finally, the resident evaluation system may not effectively
Prediction of resident performance differentiate between competence and excellence. Like the evaluations used by virtually all residency programs, the evaluation on which these conclusions has been made has not been validated independently. Potential weaknesses in the evaluation system include the large number of evaluators who may have had limited exposure to any individual resident, a lack of grading consistency among evaluators using a 9-digit Likert scale, and the relatively tight grouping of scores (mean, 7.4; range, 5.4-8.6). More research is needed in this area. The rationale for the hypothesis that emotional intelligence would correlate with resident performance was the assumption that characteristics of a good manager would correspond with those of a good resident. It is commonly presumed that a successful resident is one who works independently, functions well under stress, and exercises sound judgment. These qualities are also essential to success in management. The same characteristics, however, may not be perceived by some faculty to be desirable in a resident. Anecdotal evidence suggests that the management style of our peers varies from autocratic to democratic; the more autocratic among us tend to value flexibility and subservience to the attending physician over assertiveness, stress management, and other traits that have proven to be characteristic of a successful manager. Our finding that assertiveness correlates negatively with ABA essential attributes suggests that this may well be the case. Although the primary hypothesis clearly was not supported by this study, a number of questions that have not been addressed may warrant further study. As previously mentioned, the resident evaluation process has not been validated independently. Potential weaknesses of the process include the lack of consistency among evaluators, as well as the large number of evaluators. Normalization of evaluations based on evaluator inconsistency is not current practice. For example, a good resident possibly may have interacted with an inordinate number of faculty who tend to give lower grades, thereby resulting in a lower average evaluation than that of a weaker resident. In a large program residents interact with many faculty members, thereby resulting in greater potential for inconsistency in evaluations, especially when considering a small sample. In conclusion, this study found no significant correlation between emotional intelligence, as measured by the Bar-ON
89 EQ-I 125, and resident performance as measured by the evaluation process used by the anesthesiology residency program at the University of Pittsburgh. Although study size certainly limits the utility of this particular study, the primary purpose of this study was to determine the strength of correlation between emotional intelligence and performance and the potential design of further studies in this realm. The lack of a strong correlation between emotional intelligence and performance indicates that a large study, whether multiyear, multiinstitution, or multispecialty, would be required to determine the utility of emotional intelligence in resident selection.
References [1] Wood PS, Smith WL, Altmaier EM, Tarico VS, Franken EA Jr. A prospective study of cognitive and noncognitive selection criteria as predictors of resident performance. Invest Radiol 1990;25:855-9. [2] George JM, Young D, Metz EN. Evaluating selected internship candidates and their subsequent performances. Acad Med 1989;64:480-2. [3] Crane JT, Ferraro CM. Selection criteria for emergency medicine residency applicants. Acad Emerg Med 2000;7:54-60. [4] Clark R, Evans EB, Ivey FM, Calhoun JH, Hokanson JA. Characteristics of successful and unsuccessful applicants to orthopedic residency programs. Clin Orthop Relat Res 1989;241:257-64. [5] Warrick SS, Crumrine RS. Predictors of success in an anesthesiology residency. J Med Educ 1986;61:591-5. [6] Bell JG, Kanellitsas I, Shaffer L. Selection of obstetrics and gynecology residents on the basis of medical school performance. Am J Obstet Gynecol 2002;186:1091-4. [7] Metro DG, Talarico JF, Patel RM, Wetmore AL. The resident application process and its correlation to future performance as a resident. Anesth Analg 2005;100:502-5. [8] Martinez MN. The smarts that count. HR Magazine 1997;42:72-8. [9] Dulewicz V, Higgs M. Emotional intelligence, a review and evaluation study. J Managerial Psych 2000;15:341-72. [10] Goleman D. What makes a leader? Harv Bus Rev 1998;76:93-102. [11] Stratton TD, Elam CL, Murphy-Spencer AE, Quinlivan SL. Emotional intelligence and clinical skills: preliminary results from a comprehensive clinical performance evaluation. Acad Med 2005;80(10 suppl): S34-7. [12] Wagner PJ, Moseley GC, Grant MM, Gore JR, Owens C. Physicians' emotional intelligence and patient satisfaction. Fam Med 2002;34: 750-4. [13] Bar-On R. The Emotional Quotient Inventory (EQ-I): a test of emotional intelligence. Toronto: Multi-Health Systems; 1996. [14] Bar-On R. Bar-on emotional quotient inventory technical manual. Toronto: Multi Health Systems; 2004.