Incremental validity of the Mayer–Salovey–Caruso Emotional Intelligence Test Version 2.0 (MSCEIT) after controlling for personality and intelligence

Journal of Research in Personality 43 (2009) 60–65

Contents lists available at ScienceDirect

Journal of Research in Personality journal homepage: www.elsevier.com/locate/jrp

Incremental validity of the Mayer–Salovey–Caruso Emotional Intelligence Test Version 2.0 (MSCEIT) after controlling for personality and intelligence Eric Rossen *, John H. Kranzler University of Florida, Educational Psychology, 1703 East-West Highway, Apt. 203, Silver Spring, MD 20910, United States

a r t i c l e

i n f o

Article history: Available online 24 December 2008 Keywords: Emotional intelligence MSCEIT Big 5 Personality Cognitive ability

a b s t r a c t This study examined the incremental validity of the Mayer–Salovey–Caruso Emotional Intelligence Test Version 2.0 after controlling for general cognitive ability and the Big 5 personality factors. The criterion measures used were academic achievement, psychological well-being, peer attachment, positive relations with others, and alcohol use. Results of these analyses suggest that emotional intelligence (EI) explains a significant and moderate to large amount of unique variance for alcohol use and positive relations with others after controlling for cognitive ability and personality. Implications of these results for theory and future research on the ability model of EI are discussed. Ó 2008 Elsevier Inc. All rights reserved.

1. Introduction Of the theories of emotional intelligence (EI) that have been proposed over the past decade, none has attracted as much attention in the research literature as Mayer and Salovey’s (1997) ability model. In their model, EI consists of the ability to accurately perceive emotions, use emotions to facilitate thought, understand one’s own emotions and those of others, and manage emotions to promote adaptive behavior and personal growth. Mayer, Salovey, and Caruso (2002) developed the Mayer–Salovey–Caruso Emotional Intelligence Test Version 2.0 (MSCEIT) to measure overall EI and each of the four branches of the ability model. They have conducted an extensive program of research on the validity of the MSCEIT and its predecessors, the MSCEIT Research Version 1.1 (MSCEIT V1.1) and the Multi-Factor Emotional Intelligence Scale (MEIS; Mayer, Caruso, & Salovey, 1999). Mayer, Salovey, Caruso, and their colleagues have demonstrated that the MSCEIT can be objectively scored and its constructs reliably measured (for a review, see Mayer, Salovey, & Caruso, 2004). The pattern of correlations (convergent and discriminant) that have been reported between the MSCEIT and other measures are generally logical and consistent with theory. For example, moderate but significant correlations have been found between the MSCEIT and measures of cognitive ability and the Big 5 personality dimensions, suggesting that EI is related to but distinguishable from intelligence and personality (e.g., Bastian, Burns, & Nettelbeck, 2005; Brackett & Mayer, 2003; Brackett, Mayer, & Warner, 2004). In addition, scores on the MSCEIT have been found to correlate with important behavioral outcomes. For example, * Corresponding author. E-mail address: [email protected] (E. Rossen). 0092-6566/$ - see front matter Ó 2008 Elsevier Inc. All rights reserved. doi:10.1016/j.jrp.2008.12.002

positive correlations have been reported with academic achievement (Lyons & Schneider, 2005; Zeidner, Shani-Zinovich, Matthews, & Roberts, 2005), psychological well-being (Brackett & Mayer, 2003; Lopes, Salovey, & Straus, 2003), and peer attachment (Lopes et al., 2003; Lopes, Brackett, Nezlek, Schütz, Sellin, & Salovey, 2004), among others; and negative correlations have been reported between the MSCEIT and deviant (e.g., physical altercations, vandalism) and maladaptive behavior, such as cigarette, drug, and alcohol use (e.g., Brackett et al., 2004; Trinidad & Johnson, 2002). Mayer et al. (2004) asserted that the evidence substantiates the validity of the MSCEIT as a measure of EI and the importance of EI as a psychological construct. Despite the extensive amount of research on the ability model that has been conducted, Brody (2004) recently argued that ‘‘there is no convincing evidence that the MSCEIT provides incremental predictive validity over and above standard measures of intelligence and personality for important socially relevant outcomes” (p. 237). As he noted, a number of the studies supporting the incremental validity of the MSCEIT have not been published in peer-refereed journals (e.g., Formica, 1998; Rubin, 1999). Brody also argued that, when the MSCEIT has been found to explain a significant amount of variance beyond personality or intelligence, the amount of variance that is accounted for by EI is often ‘‘trivial”. Finally, few of the published studies examining the incremental validity of the MSCEIT have included measures of both personality and intelligence. In evaluating EI’s predictions, more can be learned by controlling for commonly used measures of socio-emotional traits such as the Big 5 and cognitive variables such as IQ to ascertain the unique contribution of EI to real-world outcomes. If EI does not explain a nontrivial amount of variance after controlling for personality and intelligence, then its importance and utility as a psychological construct is questionable.

E. Rossen, J.H. Kranzler / Journal of Research in Personality 43 (2009) 60–65

A number of studies published in peer reviewed journals have examined the incremental validity of the MSCEIT with a variety of outcome measures, several of which were published after Brody’s (2004) review (Barchard, 2003; Bastian et al., 2005; Brackett & Mayer, 2003; Brackett et al., 2004; Lopes et al., 2003; Lopes et al., 2004; Lyons & Schneider, 2005; Mestre, Guil, Lopes, Salovey, & Gil-Olarte, 2006; Márquez, Martin, & Brackett, 2006; Rode, Mooney, Arthuad-Day, Near, Baldwin, & Rubin, 2007; Rode, Mooney, Arthuad-Day, Near, Rubin, & Baldwin, 2008). Nonetheless, at the current time, only a small number of studies have used standard measures of both intelligence and personality as controls in the same study (e.g., Bastian et al.; 2005; Rode et al., 2007; Rode et al., 2008). In addition, although several studies claim to have investigated the incremental validity of the MSCEIT, they either used tasks with unsubstantiated validity to measure intelligence or personality, did not control for both personality and intelligence in the same analysis, or used performance on a brief verbal measure to estimate cognitive ability. Brief measures of verbal ability are not satisfactory estimates of general cognitive ability, or psychometric g (see Jensen, 1998). Because the g factor explains the majority of variance in performances that can be predicted from tests (e.g., Thorndike, 1985; Thorndike, 1986), determining the incremental validity of EI after controlling for g is necessary to determine its incremental predictive validity. Results of the most recent incremental validity studies that used standard measures to control for both personality and general cognitive ability are consistent with Brody’s (2004) review. For example, Bastian et al. (2005) found that scores on the MSCEIT accounted for a significant increment of variance (DR2 = 6%) in only one of five life skills assessed (viz., Anxious Thoughts) after controlling for the Big 5 and g. Results of Rode et al. (2007) indicated that the MSCEIT accounted for a statistically significant yet minimal amount of unique variance in public speaking effectiveness (R2 = 1%), but did not explain additional variance in group behavior effectiveness and academic performance. Last, Rode et al. (2008) found that overall EI did not explain a statistically significant increment of variance in GPA or life satisfaction after controlling for g, the Big 5, and long-term affect. Although results of these studies are consistent with Brody’s (2004) critique, the range of outcome measures used by Bastian et al. (2005) and Rode et al. (2007, 2008) is limited, particularly concerning maladaptive behavior. One could argue that EI may lack substantial incremental validity for the specific criteria examined in these studies, but it may explain a significant and nontrivial amount of variance in other important and socially relevant real-world outcomes that were not examined. The aim of this study, therefore, was to examine the incremental validity of the MSCEIT after controlling for both general cognitive ability and personality using standardized measures with known psychometric properties using external criteria of both prosocial and maladaptive behaviors not examined by Bastian et al. (2005) or Rode et al. (2007, 2008) is limited.

2. Method 2.1. Participants Participants were 150 undergraduate students from over 30 fields of study at a 4-year public university in North Central Florida (62 frosh, 43 sophomores, 24 juniors, and 21 seniors; 40 men and 110 women). Participants volunteered to participate to earn research credit or extra credit in an undergraduate course. Age of participants ranged from 17 to 36 years (M = 19.7, SD = 2.1). Regarding race/ethnicity, 68.7% of the participants identified themselves as White/Non-Hispanic, 14.7% as African American, 9.3% as Hispanic American, 5.3% as Asian American, 1.3% as Other, and

61

0.7% as Native American. All participants were treated in accordance with the ‘‘Ethical Principles of Psychologists and Code of Conduct” (American Psychological Association., 2002). 2.2. Instruments 2.2.1. Emotional intelligence The Mayer–Salovey–Caruso Emotional Intelligence Test Version 2.0 (MSCEIT; Mayer et al., 2002) was used to measure EI. The MSCEIT consists of 141 items measuring the ability to solve emotional problems. The MSCEIT has eight subtests that are intended to measure the four branches of the ability model (i.e., perceiving emotions, using emotions, understanding emotions, and managing emotions) with two subtests per branch. Perceiving emotions is comprised of the faces and pictures subtests. These subtests measure the ability to perceive and appraise emotions in faces and pictures/designs. Examinees must identify the intensity of emotion expressed in several images of faces and designs using the following emotions: anger, sadness, happiness, disgust, fear, surprise, and excitement. Using emotions consists of the sensations and facilitation subtests. On these tasks, examinees must identify specific emotions that may affect one’s behavior or performance on cognitive tasks. Specifically, examinees must generate emotions or moods and match them with sensations, behaviors, or tasks that typically accompany them. Understanding emotions is measured by the blends and changes subtests. On the blends subtest, examinees must demonstrate an understanding of how individuals may experience several emotions simultaneously, and how some emotions, when combined, form other emotions. The changes subtest asks respondents to identify how emotions change over time. Finally, managing emotions is measured by the emotion management and emotional regulation subtests. These tasks provide examinees with social situations and require them to select the most appropriate social response to achieve desired outcomes (Mayer, 2001). The overall EI score has a mean of 100 and standard deviation of 15. Reported split-half reliabilities range from 0.79 to 0.91 for the four factors, and 0.91 for Overall EI (Mayer et al., 2002). Test-retest reliability was 0.86 for Overall EI after three weeks (Brackett & Mayer, 2003). Results of recent confirmatory factor analysis research has raised questions about the factor structure of the MSCEIT, however (Gignac, 2005; Palmer, Gignac, Manocha, & Stough, 2005; Rossen, Kranzler, & Algina, 2008). These results suggest that the MSCEIT does not measure all the constructs it was intended to measure, with the exception of overall EI. Therefore, given the lack of agreement in the literature about the constructs measured by the first-order factors of the ability model, only the overall EI score was used in this study. The MSCEIT can be scored using either the consensus-based or expert-rating scoring method. The correlation between these two methods for the overall EI score is .98 (Mayer et al., 2002), indicating that scores obtained from either scoring method are virtually identical. The consensus scoring method was used in this study. 2.2.2. Personality The traits from the ‘‘Big 5” were measured with the International Personality Item Pool (IPIP; Goldberg, 1999). The IPIP consists of 100 items measuring the Big 5 personality dimensions (extraversion, agreeableness, conscientiousness, emotional stability, and openness to experience). Goldberg reported high internal consistencies for the scales: extraversion (a = 0.91), agreeableness (a = 0.88), conscientiousness (a = 0.88), emotional stability (a = 0.91), and openness to experience (a = 0.90). Criterion-related evidence of validity has been established with the NEO-PI-R (Costa & McCrae, 1992), with correlations ranging from 0.88 to 0.93. Results of confirmatory factor analysis also support the internal structure of the IPIP (Lim & Ployhart, 2002).

62

E. Rossen, J.H. Kranzler / Journal of Research in Personality 43 (2009) 60–65

2.2.3. General cognitive ability Intelligence was measured with the Wonderlic Personnel Test (WPT; Wonderlic, 1992). The WPT is a widely-used, objective measure of general cognitive ability (g). It can be administered to adults individually or in groups. The WPT is a timed test (12 min) consisting of 50 items that increase in difficulty as the test progresses, yielding one total score. Test item subject areas include verbal, quantitative, and spatial ability. The reliability and validity of the WPT as a measure of general cognitive ability is well-established (Geisinger, 2001). The WPT manual reports split-half reliability estimates from 0.88 to 0.94, and test-retest reliability estimates from 0.84 to 0.94 after up to a 5-year period (Dodrill, 1983; Schoenfeldt, 1985; Wonderlic, 1992). Evidence of concurrent validity with various measures of intelligence has been well-established (e.g., Dodrill, 1981; Hawkins, Faraone, Pepple, Seidman, & Tsuang, 1990; McKelvie, 1992; Mullane & McKelvie, 2001), which have indicated correlations between 0.91 and 0.93 with the Wechsler Adult Intelligence Scale (WAIS; Wechsler, 1955) and correlations between 0.84 and 0.86 with the WAIS – Revised (WAIS-R; Wechsler, 1981). 2.2.4. Academic achievement Participants provided reports of college grade point average (GPA). Self-reported GPA scores have demonstrated highly significant correlations with official records (r = 0.97) (Cassady, 2001). 2.2.5. Psychological well-being To assess psychological well-being, participants completed the Scales of Psychological Well-Being-Short Form (SPWB-SF; Ryff, 1989a). This survey consists of 43 items designed to measure six dimensions of well-being: autonomy, environmental mastery, personal growth, positive relations with others, purpose in life, and self-acceptance. Participants were asked to respond to various statements and indicate on a six-point Likert scale how true each statement is of them. Higher scores indicate greater well-being. The internal consistencies of the scales for a sample of 202 adults are moderate to good: autonomy (a = 0.77), environmental mastery (a = 0.80), personal growth (a = 0.81), positive relations with others (a = 0.79), purpose in life (a = 0.86), and self-acceptance (a = 0.84) (C.D. Ryff, personal communication, January 18, 2006). In a longitudinal study consisting of 6875 adults, the internal consistencies of the scales reported were moderate: autonomy (a = 0.70), environmental mastery (a = 0.72), personal growth (a = 0.76), positive relations with others (a = 0.78), purpose in life (a = 0.79), and self-acceptance (a = 0.79) (C.D. Ryff, personal communication, January 18, 2006). Test-retest reliability estimates are available only for the 20-item parent scales, and range from 0.81 to 0.88 (Ryff, 1989b). Correlations between the 20-item scales and related measures of positive functioning (i.e., life satisfaction, self-esteem, affect balance, internal control, and morale) were all positive and significant; whereas correlations with measures of negative functioning (i.e., external locus of control, depression) were negative and significant (Ryff, 1989a). 2.2.6. Peer attachment To assess quality of social relationships, participants completed Section 3 of the Inventory of Parent and Peer Attachment (IPPA; Armsden & Greenberg, 1987). This section contains 25 items on a five-point Likert scale focusing on peer relationships. Section 3 of the IPPA is a self-report measure with questions about relationships in three areas: trust (10 items; a = 0.91), communication (8 items; a = 0.87), and alienation (7 items; a = 0.72). An overall score, peer attachment, is also available (a = 0.92). To obtain an overall peer attachment score, negatively worded items were reversescored and then the response values in each section were summed

together. Three-week test–retest reliability for the overall peer attachment scale was .86. Evidence of criterion-related validity reported by the authors indicates that the peer attachment scale correlates with related measures of social relationships. Scores on the IPPA correlated 0.57 with the Tennessee Self-Concept Scale (TSCS; Fitts, 1965) and 0.32 with the peer utilization factor from the Inventory of Adolescent Attachment (Greenberg, Siegel, & Leitch, 1983). In addition, the IPPA was found to correlate with several affective states as measures by Bachman’s Affective States Index (Bachman, 1970). Specifically, scores correlated positively and significantly with self-esteem and life-satisfaction; scores correlated negatively and significantly with alienation, irritability/anger, depression/anxiety, and guilt (Armsden & Greenberg, 1987). 2.2.7. Alcohol use To measure alcohol use, participants completed the Alcohol Use Disorders Identification Test (AUDIT; Babor, Higgins-Biddle, Saunders, & Monteiro, 2001). This measure was developed to identify harmful drinking habits and consists of 10 items measuring alcohol use and dependency. Response format is on a five-point Likert scale, with higher scores indicating higher levels of alcohol use/abuse. The AUDIT has demonstrated concurrent validity with other measures of alcohol use/dependency, and has also successfully differentiated among hazardous and non-hazardous drinkers in college populations (Bohn, Babor, & Kranzler, 1995; Kokotailo, Egan, Gangnon, Brown, Mundt, Fleming 2004). In a study comparing scores on the AUDIT with DSM diagnostic criteria among college students, the AUDIT showed high sensitivity, correctly identifying 94% of the sample who had met diagnostic criteria as high-risk alcohol users, and moderate specificity, correctly classifying approximately two-thirds of the individuals who did not meet diagnostic high-risk alcohol use (Fleming, Barry, & MacDonald, 1991). 2.3. Procedure Participants completed all the measures in one test session. Total testing time was approximately 1.5 h. Participants completed the measures in randomized order using a balanced Latin-square to control for order effects. 2.4. Data analysis Descriptive statistics were calculated for each of the predictor and criterion measures. Pearson product–moment correlation coefficients were calculated to examine the relationships between cognitive ability, personality, and EI. Third, a series of hierarchical regression analyses were conducted to assess the incremental validity of the overall EI score of the MSCEIT for predicting each of the respective criterion variables (academic success, psychological well-being, positive relations with others, peer attachment, and alcohol use). For each analysis, general cognitive ability was entered in the first step, the Big 5 personality dimensions were entered in the second step, and the overall EI score of the MSCEIT was entered in the third step. Finally, semi-partial correlations were calculated between overall EI and the criterion variables after controlling for general cognitive ability and the Big 5. 3. Results 3.1. Descriptive statistics Table 1 presents the descriptive statistics for all predictor and criterion variables. As can be seen, these results are comparable to those found in studies with similar samples and are not unduly

63

E. Rossen, J.H. Kranzler / Journal of Research in Personality 43 (2009) 60–65 Table 1 Descriptive statistics (N = 150). Variable

M

WPT IPIP Extraversion Agreeableness Conscientiousness Emotional stability Openness MSCEIT overall EI GPA AUDIT IPPA SPWB

SD

23.4

4.9

71.9 82.6 74.2 68.2 74.9 100.1 3.3 5.1 106.7 208.4

13.6 9.2 13.7 13.9 10.7 12.1 0.47 5.1 10.9 24.8

Range

a

Criterion variables

11–37 40–99 52–98 39–100 32–100 47–99 68–130 1.7–4.0 0–29 62–124 115–257

Table 3 Hierarchical regression of criterion variables on the WPT (Step 1), IPIP (Step 2), and the MSCEIT (Step 3), and semi-partial correlations after controlling for WPT and IPIP.

0.89

GPA Psychological wellbeing Positive relations with others Peer attachment Alcohol use

Step 1 DR2 0.03* 0.03*

Step 2 DR 2

Step 3 DR2

Semi-partial r with overall EI

0.13** 0.58**

0.00 0.00

0.01 0.04

0.03*

0.46**

0.01*

0.11*

0.02 0.00

0.17** 0.13**

0.01 0.04*

0.07 0.21*

Notes: *p < 0.05, **p < 0.01. N = 148. MSCEIT = Mayer–Salovey–Caruso Emotional Intelligence Test Version 2.0; IPIP = International Personality Item Pool; WPT = Wonderlic Personnel Test.

Notes: MSCEIT = Mayer–Salovey–Caruso Emotional Intelligence Test Version 2.0; IPIP = International Personality Item Pool; WPT = Wonderlic Personnel Test; GPA = Grade Point Average; SPWB = Scales of Psychological Well-Being; IPPA = Inventory of Parent and Peer Attachment; AUDIT = Alcohol Use Disorders Identification Test.

restricted in range. Although some restriction of range is evident for the WPT in comparison with the standardization sample, the amount of range restriction is small and common in university samples. The Cronbach’s alpha coefficient for overall EI (a = 0.89) indicated acceptable internal consistency. Table 2 displays results of correlations between Overall EI and measures of the Big 5 personality dimensions and psychometric g. These findings are also comparable to the results of previous research (Mayer, Salovey, & Caruso, 2008) and suggest that scores on the MSCEIT are relatively distinct from measures of personality and intelligence as predicted by construct theory (e.g., Mayer et al., 2004). Results of the hierarchical regression analyses are summarized in Table 3. For each analysis, general cognitive ability was entered in Step 1, the Big 5 in Step 2, and overall EI score on the MSCEIT in Step 3. As can be seen in this table, the amount of variance explained by general cognitive ability for all outcome variables was generally low (R2 ranging from 0.00 to 0.03). In contrast, the amount of variance explained by the Big 5 in the second step was generally moderate to high and statistically significant, with the greatest coefficients found for psychological well-being (R2 = 0.58) and positive relations with others (R2 = 0.46). Overall EI on the MSCEIT accounted for a statistically significant portion of variance in two of the criterion measures: positive relations with others (R2 = 1%) and alcohol use (R2 = 4%). These findings are generally consistent with results of previous research (Mayer et al., 2008), which suggest that EI is related to both socially adaptive and maladaptive behaviors. Overall EI did not account for a statistically significant amount of variance in self-reported GPA, peer attachment, or psychological well-being. Semi-partial correlations between overall EI and the criterion variables are also presented in Table 3 after controlling for cognitive ability and the Big 5. Notably, the semi-partial correlation for self-reported alcohol use is negative and significant, suggesting that those with higher EI are less likely to drink alcohol regularly. 4. Discussion For EI to be considered an important psychological construct, it must predict outcomes in the real-world beyond that accounted

for by well-established constructs that are theoretically related, such as general cognitive ability and personality. Brody’s (2004) recent review of research on the incremental validity of Mayer and Salovey’s (1997) ability model raised concerns about EI as an important psychological construct. Although many published studies have examined the incremental validity of the MSCEIT, few studies published in peer reviewed journals have examined the incremental validity of the MSCEIT after controlling for both general cognitive ability and personality using standardized measures with known psychometric properties (i.e., Bastian et al. 2005; Rode et al., 2007, 2008). Moreover, the criterion measures used within those studies were limited. The aim of this study, therefore, was to examine the incremental validity of the MSCEIT, after controlling for psychometric g and the Big 5, using theoretically relevant criterion measures that were both adaptive and maladaptive in nature and were not used in this prior research (viz., academic achievement, psychological well-being, peer attachment, positive relations with others, and alcohol use). Results of the hierarchical multiple regression analyses in this study were mixed. General cognitive ability accounted for a minor amount of variance for all outcome variables, including selfreported GPA. Range restriction on cognitive ability among the university sample used in this study most likely explains this outcome. In contrast, the Big 5 personality dimensions accounted for a moderate to large amount of variance for most outcome variables after being entered in the second step of the regression. These findings support the importance of the Big 5 personality dimensions, above and beyond that of general cognitive ability, across a wide array of social–emotional and academic outcomes. After controlling for general cognitive ability and personality, overall EI did not explain a statistically significant increment of variance in academic achievement, psychological well-being, and peer attachment. Overall EI, however, contributed statistically significantly to the prediction of positive relations with others and alcohol use. Although the amount of variance accounted for in these criteria was only 1% and 4%, respectively, it is important to note that we implemented an exceptionally stringent research design to examine the incremental validity of EI. In hierarchical regression within the social and behavioral sciences, when a number of variables are entered before the final step, such as in this study (i.e., six variables), small increments to the change in R2 do

Table 2 Pearson product–moment correlations between the WPT, IPIP, and MSCEIT. WPT

Overall EI

0.24*

IPIP Extraversion

Agreeableness

Conscientiousness

Stability

Openness

0.05

0.32*

0.12

0.24*

0.22*

Notes: *p < 0.05. MSCEIT = Mayer–Salovey–Caruso Emotional Intelligence Test Version 2.0; WPT = Wonderlic Personnel Test; IPIP = International Personality Item Pool.

64

E. Rossen, J.H. Kranzler / Journal of Research in Personality 43 (2009) 60–65

not necessarily indicate a lack of meaningful contribution to prediction of the criterion, particularly when there is considerable conceptual overlap (Ahadi & Deiner, 1989; Hunsley & Meyer, 2003). Hunter and Schmidt (2004), for example, propose that reliance on the interpretation of DR2 ‘‘leads to severe underestimates of the practical and theoretical significance of relationships between variables (p. 190)” and that ‘‘variables that account for small percentages of variance often have very important effects on the dependent variable.” (p. 190). They assertated that interpretation of the semi-partial r is a more meaningful measure of a predictor variable’s contribution. Hunsley and Meyer (2003) raised the issue of defining an appropriate metric for evaluating the meaningfulness of incremental predictive validity and the semi-partial r. They asserted that a semi-partial r of 0.15–0.20 on the third step represents an important contribution to prediction and takes into account the shared variance between predictor variables. As EI was entered after IQ in the first step and five dimensions of personality in the second, semi-partial rs of 0.11 and 0.21 for positive relations with others and alcohol use can be interpreted as meaningful and substantial. According to Cohen (1988), a ‘‘large” effect in the social and behavioral sciences explains 25% of the variance, a ‘‘medium” effect about 9%, and a ‘‘small” effect about 1%. On this ‘‘scale,” the results of research on the incremental validity of EI suggest it may explain a small to large amount of additional variance on the criteria with which it is related beyond that explained by personality and cognitive ability, depending on the outcome measure. Taken as a whole, results of this study and earlier research suggest that EI represents an important construct in predicting a statistically significant amount of unique variance in a range of outcomes related to social/emotional functioning beyond that accounted for general cognitive ability and personality. Results of this study, therefore, do not substantiate Brody’s (2004) criticism of the ability model of EI. Research results suggest that the incremental validity of EI may be comparable to or somewhat better than that of cognitive group factors, such as spatial and psychomotor abilities, that explain a moderate amount of variance independent of g, typically in a narrow range of outcomes (e.g., technical or motor skills). Research has yet to examine whether the construct of EI is related to other cognitive group factors, such as those in Carroll’s (1993) three-stratum theory of human cognitive abilities, however. Is it possible that EI is a broad cognitive ability at Stratum II of Carroll’s theory? Some contemporary theories of intelligence that have attempted to broaden the definition of intelligence beyond the traditional psychometric view have included constructs that are quite similar to EI, such as the interand intra-personal intelligences in Gardner’s (1983) theory of multiple intelligences. Although Jensen (1998) argued that ‘‘social intelligence” and other related constructs ‘‘are really achievement variables that reflect how different individuals invest g in activities as affected by their particular opportunities, interests, personality traits, and motivation” (p. 133), research has not examined whether this applies to the ability model of EI. A joint confirmatory factor analysis of the MSCEIT and a well-established measure of broad cognitive abilities at Stratum II of Carroll’s theory, such as the Woodcock–Johnson Psycho-educational battery – 3rd ed. (WJ-III; Woodcock, McGrew, & Mather, 2001), would provide an empirical test of this hypothesis. It is important to note that this study examined the evidence of convergent, discriminant, and incremental validity of EI using only the overall EI score on the MSCEIT. Other studies have examined the incremental validity of the branch and area scores of the MSCEIT (e.g., Brackett et al., 2004; Lopes et al., 2003; Mestre et al., 2006). Our decision to use only the overall EI score was based on recent research suggesting that, with the exception of the overall EI score, the MSCEIT may not measure all the constructs it was

intended to measure (Brody, 2004; Gignac, 2005; Palmer et al., 2005; Rossen et al., 2008). Results of these confirmatory factor analytic studies raised serious questions concerning the structural fidelity of the MSCEIT and do not support Mayer, Salovey, Panter and Caruso’s (2005) claim that one-, two-, and four-factor models of the structure of EI provide a good fit to the data. Specifically, while some studies suggest the viability of a four-factor model, several other studies have not supported the use of the lower-order Facilitating Emotions factor (Zeider, Roberts, & Matthews, 2008). One of the few consistent findings, however, is support for a unitary overall EI. While most research suggests the existence of lower-order factors, it remains unclear how these lower-order factors are defined, derived, and measured. Therefore, interpretation of the scale scores on the MSCEIT is questionable at the current time. Until the issue of the structural fidelity of the MSCEIT is resolved, only examination of the incremental validity of the overall EI appeared to be warranted. One limitation of this study was the use of several self-report measures. Self-reports are susceptible to inaccurate or distorted responses to create favorable impressions (Barrick & Mount, 1996). Despite efforts to reduce the likelihood of deceptive response styles by preserving participants’ anonymity, research should examine the relationship between EI and actual behavior in addition to self-reported behavior using a multi-trait multi-method format (Caruso, Mayer, & Salovey, 2002). Another potential limitation is the over-representation of female participants. Previous research indicates that women tend to score slightly higher on measures of EI, on average, than men (Mayer et al., 2002). However, there is no evidence to suggest that the relationship between EI and external criteria differs according to gender. Given that our study examined the incremental validity of EI, having the sample reflect the population in terms of gender did not seem crucial. In sum, results of this study suggest that EI may explain a significant and nontrivial amount of unique variance in a range of important and socially relevant outcomes related to social/emotional functioning, and additional research on EI may lead to advances in theory and practice (e.g., early identification and intervention). Future research on the predictiveness of EI is needed with other criteria and populations. For example, a paucity of research exists on the relationship between EI and rating scales and other objective measures of behavior (e.g., behavior referrals, diagnoses of emotional/behavioral disorders, school suspensions), social–emotional functioning, achievement, and temperament. Research on the development of EI among children and adolescents is also needed. Results of research in these areas will improve understanding of the contribution of EI to individual differences in important outcomes. References Ahadi, S., & Deiner, E. (1989). Multiple determinants and effect size. Journal of Personality and Social Psychology, 56, 398–406. American Psychological Association. (2002). Ethical principles of psychologists and codes of conduct. Washington, DC: Author. Armsden, G. C., & Greenberg, M. T. (1987). The inventory of parent and peer attachment: Individual differences and their relationship to psychological wellbeing in adolescence. Journal of Youth and Adolescence, 16, 427–454. Babor, T. F., Higgins-Biddle, J. C., Saunders, J. B., & Monteiro, M. G. (2001). AUDIT: The Alcohol Use Disorders Identification Test – guidelines for use in primary care (2nd ed.). Geneva: World Health Organization. Bachman, J. G. (1970). The impact of family background and intelligence on tenth-grade boys. Youth in Transition (Vol. II). Ann Arbor: University of Michigan Institute for Social Research. Barchard, K. A. (2003). Does emotional intelligence assist in the prediction of academic success? Educational and Psychological Measurement, 63, 840–858. Barrick, M. R., & Mount, M. K. (1996). Effects of impression management and selfdeception on the predictive validity of personality constructs. Journal of Applied Psychology, 81, 261–272. Bastian, V. A., Burns, N. R., & Nettelbeck, T. (2005). Emotional intelligence predicts life skills, but not as well as personality and cognitive abilities. Personality and Individual Differences, 39, 1135–1145.

E. Rossen, J.H. Kranzler / Journal of Research in Personality 43 (2009) 60–65 Bohn, M. J., Babor, T. F., & Kranzler, H. R. (1995). The Alcohol Use Disorders Identification Test (AUDIT): Validation of a screening instrument for use in medical settings. Journal of Studies on Alcohol, 56, 32–423. Brackett, M. A., & Mayer, J. D. (2003). Convergent, discriminant, and incremental validity of competing measures of emotional intelligence. Personality and Social Psychology Bulletin, 29, 1147–1158. Brackett, M. A., Mayer, J. D., & Warner, R. M. (2004). Emotional intelligence and its relation to everyday behaviour. Personality and Individual Differences, 36, 1387–1402. Brody, N. (2004). What cognitive intelligence is and what emotional intelligence is not. Psychological Inquiry, 15, 234–238. Carroll, J. B. (1993). Human cognitive abilities: A survey of factor-analytic studies. New York, Cambridge. Caruso, D. R., Mayer, J. D., & Salovey, P. (2002). Relation of an ability measure of emotional intelligence to personality. Journal of Personality Assessment, 79, 306–320. Cassady, J. C. (2001). Self-reported GPA and SAT: A methodological note. Practical Assessment, Research & Evaluation, 7(12). http://PAREonline.net/ getvn.asp?v=7&n=12 Retrieved January 10, 2006. Cohen, J. (1988). Statistical power analysis for the behavioral sciences (2nd ed.). Hillsdale, NJ: Earlbaum. Costa, P. T., Jr., & McCrae, R. R. (1992). NEO-PI-R professional manual-revised neo personality inventory (NEO-PIR) and NEO five-factor inventory (NEO-FFI). Odessa, FL: Psychological Assessment Resources. Dodrill, C. B. (1981). An economical method for the evaluation of general intelligence in adults. Journal of Consulting and Clinical Psychology, 49, 668–673. Dodrill, C. B. (1983). Long-term reliability of the Wonderlic Personnel Test. Journal of Consulting and Clinical Psychology, 51, 316–327. Fitts, W. (1965). Tennessee self-concept manual. Los Angeles: Western Psychological Services. Fleming, M. F., Barry, K. L., & MacDonald, R. (1991). The Alcohol Use Disorders Identification Test (AUDIT) in a college sample. International Journal of Addiction, 26, 1173–1185. Formica, S. (1998). Description of the socio-emotional life space: Life qualities and activities related to emotional intelligence. Unpublished thesis, University of New Hampshire, Durham, NH. Gardner, H. (1983). Frames of mind. New York: Basic Books. Geisinger, K. F. (2001). Review of the Wonderlic Personnel Test and scholastic level exam. In B. S. Plake & J. C. Impara (Eds.), The fourteenth mental measurements yearbook (pp. 1360–1363). Lincoln, NE: The Buros Institute of Mental Measurements. Gignac, G. E. (2005). Evaluating the MSCEIT V2.0 via CFA: Comment on Mayer et al. (2003). Emotion, 5, 233–235. Goldberg, L. R. (1999). A broad-bandwidth, public domain, personality inventory measuring the lower-level facets of several five-factor models. In I. Mervielde, I. Deary, F. De Fruyt, & F. Ostendorf (Eds.). Personality psychology in Europe (Vol. 7, pp. 7–28). Tilburg, The Netherlands: Tilburg University Press. Greenberg, M. T., Siegel, J. M., & Leitch, C. L. (1983). The nature and importance of attachment relationships to parents and peers during adolescence. Journal of Youth and Adolescence, 12, 373–386. Hawkins, K. A., Faraone, S. V., Pepple, J. R., Seidman, L. J., & Tsuang, M. T. (1990). WAIS-R validation of the Wonderlic Personnel Test as a brief intelligence measure in a psychiatric sample. Psychological Assessment, 2, 198–201. Hunsley, J., & Meyer, G. J. (2003). The incremental validity of psychological testing and assessment: Conceptual, methodological, and statistical issues. Psychological Assessment, 15, 446–455. Hunter, J. E., & Schmidt, F. L. (2004). Methods of meta-analysis: Correcting error and bias in research findings (2nd ed.). CA, Sage: Thousand Oaks. Jensen, A. R. (1998). The g factor: The science of mental ability. Westport, CT: Praeger. Kokotailo, P. K., Egan, J., Gangnon, R., Brown, D., Mundt, M., & Fleming, M. (2004). Validity of The Alcohol Use Disorders Identification Test in college students. Alcoholism: Clinical and Experimental Research, 28, 914–920. Lim, B. C., & Ployhart, R. E. (2002). Assessing the convergent and discriminant validity of Goldberg’s International Personality Item Pool: A multitraitmultimethod examination. In: Paper presented at the 17th annual conference of the Society for Industrial and Organizational Psychology, Toronto, Canada. Lopes, P. N., Brackett, M. A., Nezlek, J. B., Schütz, A., Sellin, I., & Salovey, P. (2004). Emotional intelligence and social interaction. Personality and Social Psychology Bulletin, 30, 1018–1034. Lopes, P. N., Salovey, P., & Straus, R. (2003). Emotional intelligence, personality, and the perceived quality of social relationships. Personality and Individual Differences, 35, 641–658. Lyons, J. B., & Schneider, T. R. (2005). The influence of emotional intelligence on performance. Personality and Individual Differences, 39, 693–703.

65

Márquez, P. G., Martin, R. P., & Brackett, M. A. (2006). Relating emotional intelligence to social competence and academic achievement in high school students. Psichothema, 18, 118–123. Mayer, J. D. (2001). Emotion, intelligence, and emotional intelligence. In J. P. Forgas (Ed.), The handbook of affect and social cognition (pp. 410–431). Mahwah, NJ: Erlbaum. Mayer, J. D., Caruso, D. R., & Salovey, P. (1999). Emotional intelligence meets traditional standards for intelligence. Intelligence, 27, 267–298. Mayer, J. D., & Salovey, P. (1997). What is emotional intelligence? In P. Salovey & D. Sluyter (Eds.), Emotional development and emotional intelligence: Implications for educators (pp. 3–31). New York, NY: Basic Books. Mayer, J. D., Salovey, P., & Caruso, D. (2002). Mayer-Salovey-Caruso Emotional Intelligence Test (MSCEIT). Toronto, ON: MHS. Mayer, J. D., Salovey, P., & Caruso, D. R. (2004). Emotional intelligence: Theory, findings, and implications. Psychological Inquiry, 15, 197–215. Mayer, J. D., Salovey, P., & Caruso, D. (2008). Emotional intelligence: New ability or eclectic traits? American Psychologist, 63, 503–517. Mayer, J. D., Salovey, P., Panter, A. T., & Caruso, D. R. (2005). A discrepancy in analyses of the MSCEIT–Resolving the mystery and understanding of its implications: A reply to Gignac (2005). Emotion, 5, 236–237. McKelvie, S. J. (1992). Does memory contaminate test–retest reliability? Journal of General Psychology, 119, 59–72. Mestre, J. M., Guil, R., Lopes, P. N., Salovey, P., & Gil-Olarte, P. (2006). Emotional intelligence and social and academic adaptation to school. Psicothema, 18, 112–117. Mullane, J., & McKelvie, S. J. (2001). Effects of removing the time limit on first and second language intelligence test performance. Practical Assessment, Research & Evaluation, 7(23). http://PAREonline.net/getvn.asp?v=7&n=23 Retrieved October 10, 2005. Palmer, B. R., Gignac, G., Manocha, R., & Stough, C. (2005). A psychometric evaluation of the Mayer–Salovey-Caruso Emotional Intelligence Test Version 2.0. Intelligence, 33, 285–305. Rode, J. C., Mooney, C. H., Arthuad-Day, M. L., Near, J. P., Baldwin, T. T., Rubin, R. S., et al. (2007). Emotional intelligence and individual performance: Evidence of direct and moderated effects. Journal of Organizational Behavior, 28, 399–421. Rode, J. C., Mooney, C. H., Arthuad-Day, M. L., Near, J. P., Rubin, R. S., Baldwin, T. T., et al. (2008). An examination of the structural, discriminant, nomological, and incremental predictive validity of the MSCEIT V2.0. Intelligence, 36, 350–366. Rossen, E., Kranzler, J. H., & Algina, J. (2008). Confirmatory Factor Analysis of the Mayer–Salovey–Caruso Emotional Intelligence Test V2.0 (MSCEIT). Personality and Individual Differences, 44, 1258–1269. Rubin, M. M. (1999). Emotional intelligence and its role in mitigating aggression: A correlational study of the relationship between emotional intelligence and aggression in urban adolescents. Unpublished dissertation, Immaculata College, Immaculata, Pennsylvania. Ryff, C. D. (1989a). Happiness is everything, or is it? Explorations of the meaning of psychological well-being. Journal of Personality and Social Psychology, 57, 1069–1081. Ryff, C. D. (1989b). Beyond Ponce de Leon and life satisfaction: New directions in the quest of successful aging. International Journal of Behavioral Development, 12, 35–55. Schoenfeldt, L. F. (1985). Review of the Wonderlic Personnel Test. In J. V. Mitchell, Jr. (Ed.). The ninth mental measurements yearbook (Vol. 2). Lincoln, NE: University of Nebraska Press. Thorndike, R. L. (1985). The central role of general ability in prediction. Multivariate Behavioral Research, 20, 241–254. Thorndike, R. L. (1986). The role of general ability in prediction. Journal of Vocational Behavior, 29, 332–339. Trinidad, D. R., & Johnson, C. A. (2002). The association between emotional intelligence and early adolescent tobacco and alcohol use. Personality and Individual Differences, 32, 95–105. Wechsler, D. (1955). Manual for the wechsler adult intelligence scale. NewYork, NY: The Psychological Corporation. Wechsler, D. (1981). Manual for the wechsler adult intelligence scale-revised. New York, NY: The Psychological Corporation. Wonderlic, E. F. (1992). Wonderlic Personnel Test user’s manual. Libertyville, IL: E. F. Wonderlic. Woodcock, R. W., McGrew, K. S., & Mather, N. (2001). Woodcock–Johnson III tests of cognitive abilities. Itasca, IL: Riverside. Zeider, M., Roberts, R. D., & Matthews, G. (2008). The science of emotional intelligence: Current consensus and controversies. European Psychologist, 13, 64–78. Zeidner, M., Shani-Zinovich, I., Matthews, G., & Roberts, R. D. (2005). Assessing emotional intelligence in gifted and non-gifted high school students: Outcomes depend on the measure. Intelligence, 33, 369–391.