- Email: [email protected]
An examination of the factor structure of the schutte self-report emotional intelligence (SSREI) scale via confirmatory factor analysis
Personality and Individual Differences 39 (2005) 1029–1042 www.elsevier.com/locate/paid
An examination of the factor structure of the schutte self-report emotional intelligence (SSREI) scale via confirmatory factor analysis Gilles E. Gignac a
a,*
, Benjamin R. Palmer b, Ramesh Manocha c, Con Stough
b
School of Psychology, Deakin University, 221 Burwood Highway, Burwood, Victoria 3125, Australia b Centre for Neuropsychology, Swinburne University, Australia c Barbara Gross Research Unit, Faculty of Medicine, University of New South Wales, Australia Received 15 July 2004; accepted 14 March 2005 Available online 16 June 2005
Abstract Research to-date on the dimensionality of the Schutte Self-Report Emotional Intelligence (SSREI; Schutte et al., 1998) scale appears to support a four-factor interpretation, corresponding to Optimism, Social Skills, Emotional Regulation and Utilization of Emotions. However, the model of EI upon which the SSREI is based (Salovey & Mayer, 1990) has never been considered when determining the number of factors to extract/model in the factor analyses. Thus, in this investigation, we examined the CFA fit of several models, comparing the four-factor model reported by Saklofske, Austin, and Minski (2003), and the sixfactor model of EI described by Salovey and Mayer (1990). The CFA results indicated that two of the six dimensions of the Salovey and Mayer (1990) model of EI could not be identified, independently of first-order general and acquiescent factors. Specifically, while Ôappraisal of emotions in the selfÕ, Ôappraisal of emotions in othersÕ, Ôemotional regulation of the selfÕ, and Ôutilizing emotions in problem solvingÕ were identified, Ôemotional regulation of othersÕ and Ôemotional expressionÕ were not. The results are discussed in
*
Corresponding author. Fax: +61 3 9214 5230. E-mail address: [email protected] (G.E. Gignac).
0191-8869/$ - see front matter Ó 2005 Elsevier Ltd. All rights reserved. doi:10.1016/j.paid.2005.03.014
1030
G.E. Gignac et al. / Personality and Individual Differences 39 (2005) 1029–1042
light of how the SSREI could be potentially improved for the purposes of measuring the dimensions within the Salovey and Mayer model (1990). Ó 2005 Elsevier Ltd. All rights reserved. Keywords: Emotional intelligence; Self-report; CFA
1. Introduction Although a few exploratory/confirmatory factor analytic investigations exist in the literature that have examined the factor structure of the Schutte Self-Report Emotional Intelligence (SSREI; Schutte et al., 1998) scale, none of the investigations have considered the EI model upon which the sale is based, when deciding the number of factors to extract/model. Schutte et al. (1998) based the SSREI on the original Salovey and MayerÕs (1990) theory of emotional intelligence, because they believed that the original model better characterized an individualÕs present level of emotional development. As reported by Schutte et al. (1998), Salovey and MayerÕs (1990) original model included three categories: (1) Ôappraisal and expression of emotionsÕ; (2) Ôregulation of emotionsÕ; and (3) Ôutilization of emotions in solving problemsÕ. However, within these three categories are sub-categories. Specifically, the appraisal aspect of the Ôappraisal and expression of emotionsÕ category can be divided into Ôappraisal of emotions in the selfÕ and Ôappraisal of emotions in othersÕ. Similarly, the Ôregulation of emotionsÕ category can be subdivided into Ôregulation of emotions in the selfÕ and Ôregulation of emotions in othersÕ. Finally, and perhaps less clearly, Ôutilization of emotionsÕ, can be subdivided into four components: (1) Ôflexible planningÕ, Ôcreative thinkingÕ, Ôredirected attentionÕ and ÔmotivationÕ. Thus, in total, there are 10 first-order categories of EI within the Salovey and Mayer (1990) model, as described by Schutte et al. (1998). However, four of these categories are subsumed by the more nebulous Ôutilisation of emotionsÕ dimension. Thus, it could be argued that there are six primary dimensions within Salovey and MayerÕs (1990) model of EI. Schutte et al. (1998) generated an initial pool of 62 items to reflect all of the categories of Salovey and MayerÕs (1990) model of EI, and then performed a principal components analysis on the 62 item questionnaire, which was administered to 346 individuals. Based on their interpretation of a scree plot, Schutte et al. (1998) extracted four components and rotated the components, orthogonally. However, Schutte et al. (1998) reported that only the first component had an appreciable number of component loadings greater than .40, which was the criterion they used to demarcate a significant loading from a non-significant loading. Perplexingly, however, Schutte et al. (1998) reported that all of the categories of the Salovey and Mayer (1990) model were represented within the first component. Consequently, only the 33 items that exhibited component loadings greater than .40 on the first component were retained for the purpose of forming the published version of the questionnaire. Schutte et al.Õs (1998) component analysis can be criticized on a number of levels. First, it would have been more appropriate to perform a factor analysis on the 62 items, rather than a component analysis. Based on a comprehensive simulation study, Widaman (1993) demonstrated that greater factor structure accuracy (i.e., loadings and factor inter-correlations) can be achieved with factor analysis, in contrast to principal component analysis, to the extent that the factor loadings are
G.E. Gignac et al. / Personality and Individual Differences 39 (2005) 1029–1042
1031
expected to be smaller, rather than larger. Thus, in the case of analysing items, which tend to be relatively unreliable, it would have been especially advisable to perform factor analysis, rather than components analysis. Further, because items tend to be relatively unreliable, the demarcation criterion of .40 used by Schutte et al. (1998) for the purpose of identifying significant factor loadings may have been too strict, even for principal components analysis, which tends to overestimate the factor reliability of the items in the analysis (Widaman, 1993). Moreover, as argued by Petrides and Furnham (2000),1 the components/factors should have been rotated obliquely, not orthogonally. Theoretically, the categories of Salovey and MayerÕs (1990) model of EI would be expected to exhibit positive intercorrelations. Orthogonal rotations preclude any correlations between components/factors (Gorsuch, 1983). Further, and perhaps most surprisingly, given the results reported in Schutte et al. (1998), an orthogonal rotation will preclude the existence of a general factor (Jensen, 1998, p. 66). Schutte et al. (1998) reported (p. 171) the eigenvalues for the rank ordered components as 10.79, 3.58, 2.90, and 2.53. These values appear to be derived from an unrotated solution. In all likelihood, an orthogonal rotation of the components would have spread the common variance relatively evenly from the first component, probably a general component, to the other components retained in the analysis. Thus, it is very possible that the component that Schutte et al. (1998) interpreted, and ultimately used as the basis for creating the current SSREI inventory, was a general component, derived from an unrotated factor solution. Another argument in favour of the general component interpretation resides in the fact that the published version of the SSREI inventory includes items from all of the broad dimensions of Salovey and MayerÕs (1990) model of EI. Had Schutte et al. (1998) interpreted the rotated component solution, it is very doubtful that any of the components would have contained component loadings of .40 from such a multifarious collection of items, which just happen to represent all of the categories from the Salovey and MayerÕs (1990) model of EI. Since the publication of Schutte et al. (1998), a few published papers have investigated the itemlevel factor structure of the SSREI. For instance, Petrides and Furnham (2000) sought to test the hypothesis, via CFA, that the SSREI inventory measured a single, general factor of EI, based on a sample of 260 university students. The general factor model was associated with close-fit index values that indicated that the general factor model was unacceptable (e.g., CFI = .51, RMSEA = .105). Consequently, Petrides and Furnham (2000) followed their CFA analysis with an unrestricted (exploratory) principal components analysis. Based on Petrides and FurnhamÕs judgement, four components were extracted and rotated both orthogonally and obliquely. The components were found to correlate below j.30j. It is unclear whether Petrides and Furnham (2000) present the factor solution for the oblique or orthogonal rotations. The four components
1
Petrides and Furnham (2000) argued that Schutte et al. (1998) performed ‘‘a ÔLittle JiffyÕ procedure’’ and that this was the most significant psychometric problem with the Schutte et al. (1998) component analysis (p. 317). However, it appears that Petrides and Furnham (2000) have confused the expression ÔLittle JiffyÕ for varimax rotation. ÔLittle JiffyÕ refers to several principles related to performing a component/factor analysis. Specifically, the method of communality estimation, the number of components/factors to extract, and the rotation transformation (among other things). Originally, part of the Little Jiffy included an orthogonal rotation procedure (Kaiser, 1960); however, KaiserÕs Ôsecond generation Little JiffyÕ (Kaiser, 1970) included an oblique rotation procedure, because Kaiser (1970) preferred oblique rotation over orthogonal rotation.
1032
G.E. Gignac et al. / Personality and Individual Differences 39 (2005) 1029–1042
extracted by Petrides and Furnham (2000) corresponded to Optimism, Appraisal of Emotions, Social Skills, and Utilisation of Emotions. Saklofske et al. (2003) performed an unrestricted, exploratory principal components analysis on the extraction of four components (oblique rotation). The factor solution obtained by Saklofske et al. (2003) corroborated closely the factor solution obtained by Petrides and Furnham (2000), with the four components corresponding to Optimism, Appraisal of Emotions, Utilisation of Emotions, and Social Skills. However, when Saklofske et al. (2003) attempted to test the four-factor model via CFA, the model was found to be poor fitting. Unfortunately, Saklofske et al. (2003) do not report the specific fit CFA close-fit values for the four-factor model. HakanenÕs (2004) EFA investigation into the dimensionality of the SSREI suffers from the same limitations reported above: component analysis and orthogonal rotation. Given that two previous CFA investigations failed to support the four-factor model within the SSREI, it was decided that a further investigation into the factor structure of the SSREI was merited. However, in contrast to Petrides and Furnham (2000) and Saklofske et al. (2003), it was hypothesized that the SSREI would conform more closely to the theoretical six-factor model, rather than the four-factor model. Further, because the SSREI inventory is extremely unbalanced (30 positively keyed items and only three negatively keyed items), it was hypothesized that it would be necessary to model a first-order acquiescence factor, in conjunction with the substantive factors, for the purpose of achieving satisfactory model fit. Several investigations have demonstrated the existence of an acquiescence factor, when analysing self-report questionnaires via CFA at the item level (see Billiet & McClendon, 2000). Prior to statistical testing, we analyzed and categorized, qualitatively, the 33 items of the SSREI inventory for the purpose of classification into the conceptual categories of Salovey and MayerÕs (1990) model of EI. Based on our qualitative analysis, the six substantive categories were identified within the SSREI inventory. The corresponding items for each category are listed in Table 1. It can be seen that two of the dimensions only have two items each: Ôappraisal of emotions in the self Õ (AES) and Ôemotional expressionÕ (EE). Further, five items could not be readily classified into any of the six theoretical dimensions. Thus, we sought to test the hypothesis, via CFA, that the six-factor model was a more plausible model of the dimensions within the SSREI inventory, in comparison to the four-factor model, currently advocated by several investigators (e.g., Petrides & Furnham, 2000; Saklofske et al., 2003). Further, we hypothesized the existence of a first-order acquiescence factor, in conjunction to the six first-order substantive factors. Table 1 Items corresponding to the theorized dimensions of the SSREI, based on a qualitative analysis of the 33 items (see Schutte et al., 1998 for item descriptions) Dimensions
Items
Appraisal of Emotions in the Self (AES) Appraisal of Emotions in Others (AEO) Emotional Expression (EE) Emotional Regulation of the Self (ERS) Emotional Regulation of Others (ERO) Utilization of Emotions in Problem Solving (UEPS) Uncategorized
9, 5, 1, 2, 4, 7, 6,
22 15, 18, 25, 29, 32, 33 11 3, 12, 14, 23, 28, 31, 10 13, 16, 24, 30 17, 20, 27 8, 19, 21, 26
G.E. Gignac et al. / Personality and Individual Differences 39 (2005) 1029–1042
1033
2. Method 2.1. Participants The sample consisted of 367 participants (107 males, 257 females, 3 unreported), ranging in age from 15 to 78 (Mean = 38.3 years, SD = 13.7). The participants were obtained from the general population across the states of Victoria and New South Wales (Australia) via advertisements. 2.2. Materials The Schutte Self-Report Emotional Intelligence (SSREI) scale by Schutte et al. (1998) is comprised of 33, 5-point Likert scale items, three of which are negatively keyed. Previous investigations have found the total scores on the SSREI to be acceptably internally consistent (e.g., .90; Schutte et al., 1998). 2.3. Procedure Once informed consent was obtained from the participants, test booklets were provided, which included the SSREI inventory and a scannable answer sheet. The participants were given an unlimited amount of time to complete the paper based inventory, and they received a small monetary reward for participating. 2.4. Data analytic strategy The CFA analyses (AMOS 5.0) will be based on the 28 items selected in the introduction above. The first model that will be tested (model 1) will be the hypothesized six-factor model based on Salovey and MayerÕs (1990) framework of EI. The approach to modeling the six-factors will be based on a nested CFA modeling strategy. Specifically, rather than model the covariance between the six factors as an oblique factor model or a higher-order general factor model, a nested factors modeling approach will be used (Gustafsson & Balke, 1993). Thus, all 28 items will be specified to load onto a first-order general factor, directly. Further, all 25 of the positively keyed items will be specified to load onto a first-order acquiescence factor. A graphical depiction of the first model that will be tested can be seen in Fig. 1. For the purposes of model comparison, a single general factor model, and a general factor model with an acquiescence factor will also be tested via CFA. It is hypothesized that the hypothesized six-factor model will be associated with practically better fit, in comparison to the general factor model, and the general factor model with an acquiescence factor. Finally, a nested factors model consistent with the EFA findings of Saklofske et al. (2003) will be tested. Thus, in comparison to the hypothesized six-factor model reported above, the Saklofske et al. (2003) model will consist of four nested factors, corresponding to Optimism, Appraisal, Utilization, and Social. For the purposes of fair comparison, the Saklofske et al. (2003) four-factor model will also be based on only 28 items, as described above for the hypothesized six-factor model. It is hypothesized that the six-factor model will be associated with practically better model fit than the Saklofske et al. (2003) four-factor model. A graphical depiction of the Saklofske et al. (2003) model can be seen in Fig. 2.
1034
G.E. Gignac et al. / Personality and Individual Differences 39 (2005) 1029–1042 10 3 23
14 ERS
12 28 2 31 11
EE
1 13 30
ERO
24 16 4
EI'g'
Acquiescence
15 32 5
AEO
29 25 18 33 9
AES
22 17 27
UEPS
7 20
Fig. 1. Hypothesized nested factors model based on Salovey and MayerÕs (1990) six-factor model of EI (ERS = Emotional Regulation of the Self; EE = Emotional Expression; ERO = Emotional Regulation of Others; AEO = Appraisal of Emotions in Others; AES = Appraisal of Emotions in the Self; UEPS = Utilization of Emotions for Problem Solving).
All CFA analyses will be based on a Pearson covariance matrix and Maximum Likelihood Estimation (MLE). Models will identified by constraining the latent variables to 1.0. In the case of a latent variable defined by only two observed variables, the factor loadings will be constrained to equality, in accordance with Little, Lindenberger, and Nesselroade (1999, pp. 204–205). In accordance with Hu and Bentler (1999), a combination approach will be used to evaluate model fit. Specifically, two absolute close-fit indices (SRMR and RMSEA) and two incremental close-fit indices were chosen (TLI and CFI). Also in accordance with Hu and Bentler (1999), models will be evaluated as good fitting, when the absolute fit indices (SRMR and RMSEA) are <.06 and the
G.E. Gignac et al. / Personality and Individual Differences 39 (2005) 1029–1042
1035
22 9 12 23 10 optimism
28 14 2 3
EI'g'
1 29 18
appraisal
25 32 5
Acquiescence
15 27 20 utilization
7 17 33 30 11 4
social
24 13 16 31
Fig. 2. A nested version of the Saklofske et al. (2003) four-factor model of the SSREI with a first-order general factor and an orthogonal acquiescence factor.
incremental fit indices (TLI and CFI) are approximately .95 or larger. Rather than compare the fit of models statistically, a practical significance difference test will be used based on TLI values. In accordance with Vandenberg and Lance (2000), a model with a TLI value .01 larger than another model will be considered practically better fitting.
3. Results Average absolute levels of item skew and kurtosis were .91 and .93, respectively, which is within the acceptable range for CFA analysis using maximum likelihood estimation (Muthen & Kaplan, 1985). A variant (see DeCarlo, 1997) of SmallÕs omnibus test of multivariate normality was rejected (VQ3(66) = 1139.13, p < .001); however, the majority (72%) of the multivariate
1036
G.E. Gignac et al. / Personality and Individual Differences 39 (2005) 1029–1042
non-normality was due to skewness (Q1(33) = 816.85, p < .001), rather than kurtosis (VQ2(33) = 322.28, p < .001), indicating that the MLE CFA results would likely be minimally affected by the multivariate non-normality. A null CFA model hypothesizing no covariation between the 28-items was rejected ðv2ð378Þ ¼ 3734.85, p < .001, RMSEA = .159, SRMR = .278, TLI = .000, CFI = .000), indicating that the inter-item covariation was appropriate for factor analysis. The hypothesized nested factors model with a first-order general factor, a first-order acquiescence factor, and nested first-order ERS, ERO, EE, AEO, AES, and UEPS factors produced a v2ð299Þ ¼ 561.16, p < .001. However, as can be seen in Table 2 (model 1), the factor loadings from the ERO and EE factors were not significant statistically. Further, an excessively large standard error (i.e., 8.69) was associated with item 30Õs unstandardized factor loading of 1.33. Finally, item 30 and 20Õs residual variances were associated with negative values of 1.33 and .14, which is unacceptable in an adequate CFA model. Thus, despite the decent close-fit index values (e.g., RMSEA = .050 and CFI = .922), the hypothesized model was not considered acceptable. A modified hypothesized model was tested which was identical to the hypothesized model, with the exception that the nested ERO and EE factors were removed from the model. The modified hypothesized model produced a v2ð305Þ ¼ 570.12, p < .001, and all of the standard errors were within an acceptable region, indicating that the model was potentially acceptable. In contrast to the original model, only one itemÕs residual variance was associated with a negative value. To test the hypothesis that the negative error variance may be due to sampling fluctuations, item 20Õs residual error variance was constrained to .0001, in accordance with Chen, Bollen, Paxton, Curran, and Kirby (2001). The modified model with the constrained error variance yielded a v2ð306Þ ¼ 570.72, p < .001, which was not statistically significantly worse fitting than the previous non-constrained model, indicating that the negative error variance was likely due to sampling fluctuations, rather than a fundamentally inappropriate model specification. An examination of the modification indices indicated that there was some unmodeled covariance between the AES and AEO factors. Because a positive association between these two factors was reasonable theoretically (i.e., were from the same domain of Ôappraisal of emotionsÕ), the modified model was retested with the addition of a covariance link between the AES and AEO factors, which produced a v2ð305Þ ¼ 559.40, p < .001. The correlation between the AES and AEO factors was estimated at .28 (CR = 3.65, p < .001). This final model was associated with absolute close-fit index values indicating good model fit (SRMR = .043 and RMSEA = .049), while the incremental close-fit index values indicated marginally good model fit (CFI = .924 and TLI = .906). An examination of the standardized residual covariances and modification index values (i.e., Lagrange multiplier) did not suggest any conspicuously large and/or meaningful changes to the model. As can be seen in Table 2 (model 2), all of the factor loadings were positive and statistically significant, with the exception of only three items on the acquiescence factor. A graphical depiction of the modified hypothesized model is presented in Fig. 3. For the purposes of model comparison, a general factor model was tested, which produced a v2ð35Þ ¼ 1358.83, p < .001, RMSEA = .091, SRMR = .079, CFI = .699, TLI = .675, indicating very poor model fit. A TLI difference test indicated substantial practical improvement in fit, favoring the modified hypothesized model, in comparison to the general factor model (DTLI = .231). Similarly, a general factor model with an acquiescence factor produced a v2ð325Þ ¼ 949.63, p < .001, RMSEA = .074, SRMR = .064, CFI = .814, TLI = .784, indicating poor fit, based on the
G.E. Gignac et al. / Personality and Individual Differences 39 (2005) 1029–1042
1037
Table 2 Maximum Likelihood Estimation standardized factor loadings for the hypothesized six-factor model Item
ÔgÕ
ERS
EE
ERO
Model 1: Hypothesized six-factor model 10 .53 .53 3 .55 .32 23 .42 .48 14 .55 .16 12 .54 .32 28 .44 .34 2 .42 .18 31 .31 .23 11 .46 .16 1 .46 .17 13 .51 30 .52 24 .58 16 .52 4 .57 15 .37 32 .34 5 .30 29 .23 25 .49 18 .39 33 .46 9 .67 22 .53 17 .39 27 .13 7 .15 20 .50 Item
ÔgÕ
ERS
Model 2: Modified hypothesized model 10 .52 .54 3 .55 .32 23 .42 .46 14 .55 .15 12 .53 .32 28 .44 .34 2 .42 .16 31 .33 .19 11 .46 1 .46 13 .52 30 .56 24 .60 16 .52
AEO
AES
UEPS
Acquiesce .08 .13 .41 .17 .33 .37 .60 .02 .16 .20 .17 .21 .28 .03 .25 .34
.04 1.67 .03 .02 .06 .44 .44 .52 .54 .60 .64 .26
.27 .18 .28 .52 .49
.19 .27 .39 .51 .40 .25
.29 .16 .15 .97 AEO
AES
UEPS
Acquiesce .09 .14 .43 .18 .34 .37 .60 .02 .16 .19 .15 .20 .28 (continued on next page)
1038
G.E. Gignac et al. / Personality and Individual Differences 39 (2005) 1029–1042
Table 2 (continued) Item
ÔgÕ
ERS
Model 2: Modified hypothesized model 4 .59 15 .35 32 .32 5 .29 29 .22 25 .47 18 .37 33 .45 9 .65 22 .51 17 .38 27 .13 7 .15 20 .50
AEO
AES
UEPS
Acquiesce .04 .24 .34
.46 .45 .52 .54 .62 .66 .27
.26 .18 .28 .55 .52 .34 .20 .19 .83
.18 .27 .39 .49 .39 .26
Note: Factor loadings in bold were not significant statistically (p > .05). The unstandardized factor loading standard errors (available upon request) for model 2 were associated with a mean of .053 and a range of .024–.076, which was considered acceptable, based on a sample size of 352 (i.e., 1/(N1/2) = 1/18.76 = .05).
incremental close-fit indexes. A TLI difference test indicated substantial practical improvement in fit in favor of the modified hypothesized model tested above, in comparison to the general + acquiescence factor model (DTLI = .122). The Saklofske et al. (2003) four-factor (28-item) model produced a v2ð298Þ ¼ 614.84, p < .001. Although the absolute close-fit index values indicated good model fit (SRMR = .050, RMSEA = .055), the incremental close-fit index values indicated that the model could not be accepted (TLI = .880, CFI = .906). Further, as can be seen in Table 3, the Optimism factor had a mix of positive and negative factor loadings. In particular, items 22 and 9 exhibited factor loadings of .32 and .33 on the Optimism factor. It will be noted that items 22 and 9 were the items that were used to form the Appraisal of Emotions in the Self-factor in the hypothesized factor modeling above. Thus, the Saklofske et al. (2003) four factor (28-item) model could not be accepted. It will also be noted that the modified hypothesized model tested above was practically significantly better fitting than the Saklofske et al. (2003) four-factor (28-item) model (DTLI = .026). 4. Discussion The results of the CFA analyses suggested that the hypothesized six-factor model of EI proposed by Salovey and Mayer, 1990 could not be completely recovered by the Schutte Self-Report Emotional Intelligence scale (Schutte et al., 1998). Specifically, a nested factors model with a firstorder general factor, and four nested factors corresponding to Appraisal of Emotions in the Self, Appraisal of Emotions in Others, Emotional Regulation of the Self, and Utilization of Emotions in Problem Solving were identified, in conjunction with a first-order acquiescence factor. There was no evidence to suggest the existence of an independent Emotional Expression or Emotional Regulation of Others factors.
G.E. Gignac et al. / Personality and Individual Differences 39 (2005) 1029–1042
1039
10 3 23
14 ERS
12 28 2 31 11 1 13 30
EI'g'
24 16 4
Acquiescence
15 32 5
AEO
29 25 18 33 9
AES
22 17 27
UEPS
7 20
Fig. 3. Final nested factor model with a first-order general EI factor, a first-order acquiescence factor, and first-order Emotional Regulation of Self (ERS), Appraisal of Emotions of Others (AEO), Appraisal of Emotions of the Self (AES), and Using Emotions for Problem Solving (UEPS) factors.
With only two items, the Ôappraisal of emotions in the selfÕ (AES) factor should be considered weak. At this stage, the two items from the AES factor should probably be considered a starting point for the potential development of a more complete subscale/factor. The addition of five or six internally consistent items would be considered a substantial step toward developing a stronger AES factor, and a more complete SSREI inventory. In contrast to the two items which formed the AES ‘‘factor’’, the two items hypothesized to form an Ôemotional expressionÕ (EE) factor did not demonstrate any unique covariance, independently of the general EI factor and the acquiescence factor. Within the TAS-20 (Bagby, Parker, & Taylor, 1994), a Ôdifficult describing feelingsÕ dimension has emerged in several factor analytic
1040
G.E. Gignac et al. / Personality and Individual Differences 39 (2005) 1029–1042
Table 3 Maximum Likelihood Estimation standardized factor loadings for the Saklofske et al. (2003) four-factor model Item
ÔgÕ
Optimism
Saklofske et al. (2003) four-factor model 22 .66 .32 9 .75 .33 12 .60 .11 23 .48 .28 10 .57 .39 28 .46 .32 14 .54 .11 2 .43 .10 3 .58 .27 1 .45 .04 29 .26 18 .44 25 .52 32 .36 5 .31 15 .41 27 .15 20 .48 7 .15 17 .38 33 .47 30 .50 11 .44 4 .52 24 .53 13 .48 16 .49 31 .29
Appraisal
Utilization
Social
Acquiesce .23 .15 .36 .49 .19 .20 .37 .17 .13 .18 .16 .08 .25
.57 .66 .59 .49 .48 .44
.16 .44 .25 .38 .37
.21 .84 .20 .36 .02 .46 .12 .28 .26 .20 .09 .26
.07 .01 .05 .18 .16 .24 .70
investigations (e.g., Haviland & Reise, 1996; Parker, Taylor, & Bagby, 2003), suggesting that an EE dimension deserves some serious consideration within a comprehension EI inventory. Despite the face validity of including an EE dimension within an EI framework, Mayer and SaloveyÕs, 1997 revised model of EI does not include a unique dimension specifically relevant to the expression of emotions (instead, it is amalgamated within the Ôappraisal of emotionsÕ dimension). The exclusion of a specific emotional expression dimension, which appears to have received greater relevance in the original model (Salovey & Mayer, 1990), may be due to the likely forbidding difficulties of measuring emotional expression from an ability model perspective. A mixed model perspective (i.e., self-report) is not plagued by the same problem, and, thus, future attempts to model a more comprehensive emotional expression factor within the SSREI is encouraged. When tested via CFA, the four-factor model reported by Saklofske et al. (2003) was found to be less well fitting than the modified hypothesized model based on Salovey and MayerÕs (1990) original model of EI. In particular, the two Ôappraisal of emotions in the selfÕ items loaded negatively on the ÔoptimismÕ factor, independently of the general factor and the acquiescence factor, support-
G.E. Gignac et al. / Personality and Individual Differences 39 (2005) 1029–1042
1041
ing the contention that these two items should be modeled as a separate factor. The observation of a negative association between items 22 and 9 and the ÔoptimismÕ factor also highlights one of the advantages of modeling a general factor as a first-order factor, within a nested factors modeling framework, because a comparable higher-order model and/or first-order oblique model failed to identify this effect (CFA results available upon request). That is, in the presence of a general factor, items 22 and 9 loaded positively (.68 and .75, respectively) on the ÔoptimismÕ factor, not because of any unique association between items 22 and 9 and the specified ÔoptimismÕ factor, but simply because of the variance shared by all of the items with a second-order general factor. Although Saklofske et al. (2003) reported CFA model fit to a comparable degree that was found in this study for their four-factor model (i.e., CFI = .90), it should be emphasized that Saklofske et al. (2003) modified their four-factor model, in a mechanical way, according to the modification indices (i.e., residual correlation matrix and Lagrange multiplier). The practice of following mechanically the information provided by modification indices has been sternly criticised by several SEM researchers (e.g., Bollen, 1990; MacCallum, Roznowski, & Necowitz, 1992). Unfortunately, Saklofske et al. (2003) did not specify how the model was modified, specifically. However, the degrees of freedom associated with their final model (455) suggests that a very large number of modifications (i.e., 34) were required to achieve a CFI criterion of .90. Although item 21, ÔI have control over my emotions,Õ was listed as ÔuncategorizedÕ within the qualitative analysis, it is believed that emotional control should be considered a possible valid dimension of self-report emotional intelligence. It is argued here that emotional control should be conceptualized as a more reactive, inhibitory process. For example, controlling ones temper, until a more appropriate time to express oneself presents itself. In contrast, ERS should be considered more proactive. For instance, conscientiously surrounding oneself with people who makes one feel good. Thus, although the emotional control item was not included in the SSREI CFA model recommended here, it is believed that emotional control should be part of a future SSREI inventory to potentially capture adequately a comprehensive model of EI. The potential development of a revised SSREI in accordance with the CFA findings in this investigation is considered a valuable objective, because it is one of the few EI inventories in the public domain. Further, in contrast to the most popular ability based model test (i.e., MSCEIT; Mayer, Salovey, & Caruso, 2000), which is plagued by the absence of a valid scoring protocol, low internal consistency, and emphasis on maximal performance rather than typical performance (see Matthews, Zeidner, & Roberts, 2002), self-report inventories such as the SSREI do appear to offer advantages with respect to scoring, reliability, emphasis on typical performance, as well as the opportunity to complement an EI assessment with 360° feedback. Although not without itÕs own problems (e.g., susceptibility to social desirable responding), the possibility of developing a self-report EI inventory with some validity should still be considered a potentially viable alternative to ability based model tests. References Bagby, R. M., Parker, J. D. A., & Taylor, G. J. (1994). The twenty-item Toronto Alexityhymia Scale. I. Item selection and cross-validation of the factor structure. Journal of Psychosomatic Research, 38(1), 23–32. Billiet, J. B., & McClendon, M. J. (2000). Modeling acquiescence in measurement models for two balanced set of items. Structural Equation Modeling, 7, 608–628.
1042
G.E. Gignac et al. / Personality and Individual Differences 39 (2005) 1029–1042
Bollen, K. A. (1990). A comment on model evaluation and modification. Structural Equation Modeling, 25(2), 181–185. Chen, F., Bollen, K. A., Paxton, P., Curran, P. J., & Kirby, J. B. (2001). Improper solutions in structural equation models: Causes, consequences, and strategies. Sociological Methods and Research, 29(4), 468–508. DeCarlo, L. T. (1997). On the meaning and use of kurtosis. Psychological Methods, 2(3), 292–307. Gorsuch, R. L. (1983). Factor analysis. Hillsdale, NJ: Lawrence Erlbaum. Gustafsson, B. (1993). General and specific abilities as predictors of school achievement. Multivariate Behavioral Research, 28(4), 407–434. Hakanen, E. A. (2004). Relation of emotional intelligence to emotional recognition and mood management. Psychological Reports, 94, 1097–1103. Haviland, M. G., & Reise, S. P. (1996). Structure of the twenty-item Toronto Alexithymia Scale. Journal of Personality Assessment, 66(1), 116–125. Hu, L., & Bentler, P. M. (1999). Cutoff criteria for fit indexes in covariance structure analysis: Conventional criteria versus new alternatives. Structural Equation Modeling, 6(1), 1–55. Jensen, A. R. (1998). The g factor. Westport, CT: Praeger. Kaiser, H. F. (1960). The application of electronic computers to factor analysis. Psychometrika, 20, 141–151. Kaiser, H. F. (1970). A second generation Little Jiffy. Psychomtrika, 35(4), 401–415. Little, T. D., Lindenberger, U., & Nesselroade, J. R. (1999). On selecting indicators for multivariate measurement and modeling with latent variables: When ‘‘good’’ indicators are bad and ‘‘bad’’ indicators are good. Psychological Methods, 4, 192–211. MacCallum, R. C., Roznowski, M., & Necowitz, L. B. (1992). Model modifications in covariance structure analysis: The problem of capitalization on chance. Psychological Bulletin, 111(3), 490–504. Matthews, G., Zeidner, M., & Roberts, R. (2002). Emotional intelligence: Science and myth. Cambridge: MIT Press. Mayer, J., & Salovey, P. (1997). What is emotional intelligence? In P. Salovey & D. Sluyter (Eds.), Emotional development and emotional intelligence: Implications for educators (pp. 3–31). New York: Basicbooks, Inc. Mayer, J. D., Salovey, P., & Caruso, D. R. (2000). The Mayer, Salovey, and Caruso emotional intelligence test: Technical manual. Toronto, ON: MHS. Muthen, B., & Kaplan, D. (1985). A comparison of some methodologies for the factor analysis of non-normal Likert variables. British Journal of Mathematic and Statistical Psychology, 38, 171–189. Parker, J. D. A., Taylor, G. J., & Bagby, R. M. (2003). The 20-item Toronto Alexithymia Scale––III. Reliability and factorial validity in a community population. Journal of Psychosomatic Research, 55, 269–275. Petrides, K. V., & Furnham, A. (2000). On the dimensional structure of emotional intelligence. Personality and Individual Differences, 29, 313–320. Saklofske, D. H., Austin, E. J., & Minski, P. S. (2003). Factor structure and validity of a trait emotional intelligence measure. Personality and Individual Differences, 34, 707–721. Salovey, P., & Mayer, J. D. (1990). Emotional intelligence. Imagination, Cognition, and Personality, 9, 185–211. Schutte, N. S., Malouff, J. M., Hall, L. E., Haggerty, D. J., Cooper, J. T., Golden, C. J., & Dornheim, L. (1998). Development and validation of a measure of emotional intelligence. Personality and Individual Differences, 25, 167–177. Vandenberg, R. J., & Lance, C. E. (2000). A review and synthesis of the measurement invariance literature: Suggestions, practices, and recommendations, for organizational research. Organizational Research Methods, 3(1), 4–70. Widaman, K. F. (1993). Common factor analysis versus principal component analysis: Differential bias in representing model parameters? Multivariate Behavioral Research, 28(3), 263–311.
An examination of the factor structure of the schutte self-report emotional intelligence (SSREI) scale via confirmatory factor analysis Gilles E. Gignac a
a,*
, Benjamin R. Palmer b, Ramesh Manocha c, Con Stough
b
School of Psychology, Deakin University, 221 Burwood Highway, Burwood, Victoria 3125, Australia b Centre for Neuropsychology, Swinburne University, Australia c Barbara Gross Research Unit, Faculty of Medicine, University of New South Wales, Australia Received 15 July 2004; accepted 14 March 2005 Available online 16 June 2005
Abstract Research to-date on the dimensionality of the Schutte Self-Report Emotional Intelligence (SSREI; Schutte et al., 1998) scale appears to support a four-factor interpretation, corresponding to Optimism, Social Skills, Emotional Regulation and Utilization of Emotions. However, the model of EI upon which the SSREI is based (Salovey & Mayer, 1990) has never been considered when determining the number of factors to extract/model in the factor analyses. Thus, in this investigation, we examined the CFA fit of several models, comparing the four-factor model reported by Saklofske, Austin, and Minski (2003), and the sixfactor model of EI described by Salovey and Mayer (1990). The CFA results indicated that two of the six dimensions of the Salovey and Mayer (1990) model of EI could not be identified, independently of first-order general and acquiescent factors. Specifically, while Ôappraisal of emotions in the selfÕ, Ôappraisal of emotions in othersÕ, Ôemotional regulation of the selfÕ, and Ôutilizing emotions in problem solvingÕ were identified, Ôemotional regulation of othersÕ and Ôemotional expressionÕ were not. The results are discussed in
*
Corresponding author. Fax: +61 3 9214 5230. E-mail address: [email protected] (G.E. Gignac).
0191-8869/$ - see front matter Ó 2005 Elsevier Ltd. All rights reserved. doi:10.1016/j.paid.2005.03.014
1030
G.E. Gignac et al. / Personality and Individual Differences 39 (2005) 1029–1042
light of how the SSREI could be potentially improved for the purposes of measuring the dimensions within the Salovey and Mayer model (1990). Ó 2005 Elsevier Ltd. All rights reserved. Keywords: Emotional intelligence; Self-report; CFA
1. Introduction Although a few exploratory/confirmatory factor analytic investigations exist in the literature that have examined the factor structure of the Schutte Self-Report Emotional Intelligence (SSREI; Schutte et al., 1998) scale, none of the investigations have considered the EI model upon which the sale is based, when deciding the number of factors to extract/model. Schutte et al. (1998) based the SSREI on the original Salovey and MayerÕs (1990) theory of emotional intelligence, because they believed that the original model better characterized an individualÕs present level of emotional development. As reported by Schutte et al. (1998), Salovey and MayerÕs (1990) original model included three categories: (1) Ôappraisal and expression of emotionsÕ; (2) Ôregulation of emotionsÕ; and (3) Ôutilization of emotions in solving problemsÕ. However, within these three categories are sub-categories. Specifically, the appraisal aspect of the Ôappraisal and expression of emotionsÕ category can be divided into Ôappraisal of emotions in the selfÕ and Ôappraisal of emotions in othersÕ. Similarly, the Ôregulation of emotionsÕ category can be subdivided into Ôregulation of emotions in the selfÕ and Ôregulation of emotions in othersÕ. Finally, and perhaps less clearly, Ôutilization of emotionsÕ, can be subdivided into four components: (1) Ôflexible planningÕ, Ôcreative thinkingÕ, Ôredirected attentionÕ and ÔmotivationÕ. Thus, in total, there are 10 first-order categories of EI within the Salovey and Mayer (1990) model, as described by Schutte et al. (1998). However, four of these categories are subsumed by the more nebulous Ôutilisation of emotionsÕ dimension. Thus, it could be argued that there are six primary dimensions within Salovey and MayerÕs (1990) model of EI. Schutte et al. (1998) generated an initial pool of 62 items to reflect all of the categories of Salovey and MayerÕs (1990) model of EI, and then performed a principal components analysis on the 62 item questionnaire, which was administered to 346 individuals. Based on their interpretation of a scree plot, Schutte et al. (1998) extracted four components and rotated the components, orthogonally. However, Schutte et al. (1998) reported that only the first component had an appreciable number of component loadings greater than .40, which was the criterion they used to demarcate a significant loading from a non-significant loading. Perplexingly, however, Schutte et al. (1998) reported that all of the categories of the Salovey and Mayer (1990) model were represented within the first component. Consequently, only the 33 items that exhibited component loadings greater than .40 on the first component were retained for the purpose of forming the published version of the questionnaire. Schutte et al.Õs (1998) component analysis can be criticized on a number of levels. First, it would have been more appropriate to perform a factor analysis on the 62 items, rather than a component analysis. Based on a comprehensive simulation study, Widaman (1993) demonstrated that greater factor structure accuracy (i.e., loadings and factor inter-correlations) can be achieved with factor analysis, in contrast to principal component analysis, to the extent that the factor loadings are
G.E. Gignac et al. / Personality and Individual Differences 39 (2005) 1029–1042
1031
expected to be smaller, rather than larger. Thus, in the case of analysing items, which tend to be relatively unreliable, it would have been especially advisable to perform factor analysis, rather than components analysis. Further, because items tend to be relatively unreliable, the demarcation criterion of .40 used by Schutte et al. (1998) for the purpose of identifying significant factor loadings may have been too strict, even for principal components analysis, which tends to overestimate the factor reliability of the items in the analysis (Widaman, 1993). Moreover, as argued by Petrides and Furnham (2000),1 the components/factors should have been rotated obliquely, not orthogonally. Theoretically, the categories of Salovey and MayerÕs (1990) model of EI would be expected to exhibit positive intercorrelations. Orthogonal rotations preclude any correlations between components/factors (Gorsuch, 1983). Further, and perhaps most surprisingly, given the results reported in Schutte et al. (1998), an orthogonal rotation will preclude the existence of a general factor (Jensen, 1998, p. 66). Schutte et al. (1998) reported (p. 171) the eigenvalues for the rank ordered components as 10.79, 3.58, 2.90, and 2.53. These values appear to be derived from an unrotated solution. In all likelihood, an orthogonal rotation of the components would have spread the common variance relatively evenly from the first component, probably a general component, to the other components retained in the analysis. Thus, it is very possible that the component that Schutte et al. (1998) interpreted, and ultimately used as the basis for creating the current SSREI inventory, was a general component, derived from an unrotated factor solution. Another argument in favour of the general component interpretation resides in the fact that the published version of the SSREI inventory includes items from all of the broad dimensions of Salovey and MayerÕs (1990) model of EI. Had Schutte et al. (1998) interpreted the rotated component solution, it is very doubtful that any of the components would have contained component loadings of .40 from such a multifarious collection of items, which just happen to represent all of the categories from the Salovey and MayerÕs (1990) model of EI. Since the publication of Schutte et al. (1998), a few published papers have investigated the itemlevel factor structure of the SSREI. For instance, Petrides and Furnham (2000) sought to test the hypothesis, via CFA, that the SSREI inventory measured a single, general factor of EI, based on a sample of 260 university students. The general factor model was associated with close-fit index values that indicated that the general factor model was unacceptable (e.g., CFI = .51, RMSEA = .105). Consequently, Petrides and Furnham (2000) followed their CFA analysis with an unrestricted (exploratory) principal components analysis. Based on Petrides and FurnhamÕs judgement, four components were extracted and rotated both orthogonally and obliquely. The components were found to correlate below j.30j. It is unclear whether Petrides and Furnham (2000) present the factor solution for the oblique or orthogonal rotations. The four components
1
Petrides and Furnham (2000) argued that Schutte et al. (1998) performed ‘‘a ÔLittle JiffyÕ procedure’’ and that this was the most significant psychometric problem with the Schutte et al. (1998) component analysis (p. 317). However, it appears that Petrides and Furnham (2000) have confused the expression ÔLittle JiffyÕ for varimax rotation. ÔLittle JiffyÕ refers to several principles related to performing a component/factor analysis. Specifically, the method of communality estimation, the number of components/factors to extract, and the rotation transformation (among other things). Originally, part of the Little Jiffy included an orthogonal rotation procedure (Kaiser, 1960); however, KaiserÕs Ôsecond generation Little JiffyÕ (Kaiser, 1970) included an oblique rotation procedure, because Kaiser (1970) preferred oblique rotation over orthogonal rotation.
1032
G.E. Gignac et al. / Personality and Individual Differences 39 (2005) 1029–1042
extracted by Petrides and Furnham (2000) corresponded to Optimism, Appraisal of Emotions, Social Skills, and Utilisation of Emotions. Saklofske et al. (2003) performed an unrestricted, exploratory principal components analysis on the extraction of four components (oblique rotation). The factor solution obtained by Saklofske et al. (2003) corroborated closely the factor solution obtained by Petrides and Furnham (2000), with the four components corresponding to Optimism, Appraisal of Emotions, Utilisation of Emotions, and Social Skills. However, when Saklofske et al. (2003) attempted to test the four-factor model via CFA, the model was found to be poor fitting. Unfortunately, Saklofske et al. (2003) do not report the specific fit CFA close-fit values for the four-factor model. HakanenÕs (2004) EFA investigation into the dimensionality of the SSREI suffers from the same limitations reported above: component analysis and orthogonal rotation. Given that two previous CFA investigations failed to support the four-factor model within the SSREI, it was decided that a further investigation into the factor structure of the SSREI was merited. However, in contrast to Petrides and Furnham (2000) and Saklofske et al. (2003), it was hypothesized that the SSREI would conform more closely to the theoretical six-factor model, rather than the four-factor model. Further, because the SSREI inventory is extremely unbalanced (30 positively keyed items and only three negatively keyed items), it was hypothesized that it would be necessary to model a first-order acquiescence factor, in conjunction with the substantive factors, for the purpose of achieving satisfactory model fit. Several investigations have demonstrated the existence of an acquiescence factor, when analysing self-report questionnaires via CFA at the item level (see Billiet & McClendon, 2000). Prior to statistical testing, we analyzed and categorized, qualitatively, the 33 items of the SSREI inventory for the purpose of classification into the conceptual categories of Salovey and MayerÕs (1990) model of EI. Based on our qualitative analysis, the six substantive categories were identified within the SSREI inventory. The corresponding items for each category are listed in Table 1. It can be seen that two of the dimensions only have two items each: Ôappraisal of emotions in the self Õ (AES) and Ôemotional expressionÕ (EE). Further, five items could not be readily classified into any of the six theoretical dimensions. Thus, we sought to test the hypothesis, via CFA, that the six-factor model was a more plausible model of the dimensions within the SSREI inventory, in comparison to the four-factor model, currently advocated by several investigators (e.g., Petrides & Furnham, 2000; Saklofske et al., 2003). Further, we hypothesized the existence of a first-order acquiescence factor, in conjunction to the six first-order substantive factors. Table 1 Items corresponding to the theorized dimensions of the SSREI, based on a qualitative analysis of the 33 items (see Schutte et al., 1998 for item descriptions) Dimensions
Items
Appraisal of Emotions in the Self (AES) Appraisal of Emotions in Others (AEO) Emotional Expression (EE) Emotional Regulation of the Self (ERS) Emotional Regulation of Others (ERO) Utilization of Emotions in Problem Solving (UEPS) Uncategorized
9, 5, 1, 2, 4, 7, 6,
22 15, 18, 25, 29, 32, 33 11 3, 12, 14, 23, 28, 31, 10 13, 16, 24, 30 17, 20, 27 8, 19, 21, 26
G.E. Gignac et al. / Personality and Individual Differences 39 (2005) 1029–1042
1033
2. Method 2.1. Participants The sample consisted of 367 participants (107 males, 257 females, 3 unreported), ranging in age from 15 to 78 (Mean = 38.3 years, SD = 13.7). The participants were obtained from the general population across the states of Victoria and New South Wales (Australia) via advertisements. 2.2. Materials The Schutte Self-Report Emotional Intelligence (SSREI) scale by Schutte et al. (1998) is comprised of 33, 5-point Likert scale items, three of which are negatively keyed. Previous investigations have found the total scores on the SSREI to be acceptably internally consistent (e.g., .90; Schutte et al., 1998). 2.3. Procedure Once informed consent was obtained from the participants, test booklets were provided, which included the SSREI inventory and a scannable answer sheet. The participants were given an unlimited amount of time to complete the paper based inventory, and they received a small monetary reward for participating. 2.4. Data analytic strategy The CFA analyses (AMOS 5.0) will be based on the 28 items selected in the introduction above. The first model that will be tested (model 1) will be the hypothesized six-factor model based on Salovey and MayerÕs (1990) framework of EI. The approach to modeling the six-factors will be based on a nested CFA modeling strategy. Specifically, rather than model the covariance between the six factors as an oblique factor model or a higher-order general factor model, a nested factors modeling approach will be used (Gustafsson & Balke, 1993). Thus, all 28 items will be specified to load onto a first-order general factor, directly. Further, all 25 of the positively keyed items will be specified to load onto a first-order acquiescence factor. A graphical depiction of the first model that will be tested can be seen in Fig. 1. For the purposes of model comparison, a single general factor model, and a general factor model with an acquiescence factor will also be tested via CFA. It is hypothesized that the hypothesized six-factor model will be associated with practically better fit, in comparison to the general factor model, and the general factor model with an acquiescence factor. Finally, a nested factors model consistent with the EFA findings of Saklofske et al. (2003) will be tested. Thus, in comparison to the hypothesized six-factor model reported above, the Saklofske et al. (2003) model will consist of four nested factors, corresponding to Optimism, Appraisal, Utilization, and Social. For the purposes of fair comparison, the Saklofske et al. (2003) four-factor model will also be based on only 28 items, as described above for the hypothesized six-factor model. It is hypothesized that the six-factor model will be associated with practically better model fit than the Saklofske et al. (2003) four-factor model. A graphical depiction of the Saklofske et al. (2003) model can be seen in Fig. 2.
1034
G.E. Gignac et al. / Personality and Individual Differences 39 (2005) 1029–1042 10 3 23
14 ERS
12 28 2 31 11
EE
1 13 30
ERO
24 16 4
EI'g'
Acquiescence
15 32 5
AEO
29 25 18 33 9
AES
22 17 27
UEPS
7 20
Fig. 1. Hypothesized nested factors model based on Salovey and MayerÕs (1990) six-factor model of EI (ERS = Emotional Regulation of the Self; EE = Emotional Expression; ERO = Emotional Regulation of Others; AEO = Appraisal of Emotions in Others; AES = Appraisal of Emotions in the Self; UEPS = Utilization of Emotions for Problem Solving).
All CFA analyses will be based on a Pearson covariance matrix and Maximum Likelihood Estimation (MLE). Models will identified by constraining the latent variables to 1.0. In the case of a latent variable defined by only two observed variables, the factor loadings will be constrained to equality, in accordance with Little, Lindenberger, and Nesselroade (1999, pp. 204–205). In accordance with Hu and Bentler (1999), a combination approach will be used to evaluate model fit. Specifically, two absolute close-fit indices (SRMR and RMSEA) and two incremental close-fit indices were chosen (TLI and CFI). Also in accordance with Hu and Bentler (1999), models will be evaluated as good fitting, when the absolute fit indices (SRMR and RMSEA) are <.06 and the
G.E. Gignac et al. / Personality and Individual Differences 39 (2005) 1029–1042
1035
22 9 12 23 10 optimism
28 14 2 3
EI'g'
1 29 18
appraisal
25 32 5
Acquiescence
15 27 20 utilization
7 17 33 30 11 4
social
24 13 16 31
Fig. 2. A nested version of the Saklofske et al. (2003) four-factor model of the SSREI with a first-order general factor and an orthogonal acquiescence factor.
incremental fit indices (TLI and CFI) are approximately .95 or larger. Rather than compare the fit of models statistically, a practical significance difference test will be used based on TLI values. In accordance with Vandenberg and Lance (2000), a model with a TLI value .01 larger than another model will be considered practically better fitting.
3. Results Average absolute levels of item skew and kurtosis were .91 and .93, respectively, which is within the acceptable range for CFA analysis using maximum likelihood estimation (Muthen & Kaplan, 1985). A variant (see DeCarlo, 1997) of SmallÕs omnibus test of multivariate normality was rejected (VQ3(66) = 1139.13, p < .001); however, the majority (72%) of the multivariate
1036
G.E. Gignac et al. / Personality and Individual Differences 39 (2005) 1029–1042
non-normality was due to skewness (Q1(33) = 816.85, p < .001), rather than kurtosis (VQ2(33) = 322.28, p < .001), indicating that the MLE CFA results would likely be minimally affected by the multivariate non-normality. A null CFA model hypothesizing no covariation between the 28-items was rejected ðv2ð378Þ ¼ 3734.85, p < .001, RMSEA = .159, SRMR = .278, TLI = .000, CFI = .000), indicating that the inter-item covariation was appropriate for factor analysis. The hypothesized nested factors model with a first-order general factor, a first-order acquiescence factor, and nested first-order ERS, ERO, EE, AEO, AES, and UEPS factors produced a v2ð299Þ ¼ 561.16, p < .001. However, as can be seen in Table 2 (model 1), the factor loadings from the ERO and EE factors were not significant statistically. Further, an excessively large standard error (i.e., 8.69) was associated with item 30Õs unstandardized factor loading of 1.33. Finally, item 30 and 20Õs residual variances were associated with negative values of 1.33 and .14, which is unacceptable in an adequate CFA model. Thus, despite the decent close-fit index values (e.g., RMSEA = .050 and CFI = .922), the hypothesized model was not considered acceptable. A modified hypothesized model was tested which was identical to the hypothesized model, with the exception that the nested ERO and EE factors were removed from the model. The modified hypothesized model produced a v2ð305Þ ¼ 570.12, p < .001, and all of the standard errors were within an acceptable region, indicating that the model was potentially acceptable. In contrast to the original model, only one itemÕs residual variance was associated with a negative value. To test the hypothesis that the negative error variance may be due to sampling fluctuations, item 20Õs residual error variance was constrained to .0001, in accordance with Chen, Bollen, Paxton, Curran, and Kirby (2001). The modified model with the constrained error variance yielded a v2ð306Þ ¼ 570.72, p < .001, which was not statistically significantly worse fitting than the previous non-constrained model, indicating that the negative error variance was likely due to sampling fluctuations, rather than a fundamentally inappropriate model specification. An examination of the modification indices indicated that there was some unmodeled covariance between the AES and AEO factors. Because a positive association between these two factors was reasonable theoretically (i.e., were from the same domain of Ôappraisal of emotionsÕ), the modified model was retested with the addition of a covariance link between the AES and AEO factors, which produced a v2ð305Þ ¼ 559.40, p < .001. The correlation between the AES and AEO factors was estimated at .28 (CR = 3.65, p < .001). This final model was associated with absolute close-fit index values indicating good model fit (SRMR = .043 and RMSEA = .049), while the incremental close-fit index values indicated marginally good model fit (CFI = .924 and TLI = .906). An examination of the standardized residual covariances and modification index values (i.e., Lagrange multiplier) did not suggest any conspicuously large and/or meaningful changes to the model. As can be seen in Table 2 (model 2), all of the factor loadings were positive and statistically significant, with the exception of only three items on the acquiescence factor. A graphical depiction of the modified hypothesized model is presented in Fig. 3. For the purposes of model comparison, a general factor model was tested, which produced a v2ð35Þ ¼ 1358.83, p < .001, RMSEA = .091, SRMR = .079, CFI = .699, TLI = .675, indicating very poor model fit. A TLI difference test indicated substantial practical improvement in fit, favoring the modified hypothesized model, in comparison to the general factor model (DTLI = .231). Similarly, a general factor model with an acquiescence factor produced a v2ð325Þ ¼ 949.63, p < .001, RMSEA = .074, SRMR = .064, CFI = .814, TLI = .784, indicating poor fit, based on the
G.E. Gignac et al. / Personality and Individual Differences 39 (2005) 1029–1042
1037
Table 2 Maximum Likelihood Estimation standardized factor loadings for the hypothesized six-factor model Item
ÔgÕ
ERS
EE
ERO
Model 1: Hypothesized six-factor model 10 .53 .53 3 .55 .32 23 .42 .48 14 .55 .16 12 .54 .32 28 .44 .34 2 .42 .18 31 .31 .23 11 .46 .16 1 .46 .17 13 .51 30 .52 24 .58 16 .52 4 .57 15 .37 32 .34 5 .30 29 .23 25 .49 18 .39 33 .46 9 .67 22 .53 17 .39 27 .13 7 .15 20 .50 Item
ÔgÕ
ERS
Model 2: Modified hypothesized model 10 .52 .54 3 .55 .32 23 .42 .46 14 .55 .15 12 .53 .32 28 .44 .34 2 .42 .16 31 .33 .19 11 .46 1 .46 13 .52 30 .56 24 .60 16 .52
AEO
AES
UEPS
Acquiesce .08 .13 .41 .17 .33 .37 .60 .02 .16 .20 .17 .21 .28 .03 .25 .34
.04 1.67 .03 .02 .06 .44 .44 .52 .54 .60 .64 .26
.27 .18 .28 .52 .49
.19 .27 .39 .51 .40 .25
.29 .16 .15 .97 AEO
AES
UEPS
Acquiesce .09 .14 .43 .18 .34 .37 .60 .02 .16 .19 .15 .20 .28 (continued on next page)
1038
G.E. Gignac et al. / Personality and Individual Differences 39 (2005) 1029–1042
Table 2 (continued) Item
ÔgÕ
ERS
Model 2: Modified hypothesized model 4 .59 15 .35 32 .32 5 .29 29 .22 25 .47 18 .37 33 .45 9 .65 22 .51 17 .38 27 .13 7 .15 20 .50
AEO
AES
UEPS
Acquiesce .04 .24 .34
.46 .45 .52 .54 .62 .66 .27
.26 .18 .28 .55 .52 .34 .20 .19 .83
.18 .27 .39 .49 .39 .26
Note: Factor loadings in bold were not significant statistically (p > .05). The unstandardized factor loading standard errors (available upon request) for model 2 were associated with a mean of .053 and a range of .024–.076, which was considered acceptable, based on a sample size of 352 (i.e., 1/(N1/2) = 1/18.76 = .05).
incremental close-fit indexes. A TLI difference test indicated substantial practical improvement in fit in favor of the modified hypothesized model tested above, in comparison to the general + acquiescence factor model (DTLI = .122). The Saklofske et al. (2003) four-factor (28-item) model produced a v2ð298Þ ¼ 614.84, p < .001. Although the absolute close-fit index values indicated good model fit (SRMR = .050, RMSEA = .055), the incremental close-fit index values indicated that the model could not be accepted (TLI = .880, CFI = .906). Further, as can be seen in Table 3, the Optimism factor had a mix of positive and negative factor loadings. In particular, items 22 and 9 exhibited factor loadings of .32 and .33 on the Optimism factor. It will be noted that items 22 and 9 were the items that were used to form the Appraisal of Emotions in the Self-factor in the hypothesized factor modeling above. Thus, the Saklofske et al. (2003) four factor (28-item) model could not be accepted. It will also be noted that the modified hypothesized model tested above was practically significantly better fitting than the Saklofske et al. (2003) four-factor (28-item) model (DTLI = .026). 4. Discussion The results of the CFA analyses suggested that the hypothesized six-factor model of EI proposed by Salovey and Mayer, 1990 could not be completely recovered by the Schutte Self-Report Emotional Intelligence scale (Schutte et al., 1998). Specifically, a nested factors model with a firstorder general factor, and four nested factors corresponding to Appraisal of Emotions in the Self, Appraisal of Emotions in Others, Emotional Regulation of the Self, and Utilization of Emotions in Problem Solving were identified, in conjunction with a first-order acquiescence factor. There was no evidence to suggest the existence of an independent Emotional Expression or Emotional Regulation of Others factors.
G.E. Gignac et al. / Personality and Individual Differences 39 (2005) 1029–1042
1039
10 3 23
14 ERS
12 28 2 31 11 1 13 30
EI'g'
24 16 4
Acquiescence
15 32 5
AEO
29 25 18 33 9
AES
22 17 27
UEPS
7 20
Fig. 3. Final nested factor model with a first-order general EI factor, a first-order acquiescence factor, and first-order Emotional Regulation of Self (ERS), Appraisal of Emotions of Others (AEO), Appraisal of Emotions of the Self (AES), and Using Emotions for Problem Solving (UEPS) factors.
With only two items, the Ôappraisal of emotions in the selfÕ (AES) factor should be considered weak. At this stage, the two items from the AES factor should probably be considered a starting point for the potential development of a more complete subscale/factor. The addition of five or six internally consistent items would be considered a substantial step toward developing a stronger AES factor, and a more complete SSREI inventory. In contrast to the two items which formed the AES ‘‘factor’’, the two items hypothesized to form an Ôemotional expressionÕ (EE) factor did not demonstrate any unique covariance, independently of the general EI factor and the acquiescence factor. Within the TAS-20 (Bagby, Parker, & Taylor, 1994), a Ôdifficult describing feelingsÕ dimension has emerged in several factor analytic
1040
G.E. Gignac et al. / Personality and Individual Differences 39 (2005) 1029–1042
Table 3 Maximum Likelihood Estimation standardized factor loadings for the Saklofske et al. (2003) four-factor model Item
ÔgÕ
Optimism
Saklofske et al. (2003) four-factor model 22 .66 .32 9 .75 .33 12 .60 .11 23 .48 .28 10 .57 .39 28 .46 .32 14 .54 .11 2 .43 .10 3 .58 .27 1 .45 .04 29 .26 18 .44 25 .52 32 .36 5 .31 15 .41 27 .15 20 .48 7 .15 17 .38 33 .47 30 .50 11 .44 4 .52 24 .53 13 .48 16 .49 31 .29
Appraisal
Utilization
Social
Acquiesce .23 .15 .36 .49 .19 .20 .37 .17 .13 .18 .16 .08 .25
.57 .66 .59 .49 .48 .44
.16 .44 .25 .38 .37
.21 .84 .20 .36 .02 .46 .12 .28 .26 .20 .09 .26
.07 .01 .05 .18 .16 .24 .70
investigations (e.g., Haviland & Reise, 1996; Parker, Taylor, & Bagby, 2003), suggesting that an EE dimension deserves some serious consideration within a comprehension EI inventory. Despite the face validity of including an EE dimension within an EI framework, Mayer and SaloveyÕs, 1997 revised model of EI does not include a unique dimension specifically relevant to the expression of emotions (instead, it is amalgamated within the Ôappraisal of emotionsÕ dimension). The exclusion of a specific emotional expression dimension, which appears to have received greater relevance in the original model (Salovey & Mayer, 1990), may be due to the likely forbidding difficulties of measuring emotional expression from an ability model perspective. A mixed model perspective (i.e., self-report) is not plagued by the same problem, and, thus, future attempts to model a more comprehensive emotional expression factor within the SSREI is encouraged. When tested via CFA, the four-factor model reported by Saklofske et al. (2003) was found to be less well fitting than the modified hypothesized model based on Salovey and MayerÕs (1990) original model of EI. In particular, the two Ôappraisal of emotions in the selfÕ items loaded negatively on the ÔoptimismÕ factor, independently of the general factor and the acquiescence factor, support-
G.E. Gignac et al. / Personality and Individual Differences 39 (2005) 1029–1042
1041
ing the contention that these two items should be modeled as a separate factor. The observation of a negative association between items 22 and 9 and the ÔoptimismÕ factor also highlights one of the advantages of modeling a general factor as a first-order factor, within a nested factors modeling framework, because a comparable higher-order model and/or first-order oblique model failed to identify this effect (CFA results available upon request). That is, in the presence of a general factor, items 22 and 9 loaded positively (.68 and .75, respectively) on the ÔoptimismÕ factor, not because of any unique association between items 22 and 9 and the specified ÔoptimismÕ factor, but simply because of the variance shared by all of the items with a second-order general factor. Although Saklofske et al. (2003) reported CFA model fit to a comparable degree that was found in this study for their four-factor model (i.e., CFI = .90), it should be emphasized that Saklofske et al. (2003) modified their four-factor model, in a mechanical way, according to the modification indices (i.e., residual correlation matrix and Lagrange multiplier). The practice of following mechanically the information provided by modification indices has been sternly criticised by several SEM researchers (e.g., Bollen, 1990; MacCallum, Roznowski, & Necowitz, 1992). Unfortunately, Saklofske et al. (2003) did not specify how the model was modified, specifically. However, the degrees of freedom associated with their final model (455) suggests that a very large number of modifications (i.e., 34) were required to achieve a CFI criterion of .90. Although item 21, ÔI have control over my emotions,Õ was listed as ÔuncategorizedÕ within the qualitative analysis, it is believed that emotional control should be considered a possible valid dimension of self-report emotional intelligence. It is argued here that emotional control should be conceptualized as a more reactive, inhibitory process. For example, controlling ones temper, until a more appropriate time to express oneself presents itself. In contrast, ERS should be considered more proactive. For instance, conscientiously surrounding oneself with people who makes one feel good. Thus, although the emotional control item was not included in the SSREI CFA model recommended here, it is believed that emotional control should be part of a future SSREI inventory to potentially capture adequately a comprehensive model of EI. The potential development of a revised SSREI in accordance with the CFA findings in this investigation is considered a valuable objective, because it is one of the few EI inventories in the public domain. Further, in contrast to the most popular ability based model test (i.e., MSCEIT; Mayer, Salovey, & Caruso, 2000), which is plagued by the absence of a valid scoring protocol, low internal consistency, and emphasis on maximal performance rather than typical performance (see Matthews, Zeidner, & Roberts, 2002), self-report inventories such as the SSREI do appear to offer advantages with respect to scoring, reliability, emphasis on typical performance, as well as the opportunity to complement an EI assessment with 360° feedback. Although not without itÕs own problems (e.g., susceptibility to social desirable responding), the possibility of developing a self-report EI inventory with some validity should still be considered a potentially viable alternative to ability based model tests. References Bagby, R. M., Parker, J. D. A., & Taylor, G. J. (1994). The twenty-item Toronto Alexityhymia Scale. I. Item selection and cross-validation of the factor structure. Journal of Psychosomatic Research, 38(1), 23–32. Billiet, J. B., & McClendon, M. J. (2000). Modeling acquiescence in measurement models for two balanced set of items. Structural Equation Modeling, 7, 608–628.
1042
G.E. Gignac et al. / Personality and Individual Differences 39 (2005) 1029–1042
Bollen, K. A. (1990). A comment on model evaluation and modification. Structural Equation Modeling, 25(2), 181–185. Chen, F., Bollen, K. A., Paxton, P., Curran, P. J., & Kirby, J. B. (2001). Improper solutions in structural equation models: Causes, consequences, and strategies. Sociological Methods and Research, 29(4), 468–508. DeCarlo, L. T. (1997). On the meaning and use of kurtosis. Psychological Methods, 2(3), 292–307. Gorsuch, R. L. (1983). Factor analysis. Hillsdale, NJ: Lawrence Erlbaum. Gustafsson, B. (1993). General and specific abilities as predictors of school achievement. Multivariate Behavioral Research, 28(4), 407–434. Hakanen, E. A. (2004). Relation of emotional intelligence to emotional recognition and mood management. Psychological Reports, 94, 1097–1103. Haviland, M. G., & Reise, S. P. (1996). Structure of the twenty-item Toronto Alexithymia Scale. Journal of Personality Assessment, 66(1), 116–125. Hu, L., & Bentler, P. M. (1999). Cutoff criteria for fit indexes in covariance structure analysis: Conventional criteria versus new alternatives. Structural Equation Modeling, 6(1), 1–55. Jensen, A. R. (1998). The g factor. Westport, CT: Praeger. Kaiser, H. F. (1960). The application of electronic computers to factor analysis. Psychometrika, 20, 141–151. Kaiser, H. F. (1970). A second generation Little Jiffy. Psychomtrika, 35(4), 401–415. Little, T. D., Lindenberger, U., & Nesselroade, J. R. (1999). On selecting indicators for multivariate measurement and modeling with latent variables: When ‘‘good’’ indicators are bad and ‘‘bad’’ indicators are good. Psychological Methods, 4, 192–211. MacCallum, R. C., Roznowski, M., & Necowitz, L. B. (1992). Model modifications in covariance structure analysis: The problem of capitalization on chance. Psychological Bulletin, 111(3), 490–504. Matthews, G., Zeidner, M., & Roberts, R. (2002). Emotional intelligence: Science and myth. Cambridge: MIT Press. Mayer, J., & Salovey, P. (1997). What is emotional intelligence? In P. Salovey & D. Sluyter (Eds.), Emotional development and emotional intelligence: Implications for educators (pp. 3–31). New York: Basicbooks, Inc. Mayer, J. D., Salovey, P., & Caruso, D. R. (2000). The Mayer, Salovey, and Caruso emotional intelligence test: Technical manual. Toronto, ON: MHS. Muthen, B., & Kaplan, D. (1985). A comparison of some methodologies for the factor analysis of non-normal Likert variables. British Journal of Mathematic and Statistical Psychology, 38, 171–189. Parker, J. D. A., Taylor, G. J., & Bagby, R. M. (2003). The 20-item Toronto Alexithymia Scale––III. Reliability and factorial validity in a community population. Journal of Psychosomatic Research, 55, 269–275. Petrides, K. V., & Furnham, A. (2000). On the dimensional structure of emotional intelligence. Personality and Individual Differences, 29, 313–320. Saklofske, D. H., Austin, E. J., & Minski, P. S. (2003). Factor structure and validity of a trait emotional intelligence measure. Personality and Individual Differences, 34, 707–721. Salovey, P., & Mayer, J. D. (1990). Emotional intelligence. Imagination, Cognition, and Personality, 9, 185–211. Schutte, N. S., Malouff, J. M., Hall, L. E., Haggerty, D. J., Cooper, J. T., Golden, C. J., & Dornheim, L. (1998). Development and validation of a measure of emotional intelligence. Personality and Individual Differences, 25, 167–177. Vandenberg, R. J., & Lance, C. E. (2000). A review and synthesis of the measurement invariance literature: Suggestions, practices, and recommendations, for organizational research. Organizational Research Methods, 3(1), 4–70. Widaman, K. F. (1993). Common factor analysis versus principal component analysis: Differential bias in representing model parameters? Multivariate Behavioral Research, 28(3), 263–311.