- Email: [email protected]
Typical intellectual engagement and achievement in math and the sciences in secondary education
Learning and Individual Differences 43 (2015) 31–38
Contents lists available at ScienceDirect
Learning and Individual Differences journal homepage: www.elsevier.com/locate/lindif
Typical intellectual engagement and achievement in math and the sciences in secondary education☆ Ulrich Schroeders a,⁎, Stefan Schipolowski b, Katrin Böhme c a b c
Department of Educational Science, University of Bamberg, Feldkirchenstraße 21, 96045 Bamberg, Germany Department of Psychology, Humboldt-Universität zu Berlin Inclusive Education, University of Potsdam
a r t i c l e
i n f o
Article history: Received 13 August 2014 Received in revised form 14 August 2015 Accepted 18 August 2015 Available online xxxx Keywords: Typical intellectual engagement Math achievement Science achievement Fluid intelligence Subject-specific interest
a b s t r a c t Typical Intellectual Engagement (TIE) is considered a key trait in explaining individual differences in educational achievement in advanced academic or professional settings. Research in secondary education, however, has focused on cognitive and conative factors rather than personality. In the present large-scale study, we investigated the relation between TIE and achievement tests in math and science in Grade 9. A three-dimensional model (reading, contemplation, intellectual curiosity) provided high theoretical plausibility and satisfactory model fit. We quantified the predictive power of TIE with hierarchical regression models. After controlling for gender, migration background, and socioeconomic status, TIE contributed substantially to the explanation of math and science achievement. However, this effect almost disappeared after fluid intelligence and interest were added into the model. Thus, we found only limited support for the significance of TIE on educational achievement, at least for subjects more strongly relying on fluid abilities such as math and science. © 2015 Elsevier Inc. All rights reserved.
Individual differences in academic performance have been linked to a plethora of cognitive, conative, and affective factors. The use of cognitive abilities to predict school performance has a long tradition, dating back to the first intelligence test (Binet & Simon, 1904). Since then, the power of cognitive factors such as fluid intelligence or domain-specific knowledge for the prediction of performance in secondary and tertiary education has been repeatedly demonstrated (e.g., Hambrick, 2004; Kuncel, Hezlett, & Ones, 2004). Conative factors such as academic self-concept or subject-specific interest also affect students' performance. For instance, math self-concept in Grade 7 explained math achievement in Grade 10 over and above math ability in Grade 7 (Marsh, Trautwein, Lüdtke, Köller, & Baumert, 2005). In contrast to cognitive and motivational constructs, personality and other affective factors still play a minor role in educational research. Several reasons can be given for this circumstance: First, as cognitive abilities are by far the most powerful predictors of academic achievement (e.g., Furnham, Monsen, & Ahmetoglu, 2009), the potential benefit of noncognitive variables such as personality constructs in terms of incremental validity appears marginal. Second, because nonability traits operationalized with measures of typical behavior (e.g., questionnaires; see Cronbach, 1949) are prone to faking good and social desirability effects, they have received less ☆ During the preparation of this manuscript, Stefan Schipolowski was a fellow of the International Max Planck Research School The Life Course: Evolutionary and Ontogenetic Dynamics (LIFE). ⁎ Corresponding author. E-mail address: [email protected] (U. Schroeders).
http://dx.doi.org/10.1016/j.lindif.2015.08.030 1041-6080/© 2015 Elsevier Inc. All rights reserved.
attention in the high-stakes contexts (e.g., college admission) that have steered a lot of research. Finally, compared to other noncognitive characteristics (e.g., interest or self-efficacy), personality is understood as a set of more stable behavioral dispositions which are only marginally sensitive to intervention and schooling. In the context of educational research with pupils, however, the focus lies explicitly on the more malleable aspects of human behavior. 1. Typical Intellectual Engagement: A Promising Candidate The term personality represents a broad set of diverse constructs. Among the constructs that are particularly promising for predicting individual differences in academic performance are the so-called intellectual investment traits such as need for cognition, the Big Five trait openness, and typical intellectual engagement (TIE). In this paper, we focus on typical intellectual engagement (Ackerman & Goff, 1994; Goff & Ackerman, 1992). TIE describes a person's engagement in intellectual activity and his or her interest in and need for a profound understanding of complex issues. Therefore, TIE characterizes the attraction/aversion that an intellectually demanding task exerts on an individual. Individuals with high engagement receive better grades, score significantly higher on standardized ability tests (Chamorro-Premuzic, Furnham, & Ackerman, 2006b; Wilhelm, Schulze, Schmiedek, & Süß, 2003), and possess better general knowledge (Chamorro-Premuzic, Furnham, & Ackerman, 2006a). The construct TIE seems predestined for the prediction of scholastic achievement. From a theoretical point of view, TIE should have a
32
U. Schroeders et al. / Learning and Individual Differences 43 (2015) 31–38
positive impact on the acquisition of skills and knowledge in adulthood. The willingness to solve complex issues, to contemplate and to deal with new information mirrors the conditions required for successful learning. For two persons with the same level of cognitive ability, TIE might be a key construct for explaining interindividual differences in learning and academic achievement. Furthermore, as a measure of typical behavior, TIE may “play an important cumulative role in knowledge acquisition and retention” (Rolfhus & Ackerman, 1999, p. 513) and may provide a link between cognitive and conative factors, building clusters or “trait complexes” (Ackerman & Heggestad, 1997). In PPIK theory (intelligence-as-process, personality, interests, and intelligence-asknowledge; Ackerman, 1996), so-called intellectual investment traits are conceived as crucial for the transition from process to knowledge (Ackerman & Rolfhus, 1999; Rolfhus & Ackerman, 1999; von Stumm & Ackerman, 2012). From an empirical perspective, TIE has been shown to be moderately correlated with scholastic and academic performance. For example, TIE was positively correlated with grade point average, college admission test scores (Wilhelm et al., 2003), and course measures of academic performance such as tutorial reports, essays, or project reports (Chamorro-Premuzic et al., 2006b). A recent meta-analysis reported substantial relations of TIE with crystallized intelligence and knowledge (von Stumm & Ackerman, 2012). Accordingly, intellectual curiosity as indicated by TIE was identified as the “third pillar of academic performance” in addition to intelligence and effort (von Stumm, Hell, & Chamorro-Premuzic, 2011, p. 574). 2. Research Questions Hitherto, TIE has been studied mainly in adult samples with a high educational level (i.e., university students). We want to expand the knowledge on TIE by examining the impact of TIE on achievement in math and three natural sciences—biology, chemistry, and physics—in a heterogeneous sample of students at the end of compulsory education. TIE has been seen as particularly informative in explaining performance differences in advanced academic or professional settings that require consolidated cognitive effort and constant commitment (Ackerman & Beier, 2004). However, the influence that TIE exerts in secondary education has rarely been studied. In general, little is known about the development and significance of TIE in this period of time when first decisions with respect to later academic or vocational training are made and formerly homogeneous learning environments begin to diverge. In contrast to a population of university students, the representative sample of students in secondary education examined in this study is not ability-restricted in any sense; therefore, the impact of TIE on achievement might be more pronounced. On the other hand, compared to adulthood the learning environments in secondary school are still relatively homogeneous given that learning is guided by curricula and school attendance is mandatory in Germany. Furthermore, the impact of TIE on educational achievement may be cumulative in nature. These arguments advocate a lower impact of TIE in secondary than in tertiary education. In order to examine the influence of different learning environments on TIE in secondary education more thoroughly, we also compare Gymnasium (i.e., the academic-track school type) to all remaining school types (nonacademic-track schools). Given at least strong measurement invariance of the instrument across school types, we compare both groups: Beside higher means of the latent variables, we assume a more pronounced differentiation of the TIE facets in the academictrack subsample (Ceci, 1991). In this context, we also consider gender differences by means of multi-group confirmatory factor analysis. Since girls have been repeatedly shown to outperform boys in reading achievement (e.g., Brunner et al., 2013) and in accordance with previous research (Wilhelm et al., 2003), one could assume an advantage of girls on this TIE facet. However, previous findings on gender differences in TIE are inconsistent (Chamorro-Premuzic et al., 2006a).
Based on the initial 59-item questionnaire (Goff & Ackerman, 1992), Wilhelm et al. (2003) developed a 18-item short scale in German measuring three core facets of TIE—reading, contemplation, and intellectual curiosity. We adapted the questionnaire to the target population of secondary students and addressed some drawbacks found in earlier studies (i.e., items with cross-loadings, ambiguous or complicated wordings; Wilhelm et al., 2003). Due to the substantial revision of the questionnaire items we investigated the psychometric quality and internal structure of the revised measure first in order to make valid statements about the construct. Our main research question was aimed at quantifying the impact of TIE on academic performance in an unselected student sample over and above the powerful predictors usually employed in educational research. Since problem solving and modeling in mathematics and the sciences depend to a large degree on abstract thinking and scientific curiosity, a positive relation between the personality construct TIE and scholastic achievement in these subjects seems plausible. 3. Method 3.1. Design and Participants Data were collected in May and June 2012 in the German National Educational Assessment 2012, a large-scale nation-wide educational assessment study in math and the sciences based on the National Educational Standards in Germany (Pant et al., 2013). Aims and scope of the study are comparable to the National Assessment of Educational Progress (NAEP). Standardized tests and questionnaires were administered in a balanced incomplete block design (Gonzalez & Rutkowski, 2010). The test session took a total of 3 h; 2 h for the achievement tests and 1 h for a student questionnaire and additional measures. Analyses presented in this paper were based on a subsample of n = 7,207 ninth-grade students from 389 schools who completed the TIE questionnaire. All common school types from the German secondary educational system were included in the sample: 36.3% of the students attended academic-track Gymnasium, the remaining school types included vocational-track and mixed-track schools. Participation was mandatory for the achievement tests (participation rate: 92%) but voluntary for the questionnaires in most federal states (participation rate: 79%). Students were not rewarded or graded in any way. Mean age was 15.5 years (SD = 0.6; n = 7,206), and half of the sample was female (49.4%). The subsample can be seen as representative for the population of German ninth-graders with respect to migration status (66.5% always spoke German at home, n = 5,845) and socio-economic status (M = 51.8, SD = 20.6, n = 5,802). 3.2. Measures 3.2.1. TIE questionnaire TIE was assessed with a thoroughly revised version of Wilhelm et al.’s (2003) German short scale. Students indicated their agreement/disagreement with 18 statements on a 4-point scale ranging from strongly agree to strongly disagree. Item translations are given in Table 2; the original German version is provided in the online supplement. 3.2.2. Achievement tests Achievement in mathematics, biology, chemistry, and physics was assessed with standardized tests based on the National Educational Standards for Mathematics and Science in Germany (Pant et al., 2013). For math, we used the global scale; in the sciences, we used the dimension scientific inquiry in biology, chemistry, and physics; example items are given in the online supplement. The math part consisted of 374 items including closed response items (i.e., true/false and multiplechoice), short answer und extended answer items. The science part comprised 118 biology, 134 chemistry, and 134 physics items.
U. Schroeders et al. / Learning and Individual Differences 43 (2015) 31–38
33
Table 1 Correlations Between the Variables on the Manifest and the Latent Level. Construct 1. 2. 3. 4. 5. 6. 7. 8. 9. 10.
Reading Contemplation Curiosity TIEa Fluid intelligence Interest Math achievement Gender Migration HISEI
1.
2.
3.
4.
5.
6.
7.
8.
9.
10.
.92 .53 .43 .63 .19 .11 .19 −.16 −.01 .15
.37 .68 .57 .84 .25 .14 .26 −.21 −.01 .20
.29 .34 .60 .68 .20 .12 .20 −.17 −.01 .16
.85 .73 .61 .88 .30 .17 .30 −.25 −.02 .24
.17 .20 .03 .20 .83 .23 .74 .03 −.16 .25
−.02 .19 .12 .10 .17 .95 .29 .27 .05 −.02
.22 .22 .15 .27 .63 .26 .91 .12 −.22 .37
−.32 −.05 .14 −.18 .02 .26 .13 – −.01 .01
−.02 .01 .00 −.01 −.11 .04 −.20 −.01 – −.14
.21 .10 .14 .21 .22 −.02 .39 .01 −.12 –
Note. nmath = 4,032; On the diagonal: reliability estimates (for variables 1–6, McDonald's ω, 1999; for variable 7 plausible values reliability). Below the diagonal: correlations between the latent variables (average of 15 imputed data sets). Above the diagonal: correlations between manifest variables (i.e., means for the variables 1–6; math achievement is the 1st plausible value of the Global Scale). The remaining variables are coded as follows: gender (1 = female, 2 = male), migration assessed by how frequently German is spoken at home (1 = always, 2 = sometimes, 3 = never), and HISEI (highest International Socio-Economic Index of Occupational Status, z-standardized score). a TIE refers to the second-order TIE factor. For this hierarchical construct, ωh is .64 (see Brunner, Nagy, & Wilhelm, 2012). For the TIE subdimensions the cells contain factor loadings on the TIE factor.
3.2.3. Figural fluid intelligence All participants were administered 16 items of the figural fluid intelligence scale from the Berlin Test of Fluid and Crystallized Intelligence (Wilhelm, Schroeders, & Schipolowski, 2014). They had to detect regularities in a sequence of geometric figures that changed, for instance, in shape and position. The scale is a good proxy for fluid intelligence, because the figural content is seen as prototypical for the construct (Wilhelm, 2004). 3.2.4. Domain-specific interest Interest in math, biology, chemistry, and physics was assessed using four subject-specific, similarly worded items (e.g., “I'm interested in math.”). Students had to indicate how much they agreed with the statements on a 4-point scale. 3.2.5. Biographical information In the present analyses, we accounted for the following background information: gender, socioeconomic status, and migration background. More precisely, students' information on the occupations of their parents was used to determine the highest International SocioEconomic Index of Occupational Status (HISEI; Ganzeboom, De Graaf, & Treiman, 1992) within the family. The index is based on international data on income and educational background of different professions. HISEI can take values between 16 (e.g., cleaners) and 90 (e.g., judges). Migration background was assessed with a proxy of how frequently German as the language of instruction was spoken at home (always, sometimes, or never). Compared to statements about place of birth, this proxy is not limited to migration that occurred in a specific generation (e.g., students' parents). 3.3. Statistical Analyses Because we had no prior experience with the revised TIE scale, we first conducted an exploratory factor analysis (EFA) with half of the sample and an oblique GEOMIN rotation. Second, confirmatory factor analysis (CFA) was used to replicate the structure with the remaining half of the sample. For both analyses, the Weighted Least Squares Mean and Variance adjusted (WLSMV) estimator was used, because it has been shown to be superior to a maximum likelihood estimator for categorical data both in terms of model rejection rates and appropriate estimation of factor loadings (Beauducel & Herzberg, 2006). The following values of the Comparative Fit Index (CFI) and the Root Mean Square Error of Approximation (RMSEA) were taken to indicate good model fit: CFI ≥ .95, RMSEA ≤ .08 (Hu & Bentler, 1999). The measurement model established with CFA was the basis for measurement invariance testing with regard to gender and school
track. In comparison to the continuous case, the steps of invariance testing for the categorical data differ (for a detailed account of the method see Schroeders & Wilhelm, 2011). In the configural invariance model,1 factor loadings and thresholds are freely estimated, whereas residual variances are fixed to 1. Because factor loadings and thresholds have to be varied in tandem in the categorical case, there is no equivalent for the metric invariance testing of the continuous case. For strong invariance both models are invariant with respect to their factor loadings and thresholds, whereas residual variances are fixed at 1 in one group and freed in the other. In the most restrictive model, factor loadings and thresholds are fixed to equity across groups, whereas residual variances are set to 1. This testing procedure checks for different forms of measurement invariance by testing whether additionally constraining measurement parameters leads to a considerable deterioration in model fit between two consecutive models (e.g., ΔCFI N .01 Cheung & Rensvold, 2002). Finally, hierarchical latent regression models were applied to the complete sample that worked on either a math or science achievement test. In order to evaluate potential effects of TIE on achievement irrespective of personal and environmental characteristics, we controlled for background variables usually considered in educational large-scale assessments (i.e., gender, socioeconomic status, and migration background) in all regression models. In a first step (model 1), we considered only the aforementioned background variables as predictors. Secondly, we added TIE to this model to establish its influence on academic achievement while controlling for the background variables (model 2). In a third model fluid intelligence and subject-specific interest were considered as predictors in addition to the background variables. In the most complex model (model 4) TIE was added to calculate the increment in the amount of explained variance over and above fluid intelligence and the subject-specific interest scale. TIE, fluid intelligence, and interest were modeled as latent variables using the respective items as indicators. Students' educational achievement was represented by 15 plausible values per subject which were computed with the Rasch model while considering individual background characteristics (von Davier, Gonzalez, & Mislevy, 2009). In all regression models, gender, migration background, and socioeconomic status were included as control variables (i.e., all latent variables and achievement were regressed on these covariates). Standard errors of the parameters were corrected for the nested structure of the data. All analyses were computed with Mplus 7.11 (Muthén & Muthén, 1998–2014).
1 It should be noted that this testing procedure, outlined in the Mplus User's Guide 7.1 (Muthén & Muthén, 1998-2014) and described in more detail in Schroeders and Wilhelm (2011), diverges from the approach provided by Millsap & Yun-Tein (2004) where for the baseline model partial threshold constraints are introduced.
34
U. Schroeders et al. / Learning and Individual Differences 43 (2015) 31–38
Table 2 Factor Loading Matrix and Items from the TIE Questionnaire. Reading
Contemplation
Avoid Thinking
Intell. Curiosity
I
II
III
IV
TIE 07 TIE 01 TIE 16 TIE 04 TIE 10 TIE 09 TIE 18* TIE 02 TIE 05 TIE 17 TIE 14 TIE 03* TIE 12* TIE 11* TIE 08* TIE 15 TIE 13 TIE 06
.95 .88 .87 .73 .71 −.04 −.16 .14 .06 .15 .09 .14 −.06 .03 −.03 .01 .26 −.07
−.03 .03 .01 .03 .03 .69 .67 .53 .51 .50 .37 .25 −.05 −.16 .08 .03 −.04 .29
.01 .00 −.01 .02 −.04 −.07 −.12 .06 .18 −.15 .13 −.03 .51 .49 .45 −.08 −.01 .06
Correlations II III IV
.34 −.01 .29
−.21 .39
−.13
Item
Item −.07 .01 .02 .05 .03 .04 .01 .19 −.07 −.05 .13 .21 .00 .02 −.11 .79 .44 .42
I'm reading a lot, at least one book each month. I read books on various topics. I like to read different kinds of books, including novels and nonfiction books. I like going to the library. Even in elementary school, I read all sorts of things. I like to get to the bottom of things. I would like to understand exactly how everything works. I think a lot about general questions, even if I do not always find an answer. I can delve into a problem deeply so that I forget the world around me. I like to think about challenging puzzles. I enjoy thinking abstractly. There are very few subjects that bore me. I think it is often troublesome to learn entirely new things. I avoid complex problems. In most cases, more thinking will lead only to more errors. I have been following social and political issues on television or the Internet. I regularly read newspapers and magazines. I like watching documentaries on TV.
Note. Factor loadings above .30 are printed in bold type. nEFA = 3,604; Model fit: χ2 = 787.7, df = 87, RMSEA = .047, CFI = .985, SRMR = .023. * Item was excluded for the latent regression models.
As a result of the estimation of plausible values, achievement data were available for all students who worked on a specific math or science booklet. There was no significant amount of missing data for gender, fluid intelligence, interest, or TIE (on the item level between 0 and 3.9%). Missingness was noteworthy for migration background (18.9%) and HISEI (19.5%). To model missing data on the manifest covariates, they were integrated into the model as single indicators of latent constructs by estimating their (co)variances. The variance-covariance matrix made use of all pairwise present information (Muthén & Muthén, 1998–2014). To ensure that the results were not biased by the handling of missing data, the models were double-checked with multiple imputations.
4. Results Table 1 gives the correlation matrix for all variables included in subsequent analyses both on the manifest (above the diagonal) and the latent level.2 Since hierarchical latent regression models are conducted, the factor saturation, McDonald's ω (1999), is given as a reliability measure of the scale on the diagonal.
4.1. Psychometric Evaluation of the TIE Measure In a first step, we conducted an EFA of the TIE measure. Parallel analysis and the scree plot suggested a four-factor solution. The factor loadings for the four-factor solution are given in Table 2. The first factor was comprised of items describing the desire and enjoyment of reading texts with different topics. Activities under the second factor can be summed up as contemplation or abstract reasoning (e.g., “I like to think about challenging puzzles”) which is neither domain-specific nor bound to a particular purpose. The third factor depicted avoidance of cognitive effort (“I think it is often troublesome to learn entirely new things”). The fourth factor can be labeled intellectual curiosity and 2 Due to the multiple-matrix design, the correlation matrix for the math part and the natural science part (i.e., biology, chemistry, and physics) rely on different but overlapping subsamples. The respective correlations of the subsample that worked on the science tests can be found in the online supplement.
described information seeking which is not restricted to a specific news medium (e.g., newspapers). Although the results of the exploratory factor analysis indicated a four factor solution, we favored a three-factor solution, dropping the factor avoidance of cognitive effort. This was done for three reasons: First, this dimension was only weakly correlated to the other dimensions (e.g., ρ(reading, avoidance) = −.10; for the complete four factor model, please see Fig. 1 in the online supplement). Second, the avoidance factor was constituted by the only three items of the questionnaire that were, in a logical sense, negatively formulated (i.e., higher agreement equals lower values on this factor). There is substantive evidence in the literature showing that negatively and positively worded items function differently (e.g., higher difficulty and lower discrimination parameters for negatively worded items), even leading to the notion that such items “impair the functioning of scales” in general (see also DiStefano & Motl, 2006; Sliter & Zickar, 2014, p. 223). In order to account for the additional variance caused by the negative item formulation, we also specified a correlated-trait-correlateduniqueness (CTCU) and a nested factor model. However, these models were rejected because the residual correlations in the CTCU model and the loadings on the nested factor, respectively, proved to be very low and unsystematic.3 Third, the avoidance scale was not part of the original questionnaire (Wilhelm et al., 2003). Taken together, this suggests that the factor termed avoidance of cognitive effort is not part of the TIE construct but rather a method artifact. We replicated the three-dimensional structure of the TIE questionnaire with a tau-congeneric measurement model (i.e., varying factor loadings) and no cross-loadings with the second half of the sample (compare Fig. 1). This model yielded satisfactory fit to the data (χ2 = 1,775.1, df = 74, CFI = .958, RMSEA = .080). Removing the last item, however, improved the model fit considerably (χ2 = 1,212.1, df = 62, CFI = .971, RMSEA = .072). In total, five items were excluded from the revised questionnaire: The three negatively worded items loading on a fourth factor (see above), one item that only had small loadings 3 Comparable models were also specified in the study of DiSteffano and Motl (2006). The authors demonstrated that the method factors were present across different questionnaires and suggested that effects associated with negatively worded items may be more adequately interpreted as response style rather than content-related variance.
U. Schroeders et al. / Learning and Individual Differences 43 (2015) 31–38
35
Fig. 1. Predicting achievement in physics with TIE. Nonsignificant parameters were omitted. Model fit statistics represent the mean of 15 replications: n = 4,011, χ2 = 2,742.7, df = 110, CFI = .936, RMSEA = .077.
on all factors of the EFA (TIE-03) and the last item due to model misfit. Correlations between the latent variables were as follows—ρ(reading, contemplation) = .45, ρ(reading, curiosity) = .42, ρ(contemplation, curiosity) = .57—and similar to those reported previously (Wilhelm et al., 2003). Note that the three factors reading, contemplation, and intellectual curiosity could equivalently be modeled as first-order factors below an overarching TIE factor. 4.2. Measurement Invariance Across School Tracks and Gender In Table 3 the results of the measurement invariance testing are summarized.4 In the first step (configural invariance, T1/G1), factor loadings and thresholds were freely estimated, whereas residual variances were set to 1 in all groups. In the second step (T2/G2), that is strong invariance, factor loadings and thresholds were set to be equal across groups. In the last step (T3/G3), in addition to the constraints of step 2, the residual variances were set to 1 in both groups. Differences in model fit were calculated between two consecutive models. A difference of ΔCFI N .01 is considered a significant deterioration in model fit (Cheung & Rensvold, 2002). For both invariance analyses, even imposing the most restrictive constraints did not lead to significant deterioration in model fit. Strict measurement invariance allows for comparisons of means on the latent (and manifest) level. In line with theoretical considerations, the means of all three TIE facets were higher in the academic-track schools 4 Given that three-dimensional model and the higher-order model are equivalent, the complete measurement invariance testing of both models yielded the same results. Furthermore, a higher-order factor model, in which beside the first-order factor loadings and the thresholds, also the second-order factor loadings were set to be equal across groups, did not lead to a deterioration in model fit (χ2 = 1210.2, df = 160, CFI = .971, RMSEA = .060), indicating that also this more restrictive form of measurement invariance was given (see also Chen, Sousa, & West, 2005). This model is equivalent to the strong measurement invariance testing of the three-dimensional model with additionally constrained factor covariances.
(standardized estimated mean differences Δα; reading: 1.14, contemplation: 0.39, intellectual curiosity: 0.30). Furthermore, the differentiation of the factor structure was more pronounced as indicated by the factor correlations5 (academic/nonacademic track: ρ(reading, contemplation) = .40/.52, ρ(reading, curiosity) = .27/.44, ρ(contemplation, curiosity) = .46/.59). In accordance with the ability-differentiation hypothesis (e.g., Tucker-Drob, 2009), it seemed plausible to assume a differentiation of TIE—without TIE being an ability measure itself—in more capable groups, since there is mutual dependency between typical intellectual performance (as expressed in TIE) and maximal intellectual performance. With respect to gender, girls outperformed boys in reading (Δα = −1.38) and contemplation (Δα= −0.12), whereas boys were more curious (Δ = 0.30). 4.3. Hierarchical Latent Regression Models For each subject, four consecutive models were specified (see Table 4). In the first model, achievement was regressed on background variables only (i.e., gender, migration background, and socioeconomic status). In the second model, TIE was added as a higher order factor capturing the core of the different TIE facets. This model was preferred over the three-dimensional model because the first-order factors were based on few items which limits the construct representation on the level of subdimensions and is in general associated with a lower reliability of the scale. Furthermore, the correlation between contemplation and curiosity was substantial (ρ=.57), resulting in distorted estimations of the regression weights within the three-dimensional model (see online supplement). In the third model, fluid intelligence and subject-specific interest were included as latent predictors besides the background variables. In the fourth model, model 3 was extended with TIE. The column 5 Variances were comparable across groups (academic/non-academic track: σ (reading) = 3.60/4.00 (SE .26/.26); σ 2(contemplation) = 1.41/1.46 (SE .14/.14); σ 2 (intellectual curiosity) = 0.46/0.56) (SE .06/.06). 2
36
U. Schroeders et al. / Learning and Individual Differences 43 (2015) 31–38
Table 3 Measurement invariance analyses of the TIE questionnaire across school tracks and gender.
School track (T1) Configural invariance (T2) Strong invariance (T3) Strict invariance Gender (G1) Configural invariance (G2) Strong invariance (G3) Strict invariance
χ2/df
p
CFI
RMSEA
1250.1/124
b.01
.968
.071
1310.6/157 1284.4/170
b.01 b.01
.968 .969
.064 .060
1189.9/124
b.01
.970
.069
1557.9/157 1536.8/170
b.01 b.01
.961 .962
.070 .067
Δ CFI
more pronounced than for the natural sciences. Furthermore, interest in mathematics outperformed TIE as a predictor of mathematics achievement.
Δ RMSEA
5. Discussion .000 −.001
−.007 −.004
.009 −.001
.001 -.003
Note. CFI = Comparative Fit Index. RMSEA = Root Mean Square Error of Approximation. Differences in model fit refer to consecutive models (e.g., T3–T2). Theta parameterization was used for all analyses (B. O. Muthén & Asparouhov, 2002). When using the WLSMV estimator, df could not be computed directly, instead they “are calculated from the estimated covariance matrix and a diagonal weight matrix” (Beauducel & Herzberg, 2006, p. 192). For this reason, the χ2 statistics between two (consecutive) models cannot be compared directly.
R2 in Table 4 denotes the proportion of variance explained by all predictors in the model; the column ΔR2 gives the incremental variance explained by TIE over and above all other predictors included in the preceding model (e.g., P4–P3 and P2–P1). The pattern of results was similar for all science subjects. For example, 16% of the interindividual differences in physics were explained by the background variables. The value increased to 24.9% after adding TIE to the model (see Fig. 1 and model P2 in Table 4). In other words, TIE explained about 9% of the interindividual differences in physics achievement beyond the background variables. In the last model (P4, Table 4), fluid intelligence and interest were entered as additional predictors. As expected, fluid intelligence was the strongest predictor, explaining about 28% of the interindividual differences in physics achievement, whereas the contribution of subject-specific interest was close to 0 and not statistically significant. TIE exerted only a small influence over and above intelligence and interest. Specifically, individual differences in the second-order TIE factor accounted for 1.3% of the variance in physics achievement after controlling for individual differences in background variables, fluid intelligence, and interest. Results for mathematics were similar, but the influence of fluid intelligence was even
The influence of the personality trait TIE on academic achievement has been repeatedly emphasized in the literature. However, the reported effects may be biased or overestimated because the studies did not account for differences in relevant background variables (e.g., SES), mostly relied on regression analyses with manifest indicators, and were based on small selective samples (e.g., university students). Therefore, we adapted and revised an existing short scale measuring the essential core areas of TIE (Wilhelm et al., 2003), administered it to a heterogeneous sample of students in Grade 9, and modeled its relations to standardized achievement tests with a latent variable approach. In the following, we (a) discuss the meaning and impact of the three factors of the TIE questionnaire in more detail, (b) relate the results of the hierarchical latent regression to the existing research literature, and (c) point out possible study objectives for future research. Considering that reading is one of the most powerful sources of learning, reading books, articles, and newspapers is evidently a crucial concomitant of intellectual engagement. Because TIE reading is not limited with respect to content, it comes close to the “hungry mind” that is open to new ideas (von Stumm et al., 2011, p. 583). TIE has also been described as the “desire to engage and understand [one's] world [...], a need to know” (Goff & Ackerman, 1992, p. 539), which seems best expressed by contemplation. This is also reflected by the fact that this factor had the highest loading on the second-order factor. Furthermore, the theoretical and empirical overlap between TIE contemplation and fluid intelligence was most pronounced (about ρ=.30), advocating the close interplay between typical intellectual behavior and psychometric intelligence. The factor intellectual curiosity, which has been labeled the “third pillar of academic performance” besides intelligence and effort (von Stumm et al., 2011, p. 574), was measured with only three items, thus narrowing the breadth of the dimension. The questions focused on topics that receive attention in mass media (i.e., politics and current event knowledge). If the items had been more strongly related to scientific inquiry, relations with mathematics and science achievement may have been stronger.
Table 4 Standardized Regression Coefficients of Hierarchical Prediction Models. Gender
Math
Biology
Chemistry
Physics
Migration
SES
TIE
b
(SE)
b
(SE)
b
(SE)
M1 M2 M3 M4 B1 B2 B3 B4
.12 .20 .07 .09 −.06 .02 −.07 −.04
(.02) (.02) (.01) (.01) (.02) (.02) (.02) (.02)
−.17 −.17 −.09 −.09 −.20 −.22 −.13 −.15
(.02) (.02) (.02) (.01) (.02) (.02) (.02) (.02)
.35 .27 .19 .18 .32 .24 .18 .15
(.02) (.02) (.02) (.02) (.02) (.02) (.02) (.02)
C1 C2 C3 C4 P1 P2 P3 P4
.03 .12 −.01 .03 .03 .12 −.02 .03
(.02) (.02) (.02) (.02) (.02) (.02) (.02) (.02)
−.21 −.23 −.14 −.15 −.20 −.22 −.12 −.14
(.02) (.02) (.02) (.01) (.02) (.02) (.02) (.02)
.31 .22 .16 .13 .32 .24 .17 .15
(.02) (.02) (.02) (.02) (.02) (.02) (.02) (.02)
Gf
b
(SE)
.28
(.02)
.07
(.02)
.31
(.02)
.14
(.03)
.34
(.02)
.15
(.02)
.33
(.02)
.14
(.02)
b
R2
Interest (SE)
b
ΔR2
(SE)
.65 .63
(.02) (.01)
.13 .12
(.02) (.02)
.54 .50
(.02) (.02)
.06 .00
(.02) (.02)
.56 .52
(.02) (.02)
.09 .04
(.02) (.02)
.56 .53
(.02) (.02)
.08 .03
(.02) (.02)
Model fit CFI
18.0 24.3 61.8 62.3 16.2 24.1 43.0 44.5 15.6 25.4 45.9 47.7 16.0 24.9 46.3 47.6
6.3 .5 7.9 1.5
9.8 1.8 8.9 1.3
RMSEA
– .94 1.00 .96 – .94 1.00 .97
– .08 .02 .04 – .08 .02 .03
– .94 1.00 .97 – .94 1.00 .97
– .08 .02 .03 – .08 .02 .04
Note. nmath = 4,032; nscience = 4,011. Regression weights (b) in bold type were significant (p b .05). ΔR2 is the incremental variance explained by TIE over and above all other predictors in the model. SES = socioeconomic status, TIE = typical intellectual engagement, Gf = (figural) fluid intelligence. TIE, gf, and subject-specific interest were included in the hierarchical regression models as latent variables. CFI = Comparative Fit index. RMSEA = Root Mean Square of Approximation.
U. Schroeders et al. / Learning and Individual Differences 43 (2015) 31–38
When interpreting the impact of TIE on school performance it has to be taken into account that the effect was determined after controlling for influential background variables and intelligence, respectively, which is a conservative approach. Consistent with previous research (e.g., see Deary, Strand, Smith, & Fernandes, 2007; Neisser et al., 1996), intelligence already explained a large amount of variance in achievement when investigated on the latent level. Nevertheless, even a powerful predictor such as fluid intelligence still leaves about 50% of the variance in academic achievement unexplained. Our results show that the contribution of TIE in closing this gap is almost negligible. Importantly, the contribution of TIE should be interpreted in comparison with the explanatory power of other noncognitive variables such as subject-specific interest. For all science subjects, domain-general TIE outperformed subject-specific interest as a predictor of academic achievement. However, in contrast to previous studies (Marsh et al., 2005), we found a significant influence of interest that exceeded the influence of TIE for math achievement. This is first evidence that there are differential patterns depending on the subject in question. It has also been argued that TIE exhibits “stronger relations to humanities-type knowledge than to the sciences” (Rolfhus & Ackerman, 1999, p. 513). Therefore, the predictive power of TIE might be greater for other subjects such as art or literature. Since Mussel (2010, 2013) established a high conceptual overlap between TIE and related constructs such as need for cognition and the facet openness to ideas of the Big Five and also provided empirical evidence (all belong within the operation Think and the process Seek within his two-dimensional theory of intellect), the present findings can give indication that the power of such constructs in predicting math and science achievement might also be marginal. The findings for the student sample at hand were in line with previous studies on university students (e.g., Chamorro-Premuzic et al., 2006a, 2006b; Wilhelm et al., 2003). For example, Powell and Nettelbeck (2014) reported that TIE explained only 1.8% of the individual differences in academic achievement over and above fluid intelligence at the time of college admission. We found no indication that the influence of TIE was more pronounced in a non-selective, abilityheterogeneous sample. Comparing the results of both studies, the increment of TIE beyond fluid intelligence seems to be stable at different stages of the educational biography. As a limitation of the present study, we have to take into account that the original TIE questionnaire (Wilhelm et al., 2003) had to be substantially revised in order to suit the target population of ninth-graders and that additional evidence concerning the criterion-related and construct validity of the measure is pending. However, given the psychometric soundness of the revised measure, one essential question is why—in spite of the sound theoretical assumptions—the influence of intellectual investment traits such as TIE on academic achievement is marginal when the influence of the predominant fluid intelligence is taken into account. On the one hand, the learning environment in secondary education is—in comparison to adulthood—relatively homogeneous, which may entail that interindividual differences in academic performance can be largely attributed to differences in fluid intelligence, thus preventing TIE from exerting its potential influence. On the other hand, the results found here may be specific for the mathematicalscientific domain. Higher correlations would be expected for subjects that rely more strongly on crystallized intelligence (e.g., history or literature). Therefore, future research should consider a broader range of educational achievements (e.g., humanities and social sciences), and samples of adults with heterogeneous learning opportunities in order to investigate the interplay of TIE with cognitive and motivational factors in more detail. Appendix A. Supplementary data Supplementary data to this article can be found online at http://dx. doi.org/10.1016/j.lindif.2015.08.030.
37
References Ackerman, P. L. (1996). A theory of adult intellectual development: Process, personality, interests, and knowledge. Intelligence, 22, 227–257. http://dx.doi.org/10.1016/ S0160-2896(96)90016–1. Ackerman, P. L., & Beier, M. E. (2004). Knowledge and intelligence. In O. Wilhelm, & R. W. Engle (Eds.), Handbook of understanding and measuring intelligence (pp. 125–139). Thousand Oaks, CA: Sage Publications. Ackerman, P. L., & Goff, M. (1994). Typical intellectual engagement and personality: Reply to Rocklin (1994). Journal of Educational Psychology, 86, 150–153. http://dx.doi.org/ 10.1037/0022-0663.86.1.150. Ackerman, P. L., & Heggestad, E. D. (1997). Intelligence, personality, and interests: evidence for overlapping traits. Psychological Bulletin, 121, 219–245. http://dx.doi. org/10.1177/1069072703011002006. Ackerman, P. L., & Rolfhus, E. L. (1999). The locus of adult intelligence: Knowledge, abilities, and nonability traits. Psychology and Aging, 14, 314–330. http://dx.doi.org/ 10.1037/0882-7974.14.2.314. Beauducel, A., & Herzberg, P. Y. (2006). On the performance of maximum likelihood versus means and variance adjusted weighted least squares estimation in CFA. Structural Equation Modeling: A Multidisciplinary Journal, 13, 186–203. Binet, A., & Simon, T. (1904). Méthodes nouvelles pour le diagnostic du niveau intellectuel des anormaux [New methods for the diagnosis of the intellectual level of subnormals]. L'Année Psychologique, 11, 191–244. http://dx.doi.org/10.3406/psy.1904.3675. Brunner, M., Gogol, K. M., Sonnleitner, P., Keller, U., Krauss, S., & Preckel, F. (2013). Gender differences in the mean level, variability, and profile shape of student achievement: Results from 41 countries. Intelligence, 41, 378–395. http://dx.doi.org/10.1016/j. intell.2013.05.009. Brunner, M., Nagy, G., & Wilhelm, O. (2012). A tutorial on hierarchically structured constructs. Journal of Personality, 80, 796–846. http://dx.doi.org/10.1111/j.14676494.2011.00749.x. Ceci, S. J. (1991). How much does schooling influence general intelligence and its cognitive components? A reassessment of the evidence. Developmental Psychology, 27, 703–722. Chamorro-Premuzic, T., Furnham, A., & Ackerman, P. L. (2006a). Ability and personality correlates of general knowledge. Personality and Individual Differences, 41, 419–429. http://dx.doi.org/10.1016/j.paid.2005.11.036. Chamorro-Premuzic, T., Furnham, A., & Ackerman, P. L. (2006b). Incremental validity of the typical intellectual engagement scale as predictor of different academic performance measures. Journal of Personality Assessment, 87, 261–268. http://dx.doi.org/ 10.1207/s15327752jpa8703_07. Chen, F. F., Sousa, K. H., & West, S. G. (2005). Teacher's corner: testing measurement invariance of second-order factor models. Structural Equation Modeling, 12, 471–492. Cheung, G. W., & Rensvold, R. B. (2002). Evaluating goodness-of-fit indexes for testing measurement invariance. Structural Equation Modeling: A Multidisciplinary Journal, 9, 233–255. Cronbach, L. J. (1949). Essentials of psychological testing. New York, NY: Harper. Deary, I. J., Strand, S., Smith, P., & Fernandes, C. (2007). Intelligence and educational achievement. Intelligence, 35, 13–21. http://dx.doi.org/10.1016/j.intell.2006.02.001. DiStefano, C., & Motl, R. W. (2006). Further investigating method effects associated with negatively worded items on self-report surveys. Structural Equation Modeling: A Multidisciplinary Journal, 13, 440–464. http://dx.doi.org/10.1207/ s15328007sem1303_6. Furnham, A., Monsen, J., & Ahmetoglu, G. (2009). Typical intellectual engagement, Big Five personality traits, approaches to learning and cognitive ability predictors of academic performance. British Journal of Educational Psychology, 79, 769–782. http://dx.doi.org/10.1348/978185409X412147. Ganzeboom, H. B. G., De Graaf, P. M., & Treiman, D. J. (1992). A standard international socio-economic index of occupational status. Social Science Research, 21, 1–56. http://dx.doi.org/10.1016/0049-089X(92)90017-B. Goff, M., & Ackerman, P. L. (1992). Personality-intelligence relations: Assessment of typical intellectual engagement. Journal of Educational Psychology, 84, 537–552. http:// dx.doi.org/10.1037/0022-0663.84.4.537. Gonzalez, E., & Rutkowski, L. (2010). Principles of multiple matrix booklet designs and parameter recovery in large-scale assessments. In D. Hastedt, & D. von Davier (Eds.), IERI Monograph Series: Issues and Methodologies in Large-Scale Assessments, Vol. 3. (pp. 125–156). Hamburg: IEA-ETS Research Institute. Hambrick, D. Z. (2004). The role of domain knowledge in higher-level cognition. In O. Wilhelm, & R. W. Engle (Eds.), Handbook of understanding and measuring intelligence (pp. 361–372). Thousand Oaks, CA: Sage Publications. Hu, L., & Bentler, P. M. (1999). Cutoff criteria for fit indexes in covariance structure analysis: Conventional criteria versus new alternatives. Structural Equation Modeling: A Multidisciplinary Journal, 6, 1–55. http://dx.doi.org/10.1080/10705519909540118. Kuncel, N. R., Hezlett, S. A., & Ones, D. S. (2004). Academic performance, career potential, creativity, and job performance: can one construct predict them all? Journal of Personality and Social Psychology, 86, 148–161. http://dx.doi.org/10.1037/00223514.86.1.148. Marsh, H. W., Trautwein, U., Lüdtke, O., Köller, O., & Baumert, J. (2005). Academic selfconcept, interest, grades, and standardized test scores: reciprocal effects models of causal ordering. Child Development, 76, 397–416. http://dx.doi.org/10.1111/j.14678624.2005.00853.x. McDonald, R. P. (1999). Test theory: A unified treatment. Mahwah, NJ: Erlbaum. Millsap, R. E., & Yun-Tein, J. (2004). Assessing factorial invariance in ordered-categorical measures. Multivariate Behavioral Research, 39, 479–515. Mussel, P. (2010). Epistemic curiosity and related constructs: Lacking evidence of discriminant validity. Personality and Individual Differences, 49, 506–510. http://dx.doi. org/10.1016/j.paid.2010.05.014.
38
U. Schroeders et al. / Learning and Individual Differences 43 (2015) 31–38
Mussel, P. (2013). Intellect: A theoretical framework for personality traits related to intellectual achievements. Journal of Personality and Social Psychology, 104, 885–906. http://dx.doi.org/10.1037/a0031918. Muthén, B. O., & Asparouhov, T. (2002). Latent variable analysis with categorical outcomes: multiple-group and growth modeling in mplus. Mplus Web Notes, 4, 1–22. Muthén, L. K., & Muthén, B. O. (1998-2014). Mplus user's guide (7th ed.). Los Angeles, CA: Muthén & Muthén. Neisser, U., Boodoo, G., Bouchard, T. J., Jr., Boykin, A. W., Brody, N., Ceci, S. J., ... Urbina, S. (1996). Intelligence: Knowns and unknowns. American Psychologist, 51, 77–101. http://dx.doi.org/10.1037/0003-066X.51.2.77. Pant, H. A., Stanat, P., Schroeders, U., Roppelt, A., Siegle, T., & Pöhlmann, C. (2013). The IQB National Assessment Study 2012: Competencies in mathematics and the sciences at the end of secondary level I. Münster: Waxmann. Powell, C., & Nettelbeck, T. (2014). Intellectual curiosity may not incrementally predict academic success. Personality and Individual Differences, 64, 7–11. http://dx.doi.org/ 10.1016/j.paid.2014.01.045. Rolfhus, E. L., & Ackerman, P. L. (1999). Assessing individual differences in knowledge: Knowledge, intelligence, and related traits. Journal of Educational Psychology, 91, 511–526. http://dx.doi.org/10.1037/0022-0663.91.3.511. Schroeders, U., & Wilhelm, O. (2011). Equivalence of reading and listening comprehension across test media. Educational and Psychological Measurement, 71, 849–869. http://dx.doi.org/10.1177/0013164410391468.
Sliter, K. A., & Zickar, M. J. (2014). An IRT examination of the psychometric functioning of negatively worded personality items. Educational and Psychological Measurement, 74, 214–226. http://dx.doi.org/10.1177/0013164413504584. Tucker-Drob, E. M. (2009). Differentiation of cognitive abilities across the life span. Developmental Psychology, 45, 1097–1118. http://dx.doi.org/10.1037/a0015864. von Davier, M., Gonzalez, E., & Mislevy, R. J. (2009). What are plausible values and why are they useful. IERI Monograph Series, 4. (pp. 9–36). von Stumm, S., & Ackerman, P. L. (2012). Investment and intellect: A review and metaanalysis. Psychological Bulletin, 139, 841–869. http://dx.doi.org/10.1037/a0030746. von Stumm, S., Hell, B., & Chamorro-Premuzic, T. (2011). The hungry mind: Intellectual curiosity is the third pillar of academic performance. Perspectives on Psychological Science, 6, 574–588. http://dx.doi.org/10.1177/1745691611421204. Wilhelm, O. (2004). Measuring reasoning ability. In O. Wilhelm, & R. W. Engle (Eds.), Handbook of understanding and measuring intelligence (pp. 373–392). Thousand Oaks, CA: Sage Publications. Wilhelm, O., Schroeders, U., & Schipolowski, S. (2014). Berliner Test zur Erfassung fluider und kristalliner Intelligenz für die 8. bis 10. Jahrgangsstufe [Berlin Test of fluid and crystallized intelligence for grades 8–10]. Göttingen: Hogrefe. Wilhelm, O., Schulze, R., Schmiedek, F., & Süß, H. -M. (2003). Interindividuelle Unterschiede im typischen intellektuellen Engagement [Individual differences in typical intellectual engagement]. Diagnostica, 49, 49–60. http://dx.doi.org/10.1026// 0012-1924.49.2.49.
Contents lists available at ScienceDirect
Learning and Individual Differences journal homepage: www.elsevier.com/locate/lindif
Typical intellectual engagement and achievement in math and the sciences in secondary education☆ Ulrich Schroeders a,⁎, Stefan Schipolowski b, Katrin Böhme c a b c
Department of Educational Science, University of Bamberg, Feldkirchenstraße 21, 96045 Bamberg, Germany Department of Psychology, Humboldt-Universität zu Berlin Inclusive Education, University of Potsdam
a r t i c l e
i n f o
Article history: Received 13 August 2014 Received in revised form 14 August 2015 Accepted 18 August 2015 Available online xxxx Keywords: Typical intellectual engagement Math achievement Science achievement Fluid intelligence Subject-specific interest
a b s t r a c t Typical Intellectual Engagement (TIE) is considered a key trait in explaining individual differences in educational achievement in advanced academic or professional settings. Research in secondary education, however, has focused on cognitive and conative factors rather than personality. In the present large-scale study, we investigated the relation between TIE and achievement tests in math and science in Grade 9. A three-dimensional model (reading, contemplation, intellectual curiosity) provided high theoretical plausibility and satisfactory model fit. We quantified the predictive power of TIE with hierarchical regression models. After controlling for gender, migration background, and socioeconomic status, TIE contributed substantially to the explanation of math and science achievement. However, this effect almost disappeared after fluid intelligence and interest were added into the model. Thus, we found only limited support for the significance of TIE on educational achievement, at least for subjects more strongly relying on fluid abilities such as math and science. © 2015 Elsevier Inc. All rights reserved.
Individual differences in academic performance have been linked to a plethora of cognitive, conative, and affective factors. The use of cognitive abilities to predict school performance has a long tradition, dating back to the first intelligence test (Binet & Simon, 1904). Since then, the power of cognitive factors such as fluid intelligence or domain-specific knowledge for the prediction of performance in secondary and tertiary education has been repeatedly demonstrated (e.g., Hambrick, 2004; Kuncel, Hezlett, & Ones, 2004). Conative factors such as academic self-concept or subject-specific interest also affect students' performance. For instance, math self-concept in Grade 7 explained math achievement in Grade 10 over and above math ability in Grade 7 (Marsh, Trautwein, Lüdtke, Köller, & Baumert, 2005). In contrast to cognitive and motivational constructs, personality and other affective factors still play a minor role in educational research. Several reasons can be given for this circumstance: First, as cognitive abilities are by far the most powerful predictors of academic achievement (e.g., Furnham, Monsen, & Ahmetoglu, 2009), the potential benefit of noncognitive variables such as personality constructs in terms of incremental validity appears marginal. Second, because nonability traits operationalized with measures of typical behavior (e.g., questionnaires; see Cronbach, 1949) are prone to faking good and social desirability effects, they have received less ☆ During the preparation of this manuscript, Stefan Schipolowski was a fellow of the International Max Planck Research School The Life Course: Evolutionary and Ontogenetic Dynamics (LIFE). ⁎ Corresponding author. E-mail address: [email protected] (U. Schroeders).
http://dx.doi.org/10.1016/j.lindif.2015.08.030 1041-6080/© 2015 Elsevier Inc. All rights reserved.
attention in the high-stakes contexts (e.g., college admission) that have steered a lot of research. Finally, compared to other noncognitive characteristics (e.g., interest or self-efficacy), personality is understood as a set of more stable behavioral dispositions which are only marginally sensitive to intervention and schooling. In the context of educational research with pupils, however, the focus lies explicitly on the more malleable aspects of human behavior. 1. Typical Intellectual Engagement: A Promising Candidate The term personality represents a broad set of diverse constructs. Among the constructs that are particularly promising for predicting individual differences in academic performance are the so-called intellectual investment traits such as need for cognition, the Big Five trait openness, and typical intellectual engagement (TIE). In this paper, we focus on typical intellectual engagement (Ackerman & Goff, 1994; Goff & Ackerman, 1992). TIE describes a person's engagement in intellectual activity and his or her interest in and need for a profound understanding of complex issues. Therefore, TIE characterizes the attraction/aversion that an intellectually demanding task exerts on an individual. Individuals with high engagement receive better grades, score significantly higher on standardized ability tests (Chamorro-Premuzic, Furnham, & Ackerman, 2006b; Wilhelm, Schulze, Schmiedek, & Süß, 2003), and possess better general knowledge (Chamorro-Premuzic, Furnham, & Ackerman, 2006a). The construct TIE seems predestined for the prediction of scholastic achievement. From a theoretical point of view, TIE should have a
32
U. Schroeders et al. / Learning and Individual Differences 43 (2015) 31–38
positive impact on the acquisition of skills and knowledge in adulthood. The willingness to solve complex issues, to contemplate and to deal with new information mirrors the conditions required for successful learning. For two persons with the same level of cognitive ability, TIE might be a key construct for explaining interindividual differences in learning and academic achievement. Furthermore, as a measure of typical behavior, TIE may “play an important cumulative role in knowledge acquisition and retention” (Rolfhus & Ackerman, 1999, p. 513) and may provide a link between cognitive and conative factors, building clusters or “trait complexes” (Ackerman & Heggestad, 1997). In PPIK theory (intelligence-as-process, personality, interests, and intelligence-asknowledge; Ackerman, 1996), so-called intellectual investment traits are conceived as crucial for the transition from process to knowledge (Ackerman & Rolfhus, 1999; Rolfhus & Ackerman, 1999; von Stumm & Ackerman, 2012). From an empirical perspective, TIE has been shown to be moderately correlated with scholastic and academic performance. For example, TIE was positively correlated with grade point average, college admission test scores (Wilhelm et al., 2003), and course measures of academic performance such as tutorial reports, essays, or project reports (Chamorro-Premuzic et al., 2006b). A recent meta-analysis reported substantial relations of TIE with crystallized intelligence and knowledge (von Stumm & Ackerman, 2012). Accordingly, intellectual curiosity as indicated by TIE was identified as the “third pillar of academic performance” in addition to intelligence and effort (von Stumm, Hell, & Chamorro-Premuzic, 2011, p. 574). 2. Research Questions Hitherto, TIE has been studied mainly in adult samples with a high educational level (i.e., university students). We want to expand the knowledge on TIE by examining the impact of TIE on achievement in math and three natural sciences—biology, chemistry, and physics—in a heterogeneous sample of students at the end of compulsory education. TIE has been seen as particularly informative in explaining performance differences in advanced academic or professional settings that require consolidated cognitive effort and constant commitment (Ackerman & Beier, 2004). However, the influence that TIE exerts in secondary education has rarely been studied. In general, little is known about the development and significance of TIE in this period of time when first decisions with respect to later academic or vocational training are made and formerly homogeneous learning environments begin to diverge. In contrast to a population of university students, the representative sample of students in secondary education examined in this study is not ability-restricted in any sense; therefore, the impact of TIE on achievement might be more pronounced. On the other hand, compared to adulthood the learning environments in secondary school are still relatively homogeneous given that learning is guided by curricula and school attendance is mandatory in Germany. Furthermore, the impact of TIE on educational achievement may be cumulative in nature. These arguments advocate a lower impact of TIE in secondary than in tertiary education. In order to examine the influence of different learning environments on TIE in secondary education more thoroughly, we also compare Gymnasium (i.e., the academic-track school type) to all remaining school types (nonacademic-track schools). Given at least strong measurement invariance of the instrument across school types, we compare both groups: Beside higher means of the latent variables, we assume a more pronounced differentiation of the TIE facets in the academictrack subsample (Ceci, 1991). In this context, we also consider gender differences by means of multi-group confirmatory factor analysis. Since girls have been repeatedly shown to outperform boys in reading achievement (e.g., Brunner et al., 2013) and in accordance with previous research (Wilhelm et al., 2003), one could assume an advantage of girls on this TIE facet. However, previous findings on gender differences in TIE are inconsistent (Chamorro-Premuzic et al., 2006a).
Based on the initial 59-item questionnaire (Goff & Ackerman, 1992), Wilhelm et al. (2003) developed a 18-item short scale in German measuring three core facets of TIE—reading, contemplation, and intellectual curiosity. We adapted the questionnaire to the target population of secondary students and addressed some drawbacks found in earlier studies (i.e., items with cross-loadings, ambiguous or complicated wordings; Wilhelm et al., 2003). Due to the substantial revision of the questionnaire items we investigated the psychometric quality and internal structure of the revised measure first in order to make valid statements about the construct. Our main research question was aimed at quantifying the impact of TIE on academic performance in an unselected student sample over and above the powerful predictors usually employed in educational research. Since problem solving and modeling in mathematics and the sciences depend to a large degree on abstract thinking and scientific curiosity, a positive relation between the personality construct TIE and scholastic achievement in these subjects seems plausible. 3. Method 3.1. Design and Participants Data were collected in May and June 2012 in the German National Educational Assessment 2012, a large-scale nation-wide educational assessment study in math and the sciences based on the National Educational Standards in Germany (Pant et al., 2013). Aims and scope of the study are comparable to the National Assessment of Educational Progress (NAEP). Standardized tests and questionnaires were administered in a balanced incomplete block design (Gonzalez & Rutkowski, 2010). The test session took a total of 3 h; 2 h for the achievement tests and 1 h for a student questionnaire and additional measures. Analyses presented in this paper were based on a subsample of n = 7,207 ninth-grade students from 389 schools who completed the TIE questionnaire. All common school types from the German secondary educational system were included in the sample: 36.3% of the students attended academic-track Gymnasium, the remaining school types included vocational-track and mixed-track schools. Participation was mandatory for the achievement tests (participation rate: 92%) but voluntary for the questionnaires in most federal states (participation rate: 79%). Students were not rewarded or graded in any way. Mean age was 15.5 years (SD = 0.6; n = 7,206), and half of the sample was female (49.4%). The subsample can be seen as representative for the population of German ninth-graders with respect to migration status (66.5% always spoke German at home, n = 5,845) and socio-economic status (M = 51.8, SD = 20.6, n = 5,802). 3.2. Measures 3.2.1. TIE questionnaire TIE was assessed with a thoroughly revised version of Wilhelm et al.’s (2003) German short scale. Students indicated their agreement/disagreement with 18 statements on a 4-point scale ranging from strongly agree to strongly disagree. Item translations are given in Table 2; the original German version is provided in the online supplement. 3.2.2. Achievement tests Achievement in mathematics, biology, chemistry, and physics was assessed with standardized tests based on the National Educational Standards for Mathematics and Science in Germany (Pant et al., 2013). For math, we used the global scale; in the sciences, we used the dimension scientific inquiry in biology, chemistry, and physics; example items are given in the online supplement. The math part consisted of 374 items including closed response items (i.e., true/false and multiplechoice), short answer und extended answer items. The science part comprised 118 biology, 134 chemistry, and 134 physics items.
U. Schroeders et al. / Learning and Individual Differences 43 (2015) 31–38
33
Table 1 Correlations Between the Variables on the Manifest and the Latent Level. Construct 1. 2. 3. 4. 5. 6. 7. 8. 9. 10.
Reading Contemplation Curiosity TIEa Fluid intelligence Interest Math achievement Gender Migration HISEI
1.
2.
3.
4.
5.
6.
7.
8.
9.
10.
.92 .53 .43 .63 .19 .11 .19 −.16 −.01 .15
.37 .68 .57 .84 .25 .14 .26 −.21 −.01 .20
.29 .34 .60 .68 .20 .12 .20 −.17 −.01 .16
.85 .73 .61 .88 .30 .17 .30 −.25 −.02 .24
.17 .20 .03 .20 .83 .23 .74 .03 −.16 .25
−.02 .19 .12 .10 .17 .95 .29 .27 .05 −.02
.22 .22 .15 .27 .63 .26 .91 .12 −.22 .37
−.32 −.05 .14 −.18 .02 .26 .13 – −.01 .01
−.02 .01 .00 −.01 −.11 .04 −.20 −.01 – −.14
.21 .10 .14 .21 .22 −.02 .39 .01 −.12 –
Note. nmath = 4,032; On the diagonal: reliability estimates (for variables 1–6, McDonald's ω, 1999; for variable 7 plausible values reliability). Below the diagonal: correlations between the latent variables (average of 15 imputed data sets). Above the diagonal: correlations between manifest variables (i.e., means for the variables 1–6; math achievement is the 1st plausible value of the Global Scale). The remaining variables are coded as follows: gender (1 = female, 2 = male), migration assessed by how frequently German is spoken at home (1 = always, 2 = sometimes, 3 = never), and HISEI (highest International Socio-Economic Index of Occupational Status, z-standardized score). a TIE refers to the second-order TIE factor. For this hierarchical construct, ωh is .64 (see Brunner, Nagy, & Wilhelm, 2012). For the TIE subdimensions the cells contain factor loadings on the TIE factor.
3.2.3. Figural fluid intelligence All participants were administered 16 items of the figural fluid intelligence scale from the Berlin Test of Fluid and Crystallized Intelligence (Wilhelm, Schroeders, & Schipolowski, 2014). They had to detect regularities in a sequence of geometric figures that changed, for instance, in shape and position. The scale is a good proxy for fluid intelligence, because the figural content is seen as prototypical for the construct (Wilhelm, 2004). 3.2.4. Domain-specific interest Interest in math, biology, chemistry, and physics was assessed using four subject-specific, similarly worded items (e.g., “I'm interested in math.”). Students had to indicate how much they agreed with the statements on a 4-point scale. 3.2.5. Biographical information In the present analyses, we accounted for the following background information: gender, socioeconomic status, and migration background. More precisely, students' information on the occupations of their parents was used to determine the highest International SocioEconomic Index of Occupational Status (HISEI; Ganzeboom, De Graaf, & Treiman, 1992) within the family. The index is based on international data on income and educational background of different professions. HISEI can take values between 16 (e.g., cleaners) and 90 (e.g., judges). Migration background was assessed with a proxy of how frequently German as the language of instruction was spoken at home (always, sometimes, or never). Compared to statements about place of birth, this proxy is not limited to migration that occurred in a specific generation (e.g., students' parents). 3.3. Statistical Analyses Because we had no prior experience with the revised TIE scale, we first conducted an exploratory factor analysis (EFA) with half of the sample and an oblique GEOMIN rotation. Second, confirmatory factor analysis (CFA) was used to replicate the structure with the remaining half of the sample. For both analyses, the Weighted Least Squares Mean and Variance adjusted (WLSMV) estimator was used, because it has been shown to be superior to a maximum likelihood estimator for categorical data both in terms of model rejection rates and appropriate estimation of factor loadings (Beauducel & Herzberg, 2006). The following values of the Comparative Fit Index (CFI) and the Root Mean Square Error of Approximation (RMSEA) were taken to indicate good model fit: CFI ≥ .95, RMSEA ≤ .08 (Hu & Bentler, 1999). The measurement model established with CFA was the basis for measurement invariance testing with regard to gender and school
track. In comparison to the continuous case, the steps of invariance testing for the categorical data differ (for a detailed account of the method see Schroeders & Wilhelm, 2011). In the configural invariance model,1 factor loadings and thresholds are freely estimated, whereas residual variances are fixed to 1. Because factor loadings and thresholds have to be varied in tandem in the categorical case, there is no equivalent for the metric invariance testing of the continuous case. For strong invariance both models are invariant with respect to their factor loadings and thresholds, whereas residual variances are fixed at 1 in one group and freed in the other. In the most restrictive model, factor loadings and thresholds are fixed to equity across groups, whereas residual variances are set to 1. This testing procedure checks for different forms of measurement invariance by testing whether additionally constraining measurement parameters leads to a considerable deterioration in model fit between two consecutive models (e.g., ΔCFI N .01 Cheung & Rensvold, 2002). Finally, hierarchical latent regression models were applied to the complete sample that worked on either a math or science achievement test. In order to evaluate potential effects of TIE on achievement irrespective of personal and environmental characteristics, we controlled for background variables usually considered in educational large-scale assessments (i.e., gender, socioeconomic status, and migration background) in all regression models. In a first step (model 1), we considered only the aforementioned background variables as predictors. Secondly, we added TIE to this model to establish its influence on academic achievement while controlling for the background variables (model 2). In a third model fluid intelligence and subject-specific interest were considered as predictors in addition to the background variables. In the most complex model (model 4) TIE was added to calculate the increment in the amount of explained variance over and above fluid intelligence and the subject-specific interest scale. TIE, fluid intelligence, and interest were modeled as latent variables using the respective items as indicators. Students' educational achievement was represented by 15 plausible values per subject which were computed with the Rasch model while considering individual background characteristics (von Davier, Gonzalez, & Mislevy, 2009). In all regression models, gender, migration background, and socioeconomic status were included as control variables (i.e., all latent variables and achievement were regressed on these covariates). Standard errors of the parameters were corrected for the nested structure of the data. All analyses were computed with Mplus 7.11 (Muthén & Muthén, 1998–2014).
1 It should be noted that this testing procedure, outlined in the Mplus User's Guide 7.1 (Muthén & Muthén, 1998-2014) and described in more detail in Schroeders and Wilhelm (2011), diverges from the approach provided by Millsap & Yun-Tein (2004) where for the baseline model partial threshold constraints are introduced.
34
U. Schroeders et al. / Learning and Individual Differences 43 (2015) 31–38
Table 2 Factor Loading Matrix and Items from the TIE Questionnaire. Reading
Contemplation
Avoid Thinking
Intell. Curiosity
I
II
III
IV
TIE 07 TIE 01 TIE 16 TIE 04 TIE 10 TIE 09 TIE 18* TIE 02 TIE 05 TIE 17 TIE 14 TIE 03* TIE 12* TIE 11* TIE 08* TIE 15 TIE 13 TIE 06
.95 .88 .87 .73 .71 −.04 −.16 .14 .06 .15 .09 .14 −.06 .03 −.03 .01 .26 −.07
−.03 .03 .01 .03 .03 .69 .67 .53 .51 .50 .37 .25 −.05 −.16 .08 .03 −.04 .29
.01 .00 −.01 .02 −.04 −.07 −.12 .06 .18 −.15 .13 −.03 .51 .49 .45 −.08 −.01 .06
Correlations II III IV
.34 −.01 .29
−.21 .39
−.13
Item
Item −.07 .01 .02 .05 .03 .04 .01 .19 −.07 −.05 .13 .21 .00 .02 −.11 .79 .44 .42
I'm reading a lot, at least one book each month. I read books on various topics. I like to read different kinds of books, including novels and nonfiction books. I like going to the library. Even in elementary school, I read all sorts of things. I like to get to the bottom of things. I would like to understand exactly how everything works. I think a lot about general questions, even if I do not always find an answer. I can delve into a problem deeply so that I forget the world around me. I like to think about challenging puzzles. I enjoy thinking abstractly. There are very few subjects that bore me. I think it is often troublesome to learn entirely new things. I avoid complex problems. In most cases, more thinking will lead only to more errors. I have been following social and political issues on television or the Internet. I regularly read newspapers and magazines. I like watching documentaries on TV.
Note. Factor loadings above .30 are printed in bold type. nEFA = 3,604; Model fit: χ2 = 787.7, df = 87, RMSEA = .047, CFI = .985, SRMR = .023. * Item was excluded for the latent regression models.
As a result of the estimation of plausible values, achievement data were available for all students who worked on a specific math or science booklet. There was no significant amount of missing data for gender, fluid intelligence, interest, or TIE (on the item level between 0 and 3.9%). Missingness was noteworthy for migration background (18.9%) and HISEI (19.5%). To model missing data on the manifest covariates, they were integrated into the model as single indicators of latent constructs by estimating their (co)variances. The variance-covariance matrix made use of all pairwise present information (Muthén & Muthén, 1998–2014). To ensure that the results were not biased by the handling of missing data, the models were double-checked with multiple imputations.
4. Results Table 1 gives the correlation matrix for all variables included in subsequent analyses both on the manifest (above the diagonal) and the latent level.2 Since hierarchical latent regression models are conducted, the factor saturation, McDonald's ω (1999), is given as a reliability measure of the scale on the diagonal.
4.1. Psychometric Evaluation of the TIE Measure In a first step, we conducted an EFA of the TIE measure. Parallel analysis and the scree plot suggested a four-factor solution. The factor loadings for the four-factor solution are given in Table 2. The first factor was comprised of items describing the desire and enjoyment of reading texts with different topics. Activities under the second factor can be summed up as contemplation or abstract reasoning (e.g., “I like to think about challenging puzzles”) which is neither domain-specific nor bound to a particular purpose. The third factor depicted avoidance of cognitive effort (“I think it is often troublesome to learn entirely new things”). The fourth factor can be labeled intellectual curiosity and 2 Due to the multiple-matrix design, the correlation matrix for the math part and the natural science part (i.e., biology, chemistry, and physics) rely on different but overlapping subsamples. The respective correlations of the subsample that worked on the science tests can be found in the online supplement.
described information seeking which is not restricted to a specific news medium (e.g., newspapers). Although the results of the exploratory factor analysis indicated a four factor solution, we favored a three-factor solution, dropping the factor avoidance of cognitive effort. This was done for three reasons: First, this dimension was only weakly correlated to the other dimensions (e.g., ρ(reading, avoidance) = −.10; for the complete four factor model, please see Fig. 1 in the online supplement). Second, the avoidance factor was constituted by the only three items of the questionnaire that were, in a logical sense, negatively formulated (i.e., higher agreement equals lower values on this factor). There is substantive evidence in the literature showing that negatively and positively worded items function differently (e.g., higher difficulty and lower discrimination parameters for negatively worded items), even leading to the notion that such items “impair the functioning of scales” in general (see also DiStefano & Motl, 2006; Sliter & Zickar, 2014, p. 223). In order to account for the additional variance caused by the negative item formulation, we also specified a correlated-trait-correlateduniqueness (CTCU) and a nested factor model. However, these models were rejected because the residual correlations in the CTCU model and the loadings on the nested factor, respectively, proved to be very low and unsystematic.3 Third, the avoidance scale was not part of the original questionnaire (Wilhelm et al., 2003). Taken together, this suggests that the factor termed avoidance of cognitive effort is not part of the TIE construct but rather a method artifact. We replicated the three-dimensional structure of the TIE questionnaire with a tau-congeneric measurement model (i.e., varying factor loadings) and no cross-loadings with the second half of the sample (compare Fig. 1). This model yielded satisfactory fit to the data (χ2 = 1,775.1, df = 74, CFI = .958, RMSEA = .080). Removing the last item, however, improved the model fit considerably (χ2 = 1,212.1, df = 62, CFI = .971, RMSEA = .072). In total, five items were excluded from the revised questionnaire: The three negatively worded items loading on a fourth factor (see above), one item that only had small loadings 3 Comparable models were also specified in the study of DiSteffano and Motl (2006). The authors demonstrated that the method factors were present across different questionnaires and suggested that effects associated with negatively worded items may be more adequately interpreted as response style rather than content-related variance.
U. Schroeders et al. / Learning and Individual Differences 43 (2015) 31–38
35
Fig. 1. Predicting achievement in physics with TIE. Nonsignificant parameters were omitted. Model fit statistics represent the mean of 15 replications: n = 4,011, χ2 = 2,742.7, df = 110, CFI = .936, RMSEA = .077.
on all factors of the EFA (TIE-03) and the last item due to model misfit. Correlations between the latent variables were as follows—ρ(reading, contemplation) = .45, ρ(reading, curiosity) = .42, ρ(contemplation, curiosity) = .57—and similar to those reported previously (Wilhelm et al., 2003). Note that the three factors reading, contemplation, and intellectual curiosity could equivalently be modeled as first-order factors below an overarching TIE factor. 4.2. Measurement Invariance Across School Tracks and Gender In Table 3 the results of the measurement invariance testing are summarized.4 In the first step (configural invariance, T1/G1), factor loadings and thresholds were freely estimated, whereas residual variances were set to 1 in all groups. In the second step (T2/G2), that is strong invariance, factor loadings and thresholds were set to be equal across groups. In the last step (T3/G3), in addition to the constraints of step 2, the residual variances were set to 1 in both groups. Differences in model fit were calculated between two consecutive models. A difference of ΔCFI N .01 is considered a significant deterioration in model fit (Cheung & Rensvold, 2002). For both invariance analyses, even imposing the most restrictive constraints did not lead to significant deterioration in model fit. Strict measurement invariance allows for comparisons of means on the latent (and manifest) level. In line with theoretical considerations, the means of all three TIE facets were higher in the academic-track schools 4 Given that three-dimensional model and the higher-order model are equivalent, the complete measurement invariance testing of both models yielded the same results. Furthermore, a higher-order factor model, in which beside the first-order factor loadings and the thresholds, also the second-order factor loadings were set to be equal across groups, did not lead to a deterioration in model fit (χ2 = 1210.2, df = 160, CFI = .971, RMSEA = .060), indicating that also this more restrictive form of measurement invariance was given (see also Chen, Sousa, & West, 2005). This model is equivalent to the strong measurement invariance testing of the three-dimensional model with additionally constrained factor covariances.
(standardized estimated mean differences Δα; reading: 1.14, contemplation: 0.39, intellectual curiosity: 0.30). Furthermore, the differentiation of the factor structure was more pronounced as indicated by the factor correlations5 (academic/nonacademic track: ρ(reading, contemplation) = .40/.52, ρ(reading, curiosity) = .27/.44, ρ(contemplation, curiosity) = .46/.59). In accordance with the ability-differentiation hypothesis (e.g., Tucker-Drob, 2009), it seemed plausible to assume a differentiation of TIE—without TIE being an ability measure itself—in more capable groups, since there is mutual dependency between typical intellectual performance (as expressed in TIE) and maximal intellectual performance. With respect to gender, girls outperformed boys in reading (Δα = −1.38) and contemplation (Δα= −0.12), whereas boys were more curious (Δ = 0.30). 4.3. Hierarchical Latent Regression Models For each subject, four consecutive models were specified (see Table 4). In the first model, achievement was regressed on background variables only (i.e., gender, migration background, and socioeconomic status). In the second model, TIE was added as a higher order factor capturing the core of the different TIE facets. This model was preferred over the three-dimensional model because the first-order factors were based on few items which limits the construct representation on the level of subdimensions and is in general associated with a lower reliability of the scale. Furthermore, the correlation between contemplation and curiosity was substantial (ρ=.57), resulting in distorted estimations of the regression weights within the three-dimensional model (see online supplement). In the third model, fluid intelligence and subject-specific interest were included as latent predictors besides the background variables. In the fourth model, model 3 was extended with TIE. The column 5 Variances were comparable across groups (academic/non-academic track: σ (reading) = 3.60/4.00 (SE .26/.26); σ 2(contemplation) = 1.41/1.46 (SE .14/.14); σ 2 (intellectual curiosity) = 0.46/0.56) (SE .06/.06). 2
36
U. Schroeders et al. / Learning and Individual Differences 43 (2015) 31–38
Table 3 Measurement invariance analyses of the TIE questionnaire across school tracks and gender.
School track (T1) Configural invariance (T2) Strong invariance (T3) Strict invariance Gender (G1) Configural invariance (G2) Strong invariance (G3) Strict invariance
χ2/df
p
CFI
RMSEA
1250.1/124
b.01
.968
.071
1310.6/157 1284.4/170
b.01 b.01
.968 .969
.064 .060
1189.9/124
b.01
.970
.069
1557.9/157 1536.8/170
b.01 b.01
.961 .962
.070 .067
Δ CFI
more pronounced than for the natural sciences. Furthermore, interest in mathematics outperformed TIE as a predictor of mathematics achievement.
Δ RMSEA
5. Discussion .000 −.001
−.007 −.004
.009 −.001
.001 -.003
Note. CFI = Comparative Fit Index. RMSEA = Root Mean Square Error of Approximation. Differences in model fit refer to consecutive models (e.g., T3–T2). Theta parameterization was used for all analyses (B. O. Muthén & Asparouhov, 2002). When using the WLSMV estimator, df could not be computed directly, instead they “are calculated from the estimated covariance matrix and a diagonal weight matrix” (Beauducel & Herzberg, 2006, p. 192). For this reason, the χ2 statistics between two (consecutive) models cannot be compared directly.
R2 in Table 4 denotes the proportion of variance explained by all predictors in the model; the column ΔR2 gives the incremental variance explained by TIE over and above all other predictors included in the preceding model (e.g., P4–P3 and P2–P1). The pattern of results was similar for all science subjects. For example, 16% of the interindividual differences in physics were explained by the background variables. The value increased to 24.9% after adding TIE to the model (see Fig. 1 and model P2 in Table 4). In other words, TIE explained about 9% of the interindividual differences in physics achievement beyond the background variables. In the last model (P4, Table 4), fluid intelligence and interest were entered as additional predictors. As expected, fluid intelligence was the strongest predictor, explaining about 28% of the interindividual differences in physics achievement, whereas the contribution of subject-specific interest was close to 0 and not statistically significant. TIE exerted only a small influence over and above intelligence and interest. Specifically, individual differences in the second-order TIE factor accounted for 1.3% of the variance in physics achievement after controlling for individual differences in background variables, fluid intelligence, and interest. Results for mathematics were similar, but the influence of fluid intelligence was even
The influence of the personality trait TIE on academic achievement has been repeatedly emphasized in the literature. However, the reported effects may be biased or overestimated because the studies did not account for differences in relevant background variables (e.g., SES), mostly relied on regression analyses with manifest indicators, and were based on small selective samples (e.g., university students). Therefore, we adapted and revised an existing short scale measuring the essential core areas of TIE (Wilhelm et al., 2003), administered it to a heterogeneous sample of students in Grade 9, and modeled its relations to standardized achievement tests with a latent variable approach. In the following, we (a) discuss the meaning and impact of the three factors of the TIE questionnaire in more detail, (b) relate the results of the hierarchical latent regression to the existing research literature, and (c) point out possible study objectives for future research. Considering that reading is one of the most powerful sources of learning, reading books, articles, and newspapers is evidently a crucial concomitant of intellectual engagement. Because TIE reading is not limited with respect to content, it comes close to the “hungry mind” that is open to new ideas (von Stumm et al., 2011, p. 583). TIE has also been described as the “desire to engage and understand [one's] world [...], a need to know” (Goff & Ackerman, 1992, p. 539), which seems best expressed by contemplation. This is also reflected by the fact that this factor had the highest loading on the second-order factor. Furthermore, the theoretical and empirical overlap between TIE contemplation and fluid intelligence was most pronounced (about ρ=.30), advocating the close interplay between typical intellectual behavior and psychometric intelligence. The factor intellectual curiosity, which has been labeled the “third pillar of academic performance” besides intelligence and effort (von Stumm et al., 2011, p. 574), was measured with only three items, thus narrowing the breadth of the dimension. The questions focused on topics that receive attention in mass media (i.e., politics and current event knowledge). If the items had been more strongly related to scientific inquiry, relations with mathematics and science achievement may have been stronger.
Table 4 Standardized Regression Coefficients of Hierarchical Prediction Models. Gender
Math
Biology
Chemistry
Physics
Migration
SES
TIE
b
(SE)
b
(SE)
b
(SE)
M1 M2 M3 M4 B1 B2 B3 B4
.12 .20 .07 .09 −.06 .02 −.07 −.04
(.02) (.02) (.01) (.01) (.02) (.02) (.02) (.02)
−.17 −.17 −.09 −.09 −.20 −.22 −.13 −.15
(.02) (.02) (.02) (.01) (.02) (.02) (.02) (.02)
.35 .27 .19 .18 .32 .24 .18 .15
(.02) (.02) (.02) (.02) (.02) (.02) (.02) (.02)
C1 C2 C3 C4 P1 P2 P3 P4
.03 .12 −.01 .03 .03 .12 −.02 .03
(.02) (.02) (.02) (.02) (.02) (.02) (.02) (.02)
−.21 −.23 −.14 −.15 −.20 −.22 −.12 −.14
(.02) (.02) (.02) (.01) (.02) (.02) (.02) (.02)
.31 .22 .16 .13 .32 .24 .17 .15
(.02) (.02) (.02) (.02) (.02) (.02) (.02) (.02)
Gf
b
(SE)
.28
(.02)
.07
(.02)
.31
(.02)
.14
(.03)
.34
(.02)
.15
(.02)
.33
(.02)
.14
(.02)
b
R2
Interest (SE)
b
ΔR2
(SE)
.65 .63
(.02) (.01)
.13 .12
(.02) (.02)
.54 .50
(.02) (.02)
.06 .00
(.02) (.02)
.56 .52
(.02) (.02)
.09 .04
(.02) (.02)
.56 .53
(.02) (.02)
.08 .03
(.02) (.02)
Model fit CFI
18.0 24.3 61.8 62.3 16.2 24.1 43.0 44.5 15.6 25.4 45.9 47.7 16.0 24.9 46.3 47.6
6.3 .5 7.9 1.5
9.8 1.8 8.9 1.3
RMSEA
– .94 1.00 .96 – .94 1.00 .97
– .08 .02 .04 – .08 .02 .03
– .94 1.00 .97 – .94 1.00 .97
– .08 .02 .03 – .08 .02 .04
Note. nmath = 4,032; nscience = 4,011. Regression weights (b) in bold type were significant (p b .05). ΔR2 is the incremental variance explained by TIE over and above all other predictors in the model. SES = socioeconomic status, TIE = typical intellectual engagement, Gf = (figural) fluid intelligence. TIE, gf, and subject-specific interest were included in the hierarchical regression models as latent variables. CFI = Comparative Fit index. RMSEA = Root Mean Square of Approximation.
U. Schroeders et al. / Learning and Individual Differences 43 (2015) 31–38
When interpreting the impact of TIE on school performance it has to be taken into account that the effect was determined after controlling for influential background variables and intelligence, respectively, which is a conservative approach. Consistent with previous research (e.g., see Deary, Strand, Smith, & Fernandes, 2007; Neisser et al., 1996), intelligence already explained a large amount of variance in achievement when investigated on the latent level. Nevertheless, even a powerful predictor such as fluid intelligence still leaves about 50% of the variance in academic achievement unexplained. Our results show that the contribution of TIE in closing this gap is almost negligible. Importantly, the contribution of TIE should be interpreted in comparison with the explanatory power of other noncognitive variables such as subject-specific interest. For all science subjects, domain-general TIE outperformed subject-specific interest as a predictor of academic achievement. However, in contrast to previous studies (Marsh et al., 2005), we found a significant influence of interest that exceeded the influence of TIE for math achievement. This is first evidence that there are differential patterns depending on the subject in question. It has also been argued that TIE exhibits “stronger relations to humanities-type knowledge than to the sciences” (Rolfhus & Ackerman, 1999, p. 513). Therefore, the predictive power of TIE might be greater for other subjects such as art or literature. Since Mussel (2010, 2013) established a high conceptual overlap between TIE and related constructs such as need for cognition and the facet openness to ideas of the Big Five and also provided empirical evidence (all belong within the operation Think and the process Seek within his two-dimensional theory of intellect), the present findings can give indication that the power of such constructs in predicting math and science achievement might also be marginal. The findings for the student sample at hand were in line with previous studies on university students (e.g., Chamorro-Premuzic et al., 2006a, 2006b; Wilhelm et al., 2003). For example, Powell and Nettelbeck (2014) reported that TIE explained only 1.8% of the individual differences in academic achievement over and above fluid intelligence at the time of college admission. We found no indication that the influence of TIE was more pronounced in a non-selective, abilityheterogeneous sample. Comparing the results of both studies, the increment of TIE beyond fluid intelligence seems to be stable at different stages of the educational biography. As a limitation of the present study, we have to take into account that the original TIE questionnaire (Wilhelm et al., 2003) had to be substantially revised in order to suit the target population of ninth-graders and that additional evidence concerning the criterion-related and construct validity of the measure is pending. However, given the psychometric soundness of the revised measure, one essential question is why—in spite of the sound theoretical assumptions—the influence of intellectual investment traits such as TIE on academic achievement is marginal when the influence of the predominant fluid intelligence is taken into account. On the one hand, the learning environment in secondary education is—in comparison to adulthood—relatively homogeneous, which may entail that interindividual differences in academic performance can be largely attributed to differences in fluid intelligence, thus preventing TIE from exerting its potential influence. On the other hand, the results found here may be specific for the mathematicalscientific domain. Higher correlations would be expected for subjects that rely more strongly on crystallized intelligence (e.g., history or literature). Therefore, future research should consider a broader range of educational achievements (e.g., humanities and social sciences), and samples of adults with heterogeneous learning opportunities in order to investigate the interplay of TIE with cognitive and motivational factors in more detail. Appendix A. Supplementary data Supplementary data to this article can be found online at http://dx. doi.org/10.1016/j.lindif.2015.08.030.
37
References Ackerman, P. L. (1996). A theory of adult intellectual development: Process, personality, interests, and knowledge. Intelligence, 22, 227–257. http://dx.doi.org/10.1016/ S0160-2896(96)90016–1. Ackerman, P. L., & Beier, M. E. (2004). Knowledge and intelligence. In O. Wilhelm, & R. W. Engle (Eds.), Handbook of understanding and measuring intelligence (pp. 125–139). Thousand Oaks, CA: Sage Publications. Ackerman, P. L., & Goff, M. (1994). Typical intellectual engagement and personality: Reply to Rocklin (1994). Journal of Educational Psychology, 86, 150–153. http://dx.doi.org/ 10.1037/0022-0663.86.1.150. Ackerman, P. L., & Heggestad, E. D. (1997). Intelligence, personality, and interests: evidence for overlapping traits. Psychological Bulletin, 121, 219–245. http://dx.doi. org/10.1177/1069072703011002006. Ackerman, P. L., & Rolfhus, E. L. (1999). The locus of adult intelligence: Knowledge, abilities, and nonability traits. Psychology and Aging, 14, 314–330. http://dx.doi.org/ 10.1037/0882-7974.14.2.314. Beauducel, A., & Herzberg, P. Y. (2006). On the performance of maximum likelihood versus means and variance adjusted weighted least squares estimation in CFA. Structural Equation Modeling: A Multidisciplinary Journal, 13, 186–203. Binet, A., & Simon, T. (1904). Méthodes nouvelles pour le diagnostic du niveau intellectuel des anormaux [New methods for the diagnosis of the intellectual level of subnormals]. L'Année Psychologique, 11, 191–244. http://dx.doi.org/10.3406/psy.1904.3675. Brunner, M., Gogol, K. M., Sonnleitner, P., Keller, U., Krauss, S., & Preckel, F. (2013). Gender differences in the mean level, variability, and profile shape of student achievement: Results from 41 countries. Intelligence, 41, 378–395. http://dx.doi.org/10.1016/j. intell.2013.05.009. Brunner, M., Nagy, G., & Wilhelm, O. (2012). A tutorial on hierarchically structured constructs. Journal of Personality, 80, 796–846. http://dx.doi.org/10.1111/j.14676494.2011.00749.x. Ceci, S. J. (1991). How much does schooling influence general intelligence and its cognitive components? A reassessment of the evidence. Developmental Psychology, 27, 703–722. Chamorro-Premuzic, T., Furnham, A., & Ackerman, P. L. (2006a). Ability and personality correlates of general knowledge. Personality and Individual Differences, 41, 419–429. http://dx.doi.org/10.1016/j.paid.2005.11.036. Chamorro-Premuzic, T., Furnham, A., & Ackerman, P. L. (2006b). Incremental validity of the typical intellectual engagement scale as predictor of different academic performance measures. Journal of Personality Assessment, 87, 261–268. http://dx.doi.org/ 10.1207/s15327752jpa8703_07. Chen, F. F., Sousa, K. H., & West, S. G. (2005). Teacher's corner: testing measurement invariance of second-order factor models. Structural Equation Modeling, 12, 471–492. Cheung, G. W., & Rensvold, R. B. (2002). Evaluating goodness-of-fit indexes for testing measurement invariance. Structural Equation Modeling: A Multidisciplinary Journal, 9, 233–255. Cronbach, L. J. (1949). Essentials of psychological testing. New York, NY: Harper. Deary, I. J., Strand, S., Smith, P., & Fernandes, C. (2007). Intelligence and educational achievement. Intelligence, 35, 13–21. http://dx.doi.org/10.1016/j.intell.2006.02.001. DiStefano, C., & Motl, R. W. (2006). Further investigating method effects associated with negatively worded items on self-report surveys. Structural Equation Modeling: A Multidisciplinary Journal, 13, 440–464. http://dx.doi.org/10.1207/ s15328007sem1303_6. Furnham, A., Monsen, J., & Ahmetoglu, G. (2009). Typical intellectual engagement, Big Five personality traits, approaches to learning and cognitive ability predictors of academic performance. British Journal of Educational Psychology, 79, 769–782. http://dx.doi.org/10.1348/978185409X412147. Ganzeboom, H. B. G., De Graaf, P. M., & Treiman, D. J. (1992). A standard international socio-economic index of occupational status. Social Science Research, 21, 1–56. http://dx.doi.org/10.1016/0049-089X(92)90017-B. Goff, M., & Ackerman, P. L. (1992). Personality-intelligence relations: Assessment of typical intellectual engagement. Journal of Educational Psychology, 84, 537–552. http:// dx.doi.org/10.1037/0022-0663.84.4.537. Gonzalez, E., & Rutkowski, L. (2010). Principles of multiple matrix booklet designs and parameter recovery in large-scale assessments. In D. Hastedt, & D. von Davier (Eds.), IERI Monograph Series: Issues and Methodologies in Large-Scale Assessments, Vol. 3. (pp. 125–156). Hamburg: IEA-ETS Research Institute. Hambrick, D. Z. (2004). The role of domain knowledge in higher-level cognition. In O. Wilhelm, & R. W. Engle (Eds.), Handbook of understanding and measuring intelligence (pp. 361–372). Thousand Oaks, CA: Sage Publications. Hu, L., & Bentler, P. M. (1999). Cutoff criteria for fit indexes in covariance structure analysis: Conventional criteria versus new alternatives. Structural Equation Modeling: A Multidisciplinary Journal, 6, 1–55. http://dx.doi.org/10.1080/10705519909540118. Kuncel, N. R., Hezlett, S. A., & Ones, D. S. (2004). Academic performance, career potential, creativity, and job performance: can one construct predict them all? Journal of Personality and Social Psychology, 86, 148–161. http://dx.doi.org/10.1037/00223514.86.1.148. Marsh, H. W., Trautwein, U., Lüdtke, O., Köller, O., & Baumert, J. (2005). Academic selfconcept, interest, grades, and standardized test scores: reciprocal effects models of causal ordering. Child Development, 76, 397–416. http://dx.doi.org/10.1111/j.14678624.2005.00853.x. McDonald, R. P. (1999). Test theory: A unified treatment. Mahwah, NJ: Erlbaum. Millsap, R. E., & Yun-Tein, J. (2004). Assessing factorial invariance in ordered-categorical measures. Multivariate Behavioral Research, 39, 479–515. Mussel, P. (2010). Epistemic curiosity and related constructs: Lacking evidence of discriminant validity. Personality and Individual Differences, 49, 506–510. http://dx.doi. org/10.1016/j.paid.2010.05.014.
38
U. Schroeders et al. / Learning and Individual Differences 43 (2015) 31–38
Mussel, P. (2013). Intellect: A theoretical framework for personality traits related to intellectual achievements. Journal of Personality and Social Psychology, 104, 885–906. http://dx.doi.org/10.1037/a0031918. Muthén, B. O., & Asparouhov, T. (2002). Latent variable analysis with categorical outcomes: multiple-group and growth modeling in mplus. Mplus Web Notes, 4, 1–22. Muthén, L. K., & Muthén, B. O. (1998-2014). Mplus user's guide (7th ed.). Los Angeles, CA: Muthén & Muthén. Neisser, U., Boodoo, G., Bouchard, T. J., Jr., Boykin, A. W., Brody, N., Ceci, S. J., ... Urbina, S. (1996). Intelligence: Knowns and unknowns. American Psychologist, 51, 77–101. http://dx.doi.org/10.1037/0003-066X.51.2.77. Pant, H. A., Stanat, P., Schroeders, U., Roppelt, A., Siegle, T., & Pöhlmann, C. (2013). The IQB National Assessment Study 2012: Competencies in mathematics and the sciences at the end of secondary level I. Münster: Waxmann. Powell, C., & Nettelbeck, T. (2014). Intellectual curiosity may not incrementally predict academic success. Personality and Individual Differences, 64, 7–11. http://dx.doi.org/ 10.1016/j.paid.2014.01.045. Rolfhus, E. L., & Ackerman, P. L. (1999). Assessing individual differences in knowledge: Knowledge, intelligence, and related traits. Journal of Educational Psychology, 91, 511–526. http://dx.doi.org/10.1037/0022-0663.91.3.511. Schroeders, U., & Wilhelm, O. (2011). Equivalence of reading and listening comprehension across test media. Educational and Psychological Measurement, 71, 849–869. http://dx.doi.org/10.1177/0013164410391468.
Sliter, K. A., & Zickar, M. J. (2014). An IRT examination of the psychometric functioning of negatively worded personality items. Educational and Psychological Measurement, 74, 214–226. http://dx.doi.org/10.1177/0013164413504584. Tucker-Drob, E. M. (2009). Differentiation of cognitive abilities across the life span. Developmental Psychology, 45, 1097–1118. http://dx.doi.org/10.1037/a0015864. von Davier, M., Gonzalez, E., & Mislevy, R. J. (2009). What are plausible values and why are they useful. IERI Monograph Series, 4. (pp. 9–36). von Stumm, S., & Ackerman, P. L. (2012). Investment and intellect: A review and metaanalysis. Psychological Bulletin, 139, 841–869. http://dx.doi.org/10.1037/a0030746. von Stumm, S., Hell, B., & Chamorro-Premuzic, T. (2011). The hungry mind: Intellectual curiosity is the third pillar of academic performance. Perspectives on Psychological Science, 6, 574–588. http://dx.doi.org/10.1177/1745691611421204. Wilhelm, O. (2004). Measuring reasoning ability. In O. Wilhelm, & R. W. Engle (Eds.), Handbook of understanding and measuring intelligence (pp. 373–392). Thousand Oaks, CA: Sage Publications. Wilhelm, O., Schroeders, U., & Schipolowski, S. (2014). Berliner Test zur Erfassung fluider und kristalliner Intelligenz für die 8. bis 10. Jahrgangsstufe [Berlin Test of fluid and crystallized intelligence for grades 8–10]. Göttingen: Hogrefe. Wilhelm, O., Schulze, R., Schmiedek, F., & Süß, H. -M. (2003). Interindividuelle Unterschiede im typischen intellektuellen Engagement [Individual differences in typical intellectual engagement]. Diagnostica, 49, 49–60. http://dx.doi.org/10.1026// 0012-1924.49.2.49.