Incremental theories of intelligence predict multiple document comprehension

Learning and Individual Differences 31 (2014) 11–20

Contents lists available at ScienceDirect

Learning and Individual Differences journal homepage: www.elsevier.com/locate/lindif

Incremental theories of intelligence predict multiple document comprehension Jason L.G. Braasch a,b, Ivar Bråten a,⁎, Helge I. Strømsø a, Øistein Anmarkrud a a b

University of Oslo, Norway University of Memphis, USA

a r t i c l e

i n f o

Article history: Received 12 April 2013 Received in revised form 6 November 2013 Accepted 25 December 2013 Keywords: Implicit theories of intelligence Multiple document comprehension Source evaluation Intertextual inferences

a b s t r a c t This study examined implicit theories of intelligence as predictors of multiple document comprehension in a sample of 59 Norwegian upper-secondary school students. In four multiple regression analyses with multiple document comprehension indicated by students' inclusion of scientific concepts in their essays, discrimination between more and less useful documents given the reading task, consideration of document trustworthiness as a basis for making those discriminations, and ability to draw inferences across documents as outcome measures, beliefs in intelligence as a malleable, increasable quality emerged as a unique positive predictor after controlling for word recognition, prior knowledge, and working memory. However, beliefs in intelligence as a fixed, unchangeable quantity did not emerge as a unique negative predictor. The findings indicate that the benefits of endorsing an incremental theory of intelligence may be more pronounced than the costs of endorsing an entity theory of intelligence in complex reading task contexts. © 2013 Elsevier Inc. All rights reserved.

1. Introduction Many reading researchers contend that the 21st century information age has been a kind of “game changer” for comprehension, bringing about not only new affordances but also new concerns (Alexander, 2012; Brand-Gruwel & Stadtler, 2011; Britt & Gabrys, 2002; Leu, Kinzer, Coiro, Castek, & Henry, 2013). On one hand, readers can rapidly, almost instantaneously, access a wide range of up-to-date information, particularly when retrieving documents via Internet search engines. On the other hand, such access requires additional competencies, especially in considering the ease of publishing in the current information age. In this sense, readers should be more vigilant about the nature of documents as socially constructed artifacts (written by a particular author, in support of a particular agenda, for a particular publication venue, at a particular point in time, and so forth) (Britt, Rouet, & Braasch, 2013). In addition, comprehension often requires that readers integrate content information germane to their research question that is distributed across multiple documents (Afflerbach & Cho, 2009; Goldman, Braasch, Wiley, Graesser, & Brodowinska, 2012). Taken together, readers must restrict their processing towards reliable, higher quality information to successfully comprehend. It seems reasonable to assume that, if people do not employ critical reading strategies to differentiate the quality and reliability of the information they read, they may experience an overload of information, or even inappropriately integrate both accurate and inaccurate information (Graesser et al., 2007; ⁎ Corresponding author at: Department of Educational Research, University of Oslo, P.O. Box 1092, Blindern, N-0317 Oslo, Norway. Tel.: +47 22 85 52 82; fax: +47 22 85 42 50. E-mail address: [email protected] (I. Bråten). 1041-6080/$ – see front matter © 2013 Elsevier Inc. All rights reserved. http://dx.doi.org/10.1016/j.lindif.2013.12.012

Stadtler & Bromme, 2008). Moreover, if readers do not strategically infer relationships between concepts found within various documents, their resultant understandings may involve a less coherent mixture of unconnected facts and concepts (Goldman, 2004). Research to date has highlighted that a number of cognitive variables may be associated with evaluation and integration processes (Banas & Sanchez, 2012; Bråten, Anmarkrud, Brandmo, & Strømsø, 2014; Bråten, Ferguson, Anmarkrud, & Strømsø, 2013; Bråten, Strømsø, & Salmerón, 2011). However, research on psychosocial factors is surprisingly scant. In the current article, we posited that readers' implicit beliefs about the nature of intelligence – whether they believe that intelligence is a fixed and stable characteristic versus malleable and within their control to flexibly develop – might predict the extent to which they will employ strategies to evaluate source features and integrate content information in a multiple document context independent of several cognitive individual difference factors. Classic and recent empirical research findings support that source evaluation and content integration are both crucial components of multiple document comprehension (Bråten, Strømsø, & Britt, 2009; Britt, Perfetti, Sandak, & Rouet, 1999; Goldman et al., 2012; Rouet, Britt, Mason, & Perfetti, 1996; Wiley et al., 2009; Wineburg, 1991). Bråten et al. (2009) demonstrated that post-reading trust evaluations on reliable documents and considerations of source features in making these trust decisions independently predicted multiple document comprehension, after controlling for readers' prior topic knowledge. Recent studies using verbal protocols also demonstrate that strategies focused on differentiating more versus less useful documents during reading and using trustworthiness criteria when doing so relate to better postreading multiple document comprehension. For example, Anmarkrud,

12

J.L.G. Braasch et al. / Learning and Individual Differences 31 (2014) 11–20

Bråten, and Strømsø (in press) demonstrated relationships between evaluations of information sources produced during reading and argumentation sophistication and source usage in post-reading essays. Similarly, Goldman et al. (2012) contrasted the kinds of processing that better and poorer learners displayed during reading. Findings suggested that better learners were more likely to go beyond content analyses to evaluate the trustworthiness of the sources of documents compared with poorer learners. Instead, poorer learners spent more time reading unreliable documents and were more likely to include erroneous concepts in post-reading essays. Thus, there is a growing body of evidence that source evaluation strategies support the construction of accurate understandings from multiple documents. Integration of higher quality content information both within single documents and across multiple documents also appears to promote successful comprehension. In Bråten and Strømsø (2011), college student readers' identification with cross-text elaboration strategies on a post-reading survey positively predicted their intertextual comprehension, a measure of the inferred connections made across the documents. Wolfe and Goldman (2005) showed strong relationships between adolescents' self-explanation strategies during reading – the inferred connections they made both within and across multiple history texts – and their subsequent reasoning performance about the texts' topic. Using longer, more diverse documents, Goldman et al. (2012) demonstrated that more successful college student learners constructed accurate and coherent representations because they employed self-explanation strategies in response to information germane to understanding the scientific topic. Poorer learners displayed less evidence of self-explanation strategies overall, and appeared to give equal weight to more and less reliable information when they did. Thus, inferring relationships among concepts also appears to support learning from multiple documents. 1.1. Cognitive individual differences that predict multiple document comprehension Empirical research demonstrates that a number of cognitive variables contribute to the facets of multiple document comprehension outlined above. Research has investigated three cognitive variables in particular – word recognition, prior knowledge, and working memory – either as variables of interest or as control variables. For example, many empirical studies of multiple document comprehension control for prior knowledge when judging whether additional variables explain unique variance in comprehension performance. Even with additional variables included in a model, prior topic knowledge often remains a significant and substantial predictor of multiple document comprehension (Bråten et al., 2009, 2014; Strømsø, Bråten, & Samuelstuen, 2008). Moreover, Bråten et al. (2013) investigated the contribution of individual differences in word recognition skill for multiple document comprehension. As has been shown in prior research focusing on the comprehension of single texts (Andreassen & Bråten, 2010; Cunningham, Stanovich, & Wilson, 1990; Samuelstuen & Bråten, 2005; Stanovich, Cunningham, & Feeman, 1984), word recognition was a unique predictor of multiple document comprehension among lower-secondary school students. Finally, Banas and Sanchez (2012) demonstrated that individual differences in working memory capacity uniquely influenced learning of implicit relationships underlying textual materials distributed across multiple web pages. Obviously, not all individual reader characteristics of importance are cognitive in nature. Other research has demonstrated that psychosocial factors, such as personality traits or dispositions, can strongly influence individuals' propensities to expend efforts towards achievement and performance in a number of contexts and domains. In the current work, we used the cognitive individual difference factors mentioned above as control variables to rule out plausible alternative explanations that they produced the effects, as opposed to the personality predictor of focal interest in this article: readers' implicit theories of intelligence.

1.2. Implicit theories of intelligence predict performance Seminal work by Dweck and colleagues (Blackwell, Trzesniewski, & Dweck, 2007; Dweck, 1999; Dweck & Master, 2008; Hong, Chiu, Dweck, Lin, & Wan, 1999) has related learners' implicit beliefs about the nature of intelligence to their approaches and ultimate success in intellectually demanding tasks (e.g., acquiring new knowledge and skills). Some learners hold beliefs that intelligence is a fixed, relatively stable characteristic or quantity that is out of their control. Learners who identify with these beliefs are often described as holding “entity” theories of intelligence. Others believe that intelligence is malleable, a quality that is within their control to flexibly develop and change. Accordingly, learners who identify with these beliefs are often described as holding “incremental” theories of intelligence. A large body of research has established relationships between students' implicit theories of intelligence and their achievement in various learning contexts (Blackwell et al., 2007; Dupeyrat & Mariné, 2005; Faria & Fontaine, 1997; Greene, Costa, Robertson, Pan, & Deekens, 2010; Henderson & Dweck, 1990; Stipek & Gralinski, 1996). For example, Henderson and Dweck (1990), after controlling for prior achievement, found that adolescents endorsing more of an incremental theory of intelligence earned significantly higher grades in the first year of junior high school than did those endorsing more of an entity theory. Experimental research supports the relationships established through correlational work. That is, interventions focusing on the acquisition of incremental theories significantly increased adolescents' achievement test scores (Good, Aronson, & Inzlicht, 2003) and college students' grades (Aronson, Fried, & Good, 2002) relative to controls. Several empirical reports indicate that theories of intelligence orient differently towards strategic processes during learning, which contributes to ultimate success (Blackwell et al., 2007; Dupeyrat & Mariné, 2005; Hong et al., 1999; Rhodewalt, 1994; Robins & Pals, 2002). Rhodewalt (1994) demonstrated that individuals holding entity theories were more likely to “self-handicap,” withdrawing effort or procrastinating within challenging learning contexts relative to those holding incremental theories. Hong et al. (1999) compared entity and incremental theorists' responses to setbacks during learning. Incremental theorists were more likely to modify their strategies by taking remedial action after having received negative feedback compared to entity theorists. Similarly, Blackwell et al. (2007) demonstrated that junior high school students holding incremental theories were more likely to respond to failures by flexibly seeking out new learning and strategies compared to entity theorists and, as a result, increased their achievement. In the same vein, Robins and Pals (2002) showed that incremental theorists displayed a greater evidence of mastery-oriented strategies including effort escalation. In contrast, entity theorists blamed their failure on low ability, and gave up or perseverated on disadvantageous strategies in challenging learning contexts. Thus, research suggests that adaptive strategy use in challenging task contexts – something that incremental theorists do more often – supports learning and achievement. Moreover, withdrawing cognitive effort or perseverating on futile strategies in response to difficulties – something that is more common for entity theorists – proves detrimental for learning and achievement. Comprehending multiple documents in the current information age can pose quite a challenge for high school and college students for several reasons (Britt & Aglinskas, 2002; Goldman et al., 2012; Wiley et al., 2009). First, complex conceptual relationships within single texts are often left implicit, requiring that readers strategically elaborate information to establish coherence (Chi, 2000; McNamara, Kintsch, Songer, & Kintsch, 1996). Second, documents are seldom written for the student's exact inquiry purpose. That is, readers must select and re-purpose concepts from single documents and infer connections across multiple documents that are otherwise uncertain. Finally, document authors' motives are not always transparent, with documents often originating from unknown or questionable sources that may vary in expertise on the topic, biases, and so forth.

J.L.G. Braasch et al. / Learning and Individual Differences 31 (2014) 11–20

Taken together, a multiple document context presupposes that readers move beyond simply grasping the content information in each text. They must also strategically construct links among the key concepts presented by different documents, and between each key concept and the information source(s) conveying them (Britt & Rouet, 2012). Given prior research on implicit theories of intelligence (see above), it is reasonable to assume that this individual difference factor may also play a role in the challenging reading-task context of multiple document comprehension – by promoting or constraining strategic processing of multiple, variably-reliable documents. Thus, such a reading-task context might be a welcome challenge for incremental theorists leading to their flexible use of reading strategies (e.g., evaluating source information, making inferences by combining information across documents). At the same time, a multiple document comprehension context might tap into entity theorists' maladaptive strategies (e.g., perseverating on content at the expense of source evaluation, locally processing each text without concern for inferred relationships across texts). To explore this in the current work, we related secondary school students' scores on measures of implicit theories of intelligence to their evaluation and comprehension performance after reading complex, variably-reliable documents on a scientific issue. 1.3. The present study The current work extends investigations of the general detriments of holding entity theories of intelligence and the general benefits of holding incremental theories of intelligence. Whereas previous work has focused on longer-term, macro-level learning and course achievement, we investigate whether these theories are functional in more microlevel multiple document comprehension contexts. Do beliefs that intelligence is fixed confer costs for multiple document comprehension, in terms of reducing readers' use of evaluation and integration strategies when reading multiple documents? Do beliefs that intelligence is malleable, something that can be modified by using a host of appropriate strategies, benefit multiple document evaluation and integration? These central research questions were addressed in the current work. We investigated costs and benefits using four measures frequently viewed as indications of successful multiple document evaluation and comprehension. These included the number of concepts from higherquality documents that were retained and included in studentgenerated essays, propensities to differentiate between more and less useful documents, critical considerations of trustworthiness in doing so, and generation of cross-document inferences – as a measure of conceptual integration. On the basis of theoretical assumptions and previous findings concerning the influences of implicit theories of intelligence on performance outlined above, we expected that entity theories of intelligence would negatively predict multiple document evaluation and comprehension, while incremental theories would display positive prediction. 2. Method 2.1. Participants Participants were 59 students1 (M age = 17.96, SD = 0.35; 48% female) completing college preparatory courses at a public uppersecondary school in southeast Norway. The sample was relatively

1 Seventy-four students from three classrooms took part in a control condition within a larger intervention study described in Braasch et al. (2013). Of those students, 59 were present in two sessions and completed all individual difference, evaluation, and comprehension measures included in the current study. There is no reason to believe that they differed in any particular way from the 15 students that were not included in this study because they did not produce a complete data set due to absences or partial participation.

13

homogeneous (i.e., middle class) with respect to socioeconomic status. Most (77%) were native-born Norwegians who learned Norwegian as a first language. The remaining students were raised in Norway in bilingual households. 2.2. Materials 2.2.1. Individual differences 2.2.1.1. Word recognition. Strømsø, Hagtvet, Lyster, and Rygvold's (1997) assessment of word recognition skill presented 360 words written in 30 rows. In every row, four word clusters of three high-frequency words were printed without spaces (e.g., stopsleepyellow). Students indicated as many words as possible for 3 min by drawing vertical lines between the words, with the number of correct word clusters used as an index of word recognition skill. Maximum score was 120. In the present sample, the reliability estimate (Cronbach's α) for scores on this measure was .93. Please note that reading researchers who have studied individual differences in word recognition skills in relation to comprehension of text (e.g., Bell & Perfetti, 1994; Cunningham et al., 1990) have demonstrated that word recognition may be related to text comprehension even in adults and, moreover, that it may be an independent predictor (Stanovich, 2000). For example, Cunningham et al. showed that word recognition skills accounted for significant additional variance in the text comprehension of adult college readers even after measures of general intelligence, listening comprehension, sentence memory, and vocabulary were entered into a regression equation. 2.2.1.2. Prior knowledge. Ten open-ended questions assessed prereading knowledge of the central concepts underlying weather systems discussed in the documents (e.g., How are rain clouds formed?). A preliminary version of the measure was constructed in consultation with an earth scientist with expertise on the topic. Participants received 2 points if a response reflected complete, accurate understanding, 1 point if it was accurate yet incomplete, and 0 for an incorrect response or blank. Thus, prior knowledge scores reflected 20 possible points. A Cohen's Kappa interrater reliability score of .82 for 25% of the larger sample of Braasch, Bråten, Strømsø, Anmarkrud, and Ferguson (2013) was obtained. Disagreements were resolved in discussion; one rater scored the remaining participants. Cronbach's α for the sample was .76. 2.2.1.3. Working memory. Working memory capacity was assessed using an adaptation of Swanson and Trahan's (1992) Working Memory Span Task, which was derived from Daneman and Carpenter's (1980) original Reading Span Test. Twelve sets of unrelated sentences were read aloud with a 2-s interval between each sentence. Sets gradually increased from two to five sentences. For each set, participants were tasked to simultaneously a) comprehend the sentences well enough to answer a comprehension question about an unknown sentence after reading was completed, and b) remember the final words from each of the sentences. After writing a response to the comprehension question, participants recalled the sentences' final words. For each set, participants received one point for correct comprehension response and one additional point for each of the final words recalled. Because working memory refers to a cognitive system involved in simultaneous processing and storing of information (Baddeley, 1990), no points were received for a set if the comprehension response was incorrect. In this way, we ensured that participants had comprehended (i.e., processed) the sentences and not only treated the task as one of verbal memory for the final words. Maximum score was 54. Cronbach's α was .81. 2.2.1.4. Implicit theories of intelligence. A Norwegian version of Dweck's (1999) Theories of Intelligence Scale was used (Bråten & Strømsø,

14

J.L.G. Braasch et al. / Learning and Individual Differences 31 (2014) 11–20

2004). Four items assessed the degree to which students identified with an entity theory of intelligence reflecting a belief that intelligence is a fixed trait, a personal attribute or quantity that cannot be changed (e.g., You have a certain amount of intelligence, and you can't really do much to change it). Four additional items assessed the degree to which students identified with an incremental theory of intelligence reflecting a belief that intelligence is malleable, that is, that individuals can become more intelligent through their efforts (e.g., You can always substantially change how intelligent you are). Entity and incremental items were presented in a mixed order; participants rated agreement with each statement on a 6-point scale (1 = strongly disagree, 6 = strongly agree). Cronbach's α reliabilities were .84 for entity items and .89 for incremental items. 2.2.2. Documents The topic was the typical weather patterns across the Pacific Ocean and El Niño's disruption of them. The first author identified 30 core concepts underlying typical and El Niño-related weather patterns in consultation with an earth scientist with expertise on the El Niño weather phenomenon (Braasch & Goldman, 2010). The set of concepts, shown in Appendix A, guided the development of three more useful documents that were consistent with current scientific thinking and evidence. Appendix B provides a brief description of the source features and content information in each of the more useful documents. To ensure that our document set closely mirrors the kinds of diverse information sources that students might encounter when reading multiple documents, three less useful documents were also designed and presented. All were based on authentic yet unsubstantiated, less reliable accounts found on the Internet (see brief descriptions of each document's source features and content information in Appendix B). Across the three less useful documents, a total of eight erroneous concepts were reported. We note that, with authentic documents, usefulness, reliability, and accuracy are all inextricably correlated. Documents containing accurate information on a topic conveyed by reliable sources will necessarily provide the most useful information for answering an inquiry question. The converse is also true: Inaccurate information conveyed by questionable or unreliable sources will necessarily be the least useful in an inquiry context. To confirm a priori distinctions of the usefulness of the documents, we tasked six multiple document reading experts to independently read then rank-order all six documents using the same prompt that was provided for the participating students (see below). Average expert ranks for the three more useful documents, that is, for the textbook excerpt (M = 2.17, SD = 0.98), the Illustrated Science article (M = 1.83, SD = 0.75), and the forskning.no popular science article (M = 2.00, SD = 0.89), were all approximately 2 (with 1 being the most useful designation and 6 being the least). Average expert ranks for the three less useful documents, that is for the environmental blog entry (M = 4.67, SD = 0.51), the political commentary article (M = 4.33, SD = 0.52), and the astrological newsletter (M = 6.00, SD = 0.00), were considerably higher. Thus, the six documents represented a range of usefulness for answering the focal inquiry question, with experts corroborating a priori distinctions. For all documents, a bolded title (containing the key words typical weather patterns or El Niño) was followed by the source feature information (author, type, venue, publication date), which was in turn followed by the text's content information. As an indication of text difficulty, we computed readability scores for each of the six documents based on the formula proposed by Björnsson (1968), which is based on word length and sentence length. When using this formula, readability scores normally range from 20 (very easy text) to 60 (very difficult text). The mean readability score of the three more useful documents was 50.0 and the mean readability score of the three less useful documents was 44.3. According to readability standards presented by Björnsson (1983), all documents could be described as average to difficult texts typical of “factual prose”.

2.2.3. Post-reading materials and scoring 2.2.3.1. Essay task. “Explain the causes of the typical weather patterns in the Pacific Ocean and the processes that make El Niño change these weather patterns” was printed at the top of a sheet of paper. Two independent judges used the 30-concept scoring rubric in Appendix A for 25% of the larger study's essay responses. Interrater agreement (Cohen's Kappa) was .80, and all disagreements were resolved in discussion. One judge scored the remaining essays. 2.2.3.2. Rank-order and justification tasks. “You are now going to rankorder the texts from most (1) to least useful (6) for understanding the El Niño weather phenomenon” was printed at the top of another sheet of paper. The documents' titles and source features were then presented with blank spaces to write the ranks. “Below you are going to explain why you assigned the texts the ranks that you assigned to them. Provide as much detail as you can in your explanation” was written on a second page. The remainder of the worksheet presented headings (Rank 1 reason, Rank 2 reason, and so forth) with blank lines. Participants may consider a document's usefulness using a variety of different criteria based on their interpretation. Like Barzilai and Zohar (2012), we used a general usefulness prompt to investigate to which extent students raised trustworthiness issues in their ranking justifications spontaneously, that is, without a direct prompt to do so. Evaluating usefulness in light of trustworthiness is typically not done with less successful students, but when it occurs spontaneously it facilitates multiple document comprehension as is more typically the case for good students and experts (Goldman et al., 2012; Wineburg, 1991). To represent students' abilities to discriminate among diverse documents, several multiple document researchers have relied upon ranking discrimination scores (Braasch et al., 2009; Wiley et al., 2009). In alignment with this work, we computed a difference score for each student by subtracting their average rankings for the three more useful from their average rankings for the three less useful documents. As such, higher positive ranking discrimination scores reflected better differentiation; ranking discrimination scores closer to 1 signified a lack of discrimination. Rank-order justifications were coded for the presence of several types of content-based justifications that fall outside the scope of the current work (see Braasch et al., 2013, for the complete set of categories). For example, a statement like “I ranked this document third because the title interested me” received a coding of interest; a sentence like “I ranked this last because it was complicated” received a code for comprehensibility. We additionally coded for mentions of the information sources (author, publication, venue, type, or date). For example, a statement such as “I ranked this fourth because it was published in 2002” received a code of publication date. Of interest for the current work, we coded each justification for an indication of a more critical reliability assessment. These took many forms. Positive trustworthiness assessments included mentions of trustworthiness or reliability, or whether the information had been quality-checked or reviewed by experts prior to publication, to name a few. Negative assessments mentioned a lack of trustworthiness or reliability, biases to persuade or promote the author's case, or mentions of subjectivity or one-sidedness. Each justification entry was coded for assessment of the trustworthiness of the document for that particular rank-order decision. Thus, trustworthiness mention scores were out of six. Two independent judges used the various coding categories described in Braasch et al. (2013) to score a random selection of 25% of the justification protocols. The Cohen's Kappa agreement for all categories was .75. All disagreements were resolved in discussion between the two raters. One judge scored the remaining participants. 2.2.3.3. Intertextual inference verification task (IIVT). Previous research has demonstrated that intertextual verification tasks provide a fruitful

J.L.G. Braasch et al. / Learning and Individual Differences 31 (2014) 11–20

way to measure the inferred connections that readers make among concepts distributed across multiple documents (Bråten et al., 2009; Strømsø et al., 2008). In alignment with the previous research, we created a verification task to measure the degree to which students could make inferences by combining information from the more useful documents that we provided. Please note, however, that the IIVT was created for this particular research and has not been used in any previous studies. Twenty sentences (7 valid, 13 invalid inferences) were constructed such that none could be answered correctly by judging the validity of only one piece of information at a time, but rather involved crosstextual inferences about information from at least 2 of the 3 more useful documents. Thus, readers had to consider the whole, integrated meaning of each sentence to decide whether it was a valid or an invalid inference. Appendix C provides illustrative IIVT examples documenting the concepts that were covered for each item, the texts from which the concepts could be gleaned, and explanations concerning the connections among the concepts that one would have to make to respond correctly. Performance scores reflected the number of correct inference distinctions out of 20 items. Cronbach's α was .64. While the reliability estimate of the IIVT was lower than desirable, it was comparable to those obtained in other studies using similar tasks (Bråten et al., 2009; Bråten & Strømsø, 2010; Salmerón, Gil Bråten & Strømsø, 2010). Presumably, the somewhat low reliability of the IIVT was related to the relatively short length of the scale. Thus, Royer, Carlo, Dufresne, and Mestre (1996) suggested that to obtain a really high reliability estimate, a verification task might have to include almost 100 items, which was not feasible given the time available for data collection in this study. 2.3. Procedure 2.3.1. Day 1: Assessing reader characteristics Participants were first group-administered the word recognition (5 min) and working memory measures (20 min). Then, for 25 min they independently completed the prior knowledge and implicit theories of intelligence assessments. 2.3.2. Day 2: Multiple document comprehension assessments Eleven days later, participants were provided with a folder containing the six documents, the essay response sheet, the rank-order and justification tasks, and the IIVT. Students were told that the documents resulted from a Google™ search using the key words El Niño changes typical weather in the Pacific Ocean. They were allotted 30 min of reading time to be able to explain the causes of the typical weather patterns in the Pacific Ocean and the processes that make El Niño change these weather patterns. Although the documents were presented in counterbalanced orders, students could read and reread any information that they felt would help them to answer the inquiry question in any order. After the reading time expired, students replaced the documents in the folder. They then wrote their essays for up to 20 min without having access to the documents. Students then replaced the essay response sheets and took out the rank-order and justification evaluation tasks. For 15 min, students rank-ordered the documents from most (1) to least useful (6) for understanding the El Niño weather phenomenon and provided justifications for their rank-order assignments. After placing the rank-order and justification tasks back in their folders, they removed the IIVT task. Instructions were to mark each test sentence with a yes if the statement could be inferred by combining information from one or more of the documents that were just read and no if it could not. Twenty minutes were allotted for the IIVT. It should be noted that the sequence of tasks was used to minimize reactivity. That is, we intended that participants should write essays based on the mental representations that they constructed during reading without cues from source information (rank-order and justification evaluation tasks) or concepts presented in the IIVT task.

15

3. Results Descriptive statistics as well as zero-order correlations for all measured variables are reported in Table 1. To answer our research questions regarding the unique predictability of entity and incremental theories of intelligence for aspects of multiple document comprehension, we computed four regression equations. Outcome measures were students' inclusion of scientific concepts in their essays (i.e., number of scientific concepts), their abilities to discriminate amongst the more and less useful documents (i.e., ranking discriminations), their mentioning of trustworthiness issues in their ranking justifications (i.e., trustworthiness assessments), and their abilities to draw inferences across documents (i.e., IIVT scores). In addition to using the two theories of intelligence measures as predictors for each of these outcomes, we included word recognition, prior knowledge, and working memory to control for variance in the outcome measures due to these individual difference measures. Table 2 shows the results of the regression analyses for all four outcome variables. The five predictors together explained a statistically significant amount of variance in all four outcome measures, with F(5, 52) = 6.46, p = .000 (R2 = .38) for the number of scientific concepts, F(5, 52) = 4.17, p = .003 (R2 = .29) for the ranking discriminations, F(5, 52) = 3.18, p = .014 (R2 = .23) for the trustworthiness assessments, and F(5, 53) = 2.63, p = .034 (R2 = .20) for the IIVT. Moreover, for the number of scientific concepts (β = .41, p = .012), for the trustworthiness assessments (β = .35, p = .037), and for the IIVT (β = .43, p = .017), students' endorsement of an incremental theory of intelligence was a unique positive predictor, while it – for the ranking discriminations – did not quite reach a conventional level of statistical significance (β = .31, p = .055). In contrast, no statistically significant results were obtained for students' endorsement of an entity theory of intelligence. Among the control variables, prior knowledge uniquely predicted students' inclusion of scientific concepts in their essays (β = .57, p = .000) and trustworthiness assessments (β = .44, p = .003), while working memory uniquely predicted students' abilities to discriminate between more and less useful documents (β = .41, p = .003) as well as their abilities to draw inferences across documents (β = .35, p = .015). Finally, word recognition was not a unique predictor of any of the outcome measures at this educational level. 4. Discussion This study uniquely contributes to research on implicit theories of intelligence as well as on multiple document reading. To the best of our knowledge, it is the first study that examined the role of this individual difference variable in the context of learning from multiple information sources. Moreover, by including both entity and incremental views on intelligence in the same regression equations and, at the same time, controlling for several other reading-related variables, we were able to probe the relative as well as the unique contribution of the two implicit theories to multiple document evaluation and integration. As expected, beliefs in intelligence as a malleable quality uniquely and positively predicted performance for all outcome measures. Specifically, the multiple regression analyses showed that students holding such beliefs were more likely to include concepts from higher quality documents in their essays, to discriminate between more and less useful documents given the task, to consider the trustworthiness of the documents as a basis for discrimination, and to construct deeper, intertextual understandings of the documents by drawing bridging inferences between them. Presumably, in accordance with other work on the role of students' implicit theories of intelligence in learning and performance (e.g., Blackwell et al., 2007; Hong et al., 1999; Robins & Pals, 2002), beliefs in intelligence as a malleable quality facilitated students' employment of strategies that help them meet the challenges of comprehending multiple documents. Specifically, incremental theorists

16

J.L.G. Braasch et al. / Learning and Individual Differences 31 (2014) 11–20

Table 1 Descriptive statistics and zero-order correlations for all measured variables. Variable

1

1. Word recognition 2. Prior knowledge 3. Working memory 4. Entity theory 5. Incremental theory 6. Number of science concepts 7. Ranking discriminations 8. Trustworthiness assessments 9. Intertextual inferences M SD Skewness

– .37⁎⁎

2

3

– .38⁎⁎ .13 −.19 .55⁎⁎⁎ .11 .41⁎⁎

.25 −.06 .23 .16 .07 .24 .22 55.62 11.98 −.04

– .27⁎ −.21 .35⁎⁎ .25 .16 .29⁎

.08 6.18 3.71 .29

20.87 10.33 .12

4

5

– −.73⁎⁎⁎ .09 −.33⁎

– .03 .34⁎⁎

−.10 .02 4.61 .94 −.52

.19 .19 2.68 1.02 .20

6

7

– .19 .45⁎⁎⁎ .30⁎

– .36⁎⁎ .15 1.22 1.85 −.87

3.80 4.34 1.32

8

9

– .23 1.19 1.53 1.61

– 10.23 3.45 .22

⁎ p b .05. ⁎⁎ p b .01. ⁎⁎⁎ p b .001.

may have sustained their cognitive engagement in the face of uncertain outcomes, if merely to take advantage of the opportunity to gain new understandings (Dweck, 1999; Hong et al., 1999; Robins & Pals, 2002). An incremental theory of intelligence may also involve greater cognitive flexibility (Dweck, 1999), which, according to cognitive flexibility theory (Spiro, Coulson, Feltovich, & Anderson, 1994) may be needed to gain rich and flexible understandings of complex knowledge distributed across multiple sources. On the other hand, beliefs in intelligence as a fixed, unchangeable quantity did not emerge as a unique negative predictor, contrary to expectations. Thus, although prior research on implicit theories of intelligence has indicated that entity theorists are prone towards disengaging from tasks posing challenges and perseverating on inappropriate strategies (Blackwell et al., 2007; Rhodewalt, 1994; Robins & Pals, 2002), comprehension costs were not apparent in this study. In the complex reading task context that we created, the benefits of endorsing an incremental theory therefore seemed more pronounced than the costs of endorsing an entity theory of intelligence. One possible reason for this might be that an incremental theory and an entity theory are not located at the opposite ends of a single construct. This possibility is unlikely, however, given that about 85% of students (later grade school through college) have been shown to agree consistently with either an incremental theory or with an entity theory and thus hold a clear theory about intelligence (evenly split among the two theories) (Dweck, 2002; Dweck & Molden, 2005). Please also note that a strong negative correlation between the incremental and entity theory scores in this and other research (e.g., Bråten & Strømsø, 2006) shows that there is a clear distinction between the two implicit theories, in the sense that students who are high in incremental beliefs are low in entity beliefs and vice versa. We would therefore suggest that another, more likely, possibility is that students endorsing entity theories did not perceive that the task of reading to learn from multiple documents in this setting, without any real, public consequences for

them, posed enough of a risk of revealing that they were not endowed with sufficient intelligence. This would reduce the saliency and function of entity theories. However, the same task may have been enough of a welcome challenge for students endorsing incremental theories. The opportunity to learn may have oriented them towards strategic processing and possibly sustained engagement. Because the current information age guarantees that readers will need to incorporate several kinds of reading strategies when dealing with complex and variably-reliable documents, students' propensities to rise to such challenges might create gaps in knowledge acquisition and integration between those who endorse incremental and entity theories. Intriguing future research could investigate longer-term benefits of incremental theories of intelligence when secondary school students must accrue knowledge from multiple information sources over prolonged periods of time (e.g., over the course of a semester). Thus, while entity-oriented learners that are anxiously trying to prove themselves (Molden & Dweck, 2006) may be strategically handicapped when they perceive the stakes to be high – for example in a competitive setting where their performance relative to others also becomes public – the maladaptive strategic pattern found to characterize entitists (Dweck & Leggett, 1988) may not bear out in lower stakes testing environments – like the one used in this study. On the other hand, incremental-oriented learners that are excitingly exploring new realms of knowledge (Molden & Dweck, 2006) may be concerned with strategically mastering the materials whatever the stakes – and thus perceive the multiple document task context that we created as an opportunity to gain new knowledge and experience. This implies that our findings may contribute to an understanding of each implicit intelligence theory's functionality in lower stake testing environments, with entity theories only detrimental in higher stake situations and incremental theories beneficial regardless of the stakes involved. Future research in this area might experimentally manipulate the stakes

Table 2 Results of multiple regression analyses for variables predicting number of scientific concepts, ranking discriminations, trustworthiness assessments, and intertextual inferences. Predictor

Word recognition Prior knowledge Working memory Entity theory Incremental theory ⁎ p b .05. ⁎⁎ p b .01. ⁎⁎⁎ p b .001.

Scientific concepts

Ranking discriminations

Trustworthiness assessments

Intertextual inferences

B

SE B

β

B

SE B

β

B

SE B

β

B

SE B

β

−.07 .67 .08 1.17 1.73

.05 .15 .05 .77 .73

−.18 .57⁎⁎⁎

−.03 .03 .07 −.47 .55

.02 .07 .02 .36 .34

−.17 .06 .41⁎⁎ −.23 .31

−.00 .18 .01 .15 .53

.02 .06 .02 .30 .29

−.03 .44⁎⁎

.01 −.05 .12 .90 1.44

.04 .14 .05 .68 .66

.02 −.06 .35⁎ .25 .43⁎

.19 .25 .41⁎

.06 .09 .35⁎

J.L.G. Braasch et al. / Learning and Individual Differences 31 (2014) 11–20

involved – for example by putting more emphasis on competition and publicity – to investigate whether implicit theories of intelligence provide different contributions to multiple document comprehension dependent on the perceived stakes. On a related note, our findings may also suggest that entity theories of intelligence are more culturally sensitive than incremental theories – in the sense that the former operate less similarly and strongly across cultures than the latter. More specifically, in the context of the egalitarian Norwegian educational system, with relatively little emphasis on competition and individual distinction (Undheim, Nordvik, Gustafsson, & Undheim, 1995), the task that we created may have been perceived as less challenging or threatening than it would have been in a less egalitarian, more competitive system, with the cost of endorsing an entity theory of intelligence therefore not apparent on such a task in this particular context. More research is certainly warranted that investigates whether students perceive the stakes involved in similar tasks differently across cultures, with a possible consequence being that entity theories of intelligence contribute differently to performance on similar tasks in different cultures. Of note is that the unique predictability of incremental beliefs about intelligence was observed when several well-established predictors of reading competence were simultaneously controlled for. With respect to the control variables, prior knowledge and working memory were found to uniquely predict different important aspects of multiple document comprehension. It was not surprising that prior topic knowledge predicted students' inclusion of scientific concepts in their essays. However, prior knowledge was also the better predictor of students' trustworthiness assessments, suggesting that they used their knowledge of conceptual content, rather than source information, when judging the trustworthiness of the documents (cf., Bråten et al., 2011; Rouet et al., 1996). In contrast, working memory was the better predictor of students' ability to discriminate between more and less useful documents as well their performance on the IIVT. The IIVT may have required working memory resources because successful performance involves not only comprehension of and memory for individual documents, but also constructions of inferential links among them (Wiley, Griffin, & Thiede, 2005). An interpretation of the unique predictability of working memory for ranking discriminations is more speculative at present. It is important to revisit that the students did not have access to the full texts when ranking the usefulness of the documents. Thus, when using document content to help decide upon an order of usefulness, they would have to retrieve content information from two or more documents from long-term memory to compare and rank order them. Current characterizations of working memory highlight its role in accessing and utilizing information from long-term memory during complex, attention-demanding tasks (Baddeley, 2000), which may have supported rank-order performance for these participants. Finally, although students' word recognition skill has been shown to uniquely predict their learning from multiple documents among younger students (Bråten et al., 2013), one likely reason for the lack of contribution of word recognition in this study may be that most of the participants had reached a level so high that further improvement did not matter much in terms of performance (cf., Strømsø et al., 2008). Future work in this area could therefore supplement or replace measures of word recognition skills with measures of reading comprehension when trying to isolate variance due to implicit theories of intelligence. Of course, our findings are contextualized not only by the Norwegian sample of secondary school students that participated but also by the particular text materials and tasks that we presented to them. Future work examining the role of implicit theories of intelligence in multiple document evaluation and integration should therefore probe the generalizability of our findings. For example, research could include other student populations working with multiple documents on other issues and demonstrating their competence on other outcome measures. As indicated above, cross-cultural research in this area is also warranted. Moreover, future work should use larger sample sizes, also allowing for the testing of possible

17

interaction effects between implicit theories of intelligence and the other reading-related individual difference variables on multiple document evaluation and integration. Finally, because the correlational data that we analyzed do not warrant conclusions regarding causality, further experimental work is needed to more firmly establish causal relations between students' implicit theories of intelligence and aspects of their multiple document comprehension. Despite such limitations, our findings may suggest some important implications not only for theory but also for educational practice. While cognitive variables have thus far taken the front seat in work on multiple document comprehension, this study uniquely contributes to the literature by indicating that not only the cognitive but also the personality system of readers is involved in evaluating and learning from multiple information sources. Moreover, it indicates that personality in terms of self-theories concerning readers' beliefs about their intelligence (Dweck, 2008; Dweck & Leggett, 1988; Molden & Dweck, 2006) plays a role in such complex reading-task contexts independent of cognitive variables. However, in addition to contributing to fleshing out a more comprehensive theory of individual differences that underlie performance on essential 21st century literacy tasks, our study may inform educators about reader characteristics that may facilitate adaptive strategic processing of multiple documents. Importantly, these reader characteristics should not be regarded as static (Dweck & Molden, 2005). To the contrary, prior intervention work (Aronson et al., 2002; Blackwell et al., 2007; Good et al., 2003) has shown that students can be taught an incremental theory of intelligence, with incremental messages conveyed during instruction, in turn, improving student learning and performance. Future work should therefore examine the extent to which teaching an incremental theory of intelligence and how to apply it to multiple document reading may promote students' learning from heterogeneous textual materials. Appendix A. Core concepts in the domain and their distribution across the three more useful documents

Textbook excerpt Typical weather pattern concepts High air pressure in east Low air pressure in west Trade winds east to west Water is dragged west Sun heats water Warm water bulge in western Pacific Warm air temperatures High levels of evaporation less dense, air rises Cooling reduces moisture Cloud formation Heavy rainfall in west Cool water upwelling in eastern Pacific Cooler air temperatures Low levels of evaporation Denser air does not rise Less cloud formation Lack of rainfall in east Walker circulation perpetuates patterns El Niño-related weather concepts Pressure difference weakens Trade winds relax Warm water bulge is in east High evaporation Warm moist air Less dense, rises Cloud formation Heavy rainfall in east Cool water upwelling in west Low evaporation Lack of rainfall in west

✕ ✕ ✕ ✕ ✕ ✕

✕ ✕

✕

Magazine article

Research website

✕

✕ ✕ ✕ ✕ ✕ ✕ ✕ ✕ ✕ ✕ ✕ ✕ ✕ ✕ ✕ ✕ ✕ ✕ ✕ ✕ ✕ ✕ ✕ ✕ ✕

18

J.L.G. Braasch et al. / Learning and Individual Differences 31 (2014) 11–20

Appendix B. A description of more and less useful documents' source features and content information

More useful documents

Description of source features

Description of content information

Typical weather patterns in the Pacific Ocean: When the air meets the water 444 words

An excerpt from an upper secondary school geography textbook This document explains the relationships between atmospheric written by a science teacher published by a well-known processes and differences in oceanic temperatures, and that these publishing house in 2007 processes result in predictable weather patterns in the western and eastern Pacific Ocean. What causes typical weather patterns in the Popular science article for Illustrated Science magazine written by This document explains the reasons that the different ocean Pacific Ocean and how do scientists predict a researcher (oceanographer) at an ocean research institute temperatures across the Pacific Ocean result in the typical weather anomalies like El Niño? 565 words published in 2002 patterns in the western and eastern Pacific Ocean by informing on chemical processes associated with the water cycle. A world of change — the El Niño weather A popular science article from an online research magazine The document explains that El Niño events are triggered when air phenomenon 520 words published by a group of research and educational institutions pressure differentials decrease across the Pacific Ocean, causing (www.forskning.no) written by a senior researcher at a anomalous weather patterns. meteorological institute published in 2011 Less useful documents

Description of source features

El Niños on the rise — We only have ourselves to blame!!! 566 words

An environmental blog entry written by a freelance writer in 2008

Description of content information

This document explains that El Niño events are becoming stronger and more frequent over the past several decades because of changes in the direction of global warming. It claims that increases in global warming due to manmade causes have weakened the typical relationships between air pressure and air movement, thereby increasing the frequency of El Niño events. The document makes inappropriate causal interpretations of computer simulation data and uses anecdotal evidence. Is global warming increasing the occurrences A commentary article from a Norwegian liberal daily newspaper This document also claims that El Niño events are becoming stronger of El Niños? What we do and do not know written by a political editor in 2004 and more frequent over the past several decades, related to global 547 words warming. A computer simulation finding is used as support. Although this article goes on to present some researchers' cautions against conclusions based solely on computer simulation data (e.g., inaccurate prediction), the author ultimately rejects the cautions outright. Instead, she makes a “call to arms” for more serious environmental regulations on the use of fossil fuels. Thus, there are implications that her political agenda motivates what was presented. El Niño comes from space, not humanity 521 A newsletter excerpt written by a global astrologer for his This document explains that weather anomalies such as El Niño words personal website in 2011 events result from cosmic forces (e.g., the alignment of planetary bodies). As such, this document is less useful due to an inappropriate reasoning based on circumstantial, unscientific evidence.

Appendix C. Example items from the intertextual inference verification task

Example IIVT items

Concepts involved

Texts conveying these concepts

Explanation

Droughts tend to occur in areas experiencing lower evaporation.

Typically, the eastern Pacific experiences low levels of evaporation. Typically, there is a lack of rainfall in the eastern Pacific. During El Niño, the western Pacific experiences low levels of evaporation. During El Niño, there is a lack of rainfall in the western Pacific. During El Niño, the trade winds relax. Typically, there is a lack of rainfall in east.

More useful text 2

This is a valid inference because, during typical and El Niño-related weather patterns, areas with lower levels of evaporation experience a less rainfall and droughts (relative to regions of higher evaporation).

During El Niño, there is heavy rainfall in east.

More useful text 3

Typically, the eastern Pacific has higher air pressure. Typically, upwelling occurs in the eastern Pacific. During El Niño, air pressure increases in western Pacific. During El Niño, upwelling increases in western Pacific.

More useful text 1

Response YES

During an El Niño event, the relaxation of the trade winds results in a rainy season in the eastern Pacific Ocean. Response YES Within the Pacific Ocean, upwelling occurs in regions with relatively lower surface air pressure. Response NO

More useful text 2 More useful text 3 More useful text 3 More useful text 3 More useful text 1 and 2

More useful text 1 More useful text 2 More useful text 2

This is a valid inference because a relaxation of the trade winds would increase the amount of warm water, evaporation, condensation, and ultimately the amount of rainfall in the eastern Pacific relative to typically dry conditions. This is an invalid inference because, during both typical and El Niño-related weather patterns, areas with higher air pressure experience upwelling.

J.L.G. Braasch et al. / Learning and Individual Differences 31 (2014) 11–20

References Afflerbach, P., & Cho, B. -Y. (2009). Identifying and describing constructively responsive comprehension strategies in new and traditional forms of reading. In S. E. Israel, & G. G. Duffy (Eds.), Handbook of research on reading comprehension (pp. 69–90). New York: Routledge. Alexander, P. A. (2012). Reading into the future: Competence for the 21st century. Educational Psychologist, 47, 259–280. Andreassen, R., & Bråten, I. (2010). Examining the prediction of reading comprehension on different multiple-choice texts. Journal of Research in Reading, 33, 263–283. Anmarkrud, Ø., Bråten, I., & Strømsø, H. I. (2014). Multiple-documents literacy: Strategic processing, source awareness, and argumentation when reading multiple conflicting documents. Learning and Individual Differences (in press). Aronson, J., Fried, C. B., & Good, C. (2002). Reducing the effects of stereotype threat on African-American college students by shaping their theories of intelligence. Journal of Experimental Social Psychology, 38, 113–125. Baddeley, A. (1990). Human memory: Theory and practice. Hove and London: Erlbaum. Baddeley, A. (2000). The episodic buffer: A new component of working memory? Trends in Cognitive Sciences, 4, 417–423. Banas, S., & Sanchez, C. A. (2012). Working memory and learning underlying conceptual relationships across multiple documents. Applied Cognitive Psychology, 26, 594–600. Barzilai, S., & Zohar, A. (2012). Epistemic thinking in action: Evaluating and integrating online sources. Cognition and Instruction, 30, 39–85. Bell, L. C., & Perfetti, C. A. (1994). Reading skill: Some adult comparisons. Journal of Educational Psychology, 86, 244–255. Björnsson, C. H. (1968). Läsbarhet [Readability]. Stockholm: Liber. Björnsson, C. H. (1983). Readability of newspapers in 11 languages. Reading Research Quarterly, 18, 480–497. Blackwell, L. S., Trzesniewski, K. H., & Dweck, C. S. (2007). Implicit theories of intelligence predict achievement across an adolescent transition: A longitudinal study and an intervention. Child Development, 78, 246–263. Braasch, J. L. G., Bråten, I., Strømsø, H. I., Anmarkrud, Ø., & Ferguson, L. E. (2013). Promoting secondary school students' evaluation of source features of multiple documents. Contemporary Educational Psychology, 47, 447–479. Braasch, J. L. G., & Goldman, S. R. (2010). The role of prior knowledge in learning from analogies in scientific texts. Discourse Processes, 47, 447–479. Braasch, J. L. G., Lawless, K. A., Goldman, S. R., Manning, F. H., Gomez, K. W., & MacLeod, S. M. (2009). Evaluating search results: An empirical analysis of middle school students' use of source attributes to select useful sources. Journal of Educational Computing Research, 41, 63–82. Brand-Gruwel, S., & Stadtler, M. (2011). Solving information-based problems: Evaluating sources of information. Learning and Instruction, 21, 175–179. Bråten, I., Anmarkrud, Ø., Brandmo, C., & Strømsø, H. I. (2014). Developing and testing a model of direct and indirect relationships between individual differences, processing, and multiple-text comprehension. Learning and Instruction, 30, 9–24. Bråten, I., Ferguson, L. E., Anmarkrud, Ø., & Strømsø, H. I. (2013). Prediction of learning and comprehension when adolescents read multiple texts: The roles of word-level processing, strategic approach, and reading motivation. Reading and Writing: An Interdisciplinary Journal, 26, 321–348. Bråten, I., & Strømsø, H. I. (2004). Epistemological beliefs and implicit theories of intelligence as predictors of achievement goals. Contemporary Educational Psychology, 29, 371–388. Bråten, I., & Strømsø, H. I. (2006). Predicting achievement goals in two different academic contexts: A longitudinal study. Scandinavian Journal of Educational Research, 50, 127–148. Bråten, I., & Strømsø, H. I. (2010). When law students read multiple documents about global warming: Examining the role of topic-specific beliefs about the nature of knowledge and knowing. Instructional Science, 38, 635–657. Bråten, I., & Strømsø, H. I. (2011). Measuring strategic processing when students read multiple texts. Metacognition and Learning, 6, 111–130. Bråten, I., Strømsø, H. I., & Britt, M.A. (2009). Trust matters: Examining the role of source evaluation in students' construction of meaning within and across multiple texts. Reading Research Quarterly, 44, 6–28. Bråten, I., Strømsø, H. I., & Salmerón, L. (2011). Trust and mistrust when students read multiple information sources about climate change. Learning and Instruction, 21, 180–192. Britt, M.A., & Aglinskas, C. (2002). Improving students' ability to identify and use source information. Cognition and Instruction, 20, 485–522. Britt, M.A., & Gabrys, G. (2002). Implications of document-level literacy skills for web-site design. Behavior Research Methods, Instruments, and Computers, 34, 170–176. Britt, M.A., Perfetti, C. A., Sandak, R., & Rouet, J. -F. (1999). Content integration and source separation in learning from multiple texts. In S. R. Goldman, A.C. Graesser, & P. van den Broek (Eds.), Narrative, comprehension, causality, and coherence: Essays in honor of Tom Trabasso (pp. 209–233). Mahwah, NJ: Erlbaum. Britt, M.A., & Rouet, J. F. (2012). Learning with multiple documents: Component skills and their acquisition. In J. R. Kirby, & M. J. Lawson (Eds.), Enhancing the quality of learning: Dispositions, instruction, and learning processes (pp. 276–314). New York: Cambridge University Press. Britt, M.A., Rouet, J. -F., & Braasch, J. L. G. (2013). Documents experienced as entities: Extending the situation model theory of comprehension. In M.A. Britt, S. R. Goldman, & J. -F. Rouet (Eds.), Reading from words to multiple texts (pp. 160–179). New York: Routledge. Chi, M. T. H. (2000). Self-explaining expository texts: The dual processes of generating inferences and repairing mental models. In R. Glaser (Ed.), Advances in instructional psychology (pp. 161–238). Mahwah, NJ: Erlbaum. Cunningham, A. E., Stanovich, K. E., & Wilson, M. R. (1990). Cognitive variation in adult college students differing in reading ability. In T. H. Carr, & B.A. Levy (Eds.), Reading and its development: Component skills approaches (pp. 129–159). San Diego, CA: Academic Press.

19

Daneman, M., & Carpenter, P. A. (1980). Individual differences in working memory and reading. Journal of Verbal Learning and Verbal Behavior, 19, 450–466. Dupeyrat, C., & Mariné, C. (2005). Implicit theories of intelligence, goal orientation, cognitive engagement, and achievement: A test of Dweck's model with returning to school adults. Contemporary Educational Psychology, 30, 43–59. Dweck, C. S. (1999). Self-theories: Their role in motivation, personality, and development. Philadelphia, PA: Psychology Press. Dweck, C. S. (2002). The development of ability conceptions. In A. Wigfield, & J. S. Eccles (Eds.), Development of achievement motivation (pp. 57–88). San Diego, CA: Academic Press. Dweck, C. E. (2008). Can personality be changed? The role of beliefs in personality and change. Current Directions in Psychological Science, 17, 391–394. Dweck, C. S., & Leggett, E. L. (1988). A social-cognitive approach to motivation and personality. Psychological Review, 95, 256–273. Dweck, C. S., & Master, A. (2008). Self-theories motivate self-regulated learning. In D. H. Schunk, & B. J. Zimmerman (Eds.), Motivation and self-regulated learning: Theory, research, and applications (pp. 31–51). New York: Erlbaum. Dweck, C. S., & Molden, D. C. (2005). Self-theories: Their impact on competence motivation and acquisition. In A. J. Elliot, & C. S. Dweck (Eds.), Handbook of competence and motivation (pp. 122–140). New York: Guilford. Faria, L., & Fontaine, A. -M. (1997). Adolescents' personal conceptions of intelligence: The development of a new scale and some exploratory evidence. European Journal of Psychology of Education, 12, 51–62. Goldman, S. R. (2004). Cognitive aspects of constructing meaning through and across multiple texts. In N. Shuart-Faris, & D. Bloome (Eds.), Uses of intertextuality in classroom and educational research (pp. 317–351). Greenwich, CT: Information Age. Goldman, S. R., Braasch, J. L. G., Wiley, J., Graesser, A.C., & Brodowinska, K. (2012). Comprehending and learning from Internet sources: Processing patterns of better and poorer learners. Reading Research Quarterly, 47, 356–381. Good, C., Aronson, J., & Inzlicht, M. (2003). Improving adolescents' standardized test performance: An intervention to reduce the effects of stereotype threat. Journal of Applied Developmental Psychology, 24, 645–662. Graesser, A.C., Wiley, J., Goldman, S. R., O'Reilly, T., Jeon, M., & McDaniel, B. (2007). SEEK Web tutor: Fostering a critical stance while exploring the causes of volcanic eruption. Metacognition and Learning, 2, 89–105. Greene, J. A., Costa, L. -J., Robertson, J., Pan, Y., & Deekens, V. M. (2010). Exploring relations among college students' prior knowledge, implicit theories of intelligence, and self-regulated learning in a hypermedia environment. Computers & Education, 55, 1027–1043. Henderson, V. L., & Dweck, C. S. (1990). Achievement and motivation in adolescence: A new model and data. In S. Feldman, & G. Elliot (Eds.), At the threshold: The developing adolescent. Cambridge, MA: Harvard University Press. Hong, Y. Y., Chiu, C. Y., Dweck, C. S., Lin, D., & Wan, W. (1999). Implicit theories, attributions, and coping: A meaning system approach. Journal of Personality and Social Psychology, 77, 588–599. Leu, D. J., Kinzer, C. K., Coiro, J., Castek, J., & Henry, L. A. (2013). New literacies: A dual-level theory of the changing nature of literacy, instruction, and assessment. In D. E. Alvermann, N. J. Unrau, & R. B. Ruddell (Eds.), Theoretical models and processes of reading (pp. 1150–1181) (6th ed.). Newark, DE: International Reading Association. McNamara, D. S., Kintsch, E., Songer, N.B., & Kintsch, W. (1996). Are good texts always better? Interactions of text coherence, background knowledge, and levels of understanding in learning from texts. Cognition and Instruction, 14, 1–43. Molden, D. C., & Dweck, C. S. (2006). Finding "meaning" in psychology: A lay theories approach to self-regulation, social perception, and social development. American Psychologist, 61, 192–203. Rhodewalt, F. (1994). Conceptions of ability, achievement goals, and individual differences in self-handicapping behavior: On the application of implicit theories. Journal of Personality, 62, 67–85. Robins, R. W., & Pals, J. (2002). Implicit self-theories in the academic domain: Implications for goal orientation, attributions, affect, and self-esteem change. Self and Identity, 1, 313–336. Rouet, J. -F., Britt, M.A., Mason, R. A., & Perfetti, C. A. (1996). Using multiple sources of evidence to reason about history. Journal of Educational Psychology, 88, 478–493. Royer, J. M., Carlo, M. S., Dufresne, R., & Mestre, J. (1996). The assessment of levels of domain expertise while reading. Cognition and Instruction, 14, 373–408. Salmerón, L., Gil, L., Bråten, I., & Strømsø, H. I. (2010). Comprehension effects of signalling relationships between documents in search engines. Computers in Human Behavior, 26, 419–426. Samuelstuen, M. S., & Bråten, I. (2005). Decoding, knowledge, and strategies in comprehension of expository text. Scandinavian Journal of Psychology, 46, 107–117. Spiro, R. J., Coulson, R. L., Feltovich, P. J., & Anderson, D. K. (1994). Cognitive flexibility theory: Advanced knowledge acquisition in ill-structured domains. In R. B. Ruddell, M. R. Ruddell, & H. Singer (Eds.), Theoretical models and processes of reading (pp. 602–615) (4th ed.). Newark, DE: International Reading Association. Stadtler, M., & Bromme, R. (2008). Effects of the metacognitive computer-tool met.a.ware on the web search of laypersons. Computers in Human Behavior, 24, 716–737. Stanovich, K. E. (2000). Progress in understanding reading: Scientific foundations and new frontiers. New York: Guilford. Stanovich, K. E., Cunningham, A. E., & Feeman, D. J. (1984). Intelligence, cognitive skills, and early reading progress. Reading Research Quarterly, 19, 278–303. Stipek, D., & Gralinski, J. H. (1996). Children's beliefs about intelligence and school performance. Journal of Educational Psychology, 88, 397–407. Strømsø, H. I., Bråten, I., & Samuelstuen, M. S. (2008). Dimensions of topic-specific epistemological beliefs as predictors of multiple text understanding. Learning and Instruction, 18, 513–527.

20

J.L.G. Braasch et al. / Learning and Individual Differences 31 (2014) 11–20

Strømsø, H. I., Hagtvet, B. E., Lyster, S. A. H., & Rygvold, A. L. (1997). Lese- og skriveprøver for studenter på høyskole-og universitetsnivå. [Reading and spelling tests for students in higher education.]. University of Oslo: Department of Special Needs Education. Swanson, H. L., & Trahan, M. F. (1992). Learning disabled readers' comprehension of computer mediated text: The influence of working memory, metacognition, and attribution. Learning Disabilities Research & Practice, 7, 74–86. Undheim, J. O., Nordvik, H., Gustafsson, K., & Undheim, A.M. (1995). Academic achievements of high-ability students in egalitarian education: A study of able 16-year-old students in Norway. Scandinavian Journal of Educational Research, 39, 157–167.

Wiley, J., Goldman, S. R., Graesser, A.C., Sanchez, C. A., Ash, I. K., & Hemmerich, J. A. (2009). Source evaluation, comprehension, and learning in Internet science inquiry tasks. American Educational Research Journal, 46, 1060–1106. Wiley, J., Griffin, T. D., & Thiede, K. W. (2005). Putting the comprehension in metacomprehension. The Journal of General Psychology, 132, 408–428. Wineburg, S. S. (1991). Historical problem solving: A study of the cognitive processes used in the evaluation of documentary and pictorial evidence. Journal of Educational Psychology, 83, 73–87. Wolfe, M. B. W., & Goldman, S. R. (2005). Relations between adolescents' text processing and reasoning. Cognition and Instruction, 23, 467–502.