Comprehension processes and outcomes with refutation and expository texts and their contribution to learning

Learning and Instruction 41 (2016) 60e69

Contents lists available at ScienceDirect

Learning and Instruction journal homepage: www.elsevier.com/locate/learninstruc

Comprehension processes and outcomes with refutation and expository texts and their contribution to learning Irene-Anna N. Diakidoy a, *, Thalia Mouskounti a, Argyro Fella a, Christos Ioannides b a b

Department of Psychology, University of Cyprus, 1678 Nicosia, Cyprus Mikrokampos School, 61100 Kilkis, Greece

a r t i c l e i n f o

a b s t r a c t

Article history: Received 16 April 2015 Received in revised form 24 July 2015 Accepted 20 October 2015 Available online xxx

The study compared the comprehension processes and outcomes obtained with refutation and expository text and their association with learning outcomes. After a knowledge pretest, undergraduate students read an extended expository text or a corresponding refutation text that addressed three potential misconceptions about the scientific concept of energy. Think-aloud, cued recall, and posttest data indicated that the positive impact of refutation text was more associated with comprehension outcomes than processes. Refutation text did not influence comprehension processes but facilitated valid inference generation in recall and minimized the negative effects of distortions on learning. The findings suggest the timing of the refutation text effect to be later, after reading, and its nature to be that of neutralizing the influence of any misconceptions on learning from text instead of changing them. © 2015 Elsevier Ltd. All rights reserved.

Keywords: Comprehension Inferencing Refutation text Learning

1. Introduction Research exploring conceptual change in science learning has shown refutation texts that explicitly acknowledge and refute potential misconceptions to be generally more beneficial than standard expository science texts when learning requires the restructuring of prior incorrect knowledge (e.g., Braasch, Goldman, €& Wiley, 2013; Diakidoy, Kendeou, & Ioannides, 2003; Mikkila Erdmann, 2001). However, a positive refutation text effect on learning and conceptual change has not been a consistent finding (e.g., Hynd & Guzzetti, 1998; Mason, Gava, & Boldrin, 2008; Palmer, 2003). As a result, one strand of research has examined more closely reader and refutation text characteristics that may facilitate learning (e.g., Braasch et al., 2013; Kendeou, Muis, & Fulton, 2011). A parallel, yet related, line of research has focused on text comprehension, recognizing it as the basis for learning from text (Sinatra & Broughton, 2011). In this context, it has been acknowledged that the underlying mechanisms that may result in any learning gains with refutation text remain unclear. Consequently, several relatively recent studies have focused on the comprehension processes and outcomes with refutation texts (e.g., Ariasi & Mason, 2011;

* Corresponding author. Department of Psychology, University of Cyprus, P.O. Box 20537, CY-1678 Nicosia, Cyprus. E-mail address: [email protected] (I.-A.N. Diakidoy). http://dx.doi.org/10.1016/j.learninstruc.2015.10.002 0959-4752/© 2015 Elsevier Ltd. All rights reserved.

Diakidoy, Mouskounti, & Ioannides, 2011; Kendeou et al., 2011; Kendeou & van den Broek, 2007; Kendeou, Walsh, Smith, & O'Brien, 2014). This research, however, provides an incomplete picture as there has been no simultaneous focus on all three constructs of interest: comprehension processes, comprehension outcomes, and learning outcomes. Consequently, although the findings are intriguing, they also raise a set of questions regarding the association between comprehension processes and outcomes and their relative contribution to learning as a function of text structure. Therefore, the purpose of this study was to contribute to our understanding of the refutation text effect by comparing directly the comprehension processes and outcomes obtained with refutation and expository texts and their contribution to subsequent learning from text. 1.1. Refutation text effects Comprehension and learning from text depend on readers' ability to construct a coherent and well-elaborated mental representation of the information presented in text. Critical text parts need to be mentally represented in relation to each other as well as in relation to existing knowledge structures that may be relevant or related (Kintsch, 1988). The coherence of the representation is a key factor for deep comprehension as it is intimately tied with integration of new information with existing knowledge in long-term , Goldman, & Saul, 1998; McNamara & Magliano, memory (Cote

I.-A.N. Diakidoy et al. / Learning and Instruction 41 (2016) 60e69

2009). This integration, in turn, opens the way for the enrichment and/or modification of existing knowledge structures e that is, meaningful learning. Since no text explicitly specifies all the possible connections that can exist between text ideas, a lot of coherence building rests on the ability to activate and use available knowledge structures to infer within-text and knowledge connections that give rise to an understanding of the text as a whole and its integration with relevant knowledge structures facilitating, thereby, their enrichment and/or modification. Inferencing, however, is complicated in the case of expository text as readers are likely to lack an adequate knowledge base to support their coher et al., 1998). The ence building and integration efforts (e.g., Cote problem is compounded by the possibility of incompatible or inaccurate prior knowledge, as is often the case with scientific expository text. In this case, readers' misconceptions may hinder any attempts at integration or they may give rise to incorrect inferences. The findings of Kendeou and van den Broek (2005) support this latter possibility by showing readers with misconceptions to engage in the same processes, such as paraphrasing and inferencing, as readers without misconceptions. Incompatible prior knowledge, however, had a negative influence on the content of these processes resulting in more invalid inferences during reading and lower recall of text information after reading (Kendeou & van den Broek, 2005). These findings were replicated in a subsequent study that included text structure as a variable (Kendeou & van den Broek, 2007). They had young adult readers with and without misconceptions read a refutation or a non-refutation text on Newton’ laws of motion. Their think-aloud results indicated that readers with misconceptions generated fewer valid and more invalid inferences during reading regardless of text structure. These invalid inferences that were incorrect on the basis of text information were interpreted to reflect the influence of incorrect knowledge (Kendeou & van den Broek, 2007). The refutation text, however, led readers to engage in conceptual change strategies like noticing and attempting to revise discrepancies between prior knowledge and text information. This type of online processes was not observed with readers who had no misconceptions or those who read the non-refutation text. Kendeou and van den Broek (2007; van den Broek & Kendeou, 2008) interpreted their findings to indicate that the refutation text effect on learning is due to the co-activation of misconceptions and scientific explanations that supports their comparison and contrast. Having the two contrasting conceptions active in working memory at the same time increases the likelihood that the reader will notice and possibly attempt to resolve any discrepancies (see also McCrudden, 2012). Noticing and attempting to revise discrepancies between what one knows and what one reads should be reflected in longer processing times for readers with misconceptions when they read a refutation text. However, studies that have employed a reading time methodology provide mixed results showing longer, shorter, or comparable reading times for different parts of the text or the refutation text as a whole (e.g., Braasch, et al. 2013; Broughton, Sinatra, & Reynolds, 2010; Kendeou & van den Broek, 2007; experiment 2). In an eye-tracking study, Ariasi and Mason (2011) had young adult readers with misconceptions about the phenomenon of tides read either a refutation or a standard expository text on the topic. Online measures (first- and second-pass fixations) indicated that refutation text readers spent less time on refutation segments and more time (overall and during rereading, but not on first-pass) on segments that presented scientific concepts, that is, text information that conflicted with their prior knowledge. More interestingly, fixation times on refutational and scientific conception segments were positive and significant predictors of subsequent learning, while overall reading time of the text was a

61

negative predictor of learning from refutation text. Ariasi and Mason (2011) interpreted their pattern of results as indicating the strategic allocation of attention to the processing of critical text information as opposed to non-critical information. Moreover, in line with the co-activation hypothesis, they assumed this strategic processing to entail discrepancy resolution attempts on the part of their readers. Nevertheless, the longer fixation times could also reflect strategic processing to ensure the encoding and retention of this critical information in memory instead, and to minimize the influence of existing misconceptions. Considering that refutations tag misconceptions as faulty knowledge (Braasch et al., 2013), they may also function as signals to warn readers against relying on this knowledge as they process the text. In fact, in light of explicit refutations, an overreliance on prior knowledge for generating inferences could be taken as a sign of non-strategic processing and a failure to comprehend the meaning and the implications of the refutational segments. Instead, a more profitable first-step strategy would be to ensure that the memory imprint of the new incompatible information is distinct and as strong, if not stronger, as that of any misconceptions in order to counteract their influence and to ensure the availability of new information for further processing and use later on and as needed (see also €-Erdmann, 2013). Penttinen, Anto, & Mikkila If attention and processing resources are allocated to ensure the encoding of critical scientific information in memory because the refutations have warned readers against the use of prior knowledge, then readers may adopt a more text- and sentence-based  et al., 1998) resulting in fewer online (valid approach (e.g., Cote and invalid) inferences during the reading of a refutation text when compared to an expository text. This possibility, however, is not supported by results showing no effect of text on online inferences, that is, inferencing during reading (Kendeou & van den Broek, 2007). Moreover, assuming that comprehension processes are  et al., 1998; associated with comprehension outcomes (e.g., Cote Kintsch, 1988), this possibility appears to also run counter to findings showing a positive refutation text effect on offline inferences, that is, those manifested after reading in response to postereading tasks like text recall. Specifically, Diakidoy et al. (2011) had young adults with varying amounts of prior knowledge and misconceptions read either an expository text about energy or a corresponding refutation text that addressed and refuted three potential misconceptions about this concept. Comprehension was assessed after reading with a cued recall task that was scored to provide measures of both overall retention of text information as well as number and kinds of inferences generated in recall. Their results indicated a significant text effect on inferences only. Refutation text recalls contained more valid inferences than expository text recalls and regardless of prior knowledge (Diakidoy et al., 2011). 1.2. Refutation text after-effects The contrast between findings regarding online and offline inferences leaves open the possibility that the increased inference generation observed in the recall of refutation text is the product of reconstructive and more goal-directed processes operating in response to a certain task: to produce a coherent recall protocol. According to the elaborative retrieval hypothesis, recall involves top-down, monitoring, and reconstructive processes associated with the re-activation and further (re)processing of retained information (Carpenter, 2009). In the case of learning from text, the re-activated representation of its content provides the grounds for reworking, reconnecting, and restructuring the originally encoded information. It is reasonable to suppose, then, that the quality of this subsequent processing and its outcomes depend on the

62

I.-A.N. Diakidoy et al. / Learning and Instruction 41 (2016) 60e69

connectedness and the accuracy of the representation of important text content. Therefore, part of the inferencing observed at the time of refutation text recall in the Diakidoy et al. (2011) study may be the products of subsequent reconstructive processes operating on critical text information whose encoding and retention in memory may have been reinforced by the refutations in text. The elaborative retrieval hypothesis has provided the basis for explaining the association between reconstructive processes and meaningful learning (Karpicke, 2012) and the testing effect on learning (Hinze, Wiley, & Pellegrino, 2013). Similarly, it may account for the learning gains observed with refutation text and their association with comprehension outcomes. Diakidoy et al. (2011) examined directly the contribution of comprehension outcomes to subsequent learning from text by assessing learning separately from recall with a delayed posttest including generative questions. Their findings indicated overall learning gains that were larger with the refutation text than the expository text. As expected, prior knowledge had a significant influence on all outcome measures: recall, inferences, and learning. More interesting, however, was the finding that both comprehension measures, recall and inferences, were significant predictors of learning for the low-knowledge readers only (Diakidoy et al., 2011). This influence of memory (recall) on subsequent learning suggests the importance of constructing an accurate and well-connected textbase representation and implicates the possibility of online processing being devoted to its construction and retention in memory. A refutation text is more likely to facilitate this processing for several reasons. First, the co-presence of misconceptions and scientifically accepted conceptions in text alleviates the burden of activating existing yet incompatible knowledge which the reader of a corresponding expository text would have to either assume on his/her own or bypass entirely. Encountering familiar information in text ensures its fast and easy activation through a passive resonance process (Kendeou et al., 2014) explaining, in turn, the shorter fixations and reading times results for these segments (Ariasi & Mason, 2011; Kendeou & van den Broek, 2007). Second, the explicit contrast between misconceptions and scientifically accepted conceptions in clear favor of the latter may reduce the effort that the reader would have to devote to comparing conceptions with uncertain results, increase the plausibility and the perception of validity of the new conception (Chinn & Brewer, 1993; Dole & Sinatra, 1998), and direct attention and processing resources to the critical information (Ariasi & Mason, 2011). Finally, the explicit contrasts and the explanations in support of the scientific concept make for a far more cohesive and elaborated text likely to support the comprehension of readers who may need it the most (McNamara, Kintsch, Songer, & Kintsch, 1996). To summarize, prior research suggests that the refutation text effect may be associated with important changes in comprehension processes and outcomes that, in turn, impact learning from text in the case of incompatible prior knowledge. First, the differential pattern of processing times obtained for scientific information in relation to refutations and the overall negative influence of reading time on learning from text (Ariasi & Mason, 2011) is more consistent with the possibility of allocating attention to the distinct encoding of critical information than devoting time and resources to experiencing and resolving cognitive conflict online. If that is true, then readers of refutation text should manifest more processes associated with memory, like repeating and paraphrasing, and fewer processes indicative of elaboration and strategic processing, like inferences and conceptual change strategies, when compared to readers of expository text. Second, the contrasting findings regarding online and offline inference generation (Diakidoy et al., 2011; Kendeou & van den Broek, 2007), in conjunction with the elaborative retrieval hypothesis, raise the

possibility of the refutation text effect being more associated with offline reconstructive processes than online constructive processes. If that is the case, then one would expect differential effects of text on comprehension processes and outcomes, with comprehension outcomes being a stronger predictor of subsequent learning than online processes. 1.3. The present study: Questions and hypotheses The present study sought to provide further insight relative to the above issues by examining comprehension processes and outcomes in relation to each other and subsequent learning from text. The study employed a sample similar to that of previous studies (Ariasi & Mason, 2011; Braasch et al., 2013; Diakidoy et al., 2011; Kendeou & van den Broek, 2005; 2007) and the same textual and testing materials and scoring methodology as those used in the Diakidoy et al. (2011) study in order to examine the generalizability of previous comprehension and learning results and to allow for direct comparisons. Young adult readers read and recalled either a standard expository text about energy or a corresponding refutation text that explicitly acknowledged and refuted three potential misconceptions associated with the nature of energy (non-material entity), its conservation (non-producible), and its relation to force (different yet related processes) in addition to presenting the same expository text content. In this study, however, both texts were presented sentence by sentence and readers were instructed to think aloud after reading each sentence (similar to Kendeou & van den Broek, 2007). Think-aloud protocols were scored according to the types and the quality of the processes manifested (paraphrases, valid inferences, monitoring statements, invalid inferences and distortions), whereas recall protocols were scored in terms of amount of text recall, number of valid inferences, and distortions. Prior to the study, all readers were pretested on their knowledge about the concept of energy and the extent to which they adhered to the targeted misconceptions. The same pretest was used as a delayed posttest to measure any learning gains after reading and recalling the expository or the refutation text. This pre/posttest design was indicated by evidence showing differential influences of refutation text on comprehension processes, outcomes, and learning as a function of prior knowledge (e.g., Braasch et al., 2013; Diakidoy et al., 2011; Kendeou & van den Broek, 2007). By directly assessing the misconceptions addressed in the refutation text, the same test used as a posttest provided information regarding the extent to which student were able to modify these initial misconceptions as a result of reading. Thus, learning outcomes potentially involving restructuring of initial misconceptions can be distinguished more clearly from comprehension outcomes, such as text recall and spontaneous inference generation. The first issue addressed in this study concerned the nature of the online comprehension processes as a function of text type and prior knowledge. We hypothesized that readers with low and inaccurate prior knowledge when reading a refutation text would direct their attention more to encoding and retaining the scientific information than elaborating it on the basis of their prior knowledge when compared to readers with more accurate prior knowledge or readers reading the standard expository text. Therefore, we expected to observe more paraphrases and repetitions of text information than knowledge-based inferences in the think-aloud protocols obtained with refutation text when compared to those obtained with the corresponding expository text with the differences being more pronounced for readers with lower and inaccurate prior knowledge than readers with more and more accurate prior knowledge [Hypothesis 1]. Specifically, we reasoned that the lack of adequate prior knowledge and its tagging as inaccurate by the refutations in text would lead readers to focus more on

I.-A.N. Diakidoy et al. / Learning and Instruction 41 (2016) 60e69

processing explicit text information instead of inferentially elaborating it. If this is the case, then the differential patterns of reading times and fixations that have been obtained for refutation text segments (e.g., Ariasi & Mason, 2011; Kendeou & van den Broek, 2007) would be more attributable to memory-associated processes than conceptual change strategies being applied online. However, we were also interested in whether the refutation text would promote conceptual change strategies during reading e such as noticing and trying to resolve discrepancies e and in the extent to which low and inaccurate knowledge would contribute to distortions and invalid inferences (e.g., Kendeou & van den Broek, 2007). Therefore, we also examined any effects that text and prior knowledge might have on other online processes, like comprehension monitoring, conceptual change strategies, online distortions, and invalid inferences [Question 1]. The second issue addressed in this study concerned the relationship between online comprehension processes and offline comprehension outcomes as a function of text type. First, we hypothesized that if inferential comprehension outcomes are in part due to reconstructive processes operating at the time of retrieval, then online processes associated with encoding and retention as well as amount of text recall would be stronger predictors of inferences after reading than online inferencing [Hypothesis 2]. Second, in light of previous findings (e.g., Diakidoy et al., 2011; Kendeou & van den Broek, 2007), we also expected text to influence inferential comprehension outcomes more than online inferencing [Hypothesis 3]. Both of these hypotheses concern the timing of the refutation text effect (Sinatra & Broughton, 2011). According to current account, the refutation text exerts a direct influence on online processes involving comparison and contrast e that is, noticing and resolving discrepancies between prior knowledge and text information as a result of the co-activation of accurate and inaccurate conceptions in text (van den Broek & Kendeou, 2008). A confirmation of Hypotheses 2 and 3, however, would indicate the timing of the refutation text effect to be later, after reading is completed, and associated with reconstructive processes. In connection with Hypotheses 2 and 3, we also examined the possibility of differential effects of text on comprehension processes and outcomes associated with memory like paraphrasing and amount of recall [Question 2]. The third issue addressed in this study concerned the association of comprehension processes and outcomes to learning outcomes as a function of text. Given previous findings (e.g., Diakidoy et al., 2011), we hypothesized increased learning as a result of reading the refutation text when compared to the corresponding expository text [Hypothesis 4]. However, an association of the refutation text effect with postereading reconstructive processes would also indicate this learning to be in part attributable to the reactivation and reprocessing of the mental representation of the text that the readers had initially formed (Carpenter, 2009; Karpicke, 2012). Therefore, in line with the elaborative retrieval hypothesis, we also expected any subsequent learning gains to be more associated with comprehension outcomes than online processes [Hypothesis 5]. 2. Method 2.1. Participants The sample included 68 undergraduate students majoring in psychology (54 juniors and 15 seniors). The majority of the participants were females (n ¼ 62) e reflecting the typical female/male ratio of the student enrollment in this major at the University of Cyprus. Their participation in the study was voluntary in exchange for extra credit in an introductory educational psychology course.

63

The study's goals and general method appeared on the Psychology Department's secured and monitored research webpage and platform where students can select and sign up for any study they wish to participate. Students were free to withdraw their participation at any time, and study participation was only one of several methods for obtaining extra credit in any course that offered this option. All participants of this study were debriefed in detail after the completion of the study. About half of the students (n ¼ 33) were randomly assigned to read an expository text about energy, while the rest (n ¼ 35) were assigned to read a corresponding refutation text on the same topic. 2.2. Energy knowledge test The test developed by Diakidoy et al. (2011) and its scoring template were employed in this study in order to assess prior knowledge about energy before reading (pretest) and subsequent learning (posttest). The test included 28 multi-format items assessing each of three potential misconceptions. Specifically, there were two short-answer questions, 15 forced-choice items, and 11 forced-choice items followed by a written response justification question probing the extent to which students conceptualize energy as a material entity that can be stored or wasted (conservation) or as the same as force (vector vs. scalar quantities) (for more detailed test description see Diakidoy et al., 2011). The two versions of the test with the items appearing in different orders were counterbalanced between participants. The scoring template for this test included a compilation of all possible correct responses to each test item previously provided by experts in the domain and a compilation of all possible incorrect responses obtained from pilot studies and past research (see Diakidoy et al., 2011). This scoring template was used in this study for evaluating the performance of students at the pretest and the posttest. 2.3. Experimental texts The multi-section expository text on energy (832 words) and its corresponding refutation text (1092 words) from the Diakidoy et al. (2011) study were used in order to allow for direct comparisons. Both texts included four headed sections describing energy, its transformations, its relation to force, and the principle of energy conservation. In addition to the main content, the refutation version of the text contained additional information refuting three potential misconceptions. The first section distinguished energy from matter and material properties, the third section refuted notions of energy being produced out of nothing and then wasted, and the fourth section distinguished energy from force (see Diakidoy et al., 2011). 2.4. Procedure The study was completed in three sessions (see Table 1 for an overview). Session 1 lasted 50 min and involved the group administration of the Energy Knowledge Test. Participants met with three experimenters in a large classroom. They were briefed as to the general goal of the study and the specific goal of Session 1. Then the Energy Knowledge Test was distributed and participants were asked to read the printed instructions on the cover page. The instructions directed participants to read and consider carefully each test item, to mark the response they considered correct for each forced choice items, and to elaborate as much as they could on their written responses to the open-ended and explanation items. The instructions were also elaborated orally by one of the experimenters.

64

I.-A.N. Diakidoy et al. / Learning and Instruction 41 (2016) 60e69

Table 1 Procedure overview and tasks. Session 1 (group meeting lasting 50 minutes): - Briefing - Energy knowledge pretest - Prior knowledge and belief questionnaire Session 2 (after 4 weeks, individual meetings lasting 120 minutes): - Practice reading and thinking aloud - Reading and thinking aloud with assigned experimental text - Filler task - Cued recall task Session 2 (after 2 weeks, group meeting lasting 50 minutes): - Energy knowledge posttest - Debriefing

Session 2 took place approximately one month later in order to minimize, to the extent possible, memory of specific pretest item content from Session 1. Session 2 took about a week to be completed as it involved individual meetings of all students with an experimenter in small conference/interview rooms. Each meeting lasted about 120 min and involved reading, thinking aloud, and completing a cued recall task. Before reading the experimental texts, students practiced thinking aloud with a short text (432 words) on a different scientific topic (Star nursery). Subsequently, students received their assigned text in a booklet with each sentence printed on a different page. They were instructed to read for understanding and learning and to verbalize any thoughts that came to their mind as they read each sentence. They were also informed that they could not go back to re-read previous sentences. The experimenter used general prompts (“any thoughts?” “can you say more”) to ensure a verbal report for each text sentence by each participant. All verbalizations were tape recorded. After reading and thinking aloud, students completed a filler questionnaire about their learning and studying strategies before proceeding with the cued recall task. The purpose of the filler task was to minimize any short-term-memory associated serial position effects in recall. For the cued recall task, students were given a booklet with several blank pages and with the experimental text section headings printed on top of every other page. Participants were instructed to write down all they could remember from each text section using the heading as a cue. Session 3 took place two weeks later and involved the group administration of the Energy Knowledge test as a delayed posttest. The procedure, the instructions, and the location were the same as those in Session 1 (Table 1).

2.5. Scoring 2.5.1. Energy knowledge pre- and posttest All test items were scored in a trichotomous way with correct responses to the forced-choice items receiving a score of þ1, incorrect responses receiving a score of 1, and don't know responses receiving a score of 0 (see Diakidoy et al., 2011). Openended items were parsed into clauses and scored similarly on the basis of the scoring template. Clauses that corresponded to the gist of the experts' responses received a score of þ1 and clauses representing incorrect ideas received a score of 1. The scores received across test items were summed up yielding a composite energy knowledge score for each participant that could range from negative to positive values, with very low or negative total scores reflecting the presence of incorrect knowledge. The test's reliability with this sample was moderately high (Cronbach's a ¼ 0.78, Hotelling's T2 ¼ 1476.90, p ¼ .000).

2.5.2. Cued recall protocols Students' recall protocols were parsed into clauses corresponding to idea units by the second author and an independent rater (89% agreement, Cohen's K ¼ 0.78, p ¼ .000). Subsequently, the same raters independently evaluated whether each clause represented verbatim recall or a correct paraphrase of text information, a valid inference connecting or extending text information, or an invalid inference or distortion. Interrater agreement was 87% (Cohen's K ¼ 0.73, p ¼ .000). Between-rater differences at each scoring level separately were identified and resolved in conference with the raters and the first and the fourth author (an expert science educator). The scoring of the recall protocols yielded an overall recall score reflecting the proportion of text idea units recalled correctly. In the case of the refutation text, the overall recall score reflected only the recall of the information that was common across text types which represented the to-be-learned information. Finally, the total number of valid inferences in recall, and the total number of distortions and invalid inferences were also calculated for each participant. 2.5.3. Think aloud protocols Two independent raters parsed all transcribed think-aloud protocols into clauses roughly corresponding to idea units that could stand alone (84% agreement, Cohen's K ¼ .63, p ¼ .00). Subsequently, the third author and an independent rater coded each clause in waves, identifying, first, all clauses that represented text information verbatim or in terms of gist as repetitions and paraphrases. Paraphrases of explicit text information were further categorized into correct paraphrases and distortions. Subsequently, all other clauses were coded into categories adapted from previous research (Kendeou & van den Broek, 2007; McCrudden, 2012; Wolfe & Goldman, 2005). First, utterances reflecting inferences were identified and classified as valid inferences, invalid inferences, and irrelevant associations, while expressions of understanding or non-understanding, evaluations, conceptual change strategies, and questions were classified as monitoring statements. The decision to collapse all these kinds of utterances into the monitoring category was based on their very low individual frequencies relative to the total utterances in the protocol, posing normality problems. It is important to note, that conceptual change strategies, in particular, were too few (less than 1% of all monitoring utterances) to warrant their coding as a separate category (see Table 2 for examples). The categories of distortions (incorrect paraphrases) and invalid inferences were also combined into a single category for conceptual and statistical reasons: Although the distinction between correct and incorrect utterances was straightforward, the degree to which a student was adding new but incorrect information to existing text was not clear. The low frequencies of the separate categories and the fact that they both reflected misunderstanding contributed to our decision to collapse them into a single category. Interrater agreement was 95% (Cohen's K ¼ 0.91, p ¼ .000). Differences between raters at each level of coding were resolved in conference with the raters and three of the authors. The coding of the think-aloud protocols yielded scores representing the proportion of utterances classified in each category of processes for all text parts that were common between refutation and expository text versions. Finally, irrelevant associations were excluded from all analyses. 3. Results Preliminary analyses for all comprehension process variables indicated that they were normally distributed (skewness < 1) with homogeneous variances across Texts except for the combined Monitoring statements (Levene's F ¼ 6.40, p ¼ .014). Therefore, this

I.-A.N. Diakidoy et al. / Learning and Instruction 41 (2016) 60e69

65

Table 2 Examples of categories in think-aloud protocols. Categories

Examples

Paraphrases/repetitions (47%) Valid inferences (26%) Associations (<2%) Comprehension monitoring (17%)

Basically, it says that when we have a change in one part of the system, there is a change in the whole system (47-expo) So it is the transformation of kinetic energy to thermal energy that causes the balls to stop (28-ref) Solar energy comes to my mind (5-ref) OK, I get the example (38-ref) In my opinion, energy is not at all an abstract concept (9-expo) System e what system? (19-expo) So energy and force are not the same thing? (4-ref) Energy has a force that makes things happen (48-expo)

Distortions (10%)

Note. Percentages, participant number, and text assigned are shown in parentheses.

dependent variable was either excluded or entered into nonparametric analyses. Similar preliminary analyses for comprehension outcome variables indicated deviations from normality for the Number of Distortions and Valid Inferences in recall (skewness > 1.5). Both variables were normalized with a square root transformation. Finally, Pretest and Posttest Scores were also normally distributed with homogenbeous variances across Texts. Overall, Energy Pretest scores ranged from 16 (min.) to þ 45 (max.) with 52% of the sample having very low and inaccurate knowledge regarding the targeted aspects of energy concept (M ¼ 8.50, SD ¼ 11.84, Median ¼ 5.00). As it can be seen from Table 2, the Refutation Text group scored lower on the Energy Knowledge Pretest. However, this difference was not significant (t (67) ¼ 0.81, p ¼ .422). The correlation coefficients between all process and outcome measures are shown in Table 3. As expected, all corresponding comprehension process and outcome measures were positively correlated to each other, while the correlations between Distortions during and after reading and outcome measures were negative. Learning from text correlated positively with prior knowledge, memory-associated and inferential comprehension outcomes, and online inferencing, but not paraphrasing. Notably, however, paraphrasing during reading was negatively associated with online inferencing but positively with text recall (Table 3). 3.1. Effects on comprehension processes Our first hypothesis concerned the influence of text and prior knowledge on online comprehension processes. As expected, an overall higher proportion of think-aloud utterances reflected repetitions and paraphrases of text segments than online inferences regardless of text (Table 4). In order to examine the extent to which differences were influenced by Text or Prior Knowledge, MANCOVA with Text as the independent variable and Energy Pretest as the covariate was performed. Contrary to expectations, only Energy Pretest had a significant multivariate effect (Hotelling's T ¼ 15.73, F (2, 63) ¼ 495.45, p ¼ .000). Between-subject tests indicated that Energy Pretest had a significant main effect on Proportion of Valid

Table 3 Correlation coefficients between comprehension process, comprehension outcome, and learning measures. Measure

2

3

4

5

6

7

8

1. 2. 3. 4. 5. 6. 7. 8.

.77** e

.19 .33** e

.36** .55** .49** e

.14 .26* .03 .14 e

.16 .15 .30* .06 .27* e

.45** .50** .22 .41** .04 .55** e

.16 .29* .35** .28* .44** .33** .14 e

En. pretest En. posttest Recall Valid inferences Distortions Paraphrasing Inferencing Distorting

*p < .05, **p < .01.

Table 4 Means of all comprehension and knowledge measures as a function of text type. Expository text (n ¼ 33) Knowledge/Learning Energy pretest 9.70 Energy posttest 14.09 Comprehension outcomes Text recalla .15 Valid inferencesb 4.52 Distortionsb 3.70 Comprehension processes Repetitions/paraphrasesa .45 Valid inferencesa .26 a Distortions .11 a .17 Monitoring statements

Refutation text (n ¼ 35)

(11.39) (11.54)

7.37 (12.30) 15.14 (11.82)

(.10) (3.06) (3.53)

.16 (.10) 7.46 (4.09) 4.11 (4.40)

(.19) (.13) (.09) (.16)

.50 .24 .09 .17

(.17) (.12) (.07) (.10)

Note. Standard deviations are shown in parentheses. a Proportion scores. b Original, untransformed scores.

Inferences during reading (F (1, 67) ¼ 16.55, p ¼ .000), with more knowledgeable readers generating a higher proportion of inferences during reading than less knowledgeable readers regardless of Text (M ¼ .30, SD ¼ .14 and M ¼ .21, SD ¼ .10 respectively, d ¼ 0.74). Although readers of the refutation text engaged in more paraphrasing than readers of expository text, neither the main effect of Text nor its interaction with Energy Pretest were significant (p > .05). Therefore the predictions of Hypothesis 1 were partially confirmed: less knowledgeable readers focused more on encoding than inferencing during reading. Although a similar trend was manifested in connection with refutation text, neither the text differences nor the interaction with prior knowledge were significant. In connection to Hypothesis 1, we were also interested in the extent to which prior knowledge and text would influence other online processes [Question 1]. ANCOVA for Proportion of Distortions (incorrect paraphrases and invalid inferences) during reading indicated that neither Energy Pretest nor Text had any significant effects (p > .05). Finally, a ManneWhitney U test indicated that the proportion of Monitoring Statements was similarly uninfluenced by Text (p > .05) and uncorrelated with Energy Pretest scores (rs ¼ .12, p ¼ .328). Therefore, other online processes remained uninfluenced by prior knowledge and text structure. 3.2. Effects on comprehension outcomes The second and third hypotheses concerned the influence of online processes and text on comprehension outcomes. MANCOVA for Recall and Number of Valid Inferences in recall indicated significant multivariate effects for both Energy Pretest (Hotelling's T ¼ 0.24, F (2, 63) ¼ 7.55, p ¼ .001) and Text Type (Hotelling's T ¼ 0.20, F (2, 63) ¼ 6.38, p ¼ .003). However, both variables' main effects were on the Number of Valid Inferences in recall (F (1, 67) ¼ 15.20, p ¼ .000 and F (1, 67) ¼ 10.92, p ¼ .002, respectively). It

66

I.-A.N. Diakidoy et al. / Learning and Instruction 41 (2016) 60e69

can be seen from Table 4, that readers of refutation generated more inferences in their recalls than readers of expository text (d ¼ 0.81). Similarly, more knowledgeable readers included more inferences in their recalls (M ¼ 7.30, SD ¼ 4.68) than less knowledgeable readers (M ¼ 4.83, SD ¼ 2.49, d ¼ 0.66). In contrast, the effects of Energy Pretest and Text on Recall as well as their interaction were not significant (all p > .05). Finally, ANCOVA for Distortions indicated that neither Energy Pretest nor Text had any significant main or interactive effects on the Number of Distortions in recall (all p > .05). In order to examine any differential patterns of influence on comprehension processes and outcomes, a series of MANCOVA's, one for each corresponding pair of online and offline measures, was performed. The analyses for the memory-associated measures (repeating/paraphrasing and recall), which were the focus of Question 2, indicated no significant effects of either Energy Pretest or Text Type (all p > .05). Although readers of refutation text engaged in more paraphrasing than readers of expository text, this difference was not large enough to reach significance (F (1, 67) ¼ 2.97, p ¼ .089). Similar non-significant effects were obtained for online and offline Distortions (all p > .05). In contrast, the main effects of Energy Pretest and Text were significant for inferencing. Specifically, prior knowledge significantly influenced the amount of inferences generated during reading and in recall (F (1, 67) ¼ 16.55, p ¼ .000, and F (1, 67) ¼ 15.20, p ¼ .000, respectively), with more knowledgeable readers generating also a higher proportion of inferences during reading (M ¼ .30, SD ¼ .14) than less knowledgeable readers (M ¼ .21, SD ¼ .10, d ¼ 0.74) in addition to the higher Number of Inferences manifested in their recalls. Therefore, the results obtained with pairs of online and offline measures are consistent with those obtained previously for online processes only. More importantly, however, Text which had a significant effect on the Number of Inferences in recall, was not found to influence significantly the Proportion of Inferences generated during reading (F (1, 67) ¼ 0.22, p ¼ .643, see also Table 4). Therefore, consistent with our third hypothesis, refutation text contributed positively to inferential comprehension outcomes only. However, according to Hypothesis 2, online memory-associated processes and text recall were predicted to contribute to inferential comprehension outcomes. In order to examine, first, more closely the contribution of online comprehension processes to comprehension outcomes, hierarchical regression analyses were employed for each outcome of interest: Recall and Number of Valid Inferences. It can be seen from Table 5, that repeating, paraphrasing and inferencing during reading contributed significantly to better memory for text content. In contrast, prior knowledge was not a significant predictor of Recall. A different and more complex picture emerged when the analyses were performed for Number of Valid Inferences in recall which was also the outcome measure influenced most by Text. It can be seen from Table 6, that the significant contributions of Energy Pretest and Online Inferences at the time of

Table 5 Hierarchical multiple regression analyses predicting text recall (N ¼ 68). Predictor

DR2

Model 1 Energy pretest Model 2 Energy pretest Repetitions/paraphrases Online inferences Online distortions Total R2

.04

**p < .01.

b

t

p

.19

1.59

.116

.03 .53 .51 .08

0.27 3.73 3.48 0.68

.789 .000 .001 .497

.27**

.31**

Table 6 Hierarchical multiple regression analyses predicting number of valid inferences in recall (N ¼ 68). Predictor

DR2

Model 1 Energy pretest Model 2 Energy pretest Repetitions/paraphrases Online inferences Online distortions Model 3 Energy pretest Repetitions/paraphrases Online inferences Online distortions Text recall Total R2

.13**

b

t

p

.36

3.15

.002

.19 .12 .35 -.16

1.56 0.84 2.30 1.28

.123 .404 .024 .206

.18 -.09 .15 -.13 .40

1.57 0.57 0.96 1.09 3.26

.122 .571 .342 .281 .002

.12*

.11**

.36**

*p < .05, **p < .01.

entry are taken up by Recall in the final model accounting for about 10% of the variance associated with Number of Valid Inferences in recall. The same hierarchical regressions analyses for each Text separately indicated no divergence from these patterns of prediction. These results confirmed our expectation regarding the contribution of text memory on inference generation after reading [Hypothesis 2]. However, they also indicated the primary target of the influence of online processes to be memory for text. 3.3. Effects on learning Our fourth hypothesis concerned the influence of refutation text in subsequent learning. Repeated Measures analyses with Time (Energy Pretest vs. Posttest) as the within-subject factor and Text as the between-subject factor indicated a significant effect of Time only (Hotelling's T ¼ 0.61, F (1, 66) ¼ 40.19, p ¼ .000). Although readers learned more from refutation text (see Table 4), the Time  Text interaction did not reach significance (Hotelling's T ¼ 0.05, F (1, 66) ¼ 3.10, p ¼ .083). Therefore, contrary to our prediction, the positive influence of refutation text was not strong enough to reach significance. In order to examine the contribution of comprehension processes and outcomes to learning from text [Hypothesis 5], Hierarchical Regression Analyses for Energy Posttest were performed. It can be noted from Table 7, that all together, prior knowledge along with comprehension processes and outcomes, accounted for about

Table 7 Hierarchical multiple regression analyses predicting energy posttest score (N ¼ 67). Predictor

DR2

Model 1 Energy pretest Model 2 Energy pretest Repetitions/paraphrases Online inferences Online distortions Model 3 Energy pretest Repetitions/paraphrases Online inferences Online distortions Text recall Offline inferences Offline distortions Total R2

.59**

*p < .05, **p < .01.

b

t

p

.77

9.67

.000

.66 -.03 .14 -.20

7.76 0.26 1.35 2.28

.000 .796 .183 .025

.61 -.12 .01 -.11 .09 .22 -.12

7.42 1.12 0.13 1.26 0.99 2.53 1.39

.000 .267 .901 .214 .325 .014 .169

.06*

.06*

.71**

I.-A.N. Diakidoy et al. / Learning and Instruction 41 (2016) 60e69

70% of the variance in learning from text. As expected, Energy Pretest was a consistently significant predictor of learning with comprehension processes and outcomes accounting for equal amounts of variance in Energy Posttest Score (Table 7). However, the only online comprehension process that was a significant but negative predictor of subsequent learning was Online Distortions when entered prior to comprehension outcomes. Readers whose think-aloud protocols included a higher proportion of incorrect paraphrases and invalid inferences learned less than readers whose protocols had fewer distortions. In contrast, in the final model, after the entry of comprehension outcomes, the only significant and positive predictors of learning from text were prior knowledge and the Number of Valid Inferences generated in recall (Table 7). Therefore, consistent with the general prediction of the fifth hypothesis, subsequent learning from text was associated more with comprehension outcomes than comprehension processes. However, it is noteworthy, that offline inference generation emerged as the sole significant predictor of learning after accounting for the contribution of all other outcomes and processes. However, when the regression analyses were repeated for each Text separately, a different pattern of results emerged. It can be seen from Table 8, that both Energy Pretest Score and Offline Distortions are significant predictors of learning from expository text, with Distortions having an expected negative contribution. In contrast, memory-associated processes and outcomes e Online Paraphrasing and Text Recall e reach marginal significance, with Paraphrasing being a weak but negative predictor of Energy Posttest scores. In contrast, Energy Pretest Score is the only significant and positive predictor of learning from refutation text (Table 8). Although these results are consistent with our predictions regarding the timing of the refutation text effect (Hypotheses 2, 3, and 5), they do indicate that the nature of the effect may be more associated with offline distortions instead of offline inferences.

4. Discussion The primary aim of the study was to compare comprehension processes and outcomes and their contribution to learning about a science concept from corresponding refutation and expository texts. The findings indicated that although both paraphrasing and online inferencing were associated with higher text recall, neither contributed directly to inferences generated after reading. Recall, however, was positively associated with offline inferences which, in

Table 8 Hierarchical multiple regression analyses predicting energy posttest score by text. Predictor Expository text (n ¼ 33) Energy pretest Repetitions/paraphrases Online inferences Online distortions Text recall Offline inferences Offline distortions Total R2 Refutation text (n ¼ 35) Energy pretest Repetitions/paraphrases Online inferences Online distortions Text recall Offline inferences Offline distortions Total R2

R2

b

t

p

.65 -.31 -.14 -.05 .31 .11 -.31

5.44 2.07 0.95 0.45 2.02 0.80 2.35

.000 .049 .349 .658 .054 .431 .027

.58 .05 .23 -.10 -.02 .21 -.01

4.63 0.29 1.26 0.67 0.17 1.39 0.08

.000 .773 .218 .510 .865 .177 .936

.75**

.72**

Note: Final models shown only; **p < .01.

67

turn, contributed to overall learning regardless of text. Although prior knowledge did not influence any memory-associated processes and outcomes, it did contribute positively to both inferencing and learning. Finally, refutation text recalls included more valid inferences than expository text recalls. However, the refutation text effect on subsequent learning was associated more with reducing the negative influence of distortions. 4.1. Comprehension processes Our first hypothesis regarding the potential influence of prior knowledge and text on comprehension processes was only partially supported. Specifically, we had predicted less knowledgeable readers of refutation text to engage in processing that supports encoding and retention more than elaboration when compared to either more knowledgeable readers or readers of the expository text. Although there was a tendency for refutation text readers to engage in more memory-associated processes, like repeating and paraphrasing, the differences between text groups and the interaction between text and prior knowledge were not significant. Nevertheless, students with higher and more accurate prior knowledge engaged in more inferential elaboration during the reading of either text than less knowledgeable students. In contrast, readers with low and inaccurate knowledge engaged in more paraphrasing of explicit text content than inferencing. These results reinforce the role of prior knowledge in integrating and elaborating text information (e.g., McNamara et al., 1996), and indicate that an inadequate knowledge base elicits more local and memory et al., 1998; Pentinnen et al., 2013). associated processing (Cote However, these results are only partly consistent with previous findings that have shown prior knowledge to affect primarily the content of the comprehension processes but not the processes themselves (e.g., Kendeou & van den Broek, 2007). Lessknowledgeable readers in our sample did not generate more distortions and invalid inferences during reading when compared to their more-knowledgeable peers. Moreover, in contrast with previous findings (Kendeou & van den Broek, 2007; McCrudden, 2012; Sanchez & Garcia-Rodicio, 2013), the think-aloud protocols contained very few monitoring utterances that could be clearly classified as indicative of conceptual change strategies. Given the range and distribution of the pretest scores, this result cannot be attributed to lack of misconceptions in the sample. It is possible, however, to attribute it to the fact that the more naturalistic text that we employed did not provide similarly explicit refutations across the three targeted misconceptions related to energy (Diakidoy et al., 2011) inviting, thus, fewer utterances addressing the contrasts between accurate conceptions and misconceptions. The combination of a naturalistic text with a sentence-level think-aloud requirement may have prevented readers from immediately noticing and expressing a discrepancy with their prior knowledge. Nevertheless, the fact that the overall comprehension monitoring utterances were relatively few and mostly laconic, and that online distortions were also few regardless of prior knowledge is also suggestive of our readers adopting a more conservative strategy in how they approached the think-aloud task. In general, they seemed to avoid expressing thoughts that were not directly related to content or elaborations for which they were not certain about their validity. 4.2. Comprehension processes and outcomes Our second hypothesis concerning the contribution of online comprehension processes to comprehension outcomes was partly supported. Consistent with the elaborative retrieval hypothesis (Carpenter, 2009; Karpicke, 2012), we found recall to be the only

68

I.-A.N. Diakidoy et al. / Learning and Instruction 41 (2016) 60e69

significant and positive predictor of offline inferences. However, online processes associated with encoding and retention contributed only to later recall, the sole predictor of inferences after reading. Therefore, the overall picture presented by these results is that paraphrasing of text information reinforces its retention, which, in turn, supports further inferences when this mentally represented information is reactivated in response to a postereading task. The fact that online inferencing was more directly and positively associated with recall than inferential outcomes appears to indicate that our students engaged in within-text processing, focusing more on encoding and connecting text concepts and parts than extend et al., 1998; ing and integrating with their prior knowledge (Cote McNamara & Magliano, 2009; Wolfe & Goldman, 2005). This possibility is further reinforced by the finding that memory and not prior knowledge was the sole predictor of inference generation after reading, suggesting that students either did not have enough prior knowledge to support elaboration or (strategically) avoided activating and relying on it for inferential purposes. In fact, withintext processing to establish coherence between the new information may be more adaptive and productive in the case of low and inaccurate prior knowledge as it keeps out intrusions and elaborations that may be unwarranted and eventually detrimental to comprehension and learning (Kendeou & van den Broek, 2007; Penttinen et al., 2013). 4.3. Refutation text, comprehension, and learning Consistent with our third hypothesis and previous findings (Diakidoy et al., 2011), refutation text recalls included more valid inferences than expository text recalls. However, similar positive effects of refutation text were not manifested for any of the comprehension processes or text recall. This lack of influence of text on comprehension processes is partly consistent with the Kendeou and van den Broek (2007) results but not with those of Ariasi and Mason (2011). Their finding of longer total and secondpass fixations on critical text information in refutation text would lead one to expect this increase in processing time to also involve more elaboration efforts as well. Considering, however, the Ariasi and Mason (2011) findings together with the present results, one could also hypothesize that this extra processing time e especially during second-pass and look-back fixations e is devoted primarily to coherence-building efforts than elaboration or discrepancy resolution. Nevertheless, the differential pattern of results regarding online and offline inferences suggest the timing of the refutation text effect to be later, after reading is completed, than earlier, during a first encounter with new information in text. The fact that refutation text, along with recall, facilitated offline instead of online inferencing or comprehension monitoring suggests the possibility of refutations influencing not so much the amount of information and connections represented, but the nature and the quality of the connections formed. Although textbased coherence-building processing tends to be more associated with the formation of primarily local connections of relatedness  et al., 1998; Kintsch, 1988), it is also reasonable to expect (Cote refutations to draw attention to contrasts and differences, increasing, thereby, the distinctiveness of the critical but knowledge-inconsistent information. Research has shown that processing of concept-specific differences and unique features increases the distinctiveness of a concept within a conceptual network and facilitates its memory (e.g., Hunt, 2003; Rawson & Van Overschelde, 2008). Therefore, in the case of refutation text, refutations (especially the more explicit ones) may support the encoding of differences, which, in turn, become available to support further processing at retrieval.

Contrary to our expectations [Hypothesis 4], learning gains from both texts were significant but comparable. Moreover, and in line with our fifth hypothesis, only inferential comprehension outcomes along with prior knowledge contributed to overall learning regardless of text. However, the results obtained for each text separately presented a more complex picture. Invalid inferences and distortions at recall contributed negatively to learning, but significantly so only in the case of expository text. In fact, the detrimental influence of these distortions was neutralized to the point of extinction in the case of the refutation text. This finding has implications regarding the nature of the refutation text effect, suggesting its target to be the weakening of the negative effects of misconceptions instead of their change. The comparison of the prediction patterns obtained for each text separately indicated changes in the magnitude and the nature of the contribution of different online processes to learning as a function of text. These differential patterns in conjunction with the lack of influence of refutation text on amount of distortions online or in recall appear to suggest underlying qualitative differences in how the same information was encoded and inter-connected when first encountered in the different contexts that expository and refutation texts provide. This account is in line with previous findings showing that only refutations that explicitly distinguish accurate from inaccurate conceptions to positively impact learning (Braasch et al., 2013), and provide an explanation for why refutations may have no impact on amount of recall (Diakidoy et al., 2011; Kendeou & van den Broek, 2007). The present findings, however, further suggest that the explicit contrasts that refutations provide facilitate learning by ensuring the retention of accurate conceptions as distinct from any other (mis) conceptions, preventing, thereby, their intrusion in later comprehension and learning tasks. This possibility is supported by evidence obtained with functional magnetic resonance imaging (fMRI) indicating that physics experts activate brain areas involved in inhibition when evaluating a misconception-related problem (Masson, Potvin, Riopel, & Foisy, 2014). In contrast, novices having the targeted misconceptions did not manifest a similar inhibitionrelated activation (Masson et al., 2014). These contrasting patterns of activation suggest that experts may still have misconceptions but they are also better able to inhibit their intrusion, possibly due to the scientific conceptions' stronger and more distinctive memory trace. This in conjunction with our behavioral, distortion-related data has important implications regarding our understanding of conceptual change, pointing more in the direction of coexistence than replacement of misconceptions. 4.4. Limitations, implications, and conclusions At this point, the present findings and their implications are subject to qualification in light of the limitations of the study. A study-specific limitation concerns the fact that the refutation text we employed did not provide a similarly explicit refutational treatment across the three misconceptions that it addressed (see Diakidoy et al., 2011). This may have served to reduce the differences in the learning benefits obtained across text groups and the manifestation of conceptual change strategies in the think-aloud protocols. A further methodological limitation relates to the think-aloud task. Although the possibility of this task disrupting the normal reading process has been acknowledged (Penttinen et al., 2013), the amount and kind of utterances in our think-aloud protocols in conjunction with the amount of prompting that was needed to obtain them raise questions regarding not only readers' ability to verbalize automatic processes but also their inclination to do so in an experimental, learning context. It is also possible that a modified think-aloud task, in which the verbalization level is a

I.-A.N. Diakidoy et al. / Learning and Instruction 41 (2016) 60e69

larger text segment than the sentence may be more conducive for unveiling any integrative and conceptual change processes. Therefore, further research employing more controlled but extended texts and multiple on-line measures is needed to further examine the possibility of the refutation text effect being associated with the elaborative retrieval and distinctiveness hypotheses. Nevertheless, the intriguing implication of these findings is that refutation text may have more of an impact on the kind of conceptual connections formed during reading and on reconstructive elaboration after reading, than increased processing to encode and retain, to elaborate more deeply, or to strategically attempt to reconcile discrepancies online for integration purposes (Kendeou et al., 2014; Sinatra & Broughton, 2011; van den Broek & Kendeou, 2008). Although our findings do not preclude the possibility of both deeper elaboration and strategic processing, they do raise questions as to their timing e suggesting that both elaboration and discrepancy resolution seem more likely to be occurring later, when the newlyacquired information is given a chance to be re-processed and used, than on first encountering it. This, in turn, highlights the importance of the quality of the text-based representation that a reader forms or is supported to form. If this is indeed the case, then the implications extend beyond the refutation text to all learning from text, as they point to textbase construction as a way to compensate for and eventually overcome the potential deficits of lack of prior knowledge. But in the case of conceptual change learning, our distortion related findings point more in the direction of changes in the influence of prior inaccurate conceptions in subsequent learning than immediate changes in the initial conceptions themselves. This, in conjunction with the positive effect on elaboration at the time of recall suggests that the initial impression of refutation text may not be initially powerful but it is a lasting one. References Ariasi, N., & Mason, L. (2011). Uncovering the effect of text structure in learning from science text: an eye-tracking study. Instructional Science, 39, 581e601. http://dx.doi.org/10.1007/s11251-010-9142-5. Braasch, J. L. G., Goldman, S. R., & Wiley, J. (2013). The influences of text and reader characteristics on learning from refutations in science texts. Journal of Educational Psychology, 105(3), 561e578. http://dx.doi.org/10.1037/a0032627. Broughton, S. H., Sinatra, G. M., & Reynolds, R. E. (2010). The nature of the refutation text effect: an investigation of attention allocation. The Journal of Educational Research, 103(6), 407e423. Carpenter, S. K. (2009). Cue strength as a moderator of the testing effect: the benefits of elaborative retrieval. Journal of Experimental Psychology: Learning, Memory, and Cognition, 35, 1563e1569. http://dx.doi.org/10.1037/a0017021. Chinn, C. A., & Brewer, W. F. (1993). The role of anomalous data in knowledge acquisition: a theoretical framework and implications for science instruction. Review of Educational Research, 63, 1e49. , N., Goldman, S. R., & Saul, E. U. (1998). Students making sense of informational text: Cote relations between processing and representation. Discourse Processes, 25, 1e53. Diakidoy, I. N., Kendeou, P., & Ioannides, C. (2003). Reading about energy: the effects of text structure in science learning and conceptual change. Contemporary Educational Psychology, 28, 335e356. Diakidoy, I. N., Mouskounti, T., & Ioannides, C. (2011). Comprehension and learning from refutation and expository texts. Reading Research Quarterly, 46, 22e38. http://dx.doi.org/10.1598/RRQ.46.1.2. Dole, J. A., & Sinatra, G. M. (1998). Reconceptualizing change in the cognitive

69

construction of knowledge. Educational Psychologist, 33, 109e128. Hinze, S. R., Wiley, J., & Pellegrino, J. W. (2013). The importance of constructive comprehension processes in learning from tests. Journal of Memory and Language, 69, 151e164. http://dx.doi.org/10.1016/j.jml.2013.03.002. Hunt, R. R. (2003). Two contributions of distinctive processing to accurate memory. Journal of Memory and Language, 48, 811e825. http://dx.doi.org/10.1016/S0749596X(03)00018-4. Hynd, C., & Guzzetti, B. J. (1998). When knowledge contradicts intuition: conceptual change. In C. Hynd (Ed.), Learning from text across conceptual domains (pp. 139e163). Mahwah, NJ: Erlbaum. Karpicke, J. D. (2012). Retrieval-based learning: active retrieval promotes meaningful learning. Current Directions in Psychological Science, 21(3), 157e163. http://dx.doi.org/10.1177/0963721412443552. Kendeou, P., Muis, K. R., & Fulton, S. (2011). Reader and text factors in reading comprehension processes. Journal of Research in Reading, 34, 365e383. http:// dx.doi.org/10.1111/j.1467.9817.2010.01436.x. Kendeou, P., & van den Broek, P. (2005). The effects of readers' misconceptions on comprehension of scientific text. Journal of Educational Psychology, 97, 235e245. Kendeou, P., & van den Broek, P. (2007). The effects of prior knowledge and text structure on comprehension processes during reading of scientific texts. Memory and Cognition, 35, 1567e1577. Kendeou, P., Walsh, E. K., Smith, E. R., & O'Brien, E. J. (2014). Knowledge revision processes in refutation texts. Discourse Processes, 51, 374e397. http://dx.doi.org/ 10.1080/0163853X.2014.913961. Kintsch, W. (1988). The role of knowledge in discourse comprehension: a construction e integration model. Psychological Review, 95, 163e182. Mason, L., Gava, M., & Boldrin, A. (2008). On warm conceptual change: the interplay of text, epistemological beliefs, and topic interest. Journal of Educational Psychology, 100, 291e309. http://dx.doi.org/10.1037/0022-0663.100.2.291. Masson, S., Potvin, P., Riopel, M., & Foisy, L. B. (2014). Differences in brain activation between novices and experts in science during a task involving a common misconception in electricity. Mind, Brain & Education, 8(1), 44e55. McCrudden, M. T. (2012). Readers use of online discrepancy resolution strategies. Discourse Processes, 49(2), 107e136. http://dx.doi.org/10.1080/ 0163853X.2011.647618. McNamara, D. S., Kintsch, E., Songer, N. B., & Kintsch, W. (1996). Are good texts always better? Interactions of text coherence, background knowledge, and levels of understanding in learning from text. Cognition and Instruction, 14, 1e43. McNamara, D. S., & Magliano, J. (2009). Toward a comprehensive model of comprehension. The Psychology of Learning and Motivation, 51, 297e384. http:// dx.doi.org/10.1016/S0079-7421(09)51009-2. €-Erdmann, M. (2001). Improving conceptual change concerning photosynMikkila thesis through text design. Learning and Instruction, 11, 241e257. http:// dx.doi.org/10.1016/S0959-4752(00)00041-4. Palmer, D. H. (2003). Investigating the relationship between refutational texts and conceptual change. Science Education, 87, 663e684. http://dx.doi.org/10.1002/ sce.1056. €-Erdmann, M. (2013). Conceptual change, text Pentinnen, M., Anto, E., & Mikkila comprehension and eye movements during reading. Research in Science Education, 43, 1407e1434. http://dx.doi.org/10.1007/s11165-012-9313-2. Rawson, K. A., & Van Overschelde, J. P. (2008). How does knowledge promote memory? the distinctiveness theory of skilled memory. Journal of Memory and Language, 58, 646e668. http://dx.doi.org/10.1016/j.jml.2007.08.004. Sanchez, E., & Garcia-Rodicio, H. (2013). Using online measures to determine how learners process instructional explanations. Learning and Instruction, 26, 1e11. http://dx.doi.org/10.1016/j.learninstruc.2012.12.003. Sinatra, G. M., & Broughton, S. H. (2011). Bridging reading comprehension and conceptual change in science education: the promise of refutation text. Reading Research Quarterly, 46(4), 374e393. http://dx.doi.org/10.1002/RRQ.005. van den Broek, P., & Kendeou, P. (2008). Cognitive processes in comprehension of science texts: the role of co-activation in confronting misconceptions. Applied Cognitive Psychology, 22, 335e351. http://dx.doi.org/10.1002/acp.1418. Wolfe, M. B. W., & Goldman, S. R. (2005). Relations between adolescents' text processing and reasoning. Cognition & Instruction, 23(4), 467e502. http:// dx.doi.org/10.1207/s1532690xci2304_2.