- Email: [email protected]
Promoting vicarious learning of physics using deep questions with explanations
Computers & Education 58 (2012) 1042–1048
Contents lists available at SciVerse ScienceDirect
Computers & Education journal homepage: www.elsevier.com/locate/compedu
Promoting vicarious learning of physics using deep questions with explanations Scotty D. Craig*, Barry Gholson, Joshua K. Brittingham, Joah L. Williams, Keith T. Shubeck The University of Memphis, Department of Psychology, 202 psychology building, Memphis, TN 38152, United States
a r t i c l e i n f o
a b s t r a c t
Article history: Received 22 July 2011 Received in revised form 24 October 2011 Accepted 23 November 2011
Two experiments explored the role of vicarious “self” explanations in facilitating student learning gains during computer-presented instruction. In Exp. 1, college students with low or high knowledge on Newton’s laws were tested in four conditions: (a) monologue (M), (b) questions (Q), (c) explanation (E), and (d) question þ explanation (Q þ E). Those with low pre-experimental knowledge levels showed marginally significant yet consistently greater gains than those with high levels and condition Q þ E outperformed the other three (M, Q, E). Among those with high knowledge, the Q þ E presentations actually inhibited learning. In Exp. 2, high school physics students in standard and honors classes were studied during their introduction to Newton’s laws. Brief (12 min) computer videos that introduced key Newtonian concepts preceded teacher presentations in seven daily sessions. Both standard and honors students who received Q þ E presentations prior to regular classroom activities learned more in daily sessions than those who received either M or Q presentations. It was concluded that when key concepts are introduced in the context of deep questions along with explanations new learning was facilitated both in vicarious environments and in subsequent standard classroom activities. Ó 2011 Elsevier Ltd. All rights reserved.
Keywords: Vicarious learning Questions Explanations Physics
1. Introduction During the past decade research supporting the conclusion that, when contrasted with various comparison conditions, vicarious exposure (Bandura, 1962, 1977; Rosenthal & Zimmerman, 1978) to computerized presentations of curriculum content in the context of deeplevel reasoning questions (henceforth deep questions) leads to significant increases in knowledge acquisition among students in middle school (Craig et al., 2008), high school (Craig, Brittingham, Williams, Cheney, & Gholson, 2009), and college (Gholson, Coles, & Craig, 2010). Deep questions, which are designed to encourage thoughtful integrative explanations, include stems such as “How does the.?’, “Why is the.?” and “What happens when.?” (Bloom, 1956, 1977; Gholson & Craig, 2006; Graesser, Baggett, & Williams, 1996; Graesser, Franceschetti, Gholson, & Craig, 2011; Graesser & Person, 1994; Rosenshine, Meister, & Chapman, 1996). These stems may be contrasted with shallow level questions, such as “Which is the.?”, “What is the.?”, and “Does X include.?”, that usually suggest one-word answers (Driscoll et al., 2003; Graesser & Person, 1994). Developments in this research domain have been reviewed elsewhere (Gholson et al., 2010; Gholson & Craig, 2006), so we will only highlight a few findings here. In a preliminary study, Craig, Gholson, Ventura, Graesser, and The Tutoring Research Group (2000) found that college students, who listened to a dialog in which two virtual agents located on a monitor discussed course content, learned more when they overheard each content statement introduced into a dialog by a deep question than when they overheard discussion of the content that did not include deep questions. Driscoll et al. (2003), Craig, Driscoll, and Gholson (2004), McKendree, Good, and Lee, (2001) and Sweller (1999) then showed that neither overhearing discussions of content in the context of shallow questions, nor discussions involving concept repetition, led college students to the same high level of enhanced learning as introducing each new content statement with a deep question. A subsequent series of studies (Craig, Sullins, Witherspoon, & Gholson, 2006; Gholson, Witherspoon, Morgan, Brittingham, Coles, Graesser, Sullins, & Craig, 2009; Craig, Gholson, & Driscoll, 2002) included conditions in which interactive tutoring sessions with an effective intelligent tutoring system, called “AutoTutor” (Graesser et al., 2011; Graesser & Olde, 2003; Graesser, Person, Harter, & The Tutoring Research Group, 2001; Graesser, Wiemer-Hastings, Wiemer-Hastings, Harter, Person, & The Tutoring Research Group, 2000; VanLehn et al., 2007), were contrasted with vicarious conditions. In the first (Craig et al., 2006, Exp. 1), college students in an interactive
* Corresponding author. E-mail address: [email protected] (S.D. Craig). 0360-1315/$ – see front matter Ó 2011 Elsevier Ltd. All rights reserved. doi:10.1016/j.compedu.2011.11.018
S.D. Craig et al. / Computers & Education 58 (2012) 1042–1048
1043
tutoring condition were contrasted with four computer-presented vicarious conditions. A yoked vicarious condition involved presenting each learner with a recorded session taken from the interactive tutoring condition. In a content-only vicarious condition learners were presented with two sentences taken from AutoTutor’s curriculum script concerning each concept used in the interactive tutoring condition. In a third, half questions vicarious condition, learners received one of the two sentences concerning each concept in AutoTutor’s script that were used in the content-only condition, each preceded by a deep question. In the final, full questions vicarious condition, a deep question was presented prior to each of the sentences used in the content-only condition. Learners in the full questions vicarious condition significantly outperformed (learned more) those in each of the remaining four conditions, including interactive tutoring. Pretest-to-posttest learning gains in those four latter conditions were reasonably comparable (see Craig et al., 2006, Table 1). Because the impact of embedding deep questions in educational content may have important implications for those designing curricula for distance learning and for computerized learning environments in general, it was deemed necessary to replicate Exp. 1. The findings did completely replicate, college students in two vicarious deep questions conditions both significantly outperformed those in interactive tutoring and another vicarious condition (Craig et al., 2006, Exp. 2). More recently younger students, 8th, 9th, 10th, and 11th graders, were studied (Gholson, Witherspoon, et al., 2009). Experimental conditions at each grade level included (a) interactive tutoring sessions with AutoTutor, that were contrasted with two vicarious learning conditions presenting (b) content-only sentences taken from AutoTutor’s curriculum scripts, and (c) the same content sentences each preceded by a deep question. Based upon prior research with college students, we predicted that younger vicarious learners presented course content containing deep questions would exhibit greater learning gains than those in both the interactive tutoring and content-only conditions. Eighth and tenth graders learned computer literacy, while ninth and eleventh graders learned Newtonian physics. Pretest-to-posttest gains were used to assess learning. There were no significant differences at pretest among the three experimental groups at any grade level or among grade levels. Gain scores yielded only significant main effects. The difference of interest here was the effect of experimental condition (learning gains in favor of the older students and in the physics domain were also obtained). Those in the deep questions condition showed significantly greater gains (about twice as large) than those in both the interactive tutoring and contentonly conditions, which did not differ from each other. Although previously shown only among college students in the domain of computer literacy, in the Gholson, Witherspoon, et al. (2009) study the deep question effect was also shown to hold in the domain of Newtonian physics as well as computer literacy among eighth to eleventh graders. We recently borrowed the term “catalyst” from chemistry to refer to the impact of deep questions in uniting disparate sources of information in memory and providing a context for integrating new content into coherent, logical, and/or causal reasoning chains during vicarious learning (Gholson, Craig, Brittingham, Germany, Fike, & Cheney, 2009). Thus, each deep question functions as a knowledge catalyst. Most catalytic agents bind with reactants during the steps in the chemical reactions that lead to the final product, but in the final step the catalytic agent is released from that final product and regenerated. Technically, the deep questions may function as stoichiometric catalysts, because in uniting the previously separate chemical elements (disparate knowledge sources in memory) copious amounts of this catalytic agent (causal structure in deep question, see above) may be completely consumed and become an integral part of the resulting chemical compound (new causally integrated knowledge structure). We will, however, adopt the more parsimonious term, knowledge catalyst. The findings described above led us to consider other candidate features of vicarious environments that might also function as knowledge catalysts. One such potentially powerful manipulation that has not been explored in vicarious environments involves self explanations (Ainsworth & Burcham, 2007; Chi, 2000; Chi, Bassok, Lewis, Reimann, & Glaser, 1989; Chi, Roy, & Hausmann, 2008). When learners overtly explained a concept immediately after it was presented, usually by integrating it with existing knowledge (Chi, 2000), those self explanations produced large learning gains when compared to various controls (Chi et al., 1989). Self explanations are clearly distinguished from other sorts of verbal commentary, such as paraphrase or self monitoring statements (Ainsworth & Loizou, 2003; Chi, Siler, Jeons, Yamauchi, & Hausmann, 2001). They embed new content in a web of existing knowledge. As described by Chi et al. (2008), then, self explanations may, like deep questions, function as knowledge catalysts, encouraging cognitive activities that unite desperate sources of information in memory that expose knowledge gaps and provide a coherent causal context for integrating new content. Research reported below explored whether overheard (vicarious) analogs of self explanations, might, like deep questions, serve as knowledge catalysts that increase learning gains, first among college students in the laboratory (Exp. 1) and then among younger students studied individually in the classroom (Exp. 2). 2. Experiment 1 Unfortunately, students do not spontaneously generate self explanations at high rates, In fact, even when students are prompted, only a small proportion of their verbalizations are genuine self explanations. Instead students mostly verbalize self monitoring statements or paraphrase recently-presented content (Chi, 2000; Chi, de Leew, Chiu, & LaVancher, 1994). In fact, even when prompted about half of the students tested by Chi et al. (1994) failed to generate self explanations during an intensive study spanning as many as three or four daily sessions. While the students who generated explanations showed improved mental models and dramatic learning gains, those who failed to generate self explanations did not develop adequate mental models, and they performed at the same level as a comparison group who
Table 1 Pre and posttest scores for each experimental condition and knowledge level. Combined Pretest
M Q E QþE
Low knowledge Posttest
Change
Pretest
High knowledge Posttest
Change
Pretest
Posttest
Change
M
SD
M
SD
M
SD
M
SD
M
SD
M
SD
M
SD
M
SD
M
SD
.31 .31 .31 .36
.11 .10 .11 .16
.52 .52 .53 .52
.15 .17 .17 .16
.21 .21 .22 .16
.15 .16 .19 .20
.22 .25 .24 .22
.07 .06 .05 .08
.45 .45 .51 .53
.15 .15 .19 .13
.23 .21 .26 .31
.15 .17 .18 .09
.40 .42 .43 .45
.05 .06 .08 .12
.59 .63 .57 .52
.13 .13 .15 .18
.19 .21 .15 .07
.15 .15 .19 .20
1044
S.D. Craig et al. / Computers & Education 58 (2012) 1042–1048
simply read the content. Our aim in this research, then, was to present a vicarious “self” explanation, constructed according to criteria described by Chi (2000; Chi et al., 1989), following each content statement presented to the student. The question addressed in the present research is, then, might students use vicarious explanatory information of new content to expand existing knowledge structures and enhance learning. Our prediction, of course, was that presenting vicarious explanations following each sentence containing course content would, like deep questions, facilitate new learning by functioning as knowledge catalysts. The second purpose of Exp. 1 we also explore what Kalyuga, Ayres, Chandler, and Sweller (2003) called an expertise reversal effect. This effect suggests that, while low knowledge learners use programmatic instructional support to build organized knowledge structures and construct new mental models, those with more advanced knowledge structures may actually be disadvantages by targeted instructional support. This is because, for these learners, this instruction imposes considerable extrinsic cognitive load (Sweller, 1999), as they attempt to reconcile components of the instructed models with already existing advanced knowledge structures (Kalyuga, 2007). 3. Method 3.1. Participants The participants were 130 college students recruited from a research pool at a large urban university in the southern United States. Participants received credit for their participation toward introductory Psychology courses. Participants were allowed to stop their participation at any point during their experiment. 3.2. Materials 3.2.1. Assessments Two 20 item multiple-choice tests on Newtonian physics were counterbalanced as pretest and posttest for presentation to each participant. Test presentation order was counterbalanced so that approximately half received Test A as pretest with Test B as posttest, with the remainder presented the tests in reverse order. See Appendix A for example questions. 3.2.2. Experimental conditions A 30-min multimedia video was prepared for each of the four experimental conditions. Each computer-presented multimedia presentation involved two virtual agents who interacted by speaking. A virtual tutor was located on the right side of a monitor and a virtual tutee was located on the left side. The same series of relevant images were located between the two virtual agents in all conditions. In the monologue condition the virtual tutor presented 50 content statements, one at a time, while the virtual tutee simply listened. In the question condition (Q), the virtual tutee asked a deep question prior to each of the content statement presented by the virtual tutor. The deep questions used in the question condition were different from the questions used in the assessments. In the explanation condition (E), the virtual tutee listened to each of the virtual tutor’s content statement, and then provided an explanation. While these explanations elaborated on the content they did not provide any new knowledge required for a correct answer on the assessments. In the question þ explanation condition (Q þ E), the tutee asked the same question as in the question condition, and then following the content statement from the tutor, the tutee provided the same explanation that was presented in the explanation condition. Excerpts from each condition are available in Appendix B. 3.3. Procedures Each participant was designated as high or low knowledge level based on a median split on the pretest scores, yielding a 4 (experimental condition) 2 (knowledge level) design. When participants arrived at the lab for their session, they were randomized into one of the four experimental conditions. After filling out an informed consent, participants then took a 20 item pretest. They were then presented a 30 min video. Participants were only allowed to view the video one time and not allowed to rewind or fast forward. Afterward, they received a 20 item posttest. The session lasted about one hour. 3.3.1. Scoring Participants received one point for each question correct on the assessment tests. They were divided into the two knowledge levels based on the pretest data. This resulted in 70 participants in the low knowledge condition and 60 participants in the high knowledge condition. 4. Results and discussion An ANOVA performed on the pretest scores yielded no differences among the four experimental conditions at either knowledge level. A 4 (experimental condition) 2 (knowledge level) analysis of pretest-to-posttest gain scores yielded a significant interaction between knowledge level and experimental condition, F(3,122) ¼ 3.07, p < .03. Pretest-to-posttest learning gains for all conditions are presented in Table 1. We will focus on how the deep questions and explanations differentially affected learning gains among students with low previous knowledge vs. those with higher previous knowledge levels. Statistical comparisons revealed that learners in the low pre-experimental knowledge condition in the Q þ E condition showed significantly (p < .001) greater learning gains (gain ¼ 31%) than those with high pre-experimental knowledge (gain ¼ 7%). In fact, even though the former students showed considerably less pre-experimental knowledge (22% vs. 45% correct), they showed slightly higher scores on the post test. In the E condition, the gain in favor of those with low preexperimental knowledge (26%) approached significance (p < .08) when compared with those with high knowledge (15%). Comparisons among those in the four groups that exhibited low pre-experimental knowledge were also conducted. These comparisons revealed some suggestive difference. Learners in the Q þ E condition showed greater gains than those in all three of the remaining conditions (M, Q, E), but the differences were not significant (smallest p ¼ .08).
S.D. Craig et al. / Computers & Education 58 (2012) 1042–1048
1045
As predicted by the Kalyuga (2007) high knowledge reversal hypothesis, among those with high knowledge, the Q þ E condition showed very small pretest-to- to posttest gains (7%), compared to learners in the other three high-knowledge experimental conditions (14–21% gains), and actually showed significantly smaller pre-to-post gains (p < .05) than those in both the M (19%) and Q (21%) conditions. It seems, consistent with earlier findings (Kalyuga, 2007), that more knowledgeable learners were actually disadvantaged by that knowledge in the Q þ E condition, presumably due to conflict between the learner’s existing mental model and the model embodied in the newly presented instructional content. The instructional content, of course, enabled greater learning gains among those with low pre-experimental knowledge levels, particularly among those in the question þ explanation condition. 5. Experiment 2 Results of Exp. 1 indicate that vicarious self explanations of instructional content, like vicarious deep questions, may serve as knowledge catalysts that support learning. These findings, along with recent work showing that deep questions promote learning processes in the high school physics classroom (Gholson, Craig, etc submitted), provided the backdrop for Exp. 2. Brief videos were presented individually to each student in high school physics classes via laptop computers at the outset each of seven consecutive daily classroom sessions. The brief computer presentations were followed by standard classroom instruction provided by their regular teacher. Three different computer video presentations were prepared for each day of instruction, a total of 21. These were designed to provide the foundation of the teacher’s presentations each day. Because a teacher’s classroom presentations vary somewhat from class to class, even when presenting the same content, each of the three computer video was presented to approximately one third of the students in each of the five classrooms each day. 6. Method 6.1. Participants The participants were 143 students enrolled in high school physics classes in an inner-city school who were studied individually in their classrooms. Three of these classes were honors students (85 total) and the other two were standard (58 total students). Prior to the first of the seven daily classroom sessions each student’s and their parent’s informed consent was obtained. 6.2. Design and materials About one third of the students in each class were assigned to each one of three experimental conditions, question þ explanation (Q þ E), question (Q) or monologue (M). Thus, the experimental design was a 2 (achievement level: standard students vs. honors students) 3 (Experimental condition: Q þ E vs. Q vs. M). On Day 1 through Day 5 students studied Newton’s first Law and various component concepts, such as mass and friction. On Day 6 and Day 7 they studied an introduction to Newton’s second and third laws. On each day of instruction, Day 1 through Day 7, each learner was presented with two versions of a 10-item, 4-foil multiple-choice test used to measure learning gains on specific content presented during that daily session. As noted above, seven different sets of tests were constructed, one set to assess learning gains on each day of instruction. The two daily tests were presented to students in counterbalanced order (half received Test A as pretest with Test B as posttest, with the remainder presented the tests in reverse order), at the outset of each daily session prior to a computer presentation, and at the end of each session, after a teacher presentation (see below). The tests for each day were totaled. These totals were used for the analysis. Each of the three vicarious learning conditions involved a different computer-controlled multimedia presentation. The students, who wore headphones during the multimedia presentations, each worked independently at a laptop computer located on their own desk in their classroom. The multimedia presentations, provided during the first part of each of the seven consecutive classroom sessions, were each about 12 min in length. These presentations were designed to provide conceptual understanding of the content that would be presented by the teacher during the second part of that same daily classroom session. This latter part of each daily session, which was presented by the regular classroom teacher, was about 25 min in length. The teacher elaborated that day’s content while providing equations along with applications that gave quantitative expression to the concepts and relationships presented earlier in the computer-controlled vicarious learning presentations. 6.3. Experimental procedures Daily computer presentations in each experimental condition began with a brief spoken dialog between an animated virtual tutor and virtual tutee, located on two sides of each student’s monitor. This dialog served as introduction to that day’s topics. A series of images relevant to each concepts throughout the session were located on the monitor between the two agents in all conditions. Immediately following this introduction, the learner’s experimental condition was presented. In a virtual Q þ E condition, the presentation of each major concept in the curriculum involved a three-event sequence. First, the virtual tutee asked a deep question probing the concept. Second, the virtual tutor presented a content statement elaborating the meaning and use of the concept. And third, the virtual tutee provided an explanation of the concept that elaborated its meaning and attempted to provide a coherent context for the concept’s assimilation into an existing web of knowledge. An example of this three-event sequence is: (a) Virtual tutee’s question probing the concept: “What does the word mass mean as used here?”; (2) Virtual tutor’s content statement elaborating its meaning: “The mass in this case is known as ’gravitational mass’ and is measured in kilograms, or Kg”; (3) Virtual tutee’s explanation relating the concept to earlier content: “Mass, or gravitational mass is different from weight, in that mass is the same on the earth’s surface, in weightless space, and on the surface of the moon, but obviously the weight changes depending on where the object is located due to the different gravitational attractions in the different locations.” After this three-part dialog was completed, the three events involving the next concept in the curriculum were presented, etc. Each daily session included about 25 of these three-event sequences. In a virtual Q condition the only difference from the virtual Q þ E condition was that the third event, the explanation of the concept by the virtual tutee, was omitted from each three-event sequence. That is, the deep question concerning the concept was asked by the virtual tutee and
1046
S.D. Craig et al. / Computers & Education 58 (2012) 1042–1048
Table 2 Proportion gain score means and standard deviations for each experimental condition and classroom type. M ¼ Content Only; Q ¼ Questions; Q þ E ¼ Questions and explanations. Combined Pretest
M Q QþE
Standard classroom Posttest
Change
Pretest
Honors classroom
Posttest
Change
Pretest
Posttest
Change
M
SD
M
SD
M
SD
M
SD
M
SD
M
SD
M
SD
M
SD
M
SD
.32 .34 .32
.08 .08 .08
.49 .52 .54
.15 .11 .11
.17 .18 .22
.14 .10 .09
.35 .30 .32
.09 .05 .10
.45 .45 .48
.13 .11 .10
.10 .15 .16
.11 .12 .09
.29 .37 .32
.06 .09 .08
.53 .57 .58
.14 .07 .10
.24 .21 .26
.13 .08 .08
answered by the virtual tutor with a content statement, but no further explanation was provided by either virtual agent. In a virtual M condition both the first and third events in the sequence were omitted. Only the virtual tutor’s content statement was presented to the vicarious learner. 7. Results and discussion A 3 (experimental condition: Q þ E vs. Q vs. M) 2 (class type: standard vs. honors) ANOVA was performed on (pretest-to-posttest) gain scores on the tests used to evaluate daily learning across the seven sessions. It yielded significant effects of experimental condition, F(2137) ¼ 4.00, p < .01, and class type, F(1,13) ¼ 6.98, p < .01, but no interaction. The mean proportions and standard deviations of gain scores for standard and honors students in each condition may be found in Table 2. Simple effect procedures (Tukey HSD), collapsed across achievement levels, revealed that learners in the Q þ E condition significantly outperformed those in both the M (p < .01) and Q (p < .05) conditions. Learners in the Q þ E condition showed average pretest-to-posttest learning gains of 22%, while those in the M and Q conditions showed gains of 18% and 17%, respectively. Collapsed across experimental conditions, the honors students showed significantly greater gains (24%) than the standard students (15%). Because Newton’s first law (Day 1– Day 5) and the introduction to Newton’s second and third laws (Day 6 and Day 7) involved separate units in the physics course, learning data from the two units were also analyzed separately. As expected, analysis of data on the first law yielded exactly the same significant effects as the overall analyses (experimental condition, class type). The data from the introduction of the second and third laws yielded only a main effect of experimental condition, the class type effect was not significant. As results of the analyses indicate, the A þ E condition was quite successful in producing daily learning gains. The Q condition failed to produce gains relative to the M condition. We note that only about 25 concepts and questions were presented in each daily session. Previous research (e.g., Craig et al., 2006) has shown that in vicarious learning sessions in a computer learning environment about 100 content sentences preceded by deep questions per session may be needed to produce learning gains exceeding those produced by intelligent tutoring conditions or other vicarious learning conditions (Craig et al., 2006). The findings in the Q þ E condition do appear to indicate that vicarious deep explanations may, like vicarious deep questions, function as knowledge catalysts. Research now underway will further address this and related issues. 8. Conclusions The proposal that explanations which embed current course content into a web of existing knowledge might, like deep explanations, function as knowledge catalysts and increase learning received support from both Exp. 1 and 2. Among college students (Exp. 1) with low knowledge those in the Q þ E showed greater gains than any other group, and dramatically outperformed those with high knowledge in that condition (32% gain vs. 7%). Interestingly, in the high knowledge condition those in both the M and Q condition showed greater gains (19– 21%) than the 7% shown in Q þ E. This suggests, consistent with Kalyuga’s (2007) proposal, that among learners with advanced knowledge structures, providing targeted instruction designed to support model construction may actually inhibit new learning. This is because the instruction produces extrinsic cognitive load (Sweller, 1999) as learners attempt to reconcile the detailed instructional model with their existing mental model. Thus, the significantly greater learning gains among those with high knowledge in the M and Q conditions (than Q þ E) lends further support to Kalyuga’s proposal (Kalyuga et al., 2003; Sweller, 1999): greater detail in the instructed model produced more extrinsic cognitive load and smaller learning gains among learners with already refined knowledge structures. In Exp. 2, high school physics students received brief (12 min) video presentations at the outset of seven standard classroom sessions at the outset in the introduction to the study of Newton’s laws. Those in a Q þ E condition showed greater pretest-to-posttest gains than those in M and Q (no E condition was conducted) comparison groups, which did not differ from each other. As expected, students in honors classes showed greater pretest-to-posttest learning gains than students assigned to standard classes. From the two studies taken together, it seems reasonable to conclude that vicarious explanations presented immediately following course content statements function as knowledge catalysts by filling knowledge gaps that enhance learning. These findings, along with those described earlier, indicate that both deep questions and integrative explanations increase understanding and produce knowledge gains by embedding new content into webs of existing knowledge. In summary, these two studies show that integrative explanations following content statements support new learning, and are most effective with combined with deep questions that precede the content statements. While explanations alone were not shown to be effective, when combined with questions, they became effective for low knowledge learners. Moreover, as seen in Exp. 2, this combination of questions and explanations may be ideal for introducing complex concepts and topics using short videos presented prior to standard classroom instruction. Acknowledgements This research was supported by the Institute for Education Sciences (IES) Grant R305H0R0169. Any opinions, findings, and conclusions or recommendations expressed in this material are those of the authors and do not necessarily reflect the views of IES (DOE).
S.D. Craig et al. / Computers & Education 58 (2012) 1042–1048
1047
Appendix A What does the second part of Newton’s first law say about balanced forces? a. The object will move at a constant speed in a particular direction. b. Any moving object will seek its natural state of rest. c. The ancient Greeks were right and forces can never really be balanced. d. None of these. If there is a 4 kg block hanging from a rope in an elevator with resulting in a 40 N force, which of the following are true? a. If the tension on the rope is equal to 40 N, the 4 kg block will be stationary. b. If the tension on the rope is more than 40 N, the 4 kg mass will be stationary. c. If the tension on the rope is less than 40 N, the weight will be stationary. d. The tension on the rope will not affect the movement of the mass. If you tie a cart with a cable at the top of a hill so it does not fall down and scratch cars in the parking lot, what part of the weight of the cart is held up by the cable to keep it from rolling down the hill? a. Only the part of the weight that is parallel to the hill will attempt to get the cart to roll down the hill. b. The part of the weight pushing straight down due to gravity will try to push the cart down the hill c. The full weight of the cart. d. The weight of the cart parallel to the hill plus the weight of the cart perpendicular to the hill. What causes the force of tension to be observed? a. Tension occurs when an object is pushed with a rod. b. Tension occurs when an object is pulled on by a string. c. Tension occurs when an object is pulled downward by the earth. d. Tension occurs when an object is pulled down a ramp. Which of following events are due to force of tension? a. When you attempt to push a desk sideways that is not on wheels. b. When box slides down a ramp. c. When molecules attempt to stretch to counteract a weight force. d. When a book with a string attached to it is at fully support by the top of a table. How can you reduce tension on a rope? a. Increase the weight that is suspended. b. Decrease the friction by adding wheels to the object. c. Increase the gravity on the suspended object. d. Decrease the weight that is suspended.
Appendix B Below examples are given for the content statements for study one and study two (M,Q, and Q þ E only). Monolog (M) condition Tension is another force like weight. The symbol for tension is T. Tension is measured in Newtons just like weight (A large, red apple has a weight of about 1 Newton). The tension force will appear in problems where a string, line, rope, cord chain, rod, dental floss, cable and so forth are pulling on an object. In all cases the strings and things can pull but cannot push since ropes, cords and cable will collapse when pushed at one end, so the rod is the only exception here in that you can push or pull with a rod and still have tension. Questions (Q) condition Question: People talk about tension all the time, what does the physicist mean by it? Tension is another force like weight. The symbol for tension is T. Question: How does the physicist go about measuring tension? Tension is measured in Newtons just like weight (A large, red apple has a weight of about 1 Newton). Question: How does tension happen? The tension force will appear in problems where a string, line, rope, cord chain, rod, dental floss, cable and so forth are pulling on an object. Question: How can you have tension when you push on a rope or something like that? In all cases the strings and things can pull but cannot push since ropes, cords and cable will collapse when pushed at one end, so the rod is the only exception here in that you can push or pull with a rod and still have tension. Explanations (E) condition Tension is another force like weight. The symbol for tension is T. Explanation: So tension is a force like weight, but we call the Tension force T.
1048
S.D. Craig et al. / Computers & Education 58 (2012) 1042–1048
Tension is measured in Newtons just like weight (A large, red apple has a weight of about 1 Newton). Explanation: So tension T is measured in Newtons, just like weight is. The tension force will appear in problems where a string, line, rope, cord chain, rod, dental floss, cable and so forth are pulling on an object. Explanation: So tension T has to do with holding up on object pulling down on it. In all cases the strings and things can pull but cannot push since ropes, cords and cable will collapse when pushed at one end, so the rod is the only exception here in that you can push or pull with a rod and still have tension. Explanation: So of the examples used, only s rod can be used to push or pull and have tension either way. Questions and Explanations (Q þ E) condition Question: People talk about tension all the time, what does the physicist mean by it? Tension is another force like weight. The symbol for tension is T. Explanation: So tension is a force like weight, but we call the Tension force T. Question: How does the physicist go about measuring tension? Tension is measured in Newtons just like weight (A large, red apple has a weight of about 1 Newton). Explanation: So tension T is measured in Newtons, just like weight is. Question: How does tension happen? The tension force will appear in problems where a string, line, rope, cord chain, rod, dental floss, cable and so forth are pulling on an object. Explanation: So tension T has to do with holding up on object pulling down on it. Question: How can you have tension when you push on a rope or something like that? In all cases the strings and things can pull but cannot push since ropes, cords and cable will collapse when pushed at one end, so the rod is the only exception here in that you can push or pull with a rod and still have tension. Explanation: So of the examples used, only s rod can be used to push or pull and have tension either way. References Ainsworth, S., & Burcham, S. (2007). Learning and instruction. Learning and Instruction, 17, 286–303. Ainsworth, S., & Loizou, A. T. (2003). The effects of self explaining when learning with text or diagrams. Cognitive Science, 27, 669–681. Bandura, A. (1962). Social learning through imitation. In M. R. Jones (Ed.), Nebraska symposium on motivation (pp. 211–269). Lincoln: University of Nebraska Press. Bandura, A. (1977). Social learning theory. Englewood Cliffs, NJ: Prentice Hall. Bloom, B. S. (1956). Taxonomy of education objectives: the classification of educational goals. In Handbook I: Cognitive domain. New York: McKay. Bloom, B. (1977). Affective outcomes of school learning. Phi Delta Kappan, 59, 193–198. Chi, M. T. H. (2000). Self explaining expository texts: the dual process of generating inferences and repairing mental models. In R. Glaser (Ed.), Advances in instructional psychology (pp. 161–238). Mahwah, NJ: Lawrence Erlbaum Associates. Chi, M. T. H., Bassok, M., Lewis, M. W., Reimann, P., & Glaser, R. (1989). Self-explanations: how students study and use examples in learning to solve problems. Cognitive Science, 13, 145–182. Chi, M. T. H., de Leew, N., Chiu, M., & LaVancher, C. (1994). Eliciting self-explanations improves understanding. Cognitive Science, 18, 439–477. Chi, M. T. H., Roy, M., & Hausmann, R. G. M. (2008). Observing tutorial dialogues collaboratively: insights about human tutoring effectiveness from vicarious learning. Cognitive Science, 32, 301–341. Chi, M. T. H., Siler, S. A., Jeong, H., Yamauchi, T., & Hausmann, R. G. (2001). Learning from human tutoring. Cognitive Science, 25, 471–533. Craig, S. D., Brittingham, J., Williams, J., Cheney, K. R., & Gholson, B. (2009). Incorporating vicarious learning environments with discourse scaffolds into physics classrooms. In V. Dimitrova, R. Mizoguchi, B. du Boulay, & A. C. Graesser (Eds.), Artificial intelligence in education, building learning systems that care: From knowledge representation to affective modeling (pp. 680–682). Washington, DC: IOS Press. Craig, S. D., Driscoll, D., & Gholson, B. (2004). Constructing knowledge from dialogue in an intelligent tutoring system: Interactive learning, vicarious learning, and pedagogical agents. Journal of Educational Multimedia and Hypermedia, 13, 163–183. Craig, S. D., Gholson, B., & Driscoll, D. (2002). Animated pedagogical agents in multimedia educational environments: effects of agent properties, picture features, and redundancy. Journal of Educational Psychology, 94, 428–434. Craig, S. D., Gholson, B., Ventura, M., Graesser, A. C., & The Tutoring Research Group. (2000). Overhearing dialogues and monologues in virtual tutoring sessions: effects on questioning and vicarious learning. International Journal of Artificial Intelligence in Education, 11, 242–253. Craig, S. D., Graesser, A., Brittingham, J., Williams, J., Martindale, T., Williams, G., et al. (2008). An implementation of vicarious learning environments in middle school classrooms. In K. McFerrin, R. Weber, R. Weber, R. Carlsen, & D. A. Willis (Eds.), The proceedings of the 19th international conference for the Society for Information Technology & Teacher Education (pp. 1060–1064). Chesapeake, VA: AACE. Craig, S. D., Sullins, J., Witherspoon, A., & Gholson, B. (2006). The deep-level reasoning effect: the role of dialogue and deep-level-reasoning questions during vicarious learning. Cognition and Instruction, 24, 565–591. Driscoll, D., Craig, S. D., Gholson, B., Ventura, M., Hu, X., & Graesser, A. C. (2003). Vicarious learning: effects of overhearing dialogues and monologue-like discourse in a virtual tutoring session. Journal of Educational Computing Research, 29, 431–450. Gholson, B., Coles, R., & Craig, S. D. (2010). Features of computerized multimedia environments that support vicarious learning processes. In M. S. Khine, & I. M. Saleh (Eds.), New science of learning: Cognition, computers and collaboration in education (pp. 53–78). New York: Springer. Gholson, B., & Craig, S. D. (2006). Promoting constructive activities that support learning during computer-based instruction. Educational Psychology Review, 18, 119–139. Gholson, B., Craig, S. D., Brittingham, J., Germany, M.-L., & Fike, K. (2009, July). The role of deep level questions, “self-explanations,” and domain knowledge in vicarious learning. In Presented at Society for Text and Discourse, July 2009, Rotterdam, NL. Gholson, B., Witherspoon, A., Morgan, B., Brittingham, J., Coles, R., Graesser, A. C., et al. (2009). Exploring the deep-level reasoning questions effect during vicarious learning among eighth to eleventh graders in the domains of computer literacy and Newtonian physics. Instructional Science, 37, 487–493. Graesser, A. C., Baggett, W., & Williams, K. (1996). Question-driven explanatory reasoning. Applied Cognitive Psychology, 10, S17–S32. Graesser, A. C., Franceschetti, D., Gholson, B., & Craig, S. (2011). Learning Newtonian physics with conversational agents and interactive simulation. In N. Stein, & S. W. Raudenbush (Eds.), Developmental cognitive science goes to school (pp. 157–172). New York: Routledge. Graesser, A. C., & Olde, B. A. (2003). How does one know whether a person understands a device? The quality of the questions the person asks when the device breaks down. Journal of Educational Psychology, 95, 524–536. Graesser, A. C., & Person, N. (1994). Question asking during tutoring. American Educational Research Journal, 31, 104–137. Graesser, A. C., Person, N., Harter, D., & The Tutoring Research Group. (2001). Teaching tactics and dialogue in AutoTutor. International Journal of Artificial Intelligence in Education, 12, 257–279. Graesser, A. C., Wiemer-Hastings, P., Wiemer-Hastings, K., Harter, D., Person, N., & Tutoring Research Group. (2000). Using latent semantic analysis to evaluate the contributions of students in AutoTutor. Interactive Learning Environments, 8, 129–148. Kalyuga, S., Ayres, P., Chandler, P., & Sweller, J. (2003). Expertise reversal effect. Educational Psychologist, 38, 23–31. Kalyuga, S. (2007). Expertise reversal effect and its implications for learner-tailored instruction. Educational Psychology Review, 19, 509–539. Rosenshine, B., Meister, C., & Chapman, S. (1996). Teaching students to ask questions: a review of intervention studies. Review of Educational Research, 66, 181–221. Rosenthal, R. L., & Zimmerman, B. J. (1978). Social learning and cognition. New York: Academic Press. McKendree, J., Good, J., & Lee, J. (2001, June). Effects of dialogue characteristics on performance of overhearers. In Presented at the international conference on communication, problem-solving, and learning, Strathclyde. Sweller, J. (1999). Instructional design in technical areas. Melbourne, Australia: ACER Press. VanLehn, K., Graesser, A. C., Jackson, G. T., Jordan, P., Olney, A., & Rosé, C. P. (2007). Natural language tutoring: a comparison of human tutors, computer tutors and text. Cognitive Science, 31, 3–62.
Contents lists available at SciVerse ScienceDirect
Computers & Education journal homepage: www.elsevier.com/locate/compedu
Promoting vicarious learning of physics using deep questions with explanations Scotty D. Craig*, Barry Gholson, Joshua K. Brittingham, Joah L. Williams, Keith T. Shubeck The University of Memphis, Department of Psychology, 202 psychology building, Memphis, TN 38152, United States
a r t i c l e i n f o
a b s t r a c t
Article history: Received 22 July 2011 Received in revised form 24 October 2011 Accepted 23 November 2011
Two experiments explored the role of vicarious “self” explanations in facilitating student learning gains during computer-presented instruction. In Exp. 1, college students with low or high knowledge on Newton’s laws were tested in four conditions: (a) monologue (M), (b) questions (Q), (c) explanation (E), and (d) question þ explanation (Q þ E). Those with low pre-experimental knowledge levels showed marginally significant yet consistently greater gains than those with high levels and condition Q þ E outperformed the other three (M, Q, E). Among those with high knowledge, the Q þ E presentations actually inhibited learning. In Exp. 2, high school physics students in standard and honors classes were studied during their introduction to Newton’s laws. Brief (12 min) computer videos that introduced key Newtonian concepts preceded teacher presentations in seven daily sessions. Both standard and honors students who received Q þ E presentations prior to regular classroom activities learned more in daily sessions than those who received either M or Q presentations. It was concluded that when key concepts are introduced in the context of deep questions along with explanations new learning was facilitated both in vicarious environments and in subsequent standard classroom activities. Ó 2011 Elsevier Ltd. All rights reserved.
Keywords: Vicarious learning Questions Explanations Physics
1. Introduction During the past decade research supporting the conclusion that, when contrasted with various comparison conditions, vicarious exposure (Bandura, 1962, 1977; Rosenthal & Zimmerman, 1978) to computerized presentations of curriculum content in the context of deeplevel reasoning questions (henceforth deep questions) leads to significant increases in knowledge acquisition among students in middle school (Craig et al., 2008), high school (Craig, Brittingham, Williams, Cheney, & Gholson, 2009), and college (Gholson, Coles, & Craig, 2010). Deep questions, which are designed to encourage thoughtful integrative explanations, include stems such as “How does the.?’, “Why is the.?” and “What happens when.?” (Bloom, 1956, 1977; Gholson & Craig, 2006; Graesser, Baggett, & Williams, 1996; Graesser, Franceschetti, Gholson, & Craig, 2011; Graesser & Person, 1994; Rosenshine, Meister, & Chapman, 1996). These stems may be contrasted with shallow level questions, such as “Which is the.?”, “What is the.?”, and “Does X include.?”, that usually suggest one-word answers (Driscoll et al., 2003; Graesser & Person, 1994). Developments in this research domain have been reviewed elsewhere (Gholson et al., 2010; Gholson & Craig, 2006), so we will only highlight a few findings here. In a preliminary study, Craig, Gholson, Ventura, Graesser, and The Tutoring Research Group (2000) found that college students, who listened to a dialog in which two virtual agents located on a monitor discussed course content, learned more when they overheard each content statement introduced into a dialog by a deep question than when they overheard discussion of the content that did not include deep questions. Driscoll et al. (2003), Craig, Driscoll, and Gholson (2004), McKendree, Good, and Lee, (2001) and Sweller (1999) then showed that neither overhearing discussions of content in the context of shallow questions, nor discussions involving concept repetition, led college students to the same high level of enhanced learning as introducing each new content statement with a deep question. A subsequent series of studies (Craig, Sullins, Witherspoon, & Gholson, 2006; Gholson, Witherspoon, Morgan, Brittingham, Coles, Graesser, Sullins, & Craig, 2009; Craig, Gholson, & Driscoll, 2002) included conditions in which interactive tutoring sessions with an effective intelligent tutoring system, called “AutoTutor” (Graesser et al., 2011; Graesser & Olde, 2003; Graesser, Person, Harter, & The Tutoring Research Group, 2001; Graesser, Wiemer-Hastings, Wiemer-Hastings, Harter, Person, & The Tutoring Research Group, 2000; VanLehn et al., 2007), were contrasted with vicarious conditions. In the first (Craig et al., 2006, Exp. 1), college students in an interactive
* Corresponding author. E-mail address: [email protected] (S.D. Craig). 0360-1315/$ – see front matter Ó 2011 Elsevier Ltd. All rights reserved. doi:10.1016/j.compedu.2011.11.018
S.D. Craig et al. / Computers & Education 58 (2012) 1042–1048
1043
tutoring condition were contrasted with four computer-presented vicarious conditions. A yoked vicarious condition involved presenting each learner with a recorded session taken from the interactive tutoring condition. In a content-only vicarious condition learners were presented with two sentences taken from AutoTutor’s curriculum script concerning each concept used in the interactive tutoring condition. In a third, half questions vicarious condition, learners received one of the two sentences concerning each concept in AutoTutor’s script that were used in the content-only condition, each preceded by a deep question. In the final, full questions vicarious condition, a deep question was presented prior to each of the sentences used in the content-only condition. Learners in the full questions vicarious condition significantly outperformed (learned more) those in each of the remaining four conditions, including interactive tutoring. Pretest-to-posttest learning gains in those four latter conditions were reasonably comparable (see Craig et al., 2006, Table 1). Because the impact of embedding deep questions in educational content may have important implications for those designing curricula for distance learning and for computerized learning environments in general, it was deemed necessary to replicate Exp. 1. The findings did completely replicate, college students in two vicarious deep questions conditions both significantly outperformed those in interactive tutoring and another vicarious condition (Craig et al., 2006, Exp. 2). More recently younger students, 8th, 9th, 10th, and 11th graders, were studied (Gholson, Witherspoon, et al., 2009). Experimental conditions at each grade level included (a) interactive tutoring sessions with AutoTutor, that were contrasted with two vicarious learning conditions presenting (b) content-only sentences taken from AutoTutor’s curriculum scripts, and (c) the same content sentences each preceded by a deep question. Based upon prior research with college students, we predicted that younger vicarious learners presented course content containing deep questions would exhibit greater learning gains than those in both the interactive tutoring and content-only conditions. Eighth and tenth graders learned computer literacy, while ninth and eleventh graders learned Newtonian physics. Pretest-to-posttest gains were used to assess learning. There were no significant differences at pretest among the three experimental groups at any grade level or among grade levels. Gain scores yielded only significant main effects. The difference of interest here was the effect of experimental condition (learning gains in favor of the older students and in the physics domain were also obtained). Those in the deep questions condition showed significantly greater gains (about twice as large) than those in both the interactive tutoring and contentonly conditions, which did not differ from each other. Although previously shown only among college students in the domain of computer literacy, in the Gholson, Witherspoon, et al. (2009) study the deep question effect was also shown to hold in the domain of Newtonian physics as well as computer literacy among eighth to eleventh graders. We recently borrowed the term “catalyst” from chemistry to refer to the impact of deep questions in uniting disparate sources of information in memory and providing a context for integrating new content into coherent, logical, and/or causal reasoning chains during vicarious learning (Gholson, Craig, Brittingham, Germany, Fike, & Cheney, 2009). Thus, each deep question functions as a knowledge catalyst. Most catalytic agents bind with reactants during the steps in the chemical reactions that lead to the final product, but in the final step the catalytic agent is released from that final product and regenerated. Technically, the deep questions may function as stoichiometric catalysts, because in uniting the previously separate chemical elements (disparate knowledge sources in memory) copious amounts of this catalytic agent (causal structure in deep question, see above) may be completely consumed and become an integral part of the resulting chemical compound (new causally integrated knowledge structure). We will, however, adopt the more parsimonious term, knowledge catalyst. The findings described above led us to consider other candidate features of vicarious environments that might also function as knowledge catalysts. One such potentially powerful manipulation that has not been explored in vicarious environments involves self explanations (Ainsworth & Burcham, 2007; Chi, 2000; Chi, Bassok, Lewis, Reimann, & Glaser, 1989; Chi, Roy, & Hausmann, 2008). When learners overtly explained a concept immediately after it was presented, usually by integrating it with existing knowledge (Chi, 2000), those self explanations produced large learning gains when compared to various controls (Chi et al., 1989). Self explanations are clearly distinguished from other sorts of verbal commentary, such as paraphrase or self monitoring statements (Ainsworth & Loizou, 2003; Chi, Siler, Jeons, Yamauchi, & Hausmann, 2001). They embed new content in a web of existing knowledge. As described by Chi et al. (2008), then, self explanations may, like deep questions, function as knowledge catalysts, encouraging cognitive activities that unite desperate sources of information in memory that expose knowledge gaps and provide a coherent causal context for integrating new content. Research reported below explored whether overheard (vicarious) analogs of self explanations, might, like deep questions, serve as knowledge catalysts that increase learning gains, first among college students in the laboratory (Exp. 1) and then among younger students studied individually in the classroom (Exp. 2). 2. Experiment 1 Unfortunately, students do not spontaneously generate self explanations at high rates, In fact, even when students are prompted, only a small proportion of their verbalizations are genuine self explanations. Instead students mostly verbalize self monitoring statements or paraphrase recently-presented content (Chi, 2000; Chi, de Leew, Chiu, & LaVancher, 1994). In fact, even when prompted about half of the students tested by Chi et al. (1994) failed to generate self explanations during an intensive study spanning as many as three or four daily sessions. While the students who generated explanations showed improved mental models and dramatic learning gains, those who failed to generate self explanations did not develop adequate mental models, and they performed at the same level as a comparison group who
Table 1 Pre and posttest scores for each experimental condition and knowledge level. Combined Pretest
M Q E QþE
Low knowledge Posttest
Change
Pretest
High knowledge Posttest
Change
Pretest
Posttest
Change
M
SD
M
SD
M
SD
M
SD
M
SD
M
SD
M
SD
M
SD
M
SD
.31 .31 .31 .36
.11 .10 .11 .16
.52 .52 .53 .52
.15 .17 .17 .16
.21 .21 .22 .16
.15 .16 .19 .20
.22 .25 .24 .22
.07 .06 .05 .08
.45 .45 .51 .53
.15 .15 .19 .13
.23 .21 .26 .31
.15 .17 .18 .09
.40 .42 .43 .45
.05 .06 .08 .12
.59 .63 .57 .52
.13 .13 .15 .18
.19 .21 .15 .07
.15 .15 .19 .20
1044
S.D. Craig et al. / Computers & Education 58 (2012) 1042–1048
simply read the content. Our aim in this research, then, was to present a vicarious “self” explanation, constructed according to criteria described by Chi (2000; Chi et al., 1989), following each content statement presented to the student. The question addressed in the present research is, then, might students use vicarious explanatory information of new content to expand existing knowledge structures and enhance learning. Our prediction, of course, was that presenting vicarious explanations following each sentence containing course content would, like deep questions, facilitate new learning by functioning as knowledge catalysts. The second purpose of Exp. 1 we also explore what Kalyuga, Ayres, Chandler, and Sweller (2003) called an expertise reversal effect. This effect suggests that, while low knowledge learners use programmatic instructional support to build organized knowledge structures and construct new mental models, those with more advanced knowledge structures may actually be disadvantages by targeted instructional support. This is because, for these learners, this instruction imposes considerable extrinsic cognitive load (Sweller, 1999), as they attempt to reconcile components of the instructed models with already existing advanced knowledge structures (Kalyuga, 2007). 3. Method 3.1. Participants The participants were 130 college students recruited from a research pool at a large urban university in the southern United States. Participants received credit for their participation toward introductory Psychology courses. Participants were allowed to stop their participation at any point during their experiment. 3.2. Materials 3.2.1. Assessments Two 20 item multiple-choice tests on Newtonian physics were counterbalanced as pretest and posttest for presentation to each participant. Test presentation order was counterbalanced so that approximately half received Test A as pretest with Test B as posttest, with the remainder presented the tests in reverse order. See Appendix A for example questions. 3.2.2. Experimental conditions A 30-min multimedia video was prepared for each of the four experimental conditions. Each computer-presented multimedia presentation involved two virtual agents who interacted by speaking. A virtual tutor was located on the right side of a monitor and a virtual tutee was located on the left side. The same series of relevant images were located between the two virtual agents in all conditions. In the monologue condition the virtual tutor presented 50 content statements, one at a time, while the virtual tutee simply listened. In the question condition (Q), the virtual tutee asked a deep question prior to each of the content statement presented by the virtual tutor. The deep questions used in the question condition were different from the questions used in the assessments. In the explanation condition (E), the virtual tutee listened to each of the virtual tutor’s content statement, and then provided an explanation. While these explanations elaborated on the content they did not provide any new knowledge required for a correct answer on the assessments. In the question þ explanation condition (Q þ E), the tutee asked the same question as in the question condition, and then following the content statement from the tutor, the tutee provided the same explanation that was presented in the explanation condition. Excerpts from each condition are available in Appendix B. 3.3. Procedures Each participant was designated as high or low knowledge level based on a median split on the pretest scores, yielding a 4 (experimental condition) 2 (knowledge level) design. When participants arrived at the lab for their session, they were randomized into one of the four experimental conditions. After filling out an informed consent, participants then took a 20 item pretest. They were then presented a 30 min video. Participants were only allowed to view the video one time and not allowed to rewind or fast forward. Afterward, they received a 20 item posttest. The session lasted about one hour. 3.3.1. Scoring Participants received one point for each question correct on the assessment tests. They were divided into the two knowledge levels based on the pretest data. This resulted in 70 participants in the low knowledge condition and 60 participants in the high knowledge condition. 4. Results and discussion An ANOVA performed on the pretest scores yielded no differences among the four experimental conditions at either knowledge level. A 4 (experimental condition) 2 (knowledge level) analysis of pretest-to-posttest gain scores yielded a significant interaction between knowledge level and experimental condition, F(3,122) ¼ 3.07, p < .03. Pretest-to-posttest learning gains for all conditions are presented in Table 1. We will focus on how the deep questions and explanations differentially affected learning gains among students with low previous knowledge vs. those with higher previous knowledge levels. Statistical comparisons revealed that learners in the low pre-experimental knowledge condition in the Q þ E condition showed significantly (p < .001) greater learning gains (gain ¼ 31%) than those with high pre-experimental knowledge (gain ¼ 7%). In fact, even though the former students showed considerably less pre-experimental knowledge (22% vs. 45% correct), they showed slightly higher scores on the post test. In the E condition, the gain in favor of those with low preexperimental knowledge (26%) approached significance (p < .08) when compared with those with high knowledge (15%). Comparisons among those in the four groups that exhibited low pre-experimental knowledge were also conducted. These comparisons revealed some suggestive difference. Learners in the Q þ E condition showed greater gains than those in all three of the remaining conditions (M, Q, E), but the differences were not significant (smallest p ¼ .08).
S.D. Craig et al. / Computers & Education 58 (2012) 1042–1048
1045
As predicted by the Kalyuga (2007) high knowledge reversal hypothesis, among those with high knowledge, the Q þ E condition showed very small pretest-to- to posttest gains (7%), compared to learners in the other three high-knowledge experimental conditions (14–21% gains), and actually showed significantly smaller pre-to-post gains (p < .05) than those in both the M (19%) and Q (21%) conditions. It seems, consistent with earlier findings (Kalyuga, 2007), that more knowledgeable learners were actually disadvantaged by that knowledge in the Q þ E condition, presumably due to conflict between the learner’s existing mental model and the model embodied in the newly presented instructional content. The instructional content, of course, enabled greater learning gains among those with low pre-experimental knowledge levels, particularly among those in the question þ explanation condition. 5. Experiment 2 Results of Exp. 1 indicate that vicarious self explanations of instructional content, like vicarious deep questions, may serve as knowledge catalysts that support learning. These findings, along with recent work showing that deep questions promote learning processes in the high school physics classroom (Gholson, Craig, etc submitted), provided the backdrop for Exp. 2. Brief videos were presented individually to each student in high school physics classes via laptop computers at the outset each of seven consecutive daily classroom sessions. The brief computer presentations were followed by standard classroom instruction provided by their regular teacher. Three different computer video presentations were prepared for each day of instruction, a total of 21. These were designed to provide the foundation of the teacher’s presentations each day. Because a teacher’s classroom presentations vary somewhat from class to class, even when presenting the same content, each of the three computer video was presented to approximately one third of the students in each of the five classrooms each day. 6. Method 6.1. Participants The participants were 143 students enrolled in high school physics classes in an inner-city school who were studied individually in their classrooms. Three of these classes were honors students (85 total) and the other two were standard (58 total students). Prior to the first of the seven daily classroom sessions each student’s and their parent’s informed consent was obtained. 6.2. Design and materials About one third of the students in each class were assigned to each one of three experimental conditions, question þ explanation (Q þ E), question (Q) or monologue (M). Thus, the experimental design was a 2 (achievement level: standard students vs. honors students) 3 (Experimental condition: Q þ E vs. Q vs. M). On Day 1 through Day 5 students studied Newton’s first Law and various component concepts, such as mass and friction. On Day 6 and Day 7 they studied an introduction to Newton’s second and third laws. On each day of instruction, Day 1 through Day 7, each learner was presented with two versions of a 10-item, 4-foil multiple-choice test used to measure learning gains on specific content presented during that daily session. As noted above, seven different sets of tests were constructed, one set to assess learning gains on each day of instruction. The two daily tests were presented to students in counterbalanced order (half received Test A as pretest with Test B as posttest, with the remainder presented the tests in reverse order), at the outset of each daily session prior to a computer presentation, and at the end of each session, after a teacher presentation (see below). The tests for each day were totaled. These totals were used for the analysis. Each of the three vicarious learning conditions involved a different computer-controlled multimedia presentation. The students, who wore headphones during the multimedia presentations, each worked independently at a laptop computer located on their own desk in their classroom. The multimedia presentations, provided during the first part of each of the seven consecutive classroom sessions, were each about 12 min in length. These presentations were designed to provide conceptual understanding of the content that would be presented by the teacher during the second part of that same daily classroom session. This latter part of each daily session, which was presented by the regular classroom teacher, was about 25 min in length. The teacher elaborated that day’s content while providing equations along with applications that gave quantitative expression to the concepts and relationships presented earlier in the computer-controlled vicarious learning presentations. 6.3. Experimental procedures Daily computer presentations in each experimental condition began with a brief spoken dialog between an animated virtual tutor and virtual tutee, located on two sides of each student’s monitor. This dialog served as introduction to that day’s topics. A series of images relevant to each concepts throughout the session were located on the monitor between the two agents in all conditions. Immediately following this introduction, the learner’s experimental condition was presented. In a virtual Q þ E condition, the presentation of each major concept in the curriculum involved a three-event sequence. First, the virtual tutee asked a deep question probing the concept. Second, the virtual tutor presented a content statement elaborating the meaning and use of the concept. And third, the virtual tutee provided an explanation of the concept that elaborated its meaning and attempted to provide a coherent context for the concept’s assimilation into an existing web of knowledge. An example of this three-event sequence is: (a) Virtual tutee’s question probing the concept: “What does the word mass mean as used here?”; (2) Virtual tutor’s content statement elaborating its meaning: “The mass in this case is known as ’gravitational mass’ and is measured in kilograms, or Kg”; (3) Virtual tutee’s explanation relating the concept to earlier content: “Mass, or gravitational mass is different from weight, in that mass is the same on the earth’s surface, in weightless space, and on the surface of the moon, but obviously the weight changes depending on where the object is located due to the different gravitational attractions in the different locations.” After this three-part dialog was completed, the three events involving the next concept in the curriculum were presented, etc. Each daily session included about 25 of these three-event sequences. In a virtual Q condition the only difference from the virtual Q þ E condition was that the third event, the explanation of the concept by the virtual tutee, was omitted from each three-event sequence. That is, the deep question concerning the concept was asked by the virtual tutee and
1046
S.D. Craig et al. / Computers & Education 58 (2012) 1042–1048
Table 2 Proportion gain score means and standard deviations for each experimental condition and classroom type. M ¼ Content Only; Q ¼ Questions; Q þ E ¼ Questions and explanations. Combined Pretest
M Q QþE
Standard classroom Posttest
Change
Pretest
Honors classroom
Posttest
Change
Pretest
Posttest
Change
M
SD
M
SD
M
SD
M
SD
M
SD
M
SD
M
SD
M
SD
M
SD
.32 .34 .32
.08 .08 .08
.49 .52 .54
.15 .11 .11
.17 .18 .22
.14 .10 .09
.35 .30 .32
.09 .05 .10
.45 .45 .48
.13 .11 .10
.10 .15 .16
.11 .12 .09
.29 .37 .32
.06 .09 .08
.53 .57 .58
.14 .07 .10
.24 .21 .26
.13 .08 .08
answered by the virtual tutor with a content statement, but no further explanation was provided by either virtual agent. In a virtual M condition both the first and third events in the sequence were omitted. Only the virtual tutor’s content statement was presented to the vicarious learner. 7. Results and discussion A 3 (experimental condition: Q þ E vs. Q vs. M) 2 (class type: standard vs. honors) ANOVA was performed on (pretest-to-posttest) gain scores on the tests used to evaluate daily learning across the seven sessions. It yielded significant effects of experimental condition, F(2137) ¼ 4.00, p < .01, and class type, F(1,13) ¼ 6.98, p < .01, but no interaction. The mean proportions and standard deviations of gain scores for standard and honors students in each condition may be found in Table 2. Simple effect procedures (Tukey HSD), collapsed across achievement levels, revealed that learners in the Q þ E condition significantly outperformed those in both the M (p < .01) and Q (p < .05) conditions. Learners in the Q þ E condition showed average pretest-to-posttest learning gains of 22%, while those in the M and Q conditions showed gains of 18% and 17%, respectively. Collapsed across experimental conditions, the honors students showed significantly greater gains (24%) than the standard students (15%). Because Newton’s first law (Day 1– Day 5) and the introduction to Newton’s second and third laws (Day 6 and Day 7) involved separate units in the physics course, learning data from the two units were also analyzed separately. As expected, analysis of data on the first law yielded exactly the same significant effects as the overall analyses (experimental condition, class type). The data from the introduction of the second and third laws yielded only a main effect of experimental condition, the class type effect was not significant. As results of the analyses indicate, the A þ E condition was quite successful in producing daily learning gains. The Q condition failed to produce gains relative to the M condition. We note that only about 25 concepts and questions were presented in each daily session. Previous research (e.g., Craig et al., 2006) has shown that in vicarious learning sessions in a computer learning environment about 100 content sentences preceded by deep questions per session may be needed to produce learning gains exceeding those produced by intelligent tutoring conditions or other vicarious learning conditions (Craig et al., 2006). The findings in the Q þ E condition do appear to indicate that vicarious deep explanations may, like vicarious deep questions, function as knowledge catalysts. Research now underway will further address this and related issues. 8. Conclusions The proposal that explanations which embed current course content into a web of existing knowledge might, like deep explanations, function as knowledge catalysts and increase learning received support from both Exp. 1 and 2. Among college students (Exp. 1) with low knowledge those in the Q þ E showed greater gains than any other group, and dramatically outperformed those with high knowledge in that condition (32% gain vs. 7%). Interestingly, in the high knowledge condition those in both the M and Q condition showed greater gains (19– 21%) than the 7% shown in Q þ E. This suggests, consistent with Kalyuga’s (2007) proposal, that among learners with advanced knowledge structures, providing targeted instruction designed to support model construction may actually inhibit new learning. This is because the instruction produces extrinsic cognitive load (Sweller, 1999) as learners attempt to reconcile the detailed instructional model with their existing mental model. Thus, the significantly greater learning gains among those with high knowledge in the M and Q conditions (than Q þ E) lends further support to Kalyuga’s proposal (Kalyuga et al., 2003; Sweller, 1999): greater detail in the instructed model produced more extrinsic cognitive load and smaller learning gains among learners with already refined knowledge structures. In Exp. 2, high school physics students received brief (12 min) video presentations at the outset of seven standard classroom sessions at the outset in the introduction to the study of Newton’s laws. Those in a Q þ E condition showed greater pretest-to-posttest gains than those in M and Q (no E condition was conducted) comparison groups, which did not differ from each other. As expected, students in honors classes showed greater pretest-to-posttest learning gains than students assigned to standard classes. From the two studies taken together, it seems reasonable to conclude that vicarious explanations presented immediately following course content statements function as knowledge catalysts by filling knowledge gaps that enhance learning. These findings, along with those described earlier, indicate that both deep questions and integrative explanations increase understanding and produce knowledge gains by embedding new content into webs of existing knowledge. In summary, these two studies show that integrative explanations following content statements support new learning, and are most effective with combined with deep questions that precede the content statements. While explanations alone were not shown to be effective, when combined with questions, they became effective for low knowledge learners. Moreover, as seen in Exp. 2, this combination of questions and explanations may be ideal for introducing complex concepts and topics using short videos presented prior to standard classroom instruction. Acknowledgements This research was supported by the Institute for Education Sciences (IES) Grant R305H0R0169. Any opinions, findings, and conclusions or recommendations expressed in this material are those of the authors and do not necessarily reflect the views of IES (DOE).
S.D. Craig et al. / Computers & Education 58 (2012) 1042–1048
1047
Appendix A What does the second part of Newton’s first law say about balanced forces? a. The object will move at a constant speed in a particular direction. b. Any moving object will seek its natural state of rest. c. The ancient Greeks were right and forces can never really be balanced. d. None of these. If there is a 4 kg block hanging from a rope in an elevator with resulting in a 40 N force, which of the following are true? a. If the tension on the rope is equal to 40 N, the 4 kg block will be stationary. b. If the tension on the rope is more than 40 N, the 4 kg mass will be stationary. c. If the tension on the rope is less than 40 N, the weight will be stationary. d. The tension on the rope will not affect the movement of the mass. If you tie a cart with a cable at the top of a hill so it does not fall down and scratch cars in the parking lot, what part of the weight of the cart is held up by the cable to keep it from rolling down the hill? a. Only the part of the weight that is parallel to the hill will attempt to get the cart to roll down the hill. b. The part of the weight pushing straight down due to gravity will try to push the cart down the hill c. The full weight of the cart. d. The weight of the cart parallel to the hill plus the weight of the cart perpendicular to the hill. What causes the force of tension to be observed? a. Tension occurs when an object is pushed with a rod. b. Tension occurs when an object is pulled on by a string. c. Tension occurs when an object is pulled downward by the earth. d. Tension occurs when an object is pulled down a ramp. Which of following events are due to force of tension? a. When you attempt to push a desk sideways that is not on wheels. b. When box slides down a ramp. c. When molecules attempt to stretch to counteract a weight force. d. When a book with a string attached to it is at fully support by the top of a table. How can you reduce tension on a rope? a. Increase the weight that is suspended. b. Decrease the friction by adding wheels to the object. c. Increase the gravity on the suspended object. d. Decrease the weight that is suspended.
Appendix B Below examples are given for the content statements for study one and study two (M,Q, and Q þ E only). Monolog (M) condition Tension is another force like weight. The symbol for tension is T. Tension is measured in Newtons just like weight (A large, red apple has a weight of about 1 Newton). The tension force will appear in problems where a string, line, rope, cord chain, rod, dental floss, cable and so forth are pulling on an object. In all cases the strings and things can pull but cannot push since ropes, cords and cable will collapse when pushed at one end, so the rod is the only exception here in that you can push or pull with a rod and still have tension. Questions (Q) condition Question: People talk about tension all the time, what does the physicist mean by it? Tension is another force like weight. The symbol for tension is T. Question: How does the physicist go about measuring tension? Tension is measured in Newtons just like weight (A large, red apple has a weight of about 1 Newton). Question: How does tension happen? The tension force will appear in problems where a string, line, rope, cord chain, rod, dental floss, cable and so forth are pulling on an object. Question: How can you have tension when you push on a rope or something like that? In all cases the strings and things can pull but cannot push since ropes, cords and cable will collapse when pushed at one end, so the rod is the only exception here in that you can push or pull with a rod and still have tension. Explanations (E) condition Tension is another force like weight. The symbol for tension is T. Explanation: So tension is a force like weight, but we call the Tension force T.
1048
S.D. Craig et al. / Computers & Education 58 (2012) 1042–1048
Tension is measured in Newtons just like weight (A large, red apple has a weight of about 1 Newton). Explanation: So tension T is measured in Newtons, just like weight is. The tension force will appear in problems where a string, line, rope, cord chain, rod, dental floss, cable and so forth are pulling on an object. Explanation: So tension T has to do with holding up on object pulling down on it. In all cases the strings and things can pull but cannot push since ropes, cords and cable will collapse when pushed at one end, so the rod is the only exception here in that you can push or pull with a rod and still have tension. Explanation: So of the examples used, only s rod can be used to push or pull and have tension either way. Questions and Explanations (Q þ E) condition Question: People talk about tension all the time, what does the physicist mean by it? Tension is another force like weight. The symbol for tension is T. Explanation: So tension is a force like weight, but we call the Tension force T. Question: How does the physicist go about measuring tension? Tension is measured in Newtons just like weight (A large, red apple has a weight of about 1 Newton). Explanation: So tension T is measured in Newtons, just like weight is. Question: How does tension happen? The tension force will appear in problems where a string, line, rope, cord chain, rod, dental floss, cable and so forth are pulling on an object. Explanation: So tension T has to do with holding up on object pulling down on it. Question: How can you have tension when you push on a rope or something like that? In all cases the strings and things can pull but cannot push since ropes, cords and cable will collapse when pushed at one end, so the rod is the only exception here in that you can push or pull with a rod and still have tension. Explanation: So of the examples used, only s rod can be used to push or pull and have tension either way. References Ainsworth, S., & Burcham, S. (2007). Learning and instruction. Learning and Instruction, 17, 286–303. Ainsworth, S., & Loizou, A. T. (2003). The effects of self explaining when learning with text or diagrams. Cognitive Science, 27, 669–681. Bandura, A. (1962). Social learning through imitation. In M. R. Jones (Ed.), Nebraska symposium on motivation (pp. 211–269). Lincoln: University of Nebraska Press. Bandura, A. (1977). Social learning theory. Englewood Cliffs, NJ: Prentice Hall. Bloom, B. S. (1956). Taxonomy of education objectives: the classification of educational goals. In Handbook I: Cognitive domain. New York: McKay. Bloom, B. (1977). Affective outcomes of school learning. Phi Delta Kappan, 59, 193–198. Chi, M. T. H. (2000). Self explaining expository texts: the dual process of generating inferences and repairing mental models. In R. Glaser (Ed.), Advances in instructional psychology (pp. 161–238). Mahwah, NJ: Lawrence Erlbaum Associates. Chi, M. T. H., Bassok, M., Lewis, M. W., Reimann, P., & Glaser, R. (1989). Self-explanations: how students study and use examples in learning to solve problems. Cognitive Science, 13, 145–182. Chi, M. T. H., de Leew, N., Chiu, M., & LaVancher, C. (1994). Eliciting self-explanations improves understanding. Cognitive Science, 18, 439–477. Chi, M. T. H., Roy, M., & Hausmann, R. G. M. (2008). Observing tutorial dialogues collaboratively: insights about human tutoring effectiveness from vicarious learning. Cognitive Science, 32, 301–341. Chi, M. T. H., Siler, S. A., Jeong, H., Yamauchi, T., & Hausmann, R. G. (2001). Learning from human tutoring. Cognitive Science, 25, 471–533. Craig, S. D., Brittingham, J., Williams, J., Cheney, K. R., & Gholson, B. (2009). Incorporating vicarious learning environments with discourse scaffolds into physics classrooms. In V. Dimitrova, R. Mizoguchi, B. du Boulay, & A. C. Graesser (Eds.), Artificial intelligence in education, building learning systems that care: From knowledge representation to affective modeling (pp. 680–682). Washington, DC: IOS Press. Craig, S. D., Driscoll, D., & Gholson, B. (2004). Constructing knowledge from dialogue in an intelligent tutoring system: Interactive learning, vicarious learning, and pedagogical agents. Journal of Educational Multimedia and Hypermedia, 13, 163–183. Craig, S. D., Gholson, B., & Driscoll, D. (2002). Animated pedagogical agents in multimedia educational environments: effects of agent properties, picture features, and redundancy. Journal of Educational Psychology, 94, 428–434. Craig, S. D., Gholson, B., Ventura, M., Graesser, A. C., & The Tutoring Research Group. (2000). Overhearing dialogues and monologues in virtual tutoring sessions: effects on questioning and vicarious learning. International Journal of Artificial Intelligence in Education, 11, 242–253. Craig, S. D., Graesser, A., Brittingham, J., Williams, J., Martindale, T., Williams, G., et al. (2008). An implementation of vicarious learning environments in middle school classrooms. In K. McFerrin, R. Weber, R. Weber, R. Carlsen, & D. A. Willis (Eds.), The proceedings of the 19th international conference for the Society for Information Technology & Teacher Education (pp. 1060–1064). Chesapeake, VA: AACE. Craig, S. D., Sullins, J., Witherspoon, A., & Gholson, B. (2006). The deep-level reasoning effect: the role of dialogue and deep-level-reasoning questions during vicarious learning. Cognition and Instruction, 24, 565–591. Driscoll, D., Craig, S. D., Gholson, B., Ventura, M., Hu, X., & Graesser, A. C. (2003). Vicarious learning: effects of overhearing dialogues and monologue-like discourse in a virtual tutoring session. Journal of Educational Computing Research, 29, 431–450. Gholson, B., Coles, R., & Craig, S. D. (2010). Features of computerized multimedia environments that support vicarious learning processes. In M. S. Khine, & I. M. Saleh (Eds.), New science of learning: Cognition, computers and collaboration in education (pp. 53–78). New York: Springer. Gholson, B., & Craig, S. D. (2006). Promoting constructive activities that support learning during computer-based instruction. Educational Psychology Review, 18, 119–139. Gholson, B., Craig, S. D., Brittingham, J., Germany, M.-L., & Fike, K. (2009, July). The role of deep level questions, “self-explanations,” and domain knowledge in vicarious learning. In Presented at Society for Text and Discourse, July 2009, Rotterdam, NL. Gholson, B., Witherspoon, A., Morgan, B., Brittingham, J., Coles, R., Graesser, A. C., et al. (2009). Exploring the deep-level reasoning questions effect during vicarious learning among eighth to eleventh graders in the domains of computer literacy and Newtonian physics. Instructional Science, 37, 487–493. Graesser, A. C., Baggett, W., & Williams, K. (1996). Question-driven explanatory reasoning. Applied Cognitive Psychology, 10, S17–S32. Graesser, A. C., Franceschetti, D., Gholson, B., & Craig, S. (2011). Learning Newtonian physics with conversational agents and interactive simulation. In N. Stein, & S. W. Raudenbush (Eds.), Developmental cognitive science goes to school (pp. 157–172). New York: Routledge. Graesser, A. C., & Olde, B. A. (2003). How does one know whether a person understands a device? The quality of the questions the person asks when the device breaks down. Journal of Educational Psychology, 95, 524–536. Graesser, A. C., & Person, N. (1994). Question asking during tutoring. American Educational Research Journal, 31, 104–137. Graesser, A. C., Person, N., Harter, D., & The Tutoring Research Group. (2001). Teaching tactics and dialogue in AutoTutor. International Journal of Artificial Intelligence in Education, 12, 257–279. Graesser, A. C., Wiemer-Hastings, P., Wiemer-Hastings, K., Harter, D., Person, N., & Tutoring Research Group. (2000). Using latent semantic analysis to evaluate the contributions of students in AutoTutor. Interactive Learning Environments, 8, 129–148. Kalyuga, S., Ayres, P., Chandler, P., & Sweller, J. (2003). Expertise reversal effect. Educational Psychologist, 38, 23–31. Kalyuga, S. (2007). Expertise reversal effect and its implications for learner-tailored instruction. Educational Psychology Review, 19, 509–539. Rosenshine, B., Meister, C., & Chapman, S. (1996). Teaching students to ask questions: a review of intervention studies. Review of Educational Research, 66, 181–221. Rosenthal, R. L., & Zimmerman, B. J. (1978). Social learning and cognition. New York: Academic Press. McKendree, J., Good, J., & Lee, J. (2001, June). Effects of dialogue characteristics on performance of overhearers. In Presented at the international conference on communication, problem-solving, and learning, Strathclyde. Sweller, J. (1999). Instructional design in technical areas. Melbourne, Australia: ACER Press. VanLehn, K., Graesser, A. C., Jackson, G. T., Jordan, P., Olney, A., & Rosé, C. P. (2007). Natural language tutoring: a comparison of human tutors, computer tutors and text. Cognitive Science, 31, 3–62.