The effects of augmented virtual science laboratories on middle school students' understanding of gas properties

Accepted Manuscript The Effects of Augmented Virtual Science Laboratories on Middle School Students’ Understanding of Gas Properties Jennifer L. Chiu, Crystal J. DeJaegher, Jie Chao PII:

S0360-1315(15)00051-2

DOI:

10.1016/j.compedu.2015.02.007

Reference:

CAE 2798

To appear in:

Computers & Education

Received Date: 7 August 2014 Revised Date:

29 December 2014

Accepted Date: 7 February 2015

Please cite this article as: Chiu J.L., DeJaegher C.J. & Chao J., The Effects of Augmented Virtual Science Laboratories on Middle School Students’ Understanding of Gas Properties, Computers & Education (2015), doi: 10.1016/j.compedu.2015.02.007. This is a PDF file of an unedited manuscript that has been accepted for publication. As a service to our customers we are providing this early version of the manuscript. The manuscript will undergo copyediting, typesetting, and review of the resulting proof before it is published in its final form. Please note that during the production process errors may be discovered which could affect the content, and all legal disclaimers that apply to the journal pertain.

ACCEPTED MANUSCRIPT

The Effects of Augmented Virtual Science Laboratories on Middle School Students’ Understanding of Gas Properties

RI PT

Jennifer L. Chiu, Crystal J. DeJaegher, Jie Chao University of Virginia, 405 Emmet Street South, Charlottesville, VA 22904 USA

SC

Corresponding author: Jennifer L. Chiu ADDRESS: 313 Bavaro Hall, Curry School of Education, University of Virginia, Charlottesville, VA, 22904, USA. PHONE: 650 5759848, FAX: 434 9247461, EMAIL: [email protected]

EP

TE D

M AN U

Abstract The Next Generation Science Standards (NGSS) emphasize authentic scientific practices such as developing models and constructing explanations of phenomena. However, research documents how students struggle to explain observable phenomena with molecular-level behaviors with current classroom experiences. For example, physical laboratory experiences in science enable students to interact with observable scientific phenomena, but students often fail to make connections to underlying molecular-level behaviors. Virtual laboratory experiences and computer-based visualizations enable students to interact with unobservable scientific concepts, but students can have difficulties connecting to actual instantiations of the observed phenomenon. This paper investigates how combining physical and virtual experiences into augmented virtual science laboratories can help students build upon intuitive ideas and develop molecular-level explanations of macroscopic phenomena. Specifically, this study uses the Frame, a sensoraugmented virtual lab that uses sensors as physical inputs to control scientific simulations. Eighth-grade students (N=45) engaged in a Frame lab focused on the properties of gas. Results demonstrate that students using the Frame lab made progress developing molecular-level explanations of gas behavior and refining alternative and partial ideas into normative ideas about gases. This study offers insights for how augmented virtual labs can be designed to enhance science learning and encourage scientific practices as called for in the NGSS.

AC C

Highlights • Combining virtual and physical labs has potential to promote science learning. • Minimal research exists on augmented virtual technologies in authentic classrooms. • Students use physical controls of the Frame to manipulate rich, virtual simulations • Pilot tests show student improvement on explanations about gas properties. Keywords Evaluation of CAL systems; Improving classroom teaching; Interactive learning environments; Multimedia/hypermedia systems

ACCEPTED MANUSCRIPT Running head: EFFECTS OF AUGMENTED VIRTUAL LABS ON SCIENCE UNDERSTANDING

1

RI PT

The Effects of Augmented Virtual Science Laboratories on Middle School Students’ Understanding of Gas Properties

AC C

EP

TE D

M AN U

SC

1. Introduction The Next Generation Science Standards (NGSS) stress authentic scientific practices such as developing models and constructing explanations of phenomena (NGSS Lead States, 2013). However, research documents that students have difficulty developing molecular-level explanations of observable phenomena, critical to complex science understanding (Ben-Zvi, Eylon, & Silberstein, 1986; Gabel, 1999). Moreover, existing approaches to science classes can leave students with isolated or superficial ideas (Linn & Eylon, 2011). For example, hands-on laboratory experiences give students direct experience with phenomena and scientific practices (National Research Council [NRC], 2006), but are not always successful in getting students to understand underlying scientific concepts (Finkelstein et al., 2005). Virtual technology tools, software, and simulations, have been successfully implemented in science classrooms to help students develop explanations of complex science topics (Bell & Trundle, 2008; Carlsen & Andre, 1992; Chiu & Linn, 2014; Höffler & Leutner, 2007; Honey & Hilton, 2011; Jaakola, Nurmi, & Veermans, 2011; Windschitl & Andre, 1998). However, research also demonstrates how students using visualizations can focus on superficial aspects (Lowe, 2004), overestimate their understanding (Chiu & Linn, 2012), and fail to connect virtual representations to the depicted real-life scientific phenomena (Chiu, 2010). Research that combines physical and virtual labs either sequentially or side-by-side demonstrates that careful consideration of physical and virtual affordances can support the development of scientific understanding (Blikstein, Fuhrmann, Greene, & Salehi, 2012; Olympiou & Zacharia, 2012). Augmented virtual technologies offer an innovative approach to science laboratories by combining virtual and physical components to provide enhanced educational experiences (Lindgren & Johnson-Glenberg, 2013). Augmented virtual technologies use virtual tools to represent scientific phenomena (e.g., simulations, visualizations) that are enhanced by using physical real-life objects as controls. This paper focuses on the pilot testing of the Frame, a specific augmented virtual technology that uses probeware (i.e., temperature and pressure sensors) as inputs to simulations of scientific phenomena (Xie, 2012), enabling students to use real-world objects to control the simulation. For instance, instead of students moving a gas-filled piston with a mouse in a simulated environment, students impart a force on a physical spring that inputs the information into the simulation. The overall goal of this study is to explore if augmented virtual science laboratories such as the Frame can be used in authentic classroom settings to help students develop scientifically normative explanations of gas properties. In particular, this study seeks to answer the following questions: 1. Can augmented virtual Frame labs help middle school students develop explanations that connect molecular behaviors to macroscopic properties of gas? 2. Can augmented virtual Frame labs help middle school students refine alternative ideas about macroscopic gas phenomena?

ACCEPTED MANUSCRIPT EFFECTS OF AUGMENTED VIRTUAL LABS ON SCIENCE UNDERSTANDING

2

2. Background

AC C

EP

TE D

M AN U

SC

RI PT

2.1 Supporting Science Learning through Grounded Knowledge Integration Students enter the classroom with a wide range of experiences that contribute to the framework with which they interpret the world (Bransford, Brown & Cocking, 2000). Students’ experiential understandings can serve as fruitful places to build understanding, but can also counter and even interfere with developing scientifically normative explanations of phenomena (Duschl & Gitomer, 1991). Research demonstrates that students often struggle to understand difficult science concepts because they have trouble integrating scientifically normative explanations into their existing knowledge base (Clough & Driver, 1985; Erickson & Tiberghien, 1985; Jones, Carter, & Rua, 2000), especially for topics that include unobservable levels, such as gas laws and kinetic molecular theory (e.g., Nakleh, 1992; Novick & Nussbaum, 1981). Students often confuse molecular and macroscopic levels of a phenomenon (Ardac & Akaygun, 2004; Ben-Zvi, Eylon, & Silberstein, 1986) or misattribute characteristics of one level to another (e.g., Wilensky & Resnick, 1999). Students’ alternative ideas about molecular behaviors can stem from everyday macroscopic experiences. For example, many students believe that when there are more particles in a fixed container, particles have less room to move and thus move slowly (Levy, Novak, & Wilensky, 2006). Conflicts between experiential and scientific understandings can be challenging to overcome but can also serve as particularly fruitful places to help students engage in conceptual change or the restructuring of ideas (Chi, 2008; Posner, Strike, Hewson, & Gertzog, 1982; Vosniadou, 1994). Providing students with opportunities to make and refine connections between everyday and normative ideas can be particularly successful for science learning (Clark, 2006; Clark & Jorde, 2004; Levy & Wilensky, 2009; Shen & Linn, 2011). To help students make connections among molecular and macroscopic levels and help students leverage everyday ideas with the Frame, this study uses a knowledge integration (KI) learning perspective. The KI framework provides guidance for instructional strategies that encourage students to use their existing knowledge base and build understandings by making, refining and sorting connections among ideas (Clark, 2006; Linn & Eylon, 2011). Creating an environment that fosters knowledge integration elicits students’ existing understandings, adds new ideas for students to consider, gives students with an opportunity to distinguish between ideas, and provides a framework for students to sort these ideas (Linn & Eylon, 2006). KI values experiences where students bring their experiential and scientific knowledge forward, so that conflicts can be identified and connections made to create more cohesive networks of understanding (Linn & Eylon, 2011). Many studies demonstrate how KI instructional strategies help students develop connected understanding of science (Chiu & Linn, 2014; Linn, Davis & Eylon, 2004; McElhaney & Linn, 2011; Zhang & Linn, 2011). We also leverage embodied and grounded approaches to cognition as a framework for learning from augmented virtual technologies. Embodied cognition perspectives recognize the role of bodily actions on cognition (e.g., Lakoff & Johnson, 1980; Wilson, 2002). Grounded approaches to cognition contend that the mind stores information across perceptual, motor, and affective states and uses an integrated multimodal representation to build knowledge (Barsalou, 2008). Thus, conceptual understanding involves and connects to ideas that are grounded in the experiences of the learner, which complements the importance of experiential ideas in the process of knowledge integration (e.g., Clark, 2006; Lewis & Linn, 1994). A learner’s understanding of a science concept involves a network of ideas associated with what he or she

ACCEPTED MANUSCRIPT EFFECTS OF AUGMENTED VIRTUAL LABS ON SCIENCE UNDERSTANDING

3

RI PT

sees, hears, and feels during instruction, as well as past networks of ideas associated with other, prior learning. Embodied approaches to learning – that is, having students learn material through physical interaction as well as through seeing and hearing about the material - can result in improved conceptual understanding (e.g., Abrahamson, Gutiérrez, Charoenying, Negrete, & Bumbacher, 2012; Anastopoulou, Sharples, & Baber, 2011; Tolentino, Birchfield, MegowanRomanowicz, Johnson-Glenberg, Kelliher, & Martinez, 2009).

M AN U

SC

2.2 Physical Labs in Science Classrooms Science instruction has traditionally used hands-on laboratories to help activate experiential ideas and engage with scientific phenomena. Physical laboratory experiences benefit students by incorporating concrete objects in science learning; some students can better engage with science when they are able to touch, move, and examine real objects (Feisel & Rosa, 2005). Physical laboratories also give students opportunities to interact directly with the scientific phenomena being studied (Lunetta, Hofstein & Clough, 2007). Despite the widespread use of physical labs in science, hands-on (not computer-based) labs typically do not provide representations for unseen levels such as atoms and molecules (e.g., Hodson, 1996; Hofstein & Lunetta, 2004; Tobin, 1990). As a result, students can have difficulty connecting physical labs with molecular-level ideas and can have difficulties integrating observable and molecular accounts of phenomena (Gabel, 1999; Johnstone, 1991). For example, students can engage in a gas laws lab that investigates the relationship between pressure and volume to help them understand the inverse relationship that as one gets bigger, the other decreases. However, after completion students may still not understand or connect these results to the molecular behaviors that explain why this happens on an atomic level (Liu, 2006).

AC C

EP

TE D

2.3 Visualizations in Science Classrooms Scientific visualizations provide experiences for students to observe and interact with typically unobservable levels of phenomena (Honey & Hilton, 2011). Many studies demonstrate how visualizations can help students understand the particulate nature of matter, gas laws, and other molecular-level concepts (e.g., Kozma & Russell, 1997, Levy & Wilensky, 2009, Steiff and Wilensky, 2003; Wu, Krajcik, & Soloway, 2001). Visualization-based instruction can also help students make connections from molecular levels to observable levels by providing multiple, linked representations of phenomena (Kozma, 2003) or explicit guidance through instruction (Chiu & Linn, 2014; Levy, 2012; Zhang & Linn, 2011). For example, Connected Chemistry (Levy & Wilensky, 2009) anchored a visualization-based gas laws curriculum in the macroscopic world and used instruction to help students connect virtual NetLogo models to the particulate level. Students made significant gains from pretest to posttest on items assessing understanding of submicro-macroscopic connections with a medium effect size. Guiding students to make connections between visualized molecular-level phenomena and the observable world, as with the Connected Chemistry curriculum, can enhance conceptual understanding (Steiff & Wilensky, 2003; Wu, Krajcik, & Soloway, 2001). Connecting visualizations to everyday, experiential ideas can also benefit student learning. Clark & Jorde (2004) compared two versions of a simulation: one with a visualization of heat with an explicit representation of a macroscopic hand and virtual feedback about what it would feel like touching the object and one with a visualization of heat without the hand or feedback. Students who used the visualization with the hand outperformed students without the hand on post and delayed posttests and demonstrated significant improvement on written

ACCEPTED MANUSCRIPT EFFECTS OF AUGMENTED VIRTUAL LABS ON SCIENCE UNDERSTANDING

4

explanations of thermal concepts. Results demonstrate that connecting visualizations to experientially acquired ideas can help students refine and articulate understanding of complex science concepts.

AC C

EP

TE D

M AN U

SC

RI PT

2.4 Combining Physical and Virtual labs Although many empirical studies demonstrate that visualization-based instruction can improve student understanding of scientific phenomena (Dori & Belcher, 2005; Finkelstein et al., 2005; Zacharia & Anderson, 2003), studies also show that students have difficulty learning from visualizations (Tversky, Morrison, & Betrancourt, 2002). Students may focus on superficial aspects instead of targeted learning outcomes (Cook, 2006) or overestimate their understanding of material presented visually (Chiu, King-Chen, & Linn, 2013). For instance, Lowe (2004) found that students learning from an interactive weather map tended to notice salient, small aspects of space or time but had difficulty building coherent descriptions of weather that integrated aspects across the visualization. Similarly, Chiu and Linn (2012) found that students working with molecular visualizations tended to rate themselves as more knowledgeable immediately after interacting with a visualization as opposed to after asking them to explain what they learned from the visualization, suggesting that dynamic visualizations can give students an illusion of understanding the underlying concepts. Providing sequential physical and virtual experiences has shown promise in helping students to develop understanding of phenomena (Gire et al., 2010; Zacharia, 2007; Zacharia & Anderson, 2003; Zacharia & Olympiou, 2011; Zollman, Rebello, & Hogg, 2002). For example, Olympiou and Zacharia (2012) compared students experimenting with physical, virtual, and both physical and virtual manipulatives to learn optics. Students using a combination of both physical and virtual manipulatives outperformed students using either physical or virtual manipulatives alone (Zacharia, 2007). Although the effects of the order of manipulative type on student outcomes is relatively mixed (Smith & Puntambekar, 2010), the combination of physical and virtual has been shown to be successful and offers greater affordances than either approach in isolation (de Jong, Linn, & Zacharia, 2013). In addition to sequential combinations, research demonstrates that students can use both physical and virtual materials at the same time to develop sophisticated views of science. For instance, students in after-school settings developed virtual and physical models side-by-side with biology, chemistry, and physics content. Students used mismatching information from real and virtual models to develop deeper understanding of phenomena (Blikstein et al., 2012). This kind of side-by-side or bifocal approach can encourage deep learning in science by engaging students in authentic scientific practices (Blikstein, 2012). However, combining physical and virtual manipulatives either sequentially or side-byside does not guarantee that students will make connections among levels (e.g., McBride, Murphy, & Zollman, 2010). Building upon these findings, providing students with physical and virtual manipulatives simultaneously connected has potential to help students make links among molecular and observable levels in science. Mixed-reality technologies can use physical interactions to control digital environments, capitalizing on the affordances of both physical and virtual manipulatives (Borgman et al., 2008). Studies demonstrate the potential of mixed-reality technologies to enhance science learning (Johnson-Glenberg, Koziupa, Birchfield, & Li, 2011; Lui & Slotta, 2013; Novellus & Moher, 2011). However, relatively few empirical studies have investigated learning in authentic classroom settings (Lindgren & Johnson-Glenberg, 2013). The few studies that have been implemented in classrooms demonstrate the potential of mixed-reality

ACCEPTED MANUSCRIPT EFFECTS OF AUGMENTED VIRTUAL LABS ON SCIENCE UNDERSTANDING

5

SC

RI PT

approaches to enhance traditional instruction (e.g., Johnson-Glenberg, Birchfield, Tolentino, & Koziupa, 2013). Specifically, this paper explores how sensor-augmented virtual technologies may help students develop scientifically normative explanations that link molecular-level behaviors to observable phenomena. Augmented virtual technologies use virtual phenomena augmented with “real” objects, as opposed to augmented reality, where the real phenomena is augmented with virtual objects (Milgram & Kishino, 1994). Augmented virtual technologies capitalize on affordances of simulations and dynamic visualizations by enabling students to not only see but also interact with typically unobservable processes such as molecular behaviors. The “real” augmentation leverages advantages of physical labs such as direct manipulation of phenomena and tactile feedback. By simultaneously connecting real and virtual objects, augmented virtual technologies have the potential to help students connect molecular behaviors to the observable world.

AC C

EP

TE D

M AN U

2.5 The Frame Technology This paper presents exploratory work using the Frame, a sensor-augmented virtual technology that uses physical interactions with sensors to control a molecular simulation (Xie, 2012; Figure 1). The Frame engages students through a simulation of a phenomenon (in this case, gas molecules in a chamber) controlled or augmented by physical inputs. Although the

Figure 1. The Frame apparatus (top left) with a molecule “pump” on the left, spring piston controller on the right, and a jar for holding hot/cold water placed next to the temperature sensor. Students interact with the visualization (top right) through physical controls (bottom).

ACCEPTED MANUSCRIPT EFFECTS OF AUGMENTED VIRTUAL LABS ON SCIENCE UNDERSTANDING

6

AC C

EP

TE D

M AN U

SC

RI PT

Frame can be configured to help students understand various topics such as heat transfer and electricity (Xie, 2012) this study focuses on helping students understand gas properties. The particulate nature of matter (PNM) is fundamental to complex science understanding (Ozmen, 2013) as it undergirds many chemistry topics (Gabel et al., 1997; Nakleh, 1992; Snir et al., 2003) as well as topics in physical, life, and earth sciences (Benson et al., 1993; Bouwma-Gearhart et al., 2009; Lee et al., 1993; Noh & Scharmann, 1997). As a result, the PNM is a central subject in middle and high school curricula (e.g., NGSS, 2013). However, research demonstrates that students at all ages, even chemistry graduate students and teachers, have a variety of ideas about the PNM, gases, and gas laws (Bodner, 1991; Krajcik, 1995; Lin, Cheng, & Lawrenz, 2000; Nakhleh, 1992). Traditional physical labs typically have students investigate relationships between volume and pressure or temperature and pressure at the observable level with probeware or other equipment, but can leave students with alternative ideas about why these macroscopic relationships occur. Research demonstrates that technology-enhanced approaches can help students understand chemical phenomena on a molecular level (e.g., Ardac & Akaygun, 2004; Jones, Jordan & Stillings, 2005; Kelly & Jones, 2007; Madden, Jones & Rahm, 2011; Sanger et al., 2000; Stieff & Wilensky, 2003; Wu et al., 2001) and in particular, dynamic visualizations can help students develop molecular understanding of gas behavior (Levy & Wilensky, 2009). Combining physical and virtual approaches has also been beneficial for students studying gas laws. For example, Liu (2006) had high school students sequentially engage in both computer-based molecular simulations and a hands-on lab to learn about gas laws. Results demonstrated that the combination of a physical and virtual lab was better than either approach alone to develop conceptual understanding of gas laws. Additionally, evidence suggests that mixed-reality approaches may help students engage in scientific practices that contribute to understanding gas phenomena. Blikstein (2014) piloted technologies that explicitly linked the volume of a physical syringe to a molecular simulation of gas in real time. Students could press on a physical syringe and simultaneously see the volume of a virtual syringe change. Students could then compare pressure outputs from both the actual syringe and the computer model of the syringe through a bifocal pedagogical approach. Pilot results with 11 high school students over a single 6-hour period suggest that the mixed-reality technologies encouraged sophisticated scientific practices and fostered inquiry into the targeted scientific phenomena. Building from these studies, this paper investigates how the Frame can be used in classroom settings to help students learn about gas phenomena. Unlike a side-by-side approach, the Frame uses a simulation as the basis for student experimentation and augments the simulation with physical controls. Students feel as if they are directly experimenting with gas molecules, leveraging the different affordances of virtual and physical labs into a singular experience. For example, students place jars filled with hot water near a temperature sensor to increase the temperature of the simulated gas. Students can physically push on the Frame to increase force on the force sensor and thus the virtual piston in the simulation. Students can also increase or decrease the number of molecules in the simulation through a physical pump. The simulation component of the frame offers a dynamic visualization through which students can also interact using a touch-screen interface. We believe that a grounded knowledge integration perspective provides a lens to understand how the Frame can help students understand gas laws. Specifically, the physical controls of the Frame can bring forward everyday, intuitive ideas that students may hold about phenomena through the tactile interaction and feedback. When students interact with physical

ACCEPTED MANUSCRIPT EFFECTS OF AUGMENTED VIRTUAL LABS ON SCIENCE UNDERSTANDING

7

M AN U

SC

RI PT

controls, the feedback from the real world can elicit past experiences and ideas that are associated with the specific action. For example, when students physically push the spring of the Frame and feel the force of the spring pushing back, this helps activate related, everyday ideas about force. The simulation or virtual manipulative can then help students add normative ideas about the phenomenon – in this case, how a gas exerts pressure and force against a piston wall. Both the intuitive and scientific ideas are activated at the same time, enabling students to distinguish productive from less productive ideas and make connections among their ideas with proper guidance. Activating intuitive and scientific ideas simultaneously provides an opportunity for students to sort out and refine productive ideas from less productive ideas, resulting in more coherent networks of ideas (Clark, 2006; Linn & Eylon, 2006). Because students interact with physical, real world controls and see the resulting behaviors of molecules, the Frame has the potential to help learners connect macroscopic, experiential ideas to visualized molecular behaviors and refine alternative ideas. This paper explores the potential of augmented virtual labs in authentic science classrooms to help students develop normative explanations of phenomena. As an exploratory study, this paper first reports on how students used the Frame lab in authentic classroom settings, then specifically investigates if middle school students can use the Frame to construct and refine explanations that connect molecular behaviors to macroscopic gas properties as well as help students refine alternative ideas about gas phenomena. 3. Methods 3.1 Curriculum

AC C

EP

TE D

3.1.1 Development The Gas Laws curriculum used for this implementation of the Frame technology was developed in partnership among educational researchers, scientists, and teachers. The group chose gas laws as it aligned with both state and national science content standards (e.g., NGSS DCI MS-PS1.A and MS-PS1-4). The group also chose to emphasize the NGSS practice of constructing explanations because of the synergies with the goals of the lab, as “scientific explanations are explicit applications of a theory to a specific situation or phenomenon” (NRC, 2012; p. 52) and the “goal of science is to construct explanations for the causes of phenomena” (NGSS, 2013; p. 11). Specific learning objectives for scientific practices involved “constructing an explanation that includes qualitative or quantitative relationships between that predict(s) and/or describe(s) phenomenon” as well as “apply scientific ideas, principles, and/or evidence to construct, revise, and/or use an explanation for real world phenomenon, examples, or events” (NGSS, 2013; p. 11). The collaborative effort identified difficult topics for students when learning about gas laws based upon both prior research (e.g., Nakhleh, 1992) and the teachers’ experience, specifically that: gas molecules are in constant motion (Herrman-Abell & DeBoer, 2011), heating and cooling cause changes in particle motion, not size (Novick & Nussbaum, 1981), gas particles at the same temperature travel at different speeds (e.g., temperature is a measure of the average kinetic energy of the molecules), and gas pressure is a result of molecular collisions with container walls (Lin, Cheng, & Lawrenz, 2000). The Frame lab was specifically designed to target common alternative student ideas around these concepts through engaging in constructing explanations.

ACCEPTED MANUSCRIPT EFFECTS OF AUGMENTED VIRTUAL LABS ON SCIENCE UNDERSTANDING

8

TE D

M AN U

SC

RI PT

The curriculum design built upon research-based best practices with both physical and virtual labs to provide the best possible learning experience for the students (e.g., Chang, Quintana, & Krajcik, 2009; Duschl, Schweingruber, & Shouse, 2007; Hofstein & Lunetta, 2004; Levy & Wilensky, 2009; Olympiou & Zacharia, 2012; Steiff & Wilensky, 2003; Wu, Krajcik, & Soloway, 2001). Specifically, curriculum development leveraged research that demonstrates the benefit of guided inquiry activities for science learning (e.g., Hmelo-Silver, Duncan, & Chinn, 2006; Quintana et al., 2004), as well as the benefit of technology-enhanced inquiry learning environments to support scientific practices and learning (Donnelly, Linn, & Ludvigsen, 2014). The Gas Frame lab used a scaffolded knowledge integration instructional approach (Linn, Davis, & Eylon, 2004) and the Web-based Inquiry Science Environment (WISE) (Figure 1; Slotta & Linn, 2009), to support students’ investigations with the Frame. The KI instructional approach promotes design patterns that help students elicit, add, distinguish and refine their ideas about science concepts (Linn & Eylon, 2006). The Gas Frame Lab primarily leveraged a modified predict-observe-explain-reflect instructional pattern (e.g., Tien, Teichert, & Rickey, 2007; White & Gunstone, 1992) to help students engage in knowledge integration as well as provide support for students developing and refining explanations (e.g., McNeill, Lizotte, Krajcik, & Marx, 2006). The design of the unit also drew upon prior research demonstrating the effectiveness of WISE to support student learning through virtual investigations with visualizations (Linn et al., 2006; McElhaney & Linn, 2011), noting the importance of helping students to focus on relevant aspects of a phenomenon when using simulations (e.g., Chiu & Linn, 2012; Lowe, 2004). Students were given specific guidance when making observations of the simulation to pay attention to specific details important to the particular concept. The overall instructional pattern of the lab consisted of helping students (1) make predictions in the form of an initial explanation of a particular phenomenon, (2) conduct a guided investigation with the Frame of that phenomenon, (3) make targeted observations during the investigations with the Frame, (4) revise/refine their initial explanations to incorporate observations from their investigation, and (5) reflect upon how this new information applies to the context of air mattresses.

AC C

EP

3.1.2 Content Learning objectives for the unit consists of students being able to understand pressure, volume, and temperature with ideal gases on a molecular level as well as articulate relationships among pressure, volume and temperature. Students worked through a series of five activities in WISE (Figure 2) to discover scientific phenomena and to interpret data gathered from the Frame. Specifically, the first activity provided information about the context of the project (air mattresses). The second activity (“What is a gas?”) guided students through an introductory activity in the Frame that focuses on how a gas differs from liquids and solids at a particulate level through a simulation of gases, liquids, and solids. The third activity (“What is temperature?”) concentrated on helping students understand temperature on a molecular level by having students add and remove energy to the system by placing hot and cold jars next to the Frame and observing what happens to the molecules in the simulation. Specific learning objectives included: 1) gas particles at the same temperature move at different speeds and 2) adding/removing energy results in an average increase/decrease in the speed of the gas molecules (as opposed to molecules expanding/contracting in size). Activity 4 focused on helping students understand gas pressure on a molecular level by guiding them to physically push and pull on the piston and observe what happens to the gas molecules. Specific learning

ACCEPTED MANUSCRIPT 9

SC

RI PT

EFFECTS OF AUGMENTED VIRTUAL LABS ON SCIENCE UNDERSTANDING

Figure 2. The WISE Frame gas laws curriculum guided students through an investigation of air mattresses.

AC C

EP

TE D

M AN U

objectives involved understanding that decreasing volume leads to increased pressure due to increased collisions with the container walls, and vice versa. The fifth activity guided students through an investigation of the relationship between temperature and pressure by having students add energy to the system by putting a hot jar next to the Frame and having students try to hold the piston to the same location, and similarly with a cold jar. One purpose of this investigation was to have students feel that it takes more effort to keep the piston in the same place after adding energy with a hot jar (increased pressure), and that they have to pull the piston to keep it stable for the cold jar (decreased pressure). Learning objectives included understanding that increasing (or decreasing) the temperature of the molecules increases (or decreases) the average speed of the molecules, which results in increased (or decreased) pressure on the piston. In case students finished early, additional “Extra for Experts” activities provided students opportunities to conduct open investigations of the impact of the number of particles on volume and the effect of mixture of gases on pressure. The main purpose in using WISE was to facilitate knowledge integration and guide student interactions with the Frame, as well as to take advantage of research tools such as data logging capabilities to gain a rough idea of how much time students spent in each activity/step. Although WISE offers a wide variety of functionality to support knowledge integration, the Frame lab primarily used text/information steps (6 out of 32 total steps, excluding 10 extra for expert steps) and multiple-choice/open response steps (26 steps) that roughly corresponded to a paper-and-pencil lab guide. The main ways the Gas Frame lab differed from a paper-and-pencil lab were: students were typing instead of writing; one step offered students immediate feedback on a multiple-choice question covering whether actual gas molecules have colors; in the three reflect/explain steps students’ original predictions were automatically provided for the students at the top of the page; and if students started completing one portion of a step with multiple questions (steps often had combinations of multiple-choice and open response questions), WISE would automatically remind the students to answer the rest of the questions before going to the next step. The other steps did not offer any kind of feedback or enhanced capabilities. The Gas Frame Lab and accompanying WISE curriculum was implemented in one class before this study was conducted. Classroom observations demonstrated that students could easily interact with the Frame and use another computer to record results in WISE. Data also informed continued testing and revision of the curriculum and technologies for this study. For

ACCEPTED MANUSCRIPT EFFECTS OF AUGMENTED VIRTUAL LABS ON SCIENCE UNDERSTANDING

10

example, the WISE curriculum was revised from guiding students through a strict predictobserve-explain pattern to include more reflection and explanation of a real-life context of an air mattress product based on student data and feedback from student and teacher interviews.

M AN U

SC

RI PT

3.2 Participants The participants of this study were eighth grade students enrolled in physical science courses based on teacher involvement with the project; two classroom teachers volunteered to participate in this study and supplement instruction with Gas Frame activities. One teacher taught one honors section, and the other teacher taught three standard courses. The teachers had been part of the development of the Frame and supporting curricula, including a professional development workshop that familiarized the teachers with the Frame technology and provided opportunities for teachers to give feedback on curricula, assessments, and implementation in their classrooms. Teachers and students came from one middle school with around 600 students enrolled in grades 6-8. School student demographics include: 50.6% male, 49.4% female; 19.1% Black, 13.8% Hispanic, 53.9% White; 14.2 % Limited English Proficiency, 46.3% on free or reduced lunch; 14.6% Students with Disabilities and 12.5% of Students categorized as Gifted.

TE D

3.3 Treatment Four classes of students worked throughout two 90-minute class sessions on the Gas Frame lab. The Frame lab followed traditional class instruction in science topics involving atomic structure and the properties of gases. Teachers introduced the lab and researchers to the students. Researchers gave a brief introduction to the lab, the Frame technology, and then guided the students through logging into the project in WISE. Students then went through the project at their own pace. One to two researchers were present during each class. Students worked in groups of 2 or 3 chosen by the teacher to complete the Gas Frame lab. Each group had a laptop for WISE and a Frame at their lab station.

AC C

EP

3.4 Data Sources Researchers conducted classroom observations to provide insight into how students interacted with the Frame. Observations did not follow a specific protocol to keep in the spirit of the exploratory nature of the study. Field notes were taken by at least two researchers in each class. WISE also recorded student interaction with each step and activity, providing information about how long students spent in each step as well as if they revised or refined their answers during the lab. Embedded data within WISE also included predictions, observations, and explanations that students made during the lab. Pretests and posttests consisted of six open-ended questions that explicitly asked students to explain observable gas phenomena at the molecular level (Appendix A). Three open-ended items asked students to explain temperature (T), the relationship between volume and pressure (V-P), and the relationship between temperature and pressure (T-P) using the same air mattress context as the lab (proximal). An additional three open-ended items asked students to explain similar concepts (P, V-P, T-P) using different real-life contexts such as balloons and tires (distal) For example, one distal question asks students to explain why it is important to check tire pressure at various temperatures (T-P). Pretests were administered to all students in the classes a day before the implementation of the unit, and identical posttests were administered a week after the unit had been completed.

ACCEPTED MANUSCRIPT EFFECTS OF AUGMENTED VIRTUAL LABS ON SCIENCE UNDERSTANDING

11

Thus, nine days elapsed between the pretest and the posttest. Teachers covered properties of other states of matter in subsequent lessons between the Gas Frame Lab and the posttest and did not provide any additional instruction on gas properties.

AC C

EP

TE D

M AN U

SC

RI PT

3.5 Data Analysis Researchers compared field notes of classroom observations directly following the Frame implementation. From the field notes, trends in how the students used the Frame lab were generated. WISE log data were compiled to compute total time in the project as well as the duration that each student group spent in each step. Each WISE step was classified as text, predictions, observations, or explanations and then average duration in each kind of step was calculated. Analysis of open-ended items focused on the identification of connections among students’ ideas using a knowledge integration (KI) analysis approach (Liu, Lee, Hofstetter, & Linn, 2008). Higher KI scores represent more connections among scientifically relevant ideas. Pretests and posttests for each of six open-ended questions (Appendix A) were coded using a KI rubric (see Table 1 for more information as well as sample student responses). As part of the KI analysis, ideas elicited in the open-ended student responses were classified as normative, partial, alternative, and vague/irrelevant. Normative ideas are correct scientific ideas pertinent to the question. For example, a student response to a question asking why balloons keep their shape of, “the gas particles are constantly moving and hit the inside of the balloon” would be classified as normative. Partial ideas represent student ideas that could be considered normative in other contexts but are tangential to the targeted explanation. A response to the same question that states, “gas particles take the shape of the container” would be classified as a partial idea as the student recognizes that gases do not have a definite volume, but the idea is not directly applicable to balloons keeping their shape. Alternative ideas are scientifically non-normative ideas elicited by students that are relevant to the question. For example, when asked about the relationship of temperature to particle movement, a student response of, “temperature is the thing that determines the size of particles” would be classified as an alternative idea. Vague/irrelevant ideas were attributed to off-task or unclear responses (e.g., “This is fun”). To illustrate the level of normative ideas that were elicited from pretest to posttest, student responses were also coded for macroscopic, molecular, or linked macroscopic and molecular ideas present in the responses. Macroscopic ideas represent statements that contained observable elements with no explicit mention of the particulate level. For example, “balloons keep their shape because they are filled with helium” was coded as macroscopic. Molecular ideas represent statements that contained particulate elements. A response that contained, “the particles in a gas go all different directions and fast” was coded as molecular. Macroscopicmolecular ideas represent statements connecting molecular to macroscopic levels. For example, “the helium and particles make the balloon stay round” was coded as macroscopic-molecular. The codes are not mutually exclusive, so one student response could contain multiple types of ideas. The above examples all came from the same student response, so that response would be coded as having one macroscopic idea, one molecular idea, and one molecular-macroscopic idea.

ACCEPTED MANUSCRIPT EFFECTS OF AUGMENTED VIRTUAL LABS ON SCIENCE UNDERSTANDING

12

Table 1. Adapted Knowledge Integration Rubric (from Liu et al., 2008) Score Response characteristics

Examples Using Student Work

Complexlink

5

Students elicit and connect three or more normative and relevant scientific ideas

Full-link

4

Partiallink

3

No-link

2

Students elicit and connect two normative and relevant scientific ideas Students elicit normative and relevant scientific ideas as well as non-normative ideas Students elicit non-normative ideas or make invalid connections with nonnormative ideas Students do not elicit scientific ideas No response is provided

When it was day and it was hot the molecules moved faster hitting the walls inflating the mattress a little. During the night the slowed down deflating it a little. The particles cooled & slowed down creating less force which created less pressure deflating the mattress. It deflated because it was colder, when the particles become cooled they become smaller and slower. There was enough molecules during the day so it was inflated then at night they bunched together and it deflated.

No response

0

SC

M AN U

Irrelevant 1

RI PT

KI Level

I don't know

AC C

EP

TE D

Two independent raters coded all data. Cohen’s Kappa ranged from .670 to .756, indicating substantial inter-rater reliability (Cohen, 1968). Students were assigned overall KI scores based on the number of scientifically normative ideas and the connections among ideas in their responses. Student responses were classified as Complex-Link, Full-Link, Partial-Link, No-Link, Irrelevant, and No Response per the KI rubric using a 0-5 point scale (Liu, Lee, & Linn, 2011) in order to address the first research question. Paired-sample t-tests were used to determine any statistically significant differences in student performance from pretest to posttest for individual items. Cohen’s d (1988) was calculated to estimate the respective effect size of each open-ended question analyzed. Chi-squared tests were used to determine any significant shifts in categories of student ideas from pretest to posttest. Researchers examined the content of the embedded predictions, observations and explanations in WISE to complement pre/post data. As this study was conducted in an authentic school context, we had missing data. Although 77 students were enrolled in the four classes, 8 students were absent during the labs, 6 students were absent during either the pretest or the posttest and 18 students did not return IRB consent forms, resulting in a final sample of 45 (17 male; 28 female) with 14 from the honors and 31 from the standard physical science classes. We compared pretest scores for students with missing and complete data and found no statistically significant differences between groups. We present results for students who completed both a pretest and posttest assessment. The missing data is also reflected in the log data, which resulted in a final number of 17 student groups across the three classes (14 groups of 2, 3 groups of 3). We compared log data from students with complete and missing pretest and posttest data (as students missing either pretests or posttests

ACCEPTED MANUSCRIPT EFFECTS OF AUGMENTED VIRTUAL LABS ON SCIENCE UNDERSTANDING

13

were removed from final analysis) and found no statistically significant differences between groups in terms of duration. 4. Results

AC C

EP

TE D

M AN U

SC

RI PT

4.1 Overall Usage The middle school students had no difficulty operating the Frame or using a separate laptop concurrently for WISE. Students did not seem surprised that they could control the Frame using the physical controls of hot jars or spring. Students were generally very engaged and interacted with the Frame naturally, without questions as to how the physical inputs of the Frame controlled the simulation. Most groups had one student who would mainly read/record in WISE and the other student(s) would primarily conduct the investigations with the Frame. Teachers directed students to switch roles on the second day. Students spent an average of 110 minutes (SD=17.7) within the WISE project, ranging from 84 minutes to 143 minutes. Groups spent an average of 3.5 minutes per step (SD=1.2). Students tended to spend most of the time in the second and fourth activity (average time per step (min): activity 1: M(SD) = 2.0(0.7); activity 2: M(SD) = 11.8(11.9); activity 3: M(SD) = 3.2(0.9); activity 4: M(SD) = 5.1(2.3); activity 5: M(SD) = 2.5(1.3)). All groups completed all five activities in WISE, with 8 of the student groups getting through part of the extra for experts sections. All student groups interacted with WISE over 2 days. If students finished early, teachers directed the students to work on homework during the rest of the block period. On average, students tended to spend less time on text pages (M(SD) =1.54(0.8)), and more time in prediction (M(SD )=4.5(3.0)), observation (M(SD) =4.7(1.3)), and explanation steps (M(SD )=4.9(2.5)). There was a significant effect of the type of step on duration (F(3, 64) = 9.61, p < 0.001). Tukey HSD post hoc comparisons indicate that the duration of text steps significantly differed from prediction, observation, and explanation steps, but there were no significant differences in the duration among prediction, observation and explanation steps. Classroom observations aligned with the log data that students spent less time on the informational text pages and more in prediction, observation, and explanation steps that guided students’ interactions with the Frame. Classroom observations also indicated that in general, students conducted investigations with the Frame as guided in WISE. Many student groups (typically half of the class) spontaneously used the Frame to test their predictions before being specifically guided in WISE, and then went back to record their observations and explanations. Some student groups (2-3 per class) also spontaneously interacted with the Frame, conducting multiple trials of investigations like moving the hot jar back and forth towards the Frame, or conducting more open investigations like holding in the piston while the hot jar was against the Frame. Observations revealed that a few students across classes would continuously hold the spring and manipulate the piston even during discussion of the activities or without looking at the screen of the Frame. 4.2 Pretest and Posttest Performance Posttest scores were regressed with pretest scores and class (honors or standard) as explanatory variables to determine any differences in performance across classes (R2 = 0.47, F(2, 42) = 18.75, p < 0.01). Pretest score had a significant effect on posttest scores (β = .96, t(42) = 6.03, p < 0.01). After controlling for pretest score, there were no significant differences between honors and standard classes (β = .72, t(42) = .76, p = .45). Honors students (Pretest M(SD) =

ACCEPTED MANUSCRIPT EFFECTS OF AUGMENTED VIRTUAL LABS ON SCIENCE UNDERSTANDING

14

SC

RI PT

2.25(.51); Posttest M(SD) = 2.82(.71)) performed similarly to standard students (Pretest M(SD) = 2.23(.38); Posttest M(SD) = 2.65(.52)). As there were no differences between honors and standard, there were also no differences between teachers. We report the rest of the results without differentiating classes. Overall, students made significant progress from pretest to posttest with a large effect size. Table 2 displays pretest and posttest KI scores and effect sizes for each question. On average, students made progress from no-link to partial links along the KI scale. Students made significant progress on proximal items that asked students to explain temperature, the relationship between pressure and volume and the relationship between temperature and pressure in the same context as the unit. Students also made progress on distal items that asked students to explain the relationship between pressure and volume as well as temperature and pressure in a different context than that of the unit. However, scores did not significantly increase on the item that asked students to explain pressure in a different context.

M AN U

Table 2. Means and standard deviations for individual pretest and posttest KI items and overall average score Pretest Mean(SD) 2.49 (.59) 2.13 (.34)

t Temperature 3.39 Pressure-Volume 3.55 TemperaturePressure 2.16 (.67) 2.82 (.83) 4.94 Distal Pressure 2.13 (.81) 2.44 (.87) 1.76 (different Pressure-Volume 2.47 (.94) 2.91 (.82) 2.88 context) Temperature2.80 Pressure 2.07 (.94) (1.32) 4.12 Overall average KI score* 2.24 (.42) 2.70 (.58) 7.02 *Note: Overall average KI score is the average across multiple items

EP

TE D

Question Proximal (same context)

Posttest Mean(SD) 2.87 (.76) 2.36 (.61)

df 44 44

Effect Size .56 .47

2-tailed p-value < .01* < .01*

44 44 44

.88 .37 .50

< .01* .08 < .01*

44 44

.64 .91

< .01* < .01*

AC C

4.3 Developing Molecular-level Explanations In total, posttest responses included more ideas about molecular-level behavior and more ideas that linked macroscopic to molecular levels, χ2(1, N = 44) = 11.76, p = .003 (Table 3). On the proximal temperature question, responses tended to contain more macro-molecular links on the posttest. The question asks students to explain what happens to gas particles in an air mattress at different temperatures. Student responses on the pretest commonly consisted of descriptions of “air” without explicit discussion of the particulate level (“The mattresses was [sic] to [sic] full with air and it got so hot that it was abuot [sic] to deform”) whereas on the posttest students articulated particle-level behavior and connected it to the mattress (“When the mattress was hot the particles were moving really fast. But when it cooled down the particles started slowing down”). Embedded data from the Gas Frame Lab provides evidence that students were observing the behavior of gas molecules with the Frame and making connections to temperature. For example, one of the first steps within the temperature activity asked students to make observations of gas molecules at the same temperature. All but one response contained

ACCEPTED MANUSCRIPT EFFECTS OF AUGMENTED VIRTUAL LABS ON SCIENCE UNDERSTANDING

15

EP

TE D

M AN U

SC

RI PT

observations that the gas particles were in constant motion and were colliding with other molecules and the walls of the container. When students were asked to predict, observe and explain what happened to the gas molecules when they increased the temperature of the gas with the hot jar, all students’ observations included that the speed of the gas molecules increased. In the responses to the embedded reflection prompt that guided students to connect what happens inside an air mattress when heated and cooled, all but one student group explicitly referred to particle movement to explain the results. For the relationship between temperature and pressure, posttest responses included more molecular ideas and more macro-molecular links. On the proximal temperature-pressure pretest and posttest question, student responses included more molecular ideas on the posttest. This question asked students to make the connection from hot or cold temperatures to how that would translate to pressure differences in an air mattress. For instance, one student responded, “The cold air caused it to deflate because it became colder” on the pretest and on the posttest responded, “The air particles in the mattress slowed down and it deflated.” On the distal temperature-pressure question, posttest responses shifted to include more macro-molecular links. The question asked students to explain the relationship between temperature and pressure on the microscopic level in the context of tire pressure on hot and cold days. On the pretest, one student responded that tires need to be checked “so the tire doesn't explode or inplode [sic].” The posttest response from the same student stated, “In the summer the preassure [sic] builds up more because the particles move faster, in the winter the particles move slower causing the preassure to go down.” Embedded data indicate that the Gas Frame lab helped students make progress in connecting the effect of temperature on pressure on a molecular level. In activity 5, students were guided to predict, observe and explain what happens to pressure when temperature is decreased. In the predictions, 8 of the student groups incorrectly predicted that it would become harder to push in the piston if the temperature was decreased. After conducting the investigation, 5 of the 8 groups revised their explanations stating that it was easier to push the piston when the temperature decreased, with three student groups still stating that it was harder to push. Although only seven of the student groups included a particle explanation in their explanation of how temperature relates to pressure, 13 of the 17 student groups used particulate explanations on the reflection prompt to connect the results to what would happen to the pressure of an air

AC C

Table 3. Average number of normative macroscopic, molecular, and macro-molecular ideas by student response and total number of normative ideas for pretest and posttest items

Question Proximal

Temperature P-V T-P Distal Pressure P-V T-P Total No. of Ideas

Macroscopic Pre Post 0.24 0.27 0.60 0.62 0.33 0.20 0.64 0.31 0.38 0.44 0.44 0.51 119 106

Molecular Pre Post 0.00 0.04 0.00 0.00 0.24 0.82 0.49 0.53 0.13 0.24 0.02 0.09 40 78

MacroMolecular Pre Post 1.04 1.31 0.40 0.38 0.04 0.04 0.09 0.11 0.27 0.31 0.60 1.07 110 143

Total No. of Ideas Pre Post 58 73 45 45 28 46 55 43 35 45 48 75 269 327

ACCEPTED MANUSCRIPT EFFECTS OF AUGMENTED VIRTUAL LABS ON SCIENCE UNDERSTANDING

16

EP

TE D

M AN U

SC

RI PT

mattress at different temperatures. For questions relating to pressure and volume, the results were mixed. On the distal pressure-volume question, posttest responses contained slightly more molecular-level ideas. The question asked students to explain the relationship between pressure and volume in the context of a small balloon filled with air. Many students’ pretest responses attributed macroscopic properties to the molecular level, such as “this happens because it becomes harder to all of the particles to fit” or “because the air does not release out of the balloon.” Student responses on the posttest represented a shift towards explicit explanations of particulate behavior, for instance, “the atoms are moving at faster speeds in a smaller area, causing the balloon to be harder and harder to squeeze.” This response, while not a completely normative link between pressure and volume, indicates that the student was adding ideas about molecular behaviors. On the proximal pressure-volume question, there was little shift in the kinds of ideas from pretest to posttest. On the distal pressure question, there was a decrease in macroscopic ideas and little change in molecular and macro-molecular ideas from pretest to posttest and even a decrease in total normative ideas elicited. In activity 4 of the lab, students were explicitly guided to decrease the volume of the container by pushing in the piston to learn about pressure and volume. Although the lab provided specific questions to help students pay attention to how often the gas particles hit the piston wall, embedded data reveal that 4 out of the 17 student groups did not observe that as they pushed in the piston, the gas particles hit the piston more often. Of those four groups, two groups noted that they observed the pressure decreasing instead of increasing. Interestingly, on the embedded reflection question that asked students to connect the results to air mattresses by explaining how an air mattress can support weight, 11 out of the 17 student groups had responses that used molecular-level ideas (i.e., “The air mattress is able to support weight because the air mattress is a closed space that the particles move around in. When someone lays down on it, it compresses the air mattress and the gas particles inside push against the wall of the air mattress, which hold you up. "). However, responses to the reflection prompt that asked students to connect to what happens when an air mattress bursts (the proximal P-V question), only one group used molecular-level ideas to explain the result. All other student groups used macroscopic-level explanations, such as “The air in the mattress couldn't hold that many people cause they was too heavy” or “Because there is a lot more air and weight it will compress it more and POP!”

AC C

4.4 Refining Alternative Ideas Table 4 shows the percentages of normative, partial, alternative, and vague/irrelevant ideas in students’ pretest and posttest responses. Overall, there was a significant shift in categories of responses from pretest to posttest, χ2(1, N = 44) = 42.88, p < .001. Across all questions except 1, students had fewer alternative ideas in their explanations and more normative ideas from pretest to posttest. Partial ideas expressed by students decreased from pretest to posttest, with the exception of question 4. There are no notable trends for the expression of vague and irrelevant ideas. For example, the proximal T-P question asks students to consider why a mattress, inflated when it was hot, would become deflated at night. Students’ pretest responses consisted predominantly of alternative ideas (e.g. “the particals [sic] got closer together and slowed down” or “the particles were harder when it was hot but once it cooled down they became softer). Many of these alternative ideas expressed a mismatch between observed macroscopic properties

ACCEPTED MANUSCRIPT EFFECTS OF AUGMENTED VIRTUAL LABS ON SCIENCE UNDERSTANDING

17

Table 4. Average number of normative, partial, alternative and vague/irrelevant ideas by student response and total number of ideas for pretest and posttest items

123

145

231

94

Alternative Pre Post 0.36 0.49 0.09 0.04 0.60 0.29 0.18 0.18 0.11 0.13 0.36 0.24 76

62

Vague/Irrelevant Total No. of Ideas Pre Post Pre Post 0.18 0.18 82 98 0.04 0.18 51 55 0.11 0.22 62 69 0.02 0.04 66 51 0.11 0.11 46 56 0.27 0.18 79 96 33

RI PT

Partial Pre Post 0.04 0.07 0.96 0.82 0.36 0.16 0.89 0.44 0.36 0.22 0.62 0.38

36

386

425

SC

Question Proximal T P-V T-P Distal P P-V T-P Total No. of Total Ideas

Normative Pre Post 1.24 1.56 0.04 0.18 0.27 0.87 0.31 0.47 0.54 0.78 0.44 1.29

AC C

EP

TE D

M AN U

and molecular phenomena. In both examples provided, the inflation/deflation of the mattress and its respective “hardness” or “softness” was attributed to physical proximity or physical characteristics of particles, neither accurately representing the phenomenon on a microscopic level. By the post-test, students demonstrated a more scientifically normative understanding (e.g. “the particles stopped moving as fast when it cooled down” and “the temp. made the molecules speed up which caused the molecules to constantly hit the container, but once it cooled down, the molecules slowed down not hitting the container as much”). Students’ posttest ideas focus more on particulate movement and temperature and less upon alternative ideas that were common in the pretest. Embedded data indicate changes in students’ initial ideas during the lab. For example in the temperature activity, seven of the student groups incorrectly predicted that molecules would gain mass when temperature increases. However after their observations with the Frame, all of the student groups explained that molecules do not gain mass but increase in speed when temperature increases. In the T-P activity, when 8 of the student groups on their predictions stated that pressure would increase if temperature decreased, many explanations of the predictions misapplied macroscopic properties to molecular behavior (“It’s going to be harder because it going to be stuck and go slower”), indicating initial ideas that somehow gas molecules at decreased speeds relates to the gas being more viscous or harder to move. However, by the end of the activity 5 of the 8 student groups had revised their explanations to state that when temperature decreases pressure would decrease. 5. Discussion This study investigated how combining physical and virtual experiences into simultaneous, augmented virtual science laboratories could help students build upon intuitive ideas and develop molecular-level explanations of macroscopic phenomena. The findings suggest that the Gas Frame Lab helped middle school students develop an overall understanding of gas behavior as evidenced by a significant increase from pretest to posttest KI scores with a large effect size (0.91). As the KI scores represent the development of explanations that apply scientific ideas to explain real-world phenomena, results also suggest that the Gas Frame Lab helped students make progress in the NGSS practice of constructing explanations. Results align with other studies that demonstrate benefit from visualizations of molecular phenomena (e.g., Steiff & Wilensky, 2003; Wu, Krajcik, & Soloway, 2001) as well as other research that demonstrates the combination of physical and virtual materials benefits students

ACCEPTED MANUSCRIPT EFFECTS OF AUGMENTED VIRTUAL LABS ON SCIENCE UNDERSTANDING

18

AC C

EP

TE D

M AN U

SC

RI PT

(e.g., Blikstein, 2014; Liu, 2006; Olympiou & Zacharia, 2012). Results are comparable to other studies that use WISE to support inquiry around molecular concepts. For example, Zhang & Linn (2011) investigated middle school students’ (n=133) understanding of chemical reactions through a unit on hydrogen fuel cells lasting 5 instructional days (total of around 300 minutes). They found an effect size of 1.58 from students’ average pretest to posttest KI scores. Similarly, Chang and Linn (2013) found effect sizes of 0.57 (n=60), 0.63 (n=62), and 1.21 (n=83) from pretest to posttest KI scores for three studies of middle school students using a weeklong WISE unit on thermodynamics. Chiu and Linn (2014) found effect sizes of .99 (n=21) and 0.53 (n=24) for two studies of high school students using a weeklong WISE unit to learn about chemical reactions. The effect size of 0.91 found for the Gas Frame Lab is similar to if not more favorable than these studies, considering that the Gas Frame lab was roughly half of the length of instruction and assessments were administered at similar times across all studies. Pretest to posttest improvements in KI score demonstrate that students progressed in constructing explanations that used molecular-level ideas of gas particles and refined partial and alternative ideas for concepts of temperature and relating temperature to pressure. During the lab, students were guided to use hot and cold jars to control the temperature of the Frame, and feel the effect of the different temperatures on pressure by controlling the piston through the spring. Embedded data suggest that students were able to use these Frame activities to add normative molecular ideas as well as refine alternative ideas about temperature and the relationship between temperature and pressure. However, students did not make as much progress in articulating explanations that added molecular-level ideas or macro-molecular connections for pressure and the relationship between pressure and volume. Improvements in KI score were generally due to the increase in normative ideas and refinement of partial/alternative ideas. Students tended to have explanations at macroscopic levels that demonstrated a general relationship that as volume decreases, pressure increases, and vice versa. For the pressure-volume activity, the Frame lab guided students to decrease the volume of the container by pushing on the spring. Interestingly, students tended to spend more time in the pressure-volume activity as compared to the temperature and temperature-pressure activities. Although most students stated they observed increased collisions on the container wall when decreasing the volume of the container, when explaining related situations of gas behavior in air mattresses or balloons, students tended not to use molecular-level explanations, remaining at the macroscopic level. One possible explanation could be that students have robust experiential ideas for explaining everyday phenomena (diSessa, 1998; 1993) and the particular assessment item contexts of an air mattress bursting or squeezing a balloon failed to elicit students’ understanding of pressure on a molecular level, instead eliciting revision of their explanations on a macroscopic level. One major goal of this study was to explore if augmented virtual technologies could help students learn in authentic classrooms settings. Across classes, students had very little difficulty using the Frame and understanding how the physical inputs controlled the simulation. As the Frame is a novel technology and very few studies address mixed-reality approaches in classrooms (Lindgren & Johnson-Glenberg, 2013), this study indicates that augmented virtual approaches can be successful in real classroom contexts. Results from this exploratory study also point to the need to explore how particular combinations of specific physical controls and visualizations for particular content can affect student learning in authentic settings. We predicted that embodied experiences with physical controls could help students elicit relevant intuitive knowledge at the same time as adding

ACCEPTED MANUSCRIPT EFFECTS OF AUGMENTED VIRTUAL LABS ON SCIENCE UNDERSTANDING

19

TE D

M AN U

SC

RI PT

normative molecular-level ideas through visualizations. Having both sets of ideas activated at the same time could encourage students to connect molecular and macroscopic ideas and refine everyday ideas. However, the results suggest that certain combinations of augmented virtual interactions were more successful for particular concepts. The combination of using a hot and cold jar to control the temperature of the simulation as well as the spring to control the piston seemed to help students make molecular-level explanations and refine alternative ideas about temperature and the effect of temperature on pressure. The combination of students using the spring to understand pressure and the effect of volume on pressure was not as successful at helping students develop molecular-level explanations, but instead somewhat helped students add normative and refine ideas at the macroscopic level. Future work that articulates exactly how students interpret and interact with specific combinations of physical and virtual manipulatives for specific concepts can contribute to an understanding of when and why augmented virtual approaches can be successful. Similarly, the Frame was developed to help students connect real-world gas phenomena to unobservable molecular behaviors. The design and revisions of the technology were guided by the overall goal for use in authentic classrooms with teachers playing a large role in the design and refinement of the technologies. Although our augmented virtual lab proved to be generally beneficial for learning with these specific learning goals, designers may need to tailor or choose mixed-reality approaches based on targeted learning outcomes. For example, our activities did not emphasize scientific practices of modeling, where having virtual and physical representations side-by-side may have the most benefit (e.g., Blikstein, 2014). However, our study does provide guidance for other mixed-reality approaches that aim to help students connect molecular and macroscopic levels in science. Results also suggest implications for practitioners teaching or developing curricula for gas laws. Similar to other studies (e.g., de Jong, Linn, & Zacharia, 2013; Liu, 2006), results indicate that providing both hands-on and virtual experiences can support science learning. Since many teachers already incorporate probeware into labs on gas laws, teachers can consider providing complementary visualizations and simulations to maximize benefit to students.

AC C

EP

6. Limitations This study is based on a pre/post design without a control group; therefore these results serve as indications that the Frame lab can be successfully implemented in authentic classrooms. As the pretest and posttest were identical, test or practice effects may contribute to the findings of this study. Additionally, the treatment in this study involved two 90-minute class periods. More research is necessary to determine the extent of the impact that sensor-augmented virtual experiences have on student understanding after a longer duration, as well as investigating the role of the teacher in these labs. As this study focused on the development of explanations, assessment items focused on text-based open responses. However, research demonstrates that drawing tasks can provide valuable insight into how students understand molecular concepts (Marbach-Ad, Rotbain, & Stavy, 2008; Zhang & Linn, 2011). Future work should include both drawing and explanation tasks to better understand how augmented approaches might help students further develop understanding of complex molecular concepts. Another major limitation to this study involves the relative impact of the physical and virtual manipulatives, and similarly, the influence of embodied experiences on student learning. As this was an exploratory study to determine if these kinds of approaches can be successful in

ACCEPTED MANUSCRIPT EFFECTS OF AUGMENTED VIRTUAL LABS ON SCIENCE UNDERSTANDING

20

SC

RI PT

real classrooms, this study highlights the need for more detail as to if or how embodied experiences can contribute to learning in authentic settings. Although we used a grounded knowledge integration framework to shape this exploratory work, this study only provides evidence that embodied approaches can be as successful as other technology-enhanced visualization-based inquiry labs. It is possible that the gains from pretest to posttest could be attributed solely to students interacting with molecular-level visualizations and the physical controls could have a small or negligible effect. As other meta-analyses demonstrate moderate effects of visualizations on science understanding (e.g. Höffler & Leutner, 2007), future research needs to compare student learning outcomes from only visualizations to augmented approaches to isolate what, if any, benefits there are grounded or embodied experiences. Likewise, more fine-grained, qualitative studies of students working with these technologies can provide needed insight into exactly if and how students interpret and use the physical augmentations for learning.

AC C

EP

TE D

M AN U

7. Conclusion Augmented virtual science laboratories have the potential to help students develop sophisticated scientific explanations as called for by the NGSS by connecting molecular-level visualizations to observable, macroscopic phenomena. This paper investigated if students in authentic classrooms could use the Frame, an augmented virtual technology that uses physical inputs to control molecular simulations simultaneously, to develop molecular-level explanations and refine alternative ideas about gas properties. Overall results indicate students were able to make more connections among ideas as well as refine alternative to normative ideas about gas phenomena from pretest to posttest, but these results differed for different topics. This study contributes to the field of mixed-reality technologies by providing evidence that providing simultaneous hands-on and virtual experiences can promote science learning.

ACCEPTED MANUSCRIPT EFFECTS OF AUGMENTED VIRTUAL LABS ON SCIENCE UNDERSTANDING

21

AC C

EP

TE D

M AN U

SC

RI PT

References AAAS Project 2061 (n.d.) [Pilot and field test data collected between 2006 and 2010]. Unpublished raw data. Abrahamson, D., Gutiérrez, J. F., Charoenying, T., Negrete, A. G., & Bumbacher, E. (2012). Fostering hooks and shifts: tutorial tactics for guided mathematical discovery. Technology, Knowledge, and learning, 17(1-2), 61-86. DOI: 10.1007/s10758012-9192-7. Anastopoulou, S., Sharples, M., & Baber, C. (2011). An evaluation of multimodal interactions with technology while learning science concepts. British Journal of Educational Technology, 42(2), 266-290. Ardac, D., & Akaygun, S. (2004). Effectiveness of multimedia‐based instruction that emphasizes molecular representations on students' understanding of chemical change. Journal of Research in Science Teaching, 41(4), 317-337. Barsalou, L. W. (2008). Grounded cognition. Annual Review of Psychology, 59, 617–45. doi:10.1146/annurev.psych.59.103006.093639 Bell, R. L., & Trundle, K. C. (2008). The use of a computer simulation to promote scientific conceptions of moon phases. Journal of Research in Science Teaching, 45, 3, 346-372. Benson, D. L., Wittrock, M. C., & Baur, M. E. (1993). Students’ preconceptions of the nature of gases. Journal of Research in Science Teaching, 30(6), 587-597. Ben-Zvi, R., Eylon, B-S, & Silberstein, J. (1986). Is an atom of copper malleable? Journal of Chemical Education, 63, 64-66. Blikstein, P. (2012, October). Bifocal modeling: a study on the learning outcomes of comparing physical and computational models linked in real time. In Proceedings of the 14th ACM international conference on Multimodal interaction (pp. 257-264). ACM. Blikstein, P. (2014). Bifocal modeling: Promoting authentic scientific inquiry through exploring and comparing real and ideal systems linked in real-time. In A. Nijholt (Ed.) Gaming Media and Social Effects. Singapore: Springer. Blikstein, P., Fuhrmann, T., Greene, D., & Salehi, S. (2012). Bifocal modeling: mixing real and virtual labs for advanced science learning. In Proceedings of the 11th International Conference on Interaction Design and Children (pp. 296-299). ACM. Blikstein, P., & Wilensky, U. (2007). Bifocal modeling: a framework for combining computer modeling, robotics and real-world sensing. Paper presented at the Annual meeting of the American Educational Research Association. Chicago, IL. Bodner, G. (1991). I have found you an argument: The conceptual knowledge of beginning chemistry graduate students. Journal of Chemistry Education, 68, 385-388. Borgman, C. L., Abelson, H., Dirks, L, Johnson, R., Koedinger, K.R., Linn, M.C., et al. (2008). Fostering learning in the networked word: The cyberlearning opportunity and challenge. Report of the NSF Task Force on Cyberlearning. Washington, D.C.: National Science Foundation. Bouwma-Gearhart, J., Stewart, J. & Brown, K. (2009). Student misapplication of a gas-like model to explain particle movement in heated solids: Implications for curriculum and instruction towards students’ creation and revision of accurate explanatory models. International Journal of Science Education, 31(9), 1157-1174.

ACCEPTED MANUSCRIPT EFFECTS OF AUGMENTED VIRTUAL LABS ON SCIENCE UNDERSTANDING

22

AC C

EP

TE D

M AN U

SC

RI PT

Bransford, J., Brown, A. L., & Cocking, R. R. (2000). How people learn. Washington, DC: National Academy Press. Carlsen, D. D., & Andre, T. (1992). Use of a microcomputer simulation and conceptual change text to overcome student preconceptions about electric circuits. Journal of ComputerBased Instruction, 19, 4, 105-9. Chang, H. Y., Quintana, C., & Krajcik, J. S. (2010). The impact of designing and evaluating molecular animations on how well middle school students understand the particulate nature of matter. Science Education, 94(1), 73-94. Chao, J., Chiu, J. L., Pan, E., DeJaegher, C., Hazzard, E. & Xie, C. (2014). The Effects of Mixed-Reality Laboratories on High School Students’ Conceptual Understanding of Gas Laws. Paper presented at the Annual Meeting of the American Educational Research Association, Philadelphia, PA. Chiu, J. L. (2010). Developing students’ criteria for visualizations by prompting judgments of fidelity. The International Conference of the Learning Sciences. Chicago, IL. Chiu, J. L., King Chen, J. & Linn, M. C. (2013). Overcoming deceptive clarity by encouraging metacognition in the web-based inquiry science environment. In R. Azevedo & V. Aleven (Eds.) International Handbook of Metacognition and Learning Technologies (pp. 517-531). New York: Springer. Chiu, J. L. & Linn, M. C. (2012). Supporting self-monitoring with dynamic visualizations. In J. Dori & A. Zohar (Eds.) Metacognition and Science Education (pp. 133-164). Mahwah, NJ: Lawrence Erlbaum. Chiu, J. L. & Linn, M. C. (2014). Supporting knowledge integration in chemistry with a visualization-enhanced inquiry unit. Journal of Science Education and Technology, 23(1), 37-58. Clark, D. B. (2006). Longitudinal conceptual change in students' understanding of thermal equilibrium: An examination of the process of conceptual restructuring. Cognition and Instruction, 24(4), 467–563. Clark, D. B., & Jorde, D. (2004). Helping students revise disruptive experientially-supported ideas about thermodynamics: Computer visualizations and tactile models. Journal of Research in Science Teaching, 41(1), 1–23. Cohen, J. (1988). Statistical power analysis for the behavioral sciences (2nd ed.). Hillsdale, NJ: Lawrence Earlbaum Associates. de Jong, T., Linn, M. C., & Zacharia, Z. C. (2013). Physical and Virtual Laboratories in Science and Engineering Education. Science, 340(6130), 305-308. Dori, J.Y., & Belcher, J. (2005). How does technology-enabled active learning affect undergraduate students’ understanding of electromagnetism concepts? Journal of the Learning Sciences, 14, 243-279. Duschl, R. & Gitomer, D. (1991). Epistemological perspectives on conceptual change: Implications for educational practice. Journal of Research in Science Teaching, 28, 839858. Duschl, R. A., Schweingruber, H. A., & Shouse, A. E. (Eds.). (2007). Taking science to school: Learning and teaching science in grades K-8. Washington, DC: National Academies Press. Erickson, G., & Tiberghien, A. (1985). Heat and temperature. In R. Driver, E. Guesne, & Tiberghien (Eds.), Children’s ideas in science (pp. 52–83). Philadelphia: Open University Press.

ACCEPTED MANUSCRIPT EFFECTS OF AUGMENTED VIRTUAL LABS ON SCIENCE UNDERSTANDING

23

AC C

EP

TE D

M AN U

SC

RI PT

Feisel, L.D., & Rosa, A.J. (2005). The role of the laboratory in undergraduate engineering education. Journal of Engineering Education, 94, 121-130. Finkelstein, N., Adams, W., Keller, C., Kohl, P., Perkins, K., Podolefsky, N., Reid, S., et al. (2005). When learning about the real world is better done virtually: A study of substituting computer simulations for laboratory equipment. Physical Review Special Topics - Physics Education Research, 1(1), 1–8. Gabel, D. (1999). Improving teaching and learning through chemistry education research: A look to the future. Journal of Chemical Education, 76 (4), 548. Gabel, D., Samuel, K. & Hunn, D. (1987). Understanding the particulate nature of matter. Journal of Chemical Education, 64(8), 695-697. Gire, E., Carmichael, A., Chini, J.J., Rouinfar, A., Rebello, S., Smith, G., et al. (2010). The effects of physical and virtual manipulatives on students’ conceptual learning about pulleys. In K. Geomez, L. Lyons, & J. Radinsky (Eds.) Learning in the disciplines: Proceedings of the 9th international conference of the learning sciences (ICLS 2010) (Vol. 1, pp. 937-944). Herrmann-Abell, C. F., & DeBoer, G. (2011). Using distractor-driven standards-based multiplechoice assessments and Rasch modeling to investigate hierarchies of chemistry misconceptions and detect structural problems with individual items. Chemistry Education Research and Practice, 12, 184-192. Hodson, D. (1996). Laboratory work as scientific method: Three decades of confusion and distortion. Journal of Curriculum Studies, 28(2), 115-135. Höffler, T. N., & Leutner, D. (2007). Instructional animation versus static pictures: A metanalysis. Learning and Instruction, 17, 722-738. Hofstein, A., & Lunetta, V. (2004). The laboratory in science education: Foundations for the twenty-first century. Science Education, 88(1), 28-54. Honey, M. A., & Hilton, M. L. (Eds.). (2011). Learning science through computer games and simulations. Washington, DC: National Academy Press. Jaakkola, T., Nurmi, S., & Veermans, K. (2011). A comparison of students' conceptual understanding of electric circuits in simulation only and simulation-laboratory contexts. Journal of Research in Science Teaching, 48, 1, 71-93. Johnson-Glenberg, M. C., Koziupa, T., Birchfield, D., & Li, K. (2011, June). Games for learning in embodied mixed-reality environments: principles and results. In Proceedings of the 7th International Conference on Games, Learning, & Society (pp. 129-137). ETC Press. Johnstone, A. H. (1991). Why is science difficult to learn? Things are seldom what they seem. Journal of Computer Assisted Learning, 7, 75-83. Jones, L. L., Jordan, K. D. & Stillings, N. A. (2005). Molecular visualization in chemistry education: the role of multidisciplinary collaboration. Chemistry Education Research and Practice, 6(3), 136-149. Jones, M. G., Carter, G., & Rua, M. J. (2000). Exploring the development of conceptual ecologies: Communities of concepts related to convection and heat. Journal of Research in Science Teaching, 37(2), 139–159. Kelly, R. M. & Jones, L. L. (2007). Exploring how different features of animations of sodium chloride dissolution affect students’ explanations. Journal of Science Education and Technology, 16(5), 413-429. Kozma, R. (2003). The material features of multiple representations and their cognitive and social affordances for science understanding. Learning and Instruction, 13(2), 205-226.

ACCEPTED MANUSCRIPT EFFECTS OF AUGMENTED VIRTUAL LABS ON SCIENCE UNDERSTANDING

24

AC C

EP

TE D

M AN U

SC

RI PT

Kozma, R. B., & Russell, J. (1997). Multimedia and understanding: Expert and novice responses to different representations of chemical phenomena. Journal of Research in Science Teaching, 34(9), 949-968. Krajcik, J. (1991). Developing students' understandings of chemical concepts. In S. Glynn, R. Yeany, & B. Britton (Eds.), The psychology of learning science (pp. 117-147). Hillsdale, NJ: Erlbaum. Lakoff, G., & Johnson, M. (1980). Metaphors we live by. Chicago: University of Chicago Press. Lee, O., Eichinger, D. C., Anderson, C. W., Berkheimer, G. D. & Blakeslee, T. D. (1993). Changing middle school students’ conceptions of matter and molecules. Journal of Research in Science Teaching, 30(3), 249-270. Levy, D. (2012). How dynamic visualization technology can support molecular reasoning. Journal of Science Education and Technology, 22(5), 702-717. Levy, S. T., & Wilensky, U. (2009). Students’ learning with the Connected Chemistry (CC1) curriculum: navigating the complexities of the particulate world. Journal of Science Education and Technology, 18(3), 243-254. Levy, S. T., Novak, M., & Wilensky, U. (2006). Students’ foraging through the complexities of the particulate world: Scaffolding for independent inquiry in the connected chemistry (MAC) curriculum. In Annual Meeting of the American Educational Research Association, San Francisco, CA. Lewis, E., & Linn, M. C. (1994). Heat energy and temperature concepts of adolescents, adults, and experts: Implications for curricular improvements. Journal of Research in Science Teaching, 31(6), 657-677. Lin, H., Cheng, H., & Lawrenz, F. (2000). The assessment of students and teachers’ understanding of gas laws. Journal of Chemical Education, 77(2), 235-238. Lindgren, R. & Johnson-Glenberg, M. (2013). Emboldened by embodiment: Six precepts for research on embodied learning and mixed reality. Educational Researcher, 42(8), 445452. Linn, M. C. (2005). WISE design for lifelong learning—Pivotal Cases. In Peter Gärdenfors and Petter Johansson (Eds.) Cognition, Education and Communication Technology. Mahwah, NJ: Lawrence Erlbaum Associates. Linn, M. C., Davis, E. A., & Eylon, B. –S. (2004). The scaffolded knowledge integration framework for instruction. In M. C. Linn, E. A. Davis, & P. Bell (Eds.), Internet environments for science education (pp. 73-83). Mahwah, NJ: Erlbaum. Linn, M. C., & Eylon, B. -S. (2006). Science education: Integrating views of learning and instruction. Handbook of Educational Psychology, 2, 511-544. Linn, M. C. & Eylon, B.-S. (2011). Science Learning and Instruction: Taking Advantage of Technology to Promote Knowledge Integration. New York: Routledge. Liu, O. L., Lee, H. S., Hofstetter, C., & Linn, M. C. (2008). Assessing knowledge integration in science: Construct, measures, and evidence. Educational Assessment, 13(1), 33-55. Liu, X. (2006). Effects of combined hands-on laboratory and computer modeling on student learning of gas laws: A quasi-experimental study. Journal of Science Education and Technology, 15(1), 89–100. doi:10.1007/s10956-006-0359-7 Liu, L., Lee, H.-S., & Linn, M. C. (2011). Measuring knowledge integration: Validation of fouryear assessments. Journal of Research in Science Teaching, 48(9), 1079-1107. Lowe, R. (2004). Interrogation of a dynamic visualization during learning. Learning and Instruction, 14(3), 257-274.

ACCEPTED MANUSCRIPT EFFECTS OF AUGMENTED VIRTUAL LABS ON SCIENCE UNDERSTANDING

25

AC C

EP

TE D

M AN U

SC

RI PT

Lui, M., & Slotta, J. D. (2013). Exploring Evolutionary Concepts with Immersive Simulations. In Proceedings of the Tenth Computer-Supported Collaborative Learning Conference, Madison (Vol. 1, pp. 304-311). Lunetta, V. N., Hofstein, A., & Clough, M. P. (2007). Learning and teaching in the school science laboratory: An analysis of research, theory, and practice. In S. K. Abell & N. H. Lederman (Eds.), Handbook of research on science education (pp. 393-441). Mahwah, NJ: Lawrence Erlbaum Associates. Madden, S. P., Jones, L. L. & Rahm, J. (2011). The role of multiple representations in the understanding of ideal gas problems. Chemistry Education Research and Practice, 12(3), 283-293. Marbach‐Ad, G., Rotbain, Y., & Stavy, R. (2008). Using computer animation and illustration activities to improve high school students' achievement in molecular genetics. Journal of Research in Science Teaching, 45(3), 273-292. McBride, D. L., Murphy, S., & Zollman, D. A. (2010). Student understanding of the correlation between hands-on activities and computer visualizations of NMR⁄MRI. AIP Conference Proceedings (Vol. 225, pp. 225–228). McElhaney, K. W., & Linn, M. C. (2011). Investigations of a complex, realistic task: Intentional, unsystematic, and exhaustive experimenters. Journal of Research in Science Teaching, 48(7), 745-770. McNeill, K. L., Lizotte, D. J., Krajcik, J., & Marx, R. W. (2006). Supporting students’ construction of scientific explanations by fading scaffolds in instructional materials. Journal of the Learning Sciences, 15(2), 153–191. Milgram, P., & Kishino, F. (1994). A taxonomy of mixed reality visual displays. IEICE TRANSACTIONS on Information and Systems, 77(12), 1321-1329. Nakhleh, M.B. (1992). Why some students don't learn chemistry: Chemical misconceptions. Journal of Chemical Education, 69, 191-196. National Research Council [NRC]. (2006). America’s lab report: Investigations in high school science. Washington, DC: National Academy Press. NGSS Lead States. (2013). Next generation science standards: For states, by states. Washington, DC: The National Academies Press. Noh, T. & Scharmann, L. (1997). Instructional influence of a molecular-level pictorial presentation of matter on students’ conceptions and problem-solving ability. Journal of Research in Science Teaching, 34(2), 199-217. Novellis, F., & Moher, T. (2011). How real is “real enough”? Designing artifacts and procedures for embodied simulations of science practices. In Proceedings of the 10th International Conference on Interaction Design and Children - IDC ’11 (pp. 90–98). Novick, S., & Nussbaum, J. (1981). Pupils' understanding of the particulate nature of matter: A cross-age study. Science Education, 65(2), 187-196. Olympiou, G., & Zacharia, Z. (2012) Blending physical and virtual manipulatives: An effort to improve students’ conceptual understanding through science laboratory experimentation. Science Education, 96(1), 21-47. doi: 10.1002/sce.20463 Osborne, R. J. & Cosgrove, M. M. (1983). Children’s conceptions of the changes of state of water. Journal of Research in Science Teaching, 20(9), 825-838. Ozmen, H. (2013). A cross-national review of the studies on the particulate nature of matter and related concepts. Eurasian Journal of Physics and Chemistry Education, 5(2), 81-110.

ACCEPTED MANUSCRIPT EFFECTS OF AUGMENTED VIRTUAL LABS ON SCIENCE UNDERSTANDING

26

AC C

EP

TE D

M AN U

SC

RI PT

Posner, G. J., Strike, K. A., Hewson, P. W., & Gertzog, W. A. (1982). Accommodation of a scientific conception: Toward a theory of conceptual change. Science Education, 66(2), 211-227. Sanger, M., Phelps, A. & Fienhold, J. (2000). Using a computer animation to improve students’ conceptual understanding of a can-crushing demonstration. Journal of Chemical Education, 77(11), 1517-1520. Shen, J., & Linn, M. C. (2011). A technology‐enhanced unit of modeling static electricity: Integrating scientific explanations and everyday observations. International Journal of Science Education, 33(12), 1597-1623. Slotta, J. D., & Linn, M. C. (2009). WISE science: Web-based inquiry in the classroom. New York, NY: Teachers College Press. Smith, G. W., & Puntambekar, S. (2010). Examining the combination of physical and virtual experiments in an inquiry science classroom. In Proceedings of the Conference on Computer Based Learning in Science, Warsaw, Poland. Snir, J., Smith, C. L. & Raz, G. (2003). Linking phenomena with competing underlying models: A software tool for introducing students to the particulate model of matter. Science Education, 87(6), 794␣830. Steiff, M., & Wilensky, U. (2003). Connected chemistry – incorporating interactive simulations into the chemistry classroom. Journal of Science Education and Technology, 12 (3), 285302. Tien, L., Teichart, M., & Rickey, D. (2007). Effectiveness of a MORE laboratory module in prompting students to revise their molecular-level ideas about solutions. Journal of Chemical Education, 84(1), 175-181. Tobin, K. (1990). Research on science laboratory activities: In pursuit of better questions and answers to improve learning. School Science and Mathematics, 90(5), 403-418. Tolentino, L., Birchfield, D., Megowan-Romanowicz, M. C., Johnson-Glenberg, M. C., Kelliher, A., & Martinez, C. (2009). Teaching and learning in the mixed reality science classroom. Journal of Science Education and Technology, 18(6), 501-517. doi:10.1007/s10956-0099166-2 Vosniadou, S. (1994). Capturing and modeling the process of conceptual change. Learning and Instruction, 4(1), 45-69. Wilson, M. (2002). Six views of embodied cognition. Psychonomic Bulletin & Review, 9(4), 625-636. Windschitl, M., & Andre, T. (1998). Using computer simulations to enhance conceptual change: The roles of constructivist instruction and student epistemological beliefs. Journal of Research in Science Teaching, 35, 2, 145-60. White, R. T., & Gunstone, R. F. (1992). Probing understanding. London: Falmer Press. Wu, H. K., Krajcik, J. S., & Soloway, E. (2001). Promoting understanding of chemical representations: Students' use of a visualization tool in the classroom. Journal of Research in Science Teaching, 38(7), 821-842. Xie, C. (2012). Framing Mixed-Reality Labs. @Concord Newsletter, Spring 2012. Retrieved from http://concord.org/publications/newsletter/2012-spring/framing-mixed-reality-labs. Zacharia, Z. C. (2007). Comparing and combining real and virtual experimentation: an effort to enhance students’ conceptual understanding of electric circuits. Journal of Computer Assisted Learning, 23(2), 120–132.

ACCEPTED MANUSCRIPT EFFECTS OF AUGMENTED VIRTUAL LABS ON SCIENCE UNDERSTANDING

27

AC C

EP

TE D

M AN U

SC

RI PT

Zacharia, Z. C., & Anderson, O.R. (2003). The effects of an interactive computer-based simulation prior to performing a laboratory inquiry-based experiment on students’ conceptual understanding of electric circuits. American Journal of Physics, 71, 618-629. Zacharia, Z. C., & Olympiou, G. (2011). Physical versus virtual manipulative experimentation in physics learning. Learning and Instruction, 21(3), 317-331. Zhang, Z. H., & Linn, M. C. (2011). Can generating representations enhance learning with dynamic visualizations? Journal of Research in Science Teaching, 48(10), 1177-1198. Zollman, D. A., Rebello, N. S., & Hogg, K. (2002). Quantum mechanics for everyone: Hands-on activities integrated with technology. American Journal of Physics, 70, 252.

ACCEPTED MANUSCRIPT EFFECTS OF AUGMENTED VIRTUAL LABS ON SCIENCE UNDERSTANDING

28

Appendix A – Pretest/Posttest Questions and Sample Normative Responses Questions 1-3: Air mattresses Air mattresses are inflatable pads used for sleeping. A company tested air mattresses in a variety of usages and environmental conditions. Please provide explanations for each of these results.

RI PT

1. (Proximal, Temperature) Result 1: A manager put the mattress next to the hot fireplace, warming up the air inside the mattress. The manager noticed it was getting hot, so then she placed it by the window where it was cooler. Explain what happened to the gas particles inside the mattress when heated and cooled:

SC

Example normative student answer: When the mattress was next to the hot fireplace, the gas particles inside the mattress increased in speed because molecules with more energy move faster. When the mattress was placed by the window, the particles inside the mattress slowed down because less energy means the molecules will move slower.

M AN U

2. (Proximal, Pressure-Volume) Result 2: One of the company's employees sent an air mattress to his son in college. Unfortunately, the mattress burst when six students sat on the mattress to play cards. Explain why this happened:

TE D

Example normative student answer: The air mattress normally is inflated because there are air molecules inside the mattress that are constantly moving and bouncing against the walls, putting a force on the inside of the walls that pushes the mattress out. Normally there is also an external pressure on the mattress from the outside air molecules. But if too many students sit on the mattress, the outside pressure will be too much for the mattress to handle and the material will break.

EP

3. (Proximal, Temperature-Pressure) Result 3: A few employees took the air mattress out for camping. They inflated it on a hot summer day but then it became substantially deflated during the night when it was a much cooler temperature. Explain why this happened:

AC C

Example normative student answer: When the temperature cooled down, the speed of the air molecules on average decreased. This leads to them bumping into the walls of the mattress less and that decreases the pressure, which means that it will be deflated. 4. (Distal, Pressure) Why do inflated balloons keep their shape? Please explain at the particle level. Example normative student answer: The gas molecules inside the balloons are constantly moving and colliding with each other and the balloon walls. The collisions with the balloon wall exert a force and a pressure on the balloon from the inside pushing out. 5. (Distal, Pressure-Volume) A small balloon is filled with air. When you squeeze the balloon, you feel it’s harder and harder to squeeze. Please explain why this happens at the particle level. Example normative student answer: Air molecules inside the balloon are moving around and

ACCEPTED MANUSCRIPT EFFECTS OF AUGMENTED VIRTUAL LABS ON SCIENCE UNDERSTANDING

29

colliding with the walls. When you squeeze the balloon, you are decreasing the volume of the space that the molecules are in, which means that they will hit the walls on average more often than before. If you increase the collisions, that increases the force and the pressure from the gas, which makes it feel harder and harder to squeeze.

RI PT

6. (Distal, Temperature-Pressure) Car tire pressure needs to be checked and kept within a recommended range both in the summer and in the winter. Please explain why at the particle level.

AC C

EP

TE D

M AN U

SC

Example student answer: In the summer, the temperature is higher than in winter and the gas molecules in the tire will be moving faster, which means they collide with the inside of the tire more often that can result in increased pressure. In the winter, the temperature is lower and the gas molecules will be moving more slowly, which means that they collide with the inside of the tire less often which can result in decreased pressure.

ACCEPTED MANUSCRIPT

AC C

EP

TE D

M AN U

SC

RI PT

Acknowledgements This material is based upon work supported by the National Science Foundation under grants IIS-1123868 and IIS-1124281. Any opinions, findings, and conclusions or recommendations expressed in this material are those of the authors and do not necessarily reflect the views of the National Science Foundation. The authors would especially like to thank Charles Xie and Edmund Hazzard from the Concord Consortium for their collaboration on this project, as well as the teachers and students who participated in this project.