- Email: [email protected]
Computer supported collaborative learning for curriculum integration
Computers and Chemical Engineering 24 (2000) 1473-1479
ELSEVIER
Computers &Chemical Engineering www.elsevier.com/locate/compchemeng
Computer supported collaborative learning for curriculum integration Matthew Realff a,*, Pete Ludovice a, Mark Guzdial a
b Tom Morley c, Katherine Sukel d
School of Chemical Engineering, Georgia Institute of Technology, Georgia, GA 30332, USA b College of Computing, Georgia Institute of Technology, Georgia, GA 30332, USA c School of Mathematics, Georgia Institute of Technology, Georgia, GA 30332, USA d School of Psychology, Georgia Institute of Technology, Georgia, GA 30332, USA
Abstract
Computer supported collaborative learning (CSCL) has the potential for significantly improving undergraduate education. However, there are many challenges for institutions, instructors and students in achieving effective collaboration that actually improves learning. Two of the most critical are achieving teacher buy-in and motivating students to participate in a knowledgebuilding community. We have developed a new CSCL tool whose flexibility is inspiring good teachers to create exciting new collaborative learning activities, and whose persistence is an enticement to students. We are now attempting to use this tool to leverage a particular kind of educational reform goal: achieving integration across curricular boundaries. © 2000 Elsevier Science Ltd. All rights reserved.
1. Introduction
In the research described in this paper, we are utilizing computer supported collaborative learning (CSCL) to try to achieve the goal of having students integrate knowledge they have acquired in courses offered by different disciplines. The specific focus is the mathematical modeling of chemical processes. Mathematical modeling takes place throughout the chemical engineering curriculum and builds upon foundations laid in the mathematics curriculum. This subject certainly requires significant integration of knowledge and is of increasing importance as computer experiments are enjoying widespread use in design and operation decision-making. In approaching the improvement of integration, we have three key hypotheses, 1. Collaborative projects between students in classes at different points in curriculum promote transfer. 2. Student assembled, collaborative, persistent web documents serve as an appropriately indexed knowledge repository. 3. Common computational tools serve to mediate language differences across disciplines. * Corresponding author. Tel.: + 1-404-8942000.
The first hypothesis is founded on the idea that by 'reminding' the students in the more senior class that they have seen material in earlier classes via collaboration with current students in that class will help them integrate their knowledge from that class. Similarly, by 'foreminding' the students in the earlier class, they will see how their current coursework fits into the broader context of their degree. The second hypothesis argues that the venue for the collaboration should become something that the students can revisit and that putting the physical representations of knowledge 'near' each other will enable cognitive links to develop in parallel. To support this second hypothesis has been the goal of the educational technology created as part of this project. The third hypothesis uses the fact that many curricula are adopting tools such as Matlab, Maple and Mathematica 1 to support learning across many courses, and that seeing the same programming constructs and function labels in different contexts helps to make connections between knowledge more durable. In the rest of this paper, we will describe the experiments we have started to perform to test these hypotheses and the i MATLAB is a registeredtrademark of MathWorks Inc.; MAPLE is a registered trademark of Waterloo Maple Inc; Mathematica is a registered trademark of Wolfram Research Inc.
0098-1354/00/$ - see front matter © 2000 Elsevier Science Ltd. All rights reserved. PII: S0098-1354(00)00538-X
1474
M. Realff et al./ Computers and Chemical Eng#~eering 24 (2000) 1473-1479
tools that we have constructed to enable collaboration amongst students and instructors.
2. Measuring levels of integrative learning Before we can address integrative learning within the engineering curricula at Georgia Institute of Technology (Georgia Tech.), it is necessary to devise methods to test both the baseline level of integration and then to use the same tests to assess whether the collaboration is having any impact. To perform this assessment it is necessary to factor knowledge into its context and the methods applied within the context. The logic behind making this decomposition explicit is that students who are successfully integrating knowledge across disciplines should be able to complete problems of varying context and methods. If students were successfully integrating knowledge then we would expect them to be able, for example, to solve engineering problems using mathematical tools learned earlier in the curriculum. In particular, we wanted to see if Engineering seniors were able to solve problems in engineering contexts with other kinds of methods, since non-specific engineering problem-solving knowledge (e.g. calculus, computer programming) is typically taught much earlier in the curriculum. We embodied this idea in a set of questions, which were, then, administered under controlled conditions to a group of students at Georgia Tech, the next sections describe the test and its results. 2. I. Participants Thirty-two participants were recruited from Chemical Engineering and Mathematics classes at Georgia Tech. Participants were required to have completed differential equations and at least two engineering core courses in order to be eligible, and were remunerated with either $10, or extra credit in an introductory psychology course. Participants were racially diverse and ranged from Sophomores to Seniors. Half of the participants were chemical engineering majors, with smaller numbers of mechanical engineers, electrical engineers, and various other engineering, mathematics, and science majors making up the difference. Approximately one third of the sample were female (relatively equivalent to the number of women in the engineering program as a whole), and the entire group averaged a GPA of 3.15/ 4.0. 2.2. Materials We designed a test, which included combinations of mathematics, chemical engineering, and numerical methods, problem contexts and problem-solving meth-
ods. We did not consider certain combinations, such as a mathematics context and chemical engineering problem-solving knowledge, because it was impossible to determine a clear definition of chemical engineering problem-solving knowledge in a non-chemical engineering context. Numerical Methods were represented by the common engineering modeling language, MATLAB. An example problem that involves a chemical engineering context with a chemical engineering calculation (because of focus on units and need to use the Ideal Gas Law) is: A hydrogen reactor requires 154 kmol/h of hydrogen, the hydrogen costs $3/1000 ft 2 at standard conditions (T = 32 F P = 14.7 psia). If the reactor is run for 8000 h/year calculate the yearly cost of hydrogen for this reactor.
Gas Constant = 8.314 J/mol K 1 m 3 = 35.3 ft 3 1 atm-- 14.7 psi = 101.325 Kpa An example problem that involves a chemical engineering context and MATLAB problem-solving knowledge is: Write a short piece of MATLAB code that imitates a relief value. When the pressure p, is greater than 1, the function returns the value of 1. When pressure is less than 0.95 the function returns the value 0. When pressure is between 0.95 and 1 if the value was previously 1 return 1 if its previous position was 0 return 0 function output _value : valve _position (p) %begin {valve _position} %end {valve _position}
2.3. Procedure Participants were tested in groups of 2-5 individuals in a laboratory room in the School of Psychology. Each participant was seated individually at a desk and asked not to talk or discuss problems with any of the other participants. Once all participants were seated and informed consent was given, the test booklet was passed out, and participants were instructed to work through each problem in order, and not to go back to problems previously worked on. Additionally, participants were asked to note the time they started and completed each problem, as well as to attempt to answer each problem to the best of their ability. Furthermore, if they were, for some reason, unable to answer the problem, they were instructed to discuss the method they would have used to solve it, or explain their inability to solve it. Participants were, then, prompted for questions, and once their questions were answered, told to begin. Participants were given 2 h to complete the test problems. If students were still working when the 2 h were
1475
M. Realff et al./ Computers and Chemical Engineering 24 (2000) 1473-1479
Table 1 Means for each problem type Calculation
Mathematics C h e m i c a l engineering
MATLAB
2.469 2.219
1.906 1.969
Context
Mathematics Chemical engineering Numerical methods
2.906
3.625
3.00
up, the experimenter made the participant aware of the time and collected their exam. 2.4. Results
Test problems were graded on a 5-point scale. If the problem was solved successfully, a score of 5 was given. If the method was correct, but there had been some type of calculation error, a score of 4 was given. Problems solved with generally correct method were given a score of 3. A score of 2 was given when problems used partially correct method, and a score of 1 was given when participants identified variables correctly and completed preliminary calculation work successfully. Test booklets were graded by two graders independently. There was a high correlation of agreement between the two graders across problems ( r = 0.83). These scores were, then, put into a repeated measure analysis of variance to test for interactions of context and calculation. If there was integrated learning, we would expect that students would perform similarly within a context despite the calculation, within a calculation despite the context, or similarly over all. But instead, we found significant differences. A significant main effect of calculation was found, F(2, 6 2 ) = 4.49, M S E = 7.35, P = 0.02, where participants scored higher on problems with mathematical calculations over chemical engineering and M A T L A B calculations. There was a significant interaction of context and calculation, F(4,124) = 10.58, M S E = 21.58, P < 0.01. This interaction was driven by participants scoring higher on consistent context and calculation problems (e.g. chemical engineering context,
chemical engineering calculation) across chemical engineering and numerical methods areas (Table 1). Neither sex, race, GPA nor major, when added as an additional variable, constituted a significant difference. Participants scored higher on problems that were consistent across context and calculation. Surprisingly, matching a major and a context was not a guarantee of success. Electrical Engineering (EE) majors did better on Chemical Engineering context problems than did Chemical Engineers (Table 2). Additionally, we examined the length of time it took participants to complete each type of problem as a function of context and calculation. Both a significant main effect of context [F(2, 6 2 ) = 10.81, M S E = 231.82, P < 0.01] and calculation [F(2,62) = 25.80, M S E = 385.02, P < 0.01] were found. In the context main effect, participants took more time to complete mathematics problems, then chemical engineering, spending the least time on numerical methods problems. In the calculation main effect, participants took the most time on chemical engineering calculations over a relatively equal time on mathematics and Matlab calculations. Finally, there was a significant interaction of context and calculation, F(4,124) = 9.32, M S E = 211.22, P < 0.0l. This interaction was driven by participants spending more time on mathematics problems overall, but given that students scored higher on mathematics problems, it may be that this extra time was spent in completing the problems they knew how to solve. 2.5. Discussion
The results are somewhat disappointing, though not surprising. In general, students were able to solve problems well when the context and calculation matched well. But where they did not, performance was disappointing. Students' performance in numerical methods is particularly disappointing, considering the importance of these kinds of problems in professional engineering practice. But this could reflect the skill of the students with the particular program language. The results suggest that students are not integrating their knowledge across the multiple classes and years of their Engineering curriculum.
Table 2 Context problem averages, by student major
Chemical engineering major Mechanical engineering major Electrical engineering major Various engineering
Mathematics
Chemical Engineering
Numerical Method
2.229 2.167 3.222 2.143
2.375 2.167 2.889 2.286
2.562 2.556 3.111 2.952
M. Realff et al./ Computers and Chemical Engineering 24 (2000) 1473-1479
1476
operating systems, so they are available on whatever platform a teacher may have access to. The CoWeb was based on the WikiWikiWeb developed by Ward Cunningham. 3.2. Co Web in computer modeling
a~zl~q~j r.o~n~
r "n-~Sl~rt
- -
~llabor~ve i~jects ~ ~ CoU~or~ivo pt~j~t
~
13esJp ~
L
Fig. 1. CoWeb support in the curriculum. There are caveats to these results. It could have been that the mixed context-calculation problems were harder than the same context-calculation problems, something we are attempting to correct for by careful design of problems that embed the same concepts. However, the consistency of the results suggests otherwise.
3. The CoWeb collaboration space
3. I. Introduction to the Co Web
A CoWeb is a collaborative website (Guzdial et al., 1999). Simply put, any user can edit any page in the website, any user can create new pages by putting *A new page name* in the text, and old pages can be linked by inserting *An old page name* in the body of a page. Each page contains an 'Edit' button, which retrieves an editable version of the current page. After any modifications are made, the 'Save' button is used to update the page on the website. CoWeb pages generally contain plain ascii text but authors may optionally include images, simple formatting commands (for making links, lists, headings, etc.) and H T M L tags. They may also use a simple notation for creating and linking to their own pages in the CoWeb. Graphics and hyperlinks off-site can easily be entered, also without HTML. A 'Recent Changes' page lists dates in reverse chronological order and the pages that were edited on the date, so that users can easily see what's new. The software supporting the CoWeb is built on the Pluggable WebServer (PWS) implemented in the Squeak programming language (http:// squeak.cs.uiuc.edu). Squeak, and thus the CoWeb, have been run on a wide variety of platforms including Macintosh, Windows-95, Windows NT, and SunOS
The specific focus of our activities has been the integration of computer programming with mathematics and chemical engineering. Computer modeling is a key concept in engineering education, and students visit the topic in several disciplines. In introductory Computer Science, we teach them to program in MATLAB. In Calculus, we teach them differential equations and their solutions, sometimes using MATLAB and MAPLE. In Chemical Engineering, students use differential equations to model many different chemical process systems and phenomena, i.e. momentum, heat and mass transfer, reactor systems, process dynamics and control systems. However, these classes refer to one another infrequently, and in fact, different language for the same kinds of concepts is used in the different classes. Furthermore, the learning goals and objectives are often different between these disciplines. For example, a focus on existence and convergence proofs may dominate the mathematics course, the structure of the computer code, the computer course, and the physical phenomena the chemical engineering course. Thus, traditional practice actually may make it difficult for students to transfer concepts between these classes. A CoWeb is being constructed around the topic of 'Computer modeling': The Computer Modeling for Curriculum Integration (CMCI) Project (http:// math36.math.gatech.edu:8080/model). This site is meant to act as a persistent repository for students to find cross-indexed content that threads together the material presented in the different courses. The most important feature of the CoWeb is the ability of students to create their own pages and then index the site themselves. This enables them to pull together the material as they understand it, and, as they progress through the curriculum, revisit the same place to re-index the information as it fits into the context of their major discipline. An abstract representation of how this space fits within the curriculum is given in Fig. 1. The project is at an early stage and thus, we do not yet have data to confirm that this is how the Computer Modeling CoWeb is utilized by the students. Currently, we are encouraging the use of the CoWeb in the sophomore level classes in mathematics and chemical engineering, and will track student use over the next three years to validate whether or not the hypothesized use of the CoWeb happens in practice. Initial observations about the use of CoWebs in this and other contexts are that the CoWeb use goes through a progression of more and more complex activity on the part
M. Realff et al./ Computers and Chemical Engineering 24 (2000) 1473-1479
1477
Table 3 Typical collaborations Course
Task 1
Task 2
Task 3
Compare the analytical solution to the linear version and the numerical solution of the nonlinear. Compare the analytical solution to the linear version and the numerical solution of the nonlinear.
Discuss the effect of the nonlinear term on the nature and uniqueness of the solution. Discuss the conditions under which the nonlinear term can be neglected.
Analytically solve the ODEs assuming fluid velocity is constant across the reactor cross-section (plug flow assumption). Numerically solve the ODEs that describe the reaction assuming realistic flow patterns.
Determine how the variation in the velocity field can be used to customize the reactor product yield. Evaluate the conditions under which the plug flow assumption can be made.
Typical mathematics collaboration Differential equations (mathematics) Chemical engineering numerical methods
Neglect the nonlinear term below and solve this 2nd order ODE analytically. Numerically calculate the solution to a nonlinear 2nd order ODE heat transfer problem.
Typical chemical engineering collaboration Kinetics & reactor design
Construct the ODEs to describe a tubular reactor.
Chemcial engineering numerical methods
of the participants. First, use of the CoWeb always starts with prompted use, such as the requirement of the students to perform an assignment to turn-in, e.g. 'Who's Who' pages. Second, unprompted use starts with a voluntary but useful activity, such as Midterm Exam Review, On-Line Review and Critique. Once participation in the CoWeb has started, students take ownership and its use becomes exponentially greater. For example, students update their CoWeb pages after graduating the specific class in which the CoWeb was used, students maintain pointers and 'clean up' pages themselves, and finally the use diversifies to include purely social activities such as adventure games, posting songs, on-line game sign-up pages etc.
4. Synchronous collaborative projects A second thrust of the project is to initiate synchronous collaborations between groups of students in different classes. A typical set of collaborations is given in Table 3. So far, we have done this with two sets of two classes. In the first, sophomores and seniors in Chmical Engineering (ChE) collaborated to illustrate numerical methods in chemical engineering applied to empirical models for process control. In the second, students in a math class required of ChE students collaborated with ChE students in the process dynamics and control class to use phase plane representations for process dynamics. The CoWeb is used to support these collaborative projects through the simple mechanism of having students create pages and attachments to those pages. The students in the different classes then link their respective pages together. The collaboration is made meaningful by having students perform a set of activities. First, if data or programs need to be transferred, the students in the different classes must agree
to the meaning and format of the data or programs and the specification of what the programs should do. Second, the students are asked to comment on the results of the other class's work to understand the relevance of it to their specific situation, and to record those comments in the collaborative space. We have not done any controlled studies of the effect of this collaboration, but hope to do so in the upcoming semester, Spring 2000.
4.1. Example project To illustrate the above ideas one of the collaborations mentioned above will be described in more detail. An overall schematic is given in data to the sophomore class, which is shown in Fig. 2. The particular project chosen was empirical modeling of a chemical reactor. The students in process control were charged with designing two different model based controllers for a reactor that had been modeled in Matlab/Simulink. One was a feedback controller designed using direct synthesis, the other was a feed forward controller. The students tested 'the reactor' using impulse testing. They Sovhomore Numerical Methods Course
Objective: Build Numerical Integration Tool for evaluation of moments of a function Matlab Code for evaluation of function moments from Time Series Data Senior Process Control Coarse Objective: Build Empirical Model for subsequent process controller synthesis Formulae for calculation of model parameters from function moments
Value of Moments
Data File (time, outout)
Fig. 2. Schematic of example collaborative project.
1478
M. Realff et al./ Computers and Chemical Engineering 24 (2000) 1473-1479
then built an empirical transfer function model using relationships between the moments of the time response of the system to an impulse function and the transfer function parameters (Ogunike & Ray, 1994). The sophomore level class was required to take the impulse response data and develop suitable numerical integration schemes, and implementations in Matlab, to find an arbitrary number of moments of the function. These moments were then transferred back to the students in the control system class. We repeated this collaboration during the spring 1998 quarter between the same two classes. In this case, the senior class tested a reactor with the temperature presumed to be digitally sampled and with a discrete controller proposed. The senior students passed the discrete time series data to the sophomore class who carried out generalized linear regression to find a first order model with time delay. The sophomore students passed the results back to the senior class who again used this to design a control system.
5. Assessment strategy for impact of CSCL The challenging part of assessing this research is that there is very little appropriate theory about how students learning computer-supported collaborative learning in the type of context described above. Our best theory about collaborative learning says that students learn collaboratively through the process of dialogue, the construction of shared understanding, and the reference and manipulation of shared representations (Roschelle, 1992; Jeong & Chi, 1997). Yet, we have evidence that the majority of users of computer-supported collaborative learning do not engage in anywhere near the amount of dialogue for that theory to explain any learning (Guzdial, 1997). Further, there is evidence that students are learning through these environments, but learning does not correlate with amount of dialogue in the tools (Hoadley, 1998). There are two artifacts created as part of the collaboration, one is the CoWeb space itself, and the other is the logfile that shows access to the space. In addition to this electronic evidence, we can assess the students themselves to find out if there has been any impact on them by utilizing the CoWeb and paying explicit attention to curriculum integration. 5. I. Student assessment
Our student assessment takes two forms, • Assessment of student knowledge. For this, we are using two assessments. The first is an in-class test that pulls problems from the matrix described in
Section 2. We will apply this test repeatedly over the course of the next few years to gather data on problem-solving performance of students at particular points in the curriculum. If we are having a positive impact on integrative learning, then we should see that the performance on off-diagonal elements of the matrix should improve. The second is to place problems on the final examinations of certain courses that reflect the collaborative projects undertaken during the course. We should see that classes that participate in collaborative projects would have superior performance than control groups of students that do not. Assessment of student attitudes. We would like to assess whether students hold different attitudes towards the knowledge they have. For example, we will not be tracking mathematics students through all the engineering disciplines but we would like to see a shift in students' attitudes towards seeing and valuing the applications of the knowledge they gain in these earlier non-discipline specific courses.
5.2. Co Web and logfile assessment
Our current working hypothesis for how computersupported collaborative learning takes place drives our data gathering and analysis efforts. Unlike classroom-based collaborative learning, computer-supported collaborative learning leaves a trace, typically, a written transcript of the dialogue. We know that reviews of such traces can lead to significant learning (Collins & Brown, 1988). Further, in computer-supported collaborative learning, these traces are mostly structured in the form of questions and answers (Guzdial, 1997), which may be more effective for learning than a more traditional prose structure (Carroll & Smith-Kerker, 1986). Our hypothesis is that learning in a computer-supported collaborative learning results from the reading (and writing, but it is not the predominant mechanism) of a reflective trace of a dialogue that is structured in a manner conducive for learning. To measure use, our plan is to use logfile analysis, which has been useful in this kind of evaluation in the past (Guzdial & Hmelo, 1997). A logfile tracks student-actions in a tool. Through analysis of the logfile and review of the meeting space trace, we can determine what students did in a space. In particular, the logfile will inform us what postings students actually read, but cannot tell us what students attended to. Through interviews with students, we can get a better sense of what they're attending to and how they are learning. Through these analyses, we believe that we can construct a model for how students are learning in a space such as this.
M. Realff et al./ Computers and Chemical Engineering 24 (2000) 1473-1479
6. Conclusions Cross curriculum collaboration has the potential to be of significant value in promoting transfer of knowledge between the core curricula of the sophomore and freshman years and the domain curricula that overlap and continue through a degree program. There are two types of collaboration we intend to support, and in this paper, we have outlined educational technology being developed to support both types. The first is embodied in synchronous projects that involve students from multiple classes. This type of collaboration will be useful both for the students and to generate case studies of where integration is useful. The technology being employed are editable web pages with mechanisms for easy linking and embedding of appropriate representations for the domain. The second is asynchronous collaboration where students from different classes use the cases, and indices developed around those cases, to support their activities within a class and seek peer involvement in understanding and developing solutions to new problems and projects. A key factor in the success of this approach will be the development of indices both of which are useful from the standpoint of integration but which do not presuppose a sophisticated understanding of the material already existing. One of the most difficult aspects of this type of collaborative environment is assessing the impact that it has on student learning and transfer in particular. We have made some suggestions as to how this can be done but this is still an open question for educational research.
Acknowledgements Thanks to the students and faculty involved in these
1479
studies, especially the students helping to build and analyze these tools - - the Georgia Tech Squeakers. Funding for some of this work is from National Science Foundation, grant REC-9814770 and from SUN AEG award # 7826-990329-US.
References Carroll, J. M., & Smith-Kerker, P.L. (1986). The minimal manual. IBM Thomas J. Watson Research Center technical report. Collins, A., & Brown, J. S. (1988). The computer as a tool for learning through reflection. In H. Mandl,& A. Lesgold, Learning issues for intelligent tutoring systems (pp. 1-18). New York: Springer. Guzdial M. (1997). Information ecology of collaborations in educational settings: influence of tool. In: R. Hall, N. Miyake & N. Enyedy, Proceedings of computer-supported collaborative learning'97 (83-90). Toronto, Ont., Canada. Guzdial, M., & Hmelo, C. (1997). Integrating and Guiding Collaboration: lessons learned in computer-supported collaboration learning research at Georgia Tech. In: R. Hall, N. Miyake & N. Enyedy, Proceedings of computer-supported collaborative learning'97 (91-100). Toronto, Ont., Canada. Guzdial, M., Realff, M., Ludovice, P., Morely, T., Kerce, C., Lyons, E., & Sukel, K. (1999). Using a CSCL-driven shift in agency to undertake educational reform. Proceedings of CSCL99, Palo Alto, CA, USA. Hoadley, C. (1998). Scaffolding scientific discussion using socially relevant representations in networked multimedia. Unpublished PhD Dissertation, School of Education, University of California at Berkeley. Jeong, H. & Chi, M. T. H. (1997). Construction of shared knowledge during collaborative learning. In: R. Hall, N. Miyake & N. Enyedy, Proceedings of computer-supported collaborative learning'97 (124-128). Toronto, Ont., Canada. Ogunike, B. A., & Ray, W. H. (1994). Process dynamics, modeling, and control. Oxford: Oxford University Press. Roschelle, J. (1992). Learning by collaborating: convergent conceptual change. Journal of the Learning Sciences, 2(3), 235-276.
ELSEVIER
Computers &Chemical Engineering www.elsevier.com/locate/compchemeng
Computer supported collaborative learning for curriculum integration Matthew Realff a,*, Pete Ludovice a, Mark Guzdial a
b Tom Morley c, Katherine Sukel d
School of Chemical Engineering, Georgia Institute of Technology, Georgia, GA 30332, USA b College of Computing, Georgia Institute of Technology, Georgia, GA 30332, USA c School of Mathematics, Georgia Institute of Technology, Georgia, GA 30332, USA d School of Psychology, Georgia Institute of Technology, Georgia, GA 30332, USA
Abstract
Computer supported collaborative learning (CSCL) has the potential for significantly improving undergraduate education. However, there are many challenges for institutions, instructors and students in achieving effective collaboration that actually improves learning. Two of the most critical are achieving teacher buy-in and motivating students to participate in a knowledgebuilding community. We have developed a new CSCL tool whose flexibility is inspiring good teachers to create exciting new collaborative learning activities, and whose persistence is an enticement to students. We are now attempting to use this tool to leverage a particular kind of educational reform goal: achieving integration across curricular boundaries. © 2000 Elsevier Science Ltd. All rights reserved.
1. Introduction
In the research described in this paper, we are utilizing computer supported collaborative learning (CSCL) to try to achieve the goal of having students integrate knowledge they have acquired in courses offered by different disciplines. The specific focus is the mathematical modeling of chemical processes. Mathematical modeling takes place throughout the chemical engineering curriculum and builds upon foundations laid in the mathematics curriculum. This subject certainly requires significant integration of knowledge and is of increasing importance as computer experiments are enjoying widespread use in design and operation decision-making. In approaching the improvement of integration, we have three key hypotheses, 1. Collaborative projects between students in classes at different points in curriculum promote transfer. 2. Student assembled, collaborative, persistent web documents serve as an appropriately indexed knowledge repository. 3. Common computational tools serve to mediate language differences across disciplines. * Corresponding author. Tel.: + 1-404-8942000.
The first hypothesis is founded on the idea that by 'reminding' the students in the more senior class that they have seen material in earlier classes via collaboration with current students in that class will help them integrate their knowledge from that class. Similarly, by 'foreminding' the students in the earlier class, they will see how their current coursework fits into the broader context of their degree. The second hypothesis argues that the venue for the collaboration should become something that the students can revisit and that putting the physical representations of knowledge 'near' each other will enable cognitive links to develop in parallel. To support this second hypothesis has been the goal of the educational technology created as part of this project. The third hypothesis uses the fact that many curricula are adopting tools such as Matlab, Maple and Mathematica 1 to support learning across many courses, and that seeing the same programming constructs and function labels in different contexts helps to make connections between knowledge more durable. In the rest of this paper, we will describe the experiments we have started to perform to test these hypotheses and the i MATLAB is a registeredtrademark of MathWorks Inc.; MAPLE is a registered trademark of Waterloo Maple Inc; Mathematica is a registered trademark of Wolfram Research Inc.
0098-1354/00/$ - see front matter © 2000 Elsevier Science Ltd. All rights reserved. PII: S0098-1354(00)00538-X
1474
M. Realff et al./ Computers and Chemical Eng#~eering 24 (2000) 1473-1479
tools that we have constructed to enable collaboration amongst students and instructors.
2. Measuring levels of integrative learning Before we can address integrative learning within the engineering curricula at Georgia Institute of Technology (Georgia Tech.), it is necessary to devise methods to test both the baseline level of integration and then to use the same tests to assess whether the collaboration is having any impact. To perform this assessment it is necessary to factor knowledge into its context and the methods applied within the context. The logic behind making this decomposition explicit is that students who are successfully integrating knowledge across disciplines should be able to complete problems of varying context and methods. If students were successfully integrating knowledge then we would expect them to be able, for example, to solve engineering problems using mathematical tools learned earlier in the curriculum. In particular, we wanted to see if Engineering seniors were able to solve problems in engineering contexts with other kinds of methods, since non-specific engineering problem-solving knowledge (e.g. calculus, computer programming) is typically taught much earlier in the curriculum. We embodied this idea in a set of questions, which were, then, administered under controlled conditions to a group of students at Georgia Tech, the next sections describe the test and its results. 2. I. Participants Thirty-two participants were recruited from Chemical Engineering and Mathematics classes at Georgia Tech. Participants were required to have completed differential equations and at least two engineering core courses in order to be eligible, and were remunerated with either $10, or extra credit in an introductory psychology course. Participants were racially diverse and ranged from Sophomores to Seniors. Half of the participants were chemical engineering majors, with smaller numbers of mechanical engineers, electrical engineers, and various other engineering, mathematics, and science majors making up the difference. Approximately one third of the sample were female (relatively equivalent to the number of women in the engineering program as a whole), and the entire group averaged a GPA of 3.15/ 4.0. 2.2. Materials We designed a test, which included combinations of mathematics, chemical engineering, and numerical methods, problem contexts and problem-solving meth-
ods. We did not consider certain combinations, such as a mathematics context and chemical engineering problem-solving knowledge, because it was impossible to determine a clear definition of chemical engineering problem-solving knowledge in a non-chemical engineering context. Numerical Methods were represented by the common engineering modeling language, MATLAB. An example problem that involves a chemical engineering context with a chemical engineering calculation (because of focus on units and need to use the Ideal Gas Law) is: A hydrogen reactor requires 154 kmol/h of hydrogen, the hydrogen costs $3/1000 ft 2 at standard conditions (T = 32 F P = 14.7 psia). If the reactor is run for 8000 h/year calculate the yearly cost of hydrogen for this reactor.
Gas Constant = 8.314 J/mol K 1 m 3 = 35.3 ft 3 1 atm-- 14.7 psi = 101.325 Kpa An example problem that involves a chemical engineering context and MATLAB problem-solving knowledge is: Write a short piece of MATLAB code that imitates a relief value. When the pressure p, is greater than 1, the function returns the value of 1. When pressure is less than 0.95 the function returns the value 0. When pressure is between 0.95 and 1 if the value was previously 1 return 1 if its previous position was 0 return 0 function output _value : valve _position (p) %begin {valve _position} %end {valve _position}
2.3. Procedure Participants were tested in groups of 2-5 individuals in a laboratory room in the School of Psychology. Each participant was seated individually at a desk and asked not to talk or discuss problems with any of the other participants. Once all participants were seated and informed consent was given, the test booklet was passed out, and participants were instructed to work through each problem in order, and not to go back to problems previously worked on. Additionally, participants were asked to note the time they started and completed each problem, as well as to attempt to answer each problem to the best of their ability. Furthermore, if they were, for some reason, unable to answer the problem, they were instructed to discuss the method they would have used to solve it, or explain their inability to solve it. Participants were, then, prompted for questions, and once their questions were answered, told to begin. Participants were given 2 h to complete the test problems. If students were still working when the 2 h were
1475
M. Realff et al./ Computers and Chemical Engineering 24 (2000) 1473-1479
Table 1 Means for each problem type Calculation
Mathematics C h e m i c a l engineering
MATLAB
2.469 2.219
1.906 1.969
Context
Mathematics Chemical engineering Numerical methods
2.906
3.625
3.00
up, the experimenter made the participant aware of the time and collected their exam. 2.4. Results
Test problems were graded on a 5-point scale. If the problem was solved successfully, a score of 5 was given. If the method was correct, but there had been some type of calculation error, a score of 4 was given. Problems solved with generally correct method were given a score of 3. A score of 2 was given when problems used partially correct method, and a score of 1 was given when participants identified variables correctly and completed preliminary calculation work successfully. Test booklets were graded by two graders independently. There was a high correlation of agreement between the two graders across problems ( r = 0.83). These scores were, then, put into a repeated measure analysis of variance to test for interactions of context and calculation. If there was integrated learning, we would expect that students would perform similarly within a context despite the calculation, within a calculation despite the context, or similarly over all. But instead, we found significant differences. A significant main effect of calculation was found, F(2, 6 2 ) = 4.49, M S E = 7.35, P = 0.02, where participants scored higher on problems with mathematical calculations over chemical engineering and M A T L A B calculations. There was a significant interaction of context and calculation, F(4,124) = 10.58, M S E = 21.58, P < 0.01. This interaction was driven by participants scoring higher on consistent context and calculation problems (e.g. chemical engineering context,
chemical engineering calculation) across chemical engineering and numerical methods areas (Table 1). Neither sex, race, GPA nor major, when added as an additional variable, constituted a significant difference. Participants scored higher on problems that were consistent across context and calculation. Surprisingly, matching a major and a context was not a guarantee of success. Electrical Engineering (EE) majors did better on Chemical Engineering context problems than did Chemical Engineers (Table 2). Additionally, we examined the length of time it took participants to complete each type of problem as a function of context and calculation. Both a significant main effect of context [F(2, 6 2 ) = 10.81, M S E = 231.82, P < 0.01] and calculation [F(2,62) = 25.80, M S E = 385.02, P < 0.01] were found. In the context main effect, participants took more time to complete mathematics problems, then chemical engineering, spending the least time on numerical methods problems. In the calculation main effect, participants took the most time on chemical engineering calculations over a relatively equal time on mathematics and Matlab calculations. Finally, there was a significant interaction of context and calculation, F(4,124) = 9.32, M S E = 211.22, P < 0.0l. This interaction was driven by participants spending more time on mathematics problems overall, but given that students scored higher on mathematics problems, it may be that this extra time was spent in completing the problems they knew how to solve. 2.5. Discussion
The results are somewhat disappointing, though not surprising. In general, students were able to solve problems well when the context and calculation matched well. But where they did not, performance was disappointing. Students' performance in numerical methods is particularly disappointing, considering the importance of these kinds of problems in professional engineering practice. But this could reflect the skill of the students with the particular program language. The results suggest that students are not integrating their knowledge across the multiple classes and years of their Engineering curriculum.
Table 2 Context problem averages, by student major
Chemical engineering major Mechanical engineering major Electrical engineering major Various engineering
Mathematics
Chemical Engineering
Numerical Method
2.229 2.167 3.222 2.143
2.375 2.167 2.889 2.286
2.562 2.556 3.111 2.952
M. Realff et al./ Computers and Chemical Engineering 24 (2000) 1473-1479
1476
operating systems, so they are available on whatever platform a teacher may have access to. The CoWeb was based on the WikiWikiWeb developed by Ward Cunningham. 3.2. Co Web in computer modeling
a~zl~q~j r.o~n~
r "n-~Sl~rt
- -
~llabor~ve i~jects ~ ~ CoU~or~ivo pt~j~t
~
13esJp ~
L
Fig. 1. CoWeb support in the curriculum. There are caveats to these results. It could have been that the mixed context-calculation problems were harder than the same context-calculation problems, something we are attempting to correct for by careful design of problems that embed the same concepts. However, the consistency of the results suggests otherwise.
3. The CoWeb collaboration space
3. I. Introduction to the Co Web
A CoWeb is a collaborative website (Guzdial et al., 1999). Simply put, any user can edit any page in the website, any user can create new pages by putting *A new page name* in the text, and old pages can be linked by inserting *An old page name* in the body of a page. Each page contains an 'Edit' button, which retrieves an editable version of the current page. After any modifications are made, the 'Save' button is used to update the page on the website. CoWeb pages generally contain plain ascii text but authors may optionally include images, simple formatting commands (for making links, lists, headings, etc.) and H T M L tags. They may also use a simple notation for creating and linking to their own pages in the CoWeb. Graphics and hyperlinks off-site can easily be entered, also without HTML. A 'Recent Changes' page lists dates in reverse chronological order and the pages that were edited on the date, so that users can easily see what's new. The software supporting the CoWeb is built on the Pluggable WebServer (PWS) implemented in the Squeak programming language (http:// squeak.cs.uiuc.edu). Squeak, and thus the CoWeb, have been run on a wide variety of platforms including Macintosh, Windows-95, Windows NT, and SunOS
The specific focus of our activities has been the integration of computer programming with mathematics and chemical engineering. Computer modeling is a key concept in engineering education, and students visit the topic in several disciplines. In introductory Computer Science, we teach them to program in MATLAB. In Calculus, we teach them differential equations and their solutions, sometimes using MATLAB and MAPLE. In Chemical Engineering, students use differential equations to model many different chemical process systems and phenomena, i.e. momentum, heat and mass transfer, reactor systems, process dynamics and control systems. However, these classes refer to one another infrequently, and in fact, different language for the same kinds of concepts is used in the different classes. Furthermore, the learning goals and objectives are often different between these disciplines. For example, a focus on existence and convergence proofs may dominate the mathematics course, the structure of the computer code, the computer course, and the physical phenomena the chemical engineering course. Thus, traditional practice actually may make it difficult for students to transfer concepts between these classes. A CoWeb is being constructed around the topic of 'Computer modeling': The Computer Modeling for Curriculum Integration (CMCI) Project (http:// math36.math.gatech.edu:8080/model). This site is meant to act as a persistent repository for students to find cross-indexed content that threads together the material presented in the different courses. The most important feature of the CoWeb is the ability of students to create their own pages and then index the site themselves. This enables them to pull together the material as they understand it, and, as they progress through the curriculum, revisit the same place to re-index the information as it fits into the context of their major discipline. An abstract representation of how this space fits within the curriculum is given in Fig. 1. The project is at an early stage and thus, we do not yet have data to confirm that this is how the Computer Modeling CoWeb is utilized by the students. Currently, we are encouraging the use of the CoWeb in the sophomore level classes in mathematics and chemical engineering, and will track student use over the next three years to validate whether or not the hypothesized use of the CoWeb happens in practice. Initial observations about the use of CoWebs in this and other contexts are that the CoWeb use goes through a progression of more and more complex activity on the part
M. Realff et al./ Computers and Chemical Engineering 24 (2000) 1473-1479
1477
Table 3 Typical collaborations Course
Task 1
Task 2
Task 3
Compare the analytical solution to the linear version and the numerical solution of the nonlinear. Compare the analytical solution to the linear version and the numerical solution of the nonlinear.
Discuss the effect of the nonlinear term on the nature and uniqueness of the solution. Discuss the conditions under which the nonlinear term can be neglected.
Analytically solve the ODEs assuming fluid velocity is constant across the reactor cross-section (plug flow assumption). Numerically solve the ODEs that describe the reaction assuming realistic flow patterns.
Determine how the variation in the velocity field can be used to customize the reactor product yield. Evaluate the conditions under which the plug flow assumption can be made.
Typical mathematics collaboration Differential equations (mathematics) Chemical engineering numerical methods
Neglect the nonlinear term below and solve this 2nd order ODE analytically. Numerically calculate the solution to a nonlinear 2nd order ODE heat transfer problem.
Typical chemical engineering collaboration Kinetics & reactor design
Construct the ODEs to describe a tubular reactor.
Chemcial engineering numerical methods
of the participants. First, use of the CoWeb always starts with prompted use, such as the requirement of the students to perform an assignment to turn-in, e.g. 'Who's Who' pages. Second, unprompted use starts with a voluntary but useful activity, such as Midterm Exam Review, On-Line Review and Critique. Once participation in the CoWeb has started, students take ownership and its use becomes exponentially greater. For example, students update their CoWeb pages after graduating the specific class in which the CoWeb was used, students maintain pointers and 'clean up' pages themselves, and finally the use diversifies to include purely social activities such as adventure games, posting songs, on-line game sign-up pages etc.
4. Synchronous collaborative projects A second thrust of the project is to initiate synchronous collaborations between groups of students in different classes. A typical set of collaborations is given in Table 3. So far, we have done this with two sets of two classes. In the first, sophomores and seniors in Chmical Engineering (ChE) collaborated to illustrate numerical methods in chemical engineering applied to empirical models for process control. In the second, students in a math class required of ChE students collaborated with ChE students in the process dynamics and control class to use phase plane representations for process dynamics. The CoWeb is used to support these collaborative projects through the simple mechanism of having students create pages and attachments to those pages. The students in the different classes then link their respective pages together. The collaboration is made meaningful by having students perform a set of activities. First, if data or programs need to be transferred, the students in the different classes must agree
to the meaning and format of the data or programs and the specification of what the programs should do. Second, the students are asked to comment on the results of the other class's work to understand the relevance of it to their specific situation, and to record those comments in the collaborative space. We have not done any controlled studies of the effect of this collaboration, but hope to do so in the upcoming semester, Spring 2000.
4.1. Example project To illustrate the above ideas one of the collaborations mentioned above will be described in more detail. An overall schematic is given in data to the sophomore class, which is shown in Fig. 2. The particular project chosen was empirical modeling of a chemical reactor. The students in process control were charged with designing two different model based controllers for a reactor that had been modeled in Matlab/Simulink. One was a feedback controller designed using direct synthesis, the other was a feed forward controller. The students tested 'the reactor' using impulse testing. They Sovhomore Numerical Methods Course
Objective: Build Numerical Integration Tool for evaluation of moments of a function Matlab Code for evaluation of function moments from Time Series Data Senior Process Control Coarse Objective: Build Empirical Model for subsequent process controller synthesis Formulae for calculation of model parameters from function moments
Value of Moments
Data File (time, outout)
Fig. 2. Schematic of example collaborative project.
1478
M. Realff et al./ Computers and Chemical Engineering 24 (2000) 1473-1479
then built an empirical transfer function model using relationships between the moments of the time response of the system to an impulse function and the transfer function parameters (Ogunike & Ray, 1994). The sophomore level class was required to take the impulse response data and develop suitable numerical integration schemes, and implementations in Matlab, to find an arbitrary number of moments of the function. These moments were then transferred back to the students in the control system class. We repeated this collaboration during the spring 1998 quarter between the same two classes. In this case, the senior class tested a reactor with the temperature presumed to be digitally sampled and with a discrete controller proposed. The senior students passed the discrete time series data to the sophomore class who carried out generalized linear regression to find a first order model with time delay. The sophomore students passed the results back to the senior class who again used this to design a control system.
5. Assessment strategy for impact of CSCL The challenging part of assessing this research is that there is very little appropriate theory about how students learning computer-supported collaborative learning in the type of context described above. Our best theory about collaborative learning says that students learn collaboratively through the process of dialogue, the construction of shared understanding, and the reference and manipulation of shared representations (Roschelle, 1992; Jeong & Chi, 1997). Yet, we have evidence that the majority of users of computer-supported collaborative learning do not engage in anywhere near the amount of dialogue for that theory to explain any learning (Guzdial, 1997). Further, there is evidence that students are learning through these environments, but learning does not correlate with amount of dialogue in the tools (Hoadley, 1998). There are two artifacts created as part of the collaboration, one is the CoWeb space itself, and the other is the logfile that shows access to the space. In addition to this electronic evidence, we can assess the students themselves to find out if there has been any impact on them by utilizing the CoWeb and paying explicit attention to curriculum integration. 5. I. Student assessment
Our student assessment takes two forms, • Assessment of student knowledge. For this, we are using two assessments. The first is an in-class test that pulls problems from the matrix described in
Section 2. We will apply this test repeatedly over the course of the next few years to gather data on problem-solving performance of students at particular points in the curriculum. If we are having a positive impact on integrative learning, then we should see that the performance on off-diagonal elements of the matrix should improve. The second is to place problems on the final examinations of certain courses that reflect the collaborative projects undertaken during the course. We should see that classes that participate in collaborative projects would have superior performance than control groups of students that do not. Assessment of student attitudes. We would like to assess whether students hold different attitudes towards the knowledge they have. For example, we will not be tracking mathematics students through all the engineering disciplines but we would like to see a shift in students' attitudes towards seeing and valuing the applications of the knowledge they gain in these earlier non-discipline specific courses.
5.2. Co Web and logfile assessment
Our current working hypothesis for how computersupported collaborative learning takes place drives our data gathering and analysis efforts. Unlike classroom-based collaborative learning, computer-supported collaborative learning leaves a trace, typically, a written transcript of the dialogue. We know that reviews of such traces can lead to significant learning (Collins & Brown, 1988). Further, in computer-supported collaborative learning, these traces are mostly structured in the form of questions and answers (Guzdial, 1997), which may be more effective for learning than a more traditional prose structure (Carroll & Smith-Kerker, 1986). Our hypothesis is that learning in a computer-supported collaborative learning results from the reading (and writing, but it is not the predominant mechanism) of a reflective trace of a dialogue that is structured in a manner conducive for learning. To measure use, our plan is to use logfile analysis, which has been useful in this kind of evaluation in the past (Guzdial & Hmelo, 1997). A logfile tracks student-actions in a tool. Through analysis of the logfile and review of the meeting space trace, we can determine what students did in a space. In particular, the logfile will inform us what postings students actually read, but cannot tell us what students attended to. Through interviews with students, we can get a better sense of what they're attending to and how they are learning. Through these analyses, we believe that we can construct a model for how students are learning in a space such as this.
M. Realff et al./ Computers and Chemical Engineering 24 (2000) 1473-1479
6. Conclusions Cross curriculum collaboration has the potential to be of significant value in promoting transfer of knowledge between the core curricula of the sophomore and freshman years and the domain curricula that overlap and continue through a degree program. There are two types of collaboration we intend to support, and in this paper, we have outlined educational technology being developed to support both types. The first is embodied in synchronous projects that involve students from multiple classes. This type of collaboration will be useful both for the students and to generate case studies of where integration is useful. The technology being employed are editable web pages with mechanisms for easy linking and embedding of appropriate representations for the domain. The second is asynchronous collaboration where students from different classes use the cases, and indices developed around those cases, to support their activities within a class and seek peer involvement in understanding and developing solutions to new problems and projects. A key factor in the success of this approach will be the development of indices both of which are useful from the standpoint of integration but which do not presuppose a sophisticated understanding of the material already existing. One of the most difficult aspects of this type of collaborative environment is assessing the impact that it has on student learning and transfer in particular. We have made some suggestions as to how this can be done but this is still an open question for educational research.
Acknowledgements Thanks to the students and faculty involved in these
1479
studies, especially the students helping to build and analyze these tools - - the Georgia Tech Squeakers. Funding for some of this work is from National Science Foundation, grant REC-9814770 and from SUN AEG award # 7826-990329-US.
References Carroll, J. M., & Smith-Kerker, P.L. (1986). The minimal manual. IBM Thomas J. Watson Research Center technical report. Collins, A., & Brown, J. S. (1988). The computer as a tool for learning through reflection. In H. Mandl,& A. Lesgold, Learning issues for intelligent tutoring systems (pp. 1-18). New York: Springer. Guzdial M. (1997). Information ecology of collaborations in educational settings: influence of tool. In: R. Hall, N. Miyake & N. Enyedy, Proceedings of computer-supported collaborative learning'97 (83-90). Toronto, Ont., Canada. Guzdial, M., & Hmelo, C. (1997). Integrating and Guiding Collaboration: lessons learned in computer-supported collaboration learning research at Georgia Tech. In: R. Hall, N. Miyake & N. Enyedy, Proceedings of computer-supported collaborative learning'97 (91-100). Toronto, Ont., Canada. Guzdial, M., Realff, M., Ludovice, P., Morely, T., Kerce, C., Lyons, E., & Sukel, K. (1999). Using a CSCL-driven shift in agency to undertake educational reform. Proceedings of CSCL99, Palo Alto, CA, USA. Hoadley, C. (1998). Scaffolding scientific discussion using socially relevant representations in networked multimedia. Unpublished PhD Dissertation, School of Education, University of California at Berkeley. Jeong, H. & Chi, M. T. H. (1997). Construction of shared knowledge during collaborative learning. In: R. Hall, N. Miyake & N. Enyedy, Proceedings of computer-supported collaborative learning'97 (124-128). Toronto, Ont., Canada. Ogunike, B. A., & Ray, W. H. (1994). Process dynamics, modeling, and control. Oxford: Oxford University Press. Roschelle, J. (1992). Learning by collaborating: convergent conceptual change. Journal of the Learning Sciences, 2(3), 235-276.