Learning computer programming through dynamic representation of computer functioning: evaluation of a new learning package for Pascal

Int. J. Man-Machine Studies (1986) 25, 327-341

Learning computer programming through dynamic representation of computer functioning: evaluation of a new learning package for Pascal LEONARD GOODWIN AND MOHAMMAD SANATIt

Departments of Social Science & Policy Studies and t Computer Science, Worcester Polytechnic Institute, Worcester, M A 01609, U.S.A. (Received 8 January 1986) This paper describes and evaluates a new approach to teaching the beginning Pascal programming course at Worcester Polytechnic Institute. At the heart of this approach is a new computer learning package, called PASLAB, which allows students to understand what is happening inside the computer relative to statements in Pascal programs constructed by an expert. There were 322 students participating in the evaluation under traditional conditions in 1984, and 296 students participating under the new conditions in 1985. Comparison between the two groups was made by examining the respective regression equations predicting final grade for each set of students. While the characteristics of both sets of students were very similar, the regression equations were markedly different. Under traditional conditions, background characteristics of students, including programming experience, had a major impact on the grades those students received. With Paslab in place, the impact of those background characteristics decreased while the importance of psychological orientations (motivation) increased. There were no differences in performance with respect to gender. Further research is needed to test the impact of Paslab in non-technical institutions. The positive results of this study suggest the fruitfulness of developing additional computer learning packages on the same principles as Paslab to be used in other computer-science courses as well as in other fileds.

Introduction and purpose Computers are rapidly pervading and influencing many aspects of social life. Increasing numbers of people with computer knowledge will be needed, and more students will wish to study computer science. It is important, therefore, to develop educational techniques, especially in the beginning programming course, which maximize the achievement of the greatest number of students for a given amount of instructor effort. This paper briefly describes and quantitatively evaluates a new approach to teaching the beginning Pascal programming course at Worcester Polytechnic Institute (WPI). At the heart of this approach is a new computer learning package, called Paslab, which allows students to understand what is happening inside the computer relative to statements in Pascal programs constructed by an expert. Integration of this package into the course means that the class instructor no longer spends class time in explaining the details of such matters as syntax. Instead, the instructor discusses the general purposes and strategies used in Pascal, reducing the number of lectures from four to two a week. 327 0020-7373/86/090327+ 15503.00/0

© 1986Academic Press Inc. (London) Limited)

328

L. G O O D W I N A N D M. S A N A T I

The evaluation involves a comparison between the two approaches to Pascal teaching: the traditional approach; the new approach integrating Paslab into the coursework. Our basic hypothesis is that: Under the new learning conditions, as compared with traditional conditions, background characteristics of students will have much less impact on final grades, while attitudes will have much greater impact. Another way of expressing this hypothesis is that the use of Paslab will tend to compensate for a limited or negatively oriented background with respect to computer programming. Hence, achievement in computer programming as measured by final grade in the course will become less dependent on previous background and more related to psychological orientations about such matters as computer programming and self-confidence. The following sections present a brief review of relevant literature on computer learning and the rationale of the new learning package. Results of the evaluation are then presented along with the educational significance of the findings.

Previous research on computer learning The new Pascal learning package makes use of findings that have appeared in the literature on computer learning. It has been found that presenting schematic illustrations of a computer's actions facilitate the learning of programming skills (Allen, 1982; Mayer, 1981). This idea is expanded in the concept of making a program "transparent". Peele (1975) talks of the "glass-box program" explicating concepts and procedures in conncrete terms. He gives examples of how APL programs can be used to advance certain concepts in psychology and other fields. These examples are quite brief and limited. DuBoulay, O'Shea & Monk (1981) discuss the use of a "notional machine" which is an idealized model of the computer implied by the constructs of the programming language. They discuss the need for simplicity and visibility, being able to see on the screen selected parts of the program in action either through pictures or verbal messages. They then examine the attributes of three programming systems of this kind including a modification of LOGO. Lieberman (1984) presents some examples of "Tinker", which is "a programming environment for Lisp in w h i c h . . , the system displays graphically the result of each step as it is performed. The programmer can see what the program is doing". Lieberman illustrates the use of Tinker in solving a search problem. The idea of being able to see what the program is actually doing is attractive. But applying this concept systematically to a programming course and testing the result in a regular classroom setting, has not been reported by researchers. This task is fulfilled in the present study. Paslab turns the computer into a "glass box" by showing students what happens in the memory, input, and output of the computer upon the execution of each line of the given Pascal programs. Certain researchers, in an effort to improve the performance of novice programmers, have sought to elucidate the differences between novice and expert programmers. One major difference is that experts tend to place a programming problem within a "schema"--a general abstract formulation of a task whereas novices tend to think

N E W L E A R N I N G P A C K A G E FOR PASCAL

329

of concrete lines of a program (Adelson, 1984; Mayer 1981; McKeithen, Reitman, Rueter & Hirtle, 1981). Paslab allows students to see the analytical structure of the program as a whole in the course of providing a pictorial representation of what happens in the computer during the execution of the program. Students can go at their own speed, stopping on each line of the program or quickly going through the entire program. In any case they are becoming aware of the way in which an expert programmer proceeds. Some researchers have considered the personal characteristics that make for success in programming. Thus, it has been found that quantitative, mathematical skills lead to success in computer science (Campbell & McCabe, 1984; Fowler & Glorfeld, 1981; Peterson & Howe, 1979). Measures of personality and cognitive learning styles also appear to have some potentiality in predicting achievement (Lawton & Geschner, 1982). Using the Myers-Briggs measures, it was found that persons are more likely to complete a computer aided Morse code course if they have the ability to quietly concentrate and pay attention to details. Persons who like variety, action, and harmonious group projects tend to leave the course (Holiman & Waters, 1982). While learning Morse code is different from learning Pascal, it behooves us to consider preferences for learning styles and conditions as possible predictors of achievement. In a similar vein, it is reasonable to include measures of persons' interest in learning through abstract presentations or through the use of pictures or graphs (Carey, 1982). The evaluation of the WPI Pascal learning packhge presents important educational findings that go beyond those reported in the literature. Before presenting them, it is necessary to describe the WPI learning package, called Paslab, in more detail.

The Pascal learning package: Paslab The Paslab learning package is meant to be part of a beginning course in Pascal. The course at WPI using Paslab consists of two 1-h lectures per week, as compared with four 1-h lectures per week under the traditional approach. The lectures under the new approach do not deal with specific programming problems or syntax but rather are aimed at motivating students and presenting general problems solving approaches. Specific programming matters are handled by Paslab, which is used during a 3-h

Select Lab Session 1. Basic Data T y p e s / I n p u t O u t p u t / A s s i g n m e n t 2. Expressions/Conditional Statements 3. User Defined Data Types/Iterative Statements 4. Case and Repeat Statements/More Input Output 5. Structured Data Types: A R R A Y S 6. Functions and Procedures/Recursion 7. Records and Sets 8. Pointers a n d Linked Lists 9. Files 10. Return to DOS Enter Choice: FIG. 1. Main m e n u to select a laboratory session.

330

L. GOODWIN AND M. SANATI

l a b o r a t o r y p e r i o d each week. S t u d e n t s are not graded o n the Paslab exercises. T h e y are g r a d e d o n the p r o g r a m m i n g a s s i g n m e n t s given each week a n d on the e x a m i n a t i o n s . The P a s l a b period begins with students l o a d i n g the p r o g r a m o n to the m i c r o c o m p u t e r a n d selecting a specific lab session (see Fig. 1). W i t h i n that session a p a r t i c u l a r p r o g r a m m i n g exercise is c h o s e n for execution. Each exercise concentrates o n o n e p r o g r a m m i n g concept or l a n g u a g e construct. A n e x a m p l e is given in Fig. 2.

Lab Session 2 Select Exercise 1. 2. 3. 4. 5.

Arithmetic and Boolean Expressions If Statement Nested If Statement Application: Quadratic Equation Terminate This Lab Session

Enter Choice: FIG. 2. An example of exercise menu. The e x e c u t i o n of a p r o g r a m is entirely u n d e r the s t u d e n t s ' control. They m a y e n t e r the S T E P m o d e to execute the p r o g r a m line b y line, where each statement will b e highlighted b u t not executed u n t i l a key is pressed (Fig. 3). W h e n a statement is executed, the effects o f its e x e c u t i o n are illustrated graphically in a clear m a n n e r u s i n g separate d i s p l a y w i n d o w s for p r o g r a m , m e m o r y , input, a n d o u t p u t (Fig. 4). S t u d e n t s can t h e r e b y observe the d y n a m i c b e h a v i o r of a p r o g r a m at their o w n pace.

PROGRAM LAB2EX4(INPUT, OUTPUT); {program to determine the roots of a} {quadratic equation: AX**2+ BX + C = 0} VAR A, B, C :INTEGER; DELTA, ROOT1, ROOT2 :REAL;

STEP mode: Each statement is highlighted, but NOT executed until a key is pressed. AUTO mode: Statements are executed line by line. Press any key to start STEP mode. During Execution . . . . . . / Press Esc to end this exercise. Press F10 to change form STEP mode to AUTO mode or vice versa.

BEGIN {read three coefficients} READ(A, B, C); IF A< >0 THEN BEGIN DELTA: = B* B -4*A* C; IF DELTA < 0 THEN WRITE ('THE ROOTS ARE IMAGINARY.') ELSE IF DELTA = 0 THEN WRITE ('SINGLE ROOT= ', -B/(2*A):8:2) ELSE BEGIN ROOT1: = (-B + SQRT(DELTA))/(2. A); ROOT2: = (-B - SQRT(DELTA))/(2. A); WRITELN('ROOT 1 = ' ROOT1:8:2); WRITELN('ROOT 2 = ' ROOT2:8:2) END {else clause for delta = 0} END {if clause} ELSE WRITE ('NOT A QUADRATIC EQUATION.') END. {program lab2ex4}

FIG. 3. An example of a program display before execution.

331

NEW L E A R N I N G PACKAGE FOR PASCAL PROGRAM LAB2EX4(INPUT, OUTPUT);

{program to determine the roots of a} {quadratic equation: AX * * 2 + BX + C = 0} MEMORY A~I B~-5 C~4 DELTA ~- 9.00 ROOT1 ~- -1.00 ROOT2 ~- - 4 . 0 0 INPUT

1 5 4 OUTPUT ROOT 1 = - 1 - 0 0

VARA, B, C: :INTEGER; DELTA, ROOT1, ROOT2 :REAL;

BEGIN {read three coefficients} READ(A, B, C): IF A < > 0 T H E N BEGIN DELTA: = B* B - 4 * A * C ; IF D E L T A < 0 T H E N WRITE ('THE ROOTS ARE IMAGINARY.') ELSE I F DELTA = 0 T H E N WRITE ( ' S I N G L E R O O T = ', - B / ( 2 * A ) : 8 : 2 ) ELSE BEGIN ROOT1: = ( - B + SQRT(DELTA))/(2 * A): ROOT2: = ( - B - SQRT(DELTA))/(2* A): WRITELN('ROOT 1 = ', ROOT: 8 : 2); WRITELN('ROOT 2 = ', ROOT2 : 8 : 2) END {else clause for delta = 0} E N D {if clause} ELSE WRITE ('NOT A QUADRATIC EQUATION.')

END. {program lab2ex4} FIG. 4. A snapshot of a program example during execution.

The exercise programs require students to enter input data when they are needed. This allows students to investigate the behavior of programs under various input data. Students are encouraged to run each program with a set of data supplied to different possible flows of a program. After students are through with a laboratory session, they are required to practice the learned concepts and constructs by doing a set of self-test exercises. At the end of each laboratory session, they recive the programming assignment which they have 1 week to complete.

Evaluation strategy The theory behind the evaluation was that achievement in the Pascal course was a function of background characteristics of students (e.g. previous experience with computers and mathematical ability) and psychological orientations of students (e.g. interest in computers and expectation of doing well in the course). Other factors affecting achievement were seen as the conditions of the learning situation, which would include the mode of instruction and the computer laboratory situation. The general approach was to measure the background characteristics and psychological orientations of students under traditional Pascal teaching conditions and then under the new conditions at the beginning of each term. In addition, mesurements were made at mid-term regarding experiences in the course--e.g, how difficult one found the assignments. In each situation, traditional and new, a multiple regression equation was developed that predicted the final grade in the course. The final grade was derived from grades on the weekly programming assignments and the examinations. These assignments and examinations were similar under traditional and new conditions (and

332

L. G O O D W l N

A N D M. S A N A T I

the creator of Paslab was not involved in these matters). Predictor variables included those measured at the beginning of each term and at mid-term. It then was possible to determine whether different factors affected achievement under new as compared with traditional conditions.

Procedure and results The academic year at WPI is divided into four terms. The traditional course was taught in Term A 1984 and then repeated in Term B 1984 (running from September to December). The traditional course met four times a week for lecture and 2h a week in the computer laboratory (using the Data General MV8000). The text was Programming and Problem Solving with Pascal by Schneider, Weingart & Perlman. The new course using Paslab met only twice a week for lecture and 3h a week in the IBM personal computer laboratory. During the second class meeting of each term, denoted as time T1, students were asked to complete a questionnaire measuring their background characteristics, certain psychological orientations, and a short test on Pascal. Among the background characteristics were: gender; parents' education and interest in computers; numbers of courses completed in computer science, mathematics, phsysics, and chemistry; Math and Verbal SAT scores; self-rated knowledge of various computer languages; Pascal skill test measured by a 10-min exam during the second class session. Among the psychological measures were: expectation of difficulty in learning computer science; positive or negative feelings toward computers; expectation of use of computers in professional work; attitude toward social impact of computers; preference for different kinds of problems, e.g. single solution vs multiple solution problems. A total of 322 students completed the initial questionaire in terms A and B, representing 92% of all students who entered the traditional course during those terms. A total of 296 students completed the initial questionaire using the new approach during terms C and A in 1985 (January-March and September-October 1985), representing 93% of all students who entered the course. At the beginning of the fourth week of each term, denoted as time T2, students completed a second questionnaire measuring their attitudes toward the assignments completed and toward computer science. Final grades in the course, measured at time T3, were subsequently recorded on a scale from 0-3. A single regression equation was established for terms A and B 1984 (AB84) combined. A similar procedure was followed and a second regression equation established for terms C and A 1985 (CA85) when the new learning package was introduced. The form of the multiple regression equations was as follows: Final grade = blxl + b2x2 + .-. The bs, beta coefficients in this report, are the coefficients of the independent or predictor variables (the xs). A stepwise procedure was used to determine those predictor variables whose coefficients were significantly different from zero. The distribution of

NEW LEARNING PACKAGE FOR PASCAL

333 TABLE 1

Definition and distribution of final grades under traditional (AB84) and new (CA85) conditions

Definition of grader Did not pass course; completed one or two assignments/exams Did not pass course; completed half of assignments/exams Did not pass course; completed almost all assignments/exams Passed course with grade of "Acceptable" Passed course with grade of "Distinction"

Grade

Percentage of students AB84 n = 322

Percentage of students CA85 n = 296

0

2

4

1

6

3

2 3 4

15 55 22

4 68 21

100

100

t Worcester Polytecnic Institute has a three-point grade scale: No Record (not passing); Acceptable; and, Distinction. For the purpose of this study, the grade scale was extended to five points by asking the computer science instructors to give three levels of rating, 0, 1, and 2, for students who failed to pass the course.

the dependent variable, final grade in the course, for the traditional and new approaches appears in Table 1. REGRESSION RESULTS

Results of both regression equations--the significant predictors of students' final grades under traditional and new conditions--are presented in Table 2. The beta coefficients indicate the strength of the predictive variables.

Traditional conditions Knowledge of Basic has a beta coefficient o f 0.28 in the equation for traditional conditions. This means that there is a 0.28-unit increase in course grade for each unit of increase in the variable Knowledge of Basic, under traditional conditions. The probability that the coefficient for this predictor variable is really zero, is very small as indicated by the " t " ratio and the accompanying probability estimate shown in Table 2. The beta coefficients, because they are standardized, can be compared with each other. Knowledge of Basic, for example, is more than twice as important a predictor as Math SAT scores, and is the single largest predictor of final grade under traditional conditions. Knowledge of Basic represents familiarity with computers and programming, since Basic is the most ccmmon language taught in high schools. (Sixty-two percent of respondents under traditional conditions reported on the questionnaire as having fair to expert knowledge of Basic). While knowledge of Basic does not directly translate into knowledge of Pascal, the general experience and skill in programming gained, we argue, helps students achieve success in the traditional Pascal course. It is reasonable that those who came to a beginning Pascal course taught in the traditional manner with some computer experience will get higher grades than those

334

L. G O O D W l N

A N D M. S A N A T I

with little experience. This kind o f advantage is not altogether fair in a beginning course because students are being graded in part on previous experience rather than on what has been gained in the course. Math SAT scores also appear in the AB84 term equation. This connection between programming achievement and mathematical or quantitative abilities has been found in previous research as noted ealier. A point of particular concern is that interest in physics, as measured by the n u m b e r of high school courses completed, is antithetical to computer-science achievement under traditional conditions. (The beta coefficient is negative in predicting final grade in the traditional Pascal course.) The possible opposition between interest in science and performance in computer science is further seen in the sets of negative correlations in Table 3. The greater the students' Knowledge of Basic the lower the number of mathematical and physical science courses they have completed in college. The more students enjoy the computer or approve the appellation of "hacker", the less likely are they to complete college courses in mathematics, physics or chemistry. We will briefly discuss this issue again in connection with other findings, but further research is needed to fully explore the relationship between science and computer science proficiency.

TABLE 2

Regression models predicting final grades under traditional (AB84) and new (CA85) conditions Model for AB84 (n = 322) Variables

Model for CA85 (n = 296)

Beta coefficient

"t" ratiot

Beta coefficient

ratio

0.13

2-70

0.17 0.18

3.22 3.38

0.28

5-73

-0-22

-4.66

0-12

2.45 0.14

2.58

-0.16

-2.78

-0.19

-3.41

0.15

2.74

-0.09

- 1-74

"t"

Time T1 Background Math SAT score~ Test score§ Knowledge of Basic¶ Number of high school physics coursesll Major field is computer science/Electrical Engineeringtt Family has personal computers:~$ Attitude I expect the present course to be difficult for me"§§ Prefer problem with one solution¶¶ "In my opinion the best computer programmers:llll are creative" "Plan work carefully to spend as little time as possible at the terminal" "Prefer to write simple specific programs to solve particular tasks".

-0-12

-2.50

NEW LEARNING

PACKAGE FOR PASCAL

335

TAaLE 2 ( c o n t . ) M o d e l for AB84 (n = 322)

Variables Control A t t e n d a n c e in term A'84 vs Term B ' 8 4 t t t R 2 for Time 1 (adjusted without control variable)

M o d e l for CA85 (n = 296)

Beta coefficient

"t" ratiot

Beta coefficient

0"15

3"17

--

0"25

--

0"19

0.22

4.58

0.18

3.73

"t" ratio

Time T2 Attitude Easy assignment~:*:~ "The instructor for this course was h e l p f u l " "I have at times t h o u g h t seriously o f d r o p p i n g this c o u r s e " "The h o m e w o r k assignments have been m o r e beneficial t h a n the classroom p r e s e n t a t i o n s " R 2 total (Time 1 + T i m e 2 adjusted, without control variable)

0.31

-0.10

-1.75

0" 10

1"96

0"21

t Values of "t" of 1.64 or smaller indicate that the beta coefficient has a probability of 0-10 or more of actually being zero. Values of " t " of 1.96 or greater indicate that the beta coefficient has a probability of 0.05 or less of actually being zero. The item asked for Math SAT score. § The total score on a 10-min test of Pascal syntax created by the computer-science instructor and administered with the initial questionnaire at time T1. ¶ The item read, "How knowledgeable are you in Basic?" It was rated on a 4-point scale ranging from No Knowledge to Expert Knowledge. II The item read: "Approximately how many high school courses did you receive credit for in physics," and was rated on a 4-point scale: None; One or Two; Three or Four; Five or More. t t All students who indicated that their major field was computer science or electrical engineering were given a score of ' T ' , all other respondents were given a score of "0". $~ The item read, "Do you (or your family) have a personal computer at home"? 1, No; 2, Yes. §§ This item was rated on a 4-point Strongly Disagree to Strongly Agree scale. ¶¶ The item read: "Circle the number next to the statement that best describes your personal preference. 1, a well-defined problem with several possible solutions on one's approach; 2, a well-defined problem with a single unique solution, which can be proven to be correct or incorrect. IIII The following three items were rated on a 4-point Strongly Disagree to Strongly Agree scale. t t t Each respondent in Term A was given a score of "0" while each respondent in Term B was given a score of "1", thereby defining a dummy variable that distinguishes students from one or the other terms. A similar procedure was followed in Terms C and A 1985; however that dummy variable was not statistically significant and is not reported in this table. ~$:~ The component items of this measure are given in the text. Those items and the following three items in this table were rated on a 4-point Strongly Disagree to Strongly Agree scale.

The finding t h a t s t u d e n t s w h o are m a j o r i n g in c o m p u t e r science or electrical e n g i n e e r i n g d o b e t t e r t h a n o t h e r s in t h e P a s c a l c o u r s e is o n o n e h a n d n o t s u r p r i s i n g . ( M a j o r s i n c o m p u t e r s c i e n c e o r e l e c t r i c a l e n g i n e e r i n g w e r e a s s i g n e d a v a l u e o f " 1 " , all o t h e r s a v a l u e o f " 0 " . ) O n t h e o t h e r h a n d , it s u g g e s t s t h a t t e c h n i c a l s t u d e n t s i n o t h e r f i e l d s

336

L. GOODWIN AND M. SANATI

TABLE 3 Correlations relating computer knowledge and attitudes to number of college science courses completed t Number of college courses completed in: Mathematics

Physics

Chemistry

Knowledge of Basic

-0.31 (-0.26)

-0.23 (-0.19)

-0-25 (-0.12)

"I enjoy working with computers":~

-0.19 (-0-31)

-0.19 (-0.25)

-0.14 (-0.13)

"I would feel complimented if at the end of this course someone said I was a real hacker"~t

-0.22 (-0.16)

-0.24 (-0.14)

-0.16 (-0.07)

Computer knowledge or attitude

t Courses were completed in Terms A and B 1984 and, in parentheses, in terms C and A 1985. Rated on a 4-point Strongly Disagree to Strongly Agree scale.

where computers are, in fact, being used are not as strongly oriented toward mastering computer science under traditional teaching conditions as they might be. There is only one predictor at time T1 that could be considered an attitudinal measure. It deals with one's problem-solving approach: "Prefer problems with one rather than multiple solutions". This kind of rigid mind set--preferring single-solution p r o b l e m s - has negative effects on achievement, and may have even more serious repercussions in higher-level computer science courses. In addition to these predictor variables measured at the beginning of the traditional course, two predictor variables enter the equation predicting final grade from the mid-term measures (time T2). The strongest of these is constructed from responses to five separate items that indicate the ease with which one has been able to complete the homework assignments--writing Pascal programs. These items were identified as measuring a single dimension through the use of factor analysis. The scores on each item, rated on a four-point Strongly Disagree to Strongly Agree scale, were averaged together for each respondent in order to obtain a single score on a new variable which is called, "Easy Assignments", (Scores were calculated so that the higher the score the easier the assignment). The five component items are as follows: Have been more difficult than I expected; have been more time consuming than I expected; have been more frustrating than I expected; were easy to do; went smoothly. Table 2 shows that students who found the assignments difficult (had low scores on "Easy Assignments"), tended to get lower final grades. This seems reasonable at first glance. It also seems reasonable to find that students who found the instructor helpful got higher grades than students who found the instructor unhelpful. A fuller interpretation of these results will be given when we discuss the fact that neither variable appears in the equation predicting grades under the new conditions. The last variable in the equation for traditional conditions is "Attendance in term A vs term B". This is included not as a predictor of final grade, but as a control variable that takes into account any systematic differences in final grade between A term and

NEW LEARNING

PACKAGE FOR PASCAL

337

B term students. The fact that the control variable is statistically significant indicates that there is a systematic bias toward B term students having higher grades than A term students. The bias is both illuminated and taken into account by allowing the control variable to interact in the regression equation. The overall utility of the regression equation is measured by R 2, with a value o f 0.25 given in. Table 2 for the variables measured at time T1. R 2 can vary from 0 to 1. A value of zero indicates that the predictor variables are no more accurate in predicting the final grade of respondents than if one merely assigned the average grade for the whole group to each individual. A value of 1 for R 2 indicates that the predicted grades exactly match the actual grades of respondents. The value of 0.25 indicates that the variables used at time T1 predict grades 25% better than merely assigning the group average to each respondent. It is to be noted that R 2 rises from 0.25 to 0.31 when the two variables measured at time T2 are included in the regression equation. This is a sizeable increase, indicating that experiences occurring in the traditional course are having some effect upon final grade over and above the incoming background of the students. The fact that 31% o f the variance in grades is accounted for by the regression equation indicates the utility of the equation; however, 69% of the variance remains unexplained. New conditions Table 2 also presents the regression equation under the new conditions when Paslab is in use. Comparing this equation with the one obtained under traditional conditions some striking differences are observed. Background characteristics, so important under traditional conditions, become much less important when Paslab is used. More specifically, Knowledge of Basic, the single most important predictor of grades under traditional conditions, disappears under the new conditions. This suggests that Paslab permits students with little background in computers to catch up quickly with those who have substantial background so that final grade is no longer determined by familiarity with Basic. At the same time, one notes a marked increase in the importance of attitudinal variables. In particular, expectation of difficulty in the course comes in as a direct predictor of class grade. Other attitudinal predictors involve orientations toward the attributes of the good programmer. These findings are in accord with our initial hypothesis where it was indicated that Paslab would help students overcome background deficits. We also find that number of high-school physics courses is no longer related to final grade in the Pascal class. This may come about because students now see how statements in Pascal are related to what happens in the computer, and science-oriented students learn more when they can see these kinds of relationships. (Clearly, more research is needed in this area). It is true that certain background characteristics are still in the equation predicting final grade under new conditions. Test score on the initial 10-min quiz on Pascal as well as presence of a personal computer at home suggests that some familiarity with computers still contributes to the final grade. These variables, however, are much less important as predictors, relative to attitudinal variables than under traditional conditions. The point is that the movement of predictors away from background characteristics and toward students' attitudes suggests that students are becoming more responsible

338

L. G O O D W l N

A N D M. S A N A T I

for their own educational results, and their background prior to the course is becoming less influential. Our conclusions are further strengthened by considering the predictive variables that enter the equation at time T2. Under new conditions neither finding the instructor helpful nor finding the assignments easy are predictors o f final grade. An implication is that one can learn Pascal whether or not the instructor is helpful because the Paslab exercises are extremely helpful. A further implication is that students learn Pascal from the assignments whether or not they find those assignments difficult. Indeed, if we examine Table 4 students tend t o find the assignments under the new conditions less easy (or more difficult) than under traditional conditions. Under traditional conditions the difficulty of the assignments signified inability to learn Pascal, while under new conditions the difficulty encountered is overcome so that learning takes place. The intermediate variables that are effective predictors under new conditions refer to intent to drop the course and finding homework assignments more helpful than the lectures. Two points are of special interest. First, these variables do not make a major contribution to the explanation o f final grade. They contribute only an additional 0.02 to the R 2 value of the equation under new conditions--an increase of only 10%. Under traditional conditions the intermediate variables contributed 0.06 to the R 2 value o f the equation--an increse of 20%. A second point is that the intermediate variable of dropping the course does not refer to any specific aspect of the course but is a general predisposition that could occur for any number of reasons. The second intermediate variable suggests that completing the homework assignments, and by implication using Paslab as an aid, is an effective activity. The fact that only 21% of the variance of final grades is explained by the regression equation (vs 31% under traditional conditions) suggests that there are other attitudinal predictors which were not included in the study. (For example, more could be done in measuring different learning styles). Comparing old and new conditions It is of interest to note what is missing from the regression equations as well as what is in them. Gender does not enter either equation. Women do no better or worse than men in the traditional or new Pascal course. This result may be peculiar to WPI and similar schools because of the selective criteria used in admitting all students. That is, women entering an engineering-science college may be more positively oriented toward, and have a stronger background in, computers than college women in general. Further research is needed to test the applicability of the findings reported here to college women and men in general. The overall results from this study indicate that the integration of Paslab into the Pascal course has sharply moved the locus of achievement away from previous background characteristics of students and toward the orientations or motivations o f students. Before accepting this conclusion, however, it is necessary to consider the general characteristics of the students using the traditional and new pproaches. Perhaps these two groups differ substantially. Table 4 reveals that for most part the 322 students using the traditional approach are indistinguishable from the 296 students using the new approach. The only substantial difference occurs in certain of the intermediate variables--students using the new approach are less likely to think of dropping the course, find the instructor more

NEW LEARNING PACKAGE FOR PASCAL

339 TABLE 4

Mean values and standard deviations (in parentheses) of selected variables for students under traditional ( A B 8 4 ) and new ( C A 8 5 ) conditions Mean value

(S.D.)

Variables

AB84

CA85

Final grader

2.88 (0.89)

2.99 (0.85)

645 (64) 1.53 (1.41) 2.64 (0-98) 2.06 (0.55) 0.62 (0-48) 0.41 (0.49) 0.76 (0-44)

648 (62) 1.53 (1.35) 2.74 (0.96) 2.06 (0.56) 0.67 (0.47) 0.41 (0.49) 0.81 (0-39)

2.36 (0.79) 1.28 (0.45) 3-26 (0.63) 3.08 (0.73) 2-78 (0.65) 2.64§ (0.48) 2.60§ (0.58) 1-84§ (0.60) 3-12 (0.50)

2.24 (0.73) 1.34 (0.47) 3.19 (0.60) 2.98 (0.76) 2.76 (0.63) 2.42 (0.51) 2.80 (0.55) 1.65 (0.72) 2.99 (0.59)

0.55 (0.50)

--

Background~: Math SAT score Test score Knowledge of Basic Number of high-school physics courses Major field in computer science/electrical engineering (proportion) Family has personal computer at home (proportion) Gender (proportion male) Attitude "I expect the present course to be difficult for me" Prefer problem with one solution "In my opinion the best computer programmers: "are creative" "Plan work carefully to spend as little time as possible at the terminal" "Prefer to write simple, specific programs to solve particular tasks" Easy assignments "The instructor for this course was helpful" "I have at times thought seriously of dropping this course" "The homework assignments have been more beneficial than classroom presentations" Control Proportion attending Term A1984 out of AB84 Proportion attending Term C1985 out of AC85

0.55 (0.50)

t see Table 1 for a definition and description of the final grade. :~For a fuller description of variables see footnotes Table 2. § The difference between the means from AB84 and CA85 is statistically significant at the 0.05 level of probability.

340

L. ( 3 O O D W I N A N D M. S A N A T I

helpful, and find the assignments more difficult. These kinds of differences are likely the result of the different approaches used in the two courses and not the result of any initial differences between students in AB84 and students in CA85. If our our analysis is correct, we would expect more students using the new approach to pass the Pascal course than students using the traditional approach. Table 1 presents the distribution of final grades during terms AB84 and CA85. Whereas 23% of the students in the traditional course failed (received a 0, 1, or 2 score) only 11% in the new course failed. While there were different teachers for these several courses, they all used approximately the same assignments and grading procedures. (The computer science professor who developed Paslab had no role in devising the tests for, or grading, any students in the study). The sharp rise in students moving out of the failing category and into the category of acceptable performance is very likely the result of the Paslab package. The percentage of students receiving a grade of "distinction" did not alter much between traditional and new classes. The inference to be drawn is that Paslab helps students with lesser background but does not provide a boost to better students. We would have greater knowledge of the impact of Paslab if a standardized achievement test in Pascal had been given to all students before and after the course they had taken. As it is, we have only the final grade to use in the analysis. Further research should incorporate the use of an achievement test.

Summary and implications of findings This study illuminates the effectiveness of a new computer learning package, Paslab, which helps students relate Pascal statements to the dynamic functioning of the computer. It also extends our understanding of the factors influencing success in a beginning programming course. This comes through comparison of the regression equation predicting final grade in the beginning Pascal course at WPI under traditional teaching conditions and under new conditions when Paslab is integrated into the course. There were 322 students at WPI participating in the study under traditional conditions in 1984 and 296 students participating under the new conditions in 1985. Characteristics of both sets of students were very similar, yet the regression equations were markedly different. Under traditional conditions, background characteristics of students, including programming experience, had a major impact on the grades those students recieved. With Paslab in place, the impact of those background characteristics decreased. There was a corresponding increase in the impact of psychological orientations, including expectation of difficulty in the Pascal course. The higher one's expectation of difficulty, the lower the grade obtained. Moreover, one's final grade became less dependent upon how well one got along with the instructor or how difficult one found assignments. The use of Paslab apparently permitted students to learn a good bit about the mechanics of programming on their own. One of the surprising findings under traditional conditions was that students who showed a preference for Physics tended to get poor grades in Pascal programming. When Paslab is introduced, the students with an interest in physical science do as well as other students in the Pascal programming course. The meaning of this finding is not altogether clear. Further research is needed to explore the possible opposition between an interest in science and performance in computer science.

NEW LEARNING PACKAGE FOR PASCAL

341

The fact that women do as well as men in Pascal programming under both conditions needs to be tested in other educational environments. It is possible that WPI has removed a gender bias through its selection procedures--i.e, selecting only those women who have mathematical aptitude. There is a need to replicate the work reported here in other educational institutions which emphasize non-technical as well as technical programs. The achievement of the new Pascal curriculum incorporating Paslab is twofold: (1) it permits students with little computer background to rapidly catch up with students who have greater gackground, thereby shifting the basis for grade achievement away from background factors and toward students' psychological orientations--motivation; (2) it permits the instructor to relegate to Paslab the mechanical details of programming, allowing him or her to concentrate on larger issues such as programming strategy. Further research could not only cross-validate the results reported here but could explore in greater depth the relationship between performance and learning styles. Additional questions could be asked about the learning preferences of students--e.g. preferring graphs or pictures as against purely analytical explanations--as well as their study habits. The positive results of this study suggest the fruitfulness of developing additional computer learning packages on the same principles as Paslab to be used in other computer-science courses (as well as in other fields) and carrying out research on their educational impact.

References ADELSON, B. (1984). When novices surpass experts: the difficulty of a task may increase with expertise. Journal of Experimental Psychology: Learning, Memory, and Cognition, 1O,483-495. ALLEN, R. B. (1984). Cognitive Factors in Human Interaction with Computers. Behavior and Information Technology, 1, 257-258. CAMPBELL, P. F. & MCCABE, G. P. (1984). Predicting the success of freshmen in a computer science major. Communications of the ACM, 27, 1108-1113. CAREY, T. (1982). User difference in interface design. Computer, 15, 14-20. Du BOULAY, B., O'SHEA, T. & MONK, J. (1981). The black box inside the glass box: presenting computing concepts to novices. International Journal of Man-Machine Studies, 14, 237-249. FOWLER, G. C. & GLORFELD, L. W. (1981). Predicting aptitude in introductory computing: a classification model. AEDS Journal, 14, 96-109. HOFFMAN, J. L. & WATERS, K. (1982). Some effects of student personality on success with computer-assisted instruction. Educational Technology, March, 20-21. LAWTON, J. & GERSCHNER, V. T. (1982). A review of the literature on attitudes towards computers and computerized instruction. Journal of Research and development in Education, 16, 50-55. LIEBERMAN, H. (1984). Seeing what your programs are doing. InternationalJournal of M a n Machine Interaction,21, 311-33 I. M A Y E R , R. E. (1981). the psychology of how novices learn computer programming. Computing Surveys, 13, 121-141. McKEITHENEN, K. B., REITMAN, J. S., RUETER, H. H. & HIRTLE, S. C. (1981). Knowledge organization and skill differences in computer programamers. Cognitive Psychology, 13, 307-325. PEELE, H. A. (1975). Teaching with APL.--Iverson's "A Programming Language." Computers and People, November, 12-20. PETERSON, C. G. & HOWE, T. G. (1979). Predicting academic success in introduction to computers, AEDS Journal, 12, 183-191.