- Email: [email protected]
Does feedback enhance computer-assisted language learning?
Compu~rs E&c. Vol. 21, No. l/2, pp. 61-65, 1993 Printed in Great
Bntain.
Copyright
All rights reserved
DOES
c
0360- I31 S/93 $6.00 + 0.00 1993 Pergamon Press Ltd
FEEDBACK ENHANCE COMPUTER-ASSISTED LANGUAGE LEARNING? ELISABETH
University
of Amsterdam,
Spuistraat
VAN DER LINDEN 134, 1012 VB Amsterdam, The Netherlands
Abstract-Programs for computer-assisted language learning (CALL) are becoming more complex. On one hand, courseware designers develop programs of a more open-ended character, e.g. adventures, hypermedia courseware. On the other hand, courseware in the traditional drill and practice vein is more and more elaborate and offers extensive feedback to the learner. Developing this last type of courseware, in which feedback generally is based on an error analysis of a large group of learners, demands an enormous investment in time and energy. The question is whether this investment pays OR do learners indeed learn better by programs with a considerable amount of feedback? This question was addressed in a research project at the French Department of the University of Amsterdam. Students’ reactions to feedback were studied in two ways. First, log files were analyzed containing the series of tasks-responses-feedback of each student. In addition, think aloud protocols were analyzed. The results from the log files, which were confirmed by the analysis of the think aloud protocols, show that there was no overall successful strategy. While some students made optimal use of the feedback provided, others seemed to avoid using it in several ways.
INTRODUCTION In the last few years, computer-assisted language learning (CALL) has been expanding quickly but technology has been progressing even faster. There seem to be two different approaches in recent developments of CALL. On one hand, we find the traditional drill and practice programs such as the ones described by Higgins and Johns[l], on the other hand we find programs using more advanced technology, such as the one described by Viau and Lariv&e[2]. This last type of program generally makes use of authentic foreign language materials to enable learners to progress. Learners are offered considerable freedom in the way they work with the program. They can ask questions about grammatical aspects of the text, about words that they do not know, and about several other aspects of the text or the foreign language[2,3]. Programs in the more traditional vein oblige the learners to work within a rather strictly limited framework, asking them to produce precisely defined utterances, emphasizing thereby correctness of grammar or vocabulary[l]. Often, these drill and practice programs were associated with a behavioristic view of the learning process. As Swann[4] put it: “It [was] often naively assumed that what the student does is to learn grammar rules, memorize vocabulary and put the two together by means of exercises”. Nowadays, everybody knows that language learning does not work this way. “Grammar is neither necessary nor sufficient for second language learning: many people do learn a second language without any exposure to its grammatical description”[4]. Although this change in vision led to a more cognitive learning approach, it did not lead to the abandonment of drill and practice programs. Teachers and students continue to ask for them. But even these more traditional CALL programs became different. Their primary aim is no more to automatize language structures by drill exercises. They rather try to enhance learning by comprehension of these structures to “facilitate the work of the learning mechanism”[4], since retention can only take place when a language phenomenon is converted into a cognitive structure [S]. These programs offer complex feedback to the learners, guiding them to the correct answer through comprehension of the language structure involved. The steps in these programs are:
(1) Select task or question (2) Display question (3) (4) (5) (6)
Accept student’s response Match response against acceptable response Give feedback about response If response correct, go to 1 (next task); if incorrect, 61
let student
try again and go to 3.
62
ELISABETHVAN DER LINDEN
The fourth step, asking for an inventory of all acceptable answers, is time consuming but not problematical. The fifth step, construct adequate feedback, asks for theoretical as well as practical reflection on the role which feedback plays in the learning process. However, we do not know much of the real importance of feedback in CALL. Does it help learners to learn better, to understand or to retain more, or faster? It is this question that will be the subject of this paper.
LEARNING
FROM
FEEDBACK
According to cognitive learning models such as that of Rumelhart and Norman[$ learning can take place only after comprehension. Hence the emphasis on corrective feedback. In research on second language learning, little attention has been paid to the role of feedback, although it is clear that in the foreign language classroom, a considerable amount of feedback is provided to the learners. Research such as that of Cohen and Robins[6] and Long[7] does not show a convincing positive effect of feedback on second language learning. There is however a difference between traditional foreign language learning situations and computer-assisted learning. Whereas in traditional foreign language classroom situations, feedback is not very systematic nor immediate, in the CAI-context: -feedback is provided for every incorrect response; -it is given immediately; -self-correction by the learner is required [ 11. One can quite plausibly suggest that this makes feedback in CALL more effective than it is in the above described classroom situations. Theoretical as well as practical reasons make it important to understand the role of feedback in computer-assisted foreign language instruction. From a theoretical point of view, finding out how students go about learning from CALL may help to get more precise information on aspects of the learning process. From a practical point of view, this will help to design effective programs. If complex feedback is not as helpful as was thought, considerable money and time could be saved in constructing simpler CALL programs. To find out more about the role of feedback in CALL, a research program at the French Department of the University of Amsterdam, followed closely a group of first-year students working with a CALL program for French grammar.
THE
PROGRAM
First-year students at the French Department have to take an exam in French grammar at the end of term. To prepare for this exam, they can work with a computer-aided instructional program covering the most important parts of the subject matter for the exam. In this CA1 program, students are confronted with traditional grammar problem-solving tasks: the computer presents a sentence, and asks them to carry out a substitution or transformation task, such as in Fig. 1. After each provision of feedback, the student can try again to give the correct response. For each sentence, incorrect responses generally ranging between five and ten and acceptable responses are foreseen. For each of these incorrect responses specific feedback is provided. When a response is given that is not foreseen, the computer tells so to the student. Feedback concentrates always on only one error. If the student makes more than one error at a time, feedback is given for only one of them so as not to make things too complex for the student. In the next attempt, feedback for another error is given if still necessary. Whenever students give an incorrect response, they are invited to try again, using information from the feedback they have received. Students may try to produce the right answer as many times as they want. They can also ask for the right answer or go on to the next sentence whenever they want. Apart from the feedback which they automatically receive, students can ask for hints. This paper will concentrate on the exercises for the form and place in the sentence of the French personal pronouns. This part of French grammar confronts students with numerous problems: in
63
Language learning their performance, errors as well as avoidance strategies above described steps. A typical example of a possible Sentence: Response Feedback Response Feedback
are frequent [8]. The program follows the route through the program is:
Ne donnez plus de bonbons aux enfants. 1: Ne les en donnez pas. 1: Peut-on remplacer “aux enfants” par “les”? 2: Ne leur en donnez pas. 2: Oui, c’est correct.
PROCEDURE Two methods were used to study the strategies that students adopted when working with the program. First, log files of the work of 23 students were analysed to see in detail the routes they followed through the program, looking especially at the number of errors, the number of attempts and the effectiveness of the feedback. Second, since the first analysis offered mainly quantitative data that gave only a limited insight into the strategies that made students work in the way they did, a series of interviews were used to obtain think aloud protocols[9] of students working with the program. Analysis of these interviews found out more about what students think they do when they are using the CALL program. Although there is discussion about the usefulness and reliability of introspection data, the view of Ericsson and Simon [ lo] was taken that introspection is a reliable tool for research into internal cognitive processes.
RESULTS
Log files Studying the log files was expected to determine mainly the “optimal” strategy . That is, students were expected to work through the program as described above: starting by trying to respond, studying the feedback they received when their responses were incorrect, trying then to give the correct response, subsequently repeating this cycle if the responses were still incorrect and so on. This is not what was always found. The log files registered every step the students took with the aid of the computer, and on that basis the routes were divided into four: (1) (2) (3) (4)
The The The The
go on till you get it right strategy; drill strategy; once is enough strategy; browsing strategy.
1 Remplacez dans la phrase suivante les parties 1 par des pronoms personnels, y ou en. 1 Mettez ces formes a la bonne place. Maman 1 Maman
ne laisse pas les enfants
jouer dans
. . . ne
. . . jouer.
. . . laisse
. . . pas
le
soulign&es
jardin.
(space for feedback)
1 Fl Help 1 PgUp one page back
F2 Hint PgDn one page Fig.I.Example of an exercise.
F10 Stop further
1 !
64
ELISABETH VAN DER LINDEN
The first of these strategies is the “optimal” strategy that we expected-or hoped-to find with all the students. This was not the case. Only 12 students showed a clear preference for this strategy, while it frequently appeared in the log files of five students. It was very rare or non-existent in the log files of the remaining six students. The 12 students who preferred this way of working followed the expected route, going through the sequence of response-feedback-retrial sometimes up to eight or nine times. Generally however, two trials were enough to produce the correct response. The second strategy was called drill-like because it reminds one of the behavior in classical drill programs, where the only feedback that is provided concerns the correctness of the response. While second tries are possible, no feedback about the type of error is given in those programs. In this case, students who worked this way did not try to correct themselves when their response had been wrong: they just asked for the correct response and then went on to the next sentence. This way of proceeding through the program was used regularly by most students. The 12 optimal workers chose to use it occasionally; six students preferred this strategy and used it most of the time. Three students used it regularly. Students working this way seemed not to be interested in the detailed feedback they received; they probably just wanted to try once, and when wrong, to find out the correct response. As was pointed out, students might feel that this way of working is more effective because it is faster; at the same time, however, it requires less active production from the student. which suggests that it might lead to less learning. The third strategy was labelled the “once is enough” strategy: students working this way do not correct themselves when they are wrong or even look up the correct response: they just respond once, and irrespective of the correctness of this response. they go on immediately to the next sentence. This behavior is puzzling. It suggests that the students concerned, even if they find that their responses were incorrect, do not try to find out why it was incorrect or even what would have been the correct solution. Perhaps this is an example of test behavior: the students just want to know how many of their first responses are correct or incorrect in order to find out what level of proficiency they have attained. The last strategy, browsing, implies that students just look at the sentences of the exercise: the) go directly from one sentence to the next without trying to give responses. Sometimes, they then go back to the beginning and start really working through the program, sometimes they just quit and start working on a different exercise. In the first case, we are probably dealing with students who first want to have a look at what it is all about before they really start working. In the second case, perhaps the students concerned just want to do the exercises orally. It remains however surprising that they should not even take the time to look up the correct response. Interviews and thinking-aloud protocols In order to discover what brought about these different strategies, we interviewed six students and listened to them as they carried out the program while “thinking aloud”. This enabled us to relate some of the strategies described above to individual learning styles. Earlier research [&I I], had showed that idiosyncratic approaches of this type of grammar problem solving exercises explain at least as much of the achieved learning results as other factors like intelligence or general proficiency. In the analysis of the log files we found that indeed individual differences play a role in the choice of students’ strategies in CALL as well. In the interviews, these impressions were confirmed. On one side, there are the “zealous” students who use the optimal strategy until the bitter end. Other students however are impatient to get through the program and follow the drill like strategy. They sometimes pretend that this strategy is more useful to them. Reading all the feedback would confuse them. It is possible that indeed they are right: we found that the students who use up to eight or ten trials in order to find the correct response, sometimes make the same mistakes in more than one of their responses, showing that they have not retained earlier feedback received in the same sequence of responses. This suggests that the usefulness of complex feedback can perhaps not be generalized to all students: for some, it may be more effective than for others. Apart from this finding, there are a series of minor findings that seem to account for certain behavior patterns in the students, and these were confirmed in the interviews. Jn the first place. many problems arose from editing and type writing problems. As a matter of fact, we expected this kind of problem, not all students being familiar with computer aided writing. We expected it
Language
learning
65
however to be only a minor problem because the writing tasks did not differ much from those of a normal typewriting machine. Nevertheless, several students continued to get nervous when confronted with the simple tasks that had to be carried out on the computer. For this reason, some of them missed up to one-third of the sentences. Secondly, we found that, the longer the feedback is, the less it will be read. Feedback of more than three lines (existing but rare in the program) was hardly ever read until the end. Lengthy explanation does not therefore seem to work effectively. Consequently, we decide to shorten the feedback where necessary and to facilitate access to the schemes of grammatical rules that were equally present in the program, so that students could look them up separately when they wanted to see the whole rule system pertaining to an exercise. Thirdly, we discovered that, as a rule, students did not start working through the program by reading its introductory part, where the grammar rules, necessary to do the exercise, were explained. Generally, they skipped this part of the programs and it was only when they were confronted with problems they could not solve that some of the students went back to read the first section. This was another reason why we decided to elaborate on the introductory rule section and to give it the status of a background grammar to be consulted whenever necessary.
CONCLUSIONS
In conclusion, we see that students had some strategies in common, but for the rest behaved in idiosyncratic ways. Two main working strategies could be distinguished, one which we called the “optimal” one, where substantive amounts of complex feedback were consulted by the students, and the “drill-like” one, where students seemed to avoid reading feedback. We saw that the drill-like strategy, which we initially considered to be less effective, is possibly an equally effective strategy for some of the students, perhaps especially the less intelligent ones. Frequent consultation of sophisticated feedback seemed, for them, to be confusing. A practical conclusion arising from our work is that feedback, in order to be consulted, has to be concise and precise. Long feedback (exceeding three lines) is not read and for that reason is not useful. Another aspect of the use of feedback is of course its effect on the final learning results of the type of tasks we described here. In other words, what are the short- and long-term retention effects of the use or non-use of feedback. It is not unlikely that the drill-like strategy we have described here seems at first sight an effective learning strategy for some students, but that the “optimal” strategy leads to better retention. This is the question that we will investigate next. REFERENCES 1. Higgins J. and Johns T., Computers in Language Learning. Collins, London (1984). 2. Viau R. and Larivte J., Learning tools with hypertext: an experiment. Computers Educ. 20, 11-17 (1993). towards a global learning environment. In New Media in rhe Humanities (Edited 3. Rtzeau J., Minim1 and Framework: by van Loon H. and Janssen J.), pp. 86-93. Faculty of Arts, Amsterdam (1991). 4. Swann P., Computer assisted language learning for English as a foreign language. Computers Educ. 19, 251-266 (1992). D. and Norman D. A., Explorafions in Cognition. Freeman, San Francisco (1975). 5. Rumelhart 6. Cohen A. and Robins M., Errors, interaction and correction: a study of native-nonnative conversations. TESOL-Q 16, 537-547 (1976). I. Long M., Does second language instruction make a difference? TESOL-Q 17, 359-382 (1983). 8. Van der Linden E. H., Toepassing van een regelsysteem: de grammatika van het Frans (Applying a rule system: the grammar of French). Unpublished Ph. D. dissertation, University of Amsterdam (1985). Rapp.-Her Frame Boek 56, l-12 (1986). 9. Van der Linden E. H., L’analyse des protocoles. Rev. 87, 215-247 (1980). 10. Ericsson K. A. and Simon H. A., Verbal reports as data. Psychological 11. Breuker J. and Van der Linden E., Het oplossen van Franse grammatikaproblemen. (Solving French grammar problems). In Onderwijsleerprocessen (Edited by Beishuizen J., Hamaker C., Van Hout Wolters B. and Koster K. B.), pp. 113-123. Zwets & Zeitlinger, Lisse (1983).
Bntain.
Copyright
All rights reserved
DOES
c
0360- I31 S/93 $6.00 + 0.00 1993 Pergamon Press Ltd
FEEDBACK ENHANCE COMPUTER-ASSISTED LANGUAGE LEARNING? ELISABETH
University
of Amsterdam,
Spuistraat
VAN DER LINDEN 134, 1012 VB Amsterdam, The Netherlands
Abstract-Programs for computer-assisted language learning (CALL) are becoming more complex. On one hand, courseware designers develop programs of a more open-ended character, e.g. adventures, hypermedia courseware. On the other hand, courseware in the traditional drill and practice vein is more and more elaborate and offers extensive feedback to the learner. Developing this last type of courseware, in which feedback generally is based on an error analysis of a large group of learners, demands an enormous investment in time and energy. The question is whether this investment pays OR do learners indeed learn better by programs with a considerable amount of feedback? This question was addressed in a research project at the French Department of the University of Amsterdam. Students’ reactions to feedback were studied in two ways. First, log files were analyzed containing the series of tasks-responses-feedback of each student. In addition, think aloud protocols were analyzed. The results from the log files, which were confirmed by the analysis of the think aloud protocols, show that there was no overall successful strategy. While some students made optimal use of the feedback provided, others seemed to avoid using it in several ways.
INTRODUCTION In the last few years, computer-assisted language learning (CALL) has been expanding quickly but technology has been progressing even faster. There seem to be two different approaches in recent developments of CALL. On one hand, we find the traditional drill and practice programs such as the ones described by Higgins and Johns[l], on the other hand we find programs using more advanced technology, such as the one described by Viau and Lariv&e[2]. This last type of program generally makes use of authentic foreign language materials to enable learners to progress. Learners are offered considerable freedom in the way they work with the program. They can ask questions about grammatical aspects of the text, about words that they do not know, and about several other aspects of the text or the foreign language[2,3]. Programs in the more traditional vein oblige the learners to work within a rather strictly limited framework, asking them to produce precisely defined utterances, emphasizing thereby correctness of grammar or vocabulary[l]. Often, these drill and practice programs were associated with a behavioristic view of the learning process. As Swann[4] put it: “It [was] often naively assumed that what the student does is to learn grammar rules, memorize vocabulary and put the two together by means of exercises”. Nowadays, everybody knows that language learning does not work this way. “Grammar is neither necessary nor sufficient for second language learning: many people do learn a second language without any exposure to its grammatical description”[4]. Although this change in vision led to a more cognitive learning approach, it did not lead to the abandonment of drill and practice programs. Teachers and students continue to ask for them. But even these more traditional CALL programs became different. Their primary aim is no more to automatize language structures by drill exercises. They rather try to enhance learning by comprehension of these structures to “facilitate the work of the learning mechanism”[4], since retention can only take place when a language phenomenon is converted into a cognitive structure [S]. These programs offer complex feedback to the learners, guiding them to the correct answer through comprehension of the language structure involved. The steps in these programs are:
(1) Select task or question (2) Display question (3) (4) (5) (6)
Accept student’s response Match response against acceptable response Give feedback about response If response correct, go to 1 (next task); if incorrect, 61
let student
try again and go to 3.
62
ELISABETHVAN DER LINDEN
The fourth step, asking for an inventory of all acceptable answers, is time consuming but not problematical. The fifth step, construct adequate feedback, asks for theoretical as well as practical reflection on the role which feedback plays in the learning process. However, we do not know much of the real importance of feedback in CALL. Does it help learners to learn better, to understand or to retain more, or faster? It is this question that will be the subject of this paper.
LEARNING
FROM
FEEDBACK
According to cognitive learning models such as that of Rumelhart and Norman[$ learning can take place only after comprehension. Hence the emphasis on corrective feedback. In research on second language learning, little attention has been paid to the role of feedback, although it is clear that in the foreign language classroom, a considerable amount of feedback is provided to the learners. Research such as that of Cohen and Robins[6] and Long[7] does not show a convincing positive effect of feedback on second language learning. There is however a difference between traditional foreign language learning situations and computer-assisted learning. Whereas in traditional foreign language classroom situations, feedback is not very systematic nor immediate, in the CAI-context: -feedback is provided for every incorrect response; -it is given immediately; -self-correction by the learner is required [ 11. One can quite plausibly suggest that this makes feedback in CALL more effective than it is in the above described classroom situations. Theoretical as well as practical reasons make it important to understand the role of feedback in computer-assisted foreign language instruction. From a theoretical point of view, finding out how students go about learning from CALL may help to get more precise information on aspects of the learning process. From a practical point of view, this will help to design effective programs. If complex feedback is not as helpful as was thought, considerable money and time could be saved in constructing simpler CALL programs. To find out more about the role of feedback in CALL, a research program at the French Department of the University of Amsterdam, followed closely a group of first-year students working with a CALL program for French grammar.
THE
PROGRAM
First-year students at the French Department have to take an exam in French grammar at the end of term. To prepare for this exam, they can work with a computer-aided instructional program covering the most important parts of the subject matter for the exam. In this CA1 program, students are confronted with traditional grammar problem-solving tasks: the computer presents a sentence, and asks them to carry out a substitution or transformation task, such as in Fig. 1. After each provision of feedback, the student can try again to give the correct response. For each sentence, incorrect responses generally ranging between five and ten and acceptable responses are foreseen. For each of these incorrect responses specific feedback is provided. When a response is given that is not foreseen, the computer tells so to the student. Feedback concentrates always on only one error. If the student makes more than one error at a time, feedback is given for only one of them so as not to make things too complex for the student. In the next attempt, feedback for another error is given if still necessary. Whenever students give an incorrect response, they are invited to try again, using information from the feedback they have received. Students may try to produce the right answer as many times as they want. They can also ask for the right answer or go on to the next sentence whenever they want. Apart from the feedback which they automatically receive, students can ask for hints. This paper will concentrate on the exercises for the form and place in the sentence of the French personal pronouns. This part of French grammar confronts students with numerous problems: in
63
Language learning their performance, errors as well as avoidance strategies above described steps. A typical example of a possible Sentence: Response Feedback Response Feedback
are frequent [8]. The program follows the route through the program is:
Ne donnez plus de bonbons aux enfants. 1: Ne les en donnez pas. 1: Peut-on remplacer “aux enfants” par “les”? 2: Ne leur en donnez pas. 2: Oui, c’est correct.
PROCEDURE Two methods were used to study the strategies that students adopted when working with the program. First, log files of the work of 23 students were analysed to see in detail the routes they followed through the program, looking especially at the number of errors, the number of attempts and the effectiveness of the feedback. Second, since the first analysis offered mainly quantitative data that gave only a limited insight into the strategies that made students work in the way they did, a series of interviews were used to obtain think aloud protocols[9] of students working with the program. Analysis of these interviews found out more about what students think they do when they are using the CALL program. Although there is discussion about the usefulness and reliability of introspection data, the view of Ericsson and Simon [ lo] was taken that introspection is a reliable tool for research into internal cognitive processes.
RESULTS
Log files Studying the log files was expected to determine mainly the “optimal” strategy . That is, students were expected to work through the program as described above: starting by trying to respond, studying the feedback they received when their responses were incorrect, trying then to give the correct response, subsequently repeating this cycle if the responses were still incorrect and so on. This is not what was always found. The log files registered every step the students took with the aid of the computer, and on that basis the routes were divided into four: (1) (2) (3) (4)
The The The The
go on till you get it right strategy; drill strategy; once is enough strategy; browsing strategy.
1 Remplacez dans la phrase suivante les parties 1 par des pronoms personnels, y ou en. 1 Mettez ces formes a la bonne place. Maman 1 Maman
ne laisse pas les enfants
jouer dans
. . . ne
. . . jouer.
. . . laisse
. . . pas
le
soulign&es
jardin.
(space for feedback)
1 Fl Help 1 PgUp one page back
F2 Hint PgDn one page Fig.I.Example of an exercise.
F10 Stop further
1 !
64
ELISABETH VAN DER LINDEN
The first of these strategies is the “optimal” strategy that we expected-or hoped-to find with all the students. This was not the case. Only 12 students showed a clear preference for this strategy, while it frequently appeared in the log files of five students. It was very rare or non-existent in the log files of the remaining six students. The 12 students who preferred this way of working followed the expected route, going through the sequence of response-feedback-retrial sometimes up to eight or nine times. Generally however, two trials were enough to produce the correct response. The second strategy was called drill-like because it reminds one of the behavior in classical drill programs, where the only feedback that is provided concerns the correctness of the response. While second tries are possible, no feedback about the type of error is given in those programs. In this case, students who worked this way did not try to correct themselves when their response had been wrong: they just asked for the correct response and then went on to the next sentence. This way of proceeding through the program was used regularly by most students. The 12 optimal workers chose to use it occasionally; six students preferred this strategy and used it most of the time. Three students used it regularly. Students working this way seemed not to be interested in the detailed feedback they received; they probably just wanted to try once, and when wrong, to find out the correct response. As was pointed out, students might feel that this way of working is more effective because it is faster; at the same time, however, it requires less active production from the student. which suggests that it might lead to less learning. The third strategy was labelled the “once is enough” strategy: students working this way do not correct themselves when they are wrong or even look up the correct response: they just respond once, and irrespective of the correctness of this response. they go on immediately to the next sentence. This behavior is puzzling. It suggests that the students concerned, even if they find that their responses were incorrect, do not try to find out why it was incorrect or even what would have been the correct solution. Perhaps this is an example of test behavior: the students just want to know how many of their first responses are correct or incorrect in order to find out what level of proficiency they have attained. The last strategy, browsing, implies that students just look at the sentences of the exercise: the) go directly from one sentence to the next without trying to give responses. Sometimes, they then go back to the beginning and start really working through the program, sometimes they just quit and start working on a different exercise. In the first case, we are probably dealing with students who first want to have a look at what it is all about before they really start working. In the second case, perhaps the students concerned just want to do the exercises orally. It remains however surprising that they should not even take the time to look up the correct response. Interviews and thinking-aloud protocols In order to discover what brought about these different strategies, we interviewed six students and listened to them as they carried out the program while “thinking aloud”. This enabled us to relate some of the strategies described above to individual learning styles. Earlier research [&I I], had showed that idiosyncratic approaches of this type of grammar problem solving exercises explain at least as much of the achieved learning results as other factors like intelligence or general proficiency. In the analysis of the log files we found that indeed individual differences play a role in the choice of students’ strategies in CALL as well. In the interviews, these impressions were confirmed. On one side, there are the “zealous” students who use the optimal strategy until the bitter end. Other students however are impatient to get through the program and follow the drill like strategy. They sometimes pretend that this strategy is more useful to them. Reading all the feedback would confuse them. It is possible that indeed they are right: we found that the students who use up to eight or ten trials in order to find the correct response, sometimes make the same mistakes in more than one of their responses, showing that they have not retained earlier feedback received in the same sequence of responses. This suggests that the usefulness of complex feedback can perhaps not be generalized to all students: for some, it may be more effective than for others. Apart from this finding, there are a series of minor findings that seem to account for certain behavior patterns in the students, and these were confirmed in the interviews. Jn the first place. many problems arose from editing and type writing problems. As a matter of fact, we expected this kind of problem, not all students being familiar with computer aided writing. We expected it
Language
learning
65
however to be only a minor problem because the writing tasks did not differ much from those of a normal typewriting machine. Nevertheless, several students continued to get nervous when confronted with the simple tasks that had to be carried out on the computer. For this reason, some of them missed up to one-third of the sentences. Secondly, we found that, the longer the feedback is, the less it will be read. Feedback of more than three lines (existing but rare in the program) was hardly ever read until the end. Lengthy explanation does not therefore seem to work effectively. Consequently, we decide to shorten the feedback where necessary and to facilitate access to the schemes of grammatical rules that were equally present in the program, so that students could look them up separately when they wanted to see the whole rule system pertaining to an exercise. Thirdly, we discovered that, as a rule, students did not start working through the program by reading its introductory part, where the grammar rules, necessary to do the exercise, were explained. Generally, they skipped this part of the programs and it was only when they were confronted with problems they could not solve that some of the students went back to read the first section. This was another reason why we decided to elaborate on the introductory rule section and to give it the status of a background grammar to be consulted whenever necessary.
CONCLUSIONS
In conclusion, we see that students had some strategies in common, but for the rest behaved in idiosyncratic ways. Two main working strategies could be distinguished, one which we called the “optimal” one, where substantive amounts of complex feedback were consulted by the students, and the “drill-like” one, where students seemed to avoid reading feedback. We saw that the drill-like strategy, which we initially considered to be less effective, is possibly an equally effective strategy for some of the students, perhaps especially the less intelligent ones. Frequent consultation of sophisticated feedback seemed, for them, to be confusing. A practical conclusion arising from our work is that feedback, in order to be consulted, has to be concise and precise. Long feedback (exceeding three lines) is not read and for that reason is not useful. Another aspect of the use of feedback is of course its effect on the final learning results of the type of tasks we described here. In other words, what are the short- and long-term retention effects of the use or non-use of feedback. It is not unlikely that the drill-like strategy we have described here seems at first sight an effective learning strategy for some students, but that the “optimal” strategy leads to better retention. This is the question that we will investigate next. REFERENCES 1. Higgins J. and Johns T., Computers in Language Learning. Collins, London (1984). 2. Viau R. and Larivte J., Learning tools with hypertext: an experiment. Computers Educ. 20, 11-17 (1993). towards a global learning environment. In New Media in rhe Humanities (Edited 3. Rtzeau J., Minim1 and Framework: by van Loon H. and Janssen J.), pp. 86-93. Faculty of Arts, Amsterdam (1991). 4. Swann P., Computer assisted language learning for English as a foreign language. Computers Educ. 19, 251-266 (1992). D. and Norman D. A., Explorafions in Cognition. Freeman, San Francisco (1975). 5. Rumelhart 6. Cohen A. and Robins M., Errors, interaction and correction: a study of native-nonnative conversations. TESOL-Q 16, 537-547 (1976). I. Long M., Does second language instruction make a difference? TESOL-Q 17, 359-382 (1983). 8. Van der Linden E. H., Toepassing van een regelsysteem: de grammatika van het Frans (Applying a rule system: the grammar of French). Unpublished Ph. D. dissertation, University of Amsterdam (1985). Rapp.-Her Frame Boek 56, l-12 (1986). 9. Van der Linden E. H., L’analyse des protocoles. Rev. 87, 215-247 (1980). 10. Ericsson K. A. and Simon H. A., Verbal reports as data. Psychological 11. Breuker J. and Van der Linden E., Het oplossen van Franse grammatikaproblemen. (Solving French grammar problems). In Onderwijsleerprocessen (Edited by Beishuizen J., Hamaker C., Van Hout Wolters B. and Koster K. B.), pp. 113-123. Zwets & Zeitlinger, Lisse (1983).