The effects of trained peer response on ESL students' revision types and writing quality

JOURNAL

OF SECOND LANGUAGE

WRITING,

8(3), 215-241

(1999)

The Effects of Trained Peer Response on ESL Students’ Revision Types and Writing Quality E. CATHRINE University

BERG

of Pennsylvania

Since the late 198Os,peer response to writing has gained increasing attention in the English as Second Language (ESL) field. Whereas affective benefits have been reported in the literature, little is known about the effects of peer response on ESL students’ revision and writing outcomes. This study investigates these effects and also considers an often-cited suggestion for successful peer response, that is, training students to effectively participate in the peer response activity. The principal question addressed by the study is whether trained peer response shapes ESL students’ revision types and writing quality. Effects of trained peer response were investigated through a comparison of 46 ESL students divided into two groups, one trained in how to participate in peer response to writing and the other not trained. Revision types were identified based on a taxonomy that discriminates between two types of changes: those that affect text meaning and those that do not (Faigley & Witte, 198 1). Writing quality was determined by a holistic rating procedure of first versus revised drafts. Results of the investigation indicate that trained peer response positively affected ESL students’ revision types and quality of texts.

INTRODUCTION

Peer response as part of the process approach to teaching writing (e.g., Flower & Hayes, 1981) has gained increasing attention in English as Second Language (ESL) since the late 1980s. A number of studies have been conducted on peer response in the ESL classroom (e.g., Hafernik, 1983; Mangelsdorf & Schlumberger, 1992; Mendonca & Johnson, 1994; Mittan, 1989; Moore, 1986; Nelson & Murphy, 1992, 1993; Stanley, 1992; Witbeck, 1976; Zhang, 1995). Several of these studies describe classroom contexts and student roles, recommend strategies for successful peer response, or report on student attitudes and affective benefits. For example, it was noted in a questionnaire-based study by Zhang (1995)

Direct all correspondence to: E. Cathrine Berg, University tion Division, 3700 Walnut Street, Philadelphia, Pennsylvania upenn.edu

of Pennsylvania, Language in Educa19104-6216; e-mail: bergc@dolphin.

215

216

BERG

that, in terms of affective benefits, ESL writers overwhelmingly preferred teacher feedback to peer feedback. In light of these results, Zhang questioned the benefits of peer response in the ESL classroom and pointed out that no research to date “shows how the anticipated benefits are adequate to compensate for the attending affective disadvantage” (p. 219). Studies that investigate potential ESL peer response benefits other than affective ones are indeed urgently needed. Another very important, yet largely ignored, aspect of peer response to writing and its implementation in the ESL classroom concerns the role of training, that is, the preparation of students for participation in the peer response activity. Responding to writing is not a skill with which most students, ESL or not, have had extensive experience. It is therefore unrealistic to assume that they will be able to effectively read and respond to someone else’s writing, constructively react to a response to their own writing from a peer, and, based on the peer response activity, successfully revise their texts. If students are to be expected to skillfully participate in peer response and perform appropriate revisions of their texts, it appears reasonable to believe that they need to be given the opportunity to learn how. To fill the gap in knowledge about the effects of peer response on writing and the role that instruction plays in determining such effects, this study investigates the influences of trained peer response on revision types and writing outcomes. It does so by comparing two groups, one trained in how to participate in peer response to writing and the other not trained in this method. Specifically, revision types by students in the trained versus untrained group and level of improvement in trained versus untrained students’ first and second drafts are compared.

BACKGROUND

Revision of Text Meaning A fairly large number of Ll composition studies on revision exist; several of them compare revision strategies and their effects on text between experienced and inexperienced writers (e.g., Bridwell, 1980; Faigley & Witte, 1981; Perl, 1979; Sommers, 1980). To reveal types of revision, these researchers employed a number of different taxonomies. For example, Bridwell (1980) and Sommers (1980) identified revisions on a linguistic level (e.g., words, clauses, sentences) and on a level of operation (e.g., addition, deletion, substitution). Faigley and Witte (198 1) were interested in examining revisions that either affected meaning (i.e., “meaning changes”) or did not affect meaning (i.e., “surface changes”). In their study, it was found that experienced writers more frequently made meaning changes, whereas inexperienced writers mostly made surface revisions. It would seem plausible that revision types of experienced writers would be more likely to result in improved writing than types common to novice writers; in fact, such

THE EFFECTS

OF TRAINED

PEER RESPONSE

217

findings have been reported. It seems that when high-school-age and more experienced writers revise, their compositions show improvement (Ash, 1983; Bamberg, 1978; Bracewell et al., 1978; Bridwell, 1980). Additionally, among older writers, higher quality writing is related to the use of a wide variety of revision types (Bridwell, 1980), as well as revisions of more than one word (Hawisher, 1987; Maynor, 1982). On the other hand, revision may not improve writing for younger, learning disabled, or less competent college writers (Bracewell et al., 1978; MacArthur & Stoddard, 1990; Perl, 1978); in fact, revisions may actually lower the quality of drafts (Bracewell et al., 1978). Whereas research on ESL revision is scarce relative to Ll studies, ESL revision has been investigated and some results corroborate first-language findings. For example, one of the issues highlighted by Cumming and So (1996) as particularly common in ESL contexts is students’ “ tendency toward error correction in the revision or editing of texts” (p. 200). Zamel (1983), in her study on skilled and unskilled ESL writers, found that skilled writers were more likely to focus their revisions on larger rhetorical-level aspects of their texts, such as reordering paragraphs, than unskilled writers were. A crucial variation in strategies of experienced versus inexperienced ESL writers is thus similar to that of experienced versus inexperienced native speaker (NS) writers. That variation concerns the writers’ tendencies to focus revision either on meaning of text or on aspects that do not concern meaning.

Incongruities

and Viable Text Alternatives

as Causes of Revision

As Faigley and Witte (1981) pointed out, many revision studies fail to explore the important question of what causes writers to revise. However, one attempt was made by Sommers (1980). Using a case study approach, she compared revision strategies of 20 college freshmen with those of 20 experienced adult writers. She found that “incongruities between intention and execution” (p. 385) played an important role in causing writers to revise. Such incongruities can perhaps be best understood in relation to intention and comprehension. In fact, the socialinteractionist perspective on writing offers insights about revision and the mismatch between a writer’s intention and what is actually understood. From the social-interactionist perspective on writing, the focus is on the relationship between writer and reader. For example, Nystrand (1989) explained that a “misconstraint” results when there is a mismatch between the writer’s message and the reader’s comprehension. That is, the concept of misconstraint conveys a realization that what is written will not be accurately understood by the intended reader. Fitzgerald (1992) further explained that, from a social-interactionist view, the “goal of writers is to achieve reciprocity, or a state of convergence with readers. They monitor their texts for the extent to which reciprocity seems threatened. Revision is initiated if the projected state of convergence may be threatened’ (p.

218

BERG

40). Thus, from a social-interactionist perspective, a cause of revision is seen to be the discrepancy between intended and understood meaning. Nold (198 1) makes an additional point about the causes of revision. She explains that although a perceived mismatch can cause revision, it does not mean that once one is detected, the writer will revise. Furthermore, the issue is made more complex by the fact that no guarantee exists that a revision results in a less dissonant version than the original. Thus, it can be concluded that, in order to revise successfully, writers not only need to realize misconstraints, they also need the ability to produce a clearer alternative to the current text. Studies on experienced versus inexperienced writers’ revision strategies (e.g., Sommers, 1980; Zamel, 1983) indicate that experienced writers are able to revise successfully by first sensing a mismatch in meaning and then having a viable solution to the problem of incongruity. These findings are paralleled by discoveries about inexperienced writers’ revision strategies. Such discoveries suggest that the lack of skills in detecting incongruities and finding appropriate text alternatives in part prevents inexperienced writers from attempting revision of their texts (Sommers, 1980; Nold, 1981). It can be assumed, then, that if a goal for an inexperienced writer is to develop better revision strategies, this writer must develop skills in detecting incongruities and in formulating viable text alternatives. Furthermore, it can be assumed that an inexperienced writer’s revision strategies would benefit from two types of aid, one in detecting misconstraints and the other in discovering effective alternatives to unsatisfactory text. Thus, one might hypothesize that assistance in detecting incongruities between intention and understood text, especially on a text rather than on a word or sentence level, could be a first step towards learning more sophisticated revision strategies for inexperienced writers. Moreover, it can be hypothesized that peer response to writing is a potential way of helping inexperienced writers see the mismatches in their writing, especially if peer response is conducted with the explicit purpose of helping a peer discover if his or her text is clear and conveys intended meaning to another reader. Indeed, the currently emerging social-interactionist views of writing and revision might be seen to support the idea of peer response being useful in helping students discover incongruities and find viable text alternatives. Training in Peer Response The consensus among researchers of NS peer response to writing seems to be that carefully planned peer response sessions, together with appropriately prepared students, will result in better student writing (e.g., Benesch, 1984; George, 1984; Huff & Kline, 1987; Nystrand, 1984; Reither & Vipond, 1989). Although widely used and accepted among composition teachers, peer response is by no means always beneficial to all students under all circumstances. George (1984) found in her investigation of over 100 college-level peer response sessions that

THE EFFECTS OF TRAINED PEER RESPONSE

219

inexperienced writers and readers often get caught up in the subject of an essay and spend their time talking about the ideas in isolation, never in relation to the actual writing. Huff and Kline (1987) also noted that peers responding to each others’ writing can be problematic. They suggested that students’ verbal feedback can be “blatantly useless, uninformed, and often thoroughly unconstructive” (p. 150) because verbal responses do not allow students to contemplate their reactions and word them appropriately. Nystrand (1984) cautioned that peer response takes careful planning on the teacher’s part, and that students must be shown how to respond to writing in a peer context. Huff and Kline (1987) stressed the importance of providing students with appropriate peer response skills, such as giving and receiving criticism, articulating ideas about positive and negative qualities of writing, and recognizing different stages of the drafting process. Benesch (1984) suggested that before peer work on writing can begin, teachers need to ascertain students’ writing proficiency level, feedback skills, and collaborative work experience to be able to determine appropriate learning goals and effective peer response strategies. She also suggested that teachers provide models of constructive feedback in their responses to students’ writing, and that they also respond to the student feedback itself. Baker et al. (1989) asserted that peer response should involve reading and writing as the primary activities. As a matter of fact, a commonly used peer response “tool” involves those two activities. This tool is the peer response sheet (see Appendix A for an example). This sheet allows students time to contemplate and formulate appropriate feedback in writing before getting involved in a discussion with the author. The sheet usually consists of a number of specific questions that students should answer about their peer’s essay before discussing it, or the sheet might present a list of areas that should be investigated by the reader before a response is made to the peer’s writing. Whereas various researchers interested in ESL peer response have focused their attention on different aspects, such as recommendations for successful implementation, descriptions of their own methods, or analysis of the benefits for students, they have in common the belief that it is important to appropriately prepare students to participate in the peer response activity (e.g., Hafernik, 1983; Mittan, 1989; Moore, 1986; Nelson & Murphy. 1993; Stanley, 1992; Witbeck, 1976). To illustrate, Nelson and Murphy (1993) conducted a study to find out whether low-proficiency-level ESL students were capable of effectively participating in and benefiting from peer response. They concluded that these students can indeed successfully be involved in peer response and that this activity is helpful to their revision strategies. They did, however, offer three recommendations to consider for peer response to work well with less proficient ESL students. First, they suggested that students should learn the appropriate social skills needed for participation in peer collaboration. Second, they emphasized the importance of training students in the specific skill of responding to each others’ pa-

220

BERG

pers. Finally, they mentioned the importance of teaching actual writing skills to provide students with concepts with which they can discuss writing. Nelson and Murphy considered these appropriate social, response, and writing skills to be best taught through exemplification and modeling by the teacher. According to Moore (1986), peer response is useful in teaching students “important skills that are critical to effective writing . . .” (p. 23). She stressed the necessity of training students to become peer responders and outlined her method of preparing ESL students for peer response. In this method, Moore begins with an explanation of what peer response is and the following four-part pep talk: I) you are capable of critiquing each others’ essays; 2) it’s your responsibility to give and take criticism well, remembering that the writers are always ultimately responsible for their own writing, not the evaluators; 3) don’t forget to give positive comments; 4) critiquing others’ work is useful for you, to-you’ll learn skills that will enable you to better evaluate your own work (p. 23). Moore then involves the entire class in an evaluation of an anonymous essay. During this class activity, she provides written guidelines that help students focus on content and organization in their evaluations. Once students feel comfortable with the guidelines and the evaluation process, they continue the activity in groups of four. Moore reported positive feedback from students about the peer response activities. Chenoweth (1987) suggested that because unskilled ESL writers differ in their revision strategies from skilled ESL writers in that they see revision as error correction, they need to be taught “rewriting.” That is, they need to be taught how to attend to the more global issues of their writing when revising, as skilled writers do. Stanley (1992) investigated the effects of training on peer response interactions and their influence on revision. She found that the preparation resulted in “a greater level of student engagement in the task of evaluation, more productive communication about writing, and clearer guidelines for the revision of drafts” (p. 217). She also found that trained students made more revisions that directly resulted from peer response. Based on the literature to date, it seems that three important conclusions can be drawn about revision, peer response, and training. First of all, successful revision involves a focus on issues of meaning and rhetorical aspects of text, an ability to detect mismatches between intended and understood meaning, and a supply of viable text alternatives. Secondly, peer response can positively affect revision in those three areas. That is, a peer can help the writer focus on meaning of ideas and rhetorical aspects by responding to the larger issues of clarity of ideas, organization, and development. Since a peer’s access to meaning is limited to the written text, his or her ability to discover incongruities is also at least different, if not better, than the writer’s In addition, a peer can offer suggestions for alternative ways of making meaning clear in the writer’s text. Finally, the likelihood of the peer response activity being helpful to a writer in focusing on text meaning,

THE EFFECTS

OF TRAINED

PEER RESPONSE

221

detecting r&constraints, and supplying viable text alternatives would seem to increase when the peers are appropriately trained in how to participate in peer response to writing.

THE PRESENT

STUDY

Motivated by previous research in the areas of peer response and revision studies, existing gaps in this literature, and personal observations made in the ESL writing classroom, the overarching question addressed by this study is whether trained peer response shapes revision types and writing outcomes of ESL student texts. Specifically, hypotheses are that (a) trained peer response will generate a greater number of meaning changes in revised student drafts, (b) trained peer response will yield higher quality scores in revised student writing, and (c) relative effects of trained peer response on writing quality outcomes will not be influenced by a difference in writing proficiency. The present study constitutes the first part of a two-part study. Whereas the second part describes peer response negotiations and examines relationships between peer talk and revisions, the first part, discussed here, focuses on the effects of training on revision and writing quality and, thus, reports only the outcomes. Setting and Participants The setting for this investigation consisted of four writing classrooms, two levels 3 (i.e., an intermediate level corresponding to Test of English as a Foreign Language [TOEFL] scores around 375) and two levels 4 (i.e., an intermediatehigh level corresponding to TOEFL scores around 425), at a six-level (beginning through advanced) university-based intensive English program (IEP) located in a large city on the U.S. East coast. The study took place during two separate 1 lweek terms, each term involving one level 3 and one level 4 class. The IEP curriculum at the research site complies with the American Association of Intensive English Programs’ (AAIEP) qualifications as stated in its Bylaws, Article 111, Section 2, and thus involved the required 18 hours/week of classroom instruction. Of those 18 hours, 6 hours and 20 minutes were spent on written skills and 6 hours and 20 minutes on spoken skills. Students chose among a number of elective classes to make up the remaining 5 hours and 20 minutes. The written skills classes in the study met for 80 minutes every morning five days a week. The content of both the intermediate- and intermediate-high-level courses was academic writing, with level 3 focusing on the academic paragraph and level 4 on the academic essay. Students wrote five formal out-of-class compositions in each course, with each assignment involving invention activities, drafting, peer response, revision, and editing. The same textbook, Refining Composition Skills: Rhetoric and Grammarfor ESL Students (1990) by R. L. Smalley and M. K. Re-

222

BERG

utten, was used for both levels; level 3 used the first half of the book, which describes the academic paragraph, and level 4 used the second half of the book, which concerns the academic essay. The participants in this study were placed in their respective levels based on the results of a modified Test of Written English (TWE) placement test. That is, students were given 30 minutes to write a response to a prompt. Their responses were scored on a 0 to 6 scale using pluses and minuses. Students in this study who had received a 3 or 3+ on their placement essay were placed into level 3 classes and students who had received a 4or 4 were placed into level 4 classes. In this study, I was the teacher for all four classes, using an approach to teaching writing that can best be described as a “product-oriented process approach.” That is, while students were guided through explorations of their individual writing processes, they examined models and received specific guidelines for the structure of the final product. Participants’ were 46 ESL students (22 females and 24 males) from 19 different countries. No participant had been in the United States longer than three months, and none had had prior experience with peer response to writing. Educational backgrounds ranged from completion of high school to completion of a graduate degree. Ages ranged from 17 to 56. The distribution of participant variables, such as age, gender, etc., was fairly typical for an IEP writing course at the university where the study was conducted. Whereas attempts were made to recognize and account for cross-cultural variables, interaction styles and other crosscultural issues, as discussed by, for example, Carson and Nelson (1994, 1996), cannot be dismissed as unimportant or completely controlled for in this quasiexperimental study. Procedure Approximately equal numbers of students were enrolled in each of the four intact classes. The classes were assigned to the two groups as follows. During the first term, the level 3 students were trained to participate in peer response, whereas the level 4 students were not. During the second term, the level 4 students were trained to participate in peer response, whereas the level 3 students were not. Thus, the trained group was composed of one level 3 and one level 4 class with a total of 24 students (12 females and 12 males). The untrained group also consisted of one level 3 and one level 4 class and had a total of 22 students (10 females and 12 males). Writing Assignment First drafts (i.e., pre-peer response drafts) and second drafts (i.e., post-peer response drafts) for the first writing assignment* of the term were collected and examined for revisions, or lack thereof. The pre-peer response draft was a first draft written as homework on the topic of a memorable personal experience for levels

THE EFFECTS OF TRAINED PEER RESPONSE

223

3 and on the topic of a personal opinion for levels 4. The post-peer response draft was a revision of the first draft, also completed as homework. First drafts were due three days after the training program for the experimental group and the replacement activities for the control group were completed. Second drafts were due two days after the peer response activity had taken place. Level 3 students had been instructed to write one paragraph and to have an explicit topic sentence. Prior to drafting, students had employed the invention techniques of brainstorming and listing to discover the memorable experience about which to write. Class lectures and discussions presented the importance of using chronological order and specific details in the paragraphs. Level 4 students had been instructed to write an essay following the English academic format of an introduction paragraph, several body paragraphs, and a concluding paragraph. The thesis statement and topic sentences had been discussed in class, and students had been told to include a thesis statement at the end of the introduction and explicit topic sentences first in the body paragraphs. The importance of using detailed and concrete support had been stressed in class. To discover topics, level 4 students participated in group brainstorming sessions on controversial issues as well as individual invention techniques of brainstorming and listing of issues about which they felt strongly. Students in all classes had been told that the intended audience was the other students in the class. Students received no teacher comments on the first draft and, as confirmed by interviews with students, their peer was the only person to give feedback on that draft. Written permission to use student materials was obtained from students prior to the beginning of data collection. Peer Response Training The particular training program used for this study was developed over the course of several years. The training consisted of 11 steps, ranging in time from 5 to 45 minutes each. (For a brief outline of the training, see Appendix B. For a detailed account, see Berg, 1999.) In addition to being based on classroom experience, the training is also based on recommendations in the peer response literature (e.g., George, 1984; Hafemik, 1983; Hardaway, 1975; Huff & Kline. 1987; Moore, 1986; Nelson & Murphy, 1993; Nystrand, 1984) and on an awareness of problems cited in the ESL literature on writing and peer response (e.g., Connor & Asenavage, 1994; Leki, 1990; Mendonca & Johnson, 1994; Stanley, 1992). One level 3 and one level 4 class were trained in how to participate in peer response to writing. The preparation was designed to address a number of specific areas and provide students with certain response skills. These skills concerned the language used to respond to writing (e.g., asking questions, using specific words rather than making vague and general statements, and stating ideas as opinion, not fact) and the foci of discussion (e.g., a focus on larger-level aspects

224

BERG

that concern the meaning of the text as opposed to smaller-level aspects that do not concern the meaning of the text). The untrained levels 3 and 4 participated in activities similar to those in the training in terms of opportunities for students to work in pairs, participate in class discussions, and practice writing. (See Appendix B for an outline of activities for the untrained group.) Both groups learned about English academic writing and the writing process, received specific instructions about their first writing assignment, and participated in invention activities to discover topics for their paragraphs (level 3) and essays (level 4). However, the untrained group was provided with no particular response skills and received no instruction in how to participate in peer response. Instead, this group received minimal instructions in the logistics of the peer response activity and, in place of the training, students read, discussed, and wrote about a short story.”

Data Coding, Scoring Procedures, and Analysis To determine the number of meaning changes that were made by students in the revision of their drafts, all meaning changes in the second drafts were marked and counted. Whether a revision constituted a change in meaning or not was based on Faigley and Witte (1981). They define meaning changes as involving “the adding of new content or the deletion of existing content” (p. 402). Examples of meaning changes in a second draft are given in Figure 1. The student has made three meaning changes (highlighted in bold type in the paragraph). The first one involves the initial sentence, which in the first draft makes a general statement about military duty (i.e., “Every body must to be soldier by our law from my country”), but in the second draft states the main idea of the paragraph (i.e., “I have terrible experience for one month during I was soldier”). This constitutes a meaning change according to Faigley and Witte (1981) since it adds new content to the text. The second meaning change supplies an explanation of some statements that exist in the first draft. To the first draft version of, “I have no skill and I can’t do evry thing. Sometime very angry and scream to me. I don’t know what I must to do,” the student has added, “They think I know many thing because I am student of Mechanical Engineering Department. But I do not know about turbine department. I never drive the ship in my life” in the second draft to explain why he has no skill. The final meaning change in this example concerns the concluding sentence in the first draft (i.e., “I think I cannot be soldier in all of my life”), which has been deleted and replaced by, “Then I study and learn many things. But one month after I can take test very good. This test I think can I become from boy to man” in the second draft, a change which is partially new content and partially content that was moved from the middle part in the first draft to the end in the second draft. The process of determining how many meaning changes, if any, students in this study made in their second drafts involved a blind comparison of their first

THE EFFECTS OF TRAINED

PEER RESPONSE

22s

Draft 1 The TERRIBLE experience

my life

Every body must to be soldier by our law from my country. I was soldier too. I think was very terrible experience.

I have many test and very difficult test. Every day almost I must to take

test. This test I think can I become from boy to man. I have terrible experience one month I was so tired. department. thing.

I must learn many skill I never know.

I work vety hard.

I sleep 2 hours in the night.

in that.

In during

I must work in turbine

I have no skill and I can’t do evry

Sometime very angry and scream to me. I don’t know what I must to do. I think I

cannot be soldier in all of my life. THE END Draft 2 The TERRIBLE

experience

in navy

I have terrible experience soldier by our law from my country

for one month during I was soldier.

Every body must to be

I was soldier too. I think was very terrible experience.

I have

many test and very difficult test. Every day almost I must to take test. They think I know many thing because I am student of Mechanical about turbine department. evry thing.

Department.

But I do not know

I never drive the ship in my life. I have no skill and I can’t do

Sometime they very angry and scream to me. I don’t know what I must to do.1 try to

very hard in this test. I study in the nightIn skill I never know. night.

Engineering

during one month I was so tired.

I must work in turbine department.

Then I study and learn many things.

I work very hard.

I must learn many

I sleep 2 hours in the

But one month after I can take test very good.

This test I think can I become from boy to man. THE END

Figure 1. Drafts 1 (first) and 2 (revised) of a level 3 student’s paragraph memorable experience. Changes are highlighted in bold print.

about a

226

BERG

and second drafts. Each student’s first and second drafts were put side by side and compared sentence by sentence. Every instance of a difference between the first and second drafts was marked. Each difference was then judged as to whether it constituted a change in meaning or not based on Faigley and Witte (1981). No further judgments were made. A meaning change, regardless of number of words, constituted the unit of analysis. The analysis of the 46 students’ first and second drafts (i.e., a total of 92 drafts) involved making blind judgments about meaning versus non-meaning changes. A second rater independently marked approximately 25% of the drafts (i.e., a total of 24 drafts consisting of the first and second drafts from 12 students), confirming 24 of the 2.5 meaning changes determined to exist by the first rater. Besides that discrepancy in judgment, the second rater also judged an additional change as a “possible” meaning change. Thus, consistent rater agreement was achieved. To determine if peer response training influenced student writing quality, a difference score (i.e., the second draft score minus the first draft score) was calculated for each student. Quality of revisions was measured by the degree of difference between the two scores using TWE-based scoring criteria (Educational Testing Services, 1996, p. 23). The TWE testing instrument is important in several ways. First, it is a measurement of overall writing quality; readers use a holistic approach to score writing. This is important in relation to the purpose of revision, which entails a “reconceptualization” of writing (Nystrand, 1984) so that changes result not just in minor grammatical corrections but in an overall clarification of ideas and improvement of text. A measure of whether each instance of revision resulted in an improvement or not was not done since that is neither what is important in revision nor the goal or purpose of revision. For example, there was no interest in how adding a single instance of an appropriate regular plural suffix resulted in a more grammatical sentence. Rather, the focus was on whether the writing as a whole, as a unit of discourse, was improved. Another reason for using the TWE as a way to measure improvement is that it does take sentence-level aspects into consideration in its holistic approach to evaluation. Furthermore, the TWE is specifically designed to evaluate writing by academically-oriented ESL students; thus, it is appropriate for the data in this study. For instance, at the IEP where the present study was conducted, placement testing involves a modified TWE; that is, scores are modified by a plus and minus system to better differentiate student writing ability. Prior to the present study, a small pilot study was conducted in which two trained TWE readers read 10 intermediate-level ESL students’ first drafts and their revised versions and determined that the TWE with the plus and minus system was a precise enough testing instrument to measure writing improvement from a first to a revised draft. In the present study, scoring procedures of students’ first and second drafts were as follows. First and second drafts were read blindly by two trained raters.

THE EFFECTS OF TRAINED PEER RESPONSE

227

Unlike essays that are assigned the Educational Testing Services (ETS) official TWE scores (i.e., 0, 1, 2, 3, 4, 5, 6), drafts for this study were assigned the same number scores but with the possible added feature of a plus or minus to more precisely differentiate writing quality. Just as official TWE scores get averaged between the two raters’ scores, the same general principle was applied to draft scores in this study. In order to apply this principle, the +/- scores were assigned integers ranging from 1 to 19 (i.e., 0 = I, 1- = 2, I = 3, I + = 4, 2- = 5. 2 = 6,2-t = 7,3= 8,3 = 9,3+ = 10,4- = 11,4 = 12,4+ = 13,5- = 13. 5 = 1.5, 5+ = 16, 6- = 17, 6 = 18, 6+ = 19). These transformed scores were then used to determine agreement between readers and to calculate averages. For this study, the averages between the two readers’ scores were calculated when scores were the same or separated by fewer than four points on the transformed scores scale. A discrepancy was considered to exist when scores were more than three points apart (i.e., an essay with scores of a 2- ]= transformed S] and a 3 [= transformed 91 is four points apart, and thus a third reader scores the essay). An average was then calculated based on the third reader’s score and the other two scores, or the first or second reader’s score closest to the third readers. To illustrate, if the essay with a score of a 2- (= transformed 5) and a 3 (= transformed 9) is read by a third reader who assigns it a 2+ (= transformed 7) the three scores are averaged. If the third reader’s score is a 2 (= transformed 6), this score gets averaged with the first reader’s score of a 2- (= transformed 5) since it is possible to determine that this score is closer to the first reader’s score than the second reader’s 3 (= transformed 9). Rater agreement” procedures resulted in 6.5% of first drafts and 8.7% of the second drafts going to a third reader. In summary, methods of coding and scoring student drafts to measure quality of revision were as follows: a TWE-based rating scale and scoring procedures were used to score first and second drafts. The quality of revisions was measured by the degree of difference between the two scores. Results This study produced three major findings. First, the hypothesis that trained peer response would generate a greater number of meaning changes in the revised drafts was supported by the results of this study. As shown in Table 1, a two-way rank sum test5 for the frequency of meaning changes in students’ revised drafts revealed statistically significant effects for both training status and level of proficiency given the common writing assignment.6 The two-way rank sum test was a two-way extension of the one-way Kruskal-Wallis H test, allowing for a test of the interaction effect.’ In brief, interaction effect refers to the possibility that the combination of factors is more important than any one factor alone. If, for example, the training factor is only significant in combination with one of the proficiency levels, then the interaction overrides the main effect. That is, the main effect of training cannot be interpreted independently, but needs to

228

BERG

TABLE 1 Summary Statistics of Mean Number of Meaning Changes in Revised Drafts According to Training Status and Level of Proficiency Group

Means

Untrained (N = 22)

1.14

(1.08) Level 3 (N = 12)

0.67 (1.98) I .70 (0.95) 4.2 1 (2.02) 4.17 (2.62) 4.25 (I .29)

Level 4 (N = IO) Trained (N = 24) Level 3 (N = 12) Level 4 (N = 12)

Two-way

Rank Sum Test According

Source

ss

Between groups Training status Proficiency level Interaction effect Within groups Total 4 Standard deviations hp-values

4550.50 356.80 45.90 2783.87 7905.00

to Training Status and Level of Proficiency

df 1

I 1 42 45

MS

F

p-valueb

4550.50 356.80 45.90 66.28 175.67

68.65 5.38 0.69

0.001 0.02 0.41

are given below the means in parentheses.

are significant

at alpha = 0.05.

be considered in relation to the “interference” from the interaction with other variables (Hatch & Lazaraton, 1991). In this study, trained students (N = 24) exhibited significantly more (F = 68.65, df = 1, 42, p CO.001) meaning revisions than did untrained students (N = 22) (M = 4.21, SD = 2.02 and M = 1.14, SD = 1.08, respectively). Level 3 students (N = 24, M = 2.42, SD = 2.64) exhibited significantly fewer meaning revisions (F = 5.38, d! = 1,42, p = 0.02) than did level 4 students (N = 22, M = 3.09, SD = 1.72 ). This difference according to level of proficiency can be explained by the fact that level 3 students wrote paragraph-length assignments that were substantially shorter than the essays that level 4 students wrote. As a consequence, level 3 students had fewer ideas to revise. However, relative effect should be considered, and importantly, the interaction effect between training status and level of proficiency was not statistically significant (F = 0.69, df = 1,42, p = 0.41). As displayed in Table 2, the second hypothesis that trained peer response would yield higher writing quality scores (i.e., using the transformed TWE-based scores of 1 to 19) was supported by results of this study in that trained students improved their writing from a first to a second draft more than untrained students

THE EFFECTS

Summary

OF TRAINED

229

PEER RESPONSE

TABLE 2 Statistics of Writing Quality Scores According to Training Status and Level of Proficiency Means

Group Untrained

(N = 22)

Level 3 (N = 12) Level 4 (N = 10) Trained (N = 24) Level 3 (N = 12) Level 4 (N = 12)

First Draft

Second Draft

Gain

8.82 (2.04) 7.83 (1.90) 10.0 (1.56) 8.83 (1.88) 7.75 (1.48) 9.92 (1.62)

9.41 (2.26) 8.17 (2.17) 10.9 (1.29) 11.33 (2.22) 10.08 (1.38) 12.58

0.59 (1.01) 0.33 (0.98) 0.90

(2.23)

( I .OO) 2.5

( I .47) 2.33 (1.50) 2.67 (1.50)

Two-way ANOVA of Difference Scores According Training Status and Level of Proficiency ss

Source Between groups Training status Proficiency level Interaction effect Within groups Total a Standard deviations hp-values

40.54 2.31 0.16 68.90 113.15

df 1 1 1 42 45

to

MS

F

p-valueb

40.54 2.3 I 0.16 1.64 2.5 I

24.71 1.41 0.09

0.001 0.24 0.76

are given below the means in parentheses.

are significant

at alpha = 0.05.

did.* It should be noted that a difference in writing quality before treatment between the trained and the untrained group was ruled out by a two-way ANOVA test on the first draft scores. The students assigned to the trained group (N = 24, M = 8.83, SD = 1.88) did not have statistically different writing quality from those assigned to the control group (N = 22, M = 8.82, SD = 2.04) (F = 0.029, df = 1,42, p = 0.87), thus suggesting that writing quality was equal in these two groups before treatment (i.e., peer response training).9 In contrast, scores on second drafts differed between the untrained and trained groups. An ANOVA test yielded a significance for quality scores on second drafts in the trained group. That is, the training factor had a significant effect (F = 10.921, df = 1,42, p = 0.002). As displayed in Table 2, the difference scores between the first and second draft show a greater gain for the trained than the untrained group. Untrained students (N = 22) improved on average only 0.59 points (SD = 1.01) on the transformed TWE-based score scale ranging from 1 to 19, whereas the trained stu-

230

BERG

dents (N = 24) obtained an average improvement of 2.5 points (SD = 1.47). The training effect was highly significant (F = 24.71, df = 1, 42, p
THE EFFECTS OF TRAINED PEER RESPONSE

231

ing classroom, researchers (e.g., Moore, 1986; Stanley, 1992) report positive results of trained peer response on student attitudes and communication about writing. However, they offer no insights about the impact of trained peer response on revision types or writing outcomes. The issue of effects of peer response instruction on revision raises some interesting questions. Since findings in this study suggest that students provided with peer response training can have peer response sessions that positively influence revision types and subsequent writing quality, why not simply instruct students in the type of revision that would benefit writing quality? With such training in revision, would it not be possible to eliminate the entire step of peer response while still producing similarly improved writing from one draft to another? Although these are fair questions, and the issue of peer response training versus training in revision does need to be raised, the questions miss the point of the role of peer response in the writing process. Students may be able to understand how to revise for meaning versus sentence-level aspects, but how will they know where in their texts such revision is needed? That students have the ability to revise for meaning does not necessarily mean that they will perform a high number of meaning revisions or that the result will be improved writing quality. Before making appropriate meaning revisions in their texts, students need the ability to sense the incongruity between their intended meaning and what they actually communicated to a reader in their writing. And, as has been suggested by previous research (e.g., Sommers, 1980; Nold, 1981), inexperienced writers often lack this ability. I suggested earlier that one of the benefits of peer response is that it can help student writers, who have difficulties separating the meaning expressed by their written words from the supplemental knowledge they bring to their writing, discover the discrepancy between intended and understood meaning of their text. A peer who has not been involved in the creation of the text can point to unclear aspects of the writing. The peer can draw the writer’s attention to areas, ranging from individual words to organization of text, that do not make sense to someone without access to any of the writer’s additional knowledge. This, of course, also applies to more experienced writers who, for example, in submitting a manuscript for publication, consult peers for comments. Admittedly, in a classroom situation, there is another alternative to peers helping each other; the teacher could provide the response, perhaps with better results. However, it is important to remember that the classroom context is one of learning, and what can be gained from peer response in terms of learning opportunities may not be as easily gained from a teacher’s response to student writing. For example, response to writing in a teacher-student relationship entails so much more than a reader helping to point out what in a writer’s text is not clear. It involves an evaluative component of not only what is not clear, but also what is not good. The students, in a sense, have to perform for the teacher, and may simply be looking to do what pleases the teacher instead of truly considering their texts and asking themselves how their meaning can be made clearer. However, in

232

BERG

getting a response from a peer&e student cannot just take the advice as given and make the change, as is likely when the expert (i.e., teacher) provides feedback. Instead, the student will need to consider the advice from a peer, question its validity, weigh it against his or her own knowledge and ideas, and then make a decision about what, if any, changes to make. In answer to the earlier questions about replacing trained peer response with trained revision, we would have to conclude that, first of all, such a replacement would probably not yield similar results in terms of revision or writing quality since students simply would not be able to sense where in their texts they needed to revise for meaning. Peer response can be an important learning tool in a writing course because it helps student writers do what they cannot yet do for themselves, and that is to detect incongruities in their texts, which has been suggested elsewhere (e.g., Nold, 1981) to be a necessary but not sufficient condition for successful revision. Second, the learning benefits, as well as the number of affective benefits reported in the literature (e.g., Baker et al., 1989; Bruffee, 1984; Laque & Sherwood, 1977; Mittan, 1989; Nystrand, 1984), are too valuable to be eliminated. Third, since experienced writers rely on colleagues for feedback as a natural part of their writing processes, it seems fair to also allow inexperienced writers the practice of using peer response as an aid in their writing processes. Fourth, peer response can teach students about academic writing because, in discussing their own and others’ essays, they have to actively apply their knowledge of such aspects as a thesis statement, the development of ideas, and the different types of organization. Hence, peer response provides an opportunity for students to put into practice the abstract ideas about academic writing presented by their teacher and textbooks. Fifth, the discussion of ideas and language in peer response may even help students discover viable text alternatives to unclear aspects of their writing. Finally, watching a peer’s approach to reading one’s text might serve as a model for how to read text through the eyes of someone else. It may then help students develop a better sense of how to read their own texts from the perspective of an audience, what questions to ask, and how to systematically examine their text with the purpose of improving it. It appears that too much can be gained from the peer response activity to exclude it from the writing process, especially for people who are students of writing. However, in order for student writers to take advantage of the learning opportunities presented by peer response, their participation needs to be educated and involved, as has been suggested elsewhere (e.g., Connor & Asenavage, 1994; Stanley, 1992). The two most important classroom implications of the results of this study are (1) that teachers who desire to use peer response as part of their approach to teaching writing in the ESL classroom have some evidence that it can work and that it can result in improved writing and (2) that, in order for peer response to work, training seems essential. The difference in results between the trained and the untrained groups in this study suggests that training results in more successful peer response in terms of revision type and writing quality.

THE EFFECTS

OF TRAINED

PEER RESPONSE

233

Future Research Results of this study indicate a number of areas that need further investigation. Most urgently, it is important to discover what takes place during trained versus untrained peer response negotiations. Whereas positive results on revision types and writing quality were found among students who had received training in this study, we need to examine whether the effects on student writing were the results of the training itself or the trained peer response negotiations. To answer these and similar questions, we need to describe characteristics of trained and untrained peer talk and their relationships to revision strategies and writing outcomes. Furthermore, it would be useful to study the different aspects of the training program to determine the most useful activities for reaching desired outcomes on peer response talk and subsequent revisions and how extensive the training minimally needs to be. It is also important to determine how surface-level revisions affect subsequent writing outcomes. Moreover, since this investigation is limited to one writing assignment right after students were trained, we do not know how training affects peer response and revision at a much later date. Thus, long-term effects of trained peer response need investigation. Additionally, a number of interesting questions remain to be answered about the role of peer response in helping student writers discover incongruities. For example, does participation in peer response provide students with skills in detecting incongruities that could ultimately be applied to their own writing, and does the activity of peer response help supply the writer with viable text alternatives through the discussion with another person? A final suggestion for future research is to both broaden and limit the focus of this investigation. That is, while a larger-scale study than this one is needed to help substantiate results, a smaller-level study, perhaps focused on just one or a few learners, is vital to discover more precisely what takes place throughout the processes of training, peer response talk, and revision. All in all, results of this study open up a number of different areas for investigation. Acknowledgments: I am grateful to Dr. Charlene Polio for the careful reading and insightful comments on earlier versions of this paper. I would also like to acknowledge the helpful suggestions of two anonymous readers. In addition, I would like to thank Dr. David Meyerson and Dr. Yichun Wang for their assistance in the statistical analyses of the data in this study.

NOTES 1. Background information of the term from a questionnaire

on the participants was obtained at the beginning combined with interviews.

234

BERG

2. Only the first writing assignment of the term was used in order to avoid a number of additional variables, such as changes over time (e.g., changes in writing and peer response knowledge, English proficiency, and student relationships) and any other events that happen in and out of class during the course of a term that may, or may not, influence students’ peer response and revision strategies. By limiting the research to only the first assignment of the term, the number of variables to consider was reduced. Another reason for using only the first writing assignment of the term was an ethical one. Whereas my belief is that peer response training is very important to how well students can participate in peer response to writing and how successfully they are able to revise subsequent writing, the quasiexperimental design of this research project demanded a control group that received no training in peer response. Thus, I knowingly deprived 22 students of preparation that I considered essential to the successful completion of their work. However, by limiting this investigation to the first writing assignment of the term, I was able to reach a compromise in that the 22 untrained students, who completed their first assignment without any peer response training, were provided with training before work began on subsequent assignments of the term. In this way, these 22 students could benefit from trained peer response for most of their writing assignments. 3. The Lottery by Shirley Jackson. 4. Agreement was achieved when no discrepancy, as defined by TWE standards, existed (i.e., when readers’ scores were three points or fewer apart). Additionally, the correlation coefficient was calculated for interrater reliability between the first and second reader. Results for the first draft = .74, and for the second draft = .78. 5. Number of meaning changes was considered as a discrete numeric measurement, which was not normally distributed, and hence a two-way KruskalWallis Rank Sum test was employed. 6. Trained and untrained level 3 paragraphs were equal in number of words, as were level 4 essays. 7. An alternative method of testing the hypothesis is to use the Poisson distribution and generalized linear model (McCullagh & Nelder, 1983). According to this model, estimated means were as follows: untrained level 3=0.67, untrained level 4=1.7, trained level 3=4.17, and trained level 4=4.25. The number of meaning revisisons for trained students was significantly higher than that for untrained students (t = 4.83, df = 42, p
THE EFFECTS

OF TRAINED

PEER RESPONSE

23.5

revised student drafts-the parametric and non-parametric methods yielded consistent results. 8. Writing quality scores were considered as continuous numeric measurement, which was normally distributed, and hence a two-way ANOVA method was employed. 9. The two-way ANOVA also made it possible to test for effects of proficiency level on the writing quality of first drafts, and as would be expected, proficiency level was related to a significant difference in first draft quality scores (F = 19.62, df= 1,42, p = 0.08). Level 3: N = 24, M = 7.79, SD = 1.67 and level 4: N = 22, M = 9.95, SD = 1.56.

REFERENCES Ash, B. H. (1983). Selected effects of elapsed time and grade level on the revisions in eighth, tenth, and twelfth graders’ writing (Doctoral dissertation, Florida State University, 1983). Dissertation Abstracts International 43,383O. Baker, M., McQuade, D., Sommers, N., & Tratner, M. (1989). Resources for Teaching Student Writers at Work and in the Company of Other Writers: The Bedford Prizes (pp. 102-107). New York: St. Martin’s Press. Bamberg, B. (1978). Composition instruction does make a difference: a comparison of college freshman in regular and remedial English courses. Research in the Teaching of English 12,47-59. Benesch, S. (1984). Improving peer response: colluboration between teachers and students. Urbana, IL. ERIC Document Reproduction Service #ED 243 113. Berg, E. C. (1999). Preparing ESL students for peer response. TESOL Journal 8, 2@25. Bracewell, R. J., Scardamalia, M., & Bereiter, C. (1978). The development of audience awareness in writing. Urbana, IL. ERIC Document Reproduction Service #ED 154 433. Bridwell, L. S. (1980). Revising strategies in twelfth grade students’ transactional writing. Research in the Teaching of English 14, 197-222. Bruffee, K. A. (1984). Peer tutoring and the “conversation of mankind.” In G. Olson (Ed.), Writing Centers: Theory an< Administrution (pp. 3-15). Urbana, IL: National Council of Teachers of English. Carson, J. G., & Nelson, G. L. (1996). Chinese students’ perceptions of ESL peer response group interaction. Journal of Second Language Writing 5, 1-l 9. Carson, J. G., & Nelson, G. L. (1994). Writing groups: cross-cultural issues. Journul of Second Language Writing 3, 17-30. Chenoweth, A. (1987). The need to teach rewriting. ELTJournal41,25-29. Connor, U., & Asenavage, K. (1994). Peer response groups in ESL writing classes: how much impact on revision? Journal of Second Language Writing 3,257-276. Cumming, A., & So, S. (1996). Tutoring second language text revision: does the approach to instruction or language of communication make a difference? Journal of Second Language Writing 5, 197-226. Educational Testing Services. (1996). TOEFL Test qf Written English Guide (4th ed.). Princeton, NJ: Author.

236

BERG

Faigley, L., & Witte, S. (1981). Analyzing revision. College Composition and Communication 32,400-4 15. Fitzgerald, .I. (1992). Towards Knowledge in Writing: Illustrationsfrom Revision Studies. New York: Springer-Verlag New York, Inc. Flower, L., & Hayes, J. R. (1981). A cognitive process theory of writing. College Composition and Communication 32, 365-387. George, D. (1984). Working with peer groups in the composition classroom. College Composition and Communication 35, 320-326. Hafemik, J. J. (1983). The how and why of peer editing in the ESL writing class. Urbana, IL. ERIC Document Reproduction Service #ED 253 064. Hardaway, F. (1975). What students can do to take the burden off you. In R. Ohmann & W. B. Coley (Eds.), Ideas for English 101: Teaching Writing in College (pp. 22% 223). Urbana, IL: National Council of Teachers of English. Hatch, E., & Lazaraton, A. (1991). The Research Manual: Design and Statisticsfor Applied Linguistics. Boston: Heinle & Heinle. Hawisher, G. E. (1987). The effects of word processing on the revision strategies of college freshmen. Research in the Teaching ofEnglish 21, 145-159. Huff, R., & Kline, C. R. (1987). The Contemporary Writing Curriculum: Rehearsing, Composing, and Valuing. New York: Teachers College Press. Jackson, S. (1948/1988). The Lottery. In S. Barnet, M. Berman, & W. Burto (Eds.), Literature for Composition: Essays, Fiction, Poetry, and Drama (pp. 463-469). Glenview, IL: Scott, Foresman, and Company. Laque, C. F., & Sherwood, P. A. (1977). A laboratory approach to writing. Urbana, IL: National Council of Teachers of English. Leki, I. (1990). Potential problems with peer responding in ESL writing classes. CATESOL Journal 3,5-l 9. MacArthur, C., & Stoddard, B. (1990, April). Teaching learning disabled students to revise: a peer editor strategy. Paper presented at the annual meeting of the American Education Research Association, Boston, MA. Mangelsdorf, K., & Schlumberger, A. (1992). ESL student response stances in a peer review task. Journal of Second Language Writing I, 235-254. Maynor, L. C. (1982). An investigation of the revising practices of college freshman writers (Doctoral dissertation, Duke University, 1982). Dissertation Abstracts International 43,2543A. McCullagh, P., & Nelder, J. A. (1983). Generalized Linear Models. London: Chapman and Hall. Mendonca, C., & Johnson, K. (1994). Peer review negotiations: revision activities in ESL writing instruction. TESOL Quarterly 28,745-769. Mittan, R. (1989). The peer review process: harnessing students’ communicative power. In D. M. Johnson & D. H. Roen (Eds.), Richness in Writing: Empowering ESL Students (pp. 207-219). New York: Longman. Moore, L. K. (1986). Teaching students how to evaluate writing. TESOL Newsletter 20, 23-24. Nelson, G. L., & Murphy, J. M. (1992). An L2 writing group: task and social dimension. Journal of Second Language Writing I, 17 l-l 92. Nelson, G. L., & Murphy, J. M. (1993). Writing groups and the less proficient ESL student. TESOL Journal 2.23-26.

THE EFFECTS OF TRAINED PEER RESPONSE

231

Nold, E. (1981). Revising. In C. H. Frederiksen & J. F. Dominic (Eds.), Writing: The Nature, Development, and Teaching of Written Communication (pp. 67-79). Hillsdale, NJ: Erlbaum. Nystrand, M. (1984). Learning to write by talking about writing: a summary of research on intensive peer review in expository writing instruction at the University of Wisconsin-Madison. Urbana, IL. ERIC Document Reproduction Service #ED 255 914. Nystrand, M. (1989). A social-interactive model of writing. Written Communication 6, 66-85. Perl, S. (1978). Five writers writing: case studies of the composing processes of unskilled college writers (Doctoral dissertation, New York University, 1978). Dissertation Abstracts International 39,4788A. Perl, S. (1979). The composing process of unskilled college writers. Research in the Teaching of English 13, 3 17-336. Reither, J. A., & Vipond, D. (1989). Writing as collaboration. College English 51, 855867. Schumaker, J. B., Deshler, D. D., Alley, G. R., Warner, M. M., Clark, F. L., & Nolan, S. (1982). Error monitoring: a learning strategy for improving adolescent performance. In W. M. Cruickshank & J. Lemer (Eds.), Best ofACLD, (Vol. 3) (pp. 170183). Syracuse: Syracuse University Press. Smalley, R. L., & Reutten, M. K. (1990). Refining Composition Skills: Rhetoric and Grammarfor ESL Students. New York: Maxwell Macmillan. Sommers, N. (1980). Revision strategies of student writers and experienced adult writers. College Composition and Communication 3/,378-388. Stanley, J. (1992). Coaching student writers to become effective peer evaluators. Journal of Second Language Writing 1,2 17-233. Witbeck, M. C. (1976). Peer correction procedures for intermediate and advanced ESL composition lessons. TESOL Quarterly 10,321-326. Zamel, V. (1983). The composing processes of advanced ESL students. TESOL Quarterly 17, 165-187. Zhang, S. (1995). Reexamining the affective advantage of peer feedback in the ESL writing class. Journal of Second Language Writing 4,209-222.

238

BERG

APPENDIX

SAMPLE

PEER RESPONSE

A

SHEET

Author Responder Please answer the following questions, keeping peer response is to help each other write better. 1. 2. 3.

4. 5. 6.

FOR AN ESSAY

in mind that the purpose of

Can you find a thesis statement? _yes _no _I don’t know If you think you can find a thesis statement, please underline it. Does each body paragraph have a clear topic sentence? _yes _no _I don’t know Please explain your answer. Read the essay carefully. Underline everything that you don’t understand. What do you like the best about this essay? What questions, comments, and/or suggestions do you have for the author?

After you have answered these questions, discuss them and the essay with the author. Remember that you are writing for each other, so it’s important that you understand each others’ writing. Please tell the author what you think because it can help him/her write a really good essay.

APPENDIX

B

ELEVEN CONSIDERATIONS FOR PREPARING STUDENTS PARTICIPATE IN PEER RESPONSE AND THE ACTIVITIES PLACE OF TRAINING FOR STUDENTS NOT TRAINED IN PEER RESPONSE Activities 1.

2.

3.

TO IN

for Students Trained in Peer Response Writing

Comfortable classroom atmosphere and trust among students A number of in-class get-to-know each other activities and out-of-class pair and group projects (e.g., visiting a restaurant with a few classmates and orally reviewing it to the class) are conducted. The role of peer response in the writing process Writing as a process is explained. Researchers’ suggested benefits of having peers, as opposed to just teachers, respond to one’s writing are discussed. Professional writers using peer response Students examine the “acknowledgments” in their textbook. Through a class discussion, they arrive at the conclusions that all authors, student as well as professional, ask others to read their work, and that doing so is an

THE EFFECTS OF TRAINED PEER RESPONSE

4.

5.

6.

7.

8.

9.

10.

239

indication of a smart writer. Several other examples of professional writers acknowledging their “peer responders” in journals such as the TESOL Quarterly and Journal of Second Language Writing are also studied. The teacher using peer response Several drafts of a conference proposal with comments from my colleagues are examined, focusing on the progression from the first to last draft and how peers’ comments helped improve the proposal. Class peer response to writing Students respond as a class to an unknown ESL student’s paragraph, noting the clear statement of the main idea and some good details, and also some unclarities and obvious flaws in organization, support, unity, grammar, and spelling. We then discuss appropriate revisions, stressing the importance of revising for clarity of meaning and rhetorical-level aspects rather than cosmetic sentence-level errors. Appropriate vocabulary and expressions Appropriateness of language in responding to someone’s writing is addressed by comparing inappropriate comments, such as “your writing is really bad”, with appropriate ones, such as “It would be great if you gave an example here.” We also talk about and give examples of being specific and making clear to the author that the opinions expressed are not fact, but only one particular student’s impression of a peer’s writing. The response sheet The “peer response sheet for an essay” (see Appendix A for an example) is introduced to the students as a tool designed to provide adequate time to consider the writing and help them focus on some important areas of the writing assignment. Response to a collaborative writing project As an activity to practice using the peer response sheet, students get into groups of two or three and respond to an academically-structured paragraph written by another group of students. Based on the response, the student groups revise their collaborative paragraphs. Conversations among authors, responders, and the teacher Within each group, we talk about their collaborative paragraph, the peer response, the revisions they made, and students are encouraged to ask questions and express concerns. A whole-class discussion about some of the difficulties in judging classmates’ comments and students’ lack of confidence in their revision abilities concludes the activity. Revision guidelines With students’ revisions of collaborative paragraphs as a basis, students are introduced to what has been found in the literature to improve writing. We also discuss some good revision strategies and how peer response helps authors understand that there is sometimes a discrepancy between intended and perceived meaning.

BERG

240

11.

Sample peer response sessions As a last activity in the peer response training process, we view two video examples of peer response. In terms of level of student engagement, language used, and topic discussed, one video clip is an example of an unsuccessful interaction and the other is part of a successful one.

Activities for Students Not Trained in Peer Response to Writing Since the training process involved 11 activities that, on average, took about 20 minutes each, the untrained group was involved in other activities for approximately the same amount of time. Below, the first section (Part I) is an outline following the 11 steps of the preparation provided for the trained classes with an indication for each step of how much, if at all, was also provided for the untrained classes, and the second section (Part II) briefly describes the training replacement activities for the untrained classes. Part I 1.

2.

3. 4. 5. 6. 7. 8.

Comfortable classroom atmosphere and trust among students The same in-class “get-to-know each other” activities and out-of-class pair and group projects were conducted during the first 3 days of the term for the untrained groups as for the trained groups. The role of peer response in the writing process Time was spent during the second and third days on the idea of writing as a process just as was done for the trained groups. However, instead of listing the benefits of having peers, as opposed to just teachers, respond to one’s writing, students were told simply that peer response has been found to be beneficial to student writing. Professional writers using peer response This was not done. The teacher using peer response This was not done. Class peer response to writing This was not done. Appropriate vocabulary and expressions This was not done. The response sheet This was not done. Response to a collaborative writing project This was not done. However, students did participate in a collaborative writing assignment in which they created and wrote an ending to the short story we were reading in class.

THE EFFECTS OF TRAINED PEER RESPONSE

9. 10.

11.

241

Conversations among authors, responders, and the teacher This was not done. Revision guidelines Although it was not possible to use students’ revisions of collaborative paragraphs as a basis for discussing what types of changes result in improvement of writing, students were informed about findings on how to improve writing. As with the trained group, some good revision strategies were discussed. Sample peer response sessions This was not done.

Besides the practical instructions for how peer response would be conducted, which were given near the day of the response activity, this was the extent of information that the untrained group received about peer response to writing.

Part II In place of the training activities provided for the trained group, the untrained group spent time on one main activity, reading and working with a short story. Students were involved in approximately as much group and pair work, including a collaborative writing task, as trained students were. As much as possible, the context was kept the same between the two different groups so that the only main difference was the specific training that one group received.