Building students’ evaluative and productive expertise in the writing classroom

Assessing Writing 19 (2014) 66–79

Contents lists available at ScienceDirect

Assessing Writing

Building students’ evaluative and productive expertise in the writing classroom Eleanor M. Hawe ∗, Helen R. Dixon 1 Faculty of Education, The University of Auckland, Private Bag 92601, Symonds Street, Auckland 1150, New Zealand

a r t i c l e

i n f o

Article history: Available online 2 December 2013

Keywords: Assessment for learning Writing classroom Evaluative and productive expertise Peer feedback Self-monitoring

a b s t r a c t Contemporary notions of feedback involve students working alongside teachers and peers with a view to enhancing students’ learning. It is no longer sufficient or fitting for teachers to be the primary source of feedback as this runs the danger of developing dependence on external sources for information about progress and learning. If students are to move from being recipients of feedback to intelligent self-monitoring, they need to take responsibility for their learning. With this end in mind, instructional programmes should provide students with authentic opportunities to monitor and improve the quality of work during production. Three elementary teachers who articulated similar beliefs about the importance of feedback and student involvement in their learning, and who described teaching practices congruent with the development of student autonomy, were observed during the teaching of a genre based writing unit. Observations revealed qualitative differences in the opportunities created for students to gain understanding of expectations, engage in evaluative and productive activities, and make decisions about their writing. These three cases show that developing students’ evaluative knowledge and productive skills in writing involves adoption of AfL as a unitary notion and a radical transformation of the traditional taken-for-granted roles and responsibilities of teachers and students. © 2013 Elsevier Ltd. All rights reserved.

∗ Corresponding author. Tel.: +64 9 623 8899x48733. E-mail addresses: [email protected] (E.M. Hawe), [email protected] (H.R. Dixon). 1 Tel.: +64 9 623 8899x48547. 1075-2935/$ – see front matter © 2013 Elsevier Ltd. All rights reserved. http://dx.doi.org/10.1016/j.asw.2013.11.004

E.M. Hawe, H.R. Dixon / Assessing Writing 19 (2014) 66–79

67

1. Introduction Assessment for Learning (AfL) is part of everyday practice by teachers, students and peers that seeks, interprets and responds to information from dialogue, demonstration and observation in ways that enhance learning (Klenowski, 2009). The overall aim of AfL is for students to become autonomous, selfregulating learners who “judge performance relative to goals, generate internal feedback about amounts and rates of progress towards goals, and adjust further action based on that feedback” (Butler & Winne, 1995, p. 258). Influenced by socio-cultural perspectives on learning, and developments in the fields of student self-regulation and assessment for learning, present-day notions of feedback emphasize the importance of learner autonomy and agency. It is no longer sufficient or fitting for teachers to be the primary or sole source of feedback as this runs the danger of developing and maintaining dependence on others for information about progress and achievement (Sadler, 2010), rather teachers and students work in partnership to construct achievement and effect improvement (Gardner, 2006). As students move from being recipients of information to a state of ‘intelligent self-monitoring’ (Sadler, 1989), they take ownership of and responsibility for their learning. Instructional programmes must therefore provide students with opportunities to monitor and regulate the quality of their work during production. Development of students’ evaluative and productive expertise is contingent on three conditions: students need to understand the goals of learning and what constitutes quality work, compare current performance to what is expected, and have a repertoire of strategies so they can modify performance as necessary (Sadler, 1989, 2009). This paper reports on an empirical study in the instructional area of writing where the practice of three teachers who articulated similar beliefs about the importance of feedback and student involvement in learning, and described teaching practices congruent with the development of student autonomy was examined. Two complementary fields of scholarship inform this examination: Sadler’s (1989, 2007, 2009, 2010) seminal and more recent work in formative assessment, and a contemporary reading of Assessment for Learning (Swaffield, 2011; Willis, 2011). Findings from the study will be of interest to those with responsibility for teacher learning in the areas of teaching writing and/or assessment, whether these are in-house as part of school-driven initiatives, or externally driven as in the case of national, Ministry of Education endorsed professional development contracts. 2. Developing students’ evaluative and productive expertise in writing The most effective way for learners to grasp the nature of a complex activity such as writing is through direct experience in the creation, evaluation and revision of work – “no amount of telling, showing or discussing is a substitute for one’s own experience” (Sadler, 2009, p. 49). Teachers need therefore to establish a pedagogical environment in which students are deliberately inducted into the art of making substantive and comprehensive appraisals of their own and peers’ work, during production with a view to making improvements and furthering learning (Sadler, 2009, 2010). Authentic evaluative activities, operationalized through peer feedback and student self-monitoring, enable writers to assess the value and efficacy of texts while they are being created (Hout & Perry, 2009). Providing students with substantial evaluative experience is a strategic and deliberate part of the teaching design through which students are initiated into the ‘guild knowledge’ of the writing community (Marshall, 2004; Sadler, 2010). Central to this initiation is development of shared understandings between teachers and students, and among students, about the goal(s) of writing and what it is that constitutes quality when writing a particular kind of text. 2.1. The role of goals, learning intentions and success criteria If learners are to play a central role in their learning, they need to understand ‘where they are going’ (Hattie & Timperley, 2007). Failure to establish clear goals for writing is a significant obstacle to student revision of texts (Fitzgerald, 1987) and the development of successful writers (Timperley & Parr, 2009). In addition, learners need to understand what it is that comprises quality in piece of writing. New Zealand teachers have, in recent years, been encouraged to share and/or create goals of learning with students in the form of learning intentions and to use success criteria, rubrics, models

68

E.M. Hawe, H.R. Dixon / Assessing Writing 19 (2014) 66–79

and/or exemplars to communicate what counts as successful achievement of these goals (Clarke, 2001). Writing is a complex activity with no single agreed upon body of scholarship that defines the content of the subject for the purposes of learning, teaching and assessment (Marshall, 2004; Parr, 2013). Furthermore, the act of writing is an art (Marshall, 2004) and as such, “there are no formal techniques or recipes which, if followed precisely, would lead to high-quality responses” (Sadler, 2009, p. 47). For any piece of writing there is therefore a complexity and diversity of features that can be called upon to designate quality and represent ‘good writing’ (Broad, 2003). It would be unworkable for teachers to list all of these features, so a selection of the most salient is made with reference to the instructional programme. These features or properties, referred to as ‘manifest criteria’, are promulgated in rubrics, success criteria and the like (Sadler, 2009). Such devices have however been critiqued as they tend to over-emphasize the structural and more readily identifiable (often superficial) characteristics of a piece of writing (Broad, 2003; Hawe, Dixon, & Watson, 2008). Restricting attention to the manifest criteria is limiting as these criteria inevitably overlook features that emerge during production and evaluation – features that may in fact facilitate access to the writer’s message and/or contribute to a piece of work ‘standing out’ (Broad, 2003; Sadler, 2009). Writers need therefore to be aware, both during production and when making appraisals, of the ‘latent criteria’ that underpin and contribute to a piece of work (Sadler, 1989, 2009). Challenges associated with capturing the nature of quality writing do not mean the articulation of goals and what constitutes successful achievement of these should be abandoned. Rather than using precise goals and a standard set of pre-determined criteria to define progression and success, it is preferable to view progress as movement towards a broad horizon or goal where outcomes and expectations are less precisely defined at the outset and multiple pathways to successful achievement are recognized (Marshall, 2004). As pieces of writing develop, teachers and students iteratively clarify and establish what they as a community value in the work. In practice, New Zealand teachers do not rely exclusively on externally produced rubrics and/or success criteria when developing students’ understandings about quality work. They make adjustments to rubrics and criteria promulgated by the Ministry of Education, and draw on a range of pedagogical tools such as modelling, reference to carefully selected exemplars, and substantive discussions around exemplars and the nature of writing to provide students with insights into what it is that constitutes ‘good’ work (Parr & Limbrick, 2010). Taken together, these tools serve as points of reference when developing notions of ‘quality’ and making judgments about writing drafts or works-inprogress.

2.2. Making judgments about writing If students are to monitor and regulate work during production, opportunities for serious evaluations of works-in-progress need to be embedded in writing lessons (Hattie & Gan, 2011; Ward & Dix, 2004). Through the experience of evaluating their own and peers’ work, students come to realize the representation of quality in pea-sized bites underestimates the impact of the whole (Parr, 2011; Sadler, 2007). Quality in writing is reflected in and determined through all-things-considered holistic judgments where the whole is greater than the sum of the parts (Sadler, 2009). The process of constructing text requires the writer to revisit and revise at a whole-text level as well as addressing the more mechanical aspects of their work (Ward & Dix, 2004). Appraisals should therefore be conducted as flexible and open activities where dual, iterative processes are in play. The first process involves scoping the work as a whole to get a feel for its overall quality; the second pays attention to particular qualities in or properties of the work (Sadler, 2009). As they engage in evaluative activities students learn to make holistic, multi-criterion judgments, justifying these with reference to salient properties that may (or may not) be included in the manifest criteria. Moreover, they are exposed through these activities to the subject knowledge and evaluative expertise of their teachers (Marshall, 2004; Sadler, 1989). Armed with this knowledge and expertise, students are in a powerful position to generate feedback about areas of achievement and improvement in writing.

E.M. Hawe, H.R. Dixon / Assessing Writing 19 (2014) 66–79

69

2.3. Feedback and student self-monitoring Writers need feedback so they can discover the reactions and needs of readers, monitor their progress and make adjustments to their work (Zellermayer, 1989). Quality feedback provides information about progress and learning in relation to goals and expectations; encourages dialogue between the teacher and student and between students about the substantive aspects of learning; helps students develop a repertoire of alternative moves or strategies; encourages positive motivational beliefs; and enhances self-esteem (Hattie & Timperley, 2007; Nicol & Macfarlane-Dick, 2006; Sadler, 1989, 1998). When generating feedback from an audience, writing is constructed as a purposeful communication rather than completion of a task for the teacher. It is therefore critical that teachers establish an environment where students can freely exchange views about texts and mutually construct meaning in a thoughtful and reflective manner. Common features of writing lessons such as conferences (Graves, 2003) and writing circles (Gunnery, 2007) provide opportunities for student-writers to interact with and craft meaning for readers and to receive audience feedback. Effective conferences and circles help “develop the meta-cognitive awareness related to the writing process and the self-regulatory strategies needed for reflecting on [their] texts, together with the personal responsibility needed to become a writer” (Parr, 2011, p. 56). Peer feedback is a socially situated dialogic process (Askew & Lodge, 2000) where students work together, in pairs or small groups, to construct achievement and encourage improvement. Ward and Dix (2004) assert teachers rarely provide organized, formal in-depth opportunities during writing lessons for students to discuss and appraise peers’ (and their own) texts. Much of the peer feedback observed in writing classrooms occurs incidentally and informally, between friends, as they seek assistance on how to get started with their writing and ask each other for ideas (Ward & Dix, 2004). If peer feedback is planned for, it is often focused on end products and serves a summative rather than formative, forward-looking purpose (Dixon, 2011; Dixon, Hawe, & Parr, 2011). Peer response during production, using the language of writing, is however vital to developing students’ understandings about how texts work, and to furthering their writing (Ward & Dix, 2004). Authentic evaluative experiences during writing provide students with rich learning opportunities on a number of levels. While peer response is valuable in helping writers make improvements to their own work (Hyland, 2000; Keen, 2010), it helps both parties gain insights into the act of writing and what constitutes successful achievement in a particular genre or context. Students build their evaluative knowledge and productive expertise as they are exposed to ways in which others have approached the same or a similar task. They can see the challenges faced by peers, and the different moves and strategies used to bring performance closer to what is expected (Paris & Paris, 2001; Sadler, 2009). Through dialogue and discussion, students have opportunities with their peers to develop a meta-language that enables them to talk together about writing, using the language of writing (Ward & Dix, 2004). Engaging in peer feedback has the capacity to lead to, and is the pre-cursor of ‘intelligent self-monitoring’, a state whereby students generate information, during learning, about the quality of their performance (Sadler, 1989). Self-monitoring results in student ownership of and responsibility for learning and plays a part in the development of intrinsic motivation (James & Pedder, 2006). Opportunities to develop the skill of self-monitoring have enhanced students’ ability to revise and improve their writing (Xiang, 2004). Evaluative experiences can also be used to convey key learning about the writing process and the desired product. Participation in authentic evaluative activities not only facilitates deep engagement with subject matter (Hattie & Gan, 2011; Sadler, 2009), such activities help students acquire requisite knowledge and skills for self-monitoring and self-regulation (Dixon, 2011; Paris & Paris, 2001). 3. The research design Utilizing an interpretive, qualitative, case study methodology the research was conducted in two sequential phases with participants in phase two selected purposively from all who participated in phase one. The aim of phase one was two-fold: to investigate teachers’ beliefs and knowledge about feedback, and to investigate their perceptions of practice. At the time ethics approval was gained approximately 400 practicing teachers had upgraded their teaching qualification from a Diploma of

70

E.M. Hawe, H.R. Dixon / Assessing Writing 19 (2014) 66–79

Teaching to a Bachelor of Education degree at The University of Auckland. Invitations to participate were sent to a random selection of 100 of these teachers and the 20 who volunteered comprised the phase one participants. The semi-structured interview utilized in phase one tapped into teachers’ conceptions about the nature and role of feedback in the enhancement of learning; beliefs about their role and that of learners in the feedback process; and the strategies and practices teachers utilized and ascribed importance to within the feedback process, including the opportunities offered to students in relation to development of evaluative and productive knowledge and expertise (Sadler, 1989). Each interview took between 35 and 60 minutes, and with four exceptions, took place in teachers’ classrooms. The data set from phase one was complemented by the collection of teachers’ learning intentions and success criteria (Clarke, 2001). A ‘case’ is a bounded system, focused on the particular, descriptive in style, and intent on bringing about new meaning or understanding about the phenomenon under investigation (Merriam, 1998; Stake, 2000). In phase two, the case studied was teachers’ use of feedback during the teaching of writing, bounded in time and space. It was particularistic as it focused on the roles of the teacher and learners in the feedback process, as well as the nature of opportunities provided for students to develop evaluative and productive knowledge and expertise. The heuristic value of the case lay in its potential to gain insight into teachers’ espoused practice and the enactment of practice. Teachers were selected to participate in phase two because their talk in phase one was consistent with ‘best practice’ in feedback. Specifically their talk was consistent with the following notions: • goals for learning and what counted as a successful achievement were shared with students; • feedback was linked to learning goals and expectations, and was both achievement and improvement focused; • students were seen as active participants in the processes of feedback and learning; • through participation in peer feedback and self-monitoring experiences students were provided with opportunities to amass evaluative and productive knowledge and expertise similar to that held by the teacher. Of the 20 teachers interviewed in phase one, five provided strong evidence in relation to these criteria. Of the five teachers, two were unable to participate in phase two, one due to a change in teaching circumstances and the other for health reasons. The three who volunteered and were able to participate in the second phase were Kate, Marama and Audrey (pseudonyms). Based on participants’ interview responses, findings from relevant literature and a consideration of the current focus on literacy in New Zealand Ministry of Education curriculum policies and initiatives, written language was identified as the context in which there was the greatest probability of gathering rich data about teachers’ ability to implement theoretically appropriate feedback strategies. Whilst the researchers chose writing as the subject for observation, teachers chose the unit of work to be observed. Each of the observed units was genre based. Aware that observations must be undertaken in a “thoughtful and principled way” (Delamont, 1992, p. 113), a combination of Sadler’s (1989) conceptual framework of formative assessment and feedback and insights gained from phase one set the parameters around the classroom observations. Each series of observations was undertaken at strategic points: two at the introductory phase of the unit; two as students engaged in writing; and one at the unit’s end. Data were collected through field notes, the audiotaping of lessons and collection of relevant documentation. Field notes included organizational and structural details such as teacher and pupil movement from one activity to another and the nature of interactions (group, class, individual). Details pertaining to teachers’ use of and reference to learning intentions, success criteria and exemplars during lessons were also recorded, as was students’ use of these resources. The nature and length of the opportunities provided to students to make evaluative judgments and productive decisions either with the teacher or each other were also noted. Documentation included each teacher’s written language unit plan, the learning intentions and success criteria developed, models and exemplars used and handouts and worksheets given to students. A semi-structured interview was conducted at the completion of the series of observations to probe each teacher’s intentions, pedagogical decisions and embedded beliefs.

E.M. Hawe, H.R. Dixon / Assessing Writing 19 (2014) 66–79

71

Two theoretical lens associated with qualitative inquiry, hermeneutics and phenomenology (Schwandt, 2000), were used to identify, interpret and explain the subjective meanings and actions of the three teachers. Hermeneutic theory informed analysis of the teaching–learning episodes and subsequent interviews, while phenomenology drew attention to the nature of each teacher’s commonsense knowledge, beliefs and taken-for-granted assumptions about feedback, and the respective roles of the teacher and student. More specifically, analysis was grounded in the use of strategies associated with thematic analysis, the constant comparative method (open, axial and selective coding) and discourse analysis (Ezzy, 2002; Glaser & Strauss, 1967). Concepts from Sadler’s (1989) theory of formative assessment and feedback were used in the first instance to generate open, axial and selective codes, categories and themes. These were applied systematically and iteratively to the transcripts of lessons, interviews and lesson artefacts. To ensure analysis was not unduly constrained by the use of Sadler’s theory, additional codes, themes and categories were inductively developed from and applied to the data. Furthermore, data were scrutinized using the European style of discourse analysis (MacLure, 2003) to determine the nature of the power relations between teachers and students, and the impact of these relations on development of students’ evaluative and productive knowledge and expertise. 4. Findings During Phase 1 interviews, each of the three teachers emphasized the importance of sharing their evaluative knowledge with students as such knowledge was considered critical to full participation in the processes of learning and assessment. Therefore teachers aimed to make expected learning explicit through the use of learning intentions. Success criteria and exemplars were seen as a means of deconstructing and unpacking successful performance. Access to and understanding of expectations were deemed essential if students were to generate feedback information about what had been achieved and needed improvement. Student appraisal of their peers’ work was also considered valuable in the development of evaluative and productive knowledge and expertise. Making qualitative judgments and productive decisions about others’ work was seen as leading to student self monitoring, a critical component in the improvement of learning. In this paper, three case studies are presented based on the observations undertaken during the teaching of a genre based writing unit. One series of observations focuses on the teaching of narrative writing unit with Year five students (Audrey), another on a transactional writing unit taught to Year seven students (Kate), and finally a series of observations pertaining to the teaching of poetic writing with Year eight students (Marama). Whilst the three teachers employed similar strategies, the case studies illustrate the considerable variation that occurred in the nature of the goals communicated to students, the evaluative judgments and productive decisions students were asked to make and the degree of student involvement in evaluative and productive activities. 4.1. The case of Audrey Audrey expected in the narrative based unit of work that not only would her Year five students produce a piece of writing reflective of the given genre, they would also gain significant learning related to the writing process. Through engagement in three types of productive activity – planning for writing, developing a plan into a first draft and producing a ‘finished’ version ready for public viewing, she hoped students would develop understanding and skill in the structure and organization of ideas, and would spend significant time refining and re-crafting their writing. However her focus was on a single, product-oriented goal, “I am learning to write a narrative for a young child” (Obs2 ). The nature of the success criteria developed during the second lesson of the unit did little to elaborate on this goal beyond identification of a list of properties (elements) to be included in the narrative: • Setting – describe; • Characters – traits;

2

Obs – refers to classroom observation data.

72

• • • •

E.M. Hawe, H.R. Dixon / Assessing Writing 19 (2014) 66–79

Problem; Solution; Fiction, tell a story; Beginning, middle, end (Student success criteria).

Although Audrey spent considerable time establishing these success criteria with input from students, little was done to flesh out what was expected. Taken together, the learning intention and success criteria had the potential to restrict opportunities for students to develop the required evaluative knowledge and expertise necessary to make judgments about their productive attempts. On numerous occasions as students worked on their narratives, Audrey made explicit reference to success criteria as the point of reference when students were judging their work: “Don’t forget to keep checking your success criteria . . . you need to make sure all these things here from the success criteria are included in your plan” (Obs). Emphasis was placed on the fact that a ‘good’ narrative must contain all the required elements. As requested, students used the criteria as the basis for making judgments about their readiness to move to the next stage of the writing process and subsequently took plans and drafts to Audrey for individual writing conferences. At the start of a conference Audrey focused on the success criteria, looking for the given elements – “you’ve got your setting, characters, beginning, middle and end, you’ve got your problem and you’ve got your solution” (Obs). However despite presence of the required elements in the narrative, Audrey invariably judged students’ work as warranting further attention to reach the required standard – a standard known only to her. As she explained later, “I wanted more than a setting. I wanted a description of a setting and using the senses . . . and good language.” (PO Int3 ). Failure to share these tacitly held expectations with students resulted in misalignment between the evaluative judgments made by students and those made by Audrey. Furthermore, during the conferences, when Audrey identified an aspect of the narrative that detracted from her conception of quality, she took on the role of decision maker, telling students what to do to effect improvement. Feedback was often framed as a directive to be carried out, for example: “. . . so you need to describe in detail what it looks like [the setting]. What you saw there. Who was there? Thinking of your who, where, what, how, why questions” (Obs). There was little evidence of student voice or dialogue between Audrey and individual students during these conferences. While there were formal opportunities for students to share works-in-progress either during or at an end of a lesson, these were marked by formality and brevity. Typically Audrey called for volunteers to read snippets of their work to the class. During this time the audience was restricted to listening to these short extracts rather than reading and/or engaging with the work as written. Once shared with the class it was Audrey who judged the quality of the text with no input from students. At this time Audrey’s judgement was often focused on features of the work not contained within the success criteria, but which in her mind were important in relation to the quality of the writing: “Stop there, that’s awesome. What I liked Jack is the descriptive words you used to introduce your story . . . Excellent, well done.” (Obs). Audrey used this sharing time with the class to praise individuals for inclusion of specific features in their work such as use of descriptive language or development of character traits, with the hope that others might take note and make subsequent changes to their work. However this expectation was never made explicit to students. Furthermore, at no time did students ever ‘see’ others’ texts. Their access to peers’ work was fleeting, providing them with little insight and understanding of how specific strategies were used by others and how these might be incorporated into their own work. In the final lesson of the unit, as students prepared to publish their narratives, they engaged in “red pen checking” (Obs) where they marked their work to correct spelling, punctuation and gram3

PO Int – refers to Post Observation Interview data.

E.M. Hawe, H.R. Dixon / Assessing Writing 19 (2014) 66–79

73

mar. During students’ ‘final copy’ conferences Audrey made the evaluative judgments and productive decisions as her feedback to students identified errors and how to correct these: “[student writing] ‘A monkey peered down and steered.’ That’s the wrong steered, that one [indicating steered] means you’re going to steer a group of animals. So you want this one here – staring.” (Obs). Audrey’s promotion and endorsement of the “red pen checking” process was built on her belief that students learned more if they were active in making judgments and decisions about their work. If they depended on the teacher to make all the judgments “it would be too easy and they’d never learn from it. They’d never get better” (PO Int). From her perspective, this process enabled students to monitor their performance, which in turn led to improvements to work. Ironically, it was Audrey who controlled the evaluative process and it was she who made the final productive decisions. While students may have engaged in action to close the gap between current and desired performance, Audrey orchestrated their actions. The student’s role was limited to carrying out her directives. 4.2. The case of Kate Kate’s Year seven students were charged with the task of producing a single frame cartoon followed by a short comic strip, both of which were to convey a message through use of appropriate literary and visual techniques. Prior to commencing the writing unit, students had been exposed to and discussed a range of cartoons and comic strips. Initially, the class brainstormed features of cartoons and comics, and how authors ‘got their message across’ to readers. As these ideas were progressively refined, Kate recorded them on the class whiteboard where they served as a semipermanent, living record for the duration of the unit. In addition, Kate distributed a more formal assessment rubric. Through this rubric and the ideas on the whiteboard, both of which were referred to throughout the writing unit, Kate’s students were exposed to the complexities when making evaluative judgments and productive decisions about their work. Kate drew attention via these tools to the notion that successful completion of each piece of work was not solely dependent on inclusion of particular elements and techniques. Constructed using a series of descriptors, the rubric specified achievement at four levels of attainment: skilled, competent, developing strength and having difficulty. In identifying the elements and levels of attainment in the rubric, Kate hoped that it would not only clarify for students “the specific details I’m looking for” but would also convey the understanding there were “degrees of completion” (Int4 ). For example, her use and continuing discussion of relative terms from the rubric such as ‘professionally’ and ‘effectively’ with reference to properties of a cartoon (e.g.: visual traits and humour) signalled that it is the way particular traits and techniques are used that is the hallmark of a quality performance, not simply their inclusion in a piece of work. Given her belief that “there’s so many valuable ideas, information, you can get off your peers” (PO Int), Kate provided students with both formal and informal opportunities to see and discuss others’ work at various stages of production. Formal opportunities were incorporated into the lesson as time was allocated for students to engage with the work of their peers: For example, in the lessons at the mid and end points of the unit students were asked to leave cartoons on their desks for viewing: Kate: “. . . make sure that your one frame cartoon is very visible because in a second I’d like everyone to wander around very carefully, don’t hide them, I want them to be in view and I want you to go around and have a look and see what people have done so far. That’s other people in the class . . . What sort of things do you think I want you to be looking for as you look at what other people have done? Student 1: They have a hidden message. Student 2: Got visuals. Kate: Yes, like what sort of things for their visuals?” (Obs).

4

Int – refers to Phase one Interview data.

74

E.M. Hawe, H.R. Dixon / Assessing Writing 19 (2014) 66–79

Mindful that students needed to have a focus for their perusal of others’ work, Kate drew attention to elements of performance that comprised the rubric descriptor ‘professionally and effectively used visual traits’: “They used techniques with text, they used techniques with the visuals and they used techniques with the deeper meaning . . . well done”. (Obs). Planned whole class opportunities to view others’ work were not the sole means through which students enlarged their evaluative knowledge and expertise. They were also encouraged to access “what other kids in the class are doing” (PO Int) individually, informally and independently. Students were urged on numerous occasions to talk about specific aspects of their work with peers, and were observed taking advantage of this opportunity. At different points in time during one particular lesson, students spontaneously took their work to a peer/peers so they could discuss it together. Animated conversation followed as one group focused on visual and textual features of the cartoon under discussion. Knowledgeable of who was on the right track, Kate also took the opportunity to direct those in need of support or ideas, to specific peers: “If I was you, can I just suggest you have a little walk around and ask people what they’re doing because I think you might need a few more ideas, I think you’re a bit stuck . . . Have a little look around. Start with this table here. See what Jacinta’s doing and Jordan’s got a really, really good idea. Jordan was starting like you yesterday . . .. [but] now he’s come up with a really clever idea. Have you spoken to Jordan about what he’s doing?” (Obs). “Constantly seeing what other kids in the class are doing” (Int) provided a concrete example of what could be achieved and a means through which students could modify and improve their work. In Kate’s opinion, these viewings enabled students to think “Oh I can do that, I can do that” (PO Int). By providing students with sustained experiences to see the work of others during various stages of production, Kate’s students gained insight into the moves and strategies used by peers and they were then able to use this knowledge to make judgments about, and improvements to, their own works-in-progress. At the end of the unit, through completion of a formal evaluation, students were asked to make multi-criterion judgments about their works and the works of others. As they participated in a self and peer activity, students were again faced with the complexity of the evaluative process. At the start of the lesson possible differences between one level of attainment and another were brought to students’ attention, as was the need to look at work in a holistic manner. To further emphasize the challenges inherent in making judgments, the class was brought together to discuss notions of quality contained within the criteria that were easily identifiable, those that were more difficult to ascertain, and those apparent in the cartoons but not overtly captured in the rubric. The importance of looking for evidence within a piece of work that related to the judgement and remembering not to be swayed by criteria that were easily identifiable were other issues drawn to students’ attention. As Kate later explained, it was important for students to remember and understand “that it [an appraisal] took all three [criteria and in combination] . . .” (PO Int). To complete the self-assessment part of the activity, students were asked to provide a “little general comment” (Obs) unrelated to the rubric criteria, thus evoking other qualities that may have been of particular significance in regard to their work, progress or achievement. The complexity of the evaluative activity was further highlighted when students were asked in pairs to make a formal appraisal of peers’ work and compare these judgments against the selfassessments. Students were observed debating and discussing judgments made about the quality of work produced, providing justifications for their judgments. Following these deliberations, in light of arguments presented and the careful consideration of others’ work, some students revised the original assessment of their own work.

4.3. The case of Marama Marama aimed to help her Year eight students negotiate the writing process and improve their writing. Therefore throughout teaching of the poetic writing unit she drew attention to the poetic form and processes involved in production of a poem. In contrast to Audrey, a full set of learning goals

E.M. Hawe, H.R. Dixon / Assessing Writing 19 (2014) 66–79

75

and/or success criteria was not specified at the beginning of the unit. Rather, as the unit progressed these emerged in association with each stage of the writing process. For instance, at the start of the unit the goal was “to effectively prepare for poetry writing” (Obs) whereas when a majority of the class had completed drafts of their poems, two new goals were introduced – “to edit and rework” and “to critique my own writing” (Obs). To develop students’ understanding of these goals, success criteria were established and discussed with reference to surface and deep features. Cognisant that in the past students had often attended to editing and correcting superficial spelling and grammatical errors, Marama believed it important to promote understanding of the differences between re-working and editing a piece of writing. As she noted: “I mean the kids are quite good at getting their own ideas down. . . . but re-working is a huge area that they just don’t do. To them, re-working is checking the spelling, which in actual fact is more like editing . . .” (PO Int). Consequently considerable time was spent providing students with opportunities to gain a shared understanding of the deeper features that contribute to the quality of a piece of poetic writing. Partway through the unit Marama asked students to work in small groups and make judgments about a poem, written by a student of a similar age from another class, with reference to the “use of similes, alliteration; rhyme; repetition; metaphor” (Obs). Following a brief recapping of the nature of these poetic devices and what they might look like in a piece of work, students looked for evidence of each in the poem. They then shared their judgments with the class, along with the evidence supporting this judgement. Following this activity students critiqued their own works-in-progress in a similar manner, highlighting evidence of the five devices and making annotations about their use. These annotations formed the basis for further discussion with peers where the focus was on aspects of the poetic devices that required further development. During this time Marama sat with groups and listened to discussions, participating when and where she deemed appropriate. Rather than directing students to make changes to their work, she posed questions and drew attention to salient features of their work. As Marama left one of the groups, three of the students got involved in a conversation lasting more than ten minutes about one student’s draft. Focused on the criteria that had formed the basis of the critique, a substantive discussion took place about the writer’s decision to use similes as opposed to metaphors. After seeking permission from the writer, one group member marked on the draft where she thought changes might be made with a view to improvement. This then led to a discussion about the importance of using the most appropriate descriptive words when constructing similes and/or metaphors. These three students then spent some time using a thesaurus as they looked up, discussed and debated which words were the most suitable to use in their poems. Marama aimed to build “an atmosphere where the kids value what others have to contribute” and where “constructive criticism and the benefits of it” (PO Int) were brought to students’ attention. During group work, she reiterated the importance of students talking about their work and making public aspects that were causing difficulty. Students were encouraged to reveal to their peers “what you’re finding difficult . . . what sort of trouble you are experiencing” (Obs) with the intent of getting help to overcome the identified difficulties. Students responded to this call as they provided advice and feedback to peers not only as part of the formal group-work opportunities led by Marama, but also during individual writing time.

5. Discussion AfL is comprised of a set of inter-dependent strategies that teachers employ during the regular flow of teaching and learning with the dual objectives of supporting and furthering student learning, and developing autonomous, self-regulating learners (Hawe & Parr, 2013; Swaffield, 2011; Willis, 2011). These strategies entail the promotion of students’ understanding about the goal(s) of learning and what constitutes expected performance, generation of feedback by students and their teachers about the relationship between current and desired performance, student engagement in peer feedback and self-monitoring, and the taking of action by students to effect improvement (James & Pedder, 2006; Swaffield, 2011). From a socio-cultural perspective the strategies of AfL are considered “culturally

76

E.M. Hawe, H.R. Dixon / Assessing Writing 19 (2014) 66–79

situated patterns of participation [own emphasis] that allow students to develop a sense of belonging as an insider in the practice, while developing an identity of an autonomous learner. . .” (Willis, 2011, p. 402). Thus AfL necessitates a need for students and teachers to take on board new norms of behaviour as each is assigned radically changed roles and responsibilities. Students are no longer “the objects of their teacher’s behaviour, [rather] they are animators of their own effective teaching and learning processes” (James & Pedder, 2006, p. 28). In turn, teachers are charged with the responsibility for helping students acquire the reflective habits of mind necessary to become autonomous and self-regulating learners. If students are to judge performance in relation to goals, construct internal information about progress towards goals, and take action based on that feedback (Butler & Winne, 1995) teachers must have the volition and ability to share their tacitly held guild knowledge with students so that they come to hold a concept of quality generally comparable to that held by the teacher (Sadler, 1989). One of the most effective ways for students to become ‘insiders’ in the practice of writing and develop identities as autonomous writers is through involvement in the creation, evaluation and revision of texts during production. Teachers must therefore be committed to providing students with substantive and authentic opportunities for self and peer assessment during learning so students, over time, are apprenticed into full membership of the writing community (Marshall, 2004). Critical to the development of students’ evaluative and productive expertise is an understanding of the goals of learning and what constitutes quality in a piece of writing (Sadler, 1989, 2009). The ways in which learning goals/intentions, success criteria, rubrics and the like are framed influence students’ understandings of writing and the writing process, and direct their behaviour (Hawe et al., 2008; Timperley & Parr, 2009). When goals are expressed as topic or genre based tasks, as evident in the case of Audrey, they draw attention to task completion rather than the more substantive elements of a piece of writing and the writing process (Hawe & Parr, 2013; Timperley & Parr, 2009). In addition, when success criteria, rubrics and the like are focused on a pre-specified list of elements, they function as checklists where quality is equated with the presence of each element in the work (Hawe et al., 2008; Hyland, 2000). Under such circumstances it is not surprising to find teachers and students adopting an analytic approach to the appraisal of texts where presence of the manifest criteria is presumed sufficient for all cases (Sadler, 1989, 2009). Furthermore, there is an assumption that criteria do not necessarily overlap or work together. As a consequence, students adopt a convergent, mastery approach to feedback and learning (Hawe et al., 2008; Torrance & Pryor, 1998), carefully scrutinizing their work to check whether the required elements are present. Students cannot be blamed for thinking they have been successful in their work once each element is ‘ticked off’ as present – they have been inducted into the notion that quality resides in the presence of properties identified in the criteria. Being bound by confining goals and poorly expressed criteria that highlight individual elements restricts notions of quality. Writing is a complex activity – it is about the use of language as an art and the process of crafting a quality product (Marshall, 2004; Sadler, 1989). Rather than using precise goals and a standard set of pre-determined criteria to define progress and success it is more appropriate to construct progress and success as movement towards a ‘horizon of possibilities’ (James, 2008). As seen in the cases of Kate and Marama, this horizon becomes clearer and sharper as the writing unit develops. Moreover, there are opportunities during writing for teachers and students to highlight and co-construct differentiated expressions of progress and success, drawing not only on the cognitive aspects of learning but those related to the conative domain, in particular skills concomitant with goal setting, meta-cognition, self-monitoring and self-regulation (Huitt & Cain, 2005; Prawat, 1985). As evident in Audrey’s practice, it is problematic when students are introduced in the final stages of writing to new properties and more complex notions of quality, without explanation. Teachers need to share their tacit knowledge about quality writing at all stages of the writing process so students can become attuned to “the latent-to-manifest translation process, and the limitations inherent is using a fixed set of criteria” (Sadler, 2009, p. 58). As shown in the case of Kate, students need to be alerted to the notion that additional criteria of significance emerge from (or can be noted as absent from) a piece of work, encouraged to consider the ways in which criteria worked configurally to produce quality, make judgments with reference to the whole as well as the parts, and justify their appraisals (Sadler, 1989, 2009). Collectively these factors mirror the way holistic multi-criterion judgments are typically formulated by experienced teachers, experts and non-experts alike (Sadler, 2009). Seemingly Kate

E.M. Hawe, H.R. Dixon / Assessing Writing 19 (2014) 66–79

77

and Marama inducted students into relatively sophisticated understandings about what constitutes quality in a piece of work, bringing them “at least partly into the guild of professionals who are able to make valid and reliable appraisals of complex works using all the tools at their disposal” (Sadler, 2009, p. 57). Students in these teachers’ classes were then well positioned to evaluate complex works in ways that have considerable practical capacity for improving learning. Students develop their evaluative and productive knowledge and expertise as they participate in authentic appraisals of their own work and the work of their peers. Understanding what constitutes quality in texts is ‘caught’ through first hand experience in the evaluation of such works (Sadler, 2010). Despite the importance ascribed to student involvement in their learning, research indicates involvement in the context of written language is yet to be fully realized as many teachers hold on to a traditional, teacher-centred style of instruction (Dixon et al., 2011; Hawe & Parr, 2013; Marshall & Drummond, 2006). As illustrated in the case of Audrey, students may be called upon to appraise works-in-progress, but in reality these appraisals are of little consequence as the teacher retains control over the appraisal process, and the nature and scope of the feedback. An authoritative style of instruction relegates students to a peripheral, passive role in their learning. It denies access to the ‘guild knowledge’ (Sadler, 1989) that enables them to make sound judgments about the quality of their works-in-progress and areas for improvement. Over time, teacher controlled, pseudo-evaluative activities result in students’ deferral to and reliance on the teacher for judgments about their work and how to make improvements (Hawe et al., 2008; Sadler, 1989). This deferral and dependency, which teachers are often not aware of, reflects an underlying assumption that only they have the expertise to generate feedback about students’ work and a belief this expertise is not readily transferable. In contrast to Audrey, both Kate and Marama viewed students as ‘insiders’ and as autonomous writers. Their students participated in “teaching and learning processes and decision making [through the adoption of] pedagogical practices [that] further[ed] their own learning and that of their peers” (James & Pedder, 2006, p. 28). Participation in authentic and sustained evaluative experiences not only facilitate deep engagement with subject matter, these experiences also initiate students into the ‘guild knowledge’ of the writing community (Marshall, 2004; Sadler, 1989). These two teachers understood that development of student independence and autonomy is facilitated through a discourse structure where power is shared with students. Establishing such a structure acknowledges that students have valid insights into their own writing, and promotes meta-cognitive reflection, a skill critical to developing students’ understandings about writing (Ward & Dix, 2004). Through dialogic exchanges students and teachers jointly construct achievement and the way forward in ways that promote skills and understandings critical to developing students’ writing (Ward & Dix, 2004). Together, teachers and students establish what they as a community of writers value. Students are cast as partners in the learning process as opposed to passive automatons who respond to their teacher’s directives. Formal and informal opportunities for peer assessment, peer response and self-monitoring need therefore to be deliberately embedded into writing lessons. Through these activities students are exposed to a wide range of authentic works from the genre they are producing and have access to writing across the full spectrum of quality, conditions necessary for the development of evaluative knowledge and productive expertise (Sadler, 2009, 2010). As students build up this body of knowledge and expertise they become ‘insiders’, moving from dependence on teacher-supplied information to a state of ‘intelligent self monitoring’ (Sadler, 1989). 6. Conclusion In its fullest expression AfL gives students a central role in the learning process through authentic evaluative and productive experiences embodied in peer feedback and self-monitoring. Although the three teachers in this study drew on the same strategies when teaching their units, they enacted these in qualitatively different ways. In practice these differences had a significant impact on the nature and quality of the writing experiences provided for students and the role students played in their learning. In a writing classroom, the aim is to move away from students being told by their teachers about the quality of their work, towards an environment where they see and understand the reasons for quality “and in the process develop personal capability in making complex judgments” (Sadler, 2010, p. 546). The role of the writing teacher is to create and shape the conditions that support students in this endeavour.

78

E.M. Hawe, H.R. Dixon / Assessing Writing 19 (2014) 66–79

The cases presented in this paper show that developing students’ evaluative knowledge and productive skills in writing is not as simple as attaching individual AfL strategies such as peer feedback and student self-assessment to existing programmes (Dixon et al., 2011; Marshall & Drummond, 2006). Rather it involves adoption of AfL as a unitary notion and a radical transformation of the traditional taken-for-granted roles and responsibilities of teachers and students.

References Askew, S., & Lodge, C. (2000). Gifts, ping-pongs and loops – Linking feedback and learning. In S. Askew (Ed.), Feedback for learning (pp. 1–18). London: RoutledgeFalmer. Broad, B. (2003). What we really value: Beyond rubrics in teaching and assessing writing. Logan, UT: Utah State University Press. Butler, D. L., & Winne, P. H. (1995). Feedback and self-regulated learning: A theoretical synthesis. Review of Educational Research, 60(3), 245–274. Clarke, S. (2001). Unlocking formative assessment. Abingdon: Hodder & Stoughton. Delamont, S. (1992). Fieldwork in educational settings. Methods, pitfalls and perspectives. London: The Falmer Press. Dixon, H. R. (2011). Infusing peer assessment into classroom programmes: Descriptions of practice. SET: Research Information for Teachers, 2, 3–10. Dixon, H. R., Hawe, E. M., & Parr, J. M. (2011). Enacting assessment for learning: The beliefs-practice nexus. Assessment in Education: Principles, Policy & Practice, 18(4), 365–379. Ezzy, D. (2002). Qualitative analysis. Crows Nest, NSW: Allen & Unwin. Fitzgerald, J. (1987). Research on revision in writing. Review of Educational Research, 57(4), 481–506. Gardner, J. (2006). Assessment and learning: An introduction. In J. Gardner (Ed.), Assessment and learning (pp. 1–5). London: SAGE Publications. Glaser, B. G., & Strauss, A. L. (1967). The discovery of grounded theory: Strategies for qualitative research. New York: Aldine de Gruyter. Graves, D. (2003). Writing: Teachers and children at work (2nd ed.). Exeter, NH: Heinemann Educational Books. Gunnery, S. (2007). The writing circle. Markham, ON: Pembroke. Hattie, J., & Gan, M. (2011). Instruction based feedback. In R. E. Mayer, & P. A. Alexander (Eds.), Handbook of research on learning and instruction (pp. 249–271). London: Routledge. Hattie, J., & Timperley, H. (2007). The power of feedback. Review of Educational Research, 77(1), 81–112. Hawe, E., Dixon, H., & Watson, E. (2008). Oral feedback in the context of written language. Australian Journal of Language & Literacy, 31(1), 43–58. Hawe, E. M., & Parr, J. M. (2013). Assessment for learning in the writing classroom: an incomplete realisation. The Curriculum Journal, 1–27. http://dx.doi.org/10.1080/09585176.2013.862172 Hout, B., & Perry, J. (2009). Toward a new understanding for classroom writing assessment. In R. Beard, D. Myhill, J. Riley, & M. Nystrand (Eds.), The SAGE Handbook of writing development (pp. 423–436). Los Angeles: SAGE Publications. Huitt, W., & Cain, S. (2005). An overview of the conative domain. In Educational Psychology Interactive. Valdosta, GA: Valdosta State University. Retrieved from http://www.edpsycinteractive.org/papers/conative.pdf Hyland, F. (2000). ESL writers and feedback: Giving more autonomy to students. Language Teaching Research, 4(1), 33–54. James, M. (2008). Assessment and learning. In S. Swaffield (Ed.), Unlocking assessment: Understanding for reflection and application (pp. 20–35). Abingdon: Routledge. James, M., & Pedder, D. (2006). Beyond method: Assessment and learning practices and values. The Curriculum Journal, 17(2), 109–138. Keen, J. (2010). Strategic revisions in the writing of Year 7 students in the UK. The Curriculum Journal, 21(3), 255–280. Klenowski, V. (2009). Assessment for learning revisited: An Asia-Pacific perspective. Assessment in Education: Principles, Policy & Practice, 16(3), 277–282. MacLure, M. (2003). Discourse in educational and social research. Buckingham: Open University Press. Marshall, B. (2004). Goals or horizons – The conundrum of progressions in English: Or a possible way of understanding formative assessment in English. The Curriculum Journal, 15(2), 101–113. Marshall, B., & Drummond, M. J. (2006). How teachers engage with assessment for learning: Lessons from the classroom. Research Papers in Education, 21(2), 133–149. Merriam, S. (1998). Qualitative research and case study applications in education. San Francisco: Jossey Bass. Nicol, D., & Macfarlane-Dick, D. (2006). Formative assessment and self regulated learning: A model and seven principles of good feedback practice. Studies in Higher Education, 31(2), 199–218. Paris, S. G., & Paris, A. H. (2001). Classroom applications of research on self-regulated learning. Educational Psychologist, 36(2), 89–101. Parr, J. M. (2011). Writing in the curriculum. A complex act to teach and to evaluate. In C. Rubie-Davies (Ed.), Educational Psychology: Concepts, research and challenges (pp. 51–67). London: Routledge. Parr, J. M. (2013). Classroom assessment in writing. In J. H. McMillan (Ed.), SAGE handbook of research on classroom assessment (pp. 489–501). Thousand Oaks: SAGE Publications, Inc. Parr, J. M., & Limbrick, E. (2010). Contextualising practice: Hallmarks of effective teachers of writing. Teaching and Teacher Education, 26, 583–590. Prawat, R. (1985). Affective versus cognitive goal orientations in elementary teachers. American Educational Research Journal, 22(4), 587–604. Sadler, D. R. (1989). Formative assessment and the design of instructional systems. Instructional Science, 18, 119–144. Sadler, D. R. (1998). Formative assessment: Revisiting the territory. Assessment in Education: Principles, Policy & Practice, 5(1), 77–84.

E.M. Hawe, H.R. Dixon / Assessing Writing 19 (2014) 66–79

79

Sadler, D. R. (2007). Perils in the meticulous specification of goals and assessment criteria. Assessment in Education: Principles, Policy & Practice, 14, 387–392. Sadler, D. R. (2009). Transforming holistic assessment and grading into a vehicle for complex learning. In G. Joughin (Ed.), Assessment, learning and judgement in higher education (pp. 45–64). New Jersey: Springer. Sadler, D. R. (2010). Beyond feedback: Developing student capability in complex appraisal. Assessment & Evaluation in Higher Education, 35, 535–550. Schwandt, T. A. (2000). Three epistemological stances for qualitative inquiry. In N. Denzin, & Y. Lincoln (Eds.), Handbook of qualitative research (2nd ed., pp. 189–213). Thousand Oaks: Sage Publications. Stake, R. (2000). Case studies. In N. Denzin, & Y. Lincoln (Eds.), Handbook of qualitative research (pp. 435–454). Thousand Oaks: SAGE Publications, Inc. Swaffield, S. (2011). Getting to the heart of authentic Assessment for learning. Assessment in Education: Principles, Policy & Practice, 18(4), 433–449. Timperley, H. S., & Parr, J. M. (2009). What is this lesson about? Instructional processes and student understandings in writing classrooms. The Curriculum Journal, 20(1), 43–60. Torrance, H., & Pryor, J. (1998). Investigating formative assessment. Teaching, learning and assessment in the classroom. Buckingham: Open University Press. Ward, R., & Dix, S. (2004). Highlighting children’s awareness of their texts through talk. SET: Research Information for Teachers, 1, 7–11. Retrieved from http://www.nzcer.org.nz/nzcerpress/set/set-2004-no-1 Willis, J. (2011). Affiliation, autonomy and Assessment for learning. Assessment in Education: Principles, Policy & Practice, 18(4), 399–415. Xiang, W. (2004). Encouraging self-monitoring in writing by Chinese students. ELT Journal, 58(3), 238–246. Zellermayer, M. (1989). The study of teachers’ written feedback to students’ writing: Changes in theoretical considerations and the expansion of research contexts. Instructional Science, 18, 145–165. Eleanor M. Hawe is a principal lecturer in the Faculty of Education. Her research interests lie in assessment for learning across a range of educational contexts and teaching subjects. Recent publications have addressed teachers’ oral feedback during writing lessons, intra and inter-assessor reliability when administering social studies interviews and teachers’ implementation of AfL. Helen R. Dixon is a principal lecturer in the Faculty of Education. Her teaching and research interests are focused on assessment for learning, feedback and how teacher beliefs influence assessment practice. Her doctoral study which won a national award, examined teachers’ conceptions and use of feedback to enhance students’ learning.