Reflection as a visible outcome for preservice teachers

ARTICLE IN PRESS

Teaching and Teacher Education 20 (2004) 243–257

Reflection as a visible outcome for preservice teachers John R. Warda,*, Suzanne S. McCotterb a b

Millersville University, 107 Stayer, Millersville, PA 17551, USA Millersville University, 173 Stayer, Millersville, PA 17551, USA

Abstract As the standards movement progresses, efforts to encourage reflection by student teachers are often undermined. In this piece, we analyze exemplars of student teacher reflection coming from two very different approaches to outcomesbased teacher preparation. We use these exemplars to develop a rubric that illuminates the dimensions and qualities of reflection. This rubric helps clarify how meaningful reflection and an emphasis on learning are not incompatible if the focus is placed on the process of learning, rather than on outcomes alone. Finally, we contend that engagement in the process of reflection and reflection on the moral enterprise of teaching can be considered as important outcomes in their own right. r 2004 Elsevier Ltd. All rights reserved. Keywords: Critical thinking; Reflective teaching; Teacher effectiveness; Work sample tasks; Evaluation methods; Student teacher evaluation; State standards; Academic achievement; Preservice teacher education; Scoring rubrics

1. Introduction

TE:

Like many teacher educators, we have had frequent conversations with novice teachers about the lessons they have taught in field experiences and student teaching. Too frequently, these discussions turn into interrogations that sound something like this:

PT: TE: PT: TE: PT:

Teacher educator (TE): Preservice teacher (PT):

Why was this important for students to learn about? I needed to cover this material for the state test in two weeks. The whole school is getting ready for the tests.

*Corresponding author. Tel.: +1-717-872-3835. E-mail addresses: [email protected] (J.R. Ward), [email protected] (S.S. McCotter).

Do you think they learned what you wanted them to? Yes, definitely. How do you know? Well, they haven’t taken the test yet, but they didn’t have any questions. What would you change if you could teach it again? Well I would like to have developed this lesson further and teach more for understanding, but I don’t think I could have changed my approach because I have to cover this material to get students ready for the test.

Our well-intentioned questions designed to provoke deliberate thought about student learning and facilitate self-improvement fall flat. Pressing concerns about standardized tests replace and undermine thoughtful questions about teaching

0742-051X/$ - see front matter r 2004 Elsevier Ltd. All rights reserved. doi:10.1016/j.tate.2004.02.004

ARTICLE IN PRESS 244

J.R. Ward, S.S. McCotter / Teaching and Teacher Education 20 (2004) 243–257

and learning. We are perpetually challenged to get our students to reflect on their practice in meaningful ways, to consider the effect their teaching has on student learning, and develop habits that will stay with them. The increasing prevalence of standards, high-stakes testing, and outcomes assessment obscure the value of reflection, and much else, from fields of vision. In many states, district-wide performance on standardized tests is linked to school funding and governance. The Bush administration’s ‘‘No Child Left Behind’’ Act mandates annual testing by the 2005–2006 school year, and states that ‘‘If the district or school continually fails to make adequate progress toward the standards, then they will be held accountable.’’ (www. nochildleftbehind.gov). Knowing that test scores will be a visible and public indicator of perceived quality makes it difficult to see beyond the graphs and tables on the front pages of newspapers. The standards which are so widespread in basic education have been extended to teacher education programs (Cochran-Smith, 2000). A fundamental shift from an input to an output model of evaluation is taking place in the field. It is no longer enough for teacher preparation programs to demonstrate that their candidates have the knowledge, skills, and dispositions associated with effective teachers; teacher preparation programs must now demonstrate that their candidates make a positive impact on student learning (NCATE, 1999). We are all being asked to critically analyze student work in terms of how it is meeting standards. The standards movement raises both challenges and opportunities for the fields of teaching and teacher education. Beyond the oft-cited disadvantages of academic standards leading to standardization of teaching, and the danger of relying too heavily on standardized tests as the sole measure of mastery, there is a risk that the value of teacher reflection will be diminished and overwhelmed by standards. The value of reflection to the development of teachers has a growing consensus among teacher educators, although there is still some debate over the way in which reflection is defined.

For this study, we find the definition posited by Hatton and Smith (1995) to be the most straightforward and useful: ‘‘[Reflection is] deliberate thinking about action with a view to its improvement’’. As teacher educators, we place a high value on reflective thinking and practice that supports preservice teachers in broadening perspective and developing concern for others. We want preservice teachers to think about the moral enterprise of teaching and what is ‘‘taken-for granted’’. Unfortunately, standards are often viewed as closing the door on the need to ask questions about the curriculum, questions that may raise those moral concerns. Further, the process of dialogue and questioning that is at the heart of reflection is often perceived as conflicting with the ‘‘coverage’’ mentality of a standardized environment (Ketter & Pool, 2001). This is particularly true for preservice teachers. When they join the profession as first year teachers they will be immersed in the pressures of standards-driven curriculum and closely examined student outcomes. How will the habits of reflection and questioning survive under these conditions? The first specific challenge posed by the standards movement focuses on the purpose of reflection. Do standards and, more specifically, the tests that measure standards define and limit what is meant by improvement of practice? Is reflection nothing more than a tool for helping teachers increase student test scores or is there value in reflection on broader themes such as social justice? Another challenge relates to the heightened value placed on outcomes that are easily measured. As teacher educators we are facing calls to demonstrate the ‘‘value-added’’ by our work. Will the habits of reflection that we seek to develop in future teachers become devalued simply because they are difficult to evaluate, summarize, and report? The emphasis on student learning related to standards presents an opportunity, however, when it becomes the very fabric of reflection, rather than the barrier that precludes it. In fact, teacher examination of student work and student learning

ARTICLE IN PRESS J.R. Ward, S.S. McCotter / Teaching and Teacher Education 20 (2004) 243–257

can be an excellent vehicle for reflection. While single-minded attention to high-stakes tests may undermine the reflective process, a rigorous emphasis on student learning offers an opportunity for broadening the reflective focus of preservice teachers. Expecting preservice teachers to relate their teaching behaviors to student learning rather than to their own performance represents an important shift from a focus on self to a focus on others. This shift is reminiscent of Fuller’s (1969) conceptualization of the stages that preservice teachers go through in transitioning from college students to classroom teachers. In fact, examining the nature of student learning is the very form of professional development that can effectively increase student learning (Meier, 2000; Mitchell, 1996; National Commission on Teaching & America’s Future, 1996). Accomplishing this, however, means resisting the temptation to only ask the simplistic question, ‘‘Did students do well on the test?’’ One way we have begun to answer these challenges is by development of a rubric for evaluating teacher reflection. This rubric is designed to address three related questions about preservice teacher reflection that takes place within today’s standards-based educational systems. First, what are the qualities that distinguish more and less meaningful reflection? Second, how are these qualities related to a focus on student learning and student learning outcomes? Finally, how can we describe these qualities in such a way as to make them visible and valued outcomes in their own right? In the following sections we will describe our process for developing the rubric and present the rubric along with samples of preservice teacher reflective writing exemplifying different levels of the rubric. We started the development process by examining existing frameworks for describing the qualities of reflection. We found these frameworks useful for developing guiding principles for evaluating the quality of reflection. None of the existing frameworks, however, were sufficient for answering all three of our questions. In general, existing reflection frameworks are not sufficient for evaluating reflection, especially for evaluating reflection on student learning.

245

2. Reflection frameworks The essential quality of reflection is thinking about practice in order to improve (Hatton & Smith, 1995). This is clear in Dewey’s (1933, p. 17) justification of the importance of reflection for education, ‘‘By putting the consequences of different ways and lines of action before the mind, it enables us to know what we are about when we . expanded on Dewey’s work by act.’’ Schon differentiating reflection-in-action from technical rationality or the application of scientific princi. (1983), reflection is ples to practice. For Schon bound up with practice; teachers and other professionals understand and act on their situations in ways that cannot be reduced to rules or described fully by educational theory. This early work has led to a widely accepted understanding that reflection is an important process for teachers (Darling-Hammond & Snyder, 2000; Valli, 1992; Zeichner, 1996). Recent frameworks for reflection build on the . theories of Dewey and Schon, and include common process elements that owe a debt to early advocates for reflection. Dewey’s focus on reflection situated in practice is a typical characteristic of reflection in teaching and teacher education (Collier, 1999; Korthagen & Kessels, 1999; Loughran, 2002; Reiman, 1999). The tendency to reflect about problems in that practice, and to ‘‘frame and reframe’’ those problems over time was . (1983), but has been originally discussed by Schon echoed in other frameworks (e.g., Clarke, 1995; Korthagen, 1999; Loughran, 2002; Reiman, 1999; Stanley, 1998). The notion that problems cannot be easily solved lends a cyclical nature to many models of reflection (Clarke, 1995; Korthagen, 1999; Reiman, 1999; Stanley, 1998). Seeking other viewpoints, or multiple perspectives, to gain insight on problems is another common element of reflective practice (Collier, 1999; Hatton & Smith, 1995; Loughran, 2002; Rearick & Feldman, 1998). The common elements of many frameworks of reflection that we have identified (reflection is situated in practice, is cyclic in nature, and makes use of multiple perspectives) have also been applied in much of the research on how to improve the quality of teacher reflection. Several methods,

ARTICLE IN PRESS 246

J.R. Ward, S.S. McCotter / Teaching and Teacher Education 20 (2004) 243–257

such as action research (Dinkelman, 1997; Yost, Sentner, & Forlenza-Bailey, 2000; Zygouris-Coe, Pace, Malecki, & Weade, 2001), and cases (Gillespie, 1996; Harrington, Quinn-Leering, & Hodson, 1996) pointedly make use of the idea that reflection should be situated in practice. Methods such as logbooks and journals (Francis, 1995; Korthagen, 1999; Reiman, 1999) emphasize the ongoing cyclic nature of meaningful reflection. Other methods such as peer interviews and conferences (Collier, 1999) emphasize consideration of multiple perspectives. Many of these strategies, in fact, make use of all three of these qualities. In addition to qualities of reflection that are valued both from a theoretical and an applied perspective, there has been much work aimed at describing developmental or hierarchical qualities of reflection (Collier, 1999; Dinkelman, 2000; Galvez-Martin, Bowman, & Morrison, 1998; Hatton & Smith, 1995; Kitchener & King, 1981; Rearick & Feldman, 1998; Ross, 1989). In these studies, lower levels of reflection are identified by the absence of evidence of one or more qualities ascribed to higher levels (situated, cyclic, using multiple perspectives, and consideration of morally important questions). Low levels of reflection have been described as Technical (Collier, 1999), Routine (Yost et al., 2000), or simply descriptive writing (Hatton & Smith, 1995). Higher reflection is classified based on evidence of these qualities, and is often called critical reflection (Collier, 1999; Dinkelman, 2000; Hatton & Smith, 1995; Yost et al., 2000). There are missing elements from the existing frameworks, however, which make them difficult to use for evaluating the quality of preservice teacher reflection. A common shortcoming of these frameworks as tools for evaluation is a lack of attention to how teachers situate their thinking within the context of their practice (Kitchener & King, 1981; Ross, 1989). Other frameworks are incident-based, which has the advantage of being situated in practice, but misses the ongoing, cyclical aspect of framing and reframing problems (Galvez-Martin et al., 1998). Still other frameworks include many of the general principles about the process of reflection, but lack a method

to either practically use them in evaluating the reflection of teachers (Stanley, 1998; Clarke, 1995) or to evaluate both the qualities and dimensions of reflection (Hatton & Smith, 1995; Korthagen, 1999; Korthagen & Kessels, 1999). Although these frameworks have limitations, they do make some aspects of reflection visible, and are potentially helpful in working with preservice teachers. The most significant problem to date, however, is that existing frameworks do not address the challenge of outcomes-based education or deal seriously with student learning. They are designed to describe a process, but not designed to identify or recognize the qualities of reflection that are related to improvement of practice. The missing link for reflection on teaching is a tool for the comprehensive assessment of reflection that gives shape to the general principles of reflection, helps teachers visualize how reflection can improve their practice, and explicitly links reflection to student learning. In the next section we describe how we used samples of preservice teacher reflection on student learning along with general principles about the process of reflection to develop such a system.

3. Methodology After reviewing the literature on qualities of reflection and recognizing that there was no comprehensive framework for the evaluation of reflection it was important for us to focus on our data. We utilized a grounded theory approach to develop our rubric. Grounded theory is a controlled, systematically inductive approach to development of theory from data (Strauss & Corbin, 1990). 3.1. Data sources We used two rich sources of preservice teacher reflection on student learning as our data, seven exemplars from the Renaissance Teacher Work Sample (TWS) and six exemplars from a qualitative approach (Collaborative Inquiry: Reflection, Questions about Student Learning—CIRQL). TWS and CIRQL products are essentially

ARTICLE IN PRESS J.R. Ward, S.S. McCotter / Teaching and Teacher Education 20 (2004) 243–257

enhanced and improved unit plans. Each approach requires student teachers to plan and teach a unit of instruction along with a well-balanced assessment plan. Student teachers use their assessments as a basis for reflection on their teaching. The resulting student teacher products contain approximately 15–20 pages of reflective text along with lesson plans and samples of student work. These two sources were selected because they represent contrasting approaches to reflection on student learning. The TWS places greater emphasis on summative assessment and calculation of gain scores using pre- and post-assessment. CIRQL emphasizes formative assessment an ongoing reflection. The exemplars we choose were the culminating work of 13 different student teachers from 11 different teacher education institutions. The 13 units we used as our data sources were freely available online1 for purposes of training student teachers and their mentors in each of the respective methodologies. The TWS and CIRQL are methodologies developed primarily at public universities with grant funding. The TWS is supported by the Renaissance Partnership, a national consortium of teacher education programs, funded with a federal Title II grant. CIRQL was developed with the support of a National Science Foundation Collaborative Level II grant. The TWS, used for teacher licensing purposes in Oregon since 1986, has been promoted by both the National Council for Accreditation in Teacher Education (NCATE) and the American Association of Colleges for Teacher Education (AACTE) (Girod, 2002; The Newsletter of the National Council for the Accreditation of Teacher Education, 2000; Schalock, Schalock, & Myton, 1998). As a methodology, the TWS asks student teachers to clearly align learning goals, instruction, and assessment with standards; analyze gains in student learning using pre- and post-assessment measures; and reflect on their teaching given 1

Examples of TWS student teacher units may be viewed online at The Renaissance Partnership for Improving Teacher Quality (2002) website. Examples of CIRQL student teacher units are available online through the Collaborative Inquiry– Reflection, Questions about Learning (2003) website.

247

student learning gains. By requiring samples of both the teacher’s work and the students’ work, the TWS provides a structured way for novice teachers to meaningfully connect the two (Schalock & Myton, 2002). Pre- and post-assessment of students is used as a basis for measuring effectiveness: ‘‘yteacher work sampling highlights learning gains made by pupils in relation to outcomes desired from a unit of instruction. In this context, TWS measures devised by the student are used to assess learning before and after instruction’’ (Girod, 2002, p. 11). The focus on standards is apparent both in instruction, where a shift from coverage to outcomes is counseled, as well as in assessment, where clear alignment between measures and outcomes is advocated. Like TWS, CIRQL products involve a unit plan, along with robust analysis of student learning. Rather than using pre–post-test scores as the basis for analysis of student learning as in the TWS, student teachers working under the CIRQL model analyze the learning process of a few students using a case study approach. Student teachers completing a CIRQL unit are asked to use formative assessments on a daily basis for ongoing reflection. This varying selection of data sources allowed us to engage in theoretical sampling (Preissle, Tesche, & LeCompte, 1993), as we developed the theory from our data. 3.2. Analysis of reflective writing Our process of using the data to build the theory demonstrated four key procedural elements of grounded theory coding: fracturing data (identifying reflective ‘‘chunks’’ within lengthy reflective units) to gain higher levels of abstraction, using generative questions, moving toward the discovery of core categories, and ultimately integrating the entire analysis and yielding conceptual density (Strauss, 1987). Our first step in analyzing the data was to identify reflective ‘‘chunks’’ from the TWS exemplars. This gave us the first procedural element of grounded theory coding, fracturing the data. Chunks were identified as writing on a particular theme or topic. The end of a chunk was identified by a change in the focus of reflection without a

ARTICLE IN PRESS 248

J.R. Ward, S.S. McCotter / Teaching and Teacher Education 20 (2004) 243–257

clear transition or connection to previous reflection. Chunks ranged in length from several short sentences to longer pieces of several hundred words. We considered any text that focused on a specific teaching action to be reflective. The fact that the action was being described implied deliberate thinking about the action and desired improvement. This liberal definition of reflection allowed us to pull in as many chunks as possible from the exemplars. The two authors did the initial identification independently, and then agreed upon the reflective chunks to code, thus providing triangulation. Our review of the literature led us to develop generative questions, the second procedural element of grounded theory, that allowed us to interrogate our data in detail. We used three frames for our investigation of samples of preservice teacher reflection: cyclic nature, breadth of perspective, and inquisitive stance. The cyclic qualities of reflective writing referred to whether preservice teachers engaged in a process of framing and reframing problems, or whether they considered issues over a period of time or across a variety of situations. Breadth of perspective referred to whether preservice teachers considered other viewpoints, whose viewpoints they considered, and whether they considered moral questions. Inquisitive stance referred to whether reflective writing was centered on questions, and indeed whether uncertainty was expressed at all. We also considered the situated nature of reflection, although

not as one of our guiding frames. Situated reflection was operationalized as the events, problems, questions, and context that precipitated reflective writing. Specifically, the qualities of reflection related to student work and learning were considered as part of the situated nature of reflection. We analyzed these chunks asking questions such as ‘‘how well do our initial guiding frames describe this data?’’ and ‘‘What types of precipitants tended to lead to greater use of other perspectives or ongoing questions?’’ We coded reflective chunks from the TWS samples along the frames we had initially identified as part of reflection (cyclic nature, and breadth of perspective, and inquisitive stance), and noted the events or themes that precipitated and situated the reflective chunks. We conducted w2 analysis on the relationship between precipitant type and four levels of a continuum we devised sorting reflective chunks along a continuum from Routine to Critical (Table 1). There was clear significance between precipitant type and level of Reflection (w2 ¼ 121:25; df=33, p ¼ 0:000), Breadth of Perspective (w2 ¼ 123:174; df=33, p ¼ 0:000), and Cyclic Level (w2 ¼ 80:562; df=33, p ¼ 0:000). The relationship between precipitant and Inquisitive Stance was less clear (w2 ¼ 39:368; df=33, p ¼ 0:206). The most effective precipitant in terms of getting student teachers to reflect critically was when they focused on Struggling Students or Student Failure. Our original frames did not capture the

Table 1 Precipitants by reflective level Precipitant Student interest (high) Student interest (low) TWS Assessment/learning goals Content consideration External constraints Instructional strategy Prior-knowledge/experience Relations/environment Self-lauding Struggling students/failure Student learning/excitement

Routine 2

Technical

3 8 11

1 2 2 5 3 2 7 12

3 1 3

2 12 3

2 4

Dialogic

Critical

1 1 4

Total 3 2 4 9 3 5 16 23 1 5 17 6

ARTICLE IN PRESS J.R. Ward, S.S. McCotter / Teaching and Teacher Education 20 (2004) 243–257

importance of precipitants or the importance of focusing on struggling students. As we added CIRQL reflections to our data set, we saw a broader range of reflections that were not well described by our original frames. Unlike the TWS, students involved in the CIRQL process were encouraged to reflect in depth on individual students and also on a wide range of self-initiated questions. This led us to distinguish between reflections focusing on the process of learning as opposed to merely products of learning. We also saw a clear distinction between reflection on self and reflection on teaching tasks undertaken without consideration of other perspectives. This process allowed us to develop other key descriptive distinctions that appear in our final rubric. For example, it became clear that selflauding or blaming problems on constraints outside of the perceived control of the preservice teacher were associated with many of the least reflective chunks. Consideration of teaching strategies and other teaching behaviors were often associated with intention to change, but rarely led to ongoing thinking or involved breadth of perspective. The third element of grounded theory coding is the discovery of core categories. In our work this meant revising our initial frames until they provided a comprehensive method for analyzing all of our reflective chunks. Going back and forth from data to our initial guiding frames led us to develop the three dimensions of our final rubric (Focus, Inquiry, and Change). Using this process our developing rubric was revised multiple times until we reached the fourth element of grounded theory, conceptual density. In other words, we continued asking questions about our reflective chunks, until our rubric provided a good description of all of our reflective chunks and was consistent with the guiding frames we had initially identified through the literature. For the most part this process resulted in more specific descriptions for each level and dimension of our rubric, but in other cases our descriptions became more general. For example, instead of strictly considering the number of cycles of questions, insights, and resulting teacher actions our final

249

rubric describes the general qualities of ongoing reflection. Using a controlled and systematic approach to coding helped us meet the challenge of creating a rubric that can be used to define qualities that distinguish more from less meaningful reflection. It is a challenge to ask preservice teachers to use student learning and student work as a basis for reflecting on their practice. Our response to this challenge was clarified through the process of closely examining the available data. Under outcomes-based systems, the most reportable outcomes are also the most visible and valued. Our rubric achieves this goal of making the qualities of meaningful reflection more visible and valued while honoring the richness and depth of meaningful reflection on practice.

4. Reflection rubric The final rubric is depicted in Table 2, and the remainder of this section describes the dimensions and levels. 4.1. Dimensions of reflection Our cyclical examination of theory and data led us to refine our generative frames (cyclic nature, breadth of perspective, and inquisitive stance) into three dimensions (Focus, Inquiry, and Change). The Focus dimension encompassed elements of our original breadth of perspective frame with what we learned about the precipitants of reflection and reflections on student learning. The data revealed that broadening perspective was often associated with attempts to understand students and the process of their learning, especially when students were struggling or failing. Conversely, a narrow perspective was frequently revealed as a focus on problems that were perceived as caused by others and outside of the control of the teacher. The revised dimension of Focus asks, ‘‘What is the focus of concerns about practice?’’ and ranges from focus on self, to focus on students, to focus on the impact of broader concerns on student learning. This dimension most clearly describes the

ARTICLE IN PRESS 250

J.R. Ward, S.S. McCotter / Teaching and Teacher Education 20 (2004) 243–257

Table 2 Reflection rubric Levels Routine Self-disengaged from change

Technical Instrumental response to specific situations without changing perspective

Dialogic Inquiry part of a process involving cycles of situated questions and action, consideration for others’ perspectives, new insights

Transformative Fundamental questions and change

Focus (What is the focus of concerns about practice?)

Focus is on self-centered concerns (how does this affect me?) or on issues that do not involve a personal stake. Primary concerns may include control of students, time and workload, gaining recognition for personal success (including grades), avoiding blame for failure.

Focus is on specific teaching tasks such as planning and management, but does not consider connections between teaching issues. Uses assessment and observations to mark success or failure without evaluating specific qualities of student learning for formative purposes.

Focus is on students. Uses assessment and interactions with students to interpret how or in what ways students are learning in order to help them. Especially concerned with struggling students.

Focus is on personal involvement with fundamental pedagogical, ethical, moral, cultural, or historical concerns and how these impact students and others.

Inquiry (What is the process of inquiry?)

Questions about needed personal change are not asked or implied; often not acknowledging problems or blaming problems on others or limited time and resources. Critical questions and analysis are limited to critique of others. Analysis tends to be definitive and generalized.

Questions are asked by oneself about specific situations or are implied by frustration, unexpected results, exciting results, or analysis that indicates the issue is complex. Stops asking questions after initial problem is addressed.

Situated questions lead to new questions. Questions are asked with others, with open consideration of new ideas. Seeks the perspectives of students, peers, and others.

Long-term ongoing inquiry including engagement with model mentors, critical friends, critical texts, students, careful examination of critical incidents, and student learning. Asks hard questions that challenge personally held assumptions.

Change (How does inquiry change practice and perspective?)

Analysis of practice without personal response—as if analysis is done for its own sake or as if there is a distance between self and the situation.

Personally responds to a situation, but does not use the situation to change perspective.

Synthesizes situated inquiry to develop new insights about teaching or learners or about personal teaching strengths and weaknesses leading to improvement of practice.

A transformative reframing of perspective leading to fundamental change of practice.

situated qualities of reflection that tend to lead to improvement of practice. We further refined the rubric by combining the frames for cyclic nature (framing and reframing of

questions) and inquisitive stance into the single dimension of Inquiry. This dimension addresses the question ‘‘What is the process of inquiry?’’ and examines how questions are asked rather than just

ARTICLE IN PRESS J.R. Ward, S.S. McCotter / Teaching and Teacher Education 20 (2004) 243–257

their presence. In our original analysis of the data, there was an almost complete and unrealistic absence of explicit questions in student writing. We came to see this as a style of writing rather than a true absence of curiosity. This led us to consider implied questions, as well as explicit questions, as relevant to the reflective process. Analysis of student writing led us to distinguish between questions that were unconnected to practice, and situated questions. The final dimension, Change, asks the question ‘‘How does inquiry change practice and perspective?’’ Again, our data revealed several seemingly important reflective statements that did not easily fit into our original frames. These were often statements like, ‘‘The biggest thing I learned from this experience wasy’’ The problem these statements posed was that they were identified by the writer as extremely important, yet they often lacked any clear development or basis in reflective writings. Our original dimensions lacked a descriptive way to account for these insights. The change dimension evolved to include the concept of new insights arising from a synthesis of situated experiences. 4.2. Qualitative levels The labels for our first three levels of our rubric (Routine, Technical, Dialogic) are terms that have been traditionally used in the literature on reflec. tion (Hatton & Smith, 1995; Schon, 1983). In keeping with the literature, we had initially labeled the fourth level as critical. In our final rubric, the highest level has been renamed and is now called Transformative.2 This change resulted from a possible confusion we see arising between Routine and Transformative reflection. Our Transformative level includes traditional ‘‘critical’’ issues such as historical and moral concerns. However, we did not want this confused with ‘‘critical’’ in the sense of blaming problems on conditions beyond one’s 2 The use of the term Transformative is not meant to imply the Freireian notion of emancipatory reform through education. Although some teachers who reflect at the Transformative level may indeed incorporate aspects of emancipation or liberation in their practice, those ideals are not necessary for reflection at the Transformative level on our rubric.

251

control. Our finding that this type of perception was closely related to low-level reflection led us to distinguish between moral questions about one’s own practice and merely being critical about the moral practices of others (which demonstrates Routine reflection in our rubric). During the early stages of coding we viewed these levels as descriptive of markers on a continuum with important qualities being more or less present at each level. As we continued evaluating reflective writing we began to see these levels as also having clear qualitative distinctions that crossed all dimensions. This is most clearly seen in the Focus dimension. The distinctions between levels of Focus are not merely in terms of the amount of focus on the perspectives of others; each level also has a different focus. Qualitative distinctions exist between levels of the other two dimensions as well. The Inquiry dimension, for example, distinguishes between questions that are asked by oneself and questions that are asked with others. Analysis of reflective chunks using the final rubric revealed that the vast majority of student reflections were scored with a spread of no more than one level across all dimensions. For example, it was rare for a reflective chunk to fit the Routine qualities of Inquiry yet yield Change at the Transformative level. In the following sections we provide samples of preservice teacher reflective writing and explain how our rubric was used to evaluate them. We begin first with the more typical cases, where the level of reflection was relatively consistent across each dimension. Following a discussion of the four levels of reflection, we will discuss two specific and illuminating patterns of wide variance across dimensions. 4.3. Routine Routine reflections tended to contain very definitive statements that revealed either a lack of curiosity or a lack of attention to complexity. These writings usually did not focus on problems, but when they did, the tendency was to blame problems on others or on a lack of time and resources. The lack of questioning and especially the lack of a sense of responsibility for change are

ARTICLE IN PRESS J.R. Ward, S.S. McCotter / Teaching and Teacher Education 20 (2004) 243–257

252

probably most characteristic of these low-level reflections. These reflective writings also tended to be fairly short as illustrated by the following example of Routine reflection:

more active participants. But, I feel I need to find some more strategies on how to make lecture material more interesting and engaging for the students.

The other barrier I found was the ability of many of my students. As an entire class, they did not have much experience working handson. I would have liked to teach many more concepts hands-on, but due to the lack of experience in the class it was not feasible. Classroom management was a problem the first few times we tried a hands-on activity. If this had been my classroom, students would have been familiar with my mode of teaching and classroom management would not have been an issue. When I taught my fall week I did not run into any classroom management problems because they knew my expectations.

This exemplar typifies a Focus on narrow teaching tasks; in this case how to make lectures more engaging. There is a clear desire to learn more related to this topic, but Inquiry about lectures does not seem to lead to further or deeper questioning of practice, nor does this teacher use the perspectives of students or peers in reflecting on this issue. These are the hallmark characteristics of Technical Inquiry. Finally, there is an indication that this student teacher is committed to improvement of practice in this area, though there are no new insights gained from this reflection. This student teacher wants to engage students, but does not question lecturing as an approach. Narrow change or change without new insight is characteristic of Technical qualities of the Change dimension.

This exemplar was rated as Routine in all dimensions. The focus is on typical self-concerns such as the ability to control or manage students. Inquiry was also rated as Routine, there does not seem to be any doubt or questioning of the source of problems. Placing blame on others is not likely to lead the student teacher to change practice or perspective.

4.5. Dialogic

Technical reflection can best be thought of as instrumental, in that the reflection is used as a means to solve specific problems, but does not question the nature of the problem itself. Most typically, these reflections focus on teaching tasks, as seen in this exemplar:

Dialogic reflection is best thought of as ongoing process. The term itself connotes discussion and consideration of the views of others. The theme of process at this level can also be seen in a Focus on the process of learning (as contrasted with the outcomes of learning) and often in the process of Inquiry (i.e., a sustained process of asking questions, trying new approaches, and asking new questions). One of the most common forms that this reflection takes is grappling with the learning process for a struggling student, as seen in this exemplar:

I could use more professional development inygetting students more involved in ‘‘lecture’’ material and making a connection from class notes and lecture material to the overall understanding of the lesson. Sometimes there is not the opportunity to do a hands-on activity related to a particular topic. The material can be very dry, but definitely necessary to the understanding of the topic. I try to play review games, and get the students involved in the lecture by asking questions that make them

Student one, who is an English as a second language student, did very poorly on the preassessment. My first reaction was to have his ESL teacher give him the assessment. When I found out that this was not feasible, I decided to try it myself. My first step was to borrow one of the student’s English-Spanish dictionaries. I was surprised to find out that a lot of words I needed were not in the dictionary. After finding as many words as I could, I made notes on a blank assessment and set a time to meet with

4.4. Technical

ARTICLE IN PRESS J.R. Ward, S.S. McCotter / Teaching and Teacher Education 20 (2004) 243–257

student one. I went through the assessment again with this student, only to find that my efforts did not help. My analysis of this exercise, however, allowed me to understand a little better why he did poorly on the preassessment. I found that it was not just his English deficiency that hindered him on the assessment. Student one did not have the prior knowledge needed to answer the questions on the assessment. I also discovered other helpful information from this exercise. Student one is able to answer questions that require one-word answers, but could not answer questions that required him to write sentences. If I had not discovered this, I would have just assumed he didn’t know the material. Because of this discovery, I was able to make modifications on the rest of his assessments. Clearly this reflection chunk has a Focus on a struggling student. The key to a Dialogic rating of Inquiry for this sample is the ongoing nature of questions with original questions leading to new questions. Unlike many reflections at the Dialogic level, however, this reflection did not include consideration of the student’s perspective on the problem. Our rubric does not require consideration of others’ perspectives to achieve a Dialogic level; instead, another’s perspective is usually the driving force behind consideration of new ideas and new questions. In this case, the student teacher is essentially involved in a dialogue with the situation itself. The situated nature of Inquiry often motivates ongoing questions. Finally, this reflection reveals new insights and Change stemming from the process. 4.6. Transformative While Dialogic reflection considers and synthesizes new ideas, Transformative reflection questions fundamental assumptions and purpose more deeply. Although in our experience it is not unusual for preservice teachers to express Transformative ideas in response to theoretical readings, it is rare for preservice teachers to reach this level when reflecting on their own teaching. The

253

primary focus of teachers at this level on teaching tasks and self-concerns tends to crowd out deeper questioning. It is also important to point out that our rubric describes this process as taking place over a long period of time and so we would not expect to see true examples of this type of change within a single semester or year. In our sample, only one student teacher approached this level. The sample shown here is abstracted from the original writing. It seems that this student reached this higher level only after sustained inquiry and self-questioning during her experience. In this case, the student teacher was part of a cohort that met regularly with mentors to discuss ongoing questions and share student work. During the unit I found that TD worked independently on his essay and appeared confident in his work. DK in comparison who rarely asks for assistance with her writing wanted to conference all the time. Her continual need for teacher approval concerned me and forced me to question why this was occurring. After questioning her behavior I began to question the assignment. Yes, DK is a very good writer, but why was she struggling? y Wow, in the middle of my unit I then began to question the success of the unit. Am I really meeting the needs of all of my students or is this too easy? y Finally after weeks of teaching, reflecting and questioning the unit it was over and I spent hours grading countless persuasive essays. After looking over the drafts and then the final essays I found a correlation between what was occurring in the classroom and what the students were writing; they were making the connectionsy First, I would never just do a persuasive writing unit again (even though it was in conjunction with westward expansion). I would love to make a connection between a relevant issue in student’s lives and how they can utilize persuasive writing to assist them with it. I feel that it is so important to not label units, such as persuasive writing because, it deters students

ARTICLE IN PRESS 254

J.R. Ward, S.S. McCotter / Teaching and Teacher Education 20 (2004) 243–257

from getting involved. In the future I think that I will continue to ponder many of the same questions that arose throughout the teaching of my persuasive writing unit (here I go labeling it again) and even more. The Focus on students in this sample leads to a deeper questioning of what is truly valuable and meaningful for them. This sample exemplifies whole-hearted Inquiry revealing a sincere habit of continually asking questions and looking for answers over time. Though not completely reaching the full Transformative level in the Change dimension, this reflection does lead to a change of perspective on the unit that was taught, potentially having a profound effect on practice. 4.7. Patterns that cross levels of the rubric As mentioned earlier, there were two patterns in the data where student reflection showed a range of levels on the Reflection Rubric. The first pattern involved deep reflection on personal qualities that affect teaching. The following exemplar was rated as having a Routine Focus on self, but reached the Dialogic level for Inquiry: One student suggested that I teach more aggressively and to develop more confidence in myself as well as (I love this) to not ‘‘take any crap from us students’’. I understand what that student was saying. I was definitely not confident in teaching this unit, partly because probability was a topic in mathematics that I wasn’t comfortable with and partly because this was the first unit I taught with this class. As the semester went on, I did develop more confidence in teaching them as well as became more comfortable with displaying my personality, which another student had suggested. I never got to that point with my calculus students. They did ask me lots of questions, but I also was not completely comfortable with the topic I was teaching (volumes of revolution), which made it difficult to become comfortable and confident in the classroom within in the two weeks I had with them. I may have been able to develop a better relationship with them if I had more time to do so.

This example brings out an important point in using the reflection rubric. Although the Dialogic and Transformative levels of the rubric represent deeper reflection, they are not always the most appropriate or needed forms of reflection. The developmental path for many preservice teachers suggests that concern for self and gaining competency in teaching tasks is and probably should be the most immediate focus. The question is whether these beginning stages of reflection will contain the seeds for deeper reflection later or whether reflection is undertaken as a process that aims for improvement and is open to the ideas of others. In this case, the student teacher’s focus on self was clearly important: the writing reveals a struggle for personal empowerment as a teacher. Unlike other reflective writings with a Routine self-focus, however, this student teacher sought the perspectives of others in the process of reflecting on self. There is also a difference in personal responsibility as evidenced by the Focus on self-improvement rather than on blaming others for problems. A second pattern involved summative reflective statements describing teaching success as seen in the following exemplar: My greatest insight in teaching this unit was the fact that children want to learn. When concepts are presented to children in a way that is meaningful to them, they will naturally learn. My students wanted to know why, when, where, and how. They wanted to show they could apply their knowledge in a new situation. They wanted to analyze their learning by breaking it down and then they wanted to create thereby using synthesis. My students had no problems evaluating their work. They were able to justify their need and their ability to conquer the subject matter. The students had within them the desire to learn. I just had to allow that desire to express itself. I realized that I did not have the ability to ‘‘force’’ a student to learn. Instead, I recognized that I could lead the student and provide an environment that allowed the student to learn. This type of writing is perhaps the most difficult to evaluate. On the one hand, the level of generality and the definitive stance on complex

ARTICLE IN PRESS J.R. Ward, S.S. McCotter / Teaching and Teacher Education 20 (2004) 243–257

issues reveals a lack of recognition of how much there is to learn as a teacher. On the other hand, the student explicitly relates this as important. The Focus is clearly on students, but one might question whether this is more of a self-congratulatory reflection using students to illustrate personal success. This exemplar came from the TWS data; students completing the TWS know that they are being evaluated as candidates based on their reported success. This highlights the difference between evaluating and making visible the qualities of reflection (as we have attempted to do with our rubric), and merely using reflection as a means for reporting other professional outcomes. The first approach emphasizes the value of reflection as a means toward growth; the second approach tends to stifle sincere curiosity and change. In the effort to add value to the reflective process by connecting it to student learning, it is critical that quality reflection be considered a valuable outcome on its own merit, and not merely a measure of other outcomes.

5. Conclusions The standards movement poses several challenges to the value of meaningful reflection. On the most basic level, reflection is in danger of becoming nothing more than a tool toward greater student achievement. Taken to an extreme, this narrow view of the purpose of reflection redefines reflection as a means of documenting student outcomes. As teacher educators, we must be able to make a clear case for reflection as an outcome above and beyond its short-term instrumental value. Further, we should be able to define reflection in a way that makes the qualities we value visible. Development of a rubric that includes dimensions for process and emphasizes broadening perspective and fundamental questions is one means of doing so. Incorporating student learning within this rubric is one way of recognizing the importance of student learning outcomes without making reflection inferior to them. The patterns we saw in the work of preservice teachers echoed Fuller’s (1969) theory about the

255

stages that often describe preservice teacher growth. The reflections of beginning teachers reinforce the fact that reaching levels of Transformative reflection is unusual and difficult. This does not give us or them permission to aim lower, but gives us reasons to identify lower levels of reflection and provide the scaffolding needed to reach higher. We believe our matrix can help serve as some of that scaffolding. It is not supportive to ask preservice teachers to ignore their self-focused concerns, but it is more helpful (and more challenging) to engage them in a Dialogic process of carefully considering their self-related concerns. Although we hope to move our students closer to Transformative reflection, it is not our vision that most of our preservice teachers will be able to quickly reach the highest levels of our rubric, nor that all of them will do so sequentially. It is our vision that all preservice teachers should consistently reach the Dialogic level of Reflection by the end of their preparation. The process of developing the Reflection Rubric was eye opening for us. We realized that we have often asked our students to reflect on field experiences without ever discussing the qualities of good reflection, often with disappointing results. Students do not automatically know what we mean by reflection; often they assume reflection is an introspective after-the-fact description of teaching. Reflection, meant to make teaching and learning understandable and open, has itself been an invisible process to many of our preservice teachers. The dimensions of Focus, Inquiry, and Change can be used as formative guides to help preservice teachers evaluate, understand, and improve their own reflection. Though we feel the Reflection Rubric should be used as a formative tool for individual preservice teachers, it may also be used as a summative evaluation instrument for innovations such as the Teacher Work Sample, which seek to encourage reflection on student learning. As teacher education programs attempt to articulate and measure outcomes for their programs, our reflection rubric provides a means for evaluating reflection as a core program goal. Our reflection rubric would also work well as a research tool for evaluating the effectiveness of a wide variety of strategies

ARTICLE IN PRESS 256

J.R. Ward, S.S. McCotter / Teaching and Teacher Education 20 (2004) 243–257

designed to promote teacher reflection such as cases and journals as well as newer innovations such as the use of electronic portfolios and digital video. Development of the rubric using data from two models for reflection on student outcomes led us to see how an emphasis on student learning could either enhance or detract from the reflective process. When reflection is focused on formative assessment and the process of student learning, Technical questions tend to lead to ongoing reflection and broadening of perspective. Alternatively, when reflection is based on summative assessment of student learning there is little opportunity for new questions to arise. If preservice teachers reflect only on whether or not their students scored well on tests, student learning is evaluated in a very narrow way, rather than a broader reflection about the teaching and learning process. In order for reflection to be evaluated, we must overtly connect the qualities of reflection to the process of teaching and learning. It is only through this integral connection that we can prevent reflection from becoming a rote process, or seeing it eliminated altogether. Given both our passion as a profession and the political climate, we need to continue to actively search for ways to demonstrate that our student teachers understand this connection between teaching and learning. The new emphasis on student outcomes creates an opportunity for us to go to new levels of reflection and understanding of student learning. We can, should, must develop methods of stepping back to take a broad view of this essential connection without giving up the process of reflection.

References Clarke, A. (1995). Professional development in practicum settings: Reflective practice under scrutiny. Teaching and Teacher Education, 11(3), 243–261. Cochran-Smith, M. (2000). Teacher education at the turn of the century. Journal of Teacher Education, 51, 163–165. Collaborative Inquiry-Reflection, Questions about Learning. (2003). CIRQL unit exemplars. From the CIRQL web site: http://harmony2.millersville.edu/cirql.

Collier, S. T. (1999). Characteristics of effective teachers. Journal of Teacher Education, 50(3), 173–181. Darling-Hammond, L., & Snyder, J. (2000). Authentic assessment of teaching in context. Teaching and Teacher Education, 16(5–6), 523–545. Dewey, J. (1933). How we think: A restatement of the relation of reflective thinking to the educative process. Boston: Heath. Dinkelman, T. (1997). The promise of action research for critically reflective teacher education. Teacher Educator, 32(4), 250–274. Dinkelman, T. (2000). An inquiry into the development of critical reflection in secondary student teachers. Teaching and Teacher Education, 16, 195–222. Francis, D. (1995). The reflective journal: A window to preservice teachers’ practical knowledge. Teaching and Teacher Education, 11(3), 229–241. Fuller, E. F. (1969). Concerns of teaching: A developmental conceptualization. American Educational Research Journal, 6, 207–226. Galvez-Martin, M. E., Bowman, C. L., & Morrison, M. A. (1998). An exploratory study of the level of reflection attained by preservice teachers. Mid-Western Educational Researcher, 11(2), 9–18. Gillespie, M. L. (1996). Stories, cases, and practical wisdom. Innovative Higher Education, 21(1), 49–66. Girod, G. R. (2002). An overview of teacher work sample methodology. In G. Girod (Ed.), Connecting teaching and learning: A handbook for teacher educators on teacher work sample methodology. Washington, DC: AACTE. Harrington, H. L., Quinn-Leering, K., & Hodson, L. (1996). Written case analyses and critical reflection. Teaching and Teacher Education, 12, 25–27. Hatton, N., & Smith, D. (1995). Reflection in teacher education: Towards definition and implementation. Teaching and Teacher Education, 11, 33–49. Ketter, J., & Pool, J. (2001). Exploring the impact of a highstakes direct writing assessment in two high school classrooms. Research in the Teaching of English, 35, 344–393. Kitchener, K. S., & King, P. M. (1981). Reflective judgment: Concepts of justification and their relationship to age and education. Journal of Applied Developmental Psychology, 2, 89–116. Korthagen, F. A. J. (1999). Linking reflection and technical competence: The logbook as an instrument in teacher education. European Journal of Teacher Education, 22(2/3), 191–207. Korthagen, F. A. J., & Kessels, J. P. A. M. (1999). Linking theory and practice: Changing the pedagogy of teacher education. Educational Researcher, 28(4), 4–17. Loughran, J. J. (2002). Effective reflective practice: In search of meaning in learning about teaching. Journal of Teacher Education, 53(1), 33–43. Meier, D. (2000). Will standards save public education?. Boston: Beacon Press. Mitchell, R. (1996). Front-end alignment: Using standards to steer educational change. Washington, DC: The Education Trust.

ARTICLE IN PRESS J.R. Ward, S.S. McCotter / Teaching and Teacher Education 20 (2004) 243–257 National Commission on Teaching & America’s Future. (1996). What matters most: Teaching for America’s future. New York: Author. National Council for the Accreditation of Teacher Education. (1999). NCATE 2000: Continuing accreditation and beyond. Some questions and answers. Washington, DC: Author. National Council for the Accreditation of Teacher Education. (2000). Performance assessment in action. Quality Teaching: NCATE Newsletter, Spring 2000, pp. 4–8. No Child Left Behind Act (Accessed June 9, 2002). Retrieved June 5, 2002, from http://www.nochildleftbehind.gov/next/ overview/index.html. Preissle, J., Tesch, R., & LeCompte, M. D. (1997). Ethnography and qualitative design in educational research. San Diego: Academic Press. Rearick, M. L., & Feldman, A. (1998). Orientations, purposes and reflection: A framework for understanding action research. Teaching and Teacher Education, 15, 333–349. Reiman, A. J. (1999). The evolution of the social roletaking and guided reflection framework in teacher education: Recent theory and quantitative synthesis of research. Teaching and Teacher Education, 15, 597–612. Ross, D. D. (1989). First steps in developing a reflective approach. Journal of Teacher Education, 40(2), 22–30. Schalock, D., Schalock, M., & Myton, D. (1998). Effective— along with quality—should be the focus. Phi Delta Kappan, 79, 468–470. Schalock, H. D., & Myton, D. (2002). Connecting teaching and learning: An introduction to teacher work sample methodology. In G. Girod (Ed.), Connecting teaching and learning:

257

A handbook for teacher educators on teacher work sample methodology. Washington, DC: AACTE. . D. A. (1983). Reflective practitioner: How professionals Schon, think in action. New York: Basic Books. Stanley, C. (1998). A framework for teacher reflectivity. TESOL Quarterly, 32(3), 584–591. Strauss, A. L. (1987). Qualitative analysis for social scientists. New York: Cambridge University Press. Strauss, A. L., & Corbin, J. (1990). Basics of qualitative research: Grounded theory procedures and techniques. Newbury Park, CA: Sage. The Renaissance Partnership for Improving Teacher Quality. (2002). Teacher work sample: Performance prompt, teaching process standards, & scoring rubrics. From The Renaissance Partnership for Improving Teacher Quality web site: http:// fp.uni.edu/itq. Valli, L. (1992). Reflective teacher education: Cases and critiques. Albany: SUNY Press. Yost, D. S., Sentner, S. M., & Forlenza-Bailey, A. (2000). An examination of the construct of critical reflection: Implications for teacher education programming in the 21st century. Journal of Teacher Education, 51(1), 39–49. Zeichner, K. M. (1996). Teachers as reflective practitioners and the democratization of school reform. In K. M. Zeichner, S. Melnick, & M. L. Gomez (Eds.), Currents of reform in preservice teacher education (pp. 199–214). New York: Teachers College Press. Zygouris-Coe, V., Pace, B., Malecki, C., & Weade, R. (2001). Action research: A situated perspective. Qualitative Studies in Education, 14(3), 399–412.