The Chilean instructional pattern for the teaching of language: A video-survey study based on a national program for the assessment of teaching

Learning and Individual Differences 19 (2009) 1–11

Contents lists available at ScienceDirect

Learning and Individual Differences j o u r n a l h o m e p a g e : w w w. e l s ev i e r. c o m / l o c a t e / l i n d i f

The Chilean instructional pattern for the teaching of language: A video-survey study based on a national program for the assessment of teaching David D. Preiss Pontificia Universidad Católica de Chile, Escuela de Psicología, Av. Vicuña Mackenna 4860, Macul, Santiago, Chile

a r t i c l e

i n f o

Article history: Received 20 July 2007 Received in revised form 19 July 2008 Accepted 6 August 2008 Keywords: Instructional pattern Folk pedagogy Teaching competency Chile

a b s t r a c t I investigated the instructional pattern for the teaching of language at the elementary school level in Chile. The data were obtained from analyzing videos filmed for a teaching assessment system implemented by the Chilean Government. The system identified four levels of teacher competence. I treated these levels of competence as an independent factor. The dependents variables of interest were frequency of teacher questions and follow-ups in four categories, social organization of the lesson and time on task. Most of teachers studied are clearly aligned with an adult-run model of education. Evidence for adherence to this pedagogical approach can be found in (a) teacher questioning and teacher reactions to students' contributions, and (b) the place accorded to the teacher in the social organization of the lesson. The teaching pattern uncovered has its roots in the intuitive pedagogies endorsed by Chilean teachers, which in turn rest on Chilean cultural models of pedagogy. © 2008 Elsevier Inc. All rights reserved.

1. Introduction The purpose of this study is to uncover the distinguishing features of elementary school teaching of language in Chile, in particular how pedagogical conceptions transpire in the task of teaching. Thus, the focus of the study is on the instructional principles that provide cultural and personal meaning to the task of teaching in Chile. As a result, in my assessment of public school teaching I attempt to tap into teachers' intuitive or folk pedagogies, that is, the intuitive notions about learners and implicit models of pedagogy shared by theorists, educators, and children (Bruner, 1996; Olson, 2003; Olson & Bruner, 1996). In particular, this study is intended to accomplish two larger goals. First, as other video-survey studies, it intends to connect the two edges of classroom research: that is, the large-scale analysis of classroom interaction by means of low-inference indicators and the micro-genetic analysis of classroom dialogue by means of discourse analysis (Stigler, Gallimore, & Hiebert, 2000). Complementing this methodological goal is a comparative goal. Although research on teaching has accumulated a large amount of evidence about what distinguishes teaching practices at elementary schools around the world, there has not been much empirical research on teaching conducted in Latin America. For instance, the large video-surveys developed by the 1995 and 1999 versions of the Trends in International Mathematics and Science Study (hereafter, TIMSS) did

E-mail address: [email protected]. 1041-6080/$ – see front matter © 2008 Elsevier Inc. All rights reserved. doi:10.1016/j.lindif.2008.08.004

not include Latin America (Hiebert et al., 2003; Stigler, Gonzales, Kawanaka, Knoll, & Serrano, 1999). Another influential study thoroughly portrayed the nature of elementary education in the United States, France, England, Russia and India, yet it missed Latin America as well (Alexander, 2000). Where Latin America is concerned, use of video-surveys for largescale assessment of elementary school teaching has been implemented in Mexico and Chile by two government initiatives: the Programa Escuelas de Calidad (Quality Schools Program; hereafter PET; Loera, 2006) and Docente Mas (Teacher Plus; hereafter DM; Manzi, Preiss, Flotts, González, & Sun, 2008), respectively. This study reports the first analyses made for research purposes of a sample of the videos collected in the context of the Chilean initiative. 1.1. Docente mas DM is a nationwide system for the assessment of teaching, which has been implemented along a number of recent public policy initiatives intending to renovate the status of the teaching profession in Chile. DM relies on a conceptual framework named El Marco para la Buena Enseñanza (Framework for good teaching, hereafter MBE; Ministerio de Educación, 2003), which is inspired on Danielson's (1996) framework for teaching. Although the MBE sets standards for the teaching profession in Chile, it does not include strict operational definitions. In consequence, it has been DM's task to produce an assessment framework aligned to the MBE. To perform the assessment, DM uses four instruments: a selfassessment, an interview made by a peer, reports by third parties, and a portfolio, which includes, first, a video-survey of a class

2

D.D. Preiss / Learning and Individual Differences 19 (2009) 1–11

conducted by the teacher and, second, teacher planning of three lessons. Based on the scores achieved by the teachers on the four instruments, DM differentiates teachers in four performance groups: outstanding, proficient, basic, and unsatisfactory (Manzi et al., 2008). The implementation of DM started in the year 2003, when 3740 elementary school teachers were assessed. As the Chilean government scheduled a gradual implementation of the system, each year new teachers have joined the assessment process. Thus, during the first four years of implementation DM has evaluated 30.402 teachers. The current study re-analyzes the video-survey component of the portfolio teachers produced for the 2003 round of assessment. That year, DM identified 9.4% of the teachers evaluated as outstanding, 56.6% as proficient, 30.2% as basic, and 3.8% as unsatisfactory (Manzi, 2007). Although it was the first assessment ever made of teaching competence in Chile, the 2003 assessment captured a fairly representative view of the state of affairs of teaching in Chile. Indeed, the distribution of teachers across the four levels of competence remained quite stable in the following rounds of appraisal (Manzi et al., 2008). 1.2. From performance-appraisal to folk pedagogy If we are to understand why Chilean elementary school teachers teach the way they teach, we need to complement DM's performanceappraisal with an approach based on the privileged vantage point of teachers' folk pedagogies. Whereas a performance-appraisal perspective is an effective way to identify the strengths and deficits teachers might have, teachers' folk pedagogies connect teaching practices both to the cognitive dimension of teaching and its cultural background. Olson and Bruner (1996) distinguish two main kinds of folk pedagogies: internalist and externalist folk pedagogies. These, in turn they subdivide into four different categories, which I operationalized as follows: • An internalist folk pedagogy focused on the social construction of knowledge involves extensive mentalistic and elaborative dialogical talk. Lessons are structured using a collaborative or student-run format, so teachers use a moderate number of follow-up communicative moves during the lesson, allowing the students to elaborate their ideas independently. • An internalist folk pedagogy with a focus on the cultural elaboration of knowledge involves extensive mentalistic and elaborative dialogical talk. Lessons are structured using a negotiated format so dialogue is amplified by an intense use of teacher followup communicative moves, which fine-tune students' knowledge to the standards set by different cultural sources, including the teacher. • An externalist folk pedagogy which is focused on the transmission of content or skill training involves informative talk and is regulated by an extensive use of evaluative teacher follow-ups. Lessons are structured using an adult-run format, since the teacher plays a key role while delivering content knowledge. • An externalist folk pedagogy, which is focused on skill training, is regulated by an extensive use of evaluative teacher follow-ups. Lessons are structured using an adult-run format, since the teacher plays a key role while demonstrating procedural skills. Table 1 summarizes these four folk pedagogies. 1.3. An empirical approach to folk pedagogy Folk pedagogies can be approached empirically in two ways. One way is to study the folk pedagogies manifest in teachers' beliefs, using conventional surveys methods or interviews. The second way is to infer folk pedagogies from the teaching practices they inform,

Table 1 Operational definitions of folk pedagogies Folk pedagogy

Internalist

Emphasis Talk

Socialconstructionist Dialogue

Culturecentered Dialogue

Content transmission Monologue

Externalist

Follow-up use Lesson structure

Moderated Student-run

High Negotiated

Moderated Adult-run

Repeated practice Pragmatic exchange High Adult-run

using convergent and relatively indirect methods such as videosurveys or content analysis of teacher documents. This study takes the second route, aiming to infer teachers' intuitive pedagogies from the observation of two aspects of the teaching practice that heavily influence the learning process: these are, teacher talk and lesson structure. 1.3.1. Teacher talk Talk that is rich in mentalistic content is a component of quality teaching. Olson and Astington (1993) hypothesize that teachers who make more use of metacognitive talk will produce students who will be better able to understand their own and others' beliefs. Yet, the instructional importance of mentalistic talk is not only related to the understanding of beliefs but also to self-regulation (Paris & Paris, 2001). Since it helps students learn every subject matter, researchers have encouraged teachers to bring more metacognitive talk into the classroom: for instance, asking students to assess their own work, products, and processes following clearly articulated criteria (Andrade & Perkins, 1998). Despite its educational relevance, evidence of metacognitive talk is virtually absent from recorded teacher–student exchanges in real classrooms. For instance, this kind of elicitations was significantly absent in the sample of lessons the TIMSS 1995 Video Study analyzed: out of 30 sampled utterances per lesson, the mean number of metacognitive elicitations was less than one for all the countries studied (Stigler et al., 1999, p. 105). Metacognitive talk is commonly studied as a part of one of the most ubiquitous forms of classroom communication: InitiateRespond-Follow-up sequences (hereafter IRF sequences; Mehan, 1979; Sinclair & Coulthard, 1975). Within a typical school classroom these sequences are regularly initiated by a teacher question, then followed by a student answer, and commonly closed by a teacher follow-up. The assessment of IRF sequences is mixed. Some scholars are critical of its pervasiveness; others stress that it is, by design, instrumental to meeting the goals of schooling and provides a gatekeeper — the follow-up move — for the content of the lesson, which is collaboratively constructed by the teacher and the student (Newman, Griffin, & Cole, 1989). Although the most common function of the follow-up move is to evaluate a student contribution, it can be used differently. For example, it can be used to extend a student's answer or to connect the contents of the lesson with his or her life experience. On the other hand, topics can be introduced distinctly either by the teacher or the students, thereby giving shape to different kinds of IRF sequences And, finally, the contents treated can be substantially different depending on the task at hand: introducing content or reviewing homework, for instance. As regards the empirical study of these exchanges, Wells (1999) developed a coding scheme to capture this diversity of purposes. In addition, the TIMSS 1995 Video Study coded a vast number of IRF sequences, which it considered “a cohesive unit of conversational exchange” (Stigler et al., 1999, p. 113). In the three countries it studied, the opening questions with the highest incidence were those that asked students to name or state a mathematical concept, while those that required students to describe or explain, that

D.D. Preiss / Learning and Individual Differences 19 (2009) 1–11

is, those involving a higher metacognitive potential, were less common. 1.3.2. Lesson structure A focus on lesson structure is commonly related to the study of classroom management strategies. Still, successful class management strategies are not related to particular instructional approaches (Brophy, 2000). To relate patterns of time allocation to folk pedagogies, then, it is necessary to look for a framework other than class management. International comparative studies on teaching provide the kind of framework that is needed. Indeed, the TIMSS 1995 and 1999 studies performed a deep study of lesson structure, yet in domains other than language and for the middle school level (Hiebert et al., 2003; Stigler et al., 1999). Inspired by the TIMSS studies, Loera (2006) used video-surveys to investigate the lesson structure of a sample of 436 language (and mathematics) lessons in Mexico. The researchers found that the average duration of the lessons studied was of 52.18 min (S.D. = 21.08). Most of their time was devoted to student independent-work (31.62 min, S.D. = 20.78) and, second, to teacher lecturing (23.69 min, S.D. = 17.37). The time devoted by teachers to disciplining students or attending non-instructional issues was considerable (13.43 min, S.D. 5.15). The activities most implemented by the teachers were general learning activities as well as repeated practice of textbook exercises. In lesser degree, they allocated time to reading from a textbook or other alternative materials, and introducing new content. Low achieving lessons invested more time doing reading than high achieving lessons. Adopting an approach distinct than the TIMSS' videosurveys, Alexander (2000) studied elementary school teaching in France, the United Kingdom, the United States, India and Russia. Performing deep case studies, he showed that lesson structure differed between countries depending on the weight given to instructional content in the introductory and conclusion sections of the lessons, the length of time allocated to the different parts of the lessons, and whether the central section of the lessons was taken up either by a single task or a sequence of several developmental tasks. Teaching of language is a developmental process. So language teachers change their routines of allocation time depending on the grade they teach. For instance, they allocate more time to instructional tasks related to phonological processing in the beginning grades but devote more time to vocabulary and concept knowledge in the latter grades (Foorman, Goldenberg, Carlson, Saunders, & Pollard-Durodola, 2004). In addition to grade level, teacher skill, the students' proficiency level and the curriculum used play a role in shaping time allocation during language teaching (Foorman & Schatschneider, 2003). While acknowledging the developmental nature of language teaching, the study here implemented focuses its attention on the nature of cultural patterns of teaching and how they relate (or not) to teacher competence.

3

as the influx of positivism began to change the approach of the State towards education and the educational system increased its recruitment levels (Haye & Pacheco, 1995). In spite of their differences, both these models are examples of adult-run instructional models: “The teacher's job in the adult-run model is to prepare the knowledge for transmission and to motivate children to make themselves receptive” (Rogoff, Matusov, & White, 1996, p. 393). Evidently, these models are compatible with externalist folk pedagogies. A second issue this study addresses involves a problem of compatibility between educational theory and the local context of teaching. The weight given to thinking about thinking in theories of metacognition is a byproduct of the development of literate societies (Olson, 1994, 2005). However, Latin American societies, including Chile, are pre-eminently oral cultures instead of literate ones (Morandé, 1984). One of the distinctive features of oral cultures is that, paraphrasing Bruner (1996), they do not tend to “go meta”: that is, they do not treat language or mind as an object of further inspection. Therefore, the lack of a metacognitive proclivity in teaching may be the consequence of widely spread cultural patterns instead of possible teaching deficits. 2. Research questions and hypotheses Given the dominant cultural traditions informing Chilean education, we can expect that Chilean elementary school teachers will adopt teaching patterns that adhere to an adult-run model of education and, consequently, to an externalist folk pedagogy. That is to say, we expect the majority of Chilean elementary school teachers to (a) address their students by means of objective and informative talk, which is regulated by an extensive use of evaluative teacher follow-ups, and (b) structure lessons around the figure of the teacher. Despite the fact that the adult-run educational model is predominant in Chilean classrooms, I expect that some teachers will depart from the dominant model. The reason to hypothesize that some teachers will deviate from the cultural pattern rests on empirical grounds. DM has been able to identify individual differences in teaching performance: particularly, DM detected that, for the 2003 round of assessment, roughly 10% of the teachers were well aligned to the MBE, a framework for teaching that calls for a model of education more compatible to an internalist pedagogy. In consequence, I expect that those teachers DM identified as more proficient will demonstrate: (a) a greater proportion of questions inviting information for further exploration (b) a greater proportion of follow-up moves capitalizing on students' contributions, and (c) a greater proportion of time allocated to dialogue with their students than will teachers whose teaching skills are rated with lower scores. 3. Methods

1.4. The cultural modeling of instruction

3.1. Participants

By assessing the pertinence of folk pedagogy to Chilean education, the study tests the applicability to and validity of current instructional theories advanced in Anglo Saxon, mostly white, largely Protestant communities in Chile, a Spanish-speaking, mostly mestizo, and mostly Catholic country. There are some socio-cultural issues at stake that are, consequently, worth noting. The first issue this study addresses involves a consideration of the cultural traditions shaping teaching practices in Chile. Researchers have documented two main attitudes towards education in Chile: an enlightened approach, which originated in the early decades of the 19th century, when Chile won its independence from Spain, and whose main aspiration was to establish the cultural basis of the new society; and a factory approach, which developed

From the universe of 3740 elementary school teachers assessed by DM in 2003, material on 135 teachers was sampled for further analysis. Teachers participating in the study worked at schools belonging to nine different administrative regions of the country, including north, south, central and coastal geographical areas. Eightynine percent of teachers teaching in Chile's elementary schools are in urban settings, so only teachers working in urban areas were considered for sampling. From this group, teachers were selected using a simple random sampling strategy. Of the 135 teachers sampled in this study, 21 (15.55%) were assessed by DM as outstanding, 74 (54.81%) as proficient, 38 (28.14%) as basic, and 2 (1.49%) as unsatisfactory. Outstanding teachers were slightly overrepresented since they were 15% of the study's sample and around 10% in DM's

4

D.D. Preiss / Learning and Individual Differences 19 (2009) 1–11

original group. Still, the study's sample composition is similar to DM's original group composition. The mean age was 47.09 years (S.D. = 8.94). A total of 124 teachers were assessed across all the dimensions of this study. Teacher talk was assessed in 128 teachers and lesson structure in 127. Correlational analysis excluded cases with missing data. Sample size was determined based on the intended data analysis strategy as well as on practical considerations. For teacher talk, I planned to summarize the correlation between the different eight questions and follow-ups variables, using principal component analysis. The usually recommended ratio of sample size to number of variables in the literature for principal component and factor analysis ranges from 2:1 through 20:1. The ratio of sample size to number of variables in the current study is 15.5, so our final n was more than acceptable. Although the absolute sample size of the study is relatively small, I expected to detect strong correlations and distinct components, which make a smaller sample size adequate (Tabachnick & Fidell, 2001). In addition to the principal component analysis, I planned to perform a number of repeated measures analysis, which involved four, three and two repeated measures. According to Stevens (2002) in a single group repeated measures with four, three and two variables, the sample size needed for power = .8 at the.05 level, anticipating a low average correlation within variables (.3) and wishing to detect a moderate effect size (.30) is, respectively, 35, 39 and 45. Therefore, the study's final sample size is acceptable for these analyses as well. Whereas our sample size is acceptable for a single group repeated measures its size is smaller than desirable to detect an interaction between the different repeated measures and their final score on DM. Certainly, the practical issues related to the large-scale coding of micro-level variables did not make possible a bigger sample. 3.2. Procedure and materials 3.2.1. Video-surveys DM's video-surveys were the primary material used in the present study. Expert cameramen filmed each teacher performing a language lesson. Filming times were agreed upon with teachers via their school principals. Teachers were advised not to set the videotaping during the first lessons of the school day, since they commonly involve a number of non-instructional tasks. At least 80% of the students in the class had to be present, so it was recommended that teachers inform students in advance to promote their attendance. Teachers had to communicate to the cameraman beforehand which activities they would perform so the cameraman could make the necessary arrangements to adequately record the lesson. Although teachers were encouraged to perform their lessons as usual, they were asked to perform activities in which they actively address their students and interact with them. They were as well required to include at least 10 min of lecturing in the course of the videotaping. Passive activities, such watching a movie or silent reading, were explicitly discouraged. Two studies were performed using DM video-surveys. The first study focused on teacher talk during lessons' beginnings; the second study investigated lesson structure during the full lesson. 3.3. Teacher Talk study 3.3.1. Coding scheme The coding scheme for Teacher Talk (TT, hereafter) included two broad categories: teacher questions and teacher follow-ups. Teacher questions are all the questions the teacher asked to the whole class or to individual students publicly. Teacher follow-ups are all the public responses made by the teacher to students' responses. All of these codes were adjusted from Wells (1999) and from the TIMSS 1995 Video study (Stigler et al., 2000). I selected

and merged those codes that were more relevant to the Chilean context. Teacher questions were coded as follows. 1. Controls: The purpose of these questions is to regulate the lesson flow. The teacher assesses whether students are keeping track of the lesson and organizes the allocation of tasks. 2. Inform. The purpose of these questions is to check for information. The teacher assesses whether students know some information or are following the information the teacher is delivering in the class. 3. Implement. The purpose of these questions is to make students use a verbal skill. In particular, the teacher trains the students in literate language. 4. Elaborate. The purpose of these questions is to ask the students to reflect on their previous work and express their beliefs. The teacher provides students with the opportunities to think about what they have done or said, to collaborate with the class and to negotiate the meaning of their work. Usually, these questions involve the following kind of W-questions: Why? What for? Teachers' follow-ups were coded using the following codes: 1. Monosyllable/Neutral. The teacher responds to students' replies with a monosyllabic word, such as “uh uh” or “ok,” or with an answer with no instructional consequences. 2. Repeat. The teacher answers to students' contributions with a partial or total repetition of the students' responses. 3. Evaluate. The teacher makes an explicit assessment of the student responses. In addition, this code included all those follow-ups which had an evaluative intention such as “are you sure of your answer?” 4. Reformulate. The teacher capitalizes on a student contribution to explore the content. Table 2 presents examples of each category. 3.3.1.1. Coding procedure. Coding of teacher talk was focused on the lessons' beginning; that is, the first 10 min of each lesson. The decision to focus on the lessons' beginning was based on theoretical and methodological reasons. On the one hand, an inspection of how teachers launch a lesson informs the way they engage students in learning and frame the ensuing instructional tasks. On the other hand, a focus on the first 10 min of the class allowed the study to code an uninterrupted segment of TT whereas a random selection of segments might have resulted in the selection of significant amounts

Table 2 Examples of questions and follow-ups Code

Examples

–Do you follow me? –Are you done? Who else has to do sentences? Inform –What I am showing here? What is this? (pointing to an illustration) –What is a noun? Implement –What does it say in here? –Which correction do we have to make in this sentence? Elaborate –Who could explain why this is the right solution to the problem? –What do you think is the main teaching of this story? Monosyllable/ –Uh, uh…a neutral Repeat –A cat!a Evaluate –A cat! Very good answer!a Reformulate –A cat. That's true. Cats are very common pets. But other animals are also pets. For instance, dogs. Not only animals can be pets. Some people have birds.a Control

a The teacher asks for an animal that is also a pet. A student answers “a cat”. Then the teacher might continue the conversation using any of the follow-ups here illustrated.

D.D. Preiss / Learning and Individual Differences 19 (2009) 1–11

of lesson time without public talk. A transcription of the lesson was used for definitive coding. Two independent coders coded the transcripts. Coders were instructed to code as follow-ups all teacher replies to students' responses that resulted from a teacher's question. All kinds of student responses were considered, namely, individual answers, group answers, or whole class answers. If there was not a teacher question anticipating a student response, however, no follow-up was identified. Replies to students that did not result from teachers' questions, such as directions or commands, were not considered. Moreover, teacher talk that is not related to instructional content or involved disciplining students was not considered. Next, estimates of consistency and consensus are presented, using indexes recommended in the literature (Stemler, 2004; Tinsley & Weiss, 2000). It is worth noting that the main goal of the coding process was getting judges to consistently apply a scoring rubric. I did not expect judges to reach a perfect consensus since these were not nominal data and, potentially, some codes could reach very large values. Consistency was estimated using Pearson's R and all of the consistency values were acceptable (r N.8). Consensus was estimated by the index of percent agreement using adjacent categories. As there were a large number of potential values to choose from, percent agreement using adjacent categories could not lead to inflated estimates of interrater reliability. Segments of possible agreement included three possible values (1–3, 4–6, 7–10, and so on). The 0 value was added to the first segment. Consensus values were acceptable for Implement, Elaborate, Monosyllable/neutral and Reformulate (p N .7) and moderate for Inform and Evaluate (p N .6). As expected, the most frequent codes — Control and Repeat — had the weakest consensus estimates, which is a result of their larger variability. There were not codes that simultaneously had low scores in both consistency and consensus measures. Since the focus of the coding process for TT was on consistency and the consensus data did not threaten the validity of my results, all of the variables were retained for analysis. After coders reached agreement, each coder assessed half of the pending material and, when data from both were available, a coder was randomly selected to provide the data for final data analysis. 3.4. Teacher talk results 3.4.1. Descriptive statistics Observation of means and standard deviations for Questions and Follow-ups shows that there is a sizeable amount of variation in the number of questions and follow-ups teachers produce. During the first 10 min of a lesson, teachers on average asked 26.53 questions (S.D. = 14.88) and 18.60 follow-ups (S.D. = 11.77). The teacher that posed the most questions asked 61 questions of the class, whereas the teacher that introduced the least number of questions made just one. Similarly, teachers posed a maximum of 59 and a minimum of zero follow-up questions. As expected, there was a high and significant correlation between Questions and Follow-ups, r = .87, p b .0001, n = 128.

5

Table 4 Rotated component matrixa for questions and follow-ups Component

Implement Evaluate Control Inform Reformulate Repeat Elaborate Monosyllabic

1

2

3

.839 .757 .626 .206 −.085 .406 −.207 .250

.050 .095 .451 .741 .740 .703 .154 .018

−.123 .067 .292 .076 −.016 .275 .801 .786

Extraction method: principal component analysis. Rotation method: varimax with Kaiser normalization. a Rotation converged in 5 iterations.

3.4.2. Main communication patterns It was my expectation that by summarizing the correlations between different kinds of questions and follow-ups, a principal component analysis would clarify what the dominant communication patterns in the lessons studied were. Consequently, an exploratory principal component analysis with varimax rotation was run through SPSS on the eight kinds of TT variables for a sample of 128 teachers. Eigenvalues for the first three components were all larger than one and — after the fourth component — changes in successive eigenvalues were small. The first three eigenvalues from a threecomponent solution are shown in Table 3. The solution explained 65.80 percent of the total variance. Extraction communality values ranged from .556 to .735. Inform, Evaluate and Reformulate had communality values marginally close to .6, so they were only moderately defined by the solution. All of the other variables had communality values over .68 and consequently were well defined by the solution. With a cut of .5 for the inclusion of a variable in the interpretation of a component, all of the variables loaded on a component, and there were no complex variables. Loadings of variables on components are shown in Table 4. Each of these components depicts a communicative cycle. The first component involves Implement and Control questions and Evaluate follow-ups. This component, then, demonstrates a procedural communicative cycle. The second component involves Inform questions and Reformulate and Repeat follow-ups. This component we will call a declarative communicative cycle. The Third component involves Elaborate questions and Monosyllable/neutral follow-ups. This component depicts what I have called an incomplete internalist communicative cycle. 3.4.3. Teacher talk and DM measures Two-tailed Pearson correlations between TT variables and DM's portfolio score were estimated. The correlation between Elaborate and the portfolio score was small but significant (r = .28; p b .002) as was the correlation between Reformulate and the portfolio score (r = .18, p b .04). No other two-tailed significant correlations between TT variables and the portfolio score were found.

Table 3 PCA solution with varimax rotation for questions and follow-ups Component

1 2 3

Initial eigenvalues

Extraction sums of squared loadings

Rotation sums of squared loadings

Total

% of Variance

Cumulative %

Total

% of Variance

Cumulative %

Total

% of Variance

Cumulative %

2.876 1.368 1.020

35.949 17.094 12.754

35.949 53.043 65.798

2.876 1.368 1.020

35.949 17.094 12.754

35.949 53.043 65.798

1.989 1.831 1.444

24.858 22.885 18.055

24.858 47.743 65.798

Extraction method: principal component analysis.

6

D.D. Preiss / Learning and Individual Differences 19 (2009) 1–11

Next, I tested whether there were significant differences within the frequencies of the four different kinds of questions teachers produced and whether there was an interaction between these and teachers' competence level according to DM. A multivariate repeated measures analysis was performed on the four question variables (Control, Inform, Implement, and Elaborate). The grouping variable was teaching competency as determined by DM. Only two of the sampled teachers had their performance assessed as Unsatisfactory, so they were dropped from the analysis, which was performed on the three other groups. Thus, the between-subject factor had three levels (Basic, Satisfactory, and Outstanding). With Pilai's criterion, the within-subjects factor was significant F (3,121) = 68.53, p b .0001, partial ή2 = .63; but the interaction was not, F (6, 244) = 1.61, p b .14, partial ή2 = .03. Thus, the mean use of different questions by teachers did not differ according to their qualifications. A priori contrasts between the different levels of the within-subject factor were run using the SPSS command for difference contrasts. The mean of Elaborate was significantly lower than that of Implement, F (1,123) = 141.69, p b .0001. The mean of Implement was significantly lower than that of Inform, F (1,123) = 82.34, p b .0001, which in turn was significantly lower than that of Control, F (1,123) = 24.08, p b .0001. Fig. 1 shows the question means per teachers' competency levels. In addition, I tested whether there were significant differences within the frequencies of the four different kinds of follow-ups teachers produced and whether there was an interaction between these and teachers' competence level according to DM. A repeated measures analysis was performed on the four follow-up variables using SPSS' GLM Repeated measures command (Monosyllabic, Repeat, Evaluate, and Reformulate). As in the previous case, the grouping factor was teaching competency. Using Pilai's criterion, the within-subjects factor was significant F (3,121) = 49.78, p b .0001, partial ή2 = .55; but the interaction was not, F (6, 244) = 1.01, p b .418,

partial ή2 = .02. Thus, teachers' means for the different kind of follow-ups were not different depending on teachers' competency. A priori contrasts between the different levels of the within-subject factor were also run using the SPSS command for difference contrasts. The mean of Monosyllabic was significantly lower than that of Repeat, F (1,123) = 103.22, p b .0001. The mean of Evaluate, however, was not significantly lower than that of Repeat, F (1,123) = .894, p b .346. The mean of Reformulate was significantly lower than that of Evaluate, F (1,123) = 124.77, p b .0001. Fig. 2 shows the followups means per teachers' competency levels. 3.5. Summary of main results of Teacher Talk study Three communicative patterns were unveiled. Two of these patterns match the two forms adopted by an externalist folk pedagogy. They show that Chilean elementary school teachers can focus their talk either on the systematic training of skills or on the delivery of information. The third pattern is reflective of teachers' lack of ability to fully implement an internalist pedagogy. When asking questions that might tap metacognitive skills or students' personal reasoning, teachers do not pursue students' contributions further. There were no significant differences in the dependent variables between teachers rated at different levels of competence of their teaching competence by DM. Across all levels of competence, teacher questions and follow-ups are primarily focused on class regulation or on information checking and scarcely on the consideration of students' thinking. However, teachers' portfolio scores had a positive correlation with the kinds of questions and follow-ups that have a greater chance to engage students in metacognitive thinking: Elaborate and Reformulate. But these are not the more dominant kinds of questions and follow-ups used by teachers, who have a preference for, first, questions that ask for simple information and, second, follow-

Fig. 1. Question means by DM competency level (n = 126).

D.D. Preiss / Learning and Individual Differences 19 (2009) 1–11

7

Fig. 2. Follow-ups means by DM competency level (n = 126).

ups that evaluate students' contributions by simple repetition of their answers. 3.6. Lesson structure study 3.6.1. Coding scheme Two dimensions were identified as key components of lesson structure. These were social organization of the lesson, and time on task. Social organization of the lesson (hereafter, SOL). Coders had to identify what was the social organization that prevailed in the lesson at the time they had to make a coding decision. Codes included: 1. Teacher-run activity: For instance, the teacher lectures, reads a story, or conducts any public activity, so the students' attention is totally focused on him or her. 2. Whole class: The class performs a joint public task, such as reading aloud together. 3. Student-run activity: An individual student performs a public activity that engages the whole class, such as making a presentation or reading aloud. 4. Team-run activity: A group, rather than an individual student, acts together in a public task. 5. Group independent-work: The focus of attention is distributed among different groups of students who are performing an activity that is shared but not public. 6. Individual independent-work: The focus of attention is on the independent-work of each individual, since the activity is not public. 7. Dialogue: The focus of attention is on a dialogue, either between the teacher and students, or among students. The students can be answering questions from the teacher, asking questions of the teacher or discussing material between them.

Time on task (hereafter, TOT). Coders had to detect what duties teachers were really performing at the moment of coding. Codes included: 1. Non-instructional: For instance, the teacher greets the students, assigns roles to perform during an activity, disciplines students or distributes materials. 2. Lecturing: The teacher presents subject content to the whole class. Students perform tasks that allow them to make sense of what the teacher says in addition to answering or asking questions. 3. Skill Focused Activities: The teacher and the students work individually or jointly on any of the following basic skills: orthography, vocabulary, grammar or phonics. 4. Reading: Student reading aloud, teacher reading aloud, silent reading or a reading comprehension activity. 5. Writing: Teacher and students work individually or jointly on any of the following writing tasks: teacher leads collective writing, individual or group student writing and silent writing. 6. Instructional play: This category includes all instructional tasks that are embedded in play-like activities. 3.6.1.1. Coding procedure. Lesson structure coding involved observation of an entire lesson. Coders had to identify key components of lesson structure every 30 s while watching the video. They were advised to start attending to the coding dimensions 5 s in advance of the particular second they had to code. Those 5 s provided coders with contextual information. If they could supply an appropriate code based on those 5 s, then they proceeded to adopt that code. If they were uncertain, they could use the following 5 s to clarify their view. All the consistency estimates are acceptable for all the Lesson Structure variables (r N.8). Most of consensus estimates are acceptable as well, although some codes have estimates that are lower than .7. In particular, codes involving very frequent activities seem to be relatively more vulnerable to disagreement (Teacher-run Activity,

8

D.D. Preiss / Learning and Individual Differences 19 (2009) 1–11

Dialogue, Skill Focused Activities), although there are exceptions to this rule (Non-instructional). Still, all of these codes had values within the moderate range (higher than .6). Thus, all of the variables were retained for analysis. 3.7. Lesson structure results The number of coded segments depended on the duration of the lesson. A total of 101 teachers taught forty-minute lessons, producing 80 segments for coding. The 26 teachers who taught lessons that lasted less than 40 min had fewer than 80 segments coded — the shorter lesson, lasting 30 min, had 60 segments coded. Given different lesson times between teachers, further analyses were made on the percentage of segments coded using a particular code. All the analysis reported here were replicated taking into account only 40-minutes lessons and the same pattern of results and very similar means and standard deviations were found. 3.7.1. Descriptive statistics Table 5 shows the means and standard deviations for all the lesson structure related variables. Inspection of the means for each variable reveals that three kinds of SOL prevailed: Dialogue, Teacher-run activities, and Independent Individual Work. On the other hand, three kinds of tasks prevailed in the lessons: Skill Focused Activities, Instructional Play and Reading. 3.7.2. Lesson structure and teacher talk Appendix A shows the Inter-correlation Matrix for TT and percentage of lesson time allocated to SOL measures. TT measures and SOL measures have systematic correlations, suggesting that both sets of measures are valid. Dialogue had significant positive correlations with all the TT variables but Reformulate whereas the less dialogical code Teacher-run Activity had significant negative correlations with Inform, Implement and Repeat. On the other hand, as expected, the measures of Group Independent Work correlate negatively with frequent kinds of TT (Implement and Evaluate). Unexpectedly, both Student-run Activity and Team-run Activity correlated positively with Monosyllable, which was a very infrequent kind of follow-up. As shown in Appendix B, TT measures and percentage of lesson time allocated to TOT measures have systematic correlations as well, suggesting that both sets of measures are valid. Skill Focused Activities correlated positively with Implement, Evaluate, and Control, and negatively with Elaborate. Both Reading and Writing displayed negative correlations with Implement, and Evaluate. Finally, Instructional Play had a negative correlation with Implement. The results suggest a clear pattern: lessons initiated using a procedural communicative cycle, as shown by the Teacher Talk study coding,

Table 5 Lesson structure means and SDs (n = 127)

Social Arrangement

Task

Code

Mean

S.D.

Teacher-run activity Student-run activity Whole class Team-run activity Dialogue Group independent-work Individual independent-work Non-instructional Lecturing Reading Writing Skill focused activities Instructional play

19.15 4.03 1.91 1.06 24.85 8.63 19.54 7.63 5.54 10.39 8.16 33.41 13.25

7.53 4.98 2.69 2.79 12.24 15.64 15.80 7.18 6.03 17.06 17.60 25.90 17.48

develop into a skill focused lesson, as shown by the Lesson Structure study coding. 3.7.3. Lesson structure and DM measures Two-tailed Pearson correlations between lesson structure variables and DM's portfolio score were estimated. The correlation between the Percentage of Dialogue and the portfolio score was small but significant (r = .18; p b .04). No other two-tailed significant correlations between lesson structure variables and the portfolio score were found. As previously, the two sampled teachers whom had their performance assessed as Unsatisfactory were dropped from the analysis, which was performed on the three other groups. To test the hypothesis of an interaction between the two most frequent forms of SOL and teacher competency a multivariate repeated measures analysis was performed, using SPSS' GLM command, on the two variables measuring the Percentage of Teacher-run activities and the Percentage of Dialogue. The grouping factor was DM teaching competency levels (Basic, Satisfactory, and Outstanding). Box's M test was not significant (p b .907), so there is no evidence of heterogeneity of variance–covariance matrices. With Wilks' criterion, the within-subjects effect was significant, F (1, 122) = 17.18, p b .0001, partial ή2 = .12. However, there was no significant interaction between the within-subjects factor and DM teaching competency levels, F (2,122) = 1.50, p b .226, partial ή2 = .02. In summary, teachers had a larger percentage of segments coded as Dialogue than the percentage of segments coded as Teacher-run activities, independently of their level of teaching competency. To test the hypothesis of an interaction between the dominant TOT variables and teacher competency, a multivariate repeated measures analysis was performed using SPSS' GLM command, on the variables measuring the Percentage of Instructional Play, the Percentage of Skill Focused Activities, and the Percentage of Reading and Writing, which were grouped into a new variable named Percentage of Meaning Focused Activities. As reflected by their expected normal probability plots the variables presented some skewness and kurtosis. Percentage of Instructional Play was positively skewed and had positive kurtosis. Percentage of Skill Focused Activities had negative kurtosis. Percentage of Meaning Focused Activities was positively skewed. Univariate outliers were estimated by an inspection of z scores. Cases with standardized scores in excess of 3.29 (.001, two-tailed test) were considered potential outliers. Two univariate outliers were found for Percentage of Instructional Play and they were deleted. Multivariate outliers were inspected using the SPSS' REDISUALS = OUTLIERS (MAHAL) syntax. Any case with a Mahalanobis distance greater than χ2 (3) = 16.266 was considered a multivariate outlier. Two multivariate outliers were found and they were deleted. After data screening, multivariate analyses were run using SPSS' GLM module. The grouping factor was DM teaching competency (Basic, Satisfactory, and Outstanding). Box's M test was not significant (p b .99), so there was no evidence of heterogeneity of variance–covariance matrices. Mauchly's test of Sphericity was significant (p b .0001) so sphericity cannot be assumed. Consequently, the Greenhouse–Geisser criterion was used. There were differences between the different levels of the within-subject factor, F (1.581, 186.508) = 16.616, p. b 0001, partial ή2 = .123. However, the interaction was not significant F (3.161, 186.508) = 1.186, p b .317, partial ή2 = .02. A priori contrasts between the different levels of the within-subject factor were also run using the SPSS command for difference contrasts. The mean of Percentage of Instructional Play was significantly lower than that of Percentage of Meaning Focused Activities, F (1,118) = 40.56, p b .0001. The mean of Percentage of Meaning Focused Activities was significantly lower than that of Percentage of Skill Focused Activities, F (1,118) = 8.86, p b .004. To check whether the shape of the distributions had an impact on the results, a square root transformation was applied to

D.D. Preiss / Learning and Individual Differences 19 (2009) 1–11

the two variables that were positively skewed and the data were re-analyzed. Similar results were reached. 3.7.4. Summary of main results of lesson structure study There were no significant differences in the dependent variables between teachers rated at different levels of competence of their teaching competence by DM. Across all levels of competence, most of the public attention is focused on teachers. When students are performing private work, most of the time they are working alone. However, teachers' portfolio scores had a positive correlation with Teacher-lead dialogue. The content preferred by teachers involves Skill Focused Activities and, second, Meaning Focused Activities, which involve mostly reading and, very scarcely writing. 4. Discussion The results confirmed my central main hypotheses. Most of Chilean elementary school teachers are clearly aligned with an adult-run model of education. Evidence for adherence to this pedagogical approach can be found in (a) teacher questioning and teacher reactions to students' contributions, and (b) the place accorded to the teacher in the social organization of the lesson. The teaching pattern uncovered here has its roots in the intuitive pedagogies endorsed by Chilean teachers, which in turn are based on the models of pedagogy that have shaped Chile's educational system. Although most of the teachers observed are aligned with an externalist pedagogy, a small number of them deviate from this pattern. As the results are correlational it is difficult to determine what proportion of teachers departed frequently from traditional patterns. However, by way of illustration, only around 10-percent of the teachers produced more than 5 Elaborate questions, and 5percent of the teachers produced more than 5 Reformulate followups. Thus, roughly 10% of teachers appeared to depart from the prevailing canon of teaching, an estimate that is consistent with the percentage of teachers assessed by DM as outstanding (9.4) in the universe of elementary school teachers assessed in 2003. A majority of Chilean elementary school teachers address their students by means of objective and informative talk. Teacher questioning is primarily focused on class regulation or on checking for information. When teachers are not focused on class regulation or on checking for information, they tend to spend time fostering particular verbal skills. Teachers less frequently ask students to elaborate; that is, to make claims based on students' personal beliefs or to form hypothesis about the result of an instructional activity. These trends suggest, then, that Chilean teachers' talk is not that different from the talk of teachers in other parts in the world, in particular that of North American teachers, although it does differ from the relatively more elaborative talk of Japanese teachers (Stigler & Hiebert, 1999). The pattern of lesson structure found in Chilean classrooms differs from the patterns that appear to predominate elsewhere. While the Chilean lessons engaged the teacher working with the class as a whole, more than 60% of the time, summaries of existing research in the United States indicated that 65% of the lesson is commonly devoted to seat-work (Gallego, Cole, & The Laboratory of Comparative Human Cognition, 2001). The teachers can be leading short communicative exchanges back and forth with students, or they provide a lecture. In each case, students play a marginal public role, either individually or in groups. When students are performing private work, most of the time they are working individually. Despite the fact that most students' desks are arranged in a way that permits group work, either sitting at shared tables or in front of each other, private collaborative activities are scarce. The data suggest that, as in the United States (Alexander, 2000), Chilean lessons are highly irregular in structure and contents. Recent data

9

from new groups of teachers assessed by DM have shown that lack of structure is one of the main weaknesses of the lessons assessed (Manzi, 2007). Consequently, this is one of the areas calling for further research. The study has obvious limitations. This study does not present data linking teachers' pedagogies with students' achievement scores, as its focus was on the task-sense of teaching. However, future research should explore whether the prevailing practices are related to the achievement issues currently challenging Chilean education, specifically because the lack of a clear lesson structure and low use of elaborative talk have been implicated in educational deficits in other educational systems, such as the United States (Stigler et al., 1999). Second, the assessment context and, particularly, the requirement that teachers include at least 10 min of lecturing might bias a study of lesson structure. Although these results should be validated with lessons filmed in a non-assessment context, one might argue that the strength of the cultural pattern of teaching might compensate for this bias. Last but not least, my results provide clear support to the hypothesis that there are country specific patterns of teaching (Givvin, Hiebert, Jacobs, Hollingsworth, & Gallimore, 2005). Although the relation between cultural patterns of teaching and educational achievements is still a controversial one, future public policy initiatives for educational reform should take into consideration the cultural basis of the teaching patterns they want to transform. It is not clear that the recommendations from educational theories originating outside the Chilean context can be adopted without modification, due to issues of cultural compatibility raised in this paper. The time may have come for Chilean researchers, together with practicing teachers, to look to their own instructional practices and to integrate them into their scientific practices in order to find a way out of the current deficits affecting Chilean contemporary education. However, this integration has to meet contemporary educational advances, and has to look for an elective affinity with those theories that, produced outside of Chile, are more suitable to work well with the peculiarities of the Chilean imagination. For instance, it is interesting to note that a significant number of teachers assessed here produced activities that could be only coded as instructional play, which suggest that they have an untapped skill for framing content in narrative ways. As present in the videos, there is no clarity regarding the instructional values of these events coded as instructional play: they might facilitate learning, or not. However, these activities can become the groundwork of an alternative to the more conventional practice I observed repeatedly in the lessons. A closer inspection of these activities is certainly merited. Acknowledgements The research initiative reported in this paper was funded by a research grant of the Pontificia Universidad Católica de Chile's VRAID and a dissertation fellowship of the Yale Graduate School. Grant Number 11060389 from FONDECYT supported preparation of this paper. Special thanks are given to the Centro de Medición Mide UC, which granted the author access to all the databases and material used for this study. Mide UC as well funded the reproduction of tapes and written portfolios for further analysis. The current study was implemented as one of the main studies of the author's Ph.D. dissertation at Yale University. The author thanks the members of his dissertation committee for their valuable recommendations and support: Michael Cole, Elena Grigorenko, Jorge Manzi, Peter Salovey and Robert J. Sternberg, who acted as well as his main advisor. The study required the work of a large team of research assistants, without whom this work would have not been possible: Angela Haquin, Daniela Jimenez, Susana Mendive, Monica Nuñez, Carmen Ocares, Francisca Ortega, and Pia Vergara.

10

D.D. Preiss / Learning and Individual Differences 19 (2009) 1–11

Appendix A. Inter-correlation Matrix for Teacher Talk and percentage of lesson time allocated to social organization of the lesson measures (n = 124)

1. Control 2. Inform 3. Implement 4. Elaborate 5. Monosyllabic 6. Repeat 7. Evaluate 8. Reformulate 9. Teacher-run activity 10. Student-run activity 11. Whole class 12. Team-run activity 13. Dialogue 14. Group independent-work 15. Individual independent-work

1

2

3

4

5

6

7

8

9

10

11

12

13

14

15

1.000

.358⁎⁎ 1.000

.412⁎⁎ .109 1.000

.107 .089 −.179⁎ 1.000

.357⁎⁎ .184⁎ .136 .311⁎⁎ 1.000

.625⁎⁎ .615⁎⁎ .361⁎⁎ .241⁎⁎ .262⁎⁎ 1.000

.460⁎⁎ .281⁎⁎ .462⁎⁎ .065 .117 .277⁎⁎ 1.000

.289⁎⁎ .265⁎⁎ .099 .134 .079 .291⁎⁎ .056 1.000

− .169 − .202⁎ − .194⁎ − .007 − .165 − .245⁎⁎ − .147 − .169 1.000

− .115 − .049 − .097 .050 .248⁎⁎ − .062 .017 − .041 − .016 1.000

.058 .003 .062 − .119 − .134 − .020 .059 .072 .052 − .044 1.000

−.068 −.013 −.154 .000 .196⁎ −.195⁎ −.139 .000 .093 .032 .029 1.000

.351⁎⁎ .377⁎⁎ .269⁎⁎ .193⁎ .291⁎⁎ .294⁎⁎ .264⁎⁎ .143 −.065 .085 −.103 −.096 1.000

−.160 −.135 −.216⁎ −.119 −.067 −.076 −.242⁎⁎ .081 −.277⁎⁎ −.109 −.067 .081 −.416⁎⁎ 1.000

.006 −.047 .147 −.022 −.174 .019 .117 −.114 −.156 −.267⁎⁎ −.043 −.245⁎⁎ −.324⁎⁎ −.528⁎⁎ 1.000

⁎⁎Correlation is significant at the 0.01 level (2-tailed). ⁎Correlation is significant at the 0.05 level (2-tailed).

Appendix B. Inter-correlation Matrix for Teacher Talk and percentage of lesson time allocated to time on task measures (n = 124)

1. Control 2. Inform 3. Implement 4. Elaborate 5. Monosyllabic 6. Repeat 7. Evaluate 8. Reformulate 9. Non-instructional 10. Lecturing 11. Reading 12. Writing 13. Skill focused activities 14. Instructional Play

1

2

3

4

5

6

7

8

9

10

11

12

13

14

1.000

.358⁎⁎ 1.000

.412⁎⁎ .109 1.000

.107 .089 −.179⁎ 1.000

.357⁎⁎ .184⁎ .136 .311⁎⁎ 1.000

.625⁎⁎ .615⁎⁎ .361⁎⁎ .241⁎⁎ .262⁎⁎ 1.000

.460⁎⁎ .281⁎⁎ .462⁎⁎ .065 .117 .277⁎⁎ 1.000

.289⁎⁎ .265⁎⁎ .099 .134 .079 .291⁎⁎ .056 1.000

−.110 .021 −.028 .067 .070 −.092 −.042 −.188⁎ 1.000

.183⁎ .230⁎ .094 −.006 −.046 .211⁎ .015 .238⁎⁎ −.111 1.000

−.160 −.069 −.312⁎⁎ .150 .023 −.166 −.188⁎ −.058 −.091 −.104 1.000

− .172 − .117 − .229⁎ .101 .133 .024 − .243⁎⁎ .037 − .144 − .037 − .041 1.000

.250⁎⁎ .054 .507⁎⁎ −.259⁎⁎ −.097 .131 .360⁎⁎ .012 −.084 −.089 −.508⁎⁎ −.467⁎⁎ 1.000

−.060 .010 −.231⁎⁎ .105 −.027 −.098 −.084 −.013 −.009 −.072 −.115 −.223⁎ −.425⁎⁎ 1.000

⁎⁎Correlation is significant at the 0.01 level (2-tailed). ⁎Correlation is significant at the 0.05 level (2-tailed). Listwise N = 124.

References Alexander, R. (2000). Culture and Pedagogy. International comparisons in primary education. London: Blackwell Publishing. Andrade, H., & Perkins, D. (1998). Learnable intelligence and intelligent learning. In R. J. Sternberg & W. M. Williams (Eds.), Intelligence, instruction and assessment: Theory into practice (pp. 67−94). Mahwah, NJ: Lawrence Erlbaum Associates. Brophy, J. (2000). In A. Kazdin (Ed.), Classrooms: Processes and managementEncyclopedia of psychology, Vol. 2 (pp. 95−102). Washington, DC: American Psychological Association. Bruner, J. S. (1996). The Culture of Education. Cambridge, MA: Harvard University Press. Danielson, C. (1996). Enhancing professional practice: a framework for teaching. Alexandria, VA: Association for Supervision and Curriculum Development. Foorman, B. R., Goldenberg, C., Carlson, C., Saunders, W., & Pollard-Durodola, S. D. (2004). How teachers allocate time during literacy instruction in primary-grade English language learner classrooms. In P. McCardle & V. Chhabra (Eds.), The Voice of Evidence (pp. 289−328). Baltimore, MD: Brookes Publishing Co. Foorman, B. R., & Schatschneider, C. (2003). Measuring teaching practice during reading/ language arts instruction and its relation to student achievement. In S. Caughn & K. L. Briggs (Eds.), Reading in the classroom: systems for observation of teaching and learning (pp. 1−30). Baltimore, MD: Brookes Publishing. Gallego, M. A., & Cole, M.The Laboratory of Comparative Human Cognition. (2001). Classroom cultures and cultures in the classroom. In V. Richardson (Ed.), Handbook on research on teaching (pp. 951−997) 4th ed. Washington, D. C.: American Educational Research Association. Givvin, K. B., Hiebert, J., Jacobs, J., Hollingsworth, H., & Gallimore, R. (2005). Are there national patterns of teaching? Evidence from the TIMSS 1999 Video Study. Comparative Education Review, 49(3), 311−343. Haye, A., & Pacheco, V. (1995). El papel de las representaciones sociales en la crisis de la educacion chilena. [The role of social representations in the crisis of Chilean education.]. Psykhe, 4(1), 25−38.

Hiebert, J., Gallimore, R., Garnier, H., Givvin, K. B., Hollingsworth, H., Jacobs, J., et al. (2003). Teaching Mathematics in Seven Countries: Results From the TIMSS 1999 Video Study, (No. NCES 2003-013 Revised). Washington, DC: U.S. Government Printing Office. Loera, A. (2006). La práctica pedagógica videograbada. [Teaching practices videotaped]. México, DF: UPN. Manzi, J. (2007). Evaluación Docente: Antecedentes, resultados y proyecciones [Electronic Version]. Seminario Evaluación Docente en Chile: Fundamentos, experiencias y resultados. Retrieved January 2, 2008, from http://mideuc.cl/seminario/presentaciones.html Manzi, J., Preiss, D., Flotts, González, R., & Sun, Y. (2008, March). Design and implementation of a national project of teaching assessment. Paper presented at the annual meeting of the American Educational Research Assocation, New York, NY. Mehan, H. (1979). Learning lessons: social organization in the classroom. Cambridge, MA: Harvard University Press. Ministerio de Educación (2003). Marco para la buena enseñanza. [Framework for good quality teaching.]. Santiago, Chile: CPEIP. Morandé, P. (1984). Cultura y modernización en América Latina. [Culture and modernization in Latin America.]. Santiago: Instituto de Sociología, Pontificia Universidad Católica de Chile. Newman, D., Griffin, P., & Cole, M. (1989). The construction zone: working for cognitive change in school. New York: Cambridge University Press. Olson, D. R. (1994). The world on paper: the conceptual and cognitive implications of writing and reading. New York: Cambridge University Press. Olson, D. R. (2003). Psychological theory and educational reform: how school remakes mind and society. Cambridge, MA: Cambridge University Press. Olson, D. R. (2005). Technology and intelligence in a literate society. In R. J. Sternberg & D. D. Preiss (Eds.), Intelligence and Technology. The impact of tools on the nature and development of human abilities (pp. 55−68). Mahwah, NJ: Lawrence Erlbaum Associates.

D.D. Preiss / Learning and Individual Differences 19 (2009) 1–11 Olson, D. R., & Astington, J. W. (1993). Thinking about thinking: Learning how to take statements and hold beliefs. Educational Psychologist, 28(1), 7−23. Olson, D. R., & Bruner, J. S. (1996). Folk psychology and folk pedagogy. In D. R. Olson & N. Torrance (Eds.), The handbook of education and human development: new models of learning, teaching, and schooling (pp. 9−27). Cambridge, MA: Blackwell Publishers. Paris, S. G., & Paris, A. H. (2001). Classroom applications of research on self-regulated learning. Educational Psychologist, 36(2), 89−101. Rogoff, B., Matusov, E., & White, C. (1996). Models of teaching and learning: participation in a community of learners. In N. Torrance, & D. R. Olson (Eds.), The Handbook of Education and Human Development (pp. 388−414). Cambridge, MA: Blackwell. Sinclair, J. M., & Coulthard, M. (1975). Towards an analysis of discourse: the English used by teachers and pupils. London: Oxford University Press. Stemler, S. E. (2004). A comparison of consensus, consistency and measurement approaches to estimating interrater reliability. Retrieved June 20, 2005, from http:// PAREonline.net/getvn.asp?v=9&n=4 Stevens, J. (2002). Applied multivariate statistics for the social sciences, 4th ed. Mahwah, N.J.: Lawrence Erlbaum Associates.

11

Stigler, J. W., Gallimore, R., & Hiebert, J. (2000). Using video surveys to compare classrooms and teaching across cultures: Examples and lessons from the TIMSS video studies. Educational Psychologist, 35, 87−100. Stigler, J. W., Gonzales, P., Kawanaka, T., Knoll, S., & Serrano, A. (1999). The TIMSS Videotape Classroom Study: methods and findings from an exploratory research project on eighth-grade mathematics instruction in Germany, Japan, and the United States (Research and Development Report No. NCES 1999-074). Washington, D.C.: U.S. Government Printing Office. Stigler, J. W., & Hiebert, J. (1999). The Teaching Gap. Best ideas from the world's teachers for improving education in the classroom. New York: The Free Press. Tabachnick, B. G., & Fidell, L. S. (2001). Using multivariate statistics. Boston: Allyn and Bacon. Tinsley, H. E. A., & Weiss, D. J. (2000). Interrater reliability and agreement. In H. E. A. Tinsley & S. D. Brown (Eds.), Handbook of Applied Multivariate Statistics and Mathematical Modeling (pp. 95−124). London, UK: Academic Press. Wells, G. (1999). Dialogic inquiry: towards a sociocultural practice and theory of education. New York: Cambridge University Press.