Primary teachers' perceptions of baseline assessment in mathematics

PERGAMON

Studies in Educational Evaluation Studies in Educational Evaluation 25 (1999) 109-130

PRIMARY TEACHERS' PERCEPTIONS OF BASELINE ASSESSMENT IN MATHEMATICS

L. Kyriakides and R.J. Campbell Institute of Education, University of Warwick, UK

Introduction The last decade has witnessed a growing recognition of the need for signi.ficant changes in educational assessment practices (Shepard, 1989; Webb & Coxford, 1993). The calls for reform are directed not only at large scale, standardised tests but also at classroom assessment practices (DES, 1987; NCTM, 1989). There are at least three important factors which have contributed to the demands for assessment reform. The first one has to do with the changing nature of educational goals which has influenced not only the content of assessment but also the purposes of assessment (Chambers, 1993). Another factor contributing to the need for rethinking the purposes and methods of assessment is the relationship between learning, teaching and assessment. The assessment process is nowadays seen as an integral part of the educational process (Desforges, 1989; Stenmark, 1992). Finally, the limitations of current methods of recording performance and reporting credit has been pointed out by educators (Pole, 1993) and this has been an issue taken into account for the development of policy for assessment reform (DES, 1991). A theme that runs through the discussion of assessment reform and is addressed in this paper is the assessment of all children at a very early stage in their school careers which was originally developed in order to identify early children with special educational needs: baseline assessment. In this paper we attempt to deal with the purposes, processes and policies which drive, or follow from, baseline assessment. We draw on the findings of an investigation into perceptions of baseline assessment in Mathematics held by teachers who work in the highly centralised system of Cyprus.

0191-491X/99/$ - see front matter © 1999 Published by Elsevier Science Ltd. All rights reserved. PII: S0191-491 X(99)0 0016-4

110

L. Kyriakides and R. J. Campbell/Studies in Educational Evaluation 25 (1999) 109-130

One of the main characteristics of the educational system in Cyprus is that its administration is centralised and the primary schools are considered as government, and not as community, institutions. The maintenance of the centralised system has historical and political origins (Kyriakides, 1994) but also a decentralised system in a small country like Cyprus would be very demanding in manpower. With 367 schools and 2900 teachers in the primary school system, it has the same administrative range as a large English Local Educational Authority. Primary education is under the authority of the Ministry of Education, which is responsible for the educational policy making, the administration of education, and the enforcement of educational laws. Primary schools provide for six-year compulsory schooling for children from the age of 5 years and six months. There are no entrance requirements and the attendance rate is 100%. Curricula for pre-primary, primary and secondary education are prescribed by the Ministry of Education. There is a statutory time allocation for each subject. In 1994, a reform programme in mathematics common to all primary year groups was introduced which was mainly concerned with content, pedagogy and assessment (Ministry of Education, 1994). This reform can be seen as the first systematic attempt of the Ministry of Education to establish the base upon which assessment policy in Cyprus could be developed (Kyriakides, 1994; UNESCO, 1997). However, the reform was not based on any attempt to evaluate standards and to identify the strengths and weaknesses of the educational system. Two previous attempts by the Ministry of Education to investigate standards in Mathematics at the end of the 1970s were never published. It can not be therefore claimed that this reform has been designed for the specific conditions of Cyprus. The reform lacked consideration of the complex process of change which should be based on a diagnosis of the need for change. This has meant that Cypriot policy remains dependent on policy borrowing l~om other systems, namely the English and Greek educational system (Kyriakides, 1994). In 1994-95 Cyprus participated for first time in an international comparative study, namely the Third International Mathematics and Science Study (TIMSS), which was conducted under the auspices of IEA (Intemational Association for the Evaluation of Educational Achievement). The TIMSS study provided a wealth of information about curricula, teaching methods and school organisation in various countries, which was collected in a systematic way at the same point in time (Schmidt, McKnight, Valaverde, & Wiley, 1997). The results of TIMSS revealed that overall performance in mathematics in Cyprus was low, and identified a need for developing a policy on assessment which could provide information about how well the educational system is operating. In this context baseline assessment would become necessary in order to measure school effectiveness. However, the Ministry of Education has never provided guidance for teachers on how to assess pupils entering the primary school, and there were no instruments which could be used to assess such pupils. Thus, the main purpose of this research was to examine Cypriot teachers' perceptions of baseline assessment in mathematics and this study can be seen as a first step in creating evidence on how policy on baseline assessment can be developed.

L. Kyriakides and R. J. Campbell/Studies in Educational Evaluation 25 (1999) 109-130

111

A long trend in the literature (Fullan, 1991; Nisbet, 1973) supports the view that teachers' perceptions are one of the most critical factors for the effectiveness of the various models of curriculum change (Havelock, 1971) and particularly for the effectiveness of the centre-periphery model used in Cyprus (Kyriakides, 1996a). Research into teachers' beliefs about teaching and learning mathematics revealed that teachers' beliefs affect the form and type of instruction they deliver (Clark & Peterson, 1986; Pajares, 1992). It has been also claimed that if teachers' perceptions are compatible with the underlying philosophy and materials of a curriculum, there is greater likelihood that the curriculum will be implemented (Kyriakides, 1994; Richardson, 1990). The importance of teachers' perceptions is also supported by research on teachers' thinking (Day, Calderhead, & Denicolo, 1993; Zeichner, Tabachnick, & Densmore, 1987). Calderhead (1987) points out that research into teachers' thinking shows "how unrealistic it is to conceive of innovation as a set of pre-formulated ideas or principles to be implemented by teachers." (p. 17). Thus, understanding teachers' perceptions of baseline assessment is necessary for any attempt to develop a policy on baseline assessment.

Methods A randomly selected sample of Cypriot primary teachers (n=390) was surveyed by questionnaire (Appendix A), so as to establish a representative picture of the perceptions of primary teachers in Cyprus. The content of the questionnaire was derived from analysis of assessment policy in Cyprus and relevant literature on baseline assessment. There were the following four broad areas of teachers' perceptions. (A)

Teachers' Perceptions of Purposes of Baseline Assessment

There are four reasons why all school systems must have a strategy for finding out about pupils on entry (Blatchford & Cline, 1992). First, baseline assessment may produce information about what children know and what they do not know in order to help teachers decide how to meet children's learning needs. Although young pupils follow recognised patterns of development, within the group of pupils entering primary school there would be considerable variation between individuals both in their rate of learning and also in their level of attainment. Thus, information gathered t~om baseline assessment in mathematics may help teachers to plan their teaching mathematics to first year pupils. Second, information gathered from baseline assessment may also enable teachers and parents to make comparisons across pupils entering primary school. The third purpose has to do with the historical origin of baseline assessment, which is in the early identification of children with special educational needs (Lindsay, 1997). During the 1970s a number of psychologists in the UK (e.g., Lindsay & Pearson, 1981; Weddell & Raybuld, 1976; Wolfendale & Bryans, 1979) attempted to develop approaches to help teachers to consider early identification within the infant years rather than once the children had transferred to the junior school and could be seen to be failing to achieve at a satisfactory pace. As this approach moved down the age range to children entering school it became

112

L. Kyriakides and R. J. Campbell/Studies in Educational Evaluation 25 (1999) 109-130

referred to as baseline approach, Thus, information gathered from baseline assessment in mathematics may be used for identifying pupils with learning difficulties. Fourth, baseline assessment, as its name implies, provides the base from which pupils' subsequent educational progress can be measured. Measures of the educational progress made by pupils in a school, relative to that made by similar pupils in other schools, have come to be called "value added" assessment. Information gathered l~om value added assessment is more valid in exploring the effectiveness of a school unit than using outcome data only since variations in the final test results of schools reflect partly the educational attainment of pupils when they enter the school (DFEE, 1995; Fitz-Gibbon, 1995; Strand, 1997). A decision was taken to ask respondents to rank the purposes of baseline assessment in mathematics presented above. This is because in Cypriot culture assessment has been conceptualised unproblematically as a summative activity. This is reflected in the policy documents of the Ministry of Education which do not refer to the purposes of baseline assessment as though it were self-evident. Another reason is that in the recent literature there is widespread doubt that the summative and formative purposes can be achieved in a single set of assessment arrangements (Brown, 1991, pp 217-218; Kimberley, Hextall, Torrance, & Moon, 1989, p.236). (B)

Methodsof Baseline Assessment

There was concern over significant aspects of policy on baseline assessment considered important for the achievement of the purposes of baseline assessment. For example, the way in which Cypriot teachers understand the conceptual dimensions of value-added assessment and the practical dimensions it implies was examined. Special attention was also given to the content of baseline assessment and especially to whether the attitudes towards mathematics of pupils entering primary schools and their ability to apply mathematics in unfamiliar situations should be assessed. (C)

Techniquesof Baseline Assessment

The assumption that some techniques of baseline assessment are more appropriate than others was explored. In addition, an examination of the relationship between the appropriateness and ease of the various assessment techniques was needed, given the evidence about problems of manageability of assessment policy in other countries (e.g. Campbell & Neil, 1994; ENCA, 1992). (D)

Waysof Improving Assessment Practice

Finally, in order to identify how a policy on assessment will meet the needs of Cypriot teachers, their perceptions of the relative importance of the following six ways of improving assessment practice were investigated: (1) further training in baseline assessment; (2) time free of class contact; (3) smaller class size; (4) other adult in the class while assessment is occurring;

L. Kyriakides and R. J. Campbell/Studies in Educational Evaluation 25 (1999) 109-130

(5) (6)

t 13

non-statutory guidelines on baseline assessment, and a performance test published by the Ministry of Education.

Of the 390 Cypriot teachers approached 297 responded, a response rate of 76%. The response rate implies that the findings are generalisable to the population. The reliability of each of the scales which measured the dependent variables was calculated using Cronbach's Alpha. The values of Cronbach's Alpha for the four scales used to measure teachers' perceptions were high (0.74 up to 0.83). Semi-structured interviews with 15 teachers who responded to the questionnaire were also conducted in order to test the validity of the questionnaire findings by matching the qualitative data derived from interview with each teacher against the quantitative data gathered by his/her individual questionnaire. A measure of match was derived by comparing relevant parts of the questionnaire with the interview data gathered in this study. Although this measure does not necessarily imply high validity since it is possible that they are both invalid, the use of both questionnaire and interview methods provides a basis for triangulation of data (Cohen & Manion, 1994). Findings from the Questionnaire

Purposes of Assessment Figure 1 deals with teachers' perceptions of purposes of baseline assessment. The mean ranks of the perceived importance of each purpose, with 3 as the highest point and 0 as the lowest, are displayed. The Kendall Coefficient of Concordance was calculated to show the degree of consensus about purposes of baseline assessment in this ranking. A significant level of agreement amongst Cypriot teachers was revealed (W=.72, Z=6.24, V~=3.99, V2=714, p<.001). ~ The following observations arise from Figure 1. Formative assessment was considered to be the most important by almost all the teachers (95%) and thereby its mean rank (2.94) is very close to 3.00. The next most important purpose of baseline assessment is the early identification of pupils with learning difficulties which has a mean rank close to 2.00 and 80% of teachers considered it to be the second most important purpose. Since the early identification of pupils with learning difficulties and formative baseline assessment has direct feedback into the teachers' own teaching, it can be inferred that Cypriot teachers considered that the most important purpose for baseline assessment was to help them make decisions about their own teaching. It is also of interest to emphasise the low rating given to summative purposes of assessment and to value-added assessment. Their mean ranks are close to 0.50 and 1.00, respectively, which means that they are clearly differentiated from the other two purposes. As far as the summative purpose is concerned, almost all the teachers (85%) saw it as the least important purpose. Similarly, 87% saw value added assessment as the second least important purpose. Thus, Cypriot teachers perceived formative purposes of baseline assessment as more important than the purpose related to the summative and to the value added assessment.

114

L. Kyriakides and R. J. Campbell / Studies in Educational Evaluation 25 (1999) 109-130

2.94 3,

Mean 2.5,

Rank 2,

1.5.

0.5,

fl

6

C

D

Figure 1: Teachers' Perceptions of Purposes of Baseline Assessment (A: value-added assessment; B: early identification of pupils with learning difficulties; C: formative assessment; D: summative assessment)

Methods of Assessment The figures in Table 1 are based on the information derived from teachers' responses to items 12 - 27 of the questionnaire concerned with the implementation of policy on baseline assessment. Percentages of teachers agreeing and disagreeing with methods of baseline assessment, medians and modes are shown in Table 1. The following observations arise from Table 1. First, more than 70% of Cypriot teachers considered baseline assessment to be an essential part of planning and teaching mathematics to first year pupils. They argued that baseline assessment could help teachers to prepare their programmes more effectively. Moreover, almost four out of five teachers (78%) thought that information gathered from baseline assessment should be used for organising their classroom. Second, 75% of teachers supported that policy makers should use information gathered from baseline assessment in order to evaluate the current curriculum reform in mathematics. Third, more than two out of three Cypriot teachers (68%) supported the idea that baseline assessment should be used to help teachers to identify pupils with special needs and less than 15% of them (14%) rejected this idea. This implies that Cypriot teachers as a group considered important the early identification of pupils with special educational needs. On the other hand, 82% of them supported the idea that information gathered from baseline assessment should not be used for identifying low and high attainers. This

L. Kvriakides and R. J. Campbell/Studies in Educational Evaluation 25 (1999) 109-130

115

implies that Cypriot teachers as a group rejected the idea that baseline assessment should be used for labelling children. Table 1: Percentages of Cypriot Teachers Who Agree and Those Who Disagree With the Following Methods of Baseline Assessment, and Their Medians, and Modes No

Issues of policy on baseline assessment

(1)

Baseline assessment is an essential part of a) planning how to teach mathematics b) teaching mathematics to first year pupils

12.4 10.0

Information gathered from baseline assessment help teachers to organise their classroom.

(2)

% of teachers who Disagree* Agree**

Median

Mode

71.4 75.6

4.00*** 4.00

5.00 4.00

11.8

78.1

4.00

4.00

(3)

Information gathered from baseline assessment help us to evaluate the reform in mathematics

14.8

75.1

4.00

4.00

(4)

Information gathered from baseline assessment should be used for measuring pupils' progress.

14.8

75.1

4.00

4.00

(5)

Information gathered from baseline assessment should be used for identifying: a) individual pupils with special educational needs b) low and high attainers.

14.1 82.3

68.8 10.2

4.00 2.00

4.00 2.00

(6)

Baseline assessment on the basis of products than process.

59.2

26.3

2.00

2.00

(7)

Record keeping of the results of baseline assessment is an essential part of assessment.

10.4

82.4

4.00

4.00

(8)

Pupil's result of baseline assessment should be: a) announced to the whole class b) discussed between parents and teacher c) discussed between the pupil and the teacher.

89.1 1.3 21.4

5.3 93.3 68.4

2.00 4.00 4.00

1.00 5.00 4.00

Final school result in national examinations is a fair indicator of school performance.

87.2

10.1

2.00

2.00

School effectiveness in mathematics should be evaluated by measuring the educational progress made by the pupils of the school to that made by similar pupils in other schools.

25.1

48.9

3.00

4.00

Whole school decision making about methods of teaching and assessment is important.

25.4

36.4

3.00

3.00

21.4

50.0

3.50

4.00

24.5

41.4

3.00

3.00

(9)

(10)

(11)

(12)

(13)

Baseline assessment in mathematics should refer: a) to pupil's attitudes to mathematics b) to pupil's ability to apply their knowledge in unfamiliar situation.

Each school should develop its own policy in: a) baseline assessment 36.4 29.4 3.00 3.00 b) value added assessment 42.4 21.4 3.00 3.00 * This group of teachers either disagree or absolutely disagree ** This group of teachers either agree or absolutely agree *** 1: Absolutely disagree; 2: Disagree; 3: Do not know/cannot say; 4: Agree: 5: Absolutely agree

116

L. Kyriakides and R. J. Campbell/Studies in Educational Evaluation 25 (1999) 109-130

Fourth, 75% of teachers supported the idea that information gathered from baseline assessment should be used for measuring pupils' subsequent educational progress. Moreover, the great majority of Cypriot teachers (87%) did not consider the final schools results in national examinations as a "fair" indicator of school performance. But even if half of them (49%) agreed that information gathered from value added assessment should be used for exploring the effectiveness of a school unit, a quarter of them did not support the adoption of the technique of value-added assessment for measuring school effectiveness. It may be claimed that although Cypriot teachers, as a group, considered important the measurement of pupils' progress, they did not attribute its value to the need of schools to establish fair indicators of their performance by using the technique of value added analysis. Fifth, more than half of the teachers (59%) supported the view that baseline assessment should not be based on products but on the process. However, a quarter of them accepted the idea that baseline assessment should be based on products and not on the process. Thus, it cannot be claimed that Cypriot teachers, as a group, rejected the idea that baseline assessment should be based on products of learning than on processes. Similarly, there is no consensus among Cypriot teachers about whether baseline assessment should refer to pupils' attitudes to mathematics and to pupils' ability to apply their knowledge in unfamiliar situations. Although 50% of them agreed with the idea that baseline assessment should include pupils' attitudes to mathematics, more than 20% of them (21%) rejected this idea. Similarly, 41% of them agreed with the idea that baseline assessment should include pupils' ability to apply their knowledge in unfamiliar situations but a quarter of them rejected this idea. Sixth, the great majority of Cypriot teachers (82%) considered record-keeping of the results of baseline assessment to be an essential part of baseline assessment. Moreover, almost all of them (93%) supported that each pupil's result of baseline assessment should be announced to his/her parents. Furthermore, nine out of ten of them (89%) supported that pupils' results should not be announced to the whole class. Finally, 68% of them agreed that each pupil's result should be also announced to the pupil. However, 21% of them did not accept this idea. It cannot, therefore, be claimed that Cypriot teachers as a group supported that a pupil's baseline assessment results should be discussed between the pupil and the teacher. Seventh, there was no consensus about whether each school should develop its own policy on assessment and teaching since only 36% agreed with item 11 whereas 25% of them rejected this idea. As a consequence, its median and mode are equal to 3.00. Thus, the figures derived from this item suggest that there was little overall consensus among Cypriot teachers about item 11, which was concerned with the development of a school policy on assessment. Similar findings seem to derive from teachers' responses to items 13a and 13b since their median and mode are also 3.00. It can also be argued that not as many teachers agreed that each school should develop its own policy on value added assessment (item 13b) as those who agreed that it is useful for the schools to develop a policy on baseline assessment (Item 13a).

L. Kyriakides and R. J. Campbell / Studies in Educational Evaluation 25 (1999) 109-130

! 17

Techniques of Assessment (Appropriateness and Ease) Teachers were asked to rank twice five techniques of baseline assessment according to their appropriateness and their ease. The mean ranks and the Kendall Coefficients of Concordance are presented in Table 2. (The testing of the significance of the Kendall Coefficients of Concordance was based on the same approximation [Fisher's Z-distribution] as above, since the number of entities is less than 7 [see note 1]). Moreover, columns 3 and 5 show the "absolute rank" of the mean ranks which is constructed by ordering the mean ranks. (The absolute ranks are used only for display purposes, and their representation does not necessarily imply an ordering of the perceived appropriateness and ease of these five techniques.) It emerges clearly from the coefficients presented in this table that Cypriot teachers agreed among themselves in their ranking of the relative appropriateness of each technique and also agreed among themselves in their ranking of the relative ease of each technique. The mean ranks presented in column 2 of Table 2 suggest that techniques of assessment can be classified into the following groups according to their perceived appropriateness. Performance test and structured observation were considered to be the most appropriate methods. The interview was the method considered to be the next most appropriate. On the other hand, unstructured observation was seen to be the least appropriate technique and oral question-and-answer was seen to be the one but least appropriate. The distribution of the mean ranks of ease of application of these techniques is also shown in this table. Unstructured observation was considered to be the easiest technique and oral question-and-answer as the next most easy. Although the mean ranks of the other three techniques are larger than 3.5, interview seems to be seen as the least easy technique. Table 2: Mean Ranks and 'Absolute' Mean Ranks o f Baseline A s s e s s m e n t Techniques According t o Cypriot Teachers' Perceptions o f Appropriateness and Ease

Assessment Techniques

Appropriateness Mean Rank Absolute M.R.

Mean Rank

Ease Absolute M.R.

Unstructured observation

4.74*

5

1.24'*

1

Oral question-and-answer

3.33

4

2.33

2

Structured observation

2.19

2

3.79

4

Interview

2.77

3

4.17

5

Performance test for each pupil

1.76

1

3.56

3

Coefficients:

W = 0 . 5 8 - Z =4.38 V1=3.94 V2= 798 p<.001

* 1 = Most Appropriate ... 5 = Least Appropriate

W = 0 . 3 8 - Z=2-18 V1=3.14 V2= 721 p<.05

** 1 = Most Easy ... 5 = Least Easy

118

L. Kyriakides and R. J. Campbell/Studies in Educational Evaluation 25 (1999) 109-130

The last, and probably the most important, finding has to do with the well known dilemma that what is easily measured is of dubious educational value. Interview and structured observation were considered to be one of the most appropriate but least easy techniques. Likewise, the unstructured observation and oral question-and-answer were regarded to be one of the most easy but least appropriate. Calculating the Spearman Correlation Coefficient, it was possible to examine whether there was a negative relation between perceived importance and perceived ease of each technique. The values of Spearman correlation coefficients, which are presented in Table 3, reveal that there is a correlation between the appropriateness and ease of each technique of assessment. The values of Spearman correlation coefficients for three out of five techniques were particularly high. However, the values of Spearman correlation coefficient between perceived appropriateness and ease of performance test were less than 0.30. This might be attributed to the fact that the performance test was seen as the most appropriate technique but not as so difficult as structured observation and interview. It can be claimed that, with one exception, there is a consensus among Cypriot teachers that the least appropriate techniques are the easiest to implement. Table 3: The Values of Spearman Correlation Between the Perceived Appropriateness and Ease of Each Technique According to the Cypriot Teachers' Perceptions

Assessment Techniques

Values of Spearman correlation r n p

Unstructured observation

0.61

287

.001

Oral question-and-answer

0.54

287

.001

Structured observation

0.51

287

.001

Interview

0.34

287

.001

Performance test for each pupil

0.28

287

.001

Perceptions About Ways of Improving Assessment Practice Figure 2 provides information about teachers' perceptions of methods of improving assessment practice. The mean ranks of the perceived importance of each method, having 5 as the highest point and 0 as the lowest, are displayed. Kendall's Coefficient of Concordance shows that Cypriot teachers agreed among themselves in their ranking of the relative importance of the six ways of improving assessment (W=.44, Z=2.37, V~=4.99, V2=888 and p<.005). It emerges from Figure 2 that the most important ways of improving assessment were further training in techniques of baseline assessment (Mean Rank = 3.9) and smaller class size (M.R.=3.7), whereas the least important was the existence of another

L. Kyriakides and R. J. Campbell/Studies in Educational Evaluation 25 (1999) 109-130

119

adult in the classroom (M.R.=0.6). The other way o f improving assessment, which is differentiated from all others, is the one concerning time free o f class contact which was seen as the second least important way (M.R.=I.8). It is finally important to indicate that Cypriot teachers considered as neither the least nor the most important ways o f improving assessment the publication o f non-statutory guidance on baseline assessment (M.R.=2.5) and the publication o f a performance test which they could use to assess pupils entering the primary school (M.R.=2.6).

s-

Mean

4.5-

Rank 43.5-

32.52t.510.50-

fl

6

C

D

E

f

Figure 2: Teachers' Perceptions of Ways of Improving Assessment A: Further training in baseline assessment, B: Time free of class contact, C: Smaller class size, D: Other adult in the class while assessment is occurring, E: Non-statutory guidelines on baseline assessment published by the Ministry of Education, F: A performance test published by the Ministry of Education

Discussion: Implications of Findings for the Development o f Policy on Baseline Assessment in Mathematics The evidence presented above can be discussed in terms of its implications for the development of a national policy on baseline assessment in mathematics. However, it also raises more general issues regarding the development of curriculum and assessment policy in mathematics. Cypriot teachers perceived formative purposes of baseline assessment as more important than the purpose related to the value added assessment or the summative purpose o f assessment. Moreover, the interview data revealed that they were ideologically averse o f summative purposes. It can therefore be claimed that Cypriot

120

L. Kyriakides and R. J. Campbell/Studies in Educational Evaluation 25 (1999) 109-130

teachers would welcome the development of an assessment policy which promoted the formative purposes of baseline assessment, but would be less inclined to support one emphasising summative purposes. Thus, the debate on developing a policy on baseline assessment may not be restricted to workload (Kyriakides, 1994) but raises fundamental issues of educational ideology. Most systems of baseline assessment have strengths and weaknesses, and few meet all possible requirements without being excessively unwieldy (Blatchford & Cline, 1992). Policy makers in Cyprus must be clear about the objectives for policy on baseline assessment. Teachers suggest that the answer to this question lies in the fact that information provided should be of genuine assistance in determining the appropriate action to be taken in assisting each pupil's development. Cypriot teachers supported the idea that information gathered from baseline assessment can be used for measuring pupils' educational progress in mathematics. However, they did not see the measurement of pupils' progress associated with the establishment of a model of measuring school effectiveness in teaching mathematics. This is despite the fact that they did not consider schools' final results in public examinations as fair indicators of each school performance. These findings can be attributed to the lack of systematic research into curriculum policy which affects the way in which curriculum evaluation is conducted in Cyprus (UNESCO, 1997). As a consequence the concept of school effectiveness is neglected in Cyprus and there is no great demand from schools to respond to any kind of accountability. Thus, teachers can not see the need for introducing a model for measuring school effectiveness. However, the TIMSS study revealed considerable variation in mathematics achievement between Cypriot pupils in year 3 and year 4 of different primary schools. It could be claimed that the TIMSS findings revealed the importance of developing a policy on evaluating the effectiveness of schools in mathematics. It can also be argued that Cypriot teachers are not ready to adopt the conceptual and practical dimensions of value-added assessment. This could be attributed to the fact that there is no arena for professional criticism in Cyprus due to lack of any systematic evaluation of the educational system (Kyriakides, 1996a). Thus, the educational system in Cyprus remains highly centralised and matches with what Schon (1971) has described as a "stable state". However, further research on Cypriot teachers' perceptions of value added assessment is needed since teachers considered the measurement of pupils' progress important. This might enable us not only to identify teachers' perceptions about the use of information gathered from measuring pupils' progress but also to examine whether their understanding of value added assessment differs from that of policy makers in countries attempting to introduce market forces into education (DFEE, 1995). The fact that significant differences among the skills and knowledge of school entrants have been identified (Kyriakides, 1997) supports both the importance of baseline assessment for formative purposes and that spending most of teaching time working as a whole class, as is the case in Cyprus (Kyriakides 1996b), is not an appropriate way of teaching mathematics to first year pupils. Before entering school, some pupils had achieved most of the aims of mathematics teaching for first year pupils while others had

L. Kyriakides and R. J. Campbell / Studies in Educational Evaluation 25 (1999) 109-130

121

not achieved any one of them. It is, therefore, not possible to organise teaching mathematics to first year pupils without taking into account the different mathematical backgrounds of school entrants. Cypriot teachers supported that information gathered from baseline assessment should help them to respond to the learning needs of each pupil. Thus, the development of a policy supporting the formative purposes of baseline assessment may also encourage teachers to give more thought to the best way in which to respond to individual learning needs. Cypriot teachers considered baseline assessment as a natural part of teaching. Moreover, the interview data illustrate implications of this conception of baseline assessment for teaching first year pupils and specific links between methods of teaching and assessment of mathematics. It can be claimed that Cypriot policy makers should attempt to explore links between purposes, teaching activities and assessment in mathematics in order to develop an assessment policy based on the consideration of baseline assessment as natural part of teaching. Thus, analysis of the evidence of teachers' perceptions about baseline assessment in mathematics implies that the debate about baseline assessment policy in mathematics should be focused on how it can be linked to the policy on teaching mathematics to first year pupils in order to provide information to teachers about teaching mathematics to school entrants. A coherent curriculum policy and policy on baseline assessment should be developed for teacher development, irrespective of other purposes such as monitoring of schools effectiveness by emphasising the policy on value added assessment. Current curriculum reform in several countries, and in Cyprus in particular, were designed amongst other things to raise standards especially in the core subject of mathematics. However, there is disagreement about the concept of standards. One way of examining standards in a country is by measuring pupils' knowledge and skills and comparing it with policy expectations. Research, especially in Cyprus, has no impact on policy formation (UNESCO, 1997) and this has affected the way in which the new curriculum of 1994 was designed. Teachers supported the idea that information gathered from baseline assessment may contribute to the evaluation of the current curriculum reform. Thus, the development of a policy on baseline assessment in mathematics might influence the process of curriculum design in Cyprus. It is important to note that policy on baseline assessment in England has been linked with the government's intention to improve standards for all, not least for those pupils with special educational needs (Lewis, 1995; Lindsay, 1997). But this intention can only be realised if based on a coherent and valid appraisal of levels of achievement and development, using well constructed approaches which mesh with school practice. Baseline assessment is therefore potentially a very useful addition to the education system. Two significant implications emerged from the data on Cypriot teachers' perceptions about the appropriateness and ease of the five techniques of baseline assessment. First, Cypriot teachers considered as more appropriate the techniques which operate under controlled conditions. This might reflect the highly centralised educational system of Cyprus and especially a perceived need to have tangible proof to show to

122

L. Kyriakides and R. J. Campbell/Studies in Educational Evaluation 25 (1999) 109-130

parents and inspectors. By the term tangible proof Cypriot teachers meant information gathered from assessment which can be easily understood by parents and inspectors, in which numbers can be used to represent pupils' attainment. However, the appropriateness of the techniques of assessment should be judged on the kind of information they make available to teachers. Thus, teachers should be encouraged to use techniques which can help them diagnose pupils' needs irrespective of whether they are under controlled conditions. Information derived from performance tests does not clearly reveal the mathematical concept which is involved with a pupil's wrong response to a test. Thus, if policy makers emphasise only the value of performance tests, teachers will not be enabled to improve assessment practice; instead the government is provided with another way to control curriculum practice. The other implication has to do with the fact that there was an inverse relationship between assessment techniques seen as most appropriate and those seen as most easy. Logically this argues for in-service training (INSET) on how teachers can use various techniques for their baseline assessment in mathematics, as much as it argues for using only performance tests. This would be welcomed by Cypriot teachers since training on techniques of baseline assessment in mathematics was considered as the most important way of improving assessment. It can be also claimed that in-service training should give high priority to structured observations and interviews which were seen as the most appropriate but least easy techniques. Thus, INSET focused on the use of interview and structured observation may be a more effective way of improving baseline assessment than the publication of more policy documents which are rarely consulted (Kyriakides, 1994). This provides significant implications for educational policy in Cyprus, which has not systematically used INSET to bring about change and has not been directed at the implementation of the current curriculum reform at the school level. The practice therefore goes against the evidence that innovations need both external and local support to succeed (Crandal, Eiseman, & Louis, 1986; Tumbull, 1985). There is a need to identify and build upon teachers' perceptions and encourage them to promote curriculum policy at the school level in order to assess their pupils and organise their teaching according to the needs of their school entrants.

Note 1.

The testing of the significance of the observed value of W can be done according to Kendall (1970, p.98) using an approximation which is based on the Fisher's Z-distribution. We write Z=(I/2)In[(m-1)W/(I-W)], Vl=n-l-(2/m) and V~=(m-1)V1 where n is the number of entities to be ranked and m is the number of judges assigning ranks. Then, for "degree of freedom" VI and V2, Z may be tested in the existing tables of Fisher's distribution. The significant value of W can be interpreted according to Siegel and Castellan (1988, p. 237), as meaning that "the observers or judges are applying essentially the same standard in ranking the N objects under study".

L. Kyriakides and R. J. Campbell/Studies in Educational Evaluation 25 (1999) 109-130

123

Acknowledgements Dr Ann Lewis was particularly helpful in discussing many of the issues raised in this article.

References Blatchford, P., & Cline, T. (1992). Baseline assessment for school entrants. Research Papers in Education, 7 (3), 247-269.

Brown, M. (1991). Problematic issues in national assessment. Cambridge Journal of Education, 21 (2), 215-229.

Calderhead, J. (Ed.). (1987). Exploring teachers' thinking. London: Cassell. Campbell, R.J., & Neiil, S. (1994). Primary teachers at work. London: Routledge. Chambers, D.L. (1993). Integrating assessment and instruction. In N.L. Webb & A.F. Coxford (Eds.), Assessment in the mathematics classroom (pp. 17-25). Reston, Virginia: NCTM. Clark, C.M., & Peterson, P.L. (1986). Teachers' thought processes. In M.C. Wittrock (Ed.), Handbook of research on teaching (pp. 255-296). New York: Macmillan.

Cohen, D., & Manion, L. (1994). (4 th edition). Research methods in education. London: Routledge. Crandal, D.P., Eiseman, J.W., & Louis, K.S. (1986). Strategic planning issues that bear on the success of school improvement efforts. Educational Administration Quarterly, 22 (3), 21-53. Day, C., Caiderhead, J., & Denicolo, P. (1993). Research on teacher thinking: Understanding London: Falmer.

professional development.

Department of Education and Science (1987). National curriculum: Task group on assessment and testing - A report. London: HMSO.

Department of Education and Science (1991). Assessment, recording and reporting. London: HMSO. Department of Education (1995). Value added in education: A briefing paper from the London: Department of Education.

Department of Education.

Desforges, C. (1989). Testing and assessment. London: Cassell. ENCA (1992). Evaluation of national curriculum assessment at key stage 1. University of Leeds: School of Education. Fitz-Gibbon, C.T. (1995). The value added national project: General report. London: School Curriculum and Assessment Authority.

124

L. Kyriakides and R. J. Campbell/Studies in Educational Evaluation 25 (1999) 109-130 Fullan, M.G. (1991). The new meaning of educational change. London: Casseli.

Havelock (1971). Planning by innovation through dissemination and utilisation of knowledge. University of Michingan : Centre for Research and Utilisation of Scientific Knowledge - Institute of Social Research. Kendall, M.G. (1970). (4th edition). Rank correlation methods. London: Charles Griffin. Kimberley, K., Hextall, I., Torrance, H., & Moon, R. (1989). National assessment and testing: The TGAT report. British 3ournal of Sociology of Education, 10 (2), 233-251. Kyriakides, L. (1994). Primary teachers' perceptions of policy for curriculum reform in Cyprus with special reference to mathematics. Unpublished doctoral dissertation, University of Warwick, Coventry. Kyriakides, L. (1996a). "Reforming" primary education in Cyprus. Education 3-13, 24 (2), 4650. Kyriakides, L. (1996b). The implementation of curriculum policy on classroom organisation in primary mathematics. In L. Puig & A. Gutierrez (Eds.), Proceedings of the 20th International Conference for the Psychology of Mathematics Education 3, 217-232. Valencia, Spain. Kyriakides, L. (1997). The mathematical knowledge and skills of Cypriot pupils entering primary school: Implications for the development of policy on baseline assessment. In E. Pehkonen (Ed.), Proceedings of the 21st International Conference for the Psychology of Mathematics Education. 3, 176 -183. University of Helsinki, Finland. Lewis, A. (1995). Primary special needs and the national curriculum. London: Routledge. Lindsay, G. (1997). Baseline assessment: A positive or malign initiative? Coventry: University of Warwick. Lindsay, G., & Pearson, L. (1981). The infant rating scale. London: Hodder & Stoughton. Ministry of Education (1994). The new curriculum. Nicosia: Ministry of Education. Murphy, P. (1988). TGAT: A conflict of purpose. Curriculum, 9 (3), 152-158. National Council of Teachers of Mathematics (1989). Curriculum and evaluation standards for school mathematics. Reston, Virginia: NCTM. Nisbet, J. (1973). The school council. Case studies of educational innovations: 1 At the central level Paris: CERI/OECD. Pajares, M.F. (1992). Teachers' beliefs and educational research: Cleaning up a messy construct. Review of Educational Research, 62, 307-332. Pole, C.J. (1993). Assessing and recording achievement: Implementation of a new approach in schooL Buckingham: Open University Press.

L. Kyriakides and R. J. Campbell / Studies in Educational Evaluation 25 (1999) 109-130

125

Richardson, V. (1990). Significant and worthwhile change in teaching practice. Educational Researcher 19 (7), 10-18.

Schmidt, W.H., McKnight, C.C., Valaverde, G.A., & Wiley, D.E. (1997). Many visions, many aims volume 1: A cross national investigation o f curricular instructions in school mathematics. Dordecht: Kluwer. Schon, D.A. (1971). Beyond the stable state. Harmondsworth: Penguin. Shepard, L.A. (1989). Why we need better assessment. Educational Leadership, 46 (2), 4-8. Siegel, S., & Castellan, N.J. (1988). (2nd ed.). Non-parametric statistics: For the behavioural sciences. New York: McGraw-Hill.

Stenmark, J.K. (1992). Mathematics assessment." Myths, models, good questions and practical suggestions. Reston, Virginia: NCTM. Strand, S. (1997). Pupil progress during Key Stage 1: A value added analysis of school effects. British Educational Research Journal, 23 (4), 471-487.

Turnbull, B.J. (1985). Using governance and support systems to advance school improvement. The Elementary School Journal, 85 (3), 337-351

UNESCO (1997). Appraisal study on the Cyprus education system. Paris: IIEP. Webb, N.L., & Coxford, A.F. (Eds.). (1993). Assessment in the mathematics classroom. Reston, Virginia: NCTM. Wedell, K., & Raybould, E.C. (1976). The early identification of educationally "at risk" children. Educational Review, Occasional Publications No 6. Birmingham, University of Birmingham. Wolfendale, S., & Bryans, T. (1979). Identification o f learning difficulties: A model for implementation. London: National Association for Research in Education. Zeichner, M.K., Tabaehnick, B.R., & Densmore, K. (1987). Individual, institutional, and cultural influences on the development of teachers' craft knowledge. In J. Calderhead (Ed.), Exploring teachers' thinldng: London: Cassell. The Authors LEONIDAS KYRIAKIDES is a Research Officer at the Institute of Education, University o f Warwick, UK. He has published research papers in international journals. His research interests are assessment in education, baseline and value-added assessment, curriculum development, school effectiveness, and teacher professionalism.

R. J. CAMPBELL is Director o f the Institute o f Education, University o f Warwick, UK. He has published widely in the field of primary education and curriculum policy.

126

L. Kyriakides and R. J. Campbell/Studies in Educational Evaluation 25 (1999) 109-130

Appendix A: The Questionnaire I am conducting research into teachers' perceptions of baseline assessment in Mathematics. I would be grateful if you could take twenty minutes or so to complete this questionnaire, anonymously.

PART A: In this part put a tick in the appropriate box 1. S E X :

Male[---- 1

Female [ - ~

2. Length of teaching experience (including this year as 1 full year): 3. Which of the following initial teacher training qualifications do you hold? (Tick as many as apply) Teacher's Certificate ~

B.Ed./B.A.(Q.T.S.)

B.A.

PGCE ~ - 1

B.Sc. ~ - " 1

Other[-"--

4. Did you take Mathematicsas a main or subsidiarysubject during your initial training? Yes~_ 1 No [---~

5. Which of the following post-experiencequalifications do you hold? (Tick as many as apply)

M.Ed.[-'---]

AdvancedDiploma[---- 1

Other?---]

Did any of your post-experiencequalificationhave to do with Mathematics or Education? Yes No ~ - ~

7. Do you hold any formal post of responsibility for Mathematics in your school? Yesr- ~

Nor--- 1

L. Kyriakides and R. J. Campbell/Studies in Educational Evaluation 25 (1999) 109-130

127

8. Size of your class:

Under 20 ~ - - - ] 30-32

21-23["""33-35

24-26 ~

27-29

More than 36 ~ ' ]

9. Which of the following year groups are in your class? (Tick as many boxes as apply) Reception ~ - ]

Year 1

Year4[-----]

Year 5 [-------]

Year 2 [-----]

Year 3

Year 6 [------]

10. How confident would you say you are about assessing children's attainment in Mathematics in primary school? Not at all confident ~ - ] Quite confident

~'-1

Confident in a limited way [-----"] Very confident F - - 1

Don't know/can't say F-----

PART B The following question indicates four purposes of baseline assessment. Please rank the importance you attach to the four purposes using the numbers 1 to 4. Give 1 to the purpose you regard as the most important, 2 to the next most important and so on, with 4 meaning the least important. 11.

The main purpose of baseline assessment in Mathematics should be to: a) provide information to enable teachers or parents to make comparisons across pupils entering primary school b) help identify early pupils with learning difficulties c) help identify and diagnose the learning needs of pupils entering primary school d) provide information to be used for measuring pupils' subsequent educational progress

r-]

128

L. Kyriakides and R. J. Campbell/Studies in Educational Evaluation 25 (1999) 109-130

PART C Please rate each of the following items (Nos. 12-27) by circling the appropriate number. The numbers represent the following values: 1 = Absolutely disagree; 2 = Disagree; 3 = Don't know~Can't say," 4 = Agree; 5 = Absolutely agree.

12. Baseline assessment is an essential part of planning how to teach Mathematics to first year pupils.

1

2

3

4

5

13. Baseline assessment should form a natural part of teaching activities.

1

2

3

4

5

14. The methods of teaching as well as the methods of assessment should be subject to whole school decision-making.

1

2

3

4

5

15. Information gathered from baseline assessment should contribute to the evaluation of curriculum reform in Mathematics.

1

2

3

4

5

16. Information gathered from baseline assessment should be used for measuring pupils' educational progress.

1

2

3

4

5

17. The effectiveness of a school in teaching Mathematics should be evaluated by comparing the educational progress made by the pupils of the school to that made by similar pupils in other schools (i.e. through value added assessment).

1

2

3

4

5

18. Information gathered from baseline assessment should be used to identify individual children with special educational needs.

1

2

3

4

5

19. Baseline assessment provides teachers with information which should help them to organise their classroom.

1

2

3

4

5

20. Teachers should use information gathered from baseline assessment in order to identify low and high attainers.

1

2

3

4

5

21. Final school results in national examinations are a fair indicator of school performance.

1

2

3

4

5

22. Teachers should assess on the basis of pupils' learning products rather than the learning process.

1

2

3

4

5

23. Baseline assessment should include the assessment of pupils' attitudes to Mathematics.

1

2

3

4

5

24. As soon as pupils enter the primary school their ability to apply Mathematics in unfamiliar situations should be assessed.

1

2

3

4

5

25. Record keeping of the results of baseline assessment is essential.

1

2

3

4

5

L. Kyriakides and R. J. Campbell/Studies in Educational Evaluation 25 (1999) 109-130

26. A pupil's result of baseline assessment should be: a) announced to the whole class

1

2

3

4

5

b) discussed between parents and teacher

1

2

3

4

5

c) discussed between the pupil and the teacher.

1

2

3

4

5

1

2

3

4

5

1

2

3

4

5

27. Each school should develop its own policy in such a way to include suggestions about: a) baseline assessment b) value added analysis of school's result in Mathematics

129

28. Below are 5 techniques of baseline assessment. Rank these techniques twice (column A and column B). In column A give 1 to the most appropriate, 2 to the next most appropriate and so on with 5 meaning the least appropriate. In column B give 1 to the technique which you consider the easiest to use, 2 to the next easiest and so on with 5 meaning the least easy. Column A Appropriateness a) Unstructured observation of pupils' work

I

Column B Ease

7-]

b) Oral question-and-answer c) Structured observation of children's work

[-~

d) Interviewing individual children

I-----

F-]

e) Performance test for each pupil

29. Below are 6 ways of improving baseline assessment in Mathematics. Please rank these to reflect your opinion. Give 1 to the most important, 2 to the next most important and so on with 6 meaning the least important. a) Further training in baseline assessment b) Time free of class contact c) Smaller class size d) Other adult in the class while assessment is occurring e) Non-statutory guidelines on baseline assessment published by the Ministry of Education f) A performance test published by the Ministry of Education

F-] 7-1 F-q F-7

130 30.

L. Kyriakides and R. J. Campbell / Studies in Educational Evaluation 25 (1999) 109-130

Please feel free to write below any other comment about teaching and assessment of Mathematics.

Thank you very much for your help. Leonidas Kyriakides