The link between educational expenditures and student learning outcomes: Evidence from Cyprus

International Journal of Educational Development 70 (2019) 102081

Contents lists available at ScienceDirect

International Journal of Educational Development journal homepage: www.elsevier.com/locate/ijedudev

The link between educational expenditures and student learning outcomes: Evidence from Cyprus Leonidas Kyriakidesa,

⁎,1

a b

T

, Andreas Stylianoub,2, Maria Eliophotou Menona,1

Department of Education, University of Cyprus, Cyprus Ministry of Education and Culture, Cyprus

ARTICLE INFO

ABSTRACT

Keywords: Educational policies Educational finance Student outcomes Educational effectiveness

The paper investigates the relationship between educational expenditures and student learning outcomes in the Republic of Cyprus. Using Multilevel Regression Analysis and Discriminant Function Analysis, we investigate the extent to which changes in the effectiveness status of schools can be related to changes in educational investment. Based on the findings, educational investment had a positive effect on the effectiveness status of a school if invested in least effective schools and not in other types of schools (typical and most effective). Investment in specific types of equipment was found to have a significant effect on student learning outcomes.

1. Introduction In recent decades, low levels of learning have been reported in both developed and developing countries, leading to what has been described as a “learning crisis” (Winthrop et al., 2015). As a result, the investigation of factors associated with improvements in learning outcomes has gained increasing attention. The effect of educational expenditures on learning outcomes has been examined in a large number of studies in an attempt to inform educational policy, and improve the effectiveness and efficiency of educational investment. However, in many countries, policy decisions regarding the appropriate level of educational expenditures are seldom informed by relevant research. This can result in a situation where policy decisions in this area are the result of political factors with “politics … substituted for analysis” (Psacharopoulos, 1996, p. 343). In countries with scarce state funds, it is especially important that resource allocation is linked to an improvement in educational outcomes (Abayasekara and Arunatilake, 2018). The scarcity of funds is not restricted to developing countries: The financial crisis has led many developed and middle-income countries to re-examine their policies regarding investments in education through an evaluation of the impact of investments on educational outcomes. The Republic of Cyprus is a Southern European country, which has been severely affected by the financial crisis. Despite the relatively high investment in education in Cyprus

(Eurostat, 2018), there is almost no research investigating the relationship between educational investment and student learning outcomes. Moreover, the impact of school expenditures for students of different socio-economic backgrounds has not been examined. Consequently, there is insufficient data on the effectiveness of variations in expenditure levels based on different school and student characteristics. Thus, educational policy officials make relevant investment decisions without evidence on what works, even under the recent financial crisis and the associated fiscal constraints. The present study attempts to inform the literature by investigating the link between specific educational inputs and student learning outcomes, as reflected in students’ performance in university admissions examinations. Specifically, the paper investigates the link between educational expenditures and student learning outcomes in Cyprus by examining the extent to which changes in the effectiveness status of schools can be related to changes in educational investment. Using the results of admissions examinations for Cypriot and Greek Universities for a five-year period (2008–2012), we determine whether there is any relationship between educational expenditures and student learning outcomes. The analysis includes all variables in order to investigate whether certain types of variables are, as expected, more important as influences on learning outcomes than others. For instance, expenditures on technology are generally considered to have a greater (short- and medium-term) impact on learning outcomes than expenditure on new buildings (Scheerens, 2013). Beyond its significance for educational

Corresponding author. E-mail addresses: [email protected] (L. Kyriakides), [email protected] (A. Stylianou), [email protected] (M. Eliophotou Menon). 1 Department of Education, University of Cyprus, P. O. Box 20537, 1678 Nicosia, Cyprus. 2 Internal Audit Unit, Ministry of Education and Culture, Kimonos and Thoukydidou corner Akropoli, 1434 Nicosia, Cyprus. ⁎

https://doi.org/10.1016/j.ijedudev.2019.102081 Received 20 September 2018; Received in revised form 11 June 2019; Accepted 30 June 2019 Available online 04 July 2019 0738-0593/ © 2019 Elsevier Ltd. All rights reserved.

International Journal of Educational Development 70 (2019) 102081

L. Kyriakides, et al.

policy, the research reported in this paper can inform theoretical and methodological perspectives associated with educational effectiveness theory and human capital theory.

result, based on predefined aims. Achieving the goal of quality education through effective schools is a major priority of modern societies which has led to increased interest in research on educational outcomes (Creemers et al., 2010; Scheerens, 2016). Research on the factors associated with educational effectiveness is commonly associated with a variety of theoretical perspectives. On the one hand, educational effectiveness theory focuses on the development of models and theories to explain variations in educational effectiveness at different levels (Scheerens, 2013). On the other hand, research in the economics of education has attempted to examine the link between educational inputs and outputs in the framework of human capital theory. Under human capital theory, production functions have been used to investigate the relationship between educational inputs and outputs (e.g., Cheng, 1993; Hanushek, 1986, 1989; Hedges et al., 1994). Studies of educational effectiveness have sought to measure the magnitude of the effects of schools on specific outcomes, with reference to school level factors and subsequently to class level variables, focusing on the effects of such factors on student performance (Creemers and Kyriakides, 2015; Scheerens, 2016). At a later stage, advanced quantitative methods, such as multilevel modeling techniques, were used in relevant investigations. These studies reveal that factors associated with student achievement operate at different levels such as the student, the class, the school, and the system level. Moreover, factors operating at different levels were found to be related with each other. Theoretical models taking into account the dynamic character of educational effectiveness have been developed and used for promoting quality and equity in education (Creemers and Kyriakides, 2010; Scheerens, 2013). According to Hanushek (2008), evidence from the economic analysis of education points to the inefficiency of the current provision of schooling in that the most common inputs to schools, which include class size and certain teacher characteristics, exhibit very little, if any, systematic relationship to student outcomes. After a review of the literature on the relationship between student achievement and educational resources, Hanushek (1986, 1989, 1991, 1997) reported that in the majority of studies, per pupil expenditures had an insignificant or negative effect on student performance. However, Hanushek’s conclusions have been challenged by several meta-analyses which showed that an increase in resources was significantly linked to an increase in test scores (Card and Krueger, 1996). Hedges et al. (1994) conducted a meta-analysis of studies of the effects of differential school inputs on student outcomes, which included previous studies by Hanushek. Using different analytical methods, they reported systematic positive links between resource inputs and school outcomes. The authors drew attention to limitations in the data set used by Hanushek and others but it should be acknowledged that the effect size reported by Hedges et al. is very small. Additional studies also point to a small but significant association between educational inputs such as per-pupil expenditure and educational attainment (Matrix Evidence, 2009; Ram, 2004). The effect of school resources on student academic performance has been reported to be greater in low-income countries (Heyneman and Loxley, 1983). More recent research on the link between school resources and educational outcomes points to a positive association between educational expenditure and PISA (Program for International Student Assessment) performance (Anderson et al., 2007; Brunello and Rocco, 2013). Several studies attempting to measure the effect of school expenditures on educational achievement do not utilize school-level data (Ajwad, 2001; Cobb-Clark and Jha, 2013). This is an important methodological limitation of previous research which has relied on districtlevel data on spending (Condron and Roscigno, 2003). At the school level, factors such as instructional per-student expenditures, administration-related expenditures, teacher quality and commitment, and class size have been linked to positive outcomes (Abayasekara and Arunatilake, 2018; Elliott, 1998; McEwan, 2015; Verstegen and King,

2. The context In Cyprus, public educational provision takes place at the following main levels or school types: Pre-school and pre-primary education, primary education, secondary education, special education, and postsecondary tertiary education (university and non-university institutions). There are two types of public secondary schools: secondary general schools and secondary technical/vocational schools. Based on the available statistical data, more than 85% of secondary school students graduate from secondary general schools (Statistical Service of the Republic of Cyprus, 2018). The educational system of Cyprus is highly centralized, with the Ministry of Education of Culture undertaking the responsibility for the formulation of educational policy and the administration of education at all levels. Consequently, the Ministry of Education and Culture is responsible for the implementation of educational legislation, which covers all aspects of educational provision including the curriculum. In accordance with the centralized nature of the system, headmasters and teachers are appointed, transferred and promoted by the Educational Service Commission, an independent five-member body, which is appointed by the President of the Republic of Cyprus. Public schools are mainly financed from public funds. School boards undertake the responsibility for the construction, maintenance and equipment of public school buildings. These committees are public entities acting under the supervision of the technical services of the Ministry of Education and Culture. Each school board manages a predetermined number of pre-primary, primary and secondary schools. In essence, school boards are intermediaries between the Ministry of Education and Culture, and public schools. The school board prepares the annual budget for each school and is responsible for its execution. Thus, all funds raised by public schools are transferred to the school board and are not available to the school unit. School boards have been criticized for the fact that they are often slow to respond to minor requests from schools which arise on a daily basis (World Bank, 2014). Since 2006, school boards are directly elected by the citizens of the Republic of Cyprus. In 2015, the total current public cost per pupil/ student in public institutions by level of education was: Pre-school and pre-primary €4,958, primary €6,189, secondary €9800 and tertiary €10,321. In the same year, public expenditure on all levels of education accounted for 16.1% of the Government Budget (Statistical Service of the Republic of Cyprus, 2018). Based on recent data for Cyprus, the total expenditure on education as a percentage of the Gross National Product (GNP) is relatively high, placing Cyprus at the highest positions in comparison with other European Union (EU) countries. By way of illustration, according to the European Statistical Office (Eurostat, 2018), in the year 2012, the total expenditure on education as a percentage of the GDP in Cyprus amounted to 6.7% (the third highest in Europe after Denmark and Sweden), whereas for the EU countries the average rate was 5.3%. However, there is no evidence that education in Cyprus is linked to better learning outcomes than in other countries which spend lower proportions of their GDP on education. In fact, the available evidence indicates that the performance of Cypriot students is considerably lower than the EU average (OECD, 2013). Therefore, an important question arises as to whether the money invested in education has the desired results by improving educational outcomes and especially student performance. 3. The background Educational provision must deliver specific outcomes in order to ensure effectiveness. Effectiveness refers to the delivery of an expected 2

International Journal of Educational Development 70 (2019) 102081

L. Kyriakides, et al.

1998; Wenglinsky, 1997). In some cases, the evidence on the effects of these factors is mixed and contradictory. For instance, some studies have found class size reductions to have a positive effect on student achievement (Krueger, 1999; McEwan, 2015) while others fail to identify any effects (Chingos, 2012; Dee and West, 2011; Hoxby, 2000). In an analysis of evidence from the Project STAR (a four-year largescale randomized experiment in the United States), Konstantopoulos (2008) found that despite positive effects for all students, reductions in class size did not reduce the achievement gap between low and high achievers. Condron and Roscigno (2003) used school-level data from a large, North Central, urban district in the United States in order to investigate the effect of spending disparities on student achievement. They found that higher spending was positively associated with achievement through specific school resources, which included instructional spending from local sources, and spending on operations and maintenance. Better conditions of school buildings and a higher degree of order and consistency promoted student achievement in all subject areas included in the study. Given that the conditions of school buildings are likely to be worse in low-SES districts, the findings of the study point to the importance of increased investment in disadvantaged areas. In a study conducted in the United Kingdom, Nicoletti and Rabe (2012) found that school-level expenditures on educational support staff and learning resources contributed to the improvement of the achievement of disadvantaged students. Expenditures in classroom teachers were linked to higher test scores, while the opposite was the case for expenditures in substitute teachers. In an Australian study, Cobb-Clark and Jha (2013) investigated the relationship between budget allocation at the school level and student achievement. They reported a modest relationship between additional per-pupil expenditure and improvements in students’ standardized test scores. Their results point to a differential impact of per-pupil spending on achievement in that additional expenditure on ancillary teaching staff resulted in significantly greater gains in promoting achievement growth in numeracy and reading for younger students (primary and middle school years). Moreover, expenditure on leadership and management personnel resulted in faster growth in grade 5 writing skills and grade 7 reading levels. It is important to note that the differential impact of expenditures on students of different socioeconomic backgrounds has not been adequately explored in the literature. Ajwad (2001) investigated the role of educational expenditures in raising student achievement for high and low socioeconomic status students. In this study, the expenditure effect was found to be more pronounced when additional funds were invested in schools attended by low socioeconomic status students as opposed to schools attended by their higher status counterparts. Baird (2012) examined the relationship between school resources and mathematics performance of low socioeconomic status students in 19 high-income countries. Using data from the Third International Mathematics and Science Study (TIMSS) and the Oaxaca decomposition technique, she estimated the extent to which achievement gaps could be attributed to differences in classroom and school level resources provided to students from different socioeconomic backgrounds. According to the findings, in some countries, differences in measured school characteristics played a significant role in explaining achievement gaps between low and high SES students. However, in many countries this was not the case as differences in the characteristics of students appeared to be more important. Based on these findings, Baird concluded that the effect of expenditures on the performance of low SES students may vary from country to country. In a recent study, Abayasekara and Arunatilake (2018) conducted a study of school-level resource allocation and education outcomes in Sri Lanka. They found that a school’s ranking based on status, type, size and principal quality had a significant effect on student academic performance, pointing to the differential impact of several variables. In this context, the study reported in this paper aims to identify the

impact of different types of expenditures on student achievement. By doing this we aim to add to the body of evidence on the effect of school expenditures on student achievement. In addition, the study makes a significant contribution to the literature in that most studies investigating the expenditure effect are cross-sectional and consequently only a correlation between input and output variables is examined. This is not the case in the present study which uses longitudinal data to investigate the extent to which changes in student learning outcomes can be attributed to changes in expenditures at the school level. The longitudinal investigation of the effects of specific types of expenditures on student learning outcomes can provide much needed evidence regarding the link between educational expenditures and educational outcomes, resulting in more informed policy decisions. 4. Methods Based on the aims of the study, we conducted a secondary analysis of data gathered by the Ministry of Education in Cyprus on the different types of expenditures of all public secondary schools and the learning outcomes of their students for five school years. Separate multilevel regression analyses of student achievement in the university entrance examinations during five consecutive school years were conducted and changes in the effectiveness status of secondary schools were identified. Subsequently, Discriminant Function Analysis (DFA) was used to investigate the extent to which changes in school expenditures could predict changes in the effectiveness status of schools. DFA is a statistical technique used for classifying observations (Klecka, 1980) and involves the predicting of a categorical dependent variable by one or more continuous or binary independent variables. It is statistically the opposite of Multiple Analysis of Variance (MANOVA), and it is very useful in determining whether a set of variables is effective in predicting category membership. One of the benefits of DFA is that it produces a classification table showing where the data are categorized and the groups in which they are predicted to be. The table also shows the percentage of cases which were correctly classified through the prediction of group membership. Since DFA will classify cases into the largest group, a statistic, tau, can be computed showing the proportional reduction of error (PRE) when using the predicted model. The measurement of learning outcomes was based on the overall grade achieved by each student in the Pancyprian Examinations. The Pancyprian examinations are conducted at the end of each school year and are taken by all public secondary school graduating students. These tests are considered as high-stake tests since the purpose of this national examination system is two-fold: They serve as the basis for the award of the School Leaving Certificate as well as for the selection of students to be admitted to public higher education institutions in Cyprus and Greece. The input variables used in the analysis are presented below based on the school and student level.

• At the student level: gender was the only student background factor provided by students participating in the Pancyprian examinations • At the school level: • The percentage of girls • The number of low Socioeconomic Status (SES) students (based on parental income mentioned in the school records) • Educational expenditures, which were grouped into the following five categories: • Building additions/improvements/extensions: This category in• 3

cludes expenditures on the construction of new schools, new classrooms, sports rooms, I.T. rooms as well as expenditures on demolitions. Equipment for laboratories and special classrooms: This category includes expenditures on the preparation of laboratories and special classrooms, with reference to the purchase of instruments, sports equipment, and equipment for Biology and Physics laboratories.

International Journal of Educational Development 70 (2019) 102081

L. Kyriakides, et al.

• Heating/air conditioning: Expenditures in this category include heating and air-conditioning equipment. • I.T./computers: These expenditures include money spent on computers, printers, video projectors and additional I.T. equipment. • Grants provided by School Boards: This category includes grants

student level. For example, during the school year 2007–2008 the variance explained at the student level was 5.92 whereas the total variance explained was 7.19. Second, during each school year, gender was found to be related with student achievement since girls were found to perform better than boys in the university entrance examinations. Third, a relevant contextual effect was only reported during the first two school years (i.e., 2008 and 2009), revealing that schools with a higher percentage of girls had better results in the Pancyprian Examinations. Schools with smaller percentages of students coming from low SES backgrounds were not found to have better overall results in the university entrance examinations. At the next step of the analysis, variables concerned with different types of expenditures were added to Model 1. The likelihood statistic (X2) reveals a significant change (p < .001) between Model 1 and Model 2. This suggests that variables measuring different types of expenditures were found to have significant effects on student achievement during each school year. One can also see that these variables explain approximately 4% of student achievement at the university entrance examinations. Moreover, all five analyses of the expenditure effect on student achievement in different school years reveal that the effect sizes of expenditures are small. Specifically, in order to estimate the relative importance of the type of expenditures on student achievement, the fixed effect obtained with each multilevel analysis was converted to standardized effects or “Cohen’s d” by following the approach proposed by Elliot and Sammons (2004). This approach investigates the change in the outcome measure that will be produced by +/- one standard deviation in the continuous predictor variable (i.e., a specific type of expenditure), standardized by the within school standard deviation adjusted for covariates in the model. In this way, the relative strength of the effect of each type of expenditure was computed and found to be smaller than 0.10 (see Table 2). Finally, there is no type of expenditure which was found to be associated with student achievement in the majority of the school years and thereby time stability in the effect of any type of school expenditure was not identified. For this reason, we searched for time stability in the effect of the level of the overall educational expenditures on student achievement by adding this interval variable to model 1. However, overall educational expenditure was only found to have an effect on student achievement in 2009 (see model 2b, Table 1a) and in 2012 (see Table 1c) and the effect size of the level of educational expenditure was smaller than the effect of types of educational expenditure. In each analysis, the model which took into account the effect of different types of educational expenditure (i.e., model 2a) was in a position to explain more variance than the model 2b which investigates the impact of the overall level of educational expenditure. In the final part of this section, we present the results of our attempt to identify whether changes in school expenditures predict changes in the effectiveness status of schools. In order to achieve this aim, the following procedure was undertaken. Based on the results of the multilevel analyses of student achievement in each school year, an estimate of the effectiveness status of each school during each of these five consecutive school years was undertaken. Specifically, background variables (at student and school level) were taken into account in estimating the schools’ “value-added’’ contributions. These are typically referred to as the effectiveness scores of schools, but they also reflect other unmeasured factors (outside the control of the school) which were not controlled for in the analysis (Thomas, 2001). Based on the results of Model 1 (see Tables 1a-c), the difference between the expected and the actual score for each school was plotted. The standard error of estimate for each school was also taken into account and is represented by the length of a vertical line. This line can be conceptualized as the range within which we are 95% confident that the “true’’ estimate of the school’s residual lies (Goldstein, 2003). Thus, on the one hand, where this vertical line does not cross the horizontal zero line and is also situated below the zero line, the school it represents is considered as one of the least effective schools of our sample (Creemers

provided by school boards and spent on school supplies, cleaning products and other small purchases for the maintenance of school equipment.

It is important to note that other types of expenditures were not selected because of the nature of the Cyprus educational system, which is highly centralized. For instance, salaries are based on predetermined salary scales and do not vary across schools. As a result, it was not possible to investigate the effect of changes in salaries on learning outcomes. As previously mentioned, the research was based on the analysis of secondary data. The main sources of data were the Examinations Office of the Ministry of Education and Culture, the Directorate of Secondary General Education, the Accounting Office of the Ministry of Education and Culture, and the Statistical Service of Cyprus. From the Examinations Office of the Ministry of Education and Culture, we obtained data on student performance in the Pancyprian Examinations. All secondary school units in Cyprus were included in our study. Specifically, the total number of school units was 42 (from 2008 to 2010) and 44 (from 2011 to 2012, after the addition of two new secondary schools). 5. Findings For each school year, separate multilevel regression analyses of student overall achievement in the Pancyprian Examinations were initially conducted. Specifically, to investigate the expenditure effect, MLwiN (Goldstein et al., 1998) was used because the observations are interdependent and because of multi-stage sampling since students are nested within schools. The dependency has an important consequence. If students’ achievement within a school has a small range, institutional factors at school level may have contributed to it (Snijders and Bosker, 1999). Thus, the first step in each analysis was to determine the variance at the individual and school level, without explanatory variables (empty model). In subsequent steps, explanatory variables at different levels were added. Explanatory variables, except for grouping variables, were entered as Z-scores with a mean of 0 and a standard deviation of 1. This is a way of centering around the grand mean (Bryk and Raudenbush, 1992) and yields effects that are comparable. Thus, each effect expresses how much the dependent variable increases (or decreases in case of a negative sign) by each additional deviation on the independent variable (Creemers et al., 2010). Grouping variables were entered as dummies with one of the groups as baseline (e.g. boys = 0). The models presented in Tables 1a, b and c were estimated without the variables that did not have a statistically significant effect at the .05 level. A comparison of the empty models that emerged from analyzing the data during the five school years reveals that the effect of the school was greater than 15%. Moreover, in each analysis the variance at each level reaches statistical significance (p < .05) and this suggests that multilevel analysis can be used to identify the explanatory variables that are associated with achievement in the university entrance examinations (Goldstein, 2003). In Model 1, the context variables at student and school levels were added to the empty model. The following observations arise from the figures of the five columns illustrating the results of Model 1 for each analysis. For each school year, the background variables added in Model 1 were found to explain at least 10% of the total variance of student achievement. For example, during the school year 2007–2008 the explained variance was 7.19 and the total variance was 42.3. This implies that model 1 was in a position to explain 17% of the total variance. One can also see that most of the explained variance is at the 4

International Journal of Educational Development 70 (2019) 102081

L. Kyriakides, et al.

Table 1a Parameter estimates and (standard errors) for the analyses of students’ achievement in the Pancyprian Examinations per school year. Factors

Model 0

Fixed part/intercept Student Level Context Sex (boys = 0, girls = 1) School Level Context Percentage of girls Percentage of students from low SES Types of Expenditures Building additions/improvements Equipment for labs/special classrooms Heating/air-conditioning I.T./computers Grants provided by School Boards (consumables) Total expenditures Variance School Student Explained Significance test X2 Reduction of X2 Degrees of freedom p-value

7.10 (.30)

Year 2007–2008 Model 1

Model 2

Model 0

Year 2008–2009 Model 1 Model 2a

Model 2b

4.06 (.25)

3.12 (.20)

8.00 (.30)

7.00 (.20)

5.21 (.20)

5.68 (.20)

0.20 (.05)*

0.20 (.04)*

0.12 (.04)*

0.13 (.04)*

0.13 (.04)*

0.11 (.03)* –0.06 (.04)

0.12 (.04)*

0.08 (.03)* –0.07 (.05)

0.08 (.03)*

0.08 (.03)*

0.08 (.06) 0.10 (.04)* 0.05 (.04) .11 (.03) N.S.S.

0.10 0.07 0.06 0.07 0.06

(.05)* (.05) (.04) (.05) (.05)

0.09 (.04)*

6.35 (1.32) 35.96 (3.21)

5.08 (1.29) 30.03 (3.20) 7.19

3.81 (1.12) 29.19 (3.12) 9.31

8.71 (2.21) 39.69 (4.12)

7.74 (2.12) 33.88 (4.09) 6.78

6.29 (2.10) 33.40 (4.03) 8.71

7.26 (2.11) 33.40 (4.02) 7.74

712.35

601.21 111.14 2 .001

550.01 51.20 2 .001

810.81

700.01 110.80 2 .001

660.01 40.00** 1 .001

691.20 8.81** 1 .001

* Statistically significant effect at .05. ** For each alternative model 2 (i.e., model 2a and 2b), the reduction is estimated in relation to the deviance of Model 1.

et al., 2010). On the other hand, where this line does not cross the horizontal zero line and is situated above the zero line, the school it represents is characterized as one of the most effective schools. All other schools are characterized as typical. Fig. 1 illustrates how the effectiveness status of the secondary schools was defined by applying the approach mentioned above in analyzing achievement of students in the Pancyprian Examinations taken in June 2011. At the next step, we compared the effectiveness status of each school during two consecutive school years (i.e., during the school year 2010–2011 and 2011–2012). Table 3 illustrates the distribution of changes in the effectiveness status of our school sample from 2011 to

2012. The following observations arise from this table. First, no change in the effectiveness status of more than 70% of our school sample can be observed. It is also important to note that four schools were among the most effective schools in both time periods. Second, five schools managed to improve their effectiveness status whereas the effectiveness status of an almost equal number of schools (i.e., n = 6) declined. Third, extreme changes in the effectiveness status of the schools are not observed since there was no school which changed from most to least effective or managed to improve its effectiveness status from least effective to most effective during these two consecutive school years. Since the figures of Table 3 reveal that changes in the effectiveness

Table 1b Parameter estimates and (standard errors) for the analyses of students’ achievement in the Pancyprian Examinations per school year. Factors

Model 0

Fixed part/intercept Student Level Context Sex (boys = 0, girls = 1) School Level Context Percentage of girls Percentage of students from low SES Expenditures Building additions/improvements Equipment for labs/special classrooms Heating/air-conditioning I.T./computers Grants provided by School Boards Variance School Student Explained Significance test X2 Reduction of X2 Degrees of freedom p-value

5.10 (.30)

School Year 2009–2010 Model 1

Model 2

Model 0

3.26 (.25)

2.72 (.25)

8.20 (.38)

.11 (.05)*

.10 (.04)*

0.06 (.05) –0.11 (.07)

School Year 2010–2011 Model 1

Model 2

7.40 (.32)

5.14 (.30)

.12 (.04)*

.13(.04)*

0.05 (.04) –0.10 (.08) 0.11 0.09 0.07 0.05 0.04

(.03)* (.06) (.05) (.06) (.05)

0.09 0.14 0.08 0.07 0.05

(.05) (.05)* (.06) (.06) (.04)

8.89 (1.11) 37.92 (4.31)

7.49 (1.02) 34.64 (4.22) 4.68

6.55 (1.03) 33.70 (3.92) 6.55

6.85 (2.02) 33.45 (5.02)

6.45 (2.01) 29.02 (4.99) 4.84

5.64 (1.99) 28.61 (4.98) 6.05

732.35

611.21 121.14 1 .001

580.11 31.10 1 .001

460.81

370.51 90.30 1 .001

330.11 40.40 1 .001

* = Statistically significant effect at .05 level. 5

International Journal of Educational Development 70 (2019) 102081

L. Kyriakides, et al.

Table 1c Parameter estimates and (standard errors) for the analyses of students’ achievement in the Pancyprian Examinations per school year. Factors

Model 0

Fixed part/intercept Student Level Context Sex (boys = 0, girls = 1) School Level Context Percentage of girls Percentage of students from low SES Type of Expenditures Building additions/improvements Equipment for labs/special classrooms Heating/air-conditioning I.T./computers Grants provided by School Boards Total expenditures Variance School Student Explained Significance test X2 Reduction of X2 Degrees of freedom p-value

Model 1

5.40 (.30)

School Year 2011–2012 Model 2a

Model 2b

3.61 (.25)

2.79 (.25)

3.89 (.25)

0.11 (.05)*

0.10 (.04)*

0.10 (.04)*

0.04(.05) –0.08 (.05) 0.06 (.04) 0.05 (.03) 0.05 (.04) 0.08 (.03)* 0.03 (.03)

0.09 (.04)*

7.72 (2.19) 37.68 (4.45)

7.26 (2.08) 33.60 (4.32) 4.54

6.36 (2.02) 33.14 (4.22) 5.90

6.81 (2.04) 33.14 (4.24) 5.45

1732.35

1631.12 101.23 1 .001

1590.11 41.01** 1 .001

1620.10 11.02** 1 .001

* = Statistically significant effect at .05 level. **=For each alternative model 2 (i.e., model 2a and 2b), the reduction is estimated in relation to the deviance of Model 1.

status of a relatively large number of schools took place, we conducted DFA to find out whether changes in the effectiveness status of schools can be explained by taking into account the observed changes in school expenditures. The main purpose of the DFA employed for the purposes of this study was to predict to which of the following three groups each school of our sample belongs: (a) schools which managed to improve their effectiveness status, (b) schools which managed to keep their status at the same level, (c) schools which reduced their effectiveness status. In the first part of this section, a classification of the observed changes in the effectiveness status of our schools was presented. The next step in this analysis was to create a set of observations where both group membership and the values of the interval variables are known. For the purposes of this study, changes in school expenditure and school contextual factors were assumed to be the interval variables (i.e., the predictors). Thus, DFA was applied in order to distinguish our school samples into those which: (a) improved their effectiveness status, (b) did not change their effectiveness status, and (c) reduced their effectiveness status. At the first stage, DFA was used to reveal a function (i.e., Function 1) that is able to distinguish between those schools which managed to improve their status and the other two groups of schools (i.e., those which did not improve their status). Then, we identified a function (i.e., Function 2) which helped us to distinguish between those schools which did not change their status and those schools where a decline in their status was observed (see Table 4). The eigenvalues which emerged reveal that the first function accounts for 33% of the

variance, whereas the second function accounts for 28%. The significance of Wilks lambda reveals that both functions were found to be statistically significant, indicating that both can help us distinguish between the three groups of our schools. These figures also reveal that it was easier to distinguish between the schools which managed to improve their effectiveness status and those which did not improve their status rather than to distinguish between those which did not change their status and those which reduced their status. Table 5 shows that the percentage of schools which were correctly classified was 80%, whereas the percentage of the largest group was 72%. The value of PRE shows that placements based on this model increase by 28.6%, which translates into about 13 schools placed more correctly using this model. It is finally important to note that the main weakness of the classification reached by DFA was concerned with our difficulty to identify more than 42% of the declining schools. By looking more carefully at the schools which were classified correctly as improving schools, it was found that this analysis was in a position to predict all three schools which were among the least effective and managed to become typical (see Tables 3 and 5) but none of those which managed to improve their status from typical to most effective. With regard to the schools placed correctly in the category of declining schools, we can see that the three schools which were among the typical in 2011 and dropped to least effective were placed correctly whereas only one out of the four schools which were among the most effective and dropped to typical was identified. These findings suggest that DFA is in a better position to predict changes in the effectiveness status of

Table 2 Effect (in Cohen’s d values) of each type of expenditure and of the total expenditures on student achievement per school year. Expenditures

2007–2008

2008–2009

2009–2010

2010–2011

2011–2012

Types of Expenditures Building additions/improvements Equipment for labs/special classrooms I.T./Computers Total expenditures

N.S.S 0.11 0.12 N.S.S

0.09 N.S.S N.S.S 0.08

0.10 N.S.S N.S.S N.S.S

N.S.S 0.13 N.S.S N.S.S

N.S.S N.S.S 0.11 0.10

Note: N.S.S. = No statistically significant effect at the .05 level. 6

International Journal of Educational Development 70 (2019) 102081

L. Kyriakides, et al.

Fig. 1. The difference between the actual and expected score of each school (by considering the standard error) emerging from multilevel analysis of student achievement in the Pancyprian Examinations (June 2011).

variables used for this analysis is an indicator of SES at school level and refers to changes in the percentage of low SES students. The other two variables refer to specific types of expenditures that can have an impact on the learning process (i.e., equipment for laboratories and special classrooms, and I.T./computers). It can therefore be claimed that increases (during two consecutive school years) in each of these two types of school expenditures are related with student learning outcomes and increases in these expenditures seem to have an impact on improving the effectiveness status of schools which are among the least effective.

Table 3 The distribution of the school sample according to their effectiveness status during the school year 2010–2011 and during the school year.2011–2012. Groups of schools

Number of schools

A) Stability Remain Typical Remain Least Effective Remain Most Effective B) Improvement From Least Effective to Typical From Least Effective to Most Effective From Typical to Most Effective C) Declining From Most Effective to Typical From Typical to Least Effective From Most Effective to Least Effective

23 5 4 3 0 2

6. Conclusions The paper attempted to investigate the link between educational expenditures and student learning outcomes in the Republic of Cyprus through the use of multilevel regression analysis and discriminant function analysis. Multilevel regression analysis of student overall achievement in the Pancyprian examinations resulted in the estimation of three models (Models 0, 1, and 2). In Model 1, one student level variable (gender) and two school level variables (percentage of girls and low SES students) were added to the empty model. This model explained at least 10% of the total variance of student achievement, with most of the explained variance being at the student level. Moreover, gender was linked to student achievement as girls were found to perform better than boys in the Pancyprian examinations. A relevant contextual effect was identified for the first two years as schools with a higher percentage of girls exhibited better results. In Model 2, variables measuring different types of expenditures were added to Model 1. These variables were found to have significant effects on student achievement for each school year as they explained approximately 4% of student achievement in the Pancyprian examinations. In five analyses of the expenditure effect, the effect size of expenditures was found to be small. No type of expenditure was consistently linked to student achievement over the time period under investigation. In the second part of the analysis, we used the results of multilevel regression analysis to investigate whether changes in school expenditures could predict changes in the effectiveness status of schools. DFA was used to distinguish between schools which (a) improved their effectiveness status, (b) did not change their effectiveness status, and (c) reduced their effectiveness status. Through DFA, it was possible to predict changes in the effectiveness status of schools which were among

4 3 0

Table 4 Standardized canonical discriminant function coefficients based on analysis of school effectiveness. Variables concerned with changes in school factors

Function 1

Function 2

Percentage of students from low SES Equipment for labs/special classrooms I.T./Computers

.123 .091 .072

.111 .088 .074

Table 5 Classification results of changes in the school effectiveness status in each category. Groups of schools

Improvement Stability Declining

Predicted group membership

Total

Improvement

Stability

Declining

3 (60.0%) 2 (6.2%) 1 (14.3%)

1 (20.0%) 28 (87.5%) 2 (28.6%)

1 (20.0%) 2 (6.2%) 4 (57.1%)

5 32 7

schools which were among the least effective and became typical or the other way around but was not in a position to predict changes from the most effective category to the typical category. Table 4 shows the standardized weights for the model. One of the 7

International Journal of Educational Development 70 (2019) 102081

L. Kyriakides, et al.

the least effective and became typical; this was not the case for schools which changed from most effective to typical. Based on the discriminant function coefficients, it appears that investment in specific types of equipment had a significant effect on student learning outcomes in the case of Cyprus. Specifically, changes in educational expenditures in equipment for laboratories and special classrooms and in I.T./computers could predict changes in the effectiveness status of schools in terms of promoting student learning outcomes. Moreover, the use of DFA seems to reveal that changes in educational investment may have a positive effect on improving the effectiveness status of a school if invested in the least effective schools and not in other types of schools (typical and most effective). In fact, the advantage of using DFA analysis lies in that it is possible through its use to detect effects of expenditures in specific types of schools. This is not the case with regression techniques unless testing for interaction effects is possible as a result of a large data base (i.e. sufficient statistical power). In the case of Cyprus, our results appear to suggest that reductions in educational expenditure should not be made proportionally for all school units, as this can affect schools found in the category of the least effective schools, depriving them of their ability to improve their level of effectiveness. Least effective schools belong to this category for reasons commonly related to the composition of the student body (higher numbers of low SES and/or disadvantaged students). They thus require additional resources than schools with higher numbers of students with more privileged backgrounds. Consequently, this study seems to reveal that educational expenditures should not be allocated based on the number of students in a school but instead, such allocation should be based on the real needs of a school. The present study makes an important contribution to the literature on school resource effects on several levels. First, it seems to reveal that the impact of expenditures is bigger on schools which are among the least effective (in terms of promoting student learning outcomes) than in schools which are among the most effective, a topic which has not been adequately investigated in previous research. This finding seems, however, to be related with findings of synthesis of current studies which suggest that the effect of expenditures on the performance of low SES students may vary from country to country (e.g. Baird, 2012). Given that Cyprus is a small country with a highly centralized educational system, the present findings can be compared to findings of similar studies in larger and/or decentralized educational systems. Second, unlike most previous research on the topic, the present study uses longitudinal data to investigate the extent to which changes in student learning outcomes can be attributed to changes in expenditures at the school level. The methodological approach adopted in the present study, which extends beyond the mere examination of input-output correlations, attempts to provide more robust evidence on the topic. More sophisticated methods of analysis are needed in the context of an evidence-driven approach to educational decision making. An evidence-driven approach can be used to identify the types of expenditures that are more likely to have an impact on student learning. The failure to formulate an evidence-driven approach in the formulation of educational policy is likely to have a negative impact on school and student performance (Rolle, 2008). Thus, our methodology and findings may inform future attempts to examine school resource effects in a more comprehensive and thorough manner. In interpreting the findings of the study, certain limitations should be taken into account: The first limitation concerns the data used in the analysis. Specifically, the Pancyprian examinations data sets do not include information on student characteristics, except for gender. Thus, it was necessary to obtain data on certain variables of interest (such as socioeconomic status) from another source, which provided this information in a different form (school level as opposed to student level). If measures of SES at both the student and school level had been available, we would have had the chance to control for student background factors and generate a better estimate of the school effect. Nevertheless, two quantitative syntheses of studies investigating the

effect of SES on student achievement (Sirin, 2005; White, 1982) revealed that as typically defined (i.e., taking into account parents’ income, education, and/or occupation status) and typically used (i.e., treated as a student level variable), SES is only weakly correlated with academic achievement. However, when researchers use aggregated measures of SES, they usually report extremely high correlations between SES and academic achievement (i.e., the typical correlation is up to 0.70). This suggests that the approach used to estimate the school effect was under the circumstances methodologically sufficient but it is acknowledged that individual SES data should have been collected. Moreover, it was found that the aggregated scores of SES were not associated with student achievement, which is a finding reported in several studies conducted in Cyprus (e.g., Antoniou and Kyriakides, 2011; Kyriakides et al., 2015), and thereby phantom effects were not observed (Televantou et al., 2015). It should however be acknowledged that these findings are not in line with the results of relevant studies conducted in other European countries and should therefore be considered as a context specific result (see Kyriakides et al., 2018). A second limitation relates to the fact that the Pancyprian examinations are the only national test taken by public school students in Cyprus. It is possible that greater effects would have been found for students of younger ages since school effects tend to be smaller as the age of students increases (Kyriakides, 2008; Scheerens and Bosker, 1997). This suggests that future research should investigate the effect of different types of expenditures on the learning outcomes of students at lower levels of education. Overall, our results provide new evidence on the link between educational investment and student outcomes. Even though the findings did not reveal many significant effects on learning outcomes, the methodological approach adopted in the study differs from the traditional production function model, the limitations of which have been highlighted in the literature (see, for example, Levačić and Vignoles, 2002). By investigating the extent to which changes in the effectiveness status of schools can be related to changes in educational investment, we provide the basis for strategies that can be used to improve school effectiveness. Consequently, the findings of this study can be of value in educational policy making since they could inform decisions in relation to educational expenditure policy. This is especially the case for countries like Cyprus, which have implemented austerity programs in their attempt to deal with the global financial crisis. References Abayasekara, A., Arunatilake, N., 2018. School-level resource allocation and education outcomes in Sri Lanka. Int. J. Educ. Dev. 61, 127–141. Ajwad, M.I., 2001. New Evidence on the Link Between School Funding and Educational Outcomes: an Analysis Using School Campus-level Data. The World Bank, Washington, DC. Anderson, J.O., Milford, T., Ross, S.P., 2007. Using large-scale assessment datasets for research in science and mathematics education: program for International Student Assessment (PISA). Int. J. Sci. Math. Educ. 5, 591–614. Antoniou, P., Kyriakides, L., 2011. The impact of a dynamic approach to professional development on teacher instruction and student learning: results from an experimental study. Sch. Eff. Sch. Improv. 22, 291–311. Baird, K., 2012. Class in the classroom: the relationship between school resources and math performance among low socioeconomic status students in 19 rich countries. Educ. Econ. 20, 484–509. Brunello, G., Rocco, L., 2013. The effect of immigration on the school performance of natives: cross country evidence using PISA test scores. Econ. Educ. Rev. 32, 234–246. Bryk, A.S., Raudenbush, S.W., 1992. Hierarchical Linear Models in Social and Behavioral Research: Applications and Data Analysis Methods. Sage, Newbury Park, CA. Card, D., Krueger, A.B., 1996. School resources and student outcomes: an overview of the literature and new evidence from North and South Carolina. J. Econ. Perspect. 10, 31–50. Cheng, Y.C., 1993. Profiles of organizational culture and effective schools. Sch. Eff. Sch. Improv. 4, 85–110. Chingos, M.M., 2012. The impact of a universal class-size reduction policy: evidence from Florida’s statewide mandate. Econ. Educ. Rev. 31, 543–562. Cobb-Clark, A., Jha, N., 2013. Educational Achievement and the Allocation of School Resources. Discussion Paper No. 7551. Retrieved June 9, 2018 from. Institute for the Study of Labor, Bonn, Germany. http://ftp.iza.org/dp7551.pdf. Condron, D.J., Roscigno, V.J., 2003. Disparities within: unequal spending and

8

International Journal of Educational Development 70 (2019) 102081

L. Kyriakides, et al. achievement in an urban school district. Sociol. Educ. 76, 18–36. Creemers, B.P.M., Kyriakides, L., 2015. Developing, testing and using theoretical models of educational effectiveness for promoting quality in education. Sch. Eff. Sch. Improv. 26, 102–119. Creemers, B.P.M., Kyriakides, L., 2010. Explaining stability and changes in school effectiveness by looking at changes in the functioning of school factors. Sch. Eff. Sch. Improv. 21, 409–427. Creemers, B.P.M., Kyriakides, L., Sammons, P., 2010. Methodological Advances in Educational Effectiveness Research. Routledge, London and New York. Dee, T., West, M., 2011. The non-cognitive returns to class size. Educ. Eval. Policy Anal. 33, 23–46. Elliott, M., 1998. School finance and opportunities to learn: does money well spent enhance students’ achievement? Soc. Educ. 71, 223–245. Elliot, K., Sammons, P., 2004. Exploring the use of effect sizes to evaluate the impact of different influences on child outcomes. In: Elliot, K., Schagen, I. (Eds.), What Does It Mean? The Use of Effect Sizes in Educational Research. NFER, UK, pp. 6–24. Eurostat, 2018. Total General Government Expenditure on Education. 2016. Retrieved August 18, 2018, from. http://ec.europa.eu/eurostat/statistics-explained/index. php?title=File:Total_general_government_expenditure_on_education,_2016_(%25_of_ GDP_%25_of_total_expenditure).png. Goldstein, H., Rasbash, J., Plewis, I., Draper, D., Browne, W., Yang, M., Woodhouse, G., Healy, M., 1998. A User’s Guide to MLwiN. Institute of Education, London. Goldstein, H., 2003. Multilevel Statistical Models, 3rd ed. Edward Arnold, London. Hanushek, E.A., 1986. The economics of schooling: production and efficiency in public schools. J. Econ. Lit. 24, 1141–1177. Hanushek, E.A., 1989. The impact of differential expenditures on student performance. Educ. Res. 66, 397–409. Hanushek, E.A., 1991. When school finance ‘reform’ may not be a good policy. Harvard J. Legis. 28, 423–456. Hanushek, E.A., 1997. Assessing the effects of school resources on student performance: an update. Educ. Eval. Policy Anal. 19, 141–164. Hanushek, E.A., 2008. Education production functions. The New Palgrave Dictionary of Economics. Retrieved June 3, 2018, from: http://hanushek.stanford.edu/sites/ default/files/publications/Hanushek%202008%20Palgrave%20–% 20EducProdFunct.pdf. Hedges, L.V., Laine, R.D., Greenwald, R., 1994. Does money matter? A meta-analysis of studies of the effects of differential school inputs on student outcomes (an exchange: part 1). Educ. Res. 23, 5–14. Heyneman, S.P., Loxley, W.A., 1983. The effect of primary-school quality on academic achievement across twenty-nine high- and low-income countries. Am. J. Sociol. 88, 1162–1194. Hoxby, C.M., 2000. The effects of class size on student achievement. Q. J. Econ. 115, 1239–1285. Klecka, W.R., 1980. Discriminant Analysis: Quantitative Applications in the Social Sciences. Sage, Newbury Park, CA. Konstantopoulos, S., 2008. Do small classes reduce the achievement gap between low and high achievers? Evidence from Project STAR. Elem. Sch. J. 108, 275–291. Krueger, A.B., 1999. Experimental estimates of education production functions. Q. J. Econ. 114, 497–532. Kyriakides, L., 2008. Testing the validity of the comprehensive model of educational effectiveness: a step towards the development of a dynamic model of effectiveness. Sch. Eff. Sch. Improv. 19, 429–446. Kyriakides, L., Creemers, B.P.M., Antoniou, P., Demetriou, D., Charalambous, C., 2015. The impact of school policy and stakeholders’ actions on student learning: a longitudinal study. Learn. Instr. 36, 113–124. Kyriakides, L., Creemers, B.P.M., Charalambous, E., 2018. Equity and Quality Dimensions in Educational Effectiveness. Springer, Dordrecht, the Netherlands.

Levačić, R., Vignoles, A., 2002. Researching the links between school resources and student outcomes in the UK: a review of issues and evidence. Educ. Econ. 10, 313–331. Matrix Evidence, 2009. Outcomes Based Allocation of Educational Resources: Rapid Evidence Assessment. Retrieved May 16, 2018, from:. http://www.academia.edu/ 834880/Outcomes-based_allocation_of_educational_resources_rapid_evidence_ assessment. McEwan, P.J., 2015. Improving learning in primary schools of developing countries: a meta-analysis of randomized experiments. Rev. Educ. Res. 85, 353–394. Nicoletti, C., Rabe, B., 2012. Productivity of School Expenditure: Differences Across Pupils From Diverse Backgrounds. Institute for Economic and Social Research. Retrieved May 16, 2018, from:. http://www.iwaee.org/papers%20sito%202013/ Rabe_c.pdf. OECD, 2013. PISA 2012 Results: Excellence Through Equity: Giving Every Student the Chance to Succeed (Volume II). OECD Publishing, Paris. Psacharopoulos, G., 1996. Economics of education: a research agenda. Econ. Educ. Rev. 15, 339–344. Ram, R., 2004. School expenditures and student achievement: evidence from the United States. Educ. Econ. 12, 169–176. Rolle, A., 2008. Strengthening the Link Between Effective School Expenditures and State Funding Mechanisms. Retrieved February 18, 2019 from:. The Great Lakes Center for Education Research & Practice. https://greatlakescenter.org/docs/Policy_Briefs/ Rolle_Funding.pdf. Scheerens, J., 2013. The use of theory in school effectiveness research revisited. Sch. Eff. Sch. Improv. 24, 1–38. Scheerens, J., 2016. Educational Effectiveness and Ineffectiveness: A Critical Review of the Knowledge Base. Springer, Dordrecht. Scheerens, J., Bosker, R.J., 1997. The Foundations of Educational Effectiveness. Pergamon, Oxford. Sirin, S.R., 2005. Socioeconomic status and academic achievement: a meta-analytic review of research. Rev. Educ. Res. 75, 417–453. Snijders, T., Bosker, R.J., 1999. Multilevel Analysis: An Introduction to Basic and Advanced Multilevel Modeling. Sage, London. Statistical Service of the Republic of Cyprus, 2018. Statistics of Education 2015/2016. Retrieved February 18, 2019 from. http://www.mof.gov.cy/mof/cystat/statistics. nsf/All/204AA86C4060D499C22577E4002CA3E3/$file/EDUCATION-15_16-EN180618.pdf?OpenElement. Televantou, I., Marsh, H.W., Kyriakides, L., Nagengast, B., Fletcher, J., Malmberg, L.E., 2015. Phantom effects in school composition research: consequences of failure to control biases due to measurement error in traditional multilevel models. Sch. Eff. Sch. Improv. 26, 75–101. Thomas, S., 2001. Dimensions of secondary school effectiveness: comparative analyses across regions. Sch. Eff. Sch. Improv. 12, 285–322. Verstegen, D.A., King, R.A., 1998. The relationship between school spending and student achievement: a review and analysis of 35 years of production function research. J. Educ. Financ. 24, 243–262. Wenglinsky, H., 1997. How money matters: the effect of school district spending on academic achievement. Sociol. Educ. 70, 221–237. Winthrop, R., Anderson, K., Cruzalegui, I., 2015. A review of policy debates around learning in the post-2015 education and development agenda. Int. J. Educ. Dev. 40, 297–307. White, K., 1982. The relation between socioeconomic status and academic achievement. Psychol. Bull. 91, 461–481. World Bank, 2014. Analysis of the Function and Structure of the Ministry of Education and Culture of the Republic of Cyprus. Retrieved February 18, 2019 from:. http:// media.philenews.com/PDF/education.pdf.

9