The Impact of Contracting in and Contracting out Basic Health Services: The Guatemalan Experience

World Development Vol. 70, pp. 215–227, 2015 0305-750X/Ó 2015 Elsevier Ltd. All rights reserved. www.elsevier.com/locate/worlddev

http://dx.doi.org/10.1016/j.worlddev.2015.02.003

The Impact of Contracting in and Contracting out Basic Health Services: The Guatemalan Experience JULIAN CRISTIA a, ARIADNA GARCI´A PRADO b and CECILIA PELUFFO c,* a Inter-American Development Bank, Washington, USA b Public University of Navarra, Pamplona, Spain c Northwestern University, Evanston, USA Summary. — This paper uses a difference-in-difference strategy to evaluate a program in Guatemala that expanded access to health services through two contracting modalities. In the contracting-out model NGOs were responsible for all administrative and clinical procedures, while in the contracting-in model NGOs focused on administrative tasks and employed public employees to provide services. The evaluation design allows comparing results across both contracting models as well as with an area that did not receive additional services. Both models achieved modest results regarding immunization coverage and prenatal care, though contracting in performed slightly better. We also compare program phases and discuss policy implications. Ó 2015 Elsevier Ltd. All rights reserved. Key words — health services, contracting out, contracting in, Guatemala, Latin America

1. INTRODUCTION

health services by contracting to the private sector or by public provision. Unfortunately, the existing empirical evidence has not provided definite answers to those questions due to three major challenges. First, most evaluations have not used proper counterfactuals to generate plausible estimates of the effects of the programs. The review of the literature by Liu, Hotchkiss, and Bose (2008) found that, of 13 evaluations of contracting programs in primary health care, 12 used before–after or cross-sectional comparisons to estimate effects. Second, several evaluations have used data collected from health clinics as opposed to household surveys. Effects estimated using health clinic data may be biased due to incentives for providers to over-report results to meet targets in contracting arrangements. Finally, to address both questions described above it is necessary to contrast areas where: (i) additional funding is used to contract the private sector to increase health coverage; (ii) additional similar funding is used to increase health coverage through public provision; and (iii) no additional funding for improved health coverage is provided. To the best of our knowledge, no study has addressed both questions in the same context. 1 This paper evaluates a large contracting program implemented in Guatemala in 1996 that aimed to provide a basic package of child and maternal health services to rural, poor, and primarily indigenous communities. This program was named the Coverage Extension Program (“Programa de Extensio´n de Cobertura” or PEC). To provide services, the

Improvements in health services can increase longevity, improve health outcomes, and increase productivity (Mayer, 2001; Strauss & Thomas, 1998). Better immunization coverage, for instance, has proved to be crucial not only for reducing mortality but also for increasing productivity (Bloom, Canning, & Weston, 2005). Indeed, a full third of the welfare gains in developing countries in the last four decades can be attributed to improvements in longevity and health (Becker, Philipson, & Soares, 2005). Needless to say, despite this evidence, millions of individuals in developing countries remain without access to basic health care services. One popular policy option to improve health outcomes entails contracting the provision of health services to the private sector (Palmer, Strong, Wali, & Sondorp, 2006). Governments may improve health access and service delivery through contracting by selecting efficient suppliers and providing proper economic incentives by linking payments to the achievement of pre-defined targets (Loevinsohn & Harding, 2005). However, critics of contracting arrangements argue that the strategy is beset by a number of problems: high administrative costs, scarcity of potential providers that can participate in competitive bidding, the fragmentation that results from contracting, and the low capacity of governments to monitor private providers (Leonard, Bloom, Hanson, O’Farrell, & Spicer, 2013; Palmer et al., 2006). Because of the strengths of the theoretical arguments on both sides, it is an empirical issue whether contracting to the private sector can enhance the provision of health services. Studies that evaluated programs that contracted health services to the private sector have aimed to address two central questions: (a) do these programs improve coverage of health services? and (b) do these programs generate better health outcomes using the same level of resources when compared with public provision of services? Both questions are highly relevant for policy purposes. The first can shed light on whether expanding programs that contract the delivery of health services can yield improvements in health coverage. The second can help to determine whether governments should provide

* The authors are thankful to the Inter-American Development Bank for financing this study. The second author is also grateful to the Spanish Ministry of Science and Innovation, Project ECO2012-36480. We acknowledge excellent comments and suggestions by Nohora Alvarado, Hedi Deman, Gerard La Forgia, Sarah Humpage, Roberto Iunes, Cristina Maldonado, Isabel Nieves, Mariana Racimo, Silvia Raw, Jose´ Rodas, Toma´s Rosada, Guilherme Sedlacek and Nathaniel Barrett. The opinions expressed herein are those of the authors and do not necessarily reflect those of the Inter-American Development Bank. Final revision accepted: February 1, 2015. 215

216

WORLD DEVELOPMENT

government chose an outreach model in which NGOs set up medical teams that made monthly visits to targeted communities. The program experienced two distinct expansion phases. The first expansion period took place between the inception of the program in 1996 and 2000, when about 3 million individuals were covered. The second expansion period started in 2004 and by 2007 the PEC provided coverage to 4.3 million individuals, or one third of Guatemala’s population. Our empirical analysis focuses on the first expansion period, from 1996 to 2000. During this period, two different modalities of health service extension were implemented simultaneously. In the contracting-out model NGOs were responsible for all aspects related to the provision of the package of health services. NGOs contracted and supervised personnel, purchased inputs, organized services, made payments, and acted as administrators. In the contracting-in model, NGOs focused on administrative tasks and employed public workers to provide health services. In this model, NGOs acted as financial managers, purchasing non-personnel inputs, maintaining records, and making payments. The actual provision of services, however, was performed by public employees directed and supervised by Ministry of Health personnel. 2 The evaluation design involves comparing trends in health coverage indicators across three groups of communities that: (a) received additional services through a contracted-out model; (b) received additional services through a contracted-in model; and (c) did not receive additional services. Trends in health coverage indicators are estimated using two household surveys conducted in 1995 and 2002. Comparing trends in contracted-out and contracted-in areas, we assess the relative efficiency of the contracting-out model that provides full flexibility to NGOs regarding clinical and administrative functions. 3 Comparing trends in contracted-out areas with trends in areas that did not receive additional funding allows us to estimate the effects of expanding a contracted-out program on health coverage. This design aims to overcome the three major challenges described above by: (i) employing a difference-in-difference strategy to tackle baseline differences in outcomes across areas; (ii) using household surveys to avoid over-estimating effects because of biased reporting from health clinics; and (iii) tackling the central questions in the literature by comparing outcomes across the three described areas. Our results on the effects of the program during the first period of expansion complement existing evidence on the effects of the program during the second expansion period reported in Cristia, Evans, and Kim (2011). The distinct feature of our study is that we can contrast the effects of the contracting-out and contracting-in models. Cristia et al. (2011) focused on the effects of the contracting-out model because this model was the only one expanded during 2004–07 (the contracting-in model was largely discontinued by 2004). In our study, we also qualitatively compare the effects documented during the first and second expansion periods to shed light on how programs’ effects vary over time and discuss what underlying factors may drive those changes. 2. BACKGROUND (a) The Guatemalan health system The Guatemalan health system, like many other health systems in developing countries, is highly segmented and fragmented, and the lack of coordination among services often leads to duplication. It is structured and organized in such a way that a large percentage of the population is left without

access to health services. This system is characterized by a tripartite organization composed of the Ministry of Health, the Guatemalan Social Security Institute (IGSS) and the private sector. The Ministry of Health is responsible for providing curative and preventive care for the entire population and is the largest agent in the health care system, providing services at practically no charge. It is also responsible for defining health sector policies and for coordinating the different agents in the sector. The IGSS provides retirement benefits and health services to workers in the formal sector and their families, and it runs its own health facilities, which are separate from those of the Ministry of Health. While members of the IGSS can use Ministry of Health facilities, only affiliated members can use IGSS facilities. The remaining health services in Guatemala are provided by the armed forces and police health network, by NGOs and charitable organizations, and by the private, for-profit sector. There are two types of private, for-profit providers: (i) folk healers, herbalists, and other practitioners of traditional medicine; and (ii) providers in the modern private sector, which has grown primarily in urban areas, fueled by rising incomes and dissatisfaction with the quality of public sector care (Gragnolati & Marini, 2003). Regarding spending, total health expenditure per capita was $96 in 2000, while public health expenditure per capita amounted to $39 (World Bank, 2014). Ministry of Health services are divided into three levels. The first level is composed of health posts that are geographically distributed across the country, generally located in somewhat densely populated areas, and staffed by a certified or auxiliary nurse who provides basic preventive and curative services and refers the most difficult cases to higher levels. The second level is composed of health centers of varying capacity, though all are staffed with at least one physician. The health centers are generally located in county (or, as they are known in Guatemala, municipality) capitals. Finally, the third level of care is provided by hospitals located in the most populated cities. This supply of services does not adequately reach the most isolated and disadvantaged populations of Guatemala—rural, poor, and indigenous populations—and has resulted in a highly unequal concentration of health services in urban and nonindigenous areas. The problems of access to basic health services in Guatemala are found not only on the supply side, but also on the demand side (Becerril-Montekio & Lo´pez-Da´vila, 2011). Indigenous peoples, who constitute more than 40% of the population, tend to rely heavily on traditional medical services (traditional midwives and healers) and to distrust modern medicine. In general, this population has scant information about the medical benefits of preventive health measures. Moreover, limited efforts to adapt health care facilities and practices to local customs may undermine the uptake of services provided. Qualitative evidence suggests that geographic and financial factors, including the costs of transportation and medicine, may also play a role in the limited demand for formal health services (Gragnolati & Marini, 2003). These supply and demand problems are reflected in basic health indicators, which show large inequalities between rural, poor, and indigenous populations and urban, affluent, and non-indigenous populations. For instance, maternal mortality for indigenous women was 211/100,000, compared to 70/ 100,000 for non-indigenous mothers (SEGEPLAN, 2006). Similarly, 29% of indigenous women delivered their babies in health care centers, compared to 70% for non-indigenous women. Health inequalities are also found among children: while 69% of indigenous children under 5 years old were stunted, this figure drops to 36% for non-indigenous children.

THE IMPACT OF CONTRACTING IN AND CONTRACTING OUT BASIC HEALTHSERVICES:

Similar inequalities are found for rural and urban populations and rich and poor populations: 58% of rural children are stunted versus 34% of urban children, and 70% of children in the lowest income quintile are stunted versus only 14% of children in the highest income quintile (Ministerio de Salud Pu´blica y Asistencia Social de Guatemala, 2010). These differences in health indicators across socio-economic groups can be attributed to the medical care system but also, in large part, to the social inequities and poverty that prevail in rural areas. (b) The program In 1996, the government of Guatemala decided to launch a coverage extension program (PEC) that rapidly expanded to cover around 3 million beneficiaries by 2000, largely through 161 contracts between the Ministry of Health and 88 NGOs (Danel & La Forgia, 2005). The program’s main objective was to extend a package of basic health care services to rural, indigenous populations with little or no access to health services. For this, the PEC relied on NGOs that were contracted to provide the basic health package. This package mainly included preventive measures focused on maternal and child health as well as basic curative care. Table 1 describes the basic health package. A number of political and policy factors favored the decision to implement this unique expansion. First, the peace accords of 1996 officially ended the civil war and buoyed the government’s commitment to improve health in Guatemala, especially in the rural areas, where care was most needed

Table 1. Basic health care package Maternal care 1. Pre-natal care 2. Delivery care 3. Tetanus toxoid 4. Iron and folate supplementation during pregnancy 5. Postpartum care 6. Child spacing: education and referral Infant and child care 7. Immunizations 8. Care of acute respiratory infections 9. Care of diarrhea, cholera 10. Prevention, treatment of nutritional deficiencies 11. Growth and development monitoring of children younger than 2 years Care of illnesses and emergencies 12. Cholera 13. Dengue 14. Malaria 15. Tuberculosis 16. Rabies 17. Sexually transmitted diseases, HIV/AIDS 18. Emergencies (fractures, burns) Environmental health 19. Vector control 20. Zoonosis control 21. Sanitation 22. Water quality surveillance 23. Food hygiene 24. Improved household sanitary conditions Notes: This health package was used during the 1996–2004 period (World Bank, 2007). The basic health package was expanded after 2004 to include additional services (Pena, 2013).

217

and least available. Second, in 1996 the political cycle had produced a new government that was committed to the acceleration of basic health coverage extension. Third, NGOs appeared to be a good alternative for reaching the poorest populations, as public service offers less flexibility due to the difficulty of altering staffing levels and employment conditions. Moreover, two joint conditions for the successful implementation of the extension program were in place: a number of NGOs that specialized in health already existed in Guatemala, and the government was committed to being a reliable payer for these NGOs. NGOs were assigned to provide services to delimited geographical areas of around 10,000 individuals. The NGOs’ delivery strategy relied on mobile medical teams composed of a physician, an auxiliary nurse, and an institutional facilitator (rural health technician or nurse) that traveled at least once a month to “convergence centers” or meeting places for the local population where services were provided. At the community level, community facilitators would recruit and supervise local staff, generally traditional midwives and community health promoters, who assisted with the logistics of service provision (Cristia et al., 2011). The program basically complemented the existing network of health posts and health centers. In areas covered by the PEC, rural communities were visited regularly and child and maternal health preventive services were provided by the mobile medical team. In contrast, the rural population in areas not covered by the PEC typically had to travel to seek primary health services from health posts and health centers since outreach services were not provided. Because of the country’s weak transportation system and rugged geography, access to health services was difficult for many rural communities not served by the PEC. Annual management agreements were signed between the Ministry of Health and each NGO so that the latter was committed to achieving certain targets. During the first years there were no defined indicators or targets; instead, the national goals of the Ministry of Health were used (Table 2). In 1999, performance indicators were introduced into the contracting process and used to determine contract renewal. As a result, in 2000, 24 contracts were not renewed because providers failed to meet minimum performance standards and reporting requirements set up by the Ministry of Health (Danel & La Forgia, 2005). Supporting this process was a performance monitoring system that annually assessed NGO performance based on metrics related to community organization, training, service coverage, and financial management (Table 3). The program adopted two different modalities during the first years: contracting out and contracting in. Two main factors explain the difference between these two modalities. First, the role of the contracted NGOs is different: under the Table 2. Ministry of health 5-year targets Indicator 1. Vaccination of children under 5 2. Pre-natal care 3. Tetanus vaccination in women in their childbearing years 4. Growth monitoring of children under 2 5. Oral rehydration solution kits at home 6. Availability of essential medications

Target (%) 90 75 75 75 100 80

Notes: Targets 1–5 correspond to covered populations under the primary care programs (World Bank, 2007). Target 6 (availability of essential medications) applies to public health posts and NGO-provided services.

218

WORLD DEVELOPMENT Table 3. Indicators for determining NGO contract renewal

Community organization Number of volunteer community workers Number of community midwives Existence of community census Existence of community map Existence of community pharmacies Training Number of workers trained in at least 6 basic themes Coverage (%) Pre-natal care Iron and folate supplementation for pregnant women Pregnant women with a second dose of tetanus toxoid Infants with first-dose BCG and measles; third-dose DPT and Polio Children under 2 with growth monitoring Children under 2 with iron supplementation Administrative-financial Complete ledgers of all expenditures Inventory system up to date All personnel hired with formal contracts Monthly financial and bank statements Quarterly financial execution reports Evidence that all public financing used for service provision Notes: These indicators were incorporated to determine NGO contract renewal starting in 1999 (World Bank, 2007). Training of workers in basic themes include: mapping, census taking, ARI, diarrhea, and growth and promotion monitoring.

contracting-out model NGOs were direct providers of health services, while under the contracting-in model they functioned as health service administrators. Second, the staffing arrangements were also different. Under the contracting-out model, NGOs hired health care personnel to form mobile teams, which then provided services under NGO supervision; these NGOs were also responsible for the purchase of all inputs (except vaccines, provided by the Ministry of Health). Meanwhile, for the contracting-in model, the mobile teams were formed by Ministry of Health workers, who in some cases were complemented by additionally hired personnel. The NGOs hired under the contracting-in model were responsible for the purchase of supplies as well as the management of Ministry of Health books. They essentially served as financial and administrative agents for Ministry of Health-managed teams. However, their responsibilities did not include supervision, team management, or technical oversight, as these were overseen by Ministry of Health area offices. Both modalities, however, shared the same structure in terms of providing the basic package of health services, relying on the mobile teams and referral to the public health center or public hospital when necessary. About 60% of the agreements signed during 1996–2000 were awarded to NGOs that operated under the contracting-in model (Danel & La Forgia, 2005). 4 NGOs were paid by the Ministry of Health on a per capita basis. This per capita payment fluctuated over time but amounted to $8 in 2000. NGOs under the contracting-in modality received a lower capitation rate compared to those in the contracting-out modality for two reasons. First, they had to hire fewer personnel since the Ministry of Health provided the majority of the staff to conform the mobile teams. Second, contracting-in NGOs were assigned less remote areas and hence they received lower rates to account for reduced transportation costs (Danel & La Forgia, 2005). 5

(c) Four phases of the PEC The history of the PEC can be broken down into four phases. The first phase of the program (1996–2000) can be characterized as one of rapid expansion within a management environment that was weak in terms of planning, supervision, and monitoring. Elections in 2000 brought a new national administration that did not consider the program a priority. Hence, the PEC entered a second phase (2000–04) in which the program suffered deep budget cuts and it achieved only a slight increase in enrollment. The presidential election in 2004 brought a new government that envisioned the PEC as one of its key programs, and the team of individuals that had originally launched the program returned to the Ministry of Health. During this third phase (2004–08), population coverage started to increase and per capita spending recovered. The monitoring and supervision of the contracted NGOs was substantially strengthened due to two complementary major initiatives. First, an individual-level electronic medical records system was designed and implemented along with increased funding to NGOs to contract personnel for data-entry functions. 6 Second, a network of regional supervisors was introduced that regularly visited NGOs and compared the information from individual-level electronic records with information elicited from actual beneficiaries in the field. These checks reduced the ability of NGOs to over-report outputs and allowed program administrators to rely on more accurate data on actual health services provided. Finally, during a fourth phase (2008–12), the PEC declined as lack of political support again led to severe budget cuts (the supervision network was largely scrapped) and payments to NGOs were delayed. In Figure 1 we show the evolution of coverage under the PEC until 2007. The figure shows two expansion phases of the program: the first expansion (1996–2000) and the second expansion (2004–07). These periods of expansion, followed by budget cuts, result in a clear seesaw pattern. 3. RESULTS FROM THE FIRST EXPANSION (1996–2000) (a) Data We use data from two National Surveys of Maternal and Child Health (Encuesta Nacional de Salud Materno Infantil, ENSMI) that were collected in Guatemala in 1995 and 2002 to estimate effects. These surveys have a multistage, stratified sample design with weights to produce nationally representative estimates. The surveys provide information for women ages 15–49 on family planning, prenatal care, birth delivery for mothers, and their children’s vaccinations. Data on socio-economic indicators are also collected, including age, education, marital status, and household variables such as housing conditions and access to services (electricity, sewage, and running water). 7 The PEC aimed to improve child and maternal health by increasing coverage of preventive services. We provide evidence regarding the effects of the program on these dimensions by analyzing effects on the following outcomes. For children aged 0–2, we analyze changes in coverage of first doses of BCG, Polio, and DPT, as well as boosters for the latter two vaccines. For mothers aged 15–44 whose last child was born within 2 years of the survey, we assess effects on frequency and location of prenatal care, first trimester check-ups, and receipt of the tetanus vaccine. We also analyze effects on the

THE IMPACT OF CONTRACTING IN AND CONTRACTING OUT BASIC HEALTHSERVICES:

219

Figure 1. Population covered by PEC and per capita expenditure.

use of family planning methods, although this was not an outcome targeted by the program. Exploring effects on non-targeted outcomes is relevant given the potential for either positive or negative effects on them. 8 Because the program explicitly focused on improving access to health care services in rural areas, we have omitted data from urban areas, including the entire department that contains the national capital. We also excluded the department of El Pete´n from the analysis because it was not included in the 1995 survey. To estimate effects, we analyze changes in health services coverage during 1995–2002 (before and after the first expansion of the program) in three distinct geographical areas: (i) areas covered by the contracting-out model by 2002, (ii) areas covered by the contracting-in model by 2002, and (iii) areas that did not benefit from either of these health extension coverage programs (the comparison group). Table 4 presents information on levels and trends in socioeconomic indicators for these three areas. Columns 1–2 and 3–4 present statistics for the population covered by the contracting-out and contracting-in models, respectively, whereas columns 5–6 present statistics for the comparison group. Results indicate that women in the sample have low levels of education, a substantial share of them are indigenous, and they tend to have deficient access to basic housing services. Furthermore, education levels and access to housing services are lowest in areas covered by the contracting-out model, slightly higher in areas covered by the contractingin model, and highest for areas not covered by the program. This is in line with qualitative evidence suggesting that the program was targeted to the most underserved rural areas and that communities assigned to the contracting-out model were even poorer than those assigned to the contracting-in model. (b) Identification strategy The documented differences in socio-economic characteristics across the three analyzed areas suggest that simple crosssectional comparisons of outcomes may yield biased estimates of program effects. Such patterns motivate the adoption of a difference-in-difference approach. The guiding assumption for this approach is that trends in outcomes would have evolved similarly between the treatment and comparison

groups in the absence of the treatment. As mentioned, although these areas present quite different levels in terms of socio-economic indicators, they present similar trends in these same variables, suggesting that the identification assumption is likely to hold. 9 Using the repeated cross-sections from the 1995 and 2002 surveys, we can estimate the effects of the two treatments under a difference-in-difference specification. That is, we estimate effects using the following specification: y it ¼ b0 þ COit b1 þ CI it b2 þ P it b3 þ COit P it b4 þ CI it P it b5 þ xit h þ eit

ð1Þ

where y is the outcome variable; CO and CI indicate the coverage model (contracting out and contracting in, respectively); P is an indicator for the post period (2002); x is a vector that captures characteristics of the individual;b0, b1, b2, and b3 are ancillary parameters; h is the vector of coefficients for individual-level controls; and, b4 and b5 correspond to the estimate of the average effect of the contracting-out and contracting-in models, respectively; i indexes individuals and t, time. We estimate effects under two specifications. In the first specification we do not include additional controls, while in the second we include the following variables as controls: age and indicators for indigenous, educational categories, running water, dirt floor, marital status, and employment. Finally, we cluster standard errors at the community level in all regressions to account for correlated errors at this level of aggregation. It is important to note that by the time outcomes were measured for the first expansion, the majority of NGOs had been operating for a number of years in the assigned communities. That is, the bulk of the first expansion took place during 1997– 99 and outcomes were ascertained in 2002. Hence, we could expect that the majority of NGOs had been operating in these communities for about 3–5 years when effects were assessed, since only a small number of contracts were canceled in the 1997–2002 period. (c) Results This subsection presents the main results of the paper. Table 5 reports impacts on children’s vaccination rates.

220

WORLD DEVELOPMENT Table 4. Summary statistics Contracting out

Contracting in

Not covered

1995 (1)

2002 (2)

1995 (3)

2002 (4)

1995 (5)

2002 (6)

10.961 0.493 0.684 461

12.031 0.483 0.659 327

11.557 0.508 0.540 505

12.300 0.442 0.502 359

11.605 0.475 0.450 1,804

12.229 0.509 0.439 1,071

Mothers aged 15–44 with children younger than 2 years old Age 27.739 26.440 Married 0.952 0.942 Indigenous 0.686 0.652 Employed 0.106 0.112 No education 0.5 0.489 Primary education 0.399 0.480 Secondary education 0.014 0.031 Running water 0.312 0.437 Flush toilet 0.008 0.005 Electricity 0.123 0.536 Dirt floor 0.898 0.809 N women 447 308

27.497 0.946 0.544 0.143 0.457 0.522 0.021 0.381 0.028 0.255 0.807 471

26.298 0.914 0.498 0.209 0.38 0.573 0.044 0.331 0.046 0.757 0.66 334

27.425 0.926 0.448 0.167 0.432 0.526 0.037 0.392 0.051 0.376 0.717 1,724

26.920 0.942 0.432 0.208 0.384 0.562 0.054 0.418 0.086 0.640 0.616 1,010

Women aged 15–44 years old Age Indigenous No education N women N communities

26.608 0.503 0.396 1,169 42

26.919 0.382 0.342 755 35

26.962 0.386 0.355 4,451 165

27.042 0.374 0.311 2,682 124

Children age 0–2 years old Age (in months) Female Indigenous N children

27.375 0.661 0.532 980 36

25.535 0.620 0.426 704 32

Notes: This table presents statistics for areas covered by contracting-in models and contracting-out models and for areas not covered by any contracting model. Summary statistics constructed from the 1995 and the 2002 National Surveys of Maternal and Child Health. The sample includes rural population. Observations from the departments of Pete´n and Guatemala were excluded. Columns 1–2 present statistics for areas with contracting-out models. Columns 3–4 show statistics for areas with contracting-in models. Columns 5–6 report statistics for areas not covered by any contracting model.

Table 5. Estimated impacts on child immunization

Child had immunization card Immunization card was presented BCG – dose 1 DPT – dose 1 Polio – dose 1 DPT – dose 3 Polio – dose 3

Contracting out vs. not covered

Contracting in vs. not covered

Contracting out vs. contracting in

(1)

(2)

(3)

(4)

(5)

(6)

0.067 [0.046] 0.010 [0.070] 0.122 [0.050]** 0.109 [0.043]** 0.102 [0.041]*** 0.049 [0.068] 0.035 [0.067]

0.058 [0.044] 0.011 [0.071] 0.112 [0.048]** 0.109 [0.041]*** 0.102 [0.038]*** 0.048 [0.069] 0.034 [0.068]

0.066 [0.039]* 0.004 [0.053] 0.123 [0.044]*** 0.077 [0.038]** 0.074 [0.033]** 0.112 [0.060]* 0.105 [0.058]*

0.061 [0.038] 0.005 [0.052] 0.117 [0.046]** 0.091 [0.038]** 0.086 [0.033]*** 0.130 [0.059]** 0.114 [0.058]*

0.001 [0.053] 0.015 [0.080] 0.001 [0.060] 0.032 [0.051] 0.029 [0.046] 0.062 [0.080] 0.070 [0.080]

0.010 [0.050] 0.020 [0.077] 0.014 [0.056] 0.013 [0.047] 0.014 [0.042] 0.075 [0.080] 0.080 [0.082]

Notes: This table presents estimates of the effects of the contracting-out and contracting-in models on child immunization. Each cell corresponds to one OLS regression. Labels in rows correspond to dependent variables. Columns 1–2 present effects of contracting out versus no coverage. Columns 3–4 report effects for contracting in versus no coverage. Columns 5–6 present effects for contracting out versus contracting in. The sample includes rural children aged 0–2 in 1995 or 2002 not living in the departments of Pete´n or Guatemala. All regressions control for year and area dummies (contracting out, contracted in or never covered). Regressions in even-numbered columns also include age and indicators for indigenous status, educational categories, marital status, employment, running water, and dirt floor. Significance at the 1%, 5% and 10% levels is indicated by ***, ** and *, respectively.

Columns 1–2 present estimated effects of the contracting-out model versus areas not covered. Columns 3–4 report impacts of the contracting-in model versus areas not covered and

columns 5–6 present effects of the contracting-out model versus the contracting-in model. Results indicate that both models have produced statistically significant positive effects on

THE IMPACT OF CONTRACTING IN AND CONTRACTING OUT BASIC HEALTHSERVICES:

first doses of vaccines (BCG, DPT and Polio) of about 10–11 percentage points in the case of the contracting-out model and 9–12 percentage points for the contracting-in model. Regarding third doses (DPT and Polio), there are statistically significant positive effects for the contracting-in model of about 12 percentage points. Estimated effects for these outcomes are also positive, though not statistically significant, for the contracting-out model. Finally, the estimated effects on the fraction of children who have an immunization card are positive for both models though, in general, they are not statistically significant. Table 6 presents estimated effects on prenatal care and family planning. The top panel reports impacts on prenatal care of mothers of young children. In particular, we explore effects on the number of prenatal care visits, first trimester visits, location of visits, and vaccines. 10 The bottom panel presents estimated effects on family planning outcomes. There is no evidence that the contracting-out model generated effects on prenatal care or family planning. In contrast, results show that the contracting-in model has produced a 12 percentagepoint increase in coverage of the first dose of tetanus vaccine and also a similar reduction in the fraction of women reporting that their prenatal care check-ups took place at their homes or at the homes of traditional birth attendants. However, there are no statistically significant effects on other prenatal outcomes, including whether the woman had at least one prenatal care visit, the number and timing of prenatal care visits and third doses of tetanus vaccine. 11 Regarding contraceptive

221

behavior, there is some evidence suggesting positive effects in areas covered by contracting in. However, the evidence is not conclusive, as the coefficients are marginally significant in the specification without individual-level controls and not significant when these controls are added. Table 7 explores the robustness of the empirical strategy used in this paper. The identification assumption for the difference-in-difference framework is that, in the absence of treatment, outcomes variables in the treatment and comparison samples evolve similarly over the analyzed period. Although this assumption is untestable, we can check whether other covariates, which should have been unaffected by the program, have evolved similarly. To that end, we run difference-in-difference models to test for differential trends in socio-economic characteristics (e.g., age, education, and ethnicity) and variables related to household services (e.g., running water and electricity). In general, those characteristics in the three groups seem to have evolved similarly and therefore this analysis provides support for the empirical strategy followed. Finally, Tables 8 and 9 in the Appendix report average health outputs for children and women by year and coverage status. Table 8 documents that, although immunization rates improved under both modalities (contracting-out and contracting-in), neither reached levels close to 100%. First doses of the vaccine DPT and Polio are exceptions, although third doses of these same vaccines are still below 80%. Similarly, Table 9 shows that there is room for further increases in the prenatal and family planning indicators. For instance, the cov-

Table 6. Estimated impacts on pre-natal care and family planning Contracting out vs. not covered (1)

Panel B: women aged 15–44 years old Family planning: current use Any method Modern method

Contracting out vs. contracting in

(2)

(3)

(4)

(5)

(6)

0.043 [0.046] 0.601 [0.384] 0.062 [0.048] 0.042 [0.061] 0.046 [0.056] 0.031 [0.052]

0.007 [0.049] 0.119 [0.416] 0.027 [0.043] 0.004 [0.049] 0.132 [0.044]*** 0.017 [0.054]

0.002 [0.048] 0.118 [0.393] 0.035 [0.041] 0.012 [0.051] 0.115 [0.043]*** 0.005 [0.053]

0.049 [0.058] 0.432 [0.472] 0.024 [0.056] 0.056 [0.064] 0.079 [0.063] 0.014 [0.069]

0.037 [0.057] 0.494 [0.447] 0.033 [0.056] 0.044 [0.069] 0.058 [0.060] 0.028 [0.066]

0.035 [0.063]

0.028 [0.065]

0.126 [0.060]**

0.118 [0.061]*

0.161 [0.077]**

0.150 [0.080]*

0.006 [0.023] 0.006 [0.020]

0.014 [0.022] 0.024 [0.020]

0.087 [0.048]* 0.087 [0.044]**

0.045 [0.032] 0.049 [0.030]

0.093 [0.050]* 0.082 [0.046]*

0.042 [0.036] 0.035 [0.034]

Panel A: mothers aged 15–44 with children younger than 2 years old Pre-natal care (PNC): Visits and vaccine Any PNC 0.056 [0.047] Number of PNC visits 0.552 [0.385] Three or more PNC visits 0.051 [0.049] PNC visit during first trimester 0.051 [0.058] Tetanus vaccine 0.053 [0.059] Three doses of tetanus vaccine 0.032 [0.054] Pre-natal care (PNC): Place of service PNC at traditional birth attendant’s home or own home

Contracting in vs. not covered

Notes: This table presents estimates of the effects of the contracting-out and contracting-in models on pre-natal care and family planning. Each cell corresponds to one OLS regression. Labels in rows correspond to dependent variables. Columns 1–2 present effects of contracting out versus no coverage. Columns 3–4 report effects for contracting in versus no coverage. Columns 5–6 present effects for contracting out versus contracting in. The sample includes rural population not living in the departments of Pete´n or Guatemala. All regressions control for year and area dummies (contracting out, contracting in or never covered). Regressions in even-numbered columns also include age and indicators for indigenous status, educational categories, marital status, employment, running water and dirt floor. Significance at the 1%, 5% and 10% levels is indicated by ***, ** and *, respectively.

222

WORLD DEVELOPMENT Table 7. Robustness check: testing trends in covariates

Children age 0–2 years old Age (in months) Female Indigenous

Contracting out vs. not covered (1)

Contracting in vs. not covered (2)

Contracting out vs. contracting in (3)

0.446 [0.532] 0.044 [0.053] 0.015 [0.071]

0.119 [0.535] 0.100 [0.042]** 0.028 [0.068]

0.327 [0.580] 0.056 [0.062] 0.013 [0.089]

0.694 [0.739] 0.048 [0.031] 0.033 [0.065] 0.025 [0.036] 0.028 [0.045] 0.015 [0.050] 0.006 [0.020] 0.076 [0.084] 0.017 [0.019] 0.238 [0.083]*** 0.042 [0.048]

0.100 [0.898] 0.021 [0.036] 0.012 [0.086] 0.060 [0.037] 0.020 [0.058] 0.031 [0.063] 0.006 [0.022] 0.174 [0.115] 0.022 [0.013]* 0.088 [0.096] 0.054 [0.057]

0.231 [0.443] 0.109 [0.070] 0.010 [0.031]

2.151 [0.579]*** 0.081 [0.081] 0.052 [0.039]

Mothers aged 15–44 with children younger than 2 years old Age 0.794 [0.692] Married 0.027 [0.026] Indigenous 0.021 [0.070] Employed 0.035 [0.034] No education 0.047 [0.055] Primary education 0.046 [0.057] Secondary education 0.000 [0.018] Running water 0.098 [0.093] Flush toilet 0.038 [0.017]** Electricity 0.150 [0.082]* Dirt floor 0.012 [0.049] Women aged 15–44 years old Age Indigenous No education

1.920 [0.475]*** 0.028 [0.054] 0.062 [0.036]*

Notes: This table explores differential trends in covariates. Each cell corresponds to one OLS regression. Column 1 presents the effects of contracting out versus no coverage. Labels in rows correspond to dependent variables. Column 2 reports the effects of contracting in versus no coverage and column 3 shows the effect of contracting out versus contracting in. The sample includes rural population not living in the departments of Pete´n or Guatemala. All regressions control for year and area dummies (contracting out, contracting in or never covered). Standard errors are clustered at the community level. Significance at the 1%, 5% and 10% levels is indicated by ***, ** and *, respectively.

erage of tetanus vaccine among mothers in non-covered areas remains only 66%.

4. POLICY LESSONS FROM THE PEC EXPERIENCE This section discusses the empirical findings, places them in context and proposes a number of hypotheses regarding potential explanations. It also advances some tentative policy implications. (a) The first expansion: comparing contracting out and contracting in Results show that the contracting-in model seems to have performed slightly better than the contracting-out model, especially regarding the utilization of nurses and

physicians for prenatal care. Several factors may explain this result. First, while some NGOs under the contracting-out model had to start from scratch in terms of recruiting, hiring, and training personnel, the contracting-in model used Ministry of Health personnel with ample experience in rural health delivery. Furthermore, under the contracting-in model, in most cases the areas of work were chosen by Ministry of Health district managers in consultation with the rural health technicians assigned to the mobile teams. Generally, these rural technicians chose catchment areas where they had worked, lived, or had other previous experience. The use of rural technicians who were already known to the targeted population probably helped to reduce demand barriers. However, the quantitative importance of this explanation may be limited. To start with, when outcomes were measured for the first expansion, the majority of NGOs, including those under the contracting-out model, had been operating in covered

THE IMPACT OF CONTRACTING IN AND CONTRACTING OUT BASIC HEALTHSERVICES:

communities for about 3–5 years. Hence it is expected that NGOs had already developed substantial experience in the provision of services in covered areas. Moreover, some of the NGOs under the contracting-out model may have had experience in the provision of health services prior to the start of PEC. In fact, as mentioned above, the existence of a pool of NGOs with experience in health provision in rural areas was one of the motivations for adopting a contracting arrangement for the PEC. Second, the fact that the contracting-in model employed Ministry of Health workers in the mobile teams led to the involvement of regional health staff in the supervision of health service delivery. In addition, regional health authorities allowed the NGOs working under the contracting-in model to have their offices in some Ministry of Health buildings, which allowed for easier interaction with regional health managers. NGO personnel in this scenario may have been more aligned with Ministry of Health goals than NGO personnel under the contracting-out model. Indeed, previous qualitative analyses of contracting out highlighted a lack of coordination between the Ministry of Health and NGO personnel (see Ministerio de Salud de El Salvador & The World Bank, 2006 and, for Bolivia, Lavadenz, Schwad, & Straatman, 2001), leading to rivalries between private and public providers as well as to the provision of health services outside of Ministry of Health protocols. The organization of the contracting-in model seems to ameliorate such coordination problems. Finally, during the first 2 years of the first expansion the contracting-out model did not have a specific set of indicators with targets by which NGOs could be evaluated. However, as mentioned, these indicators were defined in 1999, and the following year a number of contracts were not renewed for failing to meet targets. Despite the slightly better performance of the contracting-in model, overall effects on health outputs were limited during the first expansion, and this poor performance likely had more to do with insufficient implementation capacity than the choice of modality. We return to this issue below after comparing the effects of contracting out during the first and second expansions of the PEC. (b) Comparing the effects in the first and second expansions Results from Cristia et al. (2011) indicate that during the second expansion the contracting-out modality increased first-dose vaccination rates by about 13–21 percentage points and third-dose vaccination by about 30 percentage points. There was no evidence of effects on having at least one prenatal care visit or on the number of visits. However, results indicate that the program increased the fraction of pregnant women receiving prenatal care from doctors or nurses by about 24 percentage points. 12 There were no effects on contraceptive behavior. Comparing the effects of contracting out estimated for the first expansion (1996–2000) with those from the second expansion (2004–07), we can highlight several patterns. First, the program’s effects, across modalities and periods, are typically concentrated on increasing children’s immunization and the replacement of traditional birth attendants by physicians or nurses as prenatal care providers. Second, the program seems to be unable to affect the extensive margin of prenatal care (i.e., the fraction of women having any check-ups) or the intensive margin (number of visits). Similarly, there is little evidence that the program could alter contraception choices. Third, and most importantly, the point estimates suggest substantially larger effects of the contracting-out model during the second expansion in comparison to the first. 13

223

Why did the contracting-out model perform better during the second expansion? Average per capita spending during the first and second expansions was quite similar, suggesting that differences in resources are not the driving force. Another potential explanation relates to learning at the local level by mobile medical teams and patients. As time goes by, mobile medical teams can learn procedures, acquire knowledge about how to solve common issues, and establish stronger relationships with communities. This learning curve can also be found on the patient side—as patients learn about the availability of services and the procedures to seek them, demand for health services may increase (Casabonne & Kenny, 2012). However, learning at the local level does not seem to be a likely explanation for the better results documented in the second expansion. In the evaluation of the first expansion presented in this paper, the majority of NGOs had been operating in the covered communities for 3–5 years at the time outcomes were measured. In contrast, in the evaluation of the second expansion presented by Cristia et al. (2011), NGOs had been operating in covered communities for only 1–2 years when outcomes were assessed. 14 That is, treatment communities analyzed in the second expansion had been in the program for less time compared to treatment communities evaluated in the first expansion at the time of their respective follow-ups. Hence, learning at the local level does not seem a likely explanation. A more plausible explanation for the better results during the late expansion relates to the substantial learning acquired at the central level regarding how to effectively manage the contracting process. There is qualitative evidence supporting this explanation. In particular, clear improvements in managerial capacity have been documented regarding the capacity to select providers, specify contractual provisions, negotiate and process contracts, issue payments, monitor performance, and produce and analyze information on suppliers’ production (Cristia et al., 2011; Pena, 2013). Moreover, a critical improvement seems to have been the introduction of individual-level electronic medical records, coupled with substantial investments in developing a strong supervision system. These improvements in the capacity of the program to accurately estimate health production by NGOs could have allowed improvements in selecting efficient suppliers and ensuring that production targets were actually attained. We now return to the limited effects documented during the first expansion in both the contracting-out and contracting-in models. Regarding the contracting-out model, its low effectiveness seems to be linked to the limited capabilities of the central management of the program, during the first expansion, regarding the core aspects of the contracting process (specification of targets and selection, monitoring and contract cancelation with private suppliers). In terms of the limited effects documented for the contracting-in model during the first expansion, it is difficult to provide a definite explanation. As mentioned, this model was not evaluated during the second expansion and hence there is less information to draw from. It may be the case that certain aspects that were better developed as the program was refined, such as the specification of clear targets, the implementation of an effective information system and the development of a network of supervisors, could have increased the effects of the contracting-in model. However, the capacity of the contracting-out model to improve results by canceling contracts with less effective suppliers as well as providing credible threats of termination to induce the adoption of better practices may have had a more limited role in a contracting-in model where public employees are ensured job stability. Still, whether a contracting-in model

224

WORLD DEVELOPMENT

can provide better results under a more developed management system remains an open question in this study. Finally, the poor results obtained during the first expansion could also be explained by weak demand for the services provided by the PEC during this period. Even when a program like the PEC reaches the poorest indigenous and rural communities, the utilization of health services could still be hindered by cultural barriers. These cultural barriers seemed strongest during the first period of implementation, but they persisted in the later phases of the program, leading eventually to the implementation of demand-side interventions in the form of conditional cash transfer programs (Pena, 2013). Other demand-side interventions have been implemented more recently in Guatemala, including those that focus on using community networks to foster attitudinal and behavioral changes in rural and indigenous communities (InterAmerican Development Bank, 2012). These demand-side interventions could potentially complement supply-side health coverage expansion strategies. (c) Scaling up A comparison of the results achieved during the first and second expansions of the PEC suggests important policy implications related to the optimal strategy for scaling up this type of program. During the rapid and massive scaling up of the program during the first expansion, quantity prevailed over quality, leading to poor results. The Guatemalan experience highlights the benefits of launching a pilot program that first covers a small area. This would allow the functions required at the central level to be developed and would thereby guarantee a more successful roll-out for the program, as well as greater overall impact. As King and Behrman (2009) observe: “Many governments expand programs quickly even in the absence of credible evidence, given the urgency of the problems to be addressed. However, this impatience can result in costly but avoidable mistakes and failures; it can also result in really promising new programs being terminated too soon when a rapid assessment shows negative or no impact.” Proponents of gradual expansions also highlight the importance of implementing pilots and evaluating them before scaling up (Duflo, 2004). Still, we acknowledge that, although conceptually sensible, there is no strong empirical evidence regarding the benefits of a more gradual process of expansion. Moreover, this general notion in favor of experimenting and refining processes on small scale applies to both contracting programs and those entailing pure public provision. (d) Long-term political sustainability Figure 1 documents a seesaw pattern in per capita program funding that is commonly observed in developing countries. As government changes hands, programs without broad support are undermined and, as a result, they may be discontinued or underfunded. The PEC was designed without broad consensus: the original design called only for the contracting-out model as mentioned before. Thus, protests from Ministry of Health workers resulted in the negotiation and implementation of the contracting-in model. Both modalities co-existed during the first expansion of the program (1996– 2000), only to decline with the arrival of a new government in 2000, as reflected in Figure 1. When the program administrators responsible for launching the PEC returned to the Ministry of Health during the 2004– 07 term, the contracting-out model was expanded while the

contracting-in model was mostly discontinued, in accordance with the dominant political trend at the time. Another change of government in 2008 brought a new phase (2008–11) of decline during which the PEC struggled to survive. Supervision was cut back, and government payments to NGOs were delayed due to liquidity constraints in the Ministry of Finance and to a lack of political will on the part of the Ministry of Health. As a result, some NGOs have exited the program and others have reduced the basic health package delivered by mobile teams (Pena, 2013). These swings in political support and funding have been previously linked to how programs’ expansions are funded. It has been argued that health reforms that benefit predominantly poor populations, as is the case of the PEC, are more likely to be politically supported if they are externally funded (Kaufman & Nelson, 2004; Lloyd-Sherlock, 2000). Consistent with this view, during the two periods that the PEC received a higher per capita transfer, its funding had been complemented through loans from the Inter-American Development Bank and the World Bank (Pena, 2013). While the PEC has suffered heavy budget cuts when political support has faltered, these cuts have not reduced the level of beneficiaries covered by the program, although they have affected the quality of services (Pena, 2013). Maybe the PEC is “too big to close” (4,600,000 beneficiaries). However, in other countries there have been instances in which a lack of political support brought about the total discontinuation of similar programs. This was the case, for instance, in countries such as Peru or Honduras, where contracting-out programs that were providing basic health services to poor and rural populations were completely discontinued after governmental changes, leaving those populations without health coverage (World Bank, 2007). Other countries such as Pakistan (Zaidi, Mayhew, Cleland, & Green, 2012) have also encountered problems regarding government ownership of contracting initiatives, revealing that health reforms are inherently part of the political process and are affected by the wider policy context. In order to achieve long-term sustainability, it seems necessary to compromise, that is, to implement interventions that are effective and have a positive impact while drawing on broad political and societal support that includes key stakeholders such as health workers’ unions (Kaufman & Nelson, 2004). In the case of Guatemala, one potential way to keep the PEC running—along with all of the capacities it has helped to create—is either to reestablish the contracting-in model or to add to the contracting-out model some of the favorable features of contracting in that we have pointed out, such as integrating the mobile teams with the Ministry of Health’s formal network of providers. Indeed, the possibility of reforming the PEC along the lines of the contracting-in model has been brought up in recent discussions on coverage and quality of healthcare provision in rural and poor areas of Guatemala. Results from this study are therefore relevant to this debate. 5. CONCLUSIONS This paper provides evidence on the results of one of the largest health contracting experiences to date in developing countries, a coverage extension program in rural Guatemala. Two different strategies were used during the first expansion (1996–2000) of the program: contracting-in and contractingout services to NGOs. Using a difference-in-difference model, we estimate the impact that each of these strategies had on key children’s and women’s health outputs, and found slightly

THE IMPACT OF CONTRACTING IN AND CONTRACTING OUT BASIC HEALTHSERVICES:

better results for the contracting-in model. However, the documented effects are limited and are substantially smaller than those documented during the second expansion of the program (2004–07). Our analysis of the PEC over time has two main policy implications. First, while it is important to determine which delivery strategy can generate larger effects, results point to the critical role played by the capacity of program administrators to manage the contracting process effectively. One option to ensure the development of that capacity consists of starting at a small scale and gradually developing knowledge and procedures before scaling up the program. In the case of Guatemala, a more gradual approach might have set up a more solid basis for the program to expand and generate results at scale sooner. Second, program design should aim to generate not only large effects on health outputs, but also long-term political sustainability. Political support seems key for allowing programs to develop capacity and for avoiding costly policy reversals. The current study faces two main limitations. First, because of data limitations, we are unable to assess effects on final health outcomes such as infant mortality. However, existing evidence suggests that the changes in health care use and

225

outputs documented above should produce improvements in health outcomes. There is extensive support for the claim that significant health benefits as well as productivity gains are produced by increases in vaccination rates (Bloom et al., 2005; Ehreth, 2003; Lee, 2012). Moreover, the displacement of traditional midwives by physicians and nurses as prenatal care providers should also lead to health improvements. This should be expected given the body of evidence documenting the beneficial effects of specific medical interventions performed during check-ups, and survey results suggesting that traditional midwives, even trained ones, provide sub-standard care (Goldman & Glei, 2003; Jones, Steketee, Black, Bhutta, & Morris, 2003). A second limitation is linked to the near total discontinuation of the contracting-in modality by 2004. As a result, we could not continue our comparison of contracting modalities into the second expansion of the program (2004–07), during which clear management advances were introduced. A comparison of both modalities in the first and second expansions would no doubt have shed more light on the pros and cons of each modality as well as their implementation challenges. These questions remain open for future research.

NOTES 1. Two important studies have used an experimental design and household surveys to yield unbiased estimates. Bloom et al. (2006) compared contracting-out and contracting-in models for providing health services in Cambodia. Basinga et al. (2010) contrasted clinics that received payments linked to output indicators with clinics that received fixed payments (adjusted to equalize average payment across both groups). Hence, these studies shed significant light on the relative efficiency of contracting out (versus contracting in) and linking payments to targets (versus providing flat payments). But they do not tackle the question of whether expanding programs that contract the delivery of health services can produce significant improvements in health coverage. 2. The program was originally designed to be implemented only under the contracting-out modality, but protests from public health workers, who perceived the contracting-out modality as a threat to their job stability and benefits, resulted in a negotiation between the government and unions. Implementing the contracting-in model in certain areas was a result of this negotiation. 3. Note that the contracted-out model is not compared with a model under pure public provision. However, because of the limited role of NGOs in the contracting-in model, we consider the results to be informative about that comparison. 4. Another potential modality would include the provision of services by the public sector without participation by NGOs. There are examples of mobile medical teams under a pure public provision model such as in EBAIS units in Costa Rica (Sa´enz, Acosta, Muiser, & Bermu´dez, 2011).

7. More information on the 1995 and 2002 surveys can be accessed at: http://encuestas.ccp.ucr.ac.cr/camerica/gu95.html and http://encuestas. ccp.ucr.ac.cr/camerica/gu02.html.

8. Bloom et al. (2006) described potential scenarios for positive or negative effects on non-targeted outcomes. For example, if contracting arrangements induce decreases in absenteeism from health workers, then there may be positive effects on non-targeted outcomes. On the other hand, if health workers under contracting arrangements shift resources from unmeasured care to targeted indicators, there may be negative effects on non-targeted outcomes.

9. We come back to this issue in subsection 3(c) and formally test whether there are differences among the trends in covariates across the three analyzed groups.

10. Unfortunately, there are no data on the type of prenatal care provider (traditional birth attendant, physician, nurse or other). Hence, we estimate effects on the location of prenatal care provision to shed some light on the provider type. Reductions in the fraction of pregnancy checkups taking place at the traditional birth attendant’s home or at the patient’s home may signal an increase in check-ups provided by physician or nurses.

5. We could not access reliable information to compare total per capita costs across both modalities. However, the information on how the capitation rates were determined suggests a general intent to cover the same expenses across modalities.

11. Cristia et al. (2011) also found that the effects of the contracting-out model implemented in 2006 on women’s health outcomes are concentrated in the replacement of traditional birth attendants by physicians or nurses (no effect on whether women have prenatal care or on number of visits). It seems, then, that these programs are only altering the choice of provider type without affecting other behaviors.

6. In addition to general reporting, the new computer system was used to track patients with scheduled services such as children needing vaccinations and pregnant women.

12. Effects on the increased use of physicians or nurses for prenatal care reported in Cristia et al. (2011), though large, are marginally statistically significant.

226

WORLD DEVELOPMENT

13. Estimated effects on first-dose vaccinations are about 11 percentage points for the first expansion compared to 13–21 percentage points for the second expansion. Furthermore, effects on third-dose vaccination are 3–5 percentage points during the first expansion compared to 30 percentage points during the second expansion. Finally, effects on prenatal care by physician or nurse seem to be negligible during the first expansion versus effects of 24 percentage points during the second expansion.

14. Cristia et al. (2011) estimated effects comparing outcomes in 2006 of communities that entered the PEC during 2004–05 with those of communities not covered by the program.

REFERENCES Basinga, P., Gertler, P., Binagwaho, A., Soucat, A., Sturdy, J., & Vermeersch, C. (2010). Paying primary health care centers for performance in Rwanda. Policy research working paper series, 5190. Washington, DC: World Bank. Becerril-Montekio, V., & Lo´pez-Da´vila, L. (2011). Sistema de salud de Guatemala. Salud Publica de Me´xico, 53(2), 197–208. Becker, G., Philipson, T., & Soares, R. (2005). The quantity and quality of life and the evolution of world inequality. American Economic Review, 95(1), 277–291. http://dx.doi.org/10.1257/0002828053828563. Bloom, E., Bhushan, I., Clingingsmith, D., Hong, R., King, E., Kremer, M., et al. (2006). Contracting for health: Evidence from Cambodia. Washington, DC: Brookings Institution, Mimeograph. Bloom, E., Canning, D., & Weston, M. (2005). The value of vaccination. World Economics, 6(3), 15–39. Casabonne, U., & Kenny, C. (2012). The best things in life are (nearly) free: Technology, knowledge, and global health. World Development, 40(1), 21–35. http://dx.doi.org/10.1016/j.worlddev.2011.05.009. Cristia, J., Evans, W., & Kim, B. (2011). Does contracting out primary care services work? The case of rural Guatemala. Inter-American development bank working paper series, 273. Washington, DC: InterAmerican Development Bank. Danel, I., & La Forgia, G. (2005). Contracting for basic health care in rural Guatemala. Comparison of the performance of three delivery models. In G. La Forgia (Ed.), Health system innovations in Central America: Lessons and impact of new approaches (pp. 49–88). Washington, DC: World Bank, . Duflo, E. (2004). Scaling up and evaluation. In F. Bourguignon, & B. Pleskovic (Eds.), Annual World Bank conference on development economics: Accelerating development (pp. 341–369). Washington, DC and New York, NY: World Bank and Oxford University Press. Ehreth, J. (2003). The global value of vaccination. Vaccine, 21(7–8), 596–600. http://dx.doi.org/10.1016/S0264-410X(02)00623-0. Goldman, N., & Glei, D. (2003). Evaluation of midwifery care: Results from a survey in rural Guatemala. Social Science & Medicine, 56(4), 685–700. http://dx.doi.org/10.1016/S0277-9536(02)00065-5. Inter-American Development Bank. (2012). Ana´lisis de redes sociales para comprender la toma de decisiones para la atencio´n materna en Guatemala. Mesoamerican Initiative, 2015. Washington, DC: InterAmerican Development Bank and Guatemalan Ministry of Health, Mimeograph. Gragnolati, M., & Marini, A. (2003). Health and poverty in Guatemala. Policy research working paper series, 2966. Washington, DC: World Bank. . Jones, G., Steketee, R., Black, R., Bhutta, Z., & Morris, S. (2003). How many child deaths can we prevent this year?. The Lancet, 362(9377), 65–71. http://dx.doi.org/10.1016/S0140-6736(03)13811-1. Kaufman, R., & Nelson, J. (Eds.) (2004). Crucial needs, weak incentives: Social sector reform, democratization and globalization in Latin America. Washington, DC: Woodrow Wilson Center Press. King, E., & Behrman, J. (2009). Timing and duration of exposure in evaluations of social programs. The World Bank Research Observer, 24(1), 55–82. http://dx.doi.org/10.1093/wbro/lkn009. Lavadenz, F., Schwad, N., & Straatman, H. (2001). Redes pu´blicas, descentralizadas y comunitarias de salud en Bolivia. Pan American

Journal of Public Health, 9(3), 182–189. http://dx.doi.org/10.1590/ S1020-49892001000300008. Lee, D. (2012). The impact of childhood health on adult educational attainment: Evidence from mandatory school vaccination laws. Working papers, 1202. Department of Economics, University of Missouri-Columbia. Leonard, D., Bloom, G., Hanson, K., O’Farrell, J., & Spicer, N. (2013). Institutional solutions to the asymmetric information problem in health and development services for the poor. World Development, 48, 71–87. http://dx.doi.org/10.1016/j.worlddev.2013.04.003. Liu, X., Hotchkiss, D., & Bose, S. (2008). The effectiveness of contractingout primary health care services in developing countries: A review of the evidence. Health Policy and Planning, 23(1), 1–13. http:// dx.doi.org/10.1093/heapol/czm042. Lloyd-Sherlock, P. (Ed.) (2000). Healthcare reform and poverty in Latin America. London: Institute of Latin American Studies and Brookings Institution. Loevinsohn, B., & Harding, A. (2005). Buying results? Contracting for health service delivery in developing countries. The Lancet, 366(9486), 676–681. http://dx.doi.org/10.1016/S0140-6736(05)67140-1. Mayer, D. (2001). The long-term impact of health on economic growth in Latin America. World Development, 29(6), 1025–1033. http:// dx.doi.org/10.1016/S0305-750X(01)00026-2. Ministerio de Salud de El Salvador, & The World Bank. (2006). Evaluacio´n de las proveedoras (ONGs) y equipos mo´viles institucionales de la extensio´n de cobertura de los servicios de salud. Proyecto de Reconstruccio´n de Hospitales y Extensio´n de Servicios de Salud PP145/2007 BIRF 7084/ES. Washington, DC. Ministerio de Salud Pu´blica y Asistencia Social de Guatemala. (2010). Encuesta nacional de salud materno-infantil 2008–2009. Guatemala. Palmer, N., Strong, L., Wali, A., & Sondorp, E. (2006). Contracting out health services in fragile states. British Medical Journal, 332(7543), 718–721. http://dx.doi.org/10.1136/bmj.332.7543.718. Pena, C. (2013). Improving access to health care services through the Expansion of Coverage Program (PEC): The case of Guatemala. UNICO studies series, 19. Washington, DC: World Bank. Sa´enz, M., Acosta, M., Muiser, J., & Bermu´dez, J. (2011). Sistema de Salud de Costa Rica. Salud Publica de Me´xico, 53(2), 156–167. SEGEPLAN. (2006). Estudio nacional de mortalidad maternal. Ministerio de Salud y Secretarı´a de Planificacio´n y Programacio´n de la Presidencia. Guatemala. Strauss, J., & Thomas, D. (1998). Health, nutrition and economic development. Journal of Economic Literature, 36(2), 766–817. World Bank. (2007). Key issues in Central America health sector reform: Diagnosis and strategic implications. Report No. 36426. Washington, DC: World Bank. World Bank. (2014). Data retrieved on December 1, 2014, from World Development Indicators Online (WDI) database. Zaidi, S., Mayhew, S., Cleland, J., & Green, A. (2012). Context matters in NGO-government contracting for health service delivery: A case study from Pakistan. Health Policy and Planning, 27(7), 570–581. http:// dx.doi.org/10.1093/heapol/czr081.

THE IMPACT OF CONTRACTING IN AND CONTRACTING OUT BASIC HEALTHSERVICES:

227

APPENDIX Table 8. Child immunization by year and coverage status Contracting out

Child had immunization card Immunization card was presented BCG – dose 1 DPT – dose 1 Polio – dose 1 DPT – dose 3 Polio – dose 3

Contracting in

Not covered

1995 (1)

2002 (2)

1995 (3)

2002 (4)

1995 (5)

2002 (6)

0.711 0.514 0.541 0.760 0.788 0.552 0.548

0.859 0.643 0.872 0.938 0.947 0.745 0.744

0.743 0.541 0.568 0.762 0.791 0.507 0.501

0.890 0.685 0.900 0.908 0.921 0.763 0.767

0.800 0.543 0.672 0.834 0.857 0.602 0.596

0.881 0.682 0.881 0.904 0.915 0.746 0.756

Notes: This table presents average child immunization rates in areas covered by the contracting-out model, the contracting-in model and in areas not covered by any contracting model. Summary statistics constructed from the 1995 and the 2002 National Surveys of Maternal and Child Health. The sample includes rural population. Observations from the departments of Pete´n and Guatemala were excluded. Columns 1–2 present the average immunization rates for children in areas with contracting-out models. Columns 3–4 report rates in areas covered by the contracting-in models. Columns 5– 6 indicate rates for children in areas not covered by any contracting model.

Table 9. Pre-natal care and family planning by year and coverage status Contracting out

Contracting in

Not covered

1995 (1)

2002 (2)

1995 (3)

2002 (4)

1995 (5)

2002 (6)

Panel A: mothers aged 15–44 with children younger than 2 years old Pre-natal care (PNC): visits and vaccine Any PNC 0.793 Number of PNC visits 3.938 Three or more PNC visits 0.694 PNC visit during first trimester 0.511 Tetanus vaccine 0.453 Three doses of tetanus vaccine 0.168

0.838 5.076 0.789 0.396 0.589 0.195

0.853 4.562 0.755 0.570 0.481 0.197

0.849 6.131 0.874 0.510 0.697 0.210

0.855 4.711 0.738 0.645 0.580 0.239

0.844 6.400 0.884 0.580 0.663 0.235

Pre-natal care (PNC): place of service PNC at traditional birth attendant’s home or own home

0.516

0.452

0.549

0.323

0.465

0.366

Panel B: women aged 15–44 years old Family planning: current use Any method Modern method

0.059 0.041

0.136 0.098

0.109 0.090

0.279 0.229

0.149 0.126

0.233 0.177

Notes: This table presents average statistics for women in areas with a contracting-in and a contracting-out model, as well as in areas not covered by any contracting model. Summary statistics constructed from the 1995 and the 2002 National Surveys of Maternal and Child Health. The sample includes rural population. Panel A presents health statistics for women with young children. Panel B reports family planning statistics for all women between the ages of 15 and 44. Observations from the departments of Pete´n and Guatemala were excluded. Columns 1–2 present statistics for women in areas with contracting-out models. Columns 3–4 show statistics for women in areas with contracting-in models. Columns 5–6 report statistics for women in areas not covered by any contracting model.

Available online at www.sciencedirect.com

ScienceDirect