- Email: [email protected]
Monitoring utilization and need for obstetric care in the highlands of Guatemala
International Journal of Gynecology and Obstetrics (2005) 89, 209 — 217
www.elsevier.com/locate/ijgo
AVERTING MATERNAL DEATH AND DISABILITY
Monitoring utilization and need for obstetric care in the highlands of Guatemala P. Baileya,T, E. de Bocalettib,1, G. Barriosc, Y. de Crossd a
Health Services Research, Family Health International, P.O. Box 13970, Research Triangle Park, NC, 27709, USA b MotherCare, Guatemala City, Guatemala c MotherCare, Quetzaltenango, Guatemala d MotherCare, Solola´, Guatemala Received 15 August 2004; received in revised form 11 November 2004; accepted 11 November 2004
Abstract
KEYWORDS Essential and emergency obstetric care; Maternity registry; Monitoring and evaluation; Guatemala
Objective: To monitor changes in the utilization and need for obstetric care between 1995 and 1998 at three hospitals in the highlands of Guatemala. Methods: We collected data on 5300 obstetric admissions from maternity registries and selected 10 indicators to measure intermediate outcomes. Results: Utilization of services increased at some or all hospitals as measured by the number of obstetric admissions, proportion of births in facilities, and cesarean deliveries as a proportion of all births. We observed increases in the proportion of women expected to have obstetric complications who were treated at each hospital and in the proportion of women with obstetric complications who were referrals. The changes in patient profile reflect increased service utilization and accessibility among women who traditionally used the health system the least. Conclusions: Positive changes in these indicators are likely to be associated with a reduction in maternal mortality. Without a control area, we cannot be sure of a
T Corresponding author. Fax: +1 919 544 7261. E-mail address: [email protected] (P. Bailey). 1 Currently with Save the Children in Guatemala. 0020-7292/$ - see front matter D 2005 International Federation of Gynecology and Obstetrics. Published by Elsevier Ireland Ltd. All rights reserved. doi:10.1016/j.ijgo.2004.11.038
210
P. Bailey et al. cause and effect relationship between outcomes and interventions, nevertheless, maternity registry data offer an inexpensive source of information for facilities to monitor changes. D 2005 International Federation of Gynecology and Obstetrics. Published by Elsevier Ireland Ltd. All rights reserved.
1. Introduction Since 1987 when the first major international meeting to discuss maternal mortality took place, considerable progress has been made in the search for and definition of indicators to measure programmatic or short-term advances towards reducing maternal mortality [1—4]. These process or output indicators refer to intermediate outcomes achieved at the program level, such as service utilization, and to a certain extent at the population level. Experts still debate to varying degrees the usefulness of the indicators described in this paper—number of hospital obstetric admissions, proportion of births in facilities that provide emergency obstetric care (EmOC), met need or the proportion of women estimated to have major obstetric complications who are treated in EmOC facilities, and the percentage of births delivered by cesarean [5]. The last three indicators have been used to assess program performance in Indonesia, Nepal and several African countries [6—8]. More recently they have been used extensively to measure baseline conditions prior to interventions to improve utilization and access to emergency obstetric care [9—12]. Nevertheless, few countries have produced pre- and post-intervention data [13]. Other indicators are more experimental such as percentage of hospital deliveries with complications referred din,T which sheds light on the functioning of the larger health system and of the community—hospital relationship. Also experimental are indicators that assess human rights. Non-discrimination of patients or their families is critical to prompt treatment and saving lives. Monitoring the patient profile of a particular health facility helps assess whether access to services is equitable or if certain subgroups of women are marginalized by the health system. This paper describes changes in the utilization of and need for hospital services in the Guatemalan highlands between 1995 and 1998, drawing on data from maternity registries in three district hospitals.
2. Intervention to reduce maternal mortality: the Ministry of Health and MotherCare project The Ministry of Health (MOH)/MotherCare project was designed to increase the number of women with major obstetric complications who are treated at facilities where EmOC2 is available. Two major strategies were used to meet this goal: ! improve the quality of services at hospitals, health centers and posts through in-service training in clinical and interpersonal skills and use of clinical protocols; ! increase access to and use of services through strategies focused on the community. The MOH/MotherCare project initially targeted four departments (Quetzaltenango, Solola ´, San Marcos and Totonicapa ´n) located in the southwestern region of the country with a population of approximately 1.6 million inhabitants. The majority of births in this region take place at home with the assistance of a traditional birth attendant (TBA). The 1995 and 1998 Demographic and Health Surveys (DHS) documented an increase in the proportion of births attended by a skilled attendant (physician or nurse) from 24% to 29% for this region [14,15]. The maternal mortality ratio for the region probably exceeds 200 maternal deaths per 100 000 live births [14,16]. To improve the quality of services, in-service clinical training of hospital, health center and health post staff targeted physicians, nurses and nurse auxiliaries. Protocols for the management of obstetric and neonatal emergencies were revised and played a critical role in the training of 206 hospital staff and 327 health center and health post staff. Hospitals worked to become more TBA- and family-friendly. Qualitative research was carried 2
The MotherCare project used and promoted the term bessential obstetric careQ or EOC. The term emergency obstetric care or EmOC has been used in this paper because that is what is meant. The term EOC is broader than EmOC and includes appropriate care of normal deliveries, antenatal care, care of normal and compromised newborns as well as treatment of obstetric emergencies.
Monitoring utilization and need for obstetric care out to explore why women did not come to hospitals and a major reason was the lack of dhumanizedT treatment. An institutional information, education and communication (IEC) strategy was implemented to train providers in interpersonal communication skills appropriate to the multiethnic Guatemalan context. Hospital staff identified and worked to eliminate barriers to TBA referrals and to the utilization of services by women. Some of the barriers within the hospitals were related to fear of hospital personnel, modesty, language, and certain hospital practices and procedures [17]. To increase access and utilization of services, the project also focused on interventions at the community level, which included the training of TBAs to detect women with complications and to refer these women to hospital. Groups of TBAs visited the local referral hospitals to become familiar with staff and the hospital itself. These visits were seen as one way of strengthening the relationship between these community members and hospital staff. IEC strategies emphasized the danger signs of pregnancy and what to do should one occur. Radio spots featured these same themes and women’s groups met to discuss them. More than 60 women’s groups brought together over 2000 women to talk about pregnancy and the management of complications. These interventions are discussed in greater detail elsewhere [17—19].
3. Materials and methods 3.1. Indicators The 10 indicators reflect both program level outputs such as number of obstetric admissions and population-based outcomes such as percentage of births in an EmOC facility. The maternity registry or monitoring system provides the numerator data and sometimes the information for the denominator as well. For the population-based measurements, we used projections from the 1994 Guatemala Census, adjusting by 2.6% per year, the factor recommended by the census. To estimate the expected number of live births, we multiplied the population by the crude birth rate (CBR) using the same CBR for both time periods (based on 1995 DHS estimates). The number of obstetric admissions refers to the weekly average number admitted (excluding abortions because of incomplete data). The proportion of births in an EmOC facility or bcoverageQ is
211 population-based and its numerator is the number of institutional births and the denominator the estimate of all expected births in the geographic area [3]. Since we collected data at only three hospitals, we actually measure the contribution of each hospital to this indicator. Met need is defined as the proportion of women estimated to have major obstetric complications who are treated at facilities that provide EmOC [3]. The numerator is the number of women with major complications who present and are treated at hospitals (or who develop them while in hospital) and the denominator is an estimate of how many women at the population level are expected to have major complications. The proportion of direct obstetric complications is generally estimated to be 15% of all live births. Ideally, if all women requiring EmOC receive it, met need would attain a level of 100% (greater than 100% is also possible). Met need reflects the use of services by women who are the most in need of services. The numerator consists of women with major obstetric complications including hemorrhage (hemorrhage during pregnancy, delivery or postpartum, placenta previa, retained placenta and ectopic pregnancy); dystocia (prolonged labor, cephalo-pelvic disproportion, uterine rupture and malpresentation); sepsis; hypertensive disease of pregnancy (eclampsia and pre-eclampsia); and multiple gestations [4]. The complications excluded were: premature rupture of membranes, preterm or false labor, previous cesarean section(s), sexually transmitted diseases, as well as those classified as botherQ. We calculated met need with and without abortions in the numerator since the number of abortions treated varies with the skill of local providers and the prevalence of induced abortion. Pregnancy termination is illegal in Guatemala and no attempt was made in the registries to distinguish between an incomplete abortion that was not life threatening at the time of admission and an abortion that was. The proportion of all births delivered by cesarean reflects utilization of a life-saving procedure and is defined by the number of cesarean deliveries divided by the number of live births in a specific geographic area and in a given time period. The UN Guidelines define a range from 5% to 15% as acceptable [3]. As indicators of both utilization and need, we looked at the proportion of obstetric admissions who reported that they were referred from a lower level facility or by someone in the community, and the proportion of women with complications who reported they were referred.
212
P. Bailey et al.
As health systems implement interventions to increase utilization especially among vulnerable population groups, facilities should monitor characteristics of their users to assess the extent to which they serve these groups. We chose to look for changes in several patient characteristics that suggested potential vulnerability: ethnicity, education level, distance between residence and hospital, and previous birth in a hospital.
Table 1 Numbers of obstetric admissions, abortionrelated admissions, and weekly average, by hospital and time period
372 529
53 93
3.2. Monitoring system
San Marcos 1995—1996 975 1998—1999 1053
Quetzaltenango 1995—1996 1009 1998—1999 1362
The monitoring system was also a component of the intervention package implemented by the MOH and MotherCare project at the four targeted departmental hospitals and was developed as a monitoring and evaluation tool. It was designed in collaboration with the hospitals’ obstetric staff, many of whom later implemented the system, and whose support for the system facilitated its establishment. The monitoring system used the format of a preprinted logbook or maternity registry for all women admitted for reasons related to pregnancy. The form measured 29 items that cover sociodemographic characteristics, previous use of reproductive health services, prenatal care during the index pregnancy, whether she was referred to the hospital and by whom, maternal and newborn complications, surgical intervention, medical treatment, date and hour of admission, date and hour of intervention (if there was one), attendant at birth, survival status of mother and infant, and several additional items, including date of discharge. Both nurses and doctors completed the registry. Most of the cases logged were obstetric: women with complications during pregnancy, women admitted for delivery, or admitted postpartum with complications. Some hospitals also included abortion-related cases: women with threatened or incomplete abortions. Quetzaltenango maintained a set of logbooks exclusively for their abortionrelated cases. At baseline they were included, at least partially, but at follow-up they were not. For this study, we analyzed data from two time periods (Table 1) from three of the four comprehensive EmOC hospitals where the monitoring system was originally implemented.3 The 1995— 1996 data were collected before implementation of
Obstetric Abortion- No. of admissions related weeks data excluding admissions collected abortions Solola´ 1995—1996 1998—1999
Weekly average of obsteric admissions
28.5 23
13 23 p b 0.05T
196 213
30 22
32 48 p b 0.05
127TT 9TTT
21 19
48 72 p b 0.05
T Based on calculations set out by Lentner [20]. TT Incomplete abortion registration. TTT Abortions registered in a different set of logbooks.
the package of interventions and reflected the first months of experience completing the maternity registry. We analyzed consecutive records admitted during periods of 19 to 30 weeks. Hospitals with a lower volume of admissions collected data for a longer period of time. Both data collection periods occurred at roughly the same time of year (September through January).4 The unit of analysis is a case and not a woman since in each data collection period, a number of women presented more than once during their pregnancy, delivery or postpartum period. For example, in 1998 in Solola ´, information was collected from 529 obstetric cases, 23 of whom were 10 women admitted twice and one woman admitted three times for different reasons. However, as the number of repeated admissions is small, the terms dcaseT and dwomanT are used interchangeably. The data were originally entered in Epi-Info and later converted to SPSS and SAS files for analysis. Statistical significance was set at the 0.05 level.
4. Results 4.1. Number of obstetric admissions The weekly number of obstetric admissions increased significantly at all hospitals. Solola´
3 Although the intervention targeted four hospitals in four departments, at one hospital (Totonicapa ´n) staff collected only baseline data and thus this hospital was excluded from the analysis.
4 Due to limited resources for data processing and management, only these short periods of data collection were analyzed.
Monitoring utilization and need for obstetric care Table 2
213
Proportion of births in hospital, by hospital and time period
Department
1995—1996
1998—1999
Percent increase
p-value
Solola´ Departmental population Estimated live births/year Estimated live births in study period No. of hospital births in study period Proportion of births in study hospital
227 868 8955 4908 351 7.2%
246 109 9672 4278 465 10.9%
52%
p = 0.001
San Marcos Departmental population Estimated live births/year Estimated live births in study period No. of hospital births in study period Proportion of births in study hospital
662 199 25 097 14 479 921 6.4%
715 205 27 106 11 468 997 8.7%
36%
p = 0.001
Quetzaltenango Departmental population Estimated live births/year Estimated live births in study period No. of hospital births in study period Proportion of births in study hospital
516 957 16 749 6764 989 14.6%
558 337 18 090 6610 1322 20.0%
37%
p = 0.001
experienced the largest increase in admissions (77% increase) and San Marcos and Quetzaltenango increased their numbers by half (Table 1).
4.2. Proportion of births in EmOC facilities or coverage We adjusted the weekly average of obstetric admissions shown in Table 1 down since these averages included a small number of women who were admitted, but for whom no delivery occurred, such as women admitted with premature labor but were discharged before delivery, or where the outcome of the newborn was left blank. Thus, we believe our estimate of the number of hospital births to be conservative. The proportion of births in hospital increased significantly in each department, ranging from increases of 36% to 52% (Table 2). In the depart-
Table 3
ment of Solola ´, the MOH has only one hospital and despite a 50% increase, coverage increased from 7% to 11%. Coverage by the hospital in San Marcos increased from 6 to 9%, and at the hospital in Quetzaltenango from 15 to 20%. According to the UN Guidelines, at least 15% of all births should take place in EmOC facilities since at least that many are thought to develop severe complications [3].
4.3. Met need The proportions of obstetric admissions with major complications (excluding abortion-related complications) varied across departments: 30% in Solola ´, 14% in San Marcos and 11% in Quetzaltenango (data not shown). Distinct case-mix patterns and accessibility to alternative facilities may explain the variations. Both San Marcos and Quetzalte-
Met need by hospital and time period
Hospital
1995—1996
1998—1999
Percent increase
p-value
Solola´ With abortions Without abortions
147/736 = 20.0% 110/736 = 15.0%
252/642 = 39.3% 161/642 = 25.1%
96% 67%
p = 0.001 p = 0.001
San Marcos With abortions Without abortions
335/2172 = 15.4% 139/2172 = 6.4%
355/1720 = 20.6% 152/1720 = 8.8%
34% 38%
p = 0.001 p = 0.001
Quetzaltenango With abortions Without abortions
202/1014 = 19.9% 78/1014 = 7.7%
na 149/992 = 15.0%
95%
p = 0.001
na = not available.
214 Table 4
P. Bailey et al. Referral by hospital and time period
%Obstetric admissions reporting referral Solola ´ San Marcos Quetzaltenango %Complicated cases reporting referral Solola ´ San Marcos Quetzaltenango
1995—1996 (n)
1998—1999 (n)
p-value
56.8 (310) 34.3 (775) 9.2 (934)
52.6 (517) 47.7 (1049) 15.5 (1116)
p = 0.675 p b 0.000 p b 0.000
56.2 (137) 34.0 (212) 11.9 (118)
56.0 (300) 47.2 (352) 21.1 (218)
p = 1.000 p = 0.002 p = 0.037
Missing data on the variable dreferralT accounts for the differences in denominators in row 1 (obstetric admissions) of this table and Table 1.
nango have a second MOH EmOC facility to meet the needs of the departments’ pregnant women; Solola ´ does not. Met need was consistently higher when abortions were included; the largest increases occurred in Quetzaltenango and Solola ´ (Table 3).
4.4. Proportion of all births delivered by cesarean The contribution that each hospital made to the proportion of births delivered by cesarean increased in each department, but only significantly in Solola ´ and Quetzaltenango (2.4% to 4.2% and 2.4% to 2.9%, respectively). San Marcos recorded the lowest cesarean rate (1.9% in 1998— 1999). All rates are below the 5% minimum that the UN recommends.
4.5. Proportion of obstetric admissions and proportion of complicated cases reporting referral San Marcos and Quetzaltenango witnessed significant increases in the number of women who reported that they had been referred and in the number of complicated cases reporting referral (Table 4) 5 . In Solola´ , more than half of the admissions reported that they had been referred as did women with obstetric complications. About half of the women said TBAs referred them while others were referred by lower level health facilities, by private practitioners or community members.
4.6. Profile of hospital obstetric admissions The population profile changed at all of the hospitals, but not all hospitals experienced the same changes (Table 5). Solola ´ and San Marcos 5 For this analysis, we considered all obstetric complications, not just those used to calculate met need.
served an increasingly marginalized population while Quetzaltenango appears to have provided services to an increasingly urban and educated population. Solola ´ and San Marcos experienced increases in the proportion of admissions of indigenous women, women who had never had a hospital birth (and yet were multiparous), and women with no formal schooling. In the case of San Marcos, the percentage of obstetric patients who lived 10 or more kilometers from the hospital increased from 31% to 45%. In Quetzaltenango, on the other hand, only the proportion of women who had never delivered in hospital increased. Table 5 Selected characteristics of hospital obstetric population, by hospital and time period Characteristic
Solola ´
San Marcos
Quetzaltenango
83.7 (368) 88.3 (523) p = 0.046
40.4 (900) 58.7 (1048) p b 0.000
62.7 (993) 58.9 (1351) p = 0.062
%No prior hospital birth* 1995 14.9 (368) 1998 34.1 (519) p b 0.000
22.8 (959) 28.1 (1046) p = 0.009
19.4 (989) 39.6 (1346) p b 0.000
%No formal schooling* 1995 25.9 (278) 1998 38.9 (527) p = 0.001
13.8 (848) 19.8 (1052) p = 0.001
29.2 (998) 18.6 (1362) p b 0.000
%Lives N 10 km* 1995 50.8 (372) 1998 48.8 (529) p = 0.691
30.8 (975) 45.1 (1053) p b 0.000
37.7 (1009) 24.6 (1362) p b 0.000
%Indigenous 1995 1998
Observation: The p-values refer to 1995 and 1998 distributions. Women with abortion-related complications were excluded. The asterisked variables are categorical variables with three mutually exclusive categories. In the case of the variable hospital birth, the categories were dfirst birth,T dprior hospital birth,T and dno prior hospital birthT. The education categories were dno formal schooling,T d1—6 years of schooling,T and dmore than 6 years of schoolingT. The categories for the variable distance from the hospital were dsame cityT, dlives b 10 km from the hospitalT and dlivesz 10 km from the hospitalT.
Monitoring utilization and need for obstetric care
5. Discussion 5.1. Limitations of data The monitoring system proved to be a successful tool in three of the four hospitals where it was implemented. In the fourth hospital, staff claimed they were understaffed making registry completion difficult, particularly on weekends. Improvements in the system were not difficult to achieve. With respect to data quality, several issues had to be addressed. Missing information was observed for specific variables, suggesting that the information was never obtained from the woman or that the blanks meant dnoT or dnot applicable.T Additional supervision and training of the staff completing the form resulted in more complete reporting in 1998 than in 1995. The registry data were purged of double entries for the same woman admitted for the same event, but the number of cases never registered is not known. Some speculate that the cases most likely to be omitted might be emergencies, which clearly would affect our findings if a considerable number of such cases were not included. Although no systematic effort was carried out by the project to determine the completeness of the registry of obstetric cases, no anecdotal evidence suggested serious underreporting. The reporting of complications can suffer from poor reliability and varying definitions. Efforts were made to standardize the classification of complications within hospital and across hospitals. How much clinical refinement or severity assessment might be achieved with a maternity registry will depend on the level of training and supportive supervision. However, if cases are used for trend monitoring and classification does not undergo major changes, this internal exercise can be a powerful and inexpensive tool for monitoring and process evaluation. An independent investigation of the quality of information collected in maternity registries of this region was carried out in 2000 after the closure of the MotherCare project. Only one institution overlapped—the hospital in San Marcos. The researchers found that the data in maternity registries were bhighly congruent with data in corresponding medical records.Q When maternity registries were compared with surgical registries, however, 20% of the cesareans found in the latter were missing in the former. This indicates the need to have mechanisms in place to ensure completeness and quality of the data by cross checking registries for certain items [21].
215 The strength of the indicators in this analysis lies in what they reveal about the individual hospitals over time and their use by hospital administrators and technical staff for planning, monitoring performance and even outreach. The potential for using facility-based data to provide populationbased measures is promising, if all facilities collect the same data in the same way. In this case, although not all EmOC facilities in the three departments used the maternity registry, the authors made an effort to estimate departmentwide values. They obtained service statistics on births and cesareans from the two hospitals not using the registry and from several but not all private facilities [22]. With this additional information, we estimated the proportion of births in EmOC facilities and the cesarean delivery rate at the departmental level. We estimated the proportion of births in EmOC facilities in 1998 to be 14% in the department of Solola ´, 19% in San Marcos and 55% in Quetzaltenango. Cesarean rates ranged from 3.5% in San Marcos, 5.6% in Solola ´ and 6.4% in Quetzaltenango. This estimate of the proportion of births falls within the confidence intervals of the 1998 DHS point estimate, reinforcing our confidence in the data quality. The 1998 DHS reported an increase in the population-based cesarean rate from 5.6% to 7.3% in this southwest region [14,15], slightly higher than our projections of between 3.5% and 6.4%.
6. Conclusions This paper illustrates how data collected over time from a hospital maternity registry can be used to monitor changes in the utilization of EmOC services as well as changes in who accesses these services. Although utilization increased at all three hospitals during this 3-year period of MOH and MotherCare interventions, a wide variation in the utilization of services still can be observed across these hospitals with hospital births consistently lower in San Marcos and Solola ´. Met need at each of the targeted hospitals also increased but continues low. Rates of met need tend to be considerably higher when women with abortion-related complications are included. These data confirm what other researchers have found when abortion complications are added to the estimate of met need [7,11,12]. The referral estimates suggest that the hospital in Solola ´ plays an important role as a referral center for its rural catchment area. At the other extreme was the project hospital in Quetzalte-
216 nango with the smallest proportion of referrals and the most urban catchment area but the percentage of complicated cases who said they were referred did almost double. The referral results and the changes in population profile highlight the different roles that the three hospitals play in their respective communities. Particularly encouraging from the perspective of human rights is the observation that more equitable access to services has been achieved. The overall picture of this region is one in transition from more community-based maternity care to where, increasingly, deliveries take place in EmOC facilities. This is a critical prerequisite to reducing maternal mortality as it combines the potential of a skilled attendant in an enabling environment. The relationship between the results presented here and the effectiveness of the MOH and MotherCare interventions is open to challenge. Without a control group for comparison, we cannot rule out the possibility that similar changes occurred in other regions, or that external political factors such as the signing of the 1996 Peace Accord made accessing services seem safer for women. The gold standard for attributing cause to impact in health care evaluations is the randomized clinical trial (RCT) but three characteristics make the RCT difficult for evaluating most dsafe motherhoodT programs. First, it would be unethical to withhold known life-saving procedures from any population. Second, these programs focus on communities rather than individuals, and third, like this program, they are multifaceted interventions, notoriously difficult to evaluate because more than one of the interventions can affect a single outcome [23]. Quasi-experimental designs are superior to before and after designs as this paper describes. An earlier experience with a quasi-experimental design with the MOH and MotherCare resulted in high costs and frustration with bcontrolQ areas that were difficult to maintain in the face of interventions by non-governmental organizations [24]. Nevertheless, progress was made towards the goal of increasing the number of women with obstetric complications who were attended at facilities where comprehensive EmOC is available. Project in-service training of hospital staff and use of clinical protocols may have translated into better diagnosis and treatment of complications, resulting in a more favorable image of the hospital. Improved relations between communities and hospital staff may have resulted in the increase in referrals (or compliance with referrals) and in the number of women who traditionally did not seek hospital services. Other likely contributors to the
P. Bailey et al. increase in utilization were the community IEC efforts that Pereira et al. found associated with an increase in awareness of bdangerQ signs and signals of obstetric complications in the same study area [18]. The challenge now is to maintain the momentum, if not accelerate it, to increase access and utilization of EmOC.
Acknowledgments The authors would like to thank the staff at the hospitals that participated in the implementation of the birth registry: the Regional Hospital San Juan de Dios in Quetzaltenango, the Departmental Hospital Moises Villagran in San Marcos and the Departmental Hospital of Solola ´. This study was made possible by support from MotherCare/John Snow, Inc. (Contract HRN-Q-00-00039-00), the Maternal and Neonatal Health Project/JHPIEGO (Contract HRN-A-00-98-00043-00) and the Guatemalan Mission for the United States Agency for International Development. The opinions expressed in this paper are those of the authors and do not necessarily reflect the views of the US Agency for International Development, John Snow, Inc. or JHPIEGO.
References [1] Maine D, McCarthy J, Ward VM. Guidelines for monitoring progress in the reduction of maternal mortality: a work in progress. New York7 UNICEF; 1992. [2] Bertrand J, Escudero G. Compendium of indicators for evaluating reproductive health programs. Volume One. Measure Evaluation Manual, No. 6. (2002). [3] UNICEF, WHO, UNFPA. Guidelines for monitoring the availability and use of obstetric services. New York7 UNICEF; 1997. [4] MotherCare Matters VM. Safe motherhood indicators— lessons learned in measuring progress; 1999. p. 8. [5] Ronsmans C, Campbell O, McDermott J, Koblinsky M. Questioning the indicators of need for obstetric care. Bull WHO 2002;80:317 – 24. [6] Ronsmans C, Achadi E, Sutratikto G, Zazri A, McDermott J. Use of hospital data for Safe Motherhood programmes in South Kalimantan, Indonesia. Trop Med Int Health 1999;4: 514 – 21. [7] Pathak LR, Kwast BE, Malla DS, Pradhan AS, Rajlawat R, Campbell BB. Process indicators for safe motherhood programmes: their application and implications as derived from hospital data in Nepal. Trop Med Int Health 2000; 5(12):882 – 90. [8] McGinn T. Monitoring and evaluation of PMM efforts: what have we learned? Int J Gynecol Obstet 1997;59:S245 – 51 [Suppl.]. [9] Bailey P, Paxton A. Program note: Using UN process indicators to assess needs in emergency obstetric services: Mozambique, Nepal and Senegal. Int J Gynecol Obstet 2002;76:299 – 305.
Monitoring utilization and need for obstetric care [10] AMDD Working Group on Indicators. Program note: using UN process indicators to assess needs in emergency obstetric services in Bhutan, Cameroon and Rajasthan, India. Int J Gynecol Obstet 2002;77:277 – 84. [11] AMDD Working Group on Indicators. Program note: using UN process indicators to assess needs in emergency obstetric services in Pakistan, Peru and Vietnam. Int J Gynecol Obstet 2002;78:275 – 82. [12] AMDD Working Group on Indicators. Program note: Using UN process indicators to assess needs in emergency obstetric services in Morocco, Nicaragua and Sri Lanka. Int J Gynecol Obstet 2005;89:221 – 30. [13] Hussein J, Goodburn EA, Damisoni H, Lema V, Graham W. Monitoring obstetric services: putting the UN guidelines into practice in Malawi: 3 years on. Int J Gynecol Obstet 2001;75:63 – 73. [14] INE (Instituto Nacional de Estadı´stica). Guatemala Encuesta Nacional de Salud Materno Infantil, 1995. DHS/Macro International Inc; 1996. [15] INE (Instituto Nacional de Estadı´stica). Guatemala Encuesta Nacional de Salud Materno Infantil, 1998—1999. 1999. DHS/ Macro International Inc. [16] Ministry of Public Health and Social Assistance. Study of maternal mortality in Guatemala 1989. Director General of Health Services, Department of Maternal and Child Health; 1989. [17] Hurtado E. The MotherCare/Guatemala Project Overview of IEC Strategies and Lessons Learned, Guatemala City; 1999. 14 p.
217 [18] Perreira K, Bailey P, Bocaletti E, Hurtado E, Recinos S, Matute J. Improving recognition of obstetric complications through community- and clinic-based education. Matern Child Health J 2002;6(1):19 – 28. [19] Molina S. MotherCare II Guatemala: Informe de 5 An ˜os 1994—1999. Guatemala City; 1999. 113 p. [20] Lentner C, editor. Geigy scientific tables, 8th edition. Introduction to statistics, statistical tables, mathematical formulae, compiled by K Diem and J Seldrup, vol. 2. CIBAGEIGY; 1982. p. 224 – 5. [21] Valladares R, Barillas E, Bassett Hileman S, Fonseca-Becker M, Rose M, Buekens P. Maternity registers in Guatemalan facilities: a potential source of data for monitoring obstetric care. Strengthening monitoring and evaluation of maternal health programs MEASURE evaluation bulletin, vol. 7; 2003. [22] Cross Y, Rose A. 2000. Personal communication. [23] Campbell O, Filippi V, Koblinsky M, Marshall T, Mortimer J, Pittrof R, Ronsmans C, Williams. Lessons Learnt: a decade of measuring the impact of safe motherhood programmes. DFID Research Work Programme on Population and Reproductive Health Maternal and Child Epidemiology Unit, London School of Hygiene and Tropical Medicine. 25 pages. [24] Bailey P, Sza ´szdi JA, Glover L. Obstetric complications: does training traditional birth attendants make a difference? Pan Am J Public Health 2002;11(1):15 – 22.
www.elsevier.com/locate/ijgo
AVERTING MATERNAL DEATH AND DISABILITY
Monitoring utilization and need for obstetric care in the highlands of Guatemala P. Baileya,T, E. de Bocalettib,1, G. Barriosc, Y. de Crossd a
Health Services Research, Family Health International, P.O. Box 13970, Research Triangle Park, NC, 27709, USA b MotherCare, Guatemala City, Guatemala c MotherCare, Quetzaltenango, Guatemala d MotherCare, Solola´, Guatemala Received 15 August 2004; received in revised form 11 November 2004; accepted 11 November 2004
Abstract
KEYWORDS Essential and emergency obstetric care; Maternity registry; Monitoring and evaluation; Guatemala
Objective: To monitor changes in the utilization and need for obstetric care between 1995 and 1998 at three hospitals in the highlands of Guatemala. Methods: We collected data on 5300 obstetric admissions from maternity registries and selected 10 indicators to measure intermediate outcomes. Results: Utilization of services increased at some or all hospitals as measured by the number of obstetric admissions, proportion of births in facilities, and cesarean deliveries as a proportion of all births. We observed increases in the proportion of women expected to have obstetric complications who were treated at each hospital and in the proportion of women with obstetric complications who were referrals. The changes in patient profile reflect increased service utilization and accessibility among women who traditionally used the health system the least. Conclusions: Positive changes in these indicators are likely to be associated with a reduction in maternal mortality. Without a control area, we cannot be sure of a
T Corresponding author. Fax: +1 919 544 7261. E-mail address: [email protected] (P. Bailey). 1 Currently with Save the Children in Guatemala. 0020-7292/$ - see front matter D 2005 International Federation of Gynecology and Obstetrics. Published by Elsevier Ireland Ltd. All rights reserved. doi:10.1016/j.ijgo.2004.11.038
210
P. Bailey et al. cause and effect relationship between outcomes and interventions, nevertheless, maternity registry data offer an inexpensive source of information for facilities to monitor changes. D 2005 International Federation of Gynecology and Obstetrics. Published by Elsevier Ireland Ltd. All rights reserved.
1. Introduction Since 1987 when the first major international meeting to discuss maternal mortality took place, considerable progress has been made in the search for and definition of indicators to measure programmatic or short-term advances towards reducing maternal mortality [1—4]. These process or output indicators refer to intermediate outcomes achieved at the program level, such as service utilization, and to a certain extent at the population level. Experts still debate to varying degrees the usefulness of the indicators described in this paper—number of hospital obstetric admissions, proportion of births in facilities that provide emergency obstetric care (EmOC), met need or the proportion of women estimated to have major obstetric complications who are treated in EmOC facilities, and the percentage of births delivered by cesarean [5]. The last three indicators have been used to assess program performance in Indonesia, Nepal and several African countries [6—8]. More recently they have been used extensively to measure baseline conditions prior to interventions to improve utilization and access to emergency obstetric care [9—12]. Nevertheless, few countries have produced pre- and post-intervention data [13]. Other indicators are more experimental such as percentage of hospital deliveries with complications referred din,T which sheds light on the functioning of the larger health system and of the community—hospital relationship. Also experimental are indicators that assess human rights. Non-discrimination of patients or their families is critical to prompt treatment and saving lives. Monitoring the patient profile of a particular health facility helps assess whether access to services is equitable or if certain subgroups of women are marginalized by the health system. This paper describes changes in the utilization of and need for hospital services in the Guatemalan highlands between 1995 and 1998, drawing on data from maternity registries in three district hospitals.
2. Intervention to reduce maternal mortality: the Ministry of Health and MotherCare project The Ministry of Health (MOH)/MotherCare project was designed to increase the number of women with major obstetric complications who are treated at facilities where EmOC2 is available. Two major strategies were used to meet this goal: ! improve the quality of services at hospitals, health centers and posts through in-service training in clinical and interpersonal skills and use of clinical protocols; ! increase access to and use of services through strategies focused on the community. The MOH/MotherCare project initially targeted four departments (Quetzaltenango, Solola ´, San Marcos and Totonicapa ´n) located in the southwestern region of the country with a population of approximately 1.6 million inhabitants. The majority of births in this region take place at home with the assistance of a traditional birth attendant (TBA). The 1995 and 1998 Demographic and Health Surveys (DHS) documented an increase in the proportion of births attended by a skilled attendant (physician or nurse) from 24% to 29% for this region [14,15]. The maternal mortality ratio for the region probably exceeds 200 maternal deaths per 100 000 live births [14,16]. To improve the quality of services, in-service clinical training of hospital, health center and health post staff targeted physicians, nurses and nurse auxiliaries. Protocols for the management of obstetric and neonatal emergencies were revised and played a critical role in the training of 206 hospital staff and 327 health center and health post staff. Hospitals worked to become more TBA- and family-friendly. Qualitative research was carried 2
The MotherCare project used and promoted the term bessential obstetric careQ or EOC. The term emergency obstetric care or EmOC has been used in this paper because that is what is meant. The term EOC is broader than EmOC and includes appropriate care of normal deliveries, antenatal care, care of normal and compromised newborns as well as treatment of obstetric emergencies.
Monitoring utilization and need for obstetric care out to explore why women did not come to hospitals and a major reason was the lack of dhumanizedT treatment. An institutional information, education and communication (IEC) strategy was implemented to train providers in interpersonal communication skills appropriate to the multiethnic Guatemalan context. Hospital staff identified and worked to eliminate barriers to TBA referrals and to the utilization of services by women. Some of the barriers within the hospitals were related to fear of hospital personnel, modesty, language, and certain hospital practices and procedures [17]. To increase access and utilization of services, the project also focused on interventions at the community level, which included the training of TBAs to detect women with complications and to refer these women to hospital. Groups of TBAs visited the local referral hospitals to become familiar with staff and the hospital itself. These visits were seen as one way of strengthening the relationship between these community members and hospital staff. IEC strategies emphasized the danger signs of pregnancy and what to do should one occur. Radio spots featured these same themes and women’s groups met to discuss them. More than 60 women’s groups brought together over 2000 women to talk about pregnancy and the management of complications. These interventions are discussed in greater detail elsewhere [17—19].
3. Materials and methods 3.1. Indicators The 10 indicators reflect both program level outputs such as number of obstetric admissions and population-based outcomes such as percentage of births in an EmOC facility. The maternity registry or monitoring system provides the numerator data and sometimes the information for the denominator as well. For the population-based measurements, we used projections from the 1994 Guatemala Census, adjusting by 2.6% per year, the factor recommended by the census. To estimate the expected number of live births, we multiplied the population by the crude birth rate (CBR) using the same CBR for both time periods (based on 1995 DHS estimates). The number of obstetric admissions refers to the weekly average number admitted (excluding abortions because of incomplete data). The proportion of births in an EmOC facility or bcoverageQ is
211 population-based and its numerator is the number of institutional births and the denominator the estimate of all expected births in the geographic area [3]. Since we collected data at only three hospitals, we actually measure the contribution of each hospital to this indicator. Met need is defined as the proportion of women estimated to have major obstetric complications who are treated at facilities that provide EmOC [3]. The numerator is the number of women with major complications who present and are treated at hospitals (or who develop them while in hospital) and the denominator is an estimate of how many women at the population level are expected to have major complications. The proportion of direct obstetric complications is generally estimated to be 15% of all live births. Ideally, if all women requiring EmOC receive it, met need would attain a level of 100% (greater than 100% is also possible). Met need reflects the use of services by women who are the most in need of services. The numerator consists of women with major obstetric complications including hemorrhage (hemorrhage during pregnancy, delivery or postpartum, placenta previa, retained placenta and ectopic pregnancy); dystocia (prolonged labor, cephalo-pelvic disproportion, uterine rupture and malpresentation); sepsis; hypertensive disease of pregnancy (eclampsia and pre-eclampsia); and multiple gestations [4]. The complications excluded were: premature rupture of membranes, preterm or false labor, previous cesarean section(s), sexually transmitted diseases, as well as those classified as botherQ. We calculated met need with and without abortions in the numerator since the number of abortions treated varies with the skill of local providers and the prevalence of induced abortion. Pregnancy termination is illegal in Guatemala and no attempt was made in the registries to distinguish between an incomplete abortion that was not life threatening at the time of admission and an abortion that was. The proportion of all births delivered by cesarean reflects utilization of a life-saving procedure and is defined by the number of cesarean deliveries divided by the number of live births in a specific geographic area and in a given time period. The UN Guidelines define a range from 5% to 15% as acceptable [3]. As indicators of both utilization and need, we looked at the proportion of obstetric admissions who reported that they were referred from a lower level facility or by someone in the community, and the proportion of women with complications who reported they were referred.
212
P. Bailey et al.
As health systems implement interventions to increase utilization especially among vulnerable population groups, facilities should monitor characteristics of their users to assess the extent to which they serve these groups. We chose to look for changes in several patient characteristics that suggested potential vulnerability: ethnicity, education level, distance between residence and hospital, and previous birth in a hospital.
Table 1 Numbers of obstetric admissions, abortionrelated admissions, and weekly average, by hospital and time period
372 529
53 93
3.2. Monitoring system
San Marcos 1995—1996 975 1998—1999 1053
Quetzaltenango 1995—1996 1009 1998—1999 1362
The monitoring system was also a component of the intervention package implemented by the MOH and MotherCare project at the four targeted departmental hospitals and was developed as a monitoring and evaluation tool. It was designed in collaboration with the hospitals’ obstetric staff, many of whom later implemented the system, and whose support for the system facilitated its establishment. The monitoring system used the format of a preprinted logbook or maternity registry for all women admitted for reasons related to pregnancy. The form measured 29 items that cover sociodemographic characteristics, previous use of reproductive health services, prenatal care during the index pregnancy, whether she was referred to the hospital and by whom, maternal and newborn complications, surgical intervention, medical treatment, date and hour of admission, date and hour of intervention (if there was one), attendant at birth, survival status of mother and infant, and several additional items, including date of discharge. Both nurses and doctors completed the registry. Most of the cases logged were obstetric: women with complications during pregnancy, women admitted for delivery, or admitted postpartum with complications. Some hospitals also included abortion-related cases: women with threatened or incomplete abortions. Quetzaltenango maintained a set of logbooks exclusively for their abortionrelated cases. At baseline they were included, at least partially, but at follow-up they were not. For this study, we analyzed data from two time periods (Table 1) from three of the four comprehensive EmOC hospitals where the monitoring system was originally implemented.3 The 1995— 1996 data were collected before implementation of
Obstetric Abortion- No. of admissions related weeks data excluding admissions collected abortions Solola´ 1995—1996 1998—1999
Weekly average of obsteric admissions
28.5 23
13 23 p b 0.05T
196 213
30 22
32 48 p b 0.05
127TT 9TTT
21 19
48 72 p b 0.05
T Based on calculations set out by Lentner [20]. TT Incomplete abortion registration. TTT Abortions registered in a different set of logbooks.
the package of interventions and reflected the first months of experience completing the maternity registry. We analyzed consecutive records admitted during periods of 19 to 30 weeks. Hospitals with a lower volume of admissions collected data for a longer period of time. Both data collection periods occurred at roughly the same time of year (September through January).4 The unit of analysis is a case and not a woman since in each data collection period, a number of women presented more than once during their pregnancy, delivery or postpartum period. For example, in 1998 in Solola ´, information was collected from 529 obstetric cases, 23 of whom were 10 women admitted twice and one woman admitted three times for different reasons. However, as the number of repeated admissions is small, the terms dcaseT and dwomanT are used interchangeably. The data were originally entered in Epi-Info and later converted to SPSS and SAS files for analysis. Statistical significance was set at the 0.05 level.
4. Results 4.1. Number of obstetric admissions The weekly number of obstetric admissions increased significantly at all hospitals. Solola´
3 Although the intervention targeted four hospitals in four departments, at one hospital (Totonicapa ´n) staff collected only baseline data and thus this hospital was excluded from the analysis.
4 Due to limited resources for data processing and management, only these short periods of data collection were analyzed.
Monitoring utilization and need for obstetric care Table 2
213
Proportion of births in hospital, by hospital and time period
Department
1995—1996
1998—1999
Percent increase
p-value
Solola´ Departmental population Estimated live births/year Estimated live births in study period No. of hospital births in study period Proportion of births in study hospital
227 868 8955 4908 351 7.2%
246 109 9672 4278 465 10.9%
52%
p = 0.001
San Marcos Departmental population Estimated live births/year Estimated live births in study period No. of hospital births in study period Proportion of births in study hospital
662 199 25 097 14 479 921 6.4%
715 205 27 106 11 468 997 8.7%
36%
p = 0.001
Quetzaltenango Departmental population Estimated live births/year Estimated live births in study period No. of hospital births in study period Proportion of births in study hospital
516 957 16 749 6764 989 14.6%
558 337 18 090 6610 1322 20.0%
37%
p = 0.001
experienced the largest increase in admissions (77% increase) and San Marcos and Quetzaltenango increased their numbers by half (Table 1).
4.2. Proportion of births in EmOC facilities or coverage We adjusted the weekly average of obstetric admissions shown in Table 1 down since these averages included a small number of women who were admitted, but for whom no delivery occurred, such as women admitted with premature labor but were discharged before delivery, or where the outcome of the newborn was left blank. Thus, we believe our estimate of the number of hospital births to be conservative. The proportion of births in hospital increased significantly in each department, ranging from increases of 36% to 52% (Table 2). In the depart-
Table 3
ment of Solola ´, the MOH has only one hospital and despite a 50% increase, coverage increased from 7% to 11%. Coverage by the hospital in San Marcos increased from 6 to 9%, and at the hospital in Quetzaltenango from 15 to 20%. According to the UN Guidelines, at least 15% of all births should take place in EmOC facilities since at least that many are thought to develop severe complications [3].
4.3. Met need The proportions of obstetric admissions with major complications (excluding abortion-related complications) varied across departments: 30% in Solola ´, 14% in San Marcos and 11% in Quetzaltenango (data not shown). Distinct case-mix patterns and accessibility to alternative facilities may explain the variations. Both San Marcos and Quetzalte-
Met need by hospital and time period
Hospital
1995—1996
1998—1999
Percent increase
p-value
Solola´ With abortions Without abortions
147/736 = 20.0% 110/736 = 15.0%
252/642 = 39.3% 161/642 = 25.1%
96% 67%
p = 0.001 p = 0.001
San Marcos With abortions Without abortions
335/2172 = 15.4% 139/2172 = 6.4%
355/1720 = 20.6% 152/1720 = 8.8%
34% 38%
p = 0.001 p = 0.001
Quetzaltenango With abortions Without abortions
202/1014 = 19.9% 78/1014 = 7.7%
na 149/992 = 15.0%
95%
p = 0.001
na = not available.
214 Table 4
P. Bailey et al. Referral by hospital and time period
%Obstetric admissions reporting referral Solola ´ San Marcos Quetzaltenango %Complicated cases reporting referral Solola ´ San Marcos Quetzaltenango
1995—1996 (n)
1998—1999 (n)
p-value
56.8 (310) 34.3 (775) 9.2 (934)
52.6 (517) 47.7 (1049) 15.5 (1116)
p = 0.675 p b 0.000 p b 0.000
56.2 (137) 34.0 (212) 11.9 (118)
56.0 (300) 47.2 (352) 21.1 (218)
p = 1.000 p = 0.002 p = 0.037
Missing data on the variable dreferralT accounts for the differences in denominators in row 1 (obstetric admissions) of this table and Table 1.
nango have a second MOH EmOC facility to meet the needs of the departments’ pregnant women; Solola ´ does not. Met need was consistently higher when abortions were included; the largest increases occurred in Quetzaltenango and Solola ´ (Table 3).
4.4. Proportion of all births delivered by cesarean The contribution that each hospital made to the proportion of births delivered by cesarean increased in each department, but only significantly in Solola ´ and Quetzaltenango (2.4% to 4.2% and 2.4% to 2.9%, respectively). San Marcos recorded the lowest cesarean rate (1.9% in 1998— 1999). All rates are below the 5% minimum that the UN recommends.
4.5. Proportion of obstetric admissions and proportion of complicated cases reporting referral San Marcos and Quetzaltenango witnessed significant increases in the number of women who reported that they had been referred and in the number of complicated cases reporting referral (Table 4) 5 . In Solola´ , more than half of the admissions reported that they had been referred as did women with obstetric complications. About half of the women said TBAs referred them while others were referred by lower level health facilities, by private practitioners or community members.
4.6. Profile of hospital obstetric admissions The population profile changed at all of the hospitals, but not all hospitals experienced the same changes (Table 5). Solola ´ and San Marcos 5 For this analysis, we considered all obstetric complications, not just those used to calculate met need.
served an increasingly marginalized population while Quetzaltenango appears to have provided services to an increasingly urban and educated population. Solola ´ and San Marcos experienced increases in the proportion of admissions of indigenous women, women who had never had a hospital birth (and yet were multiparous), and women with no formal schooling. In the case of San Marcos, the percentage of obstetric patients who lived 10 or more kilometers from the hospital increased from 31% to 45%. In Quetzaltenango, on the other hand, only the proportion of women who had never delivered in hospital increased. Table 5 Selected characteristics of hospital obstetric population, by hospital and time period Characteristic
Solola ´
San Marcos
Quetzaltenango
83.7 (368) 88.3 (523) p = 0.046
40.4 (900) 58.7 (1048) p b 0.000
62.7 (993) 58.9 (1351) p = 0.062
%No prior hospital birth* 1995 14.9 (368) 1998 34.1 (519) p b 0.000
22.8 (959) 28.1 (1046) p = 0.009
19.4 (989) 39.6 (1346) p b 0.000
%No formal schooling* 1995 25.9 (278) 1998 38.9 (527) p = 0.001
13.8 (848) 19.8 (1052) p = 0.001
29.2 (998) 18.6 (1362) p b 0.000
%Lives N 10 km* 1995 50.8 (372) 1998 48.8 (529) p = 0.691
30.8 (975) 45.1 (1053) p b 0.000
37.7 (1009) 24.6 (1362) p b 0.000
%Indigenous 1995 1998
Observation: The p-values refer to 1995 and 1998 distributions. Women with abortion-related complications were excluded. The asterisked variables are categorical variables with three mutually exclusive categories. In the case of the variable hospital birth, the categories were dfirst birth,T dprior hospital birth,T and dno prior hospital birthT. The education categories were dno formal schooling,T d1—6 years of schooling,T and dmore than 6 years of schoolingT. The categories for the variable distance from the hospital were dsame cityT, dlives b 10 km from the hospitalT and dlivesz 10 km from the hospitalT.
Monitoring utilization and need for obstetric care
5. Discussion 5.1. Limitations of data The monitoring system proved to be a successful tool in three of the four hospitals where it was implemented. In the fourth hospital, staff claimed they were understaffed making registry completion difficult, particularly on weekends. Improvements in the system were not difficult to achieve. With respect to data quality, several issues had to be addressed. Missing information was observed for specific variables, suggesting that the information was never obtained from the woman or that the blanks meant dnoT or dnot applicable.T Additional supervision and training of the staff completing the form resulted in more complete reporting in 1998 than in 1995. The registry data were purged of double entries for the same woman admitted for the same event, but the number of cases never registered is not known. Some speculate that the cases most likely to be omitted might be emergencies, which clearly would affect our findings if a considerable number of such cases were not included. Although no systematic effort was carried out by the project to determine the completeness of the registry of obstetric cases, no anecdotal evidence suggested serious underreporting. The reporting of complications can suffer from poor reliability and varying definitions. Efforts were made to standardize the classification of complications within hospital and across hospitals. How much clinical refinement or severity assessment might be achieved with a maternity registry will depend on the level of training and supportive supervision. However, if cases are used for trend monitoring and classification does not undergo major changes, this internal exercise can be a powerful and inexpensive tool for monitoring and process evaluation. An independent investigation of the quality of information collected in maternity registries of this region was carried out in 2000 after the closure of the MotherCare project. Only one institution overlapped—the hospital in San Marcos. The researchers found that the data in maternity registries were bhighly congruent with data in corresponding medical records.Q When maternity registries were compared with surgical registries, however, 20% of the cesareans found in the latter were missing in the former. This indicates the need to have mechanisms in place to ensure completeness and quality of the data by cross checking registries for certain items [21].
215 The strength of the indicators in this analysis lies in what they reveal about the individual hospitals over time and their use by hospital administrators and technical staff for planning, monitoring performance and even outreach. The potential for using facility-based data to provide populationbased measures is promising, if all facilities collect the same data in the same way. In this case, although not all EmOC facilities in the three departments used the maternity registry, the authors made an effort to estimate departmentwide values. They obtained service statistics on births and cesareans from the two hospitals not using the registry and from several but not all private facilities [22]. With this additional information, we estimated the proportion of births in EmOC facilities and the cesarean delivery rate at the departmental level. We estimated the proportion of births in EmOC facilities in 1998 to be 14% in the department of Solola ´, 19% in San Marcos and 55% in Quetzaltenango. Cesarean rates ranged from 3.5% in San Marcos, 5.6% in Solola ´ and 6.4% in Quetzaltenango. This estimate of the proportion of births falls within the confidence intervals of the 1998 DHS point estimate, reinforcing our confidence in the data quality. The 1998 DHS reported an increase in the population-based cesarean rate from 5.6% to 7.3% in this southwest region [14,15], slightly higher than our projections of between 3.5% and 6.4%.
6. Conclusions This paper illustrates how data collected over time from a hospital maternity registry can be used to monitor changes in the utilization of EmOC services as well as changes in who accesses these services. Although utilization increased at all three hospitals during this 3-year period of MOH and MotherCare interventions, a wide variation in the utilization of services still can be observed across these hospitals with hospital births consistently lower in San Marcos and Solola ´. Met need at each of the targeted hospitals also increased but continues low. Rates of met need tend to be considerably higher when women with abortion-related complications are included. These data confirm what other researchers have found when abortion complications are added to the estimate of met need [7,11,12]. The referral estimates suggest that the hospital in Solola ´ plays an important role as a referral center for its rural catchment area. At the other extreme was the project hospital in Quetzalte-
216 nango with the smallest proportion of referrals and the most urban catchment area but the percentage of complicated cases who said they were referred did almost double. The referral results and the changes in population profile highlight the different roles that the three hospitals play in their respective communities. Particularly encouraging from the perspective of human rights is the observation that more equitable access to services has been achieved. The overall picture of this region is one in transition from more community-based maternity care to where, increasingly, deliveries take place in EmOC facilities. This is a critical prerequisite to reducing maternal mortality as it combines the potential of a skilled attendant in an enabling environment. The relationship between the results presented here and the effectiveness of the MOH and MotherCare interventions is open to challenge. Without a control group for comparison, we cannot rule out the possibility that similar changes occurred in other regions, or that external political factors such as the signing of the 1996 Peace Accord made accessing services seem safer for women. The gold standard for attributing cause to impact in health care evaluations is the randomized clinical trial (RCT) but three characteristics make the RCT difficult for evaluating most dsafe motherhoodT programs. First, it would be unethical to withhold known life-saving procedures from any population. Second, these programs focus on communities rather than individuals, and third, like this program, they are multifaceted interventions, notoriously difficult to evaluate because more than one of the interventions can affect a single outcome [23]. Quasi-experimental designs are superior to before and after designs as this paper describes. An earlier experience with a quasi-experimental design with the MOH and MotherCare resulted in high costs and frustration with bcontrolQ areas that were difficult to maintain in the face of interventions by non-governmental organizations [24]. Nevertheless, progress was made towards the goal of increasing the number of women with obstetric complications who were attended at facilities where comprehensive EmOC is available. Project in-service training of hospital staff and use of clinical protocols may have translated into better diagnosis and treatment of complications, resulting in a more favorable image of the hospital. Improved relations between communities and hospital staff may have resulted in the increase in referrals (or compliance with referrals) and in the number of women who traditionally did not seek hospital services. Other likely contributors to the
P. Bailey et al. increase in utilization were the community IEC efforts that Pereira et al. found associated with an increase in awareness of bdangerQ signs and signals of obstetric complications in the same study area [18]. The challenge now is to maintain the momentum, if not accelerate it, to increase access and utilization of EmOC.
Acknowledgments The authors would like to thank the staff at the hospitals that participated in the implementation of the birth registry: the Regional Hospital San Juan de Dios in Quetzaltenango, the Departmental Hospital Moises Villagran in San Marcos and the Departmental Hospital of Solola ´. This study was made possible by support from MotherCare/John Snow, Inc. (Contract HRN-Q-00-00039-00), the Maternal and Neonatal Health Project/JHPIEGO (Contract HRN-A-00-98-00043-00) and the Guatemalan Mission for the United States Agency for International Development. The opinions expressed in this paper are those of the authors and do not necessarily reflect the views of the US Agency for International Development, John Snow, Inc. or JHPIEGO.
References [1] Maine D, McCarthy J, Ward VM. Guidelines for monitoring progress in the reduction of maternal mortality: a work in progress. New York7 UNICEF; 1992. [2] Bertrand J, Escudero G. Compendium of indicators for evaluating reproductive health programs. Volume One. Measure Evaluation Manual, No. 6. (2002). [3] UNICEF, WHO, UNFPA. Guidelines for monitoring the availability and use of obstetric services. New York7 UNICEF; 1997. [4] MotherCare Matters VM. Safe motherhood indicators— lessons learned in measuring progress; 1999. p. 8. [5] Ronsmans C, Campbell O, McDermott J, Koblinsky M. Questioning the indicators of need for obstetric care. Bull WHO 2002;80:317 – 24. [6] Ronsmans C, Achadi E, Sutratikto G, Zazri A, McDermott J. Use of hospital data for Safe Motherhood programmes in South Kalimantan, Indonesia. Trop Med Int Health 1999;4: 514 – 21. [7] Pathak LR, Kwast BE, Malla DS, Pradhan AS, Rajlawat R, Campbell BB. Process indicators for safe motherhood programmes: their application and implications as derived from hospital data in Nepal. Trop Med Int Health 2000; 5(12):882 – 90. [8] McGinn T. Monitoring and evaluation of PMM efforts: what have we learned? Int J Gynecol Obstet 1997;59:S245 – 51 [Suppl.]. [9] Bailey P, Paxton A. Program note: Using UN process indicators to assess needs in emergency obstetric services: Mozambique, Nepal and Senegal. Int J Gynecol Obstet 2002;76:299 – 305.
Monitoring utilization and need for obstetric care [10] AMDD Working Group on Indicators. Program note: using UN process indicators to assess needs in emergency obstetric services in Bhutan, Cameroon and Rajasthan, India. Int J Gynecol Obstet 2002;77:277 – 84. [11] AMDD Working Group on Indicators. Program note: using UN process indicators to assess needs in emergency obstetric services in Pakistan, Peru and Vietnam. Int J Gynecol Obstet 2002;78:275 – 82. [12] AMDD Working Group on Indicators. Program note: Using UN process indicators to assess needs in emergency obstetric services in Morocco, Nicaragua and Sri Lanka. Int J Gynecol Obstet 2005;89:221 – 30. [13] Hussein J, Goodburn EA, Damisoni H, Lema V, Graham W. Monitoring obstetric services: putting the UN guidelines into practice in Malawi: 3 years on. Int J Gynecol Obstet 2001;75:63 – 73. [14] INE (Instituto Nacional de Estadı´stica). Guatemala Encuesta Nacional de Salud Materno Infantil, 1995. DHS/Macro International Inc; 1996. [15] INE (Instituto Nacional de Estadı´stica). Guatemala Encuesta Nacional de Salud Materno Infantil, 1998—1999. 1999. DHS/ Macro International Inc. [16] Ministry of Public Health and Social Assistance. Study of maternal mortality in Guatemala 1989. Director General of Health Services, Department of Maternal and Child Health; 1989. [17] Hurtado E. The MotherCare/Guatemala Project Overview of IEC Strategies and Lessons Learned, Guatemala City; 1999. 14 p.
217 [18] Perreira K, Bailey P, Bocaletti E, Hurtado E, Recinos S, Matute J. Improving recognition of obstetric complications through community- and clinic-based education. Matern Child Health J 2002;6(1):19 – 28. [19] Molina S. MotherCare II Guatemala: Informe de 5 An ˜os 1994—1999. Guatemala City; 1999. 113 p. [20] Lentner C, editor. Geigy scientific tables, 8th edition. Introduction to statistics, statistical tables, mathematical formulae, compiled by K Diem and J Seldrup, vol. 2. CIBAGEIGY; 1982. p. 224 – 5. [21] Valladares R, Barillas E, Bassett Hileman S, Fonseca-Becker M, Rose M, Buekens P. Maternity registers in Guatemalan facilities: a potential source of data for monitoring obstetric care. Strengthening monitoring and evaluation of maternal health programs MEASURE evaluation bulletin, vol. 7; 2003. [22] Cross Y, Rose A. 2000. Personal communication. [23] Campbell O, Filippi V, Koblinsky M, Marshall T, Mortimer J, Pittrof R, Ronsmans C, Williams. Lessons Learnt: a decade of measuring the impact of safe motherhood programmes. DFID Research Work Programme on Population and Reproductive Health Maternal and Child Epidemiology Unit, London School of Hygiene and Tropical Medicine. 25 pages. [24] Bailey P, Sza ´szdi JA, Glover L. Obstetric complications: does training traditional birth attendants make a difference? Pan Am J Public Health 2002;11(1):15 – 22.