- Email: [email protected]
Measuring improved targeting of health interventions to the poor in the context of a community-randomised trial in rural India
Contemporary Clinical Trials 28 (2007) 382 – 390 www.elsevier.com/locate/conclintrial
Measuring improved targeting of health interventions to the poor in the context of a community-randomised trial in rural India ☆ Saul S. Morris a,⁎, M. Kent Ranson a , Tara Sinha b , Anne J. Mills a a
London School of Hygiene and Tropical Medicine, London, United Kingdom b Self-Employed Women's Association, Ahmedabad, Gujarat, India Received 15 January 2006; accepted 9 October 2006
Abstract In spite of growing interest in socioeconomic differentials in health outcomes and access to health services, little has been written about methodologies for assessing the impact of equity-enhancing policies or programs. This paper describes three methodological challenges involved in designing a randomised trial with an equity outcome, and how these were met in a trial of alternative strategies to improving the uptake of benefits of a health insurance scheme among its poorest members. The Vimo SEWA trial is nested within a community-based insurance scheme in rural India. While conducting this trial, three methodological problems were encountered: (i) measuring poverty (or “wealth”, or “socioeconomic status”) (ii) assessing beneficiaries against an appropriate reference standard population and (iii) settling on an appropriate equity measure as an outcome indicator. These problems are likely to arise in any policy or program assessment that has an equity outcome. In the Vimo SEWA trial, the socioeconomic status of beneficiaries (claimants) is assessed relative to that of all scheme members living in same sub-district by applying a rapid assessment questionnaire – which reduces to an integrated index of socioeconomic status – to both a random sample of members in each sub-district, and to all claimants. The results are used to estimate the full distribution of socioeconomic status of members in each sub-district, with each member given a rank score between 0 and 100. Interpolation is used to estimate the rank scores of claimants relative to the membership base. The primary outcome measure for the trial is the mean socioeconomic rank score of claimants. In developing country settings, using an index of socioeconomic status is simpler than assessing household income or the value of household consumption. It is also relatively straightforward to compare the socioeconomic status of health program beneficiaries with a relevant reference population, although two independent surveys are required. Expressing relative wealth on a scale from zero to 100 is conceptually appealing, and the mean value of this rank score provides an equity-specific outcome measure readily integrated into the usual analytic framework for cluster-randomised trials. © 2006 Elsevier Inc. All rights reserved. Keywords: Equity; Outcome measures; Cluster-randomised trials; India
☆
Financial support for this study was provided by the Wellcome Trust. ⁎ Corresponding author. E-mail address: [email protected] (S.S. Morris).
1551-7144/$ - see front matter © 2006 Elsevier Inc. All rights reserved. doi:10.1016/j.cct.2006.10.008
S.S. Morris et al. / Contemporary Clinical Trials 28 (2007) 382–390
383
1. Introduction Many recent reviews have highlighted the dramatic inequities in health outcomes between and within countries [17,48,12,42,43,46,18]. For example, mortality rates of children under 5 in the least-developed countries are 159 per 1000 births compared with 6 per 1000 births in high-income countries [7], and in India, the poorest 20% of the population has more than double the mortality rates, malnutrition, and fertility of the richest 20% [31]. Inequitable access to health care goods and services has long been recognised as a key determinant of these differentials — as long ago as 1971, Tudor Hart formulated the “inverse care law”, which states that “the availability of good medical care tends to vary inversely with the need for it in the population served” [39]. For example, poor children in poor countries are typically far less likely to be immunised than better-off children, and the poor are less likely than the non-poor to have access to high-impact health services, such as skilled antenatal and delivery care [47]. 2. Rationale for randomised trials with equity endpoints Building an appropriate policy response to health inequities requires, among other things, undertaking health equity impact assessments — “policies and programmes in a wide range of sectors must be subjected to such assessments so that ‘unhealthy policies’ can be identified and ‘healthier’ ones developed” ([45], p.311). While there is a large and expanding literature on the measurement of differentials between groups and individuals in wealth [8,35] and health [44,1], little has been written about methodologies for assessing the impact of equity-enhancing policies or programs. Wagstaff identifies four approaches that can shed light on the impacts of equity-enhancing policies on health inequalities [42,43]: cross-country (cross-sectional) comparative studies; country-based before-and-after studies with controls; benefit–incidence analysis; and decomposition analysis. In spite of their clear advantages for inferring causality, only a handful of evaluations of equity-enhancing policies or programs has used randomised methodologies. In the Netherlands a six year program was launched in 1994 to evaluate 12 small-scale interventions designed to reduce health inequalities, using experimental and control groups [23]. In Cambodia, the national government has experimented with contracting of management and delivery of health services on a pilot basis, carefully evaluating the experience with a before–after (1997–2001), randomised controlled trial design [2,37,34]. Two different models involving contracting of NGOs to manage health services at the district level were compared with a control group of directly managed government districts. Primary outcome indicators included targeting of health services to the poorest half of households. The results of the final survey indicated that the contracted districts performed better than the control districts with respect to most of the health service equity indictors. A similar study design has been used in Nicaragua to evaluate the impact of providing rural households with cash in exchange for keeping up-to-date with preventive health care check-ups [25]; the proportion of children under three years of age whose weight was monitored over the previous six months increased between 2000 and 2002 by 24 percentage points among the extreme poor, 12 percentage points among the poor, and 10 percentage points among the non-poor. It is possible that one of the reasons for the paucity of randomised trials with equity endpoints is the lack of methodological guidance for appropriate study design. In the following section, we introduce the context of a community-randomised trial (CRT) nested within a community-based health insurance scheme in rural India. We then describe three methodological challenges involved in designing a trial with an equity outcome, and how these challenges were tackled in the Indian CRT. 3. The Vimo SEWA equity of health insurance trial Since 1992, the Self Employed Women's Association (SEWA) – an Indian trade union for poor women working in the informal sector – has been providing voluntary assets, life and hospitalisation insurance, in a single policy, to its members and their families through Vimo SEWA (SEWA Insurance). In 2005, the premium for the least expensive policy was Rs. 100 (USD 2.3) per annum, covering the costs of inpatient care to a maximum of Rs. 2000 (USD 46) per annum. In the same year, more than 130,000 women, men and children were enrolled in rural and urban areas of the state of Gujarat. The scheme was accessible to the poorest member households, and utilization (submission of claims for hospitalisation) was equitable in Ahmedabad City, but in rural areas the financially better-off were significantly more likely to submit claims than were the poorest [32]. A variety of factors – including travel costs, poor health and transportation infrastructure, and inadequate knowledge of the scheme's benefits and processes – prevented the poorest in rural areas from accessing inpatient care or from submitting an insurance claim [36].
384
S.S. Morris et al. / Contemporary Clinical Trials 28 (2007) 382–390
Since 2004, a community-randomised trial has been seeking to improve the equity impact of Vimo SEWA in rural areas by reducing the differential in claims submission between poor and less poor members. The trial is investigating the impact of three interventions on access to scheme benefits: (i) after-sales service with supportive supervision; (ii) prospective reimbursement of hospital charges; and (iii) a combination of the two. The equity impact of the interventions will be assessed by comparing the distribution of socioeconomic status of claimants with that of the scheme's membership base in the same sub-district, and will be reported elsewhere. No previous randomised trial has assessed interventions aimed at improving the distributional impact of a health insurance scheme in a developing country. 4. Methodological challenges and how they were addressed 4.1. Identifying the poorest individuals Approaches to measuring poverty have been heavily influenced by the disciplinary background of different groups of researchers, and by geographic region characteristics. In the United Kingdom and other European countries, there has been a strong tradition of analyzing occupational class differentials in health outcomes [9], whereas in the United States there has been a preoccupation with race [21,22]. Microeconomists have developed robust methodologies for assessing direct measures of household welfare based on the parallel concepts of income and consumption (the latter term referring to the monetary value of all goods and services consumed by a household over a given period of time) [8]. Both income and consumption methodologies have been extensively applied to assess the distribution of health service users by socioeconomic status in developed and developing countries. However, income and consumption measures are problematic to assess in poor countries [13]. In such environments, incomes can be highly seasonal, and it may be difficult to separate clearly business transactions from consumption transactions. For those who are self-employed, and especially for rural households whose income comes largely from self-employment in agriculture, a huge number of different transactions need to be accounted for to determine net income, and respondents may find it difficult to recall all the values involved, or not wish to divulge the details of their incomes. And even after years of methodological development, some transactions, such as dowry receipts, for example, are hard to categorise in a household framework of accounts. Assessing consumption suffers from many similar difficulties, such as seasonality and unclear boundaries between business expenditures and consumption expenditures. Both approaches require very detailed disaggregated information collection to prevent underreporting, commonly taking up to an hour or more per household to administer, and the questions asked are often perceived to be invasive [27]. Recently, there has been a large body of literature proposing alternative measures that are simpler to operationalise [14,26,16,33]. Most of these methods rely on the observation that wealthier households are more likely to own a range of consumer durables and to benefit from utilities such as electricity or running water. They involve constructing an index of such assets and services, which is used to rank households according to their welfare levels. However, the precise values taken have no ready interpretation and because of the limited number of domains represented, some authors have questioned whether such measures can really be considered a valid representation of socioeconomic status [28,27]. For the Vimo SEWA trial, we developed a customized index of socioeconomic status. The study baseline questionnaire (see online Appendix) – used to assess socioeconomic status among the general, rural population (Survey I) – was based on a generic survey tool developed by the Consultative Group to Assist the Poorest (CGAP) and the International Food Policy Research Institute (IFPRI) to measure the poverty of microfinance clients [20]. Henry et al. compiled from the literature an exhaustive list of indicators of human, material and social capital, as well as indicators related to the fulfillment of basic needs, and then pared down the list based on eight pre-determined criteria to create a generic questionnaire. This questionnaire was modified to suit the rural Gujarati context by, for example, identifying the food grains most commonly stored. The modified questionnaire had modules on: family size and composition; quality of housing; type, number and value of assets, and level of school education and occupation of household members; hunger episodes and types of food eaten, and household expenditure on clothing and footwear. A shorter questionnaire was then used to assess the socioeconomic status of Vimo SEWA members (Survey II) and health insurance claimants (Survey III). This “rapid assessment” questionnaire included only a subset of the questions asked in Survey I — the questions necessary for a summary index based on indicators that most strongly distinguished relative levels of socioeconomic status, as described in the following paragraphs. The indicator variables retained in the questionnaire are shown in Table 1.
S.S. Morris et al. / Contemporary Clinical Trials 28 (2007) 382–390
385
Table 1 Indicators included in summary index of socioeconomic status Human • The percentage of household adults who can read and write (continuous) resources • The percentage of household adults whose maximum level of schooling was “attended college or university” (continuous) Dwelling • Number of rooms, excluding kitchen (continuous) • Whether the home's walls are made of “brick or stone with plaster” (dichotomous) • Whether the home's walls are made of materials other than brick or stone (dichotomous) • Whether the household has no electrical connection, shared connection, or its own connection (categorical) • Whether natural gas is the primary cooking fuel used (dichotomous) Food • During the last year, when cooking oil stores were highest, whether there was sufficient stock to last 1 month (dichotomous) security • During the last year, when millet or millet flour stores were highest, whether there was sufficient stock to last 12 months (dichotomous) • During the last year, when wheat or wheat flour stores were highest, whether there was sufficient stock to last 1 month (dichotomous) Assets • Number of refrigerators (continuous) • Number of electric fans (continuous) • Number of mattresses (continuous) • Number of wrist watches (continuous)
Principal Components Analysis (PCA) using Stata 7.0 (Stata Corporation, College Station, TX) was applied to the socioeconomic data to derive a single linear variable incorporating as much as possible of the information that was common to the various individual indicators. In order to do this, each indicator was standardised on the same scale (mean zero, standard deviation one), and these quantities were weighted – using empirically derived weights from the PCA – and summed. The resulting index can be represented by the formula [15]: Aj ¼ f1 d ða1j − ¯a 1 Þ=s1 þ N þ fN d ðaNj − ¯a N Þ=sN where j subscripts households, a1 to aN are the various assets, fn is the weight for the nth asset, ā indicates the mean value of an asset (over all households), and s the standard deviation. The summary index A is itself expected to have mean zero and standard deviation one, and is interpreted as an integrated measure of socioeconomic status. Thus, an index value of + 2 indicates very high socioeconomic status, whereas a value of − 2 indicates very low status, and a value of zero indicates intermediate levels. Indicator variables correlating only weakly with the summary index were dropped from the analysis at an early stage (and not included in the questionnaire for surveys II and III). These variables included total household size, which proved less discriminatory in this context than a set of variables related to educational achievement. 4.2. Assessing beneficiary distributions against an appropriate reference standard When health interventions are made available to all members of a national population, the socioeconomic status of those who use the service can be compared to an appropriate national standard. National distributions of socioeconomic status are usually available from household survey series that are commonly updated every few years (or even yearly) in most countries. For example, in Great Britain, a small proportion of individuals use private hospitals, and various indicators of their socioeconomic status, measured in facility surveys or administrative records, can be compared to the profile of the national population, which is assessed annually by the Office of National Statistics (www.statistics.gov. uk). Similarly, in India, the socioeconomic status of users of public health services in a given state can be compared with state-level representative data from surveys such as the National Family Health Survey [30]. However, many services are not available to entire populations. For example, in South Africa there are a number of programs that provide voluntary counselling and testing for HIV services to individuals living in particular cities. Whilst it may be possible to know the socioeconomic status of program beneficiaries, there is rarely adequately disaggregated data available to describe the distribution of socioeconomic status in the reference cities, and data for all urban areas combined have been used instead [38]. Another relevant example concerns the assessment of the socioeconomic status of beneficiaries of insurance schemes (i.e. successful claimants); in this context, it would be important to compare the resulting distribution of socioeconomic status to that of program members only, since nonmembers could never benefit from the scheme.
386
S.S. Morris et al. / Contemporary Clinical Trials 28 (2007) 382–390
In the Vimo SEWA trial, the socioeconomic status of claimants was contrasted with that of the membership base in their sub-districts of residence. This protects against inappropriate inferences that would result, for example, from the comparison of rural claimants with urban members. In order to permit the relevant comparisons, representative surveys of scheme member households were undertaken in the sixteen study sub-districts at baseline (2003) and again postintervention (2005), to determine the distribution of socioeconomic status in each year and sub-district. Based on these surveys, the membership base in each sub-district was ranked on socioeconomic status (using the summary index described above), so that the poorest member household was ranked zero, the wealthiest ranked 100, and the household in the middle of the range ranked 50. More specifically, because the survey was based on a sample of size k from each sub-district, it was assumed that the lowest observed value of socioeconomic status in each sub-district represented not the absolute minimum possible value, but rather centile 1 / (k + 1) * 100 of the sub-district specific distribution of members. Similarly, the highest observed value was taken to represent centile k / (k + 1) * 100 rather than the absolute maximum. Fig. 1A illustrates the association, for one sub-district, between socioeconomic index values (on the y-axis) and socioeconomic rank scores (on the x-axis). Following the characterization of the membership base in each sub-district, a separate survey of claimant households was undertaken, and a socioeconomic index value for each household was again calculated as described above. Socioeconomic index values were then converted into sub-district specific rank scores (an ordinal scale with a range of zero to 100) by linear interpolation based on the rank of the member households with the closest (i.e. the next lowest and the next highest) socioeconomic index values. Fig. 1B shows all of the claimants in one sub-district interspersed with the members in the same sub-district. Fig. 1C illustrates how the socioeconomic index values for each claimant were converted into rank scores on the basis of linear interpolation. Claimants whose socioeconomic index values were lower or higher than those of any sampled member in the same sub-district were assigned rank scores of 0 and 100, respectively. The socioeconomic rank is thus a scale, specific to each sub-district, that indicates the relative socioeconomic status of members and claimants. Scores on the scale provide no information about the absolute difference in socioeconomic
Fig. 1. Within-cluster rank versus untransformed socioeconomic index value for members (1A), members plus claimants (1B), and illustrating the linear interpolation of within-cluster centile of socioeconomic status for one claimant (1C) (Bayad sub-district, 2003).
S.S. Morris et al. / Contemporary Clinical Trials 28 (2007) 382–390
387
status between one point and another. That is, the absolute difference in wealth between households scored 5 and 10 may be entirely different from households scored 90 and 95 in the same sub-district, or between households scored 5 and 10 in a different sub-district. The socioeconomic rank score is well suited for assessing the distribution of a beneficiary population relative to a reference population, and monitoring changes in this distribution over time. 4.3. Selecting an appropriate trial outcome measure Recent work has discussed a range of statistical measures that can be used to summarise the distributional (equity) outcomes of health interventions [44,1]. Commonly used measures include: the proportion of health program benefits accruing to the poorest population groups; the relative frequency of a health program benefit or health outcome in the least poor group compared to the frequency in the poorest group, and the concentration index, which measures the covariance between the socioeconomic welfare ranks of individuals in a population and the size of the benefits that they receive [1]. This measure conveniently varies from zero (perfect equality in the distribution of benefits) to one (all benefits accrue to a single person). However, despite its conceptual elegance, our experience has been that most policymakers find the concentration index difficult to grasp. Both proportions of benefits accruing to the poorest, and rich:poor ratios, are sensitive to choice of cut-off value distinguishing the poor from the less poor. Yet in many situations, it may not be obvious what the optimal cut-off value should be. For example, many studies choose to look at the ratio of benefits accruing to the least-poor 20% of the reference population relative to the poorest 20% [6,24,18], but it may be equally valid to calculate rich:poor ratios comparing the least-poor and poorest 10%, 40% (as in Ref. [30]), or 50% (as in Ref. [34]). Other studies examine the benefits accruing to those below an appropriately defined poverty line [32]. But this creates additional problems of interpretation in a longitudinal study where the composition of the reference population is itself changing. Further complexities arise in the context of a cluster-randomised trial. In this context, the selected outcome measure will be quantified in every trial cluster, ruling out any measure which is unduly demanding in terms of sample size. Furthermore, an efficient trial analysis, utilizing information on both between- and within-cluster variance [10,19,3– 5,11,29,41], is only possible when the outcome measure has a clearly defined variance function. In the case of the concentration index, straightforward estimation of the variance function is possible only if data on both beneficiaries and the reference population are drawn from the same survey. This, as we have seen above, is not always feasible. The primary endpoint of the Vimo SEWA trial was determined to be the mean socioeconomic rank score of insurance claimants relative to their local membership base. This parameter has the advantages of: varying on a zero to 100 scale, with a value of 50 indicating an equity-neutral outcome; avoiding arbitrary cut-offs dividing the ‘poor’ from the ‘non-poor’ and being equally sensitive to changes in benefit uptake among poorer and wealthier members; being insensitive to changes over time in the socioeconomic status of the entire membership base that are exactly paralleled in the socioeconomic status of claimants; being simple to calculate, with a straightforward variance function; and demonstrating relatively limited cluster-to-cluster variability at baseline. The sample size calculations used in the trial assumed that, as the barriers to claiming benefits faced by the poorest scheme members were reduced by the interventions, the mean socioeconomic rank score of claimants relative to the local membership base would fall by 20 points. This was felt to be the minimum change that would make the distribution of benefits unequivocally ‘pro-poor’. 5. Discussion This paper has presented three methodological problems that are likely to arise in any policy or program assessment that has an equity outcome: (i) measuring poverty (or “wealth”, or “socioeconomic status”); (ii) assessing beneficiaries against an appropriate reference standard population; and (iii) settling on an appropriate equity measure as an outcome indicator. In a cluster-randomised trial which aims to optimise the equity impact of Vimo SEWA's community-insurance scheme, these challenges were addressed by: (i) developing a short socioeconomic index based on indicators that were found to distinguish relative levels of socioeconomic status most strongly; (ii) conducting purpose-specific surveys of both the member (target) and claimant (beneficiary) population, both at baseline and follow-up; and (iii) using as a primary outcome measure the mean socioeconomic rank score of claimants relative to the membership base in each intervention cluster (sub-district). Given that this is one of very few randomised and controlled public health trials that has had an equity issue as a primary outcome, useful lessons can be drawn from this experience.
388
S.S. Morris et al. / Contemporary Clinical Trials 28 (2007) 382–390
Weaknesses of the study methodology are several. First, the socioeconomic index, as a measure of poverty, is somewhat abstract, and can be difficult for people to understand vis-à-vis the more familiar concepts of income or consumption, especially because it is not expressed on a money metric. Second, the methodology was logistically challenging in that linking beneficiaries to the target population required multiple surveys — member and claimant surveys, both at baseline and follow-up. However, a single survey of members and the claimants among them would have had to be huge (given that only 2% of members submit a health insurance claim each year) in order to achieve the necessary claimant sample size. Third, like the measure of poverty, the primary outcome measure is somewhat abstract relative to, for example, health service utilization or mortality. Fourth, because the chosen endpoint is not correlated with anything else at the cluster level, the measures we took to ensure treatment group balance in this outcome measure (a priori stratification) did not result in groups that were balanced in anything else (e.g. population size of intervention areas, average socioeconomic status of members, rates of illness or hospitalisation). This is a problem in that: (i) after looking at our primary outcome, we would like to look at secondary outcomes which may not have been balanced at baseline; and (ii) these imbalances directly impact the face validity of the trial. On the other hand, there are compensating methodological strengths. First, the summary index of socioeconomic status is quite simple and transparent in that it is constructed from data on only fourteen variables. In presenting results (for example, differences in socioeconomic status between claimants and members) one can present not just the mean index values but also mean values for the fourteen underlying variables. Second, the survey instrument that is used to generate the index is short and simple, and as a result can be administered by less-skilled interviewers, and in an interview of only 10 to 15 min. Third, the instrument is strong in its content validity — it has a strong theoretical basis and a systematic approach to indicator selection. While the underlying CGAP tool is “theoretically eclectic”, the authors “carefully justify all of their recommendations on domains to be included in their index and data reduction” [27]. The tool (and corresponding methodology) is locally appropriate – incorporating variables that might be important indicators of socioeconomic status in a particular study setting – but it is also readily generalizable and has been field-tested in four different countries [20]. Finally, the outcome indicator used has certain benefits. It is relatively simple to interpret because it varies on a scale from 0 to 100, with 50 indicating an equity-neutral outcome. It is also efficient as it draws on data from claimants and members of all levels of socioeconomic status. If we had instead, for example, used “percentage of claimants falling below the 10th centile of socioeconomic status” or “falling below the poverty line”, much larger sample sizes would have been required. Methodological research on assessing the equity impacts of health interventions is still in its infancy. The conceptualization of poverty is constantly evolving [40], but this body of work urgently needs distilling around valid, operational measures. Once such measures are agreed upon, researchers will need guidance about the magnitude of impacts that are plausible from different types of interventions, in order that they can implement further randomised trials with adequate statistical power. New theoretical statistical work may be required to demonstrate how these outcome measures can be implemented within a cluster-randomised trial framework, as it is most unlikely that equityenhancing interventions would ever be allocated on an individual basis. Finally, the relevance of equity impact findings needs to be presented clearly to policy-makers in order to stimulate interest in the scaling-up of these critical interventions. Appendix A. Supplementary data Supplementary data associated with this article can be found, in the online version, at doi:10.1016/j.cct.2006.10.008. References [1] Anand S, Diderichsen F, et al. Measuring disparities in health: methods and indicators. In: Evans T, Whitehead M, Diderichsen F, Bhuiya A, Wirth M, editors. Challenging Inequities in Health: From Ethics to Action. Oxford: Oxford University Press; 2001. p. 48–67. [2] Bhushan I, Keller S, et al. Achieving the twin objectives of efficiency and equity: contracting health services in Cambodia. Economics and Research Department Policy Brief No. 6 Manila: Asian Development Bank; 2002. p. 10. [3] Campbell MK, Elbourne DR, et al. CONSORT statement: extension to cluster randomised trials. BMJ 2004;328:702–8 [(20 March 2004)]. [4] Campbell MK, Grimshaw JM, et al. Intracluster correlation coefficients in cluster randomized trials: empirical insights into how they should be reported. BMC Medical Research Methodology, vol. 4(9); 2004. [5] Campbell MK, Thomson S, et al. Sample size calculator for cluster randomized trials. Comput Biol Med 2004;34:113–25.
S.S. Morris et al. / Contemporary Clinical Trials 28 (2007) 382–390
389
[6] Castro-Leal F, Dayton J, et al. Public spending on health care in Africa: do the poor benefit? Bull World Health Org 2000;78(1):66–74. [7] Commission on Macroeconomics and Health. Macroeconomics and Health: Investing in Health for Economic Development. Geneva: World Health Organization; 2001. [8] Deaton A. The Analysis of Household Surveys: a Microeconometric Approach to Development Policy. Washington DC: The World Bank; 1997. [9] Diderichsen F, Whitehead M, et al. Sweden and Britain: the impact of policy context on inequities in health. In: Evans T, Whitehead M, Diderichsen F, Bhuiya A, Wirth M, editors. Challenging Inequities in Health: From Ethics to Action. Oxford: Oxford University Press; 2001. p. 240–55. [10] Donner A. Some aspects of the design and analysis of cluster randomization trials. Appl Stat 1998;47:95–113 [Part 1]. [11] Donner A, Klar N. Pitfalls of and controversies in cluster randomization trials. Public Health Matters 2004;94(3):416–22. [12] Evans T, Whitehead M, et al, editors. Challenging Inequities in Health: From Ethics to Action. Oxford: Oxford University Press; 2001. [13] Falkingham J, Namazie C. Measuring Health and Poverty: a Review of Approaches to Identifying the Poor. London, UK: DfID Health Systems Resource Centre; 2002. [14] Filmer D, Pritchett L. The effect of household wealth on educational attainment: evidence from 35 countries. Popul Dev Rev 1999;25 (1):85–120. [15] Filmer D, Pritchett LH. Estimating Wealth Effects Without Expenditure Data- or Tears: With an Application to Educational Enrollments in States of India. Washington DC: the World Bank; 1998. p. 38. [16] Filmer D, Pritchett LH. Estimating wealth effects without expenditure data- or tears: an application to educational enrollments in states of India. Demography 2001;38(1):115–32. [17] Gwatkin DR. Health inequalities and the health of the poor: what do we know? What can we do? Bull World Health Org 2000;78(1):3–18. [18] Gwatkin DR, Bhuiya A, et al. Making health systems more equitable. Lancet 2004;364:1273–80 [(October 2)]. [19] Hayes RJ, Bennett S. Simple sample size calculation for cluster-randomized trials. Int J Epidemiol 1999;28:319–26. [20] Henry C, Sharma M, et al. Assessing the Relative Poverty of Microfinance Clients: a CGAP Operational Tool. Wasington, D.C.: International Food Policy Research Institute; 2000. [21] Krieger N, Fee E. Measuring social inequalities in health in the United States: a historical review, 1900–1950. Int J Health Serv 1996;26 (3):391–418. [22] Krieger N, Williams DR, et al. Measuring social class in US public health research. Annu Rev Public Health 1997;18:341–78. [23] Mackenbach JP, Stronks K. A strategy for tackling health inequalities in the Netherlands. Br Med J (BMJ) 2002;325:1029–32. [24] Makinen M, Waters H, et al. Inequalities in health care use and expenditures: empirical data from eight developing countries and countries in transition. Bull World Health Org 2000;78(1):55–65. [25] Maluccio J, Flores R. Impact evaluation of a conditional cash transfer program: the Nicaraguan Red de Protección Social. FCND Discussion Paper, No. 184. Washington DC: International Food Policy Research Institute; 2004. [26] Montgomery MR, Gragnolati M, et al. Measuring Living Standards with Proxy Variables. New York: Population Council; 1999. p. 64. [27] Morris SS. Epidemilogy and the study of socio-economic inequalities in health. Measuring Health Equity in Small Areas-Findings from Demographic Surveillance Systems. INDEPTH Network. Aldershot, UK: Ashgate; 2005. p. 1–18. [28] Morris SS, Carletto C, et al. Validity of rapid estimates of household wealth and income for health surveys in rural Africa. J Epidemiol Community Health 2000;54:381–7. [29] Murray DM, Varnell SP, et al. Design and analysis of group-randomized trials: a review of recent methodological developments. Am J Public Health 2004;94(3):423–32. [30] Peters D, Rao K, et al. India: Equity effects of quality improvements on health service utilization and patient satisfaction in Uttar Pradesh State. Reaching the Poor Program Paper, vol. 5. Washington, DC: The World Bank; 2004. p. 18. [31] Peters DH, Yazbeck AS, et al. Better Health Systems for India's Poor: Findings, Analysis, and Options. Washington D.C.: The World Bank; 2002. [32] Ranson MK, Sinha T, et al. Making health insurance work for the poor: learning from the Self-Employed Women's Association's (SEWA) community-based health insurance scheme in India. Social Science and Medicine 2006;62(3):707–20. [33] Sahn DE, Stifel D. Exploring alternative measures of welfare in the absence of expenditure data. Rev Income Wealth 2003;49(4):463–89. [34] Schwartz JB, Bhushan I. Cambodia: using contracting to reduce inequity in primary health care delivery. Reaching the Poor Program Paper, No. 3. Washington, DC: The World Bank; 2004. p. 23. [35] Sen A. On Economic Inequality. Delhi: Oxford India Paperbacks; 1998. [36] Sinha T, Ranson MK, et al. Barriers to accessing benefits in a community-based health insurance scheme: lessons learnt from SEWA Insurance, Gujarat. Health Policy and Planning 2006;21(2):132–42. [37] Soeters R, Griffiths F. Improving government health services through contract management: a case from Cambodia. Health Policy Plan 2003;18 (1):74–83. [38] Thiede M, Palmer N, et al. South Africa: who goes to the public sector for voluntary HIV/AIDS counseling and testing? Reaching the Poor Program Paper, vol. 6. Washington, DC: The World Bank; 2004. p. 14. [39] Tudor Hart J. The inverse care law. Lancet February 27 1971:405–12. [40] United Nations Development Programme. Website of the International Poverty Centre. http://www.undp-povertycentre.org/md-poverty/2006 Accessed 10 January. [41] Varnell SP, Murray DM, et al. Design and analysis of group-randomized trials: a review of recent practices. Am J Public Health 2004;94 (3):393–9. [42] Wagstaff A. Inequalities in health in developing countries: swimming against the tide? Policy Research Working Paper, vol. 2795. Washington, DC: The World Bank; 2002. p. 40. [43] Wagstaff A. Poverty and health sector inequalities. Bull World Health Org 2002;80(2):97–105. [44] Wagstaff A, Paci P, et al. On the measurement of inequalities in health. Soc Sci Med 1991;33(5):545–57.
390
S.S. Morris et al. / Contemporary Clinical Trials 28 (2007) 382–390
[45] Whitehead M, Dahlgren G, et al. Developing the policy response to inequities in health: a global perspective. In: Evans T, Whitehead M, Diderichsen F, Bhuiya A, Wirth M, editors. Challenging Inequities in Health: From Ethics to Action. Oxford: Oxford University Press; 2001. p. 308–24. [46] World Bank. World Development Report 2004: Making Services Work for Poor People. Washington, DC: The World Bank; 2003. [47] World Bank (2005). World development report 2006: Equity and development. Washington DC, The World Bank (and Oxford University Press). [48] World Health Organization. The World Health Report 2000: Health Systems: Improving Performance. Geneva: WHO; 2000.
Measuring improved targeting of health interventions to the poor in the context of a community-randomised trial in rural India ☆ Saul S. Morris a,⁎, M. Kent Ranson a , Tara Sinha b , Anne J. Mills a a
London School of Hygiene and Tropical Medicine, London, United Kingdom b Self-Employed Women's Association, Ahmedabad, Gujarat, India Received 15 January 2006; accepted 9 October 2006
Abstract In spite of growing interest in socioeconomic differentials in health outcomes and access to health services, little has been written about methodologies for assessing the impact of equity-enhancing policies or programs. This paper describes three methodological challenges involved in designing a randomised trial with an equity outcome, and how these were met in a trial of alternative strategies to improving the uptake of benefits of a health insurance scheme among its poorest members. The Vimo SEWA trial is nested within a community-based insurance scheme in rural India. While conducting this trial, three methodological problems were encountered: (i) measuring poverty (or “wealth”, or “socioeconomic status”) (ii) assessing beneficiaries against an appropriate reference standard population and (iii) settling on an appropriate equity measure as an outcome indicator. These problems are likely to arise in any policy or program assessment that has an equity outcome. In the Vimo SEWA trial, the socioeconomic status of beneficiaries (claimants) is assessed relative to that of all scheme members living in same sub-district by applying a rapid assessment questionnaire – which reduces to an integrated index of socioeconomic status – to both a random sample of members in each sub-district, and to all claimants. The results are used to estimate the full distribution of socioeconomic status of members in each sub-district, with each member given a rank score between 0 and 100. Interpolation is used to estimate the rank scores of claimants relative to the membership base. The primary outcome measure for the trial is the mean socioeconomic rank score of claimants. In developing country settings, using an index of socioeconomic status is simpler than assessing household income or the value of household consumption. It is also relatively straightforward to compare the socioeconomic status of health program beneficiaries with a relevant reference population, although two independent surveys are required. Expressing relative wealth on a scale from zero to 100 is conceptually appealing, and the mean value of this rank score provides an equity-specific outcome measure readily integrated into the usual analytic framework for cluster-randomised trials. © 2006 Elsevier Inc. All rights reserved. Keywords: Equity; Outcome measures; Cluster-randomised trials; India
☆
Financial support for this study was provided by the Wellcome Trust. ⁎ Corresponding author. E-mail address: [email protected] (S.S. Morris).
1551-7144/$ - see front matter © 2006 Elsevier Inc. All rights reserved. doi:10.1016/j.cct.2006.10.008
S.S. Morris et al. / Contemporary Clinical Trials 28 (2007) 382–390
383
1. Introduction Many recent reviews have highlighted the dramatic inequities in health outcomes between and within countries [17,48,12,42,43,46,18]. For example, mortality rates of children under 5 in the least-developed countries are 159 per 1000 births compared with 6 per 1000 births in high-income countries [7], and in India, the poorest 20% of the population has more than double the mortality rates, malnutrition, and fertility of the richest 20% [31]. Inequitable access to health care goods and services has long been recognised as a key determinant of these differentials — as long ago as 1971, Tudor Hart formulated the “inverse care law”, which states that “the availability of good medical care tends to vary inversely with the need for it in the population served” [39]. For example, poor children in poor countries are typically far less likely to be immunised than better-off children, and the poor are less likely than the non-poor to have access to high-impact health services, such as skilled antenatal and delivery care [47]. 2. Rationale for randomised trials with equity endpoints Building an appropriate policy response to health inequities requires, among other things, undertaking health equity impact assessments — “policies and programmes in a wide range of sectors must be subjected to such assessments so that ‘unhealthy policies’ can be identified and ‘healthier’ ones developed” ([45], p.311). While there is a large and expanding literature on the measurement of differentials between groups and individuals in wealth [8,35] and health [44,1], little has been written about methodologies for assessing the impact of equity-enhancing policies or programs. Wagstaff identifies four approaches that can shed light on the impacts of equity-enhancing policies on health inequalities [42,43]: cross-country (cross-sectional) comparative studies; country-based before-and-after studies with controls; benefit–incidence analysis; and decomposition analysis. In spite of their clear advantages for inferring causality, only a handful of evaluations of equity-enhancing policies or programs has used randomised methodologies. In the Netherlands a six year program was launched in 1994 to evaluate 12 small-scale interventions designed to reduce health inequalities, using experimental and control groups [23]. In Cambodia, the national government has experimented with contracting of management and delivery of health services on a pilot basis, carefully evaluating the experience with a before–after (1997–2001), randomised controlled trial design [2,37,34]. Two different models involving contracting of NGOs to manage health services at the district level were compared with a control group of directly managed government districts. Primary outcome indicators included targeting of health services to the poorest half of households. The results of the final survey indicated that the contracted districts performed better than the control districts with respect to most of the health service equity indictors. A similar study design has been used in Nicaragua to evaluate the impact of providing rural households with cash in exchange for keeping up-to-date with preventive health care check-ups [25]; the proportion of children under three years of age whose weight was monitored over the previous six months increased between 2000 and 2002 by 24 percentage points among the extreme poor, 12 percentage points among the poor, and 10 percentage points among the non-poor. It is possible that one of the reasons for the paucity of randomised trials with equity endpoints is the lack of methodological guidance for appropriate study design. In the following section, we introduce the context of a community-randomised trial (CRT) nested within a community-based health insurance scheme in rural India. We then describe three methodological challenges involved in designing a trial with an equity outcome, and how these challenges were tackled in the Indian CRT. 3. The Vimo SEWA equity of health insurance trial Since 1992, the Self Employed Women's Association (SEWA) – an Indian trade union for poor women working in the informal sector – has been providing voluntary assets, life and hospitalisation insurance, in a single policy, to its members and their families through Vimo SEWA (SEWA Insurance). In 2005, the premium for the least expensive policy was Rs. 100 (USD 2.3) per annum, covering the costs of inpatient care to a maximum of Rs. 2000 (USD 46) per annum. In the same year, more than 130,000 women, men and children were enrolled in rural and urban areas of the state of Gujarat. The scheme was accessible to the poorest member households, and utilization (submission of claims for hospitalisation) was equitable in Ahmedabad City, but in rural areas the financially better-off were significantly more likely to submit claims than were the poorest [32]. A variety of factors – including travel costs, poor health and transportation infrastructure, and inadequate knowledge of the scheme's benefits and processes – prevented the poorest in rural areas from accessing inpatient care or from submitting an insurance claim [36].
384
S.S. Morris et al. / Contemporary Clinical Trials 28 (2007) 382–390
Since 2004, a community-randomised trial has been seeking to improve the equity impact of Vimo SEWA in rural areas by reducing the differential in claims submission between poor and less poor members. The trial is investigating the impact of three interventions on access to scheme benefits: (i) after-sales service with supportive supervision; (ii) prospective reimbursement of hospital charges; and (iii) a combination of the two. The equity impact of the interventions will be assessed by comparing the distribution of socioeconomic status of claimants with that of the scheme's membership base in the same sub-district, and will be reported elsewhere. No previous randomised trial has assessed interventions aimed at improving the distributional impact of a health insurance scheme in a developing country. 4. Methodological challenges and how they were addressed 4.1. Identifying the poorest individuals Approaches to measuring poverty have been heavily influenced by the disciplinary background of different groups of researchers, and by geographic region characteristics. In the United Kingdom and other European countries, there has been a strong tradition of analyzing occupational class differentials in health outcomes [9], whereas in the United States there has been a preoccupation with race [21,22]. Microeconomists have developed robust methodologies for assessing direct measures of household welfare based on the parallel concepts of income and consumption (the latter term referring to the monetary value of all goods and services consumed by a household over a given period of time) [8]. Both income and consumption methodologies have been extensively applied to assess the distribution of health service users by socioeconomic status in developed and developing countries. However, income and consumption measures are problematic to assess in poor countries [13]. In such environments, incomes can be highly seasonal, and it may be difficult to separate clearly business transactions from consumption transactions. For those who are self-employed, and especially for rural households whose income comes largely from self-employment in agriculture, a huge number of different transactions need to be accounted for to determine net income, and respondents may find it difficult to recall all the values involved, or not wish to divulge the details of their incomes. And even after years of methodological development, some transactions, such as dowry receipts, for example, are hard to categorise in a household framework of accounts. Assessing consumption suffers from many similar difficulties, such as seasonality and unclear boundaries between business expenditures and consumption expenditures. Both approaches require very detailed disaggregated information collection to prevent underreporting, commonly taking up to an hour or more per household to administer, and the questions asked are often perceived to be invasive [27]. Recently, there has been a large body of literature proposing alternative measures that are simpler to operationalise [14,26,16,33]. Most of these methods rely on the observation that wealthier households are more likely to own a range of consumer durables and to benefit from utilities such as electricity or running water. They involve constructing an index of such assets and services, which is used to rank households according to their welfare levels. However, the precise values taken have no ready interpretation and because of the limited number of domains represented, some authors have questioned whether such measures can really be considered a valid representation of socioeconomic status [28,27]. For the Vimo SEWA trial, we developed a customized index of socioeconomic status. The study baseline questionnaire (see online Appendix) – used to assess socioeconomic status among the general, rural population (Survey I) – was based on a generic survey tool developed by the Consultative Group to Assist the Poorest (CGAP) and the International Food Policy Research Institute (IFPRI) to measure the poverty of microfinance clients [20]. Henry et al. compiled from the literature an exhaustive list of indicators of human, material and social capital, as well as indicators related to the fulfillment of basic needs, and then pared down the list based on eight pre-determined criteria to create a generic questionnaire. This questionnaire was modified to suit the rural Gujarati context by, for example, identifying the food grains most commonly stored. The modified questionnaire had modules on: family size and composition; quality of housing; type, number and value of assets, and level of school education and occupation of household members; hunger episodes and types of food eaten, and household expenditure on clothing and footwear. A shorter questionnaire was then used to assess the socioeconomic status of Vimo SEWA members (Survey II) and health insurance claimants (Survey III). This “rapid assessment” questionnaire included only a subset of the questions asked in Survey I — the questions necessary for a summary index based on indicators that most strongly distinguished relative levels of socioeconomic status, as described in the following paragraphs. The indicator variables retained in the questionnaire are shown in Table 1.
S.S. Morris et al. / Contemporary Clinical Trials 28 (2007) 382–390
385
Table 1 Indicators included in summary index of socioeconomic status Human • The percentage of household adults who can read and write (continuous) resources • The percentage of household adults whose maximum level of schooling was “attended college or university” (continuous) Dwelling • Number of rooms, excluding kitchen (continuous) • Whether the home's walls are made of “brick or stone with plaster” (dichotomous) • Whether the home's walls are made of materials other than brick or stone (dichotomous) • Whether the household has no electrical connection, shared connection, or its own connection (categorical) • Whether natural gas is the primary cooking fuel used (dichotomous) Food • During the last year, when cooking oil stores were highest, whether there was sufficient stock to last 1 month (dichotomous) security • During the last year, when millet or millet flour stores were highest, whether there was sufficient stock to last 12 months (dichotomous) • During the last year, when wheat or wheat flour stores were highest, whether there was sufficient stock to last 1 month (dichotomous) Assets • Number of refrigerators (continuous) • Number of electric fans (continuous) • Number of mattresses (continuous) • Number of wrist watches (continuous)
Principal Components Analysis (PCA) using Stata 7.0 (Stata Corporation, College Station, TX) was applied to the socioeconomic data to derive a single linear variable incorporating as much as possible of the information that was common to the various individual indicators. In order to do this, each indicator was standardised on the same scale (mean zero, standard deviation one), and these quantities were weighted – using empirically derived weights from the PCA – and summed. The resulting index can be represented by the formula [15]: Aj ¼ f1 d ða1j − ¯a 1 Þ=s1 þ N þ fN d ðaNj − ¯a N Þ=sN where j subscripts households, a1 to aN are the various assets, fn is the weight for the nth asset, ā indicates the mean value of an asset (over all households), and s the standard deviation. The summary index A is itself expected to have mean zero and standard deviation one, and is interpreted as an integrated measure of socioeconomic status. Thus, an index value of + 2 indicates very high socioeconomic status, whereas a value of − 2 indicates very low status, and a value of zero indicates intermediate levels. Indicator variables correlating only weakly with the summary index were dropped from the analysis at an early stage (and not included in the questionnaire for surveys II and III). These variables included total household size, which proved less discriminatory in this context than a set of variables related to educational achievement. 4.2. Assessing beneficiary distributions against an appropriate reference standard When health interventions are made available to all members of a national population, the socioeconomic status of those who use the service can be compared to an appropriate national standard. National distributions of socioeconomic status are usually available from household survey series that are commonly updated every few years (or even yearly) in most countries. For example, in Great Britain, a small proportion of individuals use private hospitals, and various indicators of their socioeconomic status, measured in facility surveys or administrative records, can be compared to the profile of the national population, which is assessed annually by the Office of National Statistics (www.statistics.gov. uk). Similarly, in India, the socioeconomic status of users of public health services in a given state can be compared with state-level representative data from surveys such as the National Family Health Survey [30]. However, many services are not available to entire populations. For example, in South Africa there are a number of programs that provide voluntary counselling and testing for HIV services to individuals living in particular cities. Whilst it may be possible to know the socioeconomic status of program beneficiaries, there is rarely adequately disaggregated data available to describe the distribution of socioeconomic status in the reference cities, and data for all urban areas combined have been used instead [38]. Another relevant example concerns the assessment of the socioeconomic status of beneficiaries of insurance schemes (i.e. successful claimants); in this context, it would be important to compare the resulting distribution of socioeconomic status to that of program members only, since nonmembers could never benefit from the scheme.
386
S.S. Morris et al. / Contemporary Clinical Trials 28 (2007) 382–390
In the Vimo SEWA trial, the socioeconomic status of claimants was contrasted with that of the membership base in their sub-districts of residence. This protects against inappropriate inferences that would result, for example, from the comparison of rural claimants with urban members. In order to permit the relevant comparisons, representative surveys of scheme member households were undertaken in the sixteen study sub-districts at baseline (2003) and again postintervention (2005), to determine the distribution of socioeconomic status in each year and sub-district. Based on these surveys, the membership base in each sub-district was ranked on socioeconomic status (using the summary index described above), so that the poorest member household was ranked zero, the wealthiest ranked 100, and the household in the middle of the range ranked 50. More specifically, because the survey was based on a sample of size k from each sub-district, it was assumed that the lowest observed value of socioeconomic status in each sub-district represented not the absolute minimum possible value, but rather centile 1 / (k + 1) * 100 of the sub-district specific distribution of members. Similarly, the highest observed value was taken to represent centile k / (k + 1) * 100 rather than the absolute maximum. Fig. 1A illustrates the association, for one sub-district, between socioeconomic index values (on the y-axis) and socioeconomic rank scores (on the x-axis). Following the characterization of the membership base in each sub-district, a separate survey of claimant households was undertaken, and a socioeconomic index value for each household was again calculated as described above. Socioeconomic index values were then converted into sub-district specific rank scores (an ordinal scale with a range of zero to 100) by linear interpolation based on the rank of the member households with the closest (i.e. the next lowest and the next highest) socioeconomic index values. Fig. 1B shows all of the claimants in one sub-district interspersed with the members in the same sub-district. Fig. 1C illustrates how the socioeconomic index values for each claimant were converted into rank scores on the basis of linear interpolation. Claimants whose socioeconomic index values were lower or higher than those of any sampled member in the same sub-district were assigned rank scores of 0 and 100, respectively. The socioeconomic rank is thus a scale, specific to each sub-district, that indicates the relative socioeconomic status of members and claimants. Scores on the scale provide no information about the absolute difference in socioeconomic
Fig. 1. Within-cluster rank versus untransformed socioeconomic index value for members (1A), members plus claimants (1B), and illustrating the linear interpolation of within-cluster centile of socioeconomic status for one claimant (1C) (Bayad sub-district, 2003).
S.S. Morris et al. / Contemporary Clinical Trials 28 (2007) 382–390
387
status between one point and another. That is, the absolute difference in wealth between households scored 5 and 10 may be entirely different from households scored 90 and 95 in the same sub-district, or between households scored 5 and 10 in a different sub-district. The socioeconomic rank score is well suited for assessing the distribution of a beneficiary population relative to a reference population, and monitoring changes in this distribution over time. 4.3. Selecting an appropriate trial outcome measure Recent work has discussed a range of statistical measures that can be used to summarise the distributional (equity) outcomes of health interventions [44,1]. Commonly used measures include: the proportion of health program benefits accruing to the poorest population groups; the relative frequency of a health program benefit or health outcome in the least poor group compared to the frequency in the poorest group, and the concentration index, which measures the covariance between the socioeconomic welfare ranks of individuals in a population and the size of the benefits that they receive [1]. This measure conveniently varies from zero (perfect equality in the distribution of benefits) to one (all benefits accrue to a single person). However, despite its conceptual elegance, our experience has been that most policymakers find the concentration index difficult to grasp. Both proportions of benefits accruing to the poorest, and rich:poor ratios, are sensitive to choice of cut-off value distinguishing the poor from the less poor. Yet in many situations, it may not be obvious what the optimal cut-off value should be. For example, many studies choose to look at the ratio of benefits accruing to the least-poor 20% of the reference population relative to the poorest 20% [6,24,18], but it may be equally valid to calculate rich:poor ratios comparing the least-poor and poorest 10%, 40% (as in Ref. [30]), or 50% (as in Ref. [34]). Other studies examine the benefits accruing to those below an appropriately defined poverty line [32]. But this creates additional problems of interpretation in a longitudinal study where the composition of the reference population is itself changing. Further complexities arise in the context of a cluster-randomised trial. In this context, the selected outcome measure will be quantified in every trial cluster, ruling out any measure which is unduly demanding in terms of sample size. Furthermore, an efficient trial analysis, utilizing information on both between- and within-cluster variance [10,19,3– 5,11,29,41], is only possible when the outcome measure has a clearly defined variance function. In the case of the concentration index, straightforward estimation of the variance function is possible only if data on both beneficiaries and the reference population are drawn from the same survey. This, as we have seen above, is not always feasible. The primary endpoint of the Vimo SEWA trial was determined to be the mean socioeconomic rank score of insurance claimants relative to their local membership base. This parameter has the advantages of: varying on a zero to 100 scale, with a value of 50 indicating an equity-neutral outcome; avoiding arbitrary cut-offs dividing the ‘poor’ from the ‘non-poor’ and being equally sensitive to changes in benefit uptake among poorer and wealthier members; being insensitive to changes over time in the socioeconomic status of the entire membership base that are exactly paralleled in the socioeconomic status of claimants; being simple to calculate, with a straightforward variance function; and demonstrating relatively limited cluster-to-cluster variability at baseline. The sample size calculations used in the trial assumed that, as the barriers to claiming benefits faced by the poorest scheme members were reduced by the interventions, the mean socioeconomic rank score of claimants relative to the local membership base would fall by 20 points. This was felt to be the minimum change that would make the distribution of benefits unequivocally ‘pro-poor’. 5. Discussion This paper has presented three methodological problems that are likely to arise in any policy or program assessment that has an equity outcome: (i) measuring poverty (or “wealth”, or “socioeconomic status”); (ii) assessing beneficiaries against an appropriate reference standard population; and (iii) settling on an appropriate equity measure as an outcome indicator. In a cluster-randomised trial which aims to optimise the equity impact of Vimo SEWA's community-insurance scheme, these challenges were addressed by: (i) developing a short socioeconomic index based on indicators that were found to distinguish relative levels of socioeconomic status most strongly; (ii) conducting purpose-specific surveys of both the member (target) and claimant (beneficiary) population, both at baseline and follow-up; and (iii) using as a primary outcome measure the mean socioeconomic rank score of claimants relative to the membership base in each intervention cluster (sub-district). Given that this is one of very few randomised and controlled public health trials that has had an equity issue as a primary outcome, useful lessons can be drawn from this experience.
388
S.S. Morris et al. / Contemporary Clinical Trials 28 (2007) 382–390
Weaknesses of the study methodology are several. First, the socioeconomic index, as a measure of poverty, is somewhat abstract, and can be difficult for people to understand vis-à-vis the more familiar concepts of income or consumption, especially because it is not expressed on a money metric. Second, the methodology was logistically challenging in that linking beneficiaries to the target population required multiple surveys — member and claimant surveys, both at baseline and follow-up. However, a single survey of members and the claimants among them would have had to be huge (given that only 2% of members submit a health insurance claim each year) in order to achieve the necessary claimant sample size. Third, like the measure of poverty, the primary outcome measure is somewhat abstract relative to, for example, health service utilization or mortality. Fourth, because the chosen endpoint is not correlated with anything else at the cluster level, the measures we took to ensure treatment group balance in this outcome measure (a priori stratification) did not result in groups that were balanced in anything else (e.g. population size of intervention areas, average socioeconomic status of members, rates of illness or hospitalisation). This is a problem in that: (i) after looking at our primary outcome, we would like to look at secondary outcomes which may not have been balanced at baseline; and (ii) these imbalances directly impact the face validity of the trial. On the other hand, there are compensating methodological strengths. First, the summary index of socioeconomic status is quite simple and transparent in that it is constructed from data on only fourteen variables. In presenting results (for example, differences in socioeconomic status between claimants and members) one can present not just the mean index values but also mean values for the fourteen underlying variables. Second, the survey instrument that is used to generate the index is short and simple, and as a result can be administered by less-skilled interviewers, and in an interview of only 10 to 15 min. Third, the instrument is strong in its content validity — it has a strong theoretical basis and a systematic approach to indicator selection. While the underlying CGAP tool is “theoretically eclectic”, the authors “carefully justify all of their recommendations on domains to be included in their index and data reduction” [27]. The tool (and corresponding methodology) is locally appropriate – incorporating variables that might be important indicators of socioeconomic status in a particular study setting – but it is also readily generalizable and has been field-tested in four different countries [20]. Finally, the outcome indicator used has certain benefits. It is relatively simple to interpret because it varies on a scale from 0 to 100, with 50 indicating an equity-neutral outcome. It is also efficient as it draws on data from claimants and members of all levels of socioeconomic status. If we had instead, for example, used “percentage of claimants falling below the 10th centile of socioeconomic status” or “falling below the poverty line”, much larger sample sizes would have been required. Methodological research on assessing the equity impacts of health interventions is still in its infancy. The conceptualization of poverty is constantly evolving [40], but this body of work urgently needs distilling around valid, operational measures. Once such measures are agreed upon, researchers will need guidance about the magnitude of impacts that are plausible from different types of interventions, in order that they can implement further randomised trials with adequate statistical power. New theoretical statistical work may be required to demonstrate how these outcome measures can be implemented within a cluster-randomised trial framework, as it is most unlikely that equityenhancing interventions would ever be allocated on an individual basis. Finally, the relevance of equity impact findings needs to be presented clearly to policy-makers in order to stimulate interest in the scaling-up of these critical interventions. Appendix A. Supplementary data Supplementary data associated with this article can be found, in the online version, at doi:10.1016/j.cct.2006.10.008. References [1] Anand S, Diderichsen F, et al. Measuring disparities in health: methods and indicators. In: Evans T, Whitehead M, Diderichsen F, Bhuiya A, Wirth M, editors. Challenging Inequities in Health: From Ethics to Action. Oxford: Oxford University Press; 2001. p. 48–67. [2] Bhushan I, Keller S, et al. Achieving the twin objectives of efficiency and equity: contracting health services in Cambodia. Economics and Research Department Policy Brief No. 6 Manila: Asian Development Bank; 2002. p. 10. [3] Campbell MK, Elbourne DR, et al. CONSORT statement: extension to cluster randomised trials. BMJ 2004;328:702–8 [(20 March 2004)]. [4] Campbell MK, Grimshaw JM, et al. Intracluster correlation coefficients in cluster randomized trials: empirical insights into how they should be reported. BMC Medical Research Methodology, vol. 4(9); 2004. [5] Campbell MK, Thomson S, et al. Sample size calculator for cluster randomized trials. Comput Biol Med 2004;34:113–25.
S.S. Morris et al. / Contemporary Clinical Trials 28 (2007) 382–390
389
[6] Castro-Leal F, Dayton J, et al. Public spending on health care in Africa: do the poor benefit? Bull World Health Org 2000;78(1):66–74. [7] Commission on Macroeconomics and Health. Macroeconomics and Health: Investing in Health for Economic Development. Geneva: World Health Organization; 2001. [8] Deaton A. The Analysis of Household Surveys: a Microeconometric Approach to Development Policy. Washington DC: The World Bank; 1997. [9] Diderichsen F, Whitehead M, et al. Sweden and Britain: the impact of policy context on inequities in health. In: Evans T, Whitehead M, Diderichsen F, Bhuiya A, Wirth M, editors. Challenging Inequities in Health: From Ethics to Action. Oxford: Oxford University Press; 2001. p. 240–55. [10] Donner A. Some aspects of the design and analysis of cluster randomization trials. Appl Stat 1998;47:95–113 [Part 1]. [11] Donner A, Klar N. Pitfalls of and controversies in cluster randomization trials. Public Health Matters 2004;94(3):416–22. [12] Evans T, Whitehead M, et al, editors. Challenging Inequities in Health: From Ethics to Action. Oxford: Oxford University Press; 2001. [13] Falkingham J, Namazie C. Measuring Health and Poverty: a Review of Approaches to Identifying the Poor. London, UK: DfID Health Systems Resource Centre; 2002. [14] Filmer D, Pritchett L. The effect of household wealth on educational attainment: evidence from 35 countries. Popul Dev Rev 1999;25 (1):85–120. [15] Filmer D, Pritchett LH. Estimating Wealth Effects Without Expenditure Data- or Tears: With an Application to Educational Enrollments in States of India. Washington DC: the World Bank; 1998. p. 38. [16] Filmer D, Pritchett LH. Estimating wealth effects without expenditure data- or tears: an application to educational enrollments in states of India. Demography 2001;38(1):115–32. [17] Gwatkin DR. Health inequalities and the health of the poor: what do we know? What can we do? Bull World Health Org 2000;78(1):3–18. [18] Gwatkin DR, Bhuiya A, et al. Making health systems more equitable. Lancet 2004;364:1273–80 [(October 2)]. [19] Hayes RJ, Bennett S. Simple sample size calculation for cluster-randomized trials. Int J Epidemiol 1999;28:319–26. [20] Henry C, Sharma M, et al. Assessing the Relative Poverty of Microfinance Clients: a CGAP Operational Tool. Wasington, D.C.: International Food Policy Research Institute; 2000. [21] Krieger N, Fee E. Measuring social inequalities in health in the United States: a historical review, 1900–1950. Int J Health Serv 1996;26 (3):391–418. [22] Krieger N, Williams DR, et al. Measuring social class in US public health research. Annu Rev Public Health 1997;18:341–78. [23] Mackenbach JP, Stronks K. A strategy for tackling health inequalities in the Netherlands. Br Med J (BMJ) 2002;325:1029–32. [24] Makinen M, Waters H, et al. Inequalities in health care use and expenditures: empirical data from eight developing countries and countries in transition. Bull World Health Org 2000;78(1):55–65. [25] Maluccio J, Flores R. Impact evaluation of a conditional cash transfer program: the Nicaraguan Red de Protección Social. FCND Discussion Paper, No. 184. Washington DC: International Food Policy Research Institute; 2004. [26] Montgomery MR, Gragnolati M, et al. Measuring Living Standards with Proxy Variables. New York: Population Council; 1999. p. 64. [27] Morris SS. Epidemilogy and the study of socio-economic inequalities in health. Measuring Health Equity in Small Areas-Findings from Demographic Surveillance Systems. INDEPTH Network. Aldershot, UK: Ashgate; 2005. p. 1–18. [28] Morris SS, Carletto C, et al. Validity of rapid estimates of household wealth and income for health surveys in rural Africa. J Epidemiol Community Health 2000;54:381–7. [29] Murray DM, Varnell SP, et al. Design and analysis of group-randomized trials: a review of recent methodological developments. Am J Public Health 2004;94(3):423–32. [30] Peters D, Rao K, et al. India: Equity effects of quality improvements on health service utilization and patient satisfaction in Uttar Pradesh State. Reaching the Poor Program Paper, vol. 5. Washington, DC: The World Bank; 2004. p. 18. [31] Peters DH, Yazbeck AS, et al. Better Health Systems for India's Poor: Findings, Analysis, and Options. Washington D.C.: The World Bank; 2002. [32] Ranson MK, Sinha T, et al. Making health insurance work for the poor: learning from the Self-Employed Women's Association's (SEWA) community-based health insurance scheme in India. Social Science and Medicine 2006;62(3):707–20. [33] Sahn DE, Stifel D. Exploring alternative measures of welfare in the absence of expenditure data. Rev Income Wealth 2003;49(4):463–89. [34] Schwartz JB, Bhushan I. Cambodia: using contracting to reduce inequity in primary health care delivery. Reaching the Poor Program Paper, No. 3. Washington, DC: The World Bank; 2004. p. 23. [35] Sen A. On Economic Inequality. Delhi: Oxford India Paperbacks; 1998. [36] Sinha T, Ranson MK, et al. Barriers to accessing benefits in a community-based health insurance scheme: lessons learnt from SEWA Insurance, Gujarat. Health Policy and Planning 2006;21(2):132–42. [37] Soeters R, Griffiths F. Improving government health services through contract management: a case from Cambodia. Health Policy Plan 2003;18 (1):74–83. [38] Thiede M, Palmer N, et al. South Africa: who goes to the public sector for voluntary HIV/AIDS counseling and testing? Reaching the Poor Program Paper, vol. 6. Washington, DC: The World Bank; 2004. p. 14. [39] Tudor Hart J. The inverse care law. Lancet February 27 1971:405–12. [40] United Nations Development Programme. Website of the International Poverty Centre. http://www.undp-povertycentre.org/md-poverty/2006 Accessed 10 January. [41] Varnell SP, Murray DM, et al. Design and analysis of group-randomized trials: a review of recent practices. Am J Public Health 2004;94 (3):393–9. [42] Wagstaff A. Inequalities in health in developing countries: swimming against the tide? Policy Research Working Paper, vol. 2795. Washington, DC: The World Bank; 2002. p. 40. [43] Wagstaff A. Poverty and health sector inequalities. Bull World Health Org 2002;80(2):97–105. [44] Wagstaff A, Paci P, et al. On the measurement of inequalities in health. Soc Sci Med 1991;33(5):545–57.
390
S.S. Morris et al. / Contemporary Clinical Trials 28 (2007) 382–390
[45] Whitehead M, Dahlgren G, et al. Developing the policy response to inequities in health: a global perspective. In: Evans T, Whitehead M, Diderichsen F, Bhuiya A, Wirth M, editors. Challenging Inequities in Health: From Ethics to Action. Oxford: Oxford University Press; 2001. p. 308–24. [46] World Bank. World Development Report 2004: Making Services Work for Poor People. Washington, DC: The World Bank; 2003. [47] World Bank (2005). World development report 2006: Equity and development. Washington DC, The World Bank (and Oxford University Press). [48] World Health Organization. The World Health Report 2000: Health Systems: Improving Performance. Geneva: WHO; 2000.