- Email: [email protected]
Choosing the method to match the perspective: economic assessment and its implications for health-services efficiency
VIEWPOINT
Viewpoint
Choosing the method to match the perspective: economic assessment and its implications for health-services efficiency Adam Oliver, Andrew Healey, Cam Donaldson Consideration of evidence from economic assessment of health-care interventions is becoming important in several countries.1–4 The UK has established the National Institute of Clinical Excellence (NICE; www.nice.nhs.uk), which requires manufacturers or sponsors of health-care interventions to submit evidence on the clinical and cost-effectiveness of many products and services. NICE uses the submitted evidence to formulate recommendations about the interventions that they decide ought to be available within the National Health Service (NHS). The government hopes that the NICE recommendations will help to end “postcode rationing” (ie, geographical disparities in health-services provision) and increase the amount of health gain attributable to NHS resources. NICE has adopted the NHS as its perspective. Therefore, NICE should operate within the framework of the existing NHS budget and ought to have no influence over whether or not the NHS is underfunded (or overfunded). Moreover, of the three commonly defined types of economic assessment—cost-effectiveness analysis, cost-utility analysis, and cost-benefit analysis5— NICE encourages manufacturers and sponsors to use cost-effectiveness analysis or cost-utility analysis.6 Costbenefit analysis is not encouraged because of its limited application in the context of health care. However, NICE—and the health economic and public policymaking communities more generally—do not seem to recognise that the type of economic assessment chosen depends on the type of efficiency question being asked, rather than the ease with which outcomes are measured and valued.7,8 In this paper, we analyse the implications for NICE of preferring cost-effectiveness analysis and cost-utility analysis in the assessment of NHS improvements in productive efficiency (the amount of health gain generated by replacing one intervention with another for the same or lower cost) and allocative efficiency (the amount of health gain or welfare gain generated by replacing one intervention with another at greater cost).
The limitation of cost-effectiveness analysis The cost-effectiveness approach Cost-effectiveness analysis is the most common type of economic assessment,9 probably because the outcome data (effectiveness) in such analyses are relatively simple Lancet 2002; 359: 1771–74 LSE Health and Social Care, London School of Economics and Political Science, London WC2A 2AE, UK (A Oliver MSc, A Healey MSc); Departments of Economics and Community Health Sciences, University of Calgary, Calgary, Alberta, Canada (Prof C Donaldson PhD) Correspondence to: Mr Adam Oliver (e-mail: [email protected])
THE LANCET • Vol 359 • May 18, 2002 • www.thelancet.com
to calculate, and often comprise clinical measures of health (eg, percentage reduction in blood pressure, reduced incidence of hip fracture) that are routinely collected in clinical trials. In an effort to improve comparability across studies, generic measures of outcome are sometimes used, such as life-years saved or disability-days avoided. However, even if these measures are routinely collected in clinical trials and thus readily available for the health economist, they will frequently be inappropriate for the comparative analysis of different health-care interventions. For example, outcomes measured by years of life saved will be irrelevant for programmes that improve the quality of life but do not extend the length of life. Because the outcome measures generally used in costeffectiveness analyses are not comparable across a broad array of health-care programmes, the method is appropriate only for addressing questions of productive efficiency. Indeed, some argue that cost-effectiveness analysis is useful only for comparison of treatment options within a given budget and, therefore, a specific group of patients.7 In this context, if a new treatment is deemed “cost-effective”, it can be substituted for existing care without asking any other group of patients to give up resources. The same group is treated, but by a different method from before. Allocative efficiency within the NHS, which involves deciding how much to allocate to different groups of patients, cannot usually be determined by cost-effectiveness analyses. This point can be illustrated as follows: The misapplication of cost-effectiveness analysis Consider an illness, say osteoporosis, for which there is an existing treatment, X. A new treatment, Y, is being proposed. A cost-effectiveness analysis of treatment Y is undertaken to help determine whether it should replace treatment X. A cost-effectiveness ratio for treatment Y is usually calculated as: (CY–CX)/(EY–EX), where CX, CY, EX, and EY denote the costs and outcomes (effectiveness) of interventions X and Y, respectively. Assume that the outcome of treatment is the number of hip fractures prevented. Further assume that treatment X costs £2 million and prevents 1000 hip fractures annually, and that treatment Y costs £1·5 million and prevents 1500 hip fractures. The new treatment is more effective and less costly than the existing treatment. The introduction of treatment Y in place of treatment X thus offers an improvement in productive efficiency—ie, by introducing treatment Y, the health service would deliver greater health improvements for the same level of resources. Although treatment Y is likely to be introduced without too much deliberation, the cost-effectiveness ratio of –£1000 per hip fracture prevented gives an indication of the degree of productive efficiency that the new treatment offers. For example, had treatment Y cost £1·75 million, the cost-effectiveness ratio would be –£500 1771
For personal use. Only reproduce with permission from The Lancet Publishing Group.
VIEWPOINT
per hip fracture prevented, implying half as much improvement in productive efficiency compared with a situation in which treatment Y cost £1·5 million. In the circumstances described, the introduction of treatment Y has greater productive efficiency (or is “dominant”) because it is less costly and more effective than the existing treatment. Treatment Y would also offer an improvement in productive efficiency if it costs the same and is more effective, or if it costs less and is equally effective. In all of the above circumstances, the introduction of treatment Y in place of treatment X would also offer an improvement in allocative efficiency. Unfortunately, most new interventions increase costs.5 Consider again the circumstances above, but assume now that treatment Y costs £2·5 million and still prevents 1500 hip fractures. The increment in costs is £500 000 and the increment in effectiveness is 500 hip fractures prevented; hence, the incremental cost-effectiveness analysis ratio of treatment Y is £1000 per hip fracture prevented. Should we be willing to pay an extra £1000 for each additional hip fracture prevented? When a treatment increases costs, there is no explicit scientific or ethical definition of an acceptable cost-effectiveness ratio. Thus the decision about whether a particular cost-effectiveness ratio renders an intervention a worthwhile use of resources is partly political, involving a comparison of the gain of investing in the new treatment with what would be given up (ie, the opportunity cost) by funding it. Conventional wisdom, however, dictates that a treatment with a cost-effectiveness ratio of £1000 per hip fracture prevented would generally be regarded as cost-effective.10 If NICE were to appraise an osteoporosis treatment with that level of cost-effectiveness, it would probably be recommended for use within the NHS. So let us assume that NICE has recommended treatment Y to the geographically defined health authorities (the purchasers of NHS care) in England and Wales. For simplification, assume that there are ten health authorities, each of which bear 10% of the costs and enjoy 10% of the outcomes of treatment X and will be expected to experience an identical percentage and the costs and outcomes of treatment Y. Therefore, each health authority is being asked to pay for a new treatment that will cost them £250 000 (10% of £2·5 million). The old treatment that treatment Y will replace costs them £200 000 (10% of £2 million). By replacing treatment X with treatment Y each health authority has to find an additional £50 000 from somewhere else (say, by disinvesting in treatment Z), but the benefits reaped from treatment Z might outweigh the expected benefits of investing the £50 000 in treatment Y. If so, the health authority managers should ensure that the net benefit of treatment Y over treatment X exceeds the net loss of benefit from forgoing treatment Z. Since Z will be aimed at a different group of patients, this question is fundamentally one of allocative efficiency. The magnitude of costs and health outcomes, estimated in the economic assessment, will play an important part in any decision, as will other factors which were not quantified. Incidentally, given the variations in contexts across health authorities, the implementation of Y would result in different things being given up in different parts of the country, which would result in another type of postcode rationing. NICE might foresee this problem and provide explicit advice to the health authorities on the services they must cut back on to accommodate the new cost-effective but cost-increasing intervention. However, to do this with any reference to explicit decision-making criteria, NICE would have to compare the new intervention with all
1772
interventions that it regarded as expendable with reference to a common measure of outcome (eg, life-years saved). Aside from the practical difficulties of undertaking such an extensive set of comparisons, finding an appropriate common outcome measure with which to compare interventions can often be impossible in the framework of a cost-effectiveness analysis. If explicit guidance based on good assessment is unavailable, NICE could use guesswork to issue guidance about interventions on which to cut back. Alternatively, health authority managers could use their own (unsystematic) judgment to determine which services to terminate or decrease. Either way, treatment Y could be partly financed by the withdrawal of interventions that might offer significant benefit to the population. Indeed, the NICE recommendations, if based on the results of cost-effectiveness analysis, could cause greater allocative inefficiency in the NHS. In short, incremental costeffectiveness ratios for cost-increasing interventions exclude sufficient consideration of opportunity costs, and therefore cannot inform the policy maker about whether the benefits of introducing the intervention outweigh the necessary cutbacks that would have to be made elsewhere. Whither cost-effectiveness analysis? Worsening allocative inefficiency in the NHS could be avoided if the government increased the NHS budget in order to finance the new cost-increasing treatment. If the government were to do this, the health authority managers would not have to worry about cutting back on their services elsewhere. However, to increase the NHS budget, the government would be required either to cut back on other public services (eg, education, housing, defence, or law and order) or to increase taxation. Either measure involves tradeoffs, and cost-effectiveness analysis cannot inform us whether the increase in welfare from more NHS investment outweighs the foregone welfare from reducing investment in other public services or from increasing taxation. Thus, cost-effectiveness analyses cannot provide evidence of whether the NHS budget should be altered. We believe that considerations of allocative efficiency with respect to “welfare” from a societal perspective ought to take prominence over those with respect to “health” from an NHS perspective. In short, cost-effectiveness analysis is a useful framework for assessing productive efficiency within the NHS (ie, in the restrictive set of circumstances where a treatment is dominant). However, the often incomparable outcome measures across studies that use this type of economic assessment render it of limited value for ensuring allocative efficiency when introducing costincreasing treatments. Furthermore, consideration of the potential introduction of cost-increasing treatments inevitably involves comparisons of different groups of patients, which cost-effectiveness analysis is not equipped to do. To address allocative efficiency, even within the limited context of the NHS, we have to search for a measure of outcome that can be applied to most healthcare interventions.
Using “utility” The cost-utility analysis approach Cost-utility analyses are identical to cost-effectiveness analyses, except that the outcomes are measured in terms of a single composite index that combines length of life with quality of life (typically represented by qualityadjusted life-years [QALYs]), and this measure is preference based. For example, if full health is normalised with a quality weight of 1, and, by use of preference
THE LANCET • Vol 359 • May 18, 2002 • www.thelancet.com
For personal use. Only reproduce with permission from The Lancet Publishing Group.
VIEWPOINT
elicitation techniques,5,11 living with multiple sclerosis is found to offer a quality of life that is only 40% that of living in full health (ie, the health state “utility” of multiple sclerosis is 0·4), then 6 months with multiple sclerosis will give 0·2 QALYs (ie, 0·5 [years]⫻0·4). Typically, cost-utility analysis is used to generate incremental cost-utility ratios that, in themselves, have the same flaws as those generated by cost-effectiveness analysis. Namely, an incremental cost-utility ratio derived by comparing a new cost-increasing intervention with an existing comparable intervention does not offer an assessment of all of the opportunity costs associated with the new intervention. Therefore, the introduction of the new intervention necessarily leads to non-assessed cutbacks elsewhere, which might lead to an overall reduction in the benefit of health care. Consequently, ranking interventions according to their incremental costeffectiveness or cost-utility (as commonly estimated) gives a potentially misleading picture of the interventions’ relative worth. NICE must be aware of this limitation in their assessment of incremental ratios from either costeffectiveness analysis or cost-utility analysis, and the compilation of league tables of cost-effectiveness or costutility as a decision aid for policy makers ought to be avoided. However, if we extend the assessment of a new intervention beyond the derivation of the simple incremental ratio, cost-utility analysis can be used to address questions of allocative efficiency in the NHS by allowing a comparison of health outcomes across a range of health interventions.12 Although this is a theoretical ideal, NICE at least ought to recognise explicitly what the relevant trade-offs are. In the example given above, where each health authority has to find an additional £50 000 to fund treatment Y, health authority managers could theoretically measure the number of net QALYs they would gain or lose by forgoing an alternative treatment in some other therapeutic area to provide treatment Y. If treatment Y offers a net gain in QALYs after sacrifices have been considered (ie, opportunity costs in terms of QALYs foregone), then the provision of treatment Y would represent an improvement in allocative efficiency. For a given amount of resources, more QALYs would be produced overall by taking resources from one group and reallocating them to another. Cost-utility analysis represents an improvement on cost-effectiveness analysis and, if used appropriately, could potentially allow NICE to address questions of allocative efficiency as well as productive efficiency in the NHS in a scientific manner. Whither cost-utility analysis? Despite taking us further than cost-effectiveness analysis (providing that the analysis is taken further than the derivation of a simple incremental cost-utility ratio across two treatments), cost-utility analysis has two conceptual limitations. First, QALY techniques place values on health-related quality of life only. This means that other attributes of health care over which people have preferences, such as the location of care, are not accounted for. Techniques of (hypothetical) willingness to pay have been used in surveys to address people’s strengths of preference for different locations of care.13 In only one study has willingness to pay been used to assess people’s preferences for provision of increments in disparate health-care options for the community.14 That study showed that the ordering arising from willingnessto-pay values can be different from that arising from QALY estimation. Second, although cost-utility analysis can be used to
THE LANCET • Vol 359 • May 18, 2002 • www.thelancet.com
improve allocative efficiency in the NHS, how can we deal with questions of allocative efficiency across the whole of the public sector? Would it be beneficial to invest more money in the NHS at the expense of housing, or more money in education at the expense of defence? These questions are tied in with the concept of joined-up government, where different government departments take a more holistic view of what is good for society, rather than how much good their own particular government department has to offer. Since health-state utilities cannot be compared with the outcomes of education, housing, defence, &c, development of broader utility (or “welfare”) measures would be useful. Unfortunately, the development of comparable direct measures of utility across the range of public services seems impossible at present. We seemingly have to look elsewhere for an outcome measure that is appropriate for comparing the benefits of all public sector services.
The role of cost-benefit analysis Cost-benefit analysis has a broader scope than both costeffectiveness analysis and cost-utility analysis, and differs mainly in that its outcome measures are defined in monetary terms. A monetary value can be placed on a health-care treatment (ie, typically described in terms of some measure of health and other characteristics) by, for example, asking people what is the maximum amount they would be willing to pay for the treatment or how much they would be willing to pay to avoid negative characteristics, such as the debilitating effects of an illness. Willingness to pay can be measured with various other techniques.5 By measuring both costs and outcomes in money terms, cost-benefit analysis allows us to estimate the net social benefit of an intervention. Returning to our osteoporosis example, the net social benefit of treatment Y can be calculated in simple terms by: WY–CY–CS, where W is society’s willingness to pay for the benefits offered by treatment Y, CY is the NHS cost of treatment Y, and CS (included if we adopt a societal approach) are the costs associated with treatment Y that transcend NHS costs— eg, the costs of residential convalescence care. Thus, if the willingness to pay for treatment Y is £10 million, the cost of the treatment is £2·5 million and the non-NHS costs are £3 million, the net social benefit of treatment Y measures £4·5 million. By a similar token, suppose that the government is considering a scheme to increase the number of secondary school teachers, which the government projects will improve the pass rate in the national examinations for 16year-olds by 10%. The direct education costs of the scheme are estimated to be £5 million and the wider social costs (eg, an increase in NHS general practitioner visits by teachers who feel more pressure due to the new pass-rate targets) are expected to be £500 000. The willingness to pay for this scheme is £6 million; the net social benefit of the scheme is therefore estimated at £500 000. The monetary valuation of outcomes allows us to compare the net social benefit of social programmes as disparate as osteoporosis treatment and a scheme to increase the number of secondary school teachers. Therefore, if the government intended to introduce a health intervention that was calculated to have a net social benefit of £10 million, but had to forego either treatment Y or the education scheme to finance this new intervention, the simple hypothetical example outlined above suggests that abandonment of the scheme to increase the number of secondary school teachers would be sensible in terms of allocative efficiency. Theoretically,
1773
For personal use. Only reproduce with permission from The Lancet Publishing Group.
VIEWPOINT
cost-benefit analysis can be used to help determine whether the NHS is underfunded or overfunded. As with health state utilities, there are conceptual problems with cost-benefit analyses in that many decision makers might object to monetary values being placed on something as fundamental as health (although these decision makers, through making choices, will be placing monetary values on health and human life anyway). Another common criticism of willingness to pay is its association with ability to pay, which is difficult when such values are being used to aid decision making in health care, where resources are supposedly allocated on the basis of need. However, this challenge can be dealt with to some degree because there is nothing to preclude the use of “equity weights” in cost-benefit analysis.15 Moreover, the problem of values being associated with socioeconomic status and the need to derive appropriate equity weights also applies to QALYs.16 Whatever the conceptual and methodological problems of cost-benefit analysis, the method can in theory address allocative (and productive) efficiency questions when comparing all public-sector services. Rather than investing in an authority such as NICE to consider questions of productive and allocative efficiency in health care (for which modified cost-utility analysis or cost-benefit analysis ought to be recommended), the government should perhaps invest in a joined-up government agency (which would include the existing NICE personnel but, by receiving a share of other public-sector budgets, would be far larger in scale) to consider the same questions in the context of all public-sector services (for which cost-benefit analysis, and the necessary methodological developments therein, could be encouraged). This investment could lead us to a more welfare-enhancing use of existing budgets within the NHS, education, housing, defence, law and order, &c, as well as to a more welfare-enhancing distribution of public resources across the various publicsector services.
resources are distributed across the different public-sector services. History and politics play a large part in deciding this distribution. Economic assessment, if appropriately used, can provide a scientific basis for altering the distribution in a welfare-enhancing manner; but for this to happen, the outcomes of all public-sector services have to be comparable. Cost-benefit analysis is currently the only type of economic assessment that meets this criterion. In this respect, we should not be too concerned about not being able to value everything in terms of money: “. . . a good [cost-benefit analysis] will: identify relevant options for consideration; enumerate all costs and benefits to various relevant social groups; quantify as many as can be sensibly quantified; not assume the unquantified is unimportant; use discounting where relevant to derive present values; use sensitivity analysis to test the response of net benefits to changes in assumptions; and look at the distributive impact of the options.”17 If this is all cost-benefit analysis achieves—and NICE and the government encourages the use of the approach and uses the results in its management of public sector resources—it will achieve a lot.
Conclusion
6
We welcome NICE as an effort by the UK government to introduce the results of economic assessment into the public policy decision-making process. The essence of economic assessment is, after all, to show a more beneficial use of resources, and we find it difficult to believe that many people would be averse to this general concept. However, there are many types of economic assessment, each with a specific purpose. That NICE encourages manufacturers and sponsors of health-care interventions to submit evidence from cost-effectiveness and cost-utility analyses but not from cost-benefit analysis could be a mistake. The perspective adopted by NICE is the NHS. Decisions based on cost-effectiveness analysis might inadvertently cause allocative inefficiency since they do not recognise that consideration of cost-increasing technologies leads inevitably to comparisons of groups of patients, and since they do not incorporate outcomes that can be compared across the whole range of health-care interventions. To ensure allocative efficiency within the NHS, cost-utility analysis or cost-benefit analysis is required. However, we think that the perspective ought to be broader than that of the NHS. Governments are concerned not just with how specific departmental budgets are used, but also with how total public-sector
1774
We are grateful for comments, and thank the two anonymous referees for their comments on an earlier draft of the paper.
References 1
2 3 4
5
7 8
9 10
11 12 13
14
15
16
17
Commonwealth of Australia. Guidelines for the pharmaceutical industry on preparation of submissions to the pharmaceutical benefits advisory committee: including submissions involving economic analyses. Canberra: Australian Government Printing Office, 1995. Ministry of Health. Ontario guidelines for economic analysis of pharmaceutical products. Toronto: Ministry of Health, 1994. Sickness Funds Council. Dutch guidelines for pharmacoeconomic research. Amstelveen: Ziekenfondsraad, 1999. Norwegian Medicines Control Authority. Norwegian guidelines for pharmacoeconomic analysis in connection with application for reimbursement. Oslo: Norwegian Medicines Control Authority (SLK), 1999. Drummond MF, O’Brien B, Stoddart GL, Torrance, GW. Methods for the economic evaluation of health care programmes, 2nd edn. Oxford: Oxford University Press, 1997. National Institute for Clinical Excellence. Technical guidance for manufacturers and sponsors on making a submission to a technology appraisal. London: National Institute for Clinical Excellence, 2001. Donaldson C. The (near) equivalence of cost-effectiveness and costbenefit analysis: fact or fallacy. PharmacoEconomics 1998; 13: 389–96. Donaldson C, Hundley V, McIntosh E. Using economics alongside clinical trials: why we cannot choose the evaluation technique in advance. Health Econ Lett 1996; 5: 267–69. OHE-HEED. Health economic evaluations database. London: OHEIFPMA Database Ltd, 2000. National Institute for Clinical Excellence. Completed technology appraisals 2000/2001. London: National Institute for Clinical Excellence, 2001. Torrance GW. Measurement of health state utilities for economic appraisal. J Health Econ 1986; 5: 1–30. Gerard K, Mooney G. QALY league tables: handle with care. Health Econ 1993; 2: 59–64. Donaldson C, Mapp T, Farrar S, Walker A, Macphee S. Assessing community values in health care: is the ‘willingness to pay’ method feasible? Health Care Anal 1997; 5: 7–29. Olsen J-A, Donaldson C. Helicopters, hearts and hips: using willingness to pay to set priorities for public sector health care programmes. Soc Sci Med 1998; 46: 1–12. Donaldson C. Valuing the benefits of publicly-provided health care: does ‘ability to pay’ preclude the use of ‘willingness to pay’? Soc Sci Med 1999; 49: 551–63. Donaldson C, Birch S, Gafni A. The distribution problem in economic evaluation: income and the valuation of costs and consequences of health care programs. Working Paper 98/10. Hamilton: Centre for Health Economics and Policy Analysis, McMaster University, 1998. Culyer AJ. Economics. Oxford: Blackwell, 1985.
THE LANCET • Vol 359 • May 18, 2002 • www.thelancet.com
For personal use. Only reproduce with permission from The Lancet Publishing Group.
Viewpoint
Choosing the method to match the perspective: economic assessment and its implications for health-services efficiency Adam Oliver, Andrew Healey, Cam Donaldson Consideration of evidence from economic assessment of health-care interventions is becoming important in several countries.1–4 The UK has established the National Institute of Clinical Excellence (NICE; www.nice.nhs.uk), which requires manufacturers or sponsors of health-care interventions to submit evidence on the clinical and cost-effectiveness of many products and services. NICE uses the submitted evidence to formulate recommendations about the interventions that they decide ought to be available within the National Health Service (NHS). The government hopes that the NICE recommendations will help to end “postcode rationing” (ie, geographical disparities in health-services provision) and increase the amount of health gain attributable to NHS resources. NICE has adopted the NHS as its perspective. Therefore, NICE should operate within the framework of the existing NHS budget and ought to have no influence over whether or not the NHS is underfunded (or overfunded). Moreover, of the three commonly defined types of economic assessment—cost-effectiveness analysis, cost-utility analysis, and cost-benefit analysis5— NICE encourages manufacturers and sponsors to use cost-effectiveness analysis or cost-utility analysis.6 Costbenefit analysis is not encouraged because of its limited application in the context of health care. However, NICE—and the health economic and public policymaking communities more generally—do not seem to recognise that the type of economic assessment chosen depends on the type of efficiency question being asked, rather than the ease with which outcomes are measured and valued.7,8 In this paper, we analyse the implications for NICE of preferring cost-effectiveness analysis and cost-utility analysis in the assessment of NHS improvements in productive efficiency (the amount of health gain generated by replacing one intervention with another for the same or lower cost) and allocative efficiency (the amount of health gain or welfare gain generated by replacing one intervention with another at greater cost).
The limitation of cost-effectiveness analysis The cost-effectiveness approach Cost-effectiveness analysis is the most common type of economic assessment,9 probably because the outcome data (effectiveness) in such analyses are relatively simple Lancet 2002; 359: 1771–74 LSE Health and Social Care, London School of Economics and Political Science, London WC2A 2AE, UK (A Oliver MSc, A Healey MSc); Departments of Economics and Community Health Sciences, University of Calgary, Calgary, Alberta, Canada (Prof C Donaldson PhD) Correspondence to: Mr Adam Oliver (e-mail: [email protected])
THE LANCET • Vol 359 • May 18, 2002 • www.thelancet.com
to calculate, and often comprise clinical measures of health (eg, percentage reduction in blood pressure, reduced incidence of hip fracture) that are routinely collected in clinical trials. In an effort to improve comparability across studies, generic measures of outcome are sometimes used, such as life-years saved or disability-days avoided. However, even if these measures are routinely collected in clinical trials and thus readily available for the health economist, they will frequently be inappropriate for the comparative analysis of different health-care interventions. For example, outcomes measured by years of life saved will be irrelevant for programmes that improve the quality of life but do not extend the length of life. Because the outcome measures generally used in costeffectiveness analyses are not comparable across a broad array of health-care programmes, the method is appropriate only for addressing questions of productive efficiency. Indeed, some argue that cost-effectiveness analysis is useful only for comparison of treatment options within a given budget and, therefore, a specific group of patients.7 In this context, if a new treatment is deemed “cost-effective”, it can be substituted for existing care without asking any other group of patients to give up resources. The same group is treated, but by a different method from before. Allocative efficiency within the NHS, which involves deciding how much to allocate to different groups of patients, cannot usually be determined by cost-effectiveness analyses. This point can be illustrated as follows: The misapplication of cost-effectiveness analysis Consider an illness, say osteoporosis, for which there is an existing treatment, X. A new treatment, Y, is being proposed. A cost-effectiveness analysis of treatment Y is undertaken to help determine whether it should replace treatment X. A cost-effectiveness ratio for treatment Y is usually calculated as: (CY–CX)/(EY–EX), where CX, CY, EX, and EY denote the costs and outcomes (effectiveness) of interventions X and Y, respectively. Assume that the outcome of treatment is the number of hip fractures prevented. Further assume that treatment X costs £2 million and prevents 1000 hip fractures annually, and that treatment Y costs £1·5 million and prevents 1500 hip fractures. The new treatment is more effective and less costly than the existing treatment. The introduction of treatment Y in place of treatment X thus offers an improvement in productive efficiency—ie, by introducing treatment Y, the health service would deliver greater health improvements for the same level of resources. Although treatment Y is likely to be introduced without too much deliberation, the cost-effectiveness ratio of –£1000 per hip fracture prevented gives an indication of the degree of productive efficiency that the new treatment offers. For example, had treatment Y cost £1·75 million, the cost-effectiveness ratio would be –£500 1771
For personal use. Only reproduce with permission from The Lancet Publishing Group.
VIEWPOINT
per hip fracture prevented, implying half as much improvement in productive efficiency compared with a situation in which treatment Y cost £1·5 million. In the circumstances described, the introduction of treatment Y has greater productive efficiency (or is “dominant”) because it is less costly and more effective than the existing treatment. Treatment Y would also offer an improvement in productive efficiency if it costs the same and is more effective, or if it costs less and is equally effective. In all of the above circumstances, the introduction of treatment Y in place of treatment X would also offer an improvement in allocative efficiency. Unfortunately, most new interventions increase costs.5 Consider again the circumstances above, but assume now that treatment Y costs £2·5 million and still prevents 1500 hip fractures. The increment in costs is £500 000 and the increment in effectiveness is 500 hip fractures prevented; hence, the incremental cost-effectiveness analysis ratio of treatment Y is £1000 per hip fracture prevented. Should we be willing to pay an extra £1000 for each additional hip fracture prevented? When a treatment increases costs, there is no explicit scientific or ethical definition of an acceptable cost-effectiveness ratio. Thus the decision about whether a particular cost-effectiveness ratio renders an intervention a worthwhile use of resources is partly political, involving a comparison of the gain of investing in the new treatment with what would be given up (ie, the opportunity cost) by funding it. Conventional wisdom, however, dictates that a treatment with a cost-effectiveness ratio of £1000 per hip fracture prevented would generally be regarded as cost-effective.10 If NICE were to appraise an osteoporosis treatment with that level of cost-effectiveness, it would probably be recommended for use within the NHS. So let us assume that NICE has recommended treatment Y to the geographically defined health authorities (the purchasers of NHS care) in England and Wales. For simplification, assume that there are ten health authorities, each of which bear 10% of the costs and enjoy 10% of the outcomes of treatment X and will be expected to experience an identical percentage and the costs and outcomes of treatment Y. Therefore, each health authority is being asked to pay for a new treatment that will cost them £250 000 (10% of £2·5 million). The old treatment that treatment Y will replace costs them £200 000 (10% of £2 million). By replacing treatment X with treatment Y each health authority has to find an additional £50 000 from somewhere else (say, by disinvesting in treatment Z), but the benefits reaped from treatment Z might outweigh the expected benefits of investing the £50 000 in treatment Y. If so, the health authority managers should ensure that the net benefit of treatment Y over treatment X exceeds the net loss of benefit from forgoing treatment Z. Since Z will be aimed at a different group of patients, this question is fundamentally one of allocative efficiency. The magnitude of costs and health outcomes, estimated in the economic assessment, will play an important part in any decision, as will other factors which were not quantified. Incidentally, given the variations in contexts across health authorities, the implementation of Y would result in different things being given up in different parts of the country, which would result in another type of postcode rationing. NICE might foresee this problem and provide explicit advice to the health authorities on the services they must cut back on to accommodate the new cost-effective but cost-increasing intervention. However, to do this with any reference to explicit decision-making criteria, NICE would have to compare the new intervention with all
1772
interventions that it regarded as expendable with reference to a common measure of outcome (eg, life-years saved). Aside from the practical difficulties of undertaking such an extensive set of comparisons, finding an appropriate common outcome measure with which to compare interventions can often be impossible in the framework of a cost-effectiveness analysis. If explicit guidance based on good assessment is unavailable, NICE could use guesswork to issue guidance about interventions on which to cut back. Alternatively, health authority managers could use their own (unsystematic) judgment to determine which services to terminate or decrease. Either way, treatment Y could be partly financed by the withdrawal of interventions that might offer significant benefit to the population. Indeed, the NICE recommendations, if based on the results of cost-effectiveness analysis, could cause greater allocative inefficiency in the NHS. In short, incremental costeffectiveness ratios for cost-increasing interventions exclude sufficient consideration of opportunity costs, and therefore cannot inform the policy maker about whether the benefits of introducing the intervention outweigh the necessary cutbacks that would have to be made elsewhere. Whither cost-effectiveness analysis? Worsening allocative inefficiency in the NHS could be avoided if the government increased the NHS budget in order to finance the new cost-increasing treatment. If the government were to do this, the health authority managers would not have to worry about cutting back on their services elsewhere. However, to increase the NHS budget, the government would be required either to cut back on other public services (eg, education, housing, defence, or law and order) or to increase taxation. Either measure involves tradeoffs, and cost-effectiveness analysis cannot inform us whether the increase in welfare from more NHS investment outweighs the foregone welfare from reducing investment in other public services or from increasing taxation. Thus, cost-effectiveness analyses cannot provide evidence of whether the NHS budget should be altered. We believe that considerations of allocative efficiency with respect to “welfare” from a societal perspective ought to take prominence over those with respect to “health” from an NHS perspective. In short, cost-effectiveness analysis is a useful framework for assessing productive efficiency within the NHS (ie, in the restrictive set of circumstances where a treatment is dominant). However, the often incomparable outcome measures across studies that use this type of economic assessment render it of limited value for ensuring allocative efficiency when introducing costincreasing treatments. Furthermore, consideration of the potential introduction of cost-increasing treatments inevitably involves comparisons of different groups of patients, which cost-effectiveness analysis is not equipped to do. To address allocative efficiency, even within the limited context of the NHS, we have to search for a measure of outcome that can be applied to most healthcare interventions.
Using “utility” The cost-utility analysis approach Cost-utility analyses are identical to cost-effectiveness analyses, except that the outcomes are measured in terms of a single composite index that combines length of life with quality of life (typically represented by qualityadjusted life-years [QALYs]), and this measure is preference based. For example, if full health is normalised with a quality weight of 1, and, by use of preference
THE LANCET • Vol 359 • May 18, 2002 • www.thelancet.com
For personal use. Only reproduce with permission from The Lancet Publishing Group.
VIEWPOINT
elicitation techniques,5,11 living with multiple sclerosis is found to offer a quality of life that is only 40% that of living in full health (ie, the health state “utility” of multiple sclerosis is 0·4), then 6 months with multiple sclerosis will give 0·2 QALYs (ie, 0·5 [years]⫻0·4). Typically, cost-utility analysis is used to generate incremental cost-utility ratios that, in themselves, have the same flaws as those generated by cost-effectiveness analysis. Namely, an incremental cost-utility ratio derived by comparing a new cost-increasing intervention with an existing comparable intervention does not offer an assessment of all of the opportunity costs associated with the new intervention. Therefore, the introduction of the new intervention necessarily leads to non-assessed cutbacks elsewhere, which might lead to an overall reduction in the benefit of health care. Consequently, ranking interventions according to their incremental costeffectiveness or cost-utility (as commonly estimated) gives a potentially misleading picture of the interventions’ relative worth. NICE must be aware of this limitation in their assessment of incremental ratios from either costeffectiveness analysis or cost-utility analysis, and the compilation of league tables of cost-effectiveness or costutility as a decision aid for policy makers ought to be avoided. However, if we extend the assessment of a new intervention beyond the derivation of the simple incremental ratio, cost-utility analysis can be used to address questions of allocative efficiency in the NHS by allowing a comparison of health outcomes across a range of health interventions.12 Although this is a theoretical ideal, NICE at least ought to recognise explicitly what the relevant trade-offs are. In the example given above, where each health authority has to find an additional £50 000 to fund treatment Y, health authority managers could theoretically measure the number of net QALYs they would gain or lose by forgoing an alternative treatment in some other therapeutic area to provide treatment Y. If treatment Y offers a net gain in QALYs after sacrifices have been considered (ie, opportunity costs in terms of QALYs foregone), then the provision of treatment Y would represent an improvement in allocative efficiency. For a given amount of resources, more QALYs would be produced overall by taking resources from one group and reallocating them to another. Cost-utility analysis represents an improvement on cost-effectiveness analysis and, if used appropriately, could potentially allow NICE to address questions of allocative efficiency as well as productive efficiency in the NHS in a scientific manner. Whither cost-utility analysis? Despite taking us further than cost-effectiveness analysis (providing that the analysis is taken further than the derivation of a simple incremental cost-utility ratio across two treatments), cost-utility analysis has two conceptual limitations. First, QALY techniques place values on health-related quality of life only. This means that other attributes of health care over which people have preferences, such as the location of care, are not accounted for. Techniques of (hypothetical) willingness to pay have been used in surveys to address people’s strengths of preference for different locations of care.13 In only one study has willingness to pay been used to assess people’s preferences for provision of increments in disparate health-care options for the community.14 That study showed that the ordering arising from willingnessto-pay values can be different from that arising from QALY estimation. Second, although cost-utility analysis can be used to
THE LANCET • Vol 359 • May 18, 2002 • www.thelancet.com
improve allocative efficiency in the NHS, how can we deal with questions of allocative efficiency across the whole of the public sector? Would it be beneficial to invest more money in the NHS at the expense of housing, or more money in education at the expense of defence? These questions are tied in with the concept of joined-up government, where different government departments take a more holistic view of what is good for society, rather than how much good their own particular government department has to offer. Since health-state utilities cannot be compared with the outcomes of education, housing, defence, &c, development of broader utility (or “welfare”) measures would be useful. Unfortunately, the development of comparable direct measures of utility across the range of public services seems impossible at present. We seemingly have to look elsewhere for an outcome measure that is appropriate for comparing the benefits of all public sector services.
The role of cost-benefit analysis Cost-benefit analysis has a broader scope than both costeffectiveness analysis and cost-utility analysis, and differs mainly in that its outcome measures are defined in monetary terms. A monetary value can be placed on a health-care treatment (ie, typically described in terms of some measure of health and other characteristics) by, for example, asking people what is the maximum amount they would be willing to pay for the treatment or how much they would be willing to pay to avoid negative characteristics, such as the debilitating effects of an illness. Willingness to pay can be measured with various other techniques.5 By measuring both costs and outcomes in money terms, cost-benefit analysis allows us to estimate the net social benefit of an intervention. Returning to our osteoporosis example, the net social benefit of treatment Y can be calculated in simple terms by: WY–CY–CS, where W is society’s willingness to pay for the benefits offered by treatment Y, CY is the NHS cost of treatment Y, and CS (included if we adopt a societal approach) are the costs associated with treatment Y that transcend NHS costs— eg, the costs of residential convalescence care. Thus, if the willingness to pay for treatment Y is £10 million, the cost of the treatment is £2·5 million and the non-NHS costs are £3 million, the net social benefit of treatment Y measures £4·5 million. By a similar token, suppose that the government is considering a scheme to increase the number of secondary school teachers, which the government projects will improve the pass rate in the national examinations for 16year-olds by 10%. The direct education costs of the scheme are estimated to be £5 million and the wider social costs (eg, an increase in NHS general practitioner visits by teachers who feel more pressure due to the new pass-rate targets) are expected to be £500 000. The willingness to pay for this scheme is £6 million; the net social benefit of the scheme is therefore estimated at £500 000. The monetary valuation of outcomes allows us to compare the net social benefit of social programmes as disparate as osteoporosis treatment and a scheme to increase the number of secondary school teachers. Therefore, if the government intended to introduce a health intervention that was calculated to have a net social benefit of £10 million, but had to forego either treatment Y or the education scheme to finance this new intervention, the simple hypothetical example outlined above suggests that abandonment of the scheme to increase the number of secondary school teachers would be sensible in terms of allocative efficiency. Theoretically,
1773
For personal use. Only reproduce with permission from The Lancet Publishing Group.
VIEWPOINT
cost-benefit analysis can be used to help determine whether the NHS is underfunded or overfunded. As with health state utilities, there are conceptual problems with cost-benefit analyses in that many decision makers might object to monetary values being placed on something as fundamental as health (although these decision makers, through making choices, will be placing monetary values on health and human life anyway). Another common criticism of willingness to pay is its association with ability to pay, which is difficult when such values are being used to aid decision making in health care, where resources are supposedly allocated on the basis of need. However, this challenge can be dealt with to some degree because there is nothing to preclude the use of “equity weights” in cost-benefit analysis.15 Moreover, the problem of values being associated with socioeconomic status and the need to derive appropriate equity weights also applies to QALYs.16 Whatever the conceptual and methodological problems of cost-benefit analysis, the method can in theory address allocative (and productive) efficiency questions when comparing all public-sector services. Rather than investing in an authority such as NICE to consider questions of productive and allocative efficiency in health care (for which modified cost-utility analysis or cost-benefit analysis ought to be recommended), the government should perhaps invest in a joined-up government agency (which would include the existing NICE personnel but, by receiving a share of other public-sector budgets, would be far larger in scale) to consider the same questions in the context of all public-sector services (for which cost-benefit analysis, and the necessary methodological developments therein, could be encouraged). This investment could lead us to a more welfare-enhancing use of existing budgets within the NHS, education, housing, defence, law and order, &c, as well as to a more welfare-enhancing distribution of public resources across the various publicsector services.
resources are distributed across the different public-sector services. History and politics play a large part in deciding this distribution. Economic assessment, if appropriately used, can provide a scientific basis for altering the distribution in a welfare-enhancing manner; but for this to happen, the outcomes of all public-sector services have to be comparable. Cost-benefit analysis is currently the only type of economic assessment that meets this criterion. In this respect, we should not be too concerned about not being able to value everything in terms of money: “. . . a good [cost-benefit analysis] will: identify relevant options for consideration; enumerate all costs and benefits to various relevant social groups; quantify as many as can be sensibly quantified; not assume the unquantified is unimportant; use discounting where relevant to derive present values; use sensitivity analysis to test the response of net benefits to changes in assumptions; and look at the distributive impact of the options.”17 If this is all cost-benefit analysis achieves—and NICE and the government encourages the use of the approach and uses the results in its management of public sector resources—it will achieve a lot.
Conclusion
6
We welcome NICE as an effort by the UK government to introduce the results of economic assessment into the public policy decision-making process. The essence of economic assessment is, after all, to show a more beneficial use of resources, and we find it difficult to believe that many people would be averse to this general concept. However, there are many types of economic assessment, each with a specific purpose. That NICE encourages manufacturers and sponsors of health-care interventions to submit evidence from cost-effectiveness and cost-utility analyses but not from cost-benefit analysis could be a mistake. The perspective adopted by NICE is the NHS. Decisions based on cost-effectiveness analysis might inadvertently cause allocative inefficiency since they do not recognise that consideration of cost-increasing technologies leads inevitably to comparisons of groups of patients, and since they do not incorporate outcomes that can be compared across the whole range of health-care interventions. To ensure allocative efficiency within the NHS, cost-utility analysis or cost-benefit analysis is required. However, we think that the perspective ought to be broader than that of the NHS. Governments are concerned not just with how specific departmental budgets are used, but also with how total public-sector
1774
We are grateful for comments, and thank the two anonymous referees for their comments on an earlier draft of the paper.
References 1
2 3 4
5
7 8
9 10
11 12 13
14
15
16
17
Commonwealth of Australia. Guidelines for the pharmaceutical industry on preparation of submissions to the pharmaceutical benefits advisory committee: including submissions involving economic analyses. Canberra: Australian Government Printing Office, 1995. Ministry of Health. Ontario guidelines for economic analysis of pharmaceutical products. Toronto: Ministry of Health, 1994. Sickness Funds Council. Dutch guidelines for pharmacoeconomic research. Amstelveen: Ziekenfondsraad, 1999. Norwegian Medicines Control Authority. Norwegian guidelines for pharmacoeconomic analysis in connection with application for reimbursement. Oslo: Norwegian Medicines Control Authority (SLK), 1999. Drummond MF, O’Brien B, Stoddart GL, Torrance, GW. Methods for the economic evaluation of health care programmes, 2nd edn. Oxford: Oxford University Press, 1997. National Institute for Clinical Excellence. Technical guidance for manufacturers and sponsors on making a submission to a technology appraisal. London: National Institute for Clinical Excellence, 2001. Donaldson C. The (near) equivalence of cost-effectiveness and costbenefit analysis: fact or fallacy. PharmacoEconomics 1998; 13: 389–96. Donaldson C, Hundley V, McIntosh E. Using economics alongside clinical trials: why we cannot choose the evaluation technique in advance. Health Econ Lett 1996; 5: 267–69. OHE-HEED. Health economic evaluations database. London: OHEIFPMA Database Ltd, 2000. National Institute for Clinical Excellence. Completed technology appraisals 2000/2001. London: National Institute for Clinical Excellence, 2001. Torrance GW. Measurement of health state utilities for economic appraisal. J Health Econ 1986; 5: 1–30. Gerard K, Mooney G. QALY league tables: handle with care. Health Econ 1993; 2: 59–64. Donaldson C, Mapp T, Farrar S, Walker A, Macphee S. Assessing community values in health care: is the ‘willingness to pay’ method feasible? Health Care Anal 1997; 5: 7–29. Olsen J-A, Donaldson C. Helicopters, hearts and hips: using willingness to pay to set priorities for public sector health care programmes. Soc Sci Med 1998; 46: 1–12. Donaldson C. Valuing the benefits of publicly-provided health care: does ‘ability to pay’ preclude the use of ‘willingness to pay’? Soc Sci Med 1999; 49: 551–63. Donaldson C, Birch S, Gafni A. The distribution problem in economic evaluation: income and the valuation of costs and consequences of health care programs. Working Paper 98/10. Hamilton: Centre for Health Economics and Policy Analysis, McMaster University, 1998. Culyer AJ. Economics. Oxford: Blackwell, 1985.
THE LANCET • Vol 359 • May 18, 2002 • www.thelancet.com
For personal use. Only reproduce with permission from The Lancet Publishing Group.