- Email: [email protected]
WHO Guide for standardisation of economic evaluations of immunization programmes
Vaccine 28 (2010) 2356–2359
Contents lists available at ScienceDirect
Vaccine journal homepage: www.elsevier.com/locate/vaccine
WHO Guide for standardisation of economic evaluations of immunization programmes Damian G. Walker a,∗ , Raymond Hutubessy b , Philippe Beutels c a
Health Economics, Health Systems Program, Department of International Health, Bloomberg School of Public Health, Johns Hopkins University, 615 North Wolfe Street, Baltimore, MD 21205, USA Initiative for Vaccine Research World Health Organization Avenue Appia 20, CH1211-Genève 27, Switzerland c Centre for Health Economics Research and Modelling Infectious Diseases (CHERMID), Centre for the Evaluation of Vaccination, Vaccine & Infectious Disease Institute, University of Antwerp, Universiteitsplein 1, 2610 Antwerp, Belgium b
a r t i c l e
i n f o
Article history: Received 8 June 2009 Accepted 9 June 2009 Available online 28 June 2009 Keywords: Economic evaluations Cost-effectiveness analysis Guidelines Immunization programmes Vaccines
a b s t r a c t Traditional EPI vaccines are considered to be among the most efficient uses of scarce health care resources. Today, there are many under-used and new vaccines available. In the short- to medium-term, these vaccines will not cost the few cents per dose the traditional vaccines do, but will be ‘multi-dollar’ vaccines. Decision-makers will need information, among other things, on their relative cost-effectiveness. A number of reviews have indicated that there is scope for improving the transparency, completeness and comparability of economic evaluations of vaccination programmes. Thus, there is a need to improve the quality of economic evaluations of vaccination programmes. Adherence to general guidelines would increase the quality, interpretability and transferability of future analyses. However, there is reason to believe that there might also be a need for more specific advice for vaccination programmes. For example, there are inconsistencies in the methods used to estimate the future benefits of vaccination programmes and the relative efficiency of these programmes can be sensitive to some of the more controversial aspects of general guidelines, such as the inclusion of indirect costs and the discounting of health outcomes. This guide has been developed in order to meet the needs of decision-makers for relevant, reliable and consistent economic information. They aim to provide clear and concise, practical and high quality guidance for those who conduct economic evaluations. © 2009 Elsevier Ltd. All rights reserved.
1. Introduction Traditional EPI vaccines are considered to be among the most efficient uses of scarce health care resources [1]. Today, many vaccines are available that are not yet used as widely as they could be (e.g., against yellow fever, hepatitis B virus (HBV), Haemophilus influenzae type b (Hib), rotavirus, Streptococcus pneumoniae, human papilloma virus (HPV) and Japanese encephalitis). Furthermore, many more vaccines are in the pipeline, e.g., against enterotoxigenic Escherichia coli, shigellosis, dengue, hookworm, schistosomiasis, herpes simplex virus 2, human immunodeficiency virus (HIV), malaria and tuberculosis (TB). In the short- to medium-term, these vaccines will not cost the few cents per dose the traditional vaccines do, but will be ‘multi-dollar’ vaccines. Policy makers will need information, among other things, on their relative cost-effectiveness to decide whether they should be (more) widely used.
∗ Corresponding author. Tel.: +1 410 502 6023; fax: +1 410 614 1419. E-mail addresses: [email protected] (D.G. Walker), [email protected] (R. Hutubessy), [email protected] (P. Beutels). 0264-410X/$ – see front matter © 2009 Elsevier Ltd. All rights reserved. doi:10.1016/j.vaccine.2009.06.035
A number of reviews have indicated that there is scope for improving the transparency, completeness and comparability of economic evaluations of vaccination programmes [2–16]. Thus, there is a need to improve the quality of economic evaluations of vaccination programmes. Adherence to general guidelines would increase the quality, interpretability and transferability of future analyses [17]. However, there is reason to believe that there might also be a need for more specific advice for vaccination programmes. For example, there are inconsistencies in the methods used to estimate the future benefits of vaccination programmes and the relative efficiency of these programmes can be sensitive to some of the more controversial aspects of general guidelines, such as the inclusion of indirect costs and the discounting of health outcomes [18]. This WHO Guide [19] has been developed with the aim to guide economics and health services researchers in the public and private sector who conduct and critically appraise economic evaluations of immunization programmes. It aims to provide clear and concise, practical and high quality guidance for performing and presenting the results of economic evaluations, in a relevant, consistent and timely manner. It was reviewed by participants of a WHO meeting hosted by the Program for Appropriate Technology in Health (PATH)
D.G. Walker et al. / Vaccine 28 (2010) 2356–2359
that took place in Seattle, USA on 6–7 March 2007, other independent reviewers and subsequently endorsed by WHO through their Quantitative Immunization and Vaccine related Research (QUIVER) Advisory Panel. In what follows, we summarise the WHO Guide per step an analyst takes while conducting an economic evaluation. 2. Framing the evaluation The first step to conducting an economic evaluation is to frame the study. Decisions made at this stage will directly determine which costs and outcomes are considered relevant and should therefore be included in the analysis. This means that choices made in the framing of the evaluation will have an impact on the final results of an analysis. First and foremost, the study question should be well-defined, stated in an answerable form and relevant to the decision facing the target audience. The comparators under evaluation should be clearly described. The most relevant comparison for new vaccines is usually current practice. If existing practice itself appears to be a cost-ineffective option relative to other available options, the analyst should incorporate other relevant alternatives into the analysis, such as a best available alternative, a viable low-cost alternative or a do-nothing option. The form of economic evaluation should be clearly stated and justified. CUA is the preferred type of evaluation (with DALYs or QALYs as outcome measures), although a CEA, which presents outcomes using natural units as outcomes measures specific to the vaccine(s) in question, is also encouraged. Ideally analyses should take the perspective of society, and include all related effects and costs, regardless of who benefits from or pays for them. Nonetheless, the costs borne by providers (e.g., donors and governments), patients and their families and others should be disaggregated as far as possible in order to allow judgments to be made from the perspectives of the various decision-makers. It is important to clearly state the person(s) or institution(s) sponsoring the study. Finally, the time frame and analytic horizon should be clearly stated; their respective durations are contingent on the type of vaccine evaluated, the intervention and target group, and thus the type of model developed (see section below on modelling). 3. Assessing the cost of a vaccination programme The exact nature of the costs assessed will depend on the scope of the analysis and the perspective(s) adopted. Whatever the scope, the methods for the estimation of costs should be clearly stated. A summary should be provided of the expected resource use and unit costs for each alternative. This should include specifying the assumptions behind calculations of costs, e.g., amounts and types of health service use with and without the alternative, given a specific coverage of the alternative and indicating actual and potential ranges of each estimate. A full costing study should be considered if precise estimates are needed and if it is worth the additional efforts required. In all other cases it is recommended to use standardised WHO-CHOICE [20] estimates or existing country-specific cost data if available. Costs for patients and their families, including lost productivity if considered, should be reported separately. These guidelines recognize that several methods exist for valuing lost productivity; analysts should therefore make clear and justify why a particular method was chosen and what its pros and cons are. So-called future unrelated costs should not be included both because of the practical difficulties of estimation and because their inclusion involves ethical and conceptual issues, for which currently consensus is lacking in the field of health economics [21]. Finally, costs should be reported in local currency units, ideally using the most recent year as the base-year. Additionally to allow regional or global comparability, conversions to US$ using
2357
official exchange rates for the base-year in question and to I$ using purchasing power parity (PPP) exchange rates are recommended. 4. Assessing the effects of a vaccination programme Estimates of vaccine efficacy should be based upon systematic reviews of the available literature where available, taking account of the biological characteristics of the pathogen in question and how its infectious nature may have influenced the efficacy estimates derived from trials. Efficacy estimates should be adjusted for loss of potency due to heat and freeze exposure, where such data are available and multiplied by routine vaccination coverage, accounting for non-compliance (based on relevant resources depending on the type of program), if adverse events from immunization are likely to have a substantial impact on the results of the analysis, they should be included on both the costs and effects side of the analysis. The significance of the impact depends on both their likelihood of occurring as a consequence of vaccination and their severity. Finally, analysts should first present estimates of burden in natural units. These should include cases, deaths, years of life lost (YLL) and years lived with disability (YLD). These natural units can then be converted to DALYs (and these should be estimated both with and without age weighting). 5. Modelling To estimate the population impact of vaccination (i.e., to turn efficacy measures explained in the previous section into estimates of effectiveness), some form of mathematical modelling is inevitable. For economic evaluation, the guide generally advocates choosing the model that minimally meets the analytical requirements given the pathogen, the endemic situation and the intervention. Guidance was provided mainly on the generally most influential model attribute choice for the estimated costeffectiveness ratio of vaccination: choosing between static and dynamic models. A dynamic model is defined as a mathematical model in which the force of infection is a function of the proportion of infectious people in the population at each time point. The force of infection can thus change over time in this type of model. A static model, on the other hand is defined as a mathematical model in which the force of infection is assumed to be independent of the proportion of infectious people at each time point. Clearly, as transparency is key in model-based economic analysis, analysts should report on all the modelling attributes, not just on the static versus dynamic choice. Beutels et al. (forthcoming) explain the guidance on accounting for vaccine effects and choosing models in a separate publication. In the current summary, we highlight the following recommendations the guide makes. It is recommended that the mathematical model should be: • Transparent in which the structure and implicit or explicit assumptions are all clearly described. • Static, if vaccination is unlikely to change the force of infection in susceptibles or as a means to make a worst case estimate when externalities from herd-immunity cannot on the whole be adverse. • Dynamic, if vaccination is likely to change the force of infection in susceptibles, and a static model would not yield a worst case estimate, or if the worst case from a static model does not lead to a favourable outcome. • Stochastic if chance plays an important role in the transmission process of the pathogen.
2358
D.G. Walker et al. / Vaccine 28 (2010) 2356–2359
• Validated, in as many facets of validation (verification, calibration, face validity, predictive validity [22] as possible, but at least verified. 6. Discounting For curative therapies, most benefits accrue immediately or shortly after the intervention is initiated, and the cost-effectiveness of these interventions is therefore largely independent of the discount rate. Conversely, the cost-effectiveness of most vaccination programmes is highly sensitive to discounting. The guidelines recommend that costs and effects occurring in the future should initially be discounted using the rate in the country in question (for studies to inform local decision-makers) and then using a 3% discount rate (consistent with WHO-CHOICE and DCP2 [20,23]. However, it is recommended that analysts conduct sensitivity analysis using discount rates of 0%, near-zero, 5% and 10% to reflect the (probably) higher real risk-free cost of capital in developing countries, if the evaluation is being conducted in such a setting. A non-constant (declining or ‘slow’) discounting [24,25] procedure may be applied where the effects begin only long after the intervention, e.g., vaccination against HBV or HPV, or last for an exceptionally long time, e.g., polio eradication. 7. Estimating, presenting and interpreting data It is important that analysts begin by excluding those interventions that are dominated, i.e., both more costly and less effective than alternative options. The classification of costeffective and cost-ineffective interventions should not be made on the basis of point estimates of cost-effectiveness because of uncertainty about the estimates of cost, effects and hence cost-effectiveness. Therefore, as a minimum, analysts should conduct one-way sensitivity analyses of the following variables: discount rate, vaccination effectiveness (where unknown or uncertain), incidence of disease (including complication rates where relevant), direct medical costs, case fatality rates and vaccine price. Additionally, it is recommended that analysts, when possible, conduct probabilistic sensitivity analyses on all parameters and construct cost-effectiveness acceptability curves. In addition to performing such sensitivity analyses to quantify parameter uncertainty, analysts should carefully consider the extent of uncertainty introduced by the choices they made with regards to model structure and selection of data sources. If alternatives for these choices exist, they should consider and discuss, and possibly quantify how these affected the results of their analyses. Analysts should use the recommendation of the Commission on Macroeconomics and Health [26] that has been endorsed by WHO and classify the results of their evaluation according to the per capita national GDP of the country in question: less than one classifies a vaccine as ‘highly cost-effective’; between one and three times classifies an intervention as ‘cost-effective’; and more than three times classifies an intervention as ‘cost-ineffective’. However, analysts should also place their findings in broader context by comparing their findings to other economic evaluations that have been undertaken in the same or neighbouring countries after adjustments have been made for inflation. 8. The use of economic evaluation in policy and practice Although more than 200 economic evaluations of vaccination programmes have been published [27], there are not many examples in the literature of evidence about their specific use at
global, regional, national or local levels of decision-making (e.g., see Welte et al. [28], Bos et al. [29] for published accounts for The Netherlands). However, many anecdotal accounts exist about the use of such economic evaluations, mainly at a national level, to aid price negotiations and reductions in the proposed number of doses. Less clear is its role as a decisive influence, rather than a justification, for general decisions on whether or not a vaccine should be introduced in a national vaccination programme [6,13]. While some countries have a separate advisory group on vaccination, which uses evidence on cost-effectiveness and exerts great influence on policy, e.g., the United Kingdom’s Joint Committee on Vaccination and Immunisation and the United States’ Advisory Committee on Immunization Practices, most do not—or else their financial backing is only sufficient to sustain advice on the basis of the literature and ad hoc expert opinion. Most GAVI-eligible countries, however, have little or limited access to formal advisory bodies to review immunization data and provide independent advice to their respective governments. There is a need to support the establishment of national or regional processes to enhance evidence-based decision-making in immunization (and health more generally) in order to routinely and formally address questions such as whether, when and how a new vaccine should be introduced. In order to make informed decisions, countries need to have both the necessary evidence and clear processes. Much work has been done and is underway (for example see Andrus et al. [30] for Latin America) to develop the evidence base countries need for making informed decisions; little effort has thus far been expended to ensure that countries have processes in place to evaluate and use this information. 9. Other criteria to consider when making decisions While the emphasis of this guide is on value for money, i.e., whether a vaccine is worth buying and not who pays for it, if the object is to decide how to spend public funds, economic evaluation is only one of at least nine criteria relevant for priority-setting in health [31]. Cost alone matters, as do the capacities of potential beneficiaries to pay for an intervention. The other criteria that may affect priorities include horizontal equity (equal treatment for people in equal circumstances); vertical equity (priority for people with worse problems); adequacy of demand; and public attitudes and wants. Two criteria, whether an intervention is a public good and whether it yields substantial externalities, are classic justifications for public intervention, because private markets could not supply them efficiently, just as in other sectors. Equity, poverty, and risk of impoverishment from ill health may also influence priorities; so do the budgets available, and the decisions of how much to make available for buying interventions. Finally, the effectiveness of an intervention and, therefore, the degree to which it deserves priority depends on how far it is culturally appropriate or acceptable for the population it is intended to benefit. The identical intervention, technically speaking, may lead to different degrees of use or compliance in different population groups, and information and incentives may be needed to achieve the full potential outcomes. Polio vaccination is an example of where other elements in the decision-making process have ‘trumped’ the ‘logic of costeffectiveness’, in view of the achievement of polio elimination from high and middle income countries [15]. Therefore, concerns over the public’s perception about the risks associated with OPV led to its replacement by the risk-free IPV in most high income countries. Interestingly, however, one must assume that value for money has thus far been an important part of the rationale to continue using OPV in low income countries.
D.G. Walker et al. / Vaccine 28 (2010) 2356–2359
10. Conclusions Decision-making for vaccines is getting tougher [15]. Over the next decade, a number of new vaccine products will become available. With the advent of more expensive new products (“product pile-up”), countries will face a significant decision-making challenge [32]. Data regarding the relative cost-effectiveness of these products will be an important criterion for decision-makers to consider. This guide does not propose to alter the general guidelines for economic evaluations, but merely a specific interpretation of them with respect to vaccination and a more rigorous application of them in general. Economic evaluations in the field of vaccine-preventable diseases, which are often complicated by many parameters and assumptions, should first of all be explicit and transparent. All assumptions should be clearly stated and justified. Sections dealing with methods and assumptions should clearly and explicitly describe all weaknesses of the analysis. Acknowledgments We would like to thank the participants of a WHO meeting hosted by the Program for Appropriate Technology in Health (PATH) that took place in Seattle on 6–7 March 2007 to review a draft of these guidelines for their helpful comments: Jon Andrus (PAHO), Deborah Atherly (PATH), Daniel Barth-Jones (Wayne State University), Ruanne Barnabas (Fred Hutchinson Cancer), Andrew Clark (London School of Hygiene & Tropical Medicine), Nancy Engel (PATH), David Evans (WHO), Ulla Griffiths (London School of Hygiene, and Tropical Medicine), Vipat Kuruchittham (Chulalongkorn University), Carol Levin (PATH), Rosamund Lewis (GAVI Alliance), Patrick Lydon (WHO), Rocio Mosqueira (Instituto de Investigación Nutricional), Richard Rheingans (Emory University) Chutima Suraratdecha (PATH), Tessa Tan-Torres (WHO), Abdelmajid Tibouti (UNICEF), Cristiana Toscano (PAHO) Maya Vijayaraghavan (CDC) and Lara Wolfson (WHO). We would also like to thank Logan Brenzel (World Bank), Sue Goldie (Harvard University), Maarten Postma (Groningen University) and members of the International Federation of Pharmaceutical Manufacturers & Association (IFPMA) Biologicals and Vaccines Committee for detailed written comments. Finally we would like to thank the QUIVER members and meeting participants of the inaugural WHO QUIVER Advisory Group meeting that took place in Geneva on 27–28 September 2007. PB acknowledges funding from Simulation Models for Infectious Disease Processes (SIMID), a strategic basic research project funded by the Institute for the Promotion of Innovation by Science and Technology in Flanders (project number 060081). Disclaimer: Raymond Hutubessy is a staff member of the World Health Organization. The author alone is responsible for the views expressed in this publication and they do not necessarily represent the decisions, policy or views of the World Health Organization. References [1] World Bank. World development report 1993: investing in health. New York: Oxford University Press; 1993. [2] Beutels P. Economic evaluations of hepatitis B immunization: a global review of recent studies (1994–2000). Health Econ 2001;10(8):751–74. [3] Beutels P, Postma MJ. Economic evaluations of adult pneumococcal vaccination: a review of the literature. Expert Rev Pharmacoecon Outcomes Res 2001;1(1):47–58. [4] Beutels P. Potential conflicts of interest in vaccine economics research: a commentary with a case study of pneumococcal conjugate vaccination. Vaccine 2004;22(25–26):3312–22.
2359
[5] Beutels P, Thiry N, VAN Damme P. Convincing or confusing? Economic evaluations of childhood pneumococcal conjugate vaccination—a review (2002–2006). Vaccine 2007;25(8):1355–67. [6] Brinsmead R, Hill S, Walker D. Are economic evaluations of vaccines useful to decision-makers? Case study of Haemophilus influenzae type b vaccines. Pediatr Infect Dis J 2004;23(1):32–7. [7] Edmunds WJ. Universal or selective immunisation against hepatitis B virus in the United Kingdom? A review of recent cost-effectiveness studies. Commun Dis Public Health 1998;1(4):221–8. [8] Jefferson T, Demicheli V. Is vaccination against hepatitis B efficient? A review of world literature. Health Econ 1994;3(1):25–37. [9] Newall AT, Beutels P, Wood JG, Edmunds WJ, MacIntyre CR. Costeffectiveness analyses of human papillomavirus vaccination. Lancet Infect Dis 2007;7(4):289–96. [10] Pegurri E, Fox-Rushby JA, Damian W. The effects and costs of expanding the coverage of immunisation services in developing countries: a systematic literature review. Vaccine 2005;23(13):1624–35. [11] Thiry N, Beutels P, VAN Damme P, Van Doorslaer E. Economic evaluations of varicella vaccination programmes: a review of the literature. Pharmacoeconomics 2003;21(1):13–38. [12] Walker D, Rheingans R. Cost-effectiveness of rotavirus vaccines. Expert Rev Pharmacoecon Outcomes Res 2005;5(5):593–601. [13] Welte R, Trotter CL, Edmunds WJ, Postma MJ, Beutels P. The role of economic evaluation in vaccine decision making: focus on meningococcal group C conjugate vaccine. Pharmacoeconomics 2005;23(9):855–74. [14] Kim SY, Goldie SJ. Cost-effectiveness analyses of vaccination programmes: a focused review of modelling approaches. Pharmacoeconomics 2008;26(3):191–215. [15] Beutels P, Scuffham P, MacIntyre CR. Funding of drugs: do vaccines warrant a different approach? Lancet Infect Dis 2008. [16] Bilcke J, Beutels P. Reviewing the cost effectiveness of rotavirus vaccination: the importance of uncertainty in the choice of data sources. Pharmacoeconomics 2009;27(4):281–97. [17] Chiou CF, Hay JW, Wallace JF, Bloom BS, Neumann PJ, Sullivan SD, et al. Development and validation of a grading system for the quality of cost-effectiveness studies. Med Care 2003;41(1):32–44. [18] Brisson M, Edmunds WJ. Impact of model, methodological, and parameter uncertainty in the economic analysis of vaccination programs. Med Decis Making 2006;26(5):434–46. [19] World Health Organization. WHO Guide for standardization of economic evaluations of immunization programmes. Geneva: WHO; 2008. WHO/IVB/ 08.04, http://www.who.int/immunization/documents/WHO IVB 08.14/en/ index.html. [20] Tan Torres-Edejer T, Baltussen R, Adam T, Hutubessy R, Acharya A, Evans DB, et al. Making choices in health: WHO guide to cost-effectiveness. Geneva: WHO; 2003. [21] Lee RH. Future costs in cost effectiveness analysis. J Health Econ 2008;27(4): 809–18. [22] Weinstein MC, Toy EL, Sandberg EA, Neumann PJ, Evans JS, Kuntz KM, et al. Modeling for health care and other policy decisions: uses, roles, and validity. Value Health 2001;4(5):348–61. [23] Disease Control Priorities in developing countries. 2nd ed. New York: The World Bank; Oxford University Press; 2006. [24] Loewenstein G, Prelec D. Anomalies in intertemporal choice: Evidence and an interpretation. Quart J Health Econ 1992;107(2):573. [25] Harvey C. Proportional discounting of future costs and benefits. Math Oper Res 1995;20(2):381–99. [26] World Health Organization Commission on Macroeconomics and Health. Macroeconomics and health: investing in health for economic development. Report of the Commission on Macroeconomics and Health. Geneva: World Health Organization; 2001. [27] Beutels P, Van Doorslaer E, VAN Damme P, Hall J. Methodological issues and new developments in the economic evaluation of vaccines. Expert Rev Vaccines 2003;2(5):649–60. [28] Welte R, van den DG, Bos JM, de Melker H, van Alphen L, Spanjaard L, et al. Economic evaluation of meningococcal serogroup C conjugate vaccination programmes in The Netherlands and its impact on decision-making. Vaccine 2004;23(4):470–9. [29] Bos JM, Rumke H, Welte R, Postma MJ. Epidemiologic impact and costeffectiveness of universal infant vaccination with a 7-valent conjugated pneumococcal vaccine in the Netherlands. Clin Ther 2003;25(10): 2614– 30. [30] Andrus JK, Toscano CM, Lewis M, Oliveira L, Ropero AM, Davila M, et al. A model for enhancing evidence-based capacity to make informed policy decisions on the introduction of new vaccines in the Americas: PAHO’s ProVac initiative. Public Health Rep 2007;122(6):811–6. [31] Musgrove P. Public spending on health care: how are different criteria related? Health Policy 1999;47(3):207–23. [32] The GAVI Alliance’s new vaccine strategy. Lancet 2008;372(9632):2.
Contents lists available at ScienceDirect
Vaccine journal homepage: www.elsevier.com/locate/vaccine
WHO Guide for standardisation of economic evaluations of immunization programmes Damian G. Walker a,∗ , Raymond Hutubessy b , Philippe Beutels c a
Health Economics, Health Systems Program, Department of International Health, Bloomberg School of Public Health, Johns Hopkins University, 615 North Wolfe Street, Baltimore, MD 21205, USA Initiative for Vaccine Research World Health Organization Avenue Appia 20, CH1211-Genève 27, Switzerland c Centre for Health Economics Research and Modelling Infectious Diseases (CHERMID), Centre for the Evaluation of Vaccination, Vaccine & Infectious Disease Institute, University of Antwerp, Universiteitsplein 1, 2610 Antwerp, Belgium b
a r t i c l e
i n f o
Article history: Received 8 June 2009 Accepted 9 June 2009 Available online 28 June 2009 Keywords: Economic evaluations Cost-effectiveness analysis Guidelines Immunization programmes Vaccines
a b s t r a c t Traditional EPI vaccines are considered to be among the most efficient uses of scarce health care resources. Today, there are many under-used and new vaccines available. In the short- to medium-term, these vaccines will not cost the few cents per dose the traditional vaccines do, but will be ‘multi-dollar’ vaccines. Decision-makers will need information, among other things, on their relative cost-effectiveness. A number of reviews have indicated that there is scope for improving the transparency, completeness and comparability of economic evaluations of vaccination programmes. Thus, there is a need to improve the quality of economic evaluations of vaccination programmes. Adherence to general guidelines would increase the quality, interpretability and transferability of future analyses. However, there is reason to believe that there might also be a need for more specific advice for vaccination programmes. For example, there are inconsistencies in the methods used to estimate the future benefits of vaccination programmes and the relative efficiency of these programmes can be sensitive to some of the more controversial aspects of general guidelines, such as the inclusion of indirect costs and the discounting of health outcomes. This guide has been developed in order to meet the needs of decision-makers for relevant, reliable and consistent economic information. They aim to provide clear and concise, practical and high quality guidance for those who conduct economic evaluations. © 2009 Elsevier Ltd. All rights reserved.
1. Introduction Traditional EPI vaccines are considered to be among the most efficient uses of scarce health care resources [1]. Today, many vaccines are available that are not yet used as widely as they could be (e.g., against yellow fever, hepatitis B virus (HBV), Haemophilus influenzae type b (Hib), rotavirus, Streptococcus pneumoniae, human papilloma virus (HPV) and Japanese encephalitis). Furthermore, many more vaccines are in the pipeline, e.g., against enterotoxigenic Escherichia coli, shigellosis, dengue, hookworm, schistosomiasis, herpes simplex virus 2, human immunodeficiency virus (HIV), malaria and tuberculosis (TB). In the short- to medium-term, these vaccines will not cost the few cents per dose the traditional vaccines do, but will be ‘multi-dollar’ vaccines. Policy makers will need information, among other things, on their relative cost-effectiveness to decide whether they should be (more) widely used.
∗ Corresponding author. Tel.: +1 410 502 6023; fax: +1 410 614 1419. E-mail addresses: [email protected] (D.G. Walker), [email protected] (R. Hutubessy), [email protected] (P. Beutels). 0264-410X/$ – see front matter © 2009 Elsevier Ltd. All rights reserved. doi:10.1016/j.vaccine.2009.06.035
A number of reviews have indicated that there is scope for improving the transparency, completeness and comparability of economic evaluations of vaccination programmes [2–16]. Thus, there is a need to improve the quality of economic evaluations of vaccination programmes. Adherence to general guidelines would increase the quality, interpretability and transferability of future analyses [17]. However, there is reason to believe that there might also be a need for more specific advice for vaccination programmes. For example, there are inconsistencies in the methods used to estimate the future benefits of vaccination programmes and the relative efficiency of these programmes can be sensitive to some of the more controversial aspects of general guidelines, such as the inclusion of indirect costs and the discounting of health outcomes [18]. This WHO Guide [19] has been developed with the aim to guide economics and health services researchers in the public and private sector who conduct and critically appraise economic evaluations of immunization programmes. It aims to provide clear and concise, practical and high quality guidance for performing and presenting the results of economic evaluations, in a relevant, consistent and timely manner. It was reviewed by participants of a WHO meeting hosted by the Program for Appropriate Technology in Health (PATH)
D.G. Walker et al. / Vaccine 28 (2010) 2356–2359
that took place in Seattle, USA on 6–7 March 2007, other independent reviewers and subsequently endorsed by WHO through their Quantitative Immunization and Vaccine related Research (QUIVER) Advisory Panel. In what follows, we summarise the WHO Guide per step an analyst takes while conducting an economic evaluation. 2. Framing the evaluation The first step to conducting an economic evaluation is to frame the study. Decisions made at this stage will directly determine which costs and outcomes are considered relevant and should therefore be included in the analysis. This means that choices made in the framing of the evaluation will have an impact on the final results of an analysis. First and foremost, the study question should be well-defined, stated in an answerable form and relevant to the decision facing the target audience. The comparators under evaluation should be clearly described. The most relevant comparison for new vaccines is usually current practice. If existing practice itself appears to be a cost-ineffective option relative to other available options, the analyst should incorporate other relevant alternatives into the analysis, such as a best available alternative, a viable low-cost alternative or a do-nothing option. The form of economic evaluation should be clearly stated and justified. CUA is the preferred type of evaluation (with DALYs or QALYs as outcome measures), although a CEA, which presents outcomes using natural units as outcomes measures specific to the vaccine(s) in question, is also encouraged. Ideally analyses should take the perspective of society, and include all related effects and costs, regardless of who benefits from or pays for them. Nonetheless, the costs borne by providers (e.g., donors and governments), patients and their families and others should be disaggregated as far as possible in order to allow judgments to be made from the perspectives of the various decision-makers. It is important to clearly state the person(s) or institution(s) sponsoring the study. Finally, the time frame and analytic horizon should be clearly stated; their respective durations are contingent on the type of vaccine evaluated, the intervention and target group, and thus the type of model developed (see section below on modelling). 3. Assessing the cost of a vaccination programme The exact nature of the costs assessed will depend on the scope of the analysis and the perspective(s) adopted. Whatever the scope, the methods for the estimation of costs should be clearly stated. A summary should be provided of the expected resource use and unit costs for each alternative. This should include specifying the assumptions behind calculations of costs, e.g., amounts and types of health service use with and without the alternative, given a specific coverage of the alternative and indicating actual and potential ranges of each estimate. A full costing study should be considered if precise estimates are needed and if it is worth the additional efforts required. In all other cases it is recommended to use standardised WHO-CHOICE [20] estimates or existing country-specific cost data if available. Costs for patients and their families, including lost productivity if considered, should be reported separately. These guidelines recognize that several methods exist for valuing lost productivity; analysts should therefore make clear and justify why a particular method was chosen and what its pros and cons are. So-called future unrelated costs should not be included both because of the practical difficulties of estimation and because their inclusion involves ethical and conceptual issues, for which currently consensus is lacking in the field of health economics [21]. Finally, costs should be reported in local currency units, ideally using the most recent year as the base-year. Additionally to allow regional or global comparability, conversions to US$ using
2357
official exchange rates for the base-year in question and to I$ using purchasing power parity (PPP) exchange rates are recommended. 4. Assessing the effects of a vaccination programme Estimates of vaccine efficacy should be based upon systematic reviews of the available literature where available, taking account of the biological characteristics of the pathogen in question and how its infectious nature may have influenced the efficacy estimates derived from trials. Efficacy estimates should be adjusted for loss of potency due to heat and freeze exposure, where such data are available and multiplied by routine vaccination coverage, accounting for non-compliance (based on relevant resources depending on the type of program), if adverse events from immunization are likely to have a substantial impact on the results of the analysis, they should be included on both the costs and effects side of the analysis. The significance of the impact depends on both their likelihood of occurring as a consequence of vaccination and their severity. Finally, analysts should first present estimates of burden in natural units. These should include cases, deaths, years of life lost (YLL) and years lived with disability (YLD). These natural units can then be converted to DALYs (and these should be estimated both with and without age weighting). 5. Modelling To estimate the population impact of vaccination (i.e., to turn efficacy measures explained in the previous section into estimates of effectiveness), some form of mathematical modelling is inevitable. For economic evaluation, the guide generally advocates choosing the model that minimally meets the analytical requirements given the pathogen, the endemic situation and the intervention. Guidance was provided mainly on the generally most influential model attribute choice for the estimated costeffectiveness ratio of vaccination: choosing between static and dynamic models. A dynamic model is defined as a mathematical model in which the force of infection is a function of the proportion of infectious people in the population at each time point. The force of infection can thus change over time in this type of model. A static model, on the other hand is defined as a mathematical model in which the force of infection is assumed to be independent of the proportion of infectious people at each time point. Clearly, as transparency is key in model-based economic analysis, analysts should report on all the modelling attributes, not just on the static versus dynamic choice. Beutels et al. (forthcoming) explain the guidance on accounting for vaccine effects and choosing models in a separate publication. In the current summary, we highlight the following recommendations the guide makes. It is recommended that the mathematical model should be: • Transparent in which the structure and implicit or explicit assumptions are all clearly described. • Static, if vaccination is unlikely to change the force of infection in susceptibles or as a means to make a worst case estimate when externalities from herd-immunity cannot on the whole be adverse. • Dynamic, if vaccination is likely to change the force of infection in susceptibles, and a static model would not yield a worst case estimate, or if the worst case from a static model does not lead to a favourable outcome. • Stochastic if chance plays an important role in the transmission process of the pathogen.
2358
D.G. Walker et al. / Vaccine 28 (2010) 2356–2359
• Validated, in as many facets of validation (verification, calibration, face validity, predictive validity [22] as possible, but at least verified. 6. Discounting For curative therapies, most benefits accrue immediately or shortly after the intervention is initiated, and the cost-effectiveness of these interventions is therefore largely independent of the discount rate. Conversely, the cost-effectiveness of most vaccination programmes is highly sensitive to discounting. The guidelines recommend that costs and effects occurring in the future should initially be discounted using the rate in the country in question (for studies to inform local decision-makers) and then using a 3% discount rate (consistent with WHO-CHOICE and DCP2 [20,23]. However, it is recommended that analysts conduct sensitivity analysis using discount rates of 0%, near-zero, 5% and 10% to reflect the (probably) higher real risk-free cost of capital in developing countries, if the evaluation is being conducted in such a setting. A non-constant (declining or ‘slow’) discounting [24,25] procedure may be applied where the effects begin only long after the intervention, e.g., vaccination against HBV or HPV, or last for an exceptionally long time, e.g., polio eradication. 7. Estimating, presenting and interpreting data It is important that analysts begin by excluding those interventions that are dominated, i.e., both more costly and less effective than alternative options. The classification of costeffective and cost-ineffective interventions should not be made on the basis of point estimates of cost-effectiveness because of uncertainty about the estimates of cost, effects and hence cost-effectiveness. Therefore, as a minimum, analysts should conduct one-way sensitivity analyses of the following variables: discount rate, vaccination effectiveness (where unknown or uncertain), incidence of disease (including complication rates where relevant), direct medical costs, case fatality rates and vaccine price. Additionally, it is recommended that analysts, when possible, conduct probabilistic sensitivity analyses on all parameters and construct cost-effectiveness acceptability curves. In addition to performing such sensitivity analyses to quantify parameter uncertainty, analysts should carefully consider the extent of uncertainty introduced by the choices they made with regards to model structure and selection of data sources. If alternatives for these choices exist, they should consider and discuss, and possibly quantify how these affected the results of their analyses. Analysts should use the recommendation of the Commission on Macroeconomics and Health [26] that has been endorsed by WHO and classify the results of their evaluation according to the per capita national GDP of the country in question: less than one classifies a vaccine as ‘highly cost-effective’; between one and three times classifies an intervention as ‘cost-effective’; and more than three times classifies an intervention as ‘cost-ineffective’. However, analysts should also place their findings in broader context by comparing their findings to other economic evaluations that have been undertaken in the same or neighbouring countries after adjustments have been made for inflation. 8. The use of economic evaluation in policy and practice Although more than 200 economic evaluations of vaccination programmes have been published [27], there are not many examples in the literature of evidence about their specific use at
global, regional, national or local levels of decision-making (e.g., see Welte et al. [28], Bos et al. [29] for published accounts for The Netherlands). However, many anecdotal accounts exist about the use of such economic evaluations, mainly at a national level, to aid price negotiations and reductions in the proposed number of doses. Less clear is its role as a decisive influence, rather than a justification, for general decisions on whether or not a vaccine should be introduced in a national vaccination programme [6,13]. While some countries have a separate advisory group on vaccination, which uses evidence on cost-effectiveness and exerts great influence on policy, e.g., the United Kingdom’s Joint Committee on Vaccination and Immunisation and the United States’ Advisory Committee on Immunization Practices, most do not—or else their financial backing is only sufficient to sustain advice on the basis of the literature and ad hoc expert opinion. Most GAVI-eligible countries, however, have little or limited access to formal advisory bodies to review immunization data and provide independent advice to their respective governments. There is a need to support the establishment of national or regional processes to enhance evidence-based decision-making in immunization (and health more generally) in order to routinely and formally address questions such as whether, when and how a new vaccine should be introduced. In order to make informed decisions, countries need to have both the necessary evidence and clear processes. Much work has been done and is underway (for example see Andrus et al. [30] for Latin America) to develop the evidence base countries need for making informed decisions; little effort has thus far been expended to ensure that countries have processes in place to evaluate and use this information. 9. Other criteria to consider when making decisions While the emphasis of this guide is on value for money, i.e., whether a vaccine is worth buying and not who pays for it, if the object is to decide how to spend public funds, economic evaluation is only one of at least nine criteria relevant for priority-setting in health [31]. Cost alone matters, as do the capacities of potential beneficiaries to pay for an intervention. The other criteria that may affect priorities include horizontal equity (equal treatment for people in equal circumstances); vertical equity (priority for people with worse problems); adequacy of demand; and public attitudes and wants. Two criteria, whether an intervention is a public good and whether it yields substantial externalities, are classic justifications for public intervention, because private markets could not supply them efficiently, just as in other sectors. Equity, poverty, and risk of impoverishment from ill health may also influence priorities; so do the budgets available, and the decisions of how much to make available for buying interventions. Finally, the effectiveness of an intervention and, therefore, the degree to which it deserves priority depends on how far it is culturally appropriate or acceptable for the population it is intended to benefit. The identical intervention, technically speaking, may lead to different degrees of use or compliance in different population groups, and information and incentives may be needed to achieve the full potential outcomes. Polio vaccination is an example of where other elements in the decision-making process have ‘trumped’ the ‘logic of costeffectiveness’, in view of the achievement of polio elimination from high and middle income countries [15]. Therefore, concerns over the public’s perception about the risks associated with OPV led to its replacement by the risk-free IPV in most high income countries. Interestingly, however, one must assume that value for money has thus far been an important part of the rationale to continue using OPV in low income countries.
D.G. Walker et al. / Vaccine 28 (2010) 2356–2359
10. Conclusions Decision-making for vaccines is getting tougher [15]. Over the next decade, a number of new vaccine products will become available. With the advent of more expensive new products (“product pile-up”), countries will face a significant decision-making challenge [32]. Data regarding the relative cost-effectiveness of these products will be an important criterion for decision-makers to consider. This guide does not propose to alter the general guidelines for economic evaluations, but merely a specific interpretation of them with respect to vaccination and a more rigorous application of them in general. Economic evaluations in the field of vaccine-preventable diseases, which are often complicated by many parameters and assumptions, should first of all be explicit and transparent. All assumptions should be clearly stated and justified. Sections dealing with methods and assumptions should clearly and explicitly describe all weaknesses of the analysis. Acknowledgments We would like to thank the participants of a WHO meeting hosted by the Program for Appropriate Technology in Health (PATH) that took place in Seattle on 6–7 March 2007 to review a draft of these guidelines for their helpful comments: Jon Andrus (PAHO), Deborah Atherly (PATH), Daniel Barth-Jones (Wayne State University), Ruanne Barnabas (Fred Hutchinson Cancer), Andrew Clark (London School of Hygiene & Tropical Medicine), Nancy Engel (PATH), David Evans (WHO), Ulla Griffiths (London School of Hygiene, and Tropical Medicine), Vipat Kuruchittham (Chulalongkorn University), Carol Levin (PATH), Rosamund Lewis (GAVI Alliance), Patrick Lydon (WHO), Rocio Mosqueira (Instituto de Investigación Nutricional), Richard Rheingans (Emory University) Chutima Suraratdecha (PATH), Tessa Tan-Torres (WHO), Abdelmajid Tibouti (UNICEF), Cristiana Toscano (PAHO) Maya Vijayaraghavan (CDC) and Lara Wolfson (WHO). We would also like to thank Logan Brenzel (World Bank), Sue Goldie (Harvard University), Maarten Postma (Groningen University) and members of the International Federation of Pharmaceutical Manufacturers & Association (IFPMA) Biologicals and Vaccines Committee for detailed written comments. Finally we would like to thank the QUIVER members and meeting participants of the inaugural WHO QUIVER Advisory Group meeting that took place in Geneva on 27–28 September 2007. PB acknowledges funding from Simulation Models for Infectious Disease Processes (SIMID), a strategic basic research project funded by the Institute for the Promotion of Innovation by Science and Technology in Flanders (project number 060081). Disclaimer: Raymond Hutubessy is a staff member of the World Health Organization. The author alone is responsible for the views expressed in this publication and they do not necessarily represent the decisions, policy or views of the World Health Organization. References [1] World Bank. World development report 1993: investing in health. New York: Oxford University Press; 1993. [2] Beutels P. Economic evaluations of hepatitis B immunization: a global review of recent studies (1994–2000). Health Econ 2001;10(8):751–74. [3] Beutels P, Postma MJ. Economic evaluations of adult pneumococcal vaccination: a review of the literature. Expert Rev Pharmacoecon Outcomes Res 2001;1(1):47–58. [4] Beutels P. Potential conflicts of interest in vaccine economics research: a commentary with a case study of pneumococcal conjugate vaccination. Vaccine 2004;22(25–26):3312–22.
2359
[5] Beutels P, Thiry N, VAN Damme P. Convincing or confusing? Economic evaluations of childhood pneumococcal conjugate vaccination—a review (2002–2006). Vaccine 2007;25(8):1355–67. [6] Brinsmead R, Hill S, Walker D. Are economic evaluations of vaccines useful to decision-makers? Case study of Haemophilus influenzae type b vaccines. Pediatr Infect Dis J 2004;23(1):32–7. [7] Edmunds WJ. Universal or selective immunisation against hepatitis B virus in the United Kingdom? A review of recent cost-effectiveness studies. Commun Dis Public Health 1998;1(4):221–8. [8] Jefferson T, Demicheli V. Is vaccination against hepatitis B efficient? A review of world literature. Health Econ 1994;3(1):25–37. [9] Newall AT, Beutels P, Wood JG, Edmunds WJ, MacIntyre CR. Costeffectiveness analyses of human papillomavirus vaccination. Lancet Infect Dis 2007;7(4):289–96. [10] Pegurri E, Fox-Rushby JA, Damian W. The effects and costs of expanding the coverage of immunisation services in developing countries: a systematic literature review. Vaccine 2005;23(13):1624–35. [11] Thiry N, Beutels P, VAN Damme P, Van Doorslaer E. Economic evaluations of varicella vaccination programmes: a review of the literature. Pharmacoeconomics 2003;21(1):13–38. [12] Walker D, Rheingans R. Cost-effectiveness of rotavirus vaccines. Expert Rev Pharmacoecon Outcomes Res 2005;5(5):593–601. [13] Welte R, Trotter CL, Edmunds WJ, Postma MJ, Beutels P. The role of economic evaluation in vaccine decision making: focus on meningococcal group C conjugate vaccine. Pharmacoeconomics 2005;23(9):855–74. [14] Kim SY, Goldie SJ. Cost-effectiveness analyses of vaccination programmes: a focused review of modelling approaches. Pharmacoeconomics 2008;26(3):191–215. [15] Beutels P, Scuffham P, MacIntyre CR. Funding of drugs: do vaccines warrant a different approach? Lancet Infect Dis 2008. [16] Bilcke J, Beutels P. Reviewing the cost effectiveness of rotavirus vaccination: the importance of uncertainty in the choice of data sources. Pharmacoeconomics 2009;27(4):281–97. [17] Chiou CF, Hay JW, Wallace JF, Bloom BS, Neumann PJ, Sullivan SD, et al. Development and validation of a grading system for the quality of cost-effectiveness studies. Med Care 2003;41(1):32–44. [18] Brisson M, Edmunds WJ. Impact of model, methodological, and parameter uncertainty in the economic analysis of vaccination programs. Med Decis Making 2006;26(5):434–46. [19] World Health Organization. WHO Guide for standardization of economic evaluations of immunization programmes. Geneva: WHO; 2008. WHO/IVB/ 08.04, http://www.who.int/immunization/documents/WHO IVB 08.14/en/ index.html. [20] Tan Torres-Edejer T, Baltussen R, Adam T, Hutubessy R, Acharya A, Evans DB, et al. Making choices in health: WHO guide to cost-effectiveness. Geneva: WHO; 2003. [21] Lee RH. Future costs in cost effectiveness analysis. J Health Econ 2008;27(4): 809–18. [22] Weinstein MC, Toy EL, Sandberg EA, Neumann PJ, Evans JS, Kuntz KM, et al. Modeling for health care and other policy decisions: uses, roles, and validity. Value Health 2001;4(5):348–61. [23] Disease Control Priorities in developing countries. 2nd ed. New York: The World Bank; Oxford University Press; 2006. [24] Loewenstein G, Prelec D. Anomalies in intertemporal choice: Evidence and an interpretation. Quart J Health Econ 1992;107(2):573. [25] Harvey C. Proportional discounting of future costs and benefits. Math Oper Res 1995;20(2):381–99. [26] World Health Organization Commission on Macroeconomics and Health. Macroeconomics and health: investing in health for economic development. Report of the Commission on Macroeconomics and Health. Geneva: World Health Organization; 2001. [27] Beutels P, Van Doorslaer E, VAN Damme P, Hall J. Methodological issues and new developments in the economic evaluation of vaccines. Expert Rev Vaccines 2003;2(5):649–60. [28] Welte R, van den DG, Bos JM, de Melker H, van Alphen L, Spanjaard L, et al. Economic evaluation of meningococcal serogroup C conjugate vaccination programmes in The Netherlands and its impact on decision-making. Vaccine 2004;23(4):470–9. [29] Bos JM, Rumke H, Welte R, Postma MJ. Epidemiologic impact and costeffectiveness of universal infant vaccination with a 7-valent conjugated pneumococcal vaccine in the Netherlands. Clin Ther 2003;25(10): 2614– 30. [30] Andrus JK, Toscano CM, Lewis M, Oliveira L, Ropero AM, Davila M, et al. A model for enhancing evidence-based capacity to make informed policy decisions on the introduction of new vaccines in the Americas: PAHO’s ProVac initiative. Public Health Rep 2007;122(6):811–6. [31] Musgrove P. Public spending on health care: how are different criteria related? Health Policy 1999;47(3):207–23. [32] The GAVI Alliance’s new vaccine strategy. Lancet 2008;372(9632):2.