- Email: [email protected]
An Evidence Framework for Off-Patent Pharmaceutical Review for Health Technology Assessment in Emerging Markets
VALUE IN HEALTH REGIONAL ISSUES 16C (2018) 9–13
Available online at www.sciencedirect.com
journal homepage: www.elsevier.com/locate/vhri
An Evidence Framework for Off-Patent Pharmaceutical Review (EFOR) for Health Technology Assessment in Emerging Markets Diana Brixner, PhD, RPh (Professor, Executive Director)1,2,⁎, Zoltán Kaló, MSc, MD, PhD (Professor)3, Nikos Maniadakis, BSc, MSc, PhD, FESC (Professor)4, Kyoo Kim, MSc, MBA (Global Market Access and Health Economics Director)5, Kalman Wijaya, BSc, Pharm, MBA (Global Market Access and Policy Senior Manager)5 1 Department of Pharmacotherapy, University of Utah College Of Pharmacy; 2Pharmacotherapy Outcomes Research Center. Salt Lake City, UT, USA; 3Department of Health Policy and Health Economics, Eötvös Loránd University (ELTE), Budapest; Syreon Research Institute. Budapest, Hungary; 4Health Services Organization and Management, Alternate Dean, National School of Public Health. Athens, Greece; 5Abbott Established Pharmaceutical Division. Basel, Switzerland
AB STR A CT
Objective: This article introduces an Evidence Framework for OffPatent Pharmaceutical Review (EFOR), which establishes value-based criteria in a template that manufacturers use to provide evidence showing how their products meet those criteria. Health authorities in emerging markets can then use the evidence presented in the EFOR to evaluate off-patent pharmaceuticals (OPPs) in a consistent, transparent, and evidence-based manner to support policy decisions, including pricing, reimbursement, formulary listing, and drug procurement. Methods: A literature search found no multi-criteria evidence framework for evaluating OPPs in emerging markets. An International Outcomes Research Board (IORB) of academia and industry experts conducted extensive research, meetings, and workshops to define high-priority criteria to incorporate into an evidencebased health technology assessment (HTA) tool using the multicriteria decision analysis (MCDA) technique. The resulting framework was further tailored for country-specific needs in workshops in three emerging countries (Kazakhstan, Vietnam, and Indonesia). Results: The IORB defined nine criteria four categories (Product, Manufacturing, Service, and Value Assessment), which OPP manufacturers can
use to provide evidence for reimbursement and health policy decision making. Then the IORB developed the EFOR as a base case document, which can be adapted and used as a template by health authorities in emerging countries. Conclusions: Emerging countries have a significant need for an HTA tool that balances affordability with accurate evidence showing the value differentiation of OPPs. The value attributes in this setting often are different from those in developed markets, which emphasize new products and have high regulation and manufacturing standards. The EFOR is an easy-to-use, adaptable framework that emerging countries can use to increase the consistency, transparency, and effectiveness of drug decision making. The open source EFOR is available as Supplemental Materials. Keywords: dossier, emerging markets, evidence framework, EFOR, generic pharmaceuticals, health technology assessment, MCDA, multi-criteria decision analysis, off-patent pharmaceuticals.
Introduction
However, utilization of new technologies and pharmaceuticals is less common in emerging markets. Instead, emerging markets, which comprise the majority of the world’s population, rely heavily on off-patent pharmaceuticals (OPPs). OPPs include International Nonproprietary Name generics, branded generics, and off-patent originators. OPPs are an especially important component of health care, representing 60% to 80% of the volume of the pharmaceutical market in most emerging markets [3–6]. Although it is common to use the term off-patent pharmaceuticals to refer collectively to all products that are similar to the innovator product after the patent has expired, there are important distinctions between drugs that are given this label. The first type of OPP is an innovator product that receives a new name and
In developed markets, when new medications or medical technology products are introduced, health authorities have health technology assessment (HTA) tools to help them understand, evaluate, and compare the products for use in their markets. Dossier templates, such as the Academy of Managed Care Pharmacy (AMCP) Format for Formulary Submissions Version 4.0 [1] and the EUnetHTA Core Model [2], provide structured presentations of evidence to help health authorities balance known efficacy and safety results with total therapy costs, for making pricing, reimbursement, formulary listing, and product procurement choices.
& 2018 Published by Elsevier Inc. on behalf of International Society for Pharmacoeconomics and Outcomes Research (ISPOR).
The authors have indicated that they have no conflicts of interest with regard to the content of this article. * Address correspondence to: Diana Brixner, PhD, RPh, University of Utah, Department of Pharmacotherapy, 30 S 2000 E, Suite 258, Salt Lake City, UT 84112, USA. E-mail: [email protected] 2212-1099$36.00 – see front matter & 2018 Published by Elsevier Inc. on behalf of International Society for Pharmacoeconomics and Outcomes Research (ISPOR). https://doi.org/10.1016/j.vhri.2018.01.003
10
VALUE IN HEALTH REGIONAL ISSUES 16C (2018) 9–13
designation after the patent expires. All other products are considered true generic products, which are required to demonstrate their similarity to the innovator. Although often assumed to be identical to the originator drug or to other generic drugs, these OPPs, in fact, can vary greatly in pharmaceutical equivalence and bioequivalence (in which evidence shows the OPP is as effective and safe for patients in the specific indication at the same dose, form, route of administration, etc., regardless of differences in excipients or manufacturing processes). Such variations in pharmaceutical equivalence and bioequivalence are especially prevalent in emerging markets. Additionally, OPP manufacturers can vary in their implementation of good manufacturing practice (GMP) standards, which can alter or compromise the drug’s efficacy and/or safety [7–9]. To help address this problem, the World Health Organization (WHO) has published guidelines for establishing more formal procedures for countries to follow when procuring pharmaceutical products, recommending that countries procure the most costeffective drugs from the most reliable suppliers of high-quality products [10,11]. However, the WHO’s guidelines do not provide a specific template for gathering evidence across a wide range of criteria to use when evaluating products. Although current evaluation frameworks are available in developed countries, in general, they only focus on innovative medicine; there have been no equivalent tools for evaluating OPPs in emerging markets. This means that health authorities in emerging countries are often left without a consistent, practical, and objective way to evaluate an OPP and instead can only assume that the drug is reasonably equivalent. This assumption that all OPPs are equivalent to their originator drug, adhere to identical GMP standards, and have equal supply reliability, despite a lack of evidence to confirm the assumption, can cause health authorities in emerging markets to make coverage decisions based purely on price. Reliance on price alone as a decision-making tool can be detrimental to a country’s health goals when two or more OPPs based on the same originator drug are, in fact, quite different in terms of efficacy, safety, delivery, or reliability of distribution systems, for example [8,12]. In fact, a recent WHO study showed that the rate of substandard and counterfeit drugs in low- and middle-income countries was approximately 10.5% [13]. To address the significant need for systematic and evidencebased evaluations for OPPs in emerging markets, an International Outcomes Research Board (IORB) of academia and industry experts conducted extensive research, exploratory meetings, and workshops to develop a customized multi-criteria decision analysis (MCDA) tool accommodating the unique needs of these markets. This tool, the Evidence Framework for Off-Patent Pharmaceutical Review (EFOR), is an evidence-based HTA tool based on the MCDA technique [14]. The EFOR is a template that provides a consistent, defined, high-priority set of desired information, and it can be adapted to the specific needs of a particular country. Manufacturers use the EFOR to provide evidence showing how their products meet those criteria, which health policymakers and reimbursement agencies in emerging markets can then use to evaluate OPPs to make pricing, reimbursement, formulary listing, or drug purchase decisions. The EFOR provides a consistent decision-making framework based on relevant criteria specifically designed for OPPs in emerging markets. Evidence presented in the EFOR is intended to be used by payers, institutions, committees, procurement agencies, HTA organizations and health ministries.
Methods In a traditional MCDA Simple Scoring decision-making process, a set of criteria are defined, ranked by relevance and importance,
and evaluated. As MCDA became more widely used in a variety of industries, multiple MCDA methodologies appeared, and their complexity and inconsistencies grew. Therefore, in 2014, the International Society of Pharmacoeconomics and Outcome Research (ISPOR) established the MCDA Emerging Good Practices Task Force to develop good practice guidelines for implementing MCDA in health care decision making [15–17]. Even with those guidelines, however, MCDA was still considered overly complex or too context specific for use in some environments. Therefore, an IORB conducted theoretical and practical application research to develop MCDA as a streamlined evidence-based HTA methodology for OPPs, designed specifically to be adapted for use in individual emerging countries. With MCDA techniques, a complex set of research principles are translated into a practical set of criteria to weight characteristics important to relevant health policy decisions for emerging markets [14]. After an extensive literature search determined that a streamlined HTA methodology for evaluating OPPs in emerging markets did not exist, a workshop was held with emerging market country representatives at the ISPOR Annual European Meeting in Milan, Italy, in 2015 to identify and prioritize criteria that health care systems in emerging countries could use in an MCDA format to evaluate medications and determine coverage. The workshop involved 57 health care experts and decision makers from 14 countries: China, Russia, Egypt, Thailand, Turkey, Indonesia, Kazakhstan, Vietnam, the Philippines, Colombia, Taiwan, Malaysia, Algeria, and Saudi Arabia. These countries represent emerging markets in regions defined by the WHO: Africa, the Americas, Eastern Mediterranean, Europe, Southeast Asia, and Western Pacific. The participants represented several professional realms, including academia, ministries of health, drug procurement agencies, and drug pricing and reimbursement agencies. The workshop participants were asked to prioritize 22 criteria (identified from literature searches and further research). These 22 criteria and the MCDA Simple Scoring process used to identify them were published in 2017 [14]. After the MCDA criteria were prioritized in the workshop, the criteria and the MCDA process were presented and feedback was gathered from health authorities in decision-making positions regarding generic drug reimbursement at the Health Technology Assessment International (HTAi) conference in Japan in 2016 and at the ISPOR conference in Singapore in 2016. In parallel, the IORB continued to develop the EFOR template based on those criteria, comparing the EFOR to drug dossier formats and submission templates, such as the AMCP Format, the ACTD format, and the EUnetHTA HTA Core Model, to ensure that the documents will be complementary, rather than overlapping, providing health care authorities with more robust evidence to help make decisions about the incorporation of OPPs into the respective country’s health care system. After the HTAi Japan and ISPOR Singapore meetings, a small group of researchers, including a pharmacist, a medical doctor, and two health economists scrutinized all of the criteria from the viewpoint of completeness, non-redundancy, non-overlap, and preference independence. They further refined the list of criteria to reduce the number of criteria to nine, which was considered both optimal for ease of use and for adaptability to a variety of markets. By using the final recommended MCDA criteria, a “base case” EFOR that can be used and adapted to specific countries’ needs was developed. This base case template was then validated in workshops held in three emerging markets: Kazakhstan, Vietnam, and Indonesia. The resulting MCDA tool kit and the EFOR can be adapted in different settings to make decisions regarding OPPs. The EFOR provides the evidence to be evaluated in a systematic and anchored manner to facilitate an MCDA-based value-oriented assessment of OPPs.
VALUE IN HEALTH REGIONAL ISSUES 16C (2018) 9–13
Results Initial research and workshop discussions of the highest-priority factors that should be considered when evaluating an OPP yielded a set of 22 criteria in four categories.14 However, the MCDA task force report [15,16] recommends limiting the number of criteria to a manageable set for any particular decision in which MCDA is being applied. The task force found that the average number of criteria used in health care decisions was 8.2, with a range from three to 19 criteria [16,17]. With this principle in mind, the IORB evaluated the original 22 criteria for completeness, non-redundancy, non-overlap, preferential independence, local relevance, and the feasibility of measuring them within emerging markets [16]. Some criteria with potential dependencies were merged into a single criterion. For example, proving pharmaceutical equivalence is a necessary step prior to bioequivalence studies, and without evidence on bioequivalence information, therapeutic equivalence cannot be expected. Therefore, pharmaceutical, bioequivalence, and
11
therapeutic equivalences were merged into a single categorical criterion called “equivalence with the reference (original) product.” In other cases, if two criteria were correlated, one could be dropped from the list, for example, “equivalence with the reference product” had significant overlap with “level of interchangeability,” so the latter was removed. Finally, some criteria were considered complementary to each other, and so “technical assistance,” “disease awareness and education,” and “continued medical education” were merged into a single criterion: “added value services related to the product.” After merging similar criteria and eliminating criteria for which no scoring function could be identified, the IORB identified a final list of nine key criteria (shown in Table 1). The criteria were divided into four categories: Product, Manufacturer, Service, and Value Assessment. Then an EFOR “base case” was developed. This base case defines each of the nine criteria and provides a simple scoring scale for each criterion that health authorities can use to objectively measure how the OPP meets that criterion. With the scoring scale, points are awarded for each criterion and then
Table 1 – Outline of the EFOR base case. 1.0 Executive Summary Summarizes the evidence showing how the product meets the established criteria for consideration by health authorities. 2.0 Product Description 2.1 Disease Description, Incidence, and Prevalence 2.2 Product Description and Place in Therapy 2.3 Current Treatment Options
3.0 Evidence: Product Criteria 3.1 Equivalence with the Reference (Original) Product 3.2 Pharmaceutical Technology 4.0 Evidence: Manufacturer Criteria 4.1 Quality Assurance 4.2 Supply Track Record
4.3 Macroeconomic Benefit (Local Investment) 5.0 Evidence: Service Criteria 5.1 Pharmacovigilance
5.2 Value-Added Service Related to the Product 6.0 Evidence: Value Assessment Criteria 6.1 Pharmaceutical Acquisition Costs 6.2 Real-World Patient Outcomes and Costs OPP, off-patent pharmaceuticals.
Describes the disease, its incidence, and its prevalence in the target market and can include information on the clinical and economic burden of disease. Provides detailed information about the product and how it is expected to be used in this market. Describes any treatment options, standards of care, and treatment guidelines available for this disease and how this product compares with those standards, guidelines, and alternate treatment options. Provides evidence of bioequivalence or pharmaceutical equivalence to the originator, including active pharmaceutical ingredients, pharmacokinetic properties, clinical formulation, dosage form, and administration, etc. Describes the product’s technology, any technological advances, and associated patient benefit. Also describes the OPP’s excipients, process technology, and patient delivery system. Rates the quality level of the manufacturing site and provides proof of active pharmaceutical ingredient certifications and process certifications. Describes manufacturer’s historical supply reliability and sustainability, and shows how the manufacturer maintains adequate control of the distribution process, including procurement, purchasing, storage, distribution, transportation, repackaging, relabeling, documentation, and record-keeping practices Documents the manufacturer’s commitments to investing in both monetary and human resources in the local economy. Describes the manufacturer’s programs and activities designed to detect, assess, understand, and prevent adverse effects or any other drug-related problems, and identifies any reduction in adverse event reports as a result of the manufacturer’s pharmacovigilance system. Addresses any value-added services, such as technical support and education programs for patients or providers; documents any improvements in clinical outcomes, improved adherence, or adverse event reports resulting from any value-added programs. Lists all dosage forms, unit doses available, and the price for each dosage form, and cost of drug required for total number of days of intended treatment. Also compares price with originator drug, previous reference product, or comparable equivalent products. Compares efficacy and effectiveness with the national standard of care. Describes reported real-life patient outcomes, reported adverse effects, direct costs (i.e., the estimated impact on the absolute budget), and indirect costs.
12
VALUE IN HEALTH REGIONAL ISSUES 16C (2018) 9–13
totaled to calculate a final score, allowing for a more consistent comparison with standards, requirements, or competitor products. The base case is designed to be a template that health authorities can adapt as needed to fit their country’s specific health care priorities. This base case scenario has been tested for validation in emerging markets, including Indonesia, Kazakhstan, and Vietnam. The outputs of this validation will be reported in future publications. As with most documents representing an evidence synthesis, the EFOR starts with an Executive Summary (Section 1.0) of the purpose and overview of the document. The next section of the document (Section 2.0) is a Product Description, which consists of the targeted disease area and the expected place in therapy of the generic product versus the innovator. Section 3.0, Product Criteria, describes the evidence supporting the equivalence standards of the OPP to the innovator standard. In this section, any additional support to the patient can also be presented. Section 4.0 provides Manufacturer Criteria and is focused on the quality standards of the manufacturing process (including any WHO Certification Scheme ratings [11]) and whether manufacturing is done within or outside of the local market. Issues of supply are also addressed here. Processes to manage safety concerns or adverse events of the product, along with any other value-added services are discussed in Section 5.0, Evidence of Service Criteria. The final section (6.0), Value Assessment Criteria, incorporates both consideration of the dose, length of therapy, and cost per treatment episode versus standards of care, along with real-world or outcomes evidence to support improved outcomes versus the innovator product. Thus, the impact of guaranteed supply or consistent quality would be presented here. Table 1 lists the key components of the EFOR base case template, including a brief description of each criterion.
Discussion Most emerging markets are moving toward coverage expansion; however, the basic prerequisites for OPPs (i.e., pharmaceutical equivalence, GMP standards, and bioequivalence) have not yet been fully achieved. Furthermore, other key factors, such as the emerging market’s development status, the large population that needs to be covered, affordability, or limited resources, may limit access to the OPP in emerging markets. The WHO recommends that health authorities in emerging markets identify criteria for selecting high-quality, cost-effective drugs from reliable suppliers [10,11]. In addition, the WHO recommends that hospitals and other health facilities create Drug and Therapeutics Committees, bringing together people from relevant disciplines within the organization to evaluate the effective and efficient use of pharmaceutical products [18]. But the lack of standardized HTA tools or frameworks for providing the evidence that the WHO guidelines recommend severely limits the ability of decision makers to make informed decisions on drug procurement, adequately compare alternatives, and improve health care outcomes while balancing costs for their population. As the WHO points out in a recent study, the current rate of substandard and counterfeit drugs in low- and middle-income countries is approximately 10.5%. The unweighted estimated size of those combined markets is approximately $300 billion—that is, substandard and counterfeit drugs are costing those countries approximately $30.5 billion every year. The WHO recommends greater and more effective application of standards in production and supply chain management, as well as increased use of consistent standards in evidence collection, analysis, and reporting [13].
The EFOR’s identification of value-based, high-priority criteria in a uniform format helps strengthen resource allocation decisions (i.e., drug pricing, reimbursement, listing, and purchase decisions) by providing demonstrable evidence of product quality, efficacy, and supply reliability. Such a decision framework can reward current higher standards and incentivize local manufacturers to upgrade their standards over time and collect realworld data locally, which will be beneficial for the quality of care, overall health, and economic outcomes. By determining a standard set of high-priority criteria in the EFOR and requesting the evidence showing how closely the drug meets those criteria, health care authorities can clarify expectations, reduce misunderstandings, and improve communication between manufacturers, health care providers, and public decision makers. Evidence provided in the EFOR will support transparent, consistent, and relevant decisions, improving the quality of health care for a country’s population while balancing the need for affordability. The EFOR is similar to a traditional drug dossier or submission template, which presents evidence showing the clinical effectiveness and economic benefits of a drug and its place in therapy. The EFOR was based on traditional dossier formats and templates, such as the AMCP Format for Formulary Submissions [1], the ASEAN Common Technical Dossier (ACTD) for the Registration of Pharmaceuticals for Human Use [19], or the evidence submission template of the EUnetHTA Core Model [2]. However, although submission templates and dossiers used for innovative medicines and medical devices in developing countries are useful for showing their efficacy, safety, and cost effectiveness, the EFOR is uniquely well suited for emerging markets because it takes a more value-based approach to evaluating OPPs while still incorporating crucial information about the drug, including its pharmaceutical equivalence or bioequivalence to its originator or other OPPs. This is a far more beneficial method of OPP evaluation than has been available in the past, when decisions were based heavily on economic factors because of the scarcity of accurate, consistent evidence of the drug’s true composition and effectiveness. By focusing on only nine high-priority criteria, the EFOR is a simplified tool that can be readily understood and managed by both the manufacturers who supply the evidence and the health authorities who evaluate it, unlike some MCDA-based tools that have proven overly complicated and unwieldy. The EFOR’s flexibility and adaptability is one of its most useful features. To implement the EFOR within a country, it is recommended that a group of expert health authorities for that country hold workshops to evaluate the MCDA criteria from the four categories and establish a subset of criteria relevant for their particular market and the decision-making processes in which they will be used. Although criteria may be adjusted over time, a consistent list of criteria will help establish consistent decision making, accountability, and transparency within that market. The EFOR “base case” template provides internationally relevant criteria, but individual markets are encouraged to adapt this document to their own country’s needs. Proper EFOR introduction and dossier development training is highly recommended in each country prior to implementation. On the basis of information the manufacturer or health care provider presents in the EFOR, health authorities will further evaluate the drug using MCDA methodology, which includes weighting and scoring for each criterion. The MCDA Emerging Good Practices Task Force reports offer guidance for determining how criteria, weighting, and scoring can be implemented and how the set of criteria, weights, and scores may be adapted to the needs of each market [15,16]. It is essential for manufacturers and health care providers to provide accurate, measurable data for each appropriate criterion
VALUE IN HEALTH REGIONAL ISSUES 16C (2018) 9–13
in the EFOR for health care authorities to make accurate, consistent, and meaningful decisions on coverage, pricing, and other issues.
Conclusions The EFOR has two primary goals for decision makers: first, to improve communication among manufacturers, health care providers, and health authorities and, second, to provide evidence that fosters transparent, consistent, and relevant decisions about OPPs in that market. The EFOR addresses the need for a standard dossier submission framework specifically for OPPS, following a set of criteria that are relevant to emerging markets and that can be used to balance the need for affordability with factors that significantly improve the quality of health care for a country’s population, such as drug safety, level interchangeability, manufacturing site quality certification, quality assurance of active pharmaceutical ingredients, clinical efficacy and effectiveness, supply reliability, and real-life outcomes [14]. When used consistently within a market, the EFOR will help foster standardized, transparent, and more equitable decision making. As individual countries incorporate the EFOR into their decision-making settings, they can continue to tailor the framework to strengthen implementation. Publishing their own experiences with adaptation and implementation can also help spread best practices to other markets, raising the level of healthcare and patient access across the globe.
Funding These findings are the result of work supported by Abbott Established Pharmaceutical Division.
Acknowledgments The authors are grateful for the support provided by Abbott Established Pharmaceutical Division. The authors wish to thank the International Outcomes Research Board (IORB) for their input, and Kelley J. P. Lindberg for her editing assistance with this manuscript.
Supplemental Materials Supplementary data associated with this article can be found in the online version at https://doi.org/10.1016/j.vhri.2018.01.003.
13
R EF E R EN C ES
[1] The AMCP Format for Formulary Submissions. Version 4.0. Academy of Managed Care Pharmacy. 2016. Available from: http://www.amcp.org/ FormatV4/. [Accessed Janaury, 2018]. [2] EUnetHTA Joint Action 2, Work Package 8. HTA Core Model® version 3.0 (Pdf). 2016. Available from: https://meka.thl.fi/htacore/BrowseModel. aspx. [Accessed Janaury, 2018]. [3] https://www.ispor.org/Event/ReleasedPresentations/2017Glasgow# educationalforum. [Accessed Janaury, 2018]. [4] How to develop and implement a national drug policy. Gene, Switzerland: World Health Organization, 2001. Available from: http:// apps.who.int/medicinedocs/pdf/s2283e/s2283e.pdf. [Accessed Janaury, 2018]. [5] Sheppard A. Generic medicines: essential contributors to the long-term health of society. Sector sustainability challenges in Europe. IMS Health 2010. Available from: http://www.hup.hr/EasyEdit/UserFiles/ Granske_udruge/HUP-UPL/IMS.pdf. [Accessed Janaury, 2018]. [6] Hu S, Zhang Y, He J, et al. A case study of pharmaceutical pricing in China: setting the price for off-patent originators. Appl Health Econ Health Policy 2015;13(Suppl 1):S13–20. [7] IOM (Institute of Medicine). Countering the Problem of Falsified and Substandard Drugs. Washington, DC: The National Academies Press, 2013. [8] Bate R, Jin GZ, Mathur A. Does price reveal poor-quality drugs? Evidence from 17 countries. J Health Econ 2011;30(6):1150–63. [9] Alfonso-Cristancho R, Andia T, Barbosa T, Watanabe JH. Definition and classification of generic drugs across the world. Appl Health Econ Health Policy 2015;13(Suppl 1):S5–11. [10] Operational Principles for Good Pharmaceutical Procurement. Geneva, Switzerland: World Health Organization, 1999. [11] Practical Guidelines on Pharmaceutical Procurement for Countries with Small Procurement Agencies. Manila, Philippines: World Health Organization, 2002. [12] Kalo Z, Holtorf AP, Alfonso-Cristancho R, et al. Need for multicriteria evaluation of generic drug policies. Value Health 2015;18(2):346–51. [13] A Study on the Public Health and Socioeconomic Impact of Substandard and Falsified Medical Products. Geneva, Switzerland: World Health Organization, 2017. [14] Brixner D, Maniadakis N, Kalo Z, et al. Considering Multi-Criteria Decision Analysis (MCDA) Simple Scoring as an evidence-based HTA methodology for evaluating off-patent pharmaceuticals (OPPs) in emerging markets. Value Health Reg Issues 2017;13:1–6. [15] Thokala P, Devlin N, Marsh K, et al. Multiple Criteria Decision Analysis for health care decision making—an introduction: report 1 of the ISPOR MCDA Emerging Good Practices Task Force. Value Health 2016;19 (1):1–13. [16] Marsh K, IJ M, Thokala P, et al. Multiple Criteria Decision Analysis for health care decision making—emerging good practices: Report 2 of the ISPOR MCDA Emerging Good Practices Task Force. Value Health 2016;19 (2):125–37. [17] Marsh K, Lanitis T, Neasham D, et al. Assessing the value of healthcare interventions using multi-criteria decision analysis: a review of the literature. Pharmacoeconomics. 2014;32(4):345–65. [18] Holloway K, Green T. Drug and Therapeutics Committees: A Practical Guide. Geneva, Switzerland: World Health Organization, 2003. [19] The ASEAN Common Technical Dossier (ACTD) for the Registration of Pharmaceuticals for Human Use. Jakarta, Indonesia: Association of Southeast Asian Nations (ASEAN), 2016. Available from http://asean. org/storage/2017/03/68.-December-2016-ACTD.pdf. [Accessed Janaury, 2018].
Available online at www.sciencedirect.com
journal homepage: www.elsevier.com/locate/vhri
An Evidence Framework for Off-Patent Pharmaceutical Review (EFOR) for Health Technology Assessment in Emerging Markets Diana Brixner, PhD, RPh (Professor, Executive Director)1,2,⁎, Zoltán Kaló, MSc, MD, PhD (Professor)3, Nikos Maniadakis, BSc, MSc, PhD, FESC (Professor)4, Kyoo Kim, MSc, MBA (Global Market Access and Health Economics Director)5, Kalman Wijaya, BSc, Pharm, MBA (Global Market Access and Policy Senior Manager)5 1 Department of Pharmacotherapy, University of Utah College Of Pharmacy; 2Pharmacotherapy Outcomes Research Center. Salt Lake City, UT, USA; 3Department of Health Policy and Health Economics, Eötvös Loránd University (ELTE), Budapest; Syreon Research Institute. Budapest, Hungary; 4Health Services Organization and Management, Alternate Dean, National School of Public Health. Athens, Greece; 5Abbott Established Pharmaceutical Division. Basel, Switzerland
AB STR A CT
Objective: This article introduces an Evidence Framework for OffPatent Pharmaceutical Review (EFOR), which establishes value-based criteria in a template that manufacturers use to provide evidence showing how their products meet those criteria. Health authorities in emerging markets can then use the evidence presented in the EFOR to evaluate off-patent pharmaceuticals (OPPs) in a consistent, transparent, and evidence-based manner to support policy decisions, including pricing, reimbursement, formulary listing, and drug procurement. Methods: A literature search found no multi-criteria evidence framework for evaluating OPPs in emerging markets. An International Outcomes Research Board (IORB) of academia and industry experts conducted extensive research, meetings, and workshops to define high-priority criteria to incorporate into an evidencebased health technology assessment (HTA) tool using the multicriteria decision analysis (MCDA) technique. The resulting framework was further tailored for country-specific needs in workshops in three emerging countries (Kazakhstan, Vietnam, and Indonesia). Results: The IORB defined nine criteria four categories (Product, Manufacturing, Service, and Value Assessment), which OPP manufacturers can
use to provide evidence for reimbursement and health policy decision making. Then the IORB developed the EFOR as a base case document, which can be adapted and used as a template by health authorities in emerging countries. Conclusions: Emerging countries have a significant need for an HTA tool that balances affordability with accurate evidence showing the value differentiation of OPPs. The value attributes in this setting often are different from those in developed markets, which emphasize new products and have high regulation and manufacturing standards. The EFOR is an easy-to-use, adaptable framework that emerging countries can use to increase the consistency, transparency, and effectiveness of drug decision making. The open source EFOR is available as Supplemental Materials. Keywords: dossier, emerging markets, evidence framework, EFOR, generic pharmaceuticals, health technology assessment, MCDA, multi-criteria decision analysis, off-patent pharmaceuticals.
Introduction
However, utilization of new technologies and pharmaceuticals is less common in emerging markets. Instead, emerging markets, which comprise the majority of the world’s population, rely heavily on off-patent pharmaceuticals (OPPs). OPPs include International Nonproprietary Name generics, branded generics, and off-patent originators. OPPs are an especially important component of health care, representing 60% to 80% of the volume of the pharmaceutical market in most emerging markets [3–6]. Although it is common to use the term off-patent pharmaceuticals to refer collectively to all products that are similar to the innovator product after the patent has expired, there are important distinctions between drugs that are given this label. The first type of OPP is an innovator product that receives a new name and
In developed markets, when new medications or medical technology products are introduced, health authorities have health technology assessment (HTA) tools to help them understand, evaluate, and compare the products for use in their markets. Dossier templates, such as the Academy of Managed Care Pharmacy (AMCP) Format for Formulary Submissions Version 4.0 [1] and the EUnetHTA Core Model [2], provide structured presentations of evidence to help health authorities balance known efficacy and safety results with total therapy costs, for making pricing, reimbursement, formulary listing, and product procurement choices.
& 2018 Published by Elsevier Inc. on behalf of International Society for Pharmacoeconomics and Outcomes Research (ISPOR).
The authors have indicated that they have no conflicts of interest with regard to the content of this article. * Address correspondence to: Diana Brixner, PhD, RPh, University of Utah, Department of Pharmacotherapy, 30 S 2000 E, Suite 258, Salt Lake City, UT 84112, USA. E-mail: [email protected] 2212-1099$36.00 – see front matter & 2018 Published by Elsevier Inc. on behalf of International Society for Pharmacoeconomics and Outcomes Research (ISPOR). https://doi.org/10.1016/j.vhri.2018.01.003
10
VALUE IN HEALTH REGIONAL ISSUES 16C (2018) 9–13
designation after the patent expires. All other products are considered true generic products, which are required to demonstrate their similarity to the innovator. Although often assumed to be identical to the originator drug or to other generic drugs, these OPPs, in fact, can vary greatly in pharmaceutical equivalence and bioequivalence (in which evidence shows the OPP is as effective and safe for patients in the specific indication at the same dose, form, route of administration, etc., regardless of differences in excipients or manufacturing processes). Such variations in pharmaceutical equivalence and bioequivalence are especially prevalent in emerging markets. Additionally, OPP manufacturers can vary in their implementation of good manufacturing practice (GMP) standards, which can alter or compromise the drug’s efficacy and/or safety [7–9]. To help address this problem, the World Health Organization (WHO) has published guidelines for establishing more formal procedures for countries to follow when procuring pharmaceutical products, recommending that countries procure the most costeffective drugs from the most reliable suppliers of high-quality products [10,11]. However, the WHO’s guidelines do not provide a specific template for gathering evidence across a wide range of criteria to use when evaluating products. Although current evaluation frameworks are available in developed countries, in general, they only focus on innovative medicine; there have been no equivalent tools for evaluating OPPs in emerging markets. This means that health authorities in emerging countries are often left without a consistent, practical, and objective way to evaluate an OPP and instead can only assume that the drug is reasonably equivalent. This assumption that all OPPs are equivalent to their originator drug, adhere to identical GMP standards, and have equal supply reliability, despite a lack of evidence to confirm the assumption, can cause health authorities in emerging markets to make coverage decisions based purely on price. Reliance on price alone as a decision-making tool can be detrimental to a country’s health goals when two or more OPPs based on the same originator drug are, in fact, quite different in terms of efficacy, safety, delivery, or reliability of distribution systems, for example [8,12]. In fact, a recent WHO study showed that the rate of substandard and counterfeit drugs in low- and middle-income countries was approximately 10.5% [13]. To address the significant need for systematic and evidencebased evaluations for OPPs in emerging markets, an International Outcomes Research Board (IORB) of academia and industry experts conducted extensive research, exploratory meetings, and workshops to develop a customized multi-criteria decision analysis (MCDA) tool accommodating the unique needs of these markets. This tool, the Evidence Framework for Off-Patent Pharmaceutical Review (EFOR), is an evidence-based HTA tool based on the MCDA technique [14]. The EFOR is a template that provides a consistent, defined, high-priority set of desired information, and it can be adapted to the specific needs of a particular country. Manufacturers use the EFOR to provide evidence showing how their products meet those criteria, which health policymakers and reimbursement agencies in emerging markets can then use to evaluate OPPs to make pricing, reimbursement, formulary listing, or drug purchase decisions. The EFOR provides a consistent decision-making framework based on relevant criteria specifically designed for OPPs in emerging markets. Evidence presented in the EFOR is intended to be used by payers, institutions, committees, procurement agencies, HTA organizations and health ministries.
Methods In a traditional MCDA Simple Scoring decision-making process, a set of criteria are defined, ranked by relevance and importance,
and evaluated. As MCDA became more widely used in a variety of industries, multiple MCDA methodologies appeared, and their complexity and inconsistencies grew. Therefore, in 2014, the International Society of Pharmacoeconomics and Outcome Research (ISPOR) established the MCDA Emerging Good Practices Task Force to develop good practice guidelines for implementing MCDA in health care decision making [15–17]. Even with those guidelines, however, MCDA was still considered overly complex or too context specific for use in some environments. Therefore, an IORB conducted theoretical and practical application research to develop MCDA as a streamlined evidence-based HTA methodology for OPPs, designed specifically to be adapted for use in individual emerging countries. With MCDA techniques, a complex set of research principles are translated into a practical set of criteria to weight characteristics important to relevant health policy decisions for emerging markets [14]. After an extensive literature search determined that a streamlined HTA methodology for evaluating OPPs in emerging markets did not exist, a workshop was held with emerging market country representatives at the ISPOR Annual European Meeting in Milan, Italy, in 2015 to identify and prioritize criteria that health care systems in emerging countries could use in an MCDA format to evaluate medications and determine coverage. The workshop involved 57 health care experts and decision makers from 14 countries: China, Russia, Egypt, Thailand, Turkey, Indonesia, Kazakhstan, Vietnam, the Philippines, Colombia, Taiwan, Malaysia, Algeria, and Saudi Arabia. These countries represent emerging markets in regions defined by the WHO: Africa, the Americas, Eastern Mediterranean, Europe, Southeast Asia, and Western Pacific. The participants represented several professional realms, including academia, ministries of health, drug procurement agencies, and drug pricing and reimbursement agencies. The workshop participants were asked to prioritize 22 criteria (identified from literature searches and further research). These 22 criteria and the MCDA Simple Scoring process used to identify them were published in 2017 [14]. After the MCDA criteria were prioritized in the workshop, the criteria and the MCDA process were presented and feedback was gathered from health authorities in decision-making positions regarding generic drug reimbursement at the Health Technology Assessment International (HTAi) conference in Japan in 2016 and at the ISPOR conference in Singapore in 2016. In parallel, the IORB continued to develop the EFOR template based on those criteria, comparing the EFOR to drug dossier formats and submission templates, such as the AMCP Format, the ACTD format, and the EUnetHTA HTA Core Model, to ensure that the documents will be complementary, rather than overlapping, providing health care authorities with more robust evidence to help make decisions about the incorporation of OPPs into the respective country’s health care system. After the HTAi Japan and ISPOR Singapore meetings, a small group of researchers, including a pharmacist, a medical doctor, and two health economists scrutinized all of the criteria from the viewpoint of completeness, non-redundancy, non-overlap, and preference independence. They further refined the list of criteria to reduce the number of criteria to nine, which was considered both optimal for ease of use and for adaptability to a variety of markets. By using the final recommended MCDA criteria, a “base case” EFOR that can be used and adapted to specific countries’ needs was developed. This base case template was then validated in workshops held in three emerging markets: Kazakhstan, Vietnam, and Indonesia. The resulting MCDA tool kit and the EFOR can be adapted in different settings to make decisions regarding OPPs. The EFOR provides the evidence to be evaluated in a systematic and anchored manner to facilitate an MCDA-based value-oriented assessment of OPPs.
VALUE IN HEALTH REGIONAL ISSUES 16C (2018) 9–13
Results Initial research and workshop discussions of the highest-priority factors that should be considered when evaluating an OPP yielded a set of 22 criteria in four categories.14 However, the MCDA task force report [15,16] recommends limiting the number of criteria to a manageable set for any particular decision in which MCDA is being applied. The task force found that the average number of criteria used in health care decisions was 8.2, with a range from three to 19 criteria [16,17]. With this principle in mind, the IORB evaluated the original 22 criteria for completeness, non-redundancy, non-overlap, preferential independence, local relevance, and the feasibility of measuring them within emerging markets [16]. Some criteria with potential dependencies were merged into a single criterion. For example, proving pharmaceutical equivalence is a necessary step prior to bioequivalence studies, and without evidence on bioequivalence information, therapeutic equivalence cannot be expected. Therefore, pharmaceutical, bioequivalence, and
11
therapeutic equivalences were merged into a single categorical criterion called “equivalence with the reference (original) product.” In other cases, if two criteria were correlated, one could be dropped from the list, for example, “equivalence with the reference product” had significant overlap with “level of interchangeability,” so the latter was removed. Finally, some criteria were considered complementary to each other, and so “technical assistance,” “disease awareness and education,” and “continued medical education” were merged into a single criterion: “added value services related to the product.” After merging similar criteria and eliminating criteria for which no scoring function could be identified, the IORB identified a final list of nine key criteria (shown in Table 1). The criteria were divided into four categories: Product, Manufacturer, Service, and Value Assessment. Then an EFOR “base case” was developed. This base case defines each of the nine criteria and provides a simple scoring scale for each criterion that health authorities can use to objectively measure how the OPP meets that criterion. With the scoring scale, points are awarded for each criterion and then
Table 1 – Outline of the EFOR base case. 1.0 Executive Summary Summarizes the evidence showing how the product meets the established criteria for consideration by health authorities. 2.0 Product Description 2.1 Disease Description, Incidence, and Prevalence 2.2 Product Description and Place in Therapy 2.3 Current Treatment Options
3.0 Evidence: Product Criteria 3.1 Equivalence with the Reference (Original) Product 3.2 Pharmaceutical Technology 4.0 Evidence: Manufacturer Criteria 4.1 Quality Assurance 4.2 Supply Track Record
4.3 Macroeconomic Benefit (Local Investment) 5.0 Evidence: Service Criteria 5.1 Pharmacovigilance
5.2 Value-Added Service Related to the Product 6.0 Evidence: Value Assessment Criteria 6.1 Pharmaceutical Acquisition Costs 6.2 Real-World Patient Outcomes and Costs OPP, off-patent pharmaceuticals.
Describes the disease, its incidence, and its prevalence in the target market and can include information on the clinical and economic burden of disease. Provides detailed information about the product and how it is expected to be used in this market. Describes any treatment options, standards of care, and treatment guidelines available for this disease and how this product compares with those standards, guidelines, and alternate treatment options. Provides evidence of bioequivalence or pharmaceutical equivalence to the originator, including active pharmaceutical ingredients, pharmacokinetic properties, clinical formulation, dosage form, and administration, etc. Describes the product’s technology, any technological advances, and associated patient benefit. Also describes the OPP’s excipients, process technology, and patient delivery system. Rates the quality level of the manufacturing site and provides proof of active pharmaceutical ingredient certifications and process certifications. Describes manufacturer’s historical supply reliability and sustainability, and shows how the manufacturer maintains adequate control of the distribution process, including procurement, purchasing, storage, distribution, transportation, repackaging, relabeling, documentation, and record-keeping practices Documents the manufacturer’s commitments to investing in both monetary and human resources in the local economy. Describes the manufacturer’s programs and activities designed to detect, assess, understand, and prevent adverse effects or any other drug-related problems, and identifies any reduction in adverse event reports as a result of the manufacturer’s pharmacovigilance system. Addresses any value-added services, such as technical support and education programs for patients or providers; documents any improvements in clinical outcomes, improved adherence, or adverse event reports resulting from any value-added programs. Lists all dosage forms, unit doses available, and the price for each dosage form, and cost of drug required for total number of days of intended treatment. Also compares price with originator drug, previous reference product, or comparable equivalent products. Compares efficacy and effectiveness with the national standard of care. Describes reported real-life patient outcomes, reported adverse effects, direct costs (i.e., the estimated impact on the absolute budget), and indirect costs.
12
VALUE IN HEALTH REGIONAL ISSUES 16C (2018) 9–13
totaled to calculate a final score, allowing for a more consistent comparison with standards, requirements, or competitor products. The base case is designed to be a template that health authorities can adapt as needed to fit their country’s specific health care priorities. This base case scenario has been tested for validation in emerging markets, including Indonesia, Kazakhstan, and Vietnam. The outputs of this validation will be reported in future publications. As with most documents representing an evidence synthesis, the EFOR starts with an Executive Summary (Section 1.0) of the purpose and overview of the document. The next section of the document (Section 2.0) is a Product Description, which consists of the targeted disease area and the expected place in therapy of the generic product versus the innovator. Section 3.0, Product Criteria, describes the evidence supporting the equivalence standards of the OPP to the innovator standard. In this section, any additional support to the patient can also be presented. Section 4.0 provides Manufacturer Criteria and is focused on the quality standards of the manufacturing process (including any WHO Certification Scheme ratings [11]) and whether manufacturing is done within or outside of the local market. Issues of supply are also addressed here. Processes to manage safety concerns or adverse events of the product, along with any other value-added services are discussed in Section 5.0, Evidence of Service Criteria. The final section (6.0), Value Assessment Criteria, incorporates both consideration of the dose, length of therapy, and cost per treatment episode versus standards of care, along with real-world or outcomes evidence to support improved outcomes versus the innovator product. Thus, the impact of guaranteed supply or consistent quality would be presented here. Table 1 lists the key components of the EFOR base case template, including a brief description of each criterion.
Discussion Most emerging markets are moving toward coverage expansion; however, the basic prerequisites for OPPs (i.e., pharmaceutical equivalence, GMP standards, and bioequivalence) have not yet been fully achieved. Furthermore, other key factors, such as the emerging market’s development status, the large population that needs to be covered, affordability, or limited resources, may limit access to the OPP in emerging markets. The WHO recommends that health authorities in emerging markets identify criteria for selecting high-quality, cost-effective drugs from reliable suppliers [10,11]. In addition, the WHO recommends that hospitals and other health facilities create Drug and Therapeutics Committees, bringing together people from relevant disciplines within the organization to evaluate the effective and efficient use of pharmaceutical products [18]. But the lack of standardized HTA tools or frameworks for providing the evidence that the WHO guidelines recommend severely limits the ability of decision makers to make informed decisions on drug procurement, adequately compare alternatives, and improve health care outcomes while balancing costs for their population. As the WHO points out in a recent study, the current rate of substandard and counterfeit drugs in low- and middle-income countries is approximately 10.5%. The unweighted estimated size of those combined markets is approximately $300 billion—that is, substandard and counterfeit drugs are costing those countries approximately $30.5 billion every year. The WHO recommends greater and more effective application of standards in production and supply chain management, as well as increased use of consistent standards in evidence collection, analysis, and reporting [13].
The EFOR’s identification of value-based, high-priority criteria in a uniform format helps strengthen resource allocation decisions (i.e., drug pricing, reimbursement, listing, and purchase decisions) by providing demonstrable evidence of product quality, efficacy, and supply reliability. Such a decision framework can reward current higher standards and incentivize local manufacturers to upgrade their standards over time and collect realworld data locally, which will be beneficial for the quality of care, overall health, and economic outcomes. By determining a standard set of high-priority criteria in the EFOR and requesting the evidence showing how closely the drug meets those criteria, health care authorities can clarify expectations, reduce misunderstandings, and improve communication between manufacturers, health care providers, and public decision makers. Evidence provided in the EFOR will support transparent, consistent, and relevant decisions, improving the quality of health care for a country’s population while balancing the need for affordability. The EFOR is similar to a traditional drug dossier or submission template, which presents evidence showing the clinical effectiveness and economic benefits of a drug and its place in therapy. The EFOR was based on traditional dossier formats and templates, such as the AMCP Format for Formulary Submissions [1], the ASEAN Common Technical Dossier (ACTD) for the Registration of Pharmaceuticals for Human Use [19], or the evidence submission template of the EUnetHTA Core Model [2]. However, although submission templates and dossiers used for innovative medicines and medical devices in developing countries are useful for showing their efficacy, safety, and cost effectiveness, the EFOR is uniquely well suited for emerging markets because it takes a more value-based approach to evaluating OPPs while still incorporating crucial information about the drug, including its pharmaceutical equivalence or bioequivalence to its originator or other OPPs. This is a far more beneficial method of OPP evaluation than has been available in the past, when decisions were based heavily on economic factors because of the scarcity of accurate, consistent evidence of the drug’s true composition and effectiveness. By focusing on only nine high-priority criteria, the EFOR is a simplified tool that can be readily understood and managed by both the manufacturers who supply the evidence and the health authorities who evaluate it, unlike some MCDA-based tools that have proven overly complicated and unwieldy. The EFOR’s flexibility and adaptability is one of its most useful features. To implement the EFOR within a country, it is recommended that a group of expert health authorities for that country hold workshops to evaluate the MCDA criteria from the four categories and establish a subset of criteria relevant for their particular market and the decision-making processes in which they will be used. Although criteria may be adjusted over time, a consistent list of criteria will help establish consistent decision making, accountability, and transparency within that market. The EFOR “base case” template provides internationally relevant criteria, but individual markets are encouraged to adapt this document to their own country’s needs. Proper EFOR introduction and dossier development training is highly recommended in each country prior to implementation. On the basis of information the manufacturer or health care provider presents in the EFOR, health authorities will further evaluate the drug using MCDA methodology, which includes weighting and scoring for each criterion. The MCDA Emerging Good Practices Task Force reports offer guidance for determining how criteria, weighting, and scoring can be implemented and how the set of criteria, weights, and scores may be adapted to the needs of each market [15,16]. It is essential for manufacturers and health care providers to provide accurate, measurable data for each appropriate criterion
VALUE IN HEALTH REGIONAL ISSUES 16C (2018) 9–13
in the EFOR for health care authorities to make accurate, consistent, and meaningful decisions on coverage, pricing, and other issues.
Conclusions The EFOR has two primary goals for decision makers: first, to improve communication among manufacturers, health care providers, and health authorities and, second, to provide evidence that fosters transparent, consistent, and relevant decisions about OPPs in that market. The EFOR addresses the need for a standard dossier submission framework specifically for OPPS, following a set of criteria that are relevant to emerging markets and that can be used to balance the need for affordability with factors that significantly improve the quality of health care for a country’s population, such as drug safety, level interchangeability, manufacturing site quality certification, quality assurance of active pharmaceutical ingredients, clinical efficacy and effectiveness, supply reliability, and real-life outcomes [14]. When used consistently within a market, the EFOR will help foster standardized, transparent, and more equitable decision making. As individual countries incorporate the EFOR into their decision-making settings, they can continue to tailor the framework to strengthen implementation. Publishing their own experiences with adaptation and implementation can also help spread best practices to other markets, raising the level of healthcare and patient access across the globe.
Funding These findings are the result of work supported by Abbott Established Pharmaceutical Division.
Acknowledgments The authors are grateful for the support provided by Abbott Established Pharmaceutical Division. The authors wish to thank the International Outcomes Research Board (IORB) for their input, and Kelley J. P. Lindberg for her editing assistance with this manuscript.
Supplemental Materials Supplementary data associated with this article can be found in the online version at https://doi.org/10.1016/j.vhri.2018.01.003.
13
R EF E R EN C ES
[1] The AMCP Format for Formulary Submissions. Version 4.0. Academy of Managed Care Pharmacy. 2016. Available from: http://www.amcp.org/ FormatV4/. [Accessed Janaury, 2018]. [2] EUnetHTA Joint Action 2, Work Package 8. HTA Core Model® version 3.0 (Pdf). 2016. Available from: https://meka.thl.fi/htacore/BrowseModel. aspx. [Accessed Janaury, 2018]. [3] https://www.ispor.org/Event/ReleasedPresentations/2017Glasgow# educationalforum. [Accessed Janaury, 2018]. [4] How to develop and implement a national drug policy. Gene, Switzerland: World Health Organization, 2001. Available from: http:// apps.who.int/medicinedocs/pdf/s2283e/s2283e.pdf. [Accessed Janaury, 2018]. [5] Sheppard A. Generic medicines: essential contributors to the long-term health of society. Sector sustainability challenges in Europe. IMS Health 2010. Available from: http://www.hup.hr/EasyEdit/UserFiles/ Granske_udruge/HUP-UPL/IMS.pdf. [Accessed Janaury, 2018]. [6] Hu S, Zhang Y, He J, et al. A case study of pharmaceutical pricing in China: setting the price for off-patent originators. Appl Health Econ Health Policy 2015;13(Suppl 1):S13–20. [7] IOM (Institute of Medicine). Countering the Problem of Falsified and Substandard Drugs. Washington, DC: The National Academies Press, 2013. [8] Bate R, Jin GZ, Mathur A. Does price reveal poor-quality drugs? Evidence from 17 countries. J Health Econ 2011;30(6):1150–63. [9] Alfonso-Cristancho R, Andia T, Barbosa T, Watanabe JH. Definition and classification of generic drugs across the world. Appl Health Econ Health Policy 2015;13(Suppl 1):S5–11. [10] Operational Principles for Good Pharmaceutical Procurement. Geneva, Switzerland: World Health Organization, 1999. [11] Practical Guidelines on Pharmaceutical Procurement for Countries with Small Procurement Agencies. Manila, Philippines: World Health Organization, 2002. [12] Kalo Z, Holtorf AP, Alfonso-Cristancho R, et al. Need for multicriteria evaluation of generic drug policies. Value Health 2015;18(2):346–51. [13] A Study on the Public Health and Socioeconomic Impact of Substandard and Falsified Medical Products. Geneva, Switzerland: World Health Organization, 2017. [14] Brixner D, Maniadakis N, Kalo Z, et al. Considering Multi-Criteria Decision Analysis (MCDA) Simple Scoring as an evidence-based HTA methodology for evaluating off-patent pharmaceuticals (OPPs) in emerging markets. Value Health Reg Issues 2017;13:1–6. [15] Thokala P, Devlin N, Marsh K, et al. Multiple Criteria Decision Analysis for health care decision making—an introduction: report 1 of the ISPOR MCDA Emerging Good Practices Task Force. Value Health 2016;19 (1):1–13. [16] Marsh K, IJ M, Thokala P, et al. Multiple Criteria Decision Analysis for health care decision making—emerging good practices: Report 2 of the ISPOR MCDA Emerging Good Practices Task Force. Value Health 2016;19 (2):125–37. [17] Marsh K, Lanitis T, Neasham D, et al. Assessing the value of healthcare interventions using multi-criteria decision analysis: a review of the literature. Pharmacoeconomics. 2014;32(4):345–65. [18] Holloway K, Green T. Drug and Therapeutics Committees: A Practical Guide. Geneva, Switzerland: World Health Organization, 2003. [19] The ASEAN Common Technical Dossier (ACTD) for the Registration of Pharmaceuticals for Human Use. Jakarta, Indonesia: Association of Southeast Asian Nations (ASEAN), 2016. Available from http://asean. org/storage/2017/03/68.-December-2016-ACTD.pdf. [Accessed Janaury, 2018].