- Email: [email protected]
Advancing Value Assessment in the United States: A Multistakeholder Perspective
VALUE IN HEALTH 20 (2017) 299–307
Available online at www.sciencedirect.com
journal homepage: www.elsevier.com/locate/jval
Advancing Value Assessment in the United States: A Multistakeholder Perspective Corinna Sorenson, MPH, MHSA, PhD1,*, Gabriela Lavezzari, PhD, MBA2, Gregory Daniel, MPH, MS, PhD2, Randy Burkholder, BA3, Marc Boutin, JD4, Edmund Pezalla, MD, MPH5, Gillian Sanders, PhD1, Mark McClellan, MD, PhD2 1 Duke Clinical Research Institute, Duke School of Medicine, Duke University, Durham, NC, USA; 2Duke-Robert J. Margolis, MD Center for Health Policy, Duke University, Durham, NC, USA; 3Pharmaceutical Research and Manufacturers of America, Washington, DC, USA; 4National Health Council, Washington, DC, USA; 5Enlightenment Bioconsult, LLC, Wethersfield, CT, USA
AB STR A CT
Rising costs without perceived proportional improvements in quality and outcomes have motivated fundamental shifts in health care delivery and payment to achieve better value. Aligned with these efforts, several value assessment frameworks have been introduced recently to help providers, patients, and payers better understand the potential value of drugs and other interventions and make informed decisions about their use. Given their early stage of development, it is imperative to evaluate these efforts on an ongoing basis to identify how best to support and improve them moving forward. This article provides a multistakeholder perspective on the key limitations and opportunities posed by the current value assessment frameworks and areas of and actions for improvement. In particular, we outline 10
fundamental guiding principles and associated strategies that should be considered in subsequent iterations of the existing frameworks or by emerging initiatives in the future. Although value assessment frameworks may not be able to meet all the needs and preferences of stakeholders, we contend that there are common elements and potential next steps that can be supported to advance value assessment in the United States. Keywords: health care decision makers, standards, value assessment, value for money.
Introduction
determinations or price negotiations between payers and manufacturers. Current value assessment frameworks have received praise as tools to achieve higher value health care, but have also been criticized for their limitations regarding methods, processes, implementation, and potential impacts on patients’ access to care [6–11]. For example, some patients have experienced barriers obtaining new cholesterol-lowering drugs, PCSK9 inhibitors, after their (relatively unfavorable) value assessment by ICER and public debate regarding their cost [12,13]. Given their early stage of development, and their potential impact on care access and delivery, timely and ongoing evaluation of these efforts is critical to improve them and to help ensure they are implemented effectively. Such an exercise should involve all stakeholders to obtain a comprehensive, balanced, and representative assessment. In addition, we maintain that stakeholder input and, to some degree, buy-in are important to ensure the adoption and sustainability of value assessment in the United States. To meet this aim, we convened a small group of eight stakeholders representing patient advocacy organizations, drug manufacturers, payers, and academia to explore and discuss the
Rising health care costs in the United States have created significant financial burden on patients and their families, health providers, industry, and payers. New, high-priced treatments and services have faced scrutiny regarding the value they deliver in relation to their costs, especially because policymakers have accelerated efforts to shift from traditional “fee-for-service” reimbursement to “value-based” payment. Amid these efforts, a number of professional and private organizations have put forth value assessment frameworks to define and measure the value of drugs and other therapies in a more explicit and systematic manner. Such frameworks have been introduced by the American College of Cardiology and American Heart Association (ACC-AHA) [1], American Society of Clinical Oncology (ASCO) [2], Institute for Clinical and Economic Review (ICER) [3], Memorial Sloan Kettering Cancer Center (MSK) [4], and National Comprehensive Cancer Network (NCCN) [5]. The aims of these frameworks differ—some seek to help physicians and patients make more informed, evidence-based treatment decisions, whereas others are intended to aid payer coverage
Copyright & 2017, International Society for Pharmacoeconomics and Outcomes Research (ISPOR). Published by Elsevier Inc.
* Address correspondence to: Corinna Sorenson, Duke Clinical Research Institute, Duke School of Medicine, Duke University, 2400 Pratt Street, Durham, NC 27705. E-mail: [email protected]. 1098-3015$36.00 – see front matter Copyright & 2017, International Society for Pharmacoeconomics and Outcomes Research (ISPOR). Published by Elsevier Inc. http://dx.doi.org/10.1016/j.jval.2016.11.030
300
VALUE IN HEALTH 20 (2017) 299–307
following four areas: 1) main gaps and opportunities posed by the current value assessment frameworks, 2) differences and convergences in stakeholder definitions of value, 3) key guiding principles for value assessment, and 4) potential next steps for improvement. A set of questions was developed and sent to involved stakeholders to gather their respective input across these four areas. Responses to these questions were then discussed during two rounds of meetings with all stakeholder representatives to further flesh out similarities and differences across perspectives and reach consensus on key guiding principles for value assessment. This article offers the resulting multistakeholder perspective on value frameworks, outlining 10 fundamental guiding principles and associated strategies that should be considered in subsequent iterations of the existing value assessment frameworks or by emerging efforts in the future. This multistakeholder approach differs from the approaches of other groups, including the National Pharmaceutical Council and the Pharmaceutical Research and Manufacturers of America, which have recently put forth principles or practices for value assessment. A value assessment framework may not be able to meet all the needs of all the stakeholders, but we contend there are common elements and steps forward that can be supported and adopted to advance high-quality value assessment in the United States.
A Multistakeholder Perspective on Existing Value Frameworks Emerging value assessment frameworks have both commonalities and differences across intended aims, methods, processes, and applications, which are largely attributable to the interests and expertise of the developing organizations and intended uses of the respective frameworks (Table 2). For example, as professional medical societies, ASCO and the NCCN designed their frameworks to enhance shared treatment decision making between patients and physicians, whereas ICER and the MSK have developed frameworks to mainly support payer coverage and pricing determinations. These frameworks offer new tools to better understand and measure the value of new drugs and other interventions to support policy and practice and should be recognized for fostering broader conservations about these issues. Many of these efforts have strived to evolve (or plan to do so) on the basis of stakeholder input on their strengths and limitations (Table 1). Despite these steps, important issues remain that should be addressed before value assessment frameworks can be broadly implemented. First, the existing value assessment frameworks apply a somewhat narrow and static conception of value. In particular, the elements or components of value considered across frameworks principally focus on clinical and economic outcomes. While some frameworks do take into account quality of life, severity of disease, and daily functioning, for example, patient organizations, among other stakeholders, argue that insufficient consideration is given to outcomes important to patients and their caregivers, such as personal aspirations, productivity, outof-pocket costs, hope, convenience, and certainty of prognosis or treatment [14]. Patient perspectives on value, which are often individual and disease-dependent, can differ from those of payers and physicians in significant ways. For example, a recent study of patients with metastatic breast cancer showed that patients tend to emphasize value in terms of their personal benefit (e.g., ability to maintain rich relationships with family members) rather than in economic terms [15]. In addition, most of the frameworks fail to account for the range of benefits afforded by new or incremental innovation. For example,
incremental innovation may support a new delivery mechanism, such as a patch versus a pill, that results in more sustained therapeutic drug levels, improved patient treatment adherence, and enhanced clinical choice. In particular, the industry is concerned that such advances (as well as more substantial innovations, such as curative therapies) will not be recognized sufficiently for the improvements in health outcomes and potential cost offsets they deliver over time [7]. From a payer perspective, the existing frameworks are limited in their scope by focusing narrowly on patients and individual technologies, without due account of a population or societal perspective and the continuum of patient care [16]. Second, the underlying methods used by the frameworks have not been sufficiently vetted against established standards or through broad stakeholder consensus. There is limited consistency across frameworks regarding standards for grading the quality of available evidence, evidence synthesis, costing and budget impact methods, or weighing benefits versus costs (e.g., cost-utility analysis and multicriteria decision analysis). Consequently, individual assessments may differ across frameworks, as demonstrated in the case of new treatments for multiple myeloma. ASCO calculated a fairly low net health benefit (47/130) and substantially greater monthly costs for a newer regimen (bortezomib þ melphalan þ prednisone) than did the previous standard of care (melphalan þ prednisone) ($7042 vs. $279), but did not assess the quality of evidence [2]. The NCCN provided scores of 4 for efficacy, 3 for safety, 4 for quality and consistency of the evidence, and 3 for affordability (all out of 5) for the same three-drug regimen [17]. ICER examined six new second-line or later treatment regimens and determined that care value was low to intermediate, depending on the regimen, and that none represents good value in the long-term and therefore should be offered at a price discount of more than 75% [18]. The ICER also noted that the available evidence on the individual regimens was largely adequate, but that insufficient data exist to distinguish comparative net health benefit between them. Furthermore, several of the present frameworks do not consider the full range of available evidence, limiting their evidence base to clinical trials, and sometimes only a single trial. Although clinical trials are often the highest quality evidence available, the resulting evidence base is often limited and can be enhanced by considering additional study designs, such as well-designed observational studies. Similarly, none of the frameworks currently consider real-world or patient-generated/patient-desired data in assessments or outline clear policies on updating assessments if new evidence arises, which may limit capturing the value of therapies in actual clinical practice and accounting for the evolving nature of innovation. Third, the transparency of the methods and processes used by value assessment frameworks is limited. In addition to the actual frameworks themselves, the data, economic models, and underlying assumptions of individual assessments have not often been made public, which hinders interpretation of and confidence in the results. Transparency concerns extend to how therapies are selected for assessment, who is involved in the assessment, how stakeholders can engage throughout the process, and when and how stakeholder input will be considered. Finally, insufficient attention has been directed to the implementation of value assessment frameworks in the health care system. When implemented, end users have sometimes found the output challenging or confusing because of issues such as lack of transparent methods and divergent scoring approaches. Some of these issues may disproportionately affect patients, most of whom are not well versed in clinical and health economic methods. With the exception of ICER, none of the sponsoring organizations have developed detailed guidance for how to understand or use findings, which could significantly
301
VALUE IN HEALTH 20 (2017) 299–307
Table 1 – Analysis of key strengths and limitations of existing value frameworks. Framework ACC-AHA
Strengths
Addresses an important therapeutic area that impacts
ASCO
a significant patient population Promotes shared decision making to improve quality of care Considers the quality and reliability of evidence Has robust cost-effectiveness methodology drawn from evidence-based literature Measures unmet patient need qualitatively as a value factor
Addresses an important therapeutic area that impacts
a significant patient population Gives strong emphasis on shared decision making and improving clinical judgment Involves multistakeholder engagement in framework design and evaluation Incorporates patient-reported outcomes and other stakeholder concerns in second revision Has processes in place to improve iterative versions of framework
Limitations
NCCN
Compatible with the NCCN guidelines, which could aid
ICER
DrugAbacus
implementation and have potential impact on decision making Considers the affordability and overall cost of drugs Allows for efficient selection of multiple therapies Affords patients and providers to make customized value judgments on the basis of their preferences Builds off inclusive approach of recommendations in guidelines Graphic representation of measures offers more clarity and efficiency Considers multiple dimensions in the measurement of value Care value considers long-term benefits and costs to the patient Offers a number of contextual considerations, such as the novelty and uniqueness of the drug, in the value assessment Offers opportunities for stakeholder input Publishes a draft report before the official release of an assessment to improve transparency Proposes concrete policy steps to improve the value of medical technology Takes multiple evidence sources into consideration Assessments not limited to drugs Publishes modeling analysis plans for each assessment and provides other technical guides on economic models used Considers multiple dimensions of value, including innovation, research and development costs, disease rarity, etc. Provides users with the flexibility to control weights and preferences
Insufficient consideration of all available evidence Limited patient-centeredness and involvement No ability for users to customize assessment Limited consideration of broader aspects of value (e.g., productivity, social benefits, innovation) Unclear criteria of selecting interventions or care regimens for assessment Reliance on QALYs may not sufficiently capture outcomes important to patients and broader systemwide effects (e.g., long-term benefits of continuum of care) Complex scoring methodology with limited guidance on score interpretation Limited transparency in basic assumptions and model design Unclear validity of methods (e.g., reliance on point estimates of data, unclear weighting of clinical outcomes, use of median survival as benefit measure) Calculation of cost does not consider discounts, rebates, and other offsets Lack of patient engagement in framework design and value assessment Unclear process for implementation Lack of inclusion of broader elements of value (e.g., productivity, unmet need, and innovation) Over-reliance on clinical trial data Unclear selection criteria for interventions or regimens to be assessed in the framework Value definition of affordability is vague Calculation of drug costs does not consider discounts, rebates, and other offsets Panel assessment may be subjective and influenced by particular panel members Unclear or limited stakeholder engagement Lack of inclusion of broader elements of value (e.g., productivity, unmet need, and innovation) Lack of transparency in basic assumptions, model design, and measurement selection
Unclear how “other advantages/disadvantages and contextual
factors of value” are measured and considered in the final assessment Calculation of the budget impact threshold not vetted across stakeholders and focuses on short-term impacts only Assessment of drug costs does not consider potential discounts, rebates, and other offsets Uncertainty regarding how stakeholder input is considered or weighed in final assessment Lack of robust testing and open vetting process before application of initial framework in practice Limited access to economic model/underlying analysis
Assessment of drug costs does not consider discounts, rebates, and other offsets
Does not address expanded indications of drugs over time and frequent changes in measured impact as clinical data evolve Lack of patient-centeredness Limited transparency in underlying methodology Uncertain stakeholder engagement in development process
ACC-AHA, American College of Cardiology and the American Heart Association; ASCO, American Society of Clinical Oncology; ICER, Institute for Clinical and Economic Review; NCCN, National Comprehensive Cancer Network; QALY, quality-adjusted life-year.
Framework Feature
ACC-AHA
ASCO
NCCN
ICER
MSK DrugAbacus
Provide doctors and patients with information about the clinical impact and financial affordability of multiple cancer drug options Physicians and patients
Provide health providers and patients information to make informed choices when selecting systemic cancer therapies
Create a “value-based price benchmark” to help payers better link prices to patient and health system benefit
Help stakeholders determine prices for cancer drugs on the basis of proposed components of a drug’s value
Physicians and patients
Payers/providers, patients, health technology manufacturers, policymakers Primarily drugs, but has assessed devices and delivery system programs; all conditions “How much a new drug benefits a person’s health and well-being, compared to what is available on the market” 1. Care value Comparative clinical effectiveness Incremental costeffectiveness Other benefits or disadvantages Contextual considerations
Payers, providers, patients, drug makers, policymakers
Inform clinical practice guidelines and performance measure formulations on the basis of evidence of costeffectiveness
Target audience(s)
Physicians and patients
Inventions and conditions addressed
Treatment regimens, but primarily drugs; cardiovascular
Drugs, oncologic
Treatment regimens, but primarily drugs; oncologic
Definition of value
“Healthcare benefit for every dollar spent”
“Elements of clinical benefit, toxicity, and symptom palliation derived from a clinical trial”
“Best care for lower cost”; “Value in cancer care is achieved when benefits exceed costs”
Key metrics/components of value
1. Clinical benefit vs. risks Class I: benefit far exceeds risk Class II: benefit exceeds risk Class III: no benefit, no harm 2. Quality of evidence 100 points in total, measured by a questionnaire of 16 questions 3. Cost-effectiveness ratio
1. Clinical benefits (80 points) Overall survival Progression-free survival Response rate 2. Toxicity (⫾20 points) Assessed hierarchically on five levels 3. Bonus points (30 points) Palliation of symptoms Treatment-free interval Significant improvement in quality of life Each performance metric is ranked 1 (worst) to 5 (best); total points add up to 130
1. 2. 3. 4. 5.
High: o$50,000/QALY; Medium: $50,000–$150,000/QALY; Low: 4 $150,000/QALY
Additional qualitative factors measured or considered
Plans to consider unmet need
None
Efficacy Safety Quality of evidence Evidence consistency Affordability
Drugs; oncologic
Not explicitly defined
1. 2. 3. 4. 5. 6.
Efficacy Toxicity Novelty Development cost Disease rarity 6. Health burden
2. Health system value Potential health system budget impact (short-term)
Each “evidence block” is assigned a score (from 1 to 5) on the basis of the NCCN’s multiple guidelines and user’s discretion None
Yes, other benefits and contextual factors, such as disease severity, availability of treatment alternatives, public health benefits, and methods of administration
None
continued on next page
VALUE IN HEALTH 20 (2017) 299–307
Intended purpose
302
Table 2 – Comparative overview of existing value frameworks [1–5].
Mainly health economic studies from literature review
Mainly prospective randomized clinical trials comparing a new treatment with prevailing standard of care
All components ranked by clinical expert panel members on the basis of their experience and knowledge of existing evidence
Depends on technology/ therapeutic area, but mainly randomized controlled trials; could include comparative cohort studies, systematic reviews, and case series
Stakeholder involvement
Unclear
Development involved oncologists, patient advocates, payers, and industry Public comments on initial (and ongoing iterations) framework accepted
Developed mainly by the NCCN staff and members
Assessed by drug acquisition costs and patient out-of-pocket costs; average sales price used as drug acquisition cost
Does not consider budget impact
None, but does consider affordability, which refers to the overall cost of the intervention, including drug cost, required supportive care, infusions, toxicity monitoring and management, etc. Does not consider budget impact
Development involved payers, patient organizations, and industry Public comments accepted on ongoing basis on framework and individual assessments; meetings open to public Considers wholesale acquisition costs
Determination of costs and/or budget impact
Primarily drawn from relevant health economic literature; unclear if budget impact considered
Public comments accepted on ongoing basis
Cost-effectiveness threshold
Over $150,000/QALY is considered cost-ineffective
None
None
Primary “value” output(s)
Level of value (high, medium, low, uncertain, not assessed)
1. A numerical net health benefit score (up to 130) 2. Drug regimen cost
A separate score (from 1 to 5) for each of the four metrics
Applications to date
Still in concept, unclear whether it will be tested
Four case examples using draft framework, but not formally used Pilot currently being planned
Several guidelines include new evidence blocks; guidelines considered beta tests
Considers budget impact, as net change in health care costs over an initial 5-y time period; applies an annual budget impact threshold of $904 million Range from $50,000 to $150,000/QALY
1. An assessment of care value (high/intermediate/ low) 2. Numerical, value-based price benchmark Approximately 15 assessments completed to date*; planned update to framework in 2017
Randomized clinical trials (efficacy and toxicity); expert opinion (novelty); epidemiological data (rarity and health burden); research and development costs (development costs) Unclear, other than MSK staff
Feedback from users encouraged
For part B drugs: Average sales price þ 6%; for part D drugs: “full cost of drug” as reported by the Humana Prescription Drug Plan Enhanced Plan
Does not consider budget impact
Cost-effectiveness weighted in “efficacy”; can range from $12,000 to $300,000/QALY A value-based estimated price for the drug, for comparison with the drug’s list price
VALUE IN HEALTH 20 (2017) 299–307
Sources of evidence
Tool includes 52 approved drugs; unclear whether tested
ACC-AHA, American College of Cardiology and the American Heart Association; ASCO, American Society of Clinical Oncology; ICER, Institute for Clinical and Economic Review; MSK, Memorial Sloan Kettering Cancer Center; NCCN, National Comprehensive Cancer Network; QALY, quality-adjusted life-year. * These include only those assessments using new value assessment framework.
303
304
VALUE IN HEALTH 20 (2017) 299–307
hamper uptake or result in misinterpretation. In addition, most of the frameworks have not been piloted in actual practice to date. How these frameworks impact shared decision making, coverage and pricing determinations, and clinical practice guidelines, among other potential uses, is unclear. A recent survey suggests that payers are not yet considering assessments produced by the new frameworks in coverage and reimbursement decisions [19], and their role in price negotiations with manufacturers also remains uncertain.
Ten Guiding Principles to Improve the Next Generation of Value Assessment Frameworks At this time, it is clear that there is no one perfect model or framework for value assessment, or even one that will garner consensus across all stakeholders. Decision makers will benefit from multiple value frameworks to support their decisions and ensure the availability of relevant and timely information. Despite the need to accommodate a range of approaches, however, a common set of fundamental principles or guidelines may help support future efforts in this area. Considering both identified strengths and limitations of the current approaches, we propose some initial guidelines—as an important first step toward this aim—to ensure high-quality value assessment in the United States.
1.Value assessment frameworks should define and use inclusive and transparent stakeholder engagement processes Value assessment frameworks and individual evaluations should be developed through an open and transparent process that includes broad stakeholder engagement, opportunities for technical and experiential input at critical points in the assessment (e.g., priority setting, scoping, and draft report), and robust conflict of interest protections. Practicing physicians, patients, patient organizations, manufacturers, purchasers, and policymakers, among other stakeholder groups, bring essential expertise and perspective and should have a central role in the prioritization, development, and implementation of value assessments. For example, the patient community can bring different perspectives to value assessment that have been informed by their experiences, trajectory or stage of disease, level of expertise, and many other factors. Tailored mechanisms for patient education and engagement should be developed to ensure that value assessment adequately reflects the patient experience. Furthermore, clear rules for manufacturer communication of evidence should be established to facilitate exchange of data between industry and framework developers. Good engagement practices also include open and regular opportunities for public input on draft reports and transparency into how the feedback received is incorporated. Sufficient notice and time for comment periods are critical to give stakeholders ample opportunity to provide input. To aid in public trust of value assessment, all potential conflicts of interest and funding sources should be publicly disclosed. Given the importance of transparency across all stakeholders, it may be valuable for the stakeholder community to outline the minimum engagement standards required for value assessments in the United States. In determining such standards, the potential limitations of stakeholder involvement should be addressed. For example, an inclusive engagement process that is not timely will reduce the relevance of the resulting value assessment.
2.Value assessment frameworks should explicitly define their priorities and intended purpose(s) Given that value assessment frameworks differ in purpose, it is important that they clearly communicate their stated use(s) and target audience(s) to ensure that their designs, processes, and applications are appropriate for their intended objectives. Establishing explicit and transparent processes for setting priorities, for individual assessments as well as for the frameworks themselves, is widely considered best practice in health technology assessment. Such practices should involve a range of stakeholders, which will help ensure value assessment frameworks meet the needs of diverse stakeholders and yield results that will be broadly accepted. Stakeholder representation or involvement in these activities should not be limited to founding or standing committee members associated with framework developers. Avenues to expand priority-setting opportunities include annual multistakeholder dialogue meetings, citizen councils, and open public topic solicitation processes, such as those used by the Patient-Centered Outcomes Research Institute (PCORI), among others.
3.Value assessment frameworks should include patients’ perspectives Although some value assessment frameworks may target end users other than patients (e.g., payers), incorporating patient values, experiences, and needs should be a part of all value assessment initiatives. Sufficiently integrating patient voices is essential if value frameworks are to have practical utility. Different frameworks may vary in their degree of or approach to patient-centeredness, but should consider outcomes important to patients, such as functional status, health-related quality of life, productivity, and out-of-pocket costs when possible, as well as patient-centered data sources (patient-reported outcomes or patient preferences studies, registries, etc.). Although not all these outcomes will always be relevant, clear rationale for inclusion (or exclusion) of patient-centered outcomes and data should be provided. In addition, value frameworks should reflect the different responses of patient subpopulations on the basis of factors such as age, genetic variation, and comorbidities. Consideration of the patient perspective can be achieved by partnering with patient groups to develop and test value frameworks or via qualitative patient input on individual assessments.
4.Value assessment frameworks should have a holistic, systemwide scope of work that seeks to evaluate a range of interventions and care practices Consistent with PCORI’s mandate for its work on comparative effectiveness, value assessment should consider the full range of health care items and services (e.g., medicines, devices, diagnostics, and surgery), as well as the care management and delivery strategies that influence patient care, to truly evaluate the clinical utility of an intervention and foster accountability for value across the continuum of care. Frameworks that aim to help guide decisions on health care resource allocation, in particular, should be designed to apply to all relevant aspects of clinical care and patient management to support improved system efficiency. Given limited resources, establishing robust priority-setting processes, as previously discussed, is central to achieve a balanced portfolio of topics for evaluation. Greater coordination between value assessment framework developers may also help ensure that a sufficiently broad range of therapies, care strategies, and processes are assessed.
VALUE IN HEALTH 20 (2017) 299–307
5.Value assessment frameworks should be grounded in established and transparent methods Value assessment frameworks should be based on established methodologies, consistent with accepted standards. Value assessment is complex, accompanied by years of debate and investment in the development of credible and robust methods. Guidance from the PCORI, the International Society for Pharmacoeconomics and Outcomes Research, the Agency for Healthcare Research and Quality, Cochrane, and other organizations should be used to the extent possible. Value assessment frameworks should provide transparency in their methods (and analyses) to allow other organizations or analysts to replicate findings, give decision makers confidence in the outputs, and afford innovators predictability in the standards/approaches being used. This should include articulation of the types and quality of data and economic models or synthesis methods involved, assumptions made, any value thresholds employed, and approaches used to make trade-offs between potential outcomes to assess value. Users need to understand assumptions that affect results, whether they have a strong foundation, and the potential effects of alternative assumptions. Value assessment frameworks should acknowledge that the level of uncertainty in a given model and the type of assumptions made can lead to false or incomplete conclusions. To that end, sensitivity analyses—a key component of health technology assessment—should be performed to identify how results could vary in differing scenarios. Finally, because the methods and evidence used in value assessment are often complex, framework developers should ensure that review or stakeholder panels have the necessary clinical and/or analytical expertise to accurately and appropriately guide or interpret the assessment.
6.Value assessment frameworks should capture and apply the full range of available evidence Value assessment frameworks should include a broad range of rigorous and widely available scientific evidence. Although the quality of evidence should be considered, incorporating only a portion of available evidence, particularly only randomized clinical trials, will reduce the applicability of the framework, given the diverse needs and circumstances under which health care is practiced. For example, although only randomized clinical trial data may be available in some cases, other sources of observational evidence, such as comparative cohort studies and patient registries, and systematic reviews should also be considered when feasible and appropriate. Many of these approaches can capture patient activity and preferences and clinical care patterns in ways that are unachievable in clinical trials and help differentiate value between available therapies for a given indication. These considerations will be increasingly important with the ongoing advancement of real-world evidence, developed from registries, clinical care networks, patient-driven social networks, and pragmatic trials and other innovative methods using these emerging data sources. For example, consideration of observational data in value assessments of asthma therapies could capture the preferences and outcomes of important patient subgroups, such as smokers and patients with serious comorbidities and/or adherence problems, that are not often studied in premarket clinical trials. Nevertheless, established methods of assessing and rating the quality and certainty of the evidence should be used so that the quality of the evidence is evaluated and potential limitations or biases in the evidence base are transparent to the user. Several organizations (e.g., Grading of Recommendations, Assessment, Development, and Evaluation Working Group (GRADE), Institute of Medicine, Agency for Healthcare Research and Quality, and Cochrane) have done
305
extensive work on methods for identifying, assessing, and synthesizing the available evidence (e.g., use of meta-analysis), which should be considered by or reflected in value assessments [20,21]. Overall, transparent and vetted processes are needed for incorporating real-world evidence (and, in relation, patientreported outcomes) into the collection of evidence used in value assessments.
7.Value assessment frameworks should address longer term outcomes Value assessments should consider both short-term and longterm outcomes. Those assessments based only on short-term costs and benefits will likely devalue important advances, which often have longer term clinical benefits and some cost offsets through reduced complications (e.g., survival from cancer medicines and reduced heart attacks through management of cholesterol levels) and hospitalizations. Likewise, after the expiration of exclusivity on pharmaceuticals, the treatment may be widely used by a large number of patients at a lower cost, generating benefits that should be recognized in value assessments. A longterm approach should make use of appropriate methods for discounting and consideration of uncertainty around both benefits and costs.
8.Value assessment frameworks should measure and assess relevant costs and cost-effectiveness using sound, vetted methods Considerable debate has transpired regarding the costs measured in the new frameworks, whether and how cost-effectiveness should be assessed, and how affordability is operationalized. For example, some of them consider quality-adjusted life-years and a cost-effectiveness threshold, an approach that has been contested in the United States by various stakeholder groups recently and over the years as new initiatives have been introduced to assess new therapies and services [10,22]. We maintain that the aims of value assessment will not fully be realized if both cost and cost-effectiveness are not considered. At the same time, however, we acknowledge that greater consensus is required on which approaches are best aligned with stakeholder values in the US health care system, namely, regarding cost and costeffectiveness metrics and methods, the use of and criteria for decision thresholds, and potential flexibilities to allow for important exceptions or considerations. Given the different intended purposes and applications of the frameworks, appropriate cost and cost-effectiveness approaches may vary. It is important that framework developers are transparent about the significant limitations of different approaches to cost-effectiveness analysis. Developers should also clearly state the perspective (e.g., payer, societal, or patient) used, present costs and benefits separately to help users understand the magnitude of different economic and health outcomes, and assess the sensitivity of results to reasonable variations in any assumptions required to augment the available evidence.
9.Value assessment frameworks should have the ability to adapt to shifts in science, evidence, values, and the health care system more broadly Value assessment frameworks need to strike an appropriate balance between standardization across assessments, for predictability, encourage the development of the right supportive evidence, and have the flexibility to accommodate different stakeholder perceptions of value and the inherent uncertainty and evolving nature of innovation and associated evidence. To meet these goals, they should be regularly and publicly
306
VALUE IN HEALTH 20 (2017) 299–307
evaluated, to ensure they are effectively accounting for stakeholder needs and values, applying the most credible and current methods, and are having a positive impact on policy and practice. New value assessment frameworks or iterations of existing frameworks should be piloted to sufficiently pressure test their approach within a contained environment, with an opportunity to revise and refine before broad implementation. Framework developers should make these evaluation periods transparent and offer sufficient stakeholder comment opportunities to discuss their evolving strengths and limitations. They should also allow for re-assessments of therapies should new and compelling data emerge and to adequately keep pace with and account for innovation. To have impact, value assessments need to provide decision makers with the most up-to-date and timely evidence.
10.Value assessment frameworks should be developed with feasible implementation strategies reflecting practical opportunities to improve value-based care The results of value assessments should be implemented to support sound, evidence-based decision making. Although many of the frameworks still remain in the development stage and are not yet ready for use in practice, they should be designed and revised with a focus on effective implementation. For example, incentives should be built into alternative payment and delivery models to encourage the use of evidence-based tools to foster shared treatment decision making between patients and providers. New tools are needed to support better tailoring of evidence to different users, accounting for possible access barriers, such as cultural or language considerations or availability of technology. Addressing strategies for implementation in an ad hoc way will be inadequate; value assessment frameworks must be designed to be useable and relevant for their intended use (e.g., shared treatment decision making and coverage determinations). Creating an underlying infrastructure and incentives in which emerging evidence of value is disseminated and integrated throughout the health care system in a timely way will be a long-term and ongoing effort (and likely beyond the remit of framework developers alone), but it is vital to support the broader shift from volume to value of care and sustain investments currently being made in value assessment. Given their inherent limitations and variability in approaches, however, value assessments should serve as one of many tools to aid decision making.
Next Steps In this article, we put forth some common guiding principles— reflective of the diversity of stakeholder interests, needs, and values—to support high-quality value assessment at present and in the future in the United States. Our assessment has some limitations worth noting. We consider this exercise to be a first attempt at developing a set of guiding principles for value assessment from a multistakeholder perspective. This work is limited by the relatively small group of stakeholders involved. In addition, it did not use formal methods of consensus-building (e.g., Delphi or the snow card technique). In support of the key principles and to address the limitations of our analysis, we propose a number of potential next steps. First, current debates around the frameworks highlight the need for additional research and multistakeholder dialogue on defining and measuring value. Such efforts should also strive to identify convergences and differences within stakeholder groups, such as patient groups affected by different diseases. A better understanding of how different stakeholders define and operationalize value can help ensure that value frameworks reflect the
appropriate end points and attain their purported objectives. In addition, it will be important to better understand how target end users (e.g., patients and payers) make trade-offs between different elements of value. For example, how do patients consider mortality and/or morbidity against other considerations, such as value of hope, severity of disease, or convenience of treatment. Ideally, value frameworks will allow for elements of value to be weighed according to the preferences of different end users. Second, the proposed principles should be further refined and vetted to ensure they adequately capture the diversity of perspectives, content, and specificity to effectively guide developers and users. There are considerable convergences between our proposed principles and those recently published by industry and patient groups (e.g., National Pharmaceutical Council, Pharmaceutical Research and Manufacturers of America, and National Health Council), which signal the potential for stakeholder consensus on a common set of elements or practices that all value assessment frameworks can adopt to ensure assessments produced are relevant, methodologically robust, and of high quality. Third, given the divergent and, to some degree, uncertain methodological standards applied by existing frameworks, a consortium of experts and relevant stakeholders could be established to gain consensus on value assessment methodological and procedural standards, such as whether cost-utility analysis, multicriteria decision analysis, or some revision or combination of these approaches might be the best methodological foundation for value assessment. The new initiative on US value assessment frameworks by the International Society for Pharmacoeconomics and Outcomes Research is a commendable step in this direction [23]. Although a methodological foundation may indeed be helpful, identifying or constructing a single underlying theory or approach may be challenging (and potentially limiting) to account for different perspectives and decision contexts. Additional focus should be placed on standards regarding use of patient-related and real-world data (and related research networks, such as PCORnet and ASCO’s CancerLinq), given their lack of inclusion in current frameworks. Fourth, further investment on developing multistakeholder consensus on the implementation of value assessment framework is needed. This could include a series of meetings or workshops on implementation strategies and potential barriers or implications (e.g., resources and restrictions on access to care) to uptake, a needs assessment of resources and expertise required to foster and sustain high-quality value assessment, and methods education and training modules for physicians, patients, payers, and researchers. In addition, although the use of value assessment frameworks to inform shared treatment decision making, clinical guidelines, and pricing and coverage determinations has been explored, other areas of potential application, such as value-based or alternative payment models for drugs and devices, should be considered and potentially piloted. Finally, the impact of value assessment frameworks themselves should be evaluated. Do intended users adopt the framework(s)? Are their stated purpose and aims achieved (e.g., shared decision making)? Do they result in agreed-upon measures, such as reduction in overall cost or improvements in health outcomes? Have the frameworks inappropriately impacted patients’ access to care? Do they enhance transparency and accountability of decision making? Investments in value assessment will be worthwhile only if it benefits the health care system. Attaining better value in health care is a widely shared goal among stakeholders. New value assessment frameworks offer considerable potential to better understand the value of new health interventions and services and improve evidence-based decision making throughout the system. Although much work
VALUE IN HEALTH 20 (2017) 299–307
remains, with continued collaboration and commitment across stakeholders, we can make meaningful progress toward meeting these aims. Source of financial support: No financial support was received for this article. R EF E R EN CE S
[13]
[14]
[15] [1] Anderson JL, Heidenreich PA, Barnett PG, et al. ACC/AHA statement on cost/value methodology in clinical practice guidelines and performance measures. J Am Coll Cardiol 2014;63:2304–22. [2] Schnipper LE, Davidson NE, Wollins DS, et al. American Society of Clinical Oncology statement: a conceptual framework to assess the value of cancer treatment options. J Clin Oncol 2015;33:2563–77. [3] Institute for Clinical and Economic Review. ICER value assessment framework. Available from: https://icer-review.org/methodology/ icers-methods/icer-value-assessment-framework/. [Accessed July 29, 2016]. [4] Memorial Sloan Kettering Cancer Center. DrugAbacus methods. Available from: http://www.drugabacus.org/drug-abacus/methods/. [Accessed August 3, 2016]. [5] National Comprehensive Cancer Network. NCCN clinical practice guidelines in oncology (NCCN Guideliness) with NCCN Evidence Blocks™. Available from: https://www.nccn.org/evidenceblocks/. [Accessed August 3, 2016]. [6] Swanick M, Strang D. Third party drug assessors: influential new voices in the value debate. Available from: http://www.pwc.com/us/en/ health-industries/health-research-institute/publications/ third-party-drug-assessors.html. [Accessed August 3, 2016]. [7] Westrich K. Current landscape: value assessment frameworks. Available from: http://www.npcnow.org/system/files/research/ download/npc-current-landscape-value-assessment-frameworks-final. pdf/. [Accessed July 15, 2016]. [8] Angelis A, Kanavos P. Critique of the American Society of Clinical Oncology value assessment framework for cancer treatments: putting methodologic robustness first. J Clin Oncol 2016;34:2935–6. [9] Basch E. Toward a patient-centered value framework in oncology. JAMA 2016;315:2073–4. [10] Neumann PJ, Cohen JT. Measuring the value of prescription drugs. N Engl J Med 2015;373:2595–7. [11] Schnipper LE, Bastian A. New frameworks to assess value of cancer care: strengths and limitations. Oncologist 2016;21:654–8. [12] Perrone M. Breakthrough cholesterol drugs fizzle amid price pushback. Available from: http://bigstory.ap.org/article/
[16]
[17]
[18]
[19]
[20]
[21] [22]
[23]
307
a15448a756dc4bfc9bc78174b14ccb6f/breakthrough-cholesteroldrugs-fizzle-amid-price-pushback. [Accessed August 13, 2016]. Caffrey MK. For now, PBMs just say no to high-cost PCSK9 inhibitors. Available from: http://www.ajmc.com/journals/ evidence-based-diabetes-management/2015/september-2015/ for-now-pbms–just-say-no-to–high-cost-pcsk9–inhibitors. [Accessed August 13, 2016]. National Health Council. The patient voice in value: the National Health Council patient-centered value model rubric. Available from: http://www.nationalhealthcouncil.org/sites/default/files/Value-Rubric. pdf. [Accessed July 10, 2016]. Longacre ML, Charap ES, Buzaglo JS, et al. Defining value in oncology: perspectives from patients with metastatic breast cancer. Available from: http://www.cancersupportcommunity.org/sites/default/files/ uploads/our-research/presentations/access-to-care/ avbcc_value_poster_.pdf/. [Accessed July 15, 2016]. Schnipper LE, Davidson NE, Wollins DS, et al. Updating the American Society of Clinical Oncology value framework: revisions and reflections in response to comments received. J Clin Oncol 2016;34:2925–34. National Comprehensive Cancer Network. NCCN guidelines for patients. Multiple myeloma, version 1.2016. Available from: https:// www.nccn.org/patients/guidelines/myeloma/files/assets/basic-html/ page-1.html. [Accessed August 10, 2016]. Institute for Clinical and Economic Review. Multiple myeloma: final evidence report and meeting summary. Available from: https:// icer-review.org/material/mm-final-report/. [Accessed August 10, 2016]. Seidman J, Choe SH. Most health plans do not use existing value frameworks to make coverage decisions. Available from: http://avalere. com/expertise/life-sciences/insights/ most-health-plans-do-not-use-value-frameworks-to-make-coveragedecisions/. [Accessed August 3, 2016]. Stroup DF, Berlin JA, Morton SC, et al. Meta-analysis of observational studies in epidemiology: a proposal for reporting. Meta-analysis of Observational Studies in Epidemiology (MOOSE) group. JAMA 2000;283:2008–12. Ades A, Caldwell DM, Reken S, et al. Evidence synthesis for decision making: a reviewer’s checklist. Med Decis Making 2013;33:679–91. McKenna C, Soares M, Claxton K, et al. Unifying research and reimbursement decisions: case studies demonstrating the sequence of assessment and judgments required. Value Health 2015;18:865–75. International Society for Pharmacoeconomics and Outcomes Research. Initiative on US value assessment frameworks. Available from: http:// www.ispor.org/ValueAssessmentFrameworks/Index. [Accessed August 20, 2016].
Available online at www.sciencedirect.com
journal homepage: www.elsevier.com/locate/jval
Advancing Value Assessment in the United States: A Multistakeholder Perspective Corinna Sorenson, MPH, MHSA, PhD1,*, Gabriela Lavezzari, PhD, MBA2, Gregory Daniel, MPH, MS, PhD2, Randy Burkholder, BA3, Marc Boutin, JD4, Edmund Pezalla, MD, MPH5, Gillian Sanders, PhD1, Mark McClellan, MD, PhD2 1 Duke Clinical Research Institute, Duke School of Medicine, Duke University, Durham, NC, USA; 2Duke-Robert J. Margolis, MD Center for Health Policy, Duke University, Durham, NC, USA; 3Pharmaceutical Research and Manufacturers of America, Washington, DC, USA; 4National Health Council, Washington, DC, USA; 5Enlightenment Bioconsult, LLC, Wethersfield, CT, USA
AB STR A CT
Rising costs without perceived proportional improvements in quality and outcomes have motivated fundamental shifts in health care delivery and payment to achieve better value. Aligned with these efforts, several value assessment frameworks have been introduced recently to help providers, patients, and payers better understand the potential value of drugs and other interventions and make informed decisions about their use. Given their early stage of development, it is imperative to evaluate these efforts on an ongoing basis to identify how best to support and improve them moving forward. This article provides a multistakeholder perspective on the key limitations and opportunities posed by the current value assessment frameworks and areas of and actions for improvement. In particular, we outline 10
fundamental guiding principles and associated strategies that should be considered in subsequent iterations of the existing frameworks or by emerging initiatives in the future. Although value assessment frameworks may not be able to meet all the needs and preferences of stakeholders, we contend that there are common elements and potential next steps that can be supported to advance value assessment in the United States. Keywords: health care decision makers, standards, value assessment, value for money.
Introduction
determinations or price negotiations between payers and manufacturers. Current value assessment frameworks have received praise as tools to achieve higher value health care, but have also been criticized for their limitations regarding methods, processes, implementation, and potential impacts on patients’ access to care [6–11]. For example, some patients have experienced barriers obtaining new cholesterol-lowering drugs, PCSK9 inhibitors, after their (relatively unfavorable) value assessment by ICER and public debate regarding their cost [12,13]. Given their early stage of development, and their potential impact on care access and delivery, timely and ongoing evaluation of these efforts is critical to improve them and to help ensure they are implemented effectively. Such an exercise should involve all stakeholders to obtain a comprehensive, balanced, and representative assessment. In addition, we maintain that stakeholder input and, to some degree, buy-in are important to ensure the adoption and sustainability of value assessment in the United States. To meet this aim, we convened a small group of eight stakeholders representing patient advocacy organizations, drug manufacturers, payers, and academia to explore and discuss the
Rising health care costs in the United States have created significant financial burden on patients and their families, health providers, industry, and payers. New, high-priced treatments and services have faced scrutiny regarding the value they deliver in relation to their costs, especially because policymakers have accelerated efforts to shift from traditional “fee-for-service” reimbursement to “value-based” payment. Amid these efforts, a number of professional and private organizations have put forth value assessment frameworks to define and measure the value of drugs and other therapies in a more explicit and systematic manner. Such frameworks have been introduced by the American College of Cardiology and American Heart Association (ACC-AHA) [1], American Society of Clinical Oncology (ASCO) [2], Institute for Clinical and Economic Review (ICER) [3], Memorial Sloan Kettering Cancer Center (MSK) [4], and National Comprehensive Cancer Network (NCCN) [5]. The aims of these frameworks differ—some seek to help physicians and patients make more informed, evidence-based treatment decisions, whereas others are intended to aid payer coverage
Copyright & 2017, International Society for Pharmacoeconomics and Outcomes Research (ISPOR). Published by Elsevier Inc.
* Address correspondence to: Corinna Sorenson, Duke Clinical Research Institute, Duke School of Medicine, Duke University, 2400 Pratt Street, Durham, NC 27705. E-mail: [email protected]. 1098-3015$36.00 – see front matter Copyright & 2017, International Society for Pharmacoeconomics and Outcomes Research (ISPOR). Published by Elsevier Inc. http://dx.doi.org/10.1016/j.jval.2016.11.030
300
VALUE IN HEALTH 20 (2017) 299–307
following four areas: 1) main gaps and opportunities posed by the current value assessment frameworks, 2) differences and convergences in stakeholder definitions of value, 3) key guiding principles for value assessment, and 4) potential next steps for improvement. A set of questions was developed and sent to involved stakeholders to gather their respective input across these four areas. Responses to these questions were then discussed during two rounds of meetings with all stakeholder representatives to further flesh out similarities and differences across perspectives and reach consensus on key guiding principles for value assessment. This article offers the resulting multistakeholder perspective on value frameworks, outlining 10 fundamental guiding principles and associated strategies that should be considered in subsequent iterations of the existing value assessment frameworks or by emerging efforts in the future. This multistakeholder approach differs from the approaches of other groups, including the National Pharmaceutical Council and the Pharmaceutical Research and Manufacturers of America, which have recently put forth principles or practices for value assessment. A value assessment framework may not be able to meet all the needs of all the stakeholders, but we contend there are common elements and steps forward that can be supported and adopted to advance high-quality value assessment in the United States.
A Multistakeholder Perspective on Existing Value Frameworks Emerging value assessment frameworks have both commonalities and differences across intended aims, methods, processes, and applications, which are largely attributable to the interests and expertise of the developing organizations and intended uses of the respective frameworks (Table 2). For example, as professional medical societies, ASCO and the NCCN designed their frameworks to enhance shared treatment decision making between patients and physicians, whereas ICER and the MSK have developed frameworks to mainly support payer coverage and pricing determinations. These frameworks offer new tools to better understand and measure the value of new drugs and other interventions to support policy and practice and should be recognized for fostering broader conservations about these issues. Many of these efforts have strived to evolve (or plan to do so) on the basis of stakeholder input on their strengths and limitations (Table 1). Despite these steps, important issues remain that should be addressed before value assessment frameworks can be broadly implemented. First, the existing value assessment frameworks apply a somewhat narrow and static conception of value. In particular, the elements or components of value considered across frameworks principally focus on clinical and economic outcomes. While some frameworks do take into account quality of life, severity of disease, and daily functioning, for example, patient organizations, among other stakeholders, argue that insufficient consideration is given to outcomes important to patients and their caregivers, such as personal aspirations, productivity, outof-pocket costs, hope, convenience, and certainty of prognosis or treatment [14]. Patient perspectives on value, which are often individual and disease-dependent, can differ from those of payers and physicians in significant ways. For example, a recent study of patients with metastatic breast cancer showed that patients tend to emphasize value in terms of their personal benefit (e.g., ability to maintain rich relationships with family members) rather than in economic terms [15]. In addition, most of the frameworks fail to account for the range of benefits afforded by new or incremental innovation. For example,
incremental innovation may support a new delivery mechanism, such as a patch versus a pill, that results in more sustained therapeutic drug levels, improved patient treatment adherence, and enhanced clinical choice. In particular, the industry is concerned that such advances (as well as more substantial innovations, such as curative therapies) will not be recognized sufficiently for the improvements in health outcomes and potential cost offsets they deliver over time [7]. From a payer perspective, the existing frameworks are limited in their scope by focusing narrowly on patients and individual technologies, without due account of a population or societal perspective and the continuum of patient care [16]. Second, the underlying methods used by the frameworks have not been sufficiently vetted against established standards or through broad stakeholder consensus. There is limited consistency across frameworks regarding standards for grading the quality of available evidence, evidence synthesis, costing and budget impact methods, or weighing benefits versus costs (e.g., cost-utility analysis and multicriteria decision analysis). Consequently, individual assessments may differ across frameworks, as demonstrated in the case of new treatments for multiple myeloma. ASCO calculated a fairly low net health benefit (47/130) and substantially greater monthly costs for a newer regimen (bortezomib þ melphalan þ prednisone) than did the previous standard of care (melphalan þ prednisone) ($7042 vs. $279), but did not assess the quality of evidence [2]. The NCCN provided scores of 4 for efficacy, 3 for safety, 4 for quality and consistency of the evidence, and 3 for affordability (all out of 5) for the same three-drug regimen [17]. ICER examined six new second-line or later treatment regimens and determined that care value was low to intermediate, depending on the regimen, and that none represents good value in the long-term and therefore should be offered at a price discount of more than 75% [18]. The ICER also noted that the available evidence on the individual regimens was largely adequate, but that insufficient data exist to distinguish comparative net health benefit between them. Furthermore, several of the present frameworks do not consider the full range of available evidence, limiting their evidence base to clinical trials, and sometimes only a single trial. Although clinical trials are often the highest quality evidence available, the resulting evidence base is often limited and can be enhanced by considering additional study designs, such as well-designed observational studies. Similarly, none of the frameworks currently consider real-world or patient-generated/patient-desired data in assessments or outline clear policies on updating assessments if new evidence arises, which may limit capturing the value of therapies in actual clinical practice and accounting for the evolving nature of innovation. Third, the transparency of the methods and processes used by value assessment frameworks is limited. In addition to the actual frameworks themselves, the data, economic models, and underlying assumptions of individual assessments have not often been made public, which hinders interpretation of and confidence in the results. Transparency concerns extend to how therapies are selected for assessment, who is involved in the assessment, how stakeholders can engage throughout the process, and when and how stakeholder input will be considered. Finally, insufficient attention has been directed to the implementation of value assessment frameworks in the health care system. When implemented, end users have sometimes found the output challenging or confusing because of issues such as lack of transparent methods and divergent scoring approaches. Some of these issues may disproportionately affect patients, most of whom are not well versed in clinical and health economic methods. With the exception of ICER, none of the sponsoring organizations have developed detailed guidance for how to understand or use findings, which could significantly
301
VALUE IN HEALTH 20 (2017) 299–307
Table 1 – Analysis of key strengths and limitations of existing value frameworks. Framework ACC-AHA
Strengths
Addresses an important therapeutic area that impacts
ASCO
a significant patient population Promotes shared decision making to improve quality of care Considers the quality and reliability of evidence Has robust cost-effectiveness methodology drawn from evidence-based literature Measures unmet patient need qualitatively as a value factor
Addresses an important therapeutic area that impacts
a significant patient population Gives strong emphasis on shared decision making and improving clinical judgment Involves multistakeholder engagement in framework design and evaluation Incorporates patient-reported outcomes and other stakeholder concerns in second revision Has processes in place to improve iterative versions of framework
Limitations
NCCN
Compatible with the NCCN guidelines, which could aid
ICER
DrugAbacus
implementation and have potential impact on decision making Considers the affordability and overall cost of drugs Allows for efficient selection of multiple therapies Affords patients and providers to make customized value judgments on the basis of their preferences Builds off inclusive approach of recommendations in guidelines Graphic representation of measures offers more clarity and efficiency Considers multiple dimensions in the measurement of value Care value considers long-term benefits and costs to the patient Offers a number of contextual considerations, such as the novelty and uniqueness of the drug, in the value assessment Offers opportunities for stakeholder input Publishes a draft report before the official release of an assessment to improve transparency Proposes concrete policy steps to improve the value of medical technology Takes multiple evidence sources into consideration Assessments not limited to drugs Publishes modeling analysis plans for each assessment and provides other technical guides on economic models used Considers multiple dimensions of value, including innovation, research and development costs, disease rarity, etc. Provides users with the flexibility to control weights and preferences
Insufficient consideration of all available evidence Limited patient-centeredness and involvement No ability for users to customize assessment Limited consideration of broader aspects of value (e.g., productivity, social benefits, innovation) Unclear criteria of selecting interventions or care regimens for assessment Reliance on QALYs may not sufficiently capture outcomes important to patients and broader systemwide effects (e.g., long-term benefits of continuum of care) Complex scoring methodology with limited guidance on score interpretation Limited transparency in basic assumptions and model design Unclear validity of methods (e.g., reliance on point estimates of data, unclear weighting of clinical outcomes, use of median survival as benefit measure) Calculation of cost does not consider discounts, rebates, and other offsets Lack of patient engagement in framework design and value assessment Unclear process for implementation Lack of inclusion of broader elements of value (e.g., productivity, unmet need, and innovation) Over-reliance on clinical trial data Unclear selection criteria for interventions or regimens to be assessed in the framework Value definition of affordability is vague Calculation of drug costs does not consider discounts, rebates, and other offsets Panel assessment may be subjective and influenced by particular panel members Unclear or limited stakeholder engagement Lack of inclusion of broader elements of value (e.g., productivity, unmet need, and innovation) Lack of transparency in basic assumptions, model design, and measurement selection
Unclear how “other advantages/disadvantages and contextual
factors of value” are measured and considered in the final assessment Calculation of the budget impact threshold not vetted across stakeholders and focuses on short-term impacts only Assessment of drug costs does not consider potential discounts, rebates, and other offsets Uncertainty regarding how stakeholder input is considered or weighed in final assessment Lack of robust testing and open vetting process before application of initial framework in practice Limited access to economic model/underlying analysis
Assessment of drug costs does not consider discounts, rebates, and other offsets
Does not address expanded indications of drugs over time and frequent changes in measured impact as clinical data evolve Lack of patient-centeredness Limited transparency in underlying methodology Uncertain stakeholder engagement in development process
ACC-AHA, American College of Cardiology and the American Heart Association; ASCO, American Society of Clinical Oncology; ICER, Institute for Clinical and Economic Review; NCCN, National Comprehensive Cancer Network; QALY, quality-adjusted life-year.
Framework Feature
ACC-AHA
ASCO
NCCN
ICER
MSK DrugAbacus
Provide doctors and patients with information about the clinical impact and financial affordability of multiple cancer drug options Physicians and patients
Provide health providers and patients information to make informed choices when selecting systemic cancer therapies
Create a “value-based price benchmark” to help payers better link prices to patient and health system benefit
Help stakeholders determine prices for cancer drugs on the basis of proposed components of a drug’s value
Physicians and patients
Payers/providers, patients, health technology manufacturers, policymakers Primarily drugs, but has assessed devices and delivery system programs; all conditions “How much a new drug benefits a person’s health and well-being, compared to what is available on the market” 1. Care value Comparative clinical effectiveness Incremental costeffectiveness Other benefits or disadvantages Contextual considerations
Payers, providers, patients, drug makers, policymakers
Inform clinical practice guidelines and performance measure formulations on the basis of evidence of costeffectiveness
Target audience(s)
Physicians and patients
Inventions and conditions addressed
Treatment regimens, but primarily drugs; cardiovascular
Drugs, oncologic
Treatment regimens, but primarily drugs; oncologic
Definition of value
“Healthcare benefit for every dollar spent”
“Elements of clinical benefit, toxicity, and symptom palliation derived from a clinical trial”
“Best care for lower cost”; “Value in cancer care is achieved when benefits exceed costs”
Key metrics/components of value
1. Clinical benefit vs. risks Class I: benefit far exceeds risk Class II: benefit exceeds risk Class III: no benefit, no harm 2. Quality of evidence 100 points in total, measured by a questionnaire of 16 questions 3. Cost-effectiveness ratio
1. Clinical benefits (80 points) Overall survival Progression-free survival Response rate 2. Toxicity (⫾20 points) Assessed hierarchically on five levels 3. Bonus points (30 points) Palliation of symptoms Treatment-free interval Significant improvement in quality of life Each performance metric is ranked 1 (worst) to 5 (best); total points add up to 130
1. 2. 3. 4. 5.
High: o$50,000/QALY; Medium: $50,000–$150,000/QALY; Low: 4 $150,000/QALY
Additional qualitative factors measured or considered
Plans to consider unmet need
None
Efficacy Safety Quality of evidence Evidence consistency Affordability
Drugs; oncologic
Not explicitly defined
1. 2. 3. 4. 5. 6.
Efficacy Toxicity Novelty Development cost Disease rarity 6. Health burden
2. Health system value Potential health system budget impact (short-term)
Each “evidence block” is assigned a score (from 1 to 5) on the basis of the NCCN’s multiple guidelines and user’s discretion None
Yes, other benefits and contextual factors, such as disease severity, availability of treatment alternatives, public health benefits, and methods of administration
None
continued on next page
VALUE IN HEALTH 20 (2017) 299–307
Intended purpose
302
Table 2 – Comparative overview of existing value frameworks [1–5].
Mainly health economic studies from literature review
Mainly prospective randomized clinical trials comparing a new treatment with prevailing standard of care
All components ranked by clinical expert panel members on the basis of their experience and knowledge of existing evidence
Depends on technology/ therapeutic area, but mainly randomized controlled trials; could include comparative cohort studies, systematic reviews, and case series
Stakeholder involvement
Unclear
Development involved oncologists, patient advocates, payers, and industry Public comments on initial (and ongoing iterations) framework accepted
Developed mainly by the NCCN staff and members
Assessed by drug acquisition costs and patient out-of-pocket costs; average sales price used as drug acquisition cost
Does not consider budget impact
None, but does consider affordability, which refers to the overall cost of the intervention, including drug cost, required supportive care, infusions, toxicity monitoring and management, etc. Does not consider budget impact
Development involved payers, patient organizations, and industry Public comments accepted on ongoing basis on framework and individual assessments; meetings open to public Considers wholesale acquisition costs
Determination of costs and/or budget impact
Primarily drawn from relevant health economic literature; unclear if budget impact considered
Public comments accepted on ongoing basis
Cost-effectiveness threshold
Over $150,000/QALY is considered cost-ineffective
None
None
Primary “value” output(s)
Level of value (high, medium, low, uncertain, not assessed)
1. A numerical net health benefit score (up to 130) 2. Drug regimen cost
A separate score (from 1 to 5) for each of the four metrics
Applications to date
Still in concept, unclear whether it will be tested
Four case examples using draft framework, but not formally used Pilot currently being planned
Several guidelines include new evidence blocks; guidelines considered beta tests
Considers budget impact, as net change in health care costs over an initial 5-y time period; applies an annual budget impact threshold of $904 million Range from $50,000 to $150,000/QALY
1. An assessment of care value (high/intermediate/ low) 2. Numerical, value-based price benchmark Approximately 15 assessments completed to date*; planned update to framework in 2017
Randomized clinical trials (efficacy and toxicity); expert opinion (novelty); epidemiological data (rarity and health burden); research and development costs (development costs) Unclear, other than MSK staff
Feedback from users encouraged
For part B drugs: Average sales price þ 6%; for part D drugs: “full cost of drug” as reported by the Humana Prescription Drug Plan Enhanced Plan
Does not consider budget impact
Cost-effectiveness weighted in “efficacy”; can range from $12,000 to $300,000/QALY A value-based estimated price for the drug, for comparison with the drug’s list price
VALUE IN HEALTH 20 (2017) 299–307
Sources of evidence
Tool includes 52 approved drugs; unclear whether tested
ACC-AHA, American College of Cardiology and the American Heart Association; ASCO, American Society of Clinical Oncology; ICER, Institute for Clinical and Economic Review; MSK, Memorial Sloan Kettering Cancer Center; NCCN, National Comprehensive Cancer Network; QALY, quality-adjusted life-year. * These include only those assessments using new value assessment framework.
303
304
VALUE IN HEALTH 20 (2017) 299–307
hamper uptake or result in misinterpretation. In addition, most of the frameworks have not been piloted in actual practice to date. How these frameworks impact shared decision making, coverage and pricing determinations, and clinical practice guidelines, among other potential uses, is unclear. A recent survey suggests that payers are not yet considering assessments produced by the new frameworks in coverage and reimbursement decisions [19], and their role in price negotiations with manufacturers also remains uncertain.
Ten Guiding Principles to Improve the Next Generation of Value Assessment Frameworks At this time, it is clear that there is no one perfect model or framework for value assessment, or even one that will garner consensus across all stakeholders. Decision makers will benefit from multiple value frameworks to support their decisions and ensure the availability of relevant and timely information. Despite the need to accommodate a range of approaches, however, a common set of fundamental principles or guidelines may help support future efforts in this area. Considering both identified strengths and limitations of the current approaches, we propose some initial guidelines—as an important first step toward this aim—to ensure high-quality value assessment in the United States.
1.Value assessment frameworks should define and use inclusive and transparent stakeholder engagement processes Value assessment frameworks and individual evaluations should be developed through an open and transparent process that includes broad stakeholder engagement, opportunities for technical and experiential input at critical points in the assessment (e.g., priority setting, scoping, and draft report), and robust conflict of interest protections. Practicing physicians, patients, patient organizations, manufacturers, purchasers, and policymakers, among other stakeholder groups, bring essential expertise and perspective and should have a central role in the prioritization, development, and implementation of value assessments. For example, the patient community can bring different perspectives to value assessment that have been informed by their experiences, trajectory or stage of disease, level of expertise, and many other factors. Tailored mechanisms for patient education and engagement should be developed to ensure that value assessment adequately reflects the patient experience. Furthermore, clear rules for manufacturer communication of evidence should be established to facilitate exchange of data between industry and framework developers. Good engagement practices also include open and regular opportunities for public input on draft reports and transparency into how the feedback received is incorporated. Sufficient notice and time for comment periods are critical to give stakeholders ample opportunity to provide input. To aid in public trust of value assessment, all potential conflicts of interest and funding sources should be publicly disclosed. Given the importance of transparency across all stakeholders, it may be valuable for the stakeholder community to outline the minimum engagement standards required for value assessments in the United States. In determining such standards, the potential limitations of stakeholder involvement should be addressed. For example, an inclusive engagement process that is not timely will reduce the relevance of the resulting value assessment.
2.Value assessment frameworks should explicitly define their priorities and intended purpose(s) Given that value assessment frameworks differ in purpose, it is important that they clearly communicate their stated use(s) and target audience(s) to ensure that their designs, processes, and applications are appropriate for their intended objectives. Establishing explicit and transparent processes for setting priorities, for individual assessments as well as for the frameworks themselves, is widely considered best practice in health technology assessment. Such practices should involve a range of stakeholders, which will help ensure value assessment frameworks meet the needs of diverse stakeholders and yield results that will be broadly accepted. Stakeholder representation or involvement in these activities should not be limited to founding or standing committee members associated with framework developers. Avenues to expand priority-setting opportunities include annual multistakeholder dialogue meetings, citizen councils, and open public topic solicitation processes, such as those used by the Patient-Centered Outcomes Research Institute (PCORI), among others.
3.Value assessment frameworks should include patients’ perspectives Although some value assessment frameworks may target end users other than patients (e.g., payers), incorporating patient values, experiences, and needs should be a part of all value assessment initiatives. Sufficiently integrating patient voices is essential if value frameworks are to have practical utility. Different frameworks may vary in their degree of or approach to patient-centeredness, but should consider outcomes important to patients, such as functional status, health-related quality of life, productivity, and out-of-pocket costs when possible, as well as patient-centered data sources (patient-reported outcomes or patient preferences studies, registries, etc.). Although not all these outcomes will always be relevant, clear rationale for inclusion (or exclusion) of patient-centered outcomes and data should be provided. In addition, value frameworks should reflect the different responses of patient subpopulations on the basis of factors such as age, genetic variation, and comorbidities. Consideration of the patient perspective can be achieved by partnering with patient groups to develop and test value frameworks or via qualitative patient input on individual assessments.
4.Value assessment frameworks should have a holistic, systemwide scope of work that seeks to evaluate a range of interventions and care practices Consistent with PCORI’s mandate for its work on comparative effectiveness, value assessment should consider the full range of health care items and services (e.g., medicines, devices, diagnostics, and surgery), as well as the care management and delivery strategies that influence patient care, to truly evaluate the clinical utility of an intervention and foster accountability for value across the continuum of care. Frameworks that aim to help guide decisions on health care resource allocation, in particular, should be designed to apply to all relevant aspects of clinical care and patient management to support improved system efficiency. Given limited resources, establishing robust priority-setting processes, as previously discussed, is central to achieve a balanced portfolio of topics for evaluation. Greater coordination between value assessment framework developers may also help ensure that a sufficiently broad range of therapies, care strategies, and processes are assessed.
VALUE IN HEALTH 20 (2017) 299–307
5.Value assessment frameworks should be grounded in established and transparent methods Value assessment frameworks should be based on established methodologies, consistent with accepted standards. Value assessment is complex, accompanied by years of debate and investment in the development of credible and robust methods. Guidance from the PCORI, the International Society for Pharmacoeconomics and Outcomes Research, the Agency for Healthcare Research and Quality, Cochrane, and other organizations should be used to the extent possible. Value assessment frameworks should provide transparency in their methods (and analyses) to allow other organizations or analysts to replicate findings, give decision makers confidence in the outputs, and afford innovators predictability in the standards/approaches being used. This should include articulation of the types and quality of data and economic models or synthesis methods involved, assumptions made, any value thresholds employed, and approaches used to make trade-offs between potential outcomes to assess value. Users need to understand assumptions that affect results, whether they have a strong foundation, and the potential effects of alternative assumptions. Value assessment frameworks should acknowledge that the level of uncertainty in a given model and the type of assumptions made can lead to false or incomplete conclusions. To that end, sensitivity analyses—a key component of health technology assessment—should be performed to identify how results could vary in differing scenarios. Finally, because the methods and evidence used in value assessment are often complex, framework developers should ensure that review or stakeholder panels have the necessary clinical and/or analytical expertise to accurately and appropriately guide or interpret the assessment.
6.Value assessment frameworks should capture and apply the full range of available evidence Value assessment frameworks should include a broad range of rigorous and widely available scientific evidence. Although the quality of evidence should be considered, incorporating only a portion of available evidence, particularly only randomized clinical trials, will reduce the applicability of the framework, given the diverse needs and circumstances under which health care is practiced. For example, although only randomized clinical trial data may be available in some cases, other sources of observational evidence, such as comparative cohort studies and patient registries, and systematic reviews should also be considered when feasible and appropriate. Many of these approaches can capture patient activity and preferences and clinical care patterns in ways that are unachievable in clinical trials and help differentiate value between available therapies for a given indication. These considerations will be increasingly important with the ongoing advancement of real-world evidence, developed from registries, clinical care networks, patient-driven social networks, and pragmatic trials and other innovative methods using these emerging data sources. For example, consideration of observational data in value assessments of asthma therapies could capture the preferences and outcomes of important patient subgroups, such as smokers and patients with serious comorbidities and/or adherence problems, that are not often studied in premarket clinical trials. Nevertheless, established methods of assessing and rating the quality and certainty of the evidence should be used so that the quality of the evidence is evaluated and potential limitations or biases in the evidence base are transparent to the user. Several organizations (e.g., Grading of Recommendations, Assessment, Development, and Evaluation Working Group (GRADE), Institute of Medicine, Agency for Healthcare Research and Quality, and Cochrane) have done
305
extensive work on methods for identifying, assessing, and synthesizing the available evidence (e.g., use of meta-analysis), which should be considered by or reflected in value assessments [20,21]. Overall, transparent and vetted processes are needed for incorporating real-world evidence (and, in relation, patientreported outcomes) into the collection of evidence used in value assessments.
7.Value assessment frameworks should address longer term outcomes Value assessments should consider both short-term and longterm outcomes. Those assessments based only on short-term costs and benefits will likely devalue important advances, which often have longer term clinical benefits and some cost offsets through reduced complications (e.g., survival from cancer medicines and reduced heart attacks through management of cholesterol levels) and hospitalizations. Likewise, after the expiration of exclusivity on pharmaceuticals, the treatment may be widely used by a large number of patients at a lower cost, generating benefits that should be recognized in value assessments. A longterm approach should make use of appropriate methods for discounting and consideration of uncertainty around both benefits and costs.
8.Value assessment frameworks should measure and assess relevant costs and cost-effectiveness using sound, vetted methods Considerable debate has transpired regarding the costs measured in the new frameworks, whether and how cost-effectiveness should be assessed, and how affordability is operationalized. For example, some of them consider quality-adjusted life-years and a cost-effectiveness threshold, an approach that has been contested in the United States by various stakeholder groups recently and over the years as new initiatives have been introduced to assess new therapies and services [10,22]. We maintain that the aims of value assessment will not fully be realized if both cost and cost-effectiveness are not considered. At the same time, however, we acknowledge that greater consensus is required on which approaches are best aligned with stakeholder values in the US health care system, namely, regarding cost and costeffectiveness metrics and methods, the use of and criteria for decision thresholds, and potential flexibilities to allow for important exceptions or considerations. Given the different intended purposes and applications of the frameworks, appropriate cost and cost-effectiveness approaches may vary. It is important that framework developers are transparent about the significant limitations of different approaches to cost-effectiveness analysis. Developers should also clearly state the perspective (e.g., payer, societal, or patient) used, present costs and benefits separately to help users understand the magnitude of different economic and health outcomes, and assess the sensitivity of results to reasonable variations in any assumptions required to augment the available evidence.
9.Value assessment frameworks should have the ability to adapt to shifts in science, evidence, values, and the health care system more broadly Value assessment frameworks need to strike an appropriate balance between standardization across assessments, for predictability, encourage the development of the right supportive evidence, and have the flexibility to accommodate different stakeholder perceptions of value and the inherent uncertainty and evolving nature of innovation and associated evidence. To meet these goals, they should be regularly and publicly
306
VALUE IN HEALTH 20 (2017) 299–307
evaluated, to ensure they are effectively accounting for stakeholder needs and values, applying the most credible and current methods, and are having a positive impact on policy and practice. New value assessment frameworks or iterations of existing frameworks should be piloted to sufficiently pressure test their approach within a contained environment, with an opportunity to revise and refine before broad implementation. Framework developers should make these evaluation periods transparent and offer sufficient stakeholder comment opportunities to discuss their evolving strengths and limitations. They should also allow for re-assessments of therapies should new and compelling data emerge and to adequately keep pace with and account for innovation. To have impact, value assessments need to provide decision makers with the most up-to-date and timely evidence.
10.Value assessment frameworks should be developed with feasible implementation strategies reflecting practical opportunities to improve value-based care The results of value assessments should be implemented to support sound, evidence-based decision making. Although many of the frameworks still remain in the development stage and are not yet ready for use in practice, they should be designed and revised with a focus on effective implementation. For example, incentives should be built into alternative payment and delivery models to encourage the use of evidence-based tools to foster shared treatment decision making between patients and providers. New tools are needed to support better tailoring of evidence to different users, accounting for possible access barriers, such as cultural or language considerations or availability of technology. Addressing strategies for implementation in an ad hoc way will be inadequate; value assessment frameworks must be designed to be useable and relevant for their intended use (e.g., shared treatment decision making and coverage determinations). Creating an underlying infrastructure and incentives in which emerging evidence of value is disseminated and integrated throughout the health care system in a timely way will be a long-term and ongoing effort (and likely beyond the remit of framework developers alone), but it is vital to support the broader shift from volume to value of care and sustain investments currently being made in value assessment. Given their inherent limitations and variability in approaches, however, value assessments should serve as one of many tools to aid decision making.
Next Steps In this article, we put forth some common guiding principles— reflective of the diversity of stakeholder interests, needs, and values—to support high-quality value assessment at present and in the future in the United States. Our assessment has some limitations worth noting. We consider this exercise to be a first attempt at developing a set of guiding principles for value assessment from a multistakeholder perspective. This work is limited by the relatively small group of stakeholders involved. In addition, it did not use formal methods of consensus-building (e.g., Delphi or the snow card technique). In support of the key principles and to address the limitations of our analysis, we propose a number of potential next steps. First, current debates around the frameworks highlight the need for additional research and multistakeholder dialogue on defining and measuring value. Such efforts should also strive to identify convergences and differences within stakeholder groups, such as patient groups affected by different diseases. A better understanding of how different stakeholders define and operationalize value can help ensure that value frameworks reflect the
appropriate end points and attain their purported objectives. In addition, it will be important to better understand how target end users (e.g., patients and payers) make trade-offs between different elements of value. For example, how do patients consider mortality and/or morbidity against other considerations, such as value of hope, severity of disease, or convenience of treatment. Ideally, value frameworks will allow for elements of value to be weighed according to the preferences of different end users. Second, the proposed principles should be further refined and vetted to ensure they adequately capture the diversity of perspectives, content, and specificity to effectively guide developers and users. There are considerable convergences between our proposed principles and those recently published by industry and patient groups (e.g., National Pharmaceutical Council, Pharmaceutical Research and Manufacturers of America, and National Health Council), which signal the potential for stakeholder consensus on a common set of elements or practices that all value assessment frameworks can adopt to ensure assessments produced are relevant, methodologically robust, and of high quality. Third, given the divergent and, to some degree, uncertain methodological standards applied by existing frameworks, a consortium of experts and relevant stakeholders could be established to gain consensus on value assessment methodological and procedural standards, such as whether cost-utility analysis, multicriteria decision analysis, or some revision or combination of these approaches might be the best methodological foundation for value assessment. The new initiative on US value assessment frameworks by the International Society for Pharmacoeconomics and Outcomes Research is a commendable step in this direction [23]. Although a methodological foundation may indeed be helpful, identifying or constructing a single underlying theory or approach may be challenging (and potentially limiting) to account for different perspectives and decision contexts. Additional focus should be placed on standards regarding use of patient-related and real-world data (and related research networks, such as PCORnet and ASCO’s CancerLinq), given their lack of inclusion in current frameworks. Fourth, further investment on developing multistakeholder consensus on the implementation of value assessment framework is needed. This could include a series of meetings or workshops on implementation strategies and potential barriers or implications (e.g., resources and restrictions on access to care) to uptake, a needs assessment of resources and expertise required to foster and sustain high-quality value assessment, and methods education and training modules for physicians, patients, payers, and researchers. In addition, although the use of value assessment frameworks to inform shared treatment decision making, clinical guidelines, and pricing and coverage determinations has been explored, other areas of potential application, such as value-based or alternative payment models for drugs and devices, should be considered and potentially piloted. Finally, the impact of value assessment frameworks themselves should be evaluated. Do intended users adopt the framework(s)? Are their stated purpose and aims achieved (e.g., shared decision making)? Do they result in agreed-upon measures, such as reduction in overall cost or improvements in health outcomes? Have the frameworks inappropriately impacted patients’ access to care? Do they enhance transparency and accountability of decision making? Investments in value assessment will be worthwhile only if it benefits the health care system. Attaining better value in health care is a widely shared goal among stakeholders. New value assessment frameworks offer considerable potential to better understand the value of new health interventions and services and improve evidence-based decision making throughout the system. Although much work
VALUE IN HEALTH 20 (2017) 299–307
remains, with continued collaboration and commitment across stakeholders, we can make meaningful progress toward meeting these aims. Source of financial support: No financial support was received for this article. R EF E R EN CE S
[13]
[14]
[15] [1] Anderson JL, Heidenreich PA, Barnett PG, et al. ACC/AHA statement on cost/value methodology in clinical practice guidelines and performance measures. J Am Coll Cardiol 2014;63:2304–22. [2] Schnipper LE, Davidson NE, Wollins DS, et al. American Society of Clinical Oncology statement: a conceptual framework to assess the value of cancer treatment options. J Clin Oncol 2015;33:2563–77. [3] Institute for Clinical and Economic Review. ICER value assessment framework. Available from: https://icer-review.org/methodology/ icers-methods/icer-value-assessment-framework/. [Accessed July 29, 2016]. [4] Memorial Sloan Kettering Cancer Center. DrugAbacus methods. Available from: http://www.drugabacus.org/drug-abacus/methods/. [Accessed August 3, 2016]. [5] National Comprehensive Cancer Network. NCCN clinical practice guidelines in oncology (NCCN Guideliness) with NCCN Evidence Blocks™. Available from: https://www.nccn.org/evidenceblocks/. [Accessed August 3, 2016]. [6] Swanick M, Strang D. Third party drug assessors: influential new voices in the value debate. Available from: http://www.pwc.com/us/en/ health-industries/health-research-institute/publications/ third-party-drug-assessors.html. [Accessed August 3, 2016]. [7] Westrich K. Current landscape: value assessment frameworks. Available from: http://www.npcnow.org/system/files/research/ download/npc-current-landscape-value-assessment-frameworks-final. pdf/. [Accessed July 15, 2016]. [8] Angelis A, Kanavos P. Critique of the American Society of Clinical Oncology value assessment framework for cancer treatments: putting methodologic robustness first. J Clin Oncol 2016;34:2935–6. [9] Basch E. Toward a patient-centered value framework in oncology. JAMA 2016;315:2073–4. [10] Neumann PJ, Cohen JT. Measuring the value of prescription drugs. N Engl J Med 2015;373:2595–7. [11] Schnipper LE, Bastian A. New frameworks to assess value of cancer care: strengths and limitations. Oncologist 2016;21:654–8. [12] Perrone M. Breakthrough cholesterol drugs fizzle amid price pushback. Available from: http://bigstory.ap.org/article/
[16]
[17]
[18]
[19]
[20]
[21] [22]
[23]
307
a15448a756dc4bfc9bc78174b14ccb6f/breakthrough-cholesteroldrugs-fizzle-amid-price-pushback. [Accessed August 13, 2016]. Caffrey MK. For now, PBMs just say no to high-cost PCSK9 inhibitors. Available from: http://www.ajmc.com/journals/ evidence-based-diabetes-management/2015/september-2015/ for-now-pbms–just-say-no-to–high-cost-pcsk9–inhibitors. [Accessed August 13, 2016]. National Health Council. The patient voice in value: the National Health Council patient-centered value model rubric. Available from: http://www.nationalhealthcouncil.org/sites/default/files/Value-Rubric. pdf. [Accessed July 10, 2016]. Longacre ML, Charap ES, Buzaglo JS, et al. Defining value in oncology: perspectives from patients with metastatic breast cancer. Available from: http://www.cancersupportcommunity.org/sites/default/files/ uploads/our-research/presentations/access-to-care/ avbcc_value_poster_.pdf/. [Accessed July 15, 2016]. Schnipper LE, Davidson NE, Wollins DS, et al. Updating the American Society of Clinical Oncology value framework: revisions and reflections in response to comments received. J Clin Oncol 2016;34:2925–34. National Comprehensive Cancer Network. NCCN guidelines for patients. Multiple myeloma, version 1.2016. Available from: https:// www.nccn.org/patients/guidelines/myeloma/files/assets/basic-html/ page-1.html. [Accessed August 10, 2016]. Institute for Clinical and Economic Review. Multiple myeloma: final evidence report and meeting summary. Available from: https:// icer-review.org/material/mm-final-report/. [Accessed August 10, 2016]. Seidman J, Choe SH. Most health plans do not use existing value frameworks to make coverage decisions. Available from: http://avalere. com/expertise/life-sciences/insights/ most-health-plans-do-not-use-value-frameworks-to-make-coveragedecisions/. [Accessed August 3, 2016]. Stroup DF, Berlin JA, Morton SC, et al. Meta-analysis of observational studies in epidemiology: a proposal for reporting. Meta-analysis of Observational Studies in Epidemiology (MOOSE) group. JAMA 2000;283:2008–12. Ades A, Caldwell DM, Reken S, et al. Evidence synthesis for decision making: a reviewer’s checklist. Med Decis Making 2013;33:679–91. McKenna C, Soares M, Claxton K, et al. Unifying research and reimbursement decisions: case studies demonstrating the sequence of assessment and judgments required. Value Health 2015;18:865–75. International Society for Pharmacoeconomics and Outcomes Research. Initiative on US value assessment frameworks. Available from: http:// www.ispor.org/ValueAssessmentFrameworks/Index. [Accessed August 20, 2016].