- Email: [email protected]
A Proposal for Improving Evidence Generation in Nephrology
Special Article A Proposal for Improving Evidence Generation in Nephrology David C. Mendelssohn, MD, FRCPC,1 and Braden J. Manns, MD, MSc, FRCPC2,3,4 The existing framework for the evaluation and reimbursement of new drugs and other treatments in nephrology does not appear to be providing the optimal incentives to advance science and improve outcomes for patients with end-stage renal disease. This article examines reasons for this malalignment and how the field of nephrology is affected, then proceeds to show that alternative evaluation paradigms are being developed by the health technology assessment community. These alternative evaluative frameworks are complementary to traditional evidence-based medicine and comparative evaluative research and may be worthy of adaptation by those interested in seeing more pivotal kidney research (both quantity and quality) and more efficient evaluation of new therapies for patients with chronic kidney failure treated using long-term dialysis therapy. Am J Kidney Dis. 58(1):13-18. © 2011 by the National Kidney Foundation, Inc. INDEX WORDS: Comparative effectiveness research; coverage with evidence development; evidence-based medicine; field evaluation.
O
n completion of phase 3 randomized trials showing preliminary safety and effectiveness, drugs may be given a license to be marketed and sold for patient use. However, given the potential use and financial impact, many countries require further data about medication effectiveness and cost before public funding is made available for a product newly approved by a regulatory agency. This postregulatory process increasingly involves the application of strictly applied evidence-based medicine and comparative evaluative research, with outcome assessment that usually includes patient survival, clinical outcomes, and cost-effectiveness (Fig 1). Given the limited evidence base, success in achieving this postregulatory evaluation threshold has been exceedingly difficult for many drugs and devices within the field of nephrology. In the first part of this article, we discuss the multiple interacting factors that prevent nephrology from exceeding the threshold for public funding of new drugs and therapies. We then introduce new methods of postregulatory evaluation that could be applied to nephrology. These “conditionally funded field evaluations” include funding while formal collection of additional patient data continues. These methods are being used internationally and are viewed by the health technology evaluation community as a way to encourage innovation, bring promising new therapies to patients who might benefit earlier, and provide rigorous scientific evaluation of the benefits, risks, and costs. Our purpose is to introduce this new paradigm for thoughtful consideration by the renal community. Although we focus on chronic kidney failure treated using long-term dialysis therapy, the same considerations apply to patients treated using transplant and those with earlier stages of chronic kidney disease.
Am J Kidney Dis. 2011;58(1):13-18
FRAMING THE DILEMMA The discipline of nephrology finds itself facing an increasingly difficult paradox as it approaches the milestone of a half-century of maintenance dialysis therapy. The first 20 years of maintenance dialysis was characterized by rapid innovation and dramatic changes in this life-sustaining therapy. Unfortunately, innovation has reached a plateau during the past 30 years, and dialysis remains an inadequate therapy, with patients experiencing mortality rates similar to those in individuals with advanced colon cancer and facing significantly impaired quality of life. These outcomes have not substantially improved during the past 20 years. Moreover, although the prevalence of end-stage renal disease (ESRD) is ⬍0.01%, 1%-2% of the health care budget is spent on ESRD care.1,2 As caregivers and physicians for these vulnerable patients, nephrologists have an obligation to lead, support, and/or assist with research that sheds light on new approaches that have the potential to improve survival and quality of life. However, fewer randomized controlled clinical trials are performed in nephrology than in any other discipline of internal medicine.3 Furthermore, the few hard-outcome nephrology cliniFrom the 1Humber River Regional Hospital and University of Toronto, Weston; and Departments of 2Medicine and 3Community Health Sciences and 4Libin Cardiovascular Institute, University of Calgary, Calgary, Alberta, Canada. Received October 26, 2010. Accepted in revised form February 22, 2011. Originally published online April 4, 2011. Address correspondence to David C. Mendelssohn, MD, FRCPC, Humber River Regional Hospital, 200 Church St, Rm 2024, Weston, Ontario, Canada M9N 1N8. E-mail: dmendelssohn@ hrrh.on.ca © 2011 by the National Kidney Foundation, Inc. 0272-6386/$36.00 doi:10.1053/j.ajkd.2011.02.377 13
Mendelssohn and Manns
Figure 1. The traditional licensing and postregulatory process for evaluation of a new drug. Traditional postregulatory health technology assessments involve strictly applied evidencebased medicine, comparative evaluative research, and assessment of cost-effectiveness, among other factors. We propose field evaluations as an alternate pathway that might be relevant in nephrology.
cal trials that have been published often are either negative4-8 or ambiguous.9 However, nephrology has many very rich observational databases, which can be hypothesis generating and guide care. This occasionally has led to nephrology guideline targets that subsequently were shown to be incorrect when tested in formal randomized trials.10,11 Clinical research in nephrology is challenging for several reasons. Hard-outcome research with survival as the primary outcome in this small population requires multicenter studies with high cost. Given that cardiovascular disease is the major cause of death in these patients, identifying and modifying risk factors for cardiovascular disease has been a priority for the community. However, dialysis patients carry a burden of traditional risk factors and also nontraditional uremia- or dialysis-related risk factors. Single interventions that work very well in nondialysis patients have not been effective in dialysis patients. One example is therapy directed at lipid level control using statins.6 Hence, one problem is that interventions successful in general populations generally should be retested in dialysis patients. A final issue is that there are no completely validated surrogate outcomes that can substitute for survival outcomes in ESRD to decrease study size, complexity, duration, and the high cost of the required trials.12 The net result of these challenges is that much of modern nephrology practice is opinion based, and considerable uncertainty exists about the best treatments for patients with chronic kidney disease, ESRD, and the multiple complications and comorbid conditions that coexist in this patient group.
THE CURRENT POSTREGULATORY EVALUATION FRAMEWORK In the past 5-10 years, payers have adopted an increasingly more stringent approach when assessing new drugs and other therapies for coverage.13 Licensing relates to satisfying only that a new treatment is safe and effective, often compared with placebo. However, reimbursement of new therapies, particularly 14
drugs, requires satisfying the rules of strictly applied evidence-based medicine (with such hard outcomes as mortality and/or hospitalization rates) and demonstration of cost-effectiveness. In an ideal world with ample public research funds available to conclusively answer all modern medicine’s critical questions, such an approach would be unassailable. However, there is a concern that in the imperfect world the nephrology community finds itself in, consideration of these inputs alone is too onerous. It is the perception of the authors that industry already is becoming increasingly reluctant to engage in the risky development of new nephrology drugs and treatments. There is not adequate, publicly available, peerreviewed grant money to fund all the required research in nephrology and all other disciplines. Dependence on industry investments in nephrology research will continue indefinitely. Considering that Antihypertensive and Lipid-Lowering Treatment to Prevent Heart Attack Trial (ALLHAT)14 cost $180 million and Dialysis Clinical Outcomes Revisited (DCOR)9 cost ⬃$15 million, it is reasonable to suggest that a welldone nephrology study might cost a sponsor $25-$50 million for one randomized trial with survival as the primary outcome. Of course, costs incurred for the development of new molecules, animal testing, human testing, and licensing are separate from this postregulatory expense. If satisfying the new threshold is very difficult and the time from licensing to funding gets extended by years, how will industry assess its risk to reward ratio in nephrology? The current paradigm of drug and therapy evaluation may cause an unintended migration of potential industry partners to other easier areas in which profitability is more likely.
TOWARD A NEW FRAMEWORK To bring the renal community together to improve outcomes for patients with kidney disease first requires agreement about the priority goals for the discipline. Surely we all agree that we need to create an environment that fosters advances in chronic kidney disease care to a greater extent than we presently experience. Second, we need to stimulate renal research in general and randomized controlled clinical trials in particular, such that evidence-based practice in nephrology is enhanced. Third, we need to create conditions that encourage potential industry partners to join nephrology in this quest. Finally, we have to satisfy payers and governments that we are concerned about equity from discipline to discipline and are reasonable in terms of access and cost tradeoffs. Modern society faces 2 conflicting but simultaneous demands. Society desires both rapid access to innovation and high-quality evidence about real-world effecAm J Kidney Dis. 2011;58(1):13-18
Evidence Generation
tiveness. The health technology assessment community recently has begun to explore other methods of postregulatory evaluation. These newer models, although largely unknown within the nephrology community, have profound potential importance to nephrology and, if adapted to our needs, could be very helpful in stimulating evidence generation. Consider that in the United States, the Centers for Medicare & Medicaid Services (CMS) recently has introduced a new program called “coverage with evidence development” (ie, “CED”) that allows for conditional funding while formal collection of additional patient data continues.15 The intent is to qualify services or items, in rare instances in which evidence is preliminary, for which safety is thought to be reasonable, there is high potential to provide significant benefits, and there are significant barriers to conducting clinical trials. Reports of these evaluations have been published,16,17 and given the promotion of comparative evaluative research with $1.1 billion contained within the Obama stimulus package, these kinds of evaluation seem likely to grow in importance in the United States.18 Another highly innovative approach to partnerships that stimulate evidence generation are the recently completed Frequent Hemodialysis Network randomized trials that were funded jointly by the National Institutes of Health National Institute of Diabetes and Digestive and Kidney Diseases, CMS, Fresenius Medical Care Canada, Renal Research Institute, and Satellite Health Care.19 Similarly, programs involving risksharing practices and conditional pricing have been attempted in the United Kingdom and other European jurisdictions.20-22 Ontario, Canada, is a jurisdiction recognized as having a lead role internationally in developing and applying these evolving new methods.23,24 The Ministry of Health and Long Term Care provides funding to the Ontario Health Technology Advisory Committee, which then can support conditionally funded field evaluations at arm’s length from government. External content expertise is sought, and strong collaborative partnerships exist with local university-based health economic groups and health care administrative database epidemiologists. Multiple study designs are considered, depending on the specific situation. To date, 19 field evaluations have been undertaken, most have been successfully completed, and most advisory committee recommendations have been implemented by the Ontario government. One example is an observational province-wide field evaluation trial of positron emission tomography, which resulted in public funding for 8 oncology and 1 cardiac indications.25 Am J Kidney Dis. 2011;58(1):13-18
CONDITIONALLY FUNDED FIELD EVALUATIONS IN NEPHROLOGY The challenge for the nephrology community is to establish that our discipline, as argued, has exceptional circumstances such that the development and funding of new drugs and devices frequently will require an alternate paradigm. To put this in its most positive light, we propose the term evidence generation program (EGP) to describe a novel framework that could be put into place after initial acceptance of a drug or device by a national regulator, but before its formal consideration for reimbursement, if applicable. Although we propose several unique aspects to EGP, there may be existing models in place in some jurisdictions that might be amenable to partnerships with the nephrology community or different models might evolve other than the one we propose, depending on local circumstances. Recognition of the extreme challenges faced by nephrology and the need for a new framework is the key, whereas the specifics of the plan outlined next are less important. The framework proposed includes several components working in synergy to implement the EGP model (Fig 2). The first step would be creation of a program eligibility committee that would be composed of multiple qualified stakeholders. Their task would be to develop the criteria by which an item or service (drug, biologic, therapy, etc) would be eligible for assessment within the EGP program. When this task is completed, the committee then would assess applications for inclusion in the program. After acceptance of a drug or device into the EGP program, the next step is consideration of the required research by an evidence generation committee. This multistakeholder committee first would summarize existing data and then work with the sponsor to define
Figure 2. Flow chart shows the process of a hypothetical evidence generation program. Abbreviations: ⫺ve, negative outcome; ⫹ve, positive outcome. 15
Mendelssohn and Manns
the key fundamental questions that must be answered. This would include consideration about the required study design and time frame and might include one or several studies that may be observational (registry based) and/or randomized trials. The expectation would be that in a defined period, the research would unambiguously produce the data required by decision makers to enable a reimbursement decision. Implementation of the program would require the participation of clinical and academic nephrologists. They would be given the option of voluntarily opting in to the EGP or opting out. Those who opt in would gain early access to promising new items and services, but conditional on enrolling all their eligible patients into the ongoing studies. Patients who are ineligible for the formal studies or who refuse to participate might still access the trial item or service if they agree to allow their data to be entered into an observational database or registry. Patients who refuse to participate in the trials or registries remain outside the EGP and would have access to only publicly available items or services according to their insurance coverage or ability to make out-of-pocket payments. Nephrologists who opt out of the EGP would have restricted access to promising new drugs and their patients would be eligible for funds only as routinely available in that jurisdiction. The ethics of CED recently has been considered.26 Infrastructure to track patients and outcomes would be provided to opted-in physicians at no cost to them. The capital and operating cost of the research and cost of items or services would be negotiated and shared by the sponsor and payers. On completion of the EGP, the data should enable evidence-based decisions about expansion of the funding to all eligible patients, or conversely, elimination of funding for all patients. Of course, it is possible that despite the best of intentions and efforts, the evidence remains ambiguous and either an extension or revision of the EGP or an intermediate funding decision might apply. A final step after studies that support use of a new drug might consist of determination of an acceptable price by a therapy pricing committee. This multistakeholder group would review all direct and indirect costs of developing a therapy. They then would apply an agreed-on and validated method to set a fair price point that balances industries’ requirement for reasonable profit with payers’ requirement that unjustified or inflated costs are not borne by society. This step might not be required in jurisdictions in which a fair and effective process to set price already is in place. 16
AN EXAMPLE To illustrate how the EGP might work, we highlight the example of the newly developed biocompatible bicarbonate-based peritoneal dialysis (PD) solutions. These are licensed in many jurisdictions, but have not been adopted widely because of a steep price premium and uncertain effect on important long-term outcomes.27,28 However, in theory, preliminary reports suggest that these solutions might lead to better outcomes and may increase the desire of patients and health care workers to use home-based PD therapy.29 However, research to date typically has been single center, focusing only on measurement of their impact on nonclinical outcomes. To our knowledge, there are no plans for a long-term, hard-outcome, multicenter, randomized, controlled, clinical trial. In part, this relates to the high cost of attempting such a trial and the risk that the trial will show negative or ambiguous results. An industry sponsor might approach the EGP program eligibility committee for consideration of inclusion of bicarbonate-based PD solutions in the EGP. Given the low likelihood of a clinical trial under traditional paradigms of funding and evaluation, but given the evidence to date (which includes preliminary data for safety and effectiveness based on nonvalidated surrogate end points), the potential that bicarbonate-based PD solutions might be a major therapeutic advance, and the economic implications, the program eligibility committee might agree that bicarbonatebased PD solutions qualify for the EGP. A multistakeholder committee with industry and payer representatives, PD experts, methodologists, and others then would devise the best way to obtain the data required to determine whether these solutions are superior to standard PD solutions. As a preliminary determination, let us assume that an observational registry is deemed unlikely to answer the fundamental questions. Based on that, a multicenter randomized controlled trial would be designed and power calculations would be performed with outcomes that include technique survival as the primary outcome and patient survival, rates of peritonitis and encapsulating peritoneal sclerosis, and cost-effectiveness as secondary outcomes. Funding of this trial would be shared by the industry sponsor and payers. Successful completion of this randomized controlled trial would provide unambiguous data that would enable health care decision makers to decide on public funding of bicarbonate-based PD solutions. Other examples of current new therapies for which field evaluations might be considered are non–calciumbased phosphate binders, calcimimetics, and daily hemodialysis therapy. Am J Kidney Dis. 2011;58(1):13-18
Evidence Generation
CHALLENGES There are 3 key potential problems with EGP that we anticipate and have considered. First, nephrologists and/or patients might opt in, but then not really promote the trials. Clearly, an audit of compliance and comparisons with peers might be required to identify these physicians and either remedy the recruitment effort or disqualify that practitioner. Second, patients may migrate from opted-out physicians to opted-in ones. We do not see this as a problem because we hope that all nephrologists and all patients would see the value of participation, and with more physicians and more patients, trials could be completed more quickly. Finally, if the EGP was implemented in only one or a few jurisdictions that wished to study more than one item or service, those payers or governments would take on an undue cost burden. Given that the high-quality research produced by the EGP would be of international importance, the EGP must be internationalized and coordinated to be equitable and successful in participating jurisdictions. SUMMARY These principles and concepts concerning a bold new way to generate evidence in ESRD using conditionally funded field evaluations should be regarded by the community as a way to catalyze discussions about how to stimulate innovation and test new interventions that might improve outcomes for dialysis patients. It seems likely that field evaluations would allow patients earlier access to promising therapies (even if within the context of definitive randomized trials) and positively stimulate the evolution of evidence-based nephrology practice and outcomes. For nephrologists, it would create a platform to more rapidly answer fundamental research questions. For industry, field evaluations may offset some of the prohibitive costs and risks associated with developing nephrology products and might make the nephrology research and “marketplace” more inviting. For payers and governments, it offsets some of the cost and risk of allowing early (and possibly premature) access to promising therapies, delivers the evidence they need to make fair and informed coverage decisions, and possibly sets reasonable prices. Most importantly, field evaluations will stimulate an environment that promotes renal care advances. Implementation of conditional funding with field evaluations may need to be altered to align with local resources and expertise and therefore may vary from jurisdiction to jurisdiction. Nonetheless, the principles of the proposed framework are adaptable, feasible, and may be inevitable if we wish both prompt introduction and rigorous evaluation of many more promising new therapies for dialysis patients in the Am J Kidney Dis. 2011;58(1):13-18
future. Further discussion is required by health care professionals caring for dialysis patients, potential industry sponsors, and public payers.
ACKNOWLEDGEMENTS The authors acknowledge Drs Fred Finkelstein, Adeera Levin, and Marcello Tonelli, who reviewed early drafts and provided excellent suggestions and advice. Support: None. Financial Disclosure: The authors declare that they have no relevant financial interests.
REFERENCES 1. Zelmer JL. The economic burden of end-stage renal disease in Canada. Kidney Int. 2007;72(9):1122-1129. 2. Dor A, Pauly MV, Eichleay MA, Held PJ. End-stage renal disease and economic incentives: the International Study of Health Care Organization and Financing (ISHCOF). Int J Health Care Finance Econ. 2007;7(2-3):73-111. 3. Strippoli GF, Craig JC, Schena FP. The number, quality, and coverage of randomized controlled trials in nephrology. J Am Soc Nephrol. 2004;15(2):411-419. 4. Pfeffer MA, Burdmann EA, Chen CY, et al. A trial of darbepoetin alfa in type 2 diabetes and chronic kidney disease. N Engl J Med. 2009;361(21):2019-2032. 5. Cheung AK, Levin NW, Greene T, et al. Effects of high-flux hemodialysis on clinical outcomes: results of the HEMO Study. J Am Soc Nephrol. 2003;14(12):3251-3263. 6. Fellstrom BC, Jardine AG, Schmieder RE, et al. Rosuvastatin and cardiovascular events in patients undergoing hemodialysis. N Engl J Med. 2009;360(14):1395-1407. 7. Paniagua R, Amato D, Vonesh E, et al. Effects of increased peritoneal clearances on mortality rates in peritoneal dialysis: ADEMEX, a prospective, randomized, controlled trial. J Am Soc Nephrol. 2002;13(5):1307-1320. 8. Besarab A, Bolton WK, Browne JK, et al. The effects of normal as compared with low hematocrit values in patients with cardiac disease who are receiving hemodialysis and epoetin. N Engl J Med. 1998;339(9):584-590. 9. Suki WN, Zabaneh R, Cangiano JL, et al. Effects of sevelamer and calcium-based phosphate binders on mortality in hemodialysis patients. Kidney Int. 2007;72(9):1130-1137. 10. National Kidney Foundation. KDOQI Clinical Practice Guidelines and Clinical Practice Recommendations for Anemia in Chronic Kidney Disease. Am J Kidney Dis. 2006;47(5)(suppl 3):S11-145. 11. National Kidney Foundation. KDOQI Clinical Practice Guideline and Clinical Practice Recommendations for Anemia in Chronic Kidney Disease: 2007 update of hemoglobin target. Am J Kidney Dis. 2007;50(3):471-530. 12. Manns B, Owen WF Jr, Winkelmayer WC, Devereaux PJ, Tonelli M. Surrogate markers in clinical studies: problems solved or created? Am J Kidney Dis. 2006;48(1):159-166. 13. Tierney M, Manns B. Optimizing the use of prescription drugs in Canada through the Common Drug Review. CMAJ. 2008;178(4):432-435. 14. The ALLHAT Officers and Coordinators for the ALLHAT Collaborative Research Group. Major outcomes in high-risk hypertensive patients randomized to angiotensin-converting enzyme inhibitor or calcium channel blocker vs diuretic: the Antihypertensive and Lipid-Lowering Treatment to Prevent Heart Attack Trial (ALLHAT). JAMA. 2002;288(23):2981-2997. 15. Centers for Medicare and Medicaid. https://www.cms.gov/ CoverageGenInfo/03_CED.asp. Accessed March 12, 2011. 17
Mendelssohn and Manns 16. Lindsay MJ, Siegel BA, Tunis SR, et al. The National Oncologic PET Registry: expanded Medicare coverage for PET under coverage with evidence development. AJR Am J Roentgenol. 2007;188(4):1109-1113. 17. Tunis SR, Pearson SD. Coverage options for promising technologies: Medicare’s ‘coverage with evidence development’. Health Aff (Millwood). 2006;25(5):1218-1230. 18. Neumann PJ, Tunis SR. Medicare and medical technology— the growing demand for relevant outcomes. N Engl J Med. 2010; 362(5):377-379. 19. Suri RS, Garg AX, Chertow GM, et al. Frequent Hemodialysis Network (FHN) randomized trials: study design. Kidney Int. 2007;71(4):349-359. 20. McCabe C, Bergmann L, Bosanquet N, et al. Market and patient access to new oncology products in Europe: a current, multidisciplinary perspective. Ann Oncol. 2009;20(3):403-412. 21. Chapman S, Reeve E, Rajaratnam G, Neary R. Setting up an outcomes guarantee for pharmaceuticals: new approach to risk sharing in primary care. BMJ. 2003;326(7391):707-709. 22. Boggild M, Palace J, Barton P, et al. Multiple sclerosis risk sharing scheme: two year results of clinical cohort study with historical comparator. BMJ. 2009;339:b4677. 23. Whicher DM, Chalkidou K, Dhalla IA, Levin L, Tunis S. Comparative effectiveness research in Ontario, Canada: producing
18
relevant and timely information for health care decision makers. Milbank Q. 2009;87(3):585-606. 24. Levin L, Goeree R, Sikich N, et al. Establishing a comprehensive continuum from an evidentiary base to policy development for health technologies: the Ontario experience. Int J Technol Assess Health Care. 2007;23(3):299-309. 25. Evans WK, Laupacis A, Gulenchyn KY, Levin L, Levine M. Evidence-based approach to the introduction of positron emission tomography in Ontario, Canada. J Clin Oncol. 2009;27(33): 5607-5613. 26. Pearson SD, Miller FG, Emanuel EJ. Medicare’s requirement for research participation as a condition of coverage: is it ethical? JAMA. 2006;296(8):988-991. 27. Bargman JM. Peritoneal dialysis solutions and patient survival: does wishing make it so? Nephrol Dial Transplant. 2006; 21(10):2684-2686. 28. Bargman JM. Slouching towards Bethlehem: the beast of biocompatibility. Nephrol Dial Transplant. 2010;25(7):20502051. 29. Haag-Weber M, Kramer R, Haake R, et al. Low-GDP fluid (Gambrosol trio) attenuates decline of residual renal function in PD patients: a prospective randomized study. Nephrol Dial Transplant. 2010;25(7):2288-2296.
Am J Kidney Dis. 2011;58(1):13-18
O
n completion of phase 3 randomized trials showing preliminary safety and effectiveness, drugs may be given a license to be marketed and sold for patient use. However, given the potential use and financial impact, many countries require further data about medication effectiveness and cost before public funding is made available for a product newly approved by a regulatory agency. This postregulatory process increasingly involves the application of strictly applied evidence-based medicine and comparative evaluative research, with outcome assessment that usually includes patient survival, clinical outcomes, and cost-effectiveness (Fig 1). Given the limited evidence base, success in achieving this postregulatory evaluation threshold has been exceedingly difficult for many drugs and devices within the field of nephrology. In the first part of this article, we discuss the multiple interacting factors that prevent nephrology from exceeding the threshold for public funding of new drugs and therapies. We then introduce new methods of postregulatory evaluation that could be applied to nephrology. These “conditionally funded field evaluations” include funding while formal collection of additional patient data continues. These methods are being used internationally and are viewed by the health technology evaluation community as a way to encourage innovation, bring promising new therapies to patients who might benefit earlier, and provide rigorous scientific evaluation of the benefits, risks, and costs. Our purpose is to introduce this new paradigm for thoughtful consideration by the renal community. Although we focus on chronic kidney failure treated using long-term dialysis therapy, the same considerations apply to patients treated using transplant and those with earlier stages of chronic kidney disease.
Am J Kidney Dis. 2011;58(1):13-18
FRAMING THE DILEMMA The discipline of nephrology finds itself facing an increasingly difficult paradox as it approaches the milestone of a half-century of maintenance dialysis therapy. The first 20 years of maintenance dialysis was characterized by rapid innovation and dramatic changes in this life-sustaining therapy. Unfortunately, innovation has reached a plateau during the past 30 years, and dialysis remains an inadequate therapy, with patients experiencing mortality rates similar to those in individuals with advanced colon cancer and facing significantly impaired quality of life. These outcomes have not substantially improved during the past 20 years. Moreover, although the prevalence of end-stage renal disease (ESRD) is ⬍0.01%, 1%-2% of the health care budget is spent on ESRD care.1,2 As caregivers and physicians for these vulnerable patients, nephrologists have an obligation to lead, support, and/or assist with research that sheds light on new approaches that have the potential to improve survival and quality of life. However, fewer randomized controlled clinical trials are performed in nephrology than in any other discipline of internal medicine.3 Furthermore, the few hard-outcome nephrology cliniFrom the 1Humber River Regional Hospital and University of Toronto, Weston; and Departments of 2Medicine and 3Community Health Sciences and 4Libin Cardiovascular Institute, University of Calgary, Calgary, Alberta, Canada. Received October 26, 2010. Accepted in revised form February 22, 2011. Originally published online April 4, 2011. Address correspondence to David C. Mendelssohn, MD, FRCPC, Humber River Regional Hospital, 200 Church St, Rm 2024, Weston, Ontario, Canada M9N 1N8. E-mail: dmendelssohn@ hrrh.on.ca © 2011 by the National Kidney Foundation, Inc. 0272-6386/$36.00 doi:10.1053/j.ajkd.2011.02.377 13
Mendelssohn and Manns
Figure 1. The traditional licensing and postregulatory process for evaluation of a new drug. Traditional postregulatory health technology assessments involve strictly applied evidencebased medicine, comparative evaluative research, and assessment of cost-effectiveness, among other factors. We propose field evaluations as an alternate pathway that might be relevant in nephrology.
cal trials that have been published often are either negative4-8 or ambiguous.9 However, nephrology has many very rich observational databases, which can be hypothesis generating and guide care. This occasionally has led to nephrology guideline targets that subsequently were shown to be incorrect when tested in formal randomized trials.10,11 Clinical research in nephrology is challenging for several reasons. Hard-outcome research with survival as the primary outcome in this small population requires multicenter studies with high cost. Given that cardiovascular disease is the major cause of death in these patients, identifying and modifying risk factors for cardiovascular disease has been a priority for the community. However, dialysis patients carry a burden of traditional risk factors and also nontraditional uremia- or dialysis-related risk factors. Single interventions that work very well in nondialysis patients have not been effective in dialysis patients. One example is therapy directed at lipid level control using statins.6 Hence, one problem is that interventions successful in general populations generally should be retested in dialysis patients. A final issue is that there are no completely validated surrogate outcomes that can substitute for survival outcomes in ESRD to decrease study size, complexity, duration, and the high cost of the required trials.12 The net result of these challenges is that much of modern nephrology practice is opinion based, and considerable uncertainty exists about the best treatments for patients with chronic kidney disease, ESRD, and the multiple complications and comorbid conditions that coexist in this patient group.
THE CURRENT POSTREGULATORY EVALUATION FRAMEWORK In the past 5-10 years, payers have adopted an increasingly more stringent approach when assessing new drugs and other therapies for coverage.13 Licensing relates to satisfying only that a new treatment is safe and effective, often compared with placebo. However, reimbursement of new therapies, particularly 14
drugs, requires satisfying the rules of strictly applied evidence-based medicine (with such hard outcomes as mortality and/or hospitalization rates) and demonstration of cost-effectiveness. In an ideal world with ample public research funds available to conclusively answer all modern medicine’s critical questions, such an approach would be unassailable. However, there is a concern that in the imperfect world the nephrology community finds itself in, consideration of these inputs alone is too onerous. It is the perception of the authors that industry already is becoming increasingly reluctant to engage in the risky development of new nephrology drugs and treatments. There is not adequate, publicly available, peerreviewed grant money to fund all the required research in nephrology and all other disciplines. Dependence on industry investments in nephrology research will continue indefinitely. Considering that Antihypertensive and Lipid-Lowering Treatment to Prevent Heart Attack Trial (ALLHAT)14 cost $180 million and Dialysis Clinical Outcomes Revisited (DCOR)9 cost ⬃$15 million, it is reasonable to suggest that a welldone nephrology study might cost a sponsor $25-$50 million for one randomized trial with survival as the primary outcome. Of course, costs incurred for the development of new molecules, animal testing, human testing, and licensing are separate from this postregulatory expense. If satisfying the new threshold is very difficult and the time from licensing to funding gets extended by years, how will industry assess its risk to reward ratio in nephrology? The current paradigm of drug and therapy evaluation may cause an unintended migration of potential industry partners to other easier areas in which profitability is more likely.
TOWARD A NEW FRAMEWORK To bring the renal community together to improve outcomes for patients with kidney disease first requires agreement about the priority goals for the discipline. Surely we all agree that we need to create an environment that fosters advances in chronic kidney disease care to a greater extent than we presently experience. Second, we need to stimulate renal research in general and randomized controlled clinical trials in particular, such that evidence-based practice in nephrology is enhanced. Third, we need to create conditions that encourage potential industry partners to join nephrology in this quest. Finally, we have to satisfy payers and governments that we are concerned about equity from discipline to discipline and are reasonable in terms of access and cost tradeoffs. Modern society faces 2 conflicting but simultaneous demands. Society desires both rapid access to innovation and high-quality evidence about real-world effecAm J Kidney Dis. 2011;58(1):13-18
Evidence Generation
tiveness. The health technology assessment community recently has begun to explore other methods of postregulatory evaluation. These newer models, although largely unknown within the nephrology community, have profound potential importance to nephrology and, if adapted to our needs, could be very helpful in stimulating evidence generation. Consider that in the United States, the Centers for Medicare & Medicaid Services (CMS) recently has introduced a new program called “coverage with evidence development” (ie, “CED”) that allows for conditional funding while formal collection of additional patient data continues.15 The intent is to qualify services or items, in rare instances in which evidence is preliminary, for which safety is thought to be reasonable, there is high potential to provide significant benefits, and there are significant barriers to conducting clinical trials. Reports of these evaluations have been published,16,17 and given the promotion of comparative evaluative research with $1.1 billion contained within the Obama stimulus package, these kinds of evaluation seem likely to grow in importance in the United States.18 Another highly innovative approach to partnerships that stimulate evidence generation are the recently completed Frequent Hemodialysis Network randomized trials that were funded jointly by the National Institutes of Health National Institute of Diabetes and Digestive and Kidney Diseases, CMS, Fresenius Medical Care Canada, Renal Research Institute, and Satellite Health Care.19 Similarly, programs involving risksharing practices and conditional pricing have been attempted in the United Kingdom and other European jurisdictions.20-22 Ontario, Canada, is a jurisdiction recognized as having a lead role internationally in developing and applying these evolving new methods.23,24 The Ministry of Health and Long Term Care provides funding to the Ontario Health Technology Advisory Committee, which then can support conditionally funded field evaluations at arm’s length from government. External content expertise is sought, and strong collaborative partnerships exist with local university-based health economic groups and health care administrative database epidemiologists. Multiple study designs are considered, depending on the specific situation. To date, 19 field evaluations have been undertaken, most have been successfully completed, and most advisory committee recommendations have been implemented by the Ontario government. One example is an observational province-wide field evaluation trial of positron emission tomography, which resulted in public funding for 8 oncology and 1 cardiac indications.25 Am J Kidney Dis. 2011;58(1):13-18
CONDITIONALLY FUNDED FIELD EVALUATIONS IN NEPHROLOGY The challenge for the nephrology community is to establish that our discipline, as argued, has exceptional circumstances such that the development and funding of new drugs and devices frequently will require an alternate paradigm. To put this in its most positive light, we propose the term evidence generation program (EGP) to describe a novel framework that could be put into place after initial acceptance of a drug or device by a national regulator, but before its formal consideration for reimbursement, if applicable. Although we propose several unique aspects to EGP, there may be existing models in place in some jurisdictions that might be amenable to partnerships with the nephrology community or different models might evolve other than the one we propose, depending on local circumstances. Recognition of the extreme challenges faced by nephrology and the need for a new framework is the key, whereas the specifics of the plan outlined next are less important. The framework proposed includes several components working in synergy to implement the EGP model (Fig 2). The first step would be creation of a program eligibility committee that would be composed of multiple qualified stakeholders. Their task would be to develop the criteria by which an item or service (drug, biologic, therapy, etc) would be eligible for assessment within the EGP program. When this task is completed, the committee then would assess applications for inclusion in the program. After acceptance of a drug or device into the EGP program, the next step is consideration of the required research by an evidence generation committee. This multistakeholder committee first would summarize existing data and then work with the sponsor to define
Figure 2. Flow chart shows the process of a hypothetical evidence generation program. Abbreviations: ⫺ve, negative outcome; ⫹ve, positive outcome. 15
Mendelssohn and Manns
the key fundamental questions that must be answered. This would include consideration about the required study design and time frame and might include one or several studies that may be observational (registry based) and/or randomized trials. The expectation would be that in a defined period, the research would unambiguously produce the data required by decision makers to enable a reimbursement decision. Implementation of the program would require the participation of clinical and academic nephrologists. They would be given the option of voluntarily opting in to the EGP or opting out. Those who opt in would gain early access to promising new items and services, but conditional on enrolling all their eligible patients into the ongoing studies. Patients who are ineligible for the formal studies or who refuse to participate might still access the trial item or service if they agree to allow their data to be entered into an observational database or registry. Patients who refuse to participate in the trials or registries remain outside the EGP and would have access to only publicly available items or services according to their insurance coverage or ability to make out-of-pocket payments. Nephrologists who opt out of the EGP would have restricted access to promising new drugs and their patients would be eligible for funds only as routinely available in that jurisdiction. The ethics of CED recently has been considered.26 Infrastructure to track patients and outcomes would be provided to opted-in physicians at no cost to them. The capital and operating cost of the research and cost of items or services would be negotiated and shared by the sponsor and payers. On completion of the EGP, the data should enable evidence-based decisions about expansion of the funding to all eligible patients, or conversely, elimination of funding for all patients. Of course, it is possible that despite the best of intentions and efforts, the evidence remains ambiguous and either an extension or revision of the EGP or an intermediate funding decision might apply. A final step after studies that support use of a new drug might consist of determination of an acceptable price by a therapy pricing committee. This multistakeholder group would review all direct and indirect costs of developing a therapy. They then would apply an agreed-on and validated method to set a fair price point that balances industries’ requirement for reasonable profit with payers’ requirement that unjustified or inflated costs are not borne by society. This step might not be required in jurisdictions in which a fair and effective process to set price already is in place. 16
AN EXAMPLE To illustrate how the EGP might work, we highlight the example of the newly developed biocompatible bicarbonate-based peritoneal dialysis (PD) solutions. These are licensed in many jurisdictions, but have not been adopted widely because of a steep price premium and uncertain effect on important long-term outcomes.27,28 However, in theory, preliminary reports suggest that these solutions might lead to better outcomes and may increase the desire of patients and health care workers to use home-based PD therapy.29 However, research to date typically has been single center, focusing only on measurement of their impact on nonclinical outcomes. To our knowledge, there are no plans for a long-term, hard-outcome, multicenter, randomized, controlled, clinical trial. In part, this relates to the high cost of attempting such a trial and the risk that the trial will show negative or ambiguous results. An industry sponsor might approach the EGP program eligibility committee for consideration of inclusion of bicarbonate-based PD solutions in the EGP. Given the low likelihood of a clinical trial under traditional paradigms of funding and evaluation, but given the evidence to date (which includes preliminary data for safety and effectiveness based on nonvalidated surrogate end points), the potential that bicarbonate-based PD solutions might be a major therapeutic advance, and the economic implications, the program eligibility committee might agree that bicarbonatebased PD solutions qualify for the EGP. A multistakeholder committee with industry and payer representatives, PD experts, methodologists, and others then would devise the best way to obtain the data required to determine whether these solutions are superior to standard PD solutions. As a preliminary determination, let us assume that an observational registry is deemed unlikely to answer the fundamental questions. Based on that, a multicenter randomized controlled trial would be designed and power calculations would be performed with outcomes that include technique survival as the primary outcome and patient survival, rates of peritonitis and encapsulating peritoneal sclerosis, and cost-effectiveness as secondary outcomes. Funding of this trial would be shared by the industry sponsor and payers. Successful completion of this randomized controlled trial would provide unambiguous data that would enable health care decision makers to decide on public funding of bicarbonate-based PD solutions. Other examples of current new therapies for which field evaluations might be considered are non–calciumbased phosphate binders, calcimimetics, and daily hemodialysis therapy. Am J Kidney Dis. 2011;58(1):13-18
Evidence Generation
CHALLENGES There are 3 key potential problems with EGP that we anticipate and have considered. First, nephrologists and/or patients might opt in, but then not really promote the trials. Clearly, an audit of compliance and comparisons with peers might be required to identify these physicians and either remedy the recruitment effort or disqualify that practitioner. Second, patients may migrate from opted-out physicians to opted-in ones. We do not see this as a problem because we hope that all nephrologists and all patients would see the value of participation, and with more physicians and more patients, trials could be completed more quickly. Finally, if the EGP was implemented in only one or a few jurisdictions that wished to study more than one item or service, those payers or governments would take on an undue cost burden. Given that the high-quality research produced by the EGP would be of international importance, the EGP must be internationalized and coordinated to be equitable and successful in participating jurisdictions. SUMMARY These principles and concepts concerning a bold new way to generate evidence in ESRD using conditionally funded field evaluations should be regarded by the community as a way to catalyze discussions about how to stimulate innovation and test new interventions that might improve outcomes for dialysis patients. It seems likely that field evaluations would allow patients earlier access to promising therapies (even if within the context of definitive randomized trials) and positively stimulate the evolution of evidence-based nephrology practice and outcomes. For nephrologists, it would create a platform to more rapidly answer fundamental research questions. For industry, field evaluations may offset some of the prohibitive costs and risks associated with developing nephrology products and might make the nephrology research and “marketplace” more inviting. For payers and governments, it offsets some of the cost and risk of allowing early (and possibly premature) access to promising therapies, delivers the evidence they need to make fair and informed coverage decisions, and possibly sets reasonable prices. Most importantly, field evaluations will stimulate an environment that promotes renal care advances. Implementation of conditional funding with field evaluations may need to be altered to align with local resources and expertise and therefore may vary from jurisdiction to jurisdiction. Nonetheless, the principles of the proposed framework are adaptable, feasible, and may be inevitable if we wish both prompt introduction and rigorous evaluation of many more promising new therapies for dialysis patients in the Am J Kidney Dis. 2011;58(1):13-18
future. Further discussion is required by health care professionals caring for dialysis patients, potential industry sponsors, and public payers.
ACKNOWLEDGEMENTS The authors acknowledge Drs Fred Finkelstein, Adeera Levin, and Marcello Tonelli, who reviewed early drafts and provided excellent suggestions and advice. Support: None. Financial Disclosure: The authors declare that they have no relevant financial interests.
REFERENCES 1. Zelmer JL. The economic burden of end-stage renal disease in Canada. Kidney Int. 2007;72(9):1122-1129. 2. Dor A, Pauly MV, Eichleay MA, Held PJ. End-stage renal disease and economic incentives: the International Study of Health Care Organization and Financing (ISHCOF). Int J Health Care Finance Econ. 2007;7(2-3):73-111. 3. Strippoli GF, Craig JC, Schena FP. The number, quality, and coverage of randomized controlled trials in nephrology. J Am Soc Nephrol. 2004;15(2):411-419. 4. Pfeffer MA, Burdmann EA, Chen CY, et al. A trial of darbepoetin alfa in type 2 diabetes and chronic kidney disease. N Engl J Med. 2009;361(21):2019-2032. 5. Cheung AK, Levin NW, Greene T, et al. Effects of high-flux hemodialysis on clinical outcomes: results of the HEMO Study. J Am Soc Nephrol. 2003;14(12):3251-3263. 6. Fellstrom BC, Jardine AG, Schmieder RE, et al. Rosuvastatin and cardiovascular events in patients undergoing hemodialysis. N Engl J Med. 2009;360(14):1395-1407. 7. Paniagua R, Amato D, Vonesh E, et al. Effects of increased peritoneal clearances on mortality rates in peritoneal dialysis: ADEMEX, a prospective, randomized, controlled trial. J Am Soc Nephrol. 2002;13(5):1307-1320. 8. Besarab A, Bolton WK, Browne JK, et al. The effects of normal as compared with low hematocrit values in patients with cardiac disease who are receiving hemodialysis and epoetin. N Engl J Med. 1998;339(9):584-590. 9. Suki WN, Zabaneh R, Cangiano JL, et al. Effects of sevelamer and calcium-based phosphate binders on mortality in hemodialysis patients. Kidney Int. 2007;72(9):1130-1137. 10. National Kidney Foundation. KDOQI Clinical Practice Guidelines and Clinical Practice Recommendations for Anemia in Chronic Kidney Disease. Am J Kidney Dis. 2006;47(5)(suppl 3):S11-145. 11. National Kidney Foundation. KDOQI Clinical Practice Guideline and Clinical Practice Recommendations for Anemia in Chronic Kidney Disease: 2007 update of hemoglobin target. Am J Kidney Dis. 2007;50(3):471-530. 12. Manns B, Owen WF Jr, Winkelmayer WC, Devereaux PJ, Tonelli M. Surrogate markers in clinical studies: problems solved or created? Am J Kidney Dis. 2006;48(1):159-166. 13. Tierney M, Manns B. Optimizing the use of prescription drugs in Canada through the Common Drug Review. CMAJ. 2008;178(4):432-435. 14. The ALLHAT Officers and Coordinators for the ALLHAT Collaborative Research Group. Major outcomes in high-risk hypertensive patients randomized to angiotensin-converting enzyme inhibitor or calcium channel blocker vs diuretic: the Antihypertensive and Lipid-Lowering Treatment to Prevent Heart Attack Trial (ALLHAT). JAMA. 2002;288(23):2981-2997. 15. Centers for Medicare and Medicaid. https://www.cms.gov/ CoverageGenInfo/03_CED.asp. Accessed March 12, 2011. 17
Mendelssohn and Manns 16. Lindsay MJ, Siegel BA, Tunis SR, et al. The National Oncologic PET Registry: expanded Medicare coverage for PET under coverage with evidence development. AJR Am J Roentgenol. 2007;188(4):1109-1113. 17. Tunis SR, Pearson SD. Coverage options for promising technologies: Medicare’s ‘coverage with evidence development’. Health Aff (Millwood). 2006;25(5):1218-1230. 18. Neumann PJ, Tunis SR. Medicare and medical technology— the growing demand for relevant outcomes. N Engl J Med. 2010; 362(5):377-379. 19. Suri RS, Garg AX, Chertow GM, et al. Frequent Hemodialysis Network (FHN) randomized trials: study design. Kidney Int. 2007;71(4):349-359. 20. McCabe C, Bergmann L, Bosanquet N, et al. Market and patient access to new oncology products in Europe: a current, multidisciplinary perspective. Ann Oncol. 2009;20(3):403-412. 21. Chapman S, Reeve E, Rajaratnam G, Neary R. Setting up an outcomes guarantee for pharmaceuticals: new approach to risk sharing in primary care. BMJ. 2003;326(7391):707-709. 22. Boggild M, Palace J, Barton P, et al. Multiple sclerosis risk sharing scheme: two year results of clinical cohort study with historical comparator. BMJ. 2009;339:b4677. 23. Whicher DM, Chalkidou K, Dhalla IA, Levin L, Tunis S. Comparative effectiveness research in Ontario, Canada: producing
18
relevant and timely information for health care decision makers. Milbank Q. 2009;87(3):585-606. 24. Levin L, Goeree R, Sikich N, et al. Establishing a comprehensive continuum from an evidentiary base to policy development for health technologies: the Ontario experience. Int J Technol Assess Health Care. 2007;23(3):299-309. 25. Evans WK, Laupacis A, Gulenchyn KY, Levin L, Levine M. Evidence-based approach to the introduction of positron emission tomography in Ontario, Canada. J Clin Oncol. 2009;27(33): 5607-5613. 26. Pearson SD, Miller FG, Emanuel EJ. Medicare’s requirement for research participation as a condition of coverage: is it ethical? JAMA. 2006;296(8):988-991. 27. Bargman JM. Peritoneal dialysis solutions and patient survival: does wishing make it so? Nephrol Dial Transplant. 2006; 21(10):2684-2686. 28. Bargman JM. Slouching towards Bethlehem: the beast of biocompatibility. Nephrol Dial Transplant. 2010;25(7):20502051. 29. Haag-Weber M, Kramer R, Haake R, et al. Low-GDP fluid (Gambrosol trio) attenuates decline of residual renal function in PD patients: a prospective randomized study. Nephrol Dial Transplant. 2010;25(7):2288-2296.
Am J Kidney Dis. 2011;58(1):13-18