- Email: [email protected]
The Role of Clinical Registries in Health Care
C H A P T E R
5 The Role of Clinical Registries in Health Care Panagiotis Kerezoudis*,†, Clinton J. Devin‡, Sandy Goncalves*,†, Mohammed A. Alvi*,†, Anthony L. Asher§,¶, Mohamad Bydon*,†,1 *Mayo Clinic Neuro-Informatics Laboratory, Rochester, MN, United States † Department of Neurologic Surgery, Mayo Clinic, Rochester, MN, United States ‡ Department of Orthopedic Surgery, Vanderbilt University, Nashville, TN, United States § Neuroscience Institute, Carolinas Healthcare System, Charlotte, NC, Unites States ¶ Carolina Neurosurgery & Spine Associates, Charlotte, NC, United States 1 Corresponding author e-mail: [email protected]
O U T L I N E Health Care Reform in the United States
54
The Role of Registries in the Contemporary Health Care Environment 55 Implications for Neurosurgeons
57
Value-Based Purchasing and Quality Reporting
57
Successful National Registries
58
The Quality Outcomes Database
59
Curbing the Rising Costs
60
The Randomized Registry Trial
60
Quality and Safety in Neurosurgery https://doi.org/10.1016/B978-0-12-812898-5.00005-9
53
© 2018 Elsevier Inc. All rights reserved.
54
5. THE ROLE OF CLINICAL REGISTRIES IN HEALTH CARE
Device Registries
62
Current Challenges in Registry Implementation and Data Collection
63
Conclusions
64
References
65
HEALTH CARE REFORM IN THE UNITED STATES Measuring and improving the quality of surgical health care is an increasingly important goal in understanding the overall value, contribution, and impact surgical care has on the American Healthcare system. The current growth rate of health care costs in the United States is unsustainable, and therefore the ability to define real-world effectiveness of health care delivery is of vital importance. The Centers for Disease Control and Prevention (CDC) recently reported that the annual national health care expenditures have reached $3 trillion.1 Furthermore, according to the Institute of Medicine, it has been estimated that almost 30% of health care spending is provided for approaches that are cost ineffective.2 As such, there has been an unprecedented call for greater physician accountability and care optimization. The adoption of the Patient Protection and the Affordable Care Act (PPACA) in 2010 established the foundation for the national standardization of health care delivery, ultimately leading to wide-ranging legislative oversight and determination of health care value.3 Health care value, defined as the ratio of treatment safety and efficacy against cost, is the essence in the new health care environment. Measuring value-based care has been a long striving task. Researchers have been traditionally relying on the evidence-based medicine paradigm for generating guidelines with regard to safe, cost-effective medical and surgical interventions.4,5 According to this hierarchy, randomized controlled trials (RCTs) are considered to generate the highest level of evidence for clinical decision-making, as they minimize confounding variables and protect against various sources of bias.6 Nevertheless, RCTs are bound by several limitations; first, developing and maintaining randomization in surgical trials can be extremely challenging in satisfying the randomization requirement in clinical trials.7 In some cases, randomization may be either impractical or unethical, due to the lack of treatment equipoise. Furthermore, most RCTs are inherently discontinuous in nature; this hinders the utilization of data collection and analysis infrastructure for delineating and improving the continuum of care through constant bidirectional information exchange.7 More importantly, many RCT findings have failed to translate into tangible benefits for the general population.8
2. MEASURING AND IMPROVING PERFORMANCE
THE ROLE OF REGISTRIES IN THE CONTEMPORARY HEALTH CARE ENVIRONMENT
55
THE ROLE OF REGISTRIES IN THE CONTEMPORARY HEALTH CARE ENVIRONMENT Former US President Barack Obama in his 21st-century Cures Act highlighted the need for real-world evidence in order to inform therapeutic approaches and patient care, guide outcomes research, stimulate quality improvement, develop efficient safety surveillance systems, and promote well-controlled effectiveness studies.9 An important feature of real-worldevidence is that information is gathered not only in academic and researchintensive environments but also in clinical care and home or community settings.10 In addition, real-world evidence may allow researchers to answer comparative effectiveness research questions efficiently that would be otherwise prohibitively resource- and time-intensive to study with an RCT. Clinical registries have seen explosive growth in recent years and continue to demonstrate advantages in accessibility and flexibility in data collection (Fig. 5.1).11 According to the Agency for HealthCare Research and Quality, a registry is “an organized system that uses observational study methods to collect uniform data (clinical and other) to evaluate specified outcomes for a population defined by a particular disease, condition, or exposure and that serves predetermined scientific, clinical, or policy purpose(s).”12 From a scientific perspective, the primary expectation of a welldesigned registry relates to the strength of its external validity. This can be achieved through a comprehensive design that attempts to evaluate heterogeneous populations as compared to the classic method of
FIG. 5.1
Potential roles of clinical registries in health care.
2. MEASURING AND IMPROVING PERFORMANCE
56
5. THE ROLE OF CLINICAL REGISTRIES IN HEALTH CARE
interventional analysis that primarily seeks to evaluate homogenous patient groups.7 As a result of the unique design of clinical registries, the observed health outcomes analyses are often more representative of realworld practice and therefore more likely to draw inferences for broad patient populations and widespread practice settings. This is further enhanced with the popularization of advanced epidemiological methods, including propensity score methods13 and instrumental variable analysis.14 These methods allow providers and researchers to be better equipped with research tools that allow them to mitigate confounding and selection bias.15 Another asset that is particularly helpful in the setting of hospital data relates to the continuous nature of registries.16 They can easily adapt to modernizations in surgical care and facilitate small- and large-scale participation in programs that depend on continuous collection of practice quality data for performance improvement, reporting, and specialtyspecific certification purposes.16 However, certain aspects of registry design and implementation need to be well defined, such as the target population intended to be captured, the covariates and the outcomes of interest to be collected, as well as the type of randomized study that is being modeled with the registry.15 Klaiman and colleagues reported on six key characteristics for a successful clinical registry based on an exhaustive literature search of registries dedicated to the treatment of cancer, cardiovascular disease, maternity care, and joint replacement therapies.17 The following domains were identified: (1) Data standardization: data need to be standardized across multiple clinical practices, given the individual nature of different clinical charting systems. (2) Transparency: registries should be at the forefront of public data sharing, however, in the case of proprietary registries, that is, those owned by specialty societies, members’ concerns when it comes to publishing report cards should be taken into account. In a similar fashion, state-sponsored cancer registries typically avoid generating reports on provider outcomes for fear of compromising patient confidentiality. (3) Provider participation: provider motivation is critical to enhance the versatility and robustness of data and participating practices. (4) Accuracy/completeness of data: robust methods need to be in place in order to minimize the amount of missing data and support meticulous follow-up efforts, particularly for patients who travel out of state for further evaluation or treatment. (5) Financial sustainability: public and federal support is crucial in terms of registry sustainability given the associated budget constraints. For example, cardiovascular registries have found various methods to
2. MEASURING AND IMPROVING PERFORMANCE
VALUE-BASED PURCHASING AND QUALITY REPORTING
57
maintain financial stability, including subsidies from pharmaceutical companies, fees from participating providers, and collected revenues from registry data sale.17 (6) Feedback to providers: sophisticated statistical models have been developed to provide risk-adjusted feedback to providers and make publicly available online risk calculators for predicted length of stay as well as postoperative morbidity and mortality data.
IMPLICATIONS FOR NEUROSURGEONS The PPACA is laden with significant ramifications for surgical practices.3,18 As regulations are updated and new policies are adopted, goaloriented and flexible cooperation is required between governmental bodies, regulatory stakeholders, and the neurosurgical community.16 Neurosurgical practices are increasingly moving in the direction of value-based medicine requiring physicians to advocate and objectively demonstrate the quality and efficiency of their medical practices. This is particularly important given the recent finding that approximately 25% of all diagnostic and therapeutic spine procedures are unnecessary or ineffective.19,20 This reiterates the need that neurosurgeons should define and collect their own metrics and practice data.16
VALUE-BASED PURCHASING AND QUALITY REPORTING The enactment of PPACA has inaugurated an important transition toward value-based purchasing and pay-for-performance payment models.21,22 According to these models, incentives and rewards are granted to hospitals and providers for adhering to quality standards, thus creating behavior modifications through motivation for higher pay. As a result, significant efforts and resources have been expanded to develop surgical quality measures upon which surgical performance can be evaluated and benchmarked. Registries provide the necessary framework to organize data in a meaningful way from a medical and societal perspective in order to support claims received by both public and private payers. Qualified clinical data registries (QCDRs) were introduced by the Centers for Medicare and Medicaid Services CMS in 2014.23 A QCDR is a certification board, registry, or another collaborative effort that compiles and submits medical and clinical measures results under the current reporting requirements with the ultimate goal of fostering progress in the quality of care provided to patients.11,23 QCDR measures are derived from multiple different sources,
2. MEASURING AND IMPROVING PERFORMANCE
58
5. THE ROLE OF CLINICAL REGISTRIES IN HEALTH CARE
including Clinician & Group-Consumer Assessment of Healthcare Providers and Systems, National Quality Forum (NSF)-endorsed measures, and PQRS measures among other cross-disciplinary collaborations.11 However, a QCDR has the capability to develop and include measures that accommodate each specialty care, including neurological surgery. In addition, quality measures are reported in both private and public payer systems and are not limited to Medicare beneficiaries.11 According to the new model, the Merit-based Incentive System (MIPS), penalty payment reductions will reach 4% in 2018, 5% in 2019, 7% in 2020, and 9% in 2021–23.24,25
SUCCESSFUL NATIONAL REGISTRIES One of the largest surgical registries on a global scale is the National Surgical Quality Improvement Program (NSQIP) (Table 5.1). NSQIP incorporates more than 300 variables on patient demographics, comorbidities, intraoperative data as well as 30-day postoperative morbidity and mortality outcomes.26 Currently, the dataset contains data on over 1.7 million patients from more than 500 institutions, 58% of which are large academic institutions.27 One other advantage of the NSQIP registry is that the majority of the data available represents a random sample of all procedures across most surgical specialties.28 Data acquisition from each participating institution is completed by specialized data collectors, who complete extensive training. More importantly, interrater reliability is routinely audited by Surgical Clinical Reviewers with quality control processes that maintain high-fidelity data collection. According to the most recent report, the interrater reliability audit for participating sites demonstrated an overall disagreement rate of roughly 2%.29 One final feature of the NSQIP registry is a helpful tool called TABLE 5.1 Examples of Successful Registries Registry
Discipline
National Surgical Quality Improvement Program
All surgical specialties
STS National Database
Cardiothoracic surgery
Quality Outcomes Database
Spine and cranial surgery
State Cancer Registries
Cancer
CathPCI Registry of the National Cardiovascular Data Registry
PCI devices
Kaiser Permanente National Implant Registries
Devices and Implants in cardiovascular, orthopedics, spine, and endovascular
PCI, percutaneous coronary intervention; STS, Society of Thoracic Surgeons.
2. MEASURING AND IMPROVING PERFORMANCE
THE QUALITY OUTCOMES DATABASE
59
the surgical risk calculator that aims to estimate accurately 30-day complication, mortality, and readmission rates.30 Another representative example of a successful surgical registry implementation with high impact on the quality of care is the National Database of the Society of Thoracic Surgeons (STS).31 It has effectively shown the advantage of national practice data collection programs in advancing patient-centered quality care, with almost 95% of all US thoracic surgery practices currently reporting to this well-established database.32 The STS has implemented risk-adjusted regional and national benchmarks with regard to procedural quality and safety. In fact, the STS has defined STS-specific measures of quality and performance utilizing the database, which have been subsequently endorsed by the NSF.33 This allowed for creating and establishing its own quality and performance measures, while avoiding being forced to comply with outside stakeholdergenerated standards. Approximately 50% of participating sites partake in pioneering the voluntary public reporting system in an attempt to promote accountability in care. Surgeons are also offered the opportunity to leverage the data contained in the registry in order to identify targeted areas for quality improvement. Finally, and perhaps most importantly, the STS National Database has been instrumental in advancing scientific discovery and improving patient care in thoracic surgery. For example, it played a key role in establishing the superiority of internal thoracic artery bypass grafting and promoting the large-scale application of this method as the standard of care in thoracic surgery.34 State cancer registries in the United States have used robust and standardized data collection techniques in order to produce reports on national trends for cancer incidence, type, and mortality. They are funded by multiple federal organizations, including the National Program of Cancer Registries, administered by the Centers for Disease Control and Prevention, and the Surveillance, Epidemiology and End Results Program, administered by the National Cancer Institute. California cancer registry is an exemplary cancer registry that has collected comprehensive data on over 7 million cases of cancer among patients living in California diagnosed since 1988, with over 175,000 new cases added each year.35 Case ascertainment is estimated to be 99% complete.
THE QUALITY OUTCOMES DATABASE The American Association of Neurological Surgeons, in partnership with the Congress of Neurological Surgeons, the Society of Neurological Surgeons, and the American Board of Neurological Surgery, launched in March 2012 the Quality Outcomes Database (QOD) (formerly known as the National Neurosurgical Quality Outcomes Database-N2QOD).36,37
2. MEASURING AND IMPROVING PERFORMANCE
60
5. THE ROLE OF CLINICAL REGISTRIES IN HEALTH CARE
This database is a national, multiinstitutional, prospective registry that captures various preoperative patient characteristics as well as prospective surgical and patient reported outcomes at 3- and 12-months postoperatively.38 The database was created to establish a robust mechanism for risk-adjusted benchmarking, real-time analysis of individual morbidity and clinical outcomes, comparative effectiveness research, as well as evidence-based advancements in surgical procedures and facilitation of collaborative trials.36,37 Data extractors and coordinators undergo extensive training on data entry and standard operating procedures outlined in the QOD infrastructure. The first six patients who meet the prespecified eligibility criteria are enrolled into the registry via a standard sampling framework after being identified from the weekly posted surgical schedule.7 Each of the six patients are enrolled either in person during their initial clinic visit or by phone where their health status is assessed via interview. Data quality is achieved through automated and manual methods at Vanderbilt University Institute for Medicine and Public Health. Participating sites are informed weekly for missing data and periodic surgeon-ledself-audits and random audits are performed to ensure data completeness and accuracy.
CURBING THE RISING COSTS Health care systems are in a constant struggle with rising costs. Within this context, registries can also aid in identifying the most effective care paradigms and refining surgical treatments to maximize success. A multinational study querying thirteen registries from five countries (Australia, Denmark, Sweden, the United Kingdom, and the United States) suggested that by making outcomes data transparent to both the public and practitioners alike, well-managed registries allow providers to engage in constant learning with the ability to identify and disseminate best clinical practices.39 The authors estimated that the implementation of a hip replacement registry in the United States, comparable to the Swedish Hip Arthroplasty Register, would result in a reduction of $2 billion out of a total of $24 billion in 2015 by reducing reoperation rates in replacing or repairing hip prostheses.
THE RANDOMIZED REGISTRY TRIAL As mentioned above, RCT performance and completion is hampered by the design complexity and the immense costs associated.40 To overcome such challenges, health services researchers have recently began exploring the possibility of leveraging the infrastructure and the resources provided
2. MEASURING AND IMPROVING PERFORMANCE
THE RANDOMIZED REGISTRY TRIAL
61
by registries with other study designs with the aim of producing robust, “hybrid” approaches for evidence-based development. A high-quality registry retains many RCT traits, such as high follow-up rates, utilization of patient outcomes, prospective data collection, and data quality validation while mitigating sources of bias through various advanced study designs. This registry-based randomized trial, also called the randomized registry trial (RRT), attempts to expedite and enhance patient enrollment, minimize cost, and address the generalizability of findings, while maintaining the statistical rigor of randomization (Table 5.2).41–43 It represents an innovative TABLE 5.2 Comparison of Characteristics Between Conventional Randomized Controlled Trials, Registries, and Randomized Registry Trials
Characteristic
Conventional randomized controlled trial
Registry studies
Randomized registry trial
Blinding
Possible
Not possible
Possible
Randomization
Possible
Not possible
Possible
Minimize bias
Yes
No
Yes
Minimize confounding
Yes
No
Yes
Applicability
Limited
Real-world representation
Real-world representation
Cost
Expensive
Cost-effective solution
Leverage existing registry infrastructure
Amount of missing data
Minimal
Frequent
Minimal
Data quality
High
Variable and questionable
High
Other strengths
Considered the gold-standard design
Large sample size allows for identification of rare events Hypothesis generating
Flexible linkage to registry Compare enrolled to nonenrolled patients
Other limitations
Often funded by Pharma Long time to plan and implement
Data collection limited to registry capabilities Link to causality limited
Limited capacity for collection of biospecimens, pharmacodynamic, and pharmacokinetic indices
Primary purpose
Research
Research and quality reporting
Research and quality reporting
2. MEASURING AND IMPROVING PERFORMANCE
62
5. THE ROLE OF CLINICAL REGISTRIES IN HEALTH CARE
and “disruptive technology” in order to advance comparative effectiveness research and provide Level 1—prognostic and Level 2—effectiveness evidence.44 The first RRT originated in Sweden, where 7244 patients across three countries in more than 30 sites were accrued in less than 3 years.42 The estimated cost of the trial was remarkably as low as $50 per patient. Similarly, the Study of Access Site for Enhancement of Percutaneous Coronary Intervention (SAFE-PCI) for Women Trial was the first RRT in the US conducted in the United States.45 This open-label study was conducted in collaboration with the National Institutes of Health National Cardiovascular Data Registry Cath-PCI Registry platform. Women were randomized to percutaneous coronary intervention via radial versus femoral access to compare procedural bleeding complications at high volume hospitals, where participating physicians were required to be proficient at both radial and femoral access. Recruitment efficiency and overall success was attributed to the very nature of RTT design with accrual reaching 99.2% in just 22 months. 45 Reviews of this design identified key learning conclusions. According to Li and colleagues, the following lessons should be considered as this novel and promising trial design is further developed: (1) The question needs to fit the registry purposes and not the opposite. (2) The RRT design should not replace traditional RCTs, but rather offer an alternative strategy to provide answers to questions that would otherwise remain unanswered due to financial and time constraints. (3) RRTs are by definition pragmatic in their nature; therefore, they are more appropriate to investigate the safety and effectiveness of medications, devices, and treatments widely accepted in real-world settings but still variable in their use.43,46 Asking a well-defined, specific question is therefore paramount. (4) The primary aim and methods of the registry should not be affected by the RRT conduct. Balance needs to be maintained with regard to the workload and the workflow between the RRT and the registry. (5) High data quality and rigorous statistical analyses are pivotal, as accuracy of the findings is heavily dependent on data quality. (6) Strong ethos collaboration is vital and the needs and limitations of each stakeholder should be kept in mind.
DEVICE REGISTRIES Postmarket surveillance of medical devices poses unique challenges compared to drugs and biologics due to device diversity and complexity, attributes which are further complicated by the iterative nature of medical
2. MEASURING AND IMPROVING PERFORMANCE
CURRENT CHALLENGES IN REGISTRY IMPLEMENTATION AND DATA COLLECTION
63
product development and the learning curve associated with technology adoption.47Device-specific registries provide detailed information regarding patient care, procedural interventions, and device implantation not routinely collected by electronic health records and claims data. As unique device identifiers (UDI) become increasingly incorporated into electronic health information, the role of clinical registries in postmarket surveillance will continue to evolve. The Food and Drug Administration (FDA) has recently announced that it is willing to partner with institutions and, more specifically, surgeons across the county in order to leverage existing integrated data systems to monitor new technologies, evaluate associated patient outcomes, and facilitate implementation of best clinical practices.47 The value of such registries was recently brought to light with the publication of a registry-based, active surveillance of an implantable vascularclosure device safety trial. The trial investigated potential safety signals with regard to associated vascular complications among recipients of the device, with initial alerts occurring within the first 12 months of monitoring.48 Finally, Kaiser Permanente has developed and maintained longstanding device registries that have included cardiac devices, endovascular stents as well as orthopedic and spinal implants.49 These registries have been used to describe device information related to device recipients, outcomes data, and overall success of each device.
CURRENT CHALLENGES IN REGISTRY IMPLEMENTATION AND DATA COLLECTION As with other observational data sources, registries are also subject to several limitations. Owing to the very nature of data collection and analysis, there will always be challenges in eliminating all sources of bias that might arise. Challenges in ensuring the highest possible degree of data quality can be attenuated by enlisting a variety of novel analytical tools to lessen the probability of systematic error as well as errors in inference.7 The collection burden for longitudinal data represents another challenge. Efforts need to be made during the design stage to attain the least amount of user influence for data collection with the lowest number of prespecified variables without oversimplifying the data. Natural language processing of medical records, operative notes, and radiology reports constitutes a set of rapidly progressing techniques that may play a significant role in aiding data collection.50 According to the definition of value, comprehensible cost-related data are also needed to be captured and evaluated when registries are implemented. For that purpose, QOD is now engaging in pilot programs to merge its clinical outcomes data with administrative cost-related data sources.7 In addition, the need for a written informed consent form for
2. MEASURING AND IMPROVING PERFORMANCE
64
5. THE ROLE OF CLINICAL REGISTRIES IN HEALTH CARE
patients included into a registry is still a matter of debate between registry administrators and Institutional Review Board (IRB) committees, as it is primarily conducted for quality improvement and not research purposes.51 From a regulatory perspective, contention usually involves whether the registry is subject to the jurisdiction of the Common Rule (45 CFR 46) and whether data collection methods comply with the Privacy Rule.51 As such, a detailed outline of the appropriate use of protected health information, operational procedure, patient interaction, and reporting methods is routinely made available to sites for submission to their institution’s quality improvement office or IRB. Ultimately, registry data must apply to clinical practice to advance value-based health care. Evolving mechanisms for provider education and meaningful application of the surgical and functional outcomes data are critical. Most importantly, the neurological surgery profession, unlike the STS, still lacks specialty-specific quality measures endorsed by the NSF.11 National neurological surgery administration and leadership are now working on developing specialty-specific,NSF-endorsed measures for public and private reporting purposes. Finally, concerns have been previously expressed with regard to the unintended impairment of physician workflow and time of interaction with patients as a result of electronic medical record (EMR) adoption.16,52,53 Significant initiatives have been spearheaded in multiple clinical practices across the country to adopt an EMR system. Multiple teams have reported new ways to decrease the burden of registry-related manual data entry.54 Azad and colleagues recently shared their institutional experience with an EMR-integrated quality of life outcomes registry for patients who undergo spine surgery.54 Using a systematic approach to collect spine-related quality of life data within the EMR system, they developed a clinical dashboard that could track more than 25 patient variables without posing any significant impact on the patient’s clinic visit or surgeon’s clinical workflow.
CONCLUSIONS In the current era of an increasingly pay-for-performance health care industry, there is growing interest in producing objective data to determine the quality of care and inform the allocation of health care resources. Given the ongoing national debate centered around health care reform as well as the rapid cost and growth of surgical procedures for a multitude of conditions, prospective clinical registries, such as QOD, have emerged as a promising source of a diverse set of data. In contrast to RCT methods, which are primarily employed for research, registries can be used for both quality improvement and research. More importantly, registries represent
2. MEASURING AND IMPROVING PERFORMANCE
REFERENCES
65
sustainable solutions for characterizing real-world surgical care while identifying large-scale improvement opportunities. Such initiatives will ensure that surgical outcomes are appropriately measured and integrated into policy. More importantly, current metrics mandate a holistic approach to the particular needs of patients. As such, a multifaceted outcomes evaluation system that is applicable to a wide spectrum of disorders along with patient reported outcomes and cost data will constitute the linchpin of future registry collection efforts and enhance the overall value of US health care.
References 1. National Center for Health Statistics (US). Health, United States, 2013: With Special Feature on Prescription Drugs. Hyattsville, MD: National Center for Health Statistics (US); 2014. 2. Reilly BM, Evans AT. Much ado about (doing) nothing. Ann Intern Med. 2009;150(4): 270–271. 3. Menger RP, Guthikonda B, Storey CM, Nanda A, McGirt M, Asher A. Neurosurgery value and quality in the context of the affordable care act: a policy perspective. Neurosurg Focus. 2015;39(6):E5. 4. Sackett DL. Evidence-based medicine. Spine. 1998;23(10):1085–1086. 5. Guyatt GH, Oxman AD, Kunz R, et al. What is “quality of evidence” and why is it important to clinicians? BMJ. 2008;336(7651):995–998. 6. Guyatt G, Oxman AD, Akl EA, et al. GRADE guidelines: 1. Introduction—GRADE evidence profiles and summary of findings tables. J Clin Epidemiol. 2011/4;64(4):383–394. 7. Asher AL, Parker SL, Rolston JD, Selden NR, McGirt MJ. Using clinical registries to improve the quality of neurosurgical care. Neurosurg Clin N Am. 2015;26(2):253–263. ix–x. 8. Booth CM, Tannock IF. Randomised controlled trials and population-based observational research: partners in the evolution of medical evidence. Br J Cancer. 2014;110(3): 551–555. 9. Jaffe S. 21st century cures act progresses through US congress. Lancet. 2015;385 (9983):2137–2138. 10. Sherman RE, Anderson SA, Dal Pan GJ, et al. Real-world evidence—what is it and what can it tell us? N Engl J Med. 2016;375(23):2293–2297. 11. Bekelis K, McGirt MJ, Parker SL, et al. The present and future of quality measures and public reporting in neurosurgery. Neurosurg Focus. 2015;39(6):E3. 12. Gliklich R, Dreyer N, Leavy M, eds. Registries for Evaluating Patient Outcomes: A User’s Guide. 3rd ed. Two vols. Rockville, MD: Agency for Healthcare Research and Quality; 2014. (Prepared by the Outcome DEcIDE Center [Outcome Sciences, Inc., a Quintiles company] under Contract No. 290 2005 00351 TO7.) AHRQ Publication No. 13(14)EHC111. http://www.effectivehealthcare.ahrq.gov/registries-guide-3.cfm. 13. Rosenbaum PR, Rubin DB. The central role of the propensity score in observational studies for causal effects. Biometrika. 1983;70(1):41–55. 14. Martens EP, Pestman WR, de Boer A, Belitser SV, Klungel OH. Instrumental variables: application and limitations. Epidemiology. 2006;17(3):260–267. 15. Harbaugh RE. Rupture risk of unruptured aneurysms: beyond size and shape. JNS. 2016;124:286–287. 16. Selden NR, Ghogawala Z, Harbaugh RE, Litvack ZN, McGirt MJ, Asher AL. The future of practice science: challenges and opportunities for neurosurgery. Neurosurg Focus. 2013;34(1):E8.
2. MEASURING AND IMPROVING PERFORMANCE
66
5. THE ROLE OF CLINICAL REGISTRIES IN HEALTH CARE
17. Klaiman T, Pracilio V, Kimberly L, Cecil K, Legnini M. Leveraging effective clinical registries to advance medical care quality and transparency. Popul Health Manag. 2014;17(2):127–133. 18. Kim DH, Dagi TF, Bean JR. Neurosurgical practice in transition: a review. Neurosurgery. 2017;80(4S):S4–S9. 19. Committee on the Learning Health Care System in America, Institute of Medicine. In: Smith M, Saunders R, Stuckhardt L, JM MG, eds. Best Care at Lower Cost: The Path to Continuously Learning Health Care in America. Washington, DC: National Academies Press; 2014. 20. Epstein NE, Hood DC. “Unnecessary” spinal surgery: a prospective 1-year study of one surgeon’s experience. Surg Neurol Int. 2011;2:83. 21. VanLare JM, Conway PH. Value-based purchasing—national programs to move from volume to value. N Engl J Med. 2012;367(4):292–295. 22. Blumenthal D, Jena AB. Hospital value-based purchasing. J Hosp Med. 2013;8(5):271–277. 23. Center for Medicare and Medicaid Services, QCDRs and Physician Compare. 2016. https://www.cms.gov/Medicare/Quality-Initiatives-Patient-Assessment-Instruments/ physician-compare-initiative/QCDRs-and-Physician-Compare.html. Accessed May 27, 2017. 24. Swartz KR, Cheng JS. Anticipated changes in spine practice with advancing Center for Medicare and Medicaid Services-Required Changes. Neurosurgery. 2017;80(4S):S28–S33. 25. Center for Medicare and Medicaid Services, MACRA: MIPS & APMs, 2017. https:// www.cms.gov/medicare/quality-initiatives-patient-assessment-instruments/valuebased-programs/macra-mips-and-apms/macra-mips-and-apms.html. Accessed May 27, 2017. 26. Kerezoudis P, McCutcheon BA, Murphy M, et al. Predictors of 30-day perioperative morbidity and mortality of unruptured intracranial aneurysm surgery. Clin Neurol Neurosurg. 2016;149:75–80. 27. McCutcheon BA, Kerezoudis P, Porter AL, et al. Coma and stroke following surgical treatment of unruptured intracranial aneurysm: an ACS-NSQIP study. World Neurosurg. 2016;(April). https://doi.org/10.1016/j.wneu.2016.04.039. 28. Pitt HA, Kilbane M, Strasberg SM, et al. ACS-NSQIP has the potential to create an HPBNSQIP option. HPB. 2009;11(5):405–413. 29. American College of Surgeons. User Guide for the 2014 ACS NSQIP Participant Use Data File (PUF); 2015. 30. Cohen ME, Ko CY, Bilimoria KY, et al. Optimizing ACS NSQIP modeling for evaluation of surgical quality and risk: patient risk adjustment, procedure mix adjustment, shrinkage adjustment, and surgical focus. J Am Coll Surg. 2013;217(2):336–346e1. 31. Shahian DM, Jacobs JP, Edwards FH, et al. The Society of Thoracic Surgeons National Database. Heart. 2013;99(20):1494–1501. 32. Stewart JM. Abstraction techniques for the STS National Database. J Extra Corpor Technol. 2016;48(4):201–203. 33. Shahian DM, Edwards FH, Ferraris VA, et al. Quality measurement in adult cardiac surgery: part 1—conceptual framework and measure selection. Ann Thorac Surg. 2007;83(4): S3–S12. 34. Lytle BW, Blackstone EH, Sabik JF, Houghtaling P, Loop FD, Cosgrove DM. The effect of bilateral internal thoracic artery grafting on survival during 20 postoperative years. Ann Thorac Surg. 2004;78(6):2005–2012 [discussion 2012-2014]. 35. Al-Refaie WB, Vickers SM, Zhong W, Parsons H, Rothenberger D, Habermann EB. Cancer trials versus the real world in the United States. Ann Surg. 2011;254(3):438–442. [discussion 442-443]. 36. Alliance N. http://www.neuropoint.org/en/NPA-N2QOD/. Accessed 27 May 2017. 37. McGirt MJ, Speroff T, Dittus RS, Harrell Jr. FE, Asher AL. The National Neurosurgery Quality and outcomes database (N2QOD): general overview and pilot-year project description. Neurosurg Focus. 2013;34(1):E6.
2. MEASURING AND IMPROVING PERFORMANCE
REFERENCES
67
38. Asher AL, Speroff T, Dittus RS, et al. The National Neurosurgery Quality and outcomes database (N2QOD): a collaborative North American outcomes registry to advance valuebased spine care. Spine. 2014;39(22 Suppl. 1):S106–S116. 39. Larsson S, Lawyer P, Garellick G, Lindahl B, Lundstr€ om M. Use of 13 disease registries in 5 countries demonstrates the potential to use outcome data to improve health care’s value. Health Aff. 2012;31(1):220–227. 40. Liu JB, D’Angelica MI, Ko CY. The randomized registry trial: two birds, one stone. Ann Surg. 2017;265(6):1064–1065. 41. James S, Rao SV, Granger CB. Registry-based randomized clinical trials—a new clinical trial paradigm. Nat Rev Cardiol. 2015;12(5):312–316. 42. Lauer MS, D’Agostino Sr. RB. The randomized registry trial—the next disruptive technology in clinical research? N Engl J Med. 2013;369(17):1579–1581. 43. Li G, Sajobi TT, Menon BK, et al. Registry-based randomized controlled trials—what are the advantages, challenges, and areas for future research? J Clin Epidemiol. 2016;80:16–24. 44. Ball CM, Phillips RS. Evidence-Based On-Call. Edinburgh: Churchill Livingstone; 2001. 45. Hess CN, Rao SV, Kong DF, et al. Embedding a randomized clinical trial into an ongoing registry infrastructure: unique opportunities for efficiency in design of the study of access site for enhancement of percutaneous coronary intervention for women (SAFE-PCI for women). Am Heart J. 2013;166(3):421–428. 46. Registry Trials. Clinical Trials Transformation Initiative. https://www.ctti-clinicaltrials. org/projects/registry-trials. Published May 26, 2016. Accessed May 26, 2017. 47. Food US, Administration D, Others. Strengthening Our National System for Medical Device Postmarket Surveillance. Silver Spring, MD: Center for Devices and Radiological Health; 2012. 48. Resnic FS, Majithia A, Marinac-Dabic D, et al. Registry-based prospective, active surveillance of medical-device safety. N Engl J Med. 2017;376(6):526–535. 49. National Implant Registries Kaiser Permanente. https://national-implantregistries. kaiserpermanente.org/about. Accessed 26 May 2017. 50. Zhou L, Hripcsak G. Temporal reasoning with medical data—a review with emphasis on medical natural language processing. J Biomed Inform. 2007/4;40(2):183–202. 51. Asher AL, McGirt MJ, Glassman SD, et al. Regulatory considerations for prospective patient care registries: lessons learned from the National Neurosurgery Quality and outcomes database. Neurosurg Focus. 2013;34(1):E5. 52. McWilliams JM, Song Z. Implications for ACOs of variations in spending growth. N Engl J Med. 2012;366(19):e29. 53. Park SY, Lee SY, Chen Y. The effects of EMR deployment on doctors’ work practices: a qualitative study in the emergency department of a teaching hospital. Int J Med Inform. 2012;81(3):204–217. 54. Azad TD, Kalani M, Wolf T, et al. Building an electronic health record integrated quality of life outcomes registry for spine surgery. J Neurosurg Spine. 2016;24(1):176–185.
2. MEASURING AND IMPROVING PERFORMANCE
5 The Role of Clinical Registries in Health Care Panagiotis Kerezoudis*,†, Clinton J. Devin‡, Sandy Goncalves*,†, Mohammed A. Alvi*,†, Anthony L. Asher§,¶, Mohamad Bydon*,†,1 *Mayo Clinic Neuro-Informatics Laboratory, Rochester, MN, United States † Department of Neurologic Surgery, Mayo Clinic, Rochester, MN, United States ‡ Department of Orthopedic Surgery, Vanderbilt University, Nashville, TN, United States § Neuroscience Institute, Carolinas Healthcare System, Charlotte, NC, Unites States ¶ Carolina Neurosurgery & Spine Associates, Charlotte, NC, United States 1 Corresponding author e-mail: [email protected]
O U T L I N E Health Care Reform in the United States
54
The Role of Registries in the Contemporary Health Care Environment 55 Implications for Neurosurgeons
57
Value-Based Purchasing and Quality Reporting
57
Successful National Registries
58
The Quality Outcomes Database
59
Curbing the Rising Costs
60
The Randomized Registry Trial
60
Quality and Safety in Neurosurgery https://doi.org/10.1016/B978-0-12-812898-5.00005-9
53
© 2018 Elsevier Inc. All rights reserved.
54
5. THE ROLE OF CLINICAL REGISTRIES IN HEALTH CARE
Device Registries
62
Current Challenges in Registry Implementation and Data Collection
63
Conclusions
64
References
65
HEALTH CARE REFORM IN THE UNITED STATES Measuring and improving the quality of surgical health care is an increasingly important goal in understanding the overall value, contribution, and impact surgical care has on the American Healthcare system. The current growth rate of health care costs in the United States is unsustainable, and therefore the ability to define real-world effectiveness of health care delivery is of vital importance. The Centers for Disease Control and Prevention (CDC) recently reported that the annual national health care expenditures have reached $3 trillion.1 Furthermore, according to the Institute of Medicine, it has been estimated that almost 30% of health care spending is provided for approaches that are cost ineffective.2 As such, there has been an unprecedented call for greater physician accountability and care optimization. The adoption of the Patient Protection and the Affordable Care Act (PPACA) in 2010 established the foundation for the national standardization of health care delivery, ultimately leading to wide-ranging legislative oversight and determination of health care value.3 Health care value, defined as the ratio of treatment safety and efficacy against cost, is the essence in the new health care environment. Measuring value-based care has been a long striving task. Researchers have been traditionally relying on the evidence-based medicine paradigm for generating guidelines with regard to safe, cost-effective medical and surgical interventions.4,5 According to this hierarchy, randomized controlled trials (RCTs) are considered to generate the highest level of evidence for clinical decision-making, as they minimize confounding variables and protect against various sources of bias.6 Nevertheless, RCTs are bound by several limitations; first, developing and maintaining randomization in surgical trials can be extremely challenging in satisfying the randomization requirement in clinical trials.7 In some cases, randomization may be either impractical or unethical, due to the lack of treatment equipoise. Furthermore, most RCTs are inherently discontinuous in nature; this hinders the utilization of data collection and analysis infrastructure for delineating and improving the continuum of care through constant bidirectional information exchange.7 More importantly, many RCT findings have failed to translate into tangible benefits for the general population.8
2. MEASURING AND IMPROVING PERFORMANCE
THE ROLE OF REGISTRIES IN THE CONTEMPORARY HEALTH CARE ENVIRONMENT
55
THE ROLE OF REGISTRIES IN THE CONTEMPORARY HEALTH CARE ENVIRONMENT Former US President Barack Obama in his 21st-century Cures Act highlighted the need for real-world evidence in order to inform therapeutic approaches and patient care, guide outcomes research, stimulate quality improvement, develop efficient safety surveillance systems, and promote well-controlled effectiveness studies.9 An important feature of real-worldevidence is that information is gathered not only in academic and researchintensive environments but also in clinical care and home or community settings.10 In addition, real-world evidence may allow researchers to answer comparative effectiveness research questions efficiently that would be otherwise prohibitively resource- and time-intensive to study with an RCT. Clinical registries have seen explosive growth in recent years and continue to demonstrate advantages in accessibility and flexibility in data collection (Fig. 5.1).11 According to the Agency for HealthCare Research and Quality, a registry is “an organized system that uses observational study methods to collect uniform data (clinical and other) to evaluate specified outcomes for a population defined by a particular disease, condition, or exposure and that serves predetermined scientific, clinical, or policy purpose(s).”12 From a scientific perspective, the primary expectation of a welldesigned registry relates to the strength of its external validity. This can be achieved through a comprehensive design that attempts to evaluate heterogeneous populations as compared to the classic method of
FIG. 5.1
Potential roles of clinical registries in health care.
2. MEASURING AND IMPROVING PERFORMANCE
56
5. THE ROLE OF CLINICAL REGISTRIES IN HEALTH CARE
interventional analysis that primarily seeks to evaluate homogenous patient groups.7 As a result of the unique design of clinical registries, the observed health outcomes analyses are often more representative of realworld practice and therefore more likely to draw inferences for broad patient populations and widespread practice settings. This is further enhanced with the popularization of advanced epidemiological methods, including propensity score methods13 and instrumental variable analysis.14 These methods allow providers and researchers to be better equipped with research tools that allow them to mitigate confounding and selection bias.15 Another asset that is particularly helpful in the setting of hospital data relates to the continuous nature of registries.16 They can easily adapt to modernizations in surgical care and facilitate small- and large-scale participation in programs that depend on continuous collection of practice quality data for performance improvement, reporting, and specialtyspecific certification purposes.16 However, certain aspects of registry design and implementation need to be well defined, such as the target population intended to be captured, the covariates and the outcomes of interest to be collected, as well as the type of randomized study that is being modeled with the registry.15 Klaiman and colleagues reported on six key characteristics for a successful clinical registry based on an exhaustive literature search of registries dedicated to the treatment of cancer, cardiovascular disease, maternity care, and joint replacement therapies.17 The following domains were identified: (1) Data standardization: data need to be standardized across multiple clinical practices, given the individual nature of different clinical charting systems. (2) Transparency: registries should be at the forefront of public data sharing, however, in the case of proprietary registries, that is, those owned by specialty societies, members’ concerns when it comes to publishing report cards should be taken into account. In a similar fashion, state-sponsored cancer registries typically avoid generating reports on provider outcomes for fear of compromising patient confidentiality. (3) Provider participation: provider motivation is critical to enhance the versatility and robustness of data and participating practices. (4) Accuracy/completeness of data: robust methods need to be in place in order to minimize the amount of missing data and support meticulous follow-up efforts, particularly for patients who travel out of state for further evaluation or treatment. (5) Financial sustainability: public and federal support is crucial in terms of registry sustainability given the associated budget constraints. For example, cardiovascular registries have found various methods to
2. MEASURING AND IMPROVING PERFORMANCE
VALUE-BASED PURCHASING AND QUALITY REPORTING
57
maintain financial stability, including subsidies from pharmaceutical companies, fees from participating providers, and collected revenues from registry data sale.17 (6) Feedback to providers: sophisticated statistical models have been developed to provide risk-adjusted feedback to providers and make publicly available online risk calculators for predicted length of stay as well as postoperative morbidity and mortality data.
IMPLICATIONS FOR NEUROSURGEONS The PPACA is laden with significant ramifications for surgical practices.3,18 As regulations are updated and new policies are adopted, goaloriented and flexible cooperation is required between governmental bodies, regulatory stakeholders, and the neurosurgical community.16 Neurosurgical practices are increasingly moving in the direction of value-based medicine requiring physicians to advocate and objectively demonstrate the quality and efficiency of their medical practices. This is particularly important given the recent finding that approximately 25% of all diagnostic and therapeutic spine procedures are unnecessary or ineffective.19,20 This reiterates the need that neurosurgeons should define and collect their own metrics and practice data.16
VALUE-BASED PURCHASING AND QUALITY REPORTING The enactment of PPACA has inaugurated an important transition toward value-based purchasing and pay-for-performance payment models.21,22 According to these models, incentives and rewards are granted to hospitals and providers for adhering to quality standards, thus creating behavior modifications through motivation for higher pay. As a result, significant efforts and resources have been expanded to develop surgical quality measures upon which surgical performance can be evaluated and benchmarked. Registries provide the necessary framework to organize data in a meaningful way from a medical and societal perspective in order to support claims received by both public and private payers. Qualified clinical data registries (QCDRs) were introduced by the Centers for Medicare and Medicaid Services CMS in 2014.23 A QCDR is a certification board, registry, or another collaborative effort that compiles and submits medical and clinical measures results under the current reporting requirements with the ultimate goal of fostering progress in the quality of care provided to patients.11,23 QCDR measures are derived from multiple different sources,
2. MEASURING AND IMPROVING PERFORMANCE
58
5. THE ROLE OF CLINICAL REGISTRIES IN HEALTH CARE
including Clinician & Group-Consumer Assessment of Healthcare Providers and Systems, National Quality Forum (NSF)-endorsed measures, and PQRS measures among other cross-disciplinary collaborations.11 However, a QCDR has the capability to develop and include measures that accommodate each specialty care, including neurological surgery. In addition, quality measures are reported in both private and public payer systems and are not limited to Medicare beneficiaries.11 According to the new model, the Merit-based Incentive System (MIPS), penalty payment reductions will reach 4% in 2018, 5% in 2019, 7% in 2020, and 9% in 2021–23.24,25
SUCCESSFUL NATIONAL REGISTRIES One of the largest surgical registries on a global scale is the National Surgical Quality Improvement Program (NSQIP) (Table 5.1). NSQIP incorporates more than 300 variables on patient demographics, comorbidities, intraoperative data as well as 30-day postoperative morbidity and mortality outcomes.26 Currently, the dataset contains data on over 1.7 million patients from more than 500 institutions, 58% of which are large academic institutions.27 One other advantage of the NSQIP registry is that the majority of the data available represents a random sample of all procedures across most surgical specialties.28 Data acquisition from each participating institution is completed by specialized data collectors, who complete extensive training. More importantly, interrater reliability is routinely audited by Surgical Clinical Reviewers with quality control processes that maintain high-fidelity data collection. According to the most recent report, the interrater reliability audit for participating sites demonstrated an overall disagreement rate of roughly 2%.29 One final feature of the NSQIP registry is a helpful tool called TABLE 5.1 Examples of Successful Registries Registry
Discipline
National Surgical Quality Improvement Program
All surgical specialties
STS National Database
Cardiothoracic surgery
Quality Outcomes Database
Spine and cranial surgery
State Cancer Registries
Cancer
CathPCI Registry of the National Cardiovascular Data Registry
PCI devices
Kaiser Permanente National Implant Registries
Devices and Implants in cardiovascular, orthopedics, spine, and endovascular
PCI, percutaneous coronary intervention; STS, Society of Thoracic Surgeons.
2. MEASURING AND IMPROVING PERFORMANCE
THE QUALITY OUTCOMES DATABASE
59
the surgical risk calculator that aims to estimate accurately 30-day complication, mortality, and readmission rates.30 Another representative example of a successful surgical registry implementation with high impact on the quality of care is the National Database of the Society of Thoracic Surgeons (STS).31 It has effectively shown the advantage of national practice data collection programs in advancing patient-centered quality care, with almost 95% of all US thoracic surgery practices currently reporting to this well-established database.32 The STS has implemented risk-adjusted regional and national benchmarks with regard to procedural quality and safety. In fact, the STS has defined STS-specific measures of quality and performance utilizing the database, which have been subsequently endorsed by the NSF.33 This allowed for creating and establishing its own quality and performance measures, while avoiding being forced to comply with outside stakeholdergenerated standards. Approximately 50% of participating sites partake in pioneering the voluntary public reporting system in an attempt to promote accountability in care. Surgeons are also offered the opportunity to leverage the data contained in the registry in order to identify targeted areas for quality improvement. Finally, and perhaps most importantly, the STS National Database has been instrumental in advancing scientific discovery and improving patient care in thoracic surgery. For example, it played a key role in establishing the superiority of internal thoracic artery bypass grafting and promoting the large-scale application of this method as the standard of care in thoracic surgery.34 State cancer registries in the United States have used robust and standardized data collection techniques in order to produce reports on national trends for cancer incidence, type, and mortality. They are funded by multiple federal organizations, including the National Program of Cancer Registries, administered by the Centers for Disease Control and Prevention, and the Surveillance, Epidemiology and End Results Program, administered by the National Cancer Institute. California cancer registry is an exemplary cancer registry that has collected comprehensive data on over 7 million cases of cancer among patients living in California diagnosed since 1988, with over 175,000 new cases added each year.35 Case ascertainment is estimated to be 99% complete.
THE QUALITY OUTCOMES DATABASE The American Association of Neurological Surgeons, in partnership with the Congress of Neurological Surgeons, the Society of Neurological Surgeons, and the American Board of Neurological Surgery, launched in March 2012 the Quality Outcomes Database (QOD) (formerly known as the National Neurosurgical Quality Outcomes Database-N2QOD).36,37
2. MEASURING AND IMPROVING PERFORMANCE
60
5. THE ROLE OF CLINICAL REGISTRIES IN HEALTH CARE
This database is a national, multiinstitutional, prospective registry that captures various preoperative patient characteristics as well as prospective surgical and patient reported outcomes at 3- and 12-months postoperatively.38 The database was created to establish a robust mechanism for risk-adjusted benchmarking, real-time analysis of individual morbidity and clinical outcomes, comparative effectiveness research, as well as evidence-based advancements in surgical procedures and facilitation of collaborative trials.36,37 Data extractors and coordinators undergo extensive training on data entry and standard operating procedures outlined in the QOD infrastructure. The first six patients who meet the prespecified eligibility criteria are enrolled into the registry via a standard sampling framework after being identified from the weekly posted surgical schedule.7 Each of the six patients are enrolled either in person during their initial clinic visit or by phone where their health status is assessed via interview. Data quality is achieved through automated and manual methods at Vanderbilt University Institute for Medicine and Public Health. Participating sites are informed weekly for missing data and periodic surgeon-ledself-audits and random audits are performed to ensure data completeness and accuracy.
CURBING THE RISING COSTS Health care systems are in a constant struggle with rising costs. Within this context, registries can also aid in identifying the most effective care paradigms and refining surgical treatments to maximize success. A multinational study querying thirteen registries from five countries (Australia, Denmark, Sweden, the United Kingdom, and the United States) suggested that by making outcomes data transparent to both the public and practitioners alike, well-managed registries allow providers to engage in constant learning with the ability to identify and disseminate best clinical practices.39 The authors estimated that the implementation of a hip replacement registry in the United States, comparable to the Swedish Hip Arthroplasty Register, would result in a reduction of $2 billion out of a total of $24 billion in 2015 by reducing reoperation rates in replacing or repairing hip prostheses.
THE RANDOMIZED REGISTRY TRIAL As mentioned above, RCT performance and completion is hampered by the design complexity and the immense costs associated.40 To overcome such challenges, health services researchers have recently began exploring the possibility of leveraging the infrastructure and the resources provided
2. MEASURING AND IMPROVING PERFORMANCE
THE RANDOMIZED REGISTRY TRIAL
61
by registries with other study designs with the aim of producing robust, “hybrid” approaches for evidence-based development. A high-quality registry retains many RCT traits, such as high follow-up rates, utilization of patient outcomes, prospective data collection, and data quality validation while mitigating sources of bias through various advanced study designs. This registry-based randomized trial, also called the randomized registry trial (RRT), attempts to expedite and enhance patient enrollment, minimize cost, and address the generalizability of findings, while maintaining the statistical rigor of randomization (Table 5.2).41–43 It represents an innovative TABLE 5.2 Comparison of Characteristics Between Conventional Randomized Controlled Trials, Registries, and Randomized Registry Trials
Characteristic
Conventional randomized controlled trial
Registry studies
Randomized registry trial
Blinding
Possible
Not possible
Possible
Randomization
Possible
Not possible
Possible
Minimize bias
Yes
No
Yes
Minimize confounding
Yes
No
Yes
Applicability
Limited
Real-world representation
Real-world representation
Cost
Expensive
Cost-effective solution
Leverage existing registry infrastructure
Amount of missing data
Minimal
Frequent
Minimal
Data quality
High
Variable and questionable
High
Other strengths
Considered the gold-standard design
Large sample size allows for identification of rare events Hypothesis generating
Flexible linkage to registry Compare enrolled to nonenrolled patients
Other limitations
Often funded by Pharma Long time to plan and implement
Data collection limited to registry capabilities Link to causality limited
Limited capacity for collection of biospecimens, pharmacodynamic, and pharmacokinetic indices
Primary purpose
Research
Research and quality reporting
Research and quality reporting
2. MEASURING AND IMPROVING PERFORMANCE
62
5. THE ROLE OF CLINICAL REGISTRIES IN HEALTH CARE
and “disruptive technology” in order to advance comparative effectiveness research and provide Level 1—prognostic and Level 2—effectiveness evidence.44 The first RRT originated in Sweden, where 7244 patients across three countries in more than 30 sites were accrued in less than 3 years.42 The estimated cost of the trial was remarkably as low as $50 per patient. Similarly, the Study of Access Site for Enhancement of Percutaneous Coronary Intervention (SAFE-PCI) for Women Trial was the first RRT in the US conducted in the United States.45 This open-label study was conducted in collaboration with the National Institutes of Health National Cardiovascular Data Registry Cath-PCI Registry platform. Women were randomized to percutaneous coronary intervention via radial versus femoral access to compare procedural bleeding complications at high volume hospitals, where participating physicians were required to be proficient at both radial and femoral access. Recruitment efficiency and overall success was attributed to the very nature of RTT design with accrual reaching 99.2% in just 22 months. 45 Reviews of this design identified key learning conclusions. According to Li and colleagues, the following lessons should be considered as this novel and promising trial design is further developed: (1) The question needs to fit the registry purposes and not the opposite. (2) The RRT design should not replace traditional RCTs, but rather offer an alternative strategy to provide answers to questions that would otherwise remain unanswered due to financial and time constraints. (3) RRTs are by definition pragmatic in their nature; therefore, they are more appropriate to investigate the safety and effectiveness of medications, devices, and treatments widely accepted in real-world settings but still variable in their use.43,46 Asking a well-defined, specific question is therefore paramount. (4) The primary aim and methods of the registry should not be affected by the RRT conduct. Balance needs to be maintained with regard to the workload and the workflow between the RRT and the registry. (5) High data quality and rigorous statistical analyses are pivotal, as accuracy of the findings is heavily dependent on data quality. (6) Strong ethos collaboration is vital and the needs and limitations of each stakeholder should be kept in mind.
DEVICE REGISTRIES Postmarket surveillance of medical devices poses unique challenges compared to drugs and biologics due to device diversity and complexity, attributes which are further complicated by the iterative nature of medical
2. MEASURING AND IMPROVING PERFORMANCE
CURRENT CHALLENGES IN REGISTRY IMPLEMENTATION AND DATA COLLECTION
63
product development and the learning curve associated with technology adoption.47Device-specific registries provide detailed information regarding patient care, procedural interventions, and device implantation not routinely collected by electronic health records and claims data. As unique device identifiers (UDI) become increasingly incorporated into electronic health information, the role of clinical registries in postmarket surveillance will continue to evolve. The Food and Drug Administration (FDA) has recently announced that it is willing to partner with institutions and, more specifically, surgeons across the county in order to leverage existing integrated data systems to monitor new technologies, evaluate associated patient outcomes, and facilitate implementation of best clinical practices.47 The value of such registries was recently brought to light with the publication of a registry-based, active surveillance of an implantable vascularclosure device safety trial. The trial investigated potential safety signals with regard to associated vascular complications among recipients of the device, with initial alerts occurring within the first 12 months of monitoring.48 Finally, Kaiser Permanente has developed and maintained longstanding device registries that have included cardiac devices, endovascular stents as well as orthopedic and spinal implants.49 These registries have been used to describe device information related to device recipients, outcomes data, and overall success of each device.
CURRENT CHALLENGES IN REGISTRY IMPLEMENTATION AND DATA COLLECTION As with other observational data sources, registries are also subject to several limitations. Owing to the very nature of data collection and analysis, there will always be challenges in eliminating all sources of bias that might arise. Challenges in ensuring the highest possible degree of data quality can be attenuated by enlisting a variety of novel analytical tools to lessen the probability of systematic error as well as errors in inference.7 The collection burden for longitudinal data represents another challenge. Efforts need to be made during the design stage to attain the least amount of user influence for data collection with the lowest number of prespecified variables without oversimplifying the data. Natural language processing of medical records, operative notes, and radiology reports constitutes a set of rapidly progressing techniques that may play a significant role in aiding data collection.50 According to the definition of value, comprehensible cost-related data are also needed to be captured and evaluated when registries are implemented. For that purpose, QOD is now engaging in pilot programs to merge its clinical outcomes data with administrative cost-related data sources.7 In addition, the need for a written informed consent form for
2. MEASURING AND IMPROVING PERFORMANCE
64
5. THE ROLE OF CLINICAL REGISTRIES IN HEALTH CARE
patients included into a registry is still a matter of debate between registry administrators and Institutional Review Board (IRB) committees, as it is primarily conducted for quality improvement and not research purposes.51 From a regulatory perspective, contention usually involves whether the registry is subject to the jurisdiction of the Common Rule (45 CFR 46) and whether data collection methods comply with the Privacy Rule.51 As such, a detailed outline of the appropriate use of protected health information, operational procedure, patient interaction, and reporting methods is routinely made available to sites for submission to their institution’s quality improvement office or IRB. Ultimately, registry data must apply to clinical practice to advance value-based health care. Evolving mechanisms for provider education and meaningful application of the surgical and functional outcomes data are critical. Most importantly, the neurological surgery profession, unlike the STS, still lacks specialty-specific quality measures endorsed by the NSF.11 National neurological surgery administration and leadership are now working on developing specialty-specific,NSF-endorsed measures for public and private reporting purposes. Finally, concerns have been previously expressed with regard to the unintended impairment of physician workflow and time of interaction with patients as a result of electronic medical record (EMR) adoption.16,52,53 Significant initiatives have been spearheaded in multiple clinical practices across the country to adopt an EMR system. Multiple teams have reported new ways to decrease the burden of registry-related manual data entry.54 Azad and colleagues recently shared their institutional experience with an EMR-integrated quality of life outcomes registry for patients who undergo spine surgery.54 Using a systematic approach to collect spine-related quality of life data within the EMR system, they developed a clinical dashboard that could track more than 25 patient variables without posing any significant impact on the patient’s clinic visit or surgeon’s clinical workflow.
CONCLUSIONS In the current era of an increasingly pay-for-performance health care industry, there is growing interest in producing objective data to determine the quality of care and inform the allocation of health care resources. Given the ongoing national debate centered around health care reform as well as the rapid cost and growth of surgical procedures for a multitude of conditions, prospective clinical registries, such as QOD, have emerged as a promising source of a diverse set of data. In contrast to RCT methods, which are primarily employed for research, registries can be used for both quality improvement and research. More importantly, registries represent
2. MEASURING AND IMPROVING PERFORMANCE
REFERENCES
65
sustainable solutions for characterizing real-world surgical care while identifying large-scale improvement opportunities. Such initiatives will ensure that surgical outcomes are appropriately measured and integrated into policy. More importantly, current metrics mandate a holistic approach to the particular needs of patients. As such, a multifaceted outcomes evaluation system that is applicable to a wide spectrum of disorders along with patient reported outcomes and cost data will constitute the linchpin of future registry collection efforts and enhance the overall value of US health care.
References 1. National Center for Health Statistics (US). Health, United States, 2013: With Special Feature on Prescription Drugs. Hyattsville, MD: National Center for Health Statistics (US); 2014. 2. Reilly BM, Evans AT. Much ado about (doing) nothing. Ann Intern Med. 2009;150(4): 270–271. 3. Menger RP, Guthikonda B, Storey CM, Nanda A, McGirt M, Asher A. Neurosurgery value and quality in the context of the affordable care act: a policy perspective. Neurosurg Focus. 2015;39(6):E5. 4. Sackett DL. Evidence-based medicine. Spine. 1998;23(10):1085–1086. 5. Guyatt GH, Oxman AD, Kunz R, et al. What is “quality of evidence” and why is it important to clinicians? BMJ. 2008;336(7651):995–998. 6. Guyatt G, Oxman AD, Akl EA, et al. GRADE guidelines: 1. Introduction—GRADE evidence profiles and summary of findings tables. J Clin Epidemiol. 2011/4;64(4):383–394. 7. Asher AL, Parker SL, Rolston JD, Selden NR, McGirt MJ. Using clinical registries to improve the quality of neurosurgical care. Neurosurg Clin N Am. 2015;26(2):253–263. ix–x. 8. Booth CM, Tannock IF. Randomised controlled trials and population-based observational research: partners in the evolution of medical evidence. Br J Cancer. 2014;110(3): 551–555. 9. Jaffe S. 21st century cures act progresses through US congress. Lancet. 2015;385 (9983):2137–2138. 10. Sherman RE, Anderson SA, Dal Pan GJ, et al. Real-world evidence—what is it and what can it tell us? N Engl J Med. 2016;375(23):2293–2297. 11. Bekelis K, McGirt MJ, Parker SL, et al. The present and future of quality measures and public reporting in neurosurgery. Neurosurg Focus. 2015;39(6):E3. 12. Gliklich R, Dreyer N, Leavy M, eds. Registries for Evaluating Patient Outcomes: A User’s Guide. 3rd ed. Two vols. Rockville, MD: Agency for Healthcare Research and Quality; 2014. (Prepared by the Outcome DEcIDE Center [Outcome Sciences, Inc., a Quintiles company] under Contract No. 290 2005 00351 TO7.) AHRQ Publication No. 13(14)EHC111. http://www.effectivehealthcare.ahrq.gov/registries-guide-3.cfm. 13. Rosenbaum PR, Rubin DB. The central role of the propensity score in observational studies for causal effects. Biometrika. 1983;70(1):41–55. 14. Martens EP, Pestman WR, de Boer A, Belitser SV, Klungel OH. Instrumental variables: application and limitations. Epidemiology. 2006;17(3):260–267. 15. Harbaugh RE. Rupture risk of unruptured aneurysms: beyond size and shape. JNS. 2016;124:286–287. 16. Selden NR, Ghogawala Z, Harbaugh RE, Litvack ZN, McGirt MJ, Asher AL. The future of practice science: challenges and opportunities for neurosurgery. Neurosurg Focus. 2013;34(1):E8.
2. MEASURING AND IMPROVING PERFORMANCE
66
5. THE ROLE OF CLINICAL REGISTRIES IN HEALTH CARE
17. Klaiman T, Pracilio V, Kimberly L, Cecil K, Legnini M. Leveraging effective clinical registries to advance medical care quality and transparency. Popul Health Manag. 2014;17(2):127–133. 18. Kim DH, Dagi TF, Bean JR. Neurosurgical practice in transition: a review. Neurosurgery. 2017;80(4S):S4–S9. 19. Committee on the Learning Health Care System in America, Institute of Medicine. In: Smith M, Saunders R, Stuckhardt L, JM MG, eds. Best Care at Lower Cost: The Path to Continuously Learning Health Care in America. Washington, DC: National Academies Press; 2014. 20. Epstein NE, Hood DC. “Unnecessary” spinal surgery: a prospective 1-year study of one surgeon’s experience. Surg Neurol Int. 2011;2:83. 21. VanLare JM, Conway PH. Value-based purchasing—national programs to move from volume to value. N Engl J Med. 2012;367(4):292–295. 22. Blumenthal D, Jena AB. Hospital value-based purchasing. J Hosp Med. 2013;8(5):271–277. 23. Center for Medicare and Medicaid Services, QCDRs and Physician Compare. 2016. https://www.cms.gov/Medicare/Quality-Initiatives-Patient-Assessment-Instruments/ physician-compare-initiative/QCDRs-and-Physician-Compare.html. Accessed May 27, 2017. 24. Swartz KR, Cheng JS. Anticipated changes in spine practice with advancing Center for Medicare and Medicaid Services-Required Changes. Neurosurgery. 2017;80(4S):S28–S33. 25. Center for Medicare and Medicaid Services, MACRA: MIPS & APMs, 2017. https:// www.cms.gov/medicare/quality-initiatives-patient-assessment-instruments/valuebased-programs/macra-mips-and-apms/macra-mips-and-apms.html. Accessed May 27, 2017. 26. Kerezoudis P, McCutcheon BA, Murphy M, et al. Predictors of 30-day perioperative morbidity and mortality of unruptured intracranial aneurysm surgery. Clin Neurol Neurosurg. 2016;149:75–80. 27. McCutcheon BA, Kerezoudis P, Porter AL, et al. Coma and stroke following surgical treatment of unruptured intracranial aneurysm: an ACS-NSQIP study. World Neurosurg. 2016;(April). https://doi.org/10.1016/j.wneu.2016.04.039. 28. Pitt HA, Kilbane M, Strasberg SM, et al. ACS-NSQIP has the potential to create an HPBNSQIP option. HPB. 2009;11(5):405–413. 29. American College of Surgeons. User Guide for the 2014 ACS NSQIP Participant Use Data File (PUF); 2015. 30. Cohen ME, Ko CY, Bilimoria KY, et al. Optimizing ACS NSQIP modeling for evaluation of surgical quality and risk: patient risk adjustment, procedure mix adjustment, shrinkage adjustment, and surgical focus. J Am Coll Surg. 2013;217(2):336–346e1. 31. Shahian DM, Jacobs JP, Edwards FH, et al. The Society of Thoracic Surgeons National Database. Heart. 2013;99(20):1494–1501. 32. Stewart JM. Abstraction techniques for the STS National Database. J Extra Corpor Technol. 2016;48(4):201–203. 33. Shahian DM, Edwards FH, Ferraris VA, et al. Quality measurement in adult cardiac surgery: part 1—conceptual framework and measure selection. Ann Thorac Surg. 2007;83(4): S3–S12. 34. Lytle BW, Blackstone EH, Sabik JF, Houghtaling P, Loop FD, Cosgrove DM. The effect of bilateral internal thoracic artery grafting on survival during 20 postoperative years. Ann Thorac Surg. 2004;78(6):2005–2012 [discussion 2012-2014]. 35. Al-Refaie WB, Vickers SM, Zhong W, Parsons H, Rothenberger D, Habermann EB. Cancer trials versus the real world in the United States. Ann Surg. 2011;254(3):438–442. [discussion 442-443]. 36. Alliance N. http://www.neuropoint.org/en/NPA-N2QOD/. Accessed 27 May 2017. 37. McGirt MJ, Speroff T, Dittus RS, Harrell Jr. FE, Asher AL. The National Neurosurgery Quality and outcomes database (N2QOD): general overview and pilot-year project description. Neurosurg Focus. 2013;34(1):E6.
2. MEASURING AND IMPROVING PERFORMANCE
REFERENCES
67
38. Asher AL, Speroff T, Dittus RS, et al. The National Neurosurgery Quality and outcomes database (N2QOD): a collaborative North American outcomes registry to advance valuebased spine care. Spine. 2014;39(22 Suppl. 1):S106–S116. 39. Larsson S, Lawyer P, Garellick G, Lindahl B, Lundstr€ om M. Use of 13 disease registries in 5 countries demonstrates the potential to use outcome data to improve health care’s value. Health Aff. 2012;31(1):220–227. 40. Liu JB, D’Angelica MI, Ko CY. The randomized registry trial: two birds, one stone. Ann Surg. 2017;265(6):1064–1065. 41. James S, Rao SV, Granger CB. Registry-based randomized clinical trials—a new clinical trial paradigm. Nat Rev Cardiol. 2015;12(5):312–316. 42. Lauer MS, D’Agostino Sr. RB. The randomized registry trial—the next disruptive technology in clinical research? N Engl J Med. 2013;369(17):1579–1581. 43. Li G, Sajobi TT, Menon BK, et al. Registry-based randomized controlled trials—what are the advantages, challenges, and areas for future research? J Clin Epidemiol. 2016;80:16–24. 44. Ball CM, Phillips RS. Evidence-Based On-Call. Edinburgh: Churchill Livingstone; 2001. 45. Hess CN, Rao SV, Kong DF, et al. Embedding a randomized clinical trial into an ongoing registry infrastructure: unique opportunities for efficiency in design of the study of access site for enhancement of percutaneous coronary intervention for women (SAFE-PCI for women). Am Heart J. 2013;166(3):421–428. 46. Registry Trials. Clinical Trials Transformation Initiative. https://www.ctti-clinicaltrials. org/projects/registry-trials. Published May 26, 2016. Accessed May 26, 2017. 47. Food US, Administration D, Others. Strengthening Our National System for Medical Device Postmarket Surveillance. Silver Spring, MD: Center for Devices and Radiological Health; 2012. 48. Resnic FS, Majithia A, Marinac-Dabic D, et al. Registry-based prospective, active surveillance of medical-device safety. N Engl J Med. 2017;376(6):526–535. 49. National Implant Registries Kaiser Permanente. https://national-implantregistries. kaiserpermanente.org/about. Accessed 26 May 2017. 50. Zhou L, Hripcsak G. Temporal reasoning with medical data—a review with emphasis on medical natural language processing. J Biomed Inform. 2007/4;40(2):183–202. 51. Asher AL, McGirt MJ, Glassman SD, et al. Regulatory considerations for prospective patient care registries: lessons learned from the National Neurosurgery Quality and outcomes database. Neurosurg Focus. 2013;34(1):E5. 52. McWilliams JM, Song Z. Implications for ACOs of variations in spending growth. N Engl J Med. 2012;366(19):e29. 53. Park SY, Lee SY, Chen Y. The effects of EMR deployment on doctors’ work practices: a qualitative study in the emergency department of a teaching hospital. Int J Med Inform. 2012;81(3):204–217. 54. Azad TD, Kalani M, Wolf T, et al. Building an electronic health record integrated quality of life outcomes registry for spine surgery. J Neurosurg Spine. 2016;24(1):176–185.
2. MEASURING AND IMPROVING PERFORMANCE