Quality measurement and improvement in liver transplantation

Accepted Manuscript Public Health Quality measurement and improvement in liver transplantation Amit K. Mathur, Jayant Talwalkar PII: DOI: Reference:

S0168-8278(18)30173-9 https://doi.org/10.1016/j.jhep.2018.02.034 JHEPAT 6904

To appear in:

Journal of Hepatology

Received Date: Revised Date: Accepted Date:

4 August 2017 21 February 2018 27 February 2018

Please cite this article as: Mathur, A.K., Talwalkar, J., Quality measurement and improvement in liver transplantation, Journal of Hepatology (2018), doi: https://doi.org/10.1016/j.jhep.2018.02.034

This is a PDF file of an unedited manuscript that has been accepted for publication. As a service to our customers we are providing this early version of the manuscript. The manuscript will undergo copyediting, typesetting, and review of the resulting proof before it is published in its final form. Please note that during the production process errors may be discovered which could affect the content, and all legal disclaimers that apply to the journal pertain.

Quality measurement and improvement in liver transplantation Amit K. Mathur MD MS1,2, Jayant Talwalkar MD, MPH2,3 1

Division of Transplant Surgery, Mayo Clinic, Phoenix, AZ USA , 2Robert D. and Patricia E. Kern Center for the Science of Health Care Delivery, Mayo Clinic, 3Division of Gastroenterology and Hepatology, Mayo Clinic, Rochester, MN USA,

Corresponding Author Amit K. Mathur, MD MS Consultant, Division of Transplant Surgery Mayo Clinic Arizona Associate Professor of Surgery Mayo Clinic School of Medicine 5777 East Mayo Boulevard Phoenix, AZ 85054 USA. Tel: + 1 480-342-0437 Fax: + 1 480-342-2324 E-mail: [email protected]

Abstract There is growing interest in the quality of health care delivery in liver transplantation. Multiple stakeholders, including patients, transplant providers and their hospitals, payers, and regulatory bodies have an interest in measuring and monitoring quality in the liver transplant process, and understanding differences in quality across centers.

This article aims to provide an overview of quality measurement and regulatory issues in liver transplantation performed within the United States. We review how broader definitions of health care quality should be applied to liver transplant care models, outline the status quo including regulatory agencies, public reporting mechanisms, and requirements around quality assurance and performance improvement (QAPI) activities. Additionally, we further discuss unintended consequences and opportunities for growth in quality measurement. Quality measurement and the integration of quality improvement strategies into liver transplant programs holds significant promise but multiple challenges to successful implementation must be addressed in order to optimize value.

Introduction Quality measurement of health care delivery is essential to understanding and improving its value. Understanding the quality of health care delivery, the role of quality measures, and value of service has implications for every stakeholder in health care including patients, providers, hospitals, payers, and regulatory bodies. Quality measurement and quality improvement are inherently tied together – one cannot improve what cannot be measured. The measurement of quality of health care has been a science with continued innovation over the last 50 years. In 1966, Avedis Donabedian proposed that outcomes of health care are determined both by structural factors and processes of care in the delivery system (1). Quality measurement is therefore related to measuring 1) the structure of health care – capacity, personnel, expertise, available services within a health care unit, and 2) care processes, which are the clinical decisions, judgment and other related activity in the care of the patient (2). Quality improvement is a continuous, stakeholder-driven activity aimed at improving care and outcomes. Quality improvement programs must measure actionable data to produce metrics, and employ them in structured projects to develop structural resources and enrich processes of care (3).

In the United States, there are now nearly 8,000 liver transplants performed in more than 100 centers (4). While the practice of liver transplant is evolving through research and clinical care improvements, there is a growing role for the assessment and measurement of quality of liver transplant care delivery and its value. Transplant programs employ, both by training and regulation, common processes of patient evaluation, pre-transplant clinical care, surgical care, and post-transplant management, which carries the ability to

benchmark performance. Quality measurement efforts are driven by governmental and non-governmental regulatory and payer interests, demands of transparency by patients, and genuine provider interest in quality improvement. However, quality improvement in transplantation is a relatively new phenomenon on the global stage. A literature search for “liver transplantation”, “quality of care”, “quality improvement”, “QAPI”, “process improvement” reveals virtually no relevant articles in English. “Audit” (the term used by the National Health Service in the UK for quality measurement) and “liver transplantation” reveals no published data. Similar searches combined with “Eurotransplant,” other European countries, and Asian countries yield no data. Thus, the evolution of liver transplant quality measurement and improvement has been driven largely by US-based initiatives.

In this context, the intent of this manuscript is to provide a comprehensive evaluation of the current state of quality measurement and improvement in liver transplantation in the United States. This article will summarize the regulatory basis for quality measurement, tenets of public reporting of liver transplant program performance, the role of quality assurance and performance improvement programs within transplant centers, and future directions for quality measurement in the field.

The Role of Transplant Regulation in Liver Transplant Quality It is necessary to understand the structure of transplant regulation in the United States in order to understand how quality issues in liver transplantation have evolved (Table 1). Transplantation is the most regulated field in clinical medicine in the country. Transplant

regulation has been established over time through legislation passed through Congress. The National Organ Transplant Act of 1984 established the Organ Procurement and Transplant Network (OPTN) and the Scientific Registry of Transplant Recipients (SRTR) and initiated federal regulation of transplant services. The Final Rule, established by the Department of Health and Human Services in 2000, further outlined the landscape of transplant regulation (5). The Final Rule is the primary overarching principle that guides deceased donor organ procurement and transplantation practices in the United States. It delineates the value and philosophy behind organ allocation and distribution rules and also established a framework for “reviews, evaluation, and enforcement” of member organizations, as well as “record maintenance and [public] reporting” requirements (6). The ongoing evolution of national transplant policies begins with the precepts outlined in the Final Rule. These rules also placed the authority for the identification of contractors with the Health Resources and Services Administration (HRSA), a division of the Department of Health and Human Services, which oversees the contracts for the OPTN and the SRTR.

The Final Rule states that designated transplant programs must be in good standing with the Centers of Medicare and Medicaid Services (CMS). Importantly, CMS established Conditions of Participation for transplant programs in 2007, as the primary payer of transplant services in the United States (7). These regulatory bodies have codified the policies to which all solid organ transplant programs must adhere, and the requirements hospitals must meet to initiate transplant programs (7, 8). These requirements fulfill an important early role in delivering high quality liver transplant care – the rules begin to

outline the structural milieu for a liver transplant program, a critical element in the Donabedian model of quality. The structural requirements for liver transplant programs outlined by CMS and the OPTN are distinct but linked. One of the first rules outlined in the membership requirements of liver transplant program to the OPTN requires that programs must be approved for CMS reimbursement (or be a program in the Department of Veterans Affairs or another federal entity). Other structural requirements outlined in these policies include minimum volume requirements, physician, surgeon, and other staffing requirements, operative/perioperative facilities and hospital capacity, and other expectations (8).

The legal basis for transplant regulation also includes regulations on the need for quality evaluations of transplant programs within the United States. Aside from these evaluations being made available for regulatory oversight, these policies further mandated that quality reports be publicly available, and that programs monitor and selfcorrect their own performance through QAPI activities. With these rules, CMS and the OPTN were the first stakeholders to require transplant quality reporting. They continue to play a critical role in defining, monitoring and improving liver transplant quality.

Defining Quality in Liver Transplantation What defines quality in the liver transplant process? Formal comprehensive definitions for what defines quality in this field are currently lacking, although it is a central focus of regulatory bodies in the U.S.(9). As alluded to above, Donabedian’s model can be extended to liver transplantation by measuring the structural resources within a transplant

system and the processes of care that emerge in that structural milieu (1). For liver transplant programs, structural resources are defined by the capacity of the transplant hospital, transplant and non-transplant specific hospital personnel, health care services within that hospital, physical resources to manage pre-transplant and post-transplant patients, operating room facilities, financial arrangements related to transplant services and beyond, and other factors. The processes of care for a liver transplant program are the routines employed in the care of liver transplant patients in each phase of transplant care. However, defining liver transplant quality goes beyond the structure-processoutcome triad. This definition of quality was further expanded by the National Academy of Medicine (formerly the Institute of Medicine) in their 2001 report Crossing the Quality Chasm, by outlining six aims of a high quality health system: 1) Equitable 2) Effective 3) Efficient 4) Safe 5) Timely 6) Patient-Centered (10). These general principles should help guide the maturation and improvement of liver transplant programs within hospitals, and transplant programs must exert these values on all phases of transplant care, including 1) pre-transplant care, which includes multi-disciplinary management of endstage liver disease and its complications, liver transplant evaluation and wait-listing, 2) transplant care – donor and recipient matching, surgical and perioperative care 3) posttransplant care – post-operative and longitudinal management of the complex liver transplant recipient. Liver transplant programs should aim to be high quality health systems in this context, with defined quality metrics specifically linked to these ideals in each phase of care.

While transplantation was among the first medical fields to formally adopt quality

measurement in the US, the trend has extended into multiple areas of modern surgical practice and chronic disease management – quality measures are now everywhere. The effects of these efforts have been studied substantially in surgery. Over the last 15 years, significant efforts have been underway to broadly engage surgeons in measurement of their own outcomes and providing feedback on their performance at the institutional level (11-16). For nearly every surgical specialty area, including general, cardiovascular, bariatric, pediatric, trauma, as well as transplant, quality reporting on outcomes including surgical complications and mortality are a part of the clinical practice rubric. However, critics of quality reporting state it has little effect on outcomes (17-19). These studies analyzed utilization of the National Surgical Quality Improvement Program (NSQIP) by hospitals, and concluded that reporting quality metric data alone is insufficient to improve quality – action must be taken on this data. This has been confirmed in chronic disease management models of cirrhosis, where baseline data was used to justify an clinical process intervention, and outcomes were compared to post-intervention cohorts (20-23). These studies, based on patients with chronic liver disease, have demonstrated improved clinical outcomes – including better adherence, less hospitalizations, and lower mortality – through monitoring, and the creation of clinically-oriented quality improvement projects using quality improvement science (2).

If measurement is the core of performance improvement, an important step in improving liver transplant quality is defining the bounds of measurement (2). There are several challenges surrounding the metric selection to monitor quality in clinical liver transplant programs (24). The first is identifying if measures are clinically meaningful. No

transplant clinician will argue that “big” outcome measures in each phase of transplant care, i.e, wait-list mortality, volume of transplants, and post-transplant graft and patient survival are unimportant; strong cumulative performance in these areas inherently demonstrates efficient, effective, safe, and timely care. However, other quality metrics could be criticized as a matter of their relative scale, i.e. does the surgical site infection rate among high MELD patients who receive a liver transplant matter if the patients are alive? This example also raises the question of whether “big picture” metrics matter as much as process measures of dozens of small steps in the transplant process. In reality, fortunately, quality metric selection and monitoring in liver transplant program is discretionary in the United States, and clinicians therefore have an opportunity to select what they believe is clinically meaningful and robust in capturing the elements of a high quality transplant system for themselves. Regulators such as the OPTN and CMS, however, emphasize the importance of post-transplant graft and patient survival almost exclusively

The selection of these measures does have its own set of challenges, primarily related to external validation. In quality measurement, benchmarks are required to evaluate performance. These may be based on external sources (performance in other programs) or internal sources ( historical performance within the program), Some metrics are easier to benchmark in liver transplantation than others.

”Big picture” metrics like survival

are easily benchmarked based on public reporting of all US liver transplant programs. However, , it can be more difficult to identify more nuanced outcome measures due to a lack of widely accepted, externally valid benchmarks for each phase of transplant care.

These may includ a lack of benchmarks for outcomes such as pre-transplant hospitalization rates, rates of vascular and biliary complications, wound complications, post-transplant readmissions, and so on. The literature has historically been lacking in this area, particularly around process measures. However, process metrics have been proposed for chronic liver disease care which could also serve as pre-liver transplant quality metrics (2, 25). Fortunately, several centers internationally are recognizing the need for improved clinical benchmarking in the perioperative and postoperative phases of liver transplantation (26). Long term process and outcome measures have not been heavily emphasized to date.

From a liver transplant program leadership perspective, the identification of quality measures within these domains can be a challenge because many clinicians question their validity and their importance. High quality outcomes are related to structural context and process of care delivery (1). For transplant programs, the structural factors that can play a role in transplant outcomes are delivered by the parent hospital – personnel, capacity, complementary services, health care equipment, financial support, and other factors. Some of these factors are measurable through administrative data sources, but many are not. These often serve as a focal point of discussion when discussing transplant program growth between transplant leaders and hospital administration. Experienced transplant leaders understand what structural resources are required to achieve quality outcomes through training and years of clinical work. Knowing and tracking these structural data helps enhance patient safety because they may shed light on process problems, areas of vulnerability in safe care delivery, and build a case for more resources as the program

evolves. Process of care measures within transplant phases may help programs track what it deems to be important care processes and those valued by regulators, but physician buy-in on these measures may not be optimal unless there is a direct link to patient outcomes. For example, in liver transplantation, a common OPTN- mandated process measure is the timely verification of ABO blood type compatibility in the operating room prior to induction of patient anesthesia for the transplant. This process measure is tracked ostensibly to ensure that a patient does not receive an incompatible organ. However, the transplant of incompatible organs is so rare in modern clinical transplantation in the US due to multiple overlapping processes. It may be difficult for clinicians to see the value of a second time-constrained layer of verification that must occur between induction of anesthesia and organ implantation, that may be “out of compliance” if it is added to the electronic medical record a minute too early or documented too late. In this example, one limitation of a process measures is that compliance with its measurement does not always mean better care. The process measure does not capture the entirety of the ABO verification process, ABO incompatible transplants events are rare, and multiple checks add complexity to clinical work flows without a proven benefit to the patient.

Quality Reporting in Liver Transplantation

Quality assessments are predicated on the availability of data and the creation of standards for performance. CMS (a division of the Department of Health and Human Services) and the OPTN (a contract under HRSA, another division of the Department of Health and Human Services) both have defined standards for clinical performance in liver

transplantation (and for other solid organs). These standards are based on risk-adjusted outcomes derived from nationwide liver transplant activity. This risk-adjustment is driven by donor, recipient, and transplant data elements (collected from programs by the OPTN) and computed using up-to-date statistical methods by the SRTR (another contract under HRSA). The SRTR serves several functions, but one of its most critical regulatory functions is the creation of publicly-available program-specific reports (PSRs) of liver transplant outcomes. These PSRs outline whether programs have deviated from expected standards of performance, after risk-adjustment. The SRTR creates PSRs every 6 months based on an 18-month rolling cohort of patients. The report contains metrics on each phase of transplant care. Pre-transplant phase metrics include waitlist demographics, mortality, additions, removals, as well as transplant rates. Transplant phase metrics include MELD at transplant, time to transplant, median hospital length of stay, and posttransplant outcomes include 1-month, 1-year, and 3-year risk-adjusted graft and patient survival. PSRs provide metrics on all phases of transplant care relative to what is expected, what is observed in the center’s donation service area, region, and nationally. The most highlighted metrics on a liver transplant PSR are outcome based: 1-year riskadjusted graft and patient survival. These are used by both CMS and the OPTN to define the standards for high and low performing transplant programs. Programs are informed if they are performing better than expected, as expected, or worse than expected for each PSR. These flagging criteria vary by regulatory body - CMS and the OPTN have different risk-adjusted thresholds of high and low program performance relative to what is expected from that program (Table 2).

An important and recurrent criticism of liver transplant quality reports is that the riskadjustment algorithm does not fully capture the true clinical risk of some patients, and could therefore lead to spurious flagging of liver transplant programs. The riskadjustment algorithm currently applied was rigorously evaluated by the SRTR and HRSA and has evolved over time to a Bayesian framework that assumes that the baseline condition is that all programs are average, with few positive or negative outliers in evaluating graft losses and patient deaths (27). Risk adjustment, notably, is based on the data elements applied in the model-building process – stronger data elements that are truly linked to the outcome will produce a strong prediction model. Clinical risk capture is based on selection of valid and meaningful clinical variables. From a regulatory standpoint, the data elements reported to the SRTR for risk-adjustment arise from what the OPTN chooses to collect from programs. The OPTN policies governing what data are collected can be modified by transplant programs themselves, albeit through a relatively laborious process. In the event of regulatory flagging by CMS, risk factors affecting a particular program that are not accounted for by current methods can be addressed through the ‘mitigating factors’ process (28). These revision mechanisms are still often criticized, as they do not keep pace with the evolution of clinical care.

Identification and Public Reporting of Liver Transplant Program Performance CMS and the OPTN play critical roles in maintaining liver transplant quality. Both organizations have made overtures that their roles are to ensure program accountability in care delivery, and to motivate programmatic quality improvement, not just punitive action (28). If a program falls below expected performance to a significant degree, CMS

and OPTN may “flag” the program for poor performance (Table 2). Flagging programs that are performing poorly can serve two functions: 1) quality improvement for good programs that have had remediable lapses in performance and 2) removal of consistently under-performing programs in the interest of patient safety. The OPTN and CMS maintain distinct criteria for flagging and also vary individually in their approach to flagging. The review of program performance by the OPTN is executed by the Membership and Professional Standards Committee, which may entertain various levels of review, based on program performance including table and on-site surveys, peer review, interviews, and can remove membership privileges in severe circumstances. CMS is a health care regulatory agency and payer and has the responsibility to protect its beneficiaries’ health and safety concerns, and has an in-depth process for program review including site visits, peer review, and have the ability to terminate Medicare agreements with providers if warranted.

The concept of flagging programs is controversial. Liver transplant care is a high stakes endeavor, and mechanisms should exist that can stop the delivery of poor care. This is important for the transplant candidates seeking an organ, but also is an important function in organ stewardship. Donor organs should not be wasted in the name of poor care. Importantly, liver transplant programs serve an essential function for their waiting list population, and program stoppage or closure can bring significant harm to patients by potentially decreasing access to transplant. The threshold for flagging has to be high, which is a hotly debated topic in modern clinical transplantation. Fortunately, program stoppage after program flagging is a rare event and is usually related to recalcitrant

problems in the execution of liver transplant care, even after remediation, which typically underscores multiple structural and process deviations.

Stakeholder Use of Liver Transplant Quality Reports All stakeholders involved in liver transplantation have an interest in quality reporting. Transparency of outcomes from a morbid procedure with approximately 10-12% 1-year mortality rate is critical.

Perhaps the greatest benefactor of public reporting is the liver

transplant patient, and their advocates, including family members, referring physicians, and the community. Liver transplant patients are becoming increasingly sophisticated and have substantial choice in where to seek care (24). SRTR public reports have the potential to augment patient decision-making on where to seek liver transplant care. As mentioned, regulatory bodies use these reports as a structured mechanism to force quality improvement, with strict penalties if programs fail to improve. Providers have the opportunity, through structured QAPI programs, to improve quality of care in every transplant phase, far before penalties become a legitimate threat to program viability. Public reporting of liver transplant program data plays a critical role in program marketing to payers and patients, but may also serve programs to make the business case for resources within their own hospitals.

Payers use these reports to supplement their

own data to evaluate transplant program performance to guide contracting decisions, and to assess value – achieving particular outcomes standards based on a set of resources used (29). The value of public reporting goes far beyond regulatory functions – it serves all parties engaged in the liver transplant process.

Unintended Consequences of Quality Reporting in Liver Transplantation It is clear that quality reporting in liver transplantation serves important functions in maintaining patient safety and transparency. However, there is ongoing concern that policies designed to flag lower performing centers induce programs to alter their decision-making in fear of increasing the likelihood of having less than expected outcomes (Table 3) (30-32). Excessive attention to quality metrics may reduce access to transplant for high risk patients if a negative outcome is thought to put program quality assessments at risk. This alteration in decision-making, driven by risk aversion, could negatively impact access to liver transplant care for certain patients, i.e. older candidates, those with previous abdominal surgery, cardiovascular co-morbidities, or those with other risk factors. Risk aversion within a liver transplant program may affect liver transplant candidate selection, allograft selection, perioperative and post-transplant clinical management (33). Examples of this type of risk aversion are vast and may include limiting candidate age criteria, limiting transplant access for sicker candidates, avoidance of candidates with malignancy as a diagnosis, low utilization of donation after cardiac death liver allografts, decline of steatotic liver allografts, reluctance to develop living donor liver transplant programs, and prolonging hospital length of stay after surgery. In this context, risk aversion in liver transplant practices may decrease access to care for patients who could derive benefit from transplant, could lead to discard of usable liver allografts, and increase costs of care(33). The MPSC has been discussing a change in flagging criteria for kidney programs to address these disincentives and encourage programs to take on higher donor risk organs, but these initiatives are in their infancy (34). In recent years, new policy initiatives have come forward to help provide a safety

net to reduce risk aversion for liver transplant programs. MELD exceptions are available to re-transplant candidates who have ischemic cholangiopathy after transplant from a DCD donor. Another recent policy provides elevated status on the kidney transplant waiting list to all liver transplant recipients who have stage 5 chronic kidney disease or dialysis dependence within two and twelve months of the liver transplant procedure. This policy has a secondary benefit of allowing liver transplant programs to pursue higher risk organs (DCD livers, steatotic livers) that may be associated with post-transplant acute kidney injury, while simultaneously allowing the community to be good stewards of kidney allografts.

The use of graft and patient survival at one year as a bright line test for liver transplant quality also inherently undermines the importance of quality measures directed toward other vital components of liver transplant care. Figure 1 shows a broad conception of liver transplant quality as viewed through center-driven performance. Multiple metrics that are important to other stakeholders, particularly patients, are lost when focusing on the tiered ratings related to graft survival. The timeliness of transplant, organ acceptance practices, satisfaction with transplant hospital care, and other domains are de-emphasized in the current era of public reporting, which obfuscates what should be considered “true performance” – how a program performs across multiple domains, not just in keeping patients alive and grafts functioning to the arbitrary 1-year mark. This can be troubling for all stakeholders, and may have the unintended consequence of sacrificing some areas of performance for others if resources for quality improvement are constrained.

An additional concern in public reporting of liver transplant data is the ongoing tension between practice innovation and measuring quality. Many consider quality reporting a threat to innovation. In the development of new therapeutic approaches, experimental practices or novel treatments may arise that could negatively affect patient outcomes. Quality measurement of transplant programs and public reporting of outcomes may decrease the desire to participate in groundbreaking clinical trials or the development of new treatments for fear of a negative report. CMS has a ‘mitigating factors’ process to allow programs to state their case for less than ideal outcomes, but these applications are variably successful, and programs may be required to improve quality under a Systems Improvement Agreement (SIA). This agreement is a legal contract between transplant programs and CMS, which mandates improvements in transplant program processes and structure to aid in compliance with CMS Conditions of Participation for transplant programs. Failure to meet these improvements can lead to program closure.

QAPI in Liver Transplant Programs The dual regulatory requirement for transplant programs to maintain an integrated Quality Assurance and Process Improvement (QAPI) program by both CMS and the OPTN has brought the national health care quality strategy to bear on all of solid organ transplantation (35). QAPI programs place a premium on continuous assessment of clinical performance and continuous quality improvement in an infinite cycle for all phases of transplant care (8, 28, 31, 36, 37). A comprehensive transplant QAPI program has five primary elements: specific design and scope, defined governance and leadership, feedback, data systems, and monitoring, systematic analysis and systematic action, and

performance improvements. Each program is required to have a documented comprehensive QAPI plan, and to identify physician, nursing, and administrative leaders for transplant quality. To monitor performance, programs have created dashboards to track process and outcome metrics for each phase of transplant care and have designed workflows for their creation. An important QAPI function is the review of adverse events, which are done through mortality and morbidity conferences, comprehensive timeline-driven case reviews, or root cause analyses. Transplant QAPI programs are also required to integrate with larger hospital quality efforts. The liver transplant programs within Mayo Clinic’s three facilities have incorporated all of these functions in their QAPI journey, supported by well-trained personnel including quality improvement advisors with industry-based quality improvement training. In our experience, the presence of a robust QAPI program assists clinicians in the initiation of process improvement projects, contributes to a culture of safety, and allows for real-time monitoring of patient outcomes.

An important part of QAPI is the application of quality improvement methods to clinical processes. These methods have arisen from the business world, and include PDCA (the Deming Plan-Do-Check-Act iterative approach), DMAIC (the Six Sigma DefineMeasure-Analyze-Improve-Control), and LEAN (value improvement through process waste reduction). These methodologies have been applied in health care and have been applied in transplant-phase clinical processes (38, 39). Reich has commented that repeated application of these methods helps create a culture of safety and quality improvement within programs, with the goal of “generating light and not heat” through

collaborative review of events and solution engineering (35).

While there is sparse data regarding the efficacy of QAPI programs specifically on liver transplant outcomes, it is evident that QAPI can improve transplant program performance. Schold et al. showed that kidney transplant graft survival can improve in programs initially performing poorly (40). Poor program performance with clinical outcomes are rarely in isolation - CMS survey results indicated these poorly performing programs were nearly twice as likely to have deficiencies in other areas besides outcomes, including patient care, practices, and multidisciplinary planning compared to other programs (28). These areas are ripe for application of trackable, important, and actionable process and outcome metrics. Importantly, the OPTN and SRTR have created tools to augment QAPI processes within transplant programs including the use of CUSUM, a process control tool to evaluate graft losses and patient deaths relative to flagging thresholds (41, 42), and the OPTN Benchmark and Root reports which provide a plethora of data about pre-transplant care, transplantation and organ acceptance, and posttransplant care within centers.

The QAPI mandate poses challenges to liver transplant programs. Small liver programs may struggle to bring the resources to bear to have a robust QAPI program and a ubiquitous culture of safety being pursued with these regulations. These resources are expensive. Transplant programs must identify and designate quality officers and other personnel to collect required data elements from electronic medical records and other sources, submit them on designated forms for generation of PSRs, to maintain regulatory

compliance, support engagement with hospital quality programs, and in many cases to help in the execution of QAPI methodologies in quality improvement projects. Transplant quality personnel report that these efforts are generally understaffed and underfunded (43). An additional challenge for liver transplant clinicians, who are used to fixing problems and moving on, is to apply QI methodologies to fix problem areas within their programs. Defining and measuring problems, analyzing trends, creating interventions, and continuously monitoring their efficacy are not prioritized in clinical training programs (44-46). Failure to adopt these techniques could undermine the success of the QAPI mandate. Importantly, even though regulatory bodies have created an unfunded QAPI mandate on programs, they have provided data toolkits to actually impact quality, which is an important step and evidence of collaboration with transplant providers.

Opportunities to Improve Quality and Quality Measurement in Liver Transplantation

How does a program improve liver transplant quality using a QAPI framework? This is a challenge for many centers. QAPI methods have been successful in transplant, in health care, and in many other fields. Since QAPI is a mandatory activity for U.S. liver transplant programs, liver transplant program leaders need to identify barriers to adopting QAPI processes within their programs, set and accomplish concrete goals, and have a global understanding of the limitations of quality measurement and the QAPI process. Many barriers are related to resources, training, and culture. The goal of quality measurement and improvement is to affect structural and process changes in clinical care.

We have found that quality activities also require their own structure and process. Structural requirements include dedicated personnel and leadership with quality improvement experience, and computing resources to collect and collate data into organized usable dashboards for real-time review. Additionally, clinical leaders must design metrics that are benchmark-driven from historical program data or national data sources. Process requirements include regular meetings and review between clinical leaders and quality personnel to set an overall mission for the quality program, establish and maintain metrics, review indicators in a timely way from each transplant phase, review data management processes, and work through quality improvement projects when indicators deviate from a priori established thresholds. These resources are typically unfunded by payers but are necessary for robust quality improvement activity.

Since quality measurement and improvement is a mandatory component of modern transplant programs, it is both practical and necessary to train transplant providers (physicians, physician extenders, nurse coordinators, and others) on quality improvement methodologies. In multiple clinical areas, quality improvement training for clinical staff is lacking (9, 38, 45). At present, this education is lacking in clinical training programs and has to be sought out separately. Clinical transplant training should lay the foundation for quality improvement. However, it is insufficient to only train “quality champions” in established methods (PDCA, DMAIC, Lean Six Sigma). It is important for providers engaged in daily clinical care to be trained in these methods, so the methods can be readily applied by end-users. These methods can be met with trepidation, but there are data from transplant programs that support their use in a clinical context (22). Finally,

the culture of transplant programs can be barrier to quality improvement. There are grievances amongst physicians in quality monitoring activities (9), but transplant providers should embrace quality improvement and seek to integrate it in daily clinical activity. Fixing processes of care is a natural fit for transplant providers, but more published data on the application of these methods to transplant process improvement is critical to achieving buy-in.

Beyond understanding barriers to implementing quality improvement at the local level, it is imperative to understand the current limitations in measuring quality in liver transplantation. While programs can create their own metrics, there is a lack of validated generalizable metrics that cover all domains of health care quality for all phases of transplant care (47). Figure 1 illustrates that transplant center quality is related to multiple inputs, and can be measured in several domains. These measures may also feedback to the inputs as well. Key areas of measurement identified by stakeholders include more robust waiting list outcomes, organ offer acceptance metrics, and living donor outcomes (30, 47, 48).

New areas of quality measurement should be focused on the aims of health care delivery. Current metrics that are actionable by regulators are centered on 1-year graft and patient survival after transplant. This approach neglects many other important points of care that are considered the responsibility of the transplant center, such as organ acceptance patterns, waiting list survival, and long-term outcomes at 3 years and beyond. As shown in Figure 1, there are multiple points of emphasis that can be considered in evaluating

transplant program quality. For example, combining metrics into a composite that incorporates waiting list outcomes, organ acceptance patterns, and post-transplant outcomes have promise. Composite measures of liver transplant quality have not been tested, but have shown promise in other areas of surgery (49-51). Robust patientcentered metrics, such as patient-reported outcomes, in the liver transplant process are in their infancy, but could be developed, and could be used by programs to improve local care processes and patient experiences. Development of these types of metrics would require additional research and validation, but many other opportunities exist as well.

Discussion In the modern era of U.S. liver transplantation, all stakeholders expect high quality care delivery due to the improvement in clinical care over the last five decades. Quality measurement is not a new concept to solid organ transplantation in the United States, and regulatory bodies, payers, and the public hold transplant programs accountable for their performance. CMS and the OPTN, using their own individual criteria, currently flag underperforming programs based on recurring public reports (PSRs) released every 6 months. Quality reporting may have unintended consequences which may need to be addressed in the policy arena, but the need for transparency is of high value to all stakeholders. Just as quality reporting is virtually continuous, quality improvement within liver transplant programs must be a constant. Dual regulatory mandates have led to the proliferation of transplant QAPI programs, which is new to transplant clinicians and hospitals, but holds significant promise in improving local care delivery within transplant programs and rapidly identifying problem prone areas before centers are at risk

for flagging. While quality measurement in the United States currently is centered on hard clinical outcomes of 1-year graft and patient survival, there are opportunities to define liver transplant quality more comprehensively through new metrics derived from rigorous research.

How does U.S. activity in liver transplant quality compare to efforts to abroad? Perhaps one of the best examples of integration of process improvement methods into transplantation is in the United Kingdom, through National Health Service Blood and Transplantation (NHSBT) governance. Clinical audits, based on DMAIC and PDSA cycles of improvement, are required and supported for local, regional, and national projects to improve quality of care (52). These audits are performed at all levels for liver transplant care, although the results of these are not published in peer-reviewed literature. National audits on transplant activity and outcomes, akin to SRTR Annual Reports, are publically available and can be evaluated using the UK Liver Transplant Audit Database. Overall, there are several parallels in quality monitoring and improvement activities when comparing the US and the UK. Eurotransplant, which integrates the transplant activity of eight European countries, also monitors quality and produces public reports of liver transplant volumes (53). Quality monitoring is focused on adverse event review, particularly related to organ distribution across national lines and the occurrence of associated adverse events. There is also an active QAPI process active at the administrative level that performs external and internal audits. There are active qualityoriented activities in transplantation in European systems, but peer-reviewed literature on affecting change in clinical outcomes in these systems is lacking. Other areas of Europe

and Asia have made efforts to create clinical registries (i.e., Chinese Liver Transplant Registry), but it is unclear from peer-reviewed literature if this has been used to feedback into quality improvement activities using established methods.

While there are challenges in current quality measurement and barriers to instituting quality improvement, it is important to understand whether surmounting these challenges will improve value. Value can be defined roughly as quality per currency spent, but how value in liver transplantation is not well defined. There are multiple challenges in defining liver transplant value, with regards to defining quality and defining costs of care. Value cannot be defined until stakeholders can agree on what constitutes quality of liver transplant care, the numerator of the value equation. In this context, quality measurement is of the utmost importance to defining value. Another problem in understanding the quality component of the value equation is defining the bounds for what constitutes transplant care; where does it truly start and where does it truly end? The denominator of the value equation is cost, which must be conceptualized separately from quality. However, defining costs of care present similar challenges – cost has to be considered within the heretofore nebulous bounds of liver transplant care. These bounds must be defined by stakeholders, include provision for timely liver transplant versus continued end stage liver disease care, and include a reasonable period of post-liver transplant support. The path to evaluating value of transplant care is a challenge, and the quality measurement aspects represent only a part of it. As initiatives are underway to evaluate value-based purchasing in surgical care in the United States, the need to measure quality appropriately will continue to be a priority.

The future of quality measurement and improvement activities in liver transplantation hold significant promise. The alignment of stakeholder priorities in these activities is an idyllic notion, and transplant programs, regulators, and patients need to jointly put forth more effort to identify what components of care and which outcomes are most important to measure and improve.

References 1. Donabedian A. Evaluating the quality of medical care. Milbank Mem Fund Q 1966;44(3):Suppl:166-206. 2. Volk ML, Kanwal F. Quality of Care in the Cirrhotic Patient. Clin Transl Gastroenterol 2016;7:e166. 3. Atkinson S, Ingham J, Cheshire M, Went S. Defining quality and quality improvement. Clin Med (Lond) 2010;10(6):537-539. 4. Kim WR, Lake JR, Smith JM, Skeans MA, Schladt DP, Edwards EB et al. Liver. American journal of transplantation : official journal of the American Society of Transplantation and the American Society of Transplant Surgeons 2016;16 Suppl 2:69-98. 5. National Organ Transplant Act. PubL 98-507 http://legislinkorg/us/pl-98507 1984. 6. Organ Procurement and Transplantation Network. Health Resources and Services Administration, HHS. Final rule. Fed Regist 1999;64(202):56650-56661. 7. Medicare program; hospital conditions of participation: requirements for approval and re-approval of transplant centers to perform organ transplants. Final rule. Federal register 2007;72(61):15197-15280. 8. Organ Procurement and Transplantation Network Bylaws In. https://optn.transplant.hrsa.gov/governance/bylaws/, 2017. 9. Henry LR, von Holzen UW, Minarich MJ, Hardy AN, Beachy WA, Franger MS et al. Quality measurement affecting surgical practice: Utility versus utopia. Am J Surg 2017. 10. Institute of Medicine. Crossing the Quality Chasm: A New Health System for the Twenty-first Century. Washington DC: National Academy Press, 2001. 11. Ko CY, Hall BL, Hart AJ, Cohen ME, Hoyt DB. The American College of Surgeons National Surgical Quality Improvement Program: achieving better and safer surgery. Jt Comm J Qual Patient Saf 2015;41(5):199-204.

12. Bruny JL, Hall BL, Barnhart DC, Billmire DF, Dias MS, Dillon PW et al. American College of Surgeons National Surgical Quality Improvement Program Pediatric: a beta phase report. J Pediatr Surg 2013;48(1):74-80. 13. Raval MV, Dillon PW, Bruny JL, Ko CY, Hall BL, Moss RL et al. Pediatric American College of Surgeons National Surgical Quality Improvement Program: feasibility of a novel, prospective assessment of surgical outcomes. J Pediatr Surg 2011;46(1):115-121. 14. Hall BL, Richards K, Ingraham A, Ko CY. New approaches to the National Surgical Quality Improvement Program: the American College of Surgeons experience. Am J Surg 2009;198(5 Suppl):S56-62. 15. Hall BL, Hamilton BH, Richards K, Bilimoria KY, Cohen ME, Ko CY. Does surgical quality improve in the American College of Surgeons National Surgical Quality Improvement Program: an evaluation of all participating hospitals. Ann Surg 2009;250(3):363-376. 16. Birkmeyer JD, Shahian DM, Dimick JB, Finlayson SR, Flum DR, Ko CY et al. Blueprint for a new American College of Surgeons: National Surgical Quality Improvement Program. J Am Coll Surg 2008;207(5):777-782. 17. Etzioni DA, Wasif N, Dueck AC, Cima RR, Hohmann SF, Naessens JM et al. Association of hospital participation in a surgical outcomes monitoring program with inpatient complications and mortality. JAMA 2015;313(5):505-511. 18. Osborne NH, Etzioni DA. Does Participation in the ACS-NSQIP Improve Outcomes? Ann Surg 2015. 19. Osborne NH, Nicholas LH, Ryan AM, Thumma JR, Dimick JB. Association of hospital participation in a quality reporting program with surgical outcomes and expenditures for Medicare beneficiaries. JAMA 2015;313(5):496-504. 20. Wigg AJ, McCormick R, Wundke R, Woodman RJ. Efficacy of a chronic disease management model for patients with chronic liver failure. Clin Gastroenterol Hepatol 2013;11(7):850-858 e851-854. 21. Tapper EB, Finkelstein D, Mittleman MA, Piatkowski G, Chang M, Lai M. A Quality Improvement Initiative Reduces 30-Day Rate of Readmission for Patients With Cirrhosis. Clin Gastroenterol Hepatol 2016;14(5):753-759. 22. Tapper EB. Building Effective Quality Improvement Programs for Liver Disease: A Systematic Review of Quality Improvement Initiatives. Clin Gastroenterol Hepatol 2016;14(9):1256-1265 e1253. 23. Morando F, Maresio G, Piano S, Fasolato S, Cavallin M, Romano A et al. How to improve care in outpatients with cirrhosis and ascites: a new model of care coordination by consultant hepatologists. J Hepatol 2013;59(2):257-264. 24. Mathur AK, Aqel B, Moss AA. Should quality of the liver transplant candidate evaluation be measured? Clinical Liver Disease 2016;8(3):64-67. 25. Kanwal F, Kramer J, Asch SM, El-Serag H, Spiegel BM, Edmundowicz S et al. An explicit quality indicator set for measurement of quality of care in patients with cirrhosis. Clin Gastroenterol Hepatol 2010;8(8):709-717. 26. Muller X, Marcon F, Sapisochin G, Marquez M, Dondero F, Rayar M et al. Defining Benchmarks in Liver Transplantation: A Multicenter Outcome Analysis Determining Best Achievable Results. Ann Surg 2017.

27. Salkowski N, Snyder JJ, Zaun DA, Leighton T, Israni AK, Kasiske BL. Bayesian methods for assessing transplant program performance. American journal of transplantation : official journal of the American Society of Transplantation and the American Society of Transplant Surgeons 2014;14(6):1271-1276. 28. Hamilton TE. Accountability in health care--transplant community offers leadership. American journal of transplantation : official journal of the American Society of Transplantation and the American Society of Transplant Surgeons 2009;9(6):1287-1293. 29. Irwin FD, Wu C, Bannister WM, Bonagura AF, Laihinen B, Axelrod DA et al. A commercial transplant network's perspective of value in solid organ transplantation: Strategizing for value in transplant care. Transplant Rev (Orlando) 2016. 30. Goldberg DS, French B, Lewis JD, Scott FI, Mamtani R, Gilroy R et al. Liver transplant center variability in accepting organ offers and its impact on patient survival. Journal of hepatology 2016;64(4):843-851. 31. Abecassis MM, Burke R, Cosimi AB, Matas AJ, Merion RM, Millman D et al. Transplant center regulations--a mixed blessing? An ASTS Council viewpoint. American journal of transplantation : official journal of the American Society of Transplantation and the American Society of Transplant Surgeons 2008;8(12):2496-2502. 32. Abecassis MM, Burke R, Klintmalm GB, Matas AJ, Merion RM, Millman D et al. American Society of Transplant Surgeons transplant center outcomes requirements-a threat to innovation. American journal of transplantation : official journal of the American Society of Transplantation and the American Society of Transplant Surgeons 2009;9(6):1279-1286. 33. Garonzik-Wang JM, James NT, Van Arendonk KJ, Gupta N, Orandi BJ, Hall EC et al. The aggressive phenotype revisited: utilization of higher-risk liver allografts. American journal of transplantation : official journal of the American Society of Transplantation and the American Society of Transplant Surgeons 2013;13(4):936942. 34. OPTN/UNOS Membership and Professional Standards Committee. Transplant Program Performance Measures Review: A Concept Paper. In., 2016. 35. Reich DJ. Quality assessment and performance improvement in transplantation: hype or hope? Curr Opin Organ Transplant 2013;18(2):216-221. 36. Hamilton TE. Regulatory oversight in transplantation: are the patients really better off? Curr Opin Organ Transplant 2013;18(2):203-209. 37. Axelrod DA, Millman D, Abecassis MM. US Health Care Reform and Transplantation. Part I: overview and impact on access and reimbursement in the private sector. American journal of transplantation : official journal of the American Society of Transplantation and the American Society of Transplant Surgeons 2010;10(10):2197-2202. 38. Leaphart CL, Gonwa TA, Mai ML, Prendergast MB, Wadei HM, Tepas JJ, 3rd et al. Formal quality improvement curriculum and DMAIC method results in interdisciplinary collaboration and process improvement in renal transplant patients. J Surg Res 2012;177(1):7-13.

39. Toledo AH, Carroll T, Arnold E, Tulu Z, Caffey T, Kearns LE et al. Reducing liver transplant length of stay: a Lean Six Sigma approach. Prog Transplant 2013;23(4):350-364. 40. Schold JD, Buccini LD, Srinivas TR, Srinivas RT, Poggio ED, Flechner SM et al. The association of center performance evaluations and kidney transplant volume in the United States. American journal of transplantation : official journal of the American Society of Transplantation and the American Society of Transplant Surgeons 2013;13(1):67-75. 41. Axelrod DA, Kalbfleisch JD, Sun RJ, Guidinger MK, Biswas P, Levine GN et al. Innovations in the assessment of transplant center performance: implications for quality improvement. American journal of transplantation : official journal of the American Society of Transplantation and the American Society of Transplant Surgeons 2009;9(4 Pt 2):959-969. 42. Axelrod DA, Guidinger MK, Metzger RA, Wiesner RH, Webb RL, Merion RM. Transplant center quality assessment using a continuously updatable, risk-adjusted technique (CUSUM). American journal of transplantation : official journal of the American Society of Transplantation and the American Society of Transplant Surgeons 2006;6(2):313-323. 43. O'Rourke M. Quality Staffing Models for Transplant Programs. In. 4th Annual Transplant Quality Institute. Charlotte, NC: American Foundation for Donation and Transplantation, 2017. 44. Aysola J, Myers JS. Integrating Training in Quality Improvement and Health Equity in Graduate Medical Education: Two Curricula for the Price of One. Acad Med 2017. 45. Rao SK, Carballo V, Cummings BM, Millham F, Jacobson JO. Developing an Interdisciplinary, Team-Based Quality Improvement Leadership Training Program for Clinicians: The Partners Clinical Process Improvement Leadership Program. Am J Med Qual 2017;32(3):271-277. 46. Liao JM, Co JP, Kachalia A. Providing Educational Content and Context for Training the Next Generation of Physicians in Quality Improvement. Acad Med 2015;90(9):1241-1245. 47. Kasiske BL, McBride MA, Cornell DL, Gaston RS, Henry ML, Irwin FD et al. Report of a consensus conference on transplant program quality and surveillance. American journal of transplantation : official journal of the American Society of Transplantation and the American Society of Transplant Surgeons 2012;12(8):1988-1996. 48. Snyder JJ. New SRTR Metrics for Offer Acceptance. In: AST Cutting Edge of Transplantation; 2017 February 23, 2017; Pheonix, AZ; 2017. 49. Dimick JB, Birkmeyer NJ, Finks JF, Share DA, English WJ, Carlin AM et al. Composite measures for profiling hospitals on bariatric surgery performance. JAMA surgery 2014;149(1):10-16. 50. Dimick JB, Staiger DO, Hall BL, Ko CY, Birkmeyer JD. Composite measures for profiling hospitals on surgical morbidity. Annals of surgery 2013;257(1):67-72. 51. Dimick JB, Staiger DO, Osborne NH, Nicholas LH, Birkmeyer JD. Composite measures for rating hospital quality with major surgery. Health services research 2012;47(5):1861-1879.

52. The Simple Guide to Clinical Audit in NHSBT. 2017 06/13/2017; Available from: http://hospital.blood.co.uk/media/26835/simple-guide-to-clinical-audit.pdf 53. Eurotransplant. 2017 [cited 2017 11/30/2017]; Available from: https://www.eurotransplant.org/cms/index.php?page=home

Figure Legends Figure 1. A Broad Conceptual Model of Liver Transplant Center Quality Evaluating the true quality of a liver transplant center requires a multi-faceted approch across several domains, which are affected by multiple inputs in the clinical transplant environment. Current measures of liver transplant quality in the United States do not adequately capture important aspects of care delivery that hold importance for stakeholders.

Patient Factors •Attitudes toward transplantation •Medical co-morbidities •Psychosocial milieu •Socioeconomic status •Social Support •Insurance status

Facility/Program Structure •Organizational mission •Administration •Capacity •Program size •Technological resources •Teaching status •Physician/Surgeon Labor Force •RN/LPN/Allied Labor Force

Center Practices/Processes

Organ Donation Environment

•Evaluation efficiency •Waiting list management •Organ acceptance patterns •In-hospital management processes •Post-transplant care coordination/follow-up

•Population Death Rate •OPO/DSA Efficacy in Brain Dead Donor Conversion •Marginal organ placement •Donation after Cardiac Death Protocols •Technological resources •OPO Labor Force

Quality of Transplant Center

Patient-Centered Outcomes •HR Quality of life •Daily Functioning •Symptom Control •Communication with providers •Satisfaction with care provided •Robust Patient-Reported Outcomes

Clinical Outcomes •Graft Survival •Patient Survival •Medical and Surgical Complications •Waiting List Survival •Time to Transplant

Hospital Performance •Process and outcomes for common medical conditions •Process and outcomes for complex conditions •Critical care •Anesthesia •Radiology •Patient satisfaction

Center Performance •Financial margin •Cost-effective Resource Utilization •QAPI •Provider/staff satisfaction

Table 1. The Role of Regulation in Improving Quality in U.S. Liver Transplantation Policy Issuing Body Result National Organ Transplant Act of U.S. Congress  Authorized the Department of Health 1984 (and its amendments) and Human Services to create an Organ Procurement and Transplantation Network (OPTN) and the Scientific Registry of Transplant Recipients (SRTR)  The OPTN continuously facilitates organ placement and designs organ allocation and distribution policies, aims to provide data to transplant stakeholders to be used in improvement of the field of solid organ transplantation, establishes requirements for member organizations to maintain organ procurement organizations (OPOs) and solid organ transplant programs The Final Rule (2000 and later Department of  Codified the rules of engagement for revisions) Health and the OPTN and SRTR with transplant Human Services programs  Mandated ongoing and periodic reviews of transplant hospitals and OPOs  Established responsibilities of the OPTN, SRTR, and transplant prograsm to the Secretary of the Department of Health and Human Services  Identified the ability for Secretary of HHS to take punitive action against transplant programs that put public safety at risk Medicare Conditions of Participation Centers for  Further established oversight rules for of Transplant Hospitals (2007) Medicaid and CMS over transplant hospitals and Medicare linked these to approval and reServices (CMS) approval of transplant programs  “Complementary” oversight to the OPTN  Structural requirements for transplant programs  Established Flagging Criteria for transplant programs based on observed and expected graft loss and patient mortality events

Table 2. Regulatory Bodies and Quality Metrics in Solid Organ Transplantation in the United States

Organizational Entity SRTR

Outcome

Flagging Threshold Do not set specific thresholds for flagging program performance Provide semiannual reports of transplant program performance using tiered rating system Create models for risk-adjusted graft and patient survival that are applied to Flagging Criteria by other bodies (OPTN and CMS)

CMS

Volume 1-Year Graft Survival

10 transplants per year Standard Level Deficiency Observed Graft Losses – Expected Graft Losses > 3 Observed Graft Losses/Expected Graft Losses > 1.5 1-sided p-value < 0.05 Condition Level Deficiency Observed Graft Losses – Expected Graft Losses > 3 Observed Graft Losses/Expected Graft Losses > 1.85 1-sided p-value < 0.05 (on 2 of 5 consecutive PSRs)

1-Year Patient Survival

Standard Level Deficiency Observed Patient Deaths – Expected Patient Deaths > 3 Observed Patient Deaths/Expected Patient Deaths > 1.5 1-sided p-value < 0.05 Condition Level Deficiency Observed Patient Deaths – Expected Patient Deaths > 3 Observed Patient Deaths/Expected Patient Deaths > 1.85 1-sided p-value < 0.05 (on 2 of 5 consecutive PSRs)

OPTN

Volume 1-Year Graft Survival

1-Year Patient Survival

No specific number; Functional inactivity = zero transplants in 3 consecutive months Probability is greater than 75% that the hazard ratio for graft loss is greater than 1.2* Probability is greater than 10% that the hazard ratio for graft loss is greater than 2.5* Probability is greater than 75% that the hazard ratio for patient death is greater than 1.2* Probability is greater than 10% that the hazard ratio for

Advisory Committee on Transplantation (to the U.S. Secretary of Health and Human Services

patient death is greater than 2.5* No specific quality metrics related to program performance Seek input from regulatory agencies, transplant stakeholders, and other groups on the status of solid organ transplantation in order to advise the Secretary of the U.S. Department of Health and Human Services in their oversight

Table 3. Stakeholders and Barriers to Improving Solid Organ Transplant Quality Stakeholder Barrier Payers  Excessive focus on graft and patient survival  Neglect of other domains of quality of transplant care  Threatened loss of contracts with Transplant Centers; reimbursement concerns  Emphasis on the need for an embedded QAPI structure within transplant centers without further direction or support Providers  Risk aversion due to perceived pressures from other stakeholders  Excessive focus on graft and patient survival  Neglect of other domains of quality of transplant care  Failure to affect regulatory change by lack of data on clinically meaningful covariates to affect riskadjustment  Failure to apply QAPI methods to clinical process improvement Regulatory Bodies  Flagging criteria induce risk aversion in transplant centers  Focus on post-transplant metrics undermines the importance of other metrics including intention to treat at the time of waitlisting, particularly for vulnerable or high risk transplant candidates  Current metrics do not adequately capture transplant center quality or differentiate centers  Requirements for embedding QAPI programs within liver transplant centers without much further direction or support