- Email: [email protected]
Gastroenterology in a New Era of Accountability: Part 1. An Overview of Performance Measurement
CLINICAL GASTROENTEROLOGY AND HEPATOLOGY 2011;9:563–566
PRACTICE MANAGEMENT: OPPORTUNITIES AND CHALLENGES Gastroenterology in a New Era of Accountability: Part 1. An Overview of Performance Measurement SPENCER D. DORN Division of Gastroenterology and Hepatology, University of North Carolina School of Medicine, Chapel Hill, North Carolina
T
“
rust me, I’m a doctor.” This refrain was likely enough to satisfy patients throughout the “golden age of medicine,” a period during which physicians were relatively unencumbered by regulation, highly regarded by the public, had few curbs on their income-generating potential, and enjoyed a high degree of professional autonomy.1 Back then, as long as a physician did not commit egregious malpractice, his professional reputation was the sole means by which he was held accountable. Times have changed. By 1988 Arnold Relman had already predicted a new, revolutionary era of assessment and accountability,2 defined as “the procedure and process by which one party provides a justification and is held responsible for its actions by another party who has an interest in the action.”3 After decades of stalling, several converging factors—most notably, the rise of evidence-based medicine, an increasing awareness of the US health care system’s “quality chasm” and cost crisis, and a growing emphasis on patient-centered care— have finally pushed accountability to the forefront. Meanwhile, over the past 2 decades, the science of performance measurement has advanced considerably. A wide array of performance measures is now available, and electronic health records have made it easier to access clinical data. Today, gastroenterologists and the organizations within which they work are increasingly being held accountable for the quality and cost of the care that they provide. Performance measurement has begun to permeate virtually all aspects of practice, including how gastroenterologists are credentialed (eg, Maintenance of Certification Program Practice Performance Assessment Program),4 how patients evaluate them (eg, myriad public and proprietary rating sites, ranging from Medicare’s Physician Compare to Zagat’s),5 and how they are paid (ie, pay-for-performance and other programs that attempt to reward value rather than mere volume).6 The recent health care reform legislation bolsters the accountability movement through stipulations that require payers to further develop quality initiatives, such as hospital and physician value-based purchasing, as well as quality improvement and reporting programs.7 Overall, the take-home message is simple: the writing on the wall suggests that the halcyon days of yesteryear are gone. The Resources for Practical Application To view additional online resources about this topic and to access our Coding Corner, visit www.cghjournal.org/content/ practice_management.
old way of doing business will no longer suffice; accountability is here to stay. This 3-part series of articles seeks to help gastroenterologists better understand, adjust to, and potentially thrive in this new atmosphere. This first article presents a conceptual overview of performance measurement, along with its inherent challenges. Part 2 discusses the “nuts and bolts” of measurement, using the performance measure lifecycle to illustrate how measures are developed, endorsed, and implemented. Finally, part 3 discusses accountable care organizations, a burgeoning concept that, as the name suggests, attempts to push health care further down the path of accountability, and concludes by outlining ways in which gastroenterologists can best position themselves to practice in an era of accountability.
What Is Quality and How Is It Measured? Quality is a multidimensional concept whose definition varies depending on the perspective employed. Health care providers emphasize technical quality, which encompasses appropriateness and skill (both scientific and interpersonal). Patients are more interested in whether care is responsive to their specific needs and preferences, and health plans define quality based on how care is delivered to their beneficiary populations, as well as the overall performance of delivery organizations.8 The Institute of Medicine (IOM) attempted to reconcile these viewpoints when it defined quality as the “degree to which health services for individuals and populations increase the likelihood of desired health outcomes and are consistent with current professional knowledge.”9 The Institute of Medicine later added that high-quality care should be safe, effective, patient-centered, timely, efficient, and equitable.10 The Agency for Healthcare Research and Quality (AHRQ) put it more simply: quality is “doing the right thing, at the right time, in the right way, for the right person—and having the best possible result.”11 Quality is quantified using quality measures (or indicators). By and large, these measures are organized around Donabedian’s structure-process-outcome framework (Table 1).12,13 Structural measures assess attributes of the setting in which care occurs. Examples within gastroenterology include characteris© 2011 by the AGA Institute
1542-3565/$36.00 doi:10.1016/j.cgh.2011.03.037
564
SPENCER D. DORN
CLINICAL GASTROENTEROLOGY AND HEPATOLOGY Vol. 9, No. 7
Table 1. Virtues and Pitfalls of 4 Types of Quality Measures
Example Data collection burden Risk adjustment Group most meaningful Disadvantage
Structure
Process
Outcome
Patient experience
Electronic health record use Low Unnecessary Accreditation agencies Weak link to outcomes
Immunization of patients with IBD Medium Unnecessary Providers May not link to outcomes, may be overly specific, requires frequent updating
IBD-related hospitalizations or surgery High Necessary Patients Adequate risk adjustment is difficult; providers may not find outcome data “actionable”
Patient satisfaction with care Low May be necessary Patients Low response rate
IBD, inflammatory bowel disease.
tics of the care providers (eg, board certification status, staffing levels) and the facilities in which they work (eg, site of service, electronic health record use). Although structural measures are easy to measure and can apply to many different types of providers, they are not strongly linked to outcomes, and thus used only rarely.14 Process measures assess what is actually done in delivering care. In other words, are providers “doing the right thing right?” This includes adherence to evidence-based procedures (eg, appropriate interval for surveillance colonoscopy) and measures of how a process is performed (eg, colonoscopy withdrawal time). Process measures are popular among providers, who find them valid and actionable. Also, because processes can be assessed instantly and do not require risk adjustment, data collection burdens are modest. Disadvantages are that the processes measured may not meaningfully predict outcome, may be too specific to reflect overall quality of care, require frequent updating as evidence evolves, and are less significant to patients.15 Outcome measures relate to the proximate (ie, “indicators” such as adenoma detection rate) or ultimate (eg, colon cancer incidence rate) effects that care has on health status. Their main virtue is that they are easy to understand, and are thus most meaningful to patients, who tend to be interested in end results. However, because providers recognize that outcomes are often determined by factors outside of their control, they tend to view outcome measures skeptically. Additionally, as some may “cherry pick” the healthiest patients, case mix and severity adjustments are necessary, though can be difficult and often not possible (see below). Furthermore, the potentially long delay between medical care and outcome, as well as the large sample size requirements in cases of rare outcomes, creates considerable data collection burdens.16 One additional category of measures outside the Donabedian framework that deserves mention is patient experience measures. These measures are directly administered to patients in order to assess the patient centeredness and timeliness of the care that they received. Standardized patient experience questionnaires (eg, Agency for Healthcare Research and Quality [AHRQ] Consumer Assessment of Healthcare Providers and Systems [CAHPS] surveys) are available and now widely distributed. Still, their use is hindered by high cost, limited provider buy-in, a need for case mix adjustment (because older and healthier patients tend to rate their providers more positively), and low response rates (40% at best).17 In total, each type of quality measure has virtues and pitfalls, as summarized in Table 1. Quality measures are developed and
picked by balancing scientific rigor with feasibility. Because process measures tend to be more readily available and feasible, they dominate the current measurement landscape, including the Centers for Medicare and Medicaid Services Physician Quality Reporting System (PQRS).18
What Is Cost and How Is It Measured? In addition to quality, health care organizations and providers are also being increasingly held accountable for health care costs, particularly “direct costs,” defined as the value of all goods, services, and other resources that are consumed in the course of care for a condition.19 Several types of cost measures are in various stages of development. First, basic cost-related metrics such as length of stay, mean charges, or readmission rates, may be used to assess costs for a specific group of patients.17 Second, appropriateness measures use appropriateness of care criteria developed by specialty societies to consider whether a particular intervention is warranted (eg, delaying imaging for 1 month in uncomplicated acute back pain).20 Third, cost of care measures consider total resources used for a specific service.21 For example, colonoscopy cost-ofcare can be assessed by determining all related resources used immediately before, during, and in the 1 to 2 weeks after the procedure (eg, sedation/anesthesia, medications, equipment, professional and facility fees, downstream laboratory and pathology tests, imaging procedures, and hospitalization). Next, a standard price list is applied to these resources to estimate the cost of care. Finally, efficiency measures are a specific subset of cost measures that compares the cost of care associated with a specified level of quality.22 Efficiency measures are in the earliest stage of development; at present none meets the criteria for use in public reporting, value-based purchasing, or pay-for-performance programs.23
Performance Measurement Challenges Performance measurement triggers obvious concern among physicians, who worry about diminished practice autonomy and income.24 However, the details of quality and cost measurement remain far from resolved, especially with regard to the following areas: ●
Validity. Does a quality/cost measure really reflect the quality/cost of care? A measure is only valid if it is based on strong scientific evidence, and if scores on the measure can distinguish between high and low quality/cost of care.25 Many quality measures have demonstrated high validity.
July 2011
PERFORMANCE MEASUREMENT
However, because the evidence base for defining the optimal level of resource use is quite limited, the validity of cost measures is unknown.17 ●
●
●
Reliability. Is the variation in quality or cost because of measurement error or true differences in performance?26 In general, the patient panels seen by most physician groups are too small to generate reliable estimates of quality27 or cost.28 Consequently, providers may be wrongly assigned to certain high or low performance categories. One proposed remedy is to increase sample sizes by combining data across multiple health plans and/or by reporting at the physician group level (rather than individual physician).17 However, physician group profiles do not strongly predict the performance of individual physicians within the group.29 An alternative solution is to aggregate individual performance measures into a single composite score that summarizes the overall quality of care delivered. For instance, imagine a chronic liver disease composite prevention measure that includes discreet processes, such as immunizations, screening for esophageal varicies and hepatocellular carcinoma, and alcohol cessation counseling. This approach decreases sample size requirements, increases reliability, and may provide a better, more easily understood overall picture of quality of care.30 Nonetheless, achieving consensus on which measures to include in the composite and how to score them (eg, average, weighted average, all-or-none) is difficult. Also, if a composite combines unrelated metrics then important performance information may be lost.17 Time frame. Over what period should quality and cost of care be measured? For many conditions care stretches out over time, making it difficult to determine which costs and quality measurements apply. In response, many propose to assess performance over an entire “episode of care,” defined as “a series of temporally contiguous healthcare services related to the treatment of a given illness.” Episodes of care encompass initial evaluation and management, as well as follow-up and care coordination across multiple providers and various settings throughout a defined time period.26 For instance, an episode of care for a patient with Crohn’s disease would consider all related services (evaluation and management, imaging, endoscopic, surgical, laboratory, medication, etc) furnished by all providers (primary care physicians, gastroenterologists, surgeons, radiologists, etc) across all venues (ambulatory clinic, ambulatory surgical center, diagnostic imaging center, hospital, etc) for an entire year. In practice, defining an episode is tricky and typically relies on using “grouper” tools that use administrative data diagnosis codes to group related services into a single episode.31 Attribution. When multiple providers contribute to the care of a single patient, it can be difficult to determine which provider is responsible for which processes, outcomes, and costs. In such cases, a set of “attribution rules” that consider who delivered the care (primary care and/or specialists), the costs each provider was associated with, and the unit of analysis (individual, practice sites, or larger groups of physicians) are used to attribute quality and cost to specific providers.32 Again, translating this into practice is difficult because attribution rules may be somewhat
565
arbitrary and which rule is chosen significantly affects whether a provider is assigned to high or low quality/cost categories.33 ●
Risk adjustment. Certain patient characteristics can strongly affect how resources are spent and what outcomes are achieved. Statistical methods are needed to account for these patient-level characteristics and thereby “level the playing field.” However, it is impossible to adjust for unmeasured or unreported risk factors,17 a vexing problem that also afflicts epidemiology.34
●
Data availability. Whether performance data are derived from administrative claims or medical records may produce discrepant results.35 Furthermore, if claims data come from only a single payer (eg, Medicare) the results may not generalize to the provider’s entire practice. Also, the absence of price transparency36 makes it exceedingly difficult to measure costs.17 The specific challenges of measuring and reporting performance data are further discussed in part 2 of this series.
Conclusions Gastroenterologists are increasingly being held accountable for the quality and costs of the care that they deliver. Nonetheless, measuring quality and costs can be quite challenging because of issues with measure validity, reliability, time frame, provider attribution, risk adjustment, and data availability. With these limitations in mind, part 2 of this series elucidates how performance measures are developed, as well as how and why gastroenterologists can use these to assess their own performance. The series then concludes by considering accountable care organizations, which link performance measures to financial incentives designed to hold providers accountable for a given population’s care. References 1. McIntyre D, Rogers L, Heier EJ. Overview, history, and objectives of performance measurement. Health Care Financ Rev 2001;22: 7–22. 2. Relman AS. Assessment and accountability: the third revolution in medical care. N Engl J Med 1988;319:1220 –1222. 3. Emanuel EJ, Emanuel LL. What is accountability in health care? Ann Intern Med 1996;124:229 –239. 4. Weiss KB. Future of board certification in a new era of public accountability. J Am Board Fam Med 2010;23 Suppl 1:S32–S39. 5. Howell MD. A 37-year-old man trying to choose a high-quality hospital: review of hospital quality indicators. JAMA 2009;302: 2353–2360. 6. Rosenthal MB. Beyond pay for performance--emerging models of provider-payment reform. N Engl J Med 2008;359:1197–1200. 7. H.R.3590 - Patient Protection and Affordable Care Act, 2009. 8. Blumenthal D. Part 1: Quality of care--what is it? N Engl J Med 1996;335:891– 894. 9. Lohr KN, Donaldson MS, Medicare H-WJ. A strategy for quality assurance, V: Quality of care in a changing health care environment. QRB Qual Rev Bull 1992;18:120 –126. 10. Hurtado MP, Swift EK, Corrigan JM. Crossing the quality chasm: a new health system for the 21st century. Washington, DC: National Academy Press, 2001. 11. Agency for Healthcare Research and Quality. Your Guide to Choosing Quality Care. Available at: http://archive.ahrq.gov/consumer/qnt/. Accessed July 30, 2010.
566
SPENCER D. DORN
12. Donabedian A. The quality of care. How can it be assessed? JAMA 1988;260:1743–1748. 13. Donabedian A. Evaluating the quality of medical care. Milbank Mem Fund Q 1966;44 Suppl:166 –206. 14. Meyer GS, Massagli MP. The forgotten component of the quality triad: can we still learn something from “structure”? Jt Commission. J Qual Improv 2001;27:484 – 493. 15. Rubin HR, Pronovost P, Diette GB. The advantages and disadvantages of process-based measures of health care quality. Int J Qual Health Care 2001;13:469 – 474. 16. Mant J, Hicks N. Health status measurement and the assessment of medical care. Int J Qual Health Care 1996;8:107–109. 17. Romano P, Hussey P, Ritley D. Selecting Quality and Resource Use Measures: A Decision Guide for Community Quality Collaboratives. AHRQ Publication No. 09(10)-0073. Rockville, MD: Agency for Healthcare Research and Quality, 2010. 18. Pronovost PJ, Berenholtz SM, Goeschel CA. Improving the quality of measurement and evaluation in quality improvement efforts. Am J Med Qual 2008;23:143–146. 19. Gold MR, Siegel JE, Russell LB, et al. Cost-effectiveness in health and medicine. New York: Oxford University Press, 1996. 20. Baker N, Whittington JW, Resar RK, et al. Reducing Costs Through the Appropriate Use of Specialty Services. IHI Innovation Series white paper. Cambridge, MA: Insitute for Healthcare Improvement, 2010. 21. AQA. AQA principles of efficiency measures. Available at: http:// www.aqaalliance.org/files/PrinciplesofEfficiencyMeasurement. pdf. Accessed July 22, 2010. 22. AQA Alliance. AQA principles of “efficiency” measures. Available at: http://www.aqaalliance.org/files/PrinciplesofEfficiencyMeasurement April2006.doc. Accessed August 5, 2010. 23. McGlynn EA. Identifying, categorizing and evaluating health care efficiency measures. Final report. AHRQ Publication No. 080030. Rockville, MD: Agency for Healthcare Research and Quality, 2008. 24. Young PL, Olsen L, McGinnis JM. Value in health care: accounting for cost, quality, safety, outcomes, and innovation. Washington, DC: National Academies Press, 2010. 25. McGlynn EA, Asch SM. Developing a clinical performance measure. Am J Prev Med 1998;14:14 –21. 26. National Quality Forum. Measurement framework: Evaluating efficency across patient-focused episodes of care. Washington, DC: NQF, 2009. 27. Nyweide DJ, Weeks WB, Gottlieb DJ, et al. Relationship of primary care physicians’ patient caseload with measurement of quality and cost performance. JAMA 2009;302:2444 –2450.
CLINICAL GASTROENTEROLOGY AND HEPATOLOGY Vol. 9, No. 7
28. Adams JL, Mehrotra A, Thomas JW, et al. Physician cost profiling-reliability and risk of misclassification. N Engl J Med 2010;362: 1014 –1021. 29. Mehrotra A, Adams JL, Thomas JW, et al. Cost profiles: should the focus be on individual physicians or physician groups? Health Aff. (Millwood) 2010;29:1532–1538. 30. O’Brien SM, Shahian DM, DeLong ER, et al. Quality measurement in adult cardiac surgery: part 2–statistical considerations in composite measure scoring and provider rating. Ann Thorac Surg 2007;83:S13–S26. 31. Hussey PS, Sorbero ME, Mehrotra A, et al. Episode-based performance measurement and payment: making it a reality. Health Aff. (Millwood) 2009;28:1406 –1417. 32. Pacific Business Group on Health. Advancing physician performance measurement. Using administrative data to assess physician quality and efficiency, 2005. Available at: http://www. pbgh.org/storage/documents/reports/PBGHP3Report_09-01-05final. pdf. 33. Mehrotra A, Adams JL, Thomas JW, et al. The effect of different attribution rules on individual physician cost profiles. Ann Intern Med 2010;152:649 – 654. 34. Taubes G. Epidemiology faces its limits. Science 1995;269: 164 –169. 35. Benin AL, Vitkauskas G, Thornquist E, et al. Validity of using an electronic medical record for assessing quality of care in an outpatient setting. Med Care 2005;43:691– 698. 36. Reinhardt UE. The pricing of U.S. hospital services: chaos behind a veil of secrecy. Health Aff. (Millwood) 2006;25:57– 69.
Reprint requests Address requests for reprints to: Spencer D. Dorn, MD, MPH, Division of Gastroenterology and Hepatology, University of North Carolina at Chapel Hill, CB 7080, Chapel Hill, North Carolina, 27599-7080. e-mail: [email protected] ; fax: (919) 966-2250. Acknowledgments The author thanks Drs John Allen and Joel Brill for their advice on earlier drafts of the manuscript. Conflicts of interest The author discloses no conflicts. Funding Grant Support: Spencer Dorn is supported by NIH K12HS019468.
PRACTICE MANAGEMENT: OPPORTUNITIES AND CHALLENGES Gastroenterology in a New Era of Accountability: Part 1. An Overview of Performance Measurement SPENCER D. DORN Division of Gastroenterology and Hepatology, University of North Carolina School of Medicine, Chapel Hill, North Carolina
T
“
rust me, I’m a doctor.” This refrain was likely enough to satisfy patients throughout the “golden age of medicine,” a period during which physicians were relatively unencumbered by regulation, highly regarded by the public, had few curbs on their income-generating potential, and enjoyed a high degree of professional autonomy.1 Back then, as long as a physician did not commit egregious malpractice, his professional reputation was the sole means by which he was held accountable. Times have changed. By 1988 Arnold Relman had already predicted a new, revolutionary era of assessment and accountability,2 defined as “the procedure and process by which one party provides a justification and is held responsible for its actions by another party who has an interest in the action.”3 After decades of stalling, several converging factors—most notably, the rise of evidence-based medicine, an increasing awareness of the US health care system’s “quality chasm” and cost crisis, and a growing emphasis on patient-centered care— have finally pushed accountability to the forefront. Meanwhile, over the past 2 decades, the science of performance measurement has advanced considerably. A wide array of performance measures is now available, and electronic health records have made it easier to access clinical data. Today, gastroenterologists and the organizations within which they work are increasingly being held accountable for the quality and cost of the care that they provide. Performance measurement has begun to permeate virtually all aspects of practice, including how gastroenterologists are credentialed (eg, Maintenance of Certification Program Practice Performance Assessment Program),4 how patients evaluate them (eg, myriad public and proprietary rating sites, ranging from Medicare’s Physician Compare to Zagat’s),5 and how they are paid (ie, pay-for-performance and other programs that attempt to reward value rather than mere volume).6 The recent health care reform legislation bolsters the accountability movement through stipulations that require payers to further develop quality initiatives, such as hospital and physician value-based purchasing, as well as quality improvement and reporting programs.7 Overall, the take-home message is simple: the writing on the wall suggests that the halcyon days of yesteryear are gone. The Resources for Practical Application To view additional online resources about this topic and to access our Coding Corner, visit www.cghjournal.org/content/ practice_management.
old way of doing business will no longer suffice; accountability is here to stay. This 3-part series of articles seeks to help gastroenterologists better understand, adjust to, and potentially thrive in this new atmosphere. This first article presents a conceptual overview of performance measurement, along with its inherent challenges. Part 2 discusses the “nuts and bolts” of measurement, using the performance measure lifecycle to illustrate how measures are developed, endorsed, and implemented. Finally, part 3 discusses accountable care organizations, a burgeoning concept that, as the name suggests, attempts to push health care further down the path of accountability, and concludes by outlining ways in which gastroenterologists can best position themselves to practice in an era of accountability.
What Is Quality and How Is It Measured? Quality is a multidimensional concept whose definition varies depending on the perspective employed. Health care providers emphasize technical quality, which encompasses appropriateness and skill (both scientific and interpersonal). Patients are more interested in whether care is responsive to their specific needs and preferences, and health plans define quality based on how care is delivered to their beneficiary populations, as well as the overall performance of delivery organizations.8 The Institute of Medicine (IOM) attempted to reconcile these viewpoints when it defined quality as the “degree to which health services for individuals and populations increase the likelihood of desired health outcomes and are consistent with current professional knowledge.”9 The Institute of Medicine later added that high-quality care should be safe, effective, patient-centered, timely, efficient, and equitable.10 The Agency for Healthcare Research and Quality (AHRQ) put it more simply: quality is “doing the right thing, at the right time, in the right way, for the right person—and having the best possible result.”11 Quality is quantified using quality measures (or indicators). By and large, these measures are organized around Donabedian’s structure-process-outcome framework (Table 1).12,13 Structural measures assess attributes of the setting in which care occurs. Examples within gastroenterology include characteris© 2011 by the AGA Institute
1542-3565/$36.00 doi:10.1016/j.cgh.2011.03.037
564
SPENCER D. DORN
CLINICAL GASTROENTEROLOGY AND HEPATOLOGY Vol. 9, No. 7
Table 1. Virtues and Pitfalls of 4 Types of Quality Measures
Example Data collection burden Risk adjustment Group most meaningful Disadvantage
Structure
Process
Outcome
Patient experience
Electronic health record use Low Unnecessary Accreditation agencies Weak link to outcomes
Immunization of patients with IBD Medium Unnecessary Providers May not link to outcomes, may be overly specific, requires frequent updating
IBD-related hospitalizations or surgery High Necessary Patients Adequate risk adjustment is difficult; providers may not find outcome data “actionable”
Patient satisfaction with care Low May be necessary Patients Low response rate
IBD, inflammatory bowel disease.
tics of the care providers (eg, board certification status, staffing levels) and the facilities in which they work (eg, site of service, electronic health record use). Although structural measures are easy to measure and can apply to many different types of providers, they are not strongly linked to outcomes, and thus used only rarely.14 Process measures assess what is actually done in delivering care. In other words, are providers “doing the right thing right?” This includes adherence to evidence-based procedures (eg, appropriate interval for surveillance colonoscopy) and measures of how a process is performed (eg, colonoscopy withdrawal time). Process measures are popular among providers, who find them valid and actionable. Also, because processes can be assessed instantly and do not require risk adjustment, data collection burdens are modest. Disadvantages are that the processes measured may not meaningfully predict outcome, may be too specific to reflect overall quality of care, require frequent updating as evidence evolves, and are less significant to patients.15 Outcome measures relate to the proximate (ie, “indicators” such as adenoma detection rate) or ultimate (eg, colon cancer incidence rate) effects that care has on health status. Their main virtue is that they are easy to understand, and are thus most meaningful to patients, who tend to be interested in end results. However, because providers recognize that outcomes are often determined by factors outside of their control, they tend to view outcome measures skeptically. Additionally, as some may “cherry pick” the healthiest patients, case mix and severity adjustments are necessary, though can be difficult and often not possible (see below). Furthermore, the potentially long delay between medical care and outcome, as well as the large sample size requirements in cases of rare outcomes, creates considerable data collection burdens.16 One additional category of measures outside the Donabedian framework that deserves mention is patient experience measures. These measures are directly administered to patients in order to assess the patient centeredness and timeliness of the care that they received. Standardized patient experience questionnaires (eg, Agency for Healthcare Research and Quality [AHRQ] Consumer Assessment of Healthcare Providers and Systems [CAHPS] surveys) are available and now widely distributed. Still, their use is hindered by high cost, limited provider buy-in, a need for case mix adjustment (because older and healthier patients tend to rate their providers more positively), and low response rates (40% at best).17 In total, each type of quality measure has virtues and pitfalls, as summarized in Table 1. Quality measures are developed and
picked by balancing scientific rigor with feasibility. Because process measures tend to be more readily available and feasible, they dominate the current measurement landscape, including the Centers for Medicare and Medicaid Services Physician Quality Reporting System (PQRS).18
What Is Cost and How Is It Measured? In addition to quality, health care organizations and providers are also being increasingly held accountable for health care costs, particularly “direct costs,” defined as the value of all goods, services, and other resources that are consumed in the course of care for a condition.19 Several types of cost measures are in various stages of development. First, basic cost-related metrics such as length of stay, mean charges, or readmission rates, may be used to assess costs for a specific group of patients.17 Second, appropriateness measures use appropriateness of care criteria developed by specialty societies to consider whether a particular intervention is warranted (eg, delaying imaging for 1 month in uncomplicated acute back pain).20 Third, cost of care measures consider total resources used for a specific service.21 For example, colonoscopy cost-ofcare can be assessed by determining all related resources used immediately before, during, and in the 1 to 2 weeks after the procedure (eg, sedation/anesthesia, medications, equipment, professional and facility fees, downstream laboratory and pathology tests, imaging procedures, and hospitalization). Next, a standard price list is applied to these resources to estimate the cost of care. Finally, efficiency measures are a specific subset of cost measures that compares the cost of care associated with a specified level of quality.22 Efficiency measures are in the earliest stage of development; at present none meets the criteria for use in public reporting, value-based purchasing, or pay-for-performance programs.23
Performance Measurement Challenges Performance measurement triggers obvious concern among physicians, who worry about diminished practice autonomy and income.24 However, the details of quality and cost measurement remain far from resolved, especially with regard to the following areas: ●
Validity. Does a quality/cost measure really reflect the quality/cost of care? A measure is only valid if it is based on strong scientific evidence, and if scores on the measure can distinguish between high and low quality/cost of care.25 Many quality measures have demonstrated high validity.
July 2011
PERFORMANCE MEASUREMENT
However, because the evidence base for defining the optimal level of resource use is quite limited, the validity of cost measures is unknown.17 ●
●
●
Reliability. Is the variation in quality or cost because of measurement error or true differences in performance?26 In general, the patient panels seen by most physician groups are too small to generate reliable estimates of quality27 or cost.28 Consequently, providers may be wrongly assigned to certain high or low performance categories. One proposed remedy is to increase sample sizes by combining data across multiple health plans and/or by reporting at the physician group level (rather than individual physician).17 However, physician group profiles do not strongly predict the performance of individual physicians within the group.29 An alternative solution is to aggregate individual performance measures into a single composite score that summarizes the overall quality of care delivered. For instance, imagine a chronic liver disease composite prevention measure that includes discreet processes, such as immunizations, screening for esophageal varicies and hepatocellular carcinoma, and alcohol cessation counseling. This approach decreases sample size requirements, increases reliability, and may provide a better, more easily understood overall picture of quality of care.30 Nonetheless, achieving consensus on which measures to include in the composite and how to score them (eg, average, weighted average, all-or-none) is difficult. Also, if a composite combines unrelated metrics then important performance information may be lost.17 Time frame. Over what period should quality and cost of care be measured? For many conditions care stretches out over time, making it difficult to determine which costs and quality measurements apply. In response, many propose to assess performance over an entire “episode of care,” defined as “a series of temporally contiguous healthcare services related to the treatment of a given illness.” Episodes of care encompass initial evaluation and management, as well as follow-up and care coordination across multiple providers and various settings throughout a defined time period.26 For instance, an episode of care for a patient with Crohn’s disease would consider all related services (evaluation and management, imaging, endoscopic, surgical, laboratory, medication, etc) furnished by all providers (primary care physicians, gastroenterologists, surgeons, radiologists, etc) across all venues (ambulatory clinic, ambulatory surgical center, diagnostic imaging center, hospital, etc) for an entire year. In practice, defining an episode is tricky and typically relies on using “grouper” tools that use administrative data diagnosis codes to group related services into a single episode.31 Attribution. When multiple providers contribute to the care of a single patient, it can be difficult to determine which provider is responsible for which processes, outcomes, and costs. In such cases, a set of “attribution rules” that consider who delivered the care (primary care and/or specialists), the costs each provider was associated with, and the unit of analysis (individual, practice sites, or larger groups of physicians) are used to attribute quality and cost to specific providers.32 Again, translating this into practice is difficult because attribution rules may be somewhat
565
arbitrary and which rule is chosen significantly affects whether a provider is assigned to high or low quality/cost categories.33 ●
Risk adjustment. Certain patient characteristics can strongly affect how resources are spent and what outcomes are achieved. Statistical methods are needed to account for these patient-level characteristics and thereby “level the playing field.” However, it is impossible to adjust for unmeasured or unreported risk factors,17 a vexing problem that also afflicts epidemiology.34
●
Data availability. Whether performance data are derived from administrative claims or medical records may produce discrepant results.35 Furthermore, if claims data come from only a single payer (eg, Medicare) the results may not generalize to the provider’s entire practice. Also, the absence of price transparency36 makes it exceedingly difficult to measure costs.17 The specific challenges of measuring and reporting performance data are further discussed in part 2 of this series.
Conclusions Gastroenterologists are increasingly being held accountable for the quality and costs of the care that they deliver. Nonetheless, measuring quality and costs can be quite challenging because of issues with measure validity, reliability, time frame, provider attribution, risk adjustment, and data availability. With these limitations in mind, part 2 of this series elucidates how performance measures are developed, as well as how and why gastroenterologists can use these to assess their own performance. The series then concludes by considering accountable care organizations, which link performance measures to financial incentives designed to hold providers accountable for a given population’s care. References 1. McIntyre D, Rogers L, Heier EJ. Overview, history, and objectives of performance measurement. Health Care Financ Rev 2001;22: 7–22. 2. Relman AS. Assessment and accountability: the third revolution in medical care. N Engl J Med 1988;319:1220 –1222. 3. Emanuel EJ, Emanuel LL. What is accountability in health care? Ann Intern Med 1996;124:229 –239. 4. Weiss KB. Future of board certification in a new era of public accountability. J Am Board Fam Med 2010;23 Suppl 1:S32–S39. 5. Howell MD. A 37-year-old man trying to choose a high-quality hospital: review of hospital quality indicators. JAMA 2009;302: 2353–2360. 6. Rosenthal MB. Beyond pay for performance--emerging models of provider-payment reform. N Engl J Med 2008;359:1197–1200. 7. H.R.3590 - Patient Protection and Affordable Care Act, 2009. 8. Blumenthal D. Part 1: Quality of care--what is it? N Engl J Med 1996;335:891– 894. 9. Lohr KN, Donaldson MS, Medicare H-WJ. A strategy for quality assurance, V: Quality of care in a changing health care environment. QRB Qual Rev Bull 1992;18:120 –126. 10. Hurtado MP, Swift EK, Corrigan JM. Crossing the quality chasm: a new health system for the 21st century. Washington, DC: National Academy Press, 2001. 11. Agency for Healthcare Research and Quality. Your Guide to Choosing Quality Care. Available at: http://archive.ahrq.gov/consumer/qnt/. Accessed July 30, 2010.
566
SPENCER D. DORN
12. Donabedian A. The quality of care. How can it be assessed? JAMA 1988;260:1743–1748. 13. Donabedian A. Evaluating the quality of medical care. Milbank Mem Fund Q 1966;44 Suppl:166 –206. 14. Meyer GS, Massagli MP. The forgotten component of the quality triad: can we still learn something from “structure”? Jt Commission. J Qual Improv 2001;27:484 – 493. 15. Rubin HR, Pronovost P, Diette GB. The advantages and disadvantages of process-based measures of health care quality. Int J Qual Health Care 2001;13:469 – 474. 16. Mant J, Hicks N. Health status measurement and the assessment of medical care. Int J Qual Health Care 1996;8:107–109. 17. Romano P, Hussey P, Ritley D. Selecting Quality and Resource Use Measures: A Decision Guide for Community Quality Collaboratives. AHRQ Publication No. 09(10)-0073. Rockville, MD: Agency for Healthcare Research and Quality, 2010. 18. Pronovost PJ, Berenholtz SM, Goeschel CA. Improving the quality of measurement and evaluation in quality improvement efforts. Am J Med Qual 2008;23:143–146. 19. Gold MR, Siegel JE, Russell LB, et al. Cost-effectiveness in health and medicine. New York: Oxford University Press, 1996. 20. Baker N, Whittington JW, Resar RK, et al. Reducing Costs Through the Appropriate Use of Specialty Services. IHI Innovation Series white paper. Cambridge, MA: Insitute for Healthcare Improvement, 2010. 21. AQA. AQA principles of efficiency measures. Available at: http:// www.aqaalliance.org/files/PrinciplesofEfficiencyMeasurement. pdf. Accessed July 22, 2010. 22. AQA Alliance. AQA principles of “efficiency” measures. Available at: http://www.aqaalliance.org/files/PrinciplesofEfficiencyMeasurement April2006.doc. Accessed August 5, 2010. 23. McGlynn EA. Identifying, categorizing and evaluating health care efficiency measures. Final report. AHRQ Publication No. 080030. Rockville, MD: Agency for Healthcare Research and Quality, 2008. 24. Young PL, Olsen L, McGinnis JM. Value in health care: accounting for cost, quality, safety, outcomes, and innovation. Washington, DC: National Academies Press, 2010. 25. McGlynn EA, Asch SM. Developing a clinical performance measure. Am J Prev Med 1998;14:14 –21. 26. National Quality Forum. Measurement framework: Evaluating efficency across patient-focused episodes of care. Washington, DC: NQF, 2009. 27. Nyweide DJ, Weeks WB, Gottlieb DJ, et al. Relationship of primary care physicians’ patient caseload with measurement of quality and cost performance. JAMA 2009;302:2444 –2450.
CLINICAL GASTROENTEROLOGY AND HEPATOLOGY Vol. 9, No. 7
28. Adams JL, Mehrotra A, Thomas JW, et al. Physician cost profiling-reliability and risk of misclassification. N Engl J Med 2010;362: 1014 –1021. 29. Mehrotra A, Adams JL, Thomas JW, et al. Cost profiles: should the focus be on individual physicians or physician groups? Health Aff. (Millwood) 2010;29:1532–1538. 30. O’Brien SM, Shahian DM, DeLong ER, et al. Quality measurement in adult cardiac surgery: part 2–statistical considerations in composite measure scoring and provider rating. Ann Thorac Surg 2007;83:S13–S26. 31. Hussey PS, Sorbero ME, Mehrotra A, et al. Episode-based performance measurement and payment: making it a reality. Health Aff. (Millwood) 2009;28:1406 –1417. 32. Pacific Business Group on Health. Advancing physician performance measurement. Using administrative data to assess physician quality and efficiency, 2005. Available at: http://www. pbgh.org/storage/documents/reports/PBGHP3Report_09-01-05final. pdf. 33. Mehrotra A, Adams JL, Thomas JW, et al. The effect of different attribution rules on individual physician cost profiles. Ann Intern Med 2010;152:649 – 654. 34. Taubes G. Epidemiology faces its limits. Science 1995;269: 164 –169. 35. Benin AL, Vitkauskas G, Thornquist E, et al. Validity of using an electronic medical record for assessing quality of care in an outpatient setting. Med Care 2005;43:691– 698. 36. Reinhardt UE. The pricing of U.S. hospital services: chaos behind a veil of secrecy. Health Aff. (Millwood) 2006;25:57– 69.
Reprint requests Address requests for reprints to: Spencer D. Dorn, MD, MPH, Division of Gastroenterology and Hepatology, University of North Carolina at Chapel Hill, CB 7080, Chapel Hill, North Carolina, 27599-7080. e-mail: [email protected] ; fax: (919) 966-2250. Acknowledgments The author thanks Drs John Allen and Joel Brill for their advice on earlier drafts of the manuscript. Conflicts of interest The author discloses no conflicts. Funding Grant Support: Spencer Dorn is supported by NIH K12HS019468.