- Email: [email protected]
CMS Head CT Rule Under Fire
contentMain;cbsCarousel. Accessed March 2, 2012. 27. Baggerly KA, Coombes KR. Deriving chemosensitivity from cell lines: forensic bioinformatics and reproducible research in high-throughput biology. Ann Appl Stat. 2009;3:1309-1334. 28. Oransky I. The Anil Potti retraction record so far. Retraction Watch, February 14, 2012. Available at: http://retractionwatch.wordpress.com/ 2012/02/14/the-anil-potti-retractionrecord-so-far/. Accessed March 3, 2012. 29. Potti A, Dressman HK, Bild A, et al. Genomic signatures to guide the use of chemotherapeutics. Nat Med. 2006;12: 1294-1300. Retracted January 7, 2011.
30.
31. 32.
33.
doi:10.1038/nm1491. Available at: http://www.nature.com/nm/journal/ v12/n11/full/nm1491.html. Accessed March 7, 2012. Truzzi M. On the extraordinary: an attempt at clarification. Zetetic Scholar. 1978;1:11. Baggerly K. Disclose all data in publications. Nature. 2010;467:401. Institute of Medicine, Board on Health Care Services. Review of omics-based tests for predicting patient outcomes in clinical trials. PIN IOM-HCS-10-06. Available at: http://www.iom.edu/ activities/research/omicsbasedtests. aspx. Accessed March 7, 2012. Rising K, Bacchetti P, Bero L. Reporting bias in drug trials submitted to the Food
CMS Head CT Rule Under Fire by MARYN MCKENNA Special Contributor to Annals News & Perspective
A
federal rule that will score and publicly rank emergency departments (EDs) on computed tomography (CT) scan use as a way of forcing down increasing rates of imaging is provoking ire within emergency medicine and gaining attention from the medical trade press. The rule, an imaging efficiency measure promulgated by the Centers for Medicare & Medicaid Services (CMS) of the US Department of Health and Human Services and known as OP-15, “Use of Brain Computed Tomography in the Emergency Department for Atraumatic Headache,” uses diagnostic codes in Medicare billing data to determine whether CT scans were clinically appropriate.1 In its current form, it is opposed by the American College of Emergency Physicians (ACEP), publishers of Annals of Emergency Medicine, and is the subject of an article2 first published online in Annals in February. February’s article coauthored by Jeremiah D. Schuur, MD, MHS,2 director of quality and patient safety and director of performance improvement in the Department of Emer20A Annals of Emergency Medicine
gency Medicine at Brigham and Women’s Hospital, Boston, MA, found that the OP-15 rule “is not reliable, valid, or accurate and may produce misleading information about hospital ED performance.” Dr. Schuur and his coauthors, from 21 EDs across the United States, reviewed records of 748 patient visits that CMS judged to have resulted in an inappropriate CT in a “dry run” review of 2009 data. They found that for 623 of the patients, 83%, the scans were in fact appropriate, judged either by exclusions specified in the rule but not coded in the charts or by consensus standards promulgated by ACEP and other experts. “It is important for physicians, hospitals, and payers to work together to develop systems that ensure that every CT that is performed is appropriate,” Dr. Schuur said when the study was published online.3 “Our research finds that OP-15 may not be a valid measure of imaging . . . and that, when calculated from Medicare claims, [OP-15] can produce unreliable data.”
TROUBLING TREND
T
he OP-15 rule stems from recognition of a trend that no one, not even emergency physicians opposed to the
and Drug Administration: review of publication and presentation. PLoS Med. 2008;5:e217. 34. Hart B, Lundh A, Bero L. Effect of reporting bias on meta-analyses of drug trials: reanalysis of meta-analyses. BMJ. 2012;344-354. 35. Ross JS, Tse T, Zarin DA, et al. Publication of NIH funded trials registered in ClinicalTrials.gov: cross sectional analysis. BMJ. 2012;344. 36. Echt DS, Liebson PR, Mitchell LB, et al. Mortality and morbidity in patients receiving encainide, flecainide, or placebo—the Cardiac Arrhythmia Suppression Trial. N Engl J Med. 1991; 324:781-788.
rule, denies: CT scan use has increased at remarkable speed, and ED use is an important contributor. A 2007 study estimated that all CT use in US hospitals increased from 3 million scans in 1980 to 62 million in 2009, with acute care accounting for three fourths of those scans.4 A 2010 study conducted in the ED of just 1 hospital, St. Luke’s–Roosevelt Hospital Center in New York City, found that CT use more than doubled between 2001 and 2007, from 51 scans per 1,000 patient visits to 106 per 1,000.5 The biggest contributors to that increase were chest CT, which increased 6-fold, and neck CT, which increased 5-fold. Head CT, such as described in the OP-15 rule, represented the highest absolute number of scans, though its use in the St. Luke’s–Roosevelt ED increased only 60%. However, despite a broadly shared intuition that emergency CT use and specifically head CT use were increasing, no national data existed to prove it. At the 2010 ACEP Research Forum, Ali S. Raja, MD, MBA, presented an analysis of data from the 2007 edition of the National Hospital Ambulatory Medical Care Survey. Dr. Raja and coauthors estimated that of the 117 million ED visits in the United States that year, 6.7%, or 7.8 million, involved head CTs.6 In 2011, Keith E. Kocher, MD, MPH, of the University of Michigan, Ann Arbor, used the same data set to calculate changes in ED CT use across the United States between 1996 and 2007. Across those years, visits to EDs increased by 30%, but CT use in EDs increased 330%. By 2007, 1 of every 7 ED patients reVolume , . : August
ceived a CT scan as part of their evaluation. Headache was not one of the conditions causing the most CT use, but the proportion of patients complaining of headache who received a CT increased from 16.4% in 1996 to 32.5% in 2007.7 An accompanying editorial observed that CT use in EDs had increased by 14% in each year of the study period, adding, “if it were sustained, CT use would double roughly every 5 years so that we should expect almost 25% of ED visits this year [2011] to entail at least 1 CT.”8 That editorial also predicted that the standard of emergency care would impose no natural check or balance on CT use, with no significant consequences imposed for overuse and the discovery of illness in unimaged patients—plus the threat of lawsuits— discouraging underuse. “If physicians observe that many of their colleagues are using CT more liberally, it begins to seem at least okay and eventually may persuade them to join the crowd,” said the author, Robert L. Wears, MD, PhD, of the University of Florida, Jacksonville, “because, after all, the ‘standard of care’ is that which most physicians do.” It was against that backdrop that CMS proposed OP-15, one of a slate of outpatient care improvement measures. OP-15 was intended to suppress overuse, reduce health care spending, and diminish the possibility of patients being exposed to excessive ionizing radiation, with subsequent bad outcomes. The rule, relying on International Classification of Diseases, Ninth Revision codes, was drafted by CMS subcontractors the Lewin Group and submitted to the National Quality Forum’s Imaging Efficiency Steering Committee. A separate measure addressing CT use for headache, based on ACEP’s clinical policy, was submitted at the same time by Dr. Schuur, Dr. Raja, and Ron Walls, MD, chief of emergency medicine at Brigham and Women’s. The National Quality Forum rejected both measures. In the case of OP-15, the committee initially voted for it but reconsidered after public comment (including from ACEP) and voted 12 to 8 against endorsing it.9 Nevertheless, CMS in 2011 added the measure to its outpatient prospective payment system for 2012, with the eventual intention of publishing comparative rankings based on Volume , . : August
EDs’ scores on its Hospital Compare Web site (http://www.hospitalcompare. hhs.gov/). To gauge the workability of the measure, the agency last summer conducted the “dry run” with 2009 Medicare data, scored the results, and released the scores back to the hospitals from which data were used.
FEEDBACK FAILING
C
MS granted Annals an interview about the process last fall but would not allow its personnel to speak except on background. The CMS officials said the agency had already received protests from a number of hospitals about mismatches between the OP-15 results and their patients’ charts. Those reports were being considered useful feedback but were unlikely to derail full implementation, the officials said. Emergency medicine’s objections to OP-15 are based on part on the process by which the measure was enacted. ACEP’s formal response to the proposal, through a letter to then-CMS administrator Donald Berwick, MD, objected to CMS’s ignoring the National Quality Forum vote, stating: “this measure failed to meet the consensus requirement as it was not recommended for endorsement by the NQF.”10 More broadly, though, emergency physicians are concerned that the rule’s reliance of billing codes will cause it to inappropriately score down CTs that were ordered for good clinical reasons. “There are too many people who have a very reasonable reason to have a CT scan, whose data does not get into the billing information,” said Sandra Schneider, MD, who was president of ACEP when the rule was promulgated. Chronic conditions, she pointed out, rarely make it into billing codes if they are not the primary complaint yet may exert a profound effect on headache symptoms. Richard Griffey, MD, MPH, of Washington University in St. Louis School of Medicine, performed his own informal review last year of 50 charts of patients who received CT, using the new CMS rule to judge whether the scans were appropriate. “A large percentage of them ended up either having one of CMS’s own exclusion criteria or being incorrectly coded,” he said.
Compounding the problem is that there is scant evidence for or against CT use in the population who might need it the most: elderly patients, who experience the largest proportion of intracranial events yet have been left out of most studies of headache and head CT. “There is very little evidence to guide physicians on when it is safe not to do a CT,” Dr. Schuur said. The rule permits CT for atraumatic headache under a long list of exclusions, but critics point out that anticoagulants, for instance, are not mentioned, despite hemorrhages triggered by those drugs being a prime cause of new headache in elderly patients. It permits CT use in patients who have or will undergo lumbar puncture but does not address patients to whom lumbar puncture is recommended but who refuse. In their article available online at: http://www.annemergmed.com/article/ S0196-0644(11)01989-5/abstract, Dr. Schuur et al2 offer several clinical scenarios in which the CMS rule would label CTs as inappropriate. They include an 84-year-old woman receiving warfarin who slips and hits her head in a fall, a 27-year-old woman with a sudden excruciating headache who undergoes a CT that reveals nothing and who subsequently refuses a lumbar puncture, and a 48-year-old man with a previous subdural hematoma and empyema who receives a CT to check for recurrence. Underlying the description of those actual scenarios is a clear concern that the CMS rule, and the threat of poor performance rankings and diminished Medicare reimbursement that come with it, could cause emergency physicians not to order CTs when they are needed. “There really is no way other than this test to tell someone who is having a really bad migraine from someone with a subarachnoid hemorrhage,” said Dr. Schneider, who, when she was in high school, watched a friend almost die and subsequently experience paralysis from just such a bleeding event.11
UNRELIABLE AND INVALID
T
he multihospital review in Annals raises a number of questions, Dr. Schuur said. “If it is not reliable, and our review shows it is not, then it may not accurately reflect higher inappropriate use Annals of Emergency Medicine 21A
even in hospitals where that exists,” he said. “If it isn’t valid, Medicare will have a difficult time convincing providers to change practice in an area where there is not good evidence.” CMS has made no indication that it will reverse itself and cancel OP-15, though it has delayed the planned implementation of public reporting to allow for “thoughtful refinement.”12 That has left emergency physicians confused about what they might suggest to CMS to improve the rule, such as crossmatching billing and clinical data to give a full picture of appropriateness. A comment by Dr. Raja, written after the National Quality Forum voted down the rule and before CMS adopted it anyway, seems unfortunately prescient. “Had a poorly conceived utilization measure been approved,” he wrote, “it may have taken years to reverse. Meanwhile, emergency physicians would have been pressured to create workarounds and modify documentation to meet the utilization measure’s specifications.”13 Section editor: Truman J. Milling, Jr, MD. Funding and support: By Annals policy, all authors are required to disclose any and all commercial, financial, and other relationships in any way related to the subject of this article as per ICMJE conflict of interest guidelines (see www.icmje.org). The author has stated that no such relationships exist. The views expressed in News and Perspective are those of the authors, and do
not reflect the views and opinions of the American College of Emergency Physicians or the editorial board of Annals of Emergency Medicine. http://dx.doi.org/10.1016/j.annemergmed. 2012.06.012
REFERENCES 1. CMS. OP-15: use of brain computed tomography (CT) in the emergency department for atraumatic headache. 2011. Available at: http://www. qualitynet.org/dcs/ContentServer?c⫽ Page&pagename⫽QnetPublic%2FPage% 2FQnetTier2&cid⫽1228695266120. Accessed July 2, 2011. 2. Schuur JD, Brown MD, Cheung DS, et al. Assessment of Medicare’s imaging efficiency measure for emergency department patients with atraumatic headache. Ann Emerg Med. Available online at: http://www.annemergmed. com/article/S0196-0644(11)019895/abstract. Accessed June 22, 2012. 3. ACEP. Government rule designed to limit CT scans in ERs is unreliable, invalid and inaccurate [press release]. Available at: http://www.acep.org/Content. aspx?id⫽83915. Accessed February 25, 2012. 4. Brenner DJ, Hall EJ. Computed tomography—an increasing source of radiation exposure. N Engl J Med. 2007; 357:2277-2284. 5. Lee J, Kirschner J, Pawa S, et al. Computed tomography use in the adult emergency department of an academic urban hospital from 2001 to 2007. Ann Emerg Med. 2010;56:591-596.
The Gold Standard’s Flexible Alloy Adaptive Designs on the Advance
by WILLIAM B. MILLARD, PhD Special Contributor to Annals News & Perspective
A
n experiment’s integrity depends on its methodological objectivity and consistency: this is a cardinal principle of clinical research and of research 22A Annals of Emergency Medicine
generally. The progress of knowledge during decades of investigation has confirmed the indispensable value of randomization and blinding, the features that ensure greater rigor than anecdotal case reports, cohort analyses, and other study types can provide. Yet a growing number of researchers recommend an early look under the hood, with orderly interim adjustments of
6. Raja A. Utilization of neuroimaging in US emergency departments [2010 ACEP Research Forum abstract #172]. Ann Emerg Med. 2010;56:S57. 7. Kocher KE, Meurer WJ, Fazel R, et al. National trends in use of computed tomography in the emergency department. Ann Emerg Med. 2011;58: 452-462.e3. 8. Wears R. The hunting of the snark, 2011. Ann Emerg Med. 2011;58:465-466. 9. National Quality Forum. National Voluntary Consensus Standards for Imaging Efficiency: A Consensus Report. Washington, DC: National Quality Forum; 2011. Available at: http://www. qualityforum.org/projects/imaging_ efficiency.aspx. Accessed December 12, 2011. 10. ACEP. Medicare program: proposed changes to the hospital outpatient prospective payment system for 2011. Available at: http://www.acep.org/ WorkArea/DownloadAsset.aspx?id⫽ 49348. Accessed July 18, 2011. 11. Blachly L. Proposed measures on CT scans cause concern. ACEP News. Available at: http://www.acep.org/ Content.aspx?id⫽80788. Accessed August 15, 2011. 12. CMS. http://www.qualitynet.org/dcs/ ContentServer?c⫽Page&pagename⫽ QnetPublic%2FPage%2FQnetTier2&cid⫽ 1228695266120. Accessed June 22, 2012. 13. Raja AS, Walls RM, Schuur JD. Decreasing use of high-cost imaging: the danger of utilization-based performance measures. Ann Emerg Med. 2010;56:597-599.
certain variables, as a way to accelerate and improve the performance of clinical trials. The randomized clinical trial remains the acknowledged criterion standard for therapeutic discoveries, but in practice it is a complex mechanism, not entirely untarnished. Adaptive trial designs, say researchers who have implemented them and studied them, can put researchers on a fast track to reliable, realistic information. The clinical and biostatistical communities remain largely unfamiliar with adaptive designs, and certain mythologies (including the assumption that regulators are opposed) have hindered their adoption; the Food and Drug Administration (FDA) and National Institutes of Health (NIH), in fact, are not only allowing these innovations but also actively taking Volume , . : August
30.
31. 32.
33.
doi:10.1038/nm1491. Available at: http://www.nature.com/nm/journal/ v12/n11/full/nm1491.html. Accessed March 7, 2012. Truzzi M. On the extraordinary: an attempt at clarification. Zetetic Scholar. 1978;1:11. Baggerly K. Disclose all data in publications. Nature. 2010;467:401. Institute of Medicine, Board on Health Care Services. Review of omics-based tests for predicting patient outcomes in clinical trials. PIN IOM-HCS-10-06. Available at: http://www.iom.edu/ activities/research/omicsbasedtests. aspx. Accessed March 7, 2012. Rising K, Bacchetti P, Bero L. Reporting bias in drug trials submitted to the Food
CMS Head CT Rule Under Fire by MARYN MCKENNA Special Contributor to Annals News & Perspective
A
federal rule that will score and publicly rank emergency departments (EDs) on computed tomography (CT) scan use as a way of forcing down increasing rates of imaging is provoking ire within emergency medicine and gaining attention from the medical trade press. The rule, an imaging efficiency measure promulgated by the Centers for Medicare & Medicaid Services (CMS) of the US Department of Health and Human Services and known as OP-15, “Use of Brain Computed Tomography in the Emergency Department for Atraumatic Headache,” uses diagnostic codes in Medicare billing data to determine whether CT scans were clinically appropriate.1 In its current form, it is opposed by the American College of Emergency Physicians (ACEP), publishers of Annals of Emergency Medicine, and is the subject of an article2 first published online in Annals in February. February’s article coauthored by Jeremiah D. Schuur, MD, MHS,2 director of quality and patient safety and director of performance improvement in the Department of Emer20A Annals of Emergency Medicine
gency Medicine at Brigham and Women’s Hospital, Boston, MA, found that the OP-15 rule “is not reliable, valid, or accurate and may produce misleading information about hospital ED performance.” Dr. Schuur and his coauthors, from 21 EDs across the United States, reviewed records of 748 patient visits that CMS judged to have resulted in an inappropriate CT in a “dry run” review of 2009 data. They found that for 623 of the patients, 83%, the scans were in fact appropriate, judged either by exclusions specified in the rule but not coded in the charts or by consensus standards promulgated by ACEP and other experts. “It is important for physicians, hospitals, and payers to work together to develop systems that ensure that every CT that is performed is appropriate,” Dr. Schuur said when the study was published online.3 “Our research finds that OP-15 may not be a valid measure of imaging . . . and that, when calculated from Medicare claims, [OP-15] can produce unreliable data.”
TROUBLING TREND
T
he OP-15 rule stems from recognition of a trend that no one, not even emergency physicians opposed to the
and Drug Administration: review of publication and presentation. PLoS Med. 2008;5:e217. 34. Hart B, Lundh A, Bero L. Effect of reporting bias on meta-analyses of drug trials: reanalysis of meta-analyses. BMJ. 2012;344-354. 35. Ross JS, Tse T, Zarin DA, et al. Publication of NIH funded trials registered in ClinicalTrials.gov: cross sectional analysis. BMJ. 2012;344. 36. Echt DS, Liebson PR, Mitchell LB, et al. Mortality and morbidity in patients receiving encainide, flecainide, or placebo—the Cardiac Arrhythmia Suppression Trial. N Engl J Med. 1991; 324:781-788.
rule, denies: CT scan use has increased at remarkable speed, and ED use is an important contributor. A 2007 study estimated that all CT use in US hospitals increased from 3 million scans in 1980 to 62 million in 2009, with acute care accounting for three fourths of those scans.4 A 2010 study conducted in the ED of just 1 hospital, St. Luke’s–Roosevelt Hospital Center in New York City, found that CT use more than doubled between 2001 and 2007, from 51 scans per 1,000 patient visits to 106 per 1,000.5 The biggest contributors to that increase were chest CT, which increased 6-fold, and neck CT, which increased 5-fold. Head CT, such as described in the OP-15 rule, represented the highest absolute number of scans, though its use in the St. Luke’s–Roosevelt ED increased only 60%. However, despite a broadly shared intuition that emergency CT use and specifically head CT use were increasing, no national data existed to prove it. At the 2010 ACEP Research Forum, Ali S. Raja, MD, MBA, presented an analysis of data from the 2007 edition of the National Hospital Ambulatory Medical Care Survey. Dr. Raja and coauthors estimated that of the 117 million ED visits in the United States that year, 6.7%, or 7.8 million, involved head CTs.6 In 2011, Keith E. Kocher, MD, MPH, of the University of Michigan, Ann Arbor, used the same data set to calculate changes in ED CT use across the United States between 1996 and 2007. Across those years, visits to EDs increased by 30%, but CT use in EDs increased 330%. By 2007, 1 of every 7 ED patients reVolume , . : August
ceived a CT scan as part of their evaluation. Headache was not one of the conditions causing the most CT use, but the proportion of patients complaining of headache who received a CT increased from 16.4% in 1996 to 32.5% in 2007.7 An accompanying editorial observed that CT use in EDs had increased by 14% in each year of the study period, adding, “if it were sustained, CT use would double roughly every 5 years so that we should expect almost 25% of ED visits this year [2011] to entail at least 1 CT.”8 That editorial also predicted that the standard of emergency care would impose no natural check or balance on CT use, with no significant consequences imposed for overuse and the discovery of illness in unimaged patients—plus the threat of lawsuits— discouraging underuse. “If physicians observe that many of their colleagues are using CT more liberally, it begins to seem at least okay and eventually may persuade them to join the crowd,” said the author, Robert L. Wears, MD, PhD, of the University of Florida, Jacksonville, “because, after all, the ‘standard of care’ is that which most physicians do.” It was against that backdrop that CMS proposed OP-15, one of a slate of outpatient care improvement measures. OP-15 was intended to suppress overuse, reduce health care spending, and diminish the possibility of patients being exposed to excessive ionizing radiation, with subsequent bad outcomes. The rule, relying on International Classification of Diseases, Ninth Revision codes, was drafted by CMS subcontractors the Lewin Group and submitted to the National Quality Forum’s Imaging Efficiency Steering Committee. A separate measure addressing CT use for headache, based on ACEP’s clinical policy, was submitted at the same time by Dr. Schuur, Dr. Raja, and Ron Walls, MD, chief of emergency medicine at Brigham and Women’s. The National Quality Forum rejected both measures. In the case of OP-15, the committee initially voted for it but reconsidered after public comment (including from ACEP) and voted 12 to 8 against endorsing it.9 Nevertheless, CMS in 2011 added the measure to its outpatient prospective payment system for 2012, with the eventual intention of publishing comparative rankings based on Volume , . : August
EDs’ scores on its Hospital Compare Web site (http://www.hospitalcompare. hhs.gov/). To gauge the workability of the measure, the agency last summer conducted the “dry run” with 2009 Medicare data, scored the results, and released the scores back to the hospitals from which data were used.
FEEDBACK FAILING
C
MS granted Annals an interview about the process last fall but would not allow its personnel to speak except on background. The CMS officials said the agency had already received protests from a number of hospitals about mismatches between the OP-15 results and their patients’ charts. Those reports were being considered useful feedback but were unlikely to derail full implementation, the officials said. Emergency medicine’s objections to OP-15 are based on part on the process by which the measure was enacted. ACEP’s formal response to the proposal, through a letter to then-CMS administrator Donald Berwick, MD, objected to CMS’s ignoring the National Quality Forum vote, stating: “this measure failed to meet the consensus requirement as it was not recommended for endorsement by the NQF.”10 More broadly, though, emergency physicians are concerned that the rule’s reliance of billing codes will cause it to inappropriately score down CTs that were ordered for good clinical reasons. “There are too many people who have a very reasonable reason to have a CT scan, whose data does not get into the billing information,” said Sandra Schneider, MD, who was president of ACEP when the rule was promulgated. Chronic conditions, she pointed out, rarely make it into billing codes if they are not the primary complaint yet may exert a profound effect on headache symptoms. Richard Griffey, MD, MPH, of Washington University in St. Louis School of Medicine, performed his own informal review last year of 50 charts of patients who received CT, using the new CMS rule to judge whether the scans were appropriate. “A large percentage of them ended up either having one of CMS’s own exclusion criteria or being incorrectly coded,” he said.
Compounding the problem is that there is scant evidence for or against CT use in the population who might need it the most: elderly patients, who experience the largest proportion of intracranial events yet have been left out of most studies of headache and head CT. “There is very little evidence to guide physicians on when it is safe not to do a CT,” Dr. Schuur said. The rule permits CT for atraumatic headache under a long list of exclusions, but critics point out that anticoagulants, for instance, are not mentioned, despite hemorrhages triggered by those drugs being a prime cause of new headache in elderly patients. It permits CT use in patients who have or will undergo lumbar puncture but does not address patients to whom lumbar puncture is recommended but who refuse. In their article available online at: http://www.annemergmed.com/article/ S0196-0644(11)01989-5/abstract, Dr. Schuur et al2 offer several clinical scenarios in which the CMS rule would label CTs as inappropriate. They include an 84-year-old woman receiving warfarin who slips and hits her head in a fall, a 27-year-old woman with a sudden excruciating headache who undergoes a CT that reveals nothing and who subsequently refuses a lumbar puncture, and a 48-year-old man with a previous subdural hematoma and empyema who receives a CT to check for recurrence. Underlying the description of those actual scenarios is a clear concern that the CMS rule, and the threat of poor performance rankings and diminished Medicare reimbursement that come with it, could cause emergency physicians not to order CTs when they are needed. “There really is no way other than this test to tell someone who is having a really bad migraine from someone with a subarachnoid hemorrhage,” said Dr. Schneider, who, when she was in high school, watched a friend almost die and subsequently experience paralysis from just such a bleeding event.11
UNRELIABLE AND INVALID
T
he multihospital review in Annals raises a number of questions, Dr. Schuur said. “If it is not reliable, and our review shows it is not, then it may not accurately reflect higher inappropriate use Annals of Emergency Medicine 21A
even in hospitals where that exists,” he said. “If it isn’t valid, Medicare will have a difficult time convincing providers to change practice in an area where there is not good evidence.” CMS has made no indication that it will reverse itself and cancel OP-15, though it has delayed the planned implementation of public reporting to allow for “thoughtful refinement.”12 That has left emergency physicians confused about what they might suggest to CMS to improve the rule, such as crossmatching billing and clinical data to give a full picture of appropriateness. A comment by Dr. Raja, written after the National Quality Forum voted down the rule and before CMS adopted it anyway, seems unfortunately prescient. “Had a poorly conceived utilization measure been approved,” he wrote, “it may have taken years to reverse. Meanwhile, emergency physicians would have been pressured to create workarounds and modify documentation to meet the utilization measure’s specifications.”13 Section editor: Truman J. Milling, Jr, MD. Funding and support: By Annals policy, all authors are required to disclose any and all commercial, financial, and other relationships in any way related to the subject of this article as per ICMJE conflict of interest guidelines (see www.icmje.org). The author has stated that no such relationships exist. The views expressed in News and Perspective are those of the authors, and do
not reflect the views and opinions of the American College of Emergency Physicians or the editorial board of Annals of Emergency Medicine. http://dx.doi.org/10.1016/j.annemergmed. 2012.06.012
REFERENCES 1. CMS. OP-15: use of brain computed tomography (CT) in the emergency department for atraumatic headache. 2011. Available at: http://www. qualitynet.org/dcs/ContentServer?c⫽ Page&pagename⫽QnetPublic%2FPage% 2FQnetTier2&cid⫽1228695266120. Accessed July 2, 2011. 2. Schuur JD, Brown MD, Cheung DS, et al. Assessment of Medicare’s imaging efficiency measure for emergency department patients with atraumatic headache. Ann Emerg Med. Available online at: http://www.annemergmed. com/article/S0196-0644(11)019895/abstract. Accessed June 22, 2012. 3. ACEP. Government rule designed to limit CT scans in ERs is unreliable, invalid and inaccurate [press release]. Available at: http://www.acep.org/Content. aspx?id⫽83915. Accessed February 25, 2012. 4. Brenner DJ, Hall EJ. Computed tomography—an increasing source of radiation exposure. N Engl J Med. 2007; 357:2277-2284. 5. Lee J, Kirschner J, Pawa S, et al. Computed tomography use in the adult emergency department of an academic urban hospital from 2001 to 2007. Ann Emerg Med. 2010;56:591-596.
The Gold Standard’s Flexible Alloy Adaptive Designs on the Advance
by WILLIAM B. MILLARD, PhD Special Contributor to Annals News & Perspective
A
n experiment’s integrity depends on its methodological objectivity and consistency: this is a cardinal principle of clinical research and of research 22A Annals of Emergency Medicine
generally. The progress of knowledge during decades of investigation has confirmed the indispensable value of randomization and blinding, the features that ensure greater rigor than anecdotal case reports, cohort analyses, and other study types can provide. Yet a growing number of researchers recommend an early look under the hood, with orderly interim adjustments of
6. Raja A. Utilization of neuroimaging in US emergency departments [2010 ACEP Research Forum abstract #172]. Ann Emerg Med. 2010;56:S57. 7. Kocher KE, Meurer WJ, Fazel R, et al. National trends in use of computed tomography in the emergency department. Ann Emerg Med. 2011;58: 452-462.e3. 8. Wears R. The hunting of the snark, 2011. Ann Emerg Med. 2011;58:465-466. 9. National Quality Forum. National Voluntary Consensus Standards for Imaging Efficiency: A Consensus Report. Washington, DC: National Quality Forum; 2011. Available at: http://www. qualityforum.org/projects/imaging_ efficiency.aspx. Accessed December 12, 2011. 10. ACEP. Medicare program: proposed changes to the hospital outpatient prospective payment system for 2011. Available at: http://www.acep.org/ WorkArea/DownloadAsset.aspx?id⫽ 49348. Accessed July 18, 2011. 11. Blachly L. Proposed measures on CT scans cause concern. ACEP News. Available at: http://www.acep.org/ Content.aspx?id⫽80788. Accessed August 15, 2011. 12. CMS. http://www.qualitynet.org/dcs/ ContentServer?c⫽Page&pagename⫽ QnetPublic%2FPage%2FQnetTier2&cid⫽ 1228695266120. Accessed June 22, 2012. 13. Raja AS, Walls RM, Schuur JD. Decreasing use of high-cost imaging: the danger of utilization-based performance measures. Ann Emerg Med. 2010;56:597-599.
certain variables, as a way to accelerate and improve the performance of clinical trials. The randomized clinical trial remains the acknowledged criterion standard for therapeutic discoveries, but in practice it is a complex mechanism, not entirely untarnished. Adaptive trial designs, say researchers who have implemented them and studied them, can put researchers on a fast track to reliable, realistic information. The clinical and biostatistical communities remain largely unfamiliar with adaptive designs, and certain mythologies (including the assumption that regulators are opposed) have hindered their adoption; the Food and Drug Administration (FDA) and National Institutes of Health (NIH), in fact, are not only allowing these innovations but also actively taking Volume , . : August