- Email: [email protected]
The Hunting of the Snark, 2011
HEALTH POLICY/EDITORIAL
The Hunting of the Snark, 2011 Robert L. Wears, MD, MS From the Department of Emergency Medicine, University of Florida, Jacksonville, FL; and Clinical Safety Research Unit, Imperial College London, UK. 0196-0644/$-see front matter Copyright © 2011 by the American College of Emergency Physicians. doi:10.1016/j.annemergmed.2011.07.031
A podcast for this article is available at www.annemergmed.com.
SEE RELATED ARTICLE, P. 452. [Ann Emerg Med. 2011;58:465-467.] In this issue of Annals, Kocher et al1 provide quantitative detail about a change in emergency department (ED) practice, of which we are all generally aware: the explosive growth in the use of computed tomography (CT). Their data should usefully inform the ongoing discussions about why and how we use imaging and about the risks and benefits of that use.2-5 They observe that the growth in imaging has been substantial across a broad range of complaints during a 12-year period of study and further state that much of this increase may have been beneficial, in that the likelihood of hospitalization (or transfer) after a CT was less than half what it was at the beginning of their study (1996). Although this might partly have come from scanning patients who were less ill to begin with,6 it seems clear that in a substantial proportion of cases, the CT results allowed what would have otherwise been a hospitalization to be converted to a discharge with outpatient follow-up,7 a generally beneficial result for both patients and the health care system. Although the article by Kocher et al1 notes that the significant growth in CT use per ED visit, it does not emphasize its most remarkable finding, namely, that this growth has been exponential, increasing at a steady rate of 14% per year, with no apparent slowing during the 12-year study period; adjustment for patient characteristics (eg, an increase in elderly patients) does not change this fundamental result. Most of us would be deliriously happy if we could obtain half this rate of growth in our incomes or retirement plans; if it were sustained, CT use would double roughly every 5 years so that we should expect almost 25% of ED visits this year (2011) to entail at least 1 CT. Clearly, this is not sustainable; we would soon have half the population used in scanning the other half. Fortunately, “no tree grows to the sky”; because chest, abdominal, and flank pain patients, combined with trauma patients, compose roughly 30% of ED visits, we should expect some leveling off of CT use at least around 30% of ED visits.8,9 But do we really want to practice in a world in which, in effect, every complaint from bregma to smegma and every motor vehicle crash results in a CT? This pursuit of accuracy may increase overload in already beleaguered EDs to debilitating levels. Volume , . : November
Exponential growexth occurs mostly in 2 situations: breakout and positive feedback. Breakout circumstances are characterized by novelty: an organism arrives in a new ecologic niche, with an abundance of resources and few predators; or a new technology offers capabilities never before realized in an unexploited market. Clearly, some of the growth in CT use can be described in a breakout narrative; in addition, as new capabilities were developed (eg, CT angiography), minibreakouts, no doubt propelled, continued increase. But CTs’ principal breakout occurred in the 1970s to 1980s; by the mid-1990s, we should have expected subsequent advances to have contributed smaller and smaller fractions to growth, not the larger and larger increases reported by Kocher et al.1 Positive feedback, then, is an alternative explanation for exponential growth. Positive feedback loops need an initiating and sustaining mechanism (think of the initial capital investment and the annual interest rate in compound interest problems). A plausible initiating mechanism is not too difficult to imagine. A “surprise” result, such as an aortic dissection found in a patient with vague chest pain and hard-to-pin-down arm and leg numbness, or a large pulmonary embolism in a patient who feels “weak and faintified,” confronts us with the uncertainties of traditional medical problem recognition while offering an escape to greater certainty: just scan more of those vague, borderline cases and voila!— uncertainty gone, to everyone’s great relief. The occasional “near miss,” in which one manages patients without imaging, only to discover later that they had CT-detectable pathology of some sort that could have been detected sooner, reinforces the desire for greater certainty. (A nonimaging example of this quest for perfect certainty is illustrated by the recent report of a malpractice carrier urging its clients to admit all patients with chest pain,10 a report prominently repeated on personal injury law blogs.) The asymmetry in feedback about the consequences of over versus under use leads to bias favoring intervention,11 and this is strongly reinforced by the almost universal tendency to evaluate the quality of care by the quality of the result, in profound ignorance of the pervasive effects of the hindsight and outcome biases.12 The tort system is too easy a target to even mention here. The sustaining mechanism may be a bit more difficult to visualize but likely can be found in network effects among physicians.13 Network effects among interacting agents can produce positive feedback that amplifies small initial impulses into avalanches. This phenomenon has no valence; its effects Annals of Emergency Medicine 465
The Hunting of the Snark can be good (eg, the disappearance of smoking in polite society), as well as bad (the housing and other financial “bubbles” and subsequent crashes).14 Network effects result from formal or informal comparisons among physicians; we learn by discovering that our colleagues approach a problem somewhat differently than we do. If physicians observe that many of their colleagues are using CT more liberally, it begins to seem at least okay and eventually may persuade them to join the crowd because, after all, the “standard of care” is that which most physicians do. This same mechanism has been observed in financial panics, in which people standing in line to withdraw their funds during a run on a bank will admit that they do not believe the bank is likely to fail but observe that “if everyone else does it, it becomes the right thing to do.”15 In such a context, given physicians’ generally favorable views of technology, a small number of only modestly risk-averse practitioners can trigger an unconscious consensus, resulting in a quixotic search for greater and greater certainty. In Lewis Carroll’s epic poem The Hunting of the Snark,16 an improbable crew guided by a blank map pursues a mythic but dangerous creature, to their own doom. One is struck by the feeling that, in our quest for ever more perfect certainty, we are reenacting that famous tragedy. With respect to certainty, Carroll might have described us thus: They sought it with thimbles, they sought it with care; They pursued it by strong radiography; They fastened it down in a waiting-room chair, And ordered computed tomography.i The standard response to the problem of overuse of technology is a call to establish centralized command-andcontrol mechanisms, eg, decision rules, practice guidelines, or computerized decision support systems, perhaps enforced by economic sanctions.17 Centralized, feed-forward control is thought to act in the common good and to support long-term sustainability that might require local actors to sacrifice their short-term gains. But effective feed-forward control has been difficult to achieve in many health care problems.18,19 It is difficult for such guidance to be precise enough or to support the use of local knowledge to arbitrate sensibly among competing priorities.20 And there are 2 more fundamental issues. First, feed-forward guidance, even if well designed, does little to relieve practitioners from the double bind of failing to follow the guidance (when we now know they should have) versus failing to deviate appropriately from the guidance (when we now know, etc). And second, in an open system, the problems posed by illness and injury are always changing, and the technologies that might be used are also changing; thus, the guidance will inevitably be outpaced by events.21 i
Apologies to Lewis Carroll
466 Annals of Emergency Medicine
Wears It seems inevitable that any good tool will be overused at some point; in fact, that is how we learn where the boundaries of diminishing returns are. What is acceptable and appropriate use or nonuse of CT imaging is not an entirely objective question that can be neatly resolved by empirical data and formal analysis, but rather a tangled, socially constructed issue involving competing views of risk, benefit and obligation, and the elusive question of how much certainty we must have. If we cannot develop a shared view on these issues, we will be condemned to be perpetually hunting the Snark. Supervising editor: Ellen J. Weber, MD Funding and support: By Annals policy, all authors are required to disclose any and all commercial, financial, and other relationships in any way related to the subject of this article as per ICMJE conflict of interest guidelines (see www.icmje.org). The author has stated that no such relationships exist. Publication date: Available online August 11, 2011. Address for correspondence: Robert L. Wears, MD, MS, E-mail [email protected]@imperial.ac.uk. REFERENCES 1. Kocher K, Meurer WJ, Fazel R, et al. National trends in the use of computed tomography in the emergency department. Ann Emerg Med. 2011;58:452-462. 2. Newman DH, Schriger DL. Rethinking testing for pulmonary embolism: less is more. Ann Emerg Med. 2011;57:622-627, e623. 3. Wears RL. Risk, radiation, and rationality. Ann Emerg Med. 2011; 58:9-11. 4. Baumann BM, Chen EH, Mills AM, et al. Patient perceptions of computed tomographic imaging and their understanding of radiation risk and exposure. Ann Emerg Med. 2011;58:1-7. 5. Hillman BJ, Goldsmith JC. The uncritical use of high-tech medical imaging. N Engl J Med. 2010;363:4-6. 6. Korley FK, Pham JC, Kirsch TD. Use of advanced radiology during visits to US emergency departments for injury-related conditions, 1998-2007. JAMA. 2010;304:1465-1471. 7. Hollander JE, Chang AM, Shofer FS, et al. Coronary computed tomographic angiography for rapid discharge of low-risk patients with potential acute coronary syndromes. Ann Emerg Med. 2009; 53:295-304. 8. Bhuiya F, Pitts SR, McCaig LF. Emergency Department Visits for Chest Pain and Abdomnal Pain: United States, 1999-2008. Hyattsville, MD: National Center for Health Statistics; 2010. Available at: http://www.cdc.gov/nchs/data/databriefs/db43. pdf. Accessed September 3, 2010. 9. Nawar EW, Niska RW, Xu J. National Hospital Ambulatory Medical Care Survey: 2005 Emergency Department Summary. Hyattsville, MD: National Center for Health Statistics; 2007. No. 386. 10. Landro L. Hospitals overhaul ERs to reduce mistakes. Wall Street Journal. New York, NY. Available at: http://online.wsj.com/article_ email/SB10001424052748703859304576307060330715004lMyQjAxMTAxMDEwMzExNDMyWj.html#printMode. Accessed May 10, 2011. 11. Ghaffarzadegan N. Beyond personality traits and financial incentives: bias and variation in medical practice as results of experiential learning. Paper presented at 29th International System Dynamics Conference. Washington, DC; 2011.
Volume , . : November
Wears 12. Henriksen K, Kaplan H. Hindsight bias, outcome knowledge and adaptive learning. Qual Saf Health Care. 2003;12(suppl 2):ii46-ii50. 13. Christakis N, Fowler JH. Connected. New York, NY: Little, Brown & Co; 2009. 14. Sterman JD. Business Dynamics: Systems Thinking and Modeling for a Complex World. Boston, MA: Irwin McGraw-Hill; 2000. 15. Livesey B, Menon J. Northern Rock stock tumbles further amid run on bank. Available at: http://www.bloomberg.com/apps/ news?pid⫽newsarchive&sid⫽aeypCkzcRlU4. Accessed July 5, 2011. 16. Carroll L. The Hunting of the Snark: An Agony in Eight Fits. London, UK: Macmillan & Co; 1898. 17. Ostrom E, Eggertsson T, Calvert R, eds. Governing the Commons: The Evolution of Institutions for Collective Action. New York, NY: Cambridge University Press; 1990.
Volume , . : November
The Hunting of the Snark 18. Asch DA, Hershey JC. Why some health policies don’t make sense at the bedside. Ann Intern Med. 1995;122:846-850. 19. Greenberg M, Ridgely MS. Clinical decision support and malpractice risk. JAMA. 2011;306:90-91. 20. Branlat M, Woods DD. How do systems manage their adaptive capacity to successfully handle disruptions? a resilience engineering perspective. Paper presented at: AAAI Fall Symposium 2010; Association for the Advancement of Artificial Intelligence; Arlington, VA; November 11-13, 2010; 2010:26-34. Available at: http://www.aaai.org/ocs/index.php/FSS/FSS10/ paper/view/2238. 21. Shekelle PG, Ortiz E, Rhodes S, et al. Validity of the Agency for Healthcare Research and Quality Clinical Practice Guidelines: how quickly do guidelines become outdated? JAMA. 2001;286: 1461-1467.
Annals of Emergency Medicine 467
The Hunting of the Snark, 2011 Robert L. Wears, MD, MS From the Department of Emergency Medicine, University of Florida, Jacksonville, FL; and Clinical Safety Research Unit, Imperial College London, UK. 0196-0644/$-see front matter Copyright © 2011 by the American College of Emergency Physicians. doi:10.1016/j.annemergmed.2011.07.031
A podcast for this article is available at www.annemergmed.com.
SEE RELATED ARTICLE, P. 452. [Ann Emerg Med. 2011;58:465-467.] In this issue of Annals, Kocher et al1 provide quantitative detail about a change in emergency department (ED) practice, of which we are all generally aware: the explosive growth in the use of computed tomography (CT). Their data should usefully inform the ongoing discussions about why and how we use imaging and about the risks and benefits of that use.2-5 They observe that the growth in imaging has been substantial across a broad range of complaints during a 12-year period of study and further state that much of this increase may have been beneficial, in that the likelihood of hospitalization (or transfer) after a CT was less than half what it was at the beginning of their study (1996). Although this might partly have come from scanning patients who were less ill to begin with,6 it seems clear that in a substantial proportion of cases, the CT results allowed what would have otherwise been a hospitalization to be converted to a discharge with outpatient follow-up,7 a generally beneficial result for both patients and the health care system. Although the article by Kocher et al1 notes that the significant growth in CT use per ED visit, it does not emphasize its most remarkable finding, namely, that this growth has been exponential, increasing at a steady rate of 14% per year, with no apparent slowing during the 12-year study period; adjustment for patient characteristics (eg, an increase in elderly patients) does not change this fundamental result. Most of us would be deliriously happy if we could obtain half this rate of growth in our incomes or retirement plans; if it were sustained, CT use would double roughly every 5 years so that we should expect almost 25% of ED visits this year (2011) to entail at least 1 CT. Clearly, this is not sustainable; we would soon have half the population used in scanning the other half. Fortunately, “no tree grows to the sky”; because chest, abdominal, and flank pain patients, combined with trauma patients, compose roughly 30% of ED visits, we should expect some leveling off of CT use at least around 30% of ED visits.8,9 But do we really want to practice in a world in which, in effect, every complaint from bregma to smegma and every motor vehicle crash results in a CT? This pursuit of accuracy may increase overload in already beleaguered EDs to debilitating levels. Volume , . : November
Exponential growexth occurs mostly in 2 situations: breakout and positive feedback. Breakout circumstances are characterized by novelty: an organism arrives in a new ecologic niche, with an abundance of resources and few predators; or a new technology offers capabilities never before realized in an unexploited market. Clearly, some of the growth in CT use can be described in a breakout narrative; in addition, as new capabilities were developed (eg, CT angiography), minibreakouts, no doubt propelled, continued increase. But CTs’ principal breakout occurred in the 1970s to 1980s; by the mid-1990s, we should have expected subsequent advances to have contributed smaller and smaller fractions to growth, not the larger and larger increases reported by Kocher et al.1 Positive feedback, then, is an alternative explanation for exponential growth. Positive feedback loops need an initiating and sustaining mechanism (think of the initial capital investment and the annual interest rate in compound interest problems). A plausible initiating mechanism is not too difficult to imagine. A “surprise” result, such as an aortic dissection found in a patient with vague chest pain and hard-to-pin-down arm and leg numbness, or a large pulmonary embolism in a patient who feels “weak and faintified,” confronts us with the uncertainties of traditional medical problem recognition while offering an escape to greater certainty: just scan more of those vague, borderline cases and voila!— uncertainty gone, to everyone’s great relief. The occasional “near miss,” in which one manages patients without imaging, only to discover later that they had CT-detectable pathology of some sort that could have been detected sooner, reinforces the desire for greater certainty. (A nonimaging example of this quest for perfect certainty is illustrated by the recent report of a malpractice carrier urging its clients to admit all patients with chest pain,10 a report prominently repeated on personal injury law blogs.) The asymmetry in feedback about the consequences of over versus under use leads to bias favoring intervention,11 and this is strongly reinforced by the almost universal tendency to evaluate the quality of care by the quality of the result, in profound ignorance of the pervasive effects of the hindsight and outcome biases.12 The tort system is too easy a target to even mention here. The sustaining mechanism may be a bit more difficult to visualize but likely can be found in network effects among physicians.13 Network effects among interacting agents can produce positive feedback that amplifies small initial impulses into avalanches. This phenomenon has no valence; its effects Annals of Emergency Medicine 465
The Hunting of the Snark can be good (eg, the disappearance of smoking in polite society), as well as bad (the housing and other financial “bubbles” and subsequent crashes).14 Network effects result from formal or informal comparisons among physicians; we learn by discovering that our colleagues approach a problem somewhat differently than we do. If physicians observe that many of their colleagues are using CT more liberally, it begins to seem at least okay and eventually may persuade them to join the crowd because, after all, the “standard of care” is that which most physicians do. This same mechanism has been observed in financial panics, in which people standing in line to withdraw their funds during a run on a bank will admit that they do not believe the bank is likely to fail but observe that “if everyone else does it, it becomes the right thing to do.”15 In such a context, given physicians’ generally favorable views of technology, a small number of only modestly risk-averse practitioners can trigger an unconscious consensus, resulting in a quixotic search for greater and greater certainty. In Lewis Carroll’s epic poem The Hunting of the Snark,16 an improbable crew guided by a blank map pursues a mythic but dangerous creature, to their own doom. One is struck by the feeling that, in our quest for ever more perfect certainty, we are reenacting that famous tragedy. With respect to certainty, Carroll might have described us thus: They sought it with thimbles, they sought it with care; They pursued it by strong radiography; They fastened it down in a waiting-room chair, And ordered computed tomography.i The standard response to the problem of overuse of technology is a call to establish centralized command-andcontrol mechanisms, eg, decision rules, practice guidelines, or computerized decision support systems, perhaps enforced by economic sanctions.17 Centralized, feed-forward control is thought to act in the common good and to support long-term sustainability that might require local actors to sacrifice their short-term gains. But effective feed-forward control has been difficult to achieve in many health care problems.18,19 It is difficult for such guidance to be precise enough or to support the use of local knowledge to arbitrate sensibly among competing priorities.20 And there are 2 more fundamental issues. First, feed-forward guidance, even if well designed, does little to relieve practitioners from the double bind of failing to follow the guidance (when we now know they should have) versus failing to deviate appropriately from the guidance (when we now know, etc). And second, in an open system, the problems posed by illness and injury are always changing, and the technologies that might be used are also changing; thus, the guidance will inevitably be outpaced by events.21 i
Apologies to Lewis Carroll
466 Annals of Emergency Medicine
Wears It seems inevitable that any good tool will be overused at some point; in fact, that is how we learn where the boundaries of diminishing returns are. What is acceptable and appropriate use or nonuse of CT imaging is not an entirely objective question that can be neatly resolved by empirical data and formal analysis, but rather a tangled, socially constructed issue involving competing views of risk, benefit and obligation, and the elusive question of how much certainty we must have. If we cannot develop a shared view on these issues, we will be condemned to be perpetually hunting the Snark. Supervising editor: Ellen J. Weber, MD Funding and support: By Annals policy, all authors are required to disclose any and all commercial, financial, and other relationships in any way related to the subject of this article as per ICMJE conflict of interest guidelines (see www.icmje.org). The author has stated that no such relationships exist. Publication date: Available online August 11, 2011. Address for correspondence: Robert L. Wears, MD, MS, E-mail [email protected]@imperial.ac.uk. REFERENCES 1. Kocher K, Meurer WJ, Fazel R, et al. National trends in the use of computed tomography in the emergency department. Ann Emerg Med. 2011;58:452-462. 2. Newman DH, Schriger DL. Rethinking testing for pulmonary embolism: less is more. Ann Emerg Med. 2011;57:622-627, e623. 3. Wears RL. Risk, radiation, and rationality. Ann Emerg Med. 2011; 58:9-11. 4. Baumann BM, Chen EH, Mills AM, et al. Patient perceptions of computed tomographic imaging and their understanding of radiation risk and exposure. Ann Emerg Med. 2011;58:1-7. 5. Hillman BJ, Goldsmith JC. The uncritical use of high-tech medical imaging. N Engl J Med. 2010;363:4-6. 6. Korley FK, Pham JC, Kirsch TD. Use of advanced radiology during visits to US emergency departments for injury-related conditions, 1998-2007. JAMA. 2010;304:1465-1471. 7. Hollander JE, Chang AM, Shofer FS, et al. Coronary computed tomographic angiography for rapid discharge of low-risk patients with potential acute coronary syndromes. Ann Emerg Med. 2009; 53:295-304. 8. Bhuiya F, Pitts SR, McCaig LF. Emergency Department Visits for Chest Pain and Abdomnal Pain: United States, 1999-2008. Hyattsville, MD: National Center for Health Statistics; 2010. Available at: http://www.cdc.gov/nchs/data/databriefs/db43. pdf. Accessed September 3, 2010. 9. Nawar EW, Niska RW, Xu J. National Hospital Ambulatory Medical Care Survey: 2005 Emergency Department Summary. Hyattsville, MD: National Center for Health Statistics; 2007. No. 386. 10. Landro L. Hospitals overhaul ERs to reduce mistakes. Wall Street Journal. New York, NY. Available at: http://online.wsj.com/article_ email/SB10001424052748703859304576307060330715004lMyQjAxMTAxMDEwMzExNDMyWj.html#printMode. Accessed May 10, 2011. 11. Ghaffarzadegan N. Beyond personality traits and financial incentives: bias and variation in medical practice as results of experiential learning. Paper presented at 29th International System Dynamics Conference. Washington, DC; 2011.
Volume , . : November
Wears 12. Henriksen K, Kaplan H. Hindsight bias, outcome knowledge and adaptive learning. Qual Saf Health Care. 2003;12(suppl 2):ii46-ii50. 13. Christakis N, Fowler JH. Connected. New York, NY: Little, Brown & Co; 2009. 14. Sterman JD. Business Dynamics: Systems Thinking and Modeling for a Complex World. Boston, MA: Irwin McGraw-Hill; 2000. 15. Livesey B, Menon J. Northern Rock stock tumbles further amid run on bank. Available at: http://www.bloomberg.com/apps/ news?pid⫽newsarchive&sid⫽aeypCkzcRlU4. Accessed July 5, 2011. 16. Carroll L. The Hunting of the Snark: An Agony in Eight Fits. London, UK: Macmillan & Co; 1898. 17. Ostrom E, Eggertsson T, Calvert R, eds. Governing the Commons: The Evolution of Institutions for Collective Action. New York, NY: Cambridge University Press; 1990.
Volume , . : November
The Hunting of the Snark 18. Asch DA, Hershey JC. Why some health policies don’t make sense at the bedside. Ann Intern Med. 1995;122:846-850. 19. Greenberg M, Ridgely MS. Clinical decision support and malpractice risk. JAMA. 2011;306:90-91. 20. Branlat M, Woods DD. How do systems manage their adaptive capacity to successfully handle disruptions? a resilience engineering perspective. Paper presented at: AAAI Fall Symposium 2010; Association for the Advancement of Artificial Intelligence; Arlington, VA; November 11-13, 2010; 2010:26-34. Available at: http://www.aaai.org/ocs/index.php/FSS/FSS10/ paper/view/2238. 21. Shekelle PG, Ortiz E, Rhodes S, et al. Validity of the Agency for Healthcare Research and Quality Clinical Practice Guidelines: how quickly do guidelines become outdated? JAMA. 2001;286: 1461-1467.
Annals of Emergency Medicine 467