- Email: [email protected]
The allure of quantification
Magnetic Resonance Imaging 31 (2013) 1035–1036
Contents lists available at SciVerse ScienceDirect
Magnetic Resonance Imaging journal homepage: www.mrijournal.com
Letter to the Editor
The allure of quantification Everything that can be counted does not necessarily count; everything that counts cannot necessarily be counted. Albert Einstein Dr. Abramson and colleagues elegantly discuss why radiologists may be disinclined towards quantifying and stating quantifications in their reports [1]. Perhaps it might be worth also reflecting upon the allure of quantification. To appreciate the attraction of quantification it is important to understand and distinguish between certain terms: risk and uncertainty and prediction and prophesy. Henceforth, the term certainty will be used in an absolute sense and not in a scalar manner; meaning certainty is synonymous with one hundred percent certainty. The semantic antonym of certainty is uncertainty. However, this is a broad term and can be sub-divided as was done by Frank Knight, the economist, into Knightian risk and Knightian uncertainty [2]. Consider the scenario of tossing a coin. In a coin with heads on both sides one is certain that the toss will yield a head. In a coin with one side depicting a head and the other side depicting a tail it is uncertain if tossing will yield a head. However, assuming the coin is evenly weighted and the toss is fair, it can be said that there is an equal chance of attaining heads or tails – i.e. the chance of attaining head is 50%. This can be verified by repeatedly tossing the coin, thus creating a sample or universe of outcomes. When the uncertainty can be meaningfully quantified with certainty this is known as risk. Consider a fishing enthusiast who is fishing for trout for the first time in a freshwater lake. What are the chances of catching a trout over the next hour? He may know that it is not certain but not know how uncertain. There are no samples or universe of outcomes to accord certainty to the uncertainty. This is an example of unquantifiable uncertainty or, simply, uncertainty. Both predictions and prophesies deal with the likelihood of future event. Prophesies, or at least those of the more specific nature such as made by Jeremiah, are certainties about the future conditional on continuing along the same moral trajectory [3]. Predictions or prognostications also offer a glimpse in to the future. However they do so not by absolute certainty of the eventuality, unless they are general statements such as the famous one by Lord Maynard Keynes, “in the long run we are all dead” [4]; but by certainty in the degree of uncertainty. In other words they represent risk or quantifiable uncertainty. Behavioral psychologists have shown that we have a strong preference for risk over uncertainty when we are unsure of the outcome [5]. For the medical professional, the preference of risk over uncertainty is augmented by an inherent distaste of uncertainty. Uncertainty is antithetical to the physician's expertise, knowledge 0730-725X/$ – see front matter © 2013 Elsevier Inc. All rights reserved. http://dx.doi.org/10.1016/j.mri.2012.11.003
base, aura and sense of self-worth. It is much more preferable to tell a patient that their symptoms have a 24% chance of being due to a pulmonary embolus than to say that they have a “moderate likelihood” of pulmonary embolism, or even to say that they have a “10–90%” likelihood of pulmonary embolism. Diagnostic imaging reduces uncertainty. Quantification attempts to quantify uncertainty converting it from the wide latitude of Knightian uncertainty to the précis of Knightian risk. However, it begs to be asked if the precision is justified or bears semblance to reality. In the process of quantification many variables are taken, each with its own uncertainty. Assumptions are made of the distribution of the variables. Disparate studies provide disparate numbers and complex statistics are used to unite the archipelago of quantitative information. Far from a simplified universe of outcomes one can map with a fair coin toss; this is a patchwork whose contraction to a singularity of precision is as much an article of faith as it is a matter of science. Mathematician and philosopher Nassim Nicholas Taleb coined the term “Ludic Fallacy” [6]. This is the fallacy when one falsely believes they know the distribution of a variable or indeed the variable itself, used to derive risk. The trouble, as Taleb explains in his popular tome Black Swan, is not just that we are computing when we would be better off admitting to our ignorance, it is that such computations lead to complacency and the false sense that we are in control, which is periodically shattered by improbable but highly consequential events. The trouble with quantifications in clinical medicine in general and imaging in particular is not simply the lack of exactitude or residual uncertainty; it is that some of the exponents of quantification treat the numbers as if they are the temperature scale in a thermometer. This is dangerous. I never tire of explaining to referring physicians who have, to quote one example, requested cardiac CT for the patient with chest pain that there is nothing magical about 70% stenosis (cut off for consideration for catheterization). Treat the patient not the “percent stenosis”. Many radiologists are rightly circumspect of quantification. Some view its dogmatic adherence and unmitigated zeal a nuisance. Quantification does not absolve radiologists of using their clinical judgment; which is just as well. The irreverent GK Chesterton once remarked: “do not free a camel of the burden of his hump; you may be freeing him from being a camel” [7]. To slightly paraphrase: you can remove the burden of clinical judgment from a radiologist but then you will no longer have a radiologist. References [1] Abramson RG, et al. Quantitative metrics in clinical radiology reporting: a snapshot perspective from a single mixed academic-community practice. Magn Reson Imaging 2012;30(9):1357-66.
1036
Letter to the Editor / Magnetic Resonance Imaging 31 (2013) 1035–1036
[2] Knight FH. The meaning of risk and uncertainty. In Risk, Uncertainty and Profit. Mineola, NY: Dover Publication; 2006. [3] Jeremiah 29:11. [4] Keynes JM. A tract on monetary reform; 1924. [5] Ellsberg D. Risk, ambiguity and the Savage axioms. Q J Econ 1961;75:643-69. [6] Taleb NN. The Ludic Fallacy or the Uncertainty of the Nerd. In The Black Swan. 2nd ed. New York, NY: Random House; 2010. p. 127–9.
[7] http://www.brainyquote.com/quotes/authors/g/gilbert_k_chesterton_2.html (accessed on 08/28/12).
Saurabh Jha University of Pennsylvania, PA, USA E-mail address: [email protected]
Contents lists available at SciVerse ScienceDirect
Magnetic Resonance Imaging journal homepage: www.mrijournal.com
Letter to the Editor
The allure of quantification Everything that can be counted does not necessarily count; everything that counts cannot necessarily be counted. Albert Einstein Dr. Abramson and colleagues elegantly discuss why radiologists may be disinclined towards quantifying and stating quantifications in their reports [1]. Perhaps it might be worth also reflecting upon the allure of quantification. To appreciate the attraction of quantification it is important to understand and distinguish between certain terms: risk and uncertainty and prediction and prophesy. Henceforth, the term certainty will be used in an absolute sense and not in a scalar manner; meaning certainty is synonymous with one hundred percent certainty. The semantic antonym of certainty is uncertainty. However, this is a broad term and can be sub-divided as was done by Frank Knight, the economist, into Knightian risk and Knightian uncertainty [2]. Consider the scenario of tossing a coin. In a coin with heads on both sides one is certain that the toss will yield a head. In a coin with one side depicting a head and the other side depicting a tail it is uncertain if tossing will yield a head. However, assuming the coin is evenly weighted and the toss is fair, it can be said that there is an equal chance of attaining heads or tails – i.e. the chance of attaining head is 50%. This can be verified by repeatedly tossing the coin, thus creating a sample or universe of outcomes. When the uncertainty can be meaningfully quantified with certainty this is known as risk. Consider a fishing enthusiast who is fishing for trout for the first time in a freshwater lake. What are the chances of catching a trout over the next hour? He may know that it is not certain but not know how uncertain. There are no samples or universe of outcomes to accord certainty to the uncertainty. This is an example of unquantifiable uncertainty or, simply, uncertainty. Both predictions and prophesies deal with the likelihood of future event. Prophesies, or at least those of the more specific nature such as made by Jeremiah, are certainties about the future conditional on continuing along the same moral trajectory [3]. Predictions or prognostications also offer a glimpse in to the future. However they do so not by absolute certainty of the eventuality, unless they are general statements such as the famous one by Lord Maynard Keynes, “in the long run we are all dead” [4]; but by certainty in the degree of uncertainty. In other words they represent risk or quantifiable uncertainty. Behavioral psychologists have shown that we have a strong preference for risk over uncertainty when we are unsure of the outcome [5]. For the medical professional, the preference of risk over uncertainty is augmented by an inherent distaste of uncertainty. Uncertainty is antithetical to the physician's expertise, knowledge 0730-725X/$ – see front matter © 2013 Elsevier Inc. All rights reserved. http://dx.doi.org/10.1016/j.mri.2012.11.003
base, aura and sense of self-worth. It is much more preferable to tell a patient that their symptoms have a 24% chance of being due to a pulmonary embolus than to say that they have a “moderate likelihood” of pulmonary embolism, or even to say that they have a “10–90%” likelihood of pulmonary embolism. Diagnostic imaging reduces uncertainty. Quantification attempts to quantify uncertainty converting it from the wide latitude of Knightian uncertainty to the précis of Knightian risk. However, it begs to be asked if the precision is justified or bears semblance to reality. In the process of quantification many variables are taken, each with its own uncertainty. Assumptions are made of the distribution of the variables. Disparate studies provide disparate numbers and complex statistics are used to unite the archipelago of quantitative information. Far from a simplified universe of outcomes one can map with a fair coin toss; this is a patchwork whose contraction to a singularity of precision is as much an article of faith as it is a matter of science. Mathematician and philosopher Nassim Nicholas Taleb coined the term “Ludic Fallacy” [6]. This is the fallacy when one falsely believes they know the distribution of a variable or indeed the variable itself, used to derive risk. The trouble, as Taleb explains in his popular tome Black Swan, is not just that we are computing when we would be better off admitting to our ignorance, it is that such computations lead to complacency and the false sense that we are in control, which is periodically shattered by improbable but highly consequential events. The trouble with quantifications in clinical medicine in general and imaging in particular is not simply the lack of exactitude or residual uncertainty; it is that some of the exponents of quantification treat the numbers as if they are the temperature scale in a thermometer. This is dangerous. I never tire of explaining to referring physicians who have, to quote one example, requested cardiac CT for the patient with chest pain that there is nothing magical about 70% stenosis (cut off for consideration for catheterization). Treat the patient not the “percent stenosis”. Many radiologists are rightly circumspect of quantification. Some view its dogmatic adherence and unmitigated zeal a nuisance. Quantification does not absolve radiologists of using their clinical judgment; which is just as well. The irreverent GK Chesterton once remarked: “do not free a camel of the burden of his hump; you may be freeing him from being a camel” [7]. To slightly paraphrase: you can remove the burden of clinical judgment from a radiologist but then you will no longer have a radiologist. References [1] Abramson RG, et al. Quantitative metrics in clinical radiology reporting: a snapshot perspective from a single mixed academic-community practice. Magn Reson Imaging 2012;30(9):1357-66.
1036
Letter to the Editor / Magnetic Resonance Imaging 31 (2013) 1035–1036
[2] Knight FH. The meaning of risk and uncertainty. In Risk, Uncertainty and Profit. Mineola, NY: Dover Publication; 2006. [3] Jeremiah 29:11. [4] Keynes JM. A tract on monetary reform; 1924. [5] Ellsberg D. Risk, ambiguity and the Savage axioms. Q J Econ 1961;75:643-69. [6] Taleb NN. The Ludic Fallacy or the Uncertainty of the Nerd. In The Black Swan. 2nd ed. New York, NY: Random House; 2010. p. 127–9.
[7] http://www.brainyquote.com/quotes/authors/g/gilbert_k_chesterton_2.html (accessed on 08/28/12).
Saurabh Jha University of Pennsylvania, PA, USA E-mail address: [email protected]