- Email: [email protected]
Myth: Neonatology is evidence-based
Seminars in Fetal & Neonatal Medicine 16 (2011) 288e292
Contents lists available at ScienceDirect
Seminars in Fetal & Neonatal Medicine journal homepage: www.elsevier.com/locate/siny
Myth: Neonatology is evidence-based Robert E. Schumacher* Division of Neonatal/Perinatal Medicine, University of Michigan Health Systems, 200 East Hospital Drive, Ann Arbor, MI 48109-0254, USA
s u m m a r y Keywords: Decision-making Evidence-based medicine Neonatology Patient participation
The practice of evidence-based medicine involves the judicious use of current best evidence in the care of individual patients. Decisions about diagnosis, prognosis and treating patients require knowledge of the probability and value of outcomes. Decision analysis illustrates how probabilities and values help define one another, and each are important. Whereas initial probability estimates can be obtained by ‘searching for the best evidence’, values belong to individuals. Obtaining values from patients or parents is sometimes difficult and requires a respectful, thoughtful, systematic approach, but only after doing this is neonatal care evidence-based. Ó 2011 Published by Elsevier Ltd.
1. Introduction In the process of practising evidence-based medicine (EBM) physicians are always concerned with the search for the highest ‘quality’ evidence available, and the search for that evidence is regarded as a cornerstone of the process. Many feel that when the search for evidence is done and the meta-analysis has been found, the hardest part of the job is finished. However, if in seeking a definition of EBM one actually converses with a true EBM practitioner or visits an EBM website, one finds that when the evidence has been collected the job has just begun. For example, many are surprised when they are presented with some ‘pyramids of evidence’ and find the ‘N of 1’ trial (a randomized controlled trial with one subject) and not ‘meta-analysis with homogeneity of results’ perched at the top of the pyramid. In reading any definition of EBM, note must be made of the repeated reference to the treatment of individuals, not populations of individuals. ‘Evidence-based medicine is the conscientious, explicit, and judicious use of current best evidence in making decisions about the care of individual patients. The practice of evidence-based medicine means integrating individual clinical expertise with the best available external clinical evidence from systematic research . and in the more thoughtful identification and compassionate use of individual patients’ predicaments, rights, and preferences in making clinical decisions about their care.’1 Patients and families come to us with stories and anecdotes. We have all been cautioned on the problems encountered when we use ‘anecdote-based medicine.’ When providing care, physicians must
* Tel.: þ1 734 763 4109. E-mail address: [email protected]. 1744-165X/$ e see front matter Ó 2011 Published by Elsevier Ltd. doi:10.1016/j.siny.2011.04.005
make decisions about diagnoses, prognoses, and therapy. All deal with the need for estimates of probability. For example, inception cohort studies provide good estimates of prognoses. Cognitive psychologists tell us that by attaching our own personal experiences (complete with personal meaning and value) to a clinical situation, anecdotes will interfere with estimates of probability.2 Certain diagnoses might come to mind because we have seen them repetitively. Consider a 27-week newborn with grunting, flaring, retracting, and a chest radiograph that shows a reticular granular pattern. The diagnosis of respiratory distress syndrome comes immediately to mind. However, if one is asked to give a differential diagnosis, second on the list is likely to be group B streptococcal pneumonia. This may be because the one or two cases actually experienced were emotionally charged with personal value and hence easy to remember (especially if the diagnosis was missed the first time). Despite the problems encountered with anecdotes, patients bring them and they must be used. The definition of EBM demands that the patient story, narrative or ‘anecdote’ (complete with its values) be incorporated into the practice of EBM. This is part of the fundamental essence of EBM, and as such the incorporation of the patient’s story represents a major challenge to neonatologists who wish to practice EBM. Two questions need to be addressed: (i) why are stories/narratives/anecdotes/values important?; (ii) how can we use them in our practice? Narratives and anecdotes are values that are important because properly defined values (or utilities in decision analysis) dictate the necessary accuracy of probability estimates needed for making diagnoses, prognoses, and treatment decisions. (How certain does our diagnosis have to be before we decide to act?) Decision analysis (and conceptually simple algebra) tells us we require two things to
R.E. Schumacher / Seminars in Fetal & Neonatal Medicine 16 (2011) 288e292
make decisions about patient care: probability (P) and utility (value). Figure 1 illustrates in a simple way a decision that needs to be made. Should we go this way or that way? Confronted with this schema, people usually say they cannot answer until they know ‘What will happen if I go this way?’ and ‘How likely is something to happen if I go that way?’ There are consequences/outcomes/values associated with going either way e some good, some bad. Each outcome has a certain probability of occurring. Figure 2 shows the ‘this way/that way’ decision with probabilities and utilities (clearly personally subjective) added. One can now calculate the net value of going this way or that way. Two things could happen if one goes ‘this way’, and each is associated with a certain probability of occurrence (which sum to a probability of 1.0). The net value of ‘this way’ is the sum of the value of the each possibility multiplied by its probability. In this case there are only two outcomes, so that the net value (or utility) of going ‘this way’ is: (P really really good) (value really really good) þ (P pretty bad) (value pretty bad). The net value of ‘that way’ is: (P good but not great) (value good but not great ) plus (P just plain bad) (value just plain bad). The decision can be now be made; one takes the path with the higher value. (Assigning at least relative value to ‘really really good’ and ‘just plain bad’ etc. is necessary). It is therefore a kind of algebra, in which unknowns are probabilities and utilities. Accordingly, assigning utilities can allow us to solve (or estimate) the necessary probabilities required to act (and the quality of evidence needed to estimate same). Similarly, assigning probability defines utility/values and, moreover, acting without assigning value defines value to outcomes. Poor probability estimates (poor evidence) can lead to unrealistic utility/value. An interesting example of this phenomenon was demonstrated by Suresh and Clark.3 They cited reports from the Joint Commission and a statement by the American Academy of Pediatrics (AAP), both concerned with an apparent increase in the incidence of kernicterus. Subsequently, the AAP issued guidelines to be followed for the identification and follow-up of infants with the goal of preventing sequelae of hyperbilirubinemia. Suresh and Clark point out, however, that at the time the true incidence or probability of kernicterus was not well defined. Using decision analysis (‘this way’/ ‘that way’) the authors showed that if the probability of kernicterus was high enough, such a program could be cost-effective, but that at other reasonable probabilities of kernicterus the cost (utility/value) of preventing a single case could be as high as US$55,000,000. Whether or not a program to prevent kernicterus is cost-effective cannot be known until a more precise estimate of the probability of kernicterus is obtained, and a more precise estimate or ‘better’ evidence is needed. An alternative and perhaps more valid approach would be to assign an acceptable cost to prevent
Figure 1. ‘Which way should I go?’.
289
Figure 2. ‘Which way should I go?’ with probabilites and values/utilities added to frame the problem.
kernicterus and then design a program to target a population at the appropriate risk level. This is an example of how decisions regarding the treatment of populations of patients are dependent upon probabilities and utilities. What about decisions regarding individual patients? To facilitate an understanding of decision-making in our own unit we performed an exercise involving the resuscitation of extremely low birth weight (ELBW) infants. A group of nurses in our own neonatal intensive care unit was asked to assign utility (value) to three outcome states of infants born at ELBW: alive without a ‘major handicap’, alive with a major handicap, and dead. Using a linear scale, the nurses were asked to identify the best possible outcome and the worst possible outcome. These were assigned values of 1 and 10. They were then asked to assign a value between one and 10 to their choice of intermediate outcome value. All chose alive with a major handicap as the worst possible outcome and assigned a mean value of 7.5 to death. A similar exercise was performed using house officers and they assigned a mean value of 5 to death. We then assigned probability values to each outcome, first using data obtained from a cohort of babies in the ‘pre-surfactant’ era (Figure 3), and then estimated outcome probabilities for a separate cohort from follow-up studies done 10 years later.4 Using decision analysis, we determined that if nurses values were used for the outcome probabilities for the early cohort, resuscitation would not be performed (Figure 4a). Using outcome probabilities generated for the latter cohort, resuscitation would be performed. We also determined that, for these nursing-assigned values when the probability of survival without major handicap was >33%, resuscitation should be formed. Hence, the nursing values assigned to
Figure 3. Decision to resuscitate: outcomes from the control arm of a large randomized controlled trial studying the efficacy of artificial surfactant therapy.
290
R.E. Schumacher / Seminars in Fetal & Neonatal Medicine 16 (2011) 288e292
a
Survives no disability 1.00; P = 0.260 0.260
Survives with disabilty
Resuscitate 0.57
0.00; P = 0.110 0.110
Dies post resuscitation
Do I resuscitate? Resuscitate : 0.57
0.50; P = 0.630 0.630
Dies
Do Not Resuscitate
0.50
0.50 1.000
b
Survives no disability 1.00 0.260
Resuscitate
Survives with disabilty 0.73
0.00 0.110
Do I resuscitate?
Dies post resuscitation 0.75
Do Not Resuscitate : 0.75 0.630
Do Not Resuscitate
Dies 0.75
0.75; P = 1.000 1.000
Figure 4. (a) Decision analysis using utilities obtained from pediatric house officers and ‘presurfactant’ era outcome probablities. The resucitate path is favored. (b) Decision analysis using utilities obtained from neonatal intensive care unit nurses and ‘presurfactant’ era outcome probablities. The ‘Do not resucitate’ path is favored.
outcomes determined the outcome probability necessary for resuscitation. The decision to resuscitate is said to be sensitive to the probability of handicap, so the use of accurate probability estimates is imperative. Using the house officers’ outcome values, resuscitation would be favored in both cohorts of babies (Figure 4b). This is an example of how assigning different values (registered nurses vs house officers) to the same set of outcome probabilities can result in different decisions. The decision to resuscitate is sensitive to the values-assigned outcomes. Values are important. EBM ‘experts’ point out that values are important because individuals present with subjective outcome descriptions (or their own set of values) to outcomes. We see now that helping patients (or their parents) assign values is essential to practising EBM. We also see why conceptually the ‘N of 1’ trial is helpful as it transforms probability estimates of outcomes onto dichotomous patientcentric outcomes. How to assign these values is a challenge. Let us continue using the illustrative resuscitation of the ELBW example. In a series of papers, Saigal et al. demonstrated the need to obtain outcome values from patients and their families and from individual patients themselves rather than representative groups of patients. Can doctors assign improper utility? Saigal et al. examined the possible role of healthcare providers (versus parents) as surrogate assigners of health utilities for ELBW patients.5 In this study a questionnaire was given to 169 parents of surviving ELBW children, who were part of a longitudinal study of the outcome of ELBW infants, and 123 parents of term children. A similar questionnaire was completed by 98 Canadian neonatologists and 99 neonatal nurses. A significant majority of parents believed that attempts should be made to save all infants, irrespective of condition or weight at birth, compared to only 6% of health professionals. (Health professionals also believed that economic costs to society should be a factor in deciding whether to save an ELBW infant.) The investigators conclude, ‘Health care professionals must recognize
that their attitudes toward saving ELBW infants differ from those of parents. Parents, whether of term or extremely premature children, are more in favor of intervening to save the infant irrespective of its weight or condition at birth than are professionals. It therefore is imperative that there be joint decision making, combining the knowledge of the physician with the wishes of the parents.’ In another paper, a cohort of teenage former ELBW infants had their health outcome states objectively scored (health-related quality of life), and were then asked to rate themselves as to their current health state. The results of the objective scale confirmed that overall, the health utility scores of this cohort were lower than those obtained from a cohort of normal birth weight subjects. However, these teenagers subjectively ranked their own personal states of health as being better and no different from their normal birth weight contemporaries.6 We see that practicing EBM in this cohort suggests that one should use values from patients and not objective population-based health scales. In yet another study, investigators attempted to see whether values elicited from populations of teenage ELBW survivors might serve as reasonable surrogate measures for individuals from the same population. Scores were obtained from a health utilities index (HUI) based on self-assessed health status but valued using community preferences. Specifically asked in this study was: ‘Do mean directly measured utility scores of quality of life for each individual agree with group mean health utilities index (HUI) scores? The authors concluded that at the individual level, HUI are poor substitutes for directly measured scores.7 These three studies suggest that at present, there is no good substitute/surrogate/proxy for obtaining values from individuals. Values assigned to quality of life belong to individuals. How can we elicit values from patients (or parents)? To do this, at least three basic items of understanding must be in the possession of the parent: (i) a requirement that the patient has a correct understanding of all possible outcomes; (ii) a correct understanding of
R.E. Schumacher / Seminars in Fetal & Neonatal Medicine 16 (2011) 288e292
what all possible outcomes mean; and (iii) an understanding of the probability for each outcome. Examples using the decision to resuscitate an extremely premature infant are again illustrative. When first presented with the possibility of having an extremely premature baby, many parents’ first question is ‘Is my baby going to die?’ This suggests that at that point in time, parents making decisions surrounding resuscitation recognize only two outcomes, alive or dead. The nature and types of handicapping conditions that may affect such a baby are not recognized or are recognized as vague possibilities. By way of example, Hansen et al. surveyed the attitudes of parents to life-saving treatment of ELBW infants.8 The recommendations given by parents were significantly influenced by the hypothetical child’s risk of having a serious handicap, suggesting that, when counseling parents, the recognition of handicapping condition (and the probability of same) are essential parts of counseling such expectant parents. Once the existence of a handicapped state is established as a possibility, an understanding of what having a handicap means is a necessary step in decision-making. When counseling parents about the nature of a handicap, it may be important to remind them (and oneself) that the decision at hand is to resuscitate or not, and that outcomes must be valued only relative to the alternative of death. That is to say, although outcomes such as behavioral problems and learning difficulties may be relatively common in ELBW infants, their value relative to death may preclude using them as outcome measures useful in decision-making. Parents bring personal and preconceived notions about the meaning of handicap, which should be explored. Asking parents such open-ended questions as ‘What does the term cerebral palsy mean to you?’ or ‘What do you think of when you hear the words ‘mental retardation’?’ can give a counselor insight into parental understanding of these phrases. By credentialing the parents’ story, a counselor allows for open dialogue, education, and the possibility of a revision of parents’ understanding and parental values on their own terms. The assigning of correct probabilities to outcomes, and more importantly ensuring parental understanding of probabilities, is important yet can be difficult. The care provider should be very familiar with relevant probabilities. Probabilities of outcomes for populations of ELBW infants are available and can serve to anchor a discussion.9 The evidence-based counselor is, however, obliged to understand the nature of the population from which such statistics are derived and to what degree they are representative of the specific clinical situation at hand. Specific knowledge about local outcomes and the pregnancy itself is always needed to properly refine these estimates. Delivering probability estimates in an objective manner that parents can understand and use is very challenging. The introduction of cognitive bias is often hard to avoid. Hayward et al. demonstrated that the way in which information was presented or ‘framed’ had the ability to compromise risk comprehension by the decisional party. In this study, subjects were given a vignette with prognostic information framed either with survival and no-disability statistics (positive frame) or with mortality and disability statistics (negative frame). Those who received information that was in the positive frame were more likely to elect resuscitation.10 There are many other ways to introduce bias into decisionmaking. The use of decision aids (adjuncts to counseling that help clarify the process of decision-making) seems an area ready for exploration in questions of how to counsel parents in the issue of resuscitation. The International Patient Decision Aids Standards (IPDAS) Collaboration has reached agreement on criteria for judging the quality of patient decision aids.11 A simple review of these standards would benefit anyone who does counseling. For example, the collaboration provides a checklist to assist the
291
counselor in presenting probabilities of outcomes in an unbiased and understandable way.12 The literature regarding eliciting values for decision-making needed in the resuscitation of an ELBW infant is exhaustive in nature. It has just been touched on in hopes of illustrating the myriad of problems associated with such decisions and the need for the evidence-based neonatologist to continue to become educated in this area. Decisions in areas other than resuscitation are made daily e some need little discussion and little parental involvement; while some are harder to make, the approach is always similar. To many, to practice neonatology in an evidence-based fashion is to search the world of medical literature, its ‘inception cohorts’, ‘randomized controlled trials’, and ‘meta analyses’ for universal truths which can be applied directly to patient care. What these pieces of evidence actually provide is a starting point from which a practitioner can use his or her best judgment regarding the objective specifics of the individual case and the individual patient and family values involved to construct the best possible plan of care. In her book Doctor’s stories, K.M. Hunter reminds us that patients and families come to us only with narratives or stories.13 She notes that three things must happen if this narrative encounter is to contribute to the healing of the patient. ‘The first, prior to their encounter, is the physician’s understanding that a patient’s account of a malady is and will remain distinct from the medical version that patient and physician are meeting to construct. The second and third are integral parts of the patientephysician interaction: in the beginning the case must be constructed in a careful interview and physical exam; at the close, the story in its medical version must be restored to the patient.’ This is the essence of EBM, and neonatology will only be evidence-based when the individual families and their stories are so respected.
Practice points The definition of EBM demands that the family’s story, complete with its values, be incorporated into the practice of EBM. Decisions are made using probability estimates of outcomes and the utilities or ‘values’ of those outcomes. Deciding to act defines these values; a-priori assignation of values tells one when to act. Whenever possible utilities should be obtained from the individuals involved.
Research directions The role of decision aids in neonatology. What is proper surrogate decision-making?.
Conflict of interest statement None declared. Funding sources None. References 1. Sackett DL, Rosenberg WM, Gray JA, et al. Evidence based medicine: what it is and what it isn’t. BMJ 1996;312(7023):71e2. 2. Elstein AS, Schwartz A. Clinical problem solving and diagnostic decision making: selective review of the cognitive literature. BMJ 2002;324(7339):729e32.
292
R.E. Schumacher / Seminars in Fetal & Neonatal Medicine 16 (2011) 288e292
3. Suresh G, Clark R. Cost-effectiveness of strategies that are intended to prevent kernicterus in newborn infants. Pediatrics 2004;114:917e24. 4. Walther FJ, Mullett M, Schumacher R, Sundell H, Easa D, Long W. One-year follow-up of 66 premature infants weighing 500 to 699 grams treated with a single dose of synthetic surfactant or air placebo at birth: results of a doubleblind trial. American Exosurf Neonatal Study Group I. J Pediatr 1995;126:S13e9. 5. Streiner DL, Saigal S, Burrows E, Stoskopf B, Rosenbaum P. Attitudes of parents and health care professionals toward active treatment of extremely premature infants. Pediatrics 2001;108:152e7. 6. Saigal S, Feeny D, Rosenbaum P, Furlong W, Burrows E, Stoskopf B. Selfperceived health status and health-related quality of life of extremely lowbirth-weight infants at adolescence. JAMA 1996;276:453e9. 7. Feeny D, Furlong W, Saigal S, Sun J. Comparing directly measured standard gamble scores to HUI2 and HUI3 utility scores: group- and individual-level comparisons. Soc Sci Med 2004;58:799e809.
8. Hansen BM, Hoff B, Greisen G. Treatment of extremely preterm infants: parents’ attitudes. Acta Pædiatr 2003;92:715e20. 9. Tyson J, Parikh NA, Langer J, Green C, Higgins RD. National Institute of Child Health and Human Development Neonatal Research Network. Intensive care for extreme prematurity e moving beyond gestational age. N Engl J Med 2008;358:1672e81. 10. Haward M, Murphy R, Lorenz J. Message framing and perinatal decisions. Pediatrics 2008;122:109e18. 11. Elwyn G, O’Connor A, Stacey D. International Patient Decision Aids Standards (IPDAS) Collaboration. Developing a quality criteria framework for patient decision aids: online international Delphi consensus process. BMJ 2006;333(7565). 417. 12. International Patient Decision Aids Standards (IPDAS) Collaboration. [Available from: http://decisionaid.ohri.ca/methods.html. 13. Hunter KM. Doctors’ stories: the narrative structure of medical knowledge. Princeton, NJ: Princeton University Press; 1993.
Contents lists available at ScienceDirect
Seminars in Fetal & Neonatal Medicine journal homepage: www.elsevier.com/locate/siny
Myth: Neonatology is evidence-based Robert E. Schumacher* Division of Neonatal/Perinatal Medicine, University of Michigan Health Systems, 200 East Hospital Drive, Ann Arbor, MI 48109-0254, USA
s u m m a r y Keywords: Decision-making Evidence-based medicine Neonatology Patient participation
The practice of evidence-based medicine involves the judicious use of current best evidence in the care of individual patients. Decisions about diagnosis, prognosis and treating patients require knowledge of the probability and value of outcomes. Decision analysis illustrates how probabilities and values help define one another, and each are important. Whereas initial probability estimates can be obtained by ‘searching for the best evidence’, values belong to individuals. Obtaining values from patients or parents is sometimes difficult and requires a respectful, thoughtful, systematic approach, but only after doing this is neonatal care evidence-based. Ó 2011 Published by Elsevier Ltd.
1. Introduction In the process of practising evidence-based medicine (EBM) physicians are always concerned with the search for the highest ‘quality’ evidence available, and the search for that evidence is regarded as a cornerstone of the process. Many feel that when the search for evidence is done and the meta-analysis has been found, the hardest part of the job is finished. However, if in seeking a definition of EBM one actually converses with a true EBM practitioner or visits an EBM website, one finds that when the evidence has been collected the job has just begun. For example, many are surprised when they are presented with some ‘pyramids of evidence’ and find the ‘N of 1’ trial (a randomized controlled trial with one subject) and not ‘meta-analysis with homogeneity of results’ perched at the top of the pyramid. In reading any definition of EBM, note must be made of the repeated reference to the treatment of individuals, not populations of individuals. ‘Evidence-based medicine is the conscientious, explicit, and judicious use of current best evidence in making decisions about the care of individual patients. The practice of evidence-based medicine means integrating individual clinical expertise with the best available external clinical evidence from systematic research . and in the more thoughtful identification and compassionate use of individual patients’ predicaments, rights, and preferences in making clinical decisions about their care.’1 Patients and families come to us with stories and anecdotes. We have all been cautioned on the problems encountered when we use ‘anecdote-based medicine.’ When providing care, physicians must
* Tel.: þ1 734 763 4109. E-mail address: [email protected]. 1744-165X/$ e see front matter Ó 2011 Published by Elsevier Ltd. doi:10.1016/j.siny.2011.04.005
make decisions about diagnoses, prognoses, and therapy. All deal with the need for estimates of probability. For example, inception cohort studies provide good estimates of prognoses. Cognitive psychologists tell us that by attaching our own personal experiences (complete with personal meaning and value) to a clinical situation, anecdotes will interfere with estimates of probability.2 Certain diagnoses might come to mind because we have seen them repetitively. Consider a 27-week newborn with grunting, flaring, retracting, and a chest radiograph that shows a reticular granular pattern. The diagnosis of respiratory distress syndrome comes immediately to mind. However, if one is asked to give a differential diagnosis, second on the list is likely to be group B streptococcal pneumonia. This may be because the one or two cases actually experienced were emotionally charged with personal value and hence easy to remember (especially if the diagnosis was missed the first time). Despite the problems encountered with anecdotes, patients bring them and they must be used. The definition of EBM demands that the patient story, narrative or ‘anecdote’ (complete with its values) be incorporated into the practice of EBM. This is part of the fundamental essence of EBM, and as such the incorporation of the patient’s story represents a major challenge to neonatologists who wish to practice EBM. Two questions need to be addressed: (i) why are stories/narratives/anecdotes/values important?; (ii) how can we use them in our practice? Narratives and anecdotes are values that are important because properly defined values (or utilities in decision analysis) dictate the necessary accuracy of probability estimates needed for making diagnoses, prognoses, and treatment decisions. (How certain does our diagnosis have to be before we decide to act?) Decision analysis (and conceptually simple algebra) tells us we require two things to
R.E. Schumacher / Seminars in Fetal & Neonatal Medicine 16 (2011) 288e292
make decisions about patient care: probability (P) and utility (value). Figure 1 illustrates in a simple way a decision that needs to be made. Should we go this way or that way? Confronted with this schema, people usually say they cannot answer until they know ‘What will happen if I go this way?’ and ‘How likely is something to happen if I go that way?’ There are consequences/outcomes/values associated with going either way e some good, some bad. Each outcome has a certain probability of occurring. Figure 2 shows the ‘this way/that way’ decision with probabilities and utilities (clearly personally subjective) added. One can now calculate the net value of going this way or that way. Two things could happen if one goes ‘this way’, and each is associated with a certain probability of occurrence (which sum to a probability of 1.0). The net value of ‘this way’ is the sum of the value of the each possibility multiplied by its probability. In this case there are only two outcomes, so that the net value (or utility) of going ‘this way’ is: (P really really good) (value really really good) þ (P pretty bad) (value pretty bad). The net value of ‘that way’ is: (P good but not great) (value good but not great ) plus (P just plain bad) (value just plain bad). The decision can be now be made; one takes the path with the higher value. (Assigning at least relative value to ‘really really good’ and ‘just plain bad’ etc. is necessary). It is therefore a kind of algebra, in which unknowns are probabilities and utilities. Accordingly, assigning utilities can allow us to solve (or estimate) the necessary probabilities required to act (and the quality of evidence needed to estimate same). Similarly, assigning probability defines utility/values and, moreover, acting without assigning value defines value to outcomes. Poor probability estimates (poor evidence) can lead to unrealistic utility/value. An interesting example of this phenomenon was demonstrated by Suresh and Clark.3 They cited reports from the Joint Commission and a statement by the American Academy of Pediatrics (AAP), both concerned with an apparent increase in the incidence of kernicterus. Subsequently, the AAP issued guidelines to be followed for the identification and follow-up of infants with the goal of preventing sequelae of hyperbilirubinemia. Suresh and Clark point out, however, that at the time the true incidence or probability of kernicterus was not well defined. Using decision analysis (‘this way’/ ‘that way’) the authors showed that if the probability of kernicterus was high enough, such a program could be cost-effective, but that at other reasonable probabilities of kernicterus the cost (utility/value) of preventing a single case could be as high as US$55,000,000. Whether or not a program to prevent kernicterus is cost-effective cannot be known until a more precise estimate of the probability of kernicterus is obtained, and a more precise estimate or ‘better’ evidence is needed. An alternative and perhaps more valid approach would be to assign an acceptable cost to prevent
Figure 1. ‘Which way should I go?’.
289
Figure 2. ‘Which way should I go?’ with probabilites and values/utilities added to frame the problem.
kernicterus and then design a program to target a population at the appropriate risk level. This is an example of how decisions regarding the treatment of populations of patients are dependent upon probabilities and utilities. What about decisions regarding individual patients? To facilitate an understanding of decision-making in our own unit we performed an exercise involving the resuscitation of extremely low birth weight (ELBW) infants. A group of nurses in our own neonatal intensive care unit was asked to assign utility (value) to three outcome states of infants born at ELBW: alive without a ‘major handicap’, alive with a major handicap, and dead. Using a linear scale, the nurses were asked to identify the best possible outcome and the worst possible outcome. These were assigned values of 1 and 10. They were then asked to assign a value between one and 10 to their choice of intermediate outcome value. All chose alive with a major handicap as the worst possible outcome and assigned a mean value of 7.5 to death. A similar exercise was performed using house officers and they assigned a mean value of 5 to death. We then assigned probability values to each outcome, first using data obtained from a cohort of babies in the ‘pre-surfactant’ era (Figure 3), and then estimated outcome probabilities for a separate cohort from follow-up studies done 10 years later.4 Using decision analysis, we determined that if nurses values were used for the outcome probabilities for the early cohort, resuscitation would not be performed (Figure 4a). Using outcome probabilities generated for the latter cohort, resuscitation would be performed. We also determined that, for these nursing-assigned values when the probability of survival without major handicap was >33%, resuscitation should be formed. Hence, the nursing values assigned to
Figure 3. Decision to resuscitate: outcomes from the control arm of a large randomized controlled trial studying the efficacy of artificial surfactant therapy.
290
R.E. Schumacher / Seminars in Fetal & Neonatal Medicine 16 (2011) 288e292
a
Survives no disability 1.00; P = 0.260 0.260
Survives with disabilty
Resuscitate 0.57
0.00; P = 0.110 0.110
Dies post resuscitation
Do I resuscitate? Resuscitate : 0.57
0.50; P = 0.630 0.630
Dies
Do Not Resuscitate
0.50
0.50 1.000
b
Survives no disability 1.00 0.260
Resuscitate
Survives with disabilty 0.73
0.00 0.110
Do I resuscitate?
Dies post resuscitation 0.75
Do Not Resuscitate : 0.75 0.630
Do Not Resuscitate
Dies 0.75
0.75; P = 1.000 1.000
Figure 4. (a) Decision analysis using utilities obtained from pediatric house officers and ‘presurfactant’ era outcome probablities. The resucitate path is favored. (b) Decision analysis using utilities obtained from neonatal intensive care unit nurses and ‘presurfactant’ era outcome probablities. The ‘Do not resucitate’ path is favored.
outcomes determined the outcome probability necessary for resuscitation. The decision to resuscitate is said to be sensitive to the probability of handicap, so the use of accurate probability estimates is imperative. Using the house officers’ outcome values, resuscitation would be favored in both cohorts of babies (Figure 4b). This is an example of how assigning different values (registered nurses vs house officers) to the same set of outcome probabilities can result in different decisions. The decision to resuscitate is sensitive to the values-assigned outcomes. Values are important. EBM ‘experts’ point out that values are important because individuals present with subjective outcome descriptions (or their own set of values) to outcomes. We see now that helping patients (or their parents) assign values is essential to practising EBM. We also see why conceptually the ‘N of 1’ trial is helpful as it transforms probability estimates of outcomes onto dichotomous patientcentric outcomes. How to assign these values is a challenge. Let us continue using the illustrative resuscitation of the ELBW example. In a series of papers, Saigal et al. demonstrated the need to obtain outcome values from patients and their families and from individual patients themselves rather than representative groups of patients. Can doctors assign improper utility? Saigal et al. examined the possible role of healthcare providers (versus parents) as surrogate assigners of health utilities for ELBW patients.5 In this study a questionnaire was given to 169 parents of surviving ELBW children, who were part of a longitudinal study of the outcome of ELBW infants, and 123 parents of term children. A similar questionnaire was completed by 98 Canadian neonatologists and 99 neonatal nurses. A significant majority of parents believed that attempts should be made to save all infants, irrespective of condition or weight at birth, compared to only 6% of health professionals. (Health professionals also believed that economic costs to society should be a factor in deciding whether to save an ELBW infant.) The investigators conclude, ‘Health care professionals must recognize
that their attitudes toward saving ELBW infants differ from those of parents. Parents, whether of term or extremely premature children, are more in favor of intervening to save the infant irrespective of its weight or condition at birth than are professionals. It therefore is imperative that there be joint decision making, combining the knowledge of the physician with the wishes of the parents.’ In another paper, a cohort of teenage former ELBW infants had their health outcome states objectively scored (health-related quality of life), and were then asked to rate themselves as to their current health state. The results of the objective scale confirmed that overall, the health utility scores of this cohort were lower than those obtained from a cohort of normal birth weight subjects. However, these teenagers subjectively ranked their own personal states of health as being better and no different from their normal birth weight contemporaries.6 We see that practicing EBM in this cohort suggests that one should use values from patients and not objective population-based health scales. In yet another study, investigators attempted to see whether values elicited from populations of teenage ELBW survivors might serve as reasonable surrogate measures for individuals from the same population. Scores were obtained from a health utilities index (HUI) based on self-assessed health status but valued using community preferences. Specifically asked in this study was: ‘Do mean directly measured utility scores of quality of life for each individual agree with group mean health utilities index (HUI) scores? The authors concluded that at the individual level, HUI are poor substitutes for directly measured scores.7 These three studies suggest that at present, there is no good substitute/surrogate/proxy for obtaining values from individuals. Values assigned to quality of life belong to individuals. How can we elicit values from patients (or parents)? To do this, at least three basic items of understanding must be in the possession of the parent: (i) a requirement that the patient has a correct understanding of all possible outcomes; (ii) a correct understanding of
R.E. Schumacher / Seminars in Fetal & Neonatal Medicine 16 (2011) 288e292
what all possible outcomes mean; and (iii) an understanding of the probability for each outcome. Examples using the decision to resuscitate an extremely premature infant are again illustrative. When first presented with the possibility of having an extremely premature baby, many parents’ first question is ‘Is my baby going to die?’ This suggests that at that point in time, parents making decisions surrounding resuscitation recognize only two outcomes, alive or dead. The nature and types of handicapping conditions that may affect such a baby are not recognized or are recognized as vague possibilities. By way of example, Hansen et al. surveyed the attitudes of parents to life-saving treatment of ELBW infants.8 The recommendations given by parents were significantly influenced by the hypothetical child’s risk of having a serious handicap, suggesting that, when counseling parents, the recognition of handicapping condition (and the probability of same) are essential parts of counseling such expectant parents. Once the existence of a handicapped state is established as a possibility, an understanding of what having a handicap means is a necessary step in decision-making. When counseling parents about the nature of a handicap, it may be important to remind them (and oneself) that the decision at hand is to resuscitate or not, and that outcomes must be valued only relative to the alternative of death. That is to say, although outcomes such as behavioral problems and learning difficulties may be relatively common in ELBW infants, their value relative to death may preclude using them as outcome measures useful in decision-making. Parents bring personal and preconceived notions about the meaning of handicap, which should be explored. Asking parents such open-ended questions as ‘What does the term cerebral palsy mean to you?’ or ‘What do you think of when you hear the words ‘mental retardation’?’ can give a counselor insight into parental understanding of these phrases. By credentialing the parents’ story, a counselor allows for open dialogue, education, and the possibility of a revision of parents’ understanding and parental values on their own terms. The assigning of correct probabilities to outcomes, and more importantly ensuring parental understanding of probabilities, is important yet can be difficult. The care provider should be very familiar with relevant probabilities. Probabilities of outcomes for populations of ELBW infants are available and can serve to anchor a discussion.9 The evidence-based counselor is, however, obliged to understand the nature of the population from which such statistics are derived and to what degree they are representative of the specific clinical situation at hand. Specific knowledge about local outcomes and the pregnancy itself is always needed to properly refine these estimates. Delivering probability estimates in an objective manner that parents can understand and use is very challenging. The introduction of cognitive bias is often hard to avoid. Hayward et al. demonstrated that the way in which information was presented or ‘framed’ had the ability to compromise risk comprehension by the decisional party. In this study, subjects were given a vignette with prognostic information framed either with survival and no-disability statistics (positive frame) or with mortality and disability statistics (negative frame). Those who received information that was in the positive frame were more likely to elect resuscitation.10 There are many other ways to introduce bias into decisionmaking. The use of decision aids (adjuncts to counseling that help clarify the process of decision-making) seems an area ready for exploration in questions of how to counsel parents in the issue of resuscitation. The International Patient Decision Aids Standards (IPDAS) Collaboration has reached agreement on criteria for judging the quality of patient decision aids.11 A simple review of these standards would benefit anyone who does counseling. For example, the collaboration provides a checklist to assist the
291
counselor in presenting probabilities of outcomes in an unbiased and understandable way.12 The literature regarding eliciting values for decision-making needed in the resuscitation of an ELBW infant is exhaustive in nature. It has just been touched on in hopes of illustrating the myriad of problems associated with such decisions and the need for the evidence-based neonatologist to continue to become educated in this area. Decisions in areas other than resuscitation are made daily e some need little discussion and little parental involvement; while some are harder to make, the approach is always similar. To many, to practice neonatology in an evidence-based fashion is to search the world of medical literature, its ‘inception cohorts’, ‘randomized controlled trials’, and ‘meta analyses’ for universal truths which can be applied directly to patient care. What these pieces of evidence actually provide is a starting point from which a practitioner can use his or her best judgment regarding the objective specifics of the individual case and the individual patient and family values involved to construct the best possible plan of care. In her book Doctor’s stories, K.M. Hunter reminds us that patients and families come to us only with narratives or stories.13 She notes that three things must happen if this narrative encounter is to contribute to the healing of the patient. ‘The first, prior to their encounter, is the physician’s understanding that a patient’s account of a malady is and will remain distinct from the medical version that patient and physician are meeting to construct. The second and third are integral parts of the patientephysician interaction: in the beginning the case must be constructed in a careful interview and physical exam; at the close, the story in its medical version must be restored to the patient.’ This is the essence of EBM, and neonatology will only be evidence-based when the individual families and their stories are so respected.
Practice points The definition of EBM demands that the family’s story, complete with its values, be incorporated into the practice of EBM. Decisions are made using probability estimates of outcomes and the utilities or ‘values’ of those outcomes. Deciding to act defines these values; a-priori assignation of values tells one when to act. Whenever possible utilities should be obtained from the individuals involved.
Research directions The role of decision aids in neonatology. What is proper surrogate decision-making?.
Conflict of interest statement None declared. Funding sources None. References 1. Sackett DL, Rosenberg WM, Gray JA, et al. Evidence based medicine: what it is and what it isn’t. BMJ 1996;312(7023):71e2. 2. Elstein AS, Schwartz A. Clinical problem solving and diagnostic decision making: selective review of the cognitive literature. BMJ 2002;324(7339):729e32.
292
R.E. Schumacher / Seminars in Fetal & Neonatal Medicine 16 (2011) 288e292
3. Suresh G, Clark R. Cost-effectiveness of strategies that are intended to prevent kernicterus in newborn infants. Pediatrics 2004;114:917e24. 4. Walther FJ, Mullett M, Schumacher R, Sundell H, Easa D, Long W. One-year follow-up of 66 premature infants weighing 500 to 699 grams treated with a single dose of synthetic surfactant or air placebo at birth: results of a doubleblind trial. American Exosurf Neonatal Study Group I. J Pediatr 1995;126:S13e9. 5. Streiner DL, Saigal S, Burrows E, Stoskopf B, Rosenbaum P. Attitudes of parents and health care professionals toward active treatment of extremely premature infants. Pediatrics 2001;108:152e7. 6. Saigal S, Feeny D, Rosenbaum P, Furlong W, Burrows E, Stoskopf B. Selfperceived health status and health-related quality of life of extremely lowbirth-weight infants at adolescence. JAMA 1996;276:453e9. 7. Feeny D, Furlong W, Saigal S, Sun J. Comparing directly measured standard gamble scores to HUI2 and HUI3 utility scores: group- and individual-level comparisons. Soc Sci Med 2004;58:799e809.
8. Hansen BM, Hoff B, Greisen G. Treatment of extremely preterm infants: parents’ attitudes. Acta Pædiatr 2003;92:715e20. 9. Tyson J, Parikh NA, Langer J, Green C, Higgins RD. National Institute of Child Health and Human Development Neonatal Research Network. Intensive care for extreme prematurity e moving beyond gestational age. N Engl J Med 2008;358:1672e81. 10. Haward M, Murphy R, Lorenz J. Message framing and perinatal decisions. Pediatrics 2008;122:109e18. 11. Elwyn G, O’Connor A, Stacey D. International Patient Decision Aids Standards (IPDAS) Collaboration. Developing a quality criteria framework for patient decision aids: online international Delphi consensus process. BMJ 2006;333(7565). 417. 12. International Patient Decision Aids Standards (IPDAS) Collaboration. [Available from: http://decisionaid.ohri.ca/methods.html. 13. Hunter KM. Doctors’ stories: the narrative structure of medical knowledge. Princeton, NJ: Princeton University Press; 1993.