- Email: [email protected]
The Tyranny of Evidence-Based Medicine
THE TYRANNY OF EVIDENCE-BASED MEDICINE James
w. Goodwin, MD, FRCSC
Departm.ent of Obstetrics and Gynaecology, Dalhousie University Gambia-Dalhousie Maternal & Family Health Project Board of Directors. Southwest Nova District Health Authority Yarmouth NS
Evidence-based medicine, the "New Paradigm,"1 uses its ultimate instruments, the randomized controlled trial (RCT) and the meta-analysis, to generate the "best available evidence" with which to inform the decisions of medical practice. It replaces opinion based on unsystematic clinical experience and so-called "non-evidentiary" knowledge with strictly defined evidence from clinical research. There is no question that these are wonderful advances in clinical research methodology when they are used appropriately, but it seems now that, rather than assisting thoughtful management decisions, these tools have imposed a "new orthodoxy"2 that demands unquestioning adherence to its doctrine. Using a form of alchemy, the evangelists of the movement have exalted the randomized controlled trial to the rarefied heights of the "gold standard." The indispensable adjective "rigorous" is applied ad nauseum to the evidence generated in systematic reviews of properly designed RCTs. Making clinical decisions with anything less than this convicts the physician of practising inferior medicine. 3 It seems that the exercise of clinical judgement has no place within the workings of an RCT and so, after a little lip service, it is dispensed with as irrelevant to the ultimate treatment decision being made. 3 Twenty-five years ago, a now famous group in Oxford began to assemble a very large register of controlled perinatal trials (only some of which were randomized) and created so-called systematic reviews from them. Two publications resulted, Effective Care in Pregnancy and Childbirth,4 a 1500-page text in two volumes last published in1989, and The Cochrane Pregnancy and Childbirth Database,5 last published in 1995. To be sure, the current electronic database of these trials and reviews is regularly updated at the Cochrane Library in Oxford, but it can only be purchased as an extensive series of CD-ROM disks. The third edition of A Guide to Effective Care in Pregnancy and Childbirth6 has taken on the ambitious task of presenting "evidence-based information on the effects of pregnancy and childbirth care" to a very broad readership including policy-makers, obstetricians (trained and training), nurses, midwives and even childbearing women. The Guide attempts unrealistically to make the evidence from RCTs and systematic reviews warehoused in the Cochrane Library fit every clinical situation in obstetrics, normal and abnormal, and to fashion
every clinical decision. In the event, however, there prove to be few RCTs with sufficiently powerful methodology, and this shortfall forces the Guide to give disturbingly superficial coverage to such substantial topics as the assessment and augmentation oflabour, Caesarean section, post-term pregnancy, placenta previa and diabetes in pregnancy, to name only a few. Trial of labour, one of the cornerstones of obstetric management, receives no coverage at all. One of the essential components of research publication is certainly the reference citation to similar work in the scientific literature. Thus, it is disturbing that almost a third (32.4%) of the 583 reference sources given in the Guide are found in the two dated publications Effective Care ofPregnancy and Childbirth 4 and The Cochrane Pregnancy and Childbirth Databas? ("pre-Cochrane Review/6). Nearly half (245) of these sources are systematic reviews in the Cochrane Library and the remainder come from the general medical literature. None of these sources has numerical citations directed to specific statements in the Guide. Clearly, two of the elements of evidence-based medicine, rules of evidence and the critical interpretation of original literature, 1,7 cannot possibly be used well if the publications being cited are out of date, difficult to access, or are too expensive, especially for the resident in training. On the other hand, the carefully crafted randomized, multicentre Term Breech TrialS seems to leave no room at all "for disagreement about assisting breech births at term."9 Using a standard statistical comparison, this randomized trial found that perinatal outcome (perinatal mortality and serious neonatal morbidity) was significantly better for the planned Caesarean section group than for the planned vaginal birth group. Thus, the chief conclusion of this trial is that planned vaginal birth of breech presentation at term is unsafe under any circumstances. The authors seem to have assumed that the presence of an experienced obstetrician (with skill defined in four ways7) at each planned vaginal breech delivery would guarantee that sufficient judgement had been exercised at every delivery. Clearly, the application of judgement cannot possibly be measured by an RCT and thus seems out of place in clinical practice dictated by the "best available evidence."10 In the end, RCTs seem far better suited to clinical drug trials. The employment of sound judgement reaches its sharpest
jOGe.
NOVEMBER 2001
focus in the management of operative procedures such as assisted breech delivery. Any delay in the descent of the breech longer than 30 minutes after the onset of pushing following full dilatation is a dependable predictor of fetopelvic disproportion requiring abdominal delivery. The too hasty use of oxytocin augmentation here would nullifY this valuable sign and might lure an impatient (albeit experienced) obstetrician into doing an injudicious breech extraction. The management of frank or complete breech presentation at term should always be conducted as a trial of second stage labour with instant Caesarean section readiness. Any analysis of planned vaginal breech delivery must surely include the conduct of this trial labour based on the prompt recognition of any delay in the descent of the breech and the timely exercise of judgement determining the decision to proceed with assisted vaginal delivery or to stop the trial and turn to Caesarean section. Sixteen perinatal deaths were reported in the Term Breech Trial and six of the babies died following a "difficult vaginal delivery. " Unfortunately, no details have been provided about the nature of this difficulty. There were 12 deaths (2.0%) following 591 vaginal deliveries in the planned vaginal breech group and four of these were "difficult." It is entirely possible that some lapse of judgement allowed the trial to proceed to the complicated delivery and its fatal outcome. Four of the remaining eight deaths in the planned vaginal breech group (two babies dying "pre-enrolment," two dying at home after being "discharged well") could just as readily be attributed to planned Caesarean birth. Only one baby died (0.22%) among the 451 Caesarean deliveries in the planned vaginal breech delivery group, and the attempted vaginal delivery of this baby is described as "difficult." In the remaining 450 cases, however, fetopelvic disproportion was recognized in time and the decision made for abdominal delivery. Three babies (0.29%) died among those delivered to the 1041 women randomized to the planned Caesarean section group and one of these deaths followed a difficult attempted vaginal delivery before the planned Caesarean could be carried out. Ic seems clear that the timely application of judgement can improve the perinatal outcome of the planned vaginal delivery of breech presentation atterm by reducing the perinatal mortality to 0.22 percent, less than that reported for planned Caesarean section (0.29%). Thus, the chief conclusion of the Term Breech Trial cannot be considered proven. Of course, all of the foregoing is beside the point because the Term Breech Trial has been able to make its conclusion based on the statistical analysis of evidence alone without having to turn to the part played by judgement. The training of residents in this skill (assisted breech delivery) is thus superfluous because no one will be able to apply it following the verdict of the Term Breech Trial, which effectively forbids the practice of planned vaginal breech delivery in the future.
Douglas Bell of the Canadian Medical Protective Association has confirmed the legal position regarding the liability of a physician practising outside of evidence-based medicinedeveloped standards: In order to successfully defend the action, it would be necessary for the defence to convince the judge that a respected minority of physicians do continue to do planned breech vaginal deliveries and it would be necessary to cite compelling support for [their continuing to do] vaginal breech deliveries, given the results of [the] randomized [Term Breech] trial. I I
There can be little doubt that this trial has irrevocably changed the practice of obstetrics, in the developed world at least. We are left with the dilemma of how to apply its conclusions to practice in the poorer countries with dangerously limited resources. REFERENCES I.
Evidence-based medicine working group. Evidence-based medicine: a new approach to teaching the practice of medicine. JAm Med Assoc 1992;268:2420-5. 2. Editorial. Evidence-based medicine in its place. Lancet 1995;346:785. 3. Feinstein AR, Horwitz RI. Problems in the "evidence" of evidence-based medicine. Am J Med 1997; 103:529-35. 4. Chalmers I, Enkin M, Keirse MJNC, eds. Effective Care in Pregnancy and Childbirth. Oxford: Oxford University Press; 1989. 5. Cochrane Pregnancy and Childbirth Database. Oxford:The Cochrane Library; 1995. 6. Enkin M, Keirse MJNC, Neilson J. A Guide to Effective Care in Pregnancy and Childbirth. 3rd Ed. Oxford: Oxford University Press; 2000. 7. Guyatt GH. Evidence-based medicine. Ann Intem Med 1991; 114 (ACP J Club suppl 2):A- 16. 8. Hannah ME, Hannah WJ, Hewson SA, Hodnett ED, Saigal S,Willan AR. Planned caesarean section versus planned vaginal delivery for breech presentation at term: a randomised multicentre trial. Term Breech Trial Collaborative Group. Lancet 2000;356: 1375-83. 9. Lumley J. Commentary: Any room left for disagreement about assisting breech births at term? Lancet 2000;356: 1368-9. 10. Tonelli MR.The philosophical limits of evidence-based medicine. Acad Med 1998;73: 1234-40. II . Bell ED. Liability opinion (CMPA File 200 I-003527 EDB: NS).
NOVEMBER 200 I
w. Goodwin, MD, FRCSC
Departm.ent of Obstetrics and Gynaecology, Dalhousie University Gambia-Dalhousie Maternal & Family Health Project Board of Directors. Southwest Nova District Health Authority Yarmouth NS
Evidence-based medicine, the "New Paradigm,"1 uses its ultimate instruments, the randomized controlled trial (RCT) and the meta-analysis, to generate the "best available evidence" with which to inform the decisions of medical practice. It replaces opinion based on unsystematic clinical experience and so-called "non-evidentiary" knowledge with strictly defined evidence from clinical research. There is no question that these are wonderful advances in clinical research methodology when they are used appropriately, but it seems now that, rather than assisting thoughtful management decisions, these tools have imposed a "new orthodoxy"2 that demands unquestioning adherence to its doctrine. Using a form of alchemy, the evangelists of the movement have exalted the randomized controlled trial to the rarefied heights of the "gold standard." The indispensable adjective "rigorous" is applied ad nauseum to the evidence generated in systematic reviews of properly designed RCTs. Making clinical decisions with anything less than this convicts the physician of practising inferior medicine. 3 It seems that the exercise of clinical judgement has no place within the workings of an RCT and so, after a little lip service, it is dispensed with as irrelevant to the ultimate treatment decision being made. 3 Twenty-five years ago, a now famous group in Oxford began to assemble a very large register of controlled perinatal trials (only some of which were randomized) and created so-called systematic reviews from them. Two publications resulted, Effective Care in Pregnancy and Childbirth,4 a 1500-page text in two volumes last published in1989, and The Cochrane Pregnancy and Childbirth Database,5 last published in 1995. To be sure, the current electronic database of these trials and reviews is regularly updated at the Cochrane Library in Oxford, but it can only be purchased as an extensive series of CD-ROM disks. The third edition of A Guide to Effective Care in Pregnancy and Childbirth6 has taken on the ambitious task of presenting "evidence-based information on the effects of pregnancy and childbirth care" to a very broad readership including policy-makers, obstetricians (trained and training), nurses, midwives and even childbearing women. The Guide attempts unrealistically to make the evidence from RCTs and systematic reviews warehoused in the Cochrane Library fit every clinical situation in obstetrics, normal and abnormal, and to fashion
every clinical decision. In the event, however, there prove to be few RCTs with sufficiently powerful methodology, and this shortfall forces the Guide to give disturbingly superficial coverage to such substantial topics as the assessment and augmentation oflabour, Caesarean section, post-term pregnancy, placenta previa and diabetes in pregnancy, to name only a few. Trial of labour, one of the cornerstones of obstetric management, receives no coverage at all. One of the essential components of research publication is certainly the reference citation to similar work in the scientific literature. Thus, it is disturbing that almost a third (32.4%) of the 583 reference sources given in the Guide are found in the two dated publications Effective Care ofPregnancy and Childbirth 4 and The Cochrane Pregnancy and Childbirth Databas? ("pre-Cochrane Review/6). Nearly half (245) of these sources are systematic reviews in the Cochrane Library and the remainder come from the general medical literature. None of these sources has numerical citations directed to specific statements in the Guide. Clearly, two of the elements of evidence-based medicine, rules of evidence and the critical interpretation of original literature, 1,7 cannot possibly be used well if the publications being cited are out of date, difficult to access, or are too expensive, especially for the resident in training. On the other hand, the carefully crafted randomized, multicentre Term Breech TrialS seems to leave no room at all "for disagreement about assisting breech births at term."9 Using a standard statistical comparison, this randomized trial found that perinatal outcome (perinatal mortality and serious neonatal morbidity) was significantly better for the planned Caesarean section group than for the planned vaginal birth group. Thus, the chief conclusion of this trial is that planned vaginal birth of breech presentation at term is unsafe under any circumstances. The authors seem to have assumed that the presence of an experienced obstetrician (with skill defined in four ways7) at each planned vaginal breech delivery would guarantee that sufficient judgement had been exercised at every delivery. Clearly, the application of judgement cannot possibly be measured by an RCT and thus seems out of place in clinical practice dictated by the "best available evidence."10 In the end, RCTs seem far better suited to clinical drug trials. The employment of sound judgement reaches its sharpest
jOGe.
NOVEMBER 2001
focus in the management of operative procedures such as assisted breech delivery. Any delay in the descent of the breech longer than 30 minutes after the onset of pushing following full dilatation is a dependable predictor of fetopelvic disproportion requiring abdominal delivery. The too hasty use of oxytocin augmentation here would nullifY this valuable sign and might lure an impatient (albeit experienced) obstetrician into doing an injudicious breech extraction. The management of frank or complete breech presentation at term should always be conducted as a trial of second stage labour with instant Caesarean section readiness. Any analysis of planned vaginal breech delivery must surely include the conduct of this trial labour based on the prompt recognition of any delay in the descent of the breech and the timely exercise of judgement determining the decision to proceed with assisted vaginal delivery or to stop the trial and turn to Caesarean section. Sixteen perinatal deaths were reported in the Term Breech Trial and six of the babies died following a "difficult vaginal delivery. " Unfortunately, no details have been provided about the nature of this difficulty. There were 12 deaths (2.0%) following 591 vaginal deliveries in the planned vaginal breech group and four of these were "difficult." It is entirely possible that some lapse of judgement allowed the trial to proceed to the complicated delivery and its fatal outcome. Four of the remaining eight deaths in the planned vaginal breech group (two babies dying "pre-enrolment," two dying at home after being "discharged well") could just as readily be attributed to planned Caesarean birth. Only one baby died (0.22%) among the 451 Caesarean deliveries in the planned vaginal breech delivery group, and the attempted vaginal delivery of this baby is described as "difficult." In the remaining 450 cases, however, fetopelvic disproportion was recognized in time and the decision made for abdominal delivery. Three babies (0.29%) died among those delivered to the 1041 women randomized to the planned Caesarean section group and one of these deaths followed a difficult attempted vaginal delivery before the planned Caesarean could be carried out. Ic seems clear that the timely application of judgement can improve the perinatal outcome of the planned vaginal delivery of breech presentation atterm by reducing the perinatal mortality to 0.22 percent, less than that reported for planned Caesarean section (0.29%). Thus, the chief conclusion of the Term Breech Trial cannot be considered proven. Of course, all of the foregoing is beside the point because the Term Breech Trial has been able to make its conclusion based on the statistical analysis of evidence alone without having to turn to the part played by judgement. The training of residents in this skill (assisted breech delivery) is thus superfluous because no one will be able to apply it following the verdict of the Term Breech Trial, which effectively forbids the practice of planned vaginal breech delivery in the future.
Douglas Bell of the Canadian Medical Protective Association has confirmed the legal position regarding the liability of a physician practising outside of evidence-based medicinedeveloped standards: In order to successfully defend the action, it would be necessary for the defence to convince the judge that a respected minority of physicians do continue to do planned breech vaginal deliveries and it would be necessary to cite compelling support for [their continuing to do] vaginal breech deliveries, given the results of [the] randomized [Term Breech] trial. I I
There can be little doubt that this trial has irrevocably changed the practice of obstetrics, in the developed world at least. We are left with the dilemma of how to apply its conclusions to practice in the poorer countries with dangerously limited resources. REFERENCES I.
Evidence-based medicine working group. Evidence-based medicine: a new approach to teaching the practice of medicine. JAm Med Assoc 1992;268:2420-5. 2. Editorial. Evidence-based medicine in its place. Lancet 1995;346:785. 3. Feinstein AR, Horwitz RI. Problems in the "evidence" of evidence-based medicine. Am J Med 1997; 103:529-35. 4. Chalmers I, Enkin M, Keirse MJNC, eds. Effective Care in Pregnancy and Childbirth. Oxford: Oxford University Press; 1989. 5. Cochrane Pregnancy and Childbirth Database. Oxford:The Cochrane Library; 1995. 6. Enkin M, Keirse MJNC, Neilson J. A Guide to Effective Care in Pregnancy and Childbirth. 3rd Ed. Oxford: Oxford University Press; 2000. 7. Guyatt GH. Evidence-based medicine. Ann Intem Med 1991; 114 (ACP J Club suppl 2):A- 16. 8. Hannah ME, Hannah WJ, Hewson SA, Hodnett ED, Saigal S,Willan AR. Planned caesarean section versus planned vaginal delivery for breech presentation at term: a randomised multicentre trial. Term Breech Trial Collaborative Group. Lancet 2000;356: 1375-83. 9. Lumley J. Commentary: Any room left for disagreement about assisting breech births at term? Lancet 2000;356: 1368-9. 10. Tonelli MR.The philosophical limits of evidence-based medicine. Acad Med 1998;73: 1234-40. II . Bell ED. Liability opinion (CMPA File 200 I-003527 EDB: NS).
NOVEMBER 200 I