- Email: [email protected]
The limits of evidence-based medicine
Commentary
The Limits of Evidence-Based Medicine lain Kinloch Crombie
vidence-based medicine looks set to be the last major innovation to clinical practice in the twentieth century. It has been defined as "the conscientious, explicit and judicious use of current best evidence in making decisions about the care of individual patients" [18]. It seeks to replace the current medical paradigm, in which clinical experience and an understanding of the pathophysiology of disease are key elements in determining management. Instead, it proposes to root medicine in the findings of randomized controlled trials [9]. The randomized controlled trial provides the best quality of evidence about the effectiveness of a treatment. Systematic reviews provide unbiased authoritative summaries of randomized controlled trials. Thus, these should be used to inform choice of treatment, placing the practice of medicine on a sound scientific base. Evidence-based medicine is set to usher in an era of the highest-quality clinical practice. The purposes of evidence-based medicine are so clearly desirable that it might appear churlish to challenge them. Yet, this is what Linton does in his Focus article. His main concern is that the adoption of the principles of evidence-based medicine will take the "reason" out of medicine. He argues that reliance on externally produced evidence (systematic reviews) will deny the value of clinical experience and diminish its role in medicine. Although several other themes are identified in the article, this Commentary focuses on and develops the ideas that underpin the limits to the practice of evidence-based medicine.
E
From the Department of Epidemiology and Public Health, University of Dundee, Ninewells Hospital and Medical School, Dundee, United Kingdom. Reprint requests: lain Kinloch Crombie, Department of Epidemiology and Public Health, Ninewells Hospital and Medical School, Dundee DD1 9SY, United Kingdom. ©
1998 the American Pain Society
1058-9139/0701-0015$5.00/0
Pain Forum 7(1): 63-65, 1998
THE LIMITS OF EVIDENCE-BASED MEDICINE Central to evidence-based medicine is the view that systematic reviews should supplant clinical experience. Certainly, the production of systematic reviews under the auspices of the Cochrane Collaboration represents a major advance for medicine. But there are several reasons why, particularly in the field of chronic pain, it may not always be possible or desirable to practice evidence-based medicine. These can be summarized as three questions for evidence-based medicine: Is there enough evidence? Can the evidence be relied on? Is the process of evidence-based medicine appropriate?
Is There Enough Evidence? The first problem for implementing evidence-based medicine is that the evidence base is not sufficiently well established for effective treatments to be identified for each of the variety of pain conditions. Some advances have been made, for example, the systematic reviews of antidepressants [16] and anticonvulsants [15]. But for many of the treatments currently in use, this level of evidence is unavailable. For some conditions, the necessary studies have not been done. This is particularly true of those pain conditions that are difficult to diagnose and are therefore unlikely to have been studied in clinical trials. But in other circumstances the available evidence is not sufficient for' definitive conclusions. The studies may be of poor quality: Linton cites the example of the poor quality of randomized controlled trials on treatment efficacy in low back pain [13]. Likewise, Flor and colleagues found that studies of the efficacy of multidisciplinary pain clinics were generally of poor quality, either lacking control groups or using inappropriate control groups [10]. The consequence of the deficiencies in the available evidence is that there may often be little alternative than to rely, as medicine has long done, on clinical acumen and clinical experience. Denigrating these skills before the necessary evidence is acquired could be detrimental to patient care. It will be many years before much of the
63
64
COMMENTARy/Crombie
management of chronic pain has the weight of research findings that evidence-based medicine needs. Until then, we need good clinical skills.
Can the Evidence Be Relied On? A further concern is whether the findings from systematic reviews can be applied to patients seen at a local clinic. Linton raises the issue for the practitioner of "whether a treatment or diagnostic procedure is truly producing results for his or her patient. "The two areas for concern are the nature of the patient group and the nature of the treatment. To apply research findings in local practice, the types of patients treated in clinical trials need to be representative of patients seen routinely in clinic. The problem is that they are probably not. In general, patients are excluded from clinical trials if they are old or have multiple pathologies. But these are the types of patients commonly seen in clinic. This difficulty is compounded by the fact that patient referral processes vary substantially between clinics [3,6,12]. As clinical trials are often conducted in specialist tertiary referral centers, referral bias is likely to be especially marked. Patients who take part in therapeutic trials may be more highly motivated [20] and may have more severe pain. They may show a much greater improvement in pain scores than would be the case for pain patients not included in trials [7]. The second issue is whether the way the treatment is given in the clinical trial can be administered in a similar way in routine practice. Again, the answer is probably not. As Linton points out, "the way the treatment is applied, the communication between caregiver and patient, the setting, the level of anxiety, and so forth, may all influence the results of techniques designed to relieve pain. "This is likely to be particularly so for psychological therapies, but may also be true for other treatment modalities. The magnitude of the placebo effect and the extent to which it varies between studies illustrates the scope that there is for situational factors to influence treatment outcome [2,21]. These concerns about the applicability of systematic reviews are intensified by the observation that the findings from meta-analyses may be unreliable. Linton points out that the conclusions from one meta-analysis of back schools for back pain, that they are effective [13], are contradictory to those made by two other reviews [5,14]. In this instance, the explanation may lie with the criteria for selection of studies for inclusion in the analysis. The potential problem for evidence-based medicine is that it may shift the focus from the current concerns with the local assessment treatment (clinical experience) to an overreliance on the findings from clinical trials or systematic reviews. Evans has voiced similar concerns
for geriatricians: "There is a fear that in the absence of evidence clearly applicable to the case in hand a clinician might be forced by guidelines to make use of evidence which is only doubtfully relevant, generated perhaps in a different grouping of patients in another country at some other time using a similar but not identical treatment" [8]. Rather than being a boon to medical practice, systematic reviews may become a clinical straitjacket.
Is the Process of Evidence-Based Medicine Appropriate? Evidence-based medicine is described as providing a new paradigm for medical practice [9]. But there is a question of whether the paradigm will always be suitable for pain. The new paradigm states that in a consultation with a patient, the doctor should clarify the clinical problem that needs to be addressed. Published evidence on this should be sought and critically appraised, to identify the appropriate actions to be taken to deal with the clinical problem. A simple example of the new paradigm would be the management of a patient complaining of pain. First, the diagnosis would be clarified so that the literature on treatments can be accessed and evaluated to identify the treatment of choice. But this process would fall at the first step for many chronic pain patients: it can be difficult to arrive at a definitive diagnosis for many pain conditions [6]. Instead, the approach that Greenhalgh [11] has identified for general practice may be more appropriate: on the basis of signs and symptoms select a treatment and observe if it works. If it does not work then further treatment may be tried until success is achieved or treatment options are exhausted.
CONCLUSION There is little doubt that evidence-based medicine is the way of the future. The salutary example of the production of evidence in obstetric care, which revealed that many cherished interventions were of no benefit or were even harmful [4], should convince all of the potential value of systematic reviews. What Linton does in his Focus article is not to decry all of evidence-based medicine, but to outline the limits of evidence-based medicine and to suggest alternative approaches. One major issue is the extent to which evidence-based medicine will lead to clinical practice being prescribed by published evidence. The proper concern of the clinician is not limited to identifying evidence that a treatment can work, but must address whether it works for local patients. It would be unfortunate if clinicians were forced to use treatments that clinical judgment suggests may
COMMENTARY/Crombie
not work, solely because they are enshrined in quidelines. This view seems so eminently sensible one might wonder why it is not shared by the advocates of evidencebased medicine. The answer is that it is. In their outstanding book on the topic, Sackett and colleagues stress that external research evidence should be integrated with clinical experience to determine the best management for individual patients [17]. One of the five sections of this book is devoted to techniques for achieving this integration. Even the use of N-of-1 trials, which Linton proposes for evaluating treatment effects, are described in this section of the book. Why then has Linton written his Focus article? The answer lies less with what evidence-based medicine is, than with the way it could be used. Many other commentators have voiced concerns about the impact evidencebased medicine will have on clinical practice. An editorial in Lancet claimed that it had grown from: "a subversive whisper to a strident insistence that it is improper to practise medicine of any other kind" [1]. Members of the Sydenham Society "feared that the literature produced through evidence-based medicine was being misused by policymakers to create a dogmatic atmosphere in which practice guidelines had greater authority than physicians" [19]. Linton's contribution is to raise these arguments within the field of pain. That is why his article is important and deserves careful attention. Rather than advocate unthinking acceptance, Linton questions whether the innovation of evidence-based medicine will lead to what is intended: universal high quality of medical care (given the available resources). Meta-analysis is only the starting point of evidence-based medicine; if anything it is the easy part. The real challenge lies in integrating the findings of systematic reviews into clinical practice, combining research evidence with clinical experience. Linton suggests ways of doing this. The way forward is to test his ideas through research.
References 1. Anonymous: Evidence-based medicine, in its place 1. Lancet 346:785, 1995 2. Berde C, Glick R: The placebo response: powerful and still puzzling. IASP Newslett, 3-5, July/August:1994 3. Carron H, DeGood DE, Tait R: A comparison of low back pain patients in the United States and New Zealand:
65
psychosocial and economic factors affecting severity of disability. Pain 21:77-89, 1985 4. Chalmers I, Enkin M, Keirse M: Effective care in pregnancy and childbirth. Oxford University Press, London, 1989 5. Cohen JE, Goe LV, Frank VW et al: Group education interventions for people with low back pain. Spine 19:12141222,1994 6. Crombie IK, Davies HTO: Audit of outpatients: entering the loop. Br Med J 302:1437-1439,1991 7. Deyo RA, Bass JE, Walsh NE et al: Prognostic variability among chronic pain patients: implications for study design, interpretation, and reporting. Arch Physical Med and Rehab69:174-178,1988 8. Evans JG: Evidence-based medicine and evidencebiased medicine. Age Aging 24:461-463,1995 9. Evidence-based Medicine Working Group: Evidencebased medicine: a new approach to teaching the practice of medicine. JAMA 268:2420-2425, 1992 10. Flor H, Fydrich T, Turk DC: Efficacy of multidisciplinary pain treatment centers: a meta-analytic review. Pain 49: 221-230,1992 11. Greenhalgh T: Is my practice evidence-based? Br Med J 313:957-958,1996 12. Holzman AD, Rudy TE, Gerber KE et al: Chronic pain: a multiple-setting comparison of patient characteristics. J Behav Med 8:411-422,1985 13. Koes B, Bouter L, van der Heijden G: Methodological quality of randomized controlled trials on treatment efficacy in low back pain. Spine 20:228-235, 1995 14. Linton S, Kamwendo K: Low back schools: a critical review. Physical Ther 67:1375-1383, 1987 15. McQuay H, Carroll D, Jadad A et al: Anticonvulsant drugs for management of pain: a systematic review. Br Med J 311 :11047-11052, 1995 16. McQuay H, Tramer M, Nye BA et al: A systematic review of antidepressants in neurogenic pain. Pain 68:217-227, 1996 . 17. Sackett D, Richardson W, Rosenberg W, Haynes R: Evidence-based medicine: how to practice and teach EBM. Churchill Livingstone, New York, 1997 18. Sackett D, Rosenberg W, Gray J et al: Evidence-based medicine: whatit is and what it isn't. Br Med J 312:71-72, 1996 19. Shuchman M: Evidence-based medicine debated. Lancet 347:1396,1996 20. Turk DC, Rudy TE: Neglected factors in chronic pain treatment outcome studies: referral patterns, failure to enter treatment, and attrition. Pain 43:7-25, 1990 21. Turner J, Deyo R, Loeser J et al: The importance of placebo effects in pain treatment and research. JAMA 271:1609-1614, 1994
The Limits of Evidence-Based Medicine lain Kinloch Crombie
vidence-based medicine looks set to be the last major innovation to clinical practice in the twentieth century. It has been defined as "the conscientious, explicit and judicious use of current best evidence in making decisions about the care of individual patients" [18]. It seeks to replace the current medical paradigm, in which clinical experience and an understanding of the pathophysiology of disease are key elements in determining management. Instead, it proposes to root medicine in the findings of randomized controlled trials [9]. The randomized controlled trial provides the best quality of evidence about the effectiveness of a treatment. Systematic reviews provide unbiased authoritative summaries of randomized controlled trials. Thus, these should be used to inform choice of treatment, placing the practice of medicine on a sound scientific base. Evidence-based medicine is set to usher in an era of the highest-quality clinical practice. The purposes of evidence-based medicine are so clearly desirable that it might appear churlish to challenge them. Yet, this is what Linton does in his Focus article. His main concern is that the adoption of the principles of evidence-based medicine will take the "reason" out of medicine. He argues that reliance on externally produced evidence (systematic reviews) will deny the value of clinical experience and diminish its role in medicine. Although several other themes are identified in the article, this Commentary focuses on and develops the ideas that underpin the limits to the practice of evidence-based medicine.
E
From the Department of Epidemiology and Public Health, University of Dundee, Ninewells Hospital and Medical School, Dundee, United Kingdom. Reprint requests: lain Kinloch Crombie, Department of Epidemiology and Public Health, Ninewells Hospital and Medical School, Dundee DD1 9SY, United Kingdom. ©
1998 the American Pain Society
1058-9139/0701-0015$5.00/0
Pain Forum 7(1): 63-65, 1998
THE LIMITS OF EVIDENCE-BASED MEDICINE Central to evidence-based medicine is the view that systematic reviews should supplant clinical experience. Certainly, the production of systematic reviews under the auspices of the Cochrane Collaboration represents a major advance for medicine. But there are several reasons why, particularly in the field of chronic pain, it may not always be possible or desirable to practice evidence-based medicine. These can be summarized as three questions for evidence-based medicine: Is there enough evidence? Can the evidence be relied on? Is the process of evidence-based medicine appropriate?
Is There Enough Evidence? The first problem for implementing evidence-based medicine is that the evidence base is not sufficiently well established for effective treatments to be identified for each of the variety of pain conditions. Some advances have been made, for example, the systematic reviews of antidepressants [16] and anticonvulsants [15]. But for many of the treatments currently in use, this level of evidence is unavailable. For some conditions, the necessary studies have not been done. This is particularly true of those pain conditions that are difficult to diagnose and are therefore unlikely to have been studied in clinical trials. But in other circumstances the available evidence is not sufficient for' definitive conclusions. The studies may be of poor quality: Linton cites the example of the poor quality of randomized controlled trials on treatment efficacy in low back pain [13]. Likewise, Flor and colleagues found that studies of the efficacy of multidisciplinary pain clinics were generally of poor quality, either lacking control groups or using inappropriate control groups [10]. The consequence of the deficiencies in the available evidence is that there may often be little alternative than to rely, as medicine has long done, on clinical acumen and clinical experience. Denigrating these skills before the necessary evidence is acquired could be detrimental to patient care. It will be many years before much of the
63
64
COMMENTARy/Crombie
management of chronic pain has the weight of research findings that evidence-based medicine needs. Until then, we need good clinical skills.
Can the Evidence Be Relied On? A further concern is whether the findings from systematic reviews can be applied to patients seen at a local clinic. Linton raises the issue for the practitioner of "whether a treatment or diagnostic procedure is truly producing results for his or her patient. "The two areas for concern are the nature of the patient group and the nature of the treatment. To apply research findings in local practice, the types of patients treated in clinical trials need to be representative of patients seen routinely in clinic. The problem is that they are probably not. In general, patients are excluded from clinical trials if they are old or have multiple pathologies. But these are the types of patients commonly seen in clinic. This difficulty is compounded by the fact that patient referral processes vary substantially between clinics [3,6,12]. As clinical trials are often conducted in specialist tertiary referral centers, referral bias is likely to be especially marked. Patients who take part in therapeutic trials may be more highly motivated [20] and may have more severe pain. They may show a much greater improvement in pain scores than would be the case for pain patients not included in trials [7]. The second issue is whether the way the treatment is given in the clinical trial can be administered in a similar way in routine practice. Again, the answer is probably not. As Linton points out, "the way the treatment is applied, the communication between caregiver and patient, the setting, the level of anxiety, and so forth, may all influence the results of techniques designed to relieve pain. "This is likely to be particularly so for psychological therapies, but may also be true for other treatment modalities. The magnitude of the placebo effect and the extent to which it varies between studies illustrates the scope that there is for situational factors to influence treatment outcome [2,21]. These concerns about the applicability of systematic reviews are intensified by the observation that the findings from meta-analyses may be unreliable. Linton points out that the conclusions from one meta-analysis of back schools for back pain, that they are effective [13], are contradictory to those made by two other reviews [5,14]. In this instance, the explanation may lie with the criteria for selection of studies for inclusion in the analysis. The potential problem for evidence-based medicine is that it may shift the focus from the current concerns with the local assessment treatment (clinical experience) to an overreliance on the findings from clinical trials or systematic reviews. Evans has voiced similar concerns
for geriatricians: "There is a fear that in the absence of evidence clearly applicable to the case in hand a clinician might be forced by guidelines to make use of evidence which is only doubtfully relevant, generated perhaps in a different grouping of patients in another country at some other time using a similar but not identical treatment" [8]. Rather than being a boon to medical practice, systematic reviews may become a clinical straitjacket.
Is the Process of Evidence-Based Medicine Appropriate? Evidence-based medicine is described as providing a new paradigm for medical practice [9]. But there is a question of whether the paradigm will always be suitable for pain. The new paradigm states that in a consultation with a patient, the doctor should clarify the clinical problem that needs to be addressed. Published evidence on this should be sought and critically appraised, to identify the appropriate actions to be taken to deal with the clinical problem. A simple example of the new paradigm would be the management of a patient complaining of pain. First, the diagnosis would be clarified so that the literature on treatments can be accessed and evaluated to identify the treatment of choice. But this process would fall at the first step for many chronic pain patients: it can be difficult to arrive at a definitive diagnosis for many pain conditions [6]. Instead, the approach that Greenhalgh [11] has identified for general practice may be more appropriate: on the basis of signs and symptoms select a treatment and observe if it works. If it does not work then further treatment may be tried until success is achieved or treatment options are exhausted.
CONCLUSION There is little doubt that evidence-based medicine is the way of the future. The salutary example of the production of evidence in obstetric care, which revealed that many cherished interventions were of no benefit or were even harmful [4], should convince all of the potential value of systematic reviews. What Linton does in his Focus article is not to decry all of evidence-based medicine, but to outline the limits of evidence-based medicine and to suggest alternative approaches. One major issue is the extent to which evidence-based medicine will lead to clinical practice being prescribed by published evidence. The proper concern of the clinician is not limited to identifying evidence that a treatment can work, but must address whether it works for local patients. It would be unfortunate if clinicians were forced to use treatments that clinical judgment suggests may
COMMENTARY/Crombie
not work, solely because they are enshrined in quidelines. This view seems so eminently sensible one might wonder why it is not shared by the advocates of evidencebased medicine. The answer is that it is. In their outstanding book on the topic, Sackett and colleagues stress that external research evidence should be integrated with clinical experience to determine the best management for individual patients [17]. One of the five sections of this book is devoted to techniques for achieving this integration. Even the use of N-of-1 trials, which Linton proposes for evaluating treatment effects, are described in this section of the book. Why then has Linton written his Focus article? The answer lies less with what evidence-based medicine is, than with the way it could be used. Many other commentators have voiced concerns about the impact evidencebased medicine will have on clinical practice. An editorial in Lancet claimed that it had grown from: "a subversive whisper to a strident insistence that it is improper to practise medicine of any other kind" [1]. Members of the Sydenham Society "feared that the literature produced through evidence-based medicine was being misused by policymakers to create a dogmatic atmosphere in which practice guidelines had greater authority than physicians" [19]. Linton's contribution is to raise these arguments within the field of pain. That is why his article is important and deserves careful attention. Rather than advocate unthinking acceptance, Linton questions whether the innovation of evidence-based medicine will lead to what is intended: universal high quality of medical care (given the available resources). Meta-analysis is only the starting point of evidence-based medicine; if anything it is the easy part. The real challenge lies in integrating the findings of systematic reviews into clinical practice, combining research evidence with clinical experience. Linton suggests ways of doing this. The way forward is to test his ideas through research.
References 1. Anonymous: Evidence-based medicine, in its place 1. Lancet 346:785, 1995 2. Berde C, Glick R: The placebo response: powerful and still puzzling. IASP Newslett, 3-5, July/August:1994 3. Carron H, DeGood DE, Tait R: A comparison of low back pain patients in the United States and New Zealand:
65
psychosocial and economic factors affecting severity of disability. Pain 21:77-89, 1985 4. Chalmers I, Enkin M, Keirse M: Effective care in pregnancy and childbirth. Oxford University Press, London, 1989 5. Cohen JE, Goe LV, Frank VW et al: Group education interventions for people with low back pain. Spine 19:12141222,1994 6. Crombie IK, Davies HTO: Audit of outpatients: entering the loop. Br Med J 302:1437-1439,1991 7. Deyo RA, Bass JE, Walsh NE et al: Prognostic variability among chronic pain patients: implications for study design, interpretation, and reporting. Arch Physical Med and Rehab69:174-178,1988 8. Evans JG: Evidence-based medicine and evidencebiased medicine. Age Aging 24:461-463,1995 9. Evidence-based Medicine Working Group: Evidencebased medicine: a new approach to teaching the practice of medicine. JAMA 268:2420-2425, 1992 10. Flor H, Fydrich T, Turk DC: Efficacy of multidisciplinary pain treatment centers: a meta-analytic review. Pain 49: 221-230,1992 11. Greenhalgh T: Is my practice evidence-based? Br Med J 313:957-958,1996 12. Holzman AD, Rudy TE, Gerber KE et al: Chronic pain: a multiple-setting comparison of patient characteristics. J Behav Med 8:411-422,1985 13. Koes B, Bouter L, van der Heijden G: Methodological quality of randomized controlled trials on treatment efficacy in low back pain. Spine 20:228-235, 1995 14. Linton S, Kamwendo K: Low back schools: a critical review. Physical Ther 67:1375-1383, 1987 15. McQuay H, Carroll D, Jadad A et al: Anticonvulsant drugs for management of pain: a systematic review. Br Med J 311 :11047-11052, 1995 16. McQuay H, Tramer M, Nye BA et al: A systematic review of antidepressants in neurogenic pain. Pain 68:217-227, 1996 . 17. Sackett D, Richardson W, Rosenberg W, Haynes R: Evidence-based medicine: how to practice and teach EBM. Churchill Livingstone, New York, 1997 18. Sackett D, Rosenberg W, Gray J et al: Evidence-based medicine: whatit is and what it isn't. Br Med J 312:71-72, 1996 19. Shuchman M: Evidence-based medicine debated. Lancet 347:1396,1996 20. Turk DC, Rudy TE: Neglected factors in chronic pain treatment outcome studies: referral patterns, failure to enter treatment, and attrition. Pain 43:7-25, 1990 21. Turner J, Deyo R, Loeser J et al: The importance of placebo effects in pain treatment and research. JAMA 271:1609-1614, 1994