- Email: [email protected]
Evidence-based Medicine and Practice Guidelines: Solution or Problem? Part 1. Evidence-based Medicine and Evidence-based Surgery
Bignamini Asian J Oral Maxillofac Surg 2003;15:7-13. SPECIAL CONTRIBUTION
Evidence-based Medicine and Practice Guidelines: Solution or Problem? Part 1. Evidence-based Medicine and Evidence-based Surgery Angelo Antonio Bignamini Centre for Bioethics, School of Medicine, Sacred Heart Catholic University, Rome, Italy
Abstract Evidence-based medicine is widely used for decision-making in the medical field; evidence-based surgery is much less used, however, due to intrinsic and extrinsic limitations, primarily the operators’ variability of performance. Evidence-based medicine and evidence-based surgery are effectively used to synthesise, in practical statements and numbers, the large amount of clinical information published in the medical literature, leading to the exclusion from medical practice of objectively useless or damaging procedures. However, the risks associated with their incorrect use ought not to be overlooked. Evidence-based medicine and evidencebased surgery are physically limited by the published literature, since only well-studied procedures can become the object of undisputed external evidence. Evidence-based medicine and evidence-based surgery are limited in their applicability to the individual patient, since the statistical inference on which they are based is applicable to populations, not to individuals. Finally, their very generation can be biased. Evidence-based medicine and evidence-based surgery can be useful tools for the education of health care workers and an effective support for planning the requisition of available medical resources. Neither, however, can restrict the right of each patient to the best available treatment, nor should they restrict the right and duty of each physician to apply to each patient the therapeutic approach considered most suitable under the specific conditions applicable to an individual. The physician’s compliance with the information originated by evidence-based medicine and evidence-based surgery can, at most, guarantee that no major mistakes have been made, rather than to have acted in the patient’s best interest, and in an ethically appropriate way. Key Words: Evidence-based medicine, Methods
Introduction In 1992, a paper in the Journal of the American Medical Association introduced the term ‘evidencebased medicine’ (EBM).1 This paper presumably responded to a real need in medical practice because, from that single paper in 1992, the number of papers indexed annually using this term had increased to 2385 by 2001. Correspondence: Angelo Antonio Bignamini, Via G Pascoli 58, 20133 Milan, Italy Tel: (39 02) 7060 5179 Fax: (39 02) 2668 0727 Email: [email protected]
Asian J Oral Maxillofac Surg Vol 15, No 1, 2003
EBM is a technique to retrieve, screen, and compound the best external evidence with the experience and training of the physician, to best respond to the specific medical need of each individual patient. As such, EBM was supposed to cover all aspects of medicine, including diagnosis, medical treatment, and surgical treatment. It appears, however, not to be so since, in 2001, the number of papers found using the combined heading ‘evidencebased’ and ‘surgery’ was only 79. The time course of papers appearing using the 2 terms indicates that the interest in evidence-based surgery (EBS) is delayed 7
Evidence-based Medicine and Surgery
100 Evidence-based medicine (EBM) Evidence-based surgery (EBS) Oral/maxillofacial (MF) surgery
2,000
80
1,500
60
1,000
40
500
20
0
1992
1994
1996
1998
2000
Papers: EBS
Papers: EBM, oral/MF surgery
2,500
0
Time (years)
Figure 1. Number of papers that can be retrieved from Medline with the search [“evidence based medicine” (MeSH term)], with the search [(oral OR maxillofacial) AND surgery], or with the search [“evidence-based” AND surgery].
by approximately 2 years, and is approximately 30fold less than that of EBM. For comparison, in 2001, the number of papers indexed using the term ‘oral or maxillofacial surgery’ was 1983 (Figure 1). It is therefore relevant to understand why EBS is not considered as frequently as EBM, and whether this is likely to change in the near future.
Evidence-based Medicine What Is Evidence-based Medicine? ‘Evidence-based medicine’ is “the integration of best research evidence with clinical expertise and patient values”2 or “the conscientious, explicit and judicious use of current best evidence in making decisions about the care of individual patients”.3 With the term ‘best research evidence’, EBM points to “clinically relevant research ... into the accuracy and precision of diagnostic tests ..., the power of prognostic markers, and the efficacy and safety of therapeutic, rehabilitative, and preventive regimens”2 submitted to the Popperian process of verificationfalsification.4-7 The term ‘clinical expertise’ is intended as “the ability to use our clinical skills and past experience to rapidly identify each patient’s unique health state and diagnosis, their individual risks and benefits of potential interventions, and their personal values and 8
expectations”.2 Finally, EBM defines ‘patient values’ as “the unique preferences, concerns and expectations each patient brings to a clinical encounter and which must be integrated into clinical decisions if they are to serve the patient.”2 EBM maintains that: “when these three elements are integrated, clinicians and patients form a diagnostic and therapeutic alliance which optimises clinical outcomes and quality of life”.2 What Evidence-based Medicine Is Not One of the principal founders of EBM felt the need, as early as 1996, to specifically indicate what EBM is not — “Evidence-based medicine is not ‘cookbook’ medicine ... is not cost-cutting medicine ... is not restricted to randomised trials and metaanalyses”.3 This was because, as early as 4 years after its conception, EBM was considered a standardised recipe for decision-making, thus avoiding the time and effort to think about a specific medical problem; a means of cost-cutting in an institution by only supplying interventions “based on evidence”; and a ‘super-review’ of clinical papers, among which only meta-analyses and randomised controlled trials (preferentially only those published in English) were considered to be ‘best research evidence’. Sacket et al rejected 2 further arguments against the generalised use of EBM. They wrote that “evidencebased medicine is neither old-hat nor impossible to practice”, and that “the argument that evidence-based medicine can be conducted only from ivory towers and armchairs is refuted”.3 Although it is certainly true that EBM is not ‘old-hat’ and it can be put into practice, it is also true that the process of gathering, evaluating, critically appraising, and summarising the evidence into suggestions for clinical practice is not for everyone. Indeed, given the short time available for updating their knowledge and the large amount of published material, practicing clinicians need to refer to specialised groups to find and summarise the evidence, epitomised by the Cochrane Collaboration* and journals such as Evidence-Based Medicine† and Clinical Evidence‡ and others. Interestingly, despite being so widely quoted and heavily promoted, * http://www.cochrane.org † http://ebm.bmjjournals.com ‡ http://www.clinicalevidence.com
Asian J Oral Maxillofac Surg Vol 15, No 1, 2003
Bignamini
altogether, the evidence-based journals have a worldwide circulation of less than 200,0002 — for comparison, in a medium-sized country such as Italy, the combined number of family physicians, hospital physicians, and surgeons, was approximately 104,000 in 1998.8 How Is Evidence-based Medicine Developed? The practice of EBM comprises 5 steps:2 • converting the need for information into an answerable question • tracking down the best evidence to answer that question • critically appraising that evidence for validity, impact, and applicability • integrating the critical appraisal with the physician’s clinical expertise and with the patient’s unique biology, values, and circumstances • evaluating one’s own effectiveness and efficiency in executing the previous steps and seeking ways to improve for the future. It is immediately evident that none of the above steps are within practical reach of the practising physician, although they may theoretically be performed by anyone appropriately trained in this technique. However, most clinical work naturally fits within this frame. Indeed, the way a physician normally works is to informally shape the individual clinical problem into a question, find an answer to the question from a trusted source, apply the answer to the individual patient with the problem, and learn from experience how to better solve a similar problem the next time it occurs. Meanwhile, the step from normal clinical activity to EBM rests on the formalisation and rationalisation of the procedure, and this may not be easy in the current medical environment. Is Evidence-based Medicine Needed? If it is true that the 5 steps indicated above are, more or less, the normal procedure for a sound medical decision, then the modern world makes these tasks difficult. These days, there is a great and increasing need to get appropriate and valid information about diagnosis, prognosis, therapy, and follow-up for a growing number of patients; the traditional sources Asian J Oral Maxillofac Surg Vol 15, No 1, 2003
of information such as textbooks may be outdated, conselling by experts, professors and colleagues may be unreliable in relation to the specific problem, and the medical literature is too much and too variable to be rapidly assimilated — Medline alone indexed more than 507,000 papers in 2000 and more than 510,000 in 2001; the time available to find and critically apprise the evidence and to transform the evidence into an appropriate and applicable clinical action is too limited to make practical the application of the EBM for all patients. Thus, on the one hand it is apparent that EBM is needed, since it makes the clinical decisions more rational, on the other, it is apparent that EBM cannot be routinely applied by all practising physicians. The individual physician therefore has to trust the experts who critically appraised topics of interest, the most easily accessed being those of the Cochrane Collaboration and the Evidence-based On Call§ database. Evidence-based Surgery: What Is the Problem? Why is there so much material about EBM, and so little about EBS? Indeed, are the physicians correct in claiming that, since the inception of EBM, surgeons do not care about the deficiencies in the evidence base for their speciality?9 This criticism is correct, based on the number of available papers. However, there are historical and practical reasons for this. Historically, medical therapy has developed more recently than surgery, and the acceptance of new medications is based on the objective assessment through randomised controlled trials (RCTs). On the contrary, most basic surgical techniques developed at a time when observation of a direct outcome was the mainstay of clinical evaluation, rather than standardised trials. More importantly, however, medical and surgical therapy differ in some essential points. Firstly, in most cases, surgery has to correct a deranged anatomical (mechanical) condition. In this situation, there is no reasonable alternative to the surgical treatment, and this excludes all possible §
http://www.eboncall.co.uk
9
Evidence-based Medicine and Surgery
comparisons, providing an internal validity, rather than relying on an external evidence. External evidence (RCT) is therefore limited to those situations that are not self-evident. In other words, when there are alternative interventions for the same condition, and there is no superiority from any viewpoint, including patients’ preferences, of either approach — ‘equipoise’ criterion, essential for any ethically acceptable RCT. However, even when a RCT is appropriate, scientifically sound, ethically acceptable, technically feasible, and properly funded, surgery still differs substantially from diagnosis or medical therapy.10 These latter are essentially operator-independent, for example, in a well-designed RCT, the results will always be the same (within each other’s confidence interval) in a sufficiently large population, provided that population, intervention, and compliance are the same regardless of the operator’s skill, and for as long as the protocol is followed. This is not true for surgery. Surgery is a skilled, multi-step process, with a specific learning curve for any new operation or technique.11 This alone could jeopardise the validity of a RCT of a new versus an established procedure. In addition, each surgical team introduces an inherent variation on how every procedure is performed, despite all the efforts for quality control in the technical aspects of the procedure. Thus, the result of a surgical RCT is not operator-independent. Furthermore, until recently, there were no requirements in surgical RCTs for a standardised evaluation of the surgeons’ performance prior to the initiation of the trial. Much of the cynicism expressed by surgeons about RCTs stems from their concern about this inability of crude designs to acknowledge the critical importance of quality in defining surgical outcome.12 Although this does not mean that external evidence cannot be obtained in surgical RCTs, it nevertheless becomes evident that the design and conduct of surgical RCTs has to be different from those relevant to medical therapy, and probably the use of nationwide or international standardised registries should be promoted, instead of, or as well as, multicentre RCTs. 10
Evidence-base: a No-nonsense, No-problem Approach to Medicine and Surgery? There is no question that the EBM responds to several recognised problems of modern medical practice: • the practical impossibility for the individual physician to keep properly up to date with new information • awareness that the approach to diagnosis, therapy and management should incorporate population statistical data, including the concepts of significance of differences and of statistical power • awareness that the limited resources available for health care call for more effective and more efficient requisition for a given outcome. However, the approach of EBM to these problems is the epitome of the mechanistic approach to medicine. Thus, given a technical problem, the solution is the one which gives the most efficient technical answer. It is true that EBM aims at incorporating “the unique preferences, concerns and expectations”2 of patients into the medical decision. It is equally true that this is viewed from the biased viewpoint of the EBM specialist only, to the point that it becomes obvious that the patient should agree to the expert’s decisions once technically well documented,13 with the result that patients are more and more attracted to more respectful if ineffective forms of alternative unproven care, rather than EBM or EBS. This type of patient-independent, exclusively technical approach can easily be documented despite the claims of the founders of EBM. From 1997 to 2001, 7446 papers were indexed in Medline using the term ‘evidence-based medicine’. Of these, 12.8% (953 papers) also dealt with medical economics, but only 3.4% (254 papers) included medical ethics. In EBS, the difference is even more striking. Of the 227 papers containing the terms ‘evidence-based’ and ‘surgery’ published during the same period, 14.1% also included the descriptor ‘economy’, but just 1 included the term ‘ethics’ (0.4%). The second objection to EBM and EBS is that there are many more medical and surgical procedures that are unsuitable for an appraisal process than there are procedures that are suitable. Indeed, although it Asian J Oral Maxillofac Surg Vol 15, No 1, 2003
Bignamini
is stated that EBM is not only meta-analyses and RCTs,3 it is nevertheless true that, without such documentation, a procedure cannot be considered evidence-based.2 One can submit a procedure to a trial only under the ‘equipoise’ condition, and this excludes all those interventions that are undisputedly necessary, as well as those that, when submitted to a comparison, would necessitate a group of patients being exposed to unjustified, unnecessary risk. Not collecting basic haematochemical data for a patient with acute myocardial infarction is considered negligence or malpractice; consequently, the evidence demonstrating that collecting laboratory data under such circumstances is appropriate, cannot be obtained. Similarly, there are no RCTs after Semmelweis’ observations demonstrating that appropriate asepsis before surgery is necessary, although, there is no surgeon who would not perform the relevant procedures.14 This point is of such relevance, and relates to so many essential medical and surgical procedures that the recent SIGN50 methodology to write practice guidelines,15 has introduced the Good Practice Point recommending an essential procedure that cannot be experimentally verified. The third objection to the indiscriminate and uncritical application of EBM arises from its strong point, the statistical approach. Statistics is a population descriptor, but medical interventions are applied to individual patients. One can therefore properly estimate how many patients in a population can benefit or not benefit by an intervention, but can never indicate which patients will benefit. Thus, it is ethically unacceptable to justify major medical decisions for an individual patient taking into account only population-based data (often from populations other than the one the patient belongs to), without considering the benefit for the individual patient. A fourth objection lies in the fact that the process of formation of critical appraisals may introduce a bias due to a publication bias — generally, positive findings are published whereas negative findings are not, and the trend is for appraisal by English-speaking experts, who usually exclude studies published in other languages, a standardisation bias — not all the published outcomes are directly comparable and qualitative outcomes can easily be misinterpreted, Asian J Oral Maxillofac Surg Vol 15, No 1, 2003
a population bias — the population included in RCTs is often not comparable with the patients seen in clinical practice due to restrictive inclusion and exclusion criteria.16 Owing to the procedures that form EBM, and to the limits found in its application to a sick person, one can summarise the criticisms of EBM and EBS as shown in Table 1.17
Conclusion EBM is a technique to evaluate and summarise, in handy statements, the most relevant information available. EBM is a tool, but not the aim — it is a powerful tool for accessing population-based information on effective clinical conduct data. However, for a clinician, be it a physician or surgeon, the final aim is to care for and service the individual sick person. Following EBM may result in fewer incorrect decisions and more effective resource requisition, where EBM supplies sound, reliable, and applicable conclusions. However, the very nature of EBM cannot guarantee that each conclusion is applicable for each individual patient and for all necessary medical decisions. It is therefore true that EBM is not a cookbook, nor a cost-cutting procedure,3 but it is equally true that it is not a guarantee of taking the best decision. Rather, it is, at most, a guarantee not to make critical mistakes. EBM is dependent on 3 variables: the clinician with her or his expertise, experience, and training; the patient, with her or his needs, preferences, and personal history; and the evidence, which is external to and independent of the patient-physician relationship. Whichever EBM-based decision is taken, it is located at this intersection, and should respect the rights of each individual to the safeguard of life, health, and physical and psychical integrity.18,19 Therefore, the useful tools of EBM and EBS cannot, in any case and under any circumstances, replace the considerate clinical judgement of how to care for each individual patient, nor decrease or waive the physician’s moral, ethical, and judicial responsibility towards that individual sick person. 11
Evidence-based Medicine and Surgery Difficult
Should be based on scientific documentation of enormous size and of widely variable quality
Risky
Not every medical aspect can be quantitatively measured, but only quantities, better if expressed as dichotomous outcomes, can correctly be summarised in unambiguous statements
Partial
Not everything is published, thus opinions, which in the real world significantly influence the medical choices, are excluded Not everything that has been published is analysed, but only those papers that can be retrieved by, and fit with, specific principles and methods
Relative
Should be regularly verified and updated The subjects not dealt with are not immediately comparable with other already examined and tested
Absolutist
All themes not yet examined according to the accepted criteria are excluded from discussion and critical evaluation
Conditioned
Is strongly dependent from, and limited by, statistical technique
Rigid
Conclusions are valid for a population, but many are difficult or impossible to apply to the individual patient
Exclusivist
Several evidence-based medicine specialists, in particular with an economic background, are creating an ‘inner circle’ — a kind of priestly caste to which admission is by co-optation only
Circumscribed
Although designed, in principle, to evaluate every medical act, it is optimised to test the outcome of (dispensable) therapeutic treatments or procedures
Self-referential
Since it begins with an ‘answerable’ question, to formulate the question one should already know that answers exist, and since the trials are designed to answer the most likely formulated questions, evidence-based medicine questions and randomised controlled trials originate a self-referential loop
Incomplete
In the (few) cases in which practical and applicable conclusions are supplied, they are useful. However, most medical techniques cannot be evaluated or the related evidence is insufficient, yet no one has the interest to design, perform, and fund the relevant randomised controlled trials
Pitiless
When the attending physician has to answer an individual patient’s problem in terms of ‘number needed to treat’
Dyslexic
Conclusions cannot be ‘shared’ between EBM expert and physician, even less between physician and patient, because the knowledge and expertise of how the conclusions originated are too diverse
Absolutory
Since the study has already been done by other ‘specialists’, one delegates the decision and conclusion to somebody else, limiting the personal involvement and the personal obligations towards individual care
Table 1. Criticisms of the indiscriminate application of evidence-based medicine in clinical practice.
Acknowledgements The author is greatly indebted to Prof. Vito Toso, head of the Neurology-Stroke Unit of the Hospital ‘San Bortolo’, Vicenza, Italy, who, with his usual expressive capacity of synthesis, coupled with his great clinical sensibility and experience, largely contributed to Table 1.
References 1. Evidence-based Medicine Working Group. Evidence-based medicine. A new approach to teaching the practice of medicine. JAMA 1992: 268:2420-2425. 2. Centre for evidence-based medicine. http:// www.minervation.com/cebm/; Evidence-based on call. http://www.eboncall.co.uk/ 3. Sackett DL, Rosenberg WMC, Gray JAM, Haynes RB, Richardson WS. Evidence-based medicine: what it is and what it isn’t. BMJ 1996; 312:71-72. 4. Popper K. Logik der Forschung. Vienna: Julius Springer Verlag; 1935. 5. Popper K. Conjectures and refutations: the growth 12
of scientific knowledge. London: Routledge; 1963. 6. Popper K. Objective knowledge: an evolutionary approach. Oxford: Clarendon Press; 1972. 7. Shahar E. A Popperian perspective of the term ‘evidence-based medicine’. J Eval Clin Pract 1997; 3:109-116. 8. Italian Ministry of Health, tabulated by the National Institute of Statistics, ISTAT, http:// www.istat.it 9. Horton R. Surgical research or comic opera? Questions, but few answers. Lancet 1996;347: 984-985. 10. McLeod RS, Wright JG, Solomon MJ, Hu X, Walters BC, Lossing Al. Randomized controlled trials in surgery: issues and problems. Surgery 1996;119:483-486. 11. Stocchi L, Nelson H, Sargent DJ, et al. Impact of surgical and pathologic variables in rectal cancer: a United States community and cooperative group report. J Clin Oncol 2001;19:3895-3902. 12. Hermanek P, Hermanek PJ. Role of the surgeon as a variable in the treatment of rectal cancer. Asian J Oral Maxillofac Surg Vol 15, No 1, 2003
Bignamini
Semin Surg Oncol 2000;19:329-335. 13. Domenighetti G, Grilli R, Liberati A. Promoting consumers’ demand for evidence-based medicine. Int J Technol Assess Health Care 1998;14:97-105. 14. Semmelweis IP. Die aethiologie, der begriff und die prophylaxis des kindbettfiebers, Wien 1861. [English translation: Semmelweis I, Semmelweis IF, Codell Carter K (Translator)]. The etiology, concept, and prophylaxis of childbed fever. Wisconsin: Univ of Wisconsin Press: 1983. 15. SIGN 50: A guideline developers’ handbook. http://www.sign.ac.uk/guidelines/fulltext/50/ index.html
Asian J Oral Maxillofac Surg Vol 15, No 1, 2003
16. Feinstein AR, Horwitz RI. Problems in the “evidence” of “evidence-based medicine”. Am J Med 1997;103:529-435. 17. Bignamini AA. Evidence-based medicine and clinical practice guidelines: solution or part of the problem? [in Italian]. Medicina e Morale 2001; 2:225-249. 18. Mulrow CD, Cook DJ, Davidoff FD. Systematic reviews: critical links in the great chain of evidence. Ann Intern Med 1997;126:389-391. 19. Pellegrino ED. A philosophical basis of medical practice. New York: Oxford University Press; 1984.
13
Evidence-based Medicine and Practice Guidelines: Solution or Problem? Part 1. Evidence-based Medicine and Evidence-based Surgery Angelo Antonio Bignamini Centre for Bioethics, School of Medicine, Sacred Heart Catholic University, Rome, Italy
Abstract Evidence-based medicine is widely used for decision-making in the medical field; evidence-based surgery is much less used, however, due to intrinsic and extrinsic limitations, primarily the operators’ variability of performance. Evidence-based medicine and evidence-based surgery are effectively used to synthesise, in practical statements and numbers, the large amount of clinical information published in the medical literature, leading to the exclusion from medical practice of objectively useless or damaging procedures. However, the risks associated with their incorrect use ought not to be overlooked. Evidence-based medicine and evidencebased surgery are physically limited by the published literature, since only well-studied procedures can become the object of undisputed external evidence. Evidence-based medicine and evidence-based surgery are limited in their applicability to the individual patient, since the statistical inference on which they are based is applicable to populations, not to individuals. Finally, their very generation can be biased. Evidence-based medicine and evidence-based surgery can be useful tools for the education of health care workers and an effective support for planning the requisition of available medical resources. Neither, however, can restrict the right of each patient to the best available treatment, nor should they restrict the right and duty of each physician to apply to each patient the therapeutic approach considered most suitable under the specific conditions applicable to an individual. The physician’s compliance with the information originated by evidence-based medicine and evidence-based surgery can, at most, guarantee that no major mistakes have been made, rather than to have acted in the patient’s best interest, and in an ethically appropriate way. Key Words: Evidence-based medicine, Methods
Introduction In 1992, a paper in the Journal of the American Medical Association introduced the term ‘evidencebased medicine’ (EBM).1 This paper presumably responded to a real need in medical practice because, from that single paper in 1992, the number of papers indexed annually using this term had increased to 2385 by 2001. Correspondence: Angelo Antonio Bignamini, Via G Pascoli 58, 20133 Milan, Italy Tel: (39 02) 7060 5179 Fax: (39 02) 2668 0727 Email: [email protected]
Asian J Oral Maxillofac Surg Vol 15, No 1, 2003
EBM is a technique to retrieve, screen, and compound the best external evidence with the experience and training of the physician, to best respond to the specific medical need of each individual patient. As such, EBM was supposed to cover all aspects of medicine, including diagnosis, medical treatment, and surgical treatment. It appears, however, not to be so since, in 2001, the number of papers found using the combined heading ‘evidencebased’ and ‘surgery’ was only 79. The time course of papers appearing using the 2 terms indicates that the interest in evidence-based surgery (EBS) is delayed 7
Evidence-based Medicine and Surgery
100 Evidence-based medicine (EBM) Evidence-based surgery (EBS) Oral/maxillofacial (MF) surgery
2,000
80
1,500
60
1,000
40
500
20
0
1992
1994
1996
1998
2000
Papers: EBS
Papers: EBM, oral/MF surgery
2,500
0
Time (years)
Figure 1. Number of papers that can be retrieved from Medline with the search [“evidence based medicine” (MeSH term)], with the search [(oral OR maxillofacial) AND surgery], or with the search [“evidence-based” AND surgery].
by approximately 2 years, and is approximately 30fold less than that of EBM. For comparison, in 2001, the number of papers indexed using the term ‘oral or maxillofacial surgery’ was 1983 (Figure 1). It is therefore relevant to understand why EBS is not considered as frequently as EBM, and whether this is likely to change in the near future.
Evidence-based Medicine What Is Evidence-based Medicine? ‘Evidence-based medicine’ is “the integration of best research evidence with clinical expertise and patient values”2 or “the conscientious, explicit and judicious use of current best evidence in making decisions about the care of individual patients”.3 With the term ‘best research evidence’, EBM points to “clinically relevant research ... into the accuracy and precision of diagnostic tests ..., the power of prognostic markers, and the efficacy and safety of therapeutic, rehabilitative, and preventive regimens”2 submitted to the Popperian process of verificationfalsification.4-7 The term ‘clinical expertise’ is intended as “the ability to use our clinical skills and past experience to rapidly identify each patient’s unique health state and diagnosis, their individual risks and benefits of potential interventions, and their personal values and 8
expectations”.2 Finally, EBM defines ‘patient values’ as “the unique preferences, concerns and expectations each patient brings to a clinical encounter and which must be integrated into clinical decisions if they are to serve the patient.”2 EBM maintains that: “when these three elements are integrated, clinicians and patients form a diagnostic and therapeutic alliance which optimises clinical outcomes and quality of life”.2 What Evidence-based Medicine Is Not One of the principal founders of EBM felt the need, as early as 1996, to specifically indicate what EBM is not — “Evidence-based medicine is not ‘cookbook’ medicine ... is not cost-cutting medicine ... is not restricted to randomised trials and metaanalyses”.3 This was because, as early as 4 years after its conception, EBM was considered a standardised recipe for decision-making, thus avoiding the time and effort to think about a specific medical problem; a means of cost-cutting in an institution by only supplying interventions “based on evidence”; and a ‘super-review’ of clinical papers, among which only meta-analyses and randomised controlled trials (preferentially only those published in English) were considered to be ‘best research evidence’. Sacket et al rejected 2 further arguments against the generalised use of EBM. They wrote that “evidencebased medicine is neither old-hat nor impossible to practice”, and that “the argument that evidence-based medicine can be conducted only from ivory towers and armchairs is refuted”.3 Although it is certainly true that EBM is not ‘old-hat’ and it can be put into practice, it is also true that the process of gathering, evaluating, critically appraising, and summarising the evidence into suggestions for clinical practice is not for everyone. Indeed, given the short time available for updating their knowledge and the large amount of published material, practicing clinicians need to refer to specialised groups to find and summarise the evidence, epitomised by the Cochrane Collaboration* and journals such as Evidence-Based Medicine† and Clinical Evidence‡ and others. Interestingly, despite being so widely quoted and heavily promoted, * http://www.cochrane.org † http://ebm.bmjjournals.com ‡ http://www.clinicalevidence.com
Asian J Oral Maxillofac Surg Vol 15, No 1, 2003
Bignamini
altogether, the evidence-based journals have a worldwide circulation of less than 200,0002 — for comparison, in a medium-sized country such as Italy, the combined number of family physicians, hospital physicians, and surgeons, was approximately 104,000 in 1998.8 How Is Evidence-based Medicine Developed? The practice of EBM comprises 5 steps:2 • converting the need for information into an answerable question • tracking down the best evidence to answer that question • critically appraising that evidence for validity, impact, and applicability • integrating the critical appraisal with the physician’s clinical expertise and with the patient’s unique biology, values, and circumstances • evaluating one’s own effectiveness and efficiency in executing the previous steps and seeking ways to improve for the future. It is immediately evident that none of the above steps are within practical reach of the practising physician, although they may theoretically be performed by anyone appropriately trained in this technique. However, most clinical work naturally fits within this frame. Indeed, the way a physician normally works is to informally shape the individual clinical problem into a question, find an answer to the question from a trusted source, apply the answer to the individual patient with the problem, and learn from experience how to better solve a similar problem the next time it occurs. Meanwhile, the step from normal clinical activity to EBM rests on the formalisation and rationalisation of the procedure, and this may not be easy in the current medical environment. Is Evidence-based Medicine Needed? If it is true that the 5 steps indicated above are, more or less, the normal procedure for a sound medical decision, then the modern world makes these tasks difficult. These days, there is a great and increasing need to get appropriate and valid information about diagnosis, prognosis, therapy, and follow-up for a growing number of patients; the traditional sources Asian J Oral Maxillofac Surg Vol 15, No 1, 2003
of information such as textbooks may be outdated, conselling by experts, professors and colleagues may be unreliable in relation to the specific problem, and the medical literature is too much and too variable to be rapidly assimilated — Medline alone indexed more than 507,000 papers in 2000 and more than 510,000 in 2001; the time available to find and critically apprise the evidence and to transform the evidence into an appropriate and applicable clinical action is too limited to make practical the application of the EBM for all patients. Thus, on the one hand it is apparent that EBM is needed, since it makes the clinical decisions more rational, on the other, it is apparent that EBM cannot be routinely applied by all practising physicians. The individual physician therefore has to trust the experts who critically appraised topics of interest, the most easily accessed being those of the Cochrane Collaboration and the Evidence-based On Call§ database. Evidence-based Surgery: What Is the Problem? Why is there so much material about EBM, and so little about EBS? Indeed, are the physicians correct in claiming that, since the inception of EBM, surgeons do not care about the deficiencies in the evidence base for their speciality?9 This criticism is correct, based on the number of available papers. However, there are historical and practical reasons for this. Historically, medical therapy has developed more recently than surgery, and the acceptance of new medications is based on the objective assessment through randomised controlled trials (RCTs). On the contrary, most basic surgical techniques developed at a time when observation of a direct outcome was the mainstay of clinical evaluation, rather than standardised trials. More importantly, however, medical and surgical therapy differ in some essential points. Firstly, in most cases, surgery has to correct a deranged anatomical (mechanical) condition. In this situation, there is no reasonable alternative to the surgical treatment, and this excludes all possible §
http://www.eboncall.co.uk
9
Evidence-based Medicine and Surgery
comparisons, providing an internal validity, rather than relying on an external evidence. External evidence (RCT) is therefore limited to those situations that are not self-evident. In other words, when there are alternative interventions for the same condition, and there is no superiority from any viewpoint, including patients’ preferences, of either approach — ‘equipoise’ criterion, essential for any ethically acceptable RCT. However, even when a RCT is appropriate, scientifically sound, ethically acceptable, technically feasible, and properly funded, surgery still differs substantially from diagnosis or medical therapy.10 These latter are essentially operator-independent, for example, in a well-designed RCT, the results will always be the same (within each other’s confidence interval) in a sufficiently large population, provided that population, intervention, and compliance are the same regardless of the operator’s skill, and for as long as the protocol is followed. This is not true for surgery. Surgery is a skilled, multi-step process, with a specific learning curve for any new operation or technique.11 This alone could jeopardise the validity of a RCT of a new versus an established procedure. In addition, each surgical team introduces an inherent variation on how every procedure is performed, despite all the efforts for quality control in the technical aspects of the procedure. Thus, the result of a surgical RCT is not operator-independent. Furthermore, until recently, there were no requirements in surgical RCTs for a standardised evaluation of the surgeons’ performance prior to the initiation of the trial. Much of the cynicism expressed by surgeons about RCTs stems from their concern about this inability of crude designs to acknowledge the critical importance of quality in defining surgical outcome.12 Although this does not mean that external evidence cannot be obtained in surgical RCTs, it nevertheless becomes evident that the design and conduct of surgical RCTs has to be different from those relevant to medical therapy, and probably the use of nationwide or international standardised registries should be promoted, instead of, or as well as, multicentre RCTs. 10
Evidence-base: a No-nonsense, No-problem Approach to Medicine and Surgery? There is no question that the EBM responds to several recognised problems of modern medical practice: • the practical impossibility for the individual physician to keep properly up to date with new information • awareness that the approach to diagnosis, therapy and management should incorporate population statistical data, including the concepts of significance of differences and of statistical power • awareness that the limited resources available for health care call for more effective and more efficient requisition for a given outcome. However, the approach of EBM to these problems is the epitome of the mechanistic approach to medicine. Thus, given a technical problem, the solution is the one which gives the most efficient technical answer. It is true that EBM aims at incorporating “the unique preferences, concerns and expectations”2 of patients into the medical decision. It is equally true that this is viewed from the biased viewpoint of the EBM specialist only, to the point that it becomes obvious that the patient should agree to the expert’s decisions once technically well documented,13 with the result that patients are more and more attracted to more respectful if ineffective forms of alternative unproven care, rather than EBM or EBS. This type of patient-independent, exclusively technical approach can easily be documented despite the claims of the founders of EBM. From 1997 to 2001, 7446 papers were indexed in Medline using the term ‘evidence-based medicine’. Of these, 12.8% (953 papers) also dealt with medical economics, but only 3.4% (254 papers) included medical ethics. In EBS, the difference is even more striking. Of the 227 papers containing the terms ‘evidence-based’ and ‘surgery’ published during the same period, 14.1% also included the descriptor ‘economy’, but just 1 included the term ‘ethics’ (0.4%). The second objection to EBM and EBS is that there are many more medical and surgical procedures that are unsuitable for an appraisal process than there are procedures that are suitable. Indeed, although it Asian J Oral Maxillofac Surg Vol 15, No 1, 2003
Bignamini
is stated that EBM is not only meta-analyses and RCTs,3 it is nevertheless true that, without such documentation, a procedure cannot be considered evidence-based.2 One can submit a procedure to a trial only under the ‘equipoise’ condition, and this excludes all those interventions that are undisputedly necessary, as well as those that, when submitted to a comparison, would necessitate a group of patients being exposed to unjustified, unnecessary risk. Not collecting basic haematochemical data for a patient with acute myocardial infarction is considered negligence or malpractice; consequently, the evidence demonstrating that collecting laboratory data under such circumstances is appropriate, cannot be obtained. Similarly, there are no RCTs after Semmelweis’ observations demonstrating that appropriate asepsis before surgery is necessary, although, there is no surgeon who would not perform the relevant procedures.14 This point is of such relevance, and relates to so many essential medical and surgical procedures that the recent SIGN50 methodology to write practice guidelines,15 has introduced the Good Practice Point recommending an essential procedure that cannot be experimentally verified. The third objection to the indiscriminate and uncritical application of EBM arises from its strong point, the statistical approach. Statistics is a population descriptor, but medical interventions are applied to individual patients. One can therefore properly estimate how many patients in a population can benefit or not benefit by an intervention, but can never indicate which patients will benefit. Thus, it is ethically unacceptable to justify major medical decisions for an individual patient taking into account only population-based data (often from populations other than the one the patient belongs to), without considering the benefit for the individual patient. A fourth objection lies in the fact that the process of formation of critical appraisals may introduce a bias due to a publication bias — generally, positive findings are published whereas negative findings are not, and the trend is for appraisal by English-speaking experts, who usually exclude studies published in other languages, a standardisation bias — not all the published outcomes are directly comparable and qualitative outcomes can easily be misinterpreted, Asian J Oral Maxillofac Surg Vol 15, No 1, 2003
a population bias — the population included in RCTs is often not comparable with the patients seen in clinical practice due to restrictive inclusion and exclusion criteria.16 Owing to the procedures that form EBM, and to the limits found in its application to a sick person, one can summarise the criticisms of EBM and EBS as shown in Table 1.17
Conclusion EBM is a technique to evaluate and summarise, in handy statements, the most relevant information available. EBM is a tool, but not the aim — it is a powerful tool for accessing population-based information on effective clinical conduct data. However, for a clinician, be it a physician or surgeon, the final aim is to care for and service the individual sick person. Following EBM may result in fewer incorrect decisions and more effective resource requisition, where EBM supplies sound, reliable, and applicable conclusions. However, the very nature of EBM cannot guarantee that each conclusion is applicable for each individual patient and for all necessary medical decisions. It is therefore true that EBM is not a cookbook, nor a cost-cutting procedure,3 but it is equally true that it is not a guarantee of taking the best decision. Rather, it is, at most, a guarantee not to make critical mistakes. EBM is dependent on 3 variables: the clinician with her or his expertise, experience, and training; the patient, with her or his needs, preferences, and personal history; and the evidence, which is external to and independent of the patient-physician relationship. Whichever EBM-based decision is taken, it is located at this intersection, and should respect the rights of each individual to the safeguard of life, health, and physical and psychical integrity.18,19 Therefore, the useful tools of EBM and EBS cannot, in any case and under any circumstances, replace the considerate clinical judgement of how to care for each individual patient, nor decrease or waive the physician’s moral, ethical, and judicial responsibility towards that individual sick person. 11
Evidence-based Medicine and Surgery Difficult
Should be based on scientific documentation of enormous size and of widely variable quality
Risky
Not every medical aspect can be quantitatively measured, but only quantities, better if expressed as dichotomous outcomes, can correctly be summarised in unambiguous statements
Partial
Not everything is published, thus opinions, which in the real world significantly influence the medical choices, are excluded Not everything that has been published is analysed, but only those papers that can be retrieved by, and fit with, specific principles and methods
Relative
Should be regularly verified and updated The subjects not dealt with are not immediately comparable with other already examined and tested
Absolutist
All themes not yet examined according to the accepted criteria are excluded from discussion and critical evaluation
Conditioned
Is strongly dependent from, and limited by, statistical technique
Rigid
Conclusions are valid for a population, but many are difficult or impossible to apply to the individual patient
Exclusivist
Several evidence-based medicine specialists, in particular with an economic background, are creating an ‘inner circle’ — a kind of priestly caste to which admission is by co-optation only
Circumscribed
Although designed, in principle, to evaluate every medical act, it is optimised to test the outcome of (dispensable) therapeutic treatments or procedures
Self-referential
Since it begins with an ‘answerable’ question, to formulate the question one should already know that answers exist, and since the trials are designed to answer the most likely formulated questions, evidence-based medicine questions and randomised controlled trials originate a self-referential loop
Incomplete
In the (few) cases in which practical and applicable conclusions are supplied, they are useful. However, most medical techniques cannot be evaluated or the related evidence is insufficient, yet no one has the interest to design, perform, and fund the relevant randomised controlled trials
Pitiless
When the attending physician has to answer an individual patient’s problem in terms of ‘number needed to treat’
Dyslexic
Conclusions cannot be ‘shared’ between EBM expert and physician, even less between physician and patient, because the knowledge and expertise of how the conclusions originated are too diverse
Absolutory
Since the study has already been done by other ‘specialists’, one delegates the decision and conclusion to somebody else, limiting the personal involvement and the personal obligations towards individual care
Table 1. Criticisms of the indiscriminate application of evidence-based medicine in clinical practice.
Acknowledgements The author is greatly indebted to Prof. Vito Toso, head of the Neurology-Stroke Unit of the Hospital ‘San Bortolo’, Vicenza, Italy, who, with his usual expressive capacity of synthesis, coupled with his great clinical sensibility and experience, largely contributed to Table 1.
References 1. Evidence-based Medicine Working Group. Evidence-based medicine. A new approach to teaching the practice of medicine. JAMA 1992: 268:2420-2425. 2. Centre for evidence-based medicine. http:// www.minervation.com/cebm/; Evidence-based on call. http://www.eboncall.co.uk/ 3. Sackett DL, Rosenberg WMC, Gray JAM, Haynes RB, Richardson WS. Evidence-based medicine: what it is and what it isn’t. BMJ 1996; 312:71-72. 4. Popper K. Logik der Forschung. Vienna: Julius Springer Verlag; 1935. 5. Popper K. Conjectures and refutations: the growth 12
of scientific knowledge. London: Routledge; 1963. 6. Popper K. Objective knowledge: an evolutionary approach. Oxford: Clarendon Press; 1972. 7. Shahar E. A Popperian perspective of the term ‘evidence-based medicine’. J Eval Clin Pract 1997; 3:109-116. 8. Italian Ministry of Health, tabulated by the National Institute of Statistics, ISTAT, http:// www.istat.it 9. Horton R. Surgical research or comic opera? Questions, but few answers. Lancet 1996;347: 984-985. 10. McLeod RS, Wright JG, Solomon MJ, Hu X, Walters BC, Lossing Al. Randomized controlled trials in surgery: issues and problems. Surgery 1996;119:483-486. 11. Stocchi L, Nelson H, Sargent DJ, et al. Impact of surgical and pathologic variables in rectal cancer: a United States community and cooperative group report. J Clin Oncol 2001;19:3895-3902. 12. Hermanek P, Hermanek PJ. Role of the surgeon as a variable in the treatment of rectal cancer. Asian J Oral Maxillofac Surg Vol 15, No 1, 2003
Bignamini
Semin Surg Oncol 2000;19:329-335. 13. Domenighetti G, Grilli R, Liberati A. Promoting consumers’ demand for evidence-based medicine. Int J Technol Assess Health Care 1998;14:97-105. 14. Semmelweis IP. Die aethiologie, der begriff und die prophylaxis des kindbettfiebers, Wien 1861. [English translation: Semmelweis I, Semmelweis IF, Codell Carter K (Translator)]. The etiology, concept, and prophylaxis of childbed fever. Wisconsin: Univ of Wisconsin Press: 1983. 15. SIGN 50: A guideline developers’ handbook. http://www.sign.ac.uk/guidelines/fulltext/50/ index.html
Asian J Oral Maxillofac Surg Vol 15, No 1, 2003
16. Feinstein AR, Horwitz RI. Problems in the “evidence” of “evidence-based medicine”. Am J Med 1997;103:529-435. 17. Bignamini AA. Evidence-based medicine and clinical practice guidelines: solution or part of the problem? [in Italian]. Medicina e Morale 2001; 2:225-249. 18. Mulrow CD, Cook DJ, Davidoff FD. Systematic reviews: critical links in the great chain of evidence. Ann Intern Med 1997;126:389-391. 19. Pellegrino ED. A philosophical basis of medical practice. New York: Oxford University Press; 1984.
13