- Email: [email protected]
Evidence-based medicine: how do we find the evidence?
August 1998, Vol. 5, No. 3
TheJournal of the American Association of Gynecologic Laparoscopists
Evidence-Based Medicine: How Do We Find the Evidence? Kunle O. Odunsi, M.D., MRCOG, Inez E. Cooke, MRCOG, and David L. Olive, M.D.
Abstract Applying evidence-based medicine in clinical practice is an important advance in the attempt to improve clinical care. However, if appropriate evidence is unobtainable, it obviously cannot be used. Several resources are available to find quality studies in our field, detailing successes as well as shortcomings with respect to many aspects of patient care. Databases such as MEDLINE and EMBASE are helpful, but they require specific strategies to maximize the efficiency and comprehensiveness of literature searches. In addition, they are incomplete and thus do not allow access to many good studies. In an attempt to remedy this dilemma, the Cochrane Library was created. This database contains systematic reviews on a large number of subjects as well as a controlled trial registry. It approaches the criteria of an ideal database, and has proved to be an invaluable tool in the practice of evidence-based medicine. (l Am Assoc Gynecol Laparosc 5(3):313-318, 1998)
Evidence-based clinical practice was defined as "the conscientious, explicit, and judicious use of the current best evidence in making decisions about the care of individual patients; the integration of individual clinical expertise with the best available external clinical evidence from systematic research. ''~ This implies that practicing evidence-based obstetrics and gynecology requires clinical experience, judgment, and surgical skills. After history and physical examination are performed, many decisions usually have to be made: which tests, both clinical and diagnostic, will most efficiently give the diagnosis; which pointers are best prognostic factors for disease progression; what
is the best short-term treatment, or if long-term treatment is necessary, which would be best? Answers to these questions are often uncertain, and tracking down relevant information provides the opportunity to fill a gap in knowledge and practice evidence-based medicine. It also is likely to lead to a more comprehensive and holistic approach to patient management. Furthermore, by identifying lacunae in current knowledge, evidence-based practice can help establish priorities for a research agenda. A practitioner of evidence-based medicine has to retrieve information efficiently. Over 2 million articles are published annually in biomedical literature in
From the Department of Obstetrics and Gynecology (Dr. Odunsi) and Section of Reproductive Endocrinology (Dr. Olive), Yale University School of Medicine, New Haven, Connecticut; and Department of Obstetrics and Gynaecology, John Radcliffe Hospital, University of Oxford, Oxford, United Kingdom (Dr. Cooke). Address reprint requests to David L. Olive, M.D., Section of Reproductive Endocrinology, Yale School of Medicine, RO. Box 208063, New Haven, CT 06520-8063; fax 203 785 7134. Accepted for publication May 4, 1998.
313
Evidence-Based Medicine: Finding the Evidence Odunsi et al
TABLE 2. U.S. Preventive Services Task Force Classification of Levels of Evidence
over 20,000 journals. One year's worth of papers was estimated to create a 500-m tower. 2 It was suggested that general physicians have to read 19 articles/day 365 days of the year to keep abreast of their specialty? To benefit patients, the search for information must be comprehensive to obtain the best data that are available. To be practical, the process must be speedy and efficient.
Level I
I1-1 11-2
Levels of Evidence Levels of evidence exist for all types of clinical data (Tables 1 and 2).4' 5 In searching for evidence on a therapeutic agent to treat menorrhagia, for example, the best evidence will be found in a well-conducted systematic review or, failing that, a randomized controlled trial (RCT) of sufficient power. If neither of these is available, a decision may be made based on less reliable information, but with acknowledgment that it is founded on less dependable results or less strong levels of evidence. A systematic review is not merely a literature review; it gives details of methods of trial collection, reasons for inclusion and exclusion of trials, and types of statistical analyses. It does not necessarily provide results similar to a meta-analysis, especially if data are not appropriate to be presented in numerical form. However, a meta-analysis of RCTs means that data from small trials, which individually are too small to have power to be useful, can be combined, giving increased power and precision to the results. This approach allows for insignificant and irrelevant studies to be
11-3
III
II III
IV
V
Evidence obtained from at least one properly conducted randomized, controlled trial Evidence obtained from well-designed controlled trials without randomization Evidence from well-designed cohort or case control analytic studies, preferably from more than one center or research group Evidence obtained from several time series with or without intervention, or dramatic results in uncontrolled experiments (e.g., resultsof introduction of penicillin treatment in the 1940s) Opinions of respected authorities based on clinical experience, descriptive studies and case reports, or reports of expert committees.
From reference 5.
separated from salient and critical ones, so that findings may be generalized to different populations. In addition, the reviewer is able to examine studies for consistency of results, and assess what makes studies with inconsistent results different. Available Databases Over 60 biomedical databases are available (Table 3). Most physicians are familiar only with MEDLINE. Unfortunately, MEDLINE covers less than one-fourth of biomedical journals (3700). Even with a sensitive search for RCTs, only 50% of the those known to be on MEDLINE will actually be picked up.6 Furthermore, the terms "random allocation" and "randomized, controlled trial" were first used as descriptor terms in MEDLINE in 1975 and 1990, respectively, and it was not until 1991 that RCT was indexed as a publication type. There is little reason to expect better return from other general databases, although they have yet to be scrutinized in the same fashion.
TABLE 1. Classification of Levels of Evidence
Level
Quality of Evidence
Type of Evidence Strong evidence from at least one systematic review of several well-designed randomized, controlled trials Strong evidence from at least one properly designed randomized, controlled trial of appropriate size Evidence from well-designed trials without randomization, single-group pre-post, cohort, time series, or matched case controlled studies Evidence from well-designed nonexperimental studies from more than one center or research group Opinions of respected authorities based on clinical evidence, descriptive studies, or reports of expert committees
The Ideal Database The ideal and most useful database would provide systematic reviews of data related to a particular patient and specific intervention being considered. It would therefore encompass all specialties, be easily accessible, and be presented in a comprehensible and clinically relevant format. Presently, the Cochrane Librm2r produced by the Cochrane Collaboration, meets these criteria. The library is the most comprehensive source
From reference 4 with permission.
314
August 1998, Vol. 5, No. 3
The Journal of the American Association of Gynecologic Laparoscopists
TABLE 3. Available Databases and Areas of Interest
Database
Years Covered
Areas of Interest
Cochrane Library CDSR DARE CCTR ACPICOD MEDLINE
No limits
All specialties, includes previous pregnancy and childbirth reviews.
1990 on 1966 on
EMBASE
1974 on
Biological Abstracts CINAHL Nursing and Midwifery Index CANCERLIT CAB HEALTH
1990 on 1982 on 1994 on
General medicine. Electronic version of Index Medicus and International Nursing Index; strong on North American literature. Electronic version of Excerpta Medica; strong on European studies and pharmacology literature. Clinical and experimental medicine with biologic science. Nursing journals, chapters, books, theses. Nursing and midwifery in the U.K.
Science Citation Index Scientific and Technical Proceedings Current Contents
1981 on 1982 on
AIDSline PsycLit ASSIA Sociofile Bioethicsline PAlS Health Periodicals
1980 1974 1987 1974 1973 1970 1987
Health plan
1986-1987
1963 on 1973 on
Present on on on on on on on
Epidemiology, pathogenesis, immunology, and treatment of cancer. Environmental health, tropical medicine, and nutrition including obstetrics and gynecology. General science and author citation. Details of papers presented at 4000 conferences/yr, including medical conferences. Contents pages of hundreds of journals in life sciences and clinical medicine, updated weekly. AIDS-related topics. Psychology and related disciplines. Social sciences. Sociology. Ethical issues. Public and social policy, American bias. Consumerism and public health, with summaries specifically for the layperson. Administrative, financial, legal, and ethical aspects of health care.
of material for all those interested in evidence-based health care. The Cochrane Collaboration produces high-quality systematic reviews of trials that have been performed in every aspect of health care. It is an international network of individuals and institutions that addresses the need for reliable summary of information and assists in finding data required to make health care choices. The objective is to prepare, maintain, and disseminate systematic reviews of randomized trials. Since its inception in 1993, centers have been established in Australia, Canada, Italy, Scandinavia, The Netherlands, France, and the United States. The collaboration comprises review groups that are responsible for preparing and maintaining systematic reviews, hand-searching journals of interest, creating a specialized register, and providing support
to reviewers. Each group represents an area of interest and is coordinated by an editorial group. Currently, five collaborative review groups cover topics of interest to obstetricians and gynecologists: pregnancy and childbirth, subfertility, menstrual disorders, incontinence, and gynecologic cancer. The Cochrane Library contains the following: 1. Cochrane database of systematic reviews (CDSR), an electronic journal, updated quarterly, of systematic reviews using explicitly defined methods. 2. Database of abstract of effectiveness (DARE), assembled by the United Kingdom National Health Service Center for Reviews and Dissemination at the University of York, a bibliography of systematic reviews that are of good quality and have been published, but are not arranged in the format of a Cochrane review.
315
Evidence-Based Medicine: Finding the Evidence Odunsi et al
3.
Cochrane controlled trials register (CCTR), which contains 94,000 references to controlled clinical trials that were identified from MEDLINE and specialized registries, and collated by Cochrane review groups. 4. Cochrane review methodology database, a bibliography of books, journal issues, and articles on methodologic aspects of systematic reviews. Cochrane reviews are based on the analysis of RCTs, where available. The review undergoes seven stages: statement of objectives of the review; outline of eligibility criteria for trials; search for studies meeting eligibility criteria; assessment of characteristics and methodology of identified studies; decision on inclusion or exclusion of identified studies; data extraction; analysis of results, by meta-analysis if appropriate; and preparation of a structured report. Final results are disseminated in electronic form through CDSR and by publication in medical journals and bulletins. Of all specialties, obstetrics and gynecology has one of the richest resources in the Cochrane Library in pregnancy and childbirth, subfertility, and menstrual disorders groups.
Searching the Literature Evidence-based practitioners should be proficient in searching for both primary articles and systematic reviews. The process of performing a search is as follows: 1. Convert the clinical problem into a focused question. 2. Collate specific words for the search using correct Medical Subject Heading (MESH) terms, text words, and synonyms of a text word. 3. Use the words "and/or" to narrow or expand the search. 4. Reduce the search to trials with the best quality and most relevance for the topic in question. The CDSR and DARE databases are the first stop for information and, increasingly, it may be unnecessary to go further. Searches may be carried out on CDSR either by text words or by listing titles that are registered under each review group. In addition, it is possible to search by phrase and proximity, field, and MESH. The term or phrase is simply typed into the text field and the number of its occurrences in each returned document is shown as the number of hits. This is the number of paragraphs that have one or more matches in them. The search options can also be modified by
three special characters or words: & orAND; + or OR; and * the wildcard character. When information has not been reviewed in this format, the CCTR, which includes over 94,000 clinical trials, should be the next stop. Furthermore, the Cochrane Collaboration is developing software that will transfer the searcher through a series of databases starting with the Cochrane Library and proceeding to the most appropriate databases for the question (ACPJCOD, MEDLINE, EMBASE, etc.). The first step in an efficient search is to convert a clinical problem into a focused question. Ultimately, the question for any particular clinical situation is, what critical interventions are likely to improve clinical and economic outcomes (e.g., diagnostic strategies, therapy, prognosis, harm)? Using methodologic filters during the search can enhance efficient identification of high-quality studies. Retrieval properties of individual search terms 7 and combinations of terms 8 are now available. Methodologic filters for MEDLINE 8 have undergone rigorous testing for sensitivity (Table 4). Strategies developed by Cairn's Library in Oxford are published elsewhere.4 Compared with hand-searching the literature (reference standard), the four-part strategy that yields greatest sensitivity and specificity for best quality information is the RCT (publication type), drug therapy (subject heading), therapeutic use (subject heading), and all random (text word). 8 Search for a systematic review on MEDLINE or EMBASE should include meta-analysis in title (ti), abstract (ab), and MESH; review-academic in patient; review-tutorial in patient; and systematic near (review or overview). The search strategy for economic analysis and evaluation should include statistics and numeric data in MESH; cost-effective in ti, ab, or MESH; economic or cost and evaluation or benefit or effective in ti, ab, or MESH; explode costs and cost analysis; explode models-economic; and economic in MESH. For clinical decision analysis, the search on MEDLINE or EMBASE should include decision support techniques; decision making computer assisted; artificial intelligence; [(decision) and (analysis or tree or model or strategy or support or utility)]; uncertainty in ti, ab, or MESH; probability in ti, ab, or MESH; Bayes theorem; and prediction and forecasting. Clinical guidelines may also be searched in ti, ab, or MeSH key words using guidelines, consensus, protocol, standard, legislation, and recommendation.
316
August 1998, Vol. 5, No. 3
The journal of the American Association of Gynecologic Laparoscopists
TABLE 4. Search Terms for Best Studies of Treatment, Diagnosis, Prognosis, or Cause and Harm for MEDLINE
Clinical Topic Treatment
Strategy Best one term Maximum sensitivity
Prognosis
Maximum specificity Best one term Maximum sensitivity
Etiology/cause
Maximum specificity Best one term Maximum sensitivity
Diagnosis
Maximum specificity Best one term Maximum sensitivity
MEDLINE Clinical trial in pt Randomized controlled trial in pt, drug therapy in MESH, therapeutic use in MESH, random in ti, ab, MeSH (Double and blind) in ti, ab, MeSH or placebo in ti, ab, MeSH Exp cohort-studies Incidence in MESH, exp mortality, follow-up studies, mortality in MESH, prognosis in ti, ab, MESH, predict in ti, ab, MESH, course in ti, ab, MeSH Prognosis in MeSH or survival analysis Risk in ti, ab, MeSH Exp cohort studies, exp risk, (odds and ratio) in ti, ab, MESH, (relative and risk) in ti, ab, MESH, (case and control) in ti, ab, MeSH Case control studies, cohort studies Sensitivity in ti, ab, MeSH Sensitivity-and-specificity
Critical Appraisal and Synthesis of Evidence
review? How precise are they? Can they be applied to my patients? Were all clinically relevant outcomes considered? Are likely treatment benefits worth potential harm and cost? Once the process is completed, it is important to keep a record of the search strategy as well as clinically relevant results for future use. These summaries are called critically appraised topics (CATs). A CAT maker for therapy data, designed as an interactive piece of software, is available on the Center for Evidence-based Medicine's World Web page to help create these resources. In this way, the file of evidence grows as a nucleus of information for one's practice. After summarizing the evidence, the practitioner should consider anticipated benefits, harms, and costs in light of local practice and administrative constraints. In the United States, the American College of Obstetricians offers fellows ACOGNET, a free communications network to its own publications on a 24-hour basis. In addition, its clinical practice guidelines provide a formal synthesis of evidence, derived from the best available evidence of efficacy and consideration of costs, with recommendations explicitly linked to the evidence. These guidelines should also help the practitioner to improve the quality of health care, decrease cost, and diminish professional liability.
Critical appraisal of primary and secondary studies is necessary to determine their validity and usefulness. To assess the validity of a primary study such as RCT, it is important to be confident that the treatment effect, reported in both size and direction of effect, is an unbiased estimate of true effect and is not influenced to lead to a false conclusion. 9 The guide for assessing articles on trials includes the following questions9: Was the assignment of women to treatments randomized? Were all women who entered the trial properly accounted for at its conclusion? Was followup complete? Were women (and babies) analyzed in the groups to which they were randomized? Were groups similar at start of the trial? Were participants, health care workers, and study personnel blinded to treatment? Aside from experimental intervention, were groups treated equally? How large was the treatment effect? How precise was the estimate of treatment effect? Can results be applied to women in my clinical practice? Were all clinically important outcomes considered? Are likely treatment benefits worth potential harm and cost? Guidelines for assessing validity of review articles address the following issuesl~ Did the overview address a focused clinical question? Were criteria used to select articles for inclusion appropriate? Is it likely that important relevant studies were missed? Was the validity of included articles appraised? Were assessments of studies reproducible? Were results similar from study to study? What are overall results of the
Summary Two challenges for medicine at the close of the twentieth century are rationing of health resources
317
Evidence-Based Medicine: Finding the Evidence Odunsi et al
3. Davidoff F, Haynes RB, Sackett DL, et al: Evidencebased medicine: A new journal to help doctors identify the information they need. BMJ 310:1085-1086, 1995
and rapid introduction of new technologies and treatments. Increasingly, this is leading to attention to quality and outcomes of medical care. To be evidence-based practitioners, physicians must be able to search for the best evidence using efficient and comprehensive techniques. This requires construction of a specific question based on the clinical problem, design of a practical search strategy, search of appropriate database(s), and scan of retrieved abstracts for the most useful papers for appraisal. Only the Cochrane Library could be considered the ideal database and it should be the first stop for information. Other databases are also useful for retrieving information not yet included in the C o c h r a n e L i b r a r y and for h i g h l y specialized subjects. T h e Cochrane database is available on the internet and can be ordered on line by e-mail at [email protected]; by telephone at 16 299 06 6633; or by mail at EO. Box 67, St. Leonards, N S W 2065, Australia.
4. Cooke IE: Finding the evidence. In Balliere's Clinical Obstetrics and Gynecology, vol. 10. Edited by IE Cooke, DL Sackett. London, Balliere-Tindall, 1996 5. Force UPST: Guide to Clinical Preventive Services. Baltimore, Williams & Wilkins, 1995 6. Dickersin K, Scherer R, Lefebvre C: Identifying relevant studies for systematic reviews. In Systematic Reviews. Edited by I Chalmers, DG Altman. London, British Medical Journal Publishing Group, 1995 7. Wilczynski NL, Walker CJ, McKibbon KA, et al: Assessment of methodologic quality search filters in MEDLINE. Proc Annu Symp Comput Appl Med Care 17:601-605, 1994 8. Haynes RB, Wilczynski N, McKibbon A, et al: Developing optimal search strategies for detecting clinically sound studies in MEDLINE. J Am Med Informatics Assoc 1:447-458, 1994
References
1. Sackett DL, Rosenburg WMC, Gray JAM, et al: Evidence based medicine: What it is and what it isn't. BMJ 312:71-72, 1996
9. Guyatt GH, Sackett DL, Cook DJ: Users guide to medical literature. II. How to use an article about therapy or prevention. A. Are the results of the study valid? JAMA 270:2598-2601, 1993
2. Murlow CD: Rationale for systematic reviews. In Systematic Reviews. Edited by I Chalmers, DG Altman. London, British Medical Journal Publishing Group, 1995
10. Oxman AD, Cook DJ, Gutatt GH: User's guide to the medical literature. II. How to use an overview. JAMA 272:1367-1371, 1994
318
TheJournal of the American Association of Gynecologic Laparoscopists
Evidence-Based Medicine: How Do We Find the Evidence? Kunle O. Odunsi, M.D., MRCOG, Inez E. Cooke, MRCOG, and David L. Olive, M.D.
Abstract Applying evidence-based medicine in clinical practice is an important advance in the attempt to improve clinical care. However, if appropriate evidence is unobtainable, it obviously cannot be used. Several resources are available to find quality studies in our field, detailing successes as well as shortcomings with respect to many aspects of patient care. Databases such as MEDLINE and EMBASE are helpful, but they require specific strategies to maximize the efficiency and comprehensiveness of literature searches. In addition, they are incomplete and thus do not allow access to many good studies. In an attempt to remedy this dilemma, the Cochrane Library was created. This database contains systematic reviews on a large number of subjects as well as a controlled trial registry. It approaches the criteria of an ideal database, and has proved to be an invaluable tool in the practice of evidence-based medicine. (l Am Assoc Gynecol Laparosc 5(3):313-318, 1998)
Evidence-based clinical practice was defined as "the conscientious, explicit, and judicious use of the current best evidence in making decisions about the care of individual patients; the integration of individual clinical expertise with the best available external clinical evidence from systematic research. ''~ This implies that practicing evidence-based obstetrics and gynecology requires clinical experience, judgment, and surgical skills. After history and physical examination are performed, many decisions usually have to be made: which tests, both clinical and diagnostic, will most efficiently give the diagnosis; which pointers are best prognostic factors for disease progression; what
is the best short-term treatment, or if long-term treatment is necessary, which would be best? Answers to these questions are often uncertain, and tracking down relevant information provides the opportunity to fill a gap in knowledge and practice evidence-based medicine. It also is likely to lead to a more comprehensive and holistic approach to patient management. Furthermore, by identifying lacunae in current knowledge, evidence-based practice can help establish priorities for a research agenda. A practitioner of evidence-based medicine has to retrieve information efficiently. Over 2 million articles are published annually in biomedical literature in
From the Department of Obstetrics and Gynecology (Dr. Odunsi) and Section of Reproductive Endocrinology (Dr. Olive), Yale University School of Medicine, New Haven, Connecticut; and Department of Obstetrics and Gynaecology, John Radcliffe Hospital, University of Oxford, Oxford, United Kingdom (Dr. Cooke). Address reprint requests to David L. Olive, M.D., Section of Reproductive Endocrinology, Yale School of Medicine, RO. Box 208063, New Haven, CT 06520-8063; fax 203 785 7134. Accepted for publication May 4, 1998.
313
Evidence-Based Medicine: Finding the Evidence Odunsi et al
TABLE 2. U.S. Preventive Services Task Force Classification of Levels of Evidence
over 20,000 journals. One year's worth of papers was estimated to create a 500-m tower. 2 It was suggested that general physicians have to read 19 articles/day 365 days of the year to keep abreast of their specialty? To benefit patients, the search for information must be comprehensive to obtain the best data that are available. To be practical, the process must be speedy and efficient.
Level I
I1-1 11-2
Levels of Evidence Levels of evidence exist for all types of clinical data (Tables 1 and 2).4' 5 In searching for evidence on a therapeutic agent to treat menorrhagia, for example, the best evidence will be found in a well-conducted systematic review or, failing that, a randomized controlled trial (RCT) of sufficient power. If neither of these is available, a decision may be made based on less reliable information, but with acknowledgment that it is founded on less dependable results or less strong levels of evidence. A systematic review is not merely a literature review; it gives details of methods of trial collection, reasons for inclusion and exclusion of trials, and types of statistical analyses. It does not necessarily provide results similar to a meta-analysis, especially if data are not appropriate to be presented in numerical form. However, a meta-analysis of RCTs means that data from small trials, which individually are too small to have power to be useful, can be combined, giving increased power and precision to the results. This approach allows for insignificant and irrelevant studies to be
11-3
III
II III
IV
V
Evidence obtained from at least one properly conducted randomized, controlled trial Evidence obtained from well-designed controlled trials without randomization Evidence from well-designed cohort or case control analytic studies, preferably from more than one center or research group Evidence obtained from several time series with or without intervention, or dramatic results in uncontrolled experiments (e.g., resultsof introduction of penicillin treatment in the 1940s) Opinions of respected authorities based on clinical experience, descriptive studies and case reports, or reports of expert committees.
From reference 5.
separated from salient and critical ones, so that findings may be generalized to different populations. In addition, the reviewer is able to examine studies for consistency of results, and assess what makes studies with inconsistent results different. Available Databases Over 60 biomedical databases are available (Table 3). Most physicians are familiar only with MEDLINE. Unfortunately, MEDLINE covers less than one-fourth of biomedical journals (3700). Even with a sensitive search for RCTs, only 50% of the those known to be on MEDLINE will actually be picked up.6 Furthermore, the terms "random allocation" and "randomized, controlled trial" were first used as descriptor terms in MEDLINE in 1975 and 1990, respectively, and it was not until 1991 that RCT was indexed as a publication type. There is little reason to expect better return from other general databases, although they have yet to be scrutinized in the same fashion.
TABLE 1. Classification of Levels of Evidence
Level
Quality of Evidence
Type of Evidence Strong evidence from at least one systematic review of several well-designed randomized, controlled trials Strong evidence from at least one properly designed randomized, controlled trial of appropriate size Evidence from well-designed trials without randomization, single-group pre-post, cohort, time series, or matched case controlled studies Evidence from well-designed nonexperimental studies from more than one center or research group Opinions of respected authorities based on clinical evidence, descriptive studies, or reports of expert committees
The Ideal Database The ideal and most useful database would provide systematic reviews of data related to a particular patient and specific intervention being considered. It would therefore encompass all specialties, be easily accessible, and be presented in a comprehensible and clinically relevant format. Presently, the Cochrane Librm2r produced by the Cochrane Collaboration, meets these criteria. The library is the most comprehensive source
From reference 4 with permission.
314
August 1998, Vol. 5, No. 3
The Journal of the American Association of Gynecologic Laparoscopists
TABLE 3. Available Databases and Areas of Interest
Database
Years Covered
Areas of Interest
Cochrane Library CDSR DARE CCTR ACPICOD MEDLINE
No limits
All specialties, includes previous pregnancy and childbirth reviews.
1990 on 1966 on
EMBASE
1974 on
Biological Abstracts CINAHL Nursing and Midwifery Index CANCERLIT CAB HEALTH
1990 on 1982 on 1994 on
General medicine. Electronic version of Index Medicus and International Nursing Index; strong on North American literature. Electronic version of Excerpta Medica; strong on European studies and pharmacology literature. Clinical and experimental medicine with biologic science. Nursing journals, chapters, books, theses. Nursing and midwifery in the U.K.
Science Citation Index Scientific and Technical Proceedings Current Contents
1981 on 1982 on
AIDSline PsycLit ASSIA Sociofile Bioethicsline PAlS Health Periodicals
1980 1974 1987 1974 1973 1970 1987
Health plan
1986-1987
1963 on 1973 on
Present on on on on on on on
Epidemiology, pathogenesis, immunology, and treatment of cancer. Environmental health, tropical medicine, and nutrition including obstetrics and gynecology. General science and author citation. Details of papers presented at 4000 conferences/yr, including medical conferences. Contents pages of hundreds of journals in life sciences and clinical medicine, updated weekly. AIDS-related topics. Psychology and related disciplines. Social sciences. Sociology. Ethical issues. Public and social policy, American bias. Consumerism and public health, with summaries specifically for the layperson. Administrative, financial, legal, and ethical aspects of health care.
of material for all those interested in evidence-based health care. The Cochrane Collaboration produces high-quality systematic reviews of trials that have been performed in every aspect of health care. It is an international network of individuals and institutions that addresses the need for reliable summary of information and assists in finding data required to make health care choices. The objective is to prepare, maintain, and disseminate systematic reviews of randomized trials. Since its inception in 1993, centers have been established in Australia, Canada, Italy, Scandinavia, The Netherlands, France, and the United States. The collaboration comprises review groups that are responsible for preparing and maintaining systematic reviews, hand-searching journals of interest, creating a specialized register, and providing support
to reviewers. Each group represents an area of interest and is coordinated by an editorial group. Currently, five collaborative review groups cover topics of interest to obstetricians and gynecologists: pregnancy and childbirth, subfertility, menstrual disorders, incontinence, and gynecologic cancer. The Cochrane Library contains the following: 1. Cochrane database of systematic reviews (CDSR), an electronic journal, updated quarterly, of systematic reviews using explicitly defined methods. 2. Database of abstract of effectiveness (DARE), assembled by the United Kingdom National Health Service Center for Reviews and Dissemination at the University of York, a bibliography of systematic reviews that are of good quality and have been published, but are not arranged in the format of a Cochrane review.
315
Evidence-Based Medicine: Finding the Evidence Odunsi et al
3.
Cochrane controlled trials register (CCTR), which contains 94,000 references to controlled clinical trials that were identified from MEDLINE and specialized registries, and collated by Cochrane review groups. 4. Cochrane review methodology database, a bibliography of books, journal issues, and articles on methodologic aspects of systematic reviews. Cochrane reviews are based on the analysis of RCTs, where available. The review undergoes seven stages: statement of objectives of the review; outline of eligibility criteria for trials; search for studies meeting eligibility criteria; assessment of characteristics and methodology of identified studies; decision on inclusion or exclusion of identified studies; data extraction; analysis of results, by meta-analysis if appropriate; and preparation of a structured report. Final results are disseminated in electronic form through CDSR and by publication in medical journals and bulletins. Of all specialties, obstetrics and gynecology has one of the richest resources in the Cochrane Library in pregnancy and childbirth, subfertility, and menstrual disorders groups.
Searching the Literature Evidence-based practitioners should be proficient in searching for both primary articles and systematic reviews. The process of performing a search is as follows: 1. Convert the clinical problem into a focused question. 2. Collate specific words for the search using correct Medical Subject Heading (MESH) terms, text words, and synonyms of a text word. 3. Use the words "and/or" to narrow or expand the search. 4. Reduce the search to trials with the best quality and most relevance for the topic in question. The CDSR and DARE databases are the first stop for information and, increasingly, it may be unnecessary to go further. Searches may be carried out on CDSR either by text words or by listing titles that are registered under each review group. In addition, it is possible to search by phrase and proximity, field, and MESH. The term or phrase is simply typed into the text field and the number of its occurrences in each returned document is shown as the number of hits. This is the number of paragraphs that have one or more matches in them. The search options can also be modified by
three special characters or words: & orAND; + or OR; and * the wildcard character. When information has not been reviewed in this format, the CCTR, which includes over 94,000 clinical trials, should be the next stop. Furthermore, the Cochrane Collaboration is developing software that will transfer the searcher through a series of databases starting with the Cochrane Library and proceeding to the most appropriate databases for the question (ACPJCOD, MEDLINE, EMBASE, etc.). The first step in an efficient search is to convert a clinical problem into a focused question. Ultimately, the question for any particular clinical situation is, what critical interventions are likely to improve clinical and economic outcomes (e.g., diagnostic strategies, therapy, prognosis, harm)? Using methodologic filters during the search can enhance efficient identification of high-quality studies. Retrieval properties of individual search terms 7 and combinations of terms 8 are now available. Methodologic filters for MEDLINE 8 have undergone rigorous testing for sensitivity (Table 4). Strategies developed by Cairn's Library in Oxford are published elsewhere.4 Compared with hand-searching the literature (reference standard), the four-part strategy that yields greatest sensitivity and specificity for best quality information is the RCT (publication type), drug therapy (subject heading), therapeutic use (subject heading), and all random (text word). 8 Search for a systematic review on MEDLINE or EMBASE should include meta-analysis in title (ti), abstract (ab), and MESH; review-academic in patient; review-tutorial in patient; and systematic near (review or overview). The search strategy for economic analysis and evaluation should include statistics and numeric data in MESH; cost-effective in ti, ab, or MESH; economic or cost and evaluation or benefit or effective in ti, ab, or MESH; explode costs and cost analysis; explode models-economic; and economic in MESH. For clinical decision analysis, the search on MEDLINE or EMBASE should include decision support techniques; decision making computer assisted; artificial intelligence; [(decision) and (analysis or tree or model or strategy or support or utility)]; uncertainty in ti, ab, or MESH; probability in ti, ab, or MESH; Bayes theorem; and prediction and forecasting. Clinical guidelines may also be searched in ti, ab, or MeSH key words using guidelines, consensus, protocol, standard, legislation, and recommendation.
316
August 1998, Vol. 5, No. 3
The journal of the American Association of Gynecologic Laparoscopists
TABLE 4. Search Terms for Best Studies of Treatment, Diagnosis, Prognosis, or Cause and Harm for MEDLINE
Clinical Topic Treatment
Strategy Best one term Maximum sensitivity
Prognosis
Maximum specificity Best one term Maximum sensitivity
Etiology/cause
Maximum specificity Best one term Maximum sensitivity
Diagnosis
Maximum specificity Best one term Maximum sensitivity
MEDLINE Clinical trial in pt Randomized controlled trial in pt, drug therapy in MESH, therapeutic use in MESH, random in ti, ab, MeSH (Double and blind) in ti, ab, MeSH or placebo in ti, ab, MeSH Exp cohort-studies Incidence in MESH, exp mortality, follow-up studies, mortality in MESH, prognosis in ti, ab, MESH, predict in ti, ab, MESH, course in ti, ab, MeSH Prognosis in MeSH or survival analysis Risk in ti, ab, MeSH Exp cohort studies, exp risk, (odds and ratio) in ti, ab, MESH, (relative and risk) in ti, ab, MESH, (case and control) in ti, ab, MeSH Case control studies, cohort studies Sensitivity in ti, ab, MeSH Sensitivity-and-specificity
Critical Appraisal and Synthesis of Evidence
review? How precise are they? Can they be applied to my patients? Were all clinically relevant outcomes considered? Are likely treatment benefits worth potential harm and cost? Once the process is completed, it is important to keep a record of the search strategy as well as clinically relevant results for future use. These summaries are called critically appraised topics (CATs). A CAT maker for therapy data, designed as an interactive piece of software, is available on the Center for Evidence-based Medicine's World Web page to help create these resources. In this way, the file of evidence grows as a nucleus of information for one's practice. After summarizing the evidence, the practitioner should consider anticipated benefits, harms, and costs in light of local practice and administrative constraints. In the United States, the American College of Obstetricians offers fellows ACOGNET, a free communications network to its own publications on a 24-hour basis. In addition, its clinical practice guidelines provide a formal synthesis of evidence, derived from the best available evidence of efficacy and consideration of costs, with recommendations explicitly linked to the evidence. These guidelines should also help the practitioner to improve the quality of health care, decrease cost, and diminish professional liability.
Critical appraisal of primary and secondary studies is necessary to determine their validity and usefulness. To assess the validity of a primary study such as RCT, it is important to be confident that the treatment effect, reported in both size and direction of effect, is an unbiased estimate of true effect and is not influenced to lead to a false conclusion. 9 The guide for assessing articles on trials includes the following questions9: Was the assignment of women to treatments randomized? Were all women who entered the trial properly accounted for at its conclusion? Was followup complete? Were women (and babies) analyzed in the groups to which they were randomized? Were groups similar at start of the trial? Were participants, health care workers, and study personnel blinded to treatment? Aside from experimental intervention, were groups treated equally? How large was the treatment effect? How precise was the estimate of treatment effect? Can results be applied to women in my clinical practice? Were all clinically important outcomes considered? Are likely treatment benefits worth potential harm and cost? Guidelines for assessing validity of review articles address the following issuesl~ Did the overview address a focused clinical question? Were criteria used to select articles for inclusion appropriate? Is it likely that important relevant studies were missed? Was the validity of included articles appraised? Were assessments of studies reproducible? Were results similar from study to study? What are overall results of the
Summary Two challenges for medicine at the close of the twentieth century are rationing of health resources
317
Evidence-Based Medicine: Finding the Evidence Odunsi et al
3. Davidoff F, Haynes RB, Sackett DL, et al: Evidencebased medicine: A new journal to help doctors identify the information they need. BMJ 310:1085-1086, 1995
and rapid introduction of new technologies and treatments. Increasingly, this is leading to attention to quality and outcomes of medical care. To be evidence-based practitioners, physicians must be able to search for the best evidence using efficient and comprehensive techniques. This requires construction of a specific question based on the clinical problem, design of a practical search strategy, search of appropriate database(s), and scan of retrieved abstracts for the most useful papers for appraisal. Only the Cochrane Library could be considered the ideal database and it should be the first stop for information. Other databases are also useful for retrieving information not yet included in the C o c h r a n e L i b r a r y and for h i g h l y specialized subjects. T h e Cochrane database is available on the internet and can be ordered on line by e-mail at [email protected]; by telephone at 16 299 06 6633; or by mail at EO. Box 67, St. Leonards, N S W 2065, Australia.
4. Cooke IE: Finding the evidence. In Balliere's Clinical Obstetrics and Gynecology, vol. 10. Edited by IE Cooke, DL Sackett. London, Balliere-Tindall, 1996 5. Force UPST: Guide to Clinical Preventive Services. Baltimore, Williams & Wilkins, 1995 6. Dickersin K, Scherer R, Lefebvre C: Identifying relevant studies for systematic reviews. In Systematic Reviews. Edited by I Chalmers, DG Altman. London, British Medical Journal Publishing Group, 1995 7. Wilczynski NL, Walker CJ, McKibbon KA, et al: Assessment of methodologic quality search filters in MEDLINE. Proc Annu Symp Comput Appl Med Care 17:601-605, 1994 8. Haynes RB, Wilczynski N, McKibbon A, et al: Developing optimal search strategies for detecting clinically sound studies in MEDLINE. J Am Med Informatics Assoc 1:447-458, 1994
References
1. Sackett DL, Rosenburg WMC, Gray JAM, et al: Evidence based medicine: What it is and what it isn't. BMJ 312:71-72, 1996
9. Guyatt GH, Sackett DL, Cook DJ: Users guide to medical literature. II. How to use an article about therapy or prevention. A. Are the results of the study valid? JAMA 270:2598-2601, 1993
2. Murlow CD: Rationale for systematic reviews. In Systematic Reviews. Edited by I Chalmers, DG Altman. London, British Medical Journal Publishing Group, 1995
10. Oxman AD, Cook DJ, Gutatt GH: User's guide to the medical literature. II. How to use an overview. JAMA 272:1367-1371, 1994
318