- Email: [email protected]
2 Finding the evidence
2 Finding the evidence I N E Z E. C O O K E
An evidence-based practitioner needs to retrieve information efficiently. This chapter describes the variety of resources available and suggests some methods to search for answers most effectively. Over two million articles are published annually in the biomedical literature in over 20 000 journals. Health-care providers are all inundated with unmanageable amounts of information. How then do we meet the challenge of keeping up to date with current best evidence when it is hidden somewhere in this paper mountain? It seems an awesome task made more difficult by the quantity, organization, indexing and variety of journals and resources. To benefit your patient, your searching technique must find the best evidence (be comprehensive) published on the clinical problem. To be practical (and not tire your enthusiasm) the process must be speedy to retrieve the information efficiently. Two million articles, one year's worth of papers, have been estimated to rise as a 500 m tower of paper (Mulrow, 1995). WHERE DO WE GO FOR INFORMATION AT PRESENT? Needing information, whether it is a general overview of a disease that we meet rarely or a specific clinical problem recurring frequently in our practice, is not new. We need this evidence, or information, daily, and our patients would benefit from it. It was estimated, when shadowing a general physician through a typical (half-day) session in clinic, that four clinical decisions would have been altered had the particular information about the patients' conditions been available and employed (Cavell et al, 1985). Where have you usually gone looking for new information? Clinicians say they go to their textbooks and journals for this knowledge, but when these same clinicians were shadowed, the reality was quite different. They went most frequently not to their books, but to their colleagues and in particular their senior colleagues at work. With all their endeavours, however, they report that only 30% of the time are their information needs met by their normal methods. The problems are understandable and well known to us all: no time, textbooks are out of date and the sheer volume of literature is off-putting, to say the least. We are often left frustrated by our Bailli~re's Clinical Obstetrics and GynaecologyVol. 10, No. 4, December 1996 ISBN 0-7020-2260-8 0950-3501/96/040551 + 17 $12.00/00
551 Copyright © 1996, by Bailli~re Tindall All rights of reproduction in any form reserved
552
i.E. COOKE
inability to unleash what gold we imagine is hidden somewhere within the indexing system of a database. Most clinicians have less than 30 minutes a week to read around their subject, irrespective of whether they are hospital-based clinicians or general practitioners (Sackett, in press). The message is very clear: in a busy clinical week there is little time available to further our education, so whatever we do with that time must be focused. The excuse 'Our textbooks are out of date' is not just an excuse: the claims are frequently correct (see Table 1 below; Antman et al, 1992). It is estimated that for general physicians just to keep abreast of their specialty, they would have to read 133 articles per week (Davidoff et al, 1995). How do you efficiently 'distil the elixir' of clinically useful information, pertinent to your patient, from this deluge? This chapter is aimed at guiding you towards some of the most fruitful databases, especially for obstetricians and gynaecologists, to help you on your way. There will be specific examples in the forthcoming chapters, so here we discuss the types of databases, helpful rules and strategies that will help your search sift that gold. LEVELS OF EVIDENCE What is the evidence we are looking for? The answer is, of course, the best evidence, but as you will find in the forthcoming chapters, there will, for different clinical questions, be different types of evidence that will be the 'best'. The most comprehensive evidence would be a systematic review already written for every conceivably useful topic. Fortunately for us, it is a dream that many people in the Cochrane Collaboration are working towards making a reality. The methodology of systematic reviews of interventions is well organized. At present, methods of combining the results for other clinical data such as diagnostic or prognostic studies, or indeed the accuracy of so doing, is still under debate (Irwig et al, 1995). For each type of clinical data, there are 'levels of evidence'. When we search for evidence on a therapeutic agent (antihypertensives in preeclampsia, antifibrinolytics for menorrhagia) we hope that among the Table 1. Classification of levels of evidence. Level I
II
Type of evidence Strong evidence from at least one systematic review of multiple, well-designed randomized controlled trials Strong evidence from at least one properly designed randomized controlled trial of appropriate size
III
Evidence from well-designed trials without randomization, single group pre-post, cohort, time series or matched case-controlled studies
IV
Evidence from well-designed non-experimental studies from more than one centre or research group
V
Opinions of respected authorities, based on clinical evidence, descriptive studies or reports of expert committees
FINDING THE EVIDENCE
553
studies, or 'hits', retrieved there will be a well-conducted systematic review or, failing that, a well-conducted randomized controlled trial (RCT) of sufficient power. What if neither of those are available? The fact that the gold standard evidence is not available does not mean that you should not seek out and assess the less reliable evidence. You still have to make a clinical decision, so you will have to use this less reliable information to make that decision but knowing that it is founded on much less dependable results, on less strong levels of evidence (Table 1). W H Y A SYSTEMATIC R E V I E W ? A systematic review gives details of the methods of trial collection, reasons for including and excluding trials and statistical methods of analysis. Inclusion of this information means that the readers can decide for themselves whether any important sources of information have been excluded or inappropriate information included. This is what makes a systematic review so different from a traditional review. You are not dependent on the bias that the author may unwittingly bring to the article. A failure to apply the same scientific principles to the design and conduct of a review as apply to the design and conduct of the primary studies has meant that, just like our textbooks, advice in traditional reviews has often failed to reflect the strong evidence that would have emerged had the reviewers applied scientific principles systematically to reviews of RCTs (Figure 1; Antman et al, 1992). A systematic review does not necessarily have results presented as a meta-analysis; the data may simply not be appropriate to present in numerical form. A meta-analysis of RCTs means that the data from small trials, which individually are of too small a sample size to have the power to have been useful, can be combined, giving both increased power and increased precision to the results. The systematic review allows large quantities of information to be 'melted down' and presented as something palatable. It separates the insignificant and irrelevant from the salient and critical studies for us and allows us to assess the 'generalizability' of the results to different populations. The direct comparisons also enable the reviewer to examine the studies for consistency of results and to assess what makes those trials, which appear to have inconsistent results, different. Our ability to find the answer to our particular clinical problem within a systematic review (for example, pain after repair of perineal trauma following childbirth with various suture materials or the pregnancy rate associated with clomiphene use in women with unexplained infertility) does not remove from us the onus of deciding whether or not the results are applicable to our own patient or population. When the systematic review has yet to be done (or is not yet possible), there are two answers as to how to move forward. The first is to produce the review yourself under the umbrella of the Cochrane Collaboration. This is not as daunting as it may seem if the review needs to be done and one accepts the wealth of assistance and experience that is made available
554
I.E. COOKE
-t-,
E ~
d
"6
Textbook/review recommendations for giving lidocaine
Odds ratio (log scale)
;0.5 =
1 =
•
=
|
2
=
i
~
|
|
=
i
|
|=i
'10 O~ r-
1960
-~
i== 0
09
n-
W
Z
1965
1970
2 304 5 647 6 850 8 1239 9 1451
2
do
17
4
4
1
7
1
1966
1
1970
8
2
1974
2
0
1975 11 1686
1
4
12 1986
4
2
4
8
1
5
6
2
3
5
3
4
2
1980
1
1
1978
1982
1985 14 8412 1
1986
3 4
15 8745
P=>0.05
1990
5
9
4
6
1
3
2
1
1990
m
!
Favours treatment
Favours control
Figure 1. Cumulative meta-analysis by year of publication of randomized controlled trials of prophylactic lignocaine for acute myocardial infarction and recommendations of clinical expert reviewers. Adapted from Antman et al (1992).
through this organization. However, it is obviously not a quick option. The second plan is the one that will give the best evidence available to help in 'the here and now', the help we need for the patient in next week's clinic. The forthcoming chapters will discuss the mechanisms of critical appraisal and assessing the applicability of the results to your patient or population for diagnostic tests, therapies and prognostic factors. One publication type has risen above all the others as a gold standard when assessing the effectiveness of a particular intervention, that of the RCT.
FINDING THE EVIDENCE
555
WHY A R A N D O M I Z E D C O N T R O L L E D T ~ A L ? The act of true randomization and concealment of allocation is the only known way to avoid selection biases that handicap observational studies (Peto et al, 1976). While the number of RCTs produced is estimated at between 250000 and 1 000000, those identified in obstetrics and gynaecology are relatively rare (the registry for Pregnancy and Childbirth at present contains 8000 trials). It was not until 1991 that the RCT was indexed as a publication type on MEDLINE, so it is only with the march of hand searchers through past issues of journals that the true wealth of RCTs 'out there' is coming to light. Such is their place in clinical research that there are some medical journals which have agreed, in future, to publish only RCTs of interventions. 'The RCT is a very beautiful technique, of wide applicability, but as with everything else there are snags' (Cochrane, 1989). Unfortunately, many of those RCTs produced in obstetrics and gynaecology are found to violate the simple rules of conduct (Schulz et al, 1994), resulting in trials that are subject to bias, with results that, in consequence, seem misleading (Grimes, 1991). Grimes and Schulz suggest that this poor record of reporting and conducting RCTs is simply due to naivety rather than negligence. In over 80% of reported RCTs, in four major obstetric and gynaecology journals, no reference was made to methodological articles. This was reflected by an inadequate randomization system or method of allocation concealment (Grimes and Schulz, 1996). They argue that if clinicians sought help and advice before expecting to embark on a RCT, just as we would not carry out an operation without adequate training, there would be less wasted effort. Because of poor science (McDonough, 1995), leading to poor clinical practice (Grimes, 1991), one critic argues that poor-quality research is unethical (Altman, 1994). It is important, however, to realize that the effects of some health care interventions are so obvious that they can be recognized confidently without conducting carefully controlled research. Stimulating a heart to re-establish its normal rhythm after it has stopped beating (defibrillation) provides a dramatic example of an easily recognized beneficial effect of health care. RCTs are the best evaluations o f health care for therapy, prevention, quality of life and economics, and may be the most appropriate study in evaluations of harm and aetiology. There are other occasions on which RCTs would be impossible or inappropriate. To assess the effects of harmful substances such as smoking and lung cancer, the oral contraceptive pill and breast cancer or thrombosis, one cannot randomize the patients. Other research strategies, cohort or case-controlled studies, will offer the best evaluation of harm, aetiology or prognosis in these instances. That said, the RCT has pride of place among the publication types used to test the efficacy of any intervention. THE DREAM DATABASE If these are the types of evidence we want, where can we go to find them and, most importantly, find them with relative ease7 The resource that
556
I.E. COOKE
would be most useful to a clinician would provide systematic reviews of data relevant to any particular patient and the specific intervention being considered. It would therefore encompass all specialties, be easily accessible and be presented in a comprehensible and clinically relevant format to help you decide the implications for your practice. Even if the only information that the obstetrician or gynaecologist obtains from this database is that there is no reliable information as to the worth of a particular intervention, that in itself would be very useful for both present patient care and possible future areas of research. The Cochrane Collaboration aims to produce high-quality systematic reviews of trials that have been performed in every aspect of health care. The evidence will primarily be from RCTs, but other evidence will be reviewed if the RCT has not been produced or would be inappropriate. To ensure that the reviews available to us, the public, are as up to date as the trials themselves, the reviews are updated as new relevant information comes to light and are disseminated in electronic, rather than paper, format. For those of us wary of the beast with a keyboard and visual display monitor, the essence of the software and presentation of data within the Cochrane Library is transparency for the user, whether patient, nurse, student or professor. The Cochrane Library contains: 1. The Cochrane Database of Systematic Reviews (CDSR). This is an electronic journal, updated quarterly, of systematic reviews using explicitly defined methods. 2. The Database of Abstract of Effectiveness (DARE). Assembled by the UK National Health Service Centre for Reviews and Dissemination at the University of York, this is a bibliography of systematic reviews that are of good quality, and have been published elsewhere, but have not been arranged in the format of a Cochrane review. At present, there are over 1100 abstracts in this database. 3. The Cochrane Controlled Trials Register (CCTR). This contains 94 000 references to controlled clinical trials, which have been identified from MEDLINE and the specialized registries and collated by Cochrane Review Groups. 4. The Cochrane Review Methodology Database (CRMD). This is a bibliography of books, journal issues and articles on methodological aspects of systematic reviews. This sounds like the dream database for clinicians, and it really will be a very important source, if not the source, of information for health-care givers in the future. The reality at present is that while very necessary groundwork in hand searching, electronic searching and preparing protocols is taking place, there are still only a small, but rapidly growing, number of completed reviews (120 completed reviews and 200 protocols). Of all specialties, however, obstetrics and gynaecology has one of the richest resources within the Cochrane Library within the Pregnancy and Childbirth, Subfertility and Menstrual Disorders Groups. You may carry out your search on the CDSR either by text words (subject heading for the pregnancy and childbirth reviews) or by listing the titles that are registered
557
FINDING THE EVIDENCE
Refine your search strategy with the information retriev~....ll~ 1
/
/
The clinical question Jm Develop a search strategy / •
~
Decide on the best resources
~
/
Cochrane Library
1
SeconV r,iouma,s
[
4
Seek help from your librarian & | / |
~
Has your search retrieved
3. relevant
V information?
Figure 2. General search strategy.
under each Cochrane Review Group. This latter will become less efficient as the number of reviews grows and may mean that you miss titles of interest that have been listed under different Review Groups. For example, if the topic choice is bone disease after the menopause, using text word would retrieve those reviews which have been composed under either the musculoskeletal or menstrual disorders or primary care field, to name only a few possibilities. So, to go back to 'finding the evidence': Figure 2 outlines the process of performing a search: 1. Convert your clinical problem into a focused question. 2. Collate the specific words you are going to use for the search. It takes a little practice to ensure that this part of the search is as all-inclusive as possible using the correct Medical Subject Heading (MESH) terms, text words and synonyms of a text word. As you proceed through the search, certain words within abstracts often jog your thought process. Population/patient
Intervention
Outcome
premature infant or premature neonate or premature birth or preterm infant or preterm neonate
corticostereid(s) or antenatal steroids or glucocorticoids or betamethasone or adrenal cortex hormone(s)
respiratory distress syndrome or RDS
and
and
Figure 3. The risks of prematurity are numerous and the benefit of antenatal steroids now well documented. This problem is used as an example to (a) demonstrate construction of a question: 'In preterm infants does the administration of antenatal corticosteroids reduce the incidence of respiratory distress syndrome?', (b) list some of the possible search terms you may consider, and (c) demonstrate how the use of 'or" for all the terms within a column expands the search, and the use of 'and" between the columns narrows the search.
558
3.
4.
I . E . COOKE
The words 'and/or' are used to narrow or expand your search. If you imagine two search terms represented by two overlapping circles, 'or' would include everything within both circles, whereas 'and' would only include data within the area of overlap between the circles. Figure 3 shows a practical example. At this stage, you want to reduce your search to those trials of the best quality and most relevance to your topic. Several useful methodological filters for different clinical topic areas have been produced to aid you at this stage. Those described in Table 2 have undergone
Table 2. Search terms for the best studies of treatment, diagnosis, prognosis or cause/harm for MEDLINE. Clinical topic Treatment
Strategy
MEDLINE (WINSPIRS)
Best one term
clinical-trial in pt randomized-controlled-trial in pt or drug therapy in MeSH or therapeutic use in MeSH or random in ti, ab, MeSH (double and blind) in ti, ab, MeSH or placebo in ti, ab, MeSH
Maximum sensitivity
Maximum specificity Prognosis
Best one term
Maximum sensitivity
Maximum specificity Aetiologyfcause
Best one term
Maximum sensitivity
Maximum specificity Diagnosis
Best one term
Maximum sensitivity
Maximum specificity
exp cohort-studies incidence in MeSH or exp mortality or follow-up-studies or mortality in MeSH or prognos in ti, ab, MeSH or predict in ti, ab, MeSH or course in ti, ab, MeSH prognosis in MeSH or survival-analysis risk in ti, ab, MeSH exp cohort-studies or exp risk or (odds and ratio) in ti, ab, MeSH or (relative and risk) in ti, ab, MeSH or (case and control) in ti, ab, MeSH case -control-studies or cohort-studies sensitivity in ti, ab, MeSH sensitivity-and-specificity or sensitivity in ti, ab, MeSH or diagnosis in MeSH or radionuclide imaging in MeSH or diagnostic use in MeSH or specificity in ti, ab, MeSH exp sensitivity-and-specificity or (predictive and value) in ti, ab, MeSH
pt, publication type; ti, title; ab, abstract; MESH, Medical Subject Heading; hyphenated terms or exp = Thesaurus search; other terms are typed as given in free text. For the most comprehensive (as all inclusive and sensitive as possible) search possible, the Cochrane Collaboration have developed search strategies for MEDL1NE and EMBASE for RCTs and reviews. The former can be found in the methods section of reviews on the CDSR. Adapted from Haynes et al (1994a).
559
FINDING THE EVIDENCE
rigorous testing for specificity and sensitivity. You may wish to consider the strategies outlined for other topics on MEDLINE (Table 3) and EMBASE (Table 4) which have been provided by the Cairn's Library in Oxford, although these have not undergone the same stringent analysis. W H E R E TO NEXT: WHICH RESOURCES? We have already discussed that the CDSR and DARE databases are our first stop for information. Increasingly, you will need to go no further than the CDSR in the Cochrane Library, but where to next when the information you need has not been reviewed within this format (see Figure 2 above)? If this chapter had been written 6 months ago, this particular section would have now headed straight to MEDLINE as the most useful database for Table 3. Suggested search terms for systematic review, economic analysis, decision analysis and clinical guidelines on MEDLINE (WinSpirs version). The search strategies outlined have been compiled by Anne Lusber and Robin Snowball at the Cairns Library, John Radcliffe Hospital, Oxford. They have not been analysed in the same manner as the methodological filters constructed by Haynes et al (1994a) but are useful suggestions. Clinical topic
Suggested strategies
Systematic review
1. review-academic in patient (use limit) 2. review-tutorial in patient (use limit) 3. systematic near (review or overview) in ti, ab, MeSH 4. (meta?analysis or meta analysis) in ti, ab, MESH, pt
Economic analysis/evaluations
1. explode costs and cost-analysis 2. explode models---economic 3. economic in MeSH 4. statistics and numerical data in MeSH 5. (cost?effective or cost effective) in ti, ab, MeSH 6. (economic or cost) and (analysis or assessment or evaluation or benefit or effective) in ti, ab, MeSH 7. 1 or2 or 3 o r 4 o r 5 or6
Clinical decision analysis
1. 2. 3. 4. 5. 6. 7. 8.
Clinical guidelines
1. 2. 3. 4. 5. 6. 7.
explode decision support techniques explode decision making computer assisted explode artificial intelligence (decision and (analysis or tree or model or strategy or support or utility) in ti, ab, MeSH uncertainty in ti, ab, MeSH probability in ti, ab, MeSH explode probability 1 o r 2 o r 3 o r 4 o r 5 or6 or 7 guideline in ti, ab, MESH, pt consensus in ti, ab, MESH, pt protocol in ti, ab, MESH, pt standard in ti, ab, MESH, pt legislation in ti, ab, MESH, pt recommend in ti, ab, MESH, pt 1 or 2 or 3 or 4 or 5 or 6
ti, title; ab, abstract; MESH, Medical Subject Heading, hyphenated terms or exp = Thesaurus search, other terms are typed as given in free text.
560
I . E . COOKE
Table 4(a) and (b) are suggested search strategies for EMBASE for the clinical topics of (a) therapy, prognosis, harm and diagnosis based on the methodological filters which have been evaluated for MEDLINE; (b) systematic review, economic analysis, decision analysis, and clinical guidelines. The search strategies outlined have been compiled by Anne Lusher and Robin Snowball at the Cairns Library, John Radcliffe Hospital, Oxford. They have not been analysed in the same manner as the methodological filters constructed by Haynes et al (1994a) but are useful suggestions. Clinical topic
Strategy
EMBASE
Treatment
Best one-term strategy Maximum sensitivity strategy
clinical trial (explode in Thesaurus) random (ti, ab, keywords) or drug therapy (ti, ab, keywords) or drug comparison (explode in Thesaurus) or drug efficacy (Thesaurus) or controlled study (Thesaurus) or clinical trial (explode in Thesaurus double blind (ti, ab, keywords) or placebo (ti, ab, keywords)
Maximum specificity strategy Prognosis
Best one-term strategy Maximum sensitivity strategy
Maximum specificity strategy Aetiology/harm
Best one-term strategy Maximum sensitivity strategy
Maximum specificity strategy Diagnosis
Best one-term strategy Maximum sensitivity strategy
Maximum specificity strategy
prognosis (explode in Thesaurus) cohort analysis (thesaurus) or longitudinal study (Thesaurus or prospective study (Thesaurus) or follow-up (thesaurus or incidence (explode in Thesaurus) or mortality (explode in Thesaurus) or death (explode in Thesaurus) or survival (explode in Thesaurus) or mortality (ti, ab, keywords) or predict (ti, ab, keywords) or progress (ti, ab, keywords) or disease course (explode in Thesaurus) or course (ti, ab, keywords) prognosis (explode in Thesaurus) or survival (explode in Thesaurus) risk (ti, ab, keywords) cohort analysis (Thesaurus) or longitudinal study (Thesaurus) or prospective study (Thesaurus) or follow up (Thesaurus) or risk (explode in Thesaurus) or Odds Ratio (ti, ab, keywords) or relative risk (ti, ab, keywords or case control (ti, ab, keywords) case control study (Thesaurus) or cohort analysis (Thesaurus) sensitivity (ti, ab, keywords) sensitivity (ti, ab, keywords) or specificity (ti, ab, keywords) or diagnos (ti, ab, keywords) (Also consider including headings for specific diagnostic term/procedures) sensitivity (ti, ab, keywords) or specificity (ti, ab, keywords) or predictive value (ti, ab, keywords) or ROC curve (ti, ab, keywords) or diagnostic value (Thesaurus) or diagnostic accuracy (Thesaurus)
561
FINDING THE EVIDENCE Table 4b.
Clinical topic
Suggested search strategy EMBASE
Systematic review
1. systematic review (ti, ab, keywords) 2. systematic overview (ti, ab, keywords) 3. meta?analysis, meta analysis (ti, ab, keywords) 4. review (Thesaurus) 5.1,2,3,4,5
Economic analysis
1. 2. 3. 4. 5. 6. 7. 8.
Clinical decision analysis
Clinical guidelines
cost (thesaurus) cost benefit analysis (Thesaurus) cost control (Thesaurus) cost effectiveness (Thesaurus) health care cost (explode in Thesaurus) economic (ti, ab, keywords) cost/effective cost effective (ti, ab, keywords) (economic cost) + (analy, assessment, evaluation, benefi effective) (ti, ab, keywords) 9. 1,2,3,4,5,6,7,8
I. 2. 3. 4. 5. 6. 7. 8. 9. 10.
computer assisted diagnosis (explode in Thesaurus) medical decision making (Thesaurus) decision theory (Thesaurus) decision making (Thesaurus) probability (Thesaurus) artificial intelligence (Thesaurus) Bayes Theorem (Thesaurus) risk (explode in Thesaurus) prediction and forecasting (Thesaurus) decision+(analy, tree, model, strateg, support, utilit) (ti, ab, keywords) 11. uncertainty (ti, ab, keywords) 12. probability (ti, ab, keywords) 13. 1,2,3,4,5,6,7,8,9,10,11,12 1. 2. 3. 4. 5. 6. 7.
guidelines (ti, ab, keywords) consensus (ti, ab, keywords) protocol (ti, ab, keywords) standard (ti, ab, keywords) legislation (ti, ab, keywords) recommendation (ti, ab, keywords) 1,2,3,4,5,6
ti, title; ab, abstract; MESH, Medical Subject Heading, hyphenated terms or exp = Thesaurus search, other terms are typed as given in free text.
clinicians when searching for evidence from reviews or trials that have not yet been incorporated within the CDSR. In 1996 the CCTR was added to the Cochrane Library. This at present includes 94 000 clinical trials that have been retrieved either from MEDLINE or from the specialized registries built up by the Cochrane Review Groups. Text words can be used for searching this database, thereby making it relatively simple. MeSH words can also be used but will only retrieve MEDLINE articles within the CCTR. Yet again, it is already particularly useful to our specialty because the registry of trials form the Pregnancy and Childbirth, and Subfertility Groups. At present, the Cochrane Collaboration are developing software that would seamlessly transfer you through a series of databases starting
562
I . E . COOKE
with the Cochrane Library and proceeding on to the most appropriate databases for your question (ACPJCOD, MEDLINE, EMBASE etc.). Table 5 lists just a taste of other databases and information packages that are available and that are of use to the health-care profession. The UK is a leading force in winging the change towards evidence-based practice. Lack of accessible information (on the ward or in the general practitioner's surgery) has frequently been adduced as a barrier to this implementation, hence what seems like a flurry of activity in the technology of provision and dissemination of information. The list can be confusing, but there are software packages to help you decide which of the 60 biomedical databases will be most useful to you. Your librarian can help you here by accessing Table 5, Examples of databases that are available with general descriptions of the regions of interest covered by each. Database
Years covered
Areas of interest
Cochrane Library CDSR DARE CCTR ACPJCOD MEDLINE
No limits
All specialties. Includes the previous Pregnancy and Childbirth Reviews
1990 onwards 1966 onwards
EMBASE
1974 onwards
Biological Abstracts
1990 onwards
CINAHL Nursing and Midwifery Index CANCERLIT
1982 onwards 1994onwards
General medicine Electronic version of Index Medicus and International Nursing Index. Strong on North American literature Electronic version of Excerpta Medica. Strong on European studies and pharmacology literature Clinical and experimental medicine with biological science Nursing journals, chapters, books, theses Nursing and midwifery in the UK
1963 onwards
CAB HEALTH
1973 onwards
Science Citation Index Scientific and Technical Proceedings Current Contents
1981 onwards 1982 onwards Present
AIDSline PsycLit ASSIA Sociofile Bioethicsline PAlS Health Periodicals
1980 onwards 1974 onwards 1987 onwards 1974 onwards 1973 onwards 1970 onwards 1987 onwards
Healthplan
1986--1996 (previous 10 years)
Epidemiology, pathogenesis, immunology and treatment of cancer Environmental health, tropical medicine and nutrition including obstetrics and gynaecology General science and author citation Details of papers presented at 4000 conferences (including medical conferences) per year Contents pages of hundreds of journals in life sciences and clinical medicine updated weekly AIDS-related topics Psychology and related disciplines Social sciences Sociology Ethical issues Public and social policy. American bias Consumerism and public health, with summaries specifically for the lay person Administrative, financial, legal and ethical aspects of health care
This list is far from exhaustive and is simply a taster of the type of databases that are available. Your librarian will help you decide which is the most suitable for your needs and also alert you to any other locally available information sources. There are several with a UK focus, for example the King's Fund Library Database, Purchasing Innovations Database, NHS Database of economic evaluations and Project Register System.
FINDING THE EVIDENCE
563
an on-line service through, for example DataStar, called Cros search, which can rank the databases in order of priority for your search topic. To be an evidence-based practitioner, one needs to develop and hone one's searching technique. To become an efficient searcher, concentrate on a small nucleus of databases at first, to learn the few techniques that are necessary to be productive. Your librarian will be very helpful in showing you around the computer access route that your hospital library has in situ, explaining and demonstrating effective methods of using the appropriate databases and auditing your own first efforts at searching both for their inclusiveness (sensitivity) and relevance (precision). Frequently, you will discover that they may already have some search strategies stored as menu items, or they can help you do so, for recurrent use. There are 27 commercial medical products tailored for clinical use to access MEDLDIE. Haynes et al (1994b) compared the performance of each for userability. The important test is which one you, as an individual, find easiest to master. Those with the thesaurus stored electronically are especially useful. For example, our own library at the John Radcliffe Hospital, Oxford, has WinSPIRS, which has all the MeSH term information and search limitations on the screen, thus reducing the need to refer to the very large index catalogue. The latter is useful, however, for keeping up to date with any changes in MeSH terms, for example as new interventions are introduced and indexed. It also has several other databases (Biological Abstracts, PsycLit, and Sociofile and SERLINE) on the same menu. The other often-quoted database is the electronic version of Excerpta Medica (EMBASE). It covers 4700 journals with a strong coverage of the pharmacology, environmental health and European literature and an approximately 50% overlap with MEDLINE. It is at present less popular than MEDLINE because the interface is less easy to come to grips with, and in the UK it is accessed through the Joint Academic Network (JANET). In the UK, the British Medical Association allows free access to MEDLINE (the interface is CDP OVID) to all its members. With little expense (just a modem attachment and telephone call), you can carry out literature searches from home at any time of day. In the USA, the American College of Obstetricians and Gynecologists offers its members ACOGNET (a free communications network to its own publications on a 24-hour basis), but its librarians also offer complementary MEDLINE searches. However, to work beside your librarian and hence use your combined talents is usually a more productive activity, rather than just leaving a few details with your librarian and hoping that he or she will produce papers that look at the clinical problem from exactly the perspective you require. Having highlighted its usefulness, a sobering thought on MEDLINE is that it covers less than a quarter of the biomedical journals (3700). Even if we apply the optimally sensitive search for RCTs, which has been developed by Carol Lefebvre at the UK Cochrane Centre, only 50% of the RCTs known to be in MEDLINE would actually be picked up (Dickersin et al, 1995). (The figure is closer to 20% for experienced clinical searchers.) This poor 'hit' rate is thought to be due to three main problems. The first is poor indexing. 'Random allocation" and 'randomized controlled trial' were
564
I.Z. COOKE
first used as descriptor terms in 1975 and 1990 respectively, and it was not until 1991 that 'randomized controlled trial' was introduced as a publication type. Second, the index terms may be applied incorrectly by the professional indexers. That is annoying to the trialists, but in fact the third reason why RCTs are missed is that the trialists themselves do not clearly specify methods used within the study! There is little reason to expect any better return from other general databases, although they have yet to be scrutinized in the same manner. To improve this situation, MEDLINE have agreed to re-enter as an RCT (onto MEDLINE) any RCT retrieved by the electronic and hand searching efforts of the Cochrane Collaboration. To date, over 80 000 RCTs have been reclassified as such on MEDLINE. A R E A B S T R A C T S U S E F U L AS A DATA S O U R C E ? At this stage of the search, you will be left with a list of titles and abstracts. Should you find all these references in full text or can you do any further sifting with the abstracts themselves? Reading the abstracts can help you to slim down the final fist. Scan through for the specific subject; does the abstract look at the problem from the angle you are chasing? The methodology used (systematic review, RCT, cohort study, etc.) is often listed in the title or among the MeSH terms, if not in the abstract itself. Structured abstracts are usually more informative than the unstructured format as to the content of a paper. Abstracts are, however, prone to bias in that they often highlight the most positive aspects of a paper, so it is unwise to base your final decision on the information from abstracts alone.
RECORDING THE SEARCH STRATEGY Searching for the best evidence is of equal importance to the responsibility of evaluating the validity and results of the papers found. As such, it is imperative to outline the search terms and details of the databases you have used. This is not only a record for yourself, should you wish to repeat the search, but most importantly it allows others to scrutinize that search strategy for its completeness and soundness. S E C O N D A R Y PUBLICATIONS So far, this chapter has been aimed at finding evidence for specific clinical problems. You might ask whether there is any help out there for those who want to try to keep abreast of current literature. This is called 'current awareness'. If there are particular areas of interest in which you want to learn of each new publication as it arises, your librarian can set up a current awareness request for you. It means that an electronic search will be performed automatically for you, as regularly as you require, on these topics. If you prefer to continue browsing through journals, make
FIND1NG THE EVIDENCE
565
best use of that time on 'high-yield' journals: those which give the highest yield of good-quality studies on your topic. Remember, when something of interest catches your attention, you still have to validate the methods and results. There are now 'secondary' journals that pick out useful articles and those of particular relevance. In other words, you not only get the searching done for you but also the data are critically appraised. The aim of these journals is to 'read' the current literature for clinically important articles that are well executed and present them as a structured abstract with the results transferred to a clinically relevant format. Only 2% of the articles reviewed are viewed as being of appropriate quality to be considered. It is therefore little wonder that we feel swamped by the amount of literature we feel we should sift through when that sift has to filter out the other 98% of articles. At present there are only two such journals in press. The ACP Journal Club (published by the American College of Physicians) was the first of these and is made available in both paper and electronic format (ACPJCOD). Since 1990, it has scanned 50 journals for papers of specific interest to general physicians. However, the Evidence-based Medicine journal (produced by the British Medical Journal), in publication since 1995, with the same methods of selection and presentation, is aimed at a wider audience: general practice, medicine, surgery, obstetrics and gynaecology, paediatrics and psychiatry. A series of evidence-based journals for different specialties is in the pipeline from the publishing group Churchill Livingstone, one of which will be aimed at obstetricians and gynaecologists.
HOUSEKEEPING THE INFORMATION RETRIEVED The aim of the evidence-based practitioner is to use the problems you encounter daily as pointers for your continuing education. We all know that, whatever our good intentions, most of us are poor at remembering the evidence we have found when next we meet the same problem. While reference management software (for example Reference Manager, Procite or Endnote) is useful for ever-expanding reference lists, most of us would prefer to keep the clinical information we glean close to hand at all times. Professor Sackett has coined the phrase of a personal 'instant resource book'. Whenever you go to the trouble of searching and appraising the evidence for a particular question, summarize the search technique you used, list the articles retrieved and arrange the clinically relevant results obtained on a single A4 page marked with the date you did the search and a 'sell by' date to remind you of when it might be appropriate to check it out again. These summaries are called 'critically appraised topics' (CATs). A CAT-maker for therapy data, designed as an interactive piece of software, is available on the Centre for Evidence-based Medicine World Web page to help you create these resources for yourself. Surprisingly quickly, the file of
566
1. E. COOKE
evidence grows as a nucleus of information for your own practice: your own instant resource book relevant to the peculiarities of your practice.
SUMMARY When the Cochrane Collaboration fulfils its quest to organize the available evidence in a registry of trials and systematic reviews, it will mean that our task to find the best evidence will be much less daunting than it may at present seem. Meanwhile, to be evidence-based practitioners, we must learn the techniques that will make our search both comprehensive and efficient. To find this evidence requires: 1. 2. 3. 4.
the construction of a specific question based on the clinical problem; the design of a search strategy that is inclusive but practical; the search of the appropriate database(s); the scan of the retrieved abstracts for the most useful papers for critical appraisal.
Efficient searching for evidence does mean getting to grips with the manifold advantages that computerization can offer us. The use of a computer for this purpose needs, like any other new technique we take on, practice to gain competence and confidence. Within the UK, the National Health Service Research and Development Department has made the collation and dissemination of clinical evidence to practitioners a priority. Hence at present, there is an enormous effort within the UK to make sure that the evidence is in the clinical areas of need, in a format we can use it and available in a place where and when we want it, whether that is the ward or the general practitioner's surgery. The aim is to remove at least some of the barriers that prevent us from becoming individual evidencebased practitioners.
REFERENCES Altman DG (1994) The scandal of poor medical research. British Medical Journal 308: 283-284. Antman EM, Lau J, Kupelnick Bet al (1992) A comparison of results of meta-analyses of RCTs and recommendations of clinical experts. Treatments for myocardial infarction, Journal of the American Medical Association 268: 240-248. Cavell DG, Uman GC & Manning PR (1985) Information needs in office practice: are they being met? Annals of Internal Medicine 103: 596-599. Cochrane AL (1989) Effectiveness and Efficiency. London: British Medical JournallNuffield Provincial Hospitals Trust. Cochrane Library (1996) Cochrane Collaboration, Issue 1. Oxford: Update Software. Available from British Medical Journal Publishing Group, London. Davidoff E Haynes RB, Sackett DL et al (1995) Evidenced-based Medicine: a new journal to help doctors identify the information they need. British Medical Journal 310: 1085--1086. Dickersin K, Scherer R & Lefebvre C (t995) Identifying relevant studies for systematic reviews. In Chalmers I & Airman DG (eds) Systematic Reviews. London: British Medical Journal Publishing Group. Grimes DA (1991) Randomization controlled trials: "it ain't necessarily so'. Obstetrics and Gynecology 78: 703-704.
FINDING THE EVIDENCE
567
Grimes DA & Schulz KF (1996) Methodology citations and the quality of RCTs in obstetrics and gynecology. American Journal of Obstetrics and Gynecology 174(4): 1312-1315. Haynes RB (in press) Searching for the evidence. In Sackett DL, Richardson WS, Rosenberg WMC & Haynes RB (eds) Evidence-based Medicine: How to Practise and Teach EBM. London: Churchill-Livingstone. Haynes RB, Wilezynski NL, McKibbon KA et al (1994a) Developing optimal search for detecting clinically sound studies in MEDLINE. Journal of the Amer~Z'anMedical Information Association 1: 447-458. Haynes RB, Walker CJ, McKibbon KA et al (1994b) Performance of 27 MEDLINE systems tested by searches on clinical questions. Journal of the American Medical Information Association 1: 285-295. Irwig L, Macasill P, Glasziou P e t al (1995) Meta-analytic methods for diagnostic test accuracy. Journal of Clinical Epidemiology 48: 119-130. McDonough PG (1995) 'Leaky randomization': standard practice--but is it correct? Fertility and Sterility 64: 216-217. Mulrow CD (1995) Rationale for systematic reviews. In Chalmers I & Altman DG (eds) Systematic Reviews. London: British Medical Journal Publishing Group. Peto R, Pike MC, Armitage Pet al (1976) Design and analysis of randomized clinical trials requiring prolonged observation of each patient. I Introduction and design. British Journal of Cancer 34: 585-612. Sackett DL (in press) On the need for evidence-based medicine. In Sackett DL, Richardson WS, Rosenberg WMC & Haynes RB (eds) Evidence-based Medicine: How to Practise and Teach EBM. London: Churchill-Livingstone. Schulz KF, Chalmers I, Grimes DA et al (1994) Assessing the quality of randomization from reports of controlled trials published in obstetrics and gynecology journals. Journal of the American Medical Association 272: 125-128. Schulz KF, Chalmers I, Hayes RJ et al (1995) Empirical evidence of bias: dimensions of methodological quality associated with estimates of treatment effects in controlled trials. Journal o f the American Medical Association 273: 408-412.
An evidence-based practitioner needs to retrieve information efficiently. This chapter describes the variety of resources available and suggests some methods to search for answers most effectively. Over two million articles are published annually in the biomedical literature in over 20 000 journals. Health-care providers are all inundated with unmanageable amounts of information. How then do we meet the challenge of keeping up to date with current best evidence when it is hidden somewhere in this paper mountain? It seems an awesome task made more difficult by the quantity, organization, indexing and variety of journals and resources. To benefit your patient, your searching technique must find the best evidence (be comprehensive) published on the clinical problem. To be practical (and not tire your enthusiasm) the process must be speedy to retrieve the information efficiently. Two million articles, one year's worth of papers, have been estimated to rise as a 500 m tower of paper (Mulrow, 1995). WHERE DO WE GO FOR INFORMATION AT PRESENT? Needing information, whether it is a general overview of a disease that we meet rarely or a specific clinical problem recurring frequently in our practice, is not new. We need this evidence, or information, daily, and our patients would benefit from it. It was estimated, when shadowing a general physician through a typical (half-day) session in clinic, that four clinical decisions would have been altered had the particular information about the patients' conditions been available and employed (Cavell et al, 1985). Where have you usually gone looking for new information? Clinicians say they go to their textbooks and journals for this knowledge, but when these same clinicians were shadowed, the reality was quite different. They went most frequently not to their books, but to their colleagues and in particular their senior colleagues at work. With all their endeavours, however, they report that only 30% of the time are their information needs met by their normal methods. The problems are understandable and well known to us all: no time, textbooks are out of date and the sheer volume of literature is off-putting, to say the least. We are often left frustrated by our Bailli~re's Clinical Obstetrics and GynaecologyVol. 10, No. 4, December 1996 ISBN 0-7020-2260-8 0950-3501/96/040551 + 17 $12.00/00
551 Copyright © 1996, by Bailli~re Tindall All rights of reproduction in any form reserved
552
i.E. COOKE
inability to unleash what gold we imagine is hidden somewhere within the indexing system of a database. Most clinicians have less than 30 minutes a week to read around their subject, irrespective of whether they are hospital-based clinicians or general practitioners (Sackett, in press). The message is very clear: in a busy clinical week there is little time available to further our education, so whatever we do with that time must be focused. The excuse 'Our textbooks are out of date' is not just an excuse: the claims are frequently correct (see Table 1 below; Antman et al, 1992). It is estimated that for general physicians just to keep abreast of their specialty, they would have to read 133 articles per week (Davidoff et al, 1995). How do you efficiently 'distil the elixir' of clinically useful information, pertinent to your patient, from this deluge? This chapter is aimed at guiding you towards some of the most fruitful databases, especially for obstetricians and gynaecologists, to help you on your way. There will be specific examples in the forthcoming chapters, so here we discuss the types of databases, helpful rules and strategies that will help your search sift that gold. LEVELS OF EVIDENCE What is the evidence we are looking for? The answer is, of course, the best evidence, but as you will find in the forthcoming chapters, there will, for different clinical questions, be different types of evidence that will be the 'best'. The most comprehensive evidence would be a systematic review already written for every conceivably useful topic. Fortunately for us, it is a dream that many people in the Cochrane Collaboration are working towards making a reality. The methodology of systematic reviews of interventions is well organized. At present, methods of combining the results for other clinical data such as diagnostic or prognostic studies, or indeed the accuracy of so doing, is still under debate (Irwig et al, 1995). For each type of clinical data, there are 'levels of evidence'. When we search for evidence on a therapeutic agent (antihypertensives in preeclampsia, antifibrinolytics for menorrhagia) we hope that among the Table 1. Classification of levels of evidence. Level I
II
Type of evidence Strong evidence from at least one systematic review of multiple, well-designed randomized controlled trials Strong evidence from at least one properly designed randomized controlled trial of appropriate size
III
Evidence from well-designed trials without randomization, single group pre-post, cohort, time series or matched case-controlled studies
IV
Evidence from well-designed non-experimental studies from more than one centre or research group
V
Opinions of respected authorities, based on clinical evidence, descriptive studies or reports of expert committees
FINDING THE EVIDENCE
553
studies, or 'hits', retrieved there will be a well-conducted systematic review or, failing that, a well-conducted randomized controlled trial (RCT) of sufficient power. What if neither of those are available? The fact that the gold standard evidence is not available does not mean that you should not seek out and assess the less reliable evidence. You still have to make a clinical decision, so you will have to use this less reliable information to make that decision but knowing that it is founded on much less dependable results, on less strong levels of evidence (Table 1). W H Y A SYSTEMATIC R E V I E W ? A systematic review gives details of the methods of trial collection, reasons for including and excluding trials and statistical methods of analysis. Inclusion of this information means that the readers can decide for themselves whether any important sources of information have been excluded or inappropriate information included. This is what makes a systematic review so different from a traditional review. You are not dependent on the bias that the author may unwittingly bring to the article. A failure to apply the same scientific principles to the design and conduct of a review as apply to the design and conduct of the primary studies has meant that, just like our textbooks, advice in traditional reviews has often failed to reflect the strong evidence that would have emerged had the reviewers applied scientific principles systematically to reviews of RCTs (Figure 1; Antman et al, 1992). A systematic review does not necessarily have results presented as a meta-analysis; the data may simply not be appropriate to present in numerical form. A meta-analysis of RCTs means that the data from small trials, which individually are of too small a sample size to have the power to have been useful, can be combined, giving both increased power and increased precision to the results. The systematic review allows large quantities of information to be 'melted down' and presented as something palatable. It separates the insignificant and irrelevant from the salient and critical studies for us and allows us to assess the 'generalizability' of the results to different populations. The direct comparisons also enable the reviewer to examine the studies for consistency of results and to assess what makes those trials, which appear to have inconsistent results, different. Our ability to find the answer to our particular clinical problem within a systematic review (for example, pain after repair of perineal trauma following childbirth with various suture materials or the pregnancy rate associated with clomiphene use in women with unexplained infertility) does not remove from us the onus of deciding whether or not the results are applicable to our own patient or population. When the systematic review has yet to be done (or is not yet possible), there are two answers as to how to move forward. The first is to produce the review yourself under the umbrella of the Cochrane Collaboration. This is not as daunting as it may seem if the review needs to be done and one accepts the wealth of assistance and experience that is made available
554
I.E. COOKE
-t-,
E ~
d
"6
Textbook/review recommendations for giving lidocaine
Odds ratio (log scale)
;0.5 =
1 =
•
=
|
2
=
i
~
|
|
=
i
|
|=i
'10 O~ r-
1960
-~
i== 0
09
n-
W
Z
1965
1970
2 304 5 647 6 850 8 1239 9 1451
2
do
17
4
4
1
7
1
1966
1
1970
8
2
1974
2
0
1975 11 1686
1
4
12 1986
4
2
4
8
1
5
6
2
3
5
3
4
2
1980
1
1
1978
1982
1985 14 8412 1
1986
3 4
15 8745
P=>0.05
1990
5
9
4
6
1
3
2
1
1990
m
!
Favours treatment
Favours control
Figure 1. Cumulative meta-analysis by year of publication of randomized controlled trials of prophylactic lignocaine for acute myocardial infarction and recommendations of clinical expert reviewers. Adapted from Antman et al (1992).
through this organization. However, it is obviously not a quick option. The second plan is the one that will give the best evidence available to help in 'the here and now', the help we need for the patient in next week's clinic. The forthcoming chapters will discuss the mechanisms of critical appraisal and assessing the applicability of the results to your patient or population for diagnostic tests, therapies and prognostic factors. One publication type has risen above all the others as a gold standard when assessing the effectiveness of a particular intervention, that of the RCT.
FINDING THE EVIDENCE
555
WHY A R A N D O M I Z E D C O N T R O L L E D T ~ A L ? The act of true randomization and concealment of allocation is the only known way to avoid selection biases that handicap observational studies (Peto et al, 1976). While the number of RCTs produced is estimated at between 250000 and 1 000000, those identified in obstetrics and gynaecology are relatively rare (the registry for Pregnancy and Childbirth at present contains 8000 trials). It was not until 1991 that the RCT was indexed as a publication type on MEDLINE, so it is only with the march of hand searchers through past issues of journals that the true wealth of RCTs 'out there' is coming to light. Such is their place in clinical research that there are some medical journals which have agreed, in future, to publish only RCTs of interventions. 'The RCT is a very beautiful technique, of wide applicability, but as with everything else there are snags' (Cochrane, 1989). Unfortunately, many of those RCTs produced in obstetrics and gynaecology are found to violate the simple rules of conduct (Schulz et al, 1994), resulting in trials that are subject to bias, with results that, in consequence, seem misleading (Grimes, 1991). Grimes and Schulz suggest that this poor record of reporting and conducting RCTs is simply due to naivety rather than negligence. In over 80% of reported RCTs, in four major obstetric and gynaecology journals, no reference was made to methodological articles. This was reflected by an inadequate randomization system or method of allocation concealment (Grimes and Schulz, 1996). They argue that if clinicians sought help and advice before expecting to embark on a RCT, just as we would not carry out an operation without adequate training, there would be less wasted effort. Because of poor science (McDonough, 1995), leading to poor clinical practice (Grimes, 1991), one critic argues that poor-quality research is unethical (Altman, 1994). It is important, however, to realize that the effects of some health care interventions are so obvious that they can be recognized confidently without conducting carefully controlled research. Stimulating a heart to re-establish its normal rhythm after it has stopped beating (defibrillation) provides a dramatic example of an easily recognized beneficial effect of health care. RCTs are the best evaluations o f health care for therapy, prevention, quality of life and economics, and may be the most appropriate study in evaluations of harm and aetiology. There are other occasions on which RCTs would be impossible or inappropriate. To assess the effects of harmful substances such as smoking and lung cancer, the oral contraceptive pill and breast cancer or thrombosis, one cannot randomize the patients. Other research strategies, cohort or case-controlled studies, will offer the best evaluation of harm, aetiology or prognosis in these instances. That said, the RCT has pride of place among the publication types used to test the efficacy of any intervention. THE DREAM DATABASE If these are the types of evidence we want, where can we go to find them and, most importantly, find them with relative ease7 The resource that
556
I.E. COOKE
would be most useful to a clinician would provide systematic reviews of data relevant to any particular patient and the specific intervention being considered. It would therefore encompass all specialties, be easily accessible and be presented in a comprehensible and clinically relevant format to help you decide the implications for your practice. Even if the only information that the obstetrician or gynaecologist obtains from this database is that there is no reliable information as to the worth of a particular intervention, that in itself would be very useful for both present patient care and possible future areas of research. The Cochrane Collaboration aims to produce high-quality systematic reviews of trials that have been performed in every aspect of health care. The evidence will primarily be from RCTs, but other evidence will be reviewed if the RCT has not been produced or would be inappropriate. To ensure that the reviews available to us, the public, are as up to date as the trials themselves, the reviews are updated as new relevant information comes to light and are disseminated in electronic, rather than paper, format. For those of us wary of the beast with a keyboard and visual display monitor, the essence of the software and presentation of data within the Cochrane Library is transparency for the user, whether patient, nurse, student or professor. The Cochrane Library contains: 1. The Cochrane Database of Systematic Reviews (CDSR). This is an electronic journal, updated quarterly, of systematic reviews using explicitly defined methods. 2. The Database of Abstract of Effectiveness (DARE). Assembled by the UK National Health Service Centre for Reviews and Dissemination at the University of York, this is a bibliography of systematic reviews that are of good quality, and have been published elsewhere, but have not been arranged in the format of a Cochrane review. At present, there are over 1100 abstracts in this database. 3. The Cochrane Controlled Trials Register (CCTR). This contains 94 000 references to controlled clinical trials, which have been identified from MEDLINE and the specialized registries and collated by Cochrane Review Groups. 4. The Cochrane Review Methodology Database (CRMD). This is a bibliography of books, journal issues and articles on methodological aspects of systematic reviews. This sounds like the dream database for clinicians, and it really will be a very important source, if not the source, of information for health-care givers in the future. The reality at present is that while very necessary groundwork in hand searching, electronic searching and preparing protocols is taking place, there are still only a small, but rapidly growing, number of completed reviews (120 completed reviews and 200 protocols). Of all specialties, however, obstetrics and gynaecology has one of the richest resources within the Cochrane Library within the Pregnancy and Childbirth, Subfertility and Menstrual Disorders Groups. You may carry out your search on the CDSR either by text words (subject heading for the pregnancy and childbirth reviews) or by listing the titles that are registered
557
FINDING THE EVIDENCE
Refine your search strategy with the information retriev~....ll~ 1
/
/
The clinical question Jm Develop a search strategy / •
~
Decide on the best resources
~
/
Cochrane Library
1
SeconV r,iouma,s
[
4
Seek help from your librarian & | / |
~
Has your search retrieved
3. relevant
V information?
Figure 2. General search strategy.
under each Cochrane Review Group. This latter will become less efficient as the number of reviews grows and may mean that you miss titles of interest that have been listed under different Review Groups. For example, if the topic choice is bone disease after the menopause, using text word would retrieve those reviews which have been composed under either the musculoskeletal or menstrual disorders or primary care field, to name only a few possibilities. So, to go back to 'finding the evidence': Figure 2 outlines the process of performing a search: 1. Convert your clinical problem into a focused question. 2. Collate the specific words you are going to use for the search. It takes a little practice to ensure that this part of the search is as all-inclusive as possible using the correct Medical Subject Heading (MESH) terms, text words and synonyms of a text word. As you proceed through the search, certain words within abstracts often jog your thought process. Population/patient
Intervention
Outcome
premature infant or premature neonate or premature birth or preterm infant or preterm neonate
corticostereid(s) or antenatal steroids or glucocorticoids or betamethasone or adrenal cortex hormone(s)
respiratory distress syndrome or RDS
and
and
Figure 3. The risks of prematurity are numerous and the benefit of antenatal steroids now well documented. This problem is used as an example to (a) demonstrate construction of a question: 'In preterm infants does the administration of antenatal corticosteroids reduce the incidence of respiratory distress syndrome?', (b) list some of the possible search terms you may consider, and (c) demonstrate how the use of 'or" for all the terms within a column expands the search, and the use of 'and" between the columns narrows the search.
558
3.
4.
I . E . COOKE
The words 'and/or' are used to narrow or expand your search. If you imagine two search terms represented by two overlapping circles, 'or' would include everything within both circles, whereas 'and' would only include data within the area of overlap between the circles. Figure 3 shows a practical example. At this stage, you want to reduce your search to those trials of the best quality and most relevance to your topic. Several useful methodological filters for different clinical topic areas have been produced to aid you at this stage. Those described in Table 2 have undergone
Table 2. Search terms for the best studies of treatment, diagnosis, prognosis or cause/harm for MEDLINE. Clinical topic Treatment
Strategy
MEDLINE (WINSPIRS)
Best one term
clinical-trial in pt randomized-controlled-trial in pt or drug therapy in MeSH or therapeutic use in MeSH or random in ti, ab, MeSH (double and blind) in ti, ab, MeSH or placebo in ti, ab, MeSH
Maximum sensitivity
Maximum specificity Prognosis
Best one term
Maximum sensitivity
Maximum specificity Aetiologyfcause
Best one term
Maximum sensitivity
Maximum specificity Diagnosis
Best one term
Maximum sensitivity
Maximum specificity
exp cohort-studies incidence in MeSH or exp mortality or follow-up-studies or mortality in MeSH or prognos in ti, ab, MeSH or predict in ti, ab, MeSH or course in ti, ab, MeSH prognosis in MeSH or survival-analysis risk in ti, ab, MeSH exp cohort-studies or exp risk or (odds and ratio) in ti, ab, MeSH or (relative and risk) in ti, ab, MeSH or (case and control) in ti, ab, MeSH case -control-studies or cohort-studies sensitivity in ti, ab, MeSH sensitivity-and-specificity or sensitivity in ti, ab, MeSH or diagnosis in MeSH or radionuclide imaging in MeSH or diagnostic use in MeSH or specificity in ti, ab, MeSH exp sensitivity-and-specificity or (predictive and value) in ti, ab, MeSH
pt, publication type; ti, title; ab, abstract; MESH, Medical Subject Heading; hyphenated terms or exp = Thesaurus search; other terms are typed as given in free text. For the most comprehensive (as all inclusive and sensitive as possible) search possible, the Cochrane Collaboration have developed search strategies for MEDL1NE and EMBASE for RCTs and reviews. The former can be found in the methods section of reviews on the CDSR. Adapted from Haynes et al (1994a).
559
FINDING THE EVIDENCE
rigorous testing for specificity and sensitivity. You may wish to consider the strategies outlined for other topics on MEDLINE (Table 3) and EMBASE (Table 4) which have been provided by the Cairn's Library in Oxford, although these have not undergone the same stringent analysis. W H E R E TO NEXT: WHICH RESOURCES? We have already discussed that the CDSR and DARE databases are our first stop for information. Increasingly, you will need to go no further than the CDSR in the Cochrane Library, but where to next when the information you need has not been reviewed within this format (see Figure 2 above)? If this chapter had been written 6 months ago, this particular section would have now headed straight to MEDLINE as the most useful database for Table 3. Suggested search terms for systematic review, economic analysis, decision analysis and clinical guidelines on MEDLINE (WinSpirs version). The search strategies outlined have been compiled by Anne Lusber and Robin Snowball at the Cairns Library, John Radcliffe Hospital, Oxford. They have not been analysed in the same manner as the methodological filters constructed by Haynes et al (1994a) but are useful suggestions. Clinical topic
Suggested strategies
Systematic review
1. review-academic in patient (use limit) 2. review-tutorial in patient (use limit) 3. systematic near (review or overview) in ti, ab, MeSH 4. (meta?analysis or meta analysis) in ti, ab, MESH, pt
Economic analysis/evaluations
1. explode costs and cost-analysis 2. explode models---economic 3. economic in MeSH 4. statistics and numerical data in MeSH 5. (cost?effective or cost effective) in ti, ab, MeSH 6. (economic or cost) and (analysis or assessment or evaluation or benefit or effective) in ti, ab, MeSH 7. 1 or2 or 3 o r 4 o r 5 or6
Clinical decision analysis
1. 2. 3. 4. 5. 6. 7. 8.
Clinical guidelines
1. 2. 3. 4. 5. 6. 7.
explode decision support techniques explode decision making computer assisted explode artificial intelligence (decision and (analysis or tree or model or strategy or support or utility) in ti, ab, MeSH uncertainty in ti, ab, MeSH probability in ti, ab, MeSH explode probability 1 o r 2 o r 3 o r 4 o r 5 or6 or 7 guideline in ti, ab, MESH, pt consensus in ti, ab, MESH, pt protocol in ti, ab, MESH, pt standard in ti, ab, MESH, pt legislation in ti, ab, MESH, pt recommend in ti, ab, MESH, pt 1 or 2 or 3 or 4 or 5 or 6
ti, title; ab, abstract; MESH, Medical Subject Heading, hyphenated terms or exp = Thesaurus search, other terms are typed as given in free text.
560
I . E . COOKE
Table 4(a) and (b) are suggested search strategies for EMBASE for the clinical topics of (a) therapy, prognosis, harm and diagnosis based on the methodological filters which have been evaluated for MEDLINE; (b) systematic review, economic analysis, decision analysis, and clinical guidelines. The search strategies outlined have been compiled by Anne Lusher and Robin Snowball at the Cairns Library, John Radcliffe Hospital, Oxford. They have not been analysed in the same manner as the methodological filters constructed by Haynes et al (1994a) but are useful suggestions. Clinical topic
Strategy
EMBASE
Treatment
Best one-term strategy Maximum sensitivity strategy
clinical trial (explode in Thesaurus) random (ti, ab, keywords) or drug therapy (ti, ab, keywords) or drug comparison (explode in Thesaurus) or drug efficacy (Thesaurus) or controlled study (Thesaurus) or clinical trial (explode in Thesaurus double blind (ti, ab, keywords) or placebo (ti, ab, keywords)
Maximum specificity strategy Prognosis
Best one-term strategy Maximum sensitivity strategy
Maximum specificity strategy Aetiology/harm
Best one-term strategy Maximum sensitivity strategy
Maximum specificity strategy Diagnosis
Best one-term strategy Maximum sensitivity strategy
Maximum specificity strategy
prognosis (explode in Thesaurus) cohort analysis (thesaurus) or longitudinal study (Thesaurus or prospective study (Thesaurus) or follow-up (thesaurus or incidence (explode in Thesaurus) or mortality (explode in Thesaurus) or death (explode in Thesaurus) or survival (explode in Thesaurus) or mortality (ti, ab, keywords) or predict (ti, ab, keywords) or progress (ti, ab, keywords) or disease course (explode in Thesaurus) or course (ti, ab, keywords) prognosis (explode in Thesaurus) or survival (explode in Thesaurus) risk (ti, ab, keywords) cohort analysis (Thesaurus) or longitudinal study (Thesaurus) or prospective study (Thesaurus) or follow up (Thesaurus) or risk (explode in Thesaurus) or Odds Ratio (ti, ab, keywords) or relative risk (ti, ab, keywords or case control (ti, ab, keywords) case control study (Thesaurus) or cohort analysis (Thesaurus) sensitivity (ti, ab, keywords) sensitivity (ti, ab, keywords) or specificity (ti, ab, keywords) or diagnos (ti, ab, keywords) (Also consider including headings for specific diagnostic term/procedures) sensitivity (ti, ab, keywords) or specificity (ti, ab, keywords) or predictive value (ti, ab, keywords) or ROC curve (ti, ab, keywords) or diagnostic value (Thesaurus) or diagnostic accuracy (Thesaurus)
561
FINDING THE EVIDENCE Table 4b.
Clinical topic
Suggested search strategy EMBASE
Systematic review
1. systematic review (ti, ab, keywords) 2. systematic overview (ti, ab, keywords) 3. meta?analysis, meta analysis (ti, ab, keywords) 4. review (Thesaurus) 5.1,2,3,4,5
Economic analysis
1. 2. 3. 4. 5. 6. 7. 8.
Clinical decision analysis
Clinical guidelines
cost (thesaurus) cost benefit analysis (Thesaurus) cost control (Thesaurus) cost effectiveness (Thesaurus) health care cost (explode in Thesaurus) economic (ti, ab, keywords) cost/effective cost effective (ti, ab, keywords) (economic cost) + (analy, assessment, evaluation, benefi effective) (ti, ab, keywords) 9. 1,2,3,4,5,6,7,8
I. 2. 3. 4. 5. 6. 7. 8. 9. 10.
computer assisted diagnosis (explode in Thesaurus) medical decision making (Thesaurus) decision theory (Thesaurus) decision making (Thesaurus) probability (Thesaurus) artificial intelligence (Thesaurus) Bayes Theorem (Thesaurus) risk (explode in Thesaurus) prediction and forecasting (Thesaurus) decision+(analy, tree, model, strateg, support, utilit) (ti, ab, keywords) 11. uncertainty (ti, ab, keywords) 12. probability (ti, ab, keywords) 13. 1,2,3,4,5,6,7,8,9,10,11,12 1. 2. 3. 4. 5. 6. 7.
guidelines (ti, ab, keywords) consensus (ti, ab, keywords) protocol (ti, ab, keywords) standard (ti, ab, keywords) legislation (ti, ab, keywords) recommendation (ti, ab, keywords) 1,2,3,4,5,6
ti, title; ab, abstract; MESH, Medical Subject Heading, hyphenated terms or exp = Thesaurus search, other terms are typed as given in free text.
clinicians when searching for evidence from reviews or trials that have not yet been incorporated within the CDSR. In 1996 the CCTR was added to the Cochrane Library. This at present includes 94 000 clinical trials that have been retrieved either from MEDLINE or from the specialized registries built up by the Cochrane Review Groups. Text words can be used for searching this database, thereby making it relatively simple. MeSH words can also be used but will only retrieve MEDLINE articles within the CCTR. Yet again, it is already particularly useful to our specialty because the registry of trials form the Pregnancy and Childbirth, and Subfertility Groups. At present, the Cochrane Collaboration are developing software that would seamlessly transfer you through a series of databases starting
562
I . E . COOKE
with the Cochrane Library and proceeding on to the most appropriate databases for your question (ACPJCOD, MEDLINE, EMBASE etc.). Table 5 lists just a taste of other databases and information packages that are available and that are of use to the health-care profession. The UK is a leading force in winging the change towards evidence-based practice. Lack of accessible information (on the ward or in the general practitioner's surgery) has frequently been adduced as a barrier to this implementation, hence what seems like a flurry of activity in the technology of provision and dissemination of information. The list can be confusing, but there are software packages to help you decide which of the 60 biomedical databases will be most useful to you. Your librarian can help you here by accessing Table 5, Examples of databases that are available with general descriptions of the regions of interest covered by each. Database
Years covered
Areas of interest
Cochrane Library CDSR DARE CCTR ACPJCOD MEDLINE
No limits
All specialties. Includes the previous Pregnancy and Childbirth Reviews
1990 onwards 1966 onwards
EMBASE
1974 onwards
Biological Abstracts
1990 onwards
CINAHL Nursing and Midwifery Index CANCERLIT
1982 onwards 1994onwards
General medicine Electronic version of Index Medicus and International Nursing Index. Strong on North American literature Electronic version of Excerpta Medica. Strong on European studies and pharmacology literature Clinical and experimental medicine with biological science Nursing journals, chapters, books, theses Nursing and midwifery in the UK
1963 onwards
CAB HEALTH
1973 onwards
Science Citation Index Scientific and Technical Proceedings Current Contents
1981 onwards 1982 onwards Present
AIDSline PsycLit ASSIA Sociofile Bioethicsline PAlS Health Periodicals
1980 onwards 1974 onwards 1987 onwards 1974 onwards 1973 onwards 1970 onwards 1987 onwards
Healthplan
1986--1996 (previous 10 years)
Epidemiology, pathogenesis, immunology and treatment of cancer Environmental health, tropical medicine and nutrition including obstetrics and gynaecology General science and author citation Details of papers presented at 4000 conferences (including medical conferences) per year Contents pages of hundreds of journals in life sciences and clinical medicine updated weekly AIDS-related topics Psychology and related disciplines Social sciences Sociology Ethical issues Public and social policy. American bias Consumerism and public health, with summaries specifically for the lay person Administrative, financial, legal and ethical aspects of health care
This list is far from exhaustive and is simply a taster of the type of databases that are available. Your librarian will help you decide which is the most suitable for your needs and also alert you to any other locally available information sources. There are several with a UK focus, for example the King's Fund Library Database, Purchasing Innovations Database, NHS Database of economic evaluations and Project Register System.
FINDING THE EVIDENCE
563
an on-line service through, for example DataStar, called Cros search, which can rank the databases in order of priority for your search topic. To be an evidence-based practitioner, one needs to develop and hone one's searching technique. To become an efficient searcher, concentrate on a small nucleus of databases at first, to learn the few techniques that are necessary to be productive. Your librarian will be very helpful in showing you around the computer access route that your hospital library has in situ, explaining and demonstrating effective methods of using the appropriate databases and auditing your own first efforts at searching both for their inclusiveness (sensitivity) and relevance (precision). Frequently, you will discover that they may already have some search strategies stored as menu items, or they can help you do so, for recurrent use. There are 27 commercial medical products tailored for clinical use to access MEDLDIE. Haynes et al (1994b) compared the performance of each for userability. The important test is which one you, as an individual, find easiest to master. Those with the thesaurus stored electronically are especially useful. For example, our own library at the John Radcliffe Hospital, Oxford, has WinSPIRS, which has all the MeSH term information and search limitations on the screen, thus reducing the need to refer to the very large index catalogue. The latter is useful, however, for keeping up to date with any changes in MeSH terms, for example as new interventions are introduced and indexed. It also has several other databases (Biological Abstracts, PsycLit, and Sociofile and SERLINE) on the same menu. The other often-quoted database is the electronic version of Excerpta Medica (EMBASE). It covers 4700 journals with a strong coverage of the pharmacology, environmental health and European literature and an approximately 50% overlap with MEDLINE. It is at present less popular than MEDLINE because the interface is less easy to come to grips with, and in the UK it is accessed through the Joint Academic Network (JANET). In the UK, the British Medical Association allows free access to MEDLINE (the interface is CDP OVID) to all its members. With little expense (just a modem attachment and telephone call), you can carry out literature searches from home at any time of day. In the USA, the American College of Obstetricians and Gynecologists offers its members ACOGNET (a free communications network to its own publications on a 24-hour basis), but its librarians also offer complementary MEDLINE searches. However, to work beside your librarian and hence use your combined talents is usually a more productive activity, rather than just leaving a few details with your librarian and hoping that he or she will produce papers that look at the clinical problem from exactly the perspective you require. Having highlighted its usefulness, a sobering thought on MEDLINE is that it covers less than a quarter of the biomedical journals (3700). Even if we apply the optimally sensitive search for RCTs, which has been developed by Carol Lefebvre at the UK Cochrane Centre, only 50% of the RCTs known to be in MEDLINE would actually be picked up (Dickersin et al, 1995). (The figure is closer to 20% for experienced clinical searchers.) This poor 'hit' rate is thought to be due to three main problems. The first is poor indexing. 'Random allocation" and 'randomized controlled trial' were
564
I.Z. COOKE
first used as descriptor terms in 1975 and 1990 respectively, and it was not until 1991 that 'randomized controlled trial' was introduced as a publication type. Second, the index terms may be applied incorrectly by the professional indexers. That is annoying to the trialists, but in fact the third reason why RCTs are missed is that the trialists themselves do not clearly specify methods used within the study! There is little reason to expect any better return from other general databases, although they have yet to be scrutinized in the same manner. To improve this situation, MEDLINE have agreed to re-enter as an RCT (onto MEDLINE) any RCT retrieved by the electronic and hand searching efforts of the Cochrane Collaboration. To date, over 80 000 RCTs have been reclassified as such on MEDLINE. A R E A B S T R A C T S U S E F U L AS A DATA S O U R C E ? At this stage of the search, you will be left with a list of titles and abstracts. Should you find all these references in full text or can you do any further sifting with the abstracts themselves? Reading the abstracts can help you to slim down the final fist. Scan through for the specific subject; does the abstract look at the problem from the angle you are chasing? The methodology used (systematic review, RCT, cohort study, etc.) is often listed in the title or among the MeSH terms, if not in the abstract itself. Structured abstracts are usually more informative than the unstructured format as to the content of a paper. Abstracts are, however, prone to bias in that they often highlight the most positive aspects of a paper, so it is unwise to base your final decision on the information from abstracts alone.
RECORDING THE SEARCH STRATEGY Searching for the best evidence is of equal importance to the responsibility of evaluating the validity and results of the papers found. As such, it is imperative to outline the search terms and details of the databases you have used. This is not only a record for yourself, should you wish to repeat the search, but most importantly it allows others to scrutinize that search strategy for its completeness and soundness. S E C O N D A R Y PUBLICATIONS So far, this chapter has been aimed at finding evidence for specific clinical problems. You might ask whether there is any help out there for those who want to try to keep abreast of current literature. This is called 'current awareness'. If there are particular areas of interest in which you want to learn of each new publication as it arises, your librarian can set up a current awareness request for you. It means that an electronic search will be performed automatically for you, as regularly as you require, on these topics. If you prefer to continue browsing through journals, make
FIND1NG THE EVIDENCE
565
best use of that time on 'high-yield' journals: those which give the highest yield of good-quality studies on your topic. Remember, when something of interest catches your attention, you still have to validate the methods and results. There are now 'secondary' journals that pick out useful articles and those of particular relevance. In other words, you not only get the searching done for you but also the data are critically appraised. The aim of these journals is to 'read' the current literature for clinically important articles that are well executed and present them as a structured abstract with the results transferred to a clinically relevant format. Only 2% of the articles reviewed are viewed as being of appropriate quality to be considered. It is therefore little wonder that we feel swamped by the amount of literature we feel we should sift through when that sift has to filter out the other 98% of articles. At present there are only two such journals in press. The ACP Journal Club (published by the American College of Physicians) was the first of these and is made available in both paper and electronic format (ACPJCOD). Since 1990, it has scanned 50 journals for papers of specific interest to general physicians. However, the Evidence-based Medicine journal (produced by the British Medical Journal), in publication since 1995, with the same methods of selection and presentation, is aimed at a wider audience: general practice, medicine, surgery, obstetrics and gynaecology, paediatrics and psychiatry. A series of evidence-based journals for different specialties is in the pipeline from the publishing group Churchill Livingstone, one of which will be aimed at obstetricians and gynaecologists.
HOUSEKEEPING THE INFORMATION RETRIEVED The aim of the evidence-based practitioner is to use the problems you encounter daily as pointers for your continuing education. We all know that, whatever our good intentions, most of us are poor at remembering the evidence we have found when next we meet the same problem. While reference management software (for example Reference Manager, Procite or Endnote) is useful for ever-expanding reference lists, most of us would prefer to keep the clinical information we glean close to hand at all times. Professor Sackett has coined the phrase of a personal 'instant resource book'. Whenever you go to the trouble of searching and appraising the evidence for a particular question, summarize the search technique you used, list the articles retrieved and arrange the clinically relevant results obtained on a single A4 page marked with the date you did the search and a 'sell by' date to remind you of when it might be appropriate to check it out again. These summaries are called 'critically appraised topics' (CATs). A CAT-maker for therapy data, designed as an interactive piece of software, is available on the Centre for Evidence-based Medicine World Web page to help you create these resources for yourself. Surprisingly quickly, the file of
566
1. E. COOKE
evidence grows as a nucleus of information for your own practice: your own instant resource book relevant to the peculiarities of your practice.
SUMMARY When the Cochrane Collaboration fulfils its quest to organize the available evidence in a registry of trials and systematic reviews, it will mean that our task to find the best evidence will be much less daunting than it may at present seem. Meanwhile, to be evidence-based practitioners, we must learn the techniques that will make our search both comprehensive and efficient. To find this evidence requires: 1. 2. 3. 4.
the construction of a specific question based on the clinical problem; the design of a search strategy that is inclusive but practical; the search of the appropriate database(s); the scan of the retrieved abstracts for the most useful papers for critical appraisal.
Efficient searching for evidence does mean getting to grips with the manifold advantages that computerization can offer us. The use of a computer for this purpose needs, like any other new technique we take on, practice to gain competence and confidence. Within the UK, the National Health Service Research and Development Department has made the collation and dissemination of clinical evidence to practitioners a priority. Hence at present, there is an enormous effort within the UK to make sure that the evidence is in the clinical areas of need, in a format we can use it and available in a place where and when we want it, whether that is the ward or the general practitioner's surgery. The aim is to remove at least some of the barriers that prevent us from becoming individual evidencebased practitioners.
REFERENCES Altman DG (1994) The scandal of poor medical research. British Medical Journal 308: 283-284. Antman EM, Lau J, Kupelnick Bet al (1992) A comparison of results of meta-analyses of RCTs and recommendations of clinical experts. Treatments for myocardial infarction, Journal of the American Medical Association 268: 240-248. Cavell DG, Uman GC & Manning PR (1985) Information needs in office practice: are they being met? Annals of Internal Medicine 103: 596-599. Cochrane AL (1989) Effectiveness and Efficiency. London: British Medical JournallNuffield Provincial Hospitals Trust. Cochrane Library (1996) Cochrane Collaboration, Issue 1. Oxford: Update Software. Available from British Medical Journal Publishing Group, London. Davidoff E Haynes RB, Sackett DL et al (1995) Evidenced-based Medicine: a new journal to help doctors identify the information they need. British Medical Journal 310: 1085--1086. Dickersin K, Scherer R & Lefebvre C (t995) Identifying relevant studies for systematic reviews. In Chalmers I & Airman DG (eds) Systematic Reviews. London: British Medical Journal Publishing Group. Grimes DA (1991) Randomization controlled trials: "it ain't necessarily so'. Obstetrics and Gynecology 78: 703-704.
FINDING THE EVIDENCE
567
Grimes DA & Schulz KF (1996) Methodology citations and the quality of RCTs in obstetrics and gynecology. American Journal of Obstetrics and Gynecology 174(4): 1312-1315. Haynes RB (in press) Searching for the evidence. In Sackett DL, Richardson WS, Rosenberg WMC & Haynes RB (eds) Evidence-based Medicine: How to Practise and Teach EBM. London: Churchill-Livingstone. Haynes RB, Wilezynski NL, McKibbon KA et al (1994a) Developing optimal search for detecting clinically sound studies in MEDLINE. Journal of the Amer~Z'anMedical Information Association 1: 447-458. Haynes RB, Walker CJ, McKibbon KA et al (1994b) Performance of 27 MEDLINE systems tested by searches on clinical questions. Journal of the American Medical Information Association 1: 285-295. Irwig L, Macasill P, Glasziou P e t al (1995) Meta-analytic methods for diagnostic test accuracy. Journal of Clinical Epidemiology 48: 119-130. McDonough PG (1995) 'Leaky randomization': standard practice--but is it correct? Fertility and Sterility 64: 216-217. Mulrow CD (1995) Rationale for systematic reviews. In Chalmers I & Altman DG (eds) Systematic Reviews. London: British Medical Journal Publishing Group. Peto R, Pike MC, Armitage Pet al (1976) Design and analysis of randomized clinical trials requiring prolonged observation of each patient. I Introduction and design. British Journal of Cancer 34: 585-612. Sackett DL (in press) On the need for evidence-based medicine. In Sackett DL, Richardson WS, Rosenberg WMC & Haynes RB (eds) Evidence-based Medicine: How to Practise and Teach EBM. London: Churchill-Livingstone. Schulz KF, Chalmers I, Grimes DA et al (1994) Assessing the quality of randomization from reports of controlled trials published in obstetrics and gynecology journals. Journal of the American Medical Association 272: 125-128. Schulz KF, Chalmers I, Hayes RJ et al (1995) Empirical evidence of bias: dimensions of methodological quality associated with estimates of treatment effects in controlled trials. Journal o f the American Medical Association 273: 408-412.