- Email: [email protected]
The cochrane databases of randomized controlled trials and systematic reviews in pregnancy, childbirth and neonatal care
Semin Neonato11997; 2:211-219
The Cochrane databases of randomized controlled trials and systematic reviews in pregnancy, childbirth and neonatal care D a v i d J. H e n d e r s o n - S m a r t a and C a r o l i n e A n n e C r o w t h e r b
"NSW Centre for Perinatal Health Services Research, Queen Elizabeth II bLstihtte for Mothers and Infants, University of Sydney, Sydney, Attstralia bDepartment of Obstetrics and Gynaecology, University of Adelaide, Adelaide, A~tstraIia
Key words: databases, bibliographic, randomized controlled trials and standards, bias epidemiology, data collection methods, review literature, meta-analysis, obstetrics and standards, pregnancy, labour, infant, newborn, practice guidelines, quality of healthcare
Information from randomized controlled trials (RCTs) and systematic reviews of RCTs provide the strongest evidence about the benefits and harms of healthcare interventions. As such they are the comer stone of evidence-based healthcare and when integrated with the clinicians experience and skill, as well as the patients preferences, can best inform choices about care of individual patients. There are set standards to ensure that RCTs and systematic reviews are of high methodological quality so as to reduce the risk of bias and therefore ensure validity of the conclusions. This information needs to be up to date and available to clinicians in a way that text books and journals cannot be. The Cochrane Collaboration is dedicated to solve this through its electronic publication, the Cochrane Library, which has registers of RCTs and systematic reviews. The stimulus came from Archie Cochrane's challenge, quoted below, which was taken up by Iain Chalmers in the area of perinatal care. For care during pregnancy, childbirth and the neonatal period there are registers with nearly I0 000 RCTs, dating back over 50 years, and over 1000 systematic reviews, to help guide practice. The challenge ahead is to have these in an up to date form and available electronically to help clinicians to incorporate them into their practice, teaching and research.
It is surely a great criticism of our profession that we have not organized a critical summary, by speciality or subspeciality, adapted periodically, of all relevant randomised controlled trials . . . [1].
Most clinicians would agree that we should base our clinical decisions on the best evidence. Sackett and colleagues [2] have described evidence-based clinical care as 'the conscientious, explicit and judicious use of the current best evidence in making decisions about the care of individual patients'. This requires the 'integration of clinical expertise with external evidence from valid research on the effectiveness of different forms of care'. There are many sources of external evidence used in clinical Correspondence:D. I. tlenderson-Smart. NSW Centre for Perinatal Health Serx'ices Research, Queen Elizabeth II Institute for Mothers and Infants. BuildingDO2,Universityof Sydney,Sydney,NSW 2006, Australia (email: [email protected]).
1084-2756/97/030211 + 09 $12.00100
care and different questions require different research designs. Cohort studies are required for long-term prognosis, comparative studies to assess the accuracy of diagnostic tests, and qualitative methods to evaluate patient views of the care process. The strongest evidence for the effectiveness of therapeutic interventions comes from randomized controlled trials (RCTs) and systematic reviews of RCTs. Where these are available they should be used to inform therapeutic decisions. To be useful in clinical practice this information needs to be accessible in the clinical environment and be up to date. The Cochrane databases, with their registers of RCTs and systematic reviews of RCTs within an electronic publication, help provide us with the current best evidence with which to guide decisions about care for women and their newborn infants [3, 4]. © 1997 W.B. Saunders Company Ltd
D. J. Henderson-Smart & C. A. Crowther
212
Table 1. Quality of evidence ratings in clinical practice guidelines I II III-1 III-2 III-3
IV
Evidence obtained from a systematic review of all relevant randomized controlled trials. Evidence obtained from at least one properly designed randomized controlled trial. Evidence obtained from well-designed controlled trials without randomization. Evidence obtained from well-designed cohort or case control analytic studies preferably from more than one centre or research group. Evidence obtained from multiple time series with or without the intervention. Dramatic results in uncontrolled experiments (such as the results of the introduction of penicillin treatment in the 1940s) could also be regarded as this type of evidence. Opinions of respected authorities, based on clinical experience, descriptive studies, or reports of expert committees.
From National Health and Medical Research Council of Australia [81 Guidelines far the Development a~d bnpIemenlalian of Clinical Practice Guidelines. Canberra: Australian GovernmentPrintingServices,with permission.
In this article we aim to consider why randomized clinical trials and systematic reviews provide us with the best estimates of treatment effect about different forms of care in pregnancy, childbirth and the neonatal period, the historical development of the Cochrane databases and how we can incorporate the evidence they provide into our clinical practice, our teaching and our research.
Why randomized controlled trials and systematic reviews Randomized controlled trials In the hierarchy of research evidence the RCT is recognized as the most reliable way of comparing the effectiveness of different healthcare interventions (Table 1) [8]. Well designed and conducted RCTs minimize the risk of being misled about the effects of care by systematic errors or biases, and are large enough to avoid the risk of random error, or being misled that forms of care have similar effects when in fact they have different effects that are of clinical significance [5]. Assessing how adequately a RCT controls for biases is essential to judge if the results presented are valid. There are good quality trials and there are poor quality trials. Understanding which quality issues are important and how this may influence the treatment effect reported in a trial are fundamental aspects of critical appraisal [6, 7]. The greatest strength of RCTs lie in the unique process of randomisation, yet if carried out incorrectly this can cause bias [5, 7]. Appropriate randomisation eliminates biased treatment allocation and ensures that people at different prior risk, from both known and unknown factors, are
not allocated selectively to one of two different forms of care under evaluation. Evidence exists that in reports of randomized trials in which treatment allocation is poorly or not clearly concealed, the treatment effect seen is greater than in trials in which treatment allocation is adequately concealed [7]. The exact method of randomisation clearly needs to be known. Quasi-randomisation methods such as treatment allocation by date of birth or record numbers or alternate allocation, where treatment allocation is known before assignment occurs, are inferior methods. Trials using these methods have been shown to spuriously exaggerate the odds of a treatment effect by 41% [7]. Other aspects of trial design of importance in reducing the risk of bias include whether everyone entered into the trial is accounted for in the results, are people in the trial analysed according to the treatment group they were allocated to (intention to treat analysis), who is blinded to treatment allocation and are there any co-interventions likely to have influenced the outcomes measured? A rough guide to the reliability of the randomisation process is whether the baseline characteristics of the treatment groups generated by randomisation were similar. An example of the misleading results of small poor quality trials in perinatology, is that of phenobarbital prior to preterm birth to prevent neonatal intracranial haemorrhage [9]. Initial small studies with either inadequate concealment at randomisation or a large number of exclusions after randomisation, showed significant reductions in the rates of haemorrhage. Recent higher quality trials and one with an appropriately large sample size, have not confirmed the early optimism of 10 years ago. Published reports of trials need to provide sufficient information to allow a judgement to be
213
The Coehrane databases
Table 2. Guide for reporting randomized controlled trials Heading Title Abstract Introduction
Descriptor Identify the study as a randomized trial. Use a structured format. State prospectively defined hypothesis, clinical objectives, and planned subgroup or covariate analyses.
Methods Protocol
Describe: • Planned study population, together with inclusion/exclusioncriteria. • Primary and secondary outcome measure(s) and the minimum important difference(s)and indicate how the target sample size was projected. • Rationale and methods for statistical analyses, detailing main comparative analyses and whether they were completed on an intention-to-treat basis. • Prospectively defined stopping rules (if warranted). Describe: Assignment • Unit of randomization (e.g. individual, cluster, geographic). • Method used to generate the allocation schedule. • Method of allocation concealment and timing of assignment • Method to separate the generator from the executor of assignment. Masking (blinding) Describe mechanism (e.g. capsules, tablets); similarity of treatment characteristics (e.g. appearance, taste); allocation schedule control (location of code during trial and when broken); and evidence for successful blinding among participants, person doing intervention, outcome assessors, and data analysts. Provide a trial profile (figure) summarizing participant flow, numbers and timing of random Participant flow assignment, interventions and measurements for each randomized group. and follow-up Results • State estimated effect of intervention on primary and secondary outcome measures, including a Analysis point estimate and measure of precision (confidenceinterval). • State results in absolute numbers when feasible (e.g. 10120, not 50%). • Present summary data and appropriate descriptive and inferential statistics in sufficient detail to permit alternative analyses and replication. • Describe prognostic variables by treatment group and any attempt to adjust for them. • Describe protocol deviations from the study as planned, together with the reasons. • State specific interpretation of study findings, including sources of bias and imprecision (internal Comment validity) and discussion of external validity, including appropriate quantitative measures when possible. • State general interpretation of the data in light of the totality of the available evidence. Modified from Begget al [1I] ]AMA 1996; 276: 637-639, with permission.
made about the validity of the results. We need to be confident the size and direction of the treatment effect reported represents an unbiased estimate of the true treatment effect [10]. Several guidelines for the reporting of randomized clinical trials have been prepared and a unified C O N S O R T statement recently released [II]. In addition to a flow chart describing patient progress through the trial, the key items that need to be included in trial reports are given in Table 2. These items can form a checklist for critical appraisal of reports of trials. The CONSORT statement will be widely adopted by journals and should enhance the standards of
reporting of trials. Adequate reporting of RCTs is essential, given that they are the best w a y to evaluate alternative forms of care, may be combined in a systematic review and are able to establish cause and effect and so provide direct information about healthcare.
Systematic reviews Systematic reviews, with their explicit methodology, aim to improve the reliability of treatment recommendations by reducing the likelihood of
214
bias and increasing the power and the precision in estimating treatment effects [12]. Decisions about clinical care should be based on the synthesis of all existing relevant information. This quantitative synthesis is known as meta-analysis. As with primary research, where extensive efforts are usually made in design and conduct of a trial to avoid bias and random error, scientific methods need to be rigorously applied to the identification, assessment and synthesis of information within a systematic review. Until recently medical reviews did not routinely use such methods [13] and were thus at risk of making biased recommendations. The literature is replete with examples where failure to apply systematic review methods has resulted in delayed introduction into widespread clinical use of effective, and at times lifesaving, treatments. For similar reasons, worthless or indeed harmful practices have been allowed to continue. In perinatal medicine, the delay in synthesizing the information available in a systematic review of the effects of antenatal corticosteroids (first done in 1989) delayed the recognition of the beneficial effects of the administration of antenatal corticosteroids for ihe infant born preterm [14]. This failure to initially prepare and then be aware of and use the evidence from this systematic review has denied many babies the chance of life and has increased the risk of long-term disability in survivors. Good quality systematic reviews are essential to ensure that evidence from primary research trials can be translated promptly into improved health outcomes. Systematic reviews can help with rationalisation of the use of resources, can guide the providers and users of healthcare and can establish priorities for healthcare research. As with individual trials, assessing the validity of a systematic review is fundamental in assessing whether the results are valid. Several checklists for appraisal of a systematic review article have been prepared [15, 16, 17]. The key items to appraise are summarized in Table 3. These include checking whether the questions for review are clearly defined, the identification, selection and appraisal of studies is appropriate, data collection and synthesis are adequate and the conclusions are valid. The Cochrane Collaboration aims to produce quality systematic reviews on the effects of healthcare and has a commitment to regularly update the reviews to ensure the latest evidence is incorporated and accessible via the Cochrane Library
[18].
D.J. Henderson-Smart & C. A. Crowther
History of the Cochrane databases of RCTs and systematic reviews This section will briefly present the major steps taken to develop a comprehensive register of RCTs and to establish regularly updated systematic reviews of the effects of perinatal care. The most up-to-date register of RCTs and systematic reviews in pregnancy, childbirth and neonatal care, are published in the Cochrane Library, a computerized database released quarterly that has pregnancy and childbirth [19] and neonatal [20] modules. The perinatal field has led the way in trial registration, systematic reviews and database development. So how did perinatal medicine become so fortunate?
Register of randomized controlled trials Archie Cochrane recognised the importance of randomized trials in reducing the risk of bias when assessing the effects of care and the relevance of evidence derived from randomized trials for the health services [21]. Profoundly influenced by Cochrane's challenges, lain Chalmers, at the National Perinatal Epidemiology Unit in Oxford, in 1973 began to develop a register of all controlled trials in perinatal care. Electronic searching and hand searching of journals was carried out retrospectively to 1941. This Herculean task to identify all relevant information, both published and unpublished, was carried out by a dedicated team of researchers over the next 12 years. This Classified Bibliography of Controlled Trials in Perinatal Medicine 1940---I984 was first published, in book form, in 1985 [22]. This register was put into electronic form in 1988, the Oxford Database of Perinatal Trials (ODPT). The database listed published, unpublished, planned and ongoing trials and was published 6-monthly until 1992 [23]. The ongoing prospective searching for relevant trials is maintained now by the pregnancy and childbirth and neonatal collaborative review groups within the Cochrane Collaboration.
Systematic reviews With the relevant randomized clinical trial evidence identified, systematic reviews in perinatal care were prepared and first published in ODPT. It was
The Cochrane d a t a b a s e s
215
Table 3. Checklist for evaluating systematic reviews Problem Formulation Are review questions well formulated with specified key components? Were major changes in review questions avoided during the review process? Study Identification Is there a thorough search for relevant data using appropriate sources? Are there unbiased explicit searching strategies that are appropriately matched to the question? Study Selection Are appropriate inclusion and exclusion criteria used to select articles? Are selection criteria applied in a manner that limits bias? Are major changes in selection criteria avoided during the review process? Appraisal of Studies Is the validity of individual studies addressed in a reliable manner7 Are important parameters (e.g. setting, study population, study design) that could affect study results systematically addressed? Data Collection Is there a minimal amount of missing information regarding outcomes and other variables considered key to interpretation of results? Data Synthesis Are important parameters, such as study designs, considered in the synthesis? Are reasonable decisions made concerning whether and how to combine data? Are results sensitive to changes in the way the analysis was done? (Different statistical approaches; including/excluding unpublished or 'lesser quality" data) Are results precise? Discussion Are limitations of studies and the review process stated? Are review findings integrated within the context of relevant indirect evidence? Conclusions Are conclusions supported with valid studies that showed consistent, large, and significant effects? Are plausible competing explanations of observed effects addressed? Is evidence appropriately interpreted as inconclusive (no evidence of effect) or as showing a particular strategy did not work (evidence of no effect)? From CochraneCollaboration1996,with permission.
unique in being able to incorporate newly prepared systematic reviews and previous systematic reviews that had been updated as new trial evidence became available. Regular updating of systematic reviews aims to shorten the time from the publication of new evidence to its clinical implementation, by making the information easily accessible. Based on ODPT, books were published including Effective Care in Pregnancy and Childbirth (ECPC) [24], Guide to ECPC [25] and Effective Care of the Nezoborn hzfant [26]. The ECPC books have been described as 'the most advanced current example of
a basis for practising medicine founded both on empirical evidence and theory' [27]. In the foreword to ECPC, Archie Cochrane stated that the systematic reviews of randomized trials of obstetric practice presented in the book were a 'new achievement' and he expressed the hope that 'it will be widely copied by other medical specialties' [28]. The establishment of the Cochrane Collaboration in 1992, an international organization dedicated to preparing, maintaining and disseminating systematic reviews on the effects of healthcare, increases the likelihood Cochrane's hope will be realized and evidence-based healthcare
216
will become a reality for all. The reviews on pregnancy and childbirth subsequently became available in the Cochrane Collaboration Pregnancy and Childbirth Database (CCPC) which was last published in 1995 [29] and these are currently being updated in the new Cochrane format for the Cochrane Library [18]. The Cochrane systematic reviews are published quarterly on disk or CD-ROM and will soon be accessible via the internet [18]. The majority of the Cochrane systematic reviews in perinatal care within the Cochrane Library are produced by reviewers who belong to the pregnancy and childbirth and neonatal Cochrane collaborative review groups. Use of these rich information resources has been recommended for clinicians involved in perinatal care [3, 4].
Perinatal databases in clinical practice Use in direct patient care Although use of the databases at the 'bedside' has been discussed in theory [30] there is little evidence that they would be consulted by the busy clinician at the moment of patient contact, rather their role is in informing therapeutic policy for that individual clinician or for the clinical unit in which they work. For controversial areas, and particularly for audit such as in perinatal morbidity/mortality meetings, the availability of the databases is valuable in weighing the range of evidence available in any given therapeutic situation. In many such meetings there has been a swing away from opinion-based discussion based on experience, towards evidencebased decisions and this is greatly assisted by the availability of evidence from perinatal databases. We are not aware of any systematic evaluation of the use of perinatal databases at the 'bedside'.
Are perinatal databases used in clinical practice? The key issues here are knowledge about the existence of databases, their availability in the clinical setting and their use in clinical care decisions. In a 1993 survey of obstetric units in the UK [31] ECPC or ODPT were available in 55% of district general hospitals and 88% of teaching hospitals. UK users of CCPC in 1994 most c o m -
D.J. Henderson-Smart & C. A. Crowther
monly used it to improve personal knowledge and that of others, but also for guiding research, developing clinical guidelines and informing audit [32]. In Australia, Jordens et al [33] surveyed all neonatologists and a random sample of practising obstetricians on their use of systematic reviews. Of neonatologists, 72% said they used systematic reviews, principally those published in Effective Care of the Newborn hTfant [26]. In addition to computer literacy and research experience, a factor significantly associated with the use of systematic reviews was attendance at the Australian Perinatal Society meetings. Since 1990 such meetings have promoted RCTs and systematic reviews though guest speakers and workshops on critical appraisal, systematic reviews and the design of RCTs. Fewer obstetricians (44%) mentioned using systematic reviews and those that did predominantly used the electronic form, CCPC.
Use in the development of guidelines Clinical practice guidelines are being used increasingly [34]. They have been shown to change clinician behaviour and to improve patient outcomes [35]. An important factor in effective guidelines is the quality of the evidence on which they are based. In the past expert medical opinion was used to develop guidelines but such opinion may not be correct [36] and even when based on literature this was often reviewed in an unsystematic way that led to biases [10, 35]. To be valid, guidelines need to be based on the strongest evidence available, such as that from systematic reviews or randomized controlled trials (Table 1) and recommendations tempered by the level of evidence on which they are based [37, 38]. The Cochrane databases in perinatal care can help make the strongest evidence available. In Australia the National Health and Medical Research Council (NHMRC) has funded a Cochrane Centre to contribute to trial registration and development of systematic reviews. The NHMRC also has commissioned the development of guidelines including those for 'Care around Preterm Birth' based on the pregnancy and childbirth and neonatal modules of the database.
Use by funders and providers of health services In the UK (Effective Health Care 1994) and USA [34] the private and government health service
The Cochrane databases
providers have indicated that the most efficient way to fund health services is to concentrate efforts on forms of care that have been shown to be effective [38]. In a 1994 UK survey of people and organizations who had a copy of CCPC 15% of responders were purchasers of healthcare [32].
Perinatal databases in teaching In both undergraduate and postgraduate clinical education, as well as in continuing education, the databases can be used to illustrate the value of scientifically valid external information to assist in clinical care decisions [30]. These databases play an important role in curricula that are incorporating the principles of evidence-based medicine in their teaching programs. At McMaster University in Canada such an approach has been shown to produce medical graduates who are more knowledgable about current therapeutic guidelines and have greater retention of knowledge since graduation [39]. In neonatology, the databases have been used to underpin an undergraduate course at the University of Sydney, in which the databases are available 'live' during problem-based learning sessions. The ready access, completeness and graphic displays are particularly helpful in illustrating the relative effectiveness of different forms of clinical care. Students who are developing their problem-based tutorials are directed to the databases as resource material and encouraged to use their graphic displays in illustrating the effects of treatments [40]. The Royal Australian College of Obstetrics and Gynaecology has included the use of evidence from the databases in their assessments of candidates for the College. This has the important effect of avoiding a clash between the opinion of a candidate and an examiner, since the examiners have been tutored to know what evidence is available and to base their examination on that evidence. The training program for candidates includes a distance learning package with a clinical epidemiology module that incorporates the use of perinatal databases [41].
Perinatal databases in research Use of the trial register and the database of systematic reviews would seem essential prior to
217
development of any new study evaluating a therapeutic intervention. This would assist in avoiding unnecessary duplication of studies and where more studies are indicated previous trials will help in the design of new trials. Systematic reviews highlight the gaps in evidence about important outcomes and the need for additional randomized controlled trials. Reviews in the Cochrane databases are particularly valuable in that the strength and weaknesses of the various trials have been highlighted and an 'implications for research' section points towards important questions and issues in trial design needing to be addressed. It could be argued that funding bodies should require a systematic review to be part of the background to any research application for a project evaluating a therapeutic intervention. There has been a call for ethics committees also to require this [42].
The future for databases in perinatal care Prospective trial registration Trials in which statistically significant differences in outcome have not been detected are less likely to be published than those showing such differences [42, 43] and could bias any review. The Cochrane Collaboration continues to be involved in efforts to establish systems for registering RCTs at inception [42]. Registers of planned, ongoing and completed but not yet published trials, also provide valuable additional information to inform research and to promote collaboration.
Getting the systematic reviews done and kept up to date In the text Effective Care of the Nezoborn hlfant [26] 1234 RCTs were incorporated into 501 systematic reviews. Although many areas have not changed because no new trials have appeared, the newest and most controversial areas are out of date. Electronic publication, as in the Cochrane Library, is the only effective way of providing up to date systematic reviews and a high priority for the Neonatal Review Group is to do just that. The Pregnancy and Childbirth Review Group have been ahead of the field in terms of electronic publication of systematic reviews, first in the
218
OxfoM Database of Perinatal Trials and then the Cochrane Collaboration Pregnancy and Childbirth module, which had over 600 reviews. The latter, however, has not been published since 1995 due to a switch in priorities to update and convert the reviews to the new Cochrane format. At the end of 1996 that task was about 20% complete.
How can you play a role in the Cochrane collaboration? With more trained people doing reviews the process of getting the reviews done and keeping them up to date will be more efficient. Anyone interested in contributing to this enormous task can contact their local Cochrane Centre or a Collaborative Review Group via the contact addresses in the Cochrane Library. Information about training, available review topics and help finding co-authors will be provided. Enthusiasm for hard unpaid work and a commitment to follow through on updates are the main initial qualifications required.
D . J . Henderson-Smart & C. A. Crowther
MJNC (eds) Effective Care in Pregnancy and Childbirth. Vol. I. Oxford: Oxford University Press, 1989; 3-38. 6 Sackett DL, Haynes RB, Guyatt GH et al. Clinical Epidemiology: A Basic Sciencefor ClinicalMedicine. 2nd edn Boston: Little Brown Company, 199I. 7 Schultz KF Chalmers I, Hayes R et al. Empirical evidence of bias. Dimensions of methodological quality associated with estimates of treatment effects in controlled trials. ]AMA 1995; 273: 408--412. 8 National Health and Medical Research Council of Australia, Quality of Care and Health Outcomes Committee. Guidelines for the Development and hnplementation of Clinical Practice Guidelines. Canberra: Australian Government Printing Services, 1995. 9 Crowther CE, Henderson-Smart DJ. Phenobarbital prior to preterm birth. In: Neilson JP, Crowther CA, Hodnett ED, Hofmeyr GJ, Keirse MJNC, Renffew MJ (eds) Preg-
10
11
12
Communicating the results to clinicians and consumers
13 14
The need for databases to provide easy access to the best evidence for the effectiveness of different forms of care is clear. What is less clear is how to present this information to each of the groups (funders, providers and consumers) in a format that is useful; how to incorporate information into clinical care decisions about specific groups of patients or individuals [38]. It is here that clinical expertise must play a role lest we become swamped in a 'tyranny of information' [2].
References 1 Cochrane AL. 1931-1971: a critical review of the medical profession. In: Medicinefor the Year 2000. London, Office of Health Economics, 1979, 2-11. 2 Sackett DL. Evidence based medicine: what it is and what it is not. Br Med ] 1996; 312: 71-72. 3 Enkin MW. Systematic summaries and dissemination of evidence: the Cochrane pregnancy and childbirth database. Semin Perinatol 1995; 19: 155-160. 4 0 h l s s o n A. Randomized controlled trials and systematic reviews: a foundation for evidence-based practice. Aeta Paedhdr 1996; 8 5 : 647--655. 5 Chalmers I. Evaluating the effects of care during pregnancy and childbirth. In: Chalmers I, Enkin M, Keirse
15 16
17
18
19
20
nancy a~d Childbirth Module of the Cochrane Database of Systematic Reviews (Available in: the Cochrane Library database on disc and CD-ROM). The Cochrane Collaboration; Oxford: Update Software; 1996, Issue 4. Guyatt GH, Sackett DL, Cook DJ. Users guide to the medical literature II. How to use an article about therapy or prevention. A. Are the results of the study valid? ] Am Med Assoc 1993; 270: 2598-2601. Begg C, Cho M, Eastwood S, et aL Improving the quality of reporting of randomized controlled trials: the CONSORT statement, lAMA 1996; 276: 637--639. Mulrow C. Rationale for systematic reviews. Br Med ] 1994; 309: 597-599. Mulrow C The medical review article: state of the science. Ann hltenl Med 1987; 104: 485-488. Crowley P. Corticosteroids prior to preterm delivery. In: Neilson JP, Crowther CA, Hodnett ED, Hofmeyr GJ, Keirse MJNC, Renffew MJ (eds) Pregnancy and Childbirth Module of the Cochrane Database of Systematic Reviews. (Available in the Cochrane Library.) The Cochrane Collaboration; Oxford: Update Software; 1996, Issue 3. Oxman AD. Checklists for review articles. Br MedJ 1994; 3 0 9 : 648--651. Mulrow CD, Oxman AD (eds) Cochrane Collaboration Handbook. (Available in: The Cochrane Library (database on disc and CD-ROM).) The Cochrane Collaboration; Oxford: Update Software 1996, Issue 3. Thacker SB, Peterson HB, Stroup DF. Metaanalysis for the obstetrician-gynecologist. Am ] Obstet Gynecol 1996; 1 7 4 : 1403-1407. Cochrane Collaboration. Cochrane Database of Systematic Reviews. (Available in the Cochrane Library (database on disc and CD-ROM).) The Cochrane Collaboration, Oxford: Update Software; 1996, Issue 3. Neilson JP, Crowther CA, Hodnett ED, Hofmeyr GJ, Keirse MJNC, Renfrew MJ (eds) Pregnancy and Childbirth Module of the Cochrane Database of Systematic Reviews. (Available in the Cochrane Library (database on disc and CD-ROM).) The Cochrane Collaboration, Oxford: Update Software; 1996, Issue 3. Sinclair J, Bracken M, Soll RF, Horbar JD (eds) Neonatal Module of the Cochrane Database of Systematic Reviews. (Available in the Cochrane Library (database on disc and CD-ROM).) The Cochrane Collaboration; Oxford: update Software; 1996, Issue 3.
The Cochrane databases
AL. Effectiveness and Efficiency: Random Reflections on Health Services. London: Nuffield Provincial Hospital Trust, 1972. National Perinatal Epidemiology Unit. A Classified Bibliography of Controlled Trials in Perinatal Medicine 1940-1984. Oxford: Oxford University Press, 1985. Chalmers I (ed.) Oxford Database of Perinatat Trials. Version 1.2, disk issue 7 (spring), Oxford: Oxford University Press 1992. Chalmers I, Enkin MW, Keirse MJNC (eds) Effective Care in Pregnancy and Childbirth. Oxford: Oxford University Press, 1989. Enkin M, Keirse M, Renffew M & Neilson J (eds) A Guide to Effective Care in Pregnancy and Childbirth. Oxford: Oxford University Press, 1995. Sinclair JC, Bracken MB (eds) Effective Care of the Newborn Infant. Oxford: Oxford University Press, 1992. Mosteller F. Database medicine in the light of ECPC. Milbank Q 1993; 71: 523-533. Cochrane AL. Forward to effective care in pregnancy and childbirth. Chalmers I, Enkin M, Keirse MJNC (eds) Effective Care in Pregnancy and Childbirth. Vol. 1. Oxford: Oxford University Press, 1989. Enkin MW, Keirse MJNC, Renfrew MJ & Neilson JP (eds) Pregnancy and Childbirth Module of the Cochrane Database of Systematic Reviews. (Available in the Cochrane Library (database on disc and CD-ROM).) The Cochrane Colloberation; Oxford: Update Software 1995, Issue 2. Sackett DL. Applying overviews and meta-analyses at the bedside. ] Clin Epidemiol 1995; 48: 61-66. Paterson-Brown S, Fisk N & Wyatt JC. Uptake of meta-analytical overviews of effective care in English obstetric units. Br ] Obstet Gynaecol 1995; 102: 297-30I. Hyde C. Who uses the Cochrane Pregnancy and Childbirth Database? Br Med ] 1995; 310:1140-1141 Jordens C, Hawe P, Irwig I, Henderson-Smart D, Ryan M, Donoughue D, Gabb R, Fraser I. The use of
219
21 Cochrane
22
23
24
25
26 27 28
29
30 31
32 33
34
35
36
37
38
39
40
41
42 43
systematic reviews by Australian neonatologists and obstetricians. Proceedings 4th hlternational Cochrane Colloquium. Adelaide, 1996, A12. Woolf SH. Practice guidelines: a new reality in medicine. 1. Recent developments. Arch hzlern Med 1992; 150: 1811-1818. Effective Health Care. Implementing clinical practice guidelines: can guidelines be used to improve practice? Bulletin no.8, Leeds: University of Leeds, 1994 Antman EM, Lau J, Kupelnick B, Mosteller F, Chalmers TC. A comparison of results of meta-analyses of randomized control trials and recommendations of clinical experts. Treatments for myocardial infarction. JAMA 1992; 268: 240-248. Hayward SA, Wilson MC, Tunis SR, Bass EB, Guyatt G; for the Evidence-based Medicine Working Group. Users' guide to the medical literature. VII. How to use clinical practice guidelines. A. Are the recommendations valid? JAMA 1995; 274: 570-574. Invig L, Zwarenstein M, Zwi A, Chalmers I. A flow diagram to help select interventions and research for healthcare. Bull WHO (in press) Shin JH, Flaynes RB & Johnston ME. Effect of problembased, self directed undergraduate education on life-long learning. Can Med Assoc ] 1993; 148: 969-976. Jeffery HE, Henderson-Smart DJ & Hill DA. Competencybased learning in neonatology. Med Ed 1996; 30: 440444. Royal Australian College of Obstetricians and Gynaecologists. Training Program Manual. Melbourne: RACOG, 1995. Savulescu J, Chalmers I, Blunt J. Are ethics committees behaving ethically? Br Med ] 1996; 313: 1390-1393. Dickersin K, Min YI & Meinert CL. Factors influencing publication of research results: follow-up of applications submitted to two institutional review boards. ]AMA 1992; 263: 374-378.
The Cochrane databases of randomized controlled trials and systematic reviews in pregnancy, childbirth and neonatal care D a v i d J. H e n d e r s o n - S m a r t a and C a r o l i n e A n n e C r o w t h e r b
"NSW Centre for Perinatal Health Services Research, Queen Elizabeth II bLstihtte for Mothers and Infants, University of Sydney, Sydney, Attstralia bDepartment of Obstetrics and Gynaecology, University of Adelaide, Adelaide, A~tstraIia
Key words: databases, bibliographic, randomized controlled trials and standards, bias epidemiology, data collection methods, review literature, meta-analysis, obstetrics and standards, pregnancy, labour, infant, newborn, practice guidelines, quality of healthcare
Information from randomized controlled trials (RCTs) and systematic reviews of RCTs provide the strongest evidence about the benefits and harms of healthcare interventions. As such they are the comer stone of evidence-based healthcare and when integrated with the clinicians experience and skill, as well as the patients preferences, can best inform choices about care of individual patients. There are set standards to ensure that RCTs and systematic reviews are of high methodological quality so as to reduce the risk of bias and therefore ensure validity of the conclusions. This information needs to be up to date and available to clinicians in a way that text books and journals cannot be. The Cochrane Collaboration is dedicated to solve this through its electronic publication, the Cochrane Library, which has registers of RCTs and systematic reviews. The stimulus came from Archie Cochrane's challenge, quoted below, which was taken up by Iain Chalmers in the area of perinatal care. For care during pregnancy, childbirth and the neonatal period there are registers with nearly I0 000 RCTs, dating back over 50 years, and over 1000 systematic reviews, to help guide practice. The challenge ahead is to have these in an up to date form and available electronically to help clinicians to incorporate them into their practice, teaching and research.
It is surely a great criticism of our profession that we have not organized a critical summary, by speciality or subspeciality, adapted periodically, of all relevant randomised controlled trials . . . [1].
Most clinicians would agree that we should base our clinical decisions on the best evidence. Sackett and colleagues [2] have described evidence-based clinical care as 'the conscientious, explicit and judicious use of the current best evidence in making decisions about the care of individual patients'. This requires the 'integration of clinical expertise with external evidence from valid research on the effectiveness of different forms of care'. There are many sources of external evidence used in clinical Correspondence:D. I. tlenderson-Smart. NSW Centre for Perinatal Health Serx'ices Research, Queen Elizabeth II Institute for Mothers and Infants. BuildingDO2,Universityof Sydney,Sydney,NSW 2006, Australia (email: [email protected]).
1084-2756/97/030211 + 09 $12.00100
care and different questions require different research designs. Cohort studies are required for long-term prognosis, comparative studies to assess the accuracy of diagnostic tests, and qualitative methods to evaluate patient views of the care process. The strongest evidence for the effectiveness of therapeutic interventions comes from randomized controlled trials (RCTs) and systematic reviews of RCTs. Where these are available they should be used to inform therapeutic decisions. To be useful in clinical practice this information needs to be accessible in the clinical environment and be up to date. The Cochrane databases, with their registers of RCTs and systematic reviews of RCTs within an electronic publication, help provide us with the current best evidence with which to guide decisions about care for women and their newborn infants [3, 4]. © 1997 W.B. Saunders Company Ltd
D. J. Henderson-Smart & C. A. Crowther
212
Table 1. Quality of evidence ratings in clinical practice guidelines I II III-1 III-2 III-3
IV
Evidence obtained from a systematic review of all relevant randomized controlled trials. Evidence obtained from at least one properly designed randomized controlled trial. Evidence obtained from well-designed controlled trials without randomization. Evidence obtained from well-designed cohort or case control analytic studies preferably from more than one centre or research group. Evidence obtained from multiple time series with or without the intervention. Dramatic results in uncontrolled experiments (such as the results of the introduction of penicillin treatment in the 1940s) could also be regarded as this type of evidence. Opinions of respected authorities, based on clinical experience, descriptive studies, or reports of expert committees.
From National Health and Medical Research Council of Australia [81 Guidelines far the Development a~d bnpIemenlalian of Clinical Practice Guidelines. Canberra: Australian GovernmentPrintingServices,with permission.
In this article we aim to consider why randomized clinical trials and systematic reviews provide us with the best estimates of treatment effect about different forms of care in pregnancy, childbirth and the neonatal period, the historical development of the Cochrane databases and how we can incorporate the evidence they provide into our clinical practice, our teaching and our research.
Why randomized controlled trials and systematic reviews Randomized controlled trials In the hierarchy of research evidence the RCT is recognized as the most reliable way of comparing the effectiveness of different healthcare interventions (Table 1) [8]. Well designed and conducted RCTs minimize the risk of being misled about the effects of care by systematic errors or biases, and are large enough to avoid the risk of random error, or being misled that forms of care have similar effects when in fact they have different effects that are of clinical significance [5]. Assessing how adequately a RCT controls for biases is essential to judge if the results presented are valid. There are good quality trials and there are poor quality trials. Understanding which quality issues are important and how this may influence the treatment effect reported in a trial are fundamental aspects of critical appraisal [6, 7]. The greatest strength of RCTs lie in the unique process of randomisation, yet if carried out incorrectly this can cause bias [5, 7]. Appropriate randomisation eliminates biased treatment allocation and ensures that people at different prior risk, from both known and unknown factors, are
not allocated selectively to one of two different forms of care under evaluation. Evidence exists that in reports of randomized trials in which treatment allocation is poorly or not clearly concealed, the treatment effect seen is greater than in trials in which treatment allocation is adequately concealed [7]. The exact method of randomisation clearly needs to be known. Quasi-randomisation methods such as treatment allocation by date of birth or record numbers or alternate allocation, where treatment allocation is known before assignment occurs, are inferior methods. Trials using these methods have been shown to spuriously exaggerate the odds of a treatment effect by 41% [7]. Other aspects of trial design of importance in reducing the risk of bias include whether everyone entered into the trial is accounted for in the results, are people in the trial analysed according to the treatment group they were allocated to (intention to treat analysis), who is blinded to treatment allocation and are there any co-interventions likely to have influenced the outcomes measured? A rough guide to the reliability of the randomisation process is whether the baseline characteristics of the treatment groups generated by randomisation were similar. An example of the misleading results of small poor quality trials in perinatology, is that of phenobarbital prior to preterm birth to prevent neonatal intracranial haemorrhage [9]. Initial small studies with either inadequate concealment at randomisation or a large number of exclusions after randomisation, showed significant reductions in the rates of haemorrhage. Recent higher quality trials and one with an appropriately large sample size, have not confirmed the early optimism of 10 years ago. Published reports of trials need to provide sufficient information to allow a judgement to be
213
The Coehrane databases
Table 2. Guide for reporting randomized controlled trials Heading Title Abstract Introduction
Descriptor Identify the study as a randomized trial. Use a structured format. State prospectively defined hypothesis, clinical objectives, and planned subgroup or covariate analyses.
Methods Protocol
Describe: • Planned study population, together with inclusion/exclusioncriteria. • Primary and secondary outcome measure(s) and the minimum important difference(s)and indicate how the target sample size was projected. • Rationale and methods for statistical analyses, detailing main comparative analyses and whether they were completed on an intention-to-treat basis. • Prospectively defined stopping rules (if warranted). Describe: Assignment • Unit of randomization (e.g. individual, cluster, geographic). • Method used to generate the allocation schedule. • Method of allocation concealment and timing of assignment • Method to separate the generator from the executor of assignment. Masking (blinding) Describe mechanism (e.g. capsules, tablets); similarity of treatment characteristics (e.g. appearance, taste); allocation schedule control (location of code during trial and when broken); and evidence for successful blinding among participants, person doing intervention, outcome assessors, and data analysts. Provide a trial profile (figure) summarizing participant flow, numbers and timing of random Participant flow assignment, interventions and measurements for each randomized group. and follow-up Results • State estimated effect of intervention on primary and secondary outcome measures, including a Analysis point estimate and measure of precision (confidenceinterval). • State results in absolute numbers when feasible (e.g. 10120, not 50%). • Present summary data and appropriate descriptive and inferential statistics in sufficient detail to permit alternative analyses and replication. • Describe prognostic variables by treatment group and any attempt to adjust for them. • Describe protocol deviations from the study as planned, together with the reasons. • State specific interpretation of study findings, including sources of bias and imprecision (internal Comment validity) and discussion of external validity, including appropriate quantitative measures when possible. • State general interpretation of the data in light of the totality of the available evidence. Modified from Begget al [1I] ]AMA 1996; 276: 637-639, with permission.
made about the validity of the results. We need to be confident the size and direction of the treatment effect reported represents an unbiased estimate of the true treatment effect [10]. Several guidelines for the reporting of randomized clinical trials have been prepared and a unified C O N S O R T statement recently released [II]. In addition to a flow chart describing patient progress through the trial, the key items that need to be included in trial reports are given in Table 2. These items can form a checklist for critical appraisal of reports of trials. The CONSORT statement will be widely adopted by journals and should enhance the standards of
reporting of trials. Adequate reporting of RCTs is essential, given that they are the best w a y to evaluate alternative forms of care, may be combined in a systematic review and are able to establish cause and effect and so provide direct information about healthcare.
Systematic reviews Systematic reviews, with their explicit methodology, aim to improve the reliability of treatment recommendations by reducing the likelihood of
214
bias and increasing the power and the precision in estimating treatment effects [12]. Decisions about clinical care should be based on the synthesis of all existing relevant information. This quantitative synthesis is known as meta-analysis. As with primary research, where extensive efforts are usually made in design and conduct of a trial to avoid bias and random error, scientific methods need to be rigorously applied to the identification, assessment and synthesis of information within a systematic review. Until recently medical reviews did not routinely use such methods [13] and were thus at risk of making biased recommendations. The literature is replete with examples where failure to apply systematic review methods has resulted in delayed introduction into widespread clinical use of effective, and at times lifesaving, treatments. For similar reasons, worthless or indeed harmful practices have been allowed to continue. In perinatal medicine, the delay in synthesizing the information available in a systematic review of the effects of antenatal corticosteroids (first done in 1989) delayed the recognition of the beneficial effects of the administration of antenatal corticosteroids for ihe infant born preterm [14]. This failure to initially prepare and then be aware of and use the evidence from this systematic review has denied many babies the chance of life and has increased the risk of long-term disability in survivors. Good quality systematic reviews are essential to ensure that evidence from primary research trials can be translated promptly into improved health outcomes. Systematic reviews can help with rationalisation of the use of resources, can guide the providers and users of healthcare and can establish priorities for healthcare research. As with individual trials, assessing the validity of a systematic review is fundamental in assessing whether the results are valid. Several checklists for appraisal of a systematic review article have been prepared [15, 16, 17]. The key items to appraise are summarized in Table 3. These include checking whether the questions for review are clearly defined, the identification, selection and appraisal of studies is appropriate, data collection and synthesis are adequate and the conclusions are valid. The Cochrane Collaboration aims to produce quality systematic reviews on the effects of healthcare and has a commitment to regularly update the reviews to ensure the latest evidence is incorporated and accessible via the Cochrane Library
[18].
D.J. Henderson-Smart & C. A. Crowther
History of the Cochrane databases of RCTs and systematic reviews This section will briefly present the major steps taken to develop a comprehensive register of RCTs and to establish regularly updated systematic reviews of the effects of perinatal care. The most up-to-date register of RCTs and systematic reviews in pregnancy, childbirth and neonatal care, are published in the Cochrane Library, a computerized database released quarterly that has pregnancy and childbirth [19] and neonatal [20] modules. The perinatal field has led the way in trial registration, systematic reviews and database development. So how did perinatal medicine become so fortunate?
Register of randomized controlled trials Archie Cochrane recognised the importance of randomized trials in reducing the risk of bias when assessing the effects of care and the relevance of evidence derived from randomized trials for the health services [21]. Profoundly influenced by Cochrane's challenges, lain Chalmers, at the National Perinatal Epidemiology Unit in Oxford, in 1973 began to develop a register of all controlled trials in perinatal care. Electronic searching and hand searching of journals was carried out retrospectively to 1941. This Herculean task to identify all relevant information, both published and unpublished, was carried out by a dedicated team of researchers over the next 12 years. This Classified Bibliography of Controlled Trials in Perinatal Medicine 1940---I984 was first published, in book form, in 1985 [22]. This register was put into electronic form in 1988, the Oxford Database of Perinatal Trials (ODPT). The database listed published, unpublished, planned and ongoing trials and was published 6-monthly until 1992 [23]. The ongoing prospective searching for relevant trials is maintained now by the pregnancy and childbirth and neonatal collaborative review groups within the Cochrane Collaboration.
Systematic reviews With the relevant randomized clinical trial evidence identified, systematic reviews in perinatal care were prepared and first published in ODPT. It was
The Cochrane d a t a b a s e s
215
Table 3. Checklist for evaluating systematic reviews Problem Formulation Are review questions well formulated with specified key components? Were major changes in review questions avoided during the review process? Study Identification Is there a thorough search for relevant data using appropriate sources? Are there unbiased explicit searching strategies that are appropriately matched to the question? Study Selection Are appropriate inclusion and exclusion criteria used to select articles? Are selection criteria applied in a manner that limits bias? Are major changes in selection criteria avoided during the review process? Appraisal of Studies Is the validity of individual studies addressed in a reliable manner7 Are important parameters (e.g. setting, study population, study design) that could affect study results systematically addressed? Data Collection Is there a minimal amount of missing information regarding outcomes and other variables considered key to interpretation of results? Data Synthesis Are important parameters, such as study designs, considered in the synthesis? Are reasonable decisions made concerning whether and how to combine data? Are results sensitive to changes in the way the analysis was done? (Different statistical approaches; including/excluding unpublished or 'lesser quality" data) Are results precise? Discussion Are limitations of studies and the review process stated? Are review findings integrated within the context of relevant indirect evidence? Conclusions Are conclusions supported with valid studies that showed consistent, large, and significant effects? Are plausible competing explanations of observed effects addressed? Is evidence appropriately interpreted as inconclusive (no evidence of effect) or as showing a particular strategy did not work (evidence of no effect)? From CochraneCollaboration1996,with permission.
unique in being able to incorporate newly prepared systematic reviews and previous systematic reviews that had been updated as new trial evidence became available. Regular updating of systematic reviews aims to shorten the time from the publication of new evidence to its clinical implementation, by making the information easily accessible. Based on ODPT, books were published including Effective Care in Pregnancy and Childbirth (ECPC) [24], Guide to ECPC [25] and Effective Care of the Nezoborn hzfant [26]. The ECPC books have been described as 'the most advanced current example of
a basis for practising medicine founded both on empirical evidence and theory' [27]. In the foreword to ECPC, Archie Cochrane stated that the systematic reviews of randomized trials of obstetric practice presented in the book were a 'new achievement' and he expressed the hope that 'it will be widely copied by other medical specialties' [28]. The establishment of the Cochrane Collaboration in 1992, an international organization dedicated to preparing, maintaining and disseminating systematic reviews on the effects of healthcare, increases the likelihood Cochrane's hope will be realized and evidence-based healthcare
216
will become a reality for all. The reviews on pregnancy and childbirth subsequently became available in the Cochrane Collaboration Pregnancy and Childbirth Database (CCPC) which was last published in 1995 [29] and these are currently being updated in the new Cochrane format for the Cochrane Library [18]. The Cochrane systematic reviews are published quarterly on disk or CD-ROM and will soon be accessible via the internet [18]. The majority of the Cochrane systematic reviews in perinatal care within the Cochrane Library are produced by reviewers who belong to the pregnancy and childbirth and neonatal Cochrane collaborative review groups. Use of these rich information resources has been recommended for clinicians involved in perinatal care [3, 4].
Perinatal databases in clinical practice Use in direct patient care Although use of the databases at the 'bedside' has been discussed in theory [30] there is little evidence that they would be consulted by the busy clinician at the moment of patient contact, rather their role is in informing therapeutic policy for that individual clinician or for the clinical unit in which they work. For controversial areas, and particularly for audit such as in perinatal morbidity/mortality meetings, the availability of the databases is valuable in weighing the range of evidence available in any given therapeutic situation. In many such meetings there has been a swing away from opinion-based discussion based on experience, towards evidencebased decisions and this is greatly assisted by the availability of evidence from perinatal databases. We are not aware of any systematic evaluation of the use of perinatal databases at the 'bedside'.
Are perinatal databases used in clinical practice? The key issues here are knowledge about the existence of databases, their availability in the clinical setting and their use in clinical care decisions. In a 1993 survey of obstetric units in the UK [31] ECPC or ODPT were available in 55% of district general hospitals and 88% of teaching hospitals. UK users of CCPC in 1994 most c o m -
D.J. Henderson-Smart & C. A. Crowther
monly used it to improve personal knowledge and that of others, but also for guiding research, developing clinical guidelines and informing audit [32]. In Australia, Jordens et al [33] surveyed all neonatologists and a random sample of practising obstetricians on their use of systematic reviews. Of neonatologists, 72% said they used systematic reviews, principally those published in Effective Care of the Newborn hTfant [26]. In addition to computer literacy and research experience, a factor significantly associated with the use of systematic reviews was attendance at the Australian Perinatal Society meetings. Since 1990 such meetings have promoted RCTs and systematic reviews though guest speakers and workshops on critical appraisal, systematic reviews and the design of RCTs. Fewer obstetricians (44%) mentioned using systematic reviews and those that did predominantly used the electronic form, CCPC.
Use in the development of guidelines Clinical practice guidelines are being used increasingly [34]. They have been shown to change clinician behaviour and to improve patient outcomes [35]. An important factor in effective guidelines is the quality of the evidence on which they are based. In the past expert medical opinion was used to develop guidelines but such opinion may not be correct [36] and even when based on literature this was often reviewed in an unsystematic way that led to biases [10, 35]. To be valid, guidelines need to be based on the strongest evidence available, such as that from systematic reviews or randomized controlled trials (Table 1) and recommendations tempered by the level of evidence on which they are based [37, 38]. The Cochrane databases in perinatal care can help make the strongest evidence available. In Australia the National Health and Medical Research Council (NHMRC) has funded a Cochrane Centre to contribute to trial registration and development of systematic reviews. The NHMRC also has commissioned the development of guidelines including those for 'Care around Preterm Birth' based on the pregnancy and childbirth and neonatal modules of the database.
Use by funders and providers of health services In the UK (Effective Health Care 1994) and USA [34] the private and government health service
The Cochrane databases
providers have indicated that the most efficient way to fund health services is to concentrate efforts on forms of care that have been shown to be effective [38]. In a 1994 UK survey of people and organizations who had a copy of CCPC 15% of responders were purchasers of healthcare [32].
Perinatal databases in teaching In both undergraduate and postgraduate clinical education, as well as in continuing education, the databases can be used to illustrate the value of scientifically valid external information to assist in clinical care decisions [30]. These databases play an important role in curricula that are incorporating the principles of evidence-based medicine in their teaching programs. At McMaster University in Canada such an approach has been shown to produce medical graduates who are more knowledgable about current therapeutic guidelines and have greater retention of knowledge since graduation [39]. In neonatology, the databases have been used to underpin an undergraduate course at the University of Sydney, in which the databases are available 'live' during problem-based learning sessions. The ready access, completeness and graphic displays are particularly helpful in illustrating the relative effectiveness of different forms of clinical care. Students who are developing their problem-based tutorials are directed to the databases as resource material and encouraged to use their graphic displays in illustrating the effects of treatments [40]. The Royal Australian College of Obstetrics and Gynaecology has included the use of evidence from the databases in their assessments of candidates for the College. This has the important effect of avoiding a clash between the opinion of a candidate and an examiner, since the examiners have been tutored to know what evidence is available and to base their examination on that evidence. The training program for candidates includes a distance learning package with a clinical epidemiology module that incorporates the use of perinatal databases [41].
Perinatal databases in research Use of the trial register and the database of systematic reviews would seem essential prior to
217
development of any new study evaluating a therapeutic intervention. This would assist in avoiding unnecessary duplication of studies and where more studies are indicated previous trials will help in the design of new trials. Systematic reviews highlight the gaps in evidence about important outcomes and the need for additional randomized controlled trials. Reviews in the Cochrane databases are particularly valuable in that the strength and weaknesses of the various trials have been highlighted and an 'implications for research' section points towards important questions and issues in trial design needing to be addressed. It could be argued that funding bodies should require a systematic review to be part of the background to any research application for a project evaluating a therapeutic intervention. There has been a call for ethics committees also to require this [42].
The future for databases in perinatal care Prospective trial registration Trials in which statistically significant differences in outcome have not been detected are less likely to be published than those showing such differences [42, 43] and could bias any review. The Cochrane Collaboration continues to be involved in efforts to establish systems for registering RCTs at inception [42]. Registers of planned, ongoing and completed but not yet published trials, also provide valuable additional information to inform research and to promote collaboration.
Getting the systematic reviews done and kept up to date In the text Effective Care of the Nezoborn hlfant [26] 1234 RCTs were incorporated into 501 systematic reviews. Although many areas have not changed because no new trials have appeared, the newest and most controversial areas are out of date. Electronic publication, as in the Cochrane Library, is the only effective way of providing up to date systematic reviews and a high priority for the Neonatal Review Group is to do just that. The Pregnancy and Childbirth Review Group have been ahead of the field in terms of electronic publication of systematic reviews, first in the
218
OxfoM Database of Perinatal Trials and then the Cochrane Collaboration Pregnancy and Childbirth module, which had over 600 reviews. The latter, however, has not been published since 1995 due to a switch in priorities to update and convert the reviews to the new Cochrane format. At the end of 1996 that task was about 20% complete.
How can you play a role in the Cochrane collaboration? With more trained people doing reviews the process of getting the reviews done and keeping them up to date will be more efficient. Anyone interested in contributing to this enormous task can contact their local Cochrane Centre or a Collaborative Review Group via the contact addresses in the Cochrane Library. Information about training, available review topics and help finding co-authors will be provided. Enthusiasm for hard unpaid work and a commitment to follow through on updates are the main initial qualifications required.
D . J . Henderson-Smart & C. A. Crowther
MJNC (eds) Effective Care in Pregnancy and Childbirth. Vol. I. Oxford: Oxford University Press, 1989; 3-38. 6 Sackett DL, Haynes RB, Guyatt GH et al. Clinical Epidemiology: A Basic Sciencefor ClinicalMedicine. 2nd edn Boston: Little Brown Company, 199I. 7 Schultz KF Chalmers I, Hayes R et al. Empirical evidence of bias. Dimensions of methodological quality associated with estimates of treatment effects in controlled trials. ]AMA 1995; 273: 408--412. 8 National Health and Medical Research Council of Australia, Quality of Care and Health Outcomes Committee. Guidelines for the Development and hnplementation of Clinical Practice Guidelines. Canberra: Australian Government Printing Services, 1995. 9 Crowther CE, Henderson-Smart DJ. Phenobarbital prior to preterm birth. In: Neilson JP, Crowther CA, Hodnett ED, Hofmeyr GJ, Keirse MJNC, Renffew MJ (eds) Preg-
10
11
12
Communicating the results to clinicians and consumers
13 14
The need for databases to provide easy access to the best evidence for the effectiveness of different forms of care is clear. What is less clear is how to present this information to each of the groups (funders, providers and consumers) in a format that is useful; how to incorporate information into clinical care decisions about specific groups of patients or individuals [38]. It is here that clinical expertise must play a role lest we become swamped in a 'tyranny of information' [2].
References 1 Cochrane AL. 1931-1971: a critical review of the medical profession. In: Medicinefor the Year 2000. London, Office of Health Economics, 1979, 2-11. 2 Sackett DL. Evidence based medicine: what it is and what it is not. Br Med ] 1996; 312: 71-72. 3 Enkin MW. Systematic summaries and dissemination of evidence: the Cochrane pregnancy and childbirth database. Semin Perinatol 1995; 19: 155-160. 4 0 h l s s o n A. Randomized controlled trials and systematic reviews: a foundation for evidence-based practice. Aeta Paedhdr 1996; 8 5 : 647--655. 5 Chalmers I. Evaluating the effects of care during pregnancy and childbirth. In: Chalmers I, Enkin M, Keirse
15 16
17
18
19
20
nancy a~d Childbirth Module of the Cochrane Database of Systematic Reviews (Available in: the Cochrane Library database on disc and CD-ROM). The Cochrane Collaboration; Oxford: Update Software; 1996, Issue 4. Guyatt GH, Sackett DL, Cook DJ. Users guide to the medical literature II. How to use an article about therapy or prevention. A. Are the results of the study valid? ] Am Med Assoc 1993; 270: 2598-2601. Begg C, Cho M, Eastwood S, et aL Improving the quality of reporting of randomized controlled trials: the CONSORT statement, lAMA 1996; 276: 637--639. Mulrow C. Rationale for systematic reviews. Br Med ] 1994; 309: 597-599. Mulrow C The medical review article: state of the science. Ann hltenl Med 1987; 104: 485-488. Crowley P. Corticosteroids prior to preterm delivery. In: Neilson JP, Crowther CA, Hodnett ED, Hofmeyr GJ, Keirse MJNC, Renffew MJ (eds) Pregnancy and Childbirth Module of the Cochrane Database of Systematic Reviews. (Available in the Cochrane Library.) The Cochrane Collaboration; Oxford: Update Software; 1996, Issue 3. Oxman AD. Checklists for review articles. Br MedJ 1994; 3 0 9 : 648--651. Mulrow CD, Oxman AD (eds) Cochrane Collaboration Handbook. (Available in: The Cochrane Library (database on disc and CD-ROM).) The Cochrane Collaboration; Oxford: Update Software 1996, Issue 3. Thacker SB, Peterson HB, Stroup DF. Metaanalysis for the obstetrician-gynecologist. Am ] Obstet Gynecol 1996; 1 7 4 : 1403-1407. Cochrane Collaboration. Cochrane Database of Systematic Reviews. (Available in the Cochrane Library (database on disc and CD-ROM).) The Cochrane Collaboration, Oxford: Update Software; 1996, Issue 3. Neilson JP, Crowther CA, Hodnett ED, Hofmeyr GJ, Keirse MJNC, Renfrew MJ (eds) Pregnancy and Childbirth Module of the Cochrane Database of Systematic Reviews. (Available in the Cochrane Library (database on disc and CD-ROM).) The Cochrane Collaboration, Oxford: Update Software; 1996, Issue 3. Sinclair J, Bracken M, Soll RF, Horbar JD (eds) Neonatal Module of the Cochrane Database of Systematic Reviews. (Available in the Cochrane Library (database on disc and CD-ROM).) The Cochrane Collaboration; Oxford: update Software; 1996, Issue 3.
The Cochrane databases
AL. Effectiveness and Efficiency: Random Reflections on Health Services. London: Nuffield Provincial Hospital Trust, 1972. National Perinatal Epidemiology Unit. A Classified Bibliography of Controlled Trials in Perinatal Medicine 1940-1984. Oxford: Oxford University Press, 1985. Chalmers I (ed.) Oxford Database of Perinatat Trials. Version 1.2, disk issue 7 (spring), Oxford: Oxford University Press 1992. Chalmers I, Enkin MW, Keirse MJNC (eds) Effective Care in Pregnancy and Childbirth. Oxford: Oxford University Press, 1989. Enkin M, Keirse M, Renffew M & Neilson J (eds) A Guide to Effective Care in Pregnancy and Childbirth. Oxford: Oxford University Press, 1995. Sinclair JC, Bracken MB (eds) Effective Care of the Newborn Infant. Oxford: Oxford University Press, 1992. Mosteller F. Database medicine in the light of ECPC. Milbank Q 1993; 71: 523-533. Cochrane AL. Forward to effective care in pregnancy and childbirth. Chalmers I, Enkin M, Keirse MJNC (eds) Effective Care in Pregnancy and Childbirth. Vol. 1. Oxford: Oxford University Press, 1989. Enkin MW, Keirse MJNC, Renfrew MJ & Neilson JP (eds) Pregnancy and Childbirth Module of the Cochrane Database of Systematic Reviews. (Available in the Cochrane Library (database on disc and CD-ROM).) The Cochrane Colloberation; Oxford: Update Software 1995, Issue 2. Sackett DL. Applying overviews and meta-analyses at the bedside. ] Clin Epidemiol 1995; 48: 61-66. Paterson-Brown S, Fisk N & Wyatt JC. Uptake of meta-analytical overviews of effective care in English obstetric units. Br ] Obstet Gynaecol 1995; 102: 297-30I. Hyde C. Who uses the Cochrane Pregnancy and Childbirth Database? Br Med ] 1995; 310:1140-1141 Jordens C, Hawe P, Irwig I, Henderson-Smart D, Ryan M, Donoughue D, Gabb R, Fraser I. The use of
219
21 Cochrane
22
23
24
25
26 27 28
29
30 31
32 33
34
35
36
37
38
39
40
41
42 43
systematic reviews by Australian neonatologists and obstetricians. Proceedings 4th hlternational Cochrane Colloquium. Adelaide, 1996, A12. Woolf SH. Practice guidelines: a new reality in medicine. 1. Recent developments. Arch hzlern Med 1992; 150: 1811-1818. Effective Health Care. Implementing clinical practice guidelines: can guidelines be used to improve practice? Bulletin no.8, Leeds: University of Leeds, 1994 Antman EM, Lau J, Kupelnick B, Mosteller F, Chalmers TC. A comparison of results of meta-analyses of randomized control trials and recommendations of clinical experts. Treatments for myocardial infarction. JAMA 1992; 268: 240-248. Hayward SA, Wilson MC, Tunis SR, Bass EB, Guyatt G; for the Evidence-based Medicine Working Group. Users' guide to the medical literature. VII. How to use clinical practice guidelines. A. Are the recommendations valid? JAMA 1995; 274: 570-574. Invig L, Zwarenstein M, Zwi A, Chalmers I. A flow diagram to help select interventions and research for healthcare. Bull WHO (in press) Shin JH, Flaynes RB & Johnston ME. Effect of problembased, self directed undergraduate education on life-long learning. Can Med Assoc ] 1993; 148: 969-976. Jeffery HE, Henderson-Smart DJ & Hill DA. Competencybased learning in neonatology. Med Ed 1996; 30: 440444. Royal Australian College of Obstetricians and Gynaecologists. Training Program Manual. Melbourne: RACOG, 1995. Savulescu J, Chalmers I, Blunt J. Are ethics committees behaving ethically? Br Med ] 1996; 313: 1390-1393. Dickersin K, Min YI & Meinert CL. Factors influencing publication of research results: follow-up of applications submitted to two institutional review boards. ]AMA 1992; 263: 374-378.