- Email: [email protected]
A view of reviews
Editorial A view of reviews A busy clinician does not usually have the time to sift through piles of journals to keep abreast of new developments or to answer questions that arise during patient care. Evidence-based medicine requires incorporating the results of well-conducted research studies with clinical expertise but because of the staggering volume of published information, many clinicians rely on review articles to provide them with summaries of the medical literature. Reviews can compare and evaluate the results of multiple studies, summarize large amounts of information, identify deficiencies in reported research, improve the accuracy and precision of estimates of effectiveness, and investigate discrepancies between individual studies and populations. Thus, methodologically sound review articles can form a cornerstone of modern medical practice. Traditional narrative reviews are much like textbook chapters. They usually examine broad clinical questions and can provide useful background information, review pathophysiology, and explore mechanisms of disease. However, when it comes to practice recommendations, narrative reviews are usually subjective and therefore susceptible to bias in the selection and interpretation of research findings. The basic problem with narrative reviews is that the information from previous publications is combined with the reviewer’s own experience, opinions, and conjecture, each in varying and unstated proportions. Narrative reviews do not usually give any information as to the procedures by which articles were identified, selected for inclusion, or analyzed; often they merely recite the conclusions given in the articles they review, with little or nothing in the way of critical analysis. The more modern approach is that of the systematic review. Although there is no ironclad definition of this term, it is possible to identify several essential characteristics of systematic reviews: 1. A specific focused question is defined a priori. This question includes the population studied, the spe-
VOL. 95, NO. 4, APRIL 2000
cific intervention, exposure, or test, and the specific outcomes to be assessed. 2. The literature search is comprehensive. It is based on a computerized database such as MEDLINE, but because this may only recover 50 – 60% of relevant references, the search also involves other electronic databases, hand searching of bibliographies of articles identified, clinical trial registries, and consultation with experts. 3. A written protocol is developed a priori. Specific inclusion and exclusion and quality assessment criteria are stated and decisions on inclusion of studies and data abstraction are made independently by multiple reviewers, and results are reconciled. 4. The results are presented in an objective manner. Characteristics of both included and excluded studies are stated explicitly, along with assessments of study quality, to allow the reader to see exactly what was done by the reviewers and how they did it. Findings from different studies are related to one another and, whenever possible, integrated. A specific type of systematic review that uses statistical methods to synthesize quantitatively the results of several studies is called a meta-analysis. Done correctly, meta-analysis can provide precise and valid estimates of effectiveness, and it is regarded by many as the ultimate authority in evidence-based medicine. Its results are often presented in a figure showing point estimates and confidence intervals from the individual studies, along with the pooled estimate and confidence interval. This figure (forest plot) provides the results of the individual studies, displays heterogeneity of effects, and shows how the pooled estimate compares with the results of the individual studies. Meta-analysis often uses techniques such as sensitivity analysis (eliminating certain studies from the calculations) to evaluate the effects of study design, subject population, or other study characteristics on summary results and to estimate the robustness of the findings. Although it can provide the most reliable source of information, meta-analysis is not without limitations. It is usually not advisable when there is clinical or statistical heterogeneity; statistical heterogeneity occurs when the results of each individual trial are mathematically incompatible with the results of any of the others, and clinical heterogeneity refers to inconsistencies in design, study quality, interventions, outcomes, or populations. Either may be evidence that the studies included in the meta-analysis are measuring different things, so that a summary estimate may not be meaningful. Meta-analysis was developed originally for combining randomized controlled trials and this remains its major application; it has also been applied to observa-
0029-7844/00/$20.00 PII S0029-7844(00)00803-6
605
tional studies and diagnostic test evaluations, but these uses are somewhat controversial. The Cochrane Collaboration (http://www. cochrane. org) is an international nonprofit organization whose mission is to help make well-informed decisions about health care by preparing, maintaining, and ensuring the accessibility of systematic reviews of the effects of health care interventions. The Cochrane Library consists of four databases: the Cochrane Library of Systematic Reviews, the Database of Abstracts of Reviews of Effectiveness, the Cochrane Controlled Trials Register, and the Cochrane Review Methodology Database. All four are available by subscription either on CD-ROM or via the Internet from Update Software, Inc. (Vista, CA). The Cochrane Library of Systematic Reviews is also available through Ovid Technologies, Inc. (New York, NY). Additionally, abstracts of reviews will soon be available in PubMed. Obstetrics & Gynecology, from its beginnings, has published reviews, and our periodic surveys indicate that readers especially value this type of article. However, since our primary purpose is to report original research, the space we can allocate to other types of papers, including reviews, is limited. We generally aim to have no more than one review per issue, except for the semi-annual part II issues where we may have two or three. Thus, over the course of a year, we will publish 15–18 review articles. We do not solicit or commission reviews, so all we publish have been submitted spon-
606 Nanda and Pitkin
View of Reviews
taneously. Several years ago, the Editorial Board established a policy that we would not accept reviews authored by anyone with a possible conflict of interest (see “Conflict of interest revisited.” Obstet Gynecol 1995;86:293) because this type of article depends particularly on interpretation. Recently, we have seen an increase in the number submitted which, coupled with a declining number of reports of original research (see “A Cause for Concern.” Obstet Gynecol 1998;92:471), may reflect authors’ continuing need to publish in the face of declining resources for research. Whatever the cause, the increase in reviews submitted means that the acceptance rate for this type of article will decrease. We have a strong preference for systematic reviews over the narrative type because of their much greater scientific rigor, and the subjects need to be broad enough to appeal to the range of our general readership. Systematic reviews constitute an essential aspect of modern evidence-based medicine, what some have called “best evidence.” Of course, even the most rigorously conducted review cannot replace sound clinical judgement. Decisions about patient care are best made by a combination of relevant clinical experience and judgement and evidence from well conducted systematic reviews.
Kavita Nanda Roy M. Pitkin
Obstetrics & Gynecology
VOL. 95, NO. 4, APRIL 2000
cific intervention, exposure, or test, and the specific outcomes to be assessed. 2. The literature search is comprehensive. It is based on a computerized database such as MEDLINE, but because this may only recover 50 – 60% of relevant references, the search also involves other electronic databases, hand searching of bibliographies of articles identified, clinical trial registries, and consultation with experts. 3. A written protocol is developed a priori. Specific inclusion and exclusion and quality assessment criteria are stated and decisions on inclusion of studies and data abstraction are made independently by multiple reviewers, and results are reconciled. 4. The results are presented in an objective manner. Characteristics of both included and excluded studies are stated explicitly, along with assessments of study quality, to allow the reader to see exactly what was done by the reviewers and how they did it. Findings from different studies are related to one another and, whenever possible, integrated. A specific type of systematic review that uses statistical methods to synthesize quantitatively the results of several studies is called a meta-analysis. Done correctly, meta-analysis can provide precise and valid estimates of effectiveness, and it is regarded by many as the ultimate authority in evidence-based medicine. Its results are often presented in a figure showing point estimates and confidence intervals from the individual studies, along with the pooled estimate and confidence interval. This figure (forest plot) provides the results of the individual studies, displays heterogeneity of effects, and shows how the pooled estimate compares with the results of the individual studies. Meta-analysis often uses techniques such as sensitivity analysis (eliminating certain studies from the calculations) to evaluate the effects of study design, subject population, or other study characteristics on summary results and to estimate the robustness of the findings. Although it can provide the most reliable source of information, meta-analysis is not without limitations. It is usually not advisable when there is clinical or statistical heterogeneity; statistical heterogeneity occurs when the results of each individual trial are mathematically incompatible with the results of any of the others, and clinical heterogeneity refers to inconsistencies in design, study quality, interventions, outcomes, or populations. Either may be evidence that the studies included in the meta-analysis are measuring different things, so that a summary estimate may not be meaningful. Meta-analysis was developed originally for combining randomized controlled trials and this remains its major application; it has also been applied to observa-
0029-7844/00/$20.00 PII S0029-7844(00)00803-6
605
tional studies and diagnostic test evaluations, but these uses are somewhat controversial. The Cochrane Collaboration (http://www. cochrane. org) is an international nonprofit organization whose mission is to help make well-informed decisions about health care by preparing, maintaining, and ensuring the accessibility of systematic reviews of the effects of health care interventions. The Cochrane Library consists of four databases: the Cochrane Library of Systematic Reviews, the Database of Abstracts of Reviews of Effectiveness, the Cochrane Controlled Trials Register, and the Cochrane Review Methodology Database. All four are available by subscription either on CD-ROM or via the Internet from Update Software, Inc. (Vista, CA). The Cochrane Library of Systematic Reviews is also available through Ovid Technologies, Inc. (New York, NY). Additionally, abstracts of reviews will soon be available in PubMed. Obstetrics & Gynecology, from its beginnings, has published reviews, and our periodic surveys indicate that readers especially value this type of article. However, since our primary purpose is to report original research, the space we can allocate to other types of papers, including reviews, is limited. We generally aim to have no more than one review per issue, except for the semi-annual part II issues where we may have two or three. Thus, over the course of a year, we will publish 15–18 review articles. We do not solicit or commission reviews, so all we publish have been submitted spon-
606 Nanda and Pitkin
View of Reviews
taneously. Several years ago, the Editorial Board established a policy that we would not accept reviews authored by anyone with a possible conflict of interest (see “Conflict of interest revisited.” Obstet Gynecol 1995;86:293) because this type of article depends particularly on interpretation. Recently, we have seen an increase in the number submitted which, coupled with a declining number of reports of original research (see “A Cause for Concern.” Obstet Gynecol 1998;92:471), may reflect authors’ continuing need to publish in the face of declining resources for research. Whatever the cause, the increase in reviews submitted means that the acceptance rate for this type of article will decrease. We have a strong preference for systematic reviews over the narrative type because of their much greater scientific rigor, and the subjects need to be broad enough to appeal to the range of our general readership. Systematic reviews constitute an essential aspect of modern evidence-based medicine, what some have called “best evidence.” Of course, even the most rigorously conducted review cannot replace sound clinical judgement. Decisions about patient care are best made by a combination of relevant clinical experience and judgement and evidence from well conducted systematic reviews.
Kavita Nanda Roy M. Pitkin
Obstetrics & Gynecology