- Email: [email protected]
Meta-analysis of randomized controlled trials of antenatal corticosteroid for the prevention of respiratory distress syndrome: Discussion
75.
Mugford M, Piercy J. Chalmers I. Cost implications of different approaches to the prevention of respiratory distress syndrome. Arch Dis Child 1991;66:757-64. Crowley P, Chalmers I, Keirse MJ. The effects of corticosteroid administration before preterm delivery: an overview of the evidence from controlled trials. Br J Obstet Gynaecol 1990;97: 1 l-25. Garite TJ, Freeman RK, Linzey EM, Braly PS, Dorchester WL. Prospective randomized study of corticosteroids in the management of premature rupture of the membranes and the premature gestation. AM J OBSTET GYNECOL 1981; 141:508-15. Iams JD, Talbert ML, Barrows H, Sachs L. Management of prematurely ruptured membranes: a prospective randomized comparison of observation versus steroids and timed delivery. AM J OBSTET GYNECOL 1985;151:32-8. Nelson LH, Meis PJ, Hatjis CG, Ernest JM, Dillard R, Schey HM. Premature rupture of membranes; a prospective, randomized evaluation of steroids, latent phase and expectant management. Obstet Gynecol 1985;66:55-8.
76.
77.
78.
79.
80.
Farrag OAM. Prospective study of three metabolic regimensin pregnant diabetics. A&t N Z J Obstet Gynaecol 1987;27:6-9. Keirse MJNC, Kanhai HHH. An obstetrical viewpoint on preterm birth with particular reference to perinatal mortality and morbidity. In: Huisjes HJ, ed. Aspects of perinatal morbidity. Groningen: Universitaire Boekhandel Nederaland, 1981:1-35. Tubman TRJ, Rollins MD, Patterson C, Halliday HL. Increased incidence of respiratory distress syndrome in babies of hypertensive mothers. Arch Dis Child 1991;66: 52-4. Lamont RF, Dunlop PDM, Levene MI, Elder MI. Use of glucocorticoids in pregnancies complicated by severe hypertension and proteinuria. Br J Obstet Gynaecol 1983; 90:199-202. Farrell PM, Engle MJ, Zachman RD, et al., and the Collaborative Group on Antenatal Steroid Therapy. Amniotic fluid phospholipids after maternal administration of dexamethasone. AV J OSSTET GYNECOL 1983;145:484-90.
81.
82.
83.
84.
Meta-analysis of randomized controlled trials of antenatal corticosteroid for the prevention of respiratory distress syndrome: Discussion*“f John
C. Sinclair,
Hamilton,
MD
Ontario,
Canada
A meta-analysis rize
results
research.
estimate
defined
population because
trials,
it must
a systematic in
the
the
to obtain
effect
for
each
of
estimate
of
that
the can
lead
trials
treatment of interest.
results
Table I. Structure effect of treatment
to summaof scientific
an unbiased,
outcome
include review
review
of randomized
seeks of
methods
a systematic review
intervention
mary
sion
quantitative
from
A systematic
medical
over,
uses
derived
of
a
in
a
More-
$3.00
-t 0
No event
Total
a
b
n,
C
d m.
no T
Treated Control
ml
effect and greater than can review of any single
treatment
generalizability of findings study. The opportunity to draw more definitive conclusions from a series of similar studies is not restricted to medicine or to randomized trials. Some of the first
0002.9378/95
Event
Exposure
preci-
From the Departments of Pediatrics and Clinrcal Epidemiolog? and Biostatistics, McMaster University. Reprint requests: John C. Sinclair, MD, Departments of Pedialrics and Clinical Epidemiology and Biostatistics, McMaster University, 1200 Main St. West, Hamilton, Ontario, Canada, L8N 325. *This article is based on the data presented by Dr. Crowley at the Consensus Development Conference on the Effect of Corticosteroids for Fetal Maturation on Perinatal Outcomes, Bethesda, Maryland, February S-March 2, 1994. tA similar report based on this presentation has been previously published: Sinclair JC. Meta-analysis of randomized controlled trials: Antenatal corticosteroid for prevention of respiratory distress syndrome, Prospettive in Pediatria 1994;24:199-206. AM J OBSTET GYNEC~L 1995;173:335-44. Copyright 0 1995 by Mosby-Year Book, Inc. 6/O/63092
the
Outcome
sum-
of all qualifying
to increased
of a study to evaluate
Event Event
rate rate
Relative
risk
Odds
= a/n,. = c/n,.
0
risk reduction = a/n, - c/n,. risk reduction = 1 - relative risk. needed to treat = l/absolute risk reduction.
attempts clinical
= 2
group group
alb = a.
ratio
Absolute Relative Number
made
in treated in control
to synthesize
research
in the social sciences.‘, trials,
which
have
rather
in an empiric
* However, standard
way
were
randomized methodology,
particularly suited to meta-analysis.3 Since the early 198Os, there has been a sharp increase in the rate of publication of meta-analyses of randomized controlled are
trials4 which ized
Two
textbooks
essentially trials
of
consist therapy
have of covering
recently
been
meta-analyses an
entire
published, of
random-
discipline.‘,
’ 335
336
Sinclair Am J Obstet
GESTATIONAL AGE (wk) 36 35 34 33 32 IllIll
31
30 I
29 I
Table II. Overall effect of corticosteroids administered before preterm delivery on RDS
526 1
Point estimate Relative risk reduction (%) Absolute risk reduction (%) No. needed to treat
95%
-41 -7.9
I
20
I
I
40 BASELINE
I
I
60 RISK
1
13
80 (%)
Fig. 1. Number of fetuses requiring treatment with antenatal corticosteroids to prevent one case of RDS as a function of baseline risk. Number needed to treat is derived from typical relative risk reduction of 41% calculated from data of trials included in meta-analysis of Crowley.” Gestational ages corresponding to baseline risk are based on data of Usher et al.” Shaded zone indicates 95% CI.
Structure
of a trial to evaluate
treatment
Table I displays the structure of a typical study that seeks to evaluate treatment. There are two exposure groups (labeled treated or control) and two possible outcomes (labeled event or no event). Particularly important are studies in which patients are assigned to the treated or control group by a random procedure. Estimators of treatment effect for binary analysis of the results of a randomized trial are given in Table I. The same estimators are used when analyzing the results of a set of comparable trials in a meta-analysis, as will be discussed below. Meta-analysis:
Method
A systematic review of randomized trials requires that all relevant and “groupable” trials be identified, disclosing searching methods and using predefined inclusion and exclusion criteria. Next, the trials are examined for their methodologic quality; those trials whose methods fail to meet predefined criteria for methodologic quality are excluded. The results of the included trials are then extracted and tabulated, and a summary estimate of treatment effect is calculated for each outcome of interest. This summary effect (sometimes termed a “typical” effect) is some form of weighted average across trials; the weights are inversely related to the variance in the estimate of treatment effect provided by each participating trial.
Relative risk reduction (%) Absolute risk reduction (%) No. needed to treat
- 10.2
10, III)
18
of Crowley,
1994
on
Point estimate
1
-49
- 5.6,
Table III. Effect of corticosteroids administered before preterm delivery RDS in fetuses < 3 1 weeks’ gestation
I
CI
-32,
Calculated from meta-analysis (Table (15 trials, 588 events, 3438 subjects).
I
July 1995 Gym01
95%
CI
-37
-23,
-49
-19
-11,
-27
5
Calculated from meta-analysis (Table (9 trials, 203 events, 461 subjects).
4, 9 VII)
of Crowley,
1994
Table IV. Effect of corticosteroids given before preterm delivery on RDS in fetuses > 34 weeks’ gestation Point estimate Relative risk reduction (%) Absolute risk reduction (%) No. needed to treat Calculated from meta-analysis (8 trials, 29 events, 886 subjects).
95%
-37
CI
+29,
-69
-1.1
+1.3,
-3.4
94
30,
(Table
VIII)
infinite
of Crowley,
1994
For categoric outcomes, estimators of treatment effect include relative risk (and its complement, relative risk reduction), odds ratio, and absolute risk difference (and its inverse, number needed to treat); outcomes measured on a continuous scale include mean difference. Statistical methods are available for calculating each of these estimators of treatment effect and its confidence interval both for individual trials and for typical (weighted) estimates across a set of trials (e.g., refs. 7 and 8). These methods are based on either fixed-effect or random-effect models. The fixed-effect model presumes that there is no statistical heterogeneity between studies, a presumption that is tested by performing a statistical test to detect heterogeneity. The random-effect model assumes statistical heterogeneity and incorporates it into the estimate of effect. As a
Volume 173, Number Am J Obstet Gym01
EXCLUSION EFFECT
1
Sinclair
OF PATIENTS ON ABSOLUTE
DESTINED FOR NON-EVENT: RISK REDUCTION (ARR)
EXCLUSION EFFECT
337
OF PATIENTS DESTINED FOR NON-EVENT: ON NUMBER NEEDED TO TREAT (NNT)
80-
ReporkdNNTs40
60 -
Reported NNT=ZO
ReportedNNT=
10
Reported NNi-= 5 /
0
.lO
.20 FRACTION
.30 .40 EXCLUDED
Fig. 2. Relationship between signed patients excluded from tined for nonevent, reported actual absolute risk reduction. patients so excluded is 0.40 and tion is lo%, actual absolute risk
Absolute reduction
<31 31-34 234 Calculated from GA, Gestational
administered
before
.I0
20
risk (%)
meta-analysis (Tables III, VII, and VIII) age; NNT, number needed to treat.
I
I
.30
.40
.50
EXCLUDED
between proportion of randomly assigned patients excluded from analysis because they are destined for nonevent, reported number needed to treat, and actual number needed to treat. For example, if proportion of patients so excluded is 0.40 and reported number needed to treat is 10, actual number needed to treat is 17.
95% -6, -3, +3,
delivery
on RDS: Effect of gestational
CI
NNT
-42 -10 -4
of Crowley,
intervals may be somewhat wider for the random-effect model. of relevant trials. The goal of a is to identify all studies that meet inclusion criteria and to report the methods used to achieve this. There is some evidence of a “publication bias” that favors the submission and publication of trials that show “positive” results. To minimize the effect of
Reported NM = 2
I
Fig. 3. Relationship
preterm
-24 -7 -0.7
result, confidence analyses based on Ascertainment systematic review
I
FRACTION
proportion of randomly asanalysis because they are desabsolute risk reduction, and For example, if proportion of reported absolute risk reducreduction is 6%.
Table V. Effect of corticosteroids at entry GA class (wk)
01 0
.50
I
95%
4 15 145 1994
(4 trials,
285 events,
age
CI
2, 17 10, 31 25, infinite 2086
subjects).
publication bias, the reviewer may strive to identify unpublished as well as published studies, although the inclusion of unpublished studies is controversial.g Validity assessment of primary studies. Objective assessment of the methodologic quality of the primary studies requires that a systematic scheme be applied to assess validity. Methodologic standards for the design, conduct, analysis, and reporting of random-
338
Sinclair
1995 Gym01
July
Am J Obstet
RISK DIFFERENCE %, 95% Ct -40 STUDY
YEAR
CPTS.
1
Llggins
1972
1070
2
Block
1977
130
3
Morrlson
1978
126
4
Taeusch
1979
127
5
Papageorglou
1979
146
6
Dotan
1980
144
7
Schutte
1980
122
8
Teramo
1980
80
9
U.S. Collaborative
1981
743
10
Schmidt
1984
65
11
Morales
1986
245
12
Gamsu
1989
268
13
Carlan
1991
24
14
Garite
1992
82
15
Eronen
1993
66
-30
-20
-10
I
I
I
0
+lO
+20
I
3438
OVERALL
FAVORSTFiEAlMENT
FAVORS CO~IROL
Fig. 4. Conventional meta-analysis of trials testing effect of antenatal corticosteroids on incidence of RDS. Data are taken from trials included in meta-analysis of Crowley.‘3 Treatment effect, expressed as risk difference, is shown for each trial as point estimate and 95% CI. If 95% CI does not cross zero, effect is statistically significant (p < 0.05) in that trial. Six of 15 trials showed a statistically significant reduction in RDS, whereas each of other nine trials failed to demonstrate a statistically significant effect. Overall, typical estimate is that this set of trials shows a reduction in incidence of RDS that is statistically highly significant: absolute risk difference - 7.9%, 95% CI - 5.6%, - 10.2%.
ized
trials
However, weights
to
evidence critical
be
that
a basis is not
general
put
on
control
requirement
blinding cian
provide there
of the and
allocated
for
rating
different of
for
of
agreement elements.
selection
validity.‘*
randomization
patient
cannot
predict
to that
patient.
However,
validity.“, on the
bias This
‘I relative
There at
is
entry
is
is attempted
process
so that
which
treatment
other
the
cliniwill
elements
a by be
(such
as complete follow-up, blinding of the outcome measurement, and where possible of the intervention) may be equally important. Until better evidence is available on the relative weights to be attached to different aspects of the study design, it may be best simply to describe the methodologic features of each report and include these evaluations in the review (e.g., the tables that are in the appendix to each chapter in Effective Care of the Newborn Infant,” in which the reviewer reports critical methodologic ‘features of each cited trial). Reporting
treatment
effect
in
a clinically
meaning-
The results of randomized trials are important not only as hypothesis tests but also as the preferred source of valid data for informing explicit, quantitative ful
way.
judgments regarding the prescription of treatment, clinical decision analysis, and cost-effectiveness analysis. Therefore in reporting the results of individual randomized trials, as well as mgta-analyses of sets of randomized trials, it should be recognized that the size of difference and not just the fact of difference will have direct implications for clinical practice. The measures of treatment effect that are reported should facilitate their practical application. The risk ratio, or relative risk, is the traditional effect estimator from prospective studies of which randomized trials are a special case. This estimator provides an answer to the clinically important question, “What is the proportion of treated patients, relative to control patients, who experience an event?” Its complement, relative risk reduction, provides an answer to a second clinically important question, “By how much, in relative terms, is the event rate reduced?” The risk difference provides an answer to a third clinically important question, “What is the absolute difference in event rate between the treated and control groups?” Its inverse, number needed to treat, provides an answer to a fourth clinically important question,
Volume 173, Number Am J Obstet Gynecol
Sinclair
1
339
RISK DIFFERENCE %, 95% Cl -40
-30
-20
-10
I
I
I
STUDY
YEAR
IPTS.
1
Uggins
1972
1070
2
Block
1977
1200
3
Morrls0n
1978
1326
4
Taeuxh
1979
1453
5
Papageorgiou
1979
1599
-
6
Doran
1960
1743
-
7
Schutte
1980
166-l
-
8
Teramo
1980
1645
9
U.S. Collaborative
1981
2688
10 Schmidt
1984
2x3
11 Morales
1986
2998
12 Gamsu
1989
3266
13 carlan
1991
3290
14 Garite
1992
3372
15
1993
3438
Eronen
0
+lO
+20
-
-
FAVORSTREATMENT
FAVORS CONlWX
Fig. 5. Cumulative meta-analysis of trials testing effect of antenatal corticosteroids on incidence of RDS. Data are taken from trials included in meta-analysis of Crowley.” Treatment effect, expressed as risk difference, is shown as in Fig. 4 except that meta-analysis is repeated each time a new trial is published. Effect was statistically significant in first trial (Liggins, 1972) and has remained so since. Point estimate for size of effect expressed as risk difference has changed very little since reports of first trial. However, subsequent trials have added increasing precision to estimate of effect, which is indicated by narrowing of 95% CL
“How many patients does one need to treat to prevent one patient experiencing an event?” Despite the popularity of odds ratio as an effect estimator in meta-analysis, it suffers from the fact that it does not directly address a clinically important question regarding size of treatment effect. However, when the incidence of the event in the control group (i.e., the pretreatment risk) is known, the odds ratio can be converted to the clinically more meaningful risk ratio: Risk ratio
OR = 1 + PR (OR _ 1)
where OR = odds ratio and PR = pretreatment risk. For example, a highly effective treatment was tested in a high-risk population (pretreatment risk, 0.43). Treatment effect, expressed as odds ratio, was 0.37. The user can derive the corresponding risk ratio in this population, which is 0.5 1 (i.e., 49% relative risk reduction), with the above equation.
Crowley’s meta-analysis of effect of antenatal corticosteroid on respiratory distress syndrome, neonatal death, and other outcomes Crowley’” has updated the results of randomized
her existing meta-analysisI trials that have evaluated
of the
effect of antenatal corticosteroid for fetal lung maturation. This updated meta-analysis is based on the results of 18 trials, 15 of which provided information on clinical outcomes and randomized a total of 3438 fetuses. The major results of this meta-analysis are that antenatal corticosteroids produce statistically significant reductions in the odds of respiratory distress syndrome (RDS) (typical odds ratio 0.51, 95% confidence interval [CI] 0.42, 0.61) and neonatal death (typical odds ratio 0.61,95% CI 0.48, 0.76). A fixed-effect model was presumed in the metaanalysis. No statistical heterogeneity was detected with respect to the effect on RDS or death. Great care and effort were exercised to identify all relevant trials, both published and unpublished. The quality of each trial was systematically evaluated for three characteristics: blinding of randomization, whether primary analyses were based on all patients randomized, and blinding of outcome measurements. Of these three criteria, the one Crowley considered the most important was the control of selection bias at entry by the care taken to blind the randomization. However, she noted that “post-randomization exclusions occurred in all trials due to some women . remaining undelivered until term and being lost to follow-up.” She does not report the magnitude of this problem, but its
340
Sinclair Am J Obstet
July 1995 Gynecol
NUMBER NEEDED TO TREAT WITH CORTICOSTEROID TO PREVENT ONE CASE OF RDS: CUMULATIVE META-ANALYSIS
3500
Point Estimatt 95% CL
1970
1975
1980
1985
1990
1993
Fig. 6. Time trend of accumulating evidence that corticosteroids administered before preterm delivery reduce incidence of RDS. Effect was statistically significant in first trial (1972) and has remained so since. Point estimate for size of effect, expressed as number needed to treat, has changed very little since reports of first trial. However, subsequent trials have added increasing precision to estimate of effect, indicated by narrowing of 95% confidence limits (95% CL) over time. Number needed to treat is calculated from absolute risk reduction in trials reviewed by Crowley.r3 NNT = 13, 95% CI 10, 18.
Table VI. Effect of corticosteroids administered before preterm delivery neonatal deaths Point estimate
Absolute risk reduction (%) No. needed to treat
-4.5 22
on
95% -2.6,
CI -6.4
16, 39
Calculated from meta-analysis (Table XVII) of Crowley, 1994 (14 trials, 318 events, 3414 subjects).
clinical importance could be substantial. The impact of such deletions from analysis on the reported size of treatment effect will be considered later. Crowley’s meta-analysis presents only the odds ratio as a measure of treatment effect. It is unfortunate that this treatment effect estimator lacks clinical meaning. For example, an important clinical question not directly addressed by the odds ratio is: “How may fetuses does one have to expose to antenatal corticosteroid to prevent one case of RDS or one neonatal death?” Fortunately, one can use the individual trial data presented by Crowley and redo the meta-analyses to calculate clinically meaningful estimates of treatment effect. The typical estimates for efficacy of antenatal corticosteroid in reducing the incidence of RDS are as
follows: relative risk reduction 41%, absolute risk reduction 7.9%, number needed to treat 13. Table II give these results and their 95% CIs. Assuming constant relative risk, one can then calculate the absolute risk reduction over a range of baseline risks for RDS and the corresponding number needed to treat. These results are given in Fig. 1, in which it is clearly evident that the number of fetuses requiring treatment with antenatal corticosteroids to prevent one case of RDS varies with baseline risk. At a high baseline risk of 2 50% (corresponding to gestational ages of 530 weeks), the number needed to treat is low, I 5. However, at a low baseline risk of 15% (corresponding to gestational ages above 34 weeks), the number needed to treat rises sharply. This phenomenon is also demonstrated by using the data from the primary reports reviewed by Crowley to calculate clinically meaningful estimators of effect, including the number needed to treat, for babies of different gestational age classes. If gestational age is less than 31 weeks, one needs to treat five fetuses to prevent one case of RDS (Table III), which is based on data from nine trials. If gestational age is more than 34 weeks, one needs to treat 94 fetuses to prevent one case of RDS (Table IV), which is based on data from eight trials. Because the effect in this latter gestational age group is not quite statistically significant, the upper
Volume 173, Number Am J Obstet Gynecol
Sinclair
1
341
RISK DIFFERENCE X 95% Cl -lo STUDY
YEAR
t PTS.
1
Ugglns
1972
1070
2
Block
1977
130
3
uortlsan
1978
126
4
.Taeusch
1979
127
1979
146
5
Papageorglou
6
Dorm
1980
144
7
Schulte
1980
122
8
Teramo
1980
80
9
U.S. CollaboraIive
1981
743
10 Schmidt
1984
85
11 Morales
1986
245
12 Gamsu
1989
268
13 Garite
1992
52
14 Eronen
1993
66
OVERALL
-30
-20
-10
I
1
I
0
+lO
+20
1 -
3414
FAVORSTREATMENT
Fig. 7. Conventional
meta-analysis of trials testing effect of antenatal neonatal death. Data are taken from trials included in meta-analysis expressed as risk difference, is shown for each trial as point estimate 95% CI does not cross zero, effect is statistically significant Cp < 0.05) showed a statistically significant reduction in neonatal deaths, whereas to demonstrate a statistically significant effect. Overall, typical estimate a reduction in incidence of RDS that is statistically highly significant: 95% CI -2.6%, -6.4%.
bound of the 95% confidence interval for the number needed to treat includes infinity. On the basis of data from four trials that present their results by each of three gestational age strata, the number needed to treat at gestational ages c 3 1 weeks, 3 1 to 34 weeks, and >34 weeks is 4, 15, and 145, respectively (Table V). Similarly, one can estimate the effect of corticosteroids on neonatal death in terms of the number needed to treat. To prevent one neonatal death, one needs to expose 22 fetuses to corticosteroids (95% CI 16, 39) (Table VI). Again, this estimate will vary with baseline risk, following the pattern shown in Fig. 1. These estimates of absolute risk reduction and number needed to treat will be incorrect if randomly assigned babies born at term were excluded from the analysis in the primary reports of the various trials, because term births are at minimal or no risk for RDS or death. In the trials of p-mimetic tocolytics for the treatment of preterm labor, 40% of women in the treated group delivered at term.16 If postrandomization losses of such magnitude occurred in the corticosteroid trials (and even if they occurred equally in the two arms of the trials), the absolute risk reductions reported in the primary publications and in Crowley’s meta-analysis will be considerably overestimated (Fig. 2): reported
Table
VII.
FAVORStXNMlL
corticosteroids on incidence of of Crowley.13 Treatment effect, and 95% confidence interval. If in that trial. Three of 14 trials each of other 11 trials failed is that this set of trials shows absolute risk difference - 4.5%,
Losses to follow-up
Physical growth and development Aukland Amsterdam U.S. Collaborative Psychometric tests Aukland Amsterdam U.S. Collaborative
Yr
70 loss*
6
18
10-12
3
12t 36
6
18
IO-12
18
3
36
*Percentage
of those
tDifferentia1
loss treatment 7%, control 18%.
discharged.
absolute risk reduction 7.9% and actual absolute risk reduction 4.7%. As a further consequence, the number of fetuses needed to treat to prevent one case of RDS or death will be considerably underestimated (Fig. 3): reported number needed to treat 13 and actual number needed to treat 2 1. Next, one can ask the question, “At what point in the history of trials of antenatal corticosteroids for fetal lung maturation was the aggregated evidence sufficient to show that this treatment reduces the incidence of RDS and neonatal death?” One can reconstruct the history of the growth of evidence by ordering the trials by their date of publication and performing a new
342
Sinclair Am J Obstet
July 1995 Gynecol
RISK DIFFERENCE %, 95% Cl -40 STUDY
YEAR
X PTS. 1070
1
Liggins
1972
2
Block
1977
1200
3
Momlson
1978
1326
4
Taeusch
1979
1453
5
Papageorglou
1979
1599
6
Doran
1980
1743
7
Schutte
1980
1864
8
Teramo
1980
1945
9
U.S. Collaborative
1981
2688
10
Schmidt
1984
2753
11
Morales
1986
2998
12
Gamsu
1989
3266
13
Garite
1992
3348
14
Eronen
1993
3414
-30
-20
-10
FAVORSTREATh?Prr
0
+I0
+20
FAVORS CONTROL
Fig. 8. Cumulative meta-analysis of trials testing effect of antenatal corticosteroids on incidence of neonatal death. Data are taken from the trials included in meta-analysis of Crowley.13 Treatment effect, expressed as risk difference, is shown as in Fig. 4 except that meta-analysis is repeated each time a new trial is published. Effect was statistically significant in first trial (Liggins, 1972) and has remained so since. Point estimate for size of effect expressed as risk difference has changed very little since reports of first trial. However, subsequent trials have added increasing precision to estimate of effect, indicated by narrowing of 95% CI.
meta-analysis each time a new trial is published, a technique termed “cumulative meta-analysis.“‘7 Fig. 4 lists the individual trials that tested the effect of antenatal steroids on RDS in the order of their date of publication and shows the absolute risk reduction and its 95% CI for each trial. In the aggregate as previously seen, the risk reduction for RDS is highly statistically significant, although only six of 15 trials had sufficient power to show a statistically significant effect. Fig. 5 presents the same data in the form of a cumulative meta-anslysis, and the result is compelling: the effect was statistically significant in the first trial, and as subsequent trials were reported, it has remained so. Moreover, the point estimate for size of effect, expressed as absolute risk reduction, has changed very little with the addition of data from subsequent trials. However, the 95% CI has narrowed, which indicates increased precision of the estimate of effect as subsequent trials were reported. The same data can also be plotted as a time trend of the accumulating evidence that corticosteroids before preterm delivery reduce the incidence of RDS (Fig. 6). Conventional (Fig. 7) and cumulative meta-analyses (Figs. 8 and 9) are also shown for the effect on neonatal death. The risk reduction for neonatal death was statistically significant in the first trial and has remained so
as each subsequent trial is added to the cumulative meta-analysis. The point estimate for the size of effect, which is expressed as absolute risk reduction and as the number needed to treat, has changed very little since the reports of the first trial, but the precision of the estimate has increased over time. Thus we have known since 1972 and with a high degree of confidence that antenatal corticosteroids reduce the incidence of RDS and neonatal death. Therefore one might ask, “What has been the rationale for new trials during the past 20 years that have either a placebo or no-treatment control group?” Long after the evidence was in hand that antenatal corticosteroids reduced the incidence of RDS, new placebo-controlled trials were reported in which RDS was the major outcome of interest. Certainly there was a substantial lag time between the launching of a trial and the publication of its results. However, one wonders whether cumulative meta-analysis of the results of previous trials, had it been available, would have persuaded against the launching of some of the later placebo-controlled trials testing the efficacy of antenatal corticosteroids in the prevention of RDS. More understandable was the rationale for new trials designed primarily to test the effect of corticosteroids in specific subgroups, such as preterm prelabor rupture of the membranes, or to assess the
Volume 173, Number Am J Obstet Gym01
1
Sinclair
NUMBER NEEDED TO TREAT PREVENT ONE NEONATAL DEATH:
1970
1975
1980
343
WITH CORTICOSTEROID TO CUMULATIVE META-ANALYSIS
1985
1990
1993
Fig. 9. Time trend of accumulating evidence that corticosteroids administered before preterm delivery reduce incidence of neonatal death. Effect was statistically significant in first trial (1972) and has remained so since. Point estimate for size of effect, expressed as number needed to treat, has changed very little since reports of first trial. However, subsequent trials have added increased precision to estimate of effect indicated by narrowing of 95% confidence limits (95% CL) over time. Number needed to treat is calculated from absolute risk reduction in trials reviewed by Crowley.‘3 NNT = 22, 95% CI 16, 39. effect primarily on a different outcome, such as infection in the mother or newborn or late outcomes in surviving infants and children. Future research needs Long-term follow-up. Concern over late outcomes of corticosteroid-exposed fetuses has been addressed by follow-up reports in three of the trials reviewed by Crowley. She found no evidence of an adverse longterm effect as measured by neurologic abnormality at follow-up. However, the aggregate incidence of neurologic abnormality at follow-up in the treatment and control groups in these trials, 5.7% and 8.5% respectively, is small in relation to the loss to follow-up rates, which ranged from 12% to 36% in the three trials (Table VII). This must dictate caution in the interpretation of the available follow-up data. I believe that the paucity of valid late outcome data is the weak point in the evidence from randomized trials reviewed by Crowley, and therefore I disagree with her assertion that “further follow-up studies of growth or psychomotor development would seem unnecessary.” Rather, I believe that every effort should be made to trace the survivors of all trials, old and recent, that have been performed to date and to ascertain long-term outcomes in all relevant spheres.
Translation of evidence into practice. Crowley documents a low use of antenatal corticosteroids by obstetricians despite the evidence of its efficacy in reducing RDS and death rates. In addition to her examples, I would add the observation that in the Canadian multicenter trial of ritodrine for preterm labor, corticosteroids were used in only 35% and 36% of the treatment and control groups, respectively. Crowley advances several possible reasons for such low use of this treatment by obstetricians, including varied interpretations of the evidence on efficacy, innate conservatism, or fear of long-term ill effects. One could suggest several additional reasons: varied interpretation of the evidence on short-term safety (e.g., concerns about maternal or fetal infection or both, safety in subgroups such as maternal diabetes, and maternal hypertensive disease), lack of dissemination of the evidence in a clinically meaningful form, inability of physicians to recognize valid evidence when they confront it, and a style of medical practice that is not evidence based.” To the extent that the problem is in the dissemination of evidence and physicians’ performance in acting on it, there is a research opportunity to test alternative strategies for the promotion of evidence-based practice in respect to the use of antenatal corticosteroids for fetal maturation. For example, one might compare the strat-
344
Sinclair Am J Obstet
egies of “educational influential” and “audit and feedback” as was done in a randomized trial of those two strategies for increasing the offering of vaginal birth after cesarean section.” I suggest that the primary outcome of interest in such a comparison should be the extent of use of antenatal corticosteroids in threatened preterm delivery at a gestational age of 34 weeks or less.
11.
12.
13.
REFERENCES
1. Glass GV, McCaw B, Smith ML. Meta-analysis in social research. Beverley Hills, California: Sage Publications, 1981. 2. Light RJ, Pillemer DB. Summing up: the science of reviewing research. Cambridge, Massachusetts: Harvard University Press, 1984. 3. Sacks HS, Berrier J, Feitman D, et al. Meta-analyses of randomized controlled trials. New Engl J Med 1987;316: 450-5. 4. Chalmers TC, Lau J. Meta-analytic stimulus for changes in clinical trials. Stat Methods Med Res 1993:2:161-72. 5. Chalmers I, Enkin M, Keirse MJNC. Effective care in pregnancy and childbirth. Oxford, England: Oxford University Press, 1989. 6. Sinclair lC, Bracken MB. Effective care of the newborn infant. Oxford, England: Oxford University Press, 1992. 7. Bracken MB. Statistical methods for analvsis of effect of treatment in overviews of randomized trials. In: Sinclair JC, Bracken MB, eds. Effective care of the newborn infant. Oxford, England: Oxford University Press, 1992: 13-8. 8. Fleiss JL. The statistical basis of meta-analysis. Stat Methods Med Res 1993;2:121-45. 9. Cook DJ, Guyatt GH, Ryan G, et al. Should unpublished studies be included in meta-analyses? JAMA 1993;269: 2749-53. 10. Chalmers TC, Smith H, Blackburn B, et al. A method for assessing the quality of a randomized trial. Controlled Clin Trials 1981;2:31-49.
14.
15.
16.
17.
18.
19.
July 1995 Gynecol
Reisch JS, Tyson JE, Mize SG. Aid to the evaluation of therapeutic studies. Pediatrics 1989;84:815-27. Chalmers TC, Celano P, Sacks HS, Smith H Jr. Bias in treatment assignment in controlled clinical trials. New Engl J Med 1983;309:1358-61. Crowley P. Update of the antenatal steroid meta-analysis: current knowledge and future research needs. In National Institute of Child Health and Human Development (US). Report of the Consensus Development Conference on the Effect of Corticosteroids for Fetal Maturation on Perinatal Outcomes; 1994 Feb 2%Mar 2; Bethesda, MD. Bethesda (MD): The Institute; 1994 Nov. p.161-205 (NIH publication; No. 95-3784). Crowley P. Corticosteroids prior to preterm delivery. In: Enkin MW, Keirse MJNC, Renfrew MJ, Neilson JP, eds. Pregnancy and childbirth module. Cochrane database of systematic reviews: Review No. 2955, 26 April 1993. Published through Cochrane Updates on Disk, Oxford, England. Update Software, spring 1993. Usher RH, Allen AC, McLean FH. Risk of respiratory distress syndrome related to gestational age, route of delivery, and maternal diabetes. AM J OBSTET GYNECOL 1971;111:826-32. Keirse MJNC. Beta-mimetic tocolytics in preterm labor. In: Enkin MW, Keirse MJNC, Renfrew MJ, Neilson JP, eds. Pregnancy and childbirth module. Cochrane database of systematic reviews: Review No. 3237, Feb. 17, 1993. Published through Cochrane Updates on Disk, Oxford, England. Update Software, spring 1993. Lau J, Amman EM, Jimenez-Silva J, Kupelnick B, Mosteller F, Chalmers TC. Cumulative meta-analysis of therapeutic trials for myocardial infarction. New Engl J Med 1992;327:248-54. Evidence-based Medicine Working Group. Evidencebased medicine: a new approach to teaching the practice of medicine. JAMA 1992;268:2420-5. Lomas J, Enkin M, Anderson GM, Hannah WJ, Vayda E, Singer J. Opinion leaders vs audit and feedback to implement practice guidelines: delivery after previous cesarean section. JAMA 1991;265:2202-7.
Mugford M, Piercy J. Chalmers I. Cost implications of different approaches to the prevention of respiratory distress syndrome. Arch Dis Child 1991;66:757-64. Crowley P, Chalmers I, Keirse MJ. The effects of corticosteroid administration before preterm delivery: an overview of the evidence from controlled trials. Br J Obstet Gynaecol 1990;97: 1 l-25. Garite TJ, Freeman RK, Linzey EM, Braly PS, Dorchester WL. Prospective randomized study of corticosteroids in the management of premature rupture of the membranes and the premature gestation. AM J OBSTET GYNECOL 1981; 141:508-15. Iams JD, Talbert ML, Barrows H, Sachs L. Management of prematurely ruptured membranes: a prospective randomized comparison of observation versus steroids and timed delivery. AM J OBSTET GYNECOL 1985;151:32-8. Nelson LH, Meis PJ, Hatjis CG, Ernest JM, Dillard R, Schey HM. Premature rupture of membranes; a prospective, randomized evaluation of steroids, latent phase and expectant management. Obstet Gynecol 1985;66:55-8.
76.
77.
78.
79.
80.
Farrag OAM. Prospective study of three metabolic regimensin pregnant diabetics. A&t N Z J Obstet Gynaecol 1987;27:6-9. Keirse MJNC, Kanhai HHH. An obstetrical viewpoint on preterm birth with particular reference to perinatal mortality and morbidity. In: Huisjes HJ, ed. Aspects of perinatal morbidity. Groningen: Universitaire Boekhandel Nederaland, 1981:1-35. Tubman TRJ, Rollins MD, Patterson C, Halliday HL. Increased incidence of respiratory distress syndrome in babies of hypertensive mothers. Arch Dis Child 1991;66: 52-4. Lamont RF, Dunlop PDM, Levene MI, Elder MI. Use of glucocorticoids in pregnancies complicated by severe hypertension and proteinuria. Br J Obstet Gynaecol 1983; 90:199-202. Farrell PM, Engle MJ, Zachman RD, et al., and the Collaborative Group on Antenatal Steroid Therapy. Amniotic fluid phospholipids after maternal administration of dexamethasone. AV J OSSTET GYNECOL 1983;145:484-90.
81.
82.
83.
84.
Meta-analysis of randomized controlled trials of antenatal corticosteroid for the prevention of respiratory distress syndrome: Discussion*“f John
C. Sinclair,
Hamilton,
MD
Ontario,
Canada
A meta-analysis rize
results
research.
estimate
defined
population because
trials,
it must
a systematic in
the
the
to obtain
effect
for
each
of
estimate
of
that
the can
lead
trials
treatment of interest.
results
Table I. Structure effect of treatment
to summaof scientific
an unbiased,
outcome
include review
review
of randomized
seeks of
methods
a systematic review
intervention
mary
sion
quantitative
from
A systematic
medical
over,
uses
derived
of
a
in
a
More-
$3.00
-t 0
No event
Total
a
b
n,
C
d m.
no T
Treated Control
ml
effect and greater than can review of any single
treatment
generalizability of findings study. The opportunity to draw more definitive conclusions from a series of similar studies is not restricted to medicine or to randomized trials. Some of the first
0002.9378/95
Event
Exposure
preci-
From the Departments of Pediatrics and Clinrcal Epidemiolog? and Biostatistics, McMaster University. Reprint requests: John C. Sinclair, MD, Departments of Pedialrics and Clinical Epidemiology and Biostatistics, McMaster University, 1200 Main St. West, Hamilton, Ontario, Canada, L8N 325. *This article is based on the data presented by Dr. Crowley at the Consensus Development Conference on the Effect of Corticosteroids for Fetal Maturation on Perinatal Outcomes, Bethesda, Maryland, February S-March 2, 1994. tA similar report based on this presentation has been previously published: Sinclair JC. Meta-analysis of randomized controlled trials: Antenatal corticosteroid for prevention of respiratory distress syndrome, Prospettive in Pediatria 1994;24:199-206. AM J OBSTET GYNEC~L 1995;173:335-44. Copyright 0 1995 by Mosby-Year Book, Inc. 6/O/63092
the
Outcome
sum-
of all qualifying
to increased
of a study to evaluate
Event Event
rate rate
Relative
risk
Odds
= a/n,. = c/n,.
0
risk reduction = a/n, - c/n,. risk reduction = 1 - relative risk. needed to treat = l/absolute risk reduction.
attempts clinical
= 2
group group
alb = a.
ratio
Absolute Relative Number
made
in treated in control
to synthesize
research
in the social sciences.‘, trials,
which
have
rather
in an empiric
* However, standard
way
were
randomized methodology,
particularly suited to meta-analysis.3 Since the early 198Os, there has been a sharp increase in the rate of publication of meta-analyses of randomized controlled are
trials4 which ized
Two
textbooks
essentially trials
of
consist therapy
have of covering
recently
been
meta-analyses an
entire
published, of
random-
discipline.‘,
’ 335
336
Sinclair Am J Obstet
GESTATIONAL AGE (wk) 36 35 34 33 32 IllIll
31
30 I
29 I
Table II. Overall effect of corticosteroids administered before preterm delivery on RDS
526 1
Point estimate Relative risk reduction (%) Absolute risk reduction (%) No. needed to treat
95%
-41 -7.9
I
20
I
I
40 BASELINE
I
I
60 RISK
1
13
80 (%)
Fig. 1. Number of fetuses requiring treatment with antenatal corticosteroids to prevent one case of RDS as a function of baseline risk. Number needed to treat is derived from typical relative risk reduction of 41% calculated from data of trials included in meta-analysis of Crowley.” Gestational ages corresponding to baseline risk are based on data of Usher et al.” Shaded zone indicates 95% CI.
Structure
of a trial to evaluate
treatment
Table I displays the structure of a typical study that seeks to evaluate treatment. There are two exposure groups (labeled treated or control) and two possible outcomes (labeled event or no event). Particularly important are studies in which patients are assigned to the treated or control group by a random procedure. Estimators of treatment effect for binary analysis of the results of a randomized trial are given in Table I. The same estimators are used when analyzing the results of a set of comparable trials in a meta-analysis, as will be discussed below. Meta-analysis:
Method
A systematic review of randomized trials requires that all relevant and “groupable” trials be identified, disclosing searching methods and using predefined inclusion and exclusion criteria. Next, the trials are examined for their methodologic quality; those trials whose methods fail to meet predefined criteria for methodologic quality are excluded. The results of the included trials are then extracted and tabulated, and a summary estimate of treatment effect is calculated for each outcome of interest. This summary effect (sometimes termed a “typical” effect) is some form of weighted average across trials; the weights are inversely related to the variance in the estimate of treatment effect provided by each participating trial.
Relative risk reduction (%) Absolute risk reduction (%) No. needed to treat
- 10.2
10, III)
18
of Crowley,
1994
on
Point estimate
1
-49
- 5.6,
Table III. Effect of corticosteroids administered before preterm delivery RDS in fetuses < 3 1 weeks’ gestation
I
CI
-32,
Calculated from meta-analysis (Table (15 trials, 588 events, 3438 subjects).
I
July 1995 Gym01
95%
CI
-37
-23,
-49
-19
-11,
-27
5
Calculated from meta-analysis (Table (9 trials, 203 events, 461 subjects).
4, 9 VII)
of Crowley,
1994
Table IV. Effect of corticosteroids given before preterm delivery on RDS in fetuses > 34 weeks’ gestation Point estimate Relative risk reduction (%) Absolute risk reduction (%) No. needed to treat Calculated from meta-analysis (8 trials, 29 events, 886 subjects).
95%
-37
CI
+29,
-69
-1.1
+1.3,
-3.4
94
30,
(Table
VIII)
infinite
of Crowley,
1994
For categoric outcomes, estimators of treatment effect include relative risk (and its complement, relative risk reduction), odds ratio, and absolute risk difference (and its inverse, number needed to treat); outcomes measured on a continuous scale include mean difference. Statistical methods are available for calculating each of these estimators of treatment effect and its confidence interval both for individual trials and for typical (weighted) estimates across a set of trials (e.g., refs. 7 and 8). These methods are based on either fixed-effect or random-effect models. The fixed-effect model presumes that there is no statistical heterogeneity between studies, a presumption that is tested by performing a statistical test to detect heterogeneity. The random-effect model assumes statistical heterogeneity and incorporates it into the estimate of effect. As a
Volume 173, Number Am J Obstet Gym01
EXCLUSION EFFECT
1
Sinclair
OF PATIENTS ON ABSOLUTE
DESTINED FOR NON-EVENT: RISK REDUCTION (ARR)
EXCLUSION EFFECT
337
OF PATIENTS DESTINED FOR NON-EVENT: ON NUMBER NEEDED TO TREAT (NNT)
80-
ReporkdNNTs40
60 -
Reported NNT=ZO
ReportedNNT=
10
Reported NNi-= 5 /
0
.lO
.20 FRACTION
.30 .40 EXCLUDED
Fig. 2. Relationship between signed patients excluded from tined for nonevent, reported actual absolute risk reduction. patients so excluded is 0.40 and tion is lo%, actual absolute risk
Absolute reduction
<31 31-34 234 Calculated from GA, Gestational
administered
before
.I0
20
risk (%)
meta-analysis (Tables III, VII, and VIII) age; NNT, number needed to treat.
I
I
.30
.40
.50
EXCLUDED
between proportion of randomly assigned patients excluded from analysis because they are destined for nonevent, reported number needed to treat, and actual number needed to treat. For example, if proportion of patients so excluded is 0.40 and reported number needed to treat is 10, actual number needed to treat is 17.
95% -6, -3, +3,
delivery
on RDS: Effect of gestational
CI
NNT
-42 -10 -4
of Crowley,
intervals may be somewhat wider for the random-effect model. of relevant trials. The goal of a is to identify all studies that meet inclusion criteria and to report the methods used to achieve this. There is some evidence of a “publication bias” that favors the submission and publication of trials that show “positive” results. To minimize the effect of
Reported NM = 2
I
Fig. 3. Relationship
preterm
-24 -7 -0.7
result, confidence analyses based on Ascertainment systematic review
I
FRACTION
proportion of randomly asanalysis because they are desabsolute risk reduction, and For example, if proportion of reported absolute risk reducreduction is 6%.
Table V. Effect of corticosteroids at entry GA class (wk)
01 0
.50
I
95%
4 15 145 1994
(4 trials,
285 events,
age
CI
2, 17 10, 31 25, infinite 2086
subjects).
publication bias, the reviewer may strive to identify unpublished as well as published studies, although the inclusion of unpublished studies is controversial.g Validity assessment of primary studies. Objective assessment of the methodologic quality of the primary studies requires that a systematic scheme be applied to assess validity. Methodologic standards for the design, conduct, analysis, and reporting of random-
338
Sinclair
1995 Gym01
July
Am J Obstet
RISK DIFFERENCE %, 95% Ct -40 STUDY
YEAR
CPTS.
1
Llggins
1972
1070
2
Block
1977
130
3
Morrlson
1978
126
4
Taeusch
1979
127
5
Papageorglou
1979
146
6
Dotan
1980
144
7
Schutte
1980
122
8
Teramo
1980
80
9
U.S. Collaborative
1981
743
10
Schmidt
1984
65
11
Morales
1986
245
12
Gamsu
1989
268
13
Carlan
1991
24
14
Garite
1992
82
15
Eronen
1993
66
-30
-20
-10
I
I
I
0
+lO
+20
I
3438
OVERALL
FAVORSTFiEAlMENT
FAVORS CO~IROL
Fig. 4. Conventional meta-analysis of trials testing effect of antenatal corticosteroids on incidence of RDS. Data are taken from trials included in meta-analysis of Crowley.‘3 Treatment effect, expressed as risk difference, is shown for each trial as point estimate and 95% CI. If 95% CI does not cross zero, effect is statistically significant (p < 0.05) in that trial. Six of 15 trials showed a statistically significant reduction in RDS, whereas each of other nine trials failed to demonstrate a statistically significant effect. Overall, typical estimate is that this set of trials shows a reduction in incidence of RDS that is statistically highly significant: absolute risk difference - 7.9%, 95% CI - 5.6%, - 10.2%.
ized
trials
However, weights
to
evidence critical
be
that
a basis is not
general
put
on
control
requirement
blinding cian
provide there
of the and
allocated
for
rating
different of
for
of
agreement elements.
selection
validity.‘*
randomization
patient
cannot
predict
to that
patient.
However,
validity.“, on the
bias This
‘I relative
There at
is
entry
is
is attempted
process
so that
which
treatment
other
the
cliniwill
elements
a by be
(such
as complete follow-up, blinding of the outcome measurement, and where possible of the intervention) may be equally important. Until better evidence is available on the relative weights to be attached to different aspects of the study design, it may be best simply to describe the methodologic features of each report and include these evaluations in the review (e.g., the tables that are in the appendix to each chapter in Effective Care of the Newborn Infant,” in which the reviewer reports critical methodologic ‘features of each cited trial). Reporting
treatment
effect
in
a clinically
meaning-
The results of randomized trials are important not only as hypothesis tests but also as the preferred source of valid data for informing explicit, quantitative ful
way.
judgments regarding the prescription of treatment, clinical decision analysis, and cost-effectiveness analysis. Therefore in reporting the results of individual randomized trials, as well as mgta-analyses of sets of randomized trials, it should be recognized that the size of difference and not just the fact of difference will have direct implications for clinical practice. The measures of treatment effect that are reported should facilitate their practical application. The risk ratio, or relative risk, is the traditional effect estimator from prospective studies of which randomized trials are a special case. This estimator provides an answer to the clinically important question, “What is the proportion of treated patients, relative to control patients, who experience an event?” Its complement, relative risk reduction, provides an answer to a second clinically important question, “By how much, in relative terms, is the event rate reduced?” The risk difference provides an answer to a third clinically important question, “What is the absolute difference in event rate between the treated and control groups?” Its inverse, number needed to treat, provides an answer to a fourth clinically important question,
Volume 173, Number Am J Obstet Gynecol
Sinclair
1
339
RISK DIFFERENCE %, 95% Cl -40
-30
-20
-10
I
I
I
STUDY
YEAR
IPTS.
1
Uggins
1972
1070
2
Block
1977
1200
3
Morrls0n
1978
1326
4
Taeuxh
1979
1453
5
Papageorgiou
1979
1599
-
6
Doran
1960
1743
-
7
Schutte
1980
166-l
-
8
Teramo
1980
1645
9
U.S. Collaborative
1981
2688
10 Schmidt
1984
2x3
11 Morales
1986
2998
12 Gamsu
1989
3266
13 carlan
1991
3290
14 Garite
1992
3372
15
1993
3438
Eronen
0
+lO
+20
-
-
FAVORSTREATMENT
FAVORS CONlWX
Fig. 5. Cumulative meta-analysis of trials testing effect of antenatal corticosteroids on incidence of RDS. Data are taken from trials included in meta-analysis of Crowley.” Treatment effect, expressed as risk difference, is shown as in Fig. 4 except that meta-analysis is repeated each time a new trial is published. Effect was statistically significant in first trial (Liggins, 1972) and has remained so since. Point estimate for size of effect expressed as risk difference has changed very little since reports of first trial. However, subsequent trials have added increasing precision to estimate of effect, which is indicated by narrowing of 95% CL
“How many patients does one need to treat to prevent one patient experiencing an event?” Despite the popularity of odds ratio as an effect estimator in meta-analysis, it suffers from the fact that it does not directly address a clinically important question regarding size of treatment effect. However, when the incidence of the event in the control group (i.e., the pretreatment risk) is known, the odds ratio can be converted to the clinically more meaningful risk ratio: Risk ratio
OR = 1 + PR (OR _ 1)
where OR = odds ratio and PR = pretreatment risk. For example, a highly effective treatment was tested in a high-risk population (pretreatment risk, 0.43). Treatment effect, expressed as odds ratio, was 0.37. The user can derive the corresponding risk ratio in this population, which is 0.5 1 (i.e., 49% relative risk reduction), with the above equation.
Crowley’s meta-analysis of effect of antenatal corticosteroid on respiratory distress syndrome, neonatal death, and other outcomes Crowley’” has updated the results of randomized
her existing meta-analysisI trials that have evaluated
of the
effect of antenatal corticosteroid for fetal lung maturation. This updated meta-analysis is based on the results of 18 trials, 15 of which provided information on clinical outcomes and randomized a total of 3438 fetuses. The major results of this meta-analysis are that antenatal corticosteroids produce statistically significant reductions in the odds of respiratory distress syndrome (RDS) (typical odds ratio 0.51, 95% confidence interval [CI] 0.42, 0.61) and neonatal death (typical odds ratio 0.61,95% CI 0.48, 0.76). A fixed-effect model was presumed in the metaanalysis. No statistical heterogeneity was detected with respect to the effect on RDS or death. Great care and effort were exercised to identify all relevant trials, both published and unpublished. The quality of each trial was systematically evaluated for three characteristics: blinding of randomization, whether primary analyses were based on all patients randomized, and blinding of outcome measurements. Of these three criteria, the one Crowley considered the most important was the control of selection bias at entry by the care taken to blind the randomization. However, she noted that “post-randomization exclusions occurred in all trials due to some women . remaining undelivered until term and being lost to follow-up.” She does not report the magnitude of this problem, but its
340
Sinclair Am J Obstet
July 1995 Gynecol
NUMBER NEEDED TO TREAT WITH CORTICOSTEROID TO PREVENT ONE CASE OF RDS: CUMULATIVE META-ANALYSIS
3500
Point Estimatt 95% CL
1970
1975
1980
1985
1990
1993
Fig. 6. Time trend of accumulating evidence that corticosteroids administered before preterm delivery reduce incidence of RDS. Effect was statistically significant in first trial (1972) and has remained so since. Point estimate for size of effect, expressed as number needed to treat, has changed very little since reports of first trial. However, subsequent trials have added increasing precision to estimate of effect, indicated by narrowing of 95% confidence limits (95% CL) over time. Number needed to treat is calculated from absolute risk reduction in trials reviewed by Crowley.r3 NNT = 13, 95% CI 10, 18.
Table VI. Effect of corticosteroids administered before preterm delivery neonatal deaths Point estimate
Absolute risk reduction (%) No. needed to treat
-4.5 22
on
95% -2.6,
CI -6.4
16, 39
Calculated from meta-analysis (Table XVII) of Crowley, 1994 (14 trials, 318 events, 3414 subjects).
clinical importance could be substantial. The impact of such deletions from analysis on the reported size of treatment effect will be considered later. Crowley’s meta-analysis presents only the odds ratio as a measure of treatment effect. It is unfortunate that this treatment effect estimator lacks clinical meaning. For example, an important clinical question not directly addressed by the odds ratio is: “How may fetuses does one have to expose to antenatal corticosteroid to prevent one case of RDS or one neonatal death?” Fortunately, one can use the individual trial data presented by Crowley and redo the meta-analyses to calculate clinically meaningful estimates of treatment effect. The typical estimates for efficacy of antenatal corticosteroid in reducing the incidence of RDS are as
follows: relative risk reduction 41%, absolute risk reduction 7.9%, number needed to treat 13. Table II give these results and their 95% CIs. Assuming constant relative risk, one can then calculate the absolute risk reduction over a range of baseline risks for RDS and the corresponding number needed to treat. These results are given in Fig. 1, in which it is clearly evident that the number of fetuses requiring treatment with antenatal corticosteroids to prevent one case of RDS varies with baseline risk. At a high baseline risk of 2 50% (corresponding to gestational ages of 530 weeks), the number needed to treat is low, I 5. However, at a low baseline risk of 15% (corresponding to gestational ages above 34 weeks), the number needed to treat rises sharply. This phenomenon is also demonstrated by using the data from the primary reports reviewed by Crowley to calculate clinically meaningful estimators of effect, including the number needed to treat, for babies of different gestational age classes. If gestational age is less than 31 weeks, one needs to treat five fetuses to prevent one case of RDS (Table III), which is based on data from nine trials. If gestational age is more than 34 weeks, one needs to treat 94 fetuses to prevent one case of RDS (Table IV), which is based on data from eight trials. Because the effect in this latter gestational age group is not quite statistically significant, the upper
Volume 173, Number Am J Obstet Gynecol
Sinclair
1
341
RISK DIFFERENCE X 95% Cl -lo STUDY
YEAR
t PTS.
1
Ugglns
1972
1070
2
Block
1977
130
3
uortlsan
1978
126
4
.Taeusch
1979
127
1979
146
5
Papageorglou
6
Dorm
1980
144
7
Schulte
1980
122
8
Teramo
1980
80
9
U.S. CollaboraIive
1981
743
10 Schmidt
1984
85
11 Morales
1986
245
12 Gamsu
1989
268
13 Garite
1992
52
14 Eronen
1993
66
OVERALL
-30
-20
-10
I
1
I
0
+lO
+20
1 -
3414
FAVORSTREATMENT
Fig. 7. Conventional
meta-analysis of trials testing effect of antenatal neonatal death. Data are taken from trials included in meta-analysis expressed as risk difference, is shown for each trial as point estimate 95% CI does not cross zero, effect is statistically significant Cp < 0.05) showed a statistically significant reduction in neonatal deaths, whereas to demonstrate a statistically significant effect. Overall, typical estimate a reduction in incidence of RDS that is statistically highly significant: 95% CI -2.6%, -6.4%.
bound of the 95% confidence interval for the number needed to treat includes infinity. On the basis of data from four trials that present their results by each of three gestational age strata, the number needed to treat at gestational ages c 3 1 weeks, 3 1 to 34 weeks, and >34 weeks is 4, 15, and 145, respectively (Table V). Similarly, one can estimate the effect of corticosteroids on neonatal death in terms of the number needed to treat. To prevent one neonatal death, one needs to expose 22 fetuses to corticosteroids (95% CI 16, 39) (Table VI). Again, this estimate will vary with baseline risk, following the pattern shown in Fig. 1. These estimates of absolute risk reduction and number needed to treat will be incorrect if randomly assigned babies born at term were excluded from the analysis in the primary reports of the various trials, because term births are at minimal or no risk for RDS or death. In the trials of p-mimetic tocolytics for the treatment of preterm labor, 40% of women in the treated group delivered at term.16 If postrandomization losses of such magnitude occurred in the corticosteroid trials (and even if they occurred equally in the two arms of the trials), the absolute risk reductions reported in the primary publications and in Crowley’s meta-analysis will be considerably overestimated (Fig. 2): reported
Table
VII.
FAVORStXNMlL
corticosteroids on incidence of of Crowley.13 Treatment effect, and 95% confidence interval. If in that trial. Three of 14 trials each of other 11 trials failed is that this set of trials shows absolute risk difference - 4.5%,
Losses to follow-up
Physical growth and development Aukland Amsterdam U.S. Collaborative Psychometric tests Aukland Amsterdam U.S. Collaborative
Yr
70 loss*
6
18
10-12
3
12t 36
6
18
IO-12
18
3
36
*Percentage
of those
tDifferentia1
loss treatment 7%, control 18%.
discharged.
absolute risk reduction 7.9% and actual absolute risk reduction 4.7%. As a further consequence, the number of fetuses needed to treat to prevent one case of RDS or death will be considerably underestimated (Fig. 3): reported number needed to treat 13 and actual number needed to treat 2 1. Next, one can ask the question, “At what point in the history of trials of antenatal corticosteroids for fetal lung maturation was the aggregated evidence sufficient to show that this treatment reduces the incidence of RDS and neonatal death?” One can reconstruct the history of the growth of evidence by ordering the trials by their date of publication and performing a new
342
Sinclair Am J Obstet
July 1995 Gynecol
RISK DIFFERENCE %, 95% Cl -40 STUDY
YEAR
X PTS. 1070
1
Liggins
1972
2
Block
1977
1200
3
Momlson
1978
1326
4
Taeusch
1979
1453
5
Papageorglou
1979
1599
6
Doran
1980
1743
7
Schutte
1980
1864
8
Teramo
1980
1945
9
U.S. Collaborative
1981
2688
10
Schmidt
1984
2753
11
Morales
1986
2998
12
Gamsu
1989
3266
13
Garite
1992
3348
14
Eronen
1993
3414
-30
-20
-10
FAVORSTREATh?Prr
0
+I0
+20
FAVORS CONTROL
Fig. 8. Cumulative meta-analysis of trials testing effect of antenatal corticosteroids on incidence of neonatal death. Data are taken from the trials included in meta-analysis of Crowley.13 Treatment effect, expressed as risk difference, is shown as in Fig. 4 except that meta-analysis is repeated each time a new trial is published. Effect was statistically significant in first trial (Liggins, 1972) and has remained so since. Point estimate for size of effect expressed as risk difference has changed very little since reports of first trial. However, subsequent trials have added increasing precision to estimate of effect, indicated by narrowing of 95% CI.
meta-analysis each time a new trial is published, a technique termed “cumulative meta-analysis.“‘7 Fig. 4 lists the individual trials that tested the effect of antenatal steroids on RDS in the order of their date of publication and shows the absolute risk reduction and its 95% CI for each trial. In the aggregate as previously seen, the risk reduction for RDS is highly statistically significant, although only six of 15 trials had sufficient power to show a statistically significant effect. Fig. 5 presents the same data in the form of a cumulative meta-anslysis, and the result is compelling: the effect was statistically significant in the first trial, and as subsequent trials were reported, it has remained so. Moreover, the point estimate for size of effect, expressed as absolute risk reduction, has changed very little with the addition of data from subsequent trials. However, the 95% CI has narrowed, which indicates increased precision of the estimate of effect as subsequent trials were reported. The same data can also be plotted as a time trend of the accumulating evidence that corticosteroids before preterm delivery reduce the incidence of RDS (Fig. 6). Conventional (Fig. 7) and cumulative meta-analyses (Figs. 8 and 9) are also shown for the effect on neonatal death. The risk reduction for neonatal death was statistically significant in the first trial and has remained so
as each subsequent trial is added to the cumulative meta-analysis. The point estimate for the size of effect, which is expressed as absolute risk reduction and as the number needed to treat, has changed very little since the reports of the first trial, but the precision of the estimate has increased over time. Thus we have known since 1972 and with a high degree of confidence that antenatal corticosteroids reduce the incidence of RDS and neonatal death. Therefore one might ask, “What has been the rationale for new trials during the past 20 years that have either a placebo or no-treatment control group?” Long after the evidence was in hand that antenatal corticosteroids reduced the incidence of RDS, new placebo-controlled trials were reported in which RDS was the major outcome of interest. Certainly there was a substantial lag time between the launching of a trial and the publication of its results. However, one wonders whether cumulative meta-analysis of the results of previous trials, had it been available, would have persuaded against the launching of some of the later placebo-controlled trials testing the efficacy of antenatal corticosteroids in the prevention of RDS. More understandable was the rationale for new trials designed primarily to test the effect of corticosteroids in specific subgroups, such as preterm prelabor rupture of the membranes, or to assess the
Volume 173, Number Am J Obstet Gym01
1
Sinclair
NUMBER NEEDED TO TREAT PREVENT ONE NEONATAL DEATH:
1970
1975
1980
343
WITH CORTICOSTEROID TO CUMULATIVE META-ANALYSIS
1985
1990
1993
Fig. 9. Time trend of accumulating evidence that corticosteroids administered before preterm delivery reduce incidence of neonatal death. Effect was statistically significant in first trial (1972) and has remained so since. Point estimate for size of effect, expressed as number needed to treat, has changed very little since reports of first trial. However, subsequent trials have added increased precision to estimate of effect indicated by narrowing of 95% confidence limits (95% CL) over time. Number needed to treat is calculated from absolute risk reduction in trials reviewed by Crowley.‘3 NNT = 22, 95% CI 16, 39. effect primarily on a different outcome, such as infection in the mother or newborn or late outcomes in surviving infants and children. Future research needs Long-term follow-up. Concern over late outcomes of corticosteroid-exposed fetuses has been addressed by follow-up reports in three of the trials reviewed by Crowley. She found no evidence of an adverse longterm effect as measured by neurologic abnormality at follow-up. However, the aggregate incidence of neurologic abnormality at follow-up in the treatment and control groups in these trials, 5.7% and 8.5% respectively, is small in relation to the loss to follow-up rates, which ranged from 12% to 36% in the three trials (Table VII). This must dictate caution in the interpretation of the available follow-up data. I believe that the paucity of valid late outcome data is the weak point in the evidence from randomized trials reviewed by Crowley, and therefore I disagree with her assertion that “further follow-up studies of growth or psychomotor development would seem unnecessary.” Rather, I believe that every effort should be made to trace the survivors of all trials, old and recent, that have been performed to date and to ascertain long-term outcomes in all relevant spheres.
Translation of evidence into practice. Crowley documents a low use of antenatal corticosteroids by obstetricians despite the evidence of its efficacy in reducing RDS and death rates. In addition to her examples, I would add the observation that in the Canadian multicenter trial of ritodrine for preterm labor, corticosteroids were used in only 35% and 36% of the treatment and control groups, respectively. Crowley advances several possible reasons for such low use of this treatment by obstetricians, including varied interpretations of the evidence on efficacy, innate conservatism, or fear of long-term ill effects. One could suggest several additional reasons: varied interpretation of the evidence on short-term safety (e.g., concerns about maternal or fetal infection or both, safety in subgroups such as maternal diabetes, and maternal hypertensive disease), lack of dissemination of the evidence in a clinically meaningful form, inability of physicians to recognize valid evidence when they confront it, and a style of medical practice that is not evidence based.” To the extent that the problem is in the dissemination of evidence and physicians’ performance in acting on it, there is a research opportunity to test alternative strategies for the promotion of evidence-based practice in respect to the use of antenatal corticosteroids for fetal maturation. For example, one might compare the strat-
344
Sinclair Am J Obstet
egies of “educational influential” and “audit and feedback” as was done in a randomized trial of those two strategies for increasing the offering of vaginal birth after cesarean section.” I suggest that the primary outcome of interest in such a comparison should be the extent of use of antenatal corticosteroids in threatened preterm delivery at a gestational age of 34 weeks or less.
11.
12.
13.
REFERENCES
1. Glass GV, McCaw B, Smith ML. Meta-analysis in social research. Beverley Hills, California: Sage Publications, 1981. 2. Light RJ, Pillemer DB. Summing up: the science of reviewing research. Cambridge, Massachusetts: Harvard University Press, 1984. 3. Sacks HS, Berrier J, Feitman D, et al. Meta-analyses of randomized controlled trials. New Engl J Med 1987;316: 450-5. 4. Chalmers TC, Lau J. Meta-analytic stimulus for changes in clinical trials. Stat Methods Med Res 1993:2:161-72. 5. Chalmers I, Enkin M, Keirse MJNC. Effective care in pregnancy and childbirth. Oxford, England: Oxford University Press, 1989. 6. Sinclair lC, Bracken MB. Effective care of the newborn infant. Oxford, England: Oxford University Press, 1992. 7. Bracken MB. Statistical methods for analvsis of effect of treatment in overviews of randomized trials. In: Sinclair JC, Bracken MB, eds. Effective care of the newborn infant. Oxford, England: Oxford University Press, 1992: 13-8. 8. Fleiss JL. The statistical basis of meta-analysis. Stat Methods Med Res 1993;2:121-45. 9. Cook DJ, Guyatt GH, Ryan G, et al. Should unpublished studies be included in meta-analyses? JAMA 1993;269: 2749-53. 10. Chalmers TC, Smith H, Blackburn B, et al. A method for assessing the quality of a randomized trial. Controlled Clin Trials 1981;2:31-49.
14.
15.
16.
17.
18.
19.
July 1995 Gynecol
Reisch JS, Tyson JE, Mize SG. Aid to the evaluation of therapeutic studies. Pediatrics 1989;84:815-27. Chalmers TC, Celano P, Sacks HS, Smith H Jr. Bias in treatment assignment in controlled clinical trials. New Engl J Med 1983;309:1358-61. Crowley P. Update of the antenatal steroid meta-analysis: current knowledge and future research needs. In National Institute of Child Health and Human Development (US). Report of the Consensus Development Conference on the Effect of Corticosteroids for Fetal Maturation on Perinatal Outcomes; 1994 Feb 2%Mar 2; Bethesda, MD. Bethesda (MD): The Institute; 1994 Nov. p.161-205 (NIH publication; No. 95-3784). Crowley P. Corticosteroids prior to preterm delivery. In: Enkin MW, Keirse MJNC, Renfrew MJ, Neilson JP, eds. Pregnancy and childbirth module. Cochrane database of systematic reviews: Review No. 2955, 26 April 1993. Published through Cochrane Updates on Disk, Oxford, England. Update Software, spring 1993. Usher RH, Allen AC, McLean FH. Risk of respiratory distress syndrome related to gestational age, route of delivery, and maternal diabetes. AM J OBSTET GYNECOL 1971;111:826-32. Keirse MJNC. Beta-mimetic tocolytics in preterm labor. In: Enkin MW, Keirse MJNC, Renfrew MJ, Neilson JP, eds. Pregnancy and childbirth module. Cochrane database of systematic reviews: Review No. 3237, Feb. 17, 1993. Published through Cochrane Updates on Disk, Oxford, England. Update Software, spring 1993. Lau J, Amman EM, Jimenez-Silva J, Kupelnick B, Mosteller F, Chalmers TC. Cumulative meta-analysis of therapeutic trials for myocardial infarction. New Engl J Med 1992;327:248-54. Evidence-based Medicine Working Group. Evidencebased medicine: a new approach to teaching the practice of medicine. JAMA 1992;268:2420-5. Lomas J, Enkin M, Anderson GM, Hannah WJ, Vayda E, Singer J. Opinion leaders vs audit and feedback to implement practice guidelines: delivery after previous cesarean section. JAMA 1991;265:2202-7.