BRIEF REPORTS Drugs to Prevent Bronchopulmonary Dysplasia: Effect of Baseline Risk on the Number Needed to Treat Erik A. Jensen, MD, MSCE1,2, Robin S. Roberts, MSc3, and Barbara Schmidt, MD, MSc2,3 Infants born very preterm have a variable baseline risk of bronchopulmonary dysplasia (BPD). Using the example of evidence-based drug therapies to prevent BPD, we designed a visual aid that displays the “number needed to treat” with CIs for caffeine, vitamin A, and hydrocortisone over a range of baseline risks. (J Pediatr 2020;-:1-4).
B
ronchopulmonary dysplasia (BPD) is one of the most frequent and serious complications of very preterm birth, although rates vary considerably between centers and regions.1,2 BPD predisposes affected infants to long-term deficits in growth, cardiopulmonary health, and neurodevelopment.1 Understanding the likelihood of benefit from treatments intended to reduce the risk of BPD may help clinicians, parents, and policy makers select the most appropriate evidence-based therapies for individual patients, centers, and national guidelines. The “number needed to treat” (NNT) is an intuitive measure of the absolute impact of therapies.3,4 However, in contrast to relative measures of treatment effect, published NNT values typically can be applied only to patients whose baseline risks for the outcomes of interest are similar to those of the average risks in the trial participants.3-5 Individualized calculations of the NNT are required for all other patients.5 Visualizing the effect of different baseline risks on the NNT may improve our understanding of the likelihood of a treatment benefit and facilitate the comparison of different treatment options aimed at improving the same outcome. Here we present such a visual aid for a comparison of 3 drug therapies shown to reduce the risk of BPD at 36 weeks postmenstrual age in very preterm infants.6-8
Methods Our data sources were a single randomized trial of caffeine,6 a Cochrane meta-analysis on intramuscular vitamin A,7 and an individual participant data meta-analysis on prophylactic hydrocortisone.8 We estimated the risk difference (RD) and NNT values, with corresponding 95% CIs, for each 1% interval across a plausible range of control group event rates of BPD between 20% and 80%. All RD and NNT values were calculated using relative risks5 estimated from the published
BPD NNT RD
Bronchopulmonary dysplasia Number needed to treat Risk difference
rates of BPD in the study control and intervention groups.6-8 All analyses were conducted using STATA/SE 15.1 (StataCorp, College Station, Texas).
Results The characteristics of the included studies are summarized in the Table. For each therapy, the expected NNT was inversely related to the control group event rate (Figure). A visual display of the 95% CIs around the NNT values highlights the differences in certainty about the size of the treatment effect between the 3 therapies. Clinicians who wish to determine the NNT with its 95% CI for an individual infant or group of infants before choosing a particular drug therapy should consider the control group event rates in the Figure as the baseline risk of BPD for their patients. For example, if the baseline risk of BPD is 20%, then no more than 37 infants on average will need to be treated with caffeine to prevent 1 additional case of BPD. In contrast, the NNTs for vitamin A and hydrocortisone in infants at that same low risk of BPD may be as high as 271 and 861, respectively (Figure).
Discussion Several medications have been shown to reduce the risk of BPD in randomized controlled trials, but the lack of decline in BPD rates over time among very preterm infants continues to frustrate neonatal clinicians.2,9 The baseline risk of BPD among treated infants is one factor that may influence the likelihood of decreasing the rate of BPD. We present a novel visual display of individualized NNT and 95% CI values over a wide range of plausible baseline risks for 3 drug therapies to prevent BPD. For each medication, the expected NNT to prevent 1 case of BPD increased as
From the 1Division of Neonatology, Department of Pediatrics, The Children’s Hospital of Philadelphia, Philadelphia, PA; 2Department of Pediatrics, University of Pennsylvania Perelman School of Medicine, Philadelphia, PA; and 3Department of Health Research Methods, Evidence, and Impact, McMaster University, Hamilton, ON, Canada The authors declare no conflicts of interest. 0022-3476/$ - see front matter. ª 2020 Elsevier Inc. All rights reserved. https://doi.org/10.1016/j.jpeds.2020.01.070
1
2
www.jpeds.com
10 (7-16) 13 (8-108) 14 (7-397)
RR, Relative risk. *Relative risk (RR) and 95% CI values were calculated from the unweighted intervention and control group event rates shown in the table. †NNT and 95% CI values were calculated from the RR point estimate and corresponding 95% CI values shown in the table using the following formula: NNT = 1/[(1 - RR) control group event rate].5
0.78 (0.70-0.86) 0.85 (0.74-0.98) 0.84 (0.71-0.99) 46.9 (447/954) 50.5 (224/444) 43.3 (172/397) 36.3 (350/963) 43.0 (190/442) 36.3 (147/405) Single randomized trial Cochrane review of 4 trials Individual participant data meta-analysis of 4 trials Placebo Sham or no injection Placebo Caffeine6 Vitamin A (intramuscular)7 Hydrocortisone8
Control group Intervention group Control therapy Therapy
Table. Characteristics of the trials for prevention of BPD
Study design
Unweighted event rate, % (n/N)
Unadjusted RR* (95% CI)
NNT† (95% CI)
THE JOURNAL OF PEDIATRICS
Volume -
- 2020
the baseline risk of BPD decreased. Moreover, our analyses show that the use of intramuscular vitamin A and prophylactic hydrocortisone may result only in a small and potentially undiscernible reduction in the rate of BPD if these medications are used in populations with a low average baseline risk of BPD. Our visual display of the NNT with 95% CI values may help explain some apparent discrepancies in the treatment benefits that have been observed among studies. For instance, a large observational study found no association between the risk of BPD and the availability of intramuscular vitamin A during a national drug shortage, in contrast to published trial data.7,10 Notably, the rates of BPD in the treated and untreated infants in this observational study were 44% and 40%, respectively,10 lower than the pooled risk of 51% among the control group participants in the original trials.7 At this lower risk, the upper 95% CI of the NNT suggests that vitamin A may have prevented BPD in as few as 9 study infants, <1% of those who received the drug therapy.10 Although this analysis focuses on BPD, a graphical display of expected NNT values for important dichotomous outcomes could accompany any interventional trial or meta-analysis. Such a tool may enable more accurate application of study results to individual patients. For example, after becoming aware that an expected NNT may be much higher than suggested by the original trial, clinicians, patients, or families may decide against a therapy for some low-risk individuals. Conversely, high-risk patients or their families may look favorably on a treatment that demonstrated only a small and uncertain benefit in a healthier population. To calculate individualized values of the NNT, one must estimate the baseline risk of an outcome for specific patients or groups of patients. These estimations themselves will carry some degree of uncertainty. For the 3 drug therapies in this analysis, which are typically initiated early in the postnatal period, the use of center-specific rates of BPD in the target population may provide the most reliable, readily available estimates of baseline risk. For other circumstances, published outcome risk estimators may be a useful, albeit imperfect, tool. Individualized calculations of the NNT also rely on the assumption that the relative risk of a therapy remains reasonably constant across the range of plausible baseline risks. This assumption is supported by an extensive empirical investigation5 and should be questioned only in the rare instances when there is clear and convincing evidence that the relative effect of a therapy differs as a function of the baseline risk of an outcome. In summary, visual depiction of the NNT with 95% CI over a range of baseline risks for an outcome of interest, such as BPD, may facilitate the evidence-based use of this intuitive, absolute measure of a treatment effect in a heterogeneous group of patients. n
Jensen, Roberts, and Schmidt
- 2020
BRIEF REPORTS A
Caffeine
Event rate, % Caffeine Control (n=963) (n=954)
Risk difference, % (95% CI)
Number needed to treat (95% CI)
80
62.1
-17.9 (-24.3, -10.9)
6 (4, 9)
70
54.3
-15.7 (-21.2, -9.5)
6 (5, 10)
60
46.5
-13.5 (-18.2, -8.2)
7 (5, 12)
50
38.8
-11.2 (-15.2, -6.8)
9 (7, 15)
40
31.0
-9.0 (-12.1, -5.5)
11 (8, 18)
30
23.3
-6.7 (-9.1, -4.1)
15 (11, 24)
20
15.5
-4.5 (-6.1, -2.7)
22 (16, 37)
Number needed to treat (95% CI)
Original study results: Control event rate: 46.9% NNT: 10 (95% CI 7, 16)
5
7
10
25
∞
NNT for 1 less infant to develop BPD
B
Vitamin A
Event rate, % Vitamin A Control (n=442) (n=444)
Risk difference, % (95% CI)
Number needed to treat (95% CI)
80
68.2
-11.8 (-20.8, -1.5)
8 (5, 68)
70
59.6
-10.4 (-18.2, -1.3)
10 (5, 78)
60
51.1
-8.9 (-15.6, -1.1)
11 (6, 90)
50
42.6
-7.4 (-13.0, -0.92)
14 (8, 109)
40
34.1
-5.9 (-10.4, -0.74)
17 (10, 136)
30
25.6
-4.4 (-7.8, -0.55)
23 (13, 181)
20
17.0
-3.0 (-5.2, -0.37)
34 (19, 271)
Number needed to treat (95% CI)
Original study results: Control event rate: 50.5% NNT: 13 (95% CI 8, 108)
5
7
10
25
∞
NNT for 1 less infant to develop BPD
C
25 NNT for 1 more infant to develop BPD
25 NNT for 1 more infant to develop BPD
Hydrocortisone
Event rate, % Control Hydrocortisone (n=397) (n=405)
Risk difference, % (95% CI)
Number needed to treat (95% CI)
80
67.0
-13.0 (-23.5, -0.46)
8 (4, 215)
70
58.6
-11.4 (-20.6, -0.41)
9 (5, 246)
60
50.3
-9.7 (-17.6, -0.35)
10 (6, 287)
50
41.9
-8.1 (-14.7, -0.29)
12 (7, 344)
40
33.5
-6.5 (-11.8, -0.23)
15 (9, 430)
30
25.1
-4.9 (-8.8, -0.17)
21 (11, 574)
20
16.8
-3.2 (-5.9, -0.12)
31 (17, 861)
Number needed to treat (95% CI)
Original study results: Control event rate: 43.3% NNT: 14 (95% CI 8, 397)
5
7
10
25
NNT for 1 less infant to develop BPD
∞
25 NNT for 1 more infant to develop BPD
Figure. Efficacy of caffeine A, intramuscular vitamin A B, and prophylactic, low-dose hydrocortisone C, for the prevention of BPD among very preterm infants. Drugs to Prevent Bronchopulmonary Dysplasia: Effect of Baseline Risk on the Number Needed to Treat
3
THE JOURNAL OF PEDIATRICS
www.jpeds.com
Submitted for publication Nov 8, 2019; last revision received Jan 8, 2020; accepted Jan 31, 2020. Reprint requests: Erik A. Jensen, MD, MSCE, Children’s Hospital of Philadelphia, Suite 2NW21, 34th and Civic Center Blvd, Philadelphia, PA 19104. E-mail:
[email protected]
References 1. Jensen EA, Dysart K, Gantz MG, McDonald S, Bamat NA, Keszler M, et al. The diagnosis of bronchopulmonary dysplasia in very preterm infants. An evidence-based approach. Am J Respir Crit Care Med 2019;200:751-9. 2. Lui K, Lee SK, Kusuda S, Adams M, Vento M, Reichman B, et al. Trends in outcomes for neonates born very preterm and very low birth weight in 11 high-income countries. J Pediatr 2019;215:32-40.e14. 3. Laupacis A, Sackett DL, Roberts RS. An assessment of clinically useful measures of the consequences of treatment. N Engl J Med 1988;318: 1728-33. 4. Saver JL, Lewis RJ. Number needed to treat: conveying the likelihood of a therapeutic effect. JAMA 2019;321:798-9.
4
Volume -
5. Furukawa TA, Guyatt GH, Griffith LE. Can we individualize the “number needed to treat”? An empirical study of summary effect measures in meta-analyses. Int J Epidemiol 2002;31:72-6. 6. Schmidt B, Roberts RS, Davis P, Doyle LW, Barrington KJ, Ohlsson A, et al. Caffeine therapy for apnea of prematurity. N Engl J Med 2006;354:2112-21. 7. Darlow BA, Graham PJ, Rojas-Reyes MX. Vitamin A supplementation to prevent mortality and short- and long-term morbidity in very low birth weight infants. Cochrane Database Syst Rev 2016;8: CD000501. 8. Shaffer ML, Baud O, Lacaze-Masmonteil T, Peltoniemi OM, Bonsante F, Watterberg KL. Effect of prophylaxis for early adrenal insufficiency using low-dose hydrocortisone in very preterm infants: an individual patient data meta-analysis. J Pediatr 2019;207:136-42.e5. 9. Poets CF, Lorenz L. Prevention of bronchopulmonary dysplasia in extremely low gestational age neonates: current evidence. Arch Dis Child Fetal Neonatal Ed 2018;103:F285-91. 10. Tolia VN, Murthy K, McKinley PS, Bennett MM, Clark RH. The effect of the national shortage of vitamin A on death or chronic lung disease in extremely low-birth-weight infants. JAMA Pediatr 2014;168: 1039-44.
Jensen, Roberts, and Schmidt