- Email: [email protected]
17 Alpha-hydroxyprogesterone caproate for preterm prevention: issues in subgroup analysis
Editorials
ajog.org
17 Alpha-hydroxyprogesterone caproate for preterm prevention: issues in subgroup analysis Mark A. Klebanoff, MD, MPH
I
n this issue of the American Journal of Obstetrics and Gynecology, Manuck et al1 report the results of a carefullydone analysis describing the characteristics associated with response to 17 alpha-hydroxyprogesterone caproate (17OHP-C) injections among women with a history of spontaneous preterm birth. This article is unusual because it defined response not as a pregnancy lasting to a fixed cut-off such as 37 completed weeks, but rather as a pregnancy lasting at least 3 weeks longer than the most preterm gestation in the treated woman’s obstetric history. This definition may the more clinically relevant one because, at periviable gestational ages, pregnancy prolongation of even 3 weeks increases the probability of neonatal survival from 7e77%.2 Women with a history of more extreme preterm births who were treated with 17OHP-C were more likely to respond; women who experienced bleeding or abruption in the current pregnancy and women whose sister or mother had a history of preterm birth were less likely to respond than were women without these characteristics. By attempting to identify women most likely to respond to 17OHP-C therapy, the authors, in effect, are conducting a subgroup analysis of all treated women. The principles of subgroup analysis of clinical trial results were reviewed in a previous issue of the American Journal of Obstetrics and Gynecology3 and were pointed out again recently.4 The following items are the summary points: (1) “Proper” subgroups are those that can be identified up to the moment of randomization; characteristics that occur after randomization risk being markers for treatment response and, as such, might be considered part of the trial outcome. In addition, something that cannot be assessed before treatment starts is of no help to the clinician in deciding whom to treat. (2) The correct test of whether treatment effectiveness differs by subgroup is not whether the probability value for effect was
From the Departments of Pediatrics, Obstetrics and Gynecology and Epidemiology, The Ohio State University Colleges of Medicine and Public Health Center for Perinatal Research, The Research Institute at Nationwide Children’s Hospital, Columbus, OH. Received Nov. 23, 2015; accepted Dec. 7, 2015. The authors report no conflict of interest. Corresponding author: Mark A. Klebanoff, MD, MPH. mark.klebanoff@ nationwidechildrens.org 0002-9378/free ª 2016 Elsevier Inc. All rights reserved. http://dx.doi.org/10.1016/j.ajog.2015.12.002
Related article, page 376
306 American Journal of Obstetrics & Gynecology MARCH 2016
significant in 1, but not the other, group. Rather, the correct test is to evaluate whether the effect in 1 group differed significantly from the effect in the other, as assessed by the probability value for statistical interaction. (3) Even when done properly, subgroup differences often prove to be spurious.3,4 One reason for this is that subgroup comparisons often result from “fishing expeditions” rather than from a specifically focused rationale. In Bayesian terms, the prior probability that this particular subgroup difference should be “real” is very low.4 (4) Therefore, subgroup differences that are not supported by a strong, ideally a priori, rationale should be considered hypothesis-generating rather than hypothesis-testing. The report of Manuck et al1 is based on an observational cohort study, rather than a randomized clinical trial, but the correct statistical methods to evaluate subgroup differences are identical: one must test for an interaction between receiving treatment and the subgroup-defining characteristic. Instead, the authors limited their analysis to women who received 17OHP-C therapy and determined which characteristics were associated statistically significantly with response, so they cannot conduct this analysis. To their credit, the authors are cognizant of this limitation; they note that, because current American College of Obstetricians and Gynecologists practice recommendations state that all women with a history of spontaneous singleton preterm birth and a current singleton pregnancy be offered progesterone supplementation,5 deliberate withholding of 17OHP-C therapy from these women is unethical. Therefore, it seems likely that eligible women whose treating clinician fails to offer 17OHP-C therapy or who decline their clinician’s offer are very different from women who receive treatment. Indeed, in comments to the Editor, the authors presented data to show that measured characteristics of treated and untreated women are so different that it seems likely that there are numerous and important unmeasured characteristics that differ as well. Although in a randomized study we expect both measured and unmeasured baseline characteristics of treated and untreated women to be similar, the probability of large and even uncontrollable differences in this observational study would render the “correct” analysis uninterpretable, even if it had been done. Against this background, how should we consider the results of this report? In the first paragraph of the discussion, the authors themselves provide the correct answer: one should not assume causality between 17OHP-C treatment and these risk factors. Put simply, in the absence of data on the response rate of a comparable group of untreated women, we have no way to know whether 17OHP-C treatment,
Editorials
ajog.org compared with no treatment, offers greater benefit to women with a history of more extreme prematurity or to women without a family history of preterm birth than to women who do not share these characteristics. Perhaps these are markers for poor outcome in general, but 17OHP-C therapy might still be more helpful than no treatment in these women. The finding that 17OHP-C-treated women with vaginal bleeding/ abruption in the current pregnancy are at elevated risk of recurrence compared with treated women without bleeding/ abruption is even more difficult to interpret, because the bleeding in question might have occurred after 17OHP-C therapy was started. The lack of an untreated control group means that we acknowledge the possibility that bleeding/ abruption might be even a worse prognostic sign in untreated women than in women who receive 17OHP-C therapy (which suggests that, even in women with bleeding, 17OHP-C might offer benefit); therefore, this study should not be taken to mean that 17OHP-C therapy should be stopped solely because a woman experienced bleeding or abruption. And of course, bleeding that might occur after treatment starts is of no help to the clinician in deciding whether to begin treatment. In summary, the report by Manuck et al1 provides interesting data to help clinicians anticipate which women who are otherwise eligible to receive 17OHP-C therapy for
prematurity prevention might respond better to this treatment. Their findings may prove useful to researchers in their quest to understand both the mechanisms of preterm birth and the mechanisms by which 17OHP-C therapy prevents it. However, nothing in this report should be taken to contradict or negate the current American College of Obstetricians and Gynecologists practice bulletin on the use of progesterone to prevent preterm birth. All women with a history of singleton spontaneous preterm birth and a current singleton pregnancy should continue to be offered progesterone for preterm birth prevention.5 REFERENCES 1. Manuck TA, Esplin MS, Biggio J, et al. Predictors of response to 17-alpha hydroxyprogesterone caproate for prevention of recurrent spontaneous preterm birth. Am J Obstet Gynecol 2016;214:376-7. 2. Stoll BJ, Hansen NI, Bell EF, et al. Trends in care practices, morbidity, and mortality of extremely preterm neonates, 1993-2012. JAMA 2015;314:1039-51. 3. Klebanoff MA. Subgroup analysis in obstetrics clinical trials. Am J Obstet Gynecol 2007;197:119-22. 4. Burke JF, Sussman JB, Kent DM, Hayward RA. Three simple rules to ensure reasonably credible subgroup analyses. BMJ 2015;351: h5651. 5. Committee on Practice BulletinseObstetrics, American College of Obstetricians and Gynecologists. Practice bulletin no. 130: prediction and prevention of preterm birth. Obstet Gynecol 2012;120:964-73.
MARCH 2016 American Journal of Obstetrics & Gynecology
307
ajog.org
17 Alpha-hydroxyprogesterone caproate for preterm prevention: issues in subgroup analysis Mark A. Klebanoff, MD, MPH
I
n this issue of the American Journal of Obstetrics and Gynecology, Manuck et al1 report the results of a carefullydone analysis describing the characteristics associated with response to 17 alpha-hydroxyprogesterone caproate (17OHP-C) injections among women with a history of spontaneous preterm birth. This article is unusual because it defined response not as a pregnancy lasting to a fixed cut-off such as 37 completed weeks, but rather as a pregnancy lasting at least 3 weeks longer than the most preterm gestation in the treated woman’s obstetric history. This definition may the more clinically relevant one because, at periviable gestational ages, pregnancy prolongation of even 3 weeks increases the probability of neonatal survival from 7e77%.2 Women with a history of more extreme preterm births who were treated with 17OHP-C were more likely to respond; women who experienced bleeding or abruption in the current pregnancy and women whose sister or mother had a history of preterm birth were less likely to respond than were women without these characteristics. By attempting to identify women most likely to respond to 17OHP-C therapy, the authors, in effect, are conducting a subgroup analysis of all treated women. The principles of subgroup analysis of clinical trial results were reviewed in a previous issue of the American Journal of Obstetrics and Gynecology3 and were pointed out again recently.4 The following items are the summary points: (1) “Proper” subgroups are those that can be identified up to the moment of randomization; characteristics that occur after randomization risk being markers for treatment response and, as such, might be considered part of the trial outcome. In addition, something that cannot be assessed before treatment starts is of no help to the clinician in deciding whom to treat. (2) The correct test of whether treatment effectiveness differs by subgroup is not whether the probability value for effect was
From the Departments of Pediatrics, Obstetrics and Gynecology and Epidemiology, The Ohio State University Colleges of Medicine and Public Health Center for Perinatal Research, The Research Institute at Nationwide Children’s Hospital, Columbus, OH. Received Nov. 23, 2015; accepted Dec. 7, 2015. The authors report no conflict of interest. Corresponding author: Mark A. Klebanoff, MD, MPH. mark.klebanoff@ nationwidechildrens.org 0002-9378/free ª 2016 Elsevier Inc. All rights reserved. http://dx.doi.org/10.1016/j.ajog.2015.12.002
Related article, page 376
306 American Journal of Obstetrics & Gynecology MARCH 2016
significant in 1, but not the other, group. Rather, the correct test is to evaluate whether the effect in 1 group differed significantly from the effect in the other, as assessed by the probability value for statistical interaction. (3) Even when done properly, subgroup differences often prove to be spurious.3,4 One reason for this is that subgroup comparisons often result from “fishing expeditions” rather than from a specifically focused rationale. In Bayesian terms, the prior probability that this particular subgroup difference should be “real” is very low.4 (4) Therefore, subgroup differences that are not supported by a strong, ideally a priori, rationale should be considered hypothesis-generating rather than hypothesis-testing. The report of Manuck et al1 is based on an observational cohort study, rather than a randomized clinical trial, but the correct statistical methods to evaluate subgroup differences are identical: one must test for an interaction between receiving treatment and the subgroup-defining characteristic. Instead, the authors limited their analysis to women who received 17OHP-C therapy and determined which characteristics were associated statistically significantly with response, so they cannot conduct this analysis. To their credit, the authors are cognizant of this limitation; they note that, because current American College of Obstetricians and Gynecologists practice recommendations state that all women with a history of spontaneous singleton preterm birth and a current singleton pregnancy be offered progesterone supplementation,5 deliberate withholding of 17OHP-C therapy from these women is unethical. Therefore, it seems likely that eligible women whose treating clinician fails to offer 17OHP-C therapy or who decline their clinician’s offer are very different from women who receive treatment. Indeed, in comments to the Editor, the authors presented data to show that measured characteristics of treated and untreated women are so different that it seems likely that there are numerous and important unmeasured characteristics that differ as well. Although in a randomized study we expect both measured and unmeasured baseline characteristics of treated and untreated women to be similar, the probability of large and even uncontrollable differences in this observational study would render the “correct” analysis uninterpretable, even if it had been done. Against this background, how should we consider the results of this report? In the first paragraph of the discussion, the authors themselves provide the correct answer: one should not assume causality between 17OHP-C treatment and these risk factors. Put simply, in the absence of data on the response rate of a comparable group of untreated women, we have no way to know whether 17OHP-C treatment,
Editorials
ajog.org compared with no treatment, offers greater benefit to women with a history of more extreme prematurity or to women without a family history of preterm birth than to women who do not share these characteristics. Perhaps these are markers for poor outcome in general, but 17OHP-C therapy might still be more helpful than no treatment in these women. The finding that 17OHP-C-treated women with vaginal bleeding/ abruption in the current pregnancy are at elevated risk of recurrence compared with treated women without bleeding/ abruption is even more difficult to interpret, because the bleeding in question might have occurred after 17OHP-C therapy was started. The lack of an untreated control group means that we acknowledge the possibility that bleeding/ abruption might be even a worse prognostic sign in untreated women than in women who receive 17OHP-C therapy (which suggests that, even in women with bleeding, 17OHP-C might offer benefit); therefore, this study should not be taken to mean that 17OHP-C therapy should be stopped solely because a woman experienced bleeding or abruption. And of course, bleeding that might occur after treatment starts is of no help to the clinician in deciding whether to begin treatment. In summary, the report by Manuck et al1 provides interesting data to help clinicians anticipate which women who are otherwise eligible to receive 17OHP-C therapy for
prematurity prevention might respond better to this treatment. Their findings may prove useful to researchers in their quest to understand both the mechanisms of preterm birth and the mechanisms by which 17OHP-C therapy prevents it. However, nothing in this report should be taken to contradict or negate the current American College of Obstetricians and Gynecologists practice bulletin on the use of progesterone to prevent preterm birth. All women with a history of singleton spontaneous preterm birth and a current singleton pregnancy should continue to be offered progesterone for preterm birth prevention.5 REFERENCES 1. Manuck TA, Esplin MS, Biggio J, et al. Predictors of response to 17-alpha hydroxyprogesterone caproate for prevention of recurrent spontaneous preterm birth. Am J Obstet Gynecol 2016;214:376-7. 2. Stoll BJ, Hansen NI, Bell EF, et al. Trends in care practices, morbidity, and mortality of extremely preterm neonates, 1993-2012. JAMA 2015;314:1039-51. 3. Klebanoff MA. Subgroup analysis in obstetrics clinical trials. Am J Obstet Gynecol 2007;197:119-22. 4. Burke JF, Sussman JB, Kent DM, Hayward RA. Three simple rules to ensure reasonably credible subgroup analyses. BMJ 2015;351: h5651. 5. Committee on Practice BulletinseObstetrics, American College of Obstetricians and Gynecologists. Practice bulletin no. 130: prediction and prevention of preterm birth. Obstet Gynecol 2012;120:964-73.
MARCH 2016 American Journal of Obstetrics & Gynecology
307