- Email: [email protected]
Heterogeneous effects on child survival in neonatal vitamin A supplementation trials
Correspondence
Authors’ reply It is not uncommon for risk prediction tools to be based on the “worst” data from the first 24 h after admission for critical care (eg, APACHE and SNAP-II-PE for adult and neonatal critical care, respectively). Use of admission data for risk prediction represents a cross-sectional approach and facilitates a quick management response,1 whereas use of the worst data from the first several hours provides a more longitudinal perspective, probably yielding more robust risk estimates. For the reasons described in the paper,2 we based our prediction model on data obtained (before outcome occurrence) from the first 48 h, especially to permit incorporation of 24-h urine results. We re-examined the fullPIERS database to determine how the published fullPIERS model performed for prediction of outcomes within 48 h of admission using data from 1398 women collected on admission (area under the receiver operating characteristic curve [AUC] 0·857, 95% CI 0·804–0·911), and within 12 h and 24 h of admission (n=1767, AUC 0·863, 0·818–0·908; and n=1865, AUC 0·870, 0·828–0·912, respectively). Our model development strategy balanced statistical and clinical considerations. For example, given highly correlated predictors, our approach was to include the most clinically relevant of the group in the model—a process that cannot be automated and is not suitable for bootstrapping. Therefore, our internal validation assessed the potential for overfitting at the level of the stepwise backwards regression modelling (which included specification of nonlinear variables and assessment of a-priori-specified interaction terms). Internal validation showed that the prediction equation had modest optimism and performed well. However, internal validation strategies have their limitations, and overfitting can only be excluded definitively 1314
through external validation. We have invited, and are facilitating, external validation of the fullPIERS model. How should the clinical community manage women with pre-eclampsia before external validation of the fullPIERS model? The fullPIERS model was developed and internally validated using independently predictive variables from a standardised assessment and surveillance regimen associated with lower rates of adverse maternal events, institutionally and provincially.3,4 The alternative to using the fullPIERS model in clinical practice relies on deeply flawed definitions of “severe” preeclampsia.5 We believe that use of the fullPIERS model represents the better management option at present and have, therefore, hosted an openaccess calculator to facilitate risk estimation and minimise the chance of calculation errors. We declare that we have no conflicts of interest.
*Peter von Dadelszen, Beth A Payne, Jennifer A Hutcheon, K S Joseph, Laura A Magee, for the PIERS Study Group [email protected] Department of Obstetrics and Gynaecology, University of British Columbia and the Child & Family Research Institute, Vancouver, BC V6H 3N1, Canada (PvD, BAP, JAH, KSJ, LAM); School of Population and Public Health, University of British Columbia, Vancouver, BC, Canada (PvD, JAH, KSJ, LAM); and Department of Medicine, University of British Columbia, Vancouver, BC, Canada (LAM) 1
2
3
4
5
Angus DC, Laterre PF, Helterbrand J, et al. The effect of drotrecogin alfa (activated) on long-term survival after severe sepsis. Crit Care Med 2004; 32: 2199–206. von Dadelszen P, Payne B, Li J, et al. Prediction of adverse maternal outcomes in pre-eclampsia: development and validation of the fullPIERS model. Lancet 2011; 377: 219–27. Menzies J, Magee LA, MacNab YC, et al. Instituting surveillance guidelines and adverse outcomes in preeclampsia. Obstet Gynecol 2007; 110: 121–27. von Dadelszen P, Sawchuck D, McMaster R, et al. The active implementation of pregnancy hypertension guidelines in British Columbia. Obstet Gynecol 2010; 116: 659–66. Menzies J, Magee LA, Li J, et al. Current CHS and NHBPEP criteria of severe pre-eclampsia do not uniformly predict adverse maternal or perinatal outcomes. Hypertens Pregnancy 2007; 26: 447–62.
Heterogeneous effects on child survival in neonatal vitamin A supplementation trials Our report of increased female mortality in recipients of neonatal vitamin A supplementation (VAS) in two trials in Guinea-Bissau1,2 has raised concerns in relation to three WHO-sponsored neonatal VAS trials in Ghana, Tanzania, and India, which are in progress to inform global policy. Betty Kirkwood and colleagues (Nov 13, p 1643)3 did a meta-analysis of the sex-specific effects in the neonatal VAS trials, including previously unpublished data from Zimbabwe, and conclude that there is no differential effect of VAS in boys and girls. We disagree. The meta-analysis shows that the effect of VAS was significantly heterogeneous for boys (p=0·004) and overall (p=0·005).3 The interactions between VAS and sex in Guinea-Bissau and Zimbabwe were significantly different (p=0·002). This finding indicates that the effect of VAS is modified by factors that differ between sites. These apparent heterogeneous results might be due to the trial design and the higher HIV prevalence in Zimbabwe. The Guinea-Bissau trials were testing neonatal VAS versus placebo, as are the three ongoing trials. The trial in Zimbabwe was a two-bytwo factorial trial testing maternal VAS (A) or placebo (P), and neonatal VAS (a) or placebo (p), resulting in four treatment groups, Aa, Ap, Pa and Pp (table).4,5 In Zimbabwe, 32% of the mothers were HIV-positive and although mortality results were previously reported separately for HIVnegative and HIV-positive mothers (table), they were pooled in the meta-analysis.3 In previous reports, although the Ap group had borderline or significantly higher mortality than the Pp group, the two groups (Ap+Pp) were combined as the reference to conclude that neonatal VAS (Aa+Pa) www.thelancet.com Vol 377 April 16, 2011
Correspondence
9208 children of HIV-uninfected mothers (170 deaths)*4
4495 children of HIV-infected mothers (865 deaths)*5
14 110 children†‡5
Mother VAS, child VAS (Aa)
1·28 (0·83–1·98)
1·08 (0·88–1·31)
1·06 (0·89–1·26)
Mother VAS, child placebo (Ap)
1·27 (0·82–1·97)
1·24 (1·02–1·50)
1·18 (0·99–1·40)
Mother placebo, child VAS (Pa)
1·18 (0·76–1·83)§
1·21 (0·99–1·46)§
1·16 (0·98–1·38)
Mother placebo, child placebo (Pp)
Reference
Reference
Reference
*Betty Kirkwood, Jean Humphrey, Lawrence Moulton, Jose Martines [email protected]
*Unadjusted results. †Adjusted results. ‡Total included 407 children whose mothers had HIV indeterminate results or seroconverted during follow-up. §In a combined analysis of the unadjusted results from this randomised clinical trial, the mortality hazard ratio comparing Pa vs Pp was 1·21 (95% CI 1·01–1·44).
Department of Epidemiology & Population Health, London School of Hygiene and Tropical Medicine, London WC1E 7HT, UK (BK); Johns Hopkins Bloomberg School of Public Health, Baltimore, MD, USA (JH, LM); and WHO, Geneva, Switzerland (JM) 1
Kirkwood B, Humphrey J, Moulton L, Martines J. Neonatal vitamin A supplementation and infant survival. Lancet 2010; 376: 1643–44.
Table: Mortality hazard ratios (95% CI) for different vitamin A supplementation strategies compared with placebo for children according to maternal HIV status, Zimbabwe
had no effect.4,5 However, comparing Pa versus Pp as in the Guinean1,2 and the new WHO-sponsored trials, the tendency was negative for neonatal VAS (table). The newly reported sexspecific estimates were also obtained by use of the heterogeneous groups (Ap+Pp) as reference. The results are therefore not based on the same comparison as in Guinea-Bissau. Hopefully the Zimbabwe research team will contribute the data by treatment group and HIV-status. The results presented in the metaanalysis3 should not be used to assure readers that there are no sexdifferential or potentially negative effects of neonatal VAS. We declare that we have no conflicts of interest.
*Christine Stabell Benn, Ane Fisker, Peter Aaby [email protected] Bandim Health Project, Statens Serum Institut, 2300 Copenhagen S, Denmark (CSB); and Bandim Health Project, Indepth Network, Guinea-Bissau (AF, PA) 1
2
3
4
Benn CS, Diness BR, Roth A, et al. Randomised trial of the effect on mortality of 50,000 IU vitamin A given with BCG vaccine to infants in Guinea-Bissau, West-Africa. BMJ 2008; 336: 1416–20. Benn CS, Fisker AB, Napirna BM, et al. Vitamin A supplementation and BCG vaccination at birth in low birthweight neonates: two by two factorial randomised controlled trial. BMJ 2010; 340: c1101. Kirkwood B, Humphrey J, Moulton L, Martines J. Neonatal vitamin A supplementation and infant survival. Lancet 2010; 376: 1643–44. Malaba LC, Iliff PJ, Nathoo KJ, et al. Effect of postpartum maternal or neonatal vitamin A supplementation on infant mortality among infants born to HIV-negative mothers in Zimbabwe. Am J Clin Nutr 2005; 81: 454–60.
www.thelancet.com Vol 377 April 16, 2011
5
Humphrey JH, Iliff PJ, Marinda ET, et al. Effects of a single large dose of vitamin A, given during the postpartum period to HIV-positive women and their infants, on child HIV infection, HIV-free survival, and mortality. J Infect Dis 2006; 193: 860–71.
Authors’ reply Christine Stabell Benn and colleagues imply that we interpreted our metaanalysis1 to mean that there is no reason for further concern. This is not the case; rather we stated that “the results of this meta-analysis should be interpreted with caution. The overall and sex-specific confidence intervals of the effect sizes are wide and are consistent with an important benefit, no effect, or a small harm”. At present there are insufficient data to determine whether neonatal vitamin A supplementation reduces mortality, or whether it has a differential effect in boys and girls; the ongoing trials in Ghana, India, and Tanzania will provide the essential information needed to resolve this important policy issue. Our meta-analysis includes all available information, and does not arbitrarily exclude any subgroups. We would counsel against repeatedly reanalysing data with different subsets without clearly defined a-priori and scientifically justified hypotheses; the dangers of multiple testing leading to false significant findings are well known. BK and JM are involved in the ongoing trials in Ghana, India, and Tanzania. JH and LM declare that they have no conflicts of interest.
Improve the struggle against babies’ pain In his Art of Medicine essay “When is research on children ethical?” (Jan 8, p 115),1 Peter Singer praises the possibility of doing clinical research on infants if it carries “no more than a minimal risk of significant harm”, including pain. The main flaw of this argument emerges when Singer asks: “why should [parents] not be able to consent when the research carries no more than a minimal risk of significant harm?” The error is to believe that parents might “consent”, when they should only “defend”. In other words, parents should not be allowed to choose between several options that have different degrees of usefulness or stress for the baby, but they must permit uniquely the best option, avoiding any risk: babies have not given their consent, thus we should presume they would not, according to the common “precaution principle”. We2 and others3 have described how analgesia guarantees for controlgroup babies in clinical trials on pain treatment are disregarded: although the procedures were referred to as inducing minor pain, this finding is alarming because even minimal pain is disruptive and potentially harmful in neonates. Parents should follow a tight prudence principle in the baby’s interest when clinical trials are proposed; but they can err or can equivocate the level of pain and stress babies will undergo. 1315
Authors’ reply It is not uncommon for risk prediction tools to be based on the “worst” data from the first 24 h after admission for critical care (eg, APACHE and SNAP-II-PE for adult and neonatal critical care, respectively). Use of admission data for risk prediction represents a cross-sectional approach and facilitates a quick management response,1 whereas use of the worst data from the first several hours provides a more longitudinal perspective, probably yielding more robust risk estimates. For the reasons described in the paper,2 we based our prediction model on data obtained (before outcome occurrence) from the first 48 h, especially to permit incorporation of 24-h urine results. We re-examined the fullPIERS database to determine how the published fullPIERS model performed for prediction of outcomes within 48 h of admission using data from 1398 women collected on admission (area under the receiver operating characteristic curve [AUC] 0·857, 95% CI 0·804–0·911), and within 12 h and 24 h of admission (n=1767, AUC 0·863, 0·818–0·908; and n=1865, AUC 0·870, 0·828–0·912, respectively). Our model development strategy balanced statistical and clinical considerations. For example, given highly correlated predictors, our approach was to include the most clinically relevant of the group in the model—a process that cannot be automated and is not suitable for bootstrapping. Therefore, our internal validation assessed the potential for overfitting at the level of the stepwise backwards regression modelling (which included specification of nonlinear variables and assessment of a-priori-specified interaction terms). Internal validation showed that the prediction equation had modest optimism and performed well. However, internal validation strategies have their limitations, and overfitting can only be excluded definitively 1314
through external validation. We have invited, and are facilitating, external validation of the fullPIERS model. How should the clinical community manage women with pre-eclampsia before external validation of the fullPIERS model? The fullPIERS model was developed and internally validated using independently predictive variables from a standardised assessment and surveillance regimen associated with lower rates of adverse maternal events, institutionally and provincially.3,4 The alternative to using the fullPIERS model in clinical practice relies on deeply flawed definitions of “severe” preeclampsia.5 We believe that use of the fullPIERS model represents the better management option at present and have, therefore, hosted an openaccess calculator to facilitate risk estimation and minimise the chance of calculation errors. We declare that we have no conflicts of interest.
*Peter von Dadelszen, Beth A Payne, Jennifer A Hutcheon, K S Joseph, Laura A Magee, for the PIERS Study Group [email protected] Department of Obstetrics and Gynaecology, University of British Columbia and the Child & Family Research Institute, Vancouver, BC V6H 3N1, Canada (PvD, BAP, JAH, KSJ, LAM); School of Population and Public Health, University of British Columbia, Vancouver, BC, Canada (PvD, JAH, KSJ, LAM); and Department of Medicine, University of British Columbia, Vancouver, BC, Canada (LAM) 1
2
3
4
5
Angus DC, Laterre PF, Helterbrand J, et al. The effect of drotrecogin alfa (activated) on long-term survival after severe sepsis. Crit Care Med 2004; 32: 2199–206. von Dadelszen P, Payne B, Li J, et al. Prediction of adverse maternal outcomes in pre-eclampsia: development and validation of the fullPIERS model. Lancet 2011; 377: 219–27. Menzies J, Magee LA, MacNab YC, et al. Instituting surveillance guidelines and adverse outcomes in preeclampsia. Obstet Gynecol 2007; 110: 121–27. von Dadelszen P, Sawchuck D, McMaster R, et al. The active implementation of pregnancy hypertension guidelines in British Columbia. Obstet Gynecol 2010; 116: 659–66. Menzies J, Magee LA, Li J, et al. Current CHS and NHBPEP criteria of severe pre-eclampsia do not uniformly predict adverse maternal or perinatal outcomes. Hypertens Pregnancy 2007; 26: 447–62.
Heterogeneous effects on child survival in neonatal vitamin A supplementation trials Our report of increased female mortality in recipients of neonatal vitamin A supplementation (VAS) in two trials in Guinea-Bissau1,2 has raised concerns in relation to three WHO-sponsored neonatal VAS trials in Ghana, Tanzania, and India, which are in progress to inform global policy. Betty Kirkwood and colleagues (Nov 13, p 1643)3 did a meta-analysis of the sex-specific effects in the neonatal VAS trials, including previously unpublished data from Zimbabwe, and conclude that there is no differential effect of VAS in boys and girls. We disagree. The meta-analysis shows that the effect of VAS was significantly heterogeneous for boys (p=0·004) and overall (p=0·005).3 The interactions between VAS and sex in Guinea-Bissau and Zimbabwe were significantly different (p=0·002). This finding indicates that the effect of VAS is modified by factors that differ between sites. These apparent heterogeneous results might be due to the trial design and the higher HIV prevalence in Zimbabwe. The Guinea-Bissau trials were testing neonatal VAS versus placebo, as are the three ongoing trials. The trial in Zimbabwe was a two-bytwo factorial trial testing maternal VAS (A) or placebo (P), and neonatal VAS (a) or placebo (p), resulting in four treatment groups, Aa, Ap, Pa and Pp (table).4,5 In Zimbabwe, 32% of the mothers were HIV-positive and although mortality results were previously reported separately for HIVnegative and HIV-positive mothers (table), they were pooled in the meta-analysis.3 In previous reports, although the Ap group had borderline or significantly higher mortality than the Pp group, the two groups (Ap+Pp) were combined as the reference to conclude that neonatal VAS (Aa+Pa) www.thelancet.com Vol 377 April 16, 2011
Correspondence
9208 children of HIV-uninfected mothers (170 deaths)*4
4495 children of HIV-infected mothers (865 deaths)*5
14 110 children†‡5
Mother VAS, child VAS (Aa)
1·28 (0·83–1·98)
1·08 (0·88–1·31)
1·06 (0·89–1·26)
Mother VAS, child placebo (Ap)
1·27 (0·82–1·97)
1·24 (1·02–1·50)
1·18 (0·99–1·40)
Mother placebo, child VAS (Pa)
1·18 (0·76–1·83)§
1·21 (0·99–1·46)§
1·16 (0·98–1·38)
Mother placebo, child placebo (Pp)
Reference
Reference
Reference
*Betty Kirkwood, Jean Humphrey, Lawrence Moulton, Jose Martines [email protected]
*Unadjusted results. †Adjusted results. ‡Total included 407 children whose mothers had HIV indeterminate results or seroconverted during follow-up. §In a combined analysis of the unadjusted results from this randomised clinical trial, the mortality hazard ratio comparing Pa vs Pp was 1·21 (95% CI 1·01–1·44).
Department of Epidemiology & Population Health, London School of Hygiene and Tropical Medicine, London WC1E 7HT, UK (BK); Johns Hopkins Bloomberg School of Public Health, Baltimore, MD, USA (JH, LM); and WHO, Geneva, Switzerland (JM) 1
Kirkwood B, Humphrey J, Moulton L, Martines J. Neonatal vitamin A supplementation and infant survival. Lancet 2010; 376: 1643–44.
Table: Mortality hazard ratios (95% CI) for different vitamin A supplementation strategies compared with placebo for children according to maternal HIV status, Zimbabwe
had no effect.4,5 However, comparing Pa versus Pp as in the Guinean1,2 and the new WHO-sponsored trials, the tendency was negative for neonatal VAS (table). The newly reported sexspecific estimates were also obtained by use of the heterogeneous groups (Ap+Pp) as reference. The results are therefore not based on the same comparison as in Guinea-Bissau. Hopefully the Zimbabwe research team will contribute the data by treatment group and HIV-status. The results presented in the metaanalysis3 should not be used to assure readers that there are no sexdifferential or potentially negative effects of neonatal VAS. We declare that we have no conflicts of interest.
*Christine Stabell Benn, Ane Fisker, Peter Aaby [email protected] Bandim Health Project, Statens Serum Institut, 2300 Copenhagen S, Denmark (CSB); and Bandim Health Project, Indepth Network, Guinea-Bissau (AF, PA) 1
2
3
4
Benn CS, Diness BR, Roth A, et al. Randomised trial of the effect on mortality of 50,000 IU vitamin A given with BCG vaccine to infants in Guinea-Bissau, West-Africa. BMJ 2008; 336: 1416–20. Benn CS, Fisker AB, Napirna BM, et al. Vitamin A supplementation and BCG vaccination at birth in low birthweight neonates: two by two factorial randomised controlled trial. BMJ 2010; 340: c1101. Kirkwood B, Humphrey J, Moulton L, Martines J. Neonatal vitamin A supplementation and infant survival. Lancet 2010; 376: 1643–44. Malaba LC, Iliff PJ, Nathoo KJ, et al. Effect of postpartum maternal or neonatal vitamin A supplementation on infant mortality among infants born to HIV-negative mothers in Zimbabwe. Am J Clin Nutr 2005; 81: 454–60.
www.thelancet.com Vol 377 April 16, 2011
5
Humphrey JH, Iliff PJ, Marinda ET, et al. Effects of a single large dose of vitamin A, given during the postpartum period to HIV-positive women and their infants, on child HIV infection, HIV-free survival, and mortality. J Infect Dis 2006; 193: 860–71.
Authors’ reply Christine Stabell Benn and colleagues imply that we interpreted our metaanalysis1 to mean that there is no reason for further concern. This is not the case; rather we stated that “the results of this meta-analysis should be interpreted with caution. The overall and sex-specific confidence intervals of the effect sizes are wide and are consistent with an important benefit, no effect, or a small harm”. At present there are insufficient data to determine whether neonatal vitamin A supplementation reduces mortality, or whether it has a differential effect in boys and girls; the ongoing trials in Ghana, India, and Tanzania will provide the essential information needed to resolve this important policy issue. Our meta-analysis includes all available information, and does not arbitrarily exclude any subgroups. We would counsel against repeatedly reanalysing data with different subsets without clearly defined a-priori and scientifically justified hypotheses; the dangers of multiple testing leading to false significant findings are well known. BK and JM are involved in the ongoing trials in Ghana, India, and Tanzania. JH and LM declare that they have no conflicts of interest.
Improve the struggle against babies’ pain In his Art of Medicine essay “When is research on children ethical?” (Jan 8, p 115),1 Peter Singer praises the possibility of doing clinical research on infants if it carries “no more than a minimal risk of significant harm”, including pain. The main flaw of this argument emerges when Singer asks: “why should [parents] not be able to consent when the research carries no more than a minimal risk of significant harm?” The error is to believe that parents might “consent”, when they should only “defend”. In other words, parents should not be allowed to choose between several options that have different degrees of usefulness or stress for the baby, but they must permit uniquely the best option, avoiding any risk: babies have not given their consent, thus we should presume they would not, according to the common “precaution principle”. We2 and others3 have described how analgesia guarantees for controlgroup babies in clinical trials on pain treatment are disregarded: although the procedures were referred to as inducing minor pain, this finding is alarming because even minimal pain is disruptive and potentially harmful in neonates. Parents should follow a tight prudence principle in the baby’s interest when clinical trials are proposed; but they can err or can equivocate the level of pain and stress babies will undergo. 1315