- Email: [email protected]
Mortality prediction in congenital diaphragmatic hernia
Journal of Pediatric Surgery (2008) 43, 783–787
www.elsevier.com/locate/jpedsurg
Mortality prediction in congenital diaphragmatic hernia Robert Baird a,b , Ying C. MacNab c , Erik D. Skarsgard a,b,⁎ the Canadian Pediatric Surgery Network a
Division of Pediatric Surgery, British Columbia Children's Hospital, Vancouver, British Columbia, Canada V6H 3V4 Department of Surgery, University of British Columbia, Vancouver, British Columbia, Canada V5Z 4E3 c Department of Health Care and Epidemiology, University of British Columbia, Vancouver, British Columbia, Canada V6H 3V4 b
Received 12 November 2007; accepted 3 December 2007
Key words: Congenital diaphragmatic hernia; Survival; Risk-adjustment; Predictive equation
Abstract Background: A validated risk stratification tool for congenital diaphragmatic hernia (CDH) is required for accurate outcomes analyses. Existing mortality-predictive models include those of the CDH Study Group (CDHSG) based on birth weight and 5-minute Apgar score, the Canadian Neonatal Network (CNN) based on gestational age and admission score in Score for Neonatal Acute Physiology version II, and the Wilford Hall/Santa Rosa clinical prediction formula (WHSRPF) derived from blood gas measurements. The purpose of this study was to evaluate the calibration and discrimination of these predictive models using the Canadian Pediatric Surgical Network dataset. Methods: Neonatal risk variables and birth hospital survivorship were collected prospectively in 11 perinatal centers, between May 2005 and October 2006. Actual vs predicted outcomes were analyzed for each equation to measure the calibration and discrimination of each model. Results: Twenty (21.2%) of 94 infants with CDH died during birth hospitalization. The CDHSG model demonstrated superior discrimination (area under the receiver operator characteristic curve = 0.85; CNN = 0.79; WHSRPF = 0.63). Model calibration reflected by the Hosmer-Lemeshow P value was poorest with the WHSRPF = 0.37 and comparable between CDHSG and CNN (0.48 and 0.46, respectively). Conclusion: Predictive outcome models are essential for risk-adjusted outcome analysis of CDH. The ideal predictive equation should prove robust across CDH datasets. © 2008 Elsevier Inc. All rights reserved.
Despite significant advances in prenatal diagnosis and neonatal intensive care, congenital diaphragmatic hernia (CDH), continues to be a vexing congenital malformation with broadly variable cardiopulmonary disease severity at
Presented at the 39th Annual Meeting of the Canadian Association of Pediatric Surgeons, August 23-26, 2007, St John's Newfoundland, Canada. ⁎ Corresponding author. Division of Pediatric General Surgery, KO-123 ACB, 448 Oak Street, Vancouver British Columbia, Canada V6H 3V4. Tel.: +1 604 875 3744; fax: +1 604 875 2721. E-mail address: [email protected] (E.D. Skarsgard). 0022-3468/$ – see front matter © 2008 Elsevier Inc. All rights reserved. doi:10.1016/j.jpedsurg.2007.12.012
birth. Although implementation of rational treatment strategies including preoperative stabilization, lung protective ventilation, extracorporeal membrane oxygenation (ECMO), pulmonary vasodilator therapy, and delayed surgical treatment have resulted in reduced mortality trends in individual centers, there continues to be great variability in overall CDH mortality with rates ranging between 20% and 60% [1-4]. An ongoing barrier to outcome research in CDH is the lack of a validated and widely accepted disease-severity adjustment tool. Validated illness-severity assessment of the newborn with CDH should be done early in the postnatal
784 course so that the outcome prediction provided is not significantly influenced by postnatal treatment. Early outcome prediction enables anticipation of the likelihood of the need for more aggressive treatment strategies (such as ECMO) and informs physician-parent discussions on an individual patient basis. From the perspective of studying populations of CDH infants, such a tool allows comparisons of risk-adjusted patients within and between institutions. Finally, accurate risk-adjusted outcome stratification enables the rigorous evaluation of treatment strategies in prospective trials. There have been at least 3 predictive equations derived for early CDH mortality. These include the predictive equations of the CDH Study Group (CDHSG) [5], the Canadian Neonatal Network (CNN) [6], and the Wilford Hall/Santa Rosa group (WHSRPF) [7]. Each predictive equation was derived from a unique CDH patient cohort. The purpose of our study was to validate each of these outcome predictors using a single CDH dataset from the Canadian Pediatric Surgical Network (CAPSNET) database.
1. Materials and methods The dataset was obtained from the CAPSNET, a multidisciplinary group of 16 Canadian perinatal centers that collect prospective, disease-specific data on both CDH, and gastroschisis (Appendix 1). A perinatal center is defined as one with a level III neonatal intensive care unit (NICU), pediatric anesthesia, and subspecialty surgery (at least general and neurosurgery) capabilities, and a geographically or functionally adjoined maternal-fetal medicine/advanced prenatal diagnosis center. Pediatric cardiac surgery and ECMO are available in 9 and 4 CAPSNET centers, respectively. In Canada, all perinatal care for birth defects such as CDH is provided exclusively through these provincial referral centers. Eleven centers contributed data during the study period; the largest site contributed 25 patients whereas 3 sites contributed 2 patients each.
1.1. Study population Congenital diaphragmatic hernia cases for this study were accrued between May 1, 2005, and December 31, 2006. Cases were ascertained at prenatal diagnosis (if one was made) or after birth, and data were abstracted from diagnosis to death or discharge from a CAPSNET center. Infants transferred from one CAPSNET center to another were tracked back to their initial admission.
1.2. Data collection Notification of the prenatal diagnosis or birth of a case of CDH was forwarded to on-site, trained research assistants who abstracted data from maternal and infant charts using a
R. Baird et al. customized data entry program with built-in error checking and a standard manual of operations and definitions. Data were deidentifed and transmitted electronically to a centralized repository where data were cleaned, stored, and thereafter managed by a study coordinator and a geographically representative, multidisciplinary steering committee comprised of pediatric surgeons, neonatologists, maternalfetal medicine specialists, and an epidemiologist.
1.3. Predictive models Three predictive models were evaluated. Each model was derived from a separate CDH patient cohort, by testing (individually and in combination), those risk variables deemed to be predictive of mortality by multivariable logistic regression analysis within that cohort: (1) The CDHSG probability of survival equation = 1−1/ (1+e−x), where −x = −5.0240 + 0.9165 (birth weight in kilograms) + 0.4512 (Apgar score at 5 minutes) [5]. (2) The CNN predictive equation that uses a combination of 2 risk variables, the Score for Neonatal Acute Physiology version II (SNAP-II), and gestational age (GA). The SNAP-II is a standardized index, validated in other neonatal patient populations, which depicts illness severity by the magnitude of derangement in 6 physiologic parameters: mean blood pressure, lowest temperature, PO2 (mm Hg)/FIO2 (%) ratio, lowest serum pH, presence of seizure activity, and urine output (mL/kg per hour), expressed as an aggregate score [8]. (3) WHSRPF. This equation uses blood gas values (from a primarily postductal source), measured during the first 24 hours of life to calculate the equation: highest PaO2−highest PCO2, with a cutoff value of zero or greater expected to predict survival [7].
1.4. Data analysis Perinatal characteristics, characteristics of operated patients, and outcomes were recorded directly from the CAPSNET database. For each predictive equation, modeled and actual outcomes were compared using the receiver operator characteristic (ROC) curve technique of Hanley and MacNeil [9] to assess model discrimination. The area under the ROC curve (AUC) depicts model discrimination—a measure of its predictive performance. An AUC of 0.5 represents a completely random association between the modeled and actual outcomes, whereas an AUC of 1.0 represents perfect discrimination. The conformity between actual and predicted outcome is also depicted by model calibration or “goodness of fit,” where, using the HosmerLemeshow (H-L) technique, a P value of .05 or higher suggests that there is no difference between modeled and actual outcomes. The higher the P value, the better the model calibration [10]. All analyses were performed using SPSS for Windows statistical software (SPSS, Chicago, Ill).
Mortality prediction in congenital diaphragmatic hernia
2. Results 2.1. Descriptive characteristics and outcomes of CDH cohort The total number of NICU admissions during the study period from the 11 CAPSNET contributing sites was 10,094. One hundred thirteen CDH cases were entered into the database, of which 105 were liveborn. Eleven patients had open files (meaning they remained alive in hospital), whereas 94 achieved the outcome criteria of death or discharge from the hospital of birth, and these represent the study cohort. Of these, 62 (66.0%) were diagnosed in the prenatal period. Babies were born at a mean GA of 38 weeks and a mean birth weight of 3103 g. Thirty-four babies (40.4%) were either inborn or had a planned delivery at a functionally linked, specialized obstetrical center. Nine babies (10.5%) required ECMO therapy, 4 of whom (44.4%) did not survive. Seventy-six babies (84.8%) survived to surgery, which was performed an average of 3.5 ± 2.8 days after birth. Seven babies underwent surgery on ECMO. The CDH defect was left-sided in 64 (76.2%) of 84 known cases, and prosthetic patch repairs were performed in 25 (30.4%) of 82 operated babies. Overall, 74 live-born neonates survived to discharge (78.7% of live births with closed files) with a mean length of stay of 24.4 ± 18.2 days (Table 1).
2.2. Predictive performance and calibration of each model In this comparative analysis, the CDHSG predictive equation demonstrated the best discrimination with an AUC of 0.85. The CNN model had comparable discrimination using SNAP-II alone (AUC = 0.79) or in combination with GA (AUC = 0.78) but was inferior to that of the CDHSG. The WHSRPF demonstrated the poorest discrimination with an AUC of only 0.63. From the perspective of model calibration, the CDHSG performed slightly better than the CNN (SNAP-II alone) with H-L P values of .48 and .46, Table 1 Characteristics and selected outcomes in CDH patients in CAPSNET Study cohort Parameter
Value (n [%])
Female sex Weight (g) GA (wk) Cesarean delivery Prenatal diagnosis Left-sided hernia Inborn ECMO required Mesh repair Survival to discharge Length of stay (d)
46 (48.9) 3081 ± 682 38 ± 2.2 26 (27.6) 62 (68.1) 47 (81.0) 34 (40.4) 9 (9.6) 21 (33.9) 74 (78.7) 24.4 ± 18.2
785 Table 2 Predictive performance and goodness-of-fit of logistic regression models for predicting mortality among infants with CDH Variable
CNN and SNAP-II (alone)
CNN and SNAP-II + GA (combined)
CDHSG
WHSRPF
AUC χ2 H-L P
0.79 5.66 .46
0.78 12.27 .14
0.85 6.54 .48
0.63 8.67 .37
respectively. The WHSRPF H-L P value was .371, whereas the CNN combined model (SNAP-II + GA) had the poorest calibration with an H-L P value of .14 (Table 2).
3. Discussion Congenital diaphragmatic hernia presents with a spectrum of disease severity that makes it difficult to accurately predict outcome. Although some neonates are born virtually asymptomatic, others require maximal NICU treatment strategies, including high frequency oscillatory ventilation, inhaled nitric oxide, and ECMO before operative repair [11-16]. Predicting the specific interventions a patient is likely to require as well as their overall outcome based on early physiologic performance remains an important goal in CDH research. Several publications have attempted to describe an accurate outcome predictor for neonates with CDH [5-7]. Although each has proven reasonable in the context of its publication cohort, they all have yet to demonstrate reliability in subsequent investigations. The CDHSG score was derived from a large cohort (N1000) of CDH neonates and allowed for risk stratification into terciles based on the birth weight and 5-minute Apgar [5]. The CDHSG underestimated actual survival in 2 other cohorts [1,3], however, suggesting that it might be improved upon. We recently reported on the SNAP-II score as a predictor of mortality in infants with CDH [6]. The SNAP-II score, a validated outcome predictor in non-CDH neonatal populations, applies a weighted score to 6 physiologic variables within the first 12 hours of admission to the NICU [8]. In a CDH cohort of 88 CDH patients from the CNN database, multivariable logistic regression revealed that SNAP-II, in combination with GA, yielded a predictive model with comparable discrimination and superior calibration (AUC = 0.81; H-L P = .88) compared to the CDHSG predictive equation (AUC = 0.83; H-L P = .06) [6]. Recently, Schultz et al [7] describe a simplified postnatal predictor of outcome based on arterial blood gas measurements obtained during the initial 24 hours of life (but before ECMO or operative repair). The WHSRPF was developed based on known respiratory pathophysiology in CDH [17,18]. It was investigated retrospectively in a local group of 88 patients and subsequently in a CDHSG
786 cohort of 849 patients, both accrued during a 7-year period. The WHSRPF demonstrated a discrimination (AUC) of 0.87 in the local group and 0.79 in the CDHSG data set, comparable with the CDHSG's own equation (0.76) [7]. Goodness-of-fit was not reported. In this study, we collected prospective, disease-specific data from 11 Canadian institutions for an 18-month period and investigated the predictive performance of these 3 outcome models. Our results demonstrate that the CDHSG model had superior discrimination to all other models (AUC = 0.85), whereas the WHSRPF had the poorest discrimination with an AUC = 0.63. All models demonstrated reasonable goodness-of-fit, with the CDHSG model again outperforming the other 2 predictive models. The reason for the superior predictive ability of the CDHSG is unclear. The CDHSG score is generated from 2 descriptive data points (birth weight and 5-minute Apgar), the CNN score includes one descriptive data point (GA), and 6 physiologic variables (SNAP-II), whereas the WHSRPF is generated from 2 physiologic variables. The WHSRPF has the advantage of being easily generated and calculated, whereas the CDHSG score necessitates some mathematical manipulation of easily collected birth data. Both are therefore relatively easy to apply. Most components of the SNAP-II are also collected, although not all components (lowest temperature, for example) are necessarily recorded, nor is the SNAP-II score itself routinely calculated and recorded. The combined score therefore uses the greatest number of physiologic variable and theoretically may provide the best snapshot of neonatal physiology of the 3 scores. Its performance, however, may not justify its routine calculation. It must also be conceded that given the evidence favoring delayed surgical repair [19,20], delayed or sequential risk stratification may ultimately provide the most accurate prognostic information. Whereas all of the investigated models are described in the early NICU course (CDHSG at birth, SNAP b12 hours, WHSRPF b24 hours), investigating a particular predictive model at different time-points may enable more accurate outcome assessment. Finally, it is possible that some combination of existing predictive models may offer further refinement of the accuracy of outcome prediction. Several predictive models exist for the postnatal evaluation of CDH infants. Given the importance of CDH risk adjustment to outcome analysis and informing individual family counseling, further investigation is warranted to identify the optimal predictive equation.
Acknowledgments This study was supported by grant FRN no. 69050 from the Canadian Institute of Health Research (Ontario, Canada). We thank all the local CAPSNET data abstractors for their
R. Baird et al. work in data collection and Jennifer Claydon for her assistance in compiling the data.
Appendix A Participating CAPSNET Centers: Victoria General Hospital, Victoria, British Columbia B.C. Children's & Women's Health Centre, Vancouver, British Columbia Royal University Hospital, Saskatoon, Saskatchewan Winnipeg Health Sciences Centre, Winnipeg, Manitoba St. Boniface Health Centre, Winnipeg, Manitoba Hospital for Sick Children, Toronto, Ontario Mt. Sinai Hospital, Toronto, Ontario McMaster Children's Hospital, Hamilton, Ontario London Health Sciences Centre, London, Ontario Kingston General Hospital, Kingston, Ontario Children's Hospital of Eastern Ontario, Ottawa, Ontario Montreal Children's Hospital, Montreal, Quebec Hôpital Ste-Justine, Montreal, Quebec Centre Hospitalier de L'Université Laval, Ste-Foy, Quebec IWK Health Centre, Halifax, Nova Scotia Charles Janeway Child Health Centre, St. John's, Newfoundland
References [1] Downard CD, Jaksic T, Garza JJ, et al. Analysis of an improved survival rate for congenital diaphragmatic hernia. J Pediatr Surg 2003;38:729-32. [2] Stege G, Fenton A, Jaffray B. Nihilism in the 1990s: the true mortality of congenital diaphragmatic hernia. Pediatrics 2003;112: 532-5. [3] Javid PJ, Jaksic T, Skarsgard ED, et al. Survival in congenital diaphragmatic hernia: the experience of the Canadian Neonatal Network. J Pediatr Surg 2004;39:657-60. [4] The Ontario Congenital Anomalies Study Group. Apparent truth about congenital diaphragmatic hernia: a population based database is needed to establish benchmarking for clinical outcomes for CDH. J Pediatr Surg 2004;39:661-5. [5] The Congenital Diaphragmatic Study Group. Estimating disease severity of congenital diaphragmatic hernia in the first 5 minutes of life. J Pediatr Surg 2001;36:141-5. [6] Skarsgard ED, MacNab YC, the Canadian Neonatal Network, et al. SNAP-II predicts mortality among infants with congenital diaphragmatic hernia. J Perinatol 2005;25:315-9. [7] Schultz CM, DiGeronimo RJ, Yoder BA. Congenital diaphragmatic hernia: a simplified postnatal predictor of outcome. J Pediatr Surg 2007;42:510-6. [8] Richardson DK, Corcoran JD, Escobar GJ, et al. SNAP-II and SNAPPE-II: simplified newborn illness severity and mortality scores. J Pediatr 2001;138:92-100. [9] Hanley JA, McNeil BJ. The meaning and use of the area under a receiver operating characteristic (ROC) curve. Radiology 1982;143:29-36. [10] Lemeshow S, Hosmer Jr DW. A review of goodness of fit statistics for use in the development of logistic regression models. Am J Epidemiol 1982;115:92-106.
Mortality prediction in congenital diaphragmatic hernia
787
[11] Caciari A, Ruggeri G, Mondenti M, et al. High-frequency oscillatory ventilation versus conventional mechanical ventilation in congenital diaphragmatic hernia. Eur J Pediatr Surg 2001;11:3-7. [12] Miguet D, Claris O, Lapillonne A, et al. Preoperative stabilization using high-frequency oscillatory ventilation in the management of congenital diaphragmatic hernia. Crit Care Med 1994;22(9 Suppl):S77-82. [13] Okuyama H, Kubota A, Oue T, et al. Inhaled nitric oxide with early surgery improves the outcome of antenatally diagnosed congenital diaphragmatic hernia. J Pediatr Surg 2002;37:1188-90. [14] Kinsella JP, Parker TA, Ivy DD, et al. Noninvasive delivery of inhaled nitric oxide therapy for late pulmonary hypertension in newborn infants with congenital diaphragmatic hernia. J Pediatr 2003;142:397-401. [15] Atkinson JB, Kitagawa H. Extracorporeal membrane oxygenation and the management of congenital diaphragmatic hernia. Pediatr Surg Int 1993;8:200-3.
[16] Flamant C, Lorino E, Nolent P, et al. Newborn infants supported by extracorporeal membrane oxygenation: survival and clinical outcome. Arch Pediatr 2007;14:354-61. [17] Bohn DJ, James I, Filler RM, et al. The relationship between PaCo2 and ventilation parameters predicting survival in congenital diaphragmatic hernia. J Pediatr Surg 1984;16:666-71. [18] Germain JF, Farnouz C, Pinquier D, et al. Can blood gas values predict pulmonary hypoplasia in antenatally diagnosed congenital diaphragmatic hernia. J Pediatr Surg 1996;31:1634-9. [19] Langer JC, Filler RM, Bohn DJ, et al. Timing of surgery for congenital diaphragmatic hernia: is emergency operation necessary. J Pediatr Surg 1988;23:731-4. [20] West KW, Bengston K, Rescorla FJ, et al. Delayed surgical repair and ECMO improves survival in congenital diaphragmatic hernia. Ann Surg 1992;216:454-62.
www.elsevier.com/locate/jpedsurg
Mortality prediction in congenital diaphragmatic hernia Robert Baird a,b , Ying C. MacNab c , Erik D. Skarsgard a,b,⁎ the Canadian Pediatric Surgery Network a
Division of Pediatric Surgery, British Columbia Children's Hospital, Vancouver, British Columbia, Canada V6H 3V4 Department of Surgery, University of British Columbia, Vancouver, British Columbia, Canada V5Z 4E3 c Department of Health Care and Epidemiology, University of British Columbia, Vancouver, British Columbia, Canada V6H 3V4 b
Received 12 November 2007; accepted 3 December 2007
Key words: Congenital diaphragmatic hernia; Survival; Risk-adjustment; Predictive equation
Abstract Background: A validated risk stratification tool for congenital diaphragmatic hernia (CDH) is required for accurate outcomes analyses. Existing mortality-predictive models include those of the CDH Study Group (CDHSG) based on birth weight and 5-minute Apgar score, the Canadian Neonatal Network (CNN) based on gestational age and admission score in Score for Neonatal Acute Physiology version II, and the Wilford Hall/Santa Rosa clinical prediction formula (WHSRPF) derived from blood gas measurements. The purpose of this study was to evaluate the calibration and discrimination of these predictive models using the Canadian Pediatric Surgical Network dataset. Methods: Neonatal risk variables and birth hospital survivorship were collected prospectively in 11 perinatal centers, between May 2005 and October 2006. Actual vs predicted outcomes were analyzed for each equation to measure the calibration and discrimination of each model. Results: Twenty (21.2%) of 94 infants with CDH died during birth hospitalization. The CDHSG model demonstrated superior discrimination (area under the receiver operator characteristic curve = 0.85; CNN = 0.79; WHSRPF = 0.63). Model calibration reflected by the Hosmer-Lemeshow P value was poorest with the WHSRPF = 0.37 and comparable between CDHSG and CNN (0.48 and 0.46, respectively). Conclusion: Predictive outcome models are essential for risk-adjusted outcome analysis of CDH. The ideal predictive equation should prove robust across CDH datasets. © 2008 Elsevier Inc. All rights reserved.
Despite significant advances in prenatal diagnosis and neonatal intensive care, congenital diaphragmatic hernia (CDH), continues to be a vexing congenital malformation with broadly variable cardiopulmonary disease severity at
Presented at the 39th Annual Meeting of the Canadian Association of Pediatric Surgeons, August 23-26, 2007, St John's Newfoundland, Canada. ⁎ Corresponding author. Division of Pediatric General Surgery, KO-123 ACB, 448 Oak Street, Vancouver British Columbia, Canada V6H 3V4. Tel.: +1 604 875 3744; fax: +1 604 875 2721. E-mail address: [email protected] (E.D. Skarsgard). 0022-3468/$ – see front matter © 2008 Elsevier Inc. All rights reserved. doi:10.1016/j.jpedsurg.2007.12.012
birth. Although implementation of rational treatment strategies including preoperative stabilization, lung protective ventilation, extracorporeal membrane oxygenation (ECMO), pulmonary vasodilator therapy, and delayed surgical treatment have resulted in reduced mortality trends in individual centers, there continues to be great variability in overall CDH mortality with rates ranging between 20% and 60% [1-4]. An ongoing barrier to outcome research in CDH is the lack of a validated and widely accepted disease-severity adjustment tool. Validated illness-severity assessment of the newborn with CDH should be done early in the postnatal
784 course so that the outcome prediction provided is not significantly influenced by postnatal treatment. Early outcome prediction enables anticipation of the likelihood of the need for more aggressive treatment strategies (such as ECMO) and informs physician-parent discussions on an individual patient basis. From the perspective of studying populations of CDH infants, such a tool allows comparisons of risk-adjusted patients within and between institutions. Finally, accurate risk-adjusted outcome stratification enables the rigorous evaluation of treatment strategies in prospective trials. There have been at least 3 predictive equations derived for early CDH mortality. These include the predictive equations of the CDH Study Group (CDHSG) [5], the Canadian Neonatal Network (CNN) [6], and the Wilford Hall/Santa Rosa group (WHSRPF) [7]. Each predictive equation was derived from a unique CDH patient cohort. The purpose of our study was to validate each of these outcome predictors using a single CDH dataset from the Canadian Pediatric Surgical Network (CAPSNET) database.
1. Materials and methods The dataset was obtained from the CAPSNET, a multidisciplinary group of 16 Canadian perinatal centers that collect prospective, disease-specific data on both CDH, and gastroschisis (Appendix 1). A perinatal center is defined as one with a level III neonatal intensive care unit (NICU), pediatric anesthesia, and subspecialty surgery (at least general and neurosurgery) capabilities, and a geographically or functionally adjoined maternal-fetal medicine/advanced prenatal diagnosis center. Pediatric cardiac surgery and ECMO are available in 9 and 4 CAPSNET centers, respectively. In Canada, all perinatal care for birth defects such as CDH is provided exclusively through these provincial referral centers. Eleven centers contributed data during the study period; the largest site contributed 25 patients whereas 3 sites contributed 2 patients each.
1.1. Study population Congenital diaphragmatic hernia cases for this study were accrued between May 1, 2005, and December 31, 2006. Cases were ascertained at prenatal diagnosis (if one was made) or after birth, and data were abstracted from diagnosis to death or discharge from a CAPSNET center. Infants transferred from one CAPSNET center to another were tracked back to their initial admission.
1.2. Data collection Notification of the prenatal diagnosis or birth of a case of CDH was forwarded to on-site, trained research assistants who abstracted data from maternal and infant charts using a
R. Baird et al. customized data entry program with built-in error checking and a standard manual of operations and definitions. Data were deidentifed and transmitted electronically to a centralized repository where data were cleaned, stored, and thereafter managed by a study coordinator and a geographically representative, multidisciplinary steering committee comprised of pediatric surgeons, neonatologists, maternalfetal medicine specialists, and an epidemiologist.
1.3. Predictive models Three predictive models were evaluated. Each model was derived from a separate CDH patient cohort, by testing (individually and in combination), those risk variables deemed to be predictive of mortality by multivariable logistic regression analysis within that cohort: (1) The CDHSG probability of survival equation = 1−1/ (1+e−x), where −x = −5.0240 + 0.9165 (birth weight in kilograms) + 0.4512 (Apgar score at 5 minutes) [5]. (2) The CNN predictive equation that uses a combination of 2 risk variables, the Score for Neonatal Acute Physiology version II (SNAP-II), and gestational age (GA). The SNAP-II is a standardized index, validated in other neonatal patient populations, which depicts illness severity by the magnitude of derangement in 6 physiologic parameters: mean blood pressure, lowest temperature, PO2 (mm Hg)/FIO2 (%) ratio, lowest serum pH, presence of seizure activity, and urine output (mL/kg per hour), expressed as an aggregate score [8]. (3) WHSRPF. This equation uses blood gas values (from a primarily postductal source), measured during the first 24 hours of life to calculate the equation: highest PaO2−highest PCO2, with a cutoff value of zero or greater expected to predict survival [7].
1.4. Data analysis Perinatal characteristics, characteristics of operated patients, and outcomes were recorded directly from the CAPSNET database. For each predictive equation, modeled and actual outcomes were compared using the receiver operator characteristic (ROC) curve technique of Hanley and MacNeil [9] to assess model discrimination. The area under the ROC curve (AUC) depicts model discrimination—a measure of its predictive performance. An AUC of 0.5 represents a completely random association between the modeled and actual outcomes, whereas an AUC of 1.0 represents perfect discrimination. The conformity between actual and predicted outcome is also depicted by model calibration or “goodness of fit,” where, using the HosmerLemeshow (H-L) technique, a P value of .05 or higher suggests that there is no difference between modeled and actual outcomes. The higher the P value, the better the model calibration [10]. All analyses were performed using SPSS for Windows statistical software (SPSS, Chicago, Ill).
Mortality prediction in congenital diaphragmatic hernia
2. Results 2.1. Descriptive characteristics and outcomes of CDH cohort The total number of NICU admissions during the study period from the 11 CAPSNET contributing sites was 10,094. One hundred thirteen CDH cases were entered into the database, of which 105 were liveborn. Eleven patients had open files (meaning they remained alive in hospital), whereas 94 achieved the outcome criteria of death or discharge from the hospital of birth, and these represent the study cohort. Of these, 62 (66.0%) were diagnosed in the prenatal period. Babies were born at a mean GA of 38 weeks and a mean birth weight of 3103 g. Thirty-four babies (40.4%) were either inborn or had a planned delivery at a functionally linked, specialized obstetrical center. Nine babies (10.5%) required ECMO therapy, 4 of whom (44.4%) did not survive. Seventy-six babies (84.8%) survived to surgery, which was performed an average of 3.5 ± 2.8 days after birth. Seven babies underwent surgery on ECMO. The CDH defect was left-sided in 64 (76.2%) of 84 known cases, and prosthetic patch repairs were performed in 25 (30.4%) of 82 operated babies. Overall, 74 live-born neonates survived to discharge (78.7% of live births with closed files) with a mean length of stay of 24.4 ± 18.2 days (Table 1).
2.2. Predictive performance and calibration of each model In this comparative analysis, the CDHSG predictive equation demonstrated the best discrimination with an AUC of 0.85. The CNN model had comparable discrimination using SNAP-II alone (AUC = 0.79) or in combination with GA (AUC = 0.78) but was inferior to that of the CDHSG. The WHSRPF demonstrated the poorest discrimination with an AUC of only 0.63. From the perspective of model calibration, the CDHSG performed slightly better than the CNN (SNAP-II alone) with H-L P values of .48 and .46, Table 1 Characteristics and selected outcomes in CDH patients in CAPSNET Study cohort Parameter
Value (n [%])
Female sex Weight (g) GA (wk) Cesarean delivery Prenatal diagnosis Left-sided hernia Inborn ECMO required Mesh repair Survival to discharge Length of stay (d)
46 (48.9) 3081 ± 682 38 ± 2.2 26 (27.6) 62 (68.1) 47 (81.0) 34 (40.4) 9 (9.6) 21 (33.9) 74 (78.7) 24.4 ± 18.2
785 Table 2 Predictive performance and goodness-of-fit of logistic regression models for predicting mortality among infants with CDH Variable
CNN and SNAP-II (alone)
CNN and SNAP-II + GA (combined)
CDHSG
WHSRPF
AUC χ2 H-L P
0.79 5.66 .46
0.78 12.27 .14
0.85 6.54 .48
0.63 8.67 .37
respectively. The WHSRPF H-L P value was .371, whereas the CNN combined model (SNAP-II + GA) had the poorest calibration with an H-L P value of .14 (Table 2).
3. Discussion Congenital diaphragmatic hernia presents with a spectrum of disease severity that makes it difficult to accurately predict outcome. Although some neonates are born virtually asymptomatic, others require maximal NICU treatment strategies, including high frequency oscillatory ventilation, inhaled nitric oxide, and ECMO before operative repair [11-16]. Predicting the specific interventions a patient is likely to require as well as their overall outcome based on early physiologic performance remains an important goal in CDH research. Several publications have attempted to describe an accurate outcome predictor for neonates with CDH [5-7]. Although each has proven reasonable in the context of its publication cohort, they all have yet to demonstrate reliability in subsequent investigations. The CDHSG score was derived from a large cohort (N1000) of CDH neonates and allowed for risk stratification into terciles based on the birth weight and 5-minute Apgar [5]. The CDHSG underestimated actual survival in 2 other cohorts [1,3], however, suggesting that it might be improved upon. We recently reported on the SNAP-II score as a predictor of mortality in infants with CDH [6]. The SNAP-II score, a validated outcome predictor in non-CDH neonatal populations, applies a weighted score to 6 physiologic variables within the first 12 hours of admission to the NICU [8]. In a CDH cohort of 88 CDH patients from the CNN database, multivariable logistic regression revealed that SNAP-II, in combination with GA, yielded a predictive model with comparable discrimination and superior calibration (AUC = 0.81; H-L P = .88) compared to the CDHSG predictive equation (AUC = 0.83; H-L P = .06) [6]. Recently, Schultz et al [7] describe a simplified postnatal predictor of outcome based on arterial blood gas measurements obtained during the initial 24 hours of life (but before ECMO or operative repair). The WHSRPF was developed based on known respiratory pathophysiology in CDH [17,18]. It was investigated retrospectively in a local group of 88 patients and subsequently in a CDHSG
786 cohort of 849 patients, both accrued during a 7-year period. The WHSRPF demonstrated a discrimination (AUC) of 0.87 in the local group and 0.79 in the CDHSG data set, comparable with the CDHSG's own equation (0.76) [7]. Goodness-of-fit was not reported. In this study, we collected prospective, disease-specific data from 11 Canadian institutions for an 18-month period and investigated the predictive performance of these 3 outcome models. Our results demonstrate that the CDHSG model had superior discrimination to all other models (AUC = 0.85), whereas the WHSRPF had the poorest discrimination with an AUC = 0.63. All models demonstrated reasonable goodness-of-fit, with the CDHSG model again outperforming the other 2 predictive models. The reason for the superior predictive ability of the CDHSG is unclear. The CDHSG score is generated from 2 descriptive data points (birth weight and 5-minute Apgar), the CNN score includes one descriptive data point (GA), and 6 physiologic variables (SNAP-II), whereas the WHSRPF is generated from 2 physiologic variables. The WHSRPF has the advantage of being easily generated and calculated, whereas the CDHSG score necessitates some mathematical manipulation of easily collected birth data. Both are therefore relatively easy to apply. Most components of the SNAP-II are also collected, although not all components (lowest temperature, for example) are necessarily recorded, nor is the SNAP-II score itself routinely calculated and recorded. The combined score therefore uses the greatest number of physiologic variable and theoretically may provide the best snapshot of neonatal physiology of the 3 scores. Its performance, however, may not justify its routine calculation. It must also be conceded that given the evidence favoring delayed surgical repair [19,20], delayed or sequential risk stratification may ultimately provide the most accurate prognostic information. Whereas all of the investigated models are described in the early NICU course (CDHSG at birth, SNAP b12 hours, WHSRPF b24 hours), investigating a particular predictive model at different time-points may enable more accurate outcome assessment. Finally, it is possible that some combination of existing predictive models may offer further refinement of the accuracy of outcome prediction. Several predictive models exist for the postnatal evaluation of CDH infants. Given the importance of CDH risk adjustment to outcome analysis and informing individual family counseling, further investigation is warranted to identify the optimal predictive equation.
Acknowledgments This study was supported by grant FRN no. 69050 from the Canadian Institute of Health Research (Ontario, Canada). We thank all the local CAPSNET data abstractors for their
R. Baird et al. work in data collection and Jennifer Claydon for her assistance in compiling the data.
Appendix A Participating CAPSNET Centers: Victoria General Hospital, Victoria, British Columbia B.C. Children's & Women's Health Centre, Vancouver, British Columbia Royal University Hospital, Saskatoon, Saskatchewan Winnipeg Health Sciences Centre, Winnipeg, Manitoba St. Boniface Health Centre, Winnipeg, Manitoba Hospital for Sick Children, Toronto, Ontario Mt. Sinai Hospital, Toronto, Ontario McMaster Children's Hospital, Hamilton, Ontario London Health Sciences Centre, London, Ontario Kingston General Hospital, Kingston, Ontario Children's Hospital of Eastern Ontario, Ottawa, Ontario Montreal Children's Hospital, Montreal, Quebec Hôpital Ste-Justine, Montreal, Quebec Centre Hospitalier de L'Université Laval, Ste-Foy, Quebec IWK Health Centre, Halifax, Nova Scotia Charles Janeway Child Health Centre, St. John's, Newfoundland
References [1] Downard CD, Jaksic T, Garza JJ, et al. Analysis of an improved survival rate for congenital diaphragmatic hernia. J Pediatr Surg 2003;38:729-32. [2] Stege G, Fenton A, Jaffray B. Nihilism in the 1990s: the true mortality of congenital diaphragmatic hernia. Pediatrics 2003;112: 532-5. [3] Javid PJ, Jaksic T, Skarsgard ED, et al. Survival in congenital diaphragmatic hernia: the experience of the Canadian Neonatal Network. J Pediatr Surg 2004;39:657-60. [4] The Ontario Congenital Anomalies Study Group. Apparent truth about congenital diaphragmatic hernia: a population based database is needed to establish benchmarking for clinical outcomes for CDH. J Pediatr Surg 2004;39:661-5. [5] The Congenital Diaphragmatic Study Group. Estimating disease severity of congenital diaphragmatic hernia in the first 5 minutes of life. J Pediatr Surg 2001;36:141-5. [6] Skarsgard ED, MacNab YC, the Canadian Neonatal Network, et al. SNAP-II predicts mortality among infants with congenital diaphragmatic hernia. J Perinatol 2005;25:315-9. [7] Schultz CM, DiGeronimo RJ, Yoder BA. Congenital diaphragmatic hernia: a simplified postnatal predictor of outcome. J Pediatr Surg 2007;42:510-6. [8] Richardson DK, Corcoran JD, Escobar GJ, et al. SNAP-II and SNAPPE-II: simplified newborn illness severity and mortality scores. J Pediatr 2001;138:92-100. [9] Hanley JA, McNeil BJ. The meaning and use of the area under a receiver operating characteristic (ROC) curve. Radiology 1982;143:29-36. [10] Lemeshow S, Hosmer Jr DW. A review of goodness of fit statistics for use in the development of logistic regression models. Am J Epidemiol 1982;115:92-106.
Mortality prediction in congenital diaphragmatic hernia
787
[11] Caciari A, Ruggeri G, Mondenti M, et al. High-frequency oscillatory ventilation versus conventional mechanical ventilation in congenital diaphragmatic hernia. Eur J Pediatr Surg 2001;11:3-7. [12] Miguet D, Claris O, Lapillonne A, et al. Preoperative stabilization using high-frequency oscillatory ventilation in the management of congenital diaphragmatic hernia. Crit Care Med 1994;22(9 Suppl):S77-82. [13] Okuyama H, Kubota A, Oue T, et al. Inhaled nitric oxide with early surgery improves the outcome of antenatally diagnosed congenital diaphragmatic hernia. J Pediatr Surg 2002;37:1188-90. [14] Kinsella JP, Parker TA, Ivy DD, et al. Noninvasive delivery of inhaled nitric oxide therapy for late pulmonary hypertension in newborn infants with congenital diaphragmatic hernia. J Pediatr 2003;142:397-401. [15] Atkinson JB, Kitagawa H. Extracorporeal membrane oxygenation and the management of congenital diaphragmatic hernia. Pediatr Surg Int 1993;8:200-3.
[16] Flamant C, Lorino E, Nolent P, et al. Newborn infants supported by extracorporeal membrane oxygenation: survival and clinical outcome. Arch Pediatr 2007;14:354-61. [17] Bohn DJ, James I, Filler RM, et al. The relationship between PaCo2 and ventilation parameters predicting survival in congenital diaphragmatic hernia. J Pediatr Surg 1984;16:666-71. [18] Germain JF, Farnouz C, Pinquier D, et al. Can blood gas values predict pulmonary hypoplasia in antenatally diagnosed congenital diaphragmatic hernia. J Pediatr Surg 1996;31:1634-9. [19] Langer JC, Filler RM, Bohn DJ, et al. Timing of surgery for congenital diaphragmatic hernia: is emergency operation necessary. J Pediatr Surg 1988;23:731-4. [20] West KW, Bengston K, Rescorla FJ, et al. Delayed surgical repair and ECMO improves survival in congenital diaphragmatic hernia. Ann Surg 1992;216:454-62.