- Email: [email protected]
Risk-stratification enables accurate single-center outcomes assessment in congenital diaphragmatic hernia (CDH)
Journal of Pediatric Surgery 54 (2019) 932–936
Contents lists available at ScienceDirect
Journal of Pediatric Surgery journal homepage: www.elsevier.com/locate/jpedsurg
Risk-stratification enables accurate single-center outcomes assessment in congenital diaphragmatic hernia (CDH)☆ Tim Jancelewicz a,⁎, Elizabeth A. Paton a, Jorie Jones a, Mark F. Weems b, Pamela A. Lally c, Max R. Langham Jr. a, Congenital Diaphragmatic Hernia Study Group a b c
Division of Pediatric Surgery, Le Bonheur Children's Hospital, University of Tennessee Health Science Center, Memphis, TN, USA Division of Neonatal-Perinatal Medicine, Le Bonheur Children's Hospital, University of Tennessee Health Science Center, Memphis, TN, USA University of Texas McGovern Medical School and Children's Memorial Hermann Hospital, Houston, TX, USA
a r t i c l e
i n f o
Article history: Received 26 January 2019 Accepted 27 January 2019 Key words: Congenital diaphragmatic hernia CDH Extracorporeal membrane oxygenation
a b s t r a c t Background: Management of CDH is highly variable from center to center, as are patient outcomes. The purpose of this study was to examine risk-stratified survival and extracorporeal membrane oxygenation (ECMO) rates at a single center, and to determine whether adverse outcomes are related to patient characteristics or management. Methods: A retrospective single-center review of CDH patients was performed, and outcomes compared to those reported by the CDH Study Group (CDHSG) registry. Patient demographics, disparities, and clinical characteristics were examined to identify unique features of the cohort. A model derived using the registry that estimates probability of ECMO use or death in CDH newborns was used to risk-stratify patients and assess mortality rates. Observed over expected (O/E) ECMO use rates were calculated to measure whether “excess” or “appropriate” ECMO use was occurring. Results: There were 81 CDH patients treated between 2004–2017, and 5034 in the CDHSG registry. Mortality in ECMO-treated patients was higher than the registry. Socioeconomic variables were not significantly associated with outcomes. The strongest predictors of mortality were ECMO use and early blood gas variables. The risk model accurately predicted ECMO use with a c-statistic of 0.79. Compared with the registry, the disparity in mortality rates was greatest for moderate-risk patients. O/E ECMO use was highest in low and moderate-risk patients. Conclusions: ECMO use is a more consistent predictor of mortality than CDH severity at a single center, and there is relative overuse of ECMO in lower-risk patients. Risk stratification allows for more accurate institutional assessment of mortality and ECMO use, and other centers could consider such an adjusted analysis to identify opportunities for outcomes improvement. Level of Evidence: III. © 2019 Elsevier Inc. All rights reserved.
There is great variation in survival rates between centers for infants with severe congenital diaphragmatic hernia (CDH), which may be due to differences in innate patient characteristics, referral patterns, or clinical management [1,2]. Use of extracorporeal membrane oxygenation (ECMO) is known to be associated with overall decreased survival, but disease severity, comorbidities, patient selection, and quality of ECMO management all affect individual outcomes. At our center, we perceived a low survival rate for CDH patients requiring ECMO, which prompted a thorough review of our outcomes in an attempt to identify areas for improvement. We aimed to determine whether patient management or patient characteristics are the strongest determinants of survival, with a focus on assessing whether over-utilization of ECMO might be a source ☆ None of the authors have any competing interests to declare. ⁎ Corresponding author at: Le Bonheur Children's Hospital, Division of Pediatric Surgery, 49 North Dunlap St., Second Floor, Memphis, TN 38105, USA. Tel.: +1 901 545 9973. E-mail address: [email protected] (T. Jancelewicz). https://doi.org/10.1016/j.jpedsurg.2019.01.020 0022-3468/© 2019 Elsevier Inc. All rights reserved.
of increased mortality at our hospital. Our region has a high poverty rate and our population is majority black, so we also wished to examine the role of socioeconomic factors and race at the center level; these variables have previously been shown to significantly affect survival in CDH using a national database [3,4]. We hypothesized that singlecenter outcomes for CDH are more dependent on patient factors than details of clinical management. 1. Methods This study was approved by the Institutional Review Board of the University of Tennessee Health Science Center (Protocol #13–02903XP). Research using the CDH Study Group (CDHSG) registry is approved by the University of Texas McGovern Medical School in Houston Center for the Protection of Human Subjects/Institutional Review Board (#HSC-MS-03-223; Ref #118886). All statistical analyses were performed using SAS/STAT® 9.4.
T. Jancelewicz et al. / Journal of Pediatric Surgery 54 (2019) 932–936
1.1. Patients and univariate analysis A retrospective review was performed of all patients with CDH treated at a single freestanding academic children's hospital from 2004 to 2017 [Le Bonheur Children's Hospital (LBCH)]. Detailed clinical characteristics and outcomes were measured and compared where possible with patient data abstracted from the CDHSG registry (date of birth between 2008 and 2018). Pearson's chi squared test or Fisher's exact test were used for categorical variables and Student's two-sided t-test was used for normally distributed continuous variables, with p b 0.05 considered significant. The primary outcomes assessed were all case mortality and mortality in ECMO-treated patients (“ECMO mortality”). 1.2. Socioeconomic variables Maternal insurance status was obtained, and maternal zip codebased median income was estimated using 2006–2010 Census Bureau data. The effects of ethnicity, income, and insurance status on ECMO use and survival were assessed with univariate analysis. 1.3. Multivariable analysis For the outcome of mortality, significant and relevant variables from the univariate analyses were used for multivariable logistic regression with automatic stepwise forward selection, using a p b 0.1 as a threshold for variable inclusion. This was done to identify the strongest independent predictors of mortality for our center's CDH population. 1.4. Analysis with ECMO risk stratification The CDHSG has collected patient data from participating centers interested in improving the care of CDH for over 20 years. These data allow comparison of individual center results with peer institutions in a risk-adjusted manner. Multiple tools for predicting mortality risk have been derived from the CDHSG registry [5,6]. More recently, a model was created using the registry that estimates risk of ECMO treatment or death without ECMO using early postnatal covariates, with the purpose of serving as a metric for institutional assessment of ECMO usage and outcomes [7]. The model was derived using ECMO candidates only [patients with a gestational age (GA) ≥32 weeks and a birth weight ≥1.8 kg with no severe life-threatening associated anomalies]. This risk score employs four postnatal variables: 1 and 5 min Apgar scores, and highest and lowest postductal arterial partial pressure of CO2 (PCO2) during the first 24 h of life. These covariates were found to be the most accurate predictors currently available in the registry. This model was used to risk-stratify LBCH patients. First, the model was validated for predictive discrimination on the study population using logistic regression and receiver-operator characteristic (ROC) curve analysis. Model calibration was assessed using the HosmerLemeshow goodness-of-fit test. Patients were then stratified patients into five tiers of risk of ECMO (or death without ECMO): 0–20%, 20–40%, 40–60%, 60–80%, and N 80% probability. Outcomes in the riskstratified groups were compared with outcomes in identically stratified patients from the CDHSG registry. Finally, observed/expected (O/E) ECMO use and mortality were calculated for each risk tier. 2. Results 2.1. Univariate analysis There were 81 CDH patients treated during the study period. Overall mortality and rate of ECMO use were not significantly different from 5034 CDHSG registry patients. Our center had more right-sided CDH patients, lower prenatal diagnosis and inborn rates, and a higher proportion of Black patients (Table 1). Mortality in our center's ECMOtreated patients (ECMO mortality) was significantly higher than those
933
Table 1 Clinical characteristics and outcomes at index center (LBCH) compared with patients from the CDHSG registry database. Variable
LBCH (N = 81), N or mean ± SD
CDHSG (N = 5034), N or mean ± SD
p
Male (%) Mortality (%) ECMO (%) ECMO mortality (%) Birth weight (kg) EGA (weeks) Prenatal diagnosis (%) Inborn (%) Left CDH (%) Large CDH defect (C or D)⁎ (%) Race (%) Black White Hispanic ECMO duration (days) Repair rate (%)
51 (63.0) 31 (38.3) 26 (32.1) 18 (69.2) 2.99 ± 0.7 37.8 ± 2.7 27 (33.3) 25 (32.1) 54 (66.7) 24 (36.4)
2922 (58.2) 1455 (29.0) 1435 (28.5) 711 (49.7) 2.94 ± 0.7 37.5 ± 2.4 3508 (69.7) 2633 (52.4) 4165 (82.9) 1991 (47.6)
0.37 0.07 0.48 b0.0001 0.52 0.26 b0.0001 0.0001 0.0002 0.07
39 (48.1) 29 (35.8) 10 (12.3) 23.4 ± 12.7 66 (81.5)
407 (8.1) 2906 (57.7) 765 (15.2) 12.1 ± 7.7 4198 (83.4)
b0.0001 b0.0001 0.48 b0.0001 0.64
⁎ Defect size measured according to CDHSG criteria.
treated with ECMO in the registry. There were fewer patients with large defects (which was only measured in repaired patients), and an equivalent non-repair rate. When stratified by race, insurance status (private versus public or no insurance), or average income by maternal zip code, there were no significant differences in survival or ECMO use (Fig. 1). There were also no significant differences between these groups in the rate of prenatal diagnosis (data not shown). Multiple variables were significantly associated with mortality, including blood gas measurements obtained early after delivery and within the first 24 h of life (Table 2). The strongest predictor of mortality by univariate analysis was presence of a large defect size, using the CDHSG grading system [8]. Race and insurance status were not associated with mortality rates. 2.2. Multivariate analysis Stepwise selection of significant relevant variables from the univariate analysis (Table 2) into a multivariable regression model showed that the only significant independent predictors of mortality were ECMO use [odds ratio (OR) 26.6, 95% confidence interval (CI) 1.8–404, p = 0.018], and first arterial pH (OR point estimate 0.004 [95% CI b0.001–0.915], p = 0.046). Other variables that fell out of the analysis due to confounding by ECMO use included presence of a large defect (p = 0.09), Apgar scores, and other postnatal blood gas variables. 2.3. Stratification with ECMO risk model Including only ECMO candidates with complete data for all model variables (N = 44), the ECMO risk model accurately predicted the composite outcome of either ECMO use or death without ECMO use, with a c statistic of 0.79 [95% confidence interval (CI) 0.67–0.96]. The Hosmer and Lemeshow goodness-of-fit test p value was 0.75 (good fit). The model was also accurate at predicting the outcome of death alone (c statistic 0.78, 95% CI 0.63–0.92). The model was much poorer at predicting ECMO use alone (c statistic 0.65, 95% CI 0.48–0.82). The risk score was arbitrarily divided into 5 tiers of risk of ECMO (or death without ECMO): 0–20%, 20–40%, 40–60%, 60–80%, and N 80% probability. Mortality for each risk tier was not significantly higher than the registry, although there was a trend for moderate-risk patients (Fig. 2A). The rate of ECMO use for moderate-risk patients was significantly higher than the registry (Fig. 2B). ECMO mortality was higher at our center than the registry (Table 1), and this trend was apparent for all risk tiers, though not statistically different for individual tiers
934
T. Jancelewicz et al. / Journal of Pediatric Surgery 54 (2019) 932–936
Mortality
ECMO use
Zip code income
p=0.32 1/5
2/11
5/11
medium
low
p=0.63
3/5
high
18/58
18/58
Race p=0.45 Black or Hispanic
White
p=0.52
20/49
14/49
9/28
10/28
Insurance
p=0.2 Private
Public or none
p=0.3
7/25
6/25
24/56
20/56
Fig. 1. Comparison of mortality and ECMO use rates in patients stratified by race and socioeconomic variables. Zip code incomes were taken from census data for maternal zip code (low income b$29,758–$59,515, medium $59,516–$89,274, and high N$89,274).
completion of a multi-institutional randomized trial, there remains great discrepancy in patterns of care, with for example some centers offering ECMO and others not [11,12]. In this environment of uncertainty regarding best practices, it is difficult for individual centers to credibly review their outcomes in order to identify areas for improvement. Matching center performance against peer institutions is complicated 100%
A mortality
(Fig. 2C). The greatest disparity in ECMO mortality between LBCH and the registry occurred in the second-lowest risk group (p = 0.09). After risk stratification, there were still no significant differences in mortality or rate of ECMO use across risk categories between black and white patients or between those with private or public/no insurance (data not shown). For the 44 ECMO candidates with complete data to calculate an ECMO risk score, O/E ECMO use (or death without ECMO) was highest in low and moderate risk patients (Table 3). Additionally, the vast majority of mortality occurred in ECMO-treated patients. 3. Discussion
75% LBCH
50% CDHSG
25%
ECMO use
Table 2 Selected potential predictors of mortality, ranked by significance of the odds ratio (OR). Variable
Odds ratio (OR), or OR point estimate (95% CI)
P
Large CDH defect (C or D)⁎ ECMO use Prenatal diagnosis Highest arterial pCO2 during first 24 h of life First arterial pH First arterial pCO2 Prematurity ≤32 weeks 1 min Apgar 5 min Apgar Left-sided CDH Lowest arterial pCO2 during first 24 h of life Birth weight Inborn Public insurance Black race
33.3 (6.4–172.4) 7.27 (2.6–20.1) 4.86 (1.8–13.1) 1.03 (1.01–1.06)
b0.0001 b0.0001 0.001 0.001
0.02 (0.001–0.3) 1.03 (1.01–1.05) 4.6 (0.8–25.5) 0.76 (0.61–0.95) 0.73 (0.55–0.95) 3.02 (1.1–8.7) 1.05 (1.00–1.10)
0.004 0.006 0.08 0.02 0.02 0.04 0.04
0.99 (0.998–1.0) 1.8 (0.7–4.7) 1.71 (0.6–4.9) 1.25 (0.5–3.1)
0.05 0.23 0.31 0.62
⁎ Defect size measured according to CDHSG criteria.
C
*
100%
B
75% 50% 25%
100%
ECMO mortality
The treatment of patients with severe CDH is extraordinarily complex, and there is no clear management consensus between centers regarding most aspects of care [8–10]. Even amongst European centers in the CDH EURO Consortium, which has aligned care enough to allow
**
75% 50% 25%
1 2 low risk
3
4
5 high risk
Fig. 2. Comparison of index center (LBCH, N = 44, solid line) and CDHSG registry patients (N = 3586, dashed line) stratified from low to high risk of ECMO or death without ECMO use (risk tiers 1 to 5), looking at (A) mortality, (B) ECMO use rates, and (C) mortality in ECMO-treated patients (ECMO mortality). *p = 0.03 for patients with risk = 3 (40–60% risk of ECMO or death without ECMO use). **pb0.0001 comparing the plots; N for individual risk tiers are too low to show significance.
T. Jancelewicz et al. / Journal of Pediatric Surgery 54 (2019) 932–936
935
Table 3 Expected, observed, and observed/expected (O/E) ECMO use (or death without ECMO use) for five risk tiers (low to high). The average model-predicted risk was calculated for each risk tier and converted into N expected. Risk tier
N
N expected ECMO or death without ECMO use
N observed ECMO
N observed death without ECMO use
N observed ECMO or death without ECMO use
O/E ratio
1 (b20% risk) 2 (20–40% risk) 3 (40–60% risk) 4 (60–80% risk) 5 (N80% risk) TOTAL
12 10 4 10 8 44
1.5 3.1 2 6.9 7.2 20.7(47%)
2 3 4 6 3 18 (41%)
0 1 0 2 3 6 (14%)
2 4 4 8 6 24 (55%)
1.32 1.29 2.0 1.16 0.83 1.16
by both the heterogeneity of CDH severity and management differences between centers for similar patients. To properly assess our outcomes, we used the CDHSG registry to compare our results after risk stratification using a model that estimates probability of ECMO use. We hypothesized that CDH severity would be the major determinant of outcomes, but our analysis instead found that ECMO use was the strongest independent predictor of mortality. Le Bonheur Children's Hospital is a free-standing children's hospital in the American Mid-South. We serve as the safety net hospital for a region of 1.5 million people and accept all children regardless of ability to pay or insurer type. Our quality ratings are high and above many similar peer institutions. We perceived a higher-than-expected mortality rate at our institution for CDH patients treated with ECMO, which prompted us to study those factors at our hospital that may contribute to this mortality. An unadjusted review of outcomes found no significant differences in survival or ECMO use in our population compared with the CDHSG registry, despite significantly different clinical features for our population such as race composition and prenatal diagnosis rates. We noted at our center significantly higher mortality than the registry for patients treated with ECMO. However, after appropriate risk stratification using an ECMO risk model derived from the CDHSG registry and including only ECMO candidates [7], it is apparent that the mortality disparity was concentrated in moderate-risk patients. We were also able to observe a relative overuse of ECMO, with the highest O/E ratios seen in low-to-moderate risk patients. High mortality was seen in ECMO-treated patients compared with the registry regardless of CDH severity, and even the highest-risk CDH patients did not see a survival benefit from ECMO use at our center. Others have reported survival benefit in these patients [13,14]. Taken together, these data imply that ECMO-related complications may be overshadowing any potential benefit from its use. Interestingly, worse survival with ECMO use for CDH patients over the past decade has also been noted in a recent study from the Extracorporeal Life Support Organization (ELSO), though it is not clear whether this is due to patient severity or management [15]. In contrast to previous studies using national and regional databases, we found that race and socioeconomic status are not significant predictors of outcome at our center, even after risk stratification [3,16,17]. This may simply reflect a center-specific lack of disparities in care or relative uniformity in the quality of regional prenatal and perinatal care. Multivariable analysis showed that ECMO use and blood gas markers of severe pulmonary hypertension and poor gas exchange were strong predictors of mortality, in keeping with multiple previous models examining mortality risk in CDH [6,18–21]. Unsurprisingly then, the ECMO risk model, which incorporates arterial PCO2 levels, was found to perform well at our center. Other methods to stratify risk may be used to assess outcomes in CDH. For example, the CDHSG defect size categories are strongly correlated with mortality [8]; the inability to apply this risk assessment to patients in our series who were not repaired weakens the utility of this method. This study is limited by the small sample size, especially when examining risk-stratified groups. Differences between groups may not be statistically apparent due to the small number of patients seen in a single center. Furthermore, we defined an ECMO candidate as an infant with CDH who was born after at least 32 weeks' gestation with a birthweight
greater than 1.8 kg; our findings may not be applicable to those with more restrictive ECMO criteria. Another major concern is that morbidity and causes of mortality were not reported here. We do not know for instance whether complications directly attributable to ECMO were more or less important than excess pulmonary morbidity due to ventilator use that made attempted ECMO rescue necessary. A detailed analysis of complications and cause of death in our patients is underway. Finally, mortality is just one measure of outcome in CDH; measurement of longterm morbidity such as neurodevelopmental delay after ECMO and pulmonary sequelae would be essential for a compete assessment of care quality. Cost–benefit and patient quality-of-life are both important to study when examining CDH outcomes and were not analyzed in this study. For our center, the results of this analysis have reinforced that a standardized management pathway with firm criteria for transitions in care is essential, not only to improve outcomes but also to provide consistent management that can be more easily adjusted over time. We therefore developed a comprehensive CDH clinical practice guideline at our center that minimizes variability of practice. The guideline recommends ECMO for CDH patients who fail specific respiratory support measures within a lung protective strategy. In the first 30 months of implementation, total cohort CDH ECMO mortality has decreased from 69% to 60%. However, patient volume is too small to make strong conclusions. Ongoing analysis will determine if the improvement in survival will be sustained and if O/E ECMO use will approach 1:1. In summary, we have shown with this retrospective cohort study that application of an ECMO risk stratification model to provide adjusted assessment of CDH outcomes may help other individual centers determine relative overuse or underuse of ECMO and identify targets for institutional improvement in outcomes. Future studies should examine which center characteristics, as well as CDH and ECMO management strategies, are associated with the best outcomes, ideally through multicenter prospective trials. CRediT authorship contribution statement Tim Jancelewicz: Conceptualization, Data curation, Formal analysis, Writing – original draft, Writing – review & editing. Elizabeth A. Paton: Writing – original draft, Writing – review & editing. Jorie Jones: Formal analysis. Mark F. Weems: Formal analysis, Writing – original draft, Writing – review & editing. Pamela A. Lally: Data curation, Writing – original draft, Writing – review & editing. Max R. Langham Jr.: Conceptualization, Data curation, Formal analysis, Writing – original draft, Writing – review & editing. References [1] Harting MT, Hollinger L, Tsao K, et al. Aggressive surgical Management of Congenital Diaphragmatic Hernia: worth the effort?: a multicenter, prospective, Cohort Study. Ann Surg 2018;267:977–82. [2] Kays DW, Talbert JL, Islam S, et al. Improved survival in left liver-up congenital diaphragmatic hernia by early repair before extracorporeal membrane oxygenation: optimization of patient selection by multivariate risk modeling. J Am Coll Surg 2016;222:459–70. [3] Sola JE, Bronson SN, Cheung MC, et al. Survival disparities in newborns with congenital diaphragmatic hernia: a national perspective. J Pediatr Surg 2010;45:1336–42.
936
T. Jancelewicz et al. / Journal of Pediatric Surgery 54 (2019) 932–936
[4] Davis JS, Ryan ML, Perez EA, et al. ECMO hospital volume and survival in congenital diaphragmatic hernia repair. J Surg Res 2012;178:791–6. [5] Brindle ME, Cook EF, Tibboel D, et al. A clinical prediction rule for the severity of congenital diaphragmatic hernias in newborns. Pediatrics 2014;134:e413–9. [6] Schultz CM, DiGeronimo RJ, Yoder BA, et al. Congenital diaphragmatic hernia: a simplified postnatal predictor of outcome. J Pediatr Surg 2007;42:510–6. [7] Jancelewicz T, Brindle ME, Harting MT, et al. Extracorporeal membrane oxygenation (ECMO) risk stratification in newborns with congenital diaphragmatic hernia (CDH). J Pediatr Surg 2018;53:1890–5. [8] Lally KP, Lasky RE, Lally PA, et al. Standardized reporting for congenital diaphragmatic hernia–an international consensus. J Pediatr Surg 2013;48:2408–15. [9] Antonoff MB, Hustead VA, Groth SS, et al. Protocolized management of infants with congenital diaphragmatic hernia: effect on survival. J Pediatr Surg 2011;46:39–46. [10] Tracy ET, Mears SE, Smith PB, et al. Protocolized approach to the management of congenital diaphragmatic hernia: benefits of reducing variability in care. J Pediatr Surg 2010;45:1343–8. [11] Snoek KG, Capolupo I, van Rosmalen J, et al. Conventional mechanical ventilation versus high-frequency oscillatory ventilation for congenital diaphragmatic hernia: a randomized clinical trial (the VICI-trial). Ann Surg 2016;263:867–74. [12] Snoek KG, Capolupo I, Morini F, et al. Score for neonatal acute physiology-II predicts outcome in congenital diaphragmatic hernia patients. Pediatr Crit Care Med 2016; 17:540–6.
[13] Group TCDHS. Does extracorporeal membrane oxygenation improve survival in neonates with congenital diaphragmatic hernia? The Congenital Diaphragmatic Hernia Study Group. J Pediatr Surg 1999;34:720–4 [discussion 724–725]. [14] Kays DW. ECMO in CDH: is there a role? Semin Pediatr Surg 2017;26:166–70. [15] Turek JW, Nellis JR, Sherwood BG, et al. Shifting risks and conflicting outcomesECMO for neonates with congenital diaphragmatic hernia in the modern era. J Pediatr 2017;190:163–8 [e164]. [16] Aly H, Bianco-Batlles D, Mohamed MA, et al. Mortality in infants with congenital diaphragmatic hernia: a study of the United States National Database. J Perinatol 2010;30:553–7. [17] Hinton CF, Siffel C, Correa A, et al. Survival disparities associated with congenital diaphragmatic hernia. Birth Defects Res 2017;109:816–23. [18] Baird R, MacNab YC, Skarsgard ED, et al. Mortality prediction in congenital diaphragmatic hernia. J Pediatr Surg 2008;43:783–7. [19] Grizelj R, Bojanic K, Pritisanac E, et al. Survival prediction of high-risk outborn neonates with congenital diaphragmatic hernia from capillary blood gases. BMC Pediatr 2016;16(114). [20] Khmour AY, Konduri GG, Sato TT, et al. Role of admission gas exchange measurement in predicting congenital diaphragmatic hernia survival in the era of gentle ventilation. J Pediatr Surg 2014;49:1197–201. [21] Park HW, Lee BS, Lim G, et al. A simplified formula using early blood gas analysis can predict survival outcomes and the requirements for extracorporeal membrane oxygenation in congenital diaphragmatic hernia. J Korean Med Sci 2013;28:924–8.
Contents lists available at ScienceDirect
Journal of Pediatric Surgery journal homepage: www.elsevier.com/locate/jpedsurg
Risk-stratification enables accurate single-center outcomes assessment in congenital diaphragmatic hernia (CDH)☆ Tim Jancelewicz a,⁎, Elizabeth A. Paton a, Jorie Jones a, Mark F. Weems b, Pamela A. Lally c, Max R. Langham Jr. a, Congenital Diaphragmatic Hernia Study Group a b c
Division of Pediatric Surgery, Le Bonheur Children's Hospital, University of Tennessee Health Science Center, Memphis, TN, USA Division of Neonatal-Perinatal Medicine, Le Bonheur Children's Hospital, University of Tennessee Health Science Center, Memphis, TN, USA University of Texas McGovern Medical School and Children's Memorial Hermann Hospital, Houston, TX, USA
a r t i c l e
i n f o
Article history: Received 26 January 2019 Accepted 27 January 2019 Key words: Congenital diaphragmatic hernia CDH Extracorporeal membrane oxygenation
a b s t r a c t Background: Management of CDH is highly variable from center to center, as are patient outcomes. The purpose of this study was to examine risk-stratified survival and extracorporeal membrane oxygenation (ECMO) rates at a single center, and to determine whether adverse outcomes are related to patient characteristics or management. Methods: A retrospective single-center review of CDH patients was performed, and outcomes compared to those reported by the CDH Study Group (CDHSG) registry. Patient demographics, disparities, and clinical characteristics were examined to identify unique features of the cohort. A model derived using the registry that estimates probability of ECMO use or death in CDH newborns was used to risk-stratify patients and assess mortality rates. Observed over expected (O/E) ECMO use rates were calculated to measure whether “excess” or “appropriate” ECMO use was occurring. Results: There were 81 CDH patients treated between 2004–2017, and 5034 in the CDHSG registry. Mortality in ECMO-treated patients was higher than the registry. Socioeconomic variables were not significantly associated with outcomes. The strongest predictors of mortality were ECMO use and early blood gas variables. The risk model accurately predicted ECMO use with a c-statistic of 0.79. Compared with the registry, the disparity in mortality rates was greatest for moderate-risk patients. O/E ECMO use was highest in low and moderate-risk patients. Conclusions: ECMO use is a more consistent predictor of mortality than CDH severity at a single center, and there is relative overuse of ECMO in lower-risk patients. Risk stratification allows for more accurate institutional assessment of mortality and ECMO use, and other centers could consider such an adjusted analysis to identify opportunities for outcomes improvement. Level of Evidence: III. © 2019 Elsevier Inc. All rights reserved.
There is great variation in survival rates between centers for infants with severe congenital diaphragmatic hernia (CDH), which may be due to differences in innate patient characteristics, referral patterns, or clinical management [1,2]. Use of extracorporeal membrane oxygenation (ECMO) is known to be associated with overall decreased survival, but disease severity, comorbidities, patient selection, and quality of ECMO management all affect individual outcomes. At our center, we perceived a low survival rate for CDH patients requiring ECMO, which prompted a thorough review of our outcomes in an attempt to identify areas for improvement. We aimed to determine whether patient management or patient characteristics are the strongest determinants of survival, with a focus on assessing whether over-utilization of ECMO might be a source ☆ None of the authors have any competing interests to declare. ⁎ Corresponding author at: Le Bonheur Children's Hospital, Division of Pediatric Surgery, 49 North Dunlap St., Second Floor, Memphis, TN 38105, USA. Tel.: +1 901 545 9973. E-mail address: [email protected] (T. Jancelewicz). https://doi.org/10.1016/j.jpedsurg.2019.01.020 0022-3468/© 2019 Elsevier Inc. All rights reserved.
of increased mortality at our hospital. Our region has a high poverty rate and our population is majority black, so we also wished to examine the role of socioeconomic factors and race at the center level; these variables have previously been shown to significantly affect survival in CDH using a national database [3,4]. We hypothesized that singlecenter outcomes for CDH are more dependent on patient factors than details of clinical management. 1. Methods This study was approved by the Institutional Review Board of the University of Tennessee Health Science Center (Protocol #13–02903XP). Research using the CDH Study Group (CDHSG) registry is approved by the University of Texas McGovern Medical School in Houston Center for the Protection of Human Subjects/Institutional Review Board (#HSC-MS-03-223; Ref #118886). All statistical analyses were performed using SAS/STAT® 9.4.
T. Jancelewicz et al. / Journal of Pediatric Surgery 54 (2019) 932–936
1.1. Patients and univariate analysis A retrospective review was performed of all patients with CDH treated at a single freestanding academic children's hospital from 2004 to 2017 [Le Bonheur Children's Hospital (LBCH)]. Detailed clinical characteristics and outcomes were measured and compared where possible with patient data abstracted from the CDHSG registry (date of birth between 2008 and 2018). Pearson's chi squared test or Fisher's exact test were used for categorical variables and Student's two-sided t-test was used for normally distributed continuous variables, with p b 0.05 considered significant. The primary outcomes assessed were all case mortality and mortality in ECMO-treated patients (“ECMO mortality”). 1.2. Socioeconomic variables Maternal insurance status was obtained, and maternal zip codebased median income was estimated using 2006–2010 Census Bureau data. The effects of ethnicity, income, and insurance status on ECMO use and survival were assessed with univariate analysis. 1.3. Multivariable analysis For the outcome of mortality, significant and relevant variables from the univariate analyses were used for multivariable logistic regression with automatic stepwise forward selection, using a p b 0.1 as a threshold for variable inclusion. This was done to identify the strongest independent predictors of mortality for our center's CDH population. 1.4. Analysis with ECMO risk stratification The CDHSG has collected patient data from participating centers interested in improving the care of CDH for over 20 years. These data allow comparison of individual center results with peer institutions in a risk-adjusted manner. Multiple tools for predicting mortality risk have been derived from the CDHSG registry [5,6]. More recently, a model was created using the registry that estimates risk of ECMO treatment or death without ECMO using early postnatal covariates, with the purpose of serving as a metric for institutional assessment of ECMO usage and outcomes [7]. The model was derived using ECMO candidates only [patients with a gestational age (GA) ≥32 weeks and a birth weight ≥1.8 kg with no severe life-threatening associated anomalies]. This risk score employs four postnatal variables: 1 and 5 min Apgar scores, and highest and lowest postductal arterial partial pressure of CO2 (PCO2) during the first 24 h of life. These covariates were found to be the most accurate predictors currently available in the registry. This model was used to risk-stratify LBCH patients. First, the model was validated for predictive discrimination on the study population using logistic regression and receiver-operator characteristic (ROC) curve analysis. Model calibration was assessed using the HosmerLemeshow goodness-of-fit test. Patients were then stratified patients into five tiers of risk of ECMO (or death without ECMO): 0–20%, 20–40%, 40–60%, 60–80%, and N 80% probability. Outcomes in the riskstratified groups were compared with outcomes in identically stratified patients from the CDHSG registry. Finally, observed/expected (O/E) ECMO use and mortality were calculated for each risk tier. 2. Results 2.1. Univariate analysis There were 81 CDH patients treated during the study period. Overall mortality and rate of ECMO use were not significantly different from 5034 CDHSG registry patients. Our center had more right-sided CDH patients, lower prenatal diagnosis and inborn rates, and a higher proportion of Black patients (Table 1). Mortality in our center's ECMOtreated patients (ECMO mortality) was significantly higher than those
933
Table 1 Clinical characteristics and outcomes at index center (LBCH) compared with patients from the CDHSG registry database. Variable
LBCH (N = 81), N or mean ± SD
CDHSG (N = 5034), N or mean ± SD
p
Male (%) Mortality (%) ECMO (%) ECMO mortality (%) Birth weight (kg) EGA (weeks) Prenatal diagnosis (%) Inborn (%) Left CDH (%) Large CDH defect (C or D)⁎ (%) Race (%) Black White Hispanic ECMO duration (days) Repair rate (%)
51 (63.0) 31 (38.3) 26 (32.1) 18 (69.2) 2.99 ± 0.7 37.8 ± 2.7 27 (33.3) 25 (32.1) 54 (66.7) 24 (36.4)
2922 (58.2) 1455 (29.0) 1435 (28.5) 711 (49.7) 2.94 ± 0.7 37.5 ± 2.4 3508 (69.7) 2633 (52.4) 4165 (82.9) 1991 (47.6)
0.37 0.07 0.48 b0.0001 0.52 0.26 b0.0001 0.0001 0.0002 0.07
39 (48.1) 29 (35.8) 10 (12.3) 23.4 ± 12.7 66 (81.5)
407 (8.1) 2906 (57.7) 765 (15.2) 12.1 ± 7.7 4198 (83.4)
b0.0001 b0.0001 0.48 b0.0001 0.64
⁎ Defect size measured according to CDHSG criteria.
treated with ECMO in the registry. There were fewer patients with large defects (which was only measured in repaired patients), and an equivalent non-repair rate. When stratified by race, insurance status (private versus public or no insurance), or average income by maternal zip code, there were no significant differences in survival or ECMO use (Fig. 1). There were also no significant differences between these groups in the rate of prenatal diagnosis (data not shown). Multiple variables were significantly associated with mortality, including blood gas measurements obtained early after delivery and within the first 24 h of life (Table 2). The strongest predictor of mortality by univariate analysis was presence of a large defect size, using the CDHSG grading system [8]. Race and insurance status were not associated with mortality rates. 2.2. Multivariate analysis Stepwise selection of significant relevant variables from the univariate analysis (Table 2) into a multivariable regression model showed that the only significant independent predictors of mortality were ECMO use [odds ratio (OR) 26.6, 95% confidence interval (CI) 1.8–404, p = 0.018], and first arterial pH (OR point estimate 0.004 [95% CI b0.001–0.915], p = 0.046). Other variables that fell out of the analysis due to confounding by ECMO use included presence of a large defect (p = 0.09), Apgar scores, and other postnatal blood gas variables. 2.3. Stratification with ECMO risk model Including only ECMO candidates with complete data for all model variables (N = 44), the ECMO risk model accurately predicted the composite outcome of either ECMO use or death without ECMO use, with a c statistic of 0.79 [95% confidence interval (CI) 0.67–0.96]. The Hosmer and Lemeshow goodness-of-fit test p value was 0.75 (good fit). The model was also accurate at predicting the outcome of death alone (c statistic 0.78, 95% CI 0.63–0.92). The model was much poorer at predicting ECMO use alone (c statistic 0.65, 95% CI 0.48–0.82). The risk score was arbitrarily divided into 5 tiers of risk of ECMO (or death without ECMO): 0–20%, 20–40%, 40–60%, 60–80%, and N 80% probability. Mortality for each risk tier was not significantly higher than the registry, although there was a trend for moderate-risk patients (Fig. 2A). The rate of ECMO use for moderate-risk patients was significantly higher than the registry (Fig. 2B). ECMO mortality was higher at our center than the registry (Table 1), and this trend was apparent for all risk tiers, though not statistically different for individual tiers
934
T. Jancelewicz et al. / Journal of Pediatric Surgery 54 (2019) 932–936
Mortality
ECMO use
Zip code income
p=0.32 1/5
2/11
5/11
medium
low
p=0.63
3/5
high
18/58
18/58
Race p=0.45 Black or Hispanic
White
p=0.52
20/49
14/49
9/28
10/28
Insurance
p=0.2 Private
Public or none
p=0.3
7/25
6/25
24/56
20/56
Fig. 1. Comparison of mortality and ECMO use rates in patients stratified by race and socioeconomic variables. Zip code incomes were taken from census data for maternal zip code (low income b$29,758–$59,515, medium $59,516–$89,274, and high N$89,274).
completion of a multi-institutional randomized trial, there remains great discrepancy in patterns of care, with for example some centers offering ECMO and others not [11,12]. In this environment of uncertainty regarding best practices, it is difficult for individual centers to credibly review their outcomes in order to identify areas for improvement. Matching center performance against peer institutions is complicated 100%
A mortality
(Fig. 2C). The greatest disparity in ECMO mortality between LBCH and the registry occurred in the second-lowest risk group (p = 0.09). After risk stratification, there were still no significant differences in mortality or rate of ECMO use across risk categories between black and white patients or between those with private or public/no insurance (data not shown). For the 44 ECMO candidates with complete data to calculate an ECMO risk score, O/E ECMO use (or death without ECMO) was highest in low and moderate risk patients (Table 3). Additionally, the vast majority of mortality occurred in ECMO-treated patients. 3. Discussion
75% LBCH
50% CDHSG
25%
ECMO use
Table 2 Selected potential predictors of mortality, ranked by significance of the odds ratio (OR). Variable
Odds ratio (OR), or OR point estimate (95% CI)
P
Large CDH defect (C or D)⁎ ECMO use Prenatal diagnosis Highest arterial pCO2 during first 24 h of life First arterial pH First arterial pCO2 Prematurity ≤32 weeks 1 min Apgar 5 min Apgar Left-sided CDH Lowest arterial pCO2 during first 24 h of life Birth weight Inborn Public insurance Black race
33.3 (6.4–172.4) 7.27 (2.6–20.1) 4.86 (1.8–13.1) 1.03 (1.01–1.06)
b0.0001 b0.0001 0.001 0.001
0.02 (0.001–0.3) 1.03 (1.01–1.05) 4.6 (0.8–25.5) 0.76 (0.61–0.95) 0.73 (0.55–0.95) 3.02 (1.1–8.7) 1.05 (1.00–1.10)
0.004 0.006 0.08 0.02 0.02 0.04 0.04
0.99 (0.998–1.0) 1.8 (0.7–4.7) 1.71 (0.6–4.9) 1.25 (0.5–3.1)
0.05 0.23 0.31 0.62
⁎ Defect size measured according to CDHSG criteria.
C
*
100%
B
75% 50% 25%
100%
ECMO mortality
The treatment of patients with severe CDH is extraordinarily complex, and there is no clear management consensus between centers regarding most aspects of care [8–10]. Even amongst European centers in the CDH EURO Consortium, which has aligned care enough to allow
**
75% 50% 25%
1 2 low risk
3
4
5 high risk
Fig. 2. Comparison of index center (LBCH, N = 44, solid line) and CDHSG registry patients (N = 3586, dashed line) stratified from low to high risk of ECMO or death without ECMO use (risk tiers 1 to 5), looking at (A) mortality, (B) ECMO use rates, and (C) mortality in ECMO-treated patients (ECMO mortality). *p = 0.03 for patients with risk = 3 (40–60% risk of ECMO or death without ECMO use). **pb0.0001 comparing the plots; N for individual risk tiers are too low to show significance.
T. Jancelewicz et al. / Journal of Pediatric Surgery 54 (2019) 932–936
935
Table 3 Expected, observed, and observed/expected (O/E) ECMO use (or death without ECMO use) for five risk tiers (low to high). The average model-predicted risk was calculated for each risk tier and converted into N expected. Risk tier
N
N expected ECMO or death without ECMO use
N observed ECMO
N observed death without ECMO use
N observed ECMO or death without ECMO use
O/E ratio
1 (b20% risk) 2 (20–40% risk) 3 (40–60% risk) 4 (60–80% risk) 5 (N80% risk) TOTAL
12 10 4 10 8 44
1.5 3.1 2 6.9 7.2 20.7(47%)
2 3 4 6 3 18 (41%)
0 1 0 2 3 6 (14%)
2 4 4 8 6 24 (55%)
1.32 1.29 2.0 1.16 0.83 1.16
by both the heterogeneity of CDH severity and management differences between centers for similar patients. To properly assess our outcomes, we used the CDHSG registry to compare our results after risk stratification using a model that estimates probability of ECMO use. We hypothesized that CDH severity would be the major determinant of outcomes, but our analysis instead found that ECMO use was the strongest independent predictor of mortality. Le Bonheur Children's Hospital is a free-standing children's hospital in the American Mid-South. We serve as the safety net hospital for a region of 1.5 million people and accept all children regardless of ability to pay or insurer type. Our quality ratings are high and above many similar peer institutions. We perceived a higher-than-expected mortality rate at our institution for CDH patients treated with ECMO, which prompted us to study those factors at our hospital that may contribute to this mortality. An unadjusted review of outcomes found no significant differences in survival or ECMO use in our population compared with the CDHSG registry, despite significantly different clinical features for our population such as race composition and prenatal diagnosis rates. We noted at our center significantly higher mortality than the registry for patients treated with ECMO. However, after appropriate risk stratification using an ECMO risk model derived from the CDHSG registry and including only ECMO candidates [7], it is apparent that the mortality disparity was concentrated in moderate-risk patients. We were also able to observe a relative overuse of ECMO, with the highest O/E ratios seen in low-to-moderate risk patients. High mortality was seen in ECMO-treated patients compared with the registry regardless of CDH severity, and even the highest-risk CDH patients did not see a survival benefit from ECMO use at our center. Others have reported survival benefit in these patients [13,14]. Taken together, these data imply that ECMO-related complications may be overshadowing any potential benefit from its use. Interestingly, worse survival with ECMO use for CDH patients over the past decade has also been noted in a recent study from the Extracorporeal Life Support Organization (ELSO), though it is not clear whether this is due to patient severity or management [15]. In contrast to previous studies using national and regional databases, we found that race and socioeconomic status are not significant predictors of outcome at our center, even after risk stratification [3,16,17]. This may simply reflect a center-specific lack of disparities in care or relative uniformity in the quality of regional prenatal and perinatal care. Multivariable analysis showed that ECMO use and blood gas markers of severe pulmonary hypertension and poor gas exchange were strong predictors of mortality, in keeping with multiple previous models examining mortality risk in CDH [6,18–21]. Unsurprisingly then, the ECMO risk model, which incorporates arterial PCO2 levels, was found to perform well at our center. Other methods to stratify risk may be used to assess outcomes in CDH. For example, the CDHSG defect size categories are strongly correlated with mortality [8]; the inability to apply this risk assessment to patients in our series who were not repaired weakens the utility of this method. This study is limited by the small sample size, especially when examining risk-stratified groups. Differences between groups may not be statistically apparent due to the small number of patients seen in a single center. Furthermore, we defined an ECMO candidate as an infant with CDH who was born after at least 32 weeks' gestation with a birthweight
greater than 1.8 kg; our findings may not be applicable to those with more restrictive ECMO criteria. Another major concern is that morbidity and causes of mortality were not reported here. We do not know for instance whether complications directly attributable to ECMO were more or less important than excess pulmonary morbidity due to ventilator use that made attempted ECMO rescue necessary. A detailed analysis of complications and cause of death in our patients is underway. Finally, mortality is just one measure of outcome in CDH; measurement of longterm morbidity such as neurodevelopmental delay after ECMO and pulmonary sequelae would be essential for a compete assessment of care quality. Cost–benefit and patient quality-of-life are both important to study when examining CDH outcomes and were not analyzed in this study. For our center, the results of this analysis have reinforced that a standardized management pathway with firm criteria for transitions in care is essential, not only to improve outcomes but also to provide consistent management that can be more easily adjusted over time. We therefore developed a comprehensive CDH clinical practice guideline at our center that minimizes variability of practice. The guideline recommends ECMO for CDH patients who fail specific respiratory support measures within a lung protective strategy. In the first 30 months of implementation, total cohort CDH ECMO mortality has decreased from 69% to 60%. However, patient volume is too small to make strong conclusions. Ongoing analysis will determine if the improvement in survival will be sustained and if O/E ECMO use will approach 1:1. In summary, we have shown with this retrospective cohort study that application of an ECMO risk stratification model to provide adjusted assessment of CDH outcomes may help other individual centers determine relative overuse or underuse of ECMO and identify targets for institutional improvement in outcomes. Future studies should examine which center characteristics, as well as CDH and ECMO management strategies, are associated with the best outcomes, ideally through multicenter prospective trials. CRediT authorship contribution statement Tim Jancelewicz: Conceptualization, Data curation, Formal analysis, Writing – original draft, Writing – review & editing. Elizabeth A. Paton: Writing – original draft, Writing – review & editing. Jorie Jones: Formal analysis. Mark F. Weems: Formal analysis, Writing – original draft, Writing – review & editing. Pamela A. Lally: Data curation, Writing – original draft, Writing – review & editing. Max R. Langham Jr.: Conceptualization, Data curation, Formal analysis, Writing – original draft, Writing – review & editing. References [1] Harting MT, Hollinger L, Tsao K, et al. Aggressive surgical Management of Congenital Diaphragmatic Hernia: worth the effort?: a multicenter, prospective, Cohort Study. Ann Surg 2018;267:977–82. [2] Kays DW, Talbert JL, Islam S, et al. Improved survival in left liver-up congenital diaphragmatic hernia by early repair before extracorporeal membrane oxygenation: optimization of patient selection by multivariate risk modeling. J Am Coll Surg 2016;222:459–70. [3] Sola JE, Bronson SN, Cheung MC, et al. Survival disparities in newborns with congenital diaphragmatic hernia: a national perspective. J Pediatr Surg 2010;45:1336–42.
936
T. Jancelewicz et al. / Journal of Pediatric Surgery 54 (2019) 932–936
[4] Davis JS, Ryan ML, Perez EA, et al. ECMO hospital volume and survival in congenital diaphragmatic hernia repair. J Surg Res 2012;178:791–6. [5] Brindle ME, Cook EF, Tibboel D, et al. A clinical prediction rule for the severity of congenital diaphragmatic hernias in newborns. Pediatrics 2014;134:e413–9. [6] Schultz CM, DiGeronimo RJ, Yoder BA, et al. Congenital diaphragmatic hernia: a simplified postnatal predictor of outcome. J Pediatr Surg 2007;42:510–6. [7] Jancelewicz T, Brindle ME, Harting MT, et al. Extracorporeal membrane oxygenation (ECMO) risk stratification in newborns with congenital diaphragmatic hernia (CDH). J Pediatr Surg 2018;53:1890–5. [8] Lally KP, Lasky RE, Lally PA, et al. Standardized reporting for congenital diaphragmatic hernia–an international consensus. J Pediatr Surg 2013;48:2408–15. [9] Antonoff MB, Hustead VA, Groth SS, et al. Protocolized management of infants with congenital diaphragmatic hernia: effect on survival. J Pediatr Surg 2011;46:39–46. [10] Tracy ET, Mears SE, Smith PB, et al. Protocolized approach to the management of congenital diaphragmatic hernia: benefits of reducing variability in care. J Pediatr Surg 2010;45:1343–8. [11] Snoek KG, Capolupo I, van Rosmalen J, et al. Conventional mechanical ventilation versus high-frequency oscillatory ventilation for congenital diaphragmatic hernia: a randomized clinical trial (the VICI-trial). Ann Surg 2016;263:867–74. [12] Snoek KG, Capolupo I, Morini F, et al. Score for neonatal acute physiology-II predicts outcome in congenital diaphragmatic hernia patients. Pediatr Crit Care Med 2016; 17:540–6.
[13] Group TCDHS. Does extracorporeal membrane oxygenation improve survival in neonates with congenital diaphragmatic hernia? The Congenital Diaphragmatic Hernia Study Group. J Pediatr Surg 1999;34:720–4 [discussion 724–725]. [14] Kays DW. ECMO in CDH: is there a role? Semin Pediatr Surg 2017;26:166–70. [15] Turek JW, Nellis JR, Sherwood BG, et al. Shifting risks and conflicting outcomesECMO for neonates with congenital diaphragmatic hernia in the modern era. J Pediatr 2017;190:163–8 [e164]. [16] Aly H, Bianco-Batlles D, Mohamed MA, et al. Mortality in infants with congenital diaphragmatic hernia: a study of the United States National Database. J Perinatol 2010;30:553–7. [17] Hinton CF, Siffel C, Correa A, et al. Survival disparities associated with congenital diaphragmatic hernia. Birth Defects Res 2017;109:816–23. [18] Baird R, MacNab YC, Skarsgard ED, et al. Mortality prediction in congenital diaphragmatic hernia. J Pediatr Surg 2008;43:783–7. [19] Grizelj R, Bojanic K, Pritisanac E, et al. Survival prediction of high-risk outborn neonates with congenital diaphragmatic hernia from capillary blood gases. BMC Pediatr 2016;16(114). [20] Khmour AY, Konduri GG, Sato TT, et al. Role of admission gas exchange measurement in predicting congenital diaphragmatic hernia survival in the era of gentle ventilation. J Pediatr Surg 2014;49:1197–201. [21] Park HW, Lee BS, Lim G, et al. A simplified formula using early blood gas analysis can predict survival outcomes and the requirements for extracorporeal membrane oxygenation in congenital diaphragmatic hernia. J Korean Med Sci 2013;28:924–8.