- Email: [email protected]
Validation of the Prognosis for Prolonged Ventilation (ProVent) score in patients receiving 14 days of mechanical ventilation
Journal of Critical Care 44 (2018) 249–254
Contents lists available at ScienceDirect
Journal of Critical Care journal homepage: www.jccjournal.org
Validation of the Prognosis for Prolonged Ventilation (ProVent) score in patients receiving 14 days of mechanical ventilation Won-Young Kim, Eun-Jung Jo, Jung Seop Eom, Jeongha Mok, Mi-Hyun Kim, Ki Uk Kim, Hye-Kyung Park, Min Ki Lee, Kwangha Lee ⁎ Division of Pulmonary, Allergy and Critical Care Medicine, Department of Internal Medicine, Pusan National University School of Medicine, 179 Gudeok-ro, Seo-gu, Busan 49241, Republic of Korea
a r t i c l e
i n f o
Keywords: Mortality Prognosis Respiration, artificial Risk
a b s t r a c t Purpose: To evaluate the performance of the Prognosis for Prolonged Ventilation (ProVent) 14 score in patients requiring prolonged mechanical ventilation (PMV). Materials and methods: Data were obtained from 366 patients receiving at least 14 days of MV between January 2011 and December 2015 at a university-affiliated tertiary care hospital in Korea. ProVent 14 scores were assessed using the six standard variables. Model discrimination was assessed with the area under the receiver operating characteristic curve. Kaplan-Meier estimates were stratified according to the ProVent 14 score to predict 1-year survival. Results: The median age of the study group was 62 years (range, 50–72 years); 65% were male, and medical patients comprised 66% of the group. Overall mortality at 1 year was 43%. For ProVent 14 scores ranging from 0 to ≥4, 1-year mortality rates were 7%, 22%, 41%, 52%, and 75%, respectively (log-rank test, P b 0.001). The area under the receiver operating characteristic curve of the ProVent 14 score predicting 1-year mortality was 0.74 (95% confidence interval, 0.69–0.78). Conclusions: The ProVent 14 score accurately identified patients receiving PMV with a high 1-year mortality risk. Further validation in a larger sample is required. © 2017 Elsevier Inc. All rights reserved.
1. Introduction Prolonged mechanical ventilation (PMV) is defined as the requirement for invasive MV for at least 21 days [1]. The incidence of PMV has increased in recent years due to improvements in the acute management and supportive care of critically ill patients [2]. However, 1year mortality rates following PMV can be as high as 60% [3,4], and approximately 70% of survivors require life-long care in a hospital or postacute care facility; PMV is, therefore, associated with high healthcare costs both in the intensive care unit (ICU) and following discharge [5]. Several studies report difficulties in predicting long-term survival in patients receiving PMV and, therefore, challenges when communicating with families and surrogate decision makers [6,7]. Unfortunately, most intensivists have little opportunity to follow patients after they leave the ICU, and current severity of illness indexes do not perform well when used to predict mortality in patients with prolonged critical illness [8].
⁎ Corresponding author. E-mail addresses: [email protected] (W.-Y. Kim), [email protected] (E.-J. Jo), [email protected] (J.S. Eom), [email protected] (J. Mok), [email protected] (M.-H. Kim), [email protected] (K.U. Kim), [email protected] (H.-K. Park), [email protected] (M.K. Lee), [email protected] (K. Lee).
https://doi.org/10.1016/j.jcrc.2017.11.029 0883-9441/© 2017 Elsevier Inc. All rights reserved.
To reduce uncertainty surrounding the long-term outcome of patients receiving PMV, Carson et al. developed the Prognosis for Prolonged Ventilation (ProVent) scoring system, which estimates 1year mortality for patients receiving at least 21 days of MV [3,9]. The performance of the ProVent score has subsequently been validated in different patient populations [4,10]. The score is objective and can easily be measured by the clinician at the bedside and can, therefore, be widely used to predict mortality risk, inform decisions regarding allocation of limited resources, and facilitate communication with patients and families. However, important decisions regarding the care of PMV patients, such as performing tracheostomy or transferring to a long-term care facility, may be required prior to day 21 of MV [11,12]. In addition, a prolonged wait for information on a patient's prognosis can increase anxiety for the family. To address this, the group that developed the ProVent score also created and validated a mortality prediction model using data from day 14 of MV (ProVent 14) [13]. It is, however, currently not validated in non-US ICUs. The score may be calibrated to different populations and expanded to include additional parameters, as in the case of the early warning score [14], although limited data are available on the ProVent model. The aim of this study was therefore (i) to evaluate the performance of the ProVent 14 score in Korean patients requiring PMV; and (ii) to assess whether simplification, or inclusion of additional variables, improves the performance of the model.
250
W.-Y. Kim et al. / Journal of Critical Care 44 (2018) 249–254
Table 1 Baseline characteristics of the study patients.a Variable
Total (n = 366)
Survivors (n = 209)
Non-survivors (n = 157)
P
Age, years Male sex Body mass index, kg/m2 Charlson comorbidity index Intensive care unit Medical Surgical Trauma Major diagnoses leading to MV Cardiovascular Pulmonary including pneumonia Neurologic Infection other than pneumonia Postoperative state Severity of illness APACHE II score SOFA score Vasopressors (day 14) Hemodialysis (day 14) Delirium (day 14) PEEP, cm H2O (day 14) Laboratory data (day 14) Leukocytosis Hemoglobin b 7 g/dL Platelet count b 100 × 109/L Glucose ≥ 180 mg/dL PaO2/FiO2 b 200
62 (50–72) 239 (65) 22.9 (20.8–24.6) 1 (0–3)
58 (47–67) 137 (66) 23.3 (21.0–25.4) 1 (0–2)
66 (55–75) 102 (65) 22.2 (20.1–24.2) 2 (1–3)
b0.001 0.91 0.003 b0.001 0.62
243 (66) 123 (34) 149 (41)
141 (68) 68 (33) 109 (52)
102 (65) 55 (35) 40 (26)
b0.001
14 (4) 130 (36) 147 (40) 20 (6) 28 (8)
6 (3) 67 (32) 93 (45) 8 (4) 14 (7)
8 (5) 63 (40) 54 (34) 12 (8) 14 (9)
0.27 0.11 0.051 0.11 0.43
17 (14–22) 9 (6–11) 105 (29) 31 (9) 67 (18) 5 (5–8)
16 (13–20) 8 (6–11) 45 (22) 10 (5) 36 (17) 6 (5–8)
19 (15–23) 9 (7–11) 60 (38) 21 (13) 31 (20) 5 (5–8)
b0.001 0.03 b0.001 0.004 0.54 0.88
169 (46) 35 (10) 59 (16) 124 (34) 133 (36)
100 (48) 15 (7) 11 (5) 72 (34) 61 (29)
69 (44) 20 (13) 48 (31) 52 (33) 72 (46)
0.46 0.07 b0.001 0.79 0.001
MV = mechanical ventilation, APACHE = Acute Physiology and Chronic Health Evaluation, SOFA = Sequential Organ Failure Assessment, PEEP = positive end-expiratory pressure, PaO2 = arterial partial pressure of oxygen, FiO2 = fraction of inspired oxygen. a Data are presented as the median (interquartile range) or number (percentage).
2. Material and methods 2.1. Study design and patient selection The medical records of patients admitted to the ICU of a 1100-bed university-affiliated tertiary care hospital in Busan, Korea between January 2011 and December 2015 were reviewed. The hospital consists of six functionally separate ICUs with 85 beds: the medical ICU (12 beds),
the surgical ICU (10 beds), the cardiac/stroke unit (14 beds), the neurosurgical ICU (13 beds), the emergency department ICU (20 beds), and the trauma ICU (16 beds). Each ICU has full cardiovascular facilities, close airway monitoring, and at least one full-time intensivist. Adult patients receiving MV for ≥14 days after initial intubation were included in the analysis. Patients with chronic diseases requiring MV (either invasive or noninvasive) before ICU admission were not screened at the initial stage. The exclusion criteria were: age b 18 years old, irreversible
Fig. 1. Study flow diagram. MV = mechanical ventilation.
W.-Y. Kim et al. / Journal of Critical Care 44 (2018) 249–254 Table 2 Clinical outcomes of the study population.a Variable
Total (n = 366)
MV duration, d MV duration if died in ICU, d (n = 69) Tracheostomy during ICU stay ICU length of stay, d Hospital length of stay, d Mortality ICU In-hospital 1-year
24 (18–33) 28 (22–38) 243 (66) 37 (24–54) 52 (30–95) 66 (18) 70 (19) 157 (43)
MV = mechanical ventilation, d = days, ICU = intensive care unit. a Data are presented as the median (interquartile range) or number (percentage).
brain injury, acute or chronic neuromuscular disease, and a do not resuscitate order issued within 14 days of MV. These inclusion or exclusion criteria are similar to those outlined for the original model development [13]. The primary study outcome was 1-year mortality after day 14 of MV. The study protocol was approved by the Institutional Review Board of Pusan National University Hospital (C-1708-021-058), which waived the requirement for informed consent due to the retrospective nature of the study and the fact that patient records were anonymized and de-identified prior to analysis. 2.2. Data collection and definitions Data collected for all patients included age, sex, body mass index, comorbidities based on the Charlson Comorbidity Index [15], ICU type (medical/surgical), a diagnosis of trauma or non-trauma, and the major diagnosis leading to MV. The severity of illness within 24 h of ICU admission was assessed by the Acute Physiology and Chronic Health Evaluation (APACHE) II score [16] and the Sequential Organ Failure Assessment (SOFA) score [17]. To determine the ProVent 14 score, data on the patient's requirement for vasopressors, requirement for hemodialysis, and platelet count were collected on day 14 of MV. Additional variables included delirium, positive end-expiratory pressure, and laboratory data such as white blood cell count, hemoglobin level, serum glucose level, and PaO2/FiO2. These variables were selected to examine the effects of previously described prognostic indicators in
251
critically ill patients on the long-term mortality of PMV patients. Additional outcome variables assessed included duration of MV, tracheostomy during ICU stay, and other ICU and hospital outcomes (length of stay, transfer to general ward, transfer to another hospital, discharge to home, and mortality). For patients discharged from hospital, 1-year mortality after day 14 of MV was obtained from the National Health Insurance Service Database. A requirement for hemodialysis was defined as the use of any form of renal replacement therapy on, or within, 48 h of day 14 of MV. The diagnosis of delirium required psychiatric consultation and/or medications prescribed for symptom control, such as haloperidol, risperidone, and quetiapine. Leukocytosis was defined as a white blood cell count ≥ 11 × 109/L. The ProVent 14 score was calculated for each patient as described by Hough et al. [13], i.e., (i) age ≥ 65 years (+2 points); (ii) age 50–64 years (+1 point); (iii) platelet count b 100 × 109/L (+1 point); (iv) requirement for vasopressors (+1 point); (v) requirement for hemodialysis (+1 point); and (vi) non-trauma diagnosis (+1 point). 2.3. Statistical analysis Continuous variables are presented as the median and interquartile range; categorical variables are presented as percentages. Continuous variables were compared using a Mann-Whitney U test. Categorical variables were compared using a chi-square or Fisher's exact test. The number of patients included, and the 1-year mortality rates for ProVent 14 score values from 0 to ≥ 4, were compared between this study and that of Hough et al. [13]. To find out the best subset of independent variables in our population, stepwise logistic regression analysis with 1year mortality as the outcome variable was performed using the six risk variables of the ProVent 14 score. Model discrimination was assessed with the area under the receiver operating characteristic curve (AUC), and model calibration was assessed with the HosmerLemeshow test. The β coefficient values derived from multiple logistic regressions were simplified as natural numbers N 0, and the Korean ProVent 14 score was calculated as the sum of these simplified values. Kaplan-Meier estimates were stratified according to the ProVent 14 score and Korean ProVent 14 score to predict 1-year survival, and curves were compared using the log-rank test. The AUC for the ProVent 14 score and Korean ProVent 14 score predicting 1-year mortality were compared using DeLong's test, as described previously [18]. All
Fig. 2. Disposition of the study patients included in the analysis of 1-year mortality. ICU = intensive care unit, GW = general ward, MV = mechanical ventilation.
252
W.-Y. Kim et al. / Journal of Critical Care 44 (2018) 249–254
Table 3 ProVent 14 score and 1-year observed mortality in Hough et al. [13] study and the current study. ProVent 14 score
Hough et al.
Current study
n
Observed mortality, % (95% CI)
n
Observed mortality, % (95% CI)
0 1 2 3 4–6
70 99 142 117 63
4 (0–9) 28 (19–37) 43 (35–51) 61 (52–70) 92 (84–100)
42 63 88 109 63
7 (2–20) 22 (14–34) 41 (31–51) 52 (43–61) 75 (63–84)
P
0.52 0.46 0.79 0.23 0.02
CI = confidence interval.
prespecified covariables (listed in Table 1) were included in a second logistic regression analysis using stepwise backward selection with P values of b 0.10. The optimal cut-off values for body mass index and comorbidity index were selected using locally weighted scatterplot smoothing curves [19]. Cut-off values for hemoglobin level, serum glucose level, and PaO2/FiO2 were set at 7 g/dL, 180 mg/dL, and 200, respectively, based on values used in previous studies [20-22]. A modified ProVent 14 score was developed based on the β coefficient values in that model, and the performance of the score was assessed by the AUC and the Kaplan-Meier estimates. All tests of significance were twotailed, and P values b 0.05 were considered significant. All analyses were performed using SPSS version 18.0 for Windows (SPSS Inc., Chicago, IL, USA) and MedCalc Statistical Software version 16.8.4 (MedCalc Software bvba, Ostend, Belgium). 3. Results A total of 761 patients were screened during the study period; 395 were excluded from the analyses for the reasons shown in Fig. 1. Of the remaining 366 patients, 209 (57%) survived for ≥ 1 year after day 14 of MV; the remaining 157 (43%) patients died during the year following PMV. Table 1 shows the baseline characteristics of the study population. The median age was 62 (range, 50–72) years, and 65% were male. Medical patients comprised 66% of the group. The main diagnoses leading to MV were pulmonary and neurologic diseases. Non-survivors were older, had a lower body mass index and higher comorbidity index, and were less likely to be a trauma admission. Non-survivors also had significantly higher APACHE II score and SOFA score, and a higher proportion of patients required vasopressors and hemodialysis on day 14 of MV. Regarding laboratory data, the percentage of patients with a platelet count b 100 × 109/L and PaO2/FiO2 b 200 was significantly higher for non-survivors. Clinical outcomes of the study patients are shown in Table 2. Seventy patients (19%) died in the hospital; 49 patients (13%) were discharged to home. Fig. 2 shows the disposition of study patients according to clinical outcome. Table 3 summarizes the 1-year mortality data according to the ProVent 14 score in our study and that of the original study by Hough et al. [13]. No statistically significant differences in 1-year mortality rates for score values of 0 to 3 were seen between the two studies. However, mortality associated with a score of ≥4 was significantly lower in
our study (75%) than in that reported by Hough et al. (92%, P = 0.02). For the patients examined herein, the sensitivity for identifying ≥ 52% risk of 1-year mortality was 30% and specificity was 92%. Sensitivity for identifying ≥75% risk of mortality was 12% and specificity was 99%. For patients in the highest risk group in the present study (ProVent 14 score ≥ 4), in-hospital mortality was 54%, and only four patients were discharged to home. The results of logistic regression analysis with six variables of the ProVent 14 score applied to the study patients are shown in Table 4. Multivariate analysis indicated that only four variables (age ≥ 65 years, platelet count b 100 × 109/L, requirement for vasopressors, and nontrauma diagnosis) were significantly associated with 1-year mortality. According to the β coefficient values observed in this model, a Korean ProVent 14 score was proposed as follows: (i) age ≥ 65 years (+ 1 point); (ii) platelet count b 100 × 109/L (+2 points); (iii) requirement for vasopressors (+ 1 point); and (iv) non-trauma diagnosis (+ 1 point). This model had acceptable discrimination (AUC = 0.76) and calibration (Hosmer and Lemeshow chi-square = 4.23; P = 0.75). The Kaplan-Meier survival curves, stratified according to the ProVent 14 score and the Korean ProVent 14 score, are shown in Fig. 3. For ProVent 14 scores ranging from 0 to ≥4, 1-year mortality rates were 7%, 22%, 41%, 52%, and 75%, respectively (log-rank test, P b 0.001). Meanwhile, 1-year mortality rates for Korean ProVent 14 scores ranging from 0 to ≥3 were 11%, 36%, 49%, and 77%, respectively (log-rank test, P b 0.001). The AUCs for the ProVent 14 score and the Korean ProVent 14 score were 0.74 (95% confidence interval (CI), 0.69–0.78) and 0.75 (95% CI, 0.70–0.80), respectively, showing no statistically significant difference (P = 0.13, Fig. 4). Supplementary table shows the results of logistic regression analysis with risk variables. Inspection of locally weighted scatterplot smoothing curves identified the following cut-off values: body mass index b 23 kg/m2 and comorbidity index ≥ 2 (Supplementary Fig. 1). Multivariate analysis indicated that PaO2/FiO2 b 200 tended to be associated with mortality. According to the β values observed in this model, a modified ProVent 14 score was developed as follows: (i) age ≥ 65 years (+ 1 point); (ii) platelet count b 100 × 109/L (+2 points); (iii) requirement for vasopressors (+ 1 point); (iv) non-trauma diagnosis (+ 1 point); and (v) PaO2/FiO2 b 200 (+1 point). The Kaplan-Meier survival curves of 1-year survival stratified according to the values of the modified ProVent 14 score are shown in Supplementary Fig. 2. The AUC using the score to predict 1-year mortality was 0.75 (95% CI, 0.70–0.80). 4. Discussion The present study indicates that the ProVent 14 score accurately predicts the risk of 1-year mortality in patients requiring at least 14 days of MV in a non-US population. The Korean ProVent 14 score and modified ProVent 14 score did not perform better than the original model. To the best of our knowledge, this is the first comprehensive study to validate the ProVent 14 score in Korean PMV patients. The original ProVent 14 score was developed in patients requiring PMV in 2005 at five tertiary care hospitals across the US using a limited number of clinical variables that can be easily collected at the bedside or from electronic medical records [13]. Data derived from this scoring
Table 4 Logistic regression analysis with six risk variables of the ProVent 14 model. Variable
Unadjusted OR (95% CI)
P
Adjusted ORa (95% CI)
P
β value
Age ≥ 65 years Age 50–64 years Non-trauma Vasopressors Hemodialysis Platelet count b 100 × 109/L
3.73 (2.09–6.67) 1.83 (1.001–3.34) 3.19 (2.03–5.00) 2.25 (1.42–3.57) 3.06 (1.40–6.70) 7.93 (3.95–15.89)
b0.001 0.05 b0.001 0.001 0.005 b0.001
2.70 (1.41–5.16) 1.53 (0.79–2.98) 2.51 (1.53–4.13) 1.74 (1.04–2.91)
0.003 0.21 b0.001 0.04
0.99 0.43 0.92 0.55
6.71 (3.24–13.91)
b0.001
1.90
OR = odds ratio, CI = confidence interval. a All six variables were included in the multivariate analysis using stepwise backward selection procedures.
W.-Y. Kim et al. / Journal of Critical Care 44 (2018) 249–254
system, when combined with physician estimates of outcome, can provide relevant decision-making information. However, it is still unclear how this score performs in other populations, such as those in Asian ICUs, as these units are known to differ from those in Western countries in many aspects related to medical expenses, family support, the concept of death, and social customs [23]. The 43% 1-year mortality seen in the present study (where medical ICU admission comprised 66% of the group) is comparable to the 32%–51% mortality rates reported in previous studies of PMV in similar patient populations [9,13,23]. However, a 1-year mortality as high as 60%–67% was demonstrated in other two studies [4,10], which is likely due to the fact that N 70% of those populations were medical patients with pulmonary and cardiovascular diseases. Here, the 1-year mortality rate associated with a ProVent 14 score ≥ 4 (75%) was relatively low when compared with that seen in previous studies. This result is consistent with the findings of Udeh et al. [24], who showed a lower observed mortality associated with a ProVent score ≥ 4 in the 14 to 20 days PMV group (73%) than in the 21 + days PMV group (89%). We suggest that this observation results from a lower level of illness severity in patients requiring MV for 14 days when compared with those requiring MV for up to or beyond 21 days. Nevertheless, the ProVent 14 score performed well for discrimination and calibration in our patient group. In addition, the score showed very high specificity in patients at greatest risk of death. Limiting the possibility of misclassifying a patient as very high risk is particularly important as such a classification may result in the decision to withhold or withdraw life support. Despite differences in study populations, mortality, and study time frame, the similarity in the values of the AUC curves between other studies and our own emphasizes the robust nature and general application of the ProVent model. In this study, we sought to select the best subset of ProVent 14 variables in Korean PMV patients. Carson et al. [9] and Hough et al. [13] assigned one point to age 50–64 years. However, we found that age 50–64 years was not significantly associated with 1-year mortality (as did a study by Leroy et al. [4]); therefore, this was not included as a risk variable in simplified models (the French ProVent score [4] and our Korean ProVent 14 score). This discrepancy may be explained by differences in the characteristics of the study populations. Patients enrolled in the French group and our own were much older (N60 years of age), so the independent prognostic value of age 50–64 years was less apparent. Additionally, we sought to determine whether inclusion of other potential predictors of mortality would improve the model performance. The ProVent 14 score added non-trauma diagnosis as an additional variable to the original model, recognizing that non-trauma patients on PMV have worse outcomes than trauma patients. In our multivariate analysis, PaO2/FiO2 b 200 tended to be associated with mortality and was included in the modified ProVent 14 score. Simplification or addition of extra variables should be weighed carefully due to potential drawbacks such as lowering sensitivity or increasing complexity of the model. In this study, neither the Korean ProVent 14 score nor the modified ProVent 14 score derived additional benefit
253
Fig. 4. Receiver operating characteristic curves for the ProVent 14 score and Korean ProVent 14 score predicting 1-year mortality. AUC = area under the receiver operating characteristic curve.
than the original ProVent 14. Again, our findings are consistent with robustness of the original ProVent 14 model. Our study adds to the accumulating evidence of the generalizability of the ProVent 14 score. Previous studies showed that the ProVent model had good discrimination and accuracy in patients with primarily surgical diagnoses [24], in non-tertiary community hospitals [4], and even when it was used to predict short-term prognosis such as ICU mortality [25]. More studies are needed to adopt the ProVent model across broader populations such as hemato-oncologic patients requiring PMV. Moreover, the impact of the ProVent score on patient care, resource allocation, and ability to facilitate decisions with patient/family preferences should be investigated. This is a major limitation of previous ProVent studies as well as our own and is, therefore, an aspect that requires further evaluation. A recent study of a web-based decision aid for surrogate decision makers of patients receiving PMV is a good example of implementing the ProVent model for decision making process in the setting of critical illness [26]. The present study has several limitations. First, the study was singlecentered and retrospective. The ProVent 14 score was validated in patients requiring PMV at a tertiary care hospital and so it may not be possible to generalize these data to other settings, although the majority of PMV patients are managed in tertiary hospitals. Moreover, the retrospective study design could have introduced selection and ascertainment bias. Second, measuring specificity at a high mortality risk
Fig. 3. Kaplan-Meier survival curves of patients stratified according to the ProVent 14 score (A) and the Korean ProVent 14 score (B). MV = mechanical ventilation.
254
W.-Y. Kim et al. / Journal of Critical Care 44 (2018) 249–254
comes at the expense of lower sensitivity, as a substantial proportion of patients who died did not have a high ProVent 14 score. Third, the ProVent 14 variables (with the exception of age and non-trauma) are dynamic markers of illness severity and may change during the course of patient's illness, i.e., the scores assessed at day 14 may decline by day 21. Thus, knowing the outcomes of those of improved scores and worsening or unchanged scores would provide guidance to clinicians regarding decision making process and communication with families. The utility of the evolution of the ProVent score over time is interesting and should be confirmed in future studies. 5. Conclusions In conclusion, the ProVent 14 score can allow earlier identification of patients receiving PMV in Korea who are at high risk of 1-year mortality. This validation study indicated that the ProVent 14 model has ethnic and temporal generalizability. Our data indicate that a simplified approach or inclusion of additional variables did not improve the score's performance. Further validation of the ProVent 14 score in a larger sample is required. Conflicts of interest None. Funding This work was supported by a clinical research grant from Pusan National University Hospital in 2017. The funders had no role in study design, data collection and analysis, preparation of the manuscript, or decision to publish. Appendix A. Supplementary data Supplementary data to this article can be found online at https://doi. org/10.1016/j.jcrc.2017.11.029. References [1] MacIntyre NR, Epstein SK, Carson S, Scheinhorn D, Christopher K, Muldoon S, et al. Management of patients requiring prolonged mechanical ventilation: report of a NAMDRC consensus conference. Chest 2005;128:3937–54. [2] Zilberberg MD, de Wit M, Pirone JR, Shorr AF. Growth in adult prolonged acute mechanical ventilation: implications for healthcare delivery. Crit Care Med 2008;36: 1451–5. [3] Carson SS, Garrett J, Hanson LC, Lanier J, Govert J, Brake MC, et al. A prognostic model for one-year mortality in patients requiring prolonged mechanical ventilation. Crit Care Med 2008;36:2061–9. [4] Leroy G, Devos P, Lambiotte F, Thevenin D, Leroy O. One-year mortality in patients requiring prolonged mechanical ventilation: multicenter evaluation of the ProVent score. Crit Care 2014;18:R155.
[5] Wagner DP. Economics of prolonged mechanical ventilation. Am Rev Respir Dis 1989;140(2 Pt 2):S14–8. [6] White DB, Engelberg RA, Wenrich MD, Lo B, Curtis JR. Prognostication during physician-family discussions about limiting life support in intensive care units. Crit Care Med 2007;35:442–8. [7] Nelson JE, Mercado AF, Camhi SL, Tandon N, Wallenstein S, August GI, et al. Communication about chronic critical illness. Arch Intern Med 2007;167:2509–15. [8] Carson SS, Bach PB. Predicting mortality in patients suffering from prolonged critical illness: an assessment of four severity-of-illness measures. Chest 2001;120:928–33. [9] Carson SS, Kahn JM, Hough CL, Seeley EJ, White DB, Douglas IS, et al. A multicenter mortality prediction model for patients receiving prolonged mechanical ventilation. Crit Care Med 2012;40:1171–6. [10] Mok JH, Kim YH, Jeong ES, Eom JS, Kim MH, Kim KU, et al. Clinical application of the ProVent score in Korean patients requiring prolonged mechanical ventilation: a 10year experience in a university-affiliated tertiary hospital. J Crit Care 2016;33: 158–62. [11] Freeman BD, Morris PE. Tracheostomy practice in adults with acute respiratory failure. Crit Care Med 2012;40:2890–6. [12] Kahn JM, Benson NM, Appleby D, Carson SS, Iwashyna TJ. Long-term acute care hospital utilization after critical illness. JAMA 2010;303:2253–9. [13] Hough CL, Caldwell ES, Cox CE, Douglas IS, Kahn JM, White DB, et al. Development and validation of a mortality prediction model for patients receiving 14 days of mechanical ventilation. Crit Care Med 2015;43:2339–45. [14] Morgan RJ, Williams F, Wright MM. An early warning scoring system for detecting developing critical illness. Clin Intensive Care 1997;8:100. [15] Charlson ME, Pompei P, Ales KL, MacKenzie CR. A new method of classifying prognostic comorbidity in longitudinal studies: development and validation. J Chronic Dis 1987;40:373–83. [16] Knaus WA, Draper EA, Wagner DP, Zimmerman JE. APACHE II: a severity of disease classification system. Crit Care Med 1985;13:818–29. [17] Vincent JL, Moreno R, Takala J, Willatts S, De Mendonca A, Bruining H, et al. The SOFA (Sepsis-related Organ Failure Assessment) score to describe organ dysfunction/failure. On behalf of the Working Group on Sepsis-Related Problems of the European Society of Intensive Care Medicine. Intensive Care Med 1996;22:707–10. [18] DeLong ER, DeLong DM, Clarke-Pearson DL. Comparing the areas under two or more correlated receiver operating characteristic curves: a nonparametric approach. Biometrics 1988;44:837–45. [19] Cleveland WS, Grosse E, Shyu WM. Local regression models. In: Chambers JM, Hastie TJ, editors. Statistical models in S. New York: Cole/Wadsworth & Brooks; 1992. [20] Holst LB, Haase N, Wetterslev J, Wernerman J, Guttormsen AB, Karlsson S, et al. Lower versus higher hemoglobin threshold for transfusion in septic shock. N Engl J Med 2014;371:1381–91. [21] NICE-SUGAR Study Investigators, Finfer S, Chittock DR, Su SY, Blair D, Foster D, et al. Intensive versus conventional glucose control in critically ill patients. N Engl J Med 2009;360:1283–97. [22] ARDS Definition Task Force, Ranieri VM, Rubenfeld GD, Thompson BT, Ferguson ND, Caldwell E, et al. Acute respiratory distress syndrome: the Berlin Definition. JAMA 2012;307:2526–33. [23] Phua J, Joynt GM, Nishimura M, Deng Y, Myatra SN, Chan YH, et al. Withholding and withdrawal of life-sustaining treatments in intensive care units in Asia. JAMA Intern Med 2015;175:363–71. [24] Udeh CI, Hadder B, Udeh BL. Validation and extension of the prolonged mechanical ventilation prognostic model (ProVent) score for predicting 1-year mortality after prolonged mechanical ventilation. Ann Am Thorac Soc 2015;12:1845–51. [25] Boniatti MM, Giustina AD, Marin LG, França J, Dos Santos MC, Vidart J, et al. Mortality in chronically critically ill patients: expanding the use of the ProVent score. J Crit Care 2015;30:1039–42. [26] Cox CE, Wysham NG, Walton B, Jones D, Cass B, Tobin M, et al. Development and usability testing of a Web-based decision aid for families of patients receiving prolonged mechanical ventilation. Ann Intensive Care 2015;5:6.
Contents lists available at ScienceDirect
Journal of Critical Care journal homepage: www.jccjournal.org
Validation of the Prognosis for Prolonged Ventilation (ProVent) score in patients receiving 14 days of mechanical ventilation Won-Young Kim, Eun-Jung Jo, Jung Seop Eom, Jeongha Mok, Mi-Hyun Kim, Ki Uk Kim, Hye-Kyung Park, Min Ki Lee, Kwangha Lee ⁎ Division of Pulmonary, Allergy and Critical Care Medicine, Department of Internal Medicine, Pusan National University School of Medicine, 179 Gudeok-ro, Seo-gu, Busan 49241, Republic of Korea
a r t i c l e
i n f o
Keywords: Mortality Prognosis Respiration, artificial Risk
a b s t r a c t Purpose: To evaluate the performance of the Prognosis for Prolonged Ventilation (ProVent) 14 score in patients requiring prolonged mechanical ventilation (PMV). Materials and methods: Data were obtained from 366 patients receiving at least 14 days of MV between January 2011 and December 2015 at a university-affiliated tertiary care hospital in Korea. ProVent 14 scores were assessed using the six standard variables. Model discrimination was assessed with the area under the receiver operating characteristic curve. Kaplan-Meier estimates were stratified according to the ProVent 14 score to predict 1-year survival. Results: The median age of the study group was 62 years (range, 50–72 years); 65% were male, and medical patients comprised 66% of the group. Overall mortality at 1 year was 43%. For ProVent 14 scores ranging from 0 to ≥4, 1-year mortality rates were 7%, 22%, 41%, 52%, and 75%, respectively (log-rank test, P b 0.001). The area under the receiver operating characteristic curve of the ProVent 14 score predicting 1-year mortality was 0.74 (95% confidence interval, 0.69–0.78). Conclusions: The ProVent 14 score accurately identified patients receiving PMV with a high 1-year mortality risk. Further validation in a larger sample is required. © 2017 Elsevier Inc. All rights reserved.
1. Introduction Prolonged mechanical ventilation (PMV) is defined as the requirement for invasive MV for at least 21 days [1]. The incidence of PMV has increased in recent years due to improvements in the acute management and supportive care of critically ill patients [2]. However, 1year mortality rates following PMV can be as high as 60% [3,4], and approximately 70% of survivors require life-long care in a hospital or postacute care facility; PMV is, therefore, associated with high healthcare costs both in the intensive care unit (ICU) and following discharge [5]. Several studies report difficulties in predicting long-term survival in patients receiving PMV and, therefore, challenges when communicating with families and surrogate decision makers [6,7]. Unfortunately, most intensivists have little opportunity to follow patients after they leave the ICU, and current severity of illness indexes do not perform well when used to predict mortality in patients with prolonged critical illness [8].
⁎ Corresponding author. E-mail addresses: [email protected] (W.-Y. Kim), [email protected] (E.-J. Jo), [email protected] (J.S. Eom), [email protected] (J. Mok), [email protected] (M.-H. Kim), [email protected] (K.U. Kim), [email protected] (H.-K. Park), [email protected] (M.K. Lee), [email protected] (K. Lee).
https://doi.org/10.1016/j.jcrc.2017.11.029 0883-9441/© 2017 Elsevier Inc. All rights reserved.
To reduce uncertainty surrounding the long-term outcome of patients receiving PMV, Carson et al. developed the Prognosis for Prolonged Ventilation (ProVent) scoring system, which estimates 1year mortality for patients receiving at least 21 days of MV [3,9]. The performance of the ProVent score has subsequently been validated in different patient populations [4,10]. The score is objective and can easily be measured by the clinician at the bedside and can, therefore, be widely used to predict mortality risk, inform decisions regarding allocation of limited resources, and facilitate communication with patients and families. However, important decisions regarding the care of PMV patients, such as performing tracheostomy or transferring to a long-term care facility, may be required prior to day 21 of MV [11,12]. In addition, a prolonged wait for information on a patient's prognosis can increase anxiety for the family. To address this, the group that developed the ProVent score also created and validated a mortality prediction model using data from day 14 of MV (ProVent 14) [13]. It is, however, currently not validated in non-US ICUs. The score may be calibrated to different populations and expanded to include additional parameters, as in the case of the early warning score [14], although limited data are available on the ProVent model. The aim of this study was therefore (i) to evaluate the performance of the ProVent 14 score in Korean patients requiring PMV; and (ii) to assess whether simplification, or inclusion of additional variables, improves the performance of the model.
250
W.-Y. Kim et al. / Journal of Critical Care 44 (2018) 249–254
Table 1 Baseline characteristics of the study patients.a Variable
Total (n = 366)
Survivors (n = 209)
Non-survivors (n = 157)
P
Age, years Male sex Body mass index, kg/m2 Charlson comorbidity index Intensive care unit Medical Surgical Trauma Major diagnoses leading to MV Cardiovascular Pulmonary including pneumonia Neurologic Infection other than pneumonia Postoperative state Severity of illness APACHE II score SOFA score Vasopressors (day 14) Hemodialysis (day 14) Delirium (day 14) PEEP, cm H2O (day 14) Laboratory data (day 14) Leukocytosis Hemoglobin b 7 g/dL Platelet count b 100 × 109/L Glucose ≥ 180 mg/dL PaO2/FiO2 b 200
62 (50–72) 239 (65) 22.9 (20.8–24.6) 1 (0–3)
58 (47–67) 137 (66) 23.3 (21.0–25.4) 1 (0–2)
66 (55–75) 102 (65) 22.2 (20.1–24.2) 2 (1–3)
b0.001 0.91 0.003 b0.001 0.62
243 (66) 123 (34) 149 (41)
141 (68) 68 (33) 109 (52)
102 (65) 55 (35) 40 (26)
b0.001
14 (4) 130 (36) 147 (40) 20 (6) 28 (8)
6 (3) 67 (32) 93 (45) 8 (4) 14 (7)
8 (5) 63 (40) 54 (34) 12 (8) 14 (9)
0.27 0.11 0.051 0.11 0.43
17 (14–22) 9 (6–11) 105 (29) 31 (9) 67 (18) 5 (5–8)
16 (13–20) 8 (6–11) 45 (22) 10 (5) 36 (17) 6 (5–8)
19 (15–23) 9 (7–11) 60 (38) 21 (13) 31 (20) 5 (5–8)
b0.001 0.03 b0.001 0.004 0.54 0.88
169 (46) 35 (10) 59 (16) 124 (34) 133 (36)
100 (48) 15 (7) 11 (5) 72 (34) 61 (29)
69 (44) 20 (13) 48 (31) 52 (33) 72 (46)
0.46 0.07 b0.001 0.79 0.001
MV = mechanical ventilation, APACHE = Acute Physiology and Chronic Health Evaluation, SOFA = Sequential Organ Failure Assessment, PEEP = positive end-expiratory pressure, PaO2 = arterial partial pressure of oxygen, FiO2 = fraction of inspired oxygen. a Data are presented as the median (interquartile range) or number (percentage).
2. Material and methods 2.1. Study design and patient selection The medical records of patients admitted to the ICU of a 1100-bed university-affiliated tertiary care hospital in Busan, Korea between January 2011 and December 2015 were reviewed. The hospital consists of six functionally separate ICUs with 85 beds: the medical ICU (12 beds),
the surgical ICU (10 beds), the cardiac/stroke unit (14 beds), the neurosurgical ICU (13 beds), the emergency department ICU (20 beds), and the trauma ICU (16 beds). Each ICU has full cardiovascular facilities, close airway monitoring, and at least one full-time intensivist. Adult patients receiving MV for ≥14 days after initial intubation were included in the analysis. Patients with chronic diseases requiring MV (either invasive or noninvasive) before ICU admission were not screened at the initial stage. The exclusion criteria were: age b 18 years old, irreversible
Fig. 1. Study flow diagram. MV = mechanical ventilation.
W.-Y. Kim et al. / Journal of Critical Care 44 (2018) 249–254 Table 2 Clinical outcomes of the study population.a Variable
Total (n = 366)
MV duration, d MV duration if died in ICU, d (n = 69) Tracheostomy during ICU stay ICU length of stay, d Hospital length of stay, d Mortality ICU In-hospital 1-year
24 (18–33) 28 (22–38) 243 (66) 37 (24–54) 52 (30–95) 66 (18) 70 (19) 157 (43)
MV = mechanical ventilation, d = days, ICU = intensive care unit. a Data are presented as the median (interquartile range) or number (percentage).
brain injury, acute or chronic neuromuscular disease, and a do not resuscitate order issued within 14 days of MV. These inclusion or exclusion criteria are similar to those outlined for the original model development [13]. The primary study outcome was 1-year mortality after day 14 of MV. The study protocol was approved by the Institutional Review Board of Pusan National University Hospital (C-1708-021-058), which waived the requirement for informed consent due to the retrospective nature of the study and the fact that patient records were anonymized and de-identified prior to analysis. 2.2. Data collection and definitions Data collected for all patients included age, sex, body mass index, comorbidities based on the Charlson Comorbidity Index [15], ICU type (medical/surgical), a diagnosis of trauma or non-trauma, and the major diagnosis leading to MV. The severity of illness within 24 h of ICU admission was assessed by the Acute Physiology and Chronic Health Evaluation (APACHE) II score [16] and the Sequential Organ Failure Assessment (SOFA) score [17]. To determine the ProVent 14 score, data on the patient's requirement for vasopressors, requirement for hemodialysis, and platelet count were collected on day 14 of MV. Additional variables included delirium, positive end-expiratory pressure, and laboratory data such as white blood cell count, hemoglobin level, serum glucose level, and PaO2/FiO2. These variables were selected to examine the effects of previously described prognostic indicators in
251
critically ill patients on the long-term mortality of PMV patients. Additional outcome variables assessed included duration of MV, tracheostomy during ICU stay, and other ICU and hospital outcomes (length of stay, transfer to general ward, transfer to another hospital, discharge to home, and mortality). For patients discharged from hospital, 1-year mortality after day 14 of MV was obtained from the National Health Insurance Service Database. A requirement for hemodialysis was defined as the use of any form of renal replacement therapy on, or within, 48 h of day 14 of MV. The diagnosis of delirium required psychiatric consultation and/or medications prescribed for symptom control, such as haloperidol, risperidone, and quetiapine. Leukocytosis was defined as a white blood cell count ≥ 11 × 109/L. The ProVent 14 score was calculated for each patient as described by Hough et al. [13], i.e., (i) age ≥ 65 years (+2 points); (ii) age 50–64 years (+1 point); (iii) platelet count b 100 × 109/L (+1 point); (iv) requirement for vasopressors (+1 point); (v) requirement for hemodialysis (+1 point); and (vi) non-trauma diagnosis (+1 point). 2.3. Statistical analysis Continuous variables are presented as the median and interquartile range; categorical variables are presented as percentages. Continuous variables were compared using a Mann-Whitney U test. Categorical variables were compared using a chi-square or Fisher's exact test. The number of patients included, and the 1-year mortality rates for ProVent 14 score values from 0 to ≥ 4, were compared between this study and that of Hough et al. [13]. To find out the best subset of independent variables in our population, stepwise logistic regression analysis with 1year mortality as the outcome variable was performed using the six risk variables of the ProVent 14 score. Model discrimination was assessed with the area under the receiver operating characteristic curve (AUC), and model calibration was assessed with the HosmerLemeshow test. The β coefficient values derived from multiple logistic regressions were simplified as natural numbers N 0, and the Korean ProVent 14 score was calculated as the sum of these simplified values. Kaplan-Meier estimates were stratified according to the ProVent 14 score and Korean ProVent 14 score to predict 1-year survival, and curves were compared using the log-rank test. The AUC for the ProVent 14 score and Korean ProVent 14 score predicting 1-year mortality were compared using DeLong's test, as described previously [18]. All
Fig. 2. Disposition of the study patients included in the analysis of 1-year mortality. ICU = intensive care unit, GW = general ward, MV = mechanical ventilation.
252
W.-Y. Kim et al. / Journal of Critical Care 44 (2018) 249–254
Table 3 ProVent 14 score and 1-year observed mortality in Hough et al. [13] study and the current study. ProVent 14 score
Hough et al.
Current study
n
Observed mortality, % (95% CI)
n
Observed mortality, % (95% CI)
0 1 2 3 4–6
70 99 142 117 63
4 (0–9) 28 (19–37) 43 (35–51) 61 (52–70) 92 (84–100)
42 63 88 109 63
7 (2–20) 22 (14–34) 41 (31–51) 52 (43–61) 75 (63–84)
P
0.52 0.46 0.79 0.23 0.02
CI = confidence interval.
prespecified covariables (listed in Table 1) were included in a second logistic regression analysis using stepwise backward selection with P values of b 0.10. The optimal cut-off values for body mass index and comorbidity index were selected using locally weighted scatterplot smoothing curves [19]. Cut-off values for hemoglobin level, serum glucose level, and PaO2/FiO2 were set at 7 g/dL, 180 mg/dL, and 200, respectively, based on values used in previous studies [20-22]. A modified ProVent 14 score was developed based on the β coefficient values in that model, and the performance of the score was assessed by the AUC and the Kaplan-Meier estimates. All tests of significance were twotailed, and P values b 0.05 were considered significant. All analyses were performed using SPSS version 18.0 for Windows (SPSS Inc., Chicago, IL, USA) and MedCalc Statistical Software version 16.8.4 (MedCalc Software bvba, Ostend, Belgium). 3. Results A total of 761 patients were screened during the study period; 395 were excluded from the analyses for the reasons shown in Fig. 1. Of the remaining 366 patients, 209 (57%) survived for ≥ 1 year after day 14 of MV; the remaining 157 (43%) patients died during the year following PMV. Table 1 shows the baseline characteristics of the study population. The median age was 62 (range, 50–72) years, and 65% were male. Medical patients comprised 66% of the group. The main diagnoses leading to MV were pulmonary and neurologic diseases. Non-survivors were older, had a lower body mass index and higher comorbidity index, and were less likely to be a trauma admission. Non-survivors also had significantly higher APACHE II score and SOFA score, and a higher proportion of patients required vasopressors and hemodialysis on day 14 of MV. Regarding laboratory data, the percentage of patients with a platelet count b 100 × 109/L and PaO2/FiO2 b 200 was significantly higher for non-survivors. Clinical outcomes of the study patients are shown in Table 2. Seventy patients (19%) died in the hospital; 49 patients (13%) were discharged to home. Fig. 2 shows the disposition of study patients according to clinical outcome. Table 3 summarizes the 1-year mortality data according to the ProVent 14 score in our study and that of the original study by Hough et al. [13]. No statistically significant differences in 1-year mortality rates for score values of 0 to 3 were seen between the two studies. However, mortality associated with a score of ≥4 was significantly lower in
our study (75%) than in that reported by Hough et al. (92%, P = 0.02). For the patients examined herein, the sensitivity for identifying ≥ 52% risk of 1-year mortality was 30% and specificity was 92%. Sensitivity for identifying ≥75% risk of mortality was 12% and specificity was 99%. For patients in the highest risk group in the present study (ProVent 14 score ≥ 4), in-hospital mortality was 54%, and only four patients were discharged to home. The results of logistic regression analysis with six variables of the ProVent 14 score applied to the study patients are shown in Table 4. Multivariate analysis indicated that only four variables (age ≥ 65 years, platelet count b 100 × 109/L, requirement for vasopressors, and nontrauma diagnosis) were significantly associated with 1-year mortality. According to the β coefficient values observed in this model, a Korean ProVent 14 score was proposed as follows: (i) age ≥ 65 years (+ 1 point); (ii) platelet count b 100 × 109/L (+2 points); (iii) requirement for vasopressors (+ 1 point); and (iv) non-trauma diagnosis (+ 1 point). This model had acceptable discrimination (AUC = 0.76) and calibration (Hosmer and Lemeshow chi-square = 4.23; P = 0.75). The Kaplan-Meier survival curves, stratified according to the ProVent 14 score and the Korean ProVent 14 score, are shown in Fig. 3. For ProVent 14 scores ranging from 0 to ≥4, 1-year mortality rates were 7%, 22%, 41%, 52%, and 75%, respectively (log-rank test, P b 0.001). Meanwhile, 1-year mortality rates for Korean ProVent 14 scores ranging from 0 to ≥3 were 11%, 36%, 49%, and 77%, respectively (log-rank test, P b 0.001). The AUCs for the ProVent 14 score and the Korean ProVent 14 score were 0.74 (95% confidence interval (CI), 0.69–0.78) and 0.75 (95% CI, 0.70–0.80), respectively, showing no statistically significant difference (P = 0.13, Fig. 4). Supplementary table shows the results of logistic regression analysis with risk variables. Inspection of locally weighted scatterplot smoothing curves identified the following cut-off values: body mass index b 23 kg/m2 and comorbidity index ≥ 2 (Supplementary Fig. 1). Multivariate analysis indicated that PaO2/FiO2 b 200 tended to be associated with mortality. According to the β values observed in this model, a modified ProVent 14 score was developed as follows: (i) age ≥ 65 years (+ 1 point); (ii) platelet count b 100 × 109/L (+2 points); (iii) requirement for vasopressors (+ 1 point); (iv) non-trauma diagnosis (+ 1 point); and (v) PaO2/FiO2 b 200 (+1 point). The Kaplan-Meier survival curves of 1-year survival stratified according to the values of the modified ProVent 14 score are shown in Supplementary Fig. 2. The AUC using the score to predict 1-year mortality was 0.75 (95% CI, 0.70–0.80). 4. Discussion The present study indicates that the ProVent 14 score accurately predicts the risk of 1-year mortality in patients requiring at least 14 days of MV in a non-US population. The Korean ProVent 14 score and modified ProVent 14 score did not perform better than the original model. To the best of our knowledge, this is the first comprehensive study to validate the ProVent 14 score in Korean PMV patients. The original ProVent 14 score was developed in patients requiring PMV in 2005 at five tertiary care hospitals across the US using a limited number of clinical variables that can be easily collected at the bedside or from electronic medical records [13]. Data derived from this scoring
Table 4 Logistic regression analysis with six risk variables of the ProVent 14 model. Variable
Unadjusted OR (95% CI)
P
Adjusted ORa (95% CI)
P
β value
Age ≥ 65 years Age 50–64 years Non-trauma Vasopressors Hemodialysis Platelet count b 100 × 109/L
3.73 (2.09–6.67) 1.83 (1.001–3.34) 3.19 (2.03–5.00) 2.25 (1.42–3.57) 3.06 (1.40–6.70) 7.93 (3.95–15.89)
b0.001 0.05 b0.001 0.001 0.005 b0.001
2.70 (1.41–5.16) 1.53 (0.79–2.98) 2.51 (1.53–4.13) 1.74 (1.04–2.91)
0.003 0.21 b0.001 0.04
0.99 0.43 0.92 0.55
6.71 (3.24–13.91)
b0.001
1.90
OR = odds ratio, CI = confidence interval. a All six variables were included in the multivariate analysis using stepwise backward selection procedures.
W.-Y. Kim et al. / Journal of Critical Care 44 (2018) 249–254
system, when combined with physician estimates of outcome, can provide relevant decision-making information. However, it is still unclear how this score performs in other populations, such as those in Asian ICUs, as these units are known to differ from those in Western countries in many aspects related to medical expenses, family support, the concept of death, and social customs [23]. The 43% 1-year mortality seen in the present study (where medical ICU admission comprised 66% of the group) is comparable to the 32%–51% mortality rates reported in previous studies of PMV in similar patient populations [9,13,23]. However, a 1-year mortality as high as 60%–67% was demonstrated in other two studies [4,10], which is likely due to the fact that N 70% of those populations were medical patients with pulmonary and cardiovascular diseases. Here, the 1-year mortality rate associated with a ProVent 14 score ≥ 4 (75%) was relatively low when compared with that seen in previous studies. This result is consistent with the findings of Udeh et al. [24], who showed a lower observed mortality associated with a ProVent score ≥ 4 in the 14 to 20 days PMV group (73%) than in the 21 + days PMV group (89%). We suggest that this observation results from a lower level of illness severity in patients requiring MV for 14 days when compared with those requiring MV for up to or beyond 21 days. Nevertheless, the ProVent 14 score performed well for discrimination and calibration in our patient group. In addition, the score showed very high specificity in patients at greatest risk of death. Limiting the possibility of misclassifying a patient as very high risk is particularly important as such a classification may result in the decision to withhold or withdraw life support. Despite differences in study populations, mortality, and study time frame, the similarity in the values of the AUC curves between other studies and our own emphasizes the robust nature and general application of the ProVent model. In this study, we sought to select the best subset of ProVent 14 variables in Korean PMV patients. Carson et al. [9] and Hough et al. [13] assigned one point to age 50–64 years. However, we found that age 50–64 years was not significantly associated with 1-year mortality (as did a study by Leroy et al. [4]); therefore, this was not included as a risk variable in simplified models (the French ProVent score [4] and our Korean ProVent 14 score). This discrepancy may be explained by differences in the characteristics of the study populations. Patients enrolled in the French group and our own were much older (N60 years of age), so the independent prognostic value of age 50–64 years was less apparent. Additionally, we sought to determine whether inclusion of other potential predictors of mortality would improve the model performance. The ProVent 14 score added non-trauma diagnosis as an additional variable to the original model, recognizing that non-trauma patients on PMV have worse outcomes than trauma patients. In our multivariate analysis, PaO2/FiO2 b 200 tended to be associated with mortality and was included in the modified ProVent 14 score. Simplification or addition of extra variables should be weighed carefully due to potential drawbacks such as lowering sensitivity or increasing complexity of the model. In this study, neither the Korean ProVent 14 score nor the modified ProVent 14 score derived additional benefit
253
Fig. 4. Receiver operating characteristic curves for the ProVent 14 score and Korean ProVent 14 score predicting 1-year mortality. AUC = area under the receiver operating characteristic curve.
than the original ProVent 14. Again, our findings are consistent with robustness of the original ProVent 14 model. Our study adds to the accumulating evidence of the generalizability of the ProVent 14 score. Previous studies showed that the ProVent model had good discrimination and accuracy in patients with primarily surgical diagnoses [24], in non-tertiary community hospitals [4], and even when it was used to predict short-term prognosis such as ICU mortality [25]. More studies are needed to adopt the ProVent model across broader populations such as hemato-oncologic patients requiring PMV. Moreover, the impact of the ProVent score on patient care, resource allocation, and ability to facilitate decisions with patient/family preferences should be investigated. This is a major limitation of previous ProVent studies as well as our own and is, therefore, an aspect that requires further evaluation. A recent study of a web-based decision aid for surrogate decision makers of patients receiving PMV is a good example of implementing the ProVent model for decision making process in the setting of critical illness [26]. The present study has several limitations. First, the study was singlecentered and retrospective. The ProVent 14 score was validated in patients requiring PMV at a tertiary care hospital and so it may not be possible to generalize these data to other settings, although the majority of PMV patients are managed in tertiary hospitals. Moreover, the retrospective study design could have introduced selection and ascertainment bias. Second, measuring specificity at a high mortality risk
Fig. 3. Kaplan-Meier survival curves of patients stratified according to the ProVent 14 score (A) and the Korean ProVent 14 score (B). MV = mechanical ventilation.
254
W.-Y. Kim et al. / Journal of Critical Care 44 (2018) 249–254
comes at the expense of lower sensitivity, as a substantial proportion of patients who died did not have a high ProVent 14 score. Third, the ProVent 14 variables (with the exception of age and non-trauma) are dynamic markers of illness severity and may change during the course of patient's illness, i.e., the scores assessed at day 14 may decline by day 21. Thus, knowing the outcomes of those of improved scores and worsening or unchanged scores would provide guidance to clinicians regarding decision making process and communication with families. The utility of the evolution of the ProVent score over time is interesting and should be confirmed in future studies. 5. Conclusions In conclusion, the ProVent 14 score can allow earlier identification of patients receiving PMV in Korea who are at high risk of 1-year mortality. This validation study indicated that the ProVent 14 model has ethnic and temporal generalizability. Our data indicate that a simplified approach or inclusion of additional variables did not improve the score's performance. Further validation of the ProVent 14 score in a larger sample is required. Conflicts of interest None. Funding This work was supported by a clinical research grant from Pusan National University Hospital in 2017. The funders had no role in study design, data collection and analysis, preparation of the manuscript, or decision to publish. Appendix A. Supplementary data Supplementary data to this article can be found online at https://doi. org/10.1016/j.jcrc.2017.11.029. References [1] MacIntyre NR, Epstein SK, Carson S, Scheinhorn D, Christopher K, Muldoon S, et al. Management of patients requiring prolonged mechanical ventilation: report of a NAMDRC consensus conference. Chest 2005;128:3937–54. [2] Zilberberg MD, de Wit M, Pirone JR, Shorr AF. Growth in adult prolonged acute mechanical ventilation: implications for healthcare delivery. Crit Care Med 2008;36: 1451–5. [3] Carson SS, Garrett J, Hanson LC, Lanier J, Govert J, Brake MC, et al. A prognostic model for one-year mortality in patients requiring prolonged mechanical ventilation. Crit Care Med 2008;36:2061–9. [4] Leroy G, Devos P, Lambiotte F, Thevenin D, Leroy O. One-year mortality in patients requiring prolonged mechanical ventilation: multicenter evaluation of the ProVent score. Crit Care 2014;18:R155.
[5] Wagner DP. Economics of prolonged mechanical ventilation. Am Rev Respir Dis 1989;140(2 Pt 2):S14–8. [6] White DB, Engelberg RA, Wenrich MD, Lo B, Curtis JR. Prognostication during physician-family discussions about limiting life support in intensive care units. Crit Care Med 2007;35:442–8. [7] Nelson JE, Mercado AF, Camhi SL, Tandon N, Wallenstein S, August GI, et al. Communication about chronic critical illness. Arch Intern Med 2007;167:2509–15. [8] Carson SS, Bach PB. Predicting mortality in patients suffering from prolonged critical illness: an assessment of four severity-of-illness measures. Chest 2001;120:928–33. [9] Carson SS, Kahn JM, Hough CL, Seeley EJ, White DB, Douglas IS, et al. A multicenter mortality prediction model for patients receiving prolonged mechanical ventilation. Crit Care Med 2012;40:1171–6. [10] Mok JH, Kim YH, Jeong ES, Eom JS, Kim MH, Kim KU, et al. Clinical application of the ProVent score in Korean patients requiring prolonged mechanical ventilation: a 10year experience in a university-affiliated tertiary hospital. J Crit Care 2016;33: 158–62. [11] Freeman BD, Morris PE. Tracheostomy practice in adults with acute respiratory failure. Crit Care Med 2012;40:2890–6. [12] Kahn JM, Benson NM, Appleby D, Carson SS, Iwashyna TJ. Long-term acute care hospital utilization after critical illness. JAMA 2010;303:2253–9. [13] Hough CL, Caldwell ES, Cox CE, Douglas IS, Kahn JM, White DB, et al. Development and validation of a mortality prediction model for patients receiving 14 days of mechanical ventilation. Crit Care Med 2015;43:2339–45. [14] Morgan RJ, Williams F, Wright MM. An early warning scoring system for detecting developing critical illness. Clin Intensive Care 1997;8:100. [15] Charlson ME, Pompei P, Ales KL, MacKenzie CR. A new method of classifying prognostic comorbidity in longitudinal studies: development and validation. J Chronic Dis 1987;40:373–83. [16] Knaus WA, Draper EA, Wagner DP, Zimmerman JE. APACHE II: a severity of disease classification system. Crit Care Med 1985;13:818–29. [17] Vincent JL, Moreno R, Takala J, Willatts S, De Mendonca A, Bruining H, et al. The SOFA (Sepsis-related Organ Failure Assessment) score to describe organ dysfunction/failure. On behalf of the Working Group on Sepsis-Related Problems of the European Society of Intensive Care Medicine. Intensive Care Med 1996;22:707–10. [18] DeLong ER, DeLong DM, Clarke-Pearson DL. Comparing the areas under two or more correlated receiver operating characteristic curves: a nonparametric approach. Biometrics 1988;44:837–45. [19] Cleveland WS, Grosse E, Shyu WM. Local regression models. In: Chambers JM, Hastie TJ, editors. Statistical models in S. New York: Cole/Wadsworth & Brooks; 1992. [20] Holst LB, Haase N, Wetterslev J, Wernerman J, Guttormsen AB, Karlsson S, et al. Lower versus higher hemoglobin threshold for transfusion in septic shock. N Engl J Med 2014;371:1381–91. [21] NICE-SUGAR Study Investigators, Finfer S, Chittock DR, Su SY, Blair D, Foster D, et al. Intensive versus conventional glucose control in critically ill patients. N Engl J Med 2009;360:1283–97. [22] ARDS Definition Task Force, Ranieri VM, Rubenfeld GD, Thompson BT, Ferguson ND, Caldwell E, et al. Acute respiratory distress syndrome: the Berlin Definition. JAMA 2012;307:2526–33. [23] Phua J, Joynt GM, Nishimura M, Deng Y, Myatra SN, Chan YH, et al. Withholding and withdrawal of life-sustaining treatments in intensive care units in Asia. JAMA Intern Med 2015;175:363–71. [24] Udeh CI, Hadder B, Udeh BL. Validation and extension of the prolonged mechanical ventilation prognostic model (ProVent) score for predicting 1-year mortality after prolonged mechanical ventilation. Ann Am Thorac Soc 2015;12:1845–51. [25] Boniatti MM, Giustina AD, Marin LG, França J, Dos Santos MC, Vidart J, et al. Mortality in chronically critically ill patients: expanding the use of the ProVent score. J Crit Care 2015;30:1039–42. [26] Cox CE, Wysham NG, Walton B, Jones D, Cass B, Tobin M, et al. Development and usability testing of a Web-based decision aid for families of patients receiving prolonged mechanical ventilation. Ann Intensive Care 2015;5:6.