- Email: [email protected]
Prediction of Bleeding After Cardiac Surgery: Comparison of Model Performances: A Prospective Observational Study
Prediction of Bleeding After Cardiac Surgery: Comparison of Model Performances: A Prospective Observational Study Guri Greiff, MD,*‡ Hilde Pleym, MD, PhD,*§ Roar Stenseth, MD, PhD,*‡ Kristin S. Berg, MD,† Alexander Wahba, MD, PhD,*‖ and Vibeke Videm, MD, PhD†¶ Objectives: Primary aims were to (1) perform external validation of the Papworth Bleeding Risk Score, and (2) compare the usefulness of the Dyke et al universal definition of perioperative bleeding with that used in the Papworth Bleeding Risk Score. A secondary aim was to use a locally developed logistic prediction model for severe postoperative bleeding to investigate whether prediction could be improved with inclusion of the variable “surgeon” or selected intraoperative variables. Design: Single-center prospective observational study. Setting: University hospital. Participants: 7,030 adults undergoing cardiac surgery. Interventions: None. Measurements and Main Results: Papworth Bleeding Risk Score could identify the group of patients with low risk of postoperative bleeding, with negative predictive value of 0.98, when applying the Papworth Score on this population. The positive predictive value was low; only 15% of the patients who were rated high risk actually suffered from
increased postoperative bleeding when using the Papworth Score on this population. Using the universal definition of perioperative bleeding proposed by Dyke et al, 28% of patients in the Papworth high-risk group exceeded the threshold of excessive bleeding in this population. The local models showed low ability for discrimination (area under the receiver operating characteristics curve o0.75). Addition of the factor “surgeon” or selected intraoperative variables did not substantially improve the models. Conclusion: Prediction of risk for excessive bleeding after cardiac surgery was not possible using clinical variables only, independent of endpoint definition and inclusion of the variable “surgeon” or of selected intraoperative variables. These findings may be due to incomplete understanding of the causative factors underlying excessive bleeding. & 2015 Elsevier Inc. All rights reserved.
S
suggested by Dyke et al8 will be used clinically even if their properties seem attractive. The main aims of the present study, therefore, were (1) to perform external validation of the Papworth Bleeding Risk Score, and (2) to compare the usefulness of the definition of increased bleeding proposed by Dyke et al with that used in the Papworth Bleeding Risk Score. As a secondary aim, a local risk prediction model was developed to identify cardiac surgical patients at the highest risk of severe postoperative bleeding, which permitted investigation of whether prediction could be improved if the variable “surgeon” or some selected intraoperative variables were included.
EVERE BLEEDING after cardiac surgery is a relatively common complication and may occur in as many as 20% of the patients.1 Previous studies have tried to identify preoperative predictors for blood loss after coronary artery surgery, mostly without success. Wahba et al2 found that laboratory testing for hyperfibrinolysis and platelet dysfunction was useful to predict abnormal bleeding after coronary artery bypass grafting (CABG), but except for preoperative platelet count, only the postoperative variables were significant. Point-of-care tests like rotational thromboelastometry (ROTEM) to monitor multiple coagulation parameters represent an interesting alternative, but preoperative ROTEM analysis does not seem useful to predict postoperative hemorrhage in cardiac surgery patients.3,4 Identification of patients at the highest risk of excessive blood loss after cardiac surgery could lead to prophylactic interventions, such as termination of antiplatelet therapy preoperatively, meticulous surgical hemostasis, and prophylactic antifibrinolytic drug treatment. It also could lead to early postoperative treatment with platelets and fresh frozen plasma, coagulation factors, and surgical intervention, if relevant. Patients requiring reexploration for bleeding are at higher risk of adverse outcomes, which increases if time to re-exploration is prolonged.5 In 2011, Vuylsteke et al developed the Papworth Bleeding Risk Score from a prospectively created database of patients undergoing cardiac surgery with cardiopulmonary bypass (CPB).6 The authors included preoperative data from one hospital only, and the score´s external applicability and clinical utility have not been validated. From the field of epidemiology, it is clearly recognized that external validation of any prediction model is of major importance before it is useful in clinical practice.7 Dyke et al recently proposed a universal definition of perioperative bleeding in adult cardiac surgery,8 using a different definition from that used for the Papworth Bleeding Risk Score. The authors hypothesized that, without external validation, neither the Papworth Bleeding Risk Score nor the endpoint
KEY WORDS: risk score, risk prediction, hemorrhage, cardiac surgery, bleeding endpoint
METHODS The study was based on prospectively collected data from 7,137 adult patients undergoing cardiac surgery from 2000 to 2011 at St. Olav
From the Departments of *Circulation and Medical Imaging and †Laboratory Medicine, Children’s and Women’s Health, Faculty of Medicine, Norwegian University of Science and Technology, Trondheim, Norway; ‡Department of Cardiothoracic Anaesthesia and Intensive Care, §Clinic of Anaesthesia and Intensive Care, ║Department of Cardiothoracic Surgery; and ¶Department of Immunology and Transfusion Medicine; St. Olav University Hospital, Trondheim, Norway. Financial support and sponsorship was provided by the Liaison Committee between the Central Norway Regional Health Authority (RHA) and the Norwegian University of Science and Technology (NTNU). Address reprint requests to Guri Greiff, MD, Department of Cardiothoracic Anesthesia and Intensive Care, St. Olav University Hospital, N-7006 Trondheim, Norway. E-mail: [email protected] © 2015 Elsevier Inc. All rights reserved. 1053-0770/2601-0001$36.00/0 http://dx.doi.org/10.1053/j.jvca.2014.08.002
Journal of Cardiothoracic and Vascular Anesthesia, Vol 29, No 2 (April), 2015: pp 311–319
311
312
GREIFF ET AL
University Hospital, Trondheim, Norway. The present study was part of the Cardiac Surgery Outcome Study (CaSOS), and some of the data have been used for development of risk prediction models for mortality, postoperative cardiac dysfunction, prolonged ventilation, prolonged stay in the intensive care unit, acute kidney injury, and prediction of genetic and clinical risk factors for fluid overload after cardiac surgery, as well as for validation of previously published scores for several endpoints.9–14 The CaSOS study was approved by The Regional Research Ethics Committee in Medicine in Trondheim, Norway, and The Norwegian Data Inspectorate. All patients registered in the database underwent cardiac surgery with CPB. Patients treated with acetylsalicylic acid (ASA) used 75 or 160 mg. Clopidogrel was not included in the analysis because of incomplete registration. In the database, clopidogrel was registered from 2008. Before CPB, heparin (300 U/kg; Leo, Copenhagen, Denmark) was given to achieve a kaolin activated coagulation time (ACT; Medtronic Blood Management, Parker, CO) of more than 480 seconds. Additional heparin was given when needed. The perfusion circuit was primed with 1,500 mL of Ringer’s acetate solution with 7,500 U of heparin. A coated membrane oxygenator was used. After CPB, protamine sulfate (Leo, Copenhagen, Denmark) was given to achieve an ACT within 10% of the baseline value. Blood remaining in the CPB circuit was collected and retransfused to the patient. Tranexamic acid (30 mg/kg) was used routinely in the authors’ hospital from 2000 and was given before the start of CPB. High-dose aprotinin was used until 2008 in most patients with postinfarction rupture of the ventricular septum or dissection of the ascending aorta and in some patients with endocarditis, altogether approximately 150 patients (2.1%), but was not registered in the database. The study design is outlined in Figure 1.
Validation of the Papworth Bleeding Risk Score The Papworth Bleeding Risk Score is based on 5 risk factors: Surgery priority, surgery type (CABG or single valve), aortic valve disease, body mass index (BMI), and age.6 On the basis of the calculated scores (0-5 points), patients were divided into the 3 defined risk groups for the Papworth Score: low-risk (0 points), medium-risk
(1-2 points), and high-risk (3-5 points). Matching definitions was possible in the present population for all variables except aortic valve disease, for which aortic valve surgery was used. In the original publication, blood loss exceeding 2 mL/kg/h during the first 3 hours in the intensive care unit (or during a shorter period if the patient underwent transfusion with fresh-frozen plasma, platelets, or cryoprecipitate, reoperation, or died within 3 hours) was considered an adverse outcome. In the database, postoperative drainage volumes were recorded after 4 hours, so this time frame was used in the external validation. Information about transfusion timing in the intensive care unit was missing. The patients who underwent reoperation within 4 hours exceeded the threshold of 2 mL/kg/h, and no patients died within 4 hours because of bleeding. The endpoint used for excessive postoperative blood loss then was defined as blood loss exceeding 2 mL/kg/h the first 4 hours postoperatively. The negative predictive value (NPV) in the Papworth low-risk group was calculated (ie, the proportion of patients without severe bleeding in that group). The positive predictive value (PPV) in the Papworth high-risk group also was calculated (ie, the proportion of patients with severe postoperative bleeding in that group).
Alternative Endpoint Definition The composite bleeding endpoint defined by Dyke et al has 5 bleeding categories based on chest tube drainage during the first 12 postoperative hours: Transfusion of red cells, plasma, and platelets; use of cryoprecipitate, prothrombin complex concentrates, or recombinant activated factor VII; re-exploration or tamponade; and delayed sternal closure.8 The criteria were partly redundant. Cryoprecipitate or prothrombin were never used during the study period in the present population, and recombinant activated factor VII was used for approximately 10 patients, so these criteria were omitted. As in the original paper, the frequency of patients in each bleeding class was assessed (0 ¼ insignificant, 1 ¼ mild, 2 ¼ moderate, 3 ¼ severe, and 4 ¼ massive). Dyke class 3 and 4 patients were defined as having excessive postoperative bleeding and compared with patients from Dyke class 0 - 2. The NPV and PPV for the Papworth Bleeding Risk Score were calculated using this bleeding endpoint instead of the original Papworth definition.
Included patients n=7,137 Incomplete data n=107 (1.5%) Used in study n=7,030 (98.5%)
Calculated Papworth Bleeding Risk Score - compared to original publication (6)
Calculated Papworth Bleeding Risk Score using composite bleeding definition (8)
Developed local risk prediction model
- compared sensitivity, specificity, positive and negative predictive values
- compared Papworth and composite bleeding definitions - compared preoperative and intraoperative models - compared models with and without “surgeon” Fig 1.
Outline of study design.
313
PREDICTION OF BLEEDING AFTER CARDIAC SURGERY
Local Model Development Including Intraoperative Variables For development of the Papworth Bleeding Risk Score, Vuylsteke et al6 used several methods that are not in agreement with current guidelines;15 data were split once into a development and a test set, which reduces future predictive ability,16 variable selection was based on univariate testing, and all continuous variables were dichotomized. For better evaluation of the endpoint definitions and usefulness of intraoperative predictors, an alternative local risk prediction model was developed using recommended statistical methods. This reduces the risk of overfitting; ie, the resulting model becomes less dependent on the exact characteristics of the data used for development and, therefore, more useful in future data sets. A logistic regression model was developed based on 18 preoperative variables and 9 intraoperative explanatory variables (Table 1). The operation types were grouped according to the complexity of the surgical procedures and the risk of bleeding. On the basis of clinical experience and median bleeding, 4 groups were identified by surgical procedures performed; surgical procedures with approximately the same risk and complexity were grouped together. Details on model
development are shown in Appendix 1. Odds ratios and 95% confidence intervals were calculated. Four different models were developed. First, a preoperative risk prediction model for postoperative bleeding after cardiac surgery was developed using the threshold of 2 mL/kg/h of blood loss as the endpoint (model 1). Then, the authors investigated whether this model could be improved by including selected intraoperative variables (model 2). Finally, 2 models were developed (one preoperative and one intraoperative) based on the composite bleeding endpoint by Dyke et al8 (models 3 and 4). Analyses also were performed by adding “surgeon” in the preoperative models using both endpoints, ie, the threshold of 2 mL/kg/h of blood loss and the composite endpoint by Dyke et al. The variable “surgeon” was a categoric variable with 8 categories (details are shown in Appendix 1). Furthermore, models were developed by which renal dysfunction was indicated by creatinine clearance. In these models, variables used during calculation of creatinine clearance were omitted because of strong correlation. The time span for data collection was 11 years. To test whether technical changes influenced the present findings, the year of surgery was added into model 1 as an adjustment variable.
Table 1. Variable Definitions Variable
Preoperative Age Body surface area Smoking preoperatively Chronic pulmonary disease Diabetes mellitus Hypertension Chronic cardiac insufficiency Endocarditis Angina pectoris preoperatively Preoperative renal dysfunction ASA treatment LMWH treatment Warfarin treatment Angiotensin inhibitor use Hemoglobin concentration Previous cardiac surgery Operation urgency Operation type
Intraoperative CPB time Inotropic support Vasoconstrictor use Fluid balance Red blood cell transfusion Plasma transfusion Temperature Hemoglobin concentration Surgeon
Definition
Years (continuous) m2 (continuous) Current smoker or quit o6 months ago (yes/no) Use of bronchodilating agents or FEV1 below 75% (yes/no) Receiving medication (yes/no) Receiving medication or diastolic blood pressure above 90 mmHg (yes/no) Receiving medication (yes/no) Receiving antibiotic treatment (yes/no) Angina or treatment for angina pectoris preoperatively Serum creatinine 4140 mmol/L or dialysis (yes/no) Treated with acetylsalicylic acid (yes/no) Treated with low-molecular-weight heparin (yes/no) Treated with warfarin (yes/no) Treated with angiotensin converting enzyme inhibitor/ angiotensin II receptor blocker (yes/no) g/dL (continuous) (yes/no) 0 ¼ standard waiting list. 1 ¼ surgery within 1 week. 2 ¼ surgery within 24 hours 1: CABG or single valve surgery. 2: Combined AVR and CABG or combined MVR and CABG. 3: Miscellaneous (Closure of an atrial septum defect or ventricular septum defect, closure of ventricular septum rupture, AVR and MVR in combination with procedures other than CABG, operation for aneurysm of the ascending aorta, and other cardiac surgery like pericardiectomy and removal of cardiac tumors). 4: Dissection of the ascending aorta. Cardiopulmonary bypass time, minutes (continuous) For clinical indication during surgery (yes/no) For clinical indication during surgery (yes/no) Intraoperative fluid balance For clinical indication during surgery (yes/no) For clinical indication during surgery (yes/no) Lowest temperature intraoperatively Lowest hemoglobin concentration intraoperatively, g/dL (continuous) 2 categories, surgeons with equivalent amounts of postoperative bleeding (tested by analyses of variance) were grouped together
Abbreviations: ACE, angiotensin converting enzyme; ASA, acetylsalicylic acid; AVR, aortic valve replacement; CABG, coronary artery bypass grafting; CPB, cardiopulmonary bypass; FEV1, forced expired volume in 1 second; LMWH, low-molecular-weight heparin; MVR, mitral valve replacement.
314
GREIFF ET AL
Model discrimination was assessed by receiver operating characteristics (ROC) curves. An area under the curve (AUC) from 0.7 to 0.9 indicates moderate accuracy.17 Calibration of the models, ie, whether calculated risk was similar to observed risk, was assessed by the Hosmer-Lemeshow test and calibration curves (Appendix 1). With the Hosmer-Lemeshow test, a p value 40.05 indicates adequate calibration. Sensitivity is the probability of truly identifying patients with the endpoint using a certain cut-off of predicted probability, and specificity is the probability of truly identifying patients without the endpoint using the cut-off.18 NPV and PPV were calculated in the low-risk and high-risk groups at the best cut-off, identified by the Youden index19 (Appendix 1).
General Statistics Data are shown as mean with 95% confidence interval or frequencies with percentages, unless otherwise stated. For intergroup comparisons, Pearson´s chi-square test or a t-test was used. The statistical software packages SPSS (version 20.0, SPSS, Chicago, IL), R (version 3. 0.0, R Foundation, http://www.r-project.org), and SigmaPlot (version 12.0; Systat Software, San Jose, CA) were applied. All p values below 0.05 were considered statistically significant. RESULTS
Patient Characteristics A total of 7,137 patients were included, and complete data sets (7,030 patients, 98.5%) were used for model validation and comparison. Patient characteristics and other clinical variables are shown in Table 2. According to the endpoint defined by a threshold of 2 mL/kg/h of blood loss, 7.5% (527 of 7,030) of the patients had excessive postoperative bleeding. According to the composite definition by Dyke et al,8 1,258 patients (17.6%) had excessive postoperative bleeding. A total of 445 patients (6.4%) had excessive bleeding by both definitions, 82 patients (1.2%) had excessive bleeding only by the 2 mL/kg/h definition, and 813 patients (11.2%) had excessive bleeding only by the composite definition. Patients with a high Papworth Bleeding Risk Score had a higher incidence of severe postoperative bleeding compared with patients with a lower score; however, the incidence of severe bleeding was lower in the present population for the 3 risk groups defined by the Papworth Bleeding Risk Score (Table 3), with 15% of the patients in the high-risk group exceeding the bleeding threshold compared with 21% in the original population.6 Using the composite endpoint definition by Dyke et al,8 28% of the patients in the high Papworth Bleeding Risk Score group exceeded the threshold for excessive bleeding (Table 3). PPV and NPV for the Papworth Bleeding Risk Score using the 2 alternative endpoint definitions are shown in Table 3. Comparisons using novel prediction models (Logistic regression models), without the surgeon using the endpoint defined by a threshold of 2 mL/kg/h of blood loss, are shown in Table 4. ASA, lower body surface area, urgent operation, and operation type other than isolated CABG or single-valve surgery were significant predictors for excessive postoperative blood loss in model 1. Model 2 also included intraoperative variables, of which only intraoperative inotropic support and cardiopulmonary bypass time (CPB) were significant (Table 4). In this model, urgent surgery within 1 week and dissection of the ascending aorta were no longer significant. Both models had adequate calibration (Hosmer-Lemeshow test, p ¼ 0.37 for model 1 and p ¼ 0.09 for model 2). The ROC
curves showed poor-to-moderate discrimination, with AUC of 0.688 (0.664-0.712) for model 1 and 0.709 (0.698-0.743) for model 2 (p ¼ 0.008). Using creatinine clearance to define renal dysfunction did not improve model 1. The AUC for this model was 0.672 (0.648-0.696), ie, lower than for model 1; therefore, serum creatinine was retained as a simpler variable for renal dysfunction in further analyses. Inclusion of year of surgery as an adjustment variable had minor effects on the odds ratio for the variables in model 1. The models using the composite endpoint by Dyke et al8 are shown in Table 4. In these models, body surface area and surgery urgency were no longer significant; instead, preoperative renal dysfunction, previous cardiac surgery, and use of low-molecular-weight heparin were significant, in addition to ASA and operation type. The AUC were 0.739 (0.722-0.755) and 0.761 (0.745-0.777). The preoperative model had adequate calibration (Hosmer-Lemeshow test, p ¼ 0.57), but the intraoperative model was not well calibrated (p ¼ 0.04). For the preoperative model using the 2 mL/kg/h bleeding threshold endpoint (model 1), the NPV was 0.95 in the low-risk group and the PPV was 0.12 in the high-risk group, when dividing patients into 2 risk groups at the best cut-off identified by the Youden index (Appendix 1). When using the composite endpoint by Dyke et al (model 3), the NPV was 0.89 in the low-risk group and the PPV was 0.44 in the high-risk group. Surgeon was a significant predictor for excessive postoperative bleeding in the preoperative models independent of endpoint definition (p o 0.001). Calibration still was adequate (Hosmer-Lemeshow test: model 1, p ¼ 0.39; model 3, p ¼ 0.23). The AUC increased for both preoperative models with surgeon (model 1: AUC ¼ 0.713 (0.691-0.736), p o 0.001); model 3: AUC ¼ 0.757 (0.742-0.773); po0.001), but discrimination was still only moderate. Bleeding during the first 4 postoperative hours was highly overlapping among the 10 deciles of risk independent of endpoint definition (Fig 2). DISCUSSION
In this study investigating prediction models for excessive postoperative bleeding after cardiac surgery, neither the Papworth Bleeding Risk Score nor novel models based on the same endpoint definition or the Dyke et al composite endpoint definition were clinically useful. Either discrimination was too low or calibration was inadequate even when using state-of-theart methods for model development. Addition of the factor “surgeon” or selected intraoperative variables did not improve the models substantially. Severe bleeding after cardiac surgery is still a major problem, and better prediction models are needed.20 The published Papworth Bleeding Risk Score6 showed slightly worse performance in the present population, which is a common finding due to overfitting during score development and differences in patient characteristics.21 The authors’ local model developed using stricter statistical criteria and internal validation showed the same pattern of a high NPV but very low PPV. Thus, both scores will more precisely identify patients at low risk of bleeding. The models based on the Dyke et al8 endpoint definition had a
315
PREDICTION OF BLEEDING AFTER CARDIAC SURGERY
Table 2. Patient and Surgical Characteristics Bleeding o2 mL/kg/h
Characteristic
Age (years) Body surface area Smoking preoperatively Diabetes mellitus Hypertension Chronic cardiac insufficiency Endocarditis Angina pectoris preoperatively Chronic pulmonary disease Renal function Normal renal function Preoperative renal dysfunction with creatinine above 140 (mmol/L) or dialysis Previous cardiac surgery ACE inhibitor use Warfarin preoperatively Acetylsalicylic acid preoperatively Low-molecular-weight heparin preoperatively Hemoglobin preoperatively (g/dL) Hemoglobin intraoperatively (g/dL) Operation urgency Normal priority Operation within 1 week Emergency operation within 24 hours Operation type CABG or single-valve surgery AVR and CABG, MVR and CABG Miscellaneous Dissection of ascending aorta CPB time Lowest temperature intraoperatively Inotropic support Vasoconstrictor use Fluid balance (mL) Red blood cell transfusion Plasma transfusion
the First 4 Hours
Bleeding Z2 mL/kg/h
Bleeding Classes 0-2
Bleeding Classes 3-4
Postoperatively
the First 4 Hours
Defined by Dyke et al8
Defined by Dyke et al8
(n ¼ 6,518)
Postoperatively (n ¼ 527)
66.2 1.98 1,609 908 3,488 952 101 2,186 947
(65.9-66.4) (1.97-1.98) (24.7%) (13.9%) (53.5%) (14.6%) (1.5%) (33.5%) (14.5%)
67.6 1.89 105 59 270 118 19 178 77
(66.5-68.5) (1.88-1.90) (19.9%) (11.9%) (51.2%) (22.4%) (3.6%) (33.8%) (14.6%)
6,231 (95.6%) 287 (4.4%)
489 (92.8%) 38 (7.2%)
355 2,246 544 4,705 2,099
42 194 45 392 183
(5.4%) (34.5%) (8.3%) (72.2%) (32.2%)
13.7 (13.7-13.8) 10.1 (10.1-10.2) 3,523 (54.1%) 2,687 (41.25%) 307 (4.7%)
(8.0%) (36.8%) (8.5%) (74.4%) (34.7%)
13.3 (13.2-13.5) 9.7 (9.6-9.8)
p Value
0.008 o0.001 0.022 0.08 0.31 o0.001 o0.001 0.91 0.95 0.012
(n ¼ 5,823)
66.1 1.96 1,450 815 3,136 810 73 1,960 823
(65.9-66.4) (1.96-1.97) (25.2%) (14,0%) (54,0%) (13.9%) (1.3%) (33.7%) (14.2%)
5,570 (95.9%) 240 (4.1%)
0.016 0.25 0.87 0.25 0.21 o0.001 o0.001 o0.001
237 (43.5%) 229 (43.5%) 61 (11.6%)
1,993 421 4,198 1,845
0 (34.3%) (7.3%) (72.3%) (31.8%)
13.7 (13.7-13.8) 10.1 (10.1-10.2) 3,152 (54.3%) 2,385 (41.0%) 273 (4.7%)
(n ¼ 1,258)
66.6 (66.4-67.2) 1.96 (1.95-1.98) 274 (22.1%) 159 (12.5%) 638 (50.2%) 274 (21.6%) 54 (4.3%) 424 (32.6%) 213 (16.8%)
(77.9%) (10.5%) (10.5%) (1.2%) (80-82) (33.3-33.4) (23.0%) (82.3%) (2,905-2,954) (16.1%) (11.4%)
306 96 100 25 106 32.4 204 452 3,259 178 111
(58.1%) (18.2%) (19.0%) (4.7%) (101-110) (32.1-32.7) (38.7%) (85.8%) (3,166-3,401) (33.8%) (21.1%)
o0.001 o0.001 o0.001 0.047 o0.001 o0.001 o0.001
0.18 0.97 0.022 0.16 0.015 o0.001 o0.001 0.45 0.018 o0.001
1,173 (92.4%) 97 (7.0%)
410 460 180 909 183
(32.3%) (36.3%) (14.2%) (71.7%) (34.7%)
13.4 (13.4-13-6) 9.8 (9.8-9.9)
o0.001 0.18 o0.001 0.68 0.037 o0.001 o0.001 o0.001
611 (48.1%) 532 (41.9%) 127 (10.0%)
o0.001 5,077 684 682 75 81 33.4 1,556 5,366 2,930 1,049 742
p Value
o0.001 4,606 579 561 65 78 33.4 1,286 4,729 2,886 880 556
(79.3%) (10.0%) (9.7%) (1.1%) (77-79) (33.4-33.5) (22.1%) (81.4%) (2,863-2,908) (15.1%) (9.6%)
778 209 234 48 110 32.3 506 1,118 3,349 385 330
(61.3%) (16.5%) (18.4%) (3.9%) (108-114) (32.2-32.5) (39.8%) (88.0%) (3,263-3,435) (30.3%) (26.0%)
o0.001 o0.001 o0.001 o0.001 o0.001 o0.001 o0.001
NOTES. Data are shown as mean (95% confidence interval) or frequency (percentage). All p values were obtained using Pearson´s chi-square test or a t test, as appropriate. Abbreviations: ACE, angiotensin-coverting enzyme; AVR, aortic valve replacement; CABG, coronary artery bypass grafting; CPB, cardiopulmonary bypass; MVR, mitral valve replacement.
slightly higher PPV (0.28 in the high Papworth Bleeding Risk Score group and 0.44 in the local model high-risk group) and, therefore, could identify more patients experiencing excessive bleeding, but still far from the majority. However, calculating the score could give some relevant information. The low-risk group of patients needs less special care or preoperative testing. Furthermore, they may not need prophylactic treatment and, thereby, may avoid the risk of side effects of such treatment. If a patient in the low-risk group bleeds excessively, the likelihood of a surgical cause may be higher and reoperation may be performed earlier. The models also may identify groups of patients to be included in forthcoming studies.
The low ability to identify the patients at high risk of excessive bleeding using the prediction models probably is related to the large variation in amount of postoperative bleeding in all risk categories. Even if the proportion of patients with excessive postoperative blood loss increased with increasing risk score for both endpoints, the large overlap in bleeding volumes illustrated in Figure 2 indicates why accurate prediction was difficult. The underlying reason is probably that not all the factors resulting in excessive blood loss are yet identified. Increased knowledge about the multifactorial pathophysiology of bleeding in the setting of cardiac surgery is needed, so that other variables explaining more of the variation may be used for model development. The authors hypothesized
316
GREIFF ET AL
Table 3. Papworth Bleeding Risk Score, Comparing Original Population With Present Population Papworth Bleeding Risk Score Risk Groups
Original population* Patients with excessive blood loss Total number of patients Present population† Patients with excessive blood loss Total number of patients Present population, alternative endpoint definition‡ Patients with excessive blood loss Total number of patients
Low Risk
Medium Risk
High Risk
60 (3%)§ 2143 (31%)¶ NPV 0.97
287 (8%)§ 3587 (53%)¶
226 (21%)§ 1083 (16%)¶ PPV 0.21
28 (2%)§ 1396 (20%)¶ NPV 0.98
293 (7%)§ 4269 (61%)¶
203 (15%)§ 1367 (19%)¶ PPV 0.15
141 (10%)§ 1390 (20%)¶ NPV 0.90
726 (17%)§ 4274 (60%)¶
391 (28%)§ 1385 (20%)¶ PPV 0.28
Abbreviations: NPV, negative predictive value for low-risk group compared with high-risk group; PPV, positive predictive value for high-risk group compared with low-risk group. *Vuylsteke et al: The Papworth Bleeding Risk Score.6 †Blood loss exceeding 2 mL/kg/h the first 4 hours postoperatively. ‡Dyke et al: Bleeding categories including total blood loss from chest tube within 12 postoperative hours, transfusion of red cells, plasma, and platelets, surgical reexploration, tamponade.8 §Percent of the total number of patients within the risk group. ¶Percent of the total number of patients.
that factors related to microvascular bleeding and coagulation at the endothelial level as well as genetic variation may be important. This also might explain why certain patients on platelet inhibitors tend to bleed heavily. These hypotheses need further investigation. There are several differences with respect to endpoint definitions among studies trying to predict which patients will develop severe postoperative bleeding; therefore, 2 endpoint definitions were compared. The rate of bleeding per kg body weight for the first 4 hours postoperatively, which is almost equivalent to the Papworth definition, is attractive because it avoids some factors related to different traditions and department routines. The recently published definition by Dyke et al is based on factors that would be expected to be more strongly influenced by transfusion policies that vary among institutions and surgical teams, and change over time. It expands on previously used variables including red cell transfusion22,23 and need for surgical re-exploration.24 By pooling only the 2 classes with highest postoperative bleeding when defining the endpoint, some of the subjectivity in the classification probably became less influential. Even if the resulting prediction model 3 had a somewhat higher AUC and PPV, the improvement was too small to be clinically relevant. The effect of different endpoint definitions clearly was demonstrated by the small proportion of patients categorized as having excessive bleeding by both; the choice of endpoints, therefore, is not obvious. On the basis of the present study, however, none of the endpoint definitions clearly was preferable. The literature supports the significance of several risk factors identified in the present study.6 Antiplatelet therapy routinely is used in patients requiring cardiac surgery. The consistent finding of ASA as significant for prediction of postoperative bleeding is in accordance with data relating
ASA to severe bleeding after cardiac and noncardiac surgery.25,26 A meta-analysis concluded that ASA was associated with increased chest tube drainage and possibly greater requirement for blood products.25 The dose of ASA may play a role. The present patients used 75 or 160 mg, which is a relatively low dose, but still ASA was associated with postoperative bleeding. However, the decision of whether to stop antiplatelet therapy is an area of debate. Stopping may represent a risk in patients with coronary artery stents or unstable eccentric coronary artery lesions, but guidelines on antiplatelet management in cardiac surgery recommend that patients should stop ASA 2 to 10 days before surgery to reduce perioperative blood loss.26 The present findings support this recommendation. In the present study, surgeon was a significant predictor of severe bleeding after cardiac surgery, even if the model adjusted for type of surgery. Others also have found that the surgeon has a major impact on postoperative bleeding,6,27,28 which suggests that improving surgical skills may be important. However, surgeon always will be a local variable and cannot be included in a prediction model for general use. Furthermore, the inclusion of surgeon in the present models only modestly improved the predictive ability. It is important to note that the factors retained in a parsimonious risk prediction model are those best reflecting the variety of information from different underlying causal factors. They are, therefore, not necessarily the direct causal factors themselves. This explains why variables easily identified as related to the outcome may not remain in the model and why models based on different endpoint definitions contain slightly different factors. Even so, some factors were common to the models based on both endpoints and had comparable odds ratios as well (ASA, surgery type, CPB time, and inotropic support), and either may be causal or strongly associated with underlying causal factors.
317
PREDICTION OF BLEEDING AFTER CARDIAC SURGERY
Table 4. Models for Prediction of Postoperative Blood Loss Variable
Model 1 Body surface area Acetylsalicylic acid Operation urgency Normal priority Within 1 week Emergency operation within 24 hours Operation type CABG or single-valve surgery AVR and CABG, MVR and CABG Miscellaneous Dissection of ascending aorta Intercept Model 2 Body surface area Acetylsalicylic acid Operation urgency Normal priority Within 1 week Emergency operation within 24 hours Operation type CABG or single-valve surgery AVR and CABG, MVR and CABG Miscellaneous Dissection of ascending aorta Cardiopulmonary bypass time (per 10 min) Inotropic support Intercept Model 3 Acetylsalicylic acid Preoperative renal dysfunction Previous cardiac surgery Low-molecular-weight heparin Operation type CABG or single-valve surgery AVR and CABG, MVR and CABG Miscellaneous Dissection of ascending aorta Intercept Model 4 Acetylsalicylic acid Previous cardiac surgery Low-molecular-weight heparin Operation type CABG or single-valve surgery AVR and CABG, MVR and CABG Miscellaneous Dissection of ascending aorta Cardiopulmonary bypass time (per 10 min) Inotropic support Intercept
Coefficient
Odds Ratio
95% Confidence Interval
p Value
–2.077 0.388
0.125 1.474
(0.080-0.195) (1.197-1.816)
o0.001 0.001
0.250 0.812
1.000 1.290 2.252
Reference (1.054-1.564) (1.554-3.264)
0.0124 o0.001
1.000 2.306 2.858 3.772
Reference (1.797-2.959) (2.229-3.667) (2.116-6.725)
o0.001 o0.001 o0.001
–2.154 0.377
0.116 1.458
(0.075-0.178) (1.153-1.844)
o0.001 0.001
0.197 0.682
1.00 1.218 1.977
Reference (0.991-1.498) (1.390-2.813)
0.085 0.001
1.00 1.487 1.745 1.681 1.092 1.430
Reference (1.115-1.984) (1.287-2.367) (0.853-3.315) (1.071-1.114) (1.161-1.765)
0.007 o0.001 0.134 o0.001 0.001
1.576 1.620 58.520 1.418
(1.209-2.171) (1.209-2.171) (42.244-81.075) (1.213-1.658)
o0.001 0.001 o0.001 o0.001
1.00 2.582 2.156 7.349
Reference (2.117-3.149) (1.733-2.681) (4.869-11.091)
o0.001 o0.001 o0.001
1.637 44.807 1.406
(1.363-1.965) (32.218-62.315) (1.200-1.646)
o0.001 o0.001 o0.001
1.00 1.508 1.200 2.359 1.087 1.313
Reference (1.193-1.907) (0.925-1.557) (1.430-3.890) (1,065-1.111) (1.119-1.542)
0.001 0.170 0.001 o0.001 0.001
0.835 1.050 1.328 0.729
0.397 0.557 0.520 0.088 0.358 0.301 0.455 0.482 4.069 0.349
0.948 0.768 1.995 –2.632 0.493 3.802 0.340
0.411 0.182 0.852 0.084 0.273 –3.265
NOTE. Models 1 and 2: endpoint was bleeding 42 mL/kg/h the first 4 postoperative hours. Models 3 and 4: endpoint was class 3 or 4 bleeding by Dyke et al.8 Abbreviations: AVR, aortic valve replacement; CABG, coronary artery bypass grafting; MVR, mitral valve replacement.
Few of the published prediction models for bleeding after cardiac surgery are ever validated in other populations. In the present study, the Papworth Bleeding Risk Score was not only validated externally, but the authors also tried to improve prediction by using alternative statistical methods and
redefining the endpoint. This approach supported that the findings were not random, but generally were representative of prediction models based on standard clinical variables. The findings regarding PPV and NPV further illustrated that reporting model discrimination and calibration alone may
318
GREIFF ET AL
Fig 2. Median bleeding volume (left y-axis), Number of patients of class 3 and 4 by Dyke et al8 (first right y-axis), and number of patients exceeding blood loss 42 mL/kg/h for the first 4 hours postoperatively (second right y-axis). The error bars represent 25th and 75th percentiles.
render it difficult to fully evaluate the clinical usefulness of any prediction model. The present study was based on a single-center database and local factors may have influenced the results, but this would not change the direct comparisons among the models. Changes in patient characteristics, surgical techniques, and treatment strategies may influence the calibration of the present models, because the database contained data from 11 years. However, the surgeons and the surgical teams have been almost the same during these years and inclusion of the year of surgery gave very small changes to the model. The authors, therefore, believe that the results are valid despite the long data collection period. CONCLUSION
Neither the Papworth Bleeding Risk Score nor the present prediction models were suitable for identifying the patients at high risk of severe postoperative bleeding after cardiac surgery, independent of endpoint definition. Clinical variables alone probably are not sufficient to identify such patients, and the models could better identify patients at low risk of bleeding. Few patients were characterized as having excessive postoperative bleeding by both definitions, indicating that the choice of definition is important. The surgeon has an impact on postoperative bleeding but inclusion of this variable did not improve prediction much. ASA was associated with postoperative bleeding in all models. External validation of risk prediction models is important to ensure clinical usefulness. APPENDIX 1
overfitting compared with univariate screening of candidate variables. A full model including all the selected variables was fitted first, tested for predefined interactions, linearity in the logit for continuous variables, and overly-influential observations. Model reduction then was performed using limited backward step-down, keeping variables according to Akaike’s Information Criterion. To find more robust estimates of coefficients, bootstrap estimates of the covariance matrix for the regression coefficients were computed (n ¼ 400).16 The Hosmer-Lemeshow test was used to assess calibration. A nonsignificant test indicated small differences between observed and predicted values, ie, adequate goodness-of-fit. Internal validation was performed by calibration curves comparing observed and predicted probabilities after bootstrapping (n ¼ 400). The shrinkage factor also was calculated, giving an estimate of prediction accuracy in future datasets. A shrinkage factor above 0.85 is considered satisfactory, indicating that the model will predict with an estimated error of less than 15% in a future dataset. Sensitivity and specificity are related inversely according to the choice of cut-off point of the predicted probability. To determine the best cut-off , the Youden index was used (¼ maxc) (sensitivityc þ specificityc – 1); that is, the ROC curve point that maximizes the sum of sensitivity and specificity.18,19 The operations were performed by 7 surgeons in addition to a group of substitutes performing only a few operations each. The substitutes were pooled into one group, resulting in a categoric variable with 8 surgeons. This 8-level categoric variable “surgeon” then was added in the logistic regression model.
Additional Methods
Additional Results
The authors used the entire dataset for logistic regression development. Eighteen preoperative variables and 9 intraoperative explanatory variables (Table 1) were selected based on clinical knowledge and the literature, reducing the risk of
The shrinkage factor for the preoperative model using the threshold of 2 mL/kg/h of blood loss as endpoint (model 1) was 0.91, indicating that it will predict postoperative bleeding in a future dataset with an estimated error of 9%. The shrinkage
319
PREDICTION OF BLEEDING AFTER CARDIAC SURGERY
factor for the intraoperative model using the same endpoint (model 2) was 0.89. Using the endpoint defined by Dyke et al, the shrinkage factors were 0.98 and 0.99 for the preoperative (model 3) and intraoperative models (model 4), respectively. On the basis of the Youden index, the best cut-off for the preoperative model using the 2 mL/kg/h bleeding threshold endpoint (model 1) was at a predicted probability of excessive bleeding of 7%, giving a sensitivity of 65% and a specificity of
63%. A low-risk and a high-risk group were defined at this cutoff value, and the NPV was 0.95 in the low-risk group and the PPV was 0.12 in the high-risk group. When using the composite endpoint by Dyke et al, the best cut-off for the preoperative model (model 3) was at a predicted probability of 19%. At this cut-off, sensitivity was 50%, specificity was 86%, the NPV was 0.89 in the low-risk group and the PPV was 0.44 in the high-risk group.
REFERENCES 1. Paparella D, Brister SJ, Buchanan MR: Coagulation disorders of cardiopulmonary bypass: A review. Intensive Care Med 30:1873-1881, 2004 2. Wahba A, Rothe G, Lodes H, et al: Predictors of blood loss after coronary artery bypass grafting. J Cardiothorac Vasc Anesth 11: 824-827, 1997 3. Lee GC, Kicza AM, Liu KY, et al: Does rotational thromboelastometry (ROTEM) improve prediction of bleeding after cardiac surgery? Anesth Analg 115:499-506, 2012 4. Davidson SJ, McGrowder D, Roughton M, et al: Can ROTEM thromboelastometry predict postoperative bleeding after cardiac surgery? J Cardiothorac Vasc Anesth 22:655-661, 2008 5. Choong CK, Gerrard C, Goldsmith KA, et al: Delayed reexploration for bleeding after coronary artery bypass surgery results in adverse outcomes. Eur J Cardiothorac Surg 31:834-838, 2007 6. Vuylsteke A, Pagel C, Gerrard C, et al: The Papworth Bleeding Risk Score: A stratification scheme for identifying cardiac surgery patients at risk of excessive early postoperative bleeding. Eur J Cardiothorac Surg 39:924-930, 2011 7. Bleeker SE, Moll HA, Steyerberg EW, et al: External validation is necessary in prediction research: a clinical example. J Clin Epidemiol 56:826-832, 2003 8. Dyke C, Aronson S, Dietrich W, et al: Universal definition of perioperative bleeding in adult cardiac surgery. J Thorac Cardiovasc Surg 147:1458-1463, 2014 9. Berg KS, Stenseth R, Pleym H, et al: Mortality risk prediction in cardiac surgery: Comparing a novel model with the EuroSCORE. Acta Anaesthesiol Scand 55:313-321, 2011 10. Widyastuti Y, Stenseth R, Wahba A, et al: Length of intensive care unit stay following cardiac surgery: Is it impossible to find a universal prediction model? Interact Cardiovasc Thorac Surg 15: 825-832, 2012 11. Widyastuti Y, Stenseth R, Pleym H, et al: Pre-operative and intraoperative determinants for prolonged ventilation following adult cardiac surgery. Acta Anaesthesiol Scand 56:190-199, 2012 12. Widyastuti Y, Stenseth R, Berg KS, et al: Preoperative and intraoperative prediction of risk of cardiac dysfunction following open heart surgery. Eur J Anaesthesiol 29:143-151, 2012 13. Berg KS, Stenseth R, Wahba A, et al: How can we best predict acute kidney injury following cardiac surgery? A prospective observational study. Eur J Anaesthesiol 30:704-712, 2013 14. Enger TB, Pleym H, Stenseth R, et al: Genetic and clinical risk factors for fluid overload following open-heart surgery. Acta Anaesthesiol Scand 58:539-548, 2014
15. Moons KG, Kengne AP, Woodward M, et al: Risk prediction models: I. Development, internal validation, and assessing the incremental value of a new (bio)marker. Heart 98:683-690, 2012 16. Harrell FE Jr., Lee KL, Mark DB: Multivariable prognostic models: Issues in developing models, evaluating assumptions and adequacy, and measuring and reducing errors. Stat. Med 15:361-387, 1996 17. Fischer JE, Bachmann LM, Jaeschke R: A readers' guide to the interpretation of diagnostic test properties: Clinical example of sepsis. Intensive Care Med 29:1043-1051, 2003 18. Akobeng AK: Understanding diagnostic tests 3: Receiver operating characteristic curves. Acta Paediatr 96:644-647, 2007 19. Ruopp MD, Perkins NJ, Whitcomb BW, et al: Youden Index and optimal cut-point estimated from observations affected by a lower limit of detection. Biometrical journal. Biometrische Zeitschrift 50:419-430, 2008 20. Gombotz H, Knotzer H: Preoperative identification of patients with increased risk for perioperative bleeding. Curr Opin Anaesthesiol 26:82-90, 2013 21. Toll DB, Janssen KJ, Vergouwe Y, et al: Validation, updating and impact of clinical prediction rules: A review. J Clin Epidemiol 61: 1085-1094, 2008 22. Shehata N, Naglie G, Alghamdi AA, et al: Risk factors for red cell transfusion in adults undergoing coronary artery bypass surgery: A systematic review. Vox Sang 93:1-11, 2007 23. Karkouti K, O'Farrell R, Yau TM, et al: Prediction of massive blood transfusion in cardiac surgery. Can J Anaesth 53:781-794, 2006 24. Karthik S, Grayson AD, McCarron EE, et al: Reexploration for bleeding after coronary artery bypass surgery: Risk factors, outcomes, and the effect of time delay. Ann Thorac Surg 78:527-534; discussion 534, 2004 25. Alghamdi AA, Moussa F, Fremes SE: Does the use of preoperative aspirin increase the risk of bleeding in patients undergoing coronary artery bypass grafting surgery? Systematic review and metaanalysis. J Card Surg 22:247-256, 2007 26. Dunning J, Versteegh M, Fabbri A, et al: Guideline on antiplatelet and anticoagulation management in cardiac surgery. Eur J Cardiothorac Surg 34:73-92, 2008 27. Biancari F, Mikkola R, Heikkinen J, et al: Individual surgeon's impact on the risk of re-exploration for excessive bleeding after coronary artery bypass surgery. J Cardiothorac Vasc Anesth 26: 550-556, 2012 28. Dixon B, Reid D, Collins M, et al: The operating surgeon is an independent predictor of chest tube drainage following cardiac surgery. J Cardiothorac Vasc Anesth 28:242-246, 2014
increased postoperative bleeding when using the Papworth Score on this population. Using the universal definition of perioperative bleeding proposed by Dyke et al, 28% of patients in the Papworth high-risk group exceeded the threshold of excessive bleeding in this population. The local models showed low ability for discrimination (area under the receiver operating characteristics curve o0.75). Addition of the factor “surgeon” or selected intraoperative variables did not substantially improve the models. Conclusion: Prediction of risk for excessive bleeding after cardiac surgery was not possible using clinical variables only, independent of endpoint definition and inclusion of the variable “surgeon” or of selected intraoperative variables. These findings may be due to incomplete understanding of the causative factors underlying excessive bleeding. & 2015 Elsevier Inc. All rights reserved.
S
suggested by Dyke et al8 will be used clinically even if their properties seem attractive. The main aims of the present study, therefore, were (1) to perform external validation of the Papworth Bleeding Risk Score, and (2) to compare the usefulness of the definition of increased bleeding proposed by Dyke et al with that used in the Papworth Bleeding Risk Score. As a secondary aim, a local risk prediction model was developed to identify cardiac surgical patients at the highest risk of severe postoperative bleeding, which permitted investigation of whether prediction could be improved if the variable “surgeon” or some selected intraoperative variables were included.
EVERE BLEEDING after cardiac surgery is a relatively common complication and may occur in as many as 20% of the patients.1 Previous studies have tried to identify preoperative predictors for blood loss after coronary artery surgery, mostly without success. Wahba et al2 found that laboratory testing for hyperfibrinolysis and platelet dysfunction was useful to predict abnormal bleeding after coronary artery bypass grafting (CABG), but except for preoperative platelet count, only the postoperative variables were significant. Point-of-care tests like rotational thromboelastometry (ROTEM) to monitor multiple coagulation parameters represent an interesting alternative, but preoperative ROTEM analysis does not seem useful to predict postoperative hemorrhage in cardiac surgery patients.3,4 Identification of patients at the highest risk of excessive blood loss after cardiac surgery could lead to prophylactic interventions, such as termination of antiplatelet therapy preoperatively, meticulous surgical hemostasis, and prophylactic antifibrinolytic drug treatment. It also could lead to early postoperative treatment with platelets and fresh frozen plasma, coagulation factors, and surgical intervention, if relevant. Patients requiring reexploration for bleeding are at higher risk of adverse outcomes, which increases if time to re-exploration is prolonged.5 In 2011, Vuylsteke et al developed the Papworth Bleeding Risk Score from a prospectively created database of patients undergoing cardiac surgery with cardiopulmonary bypass (CPB).6 The authors included preoperative data from one hospital only, and the score´s external applicability and clinical utility have not been validated. From the field of epidemiology, it is clearly recognized that external validation of any prediction model is of major importance before it is useful in clinical practice.7 Dyke et al recently proposed a universal definition of perioperative bleeding in adult cardiac surgery,8 using a different definition from that used for the Papworth Bleeding Risk Score. The authors hypothesized that, without external validation, neither the Papworth Bleeding Risk Score nor the endpoint
KEY WORDS: risk score, risk prediction, hemorrhage, cardiac surgery, bleeding endpoint
METHODS The study was based on prospectively collected data from 7,137 adult patients undergoing cardiac surgery from 2000 to 2011 at St. Olav
From the Departments of *Circulation and Medical Imaging and †Laboratory Medicine, Children’s and Women’s Health, Faculty of Medicine, Norwegian University of Science and Technology, Trondheim, Norway; ‡Department of Cardiothoracic Anaesthesia and Intensive Care, §Clinic of Anaesthesia and Intensive Care, ║Department of Cardiothoracic Surgery; and ¶Department of Immunology and Transfusion Medicine; St. Olav University Hospital, Trondheim, Norway. Financial support and sponsorship was provided by the Liaison Committee between the Central Norway Regional Health Authority (RHA) and the Norwegian University of Science and Technology (NTNU). Address reprint requests to Guri Greiff, MD, Department of Cardiothoracic Anesthesia and Intensive Care, St. Olav University Hospital, N-7006 Trondheim, Norway. E-mail: [email protected] © 2015 Elsevier Inc. All rights reserved. 1053-0770/2601-0001$36.00/0 http://dx.doi.org/10.1053/j.jvca.2014.08.002
Journal of Cardiothoracic and Vascular Anesthesia, Vol 29, No 2 (April), 2015: pp 311–319
311
312
GREIFF ET AL
University Hospital, Trondheim, Norway. The present study was part of the Cardiac Surgery Outcome Study (CaSOS), and some of the data have been used for development of risk prediction models for mortality, postoperative cardiac dysfunction, prolonged ventilation, prolonged stay in the intensive care unit, acute kidney injury, and prediction of genetic and clinical risk factors for fluid overload after cardiac surgery, as well as for validation of previously published scores for several endpoints.9–14 The CaSOS study was approved by The Regional Research Ethics Committee in Medicine in Trondheim, Norway, and The Norwegian Data Inspectorate. All patients registered in the database underwent cardiac surgery with CPB. Patients treated with acetylsalicylic acid (ASA) used 75 or 160 mg. Clopidogrel was not included in the analysis because of incomplete registration. In the database, clopidogrel was registered from 2008. Before CPB, heparin (300 U/kg; Leo, Copenhagen, Denmark) was given to achieve a kaolin activated coagulation time (ACT; Medtronic Blood Management, Parker, CO) of more than 480 seconds. Additional heparin was given when needed. The perfusion circuit was primed with 1,500 mL of Ringer’s acetate solution with 7,500 U of heparin. A coated membrane oxygenator was used. After CPB, protamine sulfate (Leo, Copenhagen, Denmark) was given to achieve an ACT within 10% of the baseline value. Blood remaining in the CPB circuit was collected and retransfused to the patient. Tranexamic acid (30 mg/kg) was used routinely in the authors’ hospital from 2000 and was given before the start of CPB. High-dose aprotinin was used until 2008 in most patients with postinfarction rupture of the ventricular septum or dissection of the ascending aorta and in some patients with endocarditis, altogether approximately 150 patients (2.1%), but was not registered in the database. The study design is outlined in Figure 1.
Validation of the Papworth Bleeding Risk Score The Papworth Bleeding Risk Score is based on 5 risk factors: Surgery priority, surgery type (CABG or single valve), aortic valve disease, body mass index (BMI), and age.6 On the basis of the calculated scores (0-5 points), patients were divided into the 3 defined risk groups for the Papworth Score: low-risk (0 points), medium-risk
(1-2 points), and high-risk (3-5 points). Matching definitions was possible in the present population for all variables except aortic valve disease, for which aortic valve surgery was used. In the original publication, blood loss exceeding 2 mL/kg/h during the first 3 hours in the intensive care unit (or during a shorter period if the patient underwent transfusion with fresh-frozen plasma, platelets, or cryoprecipitate, reoperation, or died within 3 hours) was considered an adverse outcome. In the database, postoperative drainage volumes were recorded after 4 hours, so this time frame was used in the external validation. Information about transfusion timing in the intensive care unit was missing. The patients who underwent reoperation within 4 hours exceeded the threshold of 2 mL/kg/h, and no patients died within 4 hours because of bleeding. The endpoint used for excessive postoperative blood loss then was defined as blood loss exceeding 2 mL/kg/h the first 4 hours postoperatively. The negative predictive value (NPV) in the Papworth low-risk group was calculated (ie, the proportion of patients without severe bleeding in that group). The positive predictive value (PPV) in the Papworth high-risk group also was calculated (ie, the proportion of patients with severe postoperative bleeding in that group).
Alternative Endpoint Definition The composite bleeding endpoint defined by Dyke et al has 5 bleeding categories based on chest tube drainage during the first 12 postoperative hours: Transfusion of red cells, plasma, and platelets; use of cryoprecipitate, prothrombin complex concentrates, or recombinant activated factor VII; re-exploration or tamponade; and delayed sternal closure.8 The criteria were partly redundant. Cryoprecipitate or prothrombin were never used during the study period in the present population, and recombinant activated factor VII was used for approximately 10 patients, so these criteria were omitted. As in the original paper, the frequency of patients in each bleeding class was assessed (0 ¼ insignificant, 1 ¼ mild, 2 ¼ moderate, 3 ¼ severe, and 4 ¼ massive). Dyke class 3 and 4 patients were defined as having excessive postoperative bleeding and compared with patients from Dyke class 0 - 2. The NPV and PPV for the Papworth Bleeding Risk Score were calculated using this bleeding endpoint instead of the original Papworth definition.
Included patients n=7,137 Incomplete data n=107 (1.5%) Used in study n=7,030 (98.5%)
Calculated Papworth Bleeding Risk Score - compared to original publication (6)
Calculated Papworth Bleeding Risk Score using composite bleeding definition (8)
Developed local risk prediction model
- compared sensitivity, specificity, positive and negative predictive values
- compared Papworth and composite bleeding definitions - compared preoperative and intraoperative models - compared models with and without “surgeon” Fig 1.
Outline of study design.
313
PREDICTION OF BLEEDING AFTER CARDIAC SURGERY
Local Model Development Including Intraoperative Variables For development of the Papworth Bleeding Risk Score, Vuylsteke et al6 used several methods that are not in agreement with current guidelines;15 data were split once into a development and a test set, which reduces future predictive ability,16 variable selection was based on univariate testing, and all continuous variables were dichotomized. For better evaluation of the endpoint definitions and usefulness of intraoperative predictors, an alternative local risk prediction model was developed using recommended statistical methods. This reduces the risk of overfitting; ie, the resulting model becomes less dependent on the exact characteristics of the data used for development and, therefore, more useful in future data sets. A logistic regression model was developed based on 18 preoperative variables and 9 intraoperative explanatory variables (Table 1). The operation types were grouped according to the complexity of the surgical procedures and the risk of bleeding. On the basis of clinical experience and median bleeding, 4 groups were identified by surgical procedures performed; surgical procedures with approximately the same risk and complexity were grouped together. Details on model
development are shown in Appendix 1. Odds ratios and 95% confidence intervals were calculated. Four different models were developed. First, a preoperative risk prediction model for postoperative bleeding after cardiac surgery was developed using the threshold of 2 mL/kg/h of blood loss as the endpoint (model 1). Then, the authors investigated whether this model could be improved by including selected intraoperative variables (model 2). Finally, 2 models were developed (one preoperative and one intraoperative) based on the composite bleeding endpoint by Dyke et al8 (models 3 and 4). Analyses also were performed by adding “surgeon” in the preoperative models using both endpoints, ie, the threshold of 2 mL/kg/h of blood loss and the composite endpoint by Dyke et al. The variable “surgeon” was a categoric variable with 8 categories (details are shown in Appendix 1). Furthermore, models were developed by which renal dysfunction was indicated by creatinine clearance. In these models, variables used during calculation of creatinine clearance were omitted because of strong correlation. The time span for data collection was 11 years. To test whether technical changes influenced the present findings, the year of surgery was added into model 1 as an adjustment variable.
Table 1. Variable Definitions Variable
Preoperative Age Body surface area Smoking preoperatively Chronic pulmonary disease Diabetes mellitus Hypertension Chronic cardiac insufficiency Endocarditis Angina pectoris preoperatively Preoperative renal dysfunction ASA treatment LMWH treatment Warfarin treatment Angiotensin inhibitor use Hemoglobin concentration Previous cardiac surgery Operation urgency Operation type
Intraoperative CPB time Inotropic support Vasoconstrictor use Fluid balance Red blood cell transfusion Plasma transfusion Temperature Hemoglobin concentration Surgeon
Definition
Years (continuous) m2 (continuous) Current smoker or quit o6 months ago (yes/no) Use of bronchodilating agents or FEV1 below 75% (yes/no) Receiving medication (yes/no) Receiving medication or diastolic blood pressure above 90 mmHg (yes/no) Receiving medication (yes/no) Receiving antibiotic treatment (yes/no) Angina or treatment for angina pectoris preoperatively Serum creatinine 4140 mmol/L or dialysis (yes/no) Treated with acetylsalicylic acid (yes/no) Treated with low-molecular-weight heparin (yes/no) Treated with warfarin (yes/no) Treated with angiotensin converting enzyme inhibitor/ angiotensin II receptor blocker (yes/no) g/dL (continuous) (yes/no) 0 ¼ standard waiting list. 1 ¼ surgery within 1 week. 2 ¼ surgery within 24 hours 1: CABG or single valve surgery. 2: Combined AVR and CABG or combined MVR and CABG. 3: Miscellaneous (Closure of an atrial septum defect or ventricular septum defect, closure of ventricular septum rupture, AVR and MVR in combination with procedures other than CABG, operation for aneurysm of the ascending aorta, and other cardiac surgery like pericardiectomy and removal of cardiac tumors). 4: Dissection of the ascending aorta. Cardiopulmonary bypass time, minutes (continuous) For clinical indication during surgery (yes/no) For clinical indication during surgery (yes/no) Intraoperative fluid balance For clinical indication during surgery (yes/no) For clinical indication during surgery (yes/no) Lowest temperature intraoperatively Lowest hemoglobin concentration intraoperatively, g/dL (continuous) 2 categories, surgeons with equivalent amounts of postoperative bleeding (tested by analyses of variance) were grouped together
Abbreviations: ACE, angiotensin converting enzyme; ASA, acetylsalicylic acid; AVR, aortic valve replacement; CABG, coronary artery bypass grafting; CPB, cardiopulmonary bypass; FEV1, forced expired volume in 1 second; LMWH, low-molecular-weight heparin; MVR, mitral valve replacement.
314
GREIFF ET AL
Model discrimination was assessed by receiver operating characteristics (ROC) curves. An area under the curve (AUC) from 0.7 to 0.9 indicates moderate accuracy.17 Calibration of the models, ie, whether calculated risk was similar to observed risk, was assessed by the Hosmer-Lemeshow test and calibration curves (Appendix 1). With the Hosmer-Lemeshow test, a p value 40.05 indicates adequate calibration. Sensitivity is the probability of truly identifying patients with the endpoint using a certain cut-off of predicted probability, and specificity is the probability of truly identifying patients without the endpoint using the cut-off.18 NPV and PPV were calculated in the low-risk and high-risk groups at the best cut-off, identified by the Youden index19 (Appendix 1).
General Statistics Data are shown as mean with 95% confidence interval or frequencies with percentages, unless otherwise stated. For intergroup comparisons, Pearson´s chi-square test or a t-test was used. The statistical software packages SPSS (version 20.0, SPSS, Chicago, IL), R (version 3. 0.0, R Foundation, http://www.r-project.org), and SigmaPlot (version 12.0; Systat Software, San Jose, CA) were applied. All p values below 0.05 were considered statistically significant. RESULTS
Patient Characteristics A total of 7,137 patients were included, and complete data sets (7,030 patients, 98.5%) were used for model validation and comparison. Patient characteristics and other clinical variables are shown in Table 2. According to the endpoint defined by a threshold of 2 mL/kg/h of blood loss, 7.5% (527 of 7,030) of the patients had excessive postoperative bleeding. According to the composite definition by Dyke et al,8 1,258 patients (17.6%) had excessive postoperative bleeding. A total of 445 patients (6.4%) had excessive bleeding by both definitions, 82 patients (1.2%) had excessive bleeding only by the 2 mL/kg/h definition, and 813 patients (11.2%) had excessive bleeding only by the composite definition. Patients with a high Papworth Bleeding Risk Score had a higher incidence of severe postoperative bleeding compared with patients with a lower score; however, the incidence of severe bleeding was lower in the present population for the 3 risk groups defined by the Papworth Bleeding Risk Score (Table 3), with 15% of the patients in the high-risk group exceeding the bleeding threshold compared with 21% in the original population.6 Using the composite endpoint definition by Dyke et al,8 28% of the patients in the high Papworth Bleeding Risk Score group exceeded the threshold for excessive bleeding (Table 3). PPV and NPV for the Papworth Bleeding Risk Score using the 2 alternative endpoint definitions are shown in Table 3. Comparisons using novel prediction models (Logistic regression models), without the surgeon using the endpoint defined by a threshold of 2 mL/kg/h of blood loss, are shown in Table 4. ASA, lower body surface area, urgent operation, and operation type other than isolated CABG or single-valve surgery were significant predictors for excessive postoperative blood loss in model 1. Model 2 also included intraoperative variables, of which only intraoperative inotropic support and cardiopulmonary bypass time (CPB) were significant (Table 4). In this model, urgent surgery within 1 week and dissection of the ascending aorta were no longer significant. Both models had adequate calibration (Hosmer-Lemeshow test, p ¼ 0.37 for model 1 and p ¼ 0.09 for model 2). The ROC
curves showed poor-to-moderate discrimination, with AUC of 0.688 (0.664-0.712) for model 1 and 0.709 (0.698-0.743) for model 2 (p ¼ 0.008). Using creatinine clearance to define renal dysfunction did not improve model 1. The AUC for this model was 0.672 (0.648-0.696), ie, lower than for model 1; therefore, serum creatinine was retained as a simpler variable for renal dysfunction in further analyses. Inclusion of year of surgery as an adjustment variable had minor effects on the odds ratio for the variables in model 1. The models using the composite endpoint by Dyke et al8 are shown in Table 4. In these models, body surface area and surgery urgency were no longer significant; instead, preoperative renal dysfunction, previous cardiac surgery, and use of low-molecular-weight heparin were significant, in addition to ASA and operation type. The AUC were 0.739 (0.722-0.755) and 0.761 (0.745-0.777). The preoperative model had adequate calibration (Hosmer-Lemeshow test, p ¼ 0.57), but the intraoperative model was not well calibrated (p ¼ 0.04). For the preoperative model using the 2 mL/kg/h bleeding threshold endpoint (model 1), the NPV was 0.95 in the low-risk group and the PPV was 0.12 in the high-risk group, when dividing patients into 2 risk groups at the best cut-off identified by the Youden index (Appendix 1). When using the composite endpoint by Dyke et al (model 3), the NPV was 0.89 in the low-risk group and the PPV was 0.44 in the high-risk group. Surgeon was a significant predictor for excessive postoperative bleeding in the preoperative models independent of endpoint definition (p o 0.001). Calibration still was adequate (Hosmer-Lemeshow test: model 1, p ¼ 0.39; model 3, p ¼ 0.23). The AUC increased for both preoperative models with surgeon (model 1: AUC ¼ 0.713 (0.691-0.736), p o 0.001); model 3: AUC ¼ 0.757 (0.742-0.773); po0.001), but discrimination was still only moderate. Bleeding during the first 4 postoperative hours was highly overlapping among the 10 deciles of risk independent of endpoint definition (Fig 2). DISCUSSION
In this study investigating prediction models for excessive postoperative bleeding after cardiac surgery, neither the Papworth Bleeding Risk Score nor novel models based on the same endpoint definition or the Dyke et al composite endpoint definition were clinically useful. Either discrimination was too low or calibration was inadequate even when using state-of-theart methods for model development. Addition of the factor “surgeon” or selected intraoperative variables did not improve the models substantially. Severe bleeding after cardiac surgery is still a major problem, and better prediction models are needed.20 The published Papworth Bleeding Risk Score6 showed slightly worse performance in the present population, which is a common finding due to overfitting during score development and differences in patient characteristics.21 The authors’ local model developed using stricter statistical criteria and internal validation showed the same pattern of a high NPV but very low PPV. Thus, both scores will more precisely identify patients at low risk of bleeding. The models based on the Dyke et al8 endpoint definition had a
315
PREDICTION OF BLEEDING AFTER CARDIAC SURGERY
Table 2. Patient and Surgical Characteristics Bleeding o2 mL/kg/h
Characteristic
Age (years) Body surface area Smoking preoperatively Diabetes mellitus Hypertension Chronic cardiac insufficiency Endocarditis Angina pectoris preoperatively Chronic pulmonary disease Renal function Normal renal function Preoperative renal dysfunction with creatinine above 140 (mmol/L) or dialysis Previous cardiac surgery ACE inhibitor use Warfarin preoperatively Acetylsalicylic acid preoperatively Low-molecular-weight heparin preoperatively Hemoglobin preoperatively (g/dL) Hemoglobin intraoperatively (g/dL) Operation urgency Normal priority Operation within 1 week Emergency operation within 24 hours Operation type CABG or single-valve surgery AVR and CABG, MVR and CABG Miscellaneous Dissection of ascending aorta CPB time Lowest temperature intraoperatively Inotropic support Vasoconstrictor use Fluid balance (mL) Red blood cell transfusion Plasma transfusion
the First 4 Hours
Bleeding Z2 mL/kg/h
Bleeding Classes 0-2
Bleeding Classes 3-4
Postoperatively
the First 4 Hours
Defined by Dyke et al8
Defined by Dyke et al8
(n ¼ 6,518)
Postoperatively (n ¼ 527)
66.2 1.98 1,609 908 3,488 952 101 2,186 947
(65.9-66.4) (1.97-1.98) (24.7%) (13.9%) (53.5%) (14.6%) (1.5%) (33.5%) (14.5%)
67.6 1.89 105 59 270 118 19 178 77
(66.5-68.5) (1.88-1.90) (19.9%) (11.9%) (51.2%) (22.4%) (3.6%) (33.8%) (14.6%)
6,231 (95.6%) 287 (4.4%)
489 (92.8%) 38 (7.2%)
355 2,246 544 4,705 2,099
42 194 45 392 183
(5.4%) (34.5%) (8.3%) (72.2%) (32.2%)
13.7 (13.7-13.8) 10.1 (10.1-10.2) 3,523 (54.1%) 2,687 (41.25%) 307 (4.7%)
(8.0%) (36.8%) (8.5%) (74.4%) (34.7%)
13.3 (13.2-13.5) 9.7 (9.6-9.8)
p Value
0.008 o0.001 0.022 0.08 0.31 o0.001 o0.001 0.91 0.95 0.012
(n ¼ 5,823)
66.1 1.96 1,450 815 3,136 810 73 1,960 823
(65.9-66.4) (1.96-1.97) (25.2%) (14,0%) (54,0%) (13.9%) (1.3%) (33.7%) (14.2%)
5,570 (95.9%) 240 (4.1%)
0.016 0.25 0.87 0.25 0.21 o0.001 o0.001 o0.001
237 (43.5%) 229 (43.5%) 61 (11.6%)
1,993 421 4,198 1,845
0 (34.3%) (7.3%) (72.3%) (31.8%)
13.7 (13.7-13.8) 10.1 (10.1-10.2) 3,152 (54.3%) 2,385 (41.0%) 273 (4.7%)
(n ¼ 1,258)
66.6 (66.4-67.2) 1.96 (1.95-1.98) 274 (22.1%) 159 (12.5%) 638 (50.2%) 274 (21.6%) 54 (4.3%) 424 (32.6%) 213 (16.8%)
(77.9%) (10.5%) (10.5%) (1.2%) (80-82) (33.3-33.4) (23.0%) (82.3%) (2,905-2,954) (16.1%) (11.4%)
306 96 100 25 106 32.4 204 452 3,259 178 111
(58.1%) (18.2%) (19.0%) (4.7%) (101-110) (32.1-32.7) (38.7%) (85.8%) (3,166-3,401) (33.8%) (21.1%)
o0.001 o0.001 o0.001 0.047 o0.001 o0.001 o0.001
0.18 0.97 0.022 0.16 0.015 o0.001 o0.001 0.45 0.018 o0.001
1,173 (92.4%) 97 (7.0%)
410 460 180 909 183
(32.3%) (36.3%) (14.2%) (71.7%) (34.7%)
13.4 (13.4-13-6) 9.8 (9.8-9.9)
o0.001 0.18 o0.001 0.68 0.037 o0.001 o0.001 o0.001
611 (48.1%) 532 (41.9%) 127 (10.0%)
o0.001 5,077 684 682 75 81 33.4 1,556 5,366 2,930 1,049 742
p Value
o0.001 4,606 579 561 65 78 33.4 1,286 4,729 2,886 880 556
(79.3%) (10.0%) (9.7%) (1.1%) (77-79) (33.4-33.5) (22.1%) (81.4%) (2,863-2,908) (15.1%) (9.6%)
778 209 234 48 110 32.3 506 1,118 3,349 385 330
(61.3%) (16.5%) (18.4%) (3.9%) (108-114) (32.2-32.5) (39.8%) (88.0%) (3,263-3,435) (30.3%) (26.0%)
o0.001 o0.001 o0.001 o0.001 o0.001 o0.001 o0.001
NOTES. Data are shown as mean (95% confidence interval) or frequency (percentage). All p values were obtained using Pearson´s chi-square test or a t test, as appropriate. Abbreviations: ACE, angiotensin-coverting enzyme; AVR, aortic valve replacement; CABG, coronary artery bypass grafting; CPB, cardiopulmonary bypass; MVR, mitral valve replacement.
slightly higher PPV (0.28 in the high Papworth Bleeding Risk Score group and 0.44 in the local model high-risk group) and, therefore, could identify more patients experiencing excessive bleeding, but still far from the majority. However, calculating the score could give some relevant information. The low-risk group of patients needs less special care or preoperative testing. Furthermore, they may not need prophylactic treatment and, thereby, may avoid the risk of side effects of such treatment. If a patient in the low-risk group bleeds excessively, the likelihood of a surgical cause may be higher and reoperation may be performed earlier. The models also may identify groups of patients to be included in forthcoming studies.
The low ability to identify the patients at high risk of excessive bleeding using the prediction models probably is related to the large variation in amount of postoperative bleeding in all risk categories. Even if the proportion of patients with excessive postoperative blood loss increased with increasing risk score for both endpoints, the large overlap in bleeding volumes illustrated in Figure 2 indicates why accurate prediction was difficult. The underlying reason is probably that not all the factors resulting in excessive blood loss are yet identified. Increased knowledge about the multifactorial pathophysiology of bleeding in the setting of cardiac surgery is needed, so that other variables explaining more of the variation may be used for model development. The authors hypothesized
316
GREIFF ET AL
Table 3. Papworth Bleeding Risk Score, Comparing Original Population With Present Population Papworth Bleeding Risk Score Risk Groups
Original population* Patients with excessive blood loss Total number of patients Present population† Patients with excessive blood loss Total number of patients Present population, alternative endpoint definition‡ Patients with excessive blood loss Total number of patients
Low Risk
Medium Risk
High Risk
60 (3%)§ 2143 (31%)¶ NPV 0.97
287 (8%)§ 3587 (53%)¶
226 (21%)§ 1083 (16%)¶ PPV 0.21
28 (2%)§ 1396 (20%)¶ NPV 0.98
293 (7%)§ 4269 (61%)¶
203 (15%)§ 1367 (19%)¶ PPV 0.15
141 (10%)§ 1390 (20%)¶ NPV 0.90
726 (17%)§ 4274 (60%)¶
391 (28%)§ 1385 (20%)¶ PPV 0.28
Abbreviations: NPV, negative predictive value for low-risk group compared with high-risk group; PPV, positive predictive value for high-risk group compared with low-risk group. *Vuylsteke et al: The Papworth Bleeding Risk Score.6 †Blood loss exceeding 2 mL/kg/h the first 4 hours postoperatively. ‡Dyke et al: Bleeding categories including total blood loss from chest tube within 12 postoperative hours, transfusion of red cells, plasma, and platelets, surgical reexploration, tamponade.8 §Percent of the total number of patients within the risk group. ¶Percent of the total number of patients.
that factors related to microvascular bleeding and coagulation at the endothelial level as well as genetic variation may be important. This also might explain why certain patients on platelet inhibitors tend to bleed heavily. These hypotheses need further investigation. There are several differences with respect to endpoint definitions among studies trying to predict which patients will develop severe postoperative bleeding; therefore, 2 endpoint definitions were compared. The rate of bleeding per kg body weight for the first 4 hours postoperatively, which is almost equivalent to the Papworth definition, is attractive because it avoids some factors related to different traditions and department routines. The recently published definition by Dyke et al is based on factors that would be expected to be more strongly influenced by transfusion policies that vary among institutions and surgical teams, and change over time. It expands on previously used variables including red cell transfusion22,23 and need for surgical re-exploration.24 By pooling only the 2 classes with highest postoperative bleeding when defining the endpoint, some of the subjectivity in the classification probably became less influential. Even if the resulting prediction model 3 had a somewhat higher AUC and PPV, the improvement was too small to be clinically relevant. The effect of different endpoint definitions clearly was demonstrated by the small proportion of patients categorized as having excessive bleeding by both; the choice of endpoints, therefore, is not obvious. On the basis of the present study, however, none of the endpoint definitions clearly was preferable. The literature supports the significance of several risk factors identified in the present study.6 Antiplatelet therapy routinely is used in patients requiring cardiac surgery. The consistent finding of ASA as significant for prediction of postoperative bleeding is in accordance with data relating
ASA to severe bleeding after cardiac and noncardiac surgery.25,26 A meta-analysis concluded that ASA was associated with increased chest tube drainage and possibly greater requirement for blood products.25 The dose of ASA may play a role. The present patients used 75 or 160 mg, which is a relatively low dose, but still ASA was associated with postoperative bleeding. However, the decision of whether to stop antiplatelet therapy is an area of debate. Stopping may represent a risk in patients with coronary artery stents or unstable eccentric coronary artery lesions, but guidelines on antiplatelet management in cardiac surgery recommend that patients should stop ASA 2 to 10 days before surgery to reduce perioperative blood loss.26 The present findings support this recommendation. In the present study, surgeon was a significant predictor of severe bleeding after cardiac surgery, even if the model adjusted for type of surgery. Others also have found that the surgeon has a major impact on postoperative bleeding,6,27,28 which suggests that improving surgical skills may be important. However, surgeon always will be a local variable and cannot be included in a prediction model for general use. Furthermore, the inclusion of surgeon in the present models only modestly improved the predictive ability. It is important to note that the factors retained in a parsimonious risk prediction model are those best reflecting the variety of information from different underlying causal factors. They are, therefore, not necessarily the direct causal factors themselves. This explains why variables easily identified as related to the outcome may not remain in the model and why models based on different endpoint definitions contain slightly different factors. Even so, some factors were common to the models based on both endpoints and had comparable odds ratios as well (ASA, surgery type, CPB time, and inotropic support), and either may be causal or strongly associated with underlying causal factors.
317
PREDICTION OF BLEEDING AFTER CARDIAC SURGERY
Table 4. Models for Prediction of Postoperative Blood Loss Variable
Model 1 Body surface area Acetylsalicylic acid Operation urgency Normal priority Within 1 week Emergency operation within 24 hours Operation type CABG or single-valve surgery AVR and CABG, MVR and CABG Miscellaneous Dissection of ascending aorta Intercept Model 2 Body surface area Acetylsalicylic acid Operation urgency Normal priority Within 1 week Emergency operation within 24 hours Operation type CABG or single-valve surgery AVR and CABG, MVR and CABG Miscellaneous Dissection of ascending aorta Cardiopulmonary bypass time (per 10 min) Inotropic support Intercept Model 3 Acetylsalicylic acid Preoperative renal dysfunction Previous cardiac surgery Low-molecular-weight heparin Operation type CABG or single-valve surgery AVR and CABG, MVR and CABG Miscellaneous Dissection of ascending aorta Intercept Model 4 Acetylsalicylic acid Previous cardiac surgery Low-molecular-weight heparin Operation type CABG or single-valve surgery AVR and CABG, MVR and CABG Miscellaneous Dissection of ascending aorta Cardiopulmonary bypass time (per 10 min) Inotropic support Intercept
Coefficient
Odds Ratio
95% Confidence Interval
p Value
–2.077 0.388
0.125 1.474
(0.080-0.195) (1.197-1.816)
o0.001 0.001
0.250 0.812
1.000 1.290 2.252
Reference (1.054-1.564) (1.554-3.264)
0.0124 o0.001
1.000 2.306 2.858 3.772
Reference (1.797-2.959) (2.229-3.667) (2.116-6.725)
o0.001 o0.001 o0.001
–2.154 0.377
0.116 1.458
(0.075-0.178) (1.153-1.844)
o0.001 0.001
0.197 0.682
1.00 1.218 1.977
Reference (0.991-1.498) (1.390-2.813)
0.085 0.001
1.00 1.487 1.745 1.681 1.092 1.430
Reference (1.115-1.984) (1.287-2.367) (0.853-3.315) (1.071-1.114) (1.161-1.765)
0.007 o0.001 0.134 o0.001 0.001
1.576 1.620 58.520 1.418
(1.209-2.171) (1.209-2.171) (42.244-81.075) (1.213-1.658)
o0.001 0.001 o0.001 o0.001
1.00 2.582 2.156 7.349
Reference (2.117-3.149) (1.733-2.681) (4.869-11.091)
o0.001 o0.001 o0.001
1.637 44.807 1.406
(1.363-1.965) (32.218-62.315) (1.200-1.646)
o0.001 o0.001 o0.001
1.00 1.508 1.200 2.359 1.087 1.313
Reference (1.193-1.907) (0.925-1.557) (1.430-3.890) (1,065-1.111) (1.119-1.542)
0.001 0.170 0.001 o0.001 0.001
0.835 1.050 1.328 0.729
0.397 0.557 0.520 0.088 0.358 0.301 0.455 0.482 4.069 0.349
0.948 0.768 1.995 –2.632 0.493 3.802 0.340
0.411 0.182 0.852 0.084 0.273 –3.265
NOTE. Models 1 and 2: endpoint was bleeding 42 mL/kg/h the first 4 postoperative hours. Models 3 and 4: endpoint was class 3 or 4 bleeding by Dyke et al.8 Abbreviations: AVR, aortic valve replacement; CABG, coronary artery bypass grafting; MVR, mitral valve replacement.
Few of the published prediction models for bleeding after cardiac surgery are ever validated in other populations. In the present study, the Papworth Bleeding Risk Score was not only validated externally, but the authors also tried to improve prediction by using alternative statistical methods and
redefining the endpoint. This approach supported that the findings were not random, but generally were representative of prediction models based on standard clinical variables. The findings regarding PPV and NPV further illustrated that reporting model discrimination and calibration alone may
318
GREIFF ET AL
Fig 2. Median bleeding volume (left y-axis), Number of patients of class 3 and 4 by Dyke et al8 (first right y-axis), and number of patients exceeding blood loss 42 mL/kg/h for the first 4 hours postoperatively (second right y-axis). The error bars represent 25th and 75th percentiles.
render it difficult to fully evaluate the clinical usefulness of any prediction model. The present study was based on a single-center database and local factors may have influenced the results, but this would not change the direct comparisons among the models. Changes in patient characteristics, surgical techniques, and treatment strategies may influence the calibration of the present models, because the database contained data from 11 years. However, the surgeons and the surgical teams have been almost the same during these years and inclusion of the year of surgery gave very small changes to the model. The authors, therefore, believe that the results are valid despite the long data collection period. CONCLUSION
Neither the Papworth Bleeding Risk Score nor the present prediction models were suitable for identifying the patients at high risk of severe postoperative bleeding after cardiac surgery, independent of endpoint definition. Clinical variables alone probably are not sufficient to identify such patients, and the models could better identify patients at low risk of bleeding. Few patients were characterized as having excessive postoperative bleeding by both definitions, indicating that the choice of definition is important. The surgeon has an impact on postoperative bleeding but inclusion of this variable did not improve prediction much. ASA was associated with postoperative bleeding in all models. External validation of risk prediction models is important to ensure clinical usefulness. APPENDIX 1
overfitting compared with univariate screening of candidate variables. A full model including all the selected variables was fitted first, tested for predefined interactions, linearity in the logit for continuous variables, and overly-influential observations. Model reduction then was performed using limited backward step-down, keeping variables according to Akaike’s Information Criterion. To find more robust estimates of coefficients, bootstrap estimates of the covariance matrix for the regression coefficients were computed (n ¼ 400).16 The Hosmer-Lemeshow test was used to assess calibration. A nonsignificant test indicated small differences between observed and predicted values, ie, adequate goodness-of-fit. Internal validation was performed by calibration curves comparing observed and predicted probabilities after bootstrapping (n ¼ 400). The shrinkage factor also was calculated, giving an estimate of prediction accuracy in future datasets. A shrinkage factor above 0.85 is considered satisfactory, indicating that the model will predict with an estimated error of less than 15% in a future dataset. Sensitivity and specificity are related inversely according to the choice of cut-off point of the predicted probability. To determine the best cut-off , the Youden index was used (¼ maxc) (sensitivityc þ specificityc – 1); that is, the ROC curve point that maximizes the sum of sensitivity and specificity.18,19 The operations were performed by 7 surgeons in addition to a group of substitutes performing only a few operations each. The substitutes were pooled into one group, resulting in a categoric variable with 8 surgeons. This 8-level categoric variable “surgeon” then was added in the logistic regression model.
Additional Methods
Additional Results
The authors used the entire dataset for logistic regression development. Eighteen preoperative variables and 9 intraoperative explanatory variables (Table 1) were selected based on clinical knowledge and the literature, reducing the risk of
The shrinkage factor for the preoperative model using the threshold of 2 mL/kg/h of blood loss as endpoint (model 1) was 0.91, indicating that it will predict postoperative bleeding in a future dataset with an estimated error of 9%. The shrinkage
319
PREDICTION OF BLEEDING AFTER CARDIAC SURGERY
factor for the intraoperative model using the same endpoint (model 2) was 0.89. Using the endpoint defined by Dyke et al, the shrinkage factors were 0.98 and 0.99 for the preoperative (model 3) and intraoperative models (model 4), respectively. On the basis of the Youden index, the best cut-off for the preoperative model using the 2 mL/kg/h bleeding threshold endpoint (model 1) was at a predicted probability of excessive bleeding of 7%, giving a sensitivity of 65% and a specificity of
63%. A low-risk and a high-risk group were defined at this cutoff value, and the NPV was 0.95 in the low-risk group and the PPV was 0.12 in the high-risk group. When using the composite endpoint by Dyke et al, the best cut-off for the preoperative model (model 3) was at a predicted probability of 19%. At this cut-off, sensitivity was 50%, specificity was 86%, the NPV was 0.89 in the low-risk group and the PPV was 0.44 in the high-risk group.
REFERENCES 1. Paparella D, Brister SJ, Buchanan MR: Coagulation disorders of cardiopulmonary bypass: A review. Intensive Care Med 30:1873-1881, 2004 2. Wahba A, Rothe G, Lodes H, et al: Predictors of blood loss after coronary artery bypass grafting. J Cardiothorac Vasc Anesth 11: 824-827, 1997 3. Lee GC, Kicza AM, Liu KY, et al: Does rotational thromboelastometry (ROTEM) improve prediction of bleeding after cardiac surgery? Anesth Analg 115:499-506, 2012 4. Davidson SJ, McGrowder D, Roughton M, et al: Can ROTEM thromboelastometry predict postoperative bleeding after cardiac surgery? J Cardiothorac Vasc Anesth 22:655-661, 2008 5. Choong CK, Gerrard C, Goldsmith KA, et al: Delayed reexploration for bleeding after coronary artery bypass surgery results in adverse outcomes. Eur J Cardiothorac Surg 31:834-838, 2007 6. Vuylsteke A, Pagel C, Gerrard C, et al: The Papworth Bleeding Risk Score: A stratification scheme for identifying cardiac surgery patients at risk of excessive early postoperative bleeding. Eur J Cardiothorac Surg 39:924-930, 2011 7. Bleeker SE, Moll HA, Steyerberg EW, et al: External validation is necessary in prediction research: a clinical example. J Clin Epidemiol 56:826-832, 2003 8. Dyke C, Aronson S, Dietrich W, et al: Universal definition of perioperative bleeding in adult cardiac surgery. J Thorac Cardiovasc Surg 147:1458-1463, 2014 9. Berg KS, Stenseth R, Pleym H, et al: Mortality risk prediction in cardiac surgery: Comparing a novel model with the EuroSCORE. Acta Anaesthesiol Scand 55:313-321, 2011 10. Widyastuti Y, Stenseth R, Wahba A, et al: Length of intensive care unit stay following cardiac surgery: Is it impossible to find a universal prediction model? Interact Cardiovasc Thorac Surg 15: 825-832, 2012 11. Widyastuti Y, Stenseth R, Pleym H, et al: Pre-operative and intraoperative determinants for prolonged ventilation following adult cardiac surgery. Acta Anaesthesiol Scand 56:190-199, 2012 12. Widyastuti Y, Stenseth R, Berg KS, et al: Preoperative and intraoperative prediction of risk of cardiac dysfunction following open heart surgery. Eur J Anaesthesiol 29:143-151, 2012 13. Berg KS, Stenseth R, Wahba A, et al: How can we best predict acute kidney injury following cardiac surgery? A prospective observational study. Eur J Anaesthesiol 30:704-712, 2013 14. Enger TB, Pleym H, Stenseth R, et al: Genetic and clinical risk factors for fluid overload following open-heart surgery. Acta Anaesthesiol Scand 58:539-548, 2014
15. Moons KG, Kengne AP, Woodward M, et al: Risk prediction models: I. Development, internal validation, and assessing the incremental value of a new (bio)marker. Heart 98:683-690, 2012 16. Harrell FE Jr., Lee KL, Mark DB: Multivariable prognostic models: Issues in developing models, evaluating assumptions and adequacy, and measuring and reducing errors. Stat. Med 15:361-387, 1996 17. Fischer JE, Bachmann LM, Jaeschke R: A readers' guide to the interpretation of diagnostic test properties: Clinical example of sepsis. Intensive Care Med 29:1043-1051, 2003 18. Akobeng AK: Understanding diagnostic tests 3: Receiver operating characteristic curves. Acta Paediatr 96:644-647, 2007 19. Ruopp MD, Perkins NJ, Whitcomb BW, et al: Youden Index and optimal cut-point estimated from observations affected by a lower limit of detection. Biometrical journal. Biometrische Zeitschrift 50:419-430, 2008 20. Gombotz H, Knotzer H: Preoperative identification of patients with increased risk for perioperative bleeding. Curr Opin Anaesthesiol 26:82-90, 2013 21. Toll DB, Janssen KJ, Vergouwe Y, et al: Validation, updating and impact of clinical prediction rules: A review. J Clin Epidemiol 61: 1085-1094, 2008 22. Shehata N, Naglie G, Alghamdi AA, et al: Risk factors for red cell transfusion in adults undergoing coronary artery bypass surgery: A systematic review. Vox Sang 93:1-11, 2007 23. Karkouti K, O'Farrell R, Yau TM, et al: Prediction of massive blood transfusion in cardiac surgery. Can J Anaesth 53:781-794, 2006 24. Karthik S, Grayson AD, McCarron EE, et al: Reexploration for bleeding after coronary artery bypass surgery: Risk factors, outcomes, and the effect of time delay. Ann Thorac Surg 78:527-534; discussion 534, 2004 25. Alghamdi AA, Moussa F, Fremes SE: Does the use of preoperative aspirin increase the risk of bleeding in patients undergoing coronary artery bypass grafting surgery? Systematic review and metaanalysis. J Card Surg 22:247-256, 2007 26. Dunning J, Versteegh M, Fabbri A, et al: Guideline on antiplatelet and anticoagulation management in cardiac surgery. Eur J Cardiothorac Surg 34:73-92, 2008 27. Biancari F, Mikkola R, Heikkinen J, et al: Individual surgeon's impact on the risk of re-exploration for excessive bleeding after coronary artery bypass surgery. J Cardiothorac Vasc Anesth 26: 550-556, 2012 28. Dixon B, Reid D, Collins M, et al: The operating surgeon is an independent predictor of chest tube drainage following cardiac surgery. J Cardiothorac Vasc Anesth 28:242-246, 2014