- Email: [email protected]
Risk stratification for surgical site infections in Australia: evaluation of the US National Nosocomial Infection Surveillance risk index
Journal of Hospital Infection (2007) 66, 148e155
www.elsevierhealth.com/journals/jhin
Risk stratification for surgical site infections in Australia: evaluation of the US National Nosocomial Infection Surveillance risk index A.C.A. Clements a,b,*, E.N.C. Tong b, A.P. Morton b, M. Whitby b a
Division of Epidemiology and Social Medicine, School of Population Health, University of Queensland, Brisbane, Australia b Centre for Hospital Related Infection Surveillance and Prevention, Princess Alexandra Hospital, Brisbane, Australia Received 9 November 2006; accepted 23 February 2007 Available online 9 May 2007
KEYWORDS Surgical site infection; Risk index; Risk stratification; Risk factors; Surveillance
Summary This study evaluated the US National Nosocomial Infection Surveillance (NNIS) risk index (RI) in Australia for different surgical site infection (SSI) outcomes (overall, in-hospital, post-discharge, deep-incisional and superficial-incisional infection) and investigated local risk factors for SSI. A SSI surveillance dataset containing 43 611 records for 13 common surgical procedures, conducted in 23 hospitals between February 2001 and June 2005, was used for the analysis. The NNIS RI was evaluated against the observed SSI data using diagnostic test evaluation statistics (sensitivity, specificity, positive predictive value, negative predictive value). Sensitivity was low for all SSI outcomes (ranging from 0.47 to 0.69 and from 0.09 to 0.20 using RI thresholds of 1 and 2 respectively), while specificity varied depending on the RI threshold (0.55 and 0.93 with thresholds of 1 and 2 respectively). Mixed-effects logistic regression models were developed for the five SSI outcomes using a range of available potential risk factors. American Society of Anaesthesiologists (ASA) physical status score >2, duration of surgery, absence of antibiotic prophylaxis and type of surgical procedure were significant risk factors for one or more SSI outcomes, and risk factors varied for different SSI outcomes. The discriminatory ability of the NNIS RI was insufficient for its use as an accurate risk stratification tool for SSI surveillance in Australia and its sensitivity was too low for it to be appropriately used as a prognostic indicator. ª 2007 The Hospital Infection Society. Published by Elsevier Ltd. All rights reserved.
* Corresponding author. Address: Division of Epidemiology and Social Medicine, School of Population Health, University of Queensland, Herston Road, Brisbane, Queensland 4006, Australia. Tel.: þ61 7 3240 5952; fax: þ61 7 3240 5540. E-mail address: [email protected] 0195-6701/$ - see front matter ª 2007 The Hospital Infection Society. Published by Elsevier Ltd. All rights reserved. doi:10.1016/j.jhin.2007.02.019
Stratifying risk of surgical infections
Introduction Quality of healthcare in hospitals is of major public health importance. Surgical site infections (SSI) are some of the most common hospital-acquired infections, contributing to higher patient mortality, significantly longer length of stay and additional treatment costs.1,2 The Centre for Healthcare Related Infection Surveillance and Prevention (CHRISP) has instituted surveillance for SSI in Queensland, Australia, incorporating standardized case-definitions and risk-indexing of surgical patients based on the US Centres for Disease Control and Prevention (CDC) National Nosocomial Infections Surveillance (NNIS) protocols.3e5 The NNIS risk index (RI) has been used to stratify SSI rates for comparison and has been suggested as a prognostic tool for targeting SSI prevention methods at high-risk patients.1,3,6 A number of studies have investigated risk factors for SSI, either in the context of evaluating the NNIS RI and comparing it to alternative riskindexing approaches in local settings, or gaining an improved understanding of the epidemiology of SSI in different scenarios. Notably, while increasing NNIS RI scores have been shown to be associated with increasing risk of SSI and to be an independent predictor of SSI in multivariable models, two studies that compared the NNIS RI with locally developed RIs found the local indices to have better discriminatory performance and a third study showed that the NNIS RI was reliable for some surgical procedures but not for others.5,7e11 A recent report suggested that the NNIS RI performed well in the Australian state of Victoria as a risk stratification tool.12 While demonstrating significant correlation between NNIS RI score and incidence of infection, discriminatory performance of the index was not investigated. We aimed to evaluate the discriminatory performance of the NNIS RI in our local setting, both for overall SSI and for different SSI outcomes (in-hospital, post-discharge, superficial and deep-incisional infection). We also aimed to investigate risk factors for SSI in Australia, from a set of available potential risk factors, as preliminary work towards the development of local SSI indices.
Methods Data collection Routine SSI surveillance data were collected on a voluntary basis for 13 common surgical procedures, from 23 medium-to-large hospitals in
149 Queensland, using an electronic surveillance software tool, the Electronic Infection Control Assessment Technology version 4.2. (eICAT, CHRISP, Brisbane, Australia). Not all hospitals contributed data for all of the surgical procedures monitored. The SSI surveillance database contained records on 43 611 surgical procedures, conducted on 42 226 patients between February 2001 and June 2005. In addition to a range of SSI outcomes as defined by the CDC (whether an infection occurred, whether it was detected in-hospital or post-discharge and whether the infection was superficial-incisional or deepincisional/organ/space), procedure- and patientrelated information was also stored (Table I).4,13 Case-finding for post-discharge infection varied between hospitals, but the most common approach relied on questionnaires posted to patients and follow-up of self-reported cases with the patients’ general practitioners. Missing values occurred for varying numbers of records for each variable, with the American Society of Anesthesiologists (ASA) physical status score having the largest number of missing values. Missing values were generated via multivariate imputation by chained equations, using the ICE package of Stata 9 (Stata Corporation, College Station, TX, USA).14,15 Five replicates of the dataset were imputed using binary, ordinal or multinomial logistic regression depending on the structure of the variable. For subsequent analyses, the replicates were analysed independently and averaged to calculate point estimates with suitable standard errors derived from the withinand between-imputation variance components. This was achieved using the ‘micombine’ command in Stata 9 in conjunction with the regression commands.
Evaluation of the NNIS risk index The NNIS RI was calculated for each of the complete records in our original (pre-imputation) dataset. The index grades patients from 1 to 3 based on the number of risk factors that they exhibit out of the following: ASA score >2, contaminated or dirty surgical wound classification and surgical duration in the top quartile of all similar procedures.5 We based the procedurespecific thresholds for the upper quartile of duration of surgery on our local surveillance data (Table II). Data were available for wound classification but all procedures were clean or cleancontaminated. The sensitivity, specificity, positive predictive value (PPV) and negative predictive value (NPV) of the NNIS scores for the different SSI outcomes were calculated in Stata 9.
150 Table I
A.C.A. Clements et al. Variables included in the surgical site infection surveillance database
Variable type
Variable name
Variable description
Missing values (%)
Infection outcome
Detection mode Anatomical location
In-hospital, post-discharge, no infection Superficial-incisional, deep-incisional, organ/space, no infection
0 (0.0) 0 (0.0)
Procedure information
Date Procedure
Date (day/month/year) Hip replacement (partial, total, revision), knee replacement (total, revision), FPB, emergency LSCS, elective LSCS, CABG with graft site, CABG with no graft site, mastectomy (simple, radical), TAH Hours/minutes duration of the procedure Given >2 h pre-incision, given 0e2 h pre-incision, given post-incision, not given Emergency, elective Consultant, registrar, resident
0 (0.0) 0 (0.0)
Duration Antibiotic prophylaxis Emergency Surgeon Patient information
Age Sex ASA score
Age in years of the patient Sex of the patient 1 (healthy); 2 (mild systemic disease); 3 (severe systemic disease); 4 (severe lifethreatening systemic disease); 5 (moribund)
32 (0.1) 2215 (5.1) 546 (1.3) 1116 (2.6) 223 (0.5) 173 (0.4) 9069 (20.8)
FPB, femoro-popliteal bypass; LSCS, lower segment Caesarean section; CABG, coronary artery bypass graft; TAH, total abdominal hysterectomy; ASA, American Society of Anesthesiologists. All orthopaedic procedure data were for unilateral procedures only.
Risk factor analysis To model non-linear relationships between age, surgical duration and SSI risk, the data for patient age were categorized into <30, 31e40, 41e60, 61e70 and >70 years, and the data for surgical duration were categorized into 2, >2e3, >3e5 and >5 h. The ASA scores were also re-categorized into 1e2 and 3e5, as in the NNIS RI.5 Since we used the model validation technique of data splitting, the dataset was randomly divided into two subsets of approximately equal size. One subset was used for training (developing the models), while the other subset was used for validating the models. Separate mixed-effects logistic regression models were constructed in Stata 9 for each of the five SSI outcomes, using the training subset. The procedure- and patient-related variables previously identified were considered as potential independent variables (fixed effects) and the hospital was entered as a random effect to account for inherent clustering. Sex was not considered further as a potential independent variable due to the large number of sex-specific procedures (e.g. hysterectomy, Caesarean section). Multi-collinearity diagnostics were performed among the remaining candidate variables and none was found to be collinear.
Table II Thresholds used to define the upper quartile of surgical durations for common surgical procedures in Queensland public hospitals and thresholds used by the United States National Nosocomial Infection Surveillance (NNIS) system Procedure
Partial hip replacement Total hip replacement Revision total hip replacement Total knee replacement Revision total knee replacement Femoro-popliteal bypass Elective LSCS Emergency LSCS CABG with graft site CABG with no graft site Simple mastectomy Radical mastectomy Total abdominal hysterectomy
Threshold: NNIS (min)
Threshold: Queensland hospitals (min)
150 150 150
95 131 215
150 150
120 189
180 57 57 293 293 135 135 120
203 56 53 233 200 100 116 113
LSCS, lower segment Caesarean section; CABG, coronary artery bypass graft. All orthopaedic procedures were unilateral.
Stratifying risk of surgical infections All potential independent variables were initially entered in the models. Parsimonious models were then sought where variables were removed sequentially and those with non-significant associations with the dependent variables, as defined by a likelihood ratio test (LRT) P < 0.05, were excluded. A final model was arrived at when removal of any of the remaining variables did not result in an LRT P < 0.05. Interactions were not assessed due to the lack of prior evidence in support of their inclusion. Fitted values of each model were calculated for the validation subset, and diagnostic statistics were calculated to determine discriminatory performance of the models. The test statistic was the area under the curve (AUC) of the receiver operating characteristic (ROC), a plot of sensitivity versus one minus specificity across all fitted values, with the observed outcome taken as the gold standard. We took a value of the AUC >0.7 to indicate acceptable model performance.16
Results Evaluation of the NNIS risk index Statistics for the discriminatory performance of the NNIS RI for different SSI outcomes are presented in Table III. The highest sensitivity was found for in-hospital infections using an NNIS RI threshold of 1 (i.e. where the patient had at least one of the risk factors in the NNIS RI), but this was still low at 0.69 [95% confidence interval (CI) 0.64e 0.74]. All other sensitivities using this threshold were between 0.47 and 0.61, and specificity was 0.55, whereas using the higher NNIS RI threshold
151 of 2 resulted in sensitivities ranging from 0.09 to 0.20 and specificity of 0.93. The PPV of the NNIS RI for all outcomes was low (0.08) and NPV was high (0.95).
Risk factor analysis Overall, femoro-popliteal bypass (FPB), ASA score >2, duration of surgery >2e3, >3e5 and >5 h (with a linear increase in odds ratios, OR) and lack of antibiotic prophylaxis, had significantly higher odds than the reference categories (Table IV). Hip replacement, knee replacement, coronary artery bypass graft (CABG) with or without a graft site and elective lower segment Caesarean section (LSCS) had significantly lower odds. For in-hospital SSI, FPB, ASA score >2, duration of surgery >3e5 h and lack of antibiotic prophylaxis had significantly higher odds than the reference categories, while CABG with or without a graft site had significantly lower odds (Table V). Duration of surgery >5 h was marginally non-significant when entered into this model (OR 2.48, 95% CI 0.99e 6.23, P ¼ 0.053). For SSI detected post-discharge, FPB and duration of surgery >2e3, >3e5 and >5 h (with linearly increasing OR) had significantly higher odds than the reference categories, while hip replacement and CABG with a graft site had significantly lower odds (Table VI). Age, which had a non-linear relationship with SSI, was included in this model on the basis of the LRT, even though none of the categories was significantly different from the reference category. For deep-incisional SSI, FPB, ASA score >2 and duration of surgery >5 h had significantly higher odds than the reference categories, while elective LSCS had significantly lower odds (Table VII). For
Table III Performance indicators for the National Nosocomial Infections Surveillance (NNIS) risk index for different surgical site infection (SSI) outcomes SSI outcome
NNIS RI threshold
Sensitivity (95% CI)
Specificity (95% CI)
PPV (95% CI)
NPV (95% CI)
Overall
1 2
0.51 (0.49e0.53) 0.11 (0.09e0.12)
0.55 (0.54e0.55) 0.93 (0.93e0.93)
0.06 (0.05e0.06) 0.08 (0.07e0.09)
0.96 (0.95e0.96) 0.95 (0.95e0.95)
In-hospital
1 2
0.69 (0.64e0.74) 0.18 (0.14e0.22)
0.55 (0.54e0.55) 0.93 (0.93e0.93)
0.02 (0.01e0.02) 0.03 (0.02e0.03)
0.99 (0.99e1.00) 0.99 (0.99e0.99)
Post-discharge
1 2
0.47 (0.44e0.49) 0.09 (0.08e0.11)
0.55 (0.54e0.55) 0.93 (0.93e0.93)
0.04 (0.04e0.04) 0.05 (0.04e0.06)
0.96 (0.96e0.96) 0.96 (0.96e0.96)
Deep-incisional
1 2
0.61 (0.55e0.66) 0.20 (0.16e0.25)
0.55 (0.54e0.55) 0.93 (0.93e0.93)
0.01 (0.01e0.01) 0.02 (0.02e0.03)
0.99 (0.99e1.00) 0.99 (0.99e0.99)
Superficial-incisional
1 2
0.49 (0.47e0.52) 0.09 (0.08e0.11)
0.55 (0.54e0.55) 0.93 (0.93e0.93)
0.05 (0.04e0.05) 0.06 (0.05e0.07)
0.96 (0.96e0.96) 0.96 (0.96e0.96)
PPV, positive predictive value; NPV, negative predictive value; CI, confidence interval.
152
A.C.A. Clements et al.
Table IV Mixed-effects logistic regression model of the occurrence of surgical site infection (SSI) following common procedures in 23 hospitals in Queensland Variable
SE
P
Hip replacement (partial, revision, total) Knee replacement (total, revision) Femoro-popliteal bypass Elective lower segment Caesarean section Coronary artery bypass graft with graft site Coronary artery bypass graft without graft site Mastectomy (simple, radical) Total abdominal hysterectomy ASA score: >2
0.60 0.74 2.03 0.82 0.41 0.32 1.01 1.02 1.41
OR (95% CI) (0.45e0.79) (0.56e0.97) (1.31e3.14) (0.68e0.98) (0.27e0.61) (0.15e0.66) (0.69e1.49) (0.78e1.33) (1.14e1.74)
0.09 0.10 0.45 0.07 0.08 0.12 0.20 0.14 0.15
0.000 0.029 0.001 0.027 0.000 0.002 0.949 0.883 0.002
Duration of surgery: >2e3 h >3e5 h >5 h
1.31 (1.06e1.62) 1.55 (1.17e2.06) 3.01 (1.97e4.61)
0.14 0.22 0.66
0.014 0.002 0.000
Antibiotics: given >2 h before incision given after incision not given
1.22 (0.68e2.19) 0.88 (0.74e1.04) 1.34 (1.09e1.66)
0.36 0.08 0.15
0.513 0.146 0.006
OR, odds ratio; CI, confidence interval; SE, standard error. Reference categories were: surgical procedure, emergency lower segment Caesarean section; American Society of Anesthesiologists (ASA) score 1e2; duration of surgery 2 h; antibiotics, given 0e2 h before incision. Variance of the random-effect: 0.17; proportion of total variance explained by the random-effect (r): 0.05; area under curve (AUC) for validation subset: 0.59.
superficial-incisional SSI, FPB, ASA score >2, surgical duration >3e5 and >5 h and lack of antibiotic prophylaxis had significantly higher odds than the reference categories, while hip replacement and
CABG with or without a graft site had significantly lower odds (Table VIII). Unexplained variation arising from hospitallevel clustering was small for all models, ranging
Table V Mixed-effects logistic regression models of surgical site infection (SSI) detected in-hospital compared to no infection Variable
SE
P
Hip replacement (partial, revision, total) Knee replacement (total, revision) Femoro-popliteal bypass Elective lower segment Caesarean section Coronary artery bypass graft with graft site Coronary artery bypass graft without graft site Mastectomy (simple, radical) Total abdominal hysterectomy ASA score: >2
1.43 1.08 2.42 0.69 0.23 0.12 0.92 1.57 2.45
OR (95% CI) (0.85e2.40) (0.62e1.86) (1.14e5.13) (0.44e1.09) (0.11e0.50) (0.01e0.94) (0.39e2.15) (0.91e2.70) (1.70e3.53)
0.38 0.30 0.93 0.16 0.09 0.12 0.40 0.44 0.46
0.175 0.789 0.021 0.114 0.000 0.043 0.839 0.103 0.000
Duration of surgery: >2e3 h >3e5 h >5 h
1.20 (0.81e1.77) 1.86 (1.12e3.08) 2.48 (0.99e6.23)
0.24 0.48 1.17
0.370 0.017 0.053
Antibiotics: given >2 h before incision given after incision not given
1.81 (0.71e4.61) 1.03 (0.72e1.49) 1.81 (1.19e2.73)
0.86 0.19 0.38
0.215 0.859 0.005
OR, odds ratio; CI, confidence interval; SE, standard error. Data refer to common surgical procedures in 23 hospitals in Queensland. Reference categories were: surgical procedure, emergency lower segment Caesarean section; American Society of Anesthesiologists (ASA) score 1e2; duration of surgery 2 h; antibiotics, given 0e2 h before incision. Variance of the random-effect: 0.25; proportion of total variance explained by the random-effect (r): 0.07; area under curve (AUC) for validation subset: 0.66.
Stratifying risk of surgical infections
153
Table VI Mixed-effects logistic regression models of surgical site infection (SSI) detected post-discharge compared to no infection Variable
OR (95% CI)
SE
P
Age: 31e40 years 41e60 years 61e70 years >70 years Hip replacement (partial, revision, total) Knee replacement (total, revision) Femoro-popliteal bypass Elective lower segment Caesarean section Coronary artery bypass graft with graft site Coronary artery bypass graft without graft site Mastectomy (simple, radical) Total abdominal hysterectomy
1.03 1.13 0.94 0.71 0.53 0.87 2.37 0.83 0.49 0.58 1.10 0.88
(0.85e1.25) (0.76e1.67) (0.60e1.47) (0.45e1.12) (0.32e0.88) (0.53e1.41) (1.27e4.42) (0.68e1.01) (0.27e0.87) (0.25e1.35) (0.64e1.88) (0.58e1.32)
0.10 0.23 0.21 0.17 0.14 0.22 0.75 0.08 0.15 0.25 0.30 0.18
0.741 0.552 0.771 0.145 0.014 0.568 0.007 0.060 0.016 0.202 0.731 0.537
Duration of surgery: >2e3 h >3e5 h >5 h
1.31 (1.02e1.70) 1.46 (1.03e2.06) 2.30 (1.32e4.01)
0.17 0.26 0.65
0.037 0.032 0.003
Antibiotics: given >2 h before incision given after incision not given
0.96 (0.44e2.07) 0.86 (0.71e1.04) 1.23 (0.97e1.57)
0.38 0.08 0.15
0.911 0.115 0.093
OR, odds ratio; CI, confidence interval; SE, standard error. Data refer to common surgical procedures in 23 hospitals in Queensland. Reference categories were: surgical procedure, emergency lower segment Caesarean section; duration of surgery 2 h; antibiotics, given 0e2 h before incision, age 30 years. Variance of the random-effect: 0.21; proportion of total variance explained by the random-effect (r): 0.06; area under curve (AUC) for validation subset: 0.59.
from 5% in the overall SSI model to 9% in the deepincisional SSI model. The discriminatory performance of all models was below the acceptability threshold (AUC > 0.7), with AUC ranging from 0.59 for the overall and post-discharge SSI models to 0.66 for the in-hospital SSI model.
Discussion All of the procedures in our dataset were clean or clean-contaminated, so the NNIS RI was only able to stratify SSI risk on the basis of ASA score and surgical duration in our analysis. The low sensitivity of the NNIS RI meant that many cases of SSI did not demonstrate either of these risk factors. The specificity results reflect the fact that although few individuals not affected by SSI had both risk factors in the index, many had one risk factor. Our results for PPV (the proportion of individuals with an RI score above each threshold that developed SSI) and NPV (the proportion of individuals with an RI score below each threshold that did not develop SSI) are likely to have been highly influenced by the low incidence of SSI. The poor discriminatory
performance of the NNIS RI means that it is too crude for accurate risk stratification in clean or clean-contaminated procedures in our local setting. This finding informed the second part of our analysis, an investigation of local risk factors. ASA score >2 was found to be a risk factor for all outcomes except post-discharge SSI, while duration of surgery was found to be a risk factor for all outcomes. Long surgical duration and ASA score, both components of the NNIS RI, have been found to be independently associated with SSI in previous studies.6,9,10,17,18 Lack of antibiotic prophylaxis was found to be a risk factor for all SSI outcomes except post-discharge and deep-incisional SSI, highlighting the importance of prophylaxis as a cornerstone of SSI prevention. Lack of antibiotic prophylaxis, or administering antibiotic prophylaxis but not according to recommended procedures, has been found to be associated with increased odds of SSI, but only the former was significant in our models.17,18 FPB was found to have significantly higher odds than the reference category (emergency LSCS) for all outcomes, although this procedure is prone to non-infective complications such as serous discharge or lymphatic fistula, which might be
154
A.C.A. Clements et al.
Table VII Mixed-effects logistic regression models for deep-incisional or organ/space infections compared with no infection Variable
SE
P
Hip replacement (partial, revision, total) Knee replacement (total, revision) Femoro-popliteal bypass Elective lower segment Caesarean section Coronary artery bypass graft with graft site Coronary artery bypass graft without graft site Mastectomy (simple, radical) ASA score: >2
1.11 1.09 2.64 0.46 0.58 1.89 1.10 1.72
OR (95% CI) (0.63e1.94) (0.62e1.92) (1.09e6.42) (0.26e0.83) (0.27e1.24) (0.86e4.18) (0.55e2.22) (1.12e2.64)
0.32 0.31 1.20 0.14 0.23 0.77 0.39 0.38
0.724 0.755 0.032 0.010 0.163 0.115 0.785 0.013
Duration of surgery: >2e3 h >3e5 h >5 h
1.42 (0.89e2.25) 1.68 (0.92e3.07) 3.64 (1.55e8.56)
0.33 0.52 1.59
0.137 0.091 0.003
OR, odds ratio; CI, confidence interval; SE, standard error. Data refer to common surgical procedures in 23 hospitals in Queensland. Reference categories were: surgical procedure, emergency lower segment Caesarean section; American Society of Anesthesiologists (ASA) score 1e2; duration of surgery 2 h. Variance of the random-effect: 0.32; proportion of total variance explained by the random-effect (r): 0.09; area under curve (AUC) for validation subset: 0.60.
misdiagnosed as SSI. Other surgical procedures, including hip and knee replacement, CABG and elective LSCS, had lower odds than the reference category for different SSI outcomes. This is consistent with other studies showing that different surgical procedures were associated with different SSI risks.19,20
Table VIII
The low values for r (the proportion of residual variation explained by hospital-level clustering) suggest that hospital-level factors contribute relatively little to overall variation in the data. Data quality was likely to be an important issue in our analyses. It was particularly difficult to guarantee the quality of the post-discharge SSI outcome data as this relied
Mixed-effects logistic regression models of superficial-incisional infections compared to no infection
Variable
SE
P
Hip replacement (partial, revision, total) Knee replacement (total, revision) Femoro-popliteal bypass Elective lower segment Caesarean section Coronary artery bypass graft with graft site Coronary artery bypass graft without graft site Mastectomy (simple, radical) Total abdominal hysterectomy ASA score: >2
0.59 0.77 1.99 0.86 0.30 0.44 0.98 1.07 1.35
OR (95% CI) (0.43e0.80) (0.57e1.03) (1.23e3.22) (0.71e1.04) (0.19e0.48) (0.21e0.92) (0.64e1.49) (0.81e1.42) (1.08e1.69)
0.09 0.12 0.49 0.08 0.07 0.17 0.21 0.15 0.15
0.001 0.079 0.005 0.115 0.000 0.030 0.907 0.620 0.009
Duration of surgery: >2e3 h >3e5 h >5 h
1.20 (0.94e1.53) 1.44 (1.04e2.00) 1.88 (1.05e3.36)
0.15 0.24 0.56
0.136 0.028 0.034
Antibiotics: given >2 h before incision given after incision not given
1.08 (0.55e2.11) 0.94 (0.78e1.13) 1.36 (1.08e1.71)
0.37 0.09 0.16
0.831 0.521 0.009
OR, odds ratio; CI, confidence interval; SE, standard error. Data refer to common surgical procedures in 23 hospitals in Queensland. Reference categories were: surgical procedure, emergency lower segment Caesarean section; American Society of Anesthesiologists (ASA) score 1e2; duration of surgery 2 h; antibiotics, given 0e2 h before incision. Variance of the random-effect: 0.20; proportion of total variance explained by the random-effect (r): 0.06; area under curve (AUC) for validation subset: 0.60.
Stratifying risk of surgical infections on passive surveillance, which has been shown to give unreliable results.21 Another published SSI risk factor study that attempted model validation also showed a low predictive performance, which reiterates the need for improved surveillance methods and detailed investigations of risk factors for different SSI outcomes.8 Although our models did not have sufficient discriminatory performance for prognostic purposes, they may have a role in risk adjustment for inter-hospital comparison of infection rates. The finding that risk factors varied for different SSI outcomes suggests that RIs e such as the NNIS RI e that attempt to incorporate all SSI outcomes and surgical procedures may not be optimal. As the epidemiology of SSI is likely to vary in different locations, local RIs may provide a more robust approach to stratifying SSI risk than global indices. We have identified a need for investigations into individual patient- and procedure-level risk factors to better explain observed variation in the occurrence of SSI in Australia, with the ultimate goal of developing local, outcome- and procedure-specific RIs.
155
6.
7.
8.
9.
10.
11.
12.
Acknowledgements 13.
We thank hospital infection control staff in participating hospitals for collecting the surveillance data, and Mr Shane Doidge who maintains the infection surveillance database as surveillance co-ordinator for the Centre for Hospital Related Infection Surveillance and Prevention (CHRISP).
References
14.
15. 16. 17.
1. Coello R, Charlett A, Wilson J, Ward V, Pearson A, Borriello P. Adverse impact of surgical site infections in English hospitals. J Hosp Infect 2005;60:93e103. 2. Graves N, Nicholls TM, Morris AJ. Modeling the costs of hospital-acquired infections in New Zealand. Infect Control Hosp Epidemiol 2003;24:214e223. 3. Emori TG, Culver DH, Horan TC, et al. National nosocomial infections surveillance system (NNIS): description of surveillance methods. Am J Infect Control 1991;19:19e35. 4. Horan TC, Gaynes RP, Martone WJ, Jarvis WR, Emori TG. CDC definitions of nosocomial surgical site infections, 1992: a modification of CDC definitions of surgical wound infections. Infect Control Hosp Epidemiol 1992;13:606e608. 5. Culver DH, Horan TC, Gaynes RP, et al. Surgical wound infection rates by wound class, operative procedure, and
18.
19.
20.
21.
patient risk index. National Nosocomial Infections Surveillance System. Am J Med 1991;91:152Se157S. Kasatpibal N, Jamulitrat S, Chongsuvivatwong V. Standardized incidence rates of surgical site infection: a multicenter study in Thailand. Am J Infect Control 2005;33: 587e594. Berbari EF, Hanssen AD, Duffy MC, et al. Risk factors for prosthetic joint infection: caseecontrol study. Clin Infect Dis 1998;27:1247e1254. Muilwijk J, Walenkamp GH, Voss A, Wille JC, van den Hof S. Random effect modelling of patient-related risk factors in orthopaedic procedures: results from the Dutch nosocomial infection surveillance network ‘PREZIES’. J Hosp Infect 2006;62:319e326. Soleto L, Pirard M, Boelaert M, et al. Incidence of surgicalsite infections and the validity of the National Nosocomial Infections Surveillance System risk index in a general surgical ward in Santa Cruz, Bolivia. Infect Control Hosp Epidemiol 2003;24:26e30. de Oliveira AC, Ciosak SI, Ferraz EM, Grinbaum RS. Surgical site infection in patients submitted to digestive surgery: risk prediction and the NNIS risk index. Am J Infect Control 2006;34:201e207. Brandt C, Hansen S, Sohr D, Daschner F, Ruden H, Gastmeier P. Finding a method for optimizing risk adjustment when comparing surgical-site infection rates. Infect Control Hosp Epidemiol 2004;25:313e318. Friedman ND, Bull AL, Russo PL, Gurrin L, Richards M. Performance of the national nosocomial infections surveillance risk index in predicting surgical site infection in Australia. Infect Control Hosp Epidemiol 2007;28:55e59. Mangram AJ, Horan TC, Pearson ML, Silver LC, Jarvis WR. Guideline for Prevention of Surgical Site Infection, 1999. Centers for Disease Control and Prevention (CDC) Hospital Infection Control Practices Advisory Committee. Am J Infect Control 1999;27:97e132; [quiz 133e134; discussion 196]. van Buuren S, Boshuizen HC, Knook DL. Multiple imputation of missing blood pressure covariates in survival analysis. Stat Med 1999;18:681e694. Royston P. Multiple imputation of missing values: update of ICE. Stata J 2005;5:188e201. Hosmer DW, Lemeshow S, editors. Applied Logistic Regression. New York: John Wiley & Sons; 2000. p. 160e164. Lizan-Garcia M, Garcia-Caballero J, Asensio-Vegas A. Risk factors for surgical-wound infection in general surgery: a prospective study. Infect Control Hosp Epidemiol 1997; 18:310e315. Pessaux P, Atallah D, Lermite E, et al. Risk factors for prediction of surgical site infections in ‘clean surgery’. Am J Infect Control 2005;33:292e298. Scott JD, Forrest A, Feuerstein S, Fitzpatrick P, Schentag JJ. Factors associated with postoperative infection. Infect Control Hosp Epidemiol 2001;22:347e351. Sorensen LT, Hemmingsen U, Kallehave F, et al. Risk factors for tissue and wound complications in gastrointestinal surgery. Ann Surg 2005;241:654e658. Whitby M, McLaws ML, Collopy B, et al. Post-discharge surveillance: can patients reliably diagnose surgical wound infections? J Hosp Infect 2002;52:155e160.
www.elsevierhealth.com/journals/jhin
Risk stratification for surgical site infections in Australia: evaluation of the US National Nosocomial Infection Surveillance risk index A.C.A. Clements a,b,*, E.N.C. Tong b, A.P. Morton b, M. Whitby b a
Division of Epidemiology and Social Medicine, School of Population Health, University of Queensland, Brisbane, Australia b Centre for Hospital Related Infection Surveillance and Prevention, Princess Alexandra Hospital, Brisbane, Australia Received 9 November 2006; accepted 23 February 2007 Available online 9 May 2007
KEYWORDS Surgical site infection; Risk index; Risk stratification; Risk factors; Surveillance
Summary This study evaluated the US National Nosocomial Infection Surveillance (NNIS) risk index (RI) in Australia for different surgical site infection (SSI) outcomes (overall, in-hospital, post-discharge, deep-incisional and superficial-incisional infection) and investigated local risk factors for SSI. A SSI surveillance dataset containing 43 611 records for 13 common surgical procedures, conducted in 23 hospitals between February 2001 and June 2005, was used for the analysis. The NNIS RI was evaluated against the observed SSI data using diagnostic test evaluation statistics (sensitivity, specificity, positive predictive value, negative predictive value). Sensitivity was low for all SSI outcomes (ranging from 0.47 to 0.69 and from 0.09 to 0.20 using RI thresholds of 1 and 2 respectively), while specificity varied depending on the RI threshold (0.55 and 0.93 with thresholds of 1 and 2 respectively). Mixed-effects logistic regression models were developed for the five SSI outcomes using a range of available potential risk factors. American Society of Anaesthesiologists (ASA) physical status score >2, duration of surgery, absence of antibiotic prophylaxis and type of surgical procedure were significant risk factors for one or more SSI outcomes, and risk factors varied for different SSI outcomes. The discriminatory ability of the NNIS RI was insufficient for its use as an accurate risk stratification tool for SSI surveillance in Australia and its sensitivity was too low for it to be appropriately used as a prognostic indicator. ª 2007 The Hospital Infection Society. Published by Elsevier Ltd. All rights reserved.
* Corresponding author. Address: Division of Epidemiology and Social Medicine, School of Population Health, University of Queensland, Herston Road, Brisbane, Queensland 4006, Australia. Tel.: þ61 7 3240 5952; fax: þ61 7 3240 5540. E-mail address: [email protected] 0195-6701/$ - see front matter ª 2007 The Hospital Infection Society. Published by Elsevier Ltd. All rights reserved. doi:10.1016/j.jhin.2007.02.019
Stratifying risk of surgical infections
Introduction Quality of healthcare in hospitals is of major public health importance. Surgical site infections (SSI) are some of the most common hospital-acquired infections, contributing to higher patient mortality, significantly longer length of stay and additional treatment costs.1,2 The Centre for Healthcare Related Infection Surveillance and Prevention (CHRISP) has instituted surveillance for SSI in Queensland, Australia, incorporating standardized case-definitions and risk-indexing of surgical patients based on the US Centres for Disease Control and Prevention (CDC) National Nosocomial Infections Surveillance (NNIS) protocols.3e5 The NNIS risk index (RI) has been used to stratify SSI rates for comparison and has been suggested as a prognostic tool for targeting SSI prevention methods at high-risk patients.1,3,6 A number of studies have investigated risk factors for SSI, either in the context of evaluating the NNIS RI and comparing it to alternative riskindexing approaches in local settings, or gaining an improved understanding of the epidemiology of SSI in different scenarios. Notably, while increasing NNIS RI scores have been shown to be associated with increasing risk of SSI and to be an independent predictor of SSI in multivariable models, two studies that compared the NNIS RI with locally developed RIs found the local indices to have better discriminatory performance and a third study showed that the NNIS RI was reliable for some surgical procedures but not for others.5,7e11 A recent report suggested that the NNIS RI performed well in the Australian state of Victoria as a risk stratification tool.12 While demonstrating significant correlation between NNIS RI score and incidence of infection, discriminatory performance of the index was not investigated. We aimed to evaluate the discriminatory performance of the NNIS RI in our local setting, both for overall SSI and for different SSI outcomes (in-hospital, post-discharge, superficial and deep-incisional infection). We also aimed to investigate risk factors for SSI in Australia, from a set of available potential risk factors, as preliminary work towards the development of local SSI indices.
Methods Data collection Routine SSI surveillance data were collected on a voluntary basis for 13 common surgical procedures, from 23 medium-to-large hospitals in
149 Queensland, using an electronic surveillance software tool, the Electronic Infection Control Assessment Technology version 4.2. (eICAT, CHRISP, Brisbane, Australia). Not all hospitals contributed data for all of the surgical procedures monitored. The SSI surveillance database contained records on 43 611 surgical procedures, conducted on 42 226 patients between February 2001 and June 2005. In addition to a range of SSI outcomes as defined by the CDC (whether an infection occurred, whether it was detected in-hospital or post-discharge and whether the infection was superficial-incisional or deepincisional/organ/space), procedure- and patientrelated information was also stored (Table I).4,13 Case-finding for post-discharge infection varied between hospitals, but the most common approach relied on questionnaires posted to patients and follow-up of self-reported cases with the patients’ general practitioners. Missing values occurred for varying numbers of records for each variable, with the American Society of Anesthesiologists (ASA) physical status score having the largest number of missing values. Missing values were generated via multivariate imputation by chained equations, using the ICE package of Stata 9 (Stata Corporation, College Station, TX, USA).14,15 Five replicates of the dataset were imputed using binary, ordinal or multinomial logistic regression depending on the structure of the variable. For subsequent analyses, the replicates were analysed independently and averaged to calculate point estimates with suitable standard errors derived from the withinand between-imputation variance components. This was achieved using the ‘micombine’ command in Stata 9 in conjunction with the regression commands.
Evaluation of the NNIS risk index The NNIS RI was calculated for each of the complete records in our original (pre-imputation) dataset. The index grades patients from 1 to 3 based on the number of risk factors that they exhibit out of the following: ASA score >2, contaminated or dirty surgical wound classification and surgical duration in the top quartile of all similar procedures.5 We based the procedurespecific thresholds for the upper quartile of duration of surgery on our local surveillance data (Table II). Data were available for wound classification but all procedures were clean or cleancontaminated. The sensitivity, specificity, positive predictive value (PPV) and negative predictive value (NPV) of the NNIS scores for the different SSI outcomes were calculated in Stata 9.
150 Table I
A.C.A. Clements et al. Variables included in the surgical site infection surveillance database
Variable type
Variable name
Variable description
Missing values (%)
Infection outcome
Detection mode Anatomical location
In-hospital, post-discharge, no infection Superficial-incisional, deep-incisional, organ/space, no infection
0 (0.0) 0 (0.0)
Procedure information
Date Procedure
Date (day/month/year) Hip replacement (partial, total, revision), knee replacement (total, revision), FPB, emergency LSCS, elective LSCS, CABG with graft site, CABG with no graft site, mastectomy (simple, radical), TAH Hours/minutes duration of the procedure Given >2 h pre-incision, given 0e2 h pre-incision, given post-incision, not given Emergency, elective Consultant, registrar, resident
0 (0.0) 0 (0.0)
Duration Antibiotic prophylaxis Emergency Surgeon Patient information
Age Sex ASA score
Age in years of the patient Sex of the patient 1 (healthy); 2 (mild systemic disease); 3 (severe systemic disease); 4 (severe lifethreatening systemic disease); 5 (moribund)
32 (0.1) 2215 (5.1) 546 (1.3) 1116 (2.6) 223 (0.5) 173 (0.4) 9069 (20.8)
FPB, femoro-popliteal bypass; LSCS, lower segment Caesarean section; CABG, coronary artery bypass graft; TAH, total abdominal hysterectomy; ASA, American Society of Anesthesiologists. All orthopaedic procedure data were for unilateral procedures only.
Risk factor analysis To model non-linear relationships between age, surgical duration and SSI risk, the data for patient age were categorized into <30, 31e40, 41e60, 61e70 and >70 years, and the data for surgical duration were categorized into 2, >2e3, >3e5 and >5 h. The ASA scores were also re-categorized into 1e2 and 3e5, as in the NNIS RI.5 Since we used the model validation technique of data splitting, the dataset was randomly divided into two subsets of approximately equal size. One subset was used for training (developing the models), while the other subset was used for validating the models. Separate mixed-effects logistic regression models were constructed in Stata 9 for each of the five SSI outcomes, using the training subset. The procedure- and patient-related variables previously identified were considered as potential independent variables (fixed effects) and the hospital was entered as a random effect to account for inherent clustering. Sex was not considered further as a potential independent variable due to the large number of sex-specific procedures (e.g. hysterectomy, Caesarean section). Multi-collinearity diagnostics were performed among the remaining candidate variables and none was found to be collinear.
Table II Thresholds used to define the upper quartile of surgical durations for common surgical procedures in Queensland public hospitals and thresholds used by the United States National Nosocomial Infection Surveillance (NNIS) system Procedure
Partial hip replacement Total hip replacement Revision total hip replacement Total knee replacement Revision total knee replacement Femoro-popliteal bypass Elective LSCS Emergency LSCS CABG with graft site CABG with no graft site Simple mastectomy Radical mastectomy Total abdominal hysterectomy
Threshold: NNIS (min)
Threshold: Queensland hospitals (min)
150 150 150
95 131 215
150 150
120 189
180 57 57 293 293 135 135 120
203 56 53 233 200 100 116 113
LSCS, lower segment Caesarean section; CABG, coronary artery bypass graft. All orthopaedic procedures were unilateral.
Stratifying risk of surgical infections All potential independent variables were initially entered in the models. Parsimonious models were then sought where variables were removed sequentially and those with non-significant associations with the dependent variables, as defined by a likelihood ratio test (LRT) P < 0.05, were excluded. A final model was arrived at when removal of any of the remaining variables did not result in an LRT P < 0.05. Interactions were not assessed due to the lack of prior evidence in support of their inclusion. Fitted values of each model were calculated for the validation subset, and diagnostic statistics were calculated to determine discriminatory performance of the models. The test statistic was the area under the curve (AUC) of the receiver operating characteristic (ROC), a plot of sensitivity versus one minus specificity across all fitted values, with the observed outcome taken as the gold standard. We took a value of the AUC >0.7 to indicate acceptable model performance.16
Results Evaluation of the NNIS risk index Statistics for the discriminatory performance of the NNIS RI for different SSI outcomes are presented in Table III. The highest sensitivity was found for in-hospital infections using an NNIS RI threshold of 1 (i.e. where the patient had at least one of the risk factors in the NNIS RI), but this was still low at 0.69 [95% confidence interval (CI) 0.64e 0.74]. All other sensitivities using this threshold were between 0.47 and 0.61, and specificity was 0.55, whereas using the higher NNIS RI threshold
151 of 2 resulted in sensitivities ranging from 0.09 to 0.20 and specificity of 0.93. The PPV of the NNIS RI for all outcomes was low (0.08) and NPV was high (0.95).
Risk factor analysis Overall, femoro-popliteal bypass (FPB), ASA score >2, duration of surgery >2e3, >3e5 and >5 h (with a linear increase in odds ratios, OR) and lack of antibiotic prophylaxis, had significantly higher odds than the reference categories (Table IV). Hip replacement, knee replacement, coronary artery bypass graft (CABG) with or without a graft site and elective lower segment Caesarean section (LSCS) had significantly lower odds. For in-hospital SSI, FPB, ASA score >2, duration of surgery >3e5 h and lack of antibiotic prophylaxis had significantly higher odds than the reference categories, while CABG with or without a graft site had significantly lower odds (Table V). Duration of surgery >5 h was marginally non-significant when entered into this model (OR 2.48, 95% CI 0.99e 6.23, P ¼ 0.053). For SSI detected post-discharge, FPB and duration of surgery >2e3, >3e5 and >5 h (with linearly increasing OR) had significantly higher odds than the reference categories, while hip replacement and CABG with a graft site had significantly lower odds (Table VI). Age, which had a non-linear relationship with SSI, was included in this model on the basis of the LRT, even though none of the categories was significantly different from the reference category. For deep-incisional SSI, FPB, ASA score >2 and duration of surgery >5 h had significantly higher odds than the reference categories, while elective LSCS had significantly lower odds (Table VII). For
Table III Performance indicators for the National Nosocomial Infections Surveillance (NNIS) risk index for different surgical site infection (SSI) outcomes SSI outcome
NNIS RI threshold
Sensitivity (95% CI)
Specificity (95% CI)
PPV (95% CI)
NPV (95% CI)
Overall
1 2
0.51 (0.49e0.53) 0.11 (0.09e0.12)
0.55 (0.54e0.55) 0.93 (0.93e0.93)
0.06 (0.05e0.06) 0.08 (0.07e0.09)
0.96 (0.95e0.96) 0.95 (0.95e0.95)
In-hospital
1 2
0.69 (0.64e0.74) 0.18 (0.14e0.22)
0.55 (0.54e0.55) 0.93 (0.93e0.93)
0.02 (0.01e0.02) 0.03 (0.02e0.03)
0.99 (0.99e1.00) 0.99 (0.99e0.99)
Post-discharge
1 2
0.47 (0.44e0.49) 0.09 (0.08e0.11)
0.55 (0.54e0.55) 0.93 (0.93e0.93)
0.04 (0.04e0.04) 0.05 (0.04e0.06)
0.96 (0.96e0.96) 0.96 (0.96e0.96)
Deep-incisional
1 2
0.61 (0.55e0.66) 0.20 (0.16e0.25)
0.55 (0.54e0.55) 0.93 (0.93e0.93)
0.01 (0.01e0.01) 0.02 (0.02e0.03)
0.99 (0.99e1.00) 0.99 (0.99e0.99)
Superficial-incisional
1 2
0.49 (0.47e0.52) 0.09 (0.08e0.11)
0.55 (0.54e0.55) 0.93 (0.93e0.93)
0.05 (0.04e0.05) 0.06 (0.05e0.07)
0.96 (0.96e0.96) 0.96 (0.96e0.96)
PPV, positive predictive value; NPV, negative predictive value; CI, confidence interval.
152
A.C.A. Clements et al.
Table IV Mixed-effects logistic regression model of the occurrence of surgical site infection (SSI) following common procedures in 23 hospitals in Queensland Variable
SE
P
Hip replacement (partial, revision, total) Knee replacement (total, revision) Femoro-popliteal bypass Elective lower segment Caesarean section Coronary artery bypass graft with graft site Coronary artery bypass graft without graft site Mastectomy (simple, radical) Total abdominal hysterectomy ASA score: >2
0.60 0.74 2.03 0.82 0.41 0.32 1.01 1.02 1.41
OR (95% CI) (0.45e0.79) (0.56e0.97) (1.31e3.14) (0.68e0.98) (0.27e0.61) (0.15e0.66) (0.69e1.49) (0.78e1.33) (1.14e1.74)
0.09 0.10 0.45 0.07 0.08 0.12 0.20 0.14 0.15
0.000 0.029 0.001 0.027 0.000 0.002 0.949 0.883 0.002
Duration of surgery: >2e3 h >3e5 h >5 h
1.31 (1.06e1.62) 1.55 (1.17e2.06) 3.01 (1.97e4.61)
0.14 0.22 0.66
0.014 0.002 0.000
Antibiotics: given >2 h before incision given after incision not given
1.22 (0.68e2.19) 0.88 (0.74e1.04) 1.34 (1.09e1.66)
0.36 0.08 0.15
0.513 0.146 0.006
OR, odds ratio; CI, confidence interval; SE, standard error. Reference categories were: surgical procedure, emergency lower segment Caesarean section; American Society of Anesthesiologists (ASA) score 1e2; duration of surgery 2 h; antibiotics, given 0e2 h before incision. Variance of the random-effect: 0.17; proportion of total variance explained by the random-effect (r): 0.05; area under curve (AUC) for validation subset: 0.59.
superficial-incisional SSI, FPB, ASA score >2, surgical duration >3e5 and >5 h and lack of antibiotic prophylaxis had significantly higher odds than the reference categories, while hip replacement and
CABG with or without a graft site had significantly lower odds (Table VIII). Unexplained variation arising from hospitallevel clustering was small for all models, ranging
Table V Mixed-effects logistic regression models of surgical site infection (SSI) detected in-hospital compared to no infection Variable
SE
P
Hip replacement (partial, revision, total) Knee replacement (total, revision) Femoro-popliteal bypass Elective lower segment Caesarean section Coronary artery bypass graft with graft site Coronary artery bypass graft without graft site Mastectomy (simple, radical) Total abdominal hysterectomy ASA score: >2
1.43 1.08 2.42 0.69 0.23 0.12 0.92 1.57 2.45
OR (95% CI) (0.85e2.40) (0.62e1.86) (1.14e5.13) (0.44e1.09) (0.11e0.50) (0.01e0.94) (0.39e2.15) (0.91e2.70) (1.70e3.53)
0.38 0.30 0.93 0.16 0.09 0.12 0.40 0.44 0.46
0.175 0.789 0.021 0.114 0.000 0.043 0.839 0.103 0.000
Duration of surgery: >2e3 h >3e5 h >5 h
1.20 (0.81e1.77) 1.86 (1.12e3.08) 2.48 (0.99e6.23)
0.24 0.48 1.17
0.370 0.017 0.053
Antibiotics: given >2 h before incision given after incision not given
1.81 (0.71e4.61) 1.03 (0.72e1.49) 1.81 (1.19e2.73)
0.86 0.19 0.38
0.215 0.859 0.005
OR, odds ratio; CI, confidence interval; SE, standard error. Data refer to common surgical procedures in 23 hospitals in Queensland. Reference categories were: surgical procedure, emergency lower segment Caesarean section; American Society of Anesthesiologists (ASA) score 1e2; duration of surgery 2 h; antibiotics, given 0e2 h before incision. Variance of the random-effect: 0.25; proportion of total variance explained by the random-effect (r): 0.07; area under curve (AUC) for validation subset: 0.66.
Stratifying risk of surgical infections
153
Table VI Mixed-effects logistic regression models of surgical site infection (SSI) detected post-discharge compared to no infection Variable
OR (95% CI)
SE
P
Age: 31e40 years 41e60 years 61e70 years >70 years Hip replacement (partial, revision, total) Knee replacement (total, revision) Femoro-popliteal bypass Elective lower segment Caesarean section Coronary artery bypass graft with graft site Coronary artery bypass graft without graft site Mastectomy (simple, radical) Total abdominal hysterectomy
1.03 1.13 0.94 0.71 0.53 0.87 2.37 0.83 0.49 0.58 1.10 0.88
(0.85e1.25) (0.76e1.67) (0.60e1.47) (0.45e1.12) (0.32e0.88) (0.53e1.41) (1.27e4.42) (0.68e1.01) (0.27e0.87) (0.25e1.35) (0.64e1.88) (0.58e1.32)
0.10 0.23 0.21 0.17 0.14 0.22 0.75 0.08 0.15 0.25 0.30 0.18
0.741 0.552 0.771 0.145 0.014 0.568 0.007 0.060 0.016 0.202 0.731 0.537
Duration of surgery: >2e3 h >3e5 h >5 h
1.31 (1.02e1.70) 1.46 (1.03e2.06) 2.30 (1.32e4.01)
0.17 0.26 0.65
0.037 0.032 0.003
Antibiotics: given >2 h before incision given after incision not given
0.96 (0.44e2.07) 0.86 (0.71e1.04) 1.23 (0.97e1.57)
0.38 0.08 0.15
0.911 0.115 0.093
OR, odds ratio; CI, confidence interval; SE, standard error. Data refer to common surgical procedures in 23 hospitals in Queensland. Reference categories were: surgical procedure, emergency lower segment Caesarean section; duration of surgery 2 h; antibiotics, given 0e2 h before incision, age 30 years. Variance of the random-effect: 0.21; proportion of total variance explained by the random-effect (r): 0.06; area under curve (AUC) for validation subset: 0.59.
from 5% in the overall SSI model to 9% in the deepincisional SSI model. The discriminatory performance of all models was below the acceptability threshold (AUC > 0.7), with AUC ranging from 0.59 for the overall and post-discharge SSI models to 0.66 for the in-hospital SSI model.
Discussion All of the procedures in our dataset were clean or clean-contaminated, so the NNIS RI was only able to stratify SSI risk on the basis of ASA score and surgical duration in our analysis. The low sensitivity of the NNIS RI meant that many cases of SSI did not demonstrate either of these risk factors. The specificity results reflect the fact that although few individuals not affected by SSI had both risk factors in the index, many had one risk factor. Our results for PPV (the proportion of individuals with an RI score above each threshold that developed SSI) and NPV (the proportion of individuals with an RI score below each threshold that did not develop SSI) are likely to have been highly influenced by the low incidence of SSI. The poor discriminatory
performance of the NNIS RI means that it is too crude for accurate risk stratification in clean or clean-contaminated procedures in our local setting. This finding informed the second part of our analysis, an investigation of local risk factors. ASA score >2 was found to be a risk factor for all outcomes except post-discharge SSI, while duration of surgery was found to be a risk factor for all outcomes. Long surgical duration and ASA score, both components of the NNIS RI, have been found to be independently associated with SSI in previous studies.6,9,10,17,18 Lack of antibiotic prophylaxis was found to be a risk factor for all SSI outcomes except post-discharge and deep-incisional SSI, highlighting the importance of prophylaxis as a cornerstone of SSI prevention. Lack of antibiotic prophylaxis, or administering antibiotic prophylaxis but not according to recommended procedures, has been found to be associated with increased odds of SSI, but only the former was significant in our models.17,18 FPB was found to have significantly higher odds than the reference category (emergency LSCS) for all outcomes, although this procedure is prone to non-infective complications such as serous discharge or lymphatic fistula, which might be
154
A.C.A. Clements et al.
Table VII Mixed-effects logistic regression models for deep-incisional or organ/space infections compared with no infection Variable
SE
P
Hip replacement (partial, revision, total) Knee replacement (total, revision) Femoro-popliteal bypass Elective lower segment Caesarean section Coronary artery bypass graft with graft site Coronary artery bypass graft without graft site Mastectomy (simple, radical) ASA score: >2
1.11 1.09 2.64 0.46 0.58 1.89 1.10 1.72
OR (95% CI) (0.63e1.94) (0.62e1.92) (1.09e6.42) (0.26e0.83) (0.27e1.24) (0.86e4.18) (0.55e2.22) (1.12e2.64)
0.32 0.31 1.20 0.14 0.23 0.77 0.39 0.38
0.724 0.755 0.032 0.010 0.163 0.115 0.785 0.013
Duration of surgery: >2e3 h >3e5 h >5 h
1.42 (0.89e2.25) 1.68 (0.92e3.07) 3.64 (1.55e8.56)
0.33 0.52 1.59
0.137 0.091 0.003
OR, odds ratio; CI, confidence interval; SE, standard error. Data refer to common surgical procedures in 23 hospitals in Queensland. Reference categories were: surgical procedure, emergency lower segment Caesarean section; American Society of Anesthesiologists (ASA) score 1e2; duration of surgery 2 h. Variance of the random-effect: 0.32; proportion of total variance explained by the random-effect (r): 0.09; area under curve (AUC) for validation subset: 0.60.
misdiagnosed as SSI. Other surgical procedures, including hip and knee replacement, CABG and elective LSCS, had lower odds than the reference category for different SSI outcomes. This is consistent with other studies showing that different surgical procedures were associated with different SSI risks.19,20
Table VIII
The low values for r (the proportion of residual variation explained by hospital-level clustering) suggest that hospital-level factors contribute relatively little to overall variation in the data. Data quality was likely to be an important issue in our analyses. It was particularly difficult to guarantee the quality of the post-discharge SSI outcome data as this relied
Mixed-effects logistic regression models of superficial-incisional infections compared to no infection
Variable
SE
P
Hip replacement (partial, revision, total) Knee replacement (total, revision) Femoro-popliteal bypass Elective lower segment Caesarean section Coronary artery bypass graft with graft site Coronary artery bypass graft without graft site Mastectomy (simple, radical) Total abdominal hysterectomy ASA score: >2
0.59 0.77 1.99 0.86 0.30 0.44 0.98 1.07 1.35
OR (95% CI) (0.43e0.80) (0.57e1.03) (1.23e3.22) (0.71e1.04) (0.19e0.48) (0.21e0.92) (0.64e1.49) (0.81e1.42) (1.08e1.69)
0.09 0.12 0.49 0.08 0.07 0.17 0.21 0.15 0.15
0.001 0.079 0.005 0.115 0.000 0.030 0.907 0.620 0.009
Duration of surgery: >2e3 h >3e5 h >5 h
1.20 (0.94e1.53) 1.44 (1.04e2.00) 1.88 (1.05e3.36)
0.15 0.24 0.56
0.136 0.028 0.034
Antibiotics: given >2 h before incision given after incision not given
1.08 (0.55e2.11) 0.94 (0.78e1.13) 1.36 (1.08e1.71)
0.37 0.09 0.16
0.831 0.521 0.009
OR, odds ratio; CI, confidence interval; SE, standard error. Data refer to common surgical procedures in 23 hospitals in Queensland. Reference categories were: surgical procedure, emergency lower segment Caesarean section; American Society of Anesthesiologists (ASA) score 1e2; duration of surgery 2 h; antibiotics, given 0e2 h before incision. Variance of the random-effect: 0.20; proportion of total variance explained by the random-effect (r): 0.06; area under curve (AUC) for validation subset: 0.60.
Stratifying risk of surgical infections on passive surveillance, which has been shown to give unreliable results.21 Another published SSI risk factor study that attempted model validation also showed a low predictive performance, which reiterates the need for improved surveillance methods and detailed investigations of risk factors for different SSI outcomes.8 Although our models did not have sufficient discriminatory performance for prognostic purposes, they may have a role in risk adjustment for inter-hospital comparison of infection rates. The finding that risk factors varied for different SSI outcomes suggests that RIs e such as the NNIS RI e that attempt to incorporate all SSI outcomes and surgical procedures may not be optimal. As the epidemiology of SSI is likely to vary in different locations, local RIs may provide a more robust approach to stratifying SSI risk than global indices. We have identified a need for investigations into individual patient- and procedure-level risk factors to better explain observed variation in the occurrence of SSI in Australia, with the ultimate goal of developing local, outcome- and procedure-specific RIs.
155
6.
7.
8.
9.
10.
11.
12.
Acknowledgements 13.
We thank hospital infection control staff in participating hospitals for collecting the surveillance data, and Mr Shane Doidge who maintains the infection surveillance database as surveillance co-ordinator for the Centre for Hospital Related Infection Surveillance and Prevention (CHRISP).
References
14.
15. 16. 17.
1. Coello R, Charlett A, Wilson J, Ward V, Pearson A, Borriello P. Adverse impact of surgical site infections in English hospitals. J Hosp Infect 2005;60:93e103. 2. Graves N, Nicholls TM, Morris AJ. Modeling the costs of hospital-acquired infections in New Zealand. Infect Control Hosp Epidemiol 2003;24:214e223. 3. Emori TG, Culver DH, Horan TC, et al. National nosocomial infections surveillance system (NNIS): description of surveillance methods. Am J Infect Control 1991;19:19e35. 4. Horan TC, Gaynes RP, Martone WJ, Jarvis WR, Emori TG. CDC definitions of nosocomial surgical site infections, 1992: a modification of CDC definitions of surgical wound infections. Infect Control Hosp Epidemiol 1992;13:606e608. 5. Culver DH, Horan TC, Gaynes RP, et al. Surgical wound infection rates by wound class, operative procedure, and
18.
19.
20.
21.
patient risk index. National Nosocomial Infections Surveillance System. Am J Med 1991;91:152Se157S. Kasatpibal N, Jamulitrat S, Chongsuvivatwong V. Standardized incidence rates of surgical site infection: a multicenter study in Thailand. Am J Infect Control 2005;33: 587e594. Berbari EF, Hanssen AD, Duffy MC, et al. Risk factors for prosthetic joint infection: caseecontrol study. Clin Infect Dis 1998;27:1247e1254. Muilwijk J, Walenkamp GH, Voss A, Wille JC, van den Hof S. Random effect modelling of patient-related risk factors in orthopaedic procedures: results from the Dutch nosocomial infection surveillance network ‘PREZIES’. J Hosp Infect 2006;62:319e326. Soleto L, Pirard M, Boelaert M, et al. Incidence of surgicalsite infections and the validity of the National Nosocomial Infections Surveillance System risk index in a general surgical ward in Santa Cruz, Bolivia. Infect Control Hosp Epidemiol 2003;24:26e30. de Oliveira AC, Ciosak SI, Ferraz EM, Grinbaum RS. Surgical site infection in patients submitted to digestive surgery: risk prediction and the NNIS risk index. Am J Infect Control 2006;34:201e207. Brandt C, Hansen S, Sohr D, Daschner F, Ruden H, Gastmeier P. Finding a method for optimizing risk adjustment when comparing surgical-site infection rates. Infect Control Hosp Epidemiol 2004;25:313e318. Friedman ND, Bull AL, Russo PL, Gurrin L, Richards M. Performance of the national nosocomial infections surveillance risk index in predicting surgical site infection in Australia. Infect Control Hosp Epidemiol 2007;28:55e59. Mangram AJ, Horan TC, Pearson ML, Silver LC, Jarvis WR. Guideline for Prevention of Surgical Site Infection, 1999. Centers for Disease Control and Prevention (CDC) Hospital Infection Control Practices Advisory Committee. Am J Infect Control 1999;27:97e132; [quiz 133e134; discussion 196]. van Buuren S, Boshuizen HC, Knook DL. Multiple imputation of missing blood pressure covariates in survival analysis. Stat Med 1999;18:681e694. Royston P. Multiple imputation of missing values: update of ICE. Stata J 2005;5:188e201. Hosmer DW, Lemeshow S, editors. Applied Logistic Regression. New York: John Wiley & Sons; 2000. p. 160e164. Lizan-Garcia M, Garcia-Caballero J, Asensio-Vegas A. Risk factors for surgical-wound infection in general surgery: a prospective study. Infect Control Hosp Epidemiol 1997; 18:310e315. Pessaux P, Atallah D, Lermite E, et al. Risk factors for prediction of surgical site infections in ‘clean surgery’. Am J Infect Control 2005;33:292e298. Scott JD, Forrest A, Feuerstein S, Fitzpatrick P, Schentag JJ. Factors associated with postoperative infection. Infect Control Hosp Epidemiol 2001;22:347e351. Sorensen LT, Hemmingsen U, Kallehave F, et al. Risk factors for tissue and wound complications in gastrointestinal surgery. Ann Surg 2005;241:654e658. Whitby M, McLaws ML, Collopy B, et al. Post-discharge surveillance: can patients reliably diagnose surgical wound infections? J Hosp Infect 2002;52:155e160.