- Email: [email protected]
Two years of surgical site infection surveillance in Western Australia: analysing variation between hospitals
Healthcare Infection 2009; 14: 51–60
Two years of surgical site infection surveillance in Western Australia: analysing variation between hospitals Lynne Dailey1,4 PhD, MPH Helen van Gessel2,3 MBBS, MPH Allison Peterson3 BHSc, Cert IC 1
Western Australian Country Health Service (formally Healthcare Associated Infection Unit, HCAIU), Department of Health, Perth, WA 6892, Australia. 2 Office of Safety and Quality in Healthcare, Department of Health, Perth, WA 6849, Australia. 3 (HCAIU) Department of Health, Perth, WA 6849, Australia. 4 Corresponding author. Email: [email protected]
Abstract Surgical site infections (SSIs) following hip and knee arthroplasty are significant adverse events. The aim of this study was to present the first 2 years of aggregated and hospital-based SSI rates from the Healthcare Infection Surveillance Western Australia (HISWA) surveillance system. HISWA collects risk-adjusted data based on standardised collection methodologies and definitions. In total, 4131 hip and 4858 knee procedures were monitored at 10 public and private hospitals. There were significant variations in reported SSI rates following arthroplasty at WA hospitals (mean 1.82, range 0.67–3.83 SSIs per 100 hip procedures; mean 1.5, range 0.47– 3.67 SSIs per 100 knee procedures). Funnel plots identified hospitals with significantly high and low SSI rates, in particular hospitals with high operative volumes and significantly better outcomes. Significant variation was observed when analysis was performed by operative volume, United States National Nosocomial Infection Surveillance (NNIS) risk group and infection type (deep/organ space v. superficial). Investigations did not reveal marked differences in surveillance methodology between sites, but other causes of variation, including patient factors that are not incorporated in NNIS risk indexation, surgeon procedure volume and differences in SSI prevention policy and practice, need further evaluation. The policy implications of the association between operative volume and SSI rate merits further discussion in Australia.
Introduction Healthcare-associated infection prevention has been identified as a priority initiative by the Department of Health in Western Australia. Established in 2005, Healthcare Infection Surveillance Western Australia (HISWA) is a statewide surveillance program within the Healthcare Associated Infection Unit that includes analysis and feedback of surgical site infections (SSI) rates following hip and knee arthroplasty. Infected arthroplasty wounds are often difficult to treat and lead to repeat operations, long-term antibiotic treatment, prolonged hospital stays and increased healthcare costs.1 4 A recent Australian study showed the average excess length of stay per infection following arthroplasty was ~27 days.5 The average cost of an SSI following hip and knee arthroplasty was estimated to be AU$34 000 and AU$41 000 respectively.5 The WA hospitals that report post-arthroplasty SSI rates to HISWA use standardised surveillance definitions and incorporate conventional risk adjustment indices. Therefore, variations in reported rates should reflect real differences in the risk of a patient developing an SSI at each hospital, and comparison of rates should
Australian Infection Control Association 2009
reflect performance. However, as Wilson and colleagues6 suggest, the problem with using SSI rates alone as an indicator of performance is that the number of procedures available for analysis is relatively small and varies considerably between hospitals, making true differences difficult to detect. An alternative to simple rate comparison, funnel plots are a statistical process control method that have been used for the ongoing improvement of healthcare systems, processes and outcomes.7 9 A funnel plot is a plot of the precision of the measure (e.g. SSI rate) as the sample size (e.g. number of procedures) increases with corresponding ‘funnel shaped’ control limits. In the context of SSIs, rates based on a small number of procedures will have wider limits (reflecting less precision) than rates based on a large number of procedures. This is graphically represented on the funnel plot. As with other statistical process control methods, if a hospital rate falls outside the control limits of a funnel plot, this indicates excessive variation and that inherent differences exist, causing a significantly higher or lower infection rate than expected by random variation alone.10 Random variation occurs when a
DOI: 10.1071/HI09110
51
Healthcare Infection
hospital rate falls within the control limits, indicating that performance compared with other hospitals varied only by an amount consistent with chance. The amount of variation that can be attributed to random variation is greatest when the numbers of patients are small (left of the funnel plot) and reduces as the number of patients increases. We describe SSI rates after hip and knee arthroplasty reported by participating public and private hospitals in Western Australia over a 2-year period. The aim of the study was to present and compare aggregated and hospital-based SSI rates and to analyse rate differences.
Methodology During the study period, participation in the SSI surveillance program was voluntary and open to all public and private hospitals in WA. Between 1 July 2005 and 30 June 2007, 10 hospitals reported SSI rates after hip arthroplasty, and nine hospitals reported SSI rates after knee procedures. These included three tertiary, three non-tertiary (one regional) and four private hospitals. In order to support valid interfacility comparison, HISWA has addressed recommendations described by the Healthcare Infection Control Practices Advisory Committee11 including: 1. Use of nationally endorsed definitions by all sites for numerators (events) and denominators (see Appendix 1). 2. Provision of a detailed surveillance manual12 outlining inclusions, exclusions and stratifications, with training for contributors as required. 3. Risk-stratification based on the United States National Nosocomial Infection Surveillance (NNIS) system. 4. Similar methods and intensity of case detection. Definitions, inclusions and exclusions listed in Appendix 1 are consistent with nationally recommended criteria. Only SSIs detected during the admission period of the procedure or readmission to hospital are included in comparative data. HISWA participants are required to perform active, prospective, patient-based surveillance, although HISWA does not mandate specific case detection methods. Case detection and classification methods used by HISWA contributors were evaluated in 2006 by completion of a questionnaire with a follow-up workshop. Data has subsequently been validated by external audit in 2008, the results of which have been submitted for publication.13 Funnel plots were based on exact binomial limits (used for proportions) and display the rate of SSI (cumulative incidence) at each hospital plotted against operative volume. Confidence intervals were calculated using the exact method.14 Dashed lines represent the 95% and 99% upper and lower control limits (UCL and LCL, respectively) on the funnel plot. 52
L. Dailey et al.
Results HISWA SSI cohort From 1 July 2005 to 30 June 2007 a total of 4131 hip and 4858 knee arthroplasties were performed at hospitals participating in the HISWA program. The majority were primary procedures, with 459 (11%) hip and 280 (6%) knee revisions included. Almost threequarters of hip and knee arthroplasties were performed at private hospitals: 2921 (71%) hip and 3394 (70%) knee procedures. To our knowledge, these procedures were also performed at three private and three public WA hospitals that did not participate in this program. Using data collected by the Australian Orthopaedic Association Joint Replacement Registry, this voluntary HISWA program included 83% (1725/2078) of primary total hip and 78% (2082/ 2670) of primary total knee procedures performed in Western Australia in 2005/2006.15 Registry data was not available for the 2006/2007 period. In this cohort, the mean age of patients with an SSI was 70 years (range 34–88 years). This was comparable with the mean age of patients undergoing these procedures reported by the Australian Orthopaedic Association Joint Replacement Registry (69 years for total knee replacements, 67 years for total hip replacements).15 There was no significant difference in the average age of patients in this cohort with an SSI after hip or knee arthroplasty. A total of 79 (53%) SSIs that met the case definition were detected during the patient’s initial postoperative admission, and 69 (47%) at the time of re-admission. The median time to detection of an SSI was 13 days (range 2–308 days) following hip arthroplasty and 16 days (range 4–310 days) following knee arthroplasty. Over three-quarters (78%) of SSIs were detected within 30 days of the initial operation, and over 93% were identified within the first 4 months after the procedure. Over half, 52% (77/148), of all SSIs reported to HISWA following hip and knee arthroplasty were classified as deep/organ space infections. The proportion of infections classified as superficial at individual sites varied from 20 to 83%. A specimen from the surgical wound was obtained for microbiological culture from 115 (78%) of the 148 infections. Of these, 96 yielded positive microbiology. The majority (70%) of isolates were Gram-positive bacteria. Staphylococci were the most common, accounting for 64 (58%) isolates. There were 33 Staphylococcus aureus isolates, including nine that were methicillin resistant (MRSA: 8% of all isolates, 6% of all SSIs).
Reported SSI rates Over the 2-year period, a total of 75 SSIs following hip arthroplasty were reported by 10 participating sites, incorporating data from 4131 procedures and 222 months. The statewide quarterly
Healthcare Infection
Surgical site infection surveillance
infection rate ranged from 0.93 (95% CI 0.34–2.25) to 3.29 (95% CI 2.04–5.27) infections per 100 hip procedures. The cumulative infection rate following hip arthroplasty was 1.82 (95% CI 1.45–2.27) infections per 100 hip procedures. Over half (51%) of hip SSIs (38/75) were classified as deep/organ space infections. A total of 73 SSIs following 4858 knee arthroplasties were reported during 204 months of surveillance by nine participating sites. The statewide quarterly infection rate ranged from 0.52 (95% CI 0.11–1.60) to 2.46 (95% CI 1.50–4.01) infections per 100 knee procedures. The cumulative infection rate following knee arthroplasty over the 2 years was 1.50 (95% CI 1.20–1.89) infections per 100 procedures. Over half (53%) of knee SSIs (39/73) were classified as deep/organ space infections. Revisions accounted for 11% of hip and 6% of knee procedures. Infection rates after primary hip and knee arthroplasty procedures were lower than their respective revision procedures, although this difference was not statistically significant. The percentage of patients in each risk category is shown in Table 1. Only 4% (353/8989) of patients were in the highest risk category (risk 2) and there were 66% (5926/8989) in the lowest risk category (risk 0). The SSI rate for hip and knee arthroplasty increased with risk category (Figure 1).
Analysis of inter-hospital variation by operative volume Funnel plots were used to display control limits for SSI rate by hospital against the number of procedures6 (Figures 2 and 3). Hospitals are randomly assigned a number between 1 and 10, and those falling outside the control limits are labelled on the funnel plots (Figures 2–5). In general, a higher operative volume was associated with a lower infection rate.
Table 1.
risk category
Knee
By hospital, the knee SSI rate varied between 0.47 (95% CI 0.00–2.96) and 3.67 (95% CI 2.00–6.55) infections per 100 procedures. Two hospitals were identified as having rates outside the control limits. Knee SSI rates at hospital 4 were below the 95% LCL and below the 99% LCL for hospital 8, suggesting better overall performance (Figure 3).
Analysis of inter-hospital variation by risk index The proportion of patients classified as low risk (category 0) at each hospital ranged from 40 to 82% for hip procedures and from 29 to 81% for knee procedures. To investigate whether this variation accounted for differences between hospital SSI rates (i.e. whether lower SSI rates were reported by hospitals with a higher proportion of low-risk patients), funnel plots were constructed including only NNIS risk 0 patients (the largest single group, 66% of procedures). However, confining analysis to risk 0 patients after arthroplasty reduces the population size and potentially the chance of detecting significant variation. In a funnel plot analysis of risk 0 hip arthroplasty patients (not shown), hospitals 7 and 8, which were both below the LCLs when analysed by operative volume (Figure 2), were no longer outliers. Hospital 9 also fell below the 95% LCL when analysed by low-risk patients (risk 0), indicating better performance than expected. There were no longer any high outliers, suggesting that patient risk
Surgical site infection (SSI) rates following hip and knee arthroplasty, 2005–2007.
Procedure and Hip
The hip SSI rate varied by hospital between 0.67 (95% CI 0.00–4.13) and 3.83 (95% CI 2.37–6.11) infections per 100 procedures. Three hospitals were identified as having rates outside the control limits. Hospital 1 had a hip SSI rate above the 95% UCL, suggesting a higher overall rate than expected by chance alone (Figure 2). Hip SSI rates were below the 95% LCL for hospital 8 and below the 99% LCL for hospital 7, suggesting significantly better overall performance (Figure 2).
Number of
Number of
Superficial SSI A
Number of
Deep SSI A
Total number
SSI rateA
procedures
superficial SSIs
rate (95% CI)
deep SSIs
rate (95% CI)
of SSIs
(95% CI)
Risk 0
2725
17
0.62 (038–1.01)
14
0.51 (0.30–0.87)
31
1.14 (0.80–1.62)
Risk 1
1241
18
1.45 (0.91–2.30)
20
1.61 (1.04–2.50)
38
3.06 (2.23–4.19)
Risk 2
165
2
1.21 (0.07–4.66)
4
2.42 (0.76–6.34)
6
3.64 (1.53–7.94)
Total
4131
37
0.90 (0.65–1.24)
38
0.92 (0.67–1.27)
75
1.82 (1.45–2.27)
Risk 0
3201
20
0.62 (0.40–0.97)
14
0.44 (0.26–0.74)
34
1.06 (0.76–1.49)
Risk 1
1469
12
0.82 (0.45–1.45)
18
1.23 (0.77–1.95)
30
2.04 (1.43–2.92)
Risk 2
188
2
1.06 (0.06–4.10)
7
3.72 (1.70–7.68)
9
4.79 (2.44–9.02)
Total
4858
34
0.70 (0.50–0.98)
39
0.80 (0.59–1.10)
73
1.50 (1.20–1.89)
A
Rate per 100 procedures.
53
Healthcare Infection
L. Dailey et al.
Infection rate per 100 procedures
Figure 1. Hip and knee surgical site infection rate by risk category with 95% confidence intervals, 2005–2007. 10.0 9.0
factors accounted, at least in part, for the higher hip SSI rate at hospital 1 (Figure 2).
Hip Knee
In a funnel plot analysis of risk 0 knee arthroplasty patients, two high performers, which were both below the LCLs when analysed by operative volume (Figure 3), remained below the LCLs. Hospital 8 fell below the 95% LCL and the rate for hospital 4 was lower than the 99% LCL.
8.0 7.0 6.0 5.0 4.0 3.0
Analysis of interhospital variation in rates of deep and superficial SSI
2.0 1.0
It has been proposed that deep/organ space SSI may be more reliably detected by surveillance programs, and may be a better comparative measure.16 The rate of deep SSI after hip
0.0 Risk 0
Risk 1
Risk 2
Risk category
Figure 2.
Surgical site infection rate associated with hip arthroplasty in relation to operative volume by hospital, 2005–2007. 16.0 95% LCL 95% UCL 99% LCL 99% UCL Aggregate rate Hospital
Infection rate per 100 procedures
14.0 12.0 10.0 8.0 1
6.0
7
8
9
4.0 2.0 0.0 0
100
200
300
400
500
600
700
800
900
1000
Number of hip arthroplasties
Figure 3.
Surgical site infection rate associated with knee arthroplasty in relation to operative volume by hospital, 2005–2007. 7.0 95% LCL 95% UCL 99% LCL 99% UCL Aggregate rate Hospital
Infection rate per 100 procedures
6.0
5.0
4.0
3.0
2.0
1.0 4
8
0.0 0
200
400
600
800
Number of knee arthroplasties
54
1000
1200
1400
Healthcare Infection
Surgical site infection surveillance
Figure 4. Deep hip surgical site infection rate by hospital volume.
Deep infection rate per 100 procedures
14.0 95% LCL 95% UCL 99% LCL 99% UCL Aggregate rate Hospital
12.0
10.0
8.0
6.0 1
4.0
8
7
9
2.0 4
0.0 0
100
200
300
400
500
600
700
800
900
1000
Number of hip arthroplasties
Figure 5. Deep knee surgical site infection rate by hospital volume.
Deep infection rate per 100 procedures
6.0 95% LCL 95% UCL 99% LCL 99% UCL Aggregate rate Hospital
5.0
4.0
3.0
2.0 8
1.0
0.0 0
200
400
600
800
1000
1200
1400
Number of knee arthroplasties
arthroplasty at the 10 HISWA sites was 0.9 per 100 procedures (95% CI 0.67–1.27). One hospital (hospital 4) reported no deep hip SSIs, the highest rate was 3.2 per 100 procedures (95% CI 1.58–5.29). The rate of deep SSI after knee arthroplasty at the nine HISWA sites was 0.8 per 100 procedures (95% CI 0.6–1.1). Two hospitals (hospitals 6 and 10) reported no deep/organ space hip SSIs and the highest rate was 2.3 per 100 procedures (95% CI 1.1–4.9). Funnel plots were constructed to explore whether deep infection rates after hip and knee arthroplasty varied between hospitals to a similar extent as infection rates overall. Results are shown in Figures 4 and 5.
The rate of deep hip SSI at hospital 1 was above the 99% UCL. Hospitals 7, 8 and 9 reported deep/organ space hip SSI infection rates below the 95% LCL, showing a similar pattern to the analyses by operative volume. Figure 5 shows the deep/organ space knee infection rate at hospital 8 was below the 95% LCL (previously below the 99% LCL when analysed by operative volume).
Analysis of inter-hospital variation by surveillance methodology In 2006, infection control professionals at seven metropolitan hospitals completed a survey regarding SSI surveillance 55
Healthcare Infection
L. Dailey et al.
methodology at their site (Table 2). Following analysis by the Healthcare Associated Infection Unit, the findings were presented and discussed at a workshop and participants completed a set of hypothetical scenarios to assess inter-rate reliability. Two metropolitan hospitals contributing to the SSI surveillance indicator (hospitals 6 and 10) did not participate in this initial
survey and were not identified as outliers in any of the funnel plot analyses. Using hypothetical cases, there was an overall consensus (96%) on classification by participants, suggesting high inter-rater reliability. Following analysis and discussion, the participants
Table 2. Key findings of survey surgical site infection (SSI) surveillance methodology. ICD, international classification of disease; TMS, theatre management system. Surveillance process
% of hospitals using surveillance process (n = 7)
Obtaining denominator data Theatre management system
5/7 (71%)
Notification by theatre staff
4/7 (57%)
Other (e.g. coding and joint registry)
1/7 (14%)
Detection of SSI during the admission period of the procedure Ward rounds (at least once weekly)
6/7 (86%)
Ward rounds (twice weekly or more)
5/7 (71%)
Review of laboratory reports
6/7 (86%)
Prospective total chart review for all eligible patients
3/7 (43%)
Notification by ward staff
4/7 (57%)
ICD codes
3/7 (43%)
Review of microbiology consults
2/7 (29%)
SSI detected on readmission Identify readmissions on ward rounds
4/7 (57%)
ICD codes used to identify readmissions
5/7 (71%)
TMS reports used to identify readmissions
4/7 (57%)
Notification from theatre staff
1/7 (14%)
Notification from ward staff/surgical team
3/7 (43%)
Review of clinical microbiologist consults
5/7 (71%)
Other methods (e.g. bed management, readmission reports)
4/7 (57%)
Classification of SSI according to definition Mostly unilateral decision by infection control consultant
4/7 (57%)
Mostly consultation with surgical team
3/7 (43%)
Routine observation of wound to confirm SSI
0/7 (0%)
Internal validation of data
56
Numerator data validated
0/7 (0%)
Denominator data validated
1/7 (14%)
Healthcare Infection
Surgical site infection surveillance
expressed confidence in the sensitivity of case detection, primarily because of the use of multiple detection methods such as ward rounds combined with staff notification, chart review and coding reports. All sites used at least three methods for case detection, and despite minor differences in overall SSI surveillance processes, participants concluded a comparative intensity of surveillance practice across all hospitals. No hospitals performed regular internal validation of their SSI dataset but have subsequently participated in an external validation study. The results of this study will be published in full. However, in brief, surveillance processes resulted in an overall sensitivity of 83% and a specificity of 99%.13 In that study, the best performing (and high volume) hospitals identified using funnel plots in this analysis (hospitals 7, 8 and 9) were all confirmed to have a sensitivity and specificity for SSI detection and classification equal or superior to other participants, making it unlikely that variation is due to systematic differences in case finding and classification.
Discussion The comparison of SSI rates between hospitals or external benchmarks should be interpreted with caution,17 19 even within a standardised program such as HISWA. Potential differences can occur in case finding, case mix, interpretation of definitions and data validity, which are difficult to control for and may result in misleading conclusions from the surveillance data.20 Funnel plots demonstrated significant differences in SSI rates between participating hospitals even if analyses were confined to deep infections, or to a single risk group. Patients having hip arthroplasty at the hospital with the highest SSI rate, had 6-fold greater risk of having an SSI detected compared with the best performing hospital and, after knee arthroplasty, an 8-fold greater risk. This disparity between hospitals raises the question of potential causes for variation in reported SSI rates. If the variation represents real differences in performance, the hospitals with significantly lower rates are exemplars, or ‘positive deviants’.21 Better performance in this study could reflect systematic differences in the patients that present to various hospitals that are not accounted for in the NNIS risk index (i.e. patient factors), inaccurate data collection, case detection or classification bias (i.e. methodological variation), or meaningful differences in perioperative practices including surgical volume between the WA hospitals.
Variation in patient risk factors High hospital overall SSI rates may reflect differences in patient case mix. That is, some hospitals may operate on a higher proportion of patients with an elevated risk of infection compared with others. Our analyses using funnel plots showed persistent high performers even within a single NNIS risk group with the highest number of procedures (risk 0). This demonstrates that the
risk factors incorporated into this index can only partially account for the variation, and there are significant patient risk factors that are not captured in the NNIS classification (e.g. socioeconomic status, diabetes, obesity).22,23 It is possible that these could systematically vary between hospitals and persist as a confounder that we are unable to control for at present. Further research is required to investigate the utility of a more comprehensive risk stratification system.
Variation in methodology The variation in the proportion of SSIs classified as deep or superficial in WA is marked (20–83%), and has been noted as a possible confounder in comparing rates in a European collaborative study.24 26 However, deep SSI rates in WA reflect overall rates when analysed using funnel plots, suggesting that differences in deep versus superficial classification does not account for the inter-hospital variation. In addition, this data does not support the contention that stratification by infection type is essential, or that analyses should be confined to deep infections alone. The differences in surveillance processes in place at the participating hospitals reflect the reality of healthcare-associated infection surveillance. However, results of the external validation support the conclusions of this study, and suggest that the differences in SSI rates are not accounted for by systematic differences in surveillance methodology.13
Variation in hospital practices Examples of practices associated with better performance that may be relevant in this setting are numerous and include: better adherence to protocols for antibiotic prophylaxis, skin antisepsis and hair removal; presence of experienced surgeons with shorter operating time or superior technique; maintaining perioperative normothermia; shorter preoperative stay; optimal design of the theatre environment; higher use of active MRSA surveillance cultures to detect carriers; improved compliance with hand hygiene by healthcare workers; better perioperative glycaemic control in diabetic patients; perioperative antisepsis. We are currently unable to identify which of these factors are important in WA, but the results of this study suggest that the perioperative processes that are in place at identified high-performing hospitals are producing a meaningful reduction in adverse events and this provides opportunities for other units to learn from and improve their own practices. Hospital and surgeon-specific procedure volumes have been inversely associated with a range of surgical outcomes including infections following hip and knee replacement.27 29 Policy implications of such observations, and critical thresholds are debated in the orthopaedic literature.30 Our study supports the association of lower SSI rates with high hospital operative 57
Healthcare Infection
volume. We did not assess surgeon-specific procedure volumes in this study. Of note is the low number of MRSA (<10%) isolates detected in SSIs in this study. In comparison, rates of 30% are reported in the UK.31 This is consistent with the absence of MRSA as an endemic pathogen in WA acute care facilities, and the low rate of other healthcare-associated MRSA infections reported by WA hospitals.32,33 Funnel plots offer readily interpretable plots of multiple hospital comparisons, allow statistical process control assessment to be applied to complex data and have advantages over league tables.34,35 These plots provide an impetus for further investigation into the underlying causes of high infection rates6,35 and minimise the use of resources to investigate high rates of SSI that represent normal variation.10,36 WA hospitals are urged to continue to review and strive to improve reliable implementation of policies and practices relevant to SSI prevention, particularly if their reported rates are not decreasing and are higher than other WA sites. Active participation in improvement initiatives such as the Surgical Care Improvement Project37 and Safety and Quality Investment for Reform38 can reduce SSI rates.
Acknowledgements We would like to thank and acknowledge the contribution of the following hospitals: Armadale-Kelmscott Memorial Hospital, Fremantle Hospital, Hollywood Private Hospital, Royal Perth Hospital, Saint John of God Hospital Murdoch, Saint John of God Hospital Subiaco, Sir Charles Gairdner Hospital, Mount Hospital and Joondalup Health Campus.
References 1.
2.
Hebert CK, Williams RE, Levy RS, Barrack RL. Cost of treating an infected total knee replacement. Clin Orthop Relat Res 1996; 331: 140–5. doi:10.1097/00003086-199610000-00019 Gaine WJ, Ramamohan NA, Hussein NA, Hullin MG, McCreath SW. Wound infection in hip and knee arthroplasty. J Bone Joint Surg Br 2000; 82: 561–5. doi:10.1302/0301-620X.82B4.10305
3.
Lentino JR. Prosthetic joint infections: bane of orthopedists, challenge for infectious disease specialists. Clin Infect Dis 2003; 36: 1157–61. doi:10.1086/374554
4.
Darouiche RO. Treatment of infections associated with surgical implants. N Engl J Med 2004; 350: 1422–9. doi:10.1056/NEJMra035415
5.
VICNISS. VICNISS hospital acquired infection project, Year 4 report, August 2006. Melbourne: Victoria Department of Human Services; 2006. Available online at: http://www.vicniss.org.au/Resources/ VICNISSAnnualReport0806.pdf [verified May 2009].
6.
58
Wilson J, Ramboer I, Suetens C, HELICS-SSI working group. Hospitals in Europe link for infection control through surveillance (HELICS). Inter-country comparison of rates of surgical site infection – opportunities and limitations. J Hosp Infect 2007; 65: 165–70. doi:10.1016/S0195-6701(07)60037-1
L. Dailey et al. 7.
Carey RG. Improving healthcare with control charts: basic and advanced SPC methods and case studies. Milwaukee: Quality Press; 2003.
8.
Bolsin S. Was Bristol an outlier? Lancet 2001; 358: 2084.
9.
Colson M, Bolsin S. The use of statistical process control methods in monitoring clinical performance. Int J Qual Health Care 2003; 15: 445. doi:10.1093/intqhc/mzg070
10. Spielgelhalter DJ. Funnel plots for comparing institutional performance. Stat Med 2005; 24: 1185–202. doi:10.1002/sim.1970 11. McKibben L, Horan T, Tokars JI, Fowler G, Cardo D, Pearson ML, et al. Guidance on public reporting of healthcare-associated infections: recommendations of the Healthcare Infection Control Practices Advisory Committee. Am J Infect Control 2005; 33: 217–26. doi:10.1016/j.ajic.2005.04.001 12. HISWA. HISWA surveillance manual, Version 2, July 2008. Perth: Department of Health Western Australia; 2008. Available online at: http://www.public.health.wa.gov.au/cproot/813/2/Revised% 20Surveillance%20Manual%20July%202008.pdf [verified May 2009]. 13. Goggin L, van Gessel H, McCann R, Peterson A. Validation of surgical site infection surveillance in Perth, Western Australia. Healthcare Infect 2009; 14: in press. 14. Clopper C, Pearson S. The use of confidence or fiducial limits illustrated in the case of the binomial. Biometrika 1934; 26: 404–13. doi:10.1093/biomet/26.4.404 15. Australian Orthopaedic Association National Joint Replacement Registry. Annual Report 2007. Adelaide: AOA; 2007. 16. Mannie¨n J, van der Zeeuw AE, Willie JC, van den Hof S. Validation of surgical site infection in the Netherlands. Infect Control Hosp Epidemiol 2007; 28: 36–41. doi:10.1086/509847 17. Mertens R, Van den Berg JM, Veerman-Brenzikofer ML, Jurz X, Jans B, Klazinga N. International comparison of results of infection surveillance: The Netherlands versus Belgium. Infect Control Hosp Epidemiol 1994; 15: 574–8. 18. Coello R, Gastmeier P, de Boer AS. Surveillance of hospital-acquired infection in England, Germany, and The Netherlands: will international comparison of rates be possible? Infect Control Hosp Epidemiol 2001; 22: 393–7. doi:10.1086/501923 19. Barnes S, Salemi C, Fithian D, Akiyama L, Barron D, Eck E, et al. An enhanced benchmark for prosthetic joint replacement infection rates. Am J Infect Control 2006; 34: 669–72. doi:10.1016/j.ajic.2006.04.207 20. NNIS. Nosocomial infection rates for interhospital comparison: limitations and possible solutions. A report from the National Nosocomial Infections Surveillance (NNIS) System. Infect Control Hosp Epidemiol 1991; 12: 609–21. 21. The Positive Deviance Initiative. Boston: The Positive Deviance Initiative; 2009. Available online at: http://www.positivedeviance. org/ [verified May 2009]. 22. Agabiti N, Picciotto S, Cesaroni G, Bisanti L, Forastiere F, Onorati R, et al. The influence of socioeconomic status on utilization and outcomes of elective total hip replacement: a multicity populationbased longitudinal study. Int J Qual Health Care 2007; 19: 37–44. doi:10.1093/intqhc/mzl065 23. Cantu¨rk Z, Canturk N, Cetinarslan B, Utkan N, Tarkun I. Nosocomial infections and obesity in surgical patients. Obes Res 2003; 11: 769–75. doi:10.1038/oby.2003.107 24. Suetens C, De Laet C. Helics – SSI Statistical report 2004. 2004. Available online at: http://helics.univ-lyon1.fr/Documents/ HELICS-SSI%20Stat%20Report%202004%20Final%20Version% 20180406.pdf\ [verified May 2009]. 25. Friedman ND, Russo PL, Bull A, Richards MJ, Kelly H. Validation of coronary artery bypass graft surgical site infection surveillance data from a state-wide surveillance system in Australia. Infect Control Hosp Epidemiol 2007; 28: 812–17. doi:10.1086/518455
Surgical site infection surveillance 26. Russo PL, Gurrin L, Friedman ND, Bull A, Marasco S, Kelly H, et al. Interhospital comparisons of coronary artery bypass graft surgical site infection rates differ if donor sites are excluded. Infect Control Hosp Epidemiol 2007; 28: 1210–12. doi:10.1086/520749 27. Katz J, Losina E, Barrett J, Phillips C, Mahomed N, Lew R, et al. Association between hospital and surgeon procedure volume and outcomes in total hip replacement in the United States Medicare population. J Bone Joint Surg Am 2001; 83: 1622–9. 28. Katz J, Barrett J, Mahomed N, Baron J, Wright R, Losina E. Association between hospital and surgeon procedure volume and the outcomes of total knee replacement. J Bone Joint Surg Am 2004; 86: 1909–16. 29. Hervey SL, Purves HR, Guller U, Toth AP, Vail TP, Pietrobon R. Provider volume of total knee arthroplasties and patient outcomes in the HCUP: nationwide inpatient sample. J Bone Joint Surg Am 2003; 85: 1775–83. 30. Katz J, Losina E, Barrett J, Phillips C, Mahomed N, Lew R, et al. Volumes and outcomes in orthopaedic procedures: scientific and policy considerations. J Bone Joint Surg Am 2002. Available online at: http://www.ejbjs.org/cgi/eletters/83/11/1622 [accessed March 2009]. 31. Health Protection Agency. Third report of the mandatory surveillance of surgical site infection in orthopaedic surgery. April 2004 to March 2007. London: Health Protection Agency; 2007. Available online at: http://www.hpa.org.uk/ [verified May 2009].
Healthcare Infection 32. Ferguson JK, van Gessel H. Methicillin-resistant Staphylococcus aureus in hospitals: time for a culture change. MJA 2008; 188: 62. 33. Silla RC, Fong J, Wright J, Wood F. Infection in acute burn wounds following the Bali bombings: a comparative prospective audit. Burns 2006; 32: 139–44. doi:10.1016/j.burns.2005.09.002 34. Gale C, Roberts A, Batin P, Hall A. Funnel plots, performance variation and the Myocardial Infarction National Audit Project 2003–2004. BMC Cardiovasc Disord 2006; 6: 34. doi:10.1186/1471-22616-34 35. Morton A, Clements A, Doidge S, Stackelroth J, Curtis M, Whitby M. Surveillance of healthcare-acquired infections in Queensland, Australia: data and lessons from the first five years. Infect Control Hosp Epidemiol 2008; 29: 695–701. doi:10.1086/589904 36. Benneyan JC. Statistical quality control methods in infections control and hospital epidemiology, part 1: introduction and basic theory. Infect Control Hosp Epidemiol 1998; 19: 194–214. 37. Premier. A supporting partnership organization of the Surgical Care Improvement Project (SCIP). Available online at: http://www. premierinc.com/safety/topics/scip/index.jsp [verified May 2009]. 38. Department of Health, Western Australia. Safety and Quality Investment for Reform (SQuIRe). Available online at: http://www. safetyandquality.health.wa.gov.au/squire/index.cfm [verified May 2009].
59
Healthcare Infection
L. Dailey et al.
Appendix 1 Definitions of surgical site infection for Healthcare Infection Surveillance Western Australia surveillance system Superficial incisional surgical site infection * * *
Involves only skin and subcutaneous tissue. Occurs within 30 days of the operative procedure. Meets one of the following criteria from the incision: 1. 2. 3. 4.
purulent discharge (not stitch abscess); organisms are isolated from aseptically collected culture of fluid or tissue; diagnosis of infection or antimicrobial treatment by the operating surgeon or registrar; displays any of the following signs and symptoms at the incision site: pain or tenderness, localised swelling, redness or heat AND the incision is deliberately explored by the surgeon resulting in a positive wound culture.
Deep incisional/organ space surgical site infection * * *
Infection involves deep soft tissues (e.g. fascial and muscle layers) and/or organ spaces opened or manipulated during an operation. Occurs within 30 days after the operative procedure if implant not present or within 1 year if implant in situ. Exhibits either one or both of the following: 1. purulent drainage from deep soft tissue or drain that is placed through a stab wound into the organ/space; 2. spontaneous dehiscence at the incision site or the wound is deliberately explored by a surgeon with the patient showing evidence of one or more of the following signs or symptoms: *
* *
* *
fever >38‡C, localised pain or tenderness with culture-positive specimen. A culture-negative finding does not meet this criterion unless the patient was on antibiotics immediately prior to the wound being explored and/or the culture being taken. organisms isolated from aseptically obtained culture of fluid or tissue obtained from an organ/space. an abscess or other evidence of infection involving a deep/organ space is found on direct examination, during re-operation, or by histopathologic or radiologic examination. diagnosis of, or antimicrobial treatment of, a deep incisional or organ/space SSI by the operating surgeon or registrar.
Specific sites of an organ/space SSI include: osteomyelitis, joint or bursa.
Inclusions and exclusions for the Healthcare Infection Surveillance Western Australia surveillance system Inclusions *
All elective arthroplasty procedures: total, revision and partial.
Exclusions * * *
Revision procedures performed to remove an already infected joint. Emergency arthroplasty procedures (e.g. hemiarthroplasty of neck of femur). SSIs detected and treated as outpatients.
60
Two years of surgical site infection surveillance in Western Australia: analysing variation between hospitals Lynne Dailey1,4 PhD, MPH Helen van Gessel2,3 MBBS, MPH Allison Peterson3 BHSc, Cert IC 1
Western Australian Country Health Service (formally Healthcare Associated Infection Unit, HCAIU), Department of Health, Perth, WA 6892, Australia. 2 Office of Safety and Quality in Healthcare, Department of Health, Perth, WA 6849, Australia. 3 (HCAIU) Department of Health, Perth, WA 6849, Australia. 4 Corresponding author. Email: [email protected]
Abstract Surgical site infections (SSIs) following hip and knee arthroplasty are significant adverse events. The aim of this study was to present the first 2 years of aggregated and hospital-based SSI rates from the Healthcare Infection Surveillance Western Australia (HISWA) surveillance system. HISWA collects risk-adjusted data based on standardised collection methodologies and definitions. In total, 4131 hip and 4858 knee procedures were monitored at 10 public and private hospitals. There were significant variations in reported SSI rates following arthroplasty at WA hospitals (mean 1.82, range 0.67–3.83 SSIs per 100 hip procedures; mean 1.5, range 0.47– 3.67 SSIs per 100 knee procedures). Funnel plots identified hospitals with significantly high and low SSI rates, in particular hospitals with high operative volumes and significantly better outcomes. Significant variation was observed when analysis was performed by operative volume, United States National Nosocomial Infection Surveillance (NNIS) risk group and infection type (deep/organ space v. superficial). Investigations did not reveal marked differences in surveillance methodology between sites, but other causes of variation, including patient factors that are not incorporated in NNIS risk indexation, surgeon procedure volume and differences in SSI prevention policy and practice, need further evaluation. The policy implications of the association between operative volume and SSI rate merits further discussion in Australia.
Introduction Healthcare-associated infection prevention has been identified as a priority initiative by the Department of Health in Western Australia. Established in 2005, Healthcare Infection Surveillance Western Australia (HISWA) is a statewide surveillance program within the Healthcare Associated Infection Unit that includes analysis and feedback of surgical site infections (SSI) rates following hip and knee arthroplasty. Infected arthroplasty wounds are often difficult to treat and lead to repeat operations, long-term antibiotic treatment, prolonged hospital stays and increased healthcare costs.1 4 A recent Australian study showed the average excess length of stay per infection following arthroplasty was ~27 days.5 The average cost of an SSI following hip and knee arthroplasty was estimated to be AU$34 000 and AU$41 000 respectively.5 The WA hospitals that report post-arthroplasty SSI rates to HISWA use standardised surveillance definitions and incorporate conventional risk adjustment indices. Therefore, variations in reported rates should reflect real differences in the risk of a patient developing an SSI at each hospital, and comparison of rates should
Australian Infection Control Association 2009
reflect performance. However, as Wilson and colleagues6 suggest, the problem with using SSI rates alone as an indicator of performance is that the number of procedures available for analysis is relatively small and varies considerably between hospitals, making true differences difficult to detect. An alternative to simple rate comparison, funnel plots are a statistical process control method that have been used for the ongoing improvement of healthcare systems, processes and outcomes.7 9 A funnel plot is a plot of the precision of the measure (e.g. SSI rate) as the sample size (e.g. number of procedures) increases with corresponding ‘funnel shaped’ control limits. In the context of SSIs, rates based on a small number of procedures will have wider limits (reflecting less precision) than rates based on a large number of procedures. This is graphically represented on the funnel plot. As with other statistical process control methods, if a hospital rate falls outside the control limits of a funnel plot, this indicates excessive variation and that inherent differences exist, causing a significantly higher or lower infection rate than expected by random variation alone.10 Random variation occurs when a
DOI: 10.1071/HI09110
51
Healthcare Infection
hospital rate falls within the control limits, indicating that performance compared with other hospitals varied only by an amount consistent with chance. The amount of variation that can be attributed to random variation is greatest when the numbers of patients are small (left of the funnel plot) and reduces as the number of patients increases. We describe SSI rates after hip and knee arthroplasty reported by participating public and private hospitals in Western Australia over a 2-year period. The aim of the study was to present and compare aggregated and hospital-based SSI rates and to analyse rate differences.
Methodology During the study period, participation in the SSI surveillance program was voluntary and open to all public and private hospitals in WA. Between 1 July 2005 and 30 June 2007, 10 hospitals reported SSI rates after hip arthroplasty, and nine hospitals reported SSI rates after knee procedures. These included three tertiary, three non-tertiary (one regional) and four private hospitals. In order to support valid interfacility comparison, HISWA has addressed recommendations described by the Healthcare Infection Control Practices Advisory Committee11 including: 1. Use of nationally endorsed definitions by all sites for numerators (events) and denominators (see Appendix 1). 2. Provision of a detailed surveillance manual12 outlining inclusions, exclusions and stratifications, with training for contributors as required. 3. Risk-stratification based on the United States National Nosocomial Infection Surveillance (NNIS) system. 4. Similar methods and intensity of case detection. Definitions, inclusions and exclusions listed in Appendix 1 are consistent with nationally recommended criteria. Only SSIs detected during the admission period of the procedure or readmission to hospital are included in comparative data. HISWA participants are required to perform active, prospective, patient-based surveillance, although HISWA does not mandate specific case detection methods. Case detection and classification methods used by HISWA contributors were evaluated in 2006 by completion of a questionnaire with a follow-up workshop. Data has subsequently been validated by external audit in 2008, the results of which have been submitted for publication.13 Funnel plots were based on exact binomial limits (used for proportions) and display the rate of SSI (cumulative incidence) at each hospital plotted against operative volume. Confidence intervals were calculated using the exact method.14 Dashed lines represent the 95% and 99% upper and lower control limits (UCL and LCL, respectively) on the funnel plot. 52
L. Dailey et al.
Results HISWA SSI cohort From 1 July 2005 to 30 June 2007 a total of 4131 hip and 4858 knee arthroplasties were performed at hospitals participating in the HISWA program. The majority were primary procedures, with 459 (11%) hip and 280 (6%) knee revisions included. Almost threequarters of hip and knee arthroplasties were performed at private hospitals: 2921 (71%) hip and 3394 (70%) knee procedures. To our knowledge, these procedures were also performed at three private and three public WA hospitals that did not participate in this program. Using data collected by the Australian Orthopaedic Association Joint Replacement Registry, this voluntary HISWA program included 83% (1725/2078) of primary total hip and 78% (2082/ 2670) of primary total knee procedures performed in Western Australia in 2005/2006.15 Registry data was not available for the 2006/2007 period. In this cohort, the mean age of patients with an SSI was 70 years (range 34–88 years). This was comparable with the mean age of patients undergoing these procedures reported by the Australian Orthopaedic Association Joint Replacement Registry (69 years for total knee replacements, 67 years for total hip replacements).15 There was no significant difference in the average age of patients in this cohort with an SSI after hip or knee arthroplasty. A total of 79 (53%) SSIs that met the case definition were detected during the patient’s initial postoperative admission, and 69 (47%) at the time of re-admission. The median time to detection of an SSI was 13 days (range 2–308 days) following hip arthroplasty and 16 days (range 4–310 days) following knee arthroplasty. Over three-quarters (78%) of SSIs were detected within 30 days of the initial operation, and over 93% were identified within the first 4 months after the procedure. Over half, 52% (77/148), of all SSIs reported to HISWA following hip and knee arthroplasty were classified as deep/organ space infections. The proportion of infections classified as superficial at individual sites varied from 20 to 83%. A specimen from the surgical wound was obtained for microbiological culture from 115 (78%) of the 148 infections. Of these, 96 yielded positive microbiology. The majority (70%) of isolates were Gram-positive bacteria. Staphylococci were the most common, accounting for 64 (58%) isolates. There were 33 Staphylococcus aureus isolates, including nine that were methicillin resistant (MRSA: 8% of all isolates, 6% of all SSIs).
Reported SSI rates Over the 2-year period, a total of 75 SSIs following hip arthroplasty were reported by 10 participating sites, incorporating data from 4131 procedures and 222 months. The statewide quarterly
Healthcare Infection
Surgical site infection surveillance
infection rate ranged from 0.93 (95% CI 0.34–2.25) to 3.29 (95% CI 2.04–5.27) infections per 100 hip procedures. The cumulative infection rate following hip arthroplasty was 1.82 (95% CI 1.45–2.27) infections per 100 hip procedures. Over half (51%) of hip SSIs (38/75) were classified as deep/organ space infections. A total of 73 SSIs following 4858 knee arthroplasties were reported during 204 months of surveillance by nine participating sites. The statewide quarterly infection rate ranged from 0.52 (95% CI 0.11–1.60) to 2.46 (95% CI 1.50–4.01) infections per 100 knee procedures. The cumulative infection rate following knee arthroplasty over the 2 years was 1.50 (95% CI 1.20–1.89) infections per 100 procedures. Over half (53%) of knee SSIs (39/73) were classified as deep/organ space infections. Revisions accounted for 11% of hip and 6% of knee procedures. Infection rates after primary hip and knee arthroplasty procedures were lower than their respective revision procedures, although this difference was not statistically significant. The percentage of patients in each risk category is shown in Table 1. Only 4% (353/8989) of patients were in the highest risk category (risk 2) and there were 66% (5926/8989) in the lowest risk category (risk 0). The SSI rate for hip and knee arthroplasty increased with risk category (Figure 1).
Analysis of inter-hospital variation by operative volume Funnel plots were used to display control limits for SSI rate by hospital against the number of procedures6 (Figures 2 and 3). Hospitals are randomly assigned a number between 1 and 10, and those falling outside the control limits are labelled on the funnel plots (Figures 2–5). In general, a higher operative volume was associated with a lower infection rate.
Table 1.
risk category
Knee
By hospital, the knee SSI rate varied between 0.47 (95% CI 0.00–2.96) and 3.67 (95% CI 2.00–6.55) infections per 100 procedures. Two hospitals were identified as having rates outside the control limits. Knee SSI rates at hospital 4 were below the 95% LCL and below the 99% LCL for hospital 8, suggesting better overall performance (Figure 3).
Analysis of inter-hospital variation by risk index The proportion of patients classified as low risk (category 0) at each hospital ranged from 40 to 82% for hip procedures and from 29 to 81% for knee procedures. To investigate whether this variation accounted for differences between hospital SSI rates (i.e. whether lower SSI rates were reported by hospitals with a higher proportion of low-risk patients), funnel plots were constructed including only NNIS risk 0 patients (the largest single group, 66% of procedures). However, confining analysis to risk 0 patients after arthroplasty reduces the population size and potentially the chance of detecting significant variation. In a funnel plot analysis of risk 0 hip arthroplasty patients (not shown), hospitals 7 and 8, which were both below the LCLs when analysed by operative volume (Figure 2), were no longer outliers. Hospital 9 also fell below the 95% LCL when analysed by low-risk patients (risk 0), indicating better performance than expected. There were no longer any high outliers, suggesting that patient risk
Surgical site infection (SSI) rates following hip and knee arthroplasty, 2005–2007.
Procedure and Hip
The hip SSI rate varied by hospital between 0.67 (95% CI 0.00–4.13) and 3.83 (95% CI 2.37–6.11) infections per 100 procedures. Three hospitals were identified as having rates outside the control limits. Hospital 1 had a hip SSI rate above the 95% UCL, suggesting a higher overall rate than expected by chance alone (Figure 2). Hip SSI rates were below the 95% LCL for hospital 8 and below the 99% LCL for hospital 7, suggesting significantly better overall performance (Figure 2).
Number of
Number of
Superficial SSI A
Number of
Deep SSI A
Total number
SSI rateA
procedures
superficial SSIs
rate (95% CI)
deep SSIs
rate (95% CI)
of SSIs
(95% CI)
Risk 0
2725
17
0.62 (038–1.01)
14
0.51 (0.30–0.87)
31
1.14 (0.80–1.62)
Risk 1
1241
18
1.45 (0.91–2.30)
20
1.61 (1.04–2.50)
38
3.06 (2.23–4.19)
Risk 2
165
2
1.21 (0.07–4.66)
4
2.42 (0.76–6.34)
6
3.64 (1.53–7.94)
Total
4131
37
0.90 (0.65–1.24)
38
0.92 (0.67–1.27)
75
1.82 (1.45–2.27)
Risk 0
3201
20
0.62 (0.40–0.97)
14
0.44 (0.26–0.74)
34
1.06 (0.76–1.49)
Risk 1
1469
12
0.82 (0.45–1.45)
18
1.23 (0.77–1.95)
30
2.04 (1.43–2.92)
Risk 2
188
2
1.06 (0.06–4.10)
7
3.72 (1.70–7.68)
9
4.79 (2.44–9.02)
Total
4858
34
0.70 (0.50–0.98)
39
0.80 (0.59–1.10)
73
1.50 (1.20–1.89)
A
Rate per 100 procedures.
53
Healthcare Infection
L. Dailey et al.
Infection rate per 100 procedures
Figure 1. Hip and knee surgical site infection rate by risk category with 95% confidence intervals, 2005–2007. 10.0 9.0
factors accounted, at least in part, for the higher hip SSI rate at hospital 1 (Figure 2).
Hip Knee
In a funnel plot analysis of risk 0 knee arthroplasty patients, two high performers, which were both below the LCLs when analysed by operative volume (Figure 3), remained below the LCLs. Hospital 8 fell below the 95% LCL and the rate for hospital 4 was lower than the 99% LCL.
8.0 7.0 6.0 5.0 4.0 3.0
Analysis of interhospital variation in rates of deep and superficial SSI
2.0 1.0
It has been proposed that deep/organ space SSI may be more reliably detected by surveillance programs, and may be a better comparative measure.16 The rate of deep SSI after hip
0.0 Risk 0
Risk 1
Risk 2
Risk category
Figure 2.
Surgical site infection rate associated with hip arthroplasty in relation to operative volume by hospital, 2005–2007. 16.0 95% LCL 95% UCL 99% LCL 99% UCL Aggregate rate Hospital
Infection rate per 100 procedures
14.0 12.0 10.0 8.0 1
6.0
7
8
9
4.0 2.0 0.0 0
100
200
300
400
500
600
700
800
900
1000
Number of hip arthroplasties
Figure 3.
Surgical site infection rate associated with knee arthroplasty in relation to operative volume by hospital, 2005–2007. 7.0 95% LCL 95% UCL 99% LCL 99% UCL Aggregate rate Hospital
Infection rate per 100 procedures
6.0
5.0
4.0
3.0
2.0
1.0 4
8
0.0 0
200
400
600
800
Number of knee arthroplasties
54
1000
1200
1400
Healthcare Infection
Surgical site infection surveillance
Figure 4. Deep hip surgical site infection rate by hospital volume.
Deep infection rate per 100 procedures
14.0 95% LCL 95% UCL 99% LCL 99% UCL Aggregate rate Hospital
12.0
10.0
8.0
6.0 1
4.0
8
7
9
2.0 4
0.0 0
100
200
300
400
500
600
700
800
900
1000
Number of hip arthroplasties
Figure 5. Deep knee surgical site infection rate by hospital volume.
Deep infection rate per 100 procedures
6.0 95% LCL 95% UCL 99% LCL 99% UCL Aggregate rate Hospital
5.0
4.0
3.0
2.0 8
1.0
0.0 0
200
400
600
800
1000
1200
1400
Number of knee arthroplasties
arthroplasty at the 10 HISWA sites was 0.9 per 100 procedures (95% CI 0.67–1.27). One hospital (hospital 4) reported no deep hip SSIs, the highest rate was 3.2 per 100 procedures (95% CI 1.58–5.29). The rate of deep SSI after knee arthroplasty at the nine HISWA sites was 0.8 per 100 procedures (95% CI 0.6–1.1). Two hospitals (hospitals 6 and 10) reported no deep/organ space hip SSIs and the highest rate was 2.3 per 100 procedures (95% CI 1.1–4.9). Funnel plots were constructed to explore whether deep infection rates after hip and knee arthroplasty varied between hospitals to a similar extent as infection rates overall. Results are shown in Figures 4 and 5.
The rate of deep hip SSI at hospital 1 was above the 99% UCL. Hospitals 7, 8 and 9 reported deep/organ space hip SSI infection rates below the 95% LCL, showing a similar pattern to the analyses by operative volume. Figure 5 shows the deep/organ space knee infection rate at hospital 8 was below the 95% LCL (previously below the 99% LCL when analysed by operative volume).
Analysis of inter-hospital variation by surveillance methodology In 2006, infection control professionals at seven metropolitan hospitals completed a survey regarding SSI surveillance 55
Healthcare Infection
L. Dailey et al.
methodology at their site (Table 2). Following analysis by the Healthcare Associated Infection Unit, the findings were presented and discussed at a workshop and participants completed a set of hypothetical scenarios to assess inter-rate reliability. Two metropolitan hospitals contributing to the SSI surveillance indicator (hospitals 6 and 10) did not participate in this initial
survey and were not identified as outliers in any of the funnel plot analyses. Using hypothetical cases, there was an overall consensus (96%) on classification by participants, suggesting high inter-rater reliability. Following analysis and discussion, the participants
Table 2. Key findings of survey surgical site infection (SSI) surveillance methodology. ICD, international classification of disease; TMS, theatre management system. Surveillance process
% of hospitals using surveillance process (n = 7)
Obtaining denominator data Theatre management system
5/7 (71%)
Notification by theatre staff
4/7 (57%)
Other (e.g. coding and joint registry)
1/7 (14%)
Detection of SSI during the admission period of the procedure Ward rounds (at least once weekly)
6/7 (86%)
Ward rounds (twice weekly or more)
5/7 (71%)
Review of laboratory reports
6/7 (86%)
Prospective total chart review for all eligible patients
3/7 (43%)
Notification by ward staff
4/7 (57%)
ICD codes
3/7 (43%)
Review of microbiology consults
2/7 (29%)
SSI detected on readmission Identify readmissions on ward rounds
4/7 (57%)
ICD codes used to identify readmissions
5/7 (71%)
TMS reports used to identify readmissions
4/7 (57%)
Notification from theatre staff
1/7 (14%)
Notification from ward staff/surgical team
3/7 (43%)
Review of clinical microbiologist consults
5/7 (71%)
Other methods (e.g. bed management, readmission reports)
4/7 (57%)
Classification of SSI according to definition Mostly unilateral decision by infection control consultant
4/7 (57%)
Mostly consultation with surgical team
3/7 (43%)
Routine observation of wound to confirm SSI
0/7 (0%)
Internal validation of data
56
Numerator data validated
0/7 (0%)
Denominator data validated
1/7 (14%)
Healthcare Infection
Surgical site infection surveillance
expressed confidence in the sensitivity of case detection, primarily because of the use of multiple detection methods such as ward rounds combined with staff notification, chart review and coding reports. All sites used at least three methods for case detection, and despite minor differences in overall SSI surveillance processes, participants concluded a comparative intensity of surveillance practice across all hospitals. No hospitals performed regular internal validation of their SSI dataset but have subsequently participated in an external validation study. The results of this study will be published in full. However, in brief, surveillance processes resulted in an overall sensitivity of 83% and a specificity of 99%.13 In that study, the best performing (and high volume) hospitals identified using funnel plots in this analysis (hospitals 7, 8 and 9) were all confirmed to have a sensitivity and specificity for SSI detection and classification equal or superior to other participants, making it unlikely that variation is due to systematic differences in case finding and classification.
Discussion The comparison of SSI rates between hospitals or external benchmarks should be interpreted with caution,17 19 even within a standardised program such as HISWA. Potential differences can occur in case finding, case mix, interpretation of definitions and data validity, which are difficult to control for and may result in misleading conclusions from the surveillance data.20 Funnel plots demonstrated significant differences in SSI rates between participating hospitals even if analyses were confined to deep infections, or to a single risk group. Patients having hip arthroplasty at the hospital with the highest SSI rate, had 6-fold greater risk of having an SSI detected compared with the best performing hospital and, after knee arthroplasty, an 8-fold greater risk. This disparity between hospitals raises the question of potential causes for variation in reported SSI rates. If the variation represents real differences in performance, the hospitals with significantly lower rates are exemplars, or ‘positive deviants’.21 Better performance in this study could reflect systematic differences in the patients that present to various hospitals that are not accounted for in the NNIS risk index (i.e. patient factors), inaccurate data collection, case detection or classification bias (i.e. methodological variation), or meaningful differences in perioperative practices including surgical volume between the WA hospitals.
Variation in patient risk factors High hospital overall SSI rates may reflect differences in patient case mix. That is, some hospitals may operate on a higher proportion of patients with an elevated risk of infection compared with others. Our analyses using funnel plots showed persistent high performers even within a single NNIS risk group with the highest number of procedures (risk 0). This demonstrates that the
risk factors incorporated into this index can only partially account for the variation, and there are significant patient risk factors that are not captured in the NNIS classification (e.g. socioeconomic status, diabetes, obesity).22,23 It is possible that these could systematically vary between hospitals and persist as a confounder that we are unable to control for at present. Further research is required to investigate the utility of a more comprehensive risk stratification system.
Variation in methodology The variation in the proportion of SSIs classified as deep or superficial in WA is marked (20–83%), and has been noted as a possible confounder in comparing rates in a European collaborative study.24 26 However, deep SSI rates in WA reflect overall rates when analysed using funnel plots, suggesting that differences in deep versus superficial classification does not account for the inter-hospital variation. In addition, this data does not support the contention that stratification by infection type is essential, or that analyses should be confined to deep infections alone. The differences in surveillance processes in place at the participating hospitals reflect the reality of healthcare-associated infection surveillance. However, results of the external validation support the conclusions of this study, and suggest that the differences in SSI rates are not accounted for by systematic differences in surveillance methodology.13
Variation in hospital practices Examples of practices associated with better performance that may be relevant in this setting are numerous and include: better adherence to protocols for antibiotic prophylaxis, skin antisepsis and hair removal; presence of experienced surgeons with shorter operating time or superior technique; maintaining perioperative normothermia; shorter preoperative stay; optimal design of the theatre environment; higher use of active MRSA surveillance cultures to detect carriers; improved compliance with hand hygiene by healthcare workers; better perioperative glycaemic control in diabetic patients; perioperative antisepsis. We are currently unable to identify which of these factors are important in WA, but the results of this study suggest that the perioperative processes that are in place at identified high-performing hospitals are producing a meaningful reduction in adverse events and this provides opportunities for other units to learn from and improve their own practices. Hospital and surgeon-specific procedure volumes have been inversely associated with a range of surgical outcomes including infections following hip and knee replacement.27 29 Policy implications of such observations, and critical thresholds are debated in the orthopaedic literature.30 Our study supports the association of lower SSI rates with high hospital operative 57
Healthcare Infection
volume. We did not assess surgeon-specific procedure volumes in this study. Of note is the low number of MRSA (<10%) isolates detected in SSIs in this study. In comparison, rates of 30% are reported in the UK.31 This is consistent with the absence of MRSA as an endemic pathogen in WA acute care facilities, and the low rate of other healthcare-associated MRSA infections reported by WA hospitals.32,33 Funnel plots offer readily interpretable plots of multiple hospital comparisons, allow statistical process control assessment to be applied to complex data and have advantages over league tables.34,35 These plots provide an impetus for further investigation into the underlying causes of high infection rates6,35 and minimise the use of resources to investigate high rates of SSI that represent normal variation.10,36 WA hospitals are urged to continue to review and strive to improve reliable implementation of policies and practices relevant to SSI prevention, particularly if their reported rates are not decreasing and are higher than other WA sites. Active participation in improvement initiatives such as the Surgical Care Improvement Project37 and Safety and Quality Investment for Reform38 can reduce SSI rates.
Acknowledgements We would like to thank and acknowledge the contribution of the following hospitals: Armadale-Kelmscott Memorial Hospital, Fremantle Hospital, Hollywood Private Hospital, Royal Perth Hospital, Saint John of God Hospital Murdoch, Saint John of God Hospital Subiaco, Sir Charles Gairdner Hospital, Mount Hospital and Joondalup Health Campus.
References 1.
2.
Hebert CK, Williams RE, Levy RS, Barrack RL. Cost of treating an infected total knee replacement. Clin Orthop Relat Res 1996; 331: 140–5. doi:10.1097/00003086-199610000-00019 Gaine WJ, Ramamohan NA, Hussein NA, Hullin MG, McCreath SW. Wound infection in hip and knee arthroplasty. J Bone Joint Surg Br 2000; 82: 561–5. doi:10.1302/0301-620X.82B4.10305
3.
Lentino JR. Prosthetic joint infections: bane of orthopedists, challenge for infectious disease specialists. Clin Infect Dis 2003; 36: 1157–61. doi:10.1086/374554
4.
Darouiche RO. Treatment of infections associated with surgical implants. N Engl J Med 2004; 350: 1422–9. doi:10.1056/NEJMra035415
5.
VICNISS. VICNISS hospital acquired infection project, Year 4 report, August 2006. Melbourne: Victoria Department of Human Services; 2006. Available online at: http://www.vicniss.org.au/Resources/ VICNISSAnnualReport0806.pdf [verified May 2009].
6.
58
Wilson J, Ramboer I, Suetens C, HELICS-SSI working group. Hospitals in Europe link for infection control through surveillance (HELICS). Inter-country comparison of rates of surgical site infection – opportunities and limitations. J Hosp Infect 2007; 65: 165–70. doi:10.1016/S0195-6701(07)60037-1
L. Dailey et al. 7.
Carey RG. Improving healthcare with control charts: basic and advanced SPC methods and case studies. Milwaukee: Quality Press; 2003.
8.
Bolsin S. Was Bristol an outlier? Lancet 2001; 358: 2084.
9.
Colson M, Bolsin S. The use of statistical process control methods in monitoring clinical performance. Int J Qual Health Care 2003; 15: 445. doi:10.1093/intqhc/mzg070
10. Spielgelhalter DJ. Funnel plots for comparing institutional performance. Stat Med 2005; 24: 1185–202. doi:10.1002/sim.1970 11. McKibben L, Horan T, Tokars JI, Fowler G, Cardo D, Pearson ML, et al. Guidance on public reporting of healthcare-associated infections: recommendations of the Healthcare Infection Control Practices Advisory Committee. Am J Infect Control 2005; 33: 217–26. doi:10.1016/j.ajic.2005.04.001 12. HISWA. HISWA surveillance manual, Version 2, July 2008. Perth: Department of Health Western Australia; 2008. Available online at: http://www.public.health.wa.gov.au/cproot/813/2/Revised% 20Surveillance%20Manual%20July%202008.pdf [verified May 2009]. 13. Goggin L, van Gessel H, McCann R, Peterson A. Validation of surgical site infection surveillance in Perth, Western Australia. Healthcare Infect 2009; 14: in press. 14. Clopper C, Pearson S. The use of confidence or fiducial limits illustrated in the case of the binomial. Biometrika 1934; 26: 404–13. doi:10.1093/biomet/26.4.404 15. Australian Orthopaedic Association National Joint Replacement Registry. Annual Report 2007. Adelaide: AOA; 2007. 16. Mannie¨n J, van der Zeeuw AE, Willie JC, van den Hof S. Validation of surgical site infection in the Netherlands. Infect Control Hosp Epidemiol 2007; 28: 36–41. doi:10.1086/509847 17. Mertens R, Van den Berg JM, Veerman-Brenzikofer ML, Jurz X, Jans B, Klazinga N. International comparison of results of infection surveillance: The Netherlands versus Belgium. Infect Control Hosp Epidemiol 1994; 15: 574–8. 18. Coello R, Gastmeier P, de Boer AS. Surveillance of hospital-acquired infection in England, Germany, and The Netherlands: will international comparison of rates be possible? Infect Control Hosp Epidemiol 2001; 22: 393–7. doi:10.1086/501923 19. Barnes S, Salemi C, Fithian D, Akiyama L, Barron D, Eck E, et al. An enhanced benchmark for prosthetic joint replacement infection rates. Am J Infect Control 2006; 34: 669–72. doi:10.1016/j.ajic.2006.04.207 20. NNIS. Nosocomial infection rates for interhospital comparison: limitations and possible solutions. A report from the National Nosocomial Infections Surveillance (NNIS) System. Infect Control Hosp Epidemiol 1991; 12: 609–21. 21. The Positive Deviance Initiative. Boston: The Positive Deviance Initiative; 2009. Available online at: http://www.positivedeviance. org/ [verified May 2009]. 22. Agabiti N, Picciotto S, Cesaroni G, Bisanti L, Forastiere F, Onorati R, et al. The influence of socioeconomic status on utilization and outcomes of elective total hip replacement: a multicity populationbased longitudinal study. Int J Qual Health Care 2007; 19: 37–44. doi:10.1093/intqhc/mzl065 23. Cantu¨rk Z, Canturk N, Cetinarslan B, Utkan N, Tarkun I. Nosocomial infections and obesity in surgical patients. Obes Res 2003; 11: 769–75. doi:10.1038/oby.2003.107 24. Suetens C, De Laet C. Helics – SSI Statistical report 2004. 2004. Available online at: http://helics.univ-lyon1.fr/Documents/ HELICS-SSI%20Stat%20Report%202004%20Final%20Version% 20180406.pdf\ [verified May 2009]. 25. Friedman ND, Russo PL, Bull A, Richards MJ, Kelly H. Validation of coronary artery bypass graft surgical site infection surveillance data from a state-wide surveillance system in Australia. Infect Control Hosp Epidemiol 2007; 28: 812–17. doi:10.1086/518455
Surgical site infection surveillance 26. Russo PL, Gurrin L, Friedman ND, Bull A, Marasco S, Kelly H, et al. Interhospital comparisons of coronary artery bypass graft surgical site infection rates differ if donor sites are excluded. Infect Control Hosp Epidemiol 2007; 28: 1210–12. doi:10.1086/520749 27. Katz J, Losina E, Barrett J, Phillips C, Mahomed N, Lew R, et al. Association between hospital and surgeon procedure volume and outcomes in total hip replacement in the United States Medicare population. J Bone Joint Surg Am 2001; 83: 1622–9. 28. Katz J, Barrett J, Mahomed N, Baron J, Wright R, Losina E. Association between hospital and surgeon procedure volume and the outcomes of total knee replacement. J Bone Joint Surg Am 2004; 86: 1909–16. 29. Hervey SL, Purves HR, Guller U, Toth AP, Vail TP, Pietrobon R. Provider volume of total knee arthroplasties and patient outcomes in the HCUP: nationwide inpatient sample. J Bone Joint Surg Am 2003; 85: 1775–83. 30. Katz J, Losina E, Barrett J, Phillips C, Mahomed N, Lew R, et al. Volumes and outcomes in orthopaedic procedures: scientific and policy considerations. J Bone Joint Surg Am 2002. Available online at: http://www.ejbjs.org/cgi/eletters/83/11/1622 [accessed March 2009]. 31. Health Protection Agency. Third report of the mandatory surveillance of surgical site infection in orthopaedic surgery. April 2004 to March 2007. London: Health Protection Agency; 2007. Available online at: http://www.hpa.org.uk/ [verified May 2009].
Healthcare Infection 32. Ferguson JK, van Gessel H. Methicillin-resistant Staphylococcus aureus in hospitals: time for a culture change. MJA 2008; 188: 62. 33. Silla RC, Fong J, Wright J, Wood F. Infection in acute burn wounds following the Bali bombings: a comparative prospective audit. Burns 2006; 32: 139–44. doi:10.1016/j.burns.2005.09.002 34. Gale C, Roberts A, Batin P, Hall A. Funnel plots, performance variation and the Myocardial Infarction National Audit Project 2003–2004. BMC Cardiovasc Disord 2006; 6: 34. doi:10.1186/1471-22616-34 35. Morton A, Clements A, Doidge S, Stackelroth J, Curtis M, Whitby M. Surveillance of healthcare-acquired infections in Queensland, Australia: data and lessons from the first five years. Infect Control Hosp Epidemiol 2008; 29: 695–701. doi:10.1086/589904 36. Benneyan JC. Statistical quality control methods in infections control and hospital epidemiology, part 1: introduction and basic theory. Infect Control Hosp Epidemiol 1998; 19: 194–214. 37. Premier. A supporting partnership organization of the Surgical Care Improvement Project (SCIP). Available online at: http://www. premierinc.com/safety/topics/scip/index.jsp [verified May 2009]. 38. Department of Health, Western Australia. Safety and Quality Investment for Reform (SQuIRe). Available online at: http://www. safetyandquality.health.wa.gov.au/squire/index.cfm [verified May 2009].
59
Healthcare Infection
L. Dailey et al.
Appendix 1 Definitions of surgical site infection for Healthcare Infection Surveillance Western Australia surveillance system Superficial incisional surgical site infection * * *
Involves only skin and subcutaneous tissue. Occurs within 30 days of the operative procedure. Meets one of the following criteria from the incision: 1. 2. 3. 4.
purulent discharge (not stitch abscess); organisms are isolated from aseptically collected culture of fluid or tissue; diagnosis of infection or antimicrobial treatment by the operating surgeon or registrar; displays any of the following signs and symptoms at the incision site: pain or tenderness, localised swelling, redness or heat AND the incision is deliberately explored by the surgeon resulting in a positive wound culture.
Deep incisional/organ space surgical site infection * * *
Infection involves deep soft tissues (e.g. fascial and muscle layers) and/or organ spaces opened or manipulated during an operation. Occurs within 30 days after the operative procedure if implant not present or within 1 year if implant in situ. Exhibits either one or both of the following: 1. purulent drainage from deep soft tissue or drain that is placed through a stab wound into the organ/space; 2. spontaneous dehiscence at the incision site or the wound is deliberately explored by a surgeon with the patient showing evidence of one or more of the following signs or symptoms: *
* *
* *
fever >38‡C, localised pain or tenderness with culture-positive specimen. A culture-negative finding does not meet this criterion unless the patient was on antibiotics immediately prior to the wound being explored and/or the culture being taken. organisms isolated from aseptically obtained culture of fluid or tissue obtained from an organ/space. an abscess or other evidence of infection involving a deep/organ space is found on direct examination, during re-operation, or by histopathologic or radiologic examination. diagnosis of, or antimicrobial treatment of, a deep incisional or organ/space SSI by the operating surgeon or registrar.
Specific sites of an organ/space SSI include: osteomyelitis, joint or bursa.
Inclusions and exclusions for the Healthcare Infection Surveillance Western Australia surveillance system Inclusions *
All elective arthroplasty procedures: total, revision and partial.
Exclusions * * *
Revision procedures performed to remove an already infected joint. Emergency arthroplasty procedures (e.g. hemiarthroplasty of neck of femur). SSIs detected and treated as outpatients.
60