Administrative data has poor accuracy for surveillance of Staphylococcus aureus bacteraemia

Infection, Disease & Health (2016) 21, 162e168

Available online at www.sciencedirect.com

ScienceDirect journal homepage: http://www.journals.elsevier.com/infectiondisease-and-health/

Research

Administrative data has poor accuracy for surveillance of Staphylococcus aureus bacteraemia Anindita Das a,b,*, Karina Kennedy a,b,c, Gloria Spyropoulos b, Peter Collignon a,b,c a

ACT Pathology, Australia Canberra Hospital and Health Services, Canberra, Australia c Medical School, Australian National University, Canberra, Australia b

Received 2 August 2016; received in revised form 28 September 2016; accepted 2 October 2016

Available online 7 November 2016

KEYWORDS Clinical coding; Staphylococcus aureus; Bacteremia

Abstract Background: To determine the accuracy of the International Classification of Diseases (ICD-10) coding for Staphylococcus aureus bacteremia (SAB) compared with laboratory results during a ten-year period (January 2002eDecember 2011). Methods: A retrospective comparison of ICD-10 code A41.0 for S. aureus sepsis with SAB identified from the laboratory information system (LIS). Patients with LIS identified SAB (LISþ) and/ or the ICD-10 A41.0 code (ICD-10) were identified and classified as concordant (LISþ/ICDþ) or discordant (LISþ/ICD or LIS-/ICDþ). From July 2010 an additional code for healthcare associated SAB (HA-SAB), U90.0, was introduced and evaluated against prospectively designated episodes of HA-SAB. Results and Conclusions: There were 740 laboratory confirmed episodes of SAB however, only 408 of these were recorded by ICD-10 A41.0whilst 106 patients with negative blood cultures were miscoded as ICD-10 A41.0. The sensitivity and PPV for ICD-10 A41.0 were 55% [95% CI: 51e59%] and 72% [95% CI: 68e76%]. For the subset of HA-SAB, the sensitivity and PPV for ICD-10 U90.0 were only 12% [95% CI: 5e24%] and 32% [95% CI: 15e54%] respectively. Surveillance based solely on ICD-10 A41.0, code underestimates the true incidence of SAB even while including non-bacteremic episodes. ICD-10 U90.0 for HA-SAB has even poorer sensitivity and PPV. Laboratory culture results should become the major criterion for ICD-10 coding for SAB to improve the accuracy of surveillance data. ª 2016 The Authors. Published by Elsevier B.V. on behalf of Australasian College for Infection Prevention and Control. This is an open access article under the CC BY-NC-ND license (http:// creativecommons.org/licenses/by-nc-nd/4.0/).

* Corresponding author. 6 Barwick Place, Gowrie ACT 2904, Australia. Fax: þ61 2 62444646. E-mail address: [email protected] (A. Das). http://dx.doi.org/10.1016/j.idh.2016.10.001 2468-0451/ª 2016 The Authors. Published by Elsevier B.V. on behalf of Australasian College for Infection Prevention and Control. This is an open access article under the CC BY-NC-ND license (http://creativecommons.org/licenses/by-nc-nd/4.0/).

Administrative data has poor accuracy for surveillance of Staphylococcus aureus bacteraemia

163

Highlights  Established factors: SAB is common and associated with mortality rates of 20e30%.  Healthcare associated SAB (HA-SAB) is reportable and a marker of quality in healthcare in Australia.  New factors: ICD-10 coding has limited sensitivity and positive predictive value in identifying laboratory confirmed SAB and is even less accurate for HA-SAB.  Implications: Surveillance and reporting data for SAB should be based on laboratory data.

Introduction The International Classification of Diseases (ICD) has become a standard recording tool for epidemiology, health management and clinical purposes. Data, using this coding based on entries in the patients’ medical records, is used to monitor the incidence and prevalence of diseases, to provide mortality and morbidity statistics, and for reimbursement and resource allocation decision-making [1]. It is endorsed by the World Health Organization and used by all Members States. Accuracy of the coding system is therefore paramount for the production of reliable data. A major limitation of ICD coding is that a healthcare practitioner must record in the notes the medical condition before it can be coded. Laboratory results by themselves are not able to be used for coding. Previous studies have reported variable accuracy of ICD in the coding for infectious disease [2e10], although few have specifically examined bloodstream infections [4,5,8e10]. Staphylococcus aureus is one of the commonest causes of bacteraemia, both in community and healthcare settings, carrying mortality rates of 20e40% [11e13]. In addition, healthcare associated S. aureus bacteraemia (HA-SAB) is used as an indicator of quality in healthcare and reported publically in Australia (www.myhospitals. gov.au) [14], and the UK [15,16]. The diagnosis of SAB is relatively uncomplicated, as isolation of S. aureus from blood cultures is almost always clinically significant, with very few S. aureus isolates in blood being considered contaminants. Using laboratory data for SAB episodes will therefore give one of the most accurate measures of SAB incidence within an institution. The aim of this study was to determine the accuracy of the ICD-10 coding system for the identification of SAB patient episodes at Canberra Hospital (CH).

Methods A retrospective comparison of SAB cases identified between 1 January 2002 and 31 December 2011 in patients attending Canberra Hospital; a tertiary referral hospital within the Australian Capital Territory, providing services to a population of approximately 600,000 people. Medical record coders at Canberra Hospital are trained in clinical coding using ICD-10 (Australian Version) [17], and undertake regular competency evaluations and audits. Allocation of codes is based on identification by the coders of the specific diagnoses within the medical record, with multiple codes

often applied to each hospital admission. Patients attending the Emergency Department, who are not admitted, do not receive ICD-10 coding for that episode of care. Patients admitted during the study period with the ICD-10 code A41.0 “S. aureus sepsis” were identified from medical records. From 1 July 2010 a new supplementary code, U90.0 Healthcare associated S. aureus bacteraemia (HA-SAB), was introduced. For laboratory diagnosed episodes, the Canberra Hospital laboratory information system (LIS) was used to identify episodes of SAB based on the isolation of S. aureus from blood cultures performed on patients either via the Emergency Department or whilst admitted during the same time period. For patients with multiple positive blood cultures with S. aureus during the same hospitalisation, only the first episode was included. The results from the two datasets were compared. As coding is applied at the end of an episode of care, bacteraemia episodes coded during the study period but occurring prior to the commencement of the study period were excluded, whilst bacteraemia episodes coded after the study period but occurring during the study period ended were included in the analysis. Laboratory diagnosed cases of SAB which were coded in the patients’ medical records with an A41.0 code, were classified as concordant (LISþ/ICD-10þ). Discordant cases were those with a SAB laboratory diagnosis but without the A41.0 code (LISþ/ICD-10) or an A41.0 code without a laboratory diagnosed SAB episode (LIS-/ICD-10þ). The medical records of discordant cases were investigated further to identify factors responsible for missing or incorrect codes. The subset of HA-SAB episodes were identified by crossreferencing the episodes with a pre-existing prospectively collected infection control bacteraemia database. These episodes were compared to the admissions receiving the ICD-10 U90.0 code. HA-SAB (inpatient and non-inpatient) was defined according to Australian guidelines and were consistent with the U90.0 definitions [17,18]. The sensitivity of ICD-10 A41.0 code for SAB was determined by dividing the number of concordant episodes (LISþ/ICD-10þ) by the total number of SAB identified by the LIS. The positive predictive value (PPV) was calculated by dividing the concordant episodes by the total number of admissions with the ICD-10 A41.0 code. Similarly, the sensitivity and PPV of U90.0 for HA-SAB was calculated with reference to bacteraemia database. The ACT Health Directorate Human Research Ethics Committee approved the study in August 2012.

164

Results During the ten-year study period there were 567,338 hospital admissions. The laboratory information system identified 740 episodes of SAB. These included 439 healthcare associated (286 inpatient and 153 non-inpatient onset) and 301 community associated episodes. ICD-10 A41.0 coded 565 episodes, of which 408 were concordant (LISþ/ICD10þ) with the laboratory SAB data (Fig. 1). ICD-10 coding identified 51 patients transferred to Canberra Hospital from regional hospitals with a diagnosis of SAB, but in whom positive SAB blood cultures were not obtained from the Canberra Hospital laboratory (LIS-/ICD10þ/SABþ). A further 106 patient admissions had no laboratory or medical record evidence of SAB (LIS-/ICD-10þ/ SAB). Incorrect discharge diagnoses, misinterpretation of coagulase negative Staphylococci (CoNS) in blood cultures as SAB and culture isolation of S. aureus from sites other than blood were found to be responsible for these discordant results. Three hundred and thirty-two episodes of SAB identified on the LIS did not have the ICD-10 (A41.0) code (LISþ/ICD10). Eleven (3%) of these episodes were from patients that presented to the Emergency Department but either died (2) or were transferred to another institution (9) rather than being admitted to Canberra Hospital, and therefore were not given any ICD-10 codes. Of the remaining 321 episodes, failure to document SAB in the medical records and use of multiple other sepsis codes contributed to the discrepancy. In 227 (71%) of these discrepant cases there was at least one Staphylococcus organism code (but not A41.0). In 37 (12%) a generalised sepsis code was used and in 46 (14%) there was no infection code (Table 1). From the time of introduction of the new supplementary code, U90.0 on 1 July 2010 there were 52 episodes of HA-SAB (39 inpatient and 13 non-inpatient onset). The U90.0 coding identified 19 patient admissions during the same period, of which only 6 were concordant. Forty-six episodes of HA-SAB did not receive the U90.0 code, whilst 13 patient admissions coded with U90.0 were not associated with SAB. The sensitivity of ICD-10 (A41.0) for identifying SAB was 55% (408/740) [95% CI: 51e59%]. Excluding the Emergency Department presentations from the denominator made little impact to the sensitivity (56%; 408/729). The PPV of ICD10 (A41.0) was 72% (408/565) [95% CI: 68e76%]. The annual number of SAB episodes, and concordant (LISþ, ICD-10þ) and discordant (LIS-, ICD-10þ) A41.0 coded episodes is shown in Fig. 2. Including the SAB episodes identified at other institutions (LIS-/ICD10þ/SABþ), increased the PPV to 81% (459/565) [95% CI: 78e84%]. The sensitivity and PPV of the U90.0 code for HA-SAB was even poorer at 12% (6/52) [95% CI: 5e24%] and 32% (6/19) [95% CI: 15e54%] respectively. The annual number of SAB episodes and sensitivity and PPV of ICD-10 coding is shown in Fig. 2.

Discussion The International Classification of Diseases (ICD) is an established, standardized system used globally for the collection of health data and statistics [1,2]. The availability of reliable data is dependent on the accuracy of the

A. Das et al. coding system. A recent systematic review of the role of administrative data for surveillance of health care associated infections has reported limited and widely variable accuracy [19]. Our study demonstrates that the ICD-10 system lacks sensitivity and has only moderate positive predictive value when compared to laboratory diagnosis of SAB, and is even less accurate in identifying healthcare associated SAB at our institution. The sensitivity (56%) was particularly poor, reflecting the large number of SAB episodes missing A41.0 coding. Missed coding may be due to inaccurate and inconsistent documentation in the patient records by medical practitioners and misinterpretation of diagnoses by coding officers. Most (70%) of the SAB episodes missed by coding were given an ICD-10 code related to S. aureus localised to a specific system (B95.6), most likely due to prioritisation of focal S. aureus infection coding over the more generic and less defined A41.0 sepsis code (Table 1). S. aureus sepsis is not completely interchangeable with SAB, as sepsis may result from infection at any site (eg an infected hip joint), and can cause ambiguity in the coding process [20,21]. Emergency Department presentations without admission also contributed to the low sensitivity, as ICD-10 coding is only applicable to patient admissions, although this effect was small (3%). The positive predictive value provides an indication of the accuracy of coding for those episodes receiving the A41.0 code. The PPV for identifying SAB related to our institution was only 72%. The identification of SAB episodes diagnosed in other health facilities prior to transfer to Canberra Hospital contributed to 10% of the discordant coding, and is reflective of the tertiary level care Canberra Hospital provides to an otherwise large and isolated region. Although the coding was correct (i.e it captured discharge diagnoses of SAB), the incident bacteraemia occurred at an alternate health facility, and therefore for surveillance purposes would be better if captured in the incident health facility. Similarly, it is possible that SAB episodes identified through the LIS at our institution may have also been recorded at an alternate health facility prior to transfer, and be documented twice through both ICD-10 coding and laboratory data. For national surveillance purposes, a means of excluding such duplication is required. The remaining discrepancies in coding were due to clinical presentations with sepsis, but with S. aureus isolated from sites other than blood, such as skin, soft tissue, sputum, and joint fluids, being coded as A41.0. The ambiguity in the A41.0 code e S. aureus sepsis e may have also led to discordant coding compared to laboratory results. Isolation of coagulase negative Staphylococcus from blood cultures was found to be another source of confusion resulting in either incorrect documentation of SAB or miscoding of A41.0. In addition, medical record documentation is predominantly performed by junior medical staff and variations in training and individual interpretation of disease could contribute to discrepancies. From 1 July 2010, the supplementary code U90.0 was available for HA-SAB. This code could only be used if there was clinician documentation in the medical records of HASAB, including terms “hospital associated”, “hospital acquired” and/or “nosocomial”. This terminology however is not frequently used outside of infection control speciality

Administrative data has poor accuracy for surveillance of Staphylococcus aureus bacteraemia

Canberra Hospital Laboratory InformaƟon System IdenƟfied SAB (LIS+) 740

Canberra Hospital Medical Records ICD-10 (A41.0) Code 565

Discordant LIS+/ICD-10332

Emergency Department PresentaƟon Only

165

Concordant LIS+/ICD-10+ 408

Discordant LIS-/ICD-10 + 157

Incorrectly coded

Incorrectly coded

321

106

11

SAB idenƟfied at another insƟtuƟon prior to transfer to Canberra Hospital 51

Figure 1 Concordance and discordance between Staphylococcus aureus bacteraemia (SAB) cases identified by the laboratory information system and ICD-10 code A41.0 at Canberra Hospital from January 2002 to December 2011.

units. In addition, the definition of HA-SAB included episodes of bacteremia with onset within the community but associated with the healthcare system due to an indwelling device, surgical site infection or cytotoxic induced chemotherapy. Not unexpectedly, the sensitivity and PPV were only 12% and 32% respectively. Despite these poor measures of accuracy, administrative data based information is being used to generate comparative hospital HA-SAB rates for Australian hospitals (https://www. healthroundtable.org/). Several other studies have also investigated the inaccuracy of ICD coding for the surveillance of infectious diseases, including bacteraemia [4,5,8e10,19]. Interestingly, studies originating from the United States refer to ICD-9 codes, with the uptake of ICD-10 across American hospitals lagging behind Australia and Europe [2,21]. Inaccuracies have been found in 36e78% of ICD codes used for Clostridium difficile infection in France, Singapore and United States, and have been attributed to errors in interpretation of discharge diagnoses, inaccurate documentation in medical records and differences in awareness of health care professionals’ worldwide [6,22,23]. A wide variation in sensitivity of ICD coding has also been reported in pneumonia diagnoses (17e66%) [4,5], whereas high correlation has been found for central nervous system infection (95%) [4]. Studies specifically investigating bacteraemia have demonstrated sensitivities ranging from 5.9 to 46% [4,5,19] for septicaemia and catheter related bacteraemia, 86% for Gram negative bacteraemia [8] and low positive predictive

values of 15e20% [3,10] for ICD coding. Whilst the sensitivity (56%) and PPV (72%) in our study were favourable compared to those previously reported, we remain concerned regarding the overall accuracy of coding for HA-SAB at our institution. This being a single centre retrospective study is a limitation, but availability of standardised national coding guidelines [17] likely reflects similar reporting practices resulting in low accuracy of surveillance data. As a means of addressing the problems identified in medical record coding, and to optimise the management of SAB, since 2014 we have introduced a “SAB” sticker (Fig. 3) which is placed in the medical records by the clinical microbiologist/registrar. When applicable, the episode is also designated as “healthcare associated SAB” according to national definitions [18]. We believe laboratory information systems are more accurate for SAB surveillance; however in the absence of a means of cross-referencing data between laboratories and correlating bacteraemia data with clinical information, some limitations exist. Duplication of SAB episodes across healthcare centres when patients are tested at different laboratories, and under-appreciation of non-inpatient healthcare associated SAB are the main restrictions. In our dataset, 32% (153/439) of healthcare associated SAB would have been misidentified as community associated infection if only admission date criteria were used. Incorporation of laboratory databases into administrative coding data should augment detection and allow for improved

166

A. Das et al.

Table 1 Alternate ICD-10 codes used in 332 episodes of laboratory identified Staphylococcus aureus bacteraemia (SAB) where ICD-10 A41.0 not used, CH 2002e2011. Alternate ICD-10 codes used

SAB episodes

Staphylococcus species codesa B95.6: S. aureus as disease classified to another chapter (disease localised to specific system; e.g. respiratory, skin) A41.2: Unspecified Staphylococcus sepsis (neither S. aureus or CoNS documented in patient file) A41.1: Coagulase negative Staphylococcus (CoNS) sepsis A49.0: Unspecified site Staphylococcus (no site specified) A49.01: Unspecified site S. aureus (use from July 2010) Other sepsis codes P36.3: Neonatal sepsis Other sepsis codes No related Staphylococcus or sepsis codes Not codedb

227 225

16

2 6

43 37 17 20 57 11

a Some patients had multiple overlapping Staphylococcus species codes. b Patients presenting to the Emergency Department and either transferred to another health facility or deceased.

accuracy without additional expenditure. Sophisticated computer algorithms of automated surveillance systems combining microbiological and administrative data have improved accuracy and reliability in detection of health

care associated blood stream infections [24,25]. A limitation of the LIS in the context of hospital funding, is the potential of missing SAB cases from other institutions if blood cultures are not positive after transfer. This however does not impact on the use of the LIS for institutional surveillance purposes. The sensitivity and PPV of the HA-SAB code U90.0 was determined to be very low but this was introduced only 18 months before the end of the study period, therefore the overall numbers of this subset are small. The inaccuracy of estimates of disease burden due to missed cases and miscoded records can have significant implications on allocation of funds, policy decisions, and underestimation of burden of disease on health care infrastructure and flawed tracking and comparison of disease trends both nationally and internationally. We found that sole use of the ICD-10 coding system for surveillance of SAB is inaccurate due to the poor sensitivity (55%) resulting from missed coding, combined with moderate positive predictive value (72%) due to incorrect coding of non-SAB episodes, with even poorer accuracy in coding of the healthcare associated subset. Introducing a specific ICD-10 code for SAB, to decrease the confusion associated with the more general term “S. aureus sepsis”, and inclusion of laboratory results for classification of S. aureus bacteremia is vital to improve the accuracy of ICD-10 coding for the purpose of providing surveillance data.

Ethics statement This is a retrospective study involving analysis of medical records and microbiology laboratory results with no use of patient identifiers and no additional requirement of patient involvement or sampling. The ACT Health Directorate Human Research Ethics Committee approved the study in August 2012.

Figure 2 Annual episodes of laboratory identified Staphylococcus aureus bacteraemia (SAB) and sensitivity and positive predictive values (PPV) of ICD-10 code A41.0 at Canberra Hospital.

Administrative data has poor accuracy for surveillance of Staphylococcus aureus bacteraemia

Figure 3

167

Staphylococcus aureus bacteraemia notification and management bundle sticker used in patient medical records.

Authorship statement All the authors have contributed to the design of the project and manuscript preparation. This manuscript has not been previously published and is not under consideration in any other peer reviewed media. We have presented the provisional findings at the Australasian Society of Infectious Diseases annual meeting in Canberra 2013.

Conflicts of interest The authors of this study have no conflicts of interest to report and no funding resources to acknowledge.

Funding None to declare.

Provenance and peer review Not commissioned; externally peer reviewed.

Acknowledgements The authors would like to acknowledge Dr Heather Wilson, Registrar Microbiology for her role in the design and

scripting of the Staphylococcus aureus bacteraemia notification and management bundle sticker used in medical records.

References [1] International Classification of Diseases (ICD), World Health Organisation, http://www.who.int/classifications/icd/en/. ICD10 Version:2016, Chapter I, http://apps.who.int/classifications/ icd10/browse/2016/en#/I [Accessed June 2016]. [2] Goto M, Ohl ME, Schweizer ML, Perencevich EN. Accuracy of administrative code data for the surveillance of health-care associated infections: a systematic review and meta-analysis. Clin Infect Dis 2014;58:688e96. [3] Jhung MA, Banerjee SN. Administrative coding data and health careeassociated infections. Clin Infect Dis 2009;49:949e55. [4] Gedeborg R, Furebring M, Michaelsson K. Diagnosis-dependent misclassification of infections using administrative data variably affected incidence and mortality estimates in ICU patients. J Clin Epidemiol 2007;60:155e62. [5] Madsen KM, Schonheyder HC, Kristensen B, Nielsen GL, Sorensen HT. Can hospital discharge diagnosis be used for surveillance of bacteremia? A data quality study of a Danish hospital discharge registry. Infect Control Hosp Epidemiol 1998;19:175e80. [6] Jones G, Taright N, Boelle PY, Marty J, Lanande V, Eckert C, et al. Accuracy of ICD-10 codes for surveillance of Clostridium difficile infections, France. Emerg Infect Dis 2012;18:979e81. [7] Kuehnert MJ, Hill HA, Kupronis BA, Tokars JI, Solomon SL, Jernigan DB. Methicillin-resistant Staphylococcus aureus hospitalizations, United States. Emerg Infect Dis 2005;11:868e72.

168 [8] Sogaard KK, Thomsen RW, Schonheyder HC, Sogaard M. Positive predictive values of the International classification of diseases, 10th revision diagnoses of gram-negative septicaemia/sepsis and urosepsis for the presence of gram negative bacteremia. Clin Epidemiol 2015;7:195e9. [9] Rhee C, Murphy MV, Li L, Platt R, Klompas M. For the centers for disease control and prevention epicenters program. Comparison of trends in sepsis incidence and coding using administrative claims versus objective clinical data. Clin Infect Dis 2015;60:88e95. [10] Stevenson KB, Khan Y, Dickman J, Gillenwater T, Kulich P, Myers C, et al. Administrative coding data, compared with CDC/NHSN criteria, are poor indicators of health careassociated infections. Am J Infect Control 2008;36:155e64. [11] Laupland KB, Lyytikainen O, Sogaard M, Kennedy KJ, Knudsen JD, Ostergaard C, et al. The changing epidemiology of Staphylococcus aureus bloodstream infection: a multinational population-based surveillance study. Clin Microbiol Infect 2013;19(5):465e71. [12] Ong CW, Roberts JL, Collignon PJ. Long term survival outcome following Staphylococccus aureus bacteraemia. Healthc Infect 2013;18:102e9. http://dx.doi.org/10.1071/HI12062. [13] van Hal SJ, Jensen SO, Vaska VL, Espedido BA, Paterson DL, Gosbell IB. Predictors of mortality in Staphylococcus aureus bacteremia. Clin Microbiol Rev 2012;25:362e86. [14] Collignon PJ, Wilkinson IJ, Gilbert GL, Grayson ML, Whitby RM. Health care-associated Staphylococcus aureus blood stream infections: a clinical quality indicator for all hospitals. Med J Aust 2006;184:404e6. [15] Johnson AP, Davies J, Guy R, Abernethy J, Sheridan E, Pearson A, et al. Mandatory surveillance of methicillin resistant Staphylococcus aureus (MRSA) bacteraemia in England: the first 10 years. J Antimicrob Chemother 2012;67:802e9. [16] Pearson A, Chronias A, Murray M. Voluntary and mandatory surveillance for methicillin resistant Staphylococcus aureus (MRSA) and methicillin susceptible Staphylococcus aureus (MSSA) bacteraemia in England. J Antimicrob Chemother 2009;64:i11e7. [17] Australian Consortium for Classification Development, ICD-10AM Ninth Edition. https://www.accd.net.au/Icd10.aspx. The

A. Das et al.

[18]

[19]

[20]

[21]

[22]

[23]

[24]

[25]

National Centre for Classification in Health, University of Sydney in collaboration with Western Sydney University and KPMG [Accessed June 2016]. Surveillance validation guide for Staphylococcus aureus bacteraemia, October 2014. National Surveillance Initiative, http://www.safetyandquality.gov.au/our-work/healthcareassociated-infection/national-hai-surveillance-initiative/. Australian Commission on Safety and Quality in Health Care [Accessed June 2016]. Van Mourik Maaike SM, Van Dujin PJ, Moons KGM, Bonten MJM, Lee GM. Accuracy of administrative data for surveillance of healthcare-associated infections: a systematic review. BMJ Open 2015;5:e008424. http://dx.doi.org/10.1136/bmjopen2015-008424. Singer M, Deutschman CS, Seymour CW, Shankar-Hari M, Annane D, Bauer M, et al. The third international consensus definitions for sepsis and septic shock (Sepsis-3). JAMA 2016; 315:801e10. Gohil SK, Cao C, Phelan M, Tjoa T, Rhee C, Platt R, et al. Impact of policies on the rise in sepsis incidence, 2000-2010. Clin Infect Dis 2016;62:695e703. Dubberke ER, Butler AM, Yokoe DS, Mayer J, Hota B, Mangino JE, et al. Multicenter study of surveillance for hospital-onset Clostridium difficile infection by the use of ICD-9-CM diagnosis codes. Infect Control Hosp Epidemiol 2010; 31:262e8. Chan M, Lim PL, Chow A, Win MK, Barkham TM. Surveillance for Clostridium difficile Infection: ICD-9 coding has poor sensitivity compared to laboratory diagnosis in hospital patients, Singapore. PloS ONE 2011;6:e15603. http://dx.doi.org/10.1371/journal. pone.0015603. Leal J, Gregson DB, Church DL, Henderson EA, Ross T, Laupland KB. Development of a novel electronic surveillance system for monitoring of bloodstream infections. Infect Control Hosp Epidemiol 2010;31:740e7. Lin MY, Woeltje KF, Khan YM, Hota B, Doherty JA, Borlawsky TB, et al. Multicenter evaluation of computer automated versus traditional surveillance of hospital-acquired bloodstream infections. Infect Control Hosp Epidemiol 2014; 35:1483e90.