- Email: [email protected]
New control chart methods for monitoring MROs in Hospitals
Australian Infection Control
New control chart methods for monitoring MROs in Hospitals Anthony Morton MSc(Appl), MD, Infection Management Services, Princess Alexandra Hospital, Brisbane and School of Mathematical Sciences, Queensland University of Technology
Michelle Gatton BSc(Hons), PhD, Queensland Institute of Medical Research Edward Tong BA, BSc (Hons), Centre for Health Related Infection Surveillance and Prevention, Queensland Health Archie Clements MVM, PhD, Centre for Health Related Infection Surveillance and Prevention, Queensland Health and School of Population Health, University of Queensland
Abstract Routine surveillance of colonisations with multiple antibiotic resistant organisms (MROs) is now widespread and these data are increasingly summarised in control charts. The purpose of their analysis in this manner is to provide early warning of outbreaks or to judge the response to system changes designed to reduce colonisation rates. Conventional statistical process control (SPC) charts assume independence of observations. In addition, there needs to be a run of stable, non-trended (stationary) data values to obtain accurate control limits. Colonisation with an MRO is not an independent event as it must involve transmission from a carrier and this can lead to excessive variation. In addition, non-linear trends are often present and MRO prevalence data display temporal correlation. The latter occurs when data at particular times are more like data at related, usually contiguous times, than data from more distant times; thus they are not temporally independent. These characteristics make it difficult to implement conventional SPC charts with MRO data. To overcome these problems, we suggest the use of generalised additive models (GAMs) when there is no temporal correlation, as with new colonisations, and generalised additive mixed models (GAMMs) when temporal correlation is present; as occurs commonly with prevalence data. We illustrate their use with multi-resistant methicillin-resistant Staphylococcus aureus (mMRSA) prevalence and new colonisation data. These methods are able to deal with excess variability, trends and temporal correlation. They are easily implemented in the freely available R software package. Our analysis demonstrates an upward non-linear trend in mMRSA prevalence between January 2004 and October 2006. The mMRSA new colonisation data also display an upward trend between September 2005 and May 2006. Monthly new colonisation rates exceeded the upper control limit in April 2005 and equalled it in May 2006. There was a modest downward trend in the new colonisation rate in the latter part of 2006.
Introduction MROs are an increasing problem in many hospitals causing patient injury and death, increased lengths of stay and economic loss 1. Monthly new colonisations with MROs are routinely monitored using a variety of techniques including control charts 2; this assists with the timely detection of outbreaks and in judging responses to system changes designed to reduce colonisation rates. In addition, MRO prevalence, often referred to as MRO ‘burden’ or ‘colonisation pressure’, is being monitored increasingly because of its importance in transmission 3.
data require an underlying system that is behaving predictably and is therefore in statistical control. This is usually achieved by the successful implementation of evidence-based systems that are increasingly implemented as ‘bundles’4. These have been successful in reducing rates of surgical site infections and device-related bacteraemias 5. However, to date they have been less successful with MRO colonisations, making runs of stable non-trended (stationary) data difficult to achieve. In addition, MRO colonisation is not an independent process as it is due to transmission from another carrier. Excessive variability is a common consequence of this lack of independence.
Conventional monthly control charts require independence of observations and a run of stable, non-trended (stationary) monthly values, preferrably as many as twenty-four, so that accurate centre lines and control limits can be calculated. Stable, non-trended 14
Temporal correlation (autocorrelation) is a feature of MRO prevalence data. For example, today’s MRO prevalence count tends to be more like yesterday’s count than those in previous weeks. Vol 12
Issue 1
March 2007
Australian Infection Control
All of these factors make the use of conventional control charts difficult and consequently their use may give misleading information. Here we show that GAMs 6 for data that do not demonstrate temporal correlation and GAMMs for data that are temporally correlated provide alternative analysis tools to control charts for these types of data.
Methods Daily prevalence of mMRSA, non-multi-resistant methicillinresistant Staph. aureus (nmMRSA), UK epidemic methicillinresistant Staph. aureus (ukEMRSA), multi-resistant Klebsiella pneumoniae, vanconycin-resistant Enterococcus (VRE) and multiresistant Acinetobacter has been recorded at the Princess Alexandra Hospital (PAH), Brisbane, since November 2003. The data from November 2003 to mid-January 2007 are complete for 89% of those 1169 days. Interpolated daily prevalence values, estimated from the data immediately before and after the gaps, have been substituted for the missing data. Monthly counts of new colonisations with these organisms are the subject of the routine surveillance program at PAH. Swabbing of all ICU patients occurs twice-weekly and once-weekly for patients in other high-risk areas as recommended by the Australian Infection Control Association National Advisory Board. The data for January 2004 to December 2006 inclusive are complete and have been employed in the study. We illustrate the analysis of the mMRSA data using GAMs and GAMMs 6. These models are described in the Appendix. The average mMRSA prevalence for each month was determined and rounded to the nearest integer value. Analysis of these data between January 2004 and December 2006 was then performed. To account for the presence of temporal correlation, we have employed the GAMM in the mgcv package 6 in R 7. In addition an approximate upper two standard deviation equivalent control limit was calculated. There is further description in the Appendix. An occupied bed days (OBDs) denominator was included in the analysis. The monthly number of new mMRSA isolates from January 2004 to December 2006 was analysed using a GAM as described in the Appendix. The required R commands for these analyses are available from the corresponding author. Because some infection control practitioners may have difficulty with a statistics package, as opposed to familiar spreadsheets that are incapable of implementing GAM methods, these commands will be automated in easy-to-use functions.
Results Monthly mMRSA prevalence for the study period is displayed in Figure 1. There was significant temporal correlation (the correlogram is not shown). The GAMM model produced a highly significant result 16
(P<.0001) and indicated an upward non-linear trend in prevalence between January 2004 and October 2006. The fitted estimates using the GAMM were 19.7 cases per month (95% CI 17 to 22.4) in January 2004 reaching a peak of 36.6 cases per month (95% CI 32.9 to 40.6) in October 2006. The upper dotted line in Figure 1 shows the approximate two sigma equivalent upper control limit that quantifies the relationship of the individual monthly values to the GAMM estimates. Over the study period there were no values that exceeded the control limit. The corresponding Shewhart/EWMA SPC chart 8 gives similar information (Figure 2 produced in Microsoft Excel®). Unfortunately, because of the presence of temporal correlation and a trend, users will have no idea whether the centre line and control limits in the SPC chart correctly summarise the time series. The monthly new mMRSA colonisations did not show any significant temporal correlation; therefore, the simpler GAM was employed (Figure 3). These data show an upward non-linear trend commencing in September 2005 (GAM estimate 8.2, 95% CI 5.3 to 12.7) and reaching a peak in May 2006 (GAM estimate 19.4, 95% CI 16 to 23.3). This is highly statistically significant (P<.0005). There was a modest fall in the rate of new colonisation in the latter part of 2006. The upper dotted line in the figure shows the approximate two sigma equivalent upper control limit that quantifies the relationship of the individual monthly values to the GAM estimates. There was a count higher than this in March 2005 and one equal to it in May 2006. Figure 4 is the corresponding Shewhart/EWMA SPC chart 8. Once again, the charts give similar information but, because of the high variability and trend seen with these data, there is no way of knowing how accurately the SPC chart summarises the time-series.
Discussion Conventional control charts require observations to be independent 9. MRO colonisation and infection frequently violate this requirement as colonisation results from transmission from a carrier. This often results in excessive variability. Special methods such as employing the negative binomial distribution have been suggested to overcome this problem but these methods increase the complexity of the analysis 8. Corcoran and Speekenbrink 9 also point out that it is often difficult to get a sufficiently long run of observations that are stable and predictable (in statistical control) as well as stationary (that do not display a trend) to calculate reliable in-control centre lines and control limits. Our analysis of the prevalence and new colonisation data for mMRSA indicate that these data display nonlinear trends that make it difficult to find stable mean and accurate control limits. Finally, the MRO prevalence data display temporal correlation. Although SPC methods exist for the analysis of these data 10 their use introduces additional complexity and the interpretation of the resulting charts can be difficult For these reasons, we have developed a new approach that uses the non-linear GAM and GAMM to estimate the expected number Vol 12
Issue 1
March 2007
Australian Infection Control
of colonisations and superimposes, above the expected number, an approximate upper two sigma equivalent control limit. Although the number of monthly data values that are required is unlikely to be much reduced compared to the SPC analysis, there is not the need for a long run of data values that are in statistical control to obtain reliable control limits for the latter. In addition, weekly data that are usually too variable to use in SPC charts can be analysed with a GAM or GAMM provided there are not very small weekly counts. This may bring analysis closer to real-time thus potentially helping with earlier detection of new outbreaks. These methods successfully deal with temporal correlation, excessive variability and the presence of non-linear trends. Although GAM and GAMM charts may be slightly more difficult to interpret than conventional SPC charts, the two major points can be seen: 1. The predicted GAM estimate and its confidence interval provide reliable information about the average value at any time together with the observed trends that indicated the need for investigation and possible system change. 2. The control limit provides a means for detecting a sudden increase in the prevalence or the number of new isolates of the MRO of interest, analogous to the control limits of a conventional control chart. This occurred with the mMRSA new colonisation data and would have suggested the need to search for a possible additional cause at those times. It is well known that taking averages at intervals, for example the monthly averages of the daily prevalence values, can markedly reduce temporal correlation 10 although, with our monthly prevalence data, some temporal correlation persisted. If there is no temporal correlation, the simpler GAM procedure that is also in the mgcv library 6 in R 7 can be employed. The prevalence and new colonisation charts show marked upward trends at roughly the same time. It is expected that increasing numbers of new colonisations will also increase the prevalence numbers so a causal relationship cannot be proved with these data. However, it has been demonstrated that such a relationship can exist 3. In addition, before the monitoring of MRO prevalence commenced late in 2003, we saw an outbreak due to VRE that monopolised isolation facilities to the extent that many mMRSA carriers could not be placed in isolation beds. There was a significant rise in the rate of new mMRSA colonisations at that time 11. A serious consequence of the inappropriate use of conventional SPC charts is the occurrence of frequent false-positive signals. In a judgmental environment, this can provoke tampering with systems that are not malfunctioning or out of control. Such tampering creates uncertainty and promotes error. The approach reported here overcomes this problem by acknowledging the unique and difficult characteristics of these data that can be excessively variable, trended and temporally correlated. Vol 12
Issue 1
March 2007
As with all monitoring of infections, not all carriers or new colonisations are detected. It is not economic to screen patients in low-risk areas like psychiatry. In addition, newly colonised patients may be discharged between screenings from a high-risk area like ICU where swabbing is carried out twice weekly so occasional new colonisations will go undetected. Finally, there are small delays with the identification of the MRO in the microbiology laboratory and the procedures used are unlikely to have perfect sensitivity and specificity. However, the surveillance system at PAH remained relatively constant during the observation period so changes in MRO prevalence and new colonisations should have been detected reliably. An important issue is when the MRO burden reaches the level that overwhelms efforts to curtail transmission, for example by isolating or cohorting carriers. Since there are multiple factors involved in transmission and they must in general be the subject of observational studies, the use of statistical modeling techniques such as Bayesian networks will be necessary to utilise and understand these data fully.
References 1. 2.
3.
4.
5.
6. 7. 8.
9. 10. 11.
12.
Spelman D. Hospital-acquired Infections. MJA 2002;176:286-291. Curran E, Benneyan J & Hood J. Controlling methicillin-resistant Staphylococcus aureus: a feedback approach using annotated statistical process control charts. Infect Cont & Hosp Epidem 2002;23:13-18. Merrer J, Santoli F, Vecchi C, Tran B, De Jonghe B &d Outin H. Colonisation pressure and risk of acquisition of methicillin-resistant Staphylococcus aureus in a medical intensive care unit. Infect Cont & Hosp Epidem 2000;21:718-723. Institute for Healthcare Improvement. Raising the bar with bundles. Viewed 21 February 2007. Berwick D, Calkins D, McCannon C & Hackbarth A, The 100,000 Lives Campaign: setting a goal and a deadline for improving health care quality. JAMA 2006;295:324-327. Wood S. Generalised Additive Models. Chapman and Hall/CRC Boca Raton, 2006. Ihaka R & Gentleman R. R: a language for data analysis and graphics. J Comp & Graph Stats 1996;5:299-314. Morton A, Whitby M, McLaws M-L, Dobson A, Mcelwain S, Looke D, et al. The application of statistical process control charts to the detection and monitoring of hospital-acquired infections. J Qual Clin Prac 2001;21:112-117. Corcoran & Speekenbrink. Statistical process control charts and MRSA. J Hosp Infect 2007;65:175. Montgomery D. Introduction to statistical quality control. 5th edition New York:John Wiley & Sons, 2005. Bartley P, Schooneveldt J, Looke D, Morton A, Johnson D & Nimmo G. The relationship of a clonal outbreak of Enterococcus faecium vanA to methicillin-resistant Staphylococcus aureus incidence in an Australian hospital. J Hosp Infect 2001;48:43-54. The R core development team. Viewed 21 February 2007
Appendix R 7,12 (The R Core Development Team) is a very comprehensive open source statistics package that implements the S statistical computing language. It can be freely downloaded from
Australian Infection Control
2.5
2
RATE PER 1000 BED-DAYS
1.5
1
0.5
Observation of the plot of autocorrelations for the mMRSA prevalence data (not shown) revealed an exponentially decaying correlation function where consecutive monthly mMRSA counts were more correlated than values that were more separated in time. This suggested that a first-order autoregression (AR1) correlation object would be suitable for use with these data. The GAMM is:
Oct-06
Dec-06
Nov-06
Jul-06
Sep-06
Jun-06
Aug-06
Apr-06
Mar-06
May-06
Jan-06
Feb-06
Dec-05
Oct-05
Nov-05
Jul-05
Sep-05
Aug-05
Apr-05
Jun-05
Mar-05
May-05
Jan-05
Feb-05
Dec-04
Oct-04
Nov-04
Sep-04
Jul-04
Aug-04
Apr-04
Jun-04
MONTHS Rate
Mean rate
+2 sigma S
+3 sigma S
-3 sigma S
EWMA
+ 2.5 sigma E
- 2.5 sigma E
-2 sigma S
Figure 3. Generalised additive model for monthly MRSA new colonisations Jan 2004 to Dec 2006 with 95% confidence limits & Upper 2 Sigma-equivalent control limit. 30
25 Monthly New MRSA Colonisations
The GAM and GAMM predict the smoothed mean values for each time period together with their 95% confidence limits. Using these smoothed mean values as expected values, the proposed approximate upper two sigma equivalent control limit is at the 97.5th percentile of the Poisson distribution (Figures 1 and 3).
May-04
The GAMM analysis of the temporally correlated monthly prevalence data employs the Poisson family, a correlation object (corAR1) and log (OBDs) as an offset. The GAM analysis of the new colonisation data that are uncorrelated also employs the Poisson family with log (OBDs) as an offset but no correlation object.
Mar-04
0 Feb-04
GAMs are regression models that can be used to study non-linear relationships between variables. Non-parametric smoothers, usually based on splines, are used to explain those relationships. GAMMs have the additional capability of dealing with extraneous variability, in this case due to the temporal correlation (autocorrelation) in the prevalence data. These models are suitable for the analysis of time-series data in the form of counts that may be small and that may display excess variability, non-linear trends and temporal correlation.
Figure 2. Shewhart/EWMA Chart monthly average mMRSA prevalence.
Jan-04
GAMs. However, it has the added ability to extend to a GAMM. It can be downloaded from
logE(mMRSA)=gamm(mMRSA~s(time),family=poisson,offset=log( OBD), correlation=corAR1()), where E(mMRSA)t is the fitted value at time t. This model has only the intercept in its parametric part. The non-parametric part is the smooth-spline s(time). The offset term brings log(OBD) into the model with a fixed coefficient of one
20
15
10
5
0 Jan20
Apr20
Jul20
Oct20
Jan20
Apr20
Jul20
Oct20
Jan20
Apr20
Jul20
Oct20
Data blue, GAM estimate solid red, Confidence limits dashed red, Control limit dotted red
Figure 4. Shewhart/EWMA Chart mMRSA new colonisations. Figure 1. Generalised additive mixed model for monthly MRSA prevalence Jan 2004 to Dec 2006 with 95% confidence limits & Upper 2 Sigma-equivalent control limit.
1.6
1.4
RATE PER 1000 BED-DAYS
1.2
50
40 35
1
0.8
0.6
0.4
30 0.2
25
15 Jan04
Dec-06
Oct-06
Nov-06
Sep-06
Jul-06
Jun-06
Aug-06
Apr-06
May-06
Mar-06
Jan-06
Feb-06
Oct-05
Dec-05
Nov-05
Jul-05
Sep-05
Aug-05
Jun-05
Apr-05
Mar-05
MONTHS
Apr04
Jul04
Oct04
Jan05
Apr05
Jul05
Oct05
Jan06
Apr06
Data blue, GAM estimate solid red, Confidence limits dashed red, Control limit dotted red
18
May-05
Jan-05
Feb-05
Dec-04
Oct-04
Nov-04
Sep-04
Jul-04
Aug-04
Jun-04
May-04
Apr-04
Mar-04
Feb-04
0
20
Jan-04
Monthly New MRSA Colonisations
45
Jul06
Oct06
Rate
Mean rate
+2 sigma S
+3 sigma S
-3 sigma S
EWMA
+ 2.5 sigma E
- 2.5 sigma E
Vol 12
-2 sigma S
Issue 1
March 2007
New control chart methods for monitoring MROs in Hospitals Anthony Morton MSc(Appl), MD, Infection Management Services, Princess Alexandra Hospital, Brisbane and School of Mathematical Sciences, Queensland University of Technology
Michelle Gatton BSc(Hons), PhD, Queensland Institute of Medical Research Edward Tong BA, BSc (Hons), Centre for Health Related Infection Surveillance and Prevention, Queensland Health Archie Clements MVM, PhD, Centre for Health Related Infection Surveillance and Prevention, Queensland Health and School of Population Health, University of Queensland
Abstract Routine surveillance of colonisations with multiple antibiotic resistant organisms (MROs) is now widespread and these data are increasingly summarised in control charts. The purpose of their analysis in this manner is to provide early warning of outbreaks or to judge the response to system changes designed to reduce colonisation rates. Conventional statistical process control (SPC) charts assume independence of observations. In addition, there needs to be a run of stable, non-trended (stationary) data values to obtain accurate control limits. Colonisation with an MRO is not an independent event as it must involve transmission from a carrier and this can lead to excessive variation. In addition, non-linear trends are often present and MRO prevalence data display temporal correlation. The latter occurs when data at particular times are more like data at related, usually contiguous times, than data from more distant times; thus they are not temporally independent. These characteristics make it difficult to implement conventional SPC charts with MRO data. To overcome these problems, we suggest the use of generalised additive models (GAMs) when there is no temporal correlation, as with new colonisations, and generalised additive mixed models (GAMMs) when temporal correlation is present; as occurs commonly with prevalence data. We illustrate their use with multi-resistant methicillin-resistant Staphylococcus aureus (mMRSA) prevalence and new colonisation data. These methods are able to deal with excess variability, trends and temporal correlation. They are easily implemented in the freely available R software package. Our analysis demonstrates an upward non-linear trend in mMRSA prevalence between January 2004 and October 2006. The mMRSA new colonisation data also display an upward trend between September 2005 and May 2006. Monthly new colonisation rates exceeded the upper control limit in April 2005 and equalled it in May 2006. There was a modest downward trend in the new colonisation rate in the latter part of 2006.
Introduction MROs are an increasing problem in many hospitals causing patient injury and death, increased lengths of stay and economic loss 1. Monthly new colonisations with MROs are routinely monitored using a variety of techniques including control charts 2; this assists with the timely detection of outbreaks and in judging responses to system changes designed to reduce colonisation rates. In addition, MRO prevalence, often referred to as MRO ‘burden’ or ‘colonisation pressure’, is being monitored increasingly because of its importance in transmission 3.
data require an underlying system that is behaving predictably and is therefore in statistical control. This is usually achieved by the successful implementation of evidence-based systems that are increasingly implemented as ‘bundles’4. These have been successful in reducing rates of surgical site infections and device-related bacteraemias 5. However, to date they have been less successful with MRO colonisations, making runs of stable non-trended (stationary) data difficult to achieve. In addition, MRO colonisation is not an independent process as it is due to transmission from another carrier. Excessive variability is a common consequence of this lack of independence.
Conventional monthly control charts require independence of observations and a run of stable, non-trended (stationary) monthly values, preferrably as many as twenty-four, so that accurate centre lines and control limits can be calculated. Stable, non-trended 14
Temporal correlation (autocorrelation) is a feature of MRO prevalence data. For example, today’s MRO prevalence count tends to be more like yesterday’s count than those in previous weeks. Vol 12
Issue 1
March 2007
Australian Infection Control
All of these factors make the use of conventional control charts difficult and consequently their use may give misleading information. Here we show that GAMs 6 for data that do not demonstrate temporal correlation and GAMMs for data that are temporally correlated provide alternative analysis tools to control charts for these types of data.
Methods Daily prevalence of mMRSA, non-multi-resistant methicillinresistant Staph. aureus (nmMRSA), UK epidemic methicillinresistant Staph. aureus (ukEMRSA), multi-resistant Klebsiella pneumoniae, vanconycin-resistant Enterococcus (VRE) and multiresistant Acinetobacter has been recorded at the Princess Alexandra Hospital (PAH), Brisbane, since November 2003. The data from November 2003 to mid-January 2007 are complete for 89% of those 1169 days. Interpolated daily prevalence values, estimated from the data immediately before and after the gaps, have been substituted for the missing data. Monthly counts of new colonisations with these organisms are the subject of the routine surveillance program at PAH. Swabbing of all ICU patients occurs twice-weekly and once-weekly for patients in other high-risk areas as recommended by the Australian Infection Control Association National Advisory Board. The data for January 2004 to December 2006 inclusive are complete and have been employed in the study. We illustrate the analysis of the mMRSA data using GAMs and GAMMs 6. These models are described in the Appendix. The average mMRSA prevalence for each month was determined and rounded to the nearest integer value. Analysis of these data between January 2004 and December 2006 was then performed. To account for the presence of temporal correlation, we have employed the GAMM in the mgcv package 6 in R 7. In addition an approximate upper two standard deviation equivalent control limit was calculated. There is further description in the Appendix. An occupied bed days (OBDs) denominator was included in the analysis. The monthly number of new mMRSA isolates from January 2004 to December 2006 was analysed using a GAM as described in the Appendix. The required R commands for these analyses are available from the corresponding author. Because some infection control practitioners may have difficulty with a statistics package, as opposed to familiar spreadsheets that are incapable of implementing GAM methods, these commands will be automated in easy-to-use functions.
Results Monthly mMRSA prevalence for the study period is displayed in Figure 1. There was significant temporal correlation (the correlogram is not shown). The GAMM model produced a highly significant result 16
(P<.0001) and indicated an upward non-linear trend in prevalence between January 2004 and October 2006. The fitted estimates using the GAMM were 19.7 cases per month (95% CI 17 to 22.4) in January 2004 reaching a peak of 36.6 cases per month (95% CI 32.9 to 40.6) in October 2006. The upper dotted line in Figure 1 shows the approximate two sigma equivalent upper control limit that quantifies the relationship of the individual monthly values to the GAMM estimates. Over the study period there were no values that exceeded the control limit. The corresponding Shewhart/EWMA SPC chart 8 gives similar information (Figure 2 produced in Microsoft Excel®). Unfortunately, because of the presence of temporal correlation and a trend, users will have no idea whether the centre line and control limits in the SPC chart correctly summarise the time series. The monthly new mMRSA colonisations did not show any significant temporal correlation; therefore, the simpler GAM was employed (Figure 3). These data show an upward non-linear trend commencing in September 2005 (GAM estimate 8.2, 95% CI 5.3 to 12.7) and reaching a peak in May 2006 (GAM estimate 19.4, 95% CI 16 to 23.3). This is highly statistically significant (P<.0005). There was a modest fall in the rate of new colonisation in the latter part of 2006. The upper dotted line in the figure shows the approximate two sigma equivalent upper control limit that quantifies the relationship of the individual monthly values to the GAM estimates. There was a count higher than this in March 2005 and one equal to it in May 2006. Figure 4 is the corresponding Shewhart/EWMA SPC chart 8. Once again, the charts give similar information but, because of the high variability and trend seen with these data, there is no way of knowing how accurately the SPC chart summarises the time-series.
Discussion Conventional control charts require observations to be independent 9. MRO colonisation and infection frequently violate this requirement as colonisation results from transmission from a carrier. This often results in excessive variability. Special methods such as employing the negative binomial distribution have been suggested to overcome this problem but these methods increase the complexity of the analysis 8. Corcoran and Speekenbrink 9 also point out that it is often difficult to get a sufficiently long run of observations that are stable and predictable (in statistical control) as well as stationary (that do not display a trend) to calculate reliable in-control centre lines and control limits. Our analysis of the prevalence and new colonisation data for mMRSA indicate that these data display nonlinear trends that make it difficult to find stable mean and accurate control limits. Finally, the MRO prevalence data display temporal correlation. Although SPC methods exist for the analysis of these data 10 their use introduces additional complexity and the interpretation of the resulting charts can be difficult For these reasons, we have developed a new approach that uses the non-linear GAM and GAMM to estimate the expected number Vol 12
Issue 1
March 2007
Australian Infection Control
of colonisations and superimposes, above the expected number, an approximate upper two sigma equivalent control limit. Although the number of monthly data values that are required is unlikely to be much reduced compared to the SPC analysis, there is not the need for a long run of data values that are in statistical control to obtain reliable control limits for the latter. In addition, weekly data that are usually too variable to use in SPC charts can be analysed with a GAM or GAMM provided there are not very small weekly counts. This may bring analysis closer to real-time thus potentially helping with earlier detection of new outbreaks. These methods successfully deal with temporal correlation, excessive variability and the presence of non-linear trends. Although GAM and GAMM charts may be slightly more difficult to interpret than conventional SPC charts, the two major points can be seen: 1. The predicted GAM estimate and its confidence interval provide reliable information about the average value at any time together with the observed trends that indicated the need for investigation and possible system change. 2. The control limit provides a means for detecting a sudden increase in the prevalence or the number of new isolates of the MRO of interest, analogous to the control limits of a conventional control chart. This occurred with the mMRSA new colonisation data and would have suggested the need to search for a possible additional cause at those times. It is well known that taking averages at intervals, for example the monthly averages of the daily prevalence values, can markedly reduce temporal correlation 10 although, with our monthly prevalence data, some temporal correlation persisted. If there is no temporal correlation, the simpler GAM procedure that is also in the mgcv library 6 in R 7 can be employed. The prevalence and new colonisation charts show marked upward trends at roughly the same time. It is expected that increasing numbers of new colonisations will also increase the prevalence numbers so a causal relationship cannot be proved with these data. However, it has been demonstrated that such a relationship can exist 3. In addition, before the monitoring of MRO prevalence commenced late in 2003, we saw an outbreak due to VRE that monopolised isolation facilities to the extent that many mMRSA carriers could not be placed in isolation beds. There was a significant rise in the rate of new mMRSA colonisations at that time 11. A serious consequence of the inappropriate use of conventional SPC charts is the occurrence of frequent false-positive signals. In a judgmental environment, this can provoke tampering with systems that are not malfunctioning or out of control. Such tampering creates uncertainty and promotes error. The approach reported here overcomes this problem by acknowledging the unique and difficult characteristics of these data that can be excessively variable, trended and temporally correlated. Vol 12
Issue 1
March 2007
As with all monitoring of infections, not all carriers or new colonisations are detected. It is not economic to screen patients in low-risk areas like psychiatry. In addition, newly colonised patients may be discharged between screenings from a high-risk area like ICU where swabbing is carried out twice weekly so occasional new colonisations will go undetected. Finally, there are small delays with the identification of the MRO in the microbiology laboratory and the procedures used are unlikely to have perfect sensitivity and specificity. However, the surveillance system at PAH remained relatively constant during the observation period so changes in MRO prevalence and new colonisations should have been detected reliably. An important issue is when the MRO burden reaches the level that overwhelms efforts to curtail transmission, for example by isolating or cohorting carriers. Since there are multiple factors involved in transmission and they must in general be the subject of observational studies, the use of statistical modeling techniques such as Bayesian networks will be necessary to utilise and understand these data fully.
References 1. 2.
3.
4.
5.
6. 7. 8.
9. 10. 11.
12.
Spelman D. Hospital-acquired Infections. MJA 2002;176:286-291. Curran E, Benneyan J & Hood J. Controlling methicillin-resistant Staphylococcus aureus: a feedback approach using annotated statistical process control charts. Infect Cont & Hosp Epidem 2002;23:13-18. Merrer J, Santoli F, Vecchi C, Tran B, De Jonghe B &d Outin H. Colonisation pressure and risk of acquisition of methicillin-resistant Staphylococcus aureus in a medical intensive care unit. Infect Cont & Hosp Epidem 2000;21:718-723. Institute for Healthcare Improvement. Raising the bar with bundles. Viewed 21 February 2007
Appendix R 7,12 (The R Core Development Team) is a very comprehensive open source statistics package that implements the S statistical computing language. It can be freely downloaded from
Australian Infection Control
2.5
2
RATE PER 1000 BED-DAYS
1.5
1
0.5
Observation of the plot of autocorrelations for the mMRSA prevalence data (not shown) revealed an exponentially decaying correlation function where consecutive monthly mMRSA counts were more correlated than values that were more separated in time. This suggested that a first-order autoregression (AR1) correlation object would be suitable for use with these data. The GAMM is:
Oct-06
Dec-06
Nov-06
Jul-06
Sep-06
Jun-06
Aug-06
Apr-06
Mar-06
May-06
Jan-06
Feb-06
Dec-05
Oct-05
Nov-05
Jul-05
Sep-05
Aug-05
Apr-05
Jun-05
Mar-05
May-05
Jan-05
Feb-05
Dec-04
Oct-04
Nov-04
Sep-04
Jul-04
Aug-04
Apr-04
Jun-04
MONTHS Rate
Mean rate
+2 sigma S
+3 sigma S
-3 sigma S
EWMA
+ 2.5 sigma E
- 2.5 sigma E
-2 sigma S
Figure 3. Generalised additive model for monthly MRSA new colonisations Jan 2004 to Dec 2006 with 95% confidence limits & Upper 2 Sigma-equivalent control limit. 30
25 Monthly New MRSA Colonisations
The GAM and GAMM predict the smoothed mean values for each time period together with their 95% confidence limits. Using these smoothed mean values as expected values, the proposed approximate upper two sigma equivalent control limit is at the 97.5th percentile of the Poisson distribution (Figures 1 and 3).
May-04
The GAMM analysis of the temporally correlated monthly prevalence data employs the Poisson family, a correlation object (corAR1) and log (OBDs) as an offset. The GAM analysis of the new colonisation data that are uncorrelated also employs the Poisson family with log (OBDs) as an offset but no correlation object.
Mar-04
0 Feb-04
GAMs are regression models that can be used to study non-linear relationships between variables. Non-parametric smoothers, usually based on splines, are used to explain those relationships. GAMMs have the additional capability of dealing with extraneous variability, in this case due to the temporal correlation (autocorrelation) in the prevalence data. These models are suitable for the analysis of time-series data in the form of counts that may be small and that may display excess variability, non-linear trends and temporal correlation.
Figure 2. Shewhart/EWMA Chart monthly average mMRSA prevalence.
Jan-04
GAMs. However, it has the added ability to extend to a GAMM. It can be downloaded from
logE(mMRSA)=gamm(mMRSA~s(time),family=poisson,offset=log( OBD), correlation=corAR1()), where E(mMRSA)t is the fitted value at time t. This model has only the intercept in its parametric part. The non-parametric part is the smooth-spline s(time). The offset term brings log(OBD) into the model with a fixed coefficient of one
20
15
10
5
0 Jan20
Apr20
Jul20
Oct20
Jan20
Apr20
Jul20
Oct20
Jan20
Apr20
Jul20
Oct20
Data blue, GAM estimate solid red, Confidence limits dashed red, Control limit dotted red
Figure 4. Shewhart/EWMA Chart mMRSA new colonisations. Figure 1. Generalised additive mixed model for monthly MRSA prevalence Jan 2004 to Dec 2006 with 95% confidence limits & Upper 2 Sigma-equivalent control limit.
1.6
1.4
RATE PER 1000 BED-DAYS
1.2
50
40 35
1
0.8
0.6
0.4
30 0.2
25
15 Jan04
Dec-06
Oct-06
Nov-06
Sep-06
Jul-06
Jun-06
Aug-06
Apr-06
May-06
Mar-06
Jan-06
Feb-06
Oct-05
Dec-05
Nov-05
Jul-05
Sep-05
Aug-05
Jun-05
Apr-05
Mar-05
MONTHS
Apr04
Jul04
Oct04
Jan05
Apr05
Jul05
Oct05
Jan06
Apr06
Data blue, GAM estimate solid red, Confidence limits dashed red, Control limit dotted red
18
May-05
Jan-05
Feb-05
Dec-04
Oct-04
Nov-04
Sep-04
Jul-04
Aug-04
Jun-04
May-04
Apr-04
Mar-04
Feb-04
0
20
Jan-04
Monthly New MRSA Colonisations
45
Jul06
Oct06
Rate
Mean rate
+2 sigma S
+3 sigma S
-3 sigma S
EWMA
+ 2.5 sigma E
- 2.5 sigma E
Vol 12
-2 sigma S
Issue 1
March 2007