- Email: [email protected]
Quantifying the Hawthorne effect using overt and covert observation of hand hygiene at a tertiary care hospital in Saudi Arabia
American Journal of Infection Control 46 (2018) 930-5
Contents lists available at ScienceDirect
American Journal of Infection Control
American Journal of Infection Control
j o u r n a l h o m e p a g e : w w w. a j i c j o u r n a l . o r g
Major Article
Quantifying the Hawthorne effect using overt and covert observation of hand hygiene at a tertiary care hospital in Saudi Arabia Aiman El-Saed MD, PhD a,b,c, Seema Noushad MD a, Elias Tannous CIC d, Fatima Abdirizak MPH e, Yaseen Arabi MD b,f,j, Salih Al Azzam MD b,g, Esam Albanyan MD b,h, Hamdan Al Jahdalil MD b,h, Reem Al Sudairy MD i, Hanan H. Balkhy MD a,b,j,* a
Infection Prevention and Control Department, King Abdulaziz Medical City, Ministry of National Guard Health Affairs, Riyadh, Saudi Arabia King Saud bin Abdulaziz University for Health Sciences, Riyadh, Saudi Arabia Community Medicine Department, Faculty of Medicine, Mansoura University, Mansoura, Egypt d Quality and Patient Safety Institute, Cleveland Clinic, Abu Dhabi, United Arab Emirates e School of Public Health, Georgia State University, Atlanta, GA f Intensive Care Department, King Abdulaziz Medical City, Ministry of National Guard Health Affairs, Riyadh, Saudi Arabia g Department of Surgery, King Abdulaziz Medical City, Ministry of National Guard Health Affairs, Riyadh, Saudi Arabia h Department of Medicine, King Abdulaziz Medical City, Ministry of National Guard Health Affairs, Riyadh, Saudi Arabia i Department of Oncology, King Abdulaziz Medical City, Ministry of National Guard Health Affairs, Riyadh, Saudi Arabia j King Abdullah International Medical Research Center, Riyadh, Saudi Arabia b c
Key Words: Hospital Health care workers Hand hygiene compliance
Introduction: Although direct human observation of hand hygiene (HH) is considered the gold standard for measuring HH compliance, its accuracy is challenged by the Hawthorne effect. Objectives: To compare HH compliance using both overt and covert methods of direct observation in different professional categories, hospital settings, and HH indications. Methods: A cross-sectional study was conducted in 28 units at King Abdulaziz Medical City, Riyadh, Saudi Arabia, between October 2012 and July 2013. Compliance was defined as performing handrubbing or handwashing during 1 of the World Health Organization 5 Moments for HH indications (ie, opportunities). Overt observation was done by infection preventionists (IPs) who were doing their routine HH observation. Covert observation was done by unrecognized temporarily hired professionally trained observers. Results: A total of 15,883 opportunities were observed using overt observation and 7,040 opportunities were observed using covert observation. Overall HH compliance was 87.1% versus 44.9% using overt/ covert observations, respectively (risk ratio, 1.94; P < .001). The significant overestimation was seen across all professional categories, hospital settings, and HH indications. Conclusion: There is a considerable difference in HH compliance being observed overtly and covertly in all categories. More work is required to improve the methodology of direct observation to minimize the influence of the Hawthorne effect. © 2018 Association for Professionals in Infection Control and Epidemiology, Inc. Published by Elsevier Inc. All rights reserved.
Approximately 7.5% of patients admitted to hospitals in developed countries and 10% in developing countries end up with health care-associated infections.1 Between 20% and 40% of these infections may be directly attributed to transmission from contaminated hands of health care workers (HCWs).2 Therefore, hand hygiene (HH)
* Address correspondence to Hanan H. Balkhy, MD, Pediatric Infectious Disease, King Saud bin Abdulaziz University for Health Sciences, Infectious Diseases, King Abdullah International Medical Research Center, and Infection Prevention and Control, King Abdulaziz Medical City, PO Box 22490, Riyadh 11426, Saudi Arabia. E-mail address: [email protected] (H.H. Balkhy). Conflicts of interest: None to report.
is considered the single most important strategy to reduce the incidence of health care-associated infections.3,4 Additionally, it is a core element for preventing the spread of antimicrobial resistance and reducing colonization of multiresistant microorganisms.5,6 Therefore, it is among the critical indicators for patient safety required by hospitals to be granted accreditation.7 Although the benefits of HH are well known and noncontroversial, the HH compliance in health care settings is still suboptimal, with an average compliance rate of 40%.8 Monitoring HH compliance serves multiple functions: it stimulates HCWs to improve their performance, helps to improve infrastructure design, and works as objective assessment of the
0196-6553/© 2018 Association for Professionals in Infection Control and Epidemiology, Inc. Published by Elsevier Inc. All rights reserved. https://doi.org/10.1016/j.ajic.2018.02.025
A. El-Saed et al. / American Journal of Infection Control 46 (2018) 930-5
quality of care.9,10 Our institution has been conducting HH observations for more than 10 years. We were concerned with the accuracy of the reported high compliance. Although direct human observation of HH practices is considered the gold standard for measuring HH compliance,9,11 its accuracy is challenged by interobserver variability and the Hawthorne effect.9,12 The latter, which is defined as a change in HCW behavior because of the awareness of being observed, is known to overestimate compliance rates.13-15 The influence of Hawthorne effect may be variable in high- or low-compliant hospital locations.16 Covert observation was suggested as a tool to quantify or overcome Hawthorne effect bias.17-19 Few studies tried to quantify the amount of overestimation of HH compliance using the overt or covert methods in different hospital settings, indications, and professional categories.15,20 However, these studies either did not use the standard World Health Organization (WHO) 5 Moment methodology or observed an insufficient number of HH opportunities for appropriate stratification of data. Additionally, such data are absolutely lacking in Saudi Arabia. The objective of the current study was to compare HH compliance using overt or covert methods of direct observation in different professional categories, hospital settings, and HH indications. METHODS Setting The current study was conducted at King Abdulaziz Medical CityRiyadh (KAMC-R), Ministry of National Guard Health Affairs in Saudi Arabia. KAMC-R is a 1,000-bed tertiary care facility that is funded by the government. It provides health care services for about 750,000 Saudi National Guard soldiers, employees, and their families. The care provided ranges from primary and preventive care to tertiary care. At the time of the study, approximately 9,170 HCWs were working for KAMC-R in jobs that involved direct patient care, including approximately 1,670 physicians, 4,660 nurses, and 2,840 other HCWs. KAMC-R has a multisection emergency department (150 beds), 13 different intensive care units (ICUs) (total of 185 beds), and 36 wards covering all other specialties. At the time of the study, the emergency department was serving more than 250,000 visits a year, ICUs were serving 5,000 admissions a year, and wards were serving 27,000. The data about KAMC-R–served populations and HCWs were obtained from the annual census reports for 2013 and 2014. Population The study targeted clinical HCWs (who were directly involved in patient care) in different departments of KAMC-R, including physicians, nurses, and other HCWs. The latter included therapists, technicians, laboratory personnel, emergency medical service personnel, dental personnel, and pharmacists. Nonclinical HCWs not directly involved in patient care such as clerical, dietary, laundry, security, maintenance, and administrative jobs were not included.
931
temporarily hired professionally trained observers during the same period. Study outcome HH compliance was defined as doing either handrubbing (with alcohol-based formulation) or handwashing (with soap and water) during 1 of the WHO 5 Moment HH indications (ie, opportunities): before patient contact, before an aseptic task, after exposure to body fluids, after patient contact, and after contact with patient surroundings.21 More than 1 indication falling into the same opportunity was allowed. Data collection For the overt observation, a standard WHO HH observation form (with space for 32 opportunities) was used in collecting HH compliance per WHO methods.21 The data of more than 1 HCW of different specialty were collected on the same form. Data collection was performed by IPs who were routinely collecting HH data for at least 6 months for the infection control department. For the covert observation, WHO HH observation forms (slightly modified to allow the data of only 1 HCW to be collected on the same form) were used in collecting HH compliance per WHO methods. Observation of the same HCW was continued until the session ended or 20 HH opportunities were observed. Data collection was performed by 2 temporarily hired observers who passed a training test on official WHO HH videos containing standard scenarios as well as real hospital settings. Additionally, validation of HH observation was done by occasional concomitant observation by a senior IP, with almost identical findings (κ > 0.9). The temporarily hired observers doing covert observation wore nursing uniforms but their monitoring task was not revealed to the observed HCWs. In both observation methods, HH observation was done in a series of sessions no more than 30 minutes (mean, 20 ± 10 minutes) each. Data collection from the same unit was continued until at least 200 HH opportunities were observed. Additionally, HH observation was done quietly without attempts to promote HH compliance or provide performance feedback to HCWs. Statistical methods Categorical variables are presented as frequencies and percentages, whereas continuous variables are presented as means ± standard deviation. The χ2 or Fisher exact test, as appropriate, was used to test significant differences in HH compliance between overt and covert observation. Additionally, risk ratio (RR) and odds ratio (OR) for HH compliance using overt compared with covert observations were calculated using standard ways of calculations. Student t test or Mann-Whitney U test, as appropriate, was used to test significant differences in continuous variables between the overt and covert observations. All P values were 2-tailed. A P value < .05 was considered significant. SPSS version 23.0 (IBM-SPSS Inc, Armonk, NY) was used for all statistical analyses. RESULTS
Study design A cross-sectional study design was conducted at KAMC-R between October 2012 and July 2013. The study received all required ethical approvals from King Abdullah International Medical Research Center, Riyadh, Saudi Arabia, before data collection. Overt observation was done by presumably well-recognized infection preventionists (IPs) who were doing their routine HH observation during the study period. Covert observation was done by presumably unrecognized
As shown in Table 1, a total of 15,883 opportunities were observed during 725 observation sessions done using overt observation across 270.9 hours of observations. On the other hand, a total of 7,040 opportunities were observed during 298 sessions done using covert observation across 148.8 hours of observations. For the 2 observation methods, the majority of the observed HCWs were nurses (62.2% and 58.7%). With slight differences between the 2 observation methods, the majority (48.3%) of all opportunities were observed
932
A. El-Saed et al. / American Journal of Infection Control 46 (2018) 930-5
Table 1 Comparisons of the 2 observation methods Overt observation Observation sessions Overall number of sessions Number of sessions by hospital location Emergency department Intensive care units Wards Average duration per session (min) Total duration of sessions (h) Hand hygiene opportunities Overall number of opportunities Number of opportunities by professional category Nurses Physicians Other health care workers Number of opportunities by hospital location Emergency department Intensive care units Wards Hand hygiene indications Overall number of indications Type Before patient contact Before aseptic tasks After exposure to body fluids After patient contact After contact with surroundings Observed health care workers Nurses Physicians Other health care workers Total Observing staff
Covert observation
Total
725 (70.9)
298 (29.1)
150 (20.7) 236 (32.6) 339 (46.8) 22.4 ± 9.4 270.9
69 (23.2) 74 (24.8) 155 (52.0) 29.9 ± 7.7 148.8
219 (21.4) 310 (30.3) 494 (48.3) 24.7 ± 9.6 421.6
15,883 (69.3)
7,040 (30.7)
22,923 (100.0)
9,877 (62.2) 3,994 (25.1) 2,012 (12.7)
4,135 (58.7) 1,358 (19.3) 1,547 (22.0)
14,012 (61.1) 5,352 (23.3) 3,559 (15.5)
4,304 (27.1) 5,061 (31.9) 6,518 (41.0)
1,728 (24.5) 1,933 (27.5) 3,379 (48.0)
6,032 (26.3) 6,994 (30.5) 9,897 (43.2)
15,918 (69.3)
7,069 (30.7)
22,987 (100.0)
5,511 (34.6) 360 (2.3) 299 (1.9) 3,605 (22.6) 6,142 (38.6)
1,899 (26.9) 727 (10.3) 427 (6.0) 1,672 (23.7) 2,343 (33.1)
7,410 (32.2) 1,087 (4.7) 726 (3.2) 5,277 (23.0) 8,485 (36.9)
NA NA NA NA 11
289 (53.7) 119 (22.1) 130 (24.2) 538 (100.0) 2
1,023 (100.0)
NA NA NA NA 13
Values are presented as n (%) or mean ± standard deviation. NA, not available.
Fig 1. Hand hygiene action by observation method. Fig 2. Wearing gloves status (%) in noncompliant opportunities by observation method.
in wards, followed by ICUs (30.3%) and the emergency department (21.4%). With slight differences between the 2 observation methods that does not influence the order, overall, the most common HH indication was “after contact with the patient surroundings” (36.9%), followed by “before patient contact” (32.2%), “after patient contact” (23.0%), “before aseptic tasks” (4.7%), and lastly “after exposure to body fluids” (3.2%). As shown in Figure 1, there was a large and highly significant difference in the HH compliance between the 2 observation methods, being much higher in overt compared with covert observations (87.1% vs 44.9%; P < .001). As shown in Table 1, the RR for the overt compared with the covert observations was 1.94, with even much higher
OR (8.26). The observed difference was mainly due to difference in alcohol-based handrubbing, which represented 92.8% of appropriate actions taken. As shown in Figure 2, wearing gloves with lack of compliance (N = 807 opportunities) was lower in overt compared with covert observations (10.1% vs 15.4%; P < .001). As shown in Table 2, HH compliance was significantly higher in overt compared with covert observations after stratification by professional categories and hospital locations (P < .001 for all). For example, HH compliance in nurses was 89.8% using overt observation compared with 47.0% using covert observation (RR, 1.91; P < .001). Similarly, HH compliance in ICUs was 91.2% using overt
A. El-Saed et al. / American Journal of Infection Control 46 (2018) 930-5
933
Table 2 Hand hygiene compliance by observation method, hospital location, and professional category Overt observation (opportunities) Observed (n)
Compliant (n)
Compliance (%)
15,883
13,828
9,877 3,994 2,012 346 15,537 4,304 5,061 6,518 1,194 1,655 1k395 2k274
Overall Professional category Nurses Physicians Other health care workers Respiratory therapists Nonrespiratory therapists Hospital location Emergency department Intensive care units Ward Medical Surgical Obstetrics Pediatrics
Covert observation (opportunities) P value1
Observed (n)
Compliant (n)
Compliance (%)
P value2
Risk ratio
Odds ratio
87.1
–
7,040
3,161
44.9
–
1.94
8.26
< .001
8,870 3,212 1,746 306 13,522
89.8 80.4 86.8 88.4 87.0
< .001
4,135 1,358 1,547 600 6,440
1,942 567 652 275 2,886
47.0 41.8 42.1 45.8 44.8
< .001
1.91 1.92 2.06 1.93 1.94
9.95 5.73 9.01 9.04 8.26
< .001 < .001 < .001 < .001 < .001
3,610 4,616 5,602 1,078 1,423 1,073 2,028
83.9 91.2 85.9 90.3 86.0 76.9 89.2
< .001
1,728 1,933 3,379 542 873 627 1,337
725 827 1,609 293 446 304 566
42.0 42.8 47.6 54.1 51.1 48.5 42.3
< .001
2.00 2.13 1.80 1.67 1.68 1.59 2.11
7.20 13.87 6.73 7.90 5.87 3.54 11.23
< .001 < .001 < .001 < .001 < .001 < .001 < .001
.440
< .001
.631
< .001
P value3
Differences between groups were evaluated in overt observation using P value1 and in covert observation using P value2. Differences between overt and covert observations were evaluated using P value3. All P values were derived from χ2 or Fisher exact test, as appropriate.
Table 3 Hand hygiene compliance by observation method, professional category, and hand hygiene indication Overt observation (opportunities)
Overall Before patient contact Before aseptic tasks After exposure to body fluids After patient contact After contact with surroundings Nurses Before patient contact Before aseptic tasks After exposure to body fluids After patient contact After contact with surroundings Physicians Before patient contact Before aseptic tasks After exposure to body fluids After patient contact After contact with surroundings Other health care workers Before patient contact Before aseptic tasks After exposure to body fluids After patient contact After contact with surroundings
Observed (n)
Compliant (n)
Compliance (n)
5,511 360 299 3,605 6,142
4,934 340 291 3,227 5,068
89.5 94.4 97.3 89.5 82.5
3,377 285 239 2,254 3,749
3,129 273 232 2,079 3,182
1,379 43 41 899 1,635
1,155 39 41 757 1,223
755 32 19 452 758
650 28 18 391 663
Covert observation (opportunities) Risk ratio
Odds ratio
P value3
< .001
1.75 3.01 2.26 2.35 1.67
8.22 37.21 48.04 13.87 4.81
< .001 < .001 < .001 < .001 < .001
54.1 35.5 50.0 40.0 50.3
< .001
1.71 2.70 1.94 2.31 1.69
10.68 41.27 33.14 17.84 5.55
< .001 < .001 < .001 < .001 < .001
216 10 10 122 210
48.6 16.4 26.3 35.6 44.4
< .001
1.72 5.53 3.80 2.37 1.68
5.44 49.73 9.66 3.72
< .001 < .001 < .001 < .001 < .001
184 35 19 136 280
45.3 23.2 24.1 35.7 52.1
< .001
1.90 3.77 3.93 2.42 1.68
7.47 23.20 56.84 11.55 6.41
< .001 < .001 < .001 < .001 < .001
Observed (n)
Compliant (n)
Compliance (%)
< .001
1,899 727 427 1,672 2,343
968 228 184 637 1,160
51.0 31.4 43.1 38.1 49.5
92.7 95.8 97.1 92.2 84.9
< .001
1,049 515 310 948 1,333
568 183 155 379 670
83.8 90.7 100.0 84.2 74.8
< .001
444 61 38 343 473 406 151 79 381 537
P value1
86.1 87.5% 94.7% 86.5% 87.5%
.788
P value2
Differences between different indications were evaluated in overt observation using P value1 and in covert observation using P value2. Differences between overt and covert observations were evaluated using P value3. All P values were derived from χ2 or Fisher exact test, as appropriate.
observation compared with 42.8% using covert observation (RR, 2.13; P < .001). The difference between the 2 observation methods was more variable in hospital locations than in professional categories. For example, the RRs ranged between 1.59 in obstetric wards and 2.13 in ICUs. On the other hand, the RRs ranged between 1.91 in nurses and 2.06 in other HCWs. In both observation methods, nurses had the highest compliance, followed by other HCWs and then physicians (P < .001 for each). In both observation methods, the emergency department had the lowest compliance (P < .001 for each). On the other hand, ICUs had the highest compliance in overt observation (91.2% vs 85.1%; P < .001), whereas wards had the highest compliance in covert observation (47.6% vs 42.4%; P < .001). As shown in Table 3, HH compliance was significantly higher in overt compared with covert observations after stratification by
HH indications (either overall or professional category-specific; P < .001 for all). For example, HH compliance for “before patient contact” was 89.5% using overt observation compared with 51.0% using covert observation (RR, 1.75; P < .001). The difference between the 2 observation methods was highest with “before aseptic tasks” indication (RR, 3.01) and lowest with “after contact with surroundings” indication (RR, 1.67). Additionally, within each observation method there was great variability with regard to compliance by HH indications. For example, the compliance was highest for “after exposure to body fluids” and lowest for “after contact with surroundings” using overt observation. On the other hand, the compliance was highest for “before patient contact” and lowest for “before aseptic tasks” using covert observation. Largely similar findings were observed after stratification of HH indications by professional category.
934
A. El-Saed et al. / American Journal of Infection Control 46 (2018) 930-5
DISCUSSION The key finding of the current study was the huge difference in HH compliance using 2 different observation methods. We are reporting an overall 94% overestimation of HH compliance rates using overt compared with covert observations, which translates into approximately 42% net difference between both methods. In previous studies, the overt method has been consistently associated with overestimation of HH compliance, which is commonly attributed to the Hawthorne effect.15-17,20,22,23 However, the degree of overestimation in these studies was considerably variable, with a range of 12% to 124% relative overestimation and 7% to 50% net difference between both methods.15-17,20,22,23 The overestimation was even higher in studies that used electronic monitoring to quantify the Hawthorne effect.14,24 This variability may be related to the baseline HH compliance16 and the different and sometimes poor methods used to detect the overestimation in those studies. For example, the majority of the studies that sought to quantify the overestimation using direct observation did not use the standard WHO 5 Moments methodology, which make fair comparisons difficult.15,17,22 Additionally, some of these studies were clearly underpowered20 or even failed to have a true control group.18,19 Although the overestimation of HH compliance in our study was clearly seen in all examined groups, the degree of overestimation tends to be more variable in hospital locations and HH indications. The observed variability probably reflected the variability in overt compliance in different strata of these groups. For example, when we analyzed the data after stratifying the overt compliance as below and above the average (ie, 87.1%), the degree of overestimation was much higher among hospital locations and HH indications that had overt compliance above the study average (data not shown). This was very true with ICUs and “before aseptic tasks” indication, which had >90% overt compliance and >100% overestimation compared with the obstetrics ward and “after contact with surroundings” indication, which had <83% overt compliance and <70% overestimation. Similarly, it has been suggested that the observation of staff compliance has little influence in low-compliant hospital locations but a notable influence in high-compliant hospital locations.16 Although the same findings have not been adequately examined in other studies, reviewing the reported location-based overestimation in these studies showed conflicting findings.15,17,22 The higher overestimation with “before aseptic tasks” HH indication in both nurses and physicians may point to the effect of socially desirable behavior during overt observation. Additionally, when the overestimation was stratified by before-and-after indications (data not shown), the overestimation was significantly higher in before than in after indications (97% vs 92%; P < .001). This may be explained by the perception that patient protection (primarily before indications) is probably more socially desirable behavior than HCW protection (primarily after indications). Our study used unrecognized, temporarily hired professionally trained observers to estimate the real HH compliance during the same period recognized IPs were doing their routine HH observation. Previous studies used different methods to detect the real (covert) HH compliance. For example, several studies used medical students as unrecognized observers.16,17,19,20 Although students are readily available and usually unrecognized by unit staff, they frequently lack knowledge of several invasive procedures and are commonly restricted from some areas.17 A number of studies used the same observers in 2 different periods, 1 with announcement of HH monitoring and the other without announcement.15,23 Obviously, this method ignores the possibility of real differences between periods, requires that the covert period to be before the overt period, and cannot be repeated without some breach of the concealment because the observer is already known. Finally,
electronic monitoring systems have been used to quantify the Hawthorne effect.14,24 However, the inherent limitations of this method can further inflate the Hawthorne effect, with up to 3-fold overestimation.14 Previous studies described the overestimation of HH compliance using overt or covert methods in absolute terms and a few of them additionally calculated the OR of the difference. 15,22 Although we determined the relative overestimation using the risk ratios and ORs, we believe that the OR is not the optimal metric to quantify the overestimation because it tends to further augment the overestimation, a phenomenon that is known to naturally happen when the characteristic under study has a prevalence >10%.25 Therefore, we relied on the risk ratio in our reference to the degree of overestimation throughout this article. Although the current study is reconfirming a huge difference in HH compliance using 2 different methods of direct observation, we still believe that direct HH observation is the gold standard for measuring compliance.9,11 Direct observation is more likely to better assess the timing, indication-specific compliance, and technique of HH action than indirect or automated observation.9 Additionally, it is the monitoring method recognized by WHO and implemented in different health care settings around the world.12,21 Therefore, future work should focus on how to improve the methodology of direct observation to minimize the influence of the Hawthorne effect rather than using indirect or automated observation. It has been suggested that limiting duration of the observation session to approximately 15 minutes can minimize the Hawthorne effect.26 Additionally, frequent and unobtrusive observation may acclimatize staff to the presence of observers.27 Finally, training non-IPs to perform auditing observation in units other than their own and frequently rotating IPs to different hospital areas may help minimize the Hawthorne effect in large hospitals. Although the current study is unique in quantifying the Hawthorne effect in different professional categories, hospital settings, and HH indications using the standard WHO 5 Moments methodology in a large number of observations, we acknowledge a number of limitations. Being a single-center study, the findings of our study should be generalized with caution. Although we used unrecognized, temporarily hired observers to estimate HH compliance covertly, we cannot exclude some breaches of concealment, especially with longer sessions and recruiting multiple HCWs from the same unit. However, with the current methods the influence of this (if any) is believed to be minimal and the huge overestimation reported is by itself a sign of good covert methodology. CONCLUSIONS We are reporting an overall 94% overestimation of HH compliance rates using overt compared with covert observations. Although the overestimation was clearly seen in all examined groups, the degree of overestimation tended to be more variable in hospital locations and HH indications. More work is needed to improve the methodology of direct observation to minimize the influence of the Hawthorne effect. The effectiveness of any suggested methodology change needs to be evaluated in future studies. References 1. World Health Organization. Report on the Burden of Endemic Health CareAssociated Infection Worldwide. A systematic review of the literature. Available from http://apps.who.int/iris/bitstream/handle/10665/80135/978924150 1507_eng.pdf?sequence=1&isAllowed=y; 2011. Accessed February 1, 2015. 2. Weber DJ, Rutala WA, Miller MB, Huslage K, Sickbert-Bennett E. Role of hospital surfaces in the transmission of emerging health care-associated pathogens: norovirus, Clostridium difficile, and Acinetobacter species. Am J Infect Control 2010;38(5 Suppl 1):S25-33.
A. El-Saed et al. / American Journal of Infection Control 46 (2018) 930-5
3. Allegranzi B, Pittet D. Role of hand hygiene in healthcare-associated infection prevention. J Hosp Infect 2009;73:305-15. 4. Pittet D. Compliance with hand disinfection and its impact on hospital-acquired infections. J Hosp Infect 2001;48(Suppl A):S40-6. 5. Vernaz N, Sax H, Pittet D, Bonnabry P, Schrenzel J, Harbarth S. Temporal effects of antibiotic use and hand rub consumption on the incidence of MRSA and Clostridium difficile. J Antimicrob Chemother 2008;62:601-7. 6. Chun HK, Kim KM, Park HR. Effects of hand hygiene education and individual feedback on hand hygiene behaviour, MRSA acquisition rate and MRSA colonization pressure among intensive care unit nurses. Int J Nurs Pract 2015;21:709-15. 7. The Joint Commission: Measuring Hand Hygiene Adherence: Overcoming the Challenges. Available from https://www.jointcommission.org/assets/1/18/ hh_monograph.pdf; 2009. Accessed April 30, 2017. 8. Erasmus V, Daha TJ, Brug H, Richardus JH, Behrendt MD, Vos MC, et al. Systematic review of studies on compliance with hand hygiene guidelines in hospital care. Infect Control Hosp Epidemiol 2010;31:283-94. 9. Boyce JM. Hand hygiene compliance monitoring: current perspectives from the USA. J Hosp Infect 2008;70(Suppl 1):2-7. 10. Rosenthal VD, McCormick RD, Guzman S, Villamayor C, Orellano PW. Effect of education and performance feedback on handwashing: the benefit of administrative support in Argentinean hospitals. Am J Infect Control 2003;31:8592. 11. Jarrin Tejada C, Bearman G. Hand hygiene compliance monitoring: the state of the art. Curr Infect Dis Rep 2015;17:470. 12. Sax H, Allegranzi B, Chraiti MN, Boyce J, Larson E, Pittet D. The World Health Organization hand hygiene observation method. Am J Infect Control 2009;37:827-34. 13. Haessler S. The Hawthorne effect in measurements of hand hygiene compliance: a definite problem, but also an opportunity. BMJ Qual Saf 2014;23:965-7. 14. Srigley JA, Furness CD, Baker GR, Gardam M. Quantification of the Hawthorne effect in hand hygiene compliance monitoring using an electronic monitoring system: a retrospective cohort study. BMJ Qual Saf 2014;23:974-80. 15. Eckmanns T, Bessert J, Behnke M, Gastmeier P, Ruden H. Compliance with antiseptic hand rub use in intensive care units: the Hawthorne effect. Infect Control Hosp Epidemiol 2006;27:931-4. 16. Kohli E, Ptak J, Smith R, Taylor E, Talbot EA, Kirkland KB. Variability in the Hawthorne effect with regard to hand hygiene performance in high- and
17.
18.
19.
20.
21.
22.
23.
24.
25. 26.
27.
935
low-performing inpatient care units. Infect Control Hosp Epidemiol 2009;30:2225. Almaguer-Leyva M, Mendoza-Flores L, Padilla-Orozco M, Sanchez-Hinojosa R, Espinosa-Santacruz D, Camacho-Ortiz A. Integrating medical students as covert observers in the evaluation of hand hygiene compliance. Am J Infect Control 2014;42:937-9. van de Mortel T, Murgo M. An examination of covert observation and solution audit as tools to measure the success of hand hygiene interventions. Am J Infect Control 2006;34:95-9. Wu KS, Chen YS, Lin HS, Hsieh EL, Chen JK, Tsai HC, et al. A nationwide covert observation study using a novel method for hand hygiene compliance in health care. Am J Infect Control 2017;45:240-4. Pan SC, Tien KL, Hung IC, Lin YJ, Sheng WH, Wang MJ, et al. Compliance of health care workers with hand hygiene practices: independent advantages of overt and covert observers. PLoS ONE 2013;8:e53746. World Health Organization: World Health Organization. WHO guidelines for hand hygiene in health care. Geneva, Switzerland: World Health Organization. Available from http://whqlibdoc.who.int/publications/2009/9789241597906_eng.pdf? ua=1; 2009. Accessed April 30, 2017. Dhar S, Tansek R, Toftey EA, Dziekan BA, Chevalier TC, Bohlinger CG, et al. Observer bias in hand hygiene compliance reporting. Infect Control Hosp Epidemiol 2010;31:869-70. Maury E, Moussa N, Lakermi C, Barbut F, Offenstadt G. Compliance of health care workers to hand hygiene: awareness of being observed is important. Intensive Care Med 2006;32:2088-9. Hagel S, Reischke J, Kesselmeier M, Winning J, Gastmeier P, Brunkhorst FM, et al. Quantifying the Hawthorne effect in hand hygiene compliance through comparing direct observation with automated hand hygiene monitoring. Infect Control Hosp Epidemiol 2015;36:957-62. Viera AJ. Odds ratios and risk ratios: what’s the difference and why does it matter? South Med J 2008;101:730-4. Yin J, Reisinger HS, Vander Weg M, Schweizer ML, Jesson A, Morgan DJ, et al. Establishing evidence-based criteria for directly observed hand hygiene compliance monitoring programs: a prospective, multicenter cohort study. Infect Control Hosp Epidemiol 2014;35:1163-8. Gould DJ, Creedon S, Jeanes A, Drey NS, Chudleigh J, Moralejo D. Impact of observing hand hygiene in practice and research: a methodological reconsideration. J Hosp Infect 2017;95:169-74.
Contents lists available at ScienceDirect
American Journal of Infection Control
American Journal of Infection Control
j o u r n a l h o m e p a g e : w w w. a j i c j o u r n a l . o r g
Major Article
Quantifying the Hawthorne effect using overt and covert observation of hand hygiene at a tertiary care hospital in Saudi Arabia Aiman El-Saed MD, PhD a,b,c, Seema Noushad MD a, Elias Tannous CIC d, Fatima Abdirizak MPH e, Yaseen Arabi MD b,f,j, Salih Al Azzam MD b,g, Esam Albanyan MD b,h, Hamdan Al Jahdalil MD b,h, Reem Al Sudairy MD i, Hanan H. Balkhy MD a,b,j,* a
Infection Prevention and Control Department, King Abdulaziz Medical City, Ministry of National Guard Health Affairs, Riyadh, Saudi Arabia King Saud bin Abdulaziz University for Health Sciences, Riyadh, Saudi Arabia Community Medicine Department, Faculty of Medicine, Mansoura University, Mansoura, Egypt d Quality and Patient Safety Institute, Cleveland Clinic, Abu Dhabi, United Arab Emirates e School of Public Health, Georgia State University, Atlanta, GA f Intensive Care Department, King Abdulaziz Medical City, Ministry of National Guard Health Affairs, Riyadh, Saudi Arabia g Department of Surgery, King Abdulaziz Medical City, Ministry of National Guard Health Affairs, Riyadh, Saudi Arabia h Department of Medicine, King Abdulaziz Medical City, Ministry of National Guard Health Affairs, Riyadh, Saudi Arabia i Department of Oncology, King Abdulaziz Medical City, Ministry of National Guard Health Affairs, Riyadh, Saudi Arabia j King Abdullah International Medical Research Center, Riyadh, Saudi Arabia b c
Key Words: Hospital Health care workers Hand hygiene compliance
Introduction: Although direct human observation of hand hygiene (HH) is considered the gold standard for measuring HH compliance, its accuracy is challenged by the Hawthorne effect. Objectives: To compare HH compliance using both overt and covert methods of direct observation in different professional categories, hospital settings, and HH indications. Methods: A cross-sectional study was conducted in 28 units at King Abdulaziz Medical City, Riyadh, Saudi Arabia, between October 2012 and July 2013. Compliance was defined as performing handrubbing or handwashing during 1 of the World Health Organization 5 Moments for HH indications (ie, opportunities). Overt observation was done by infection preventionists (IPs) who were doing their routine HH observation. Covert observation was done by unrecognized temporarily hired professionally trained observers. Results: A total of 15,883 opportunities were observed using overt observation and 7,040 opportunities were observed using covert observation. Overall HH compliance was 87.1% versus 44.9% using overt/ covert observations, respectively (risk ratio, 1.94; P < .001). The significant overestimation was seen across all professional categories, hospital settings, and HH indications. Conclusion: There is a considerable difference in HH compliance being observed overtly and covertly in all categories. More work is required to improve the methodology of direct observation to minimize the influence of the Hawthorne effect. © 2018 Association for Professionals in Infection Control and Epidemiology, Inc. Published by Elsevier Inc. All rights reserved.
Approximately 7.5% of patients admitted to hospitals in developed countries and 10% in developing countries end up with health care-associated infections.1 Between 20% and 40% of these infections may be directly attributed to transmission from contaminated hands of health care workers (HCWs).2 Therefore, hand hygiene (HH)
* Address correspondence to Hanan H. Balkhy, MD, Pediatric Infectious Disease, King Saud bin Abdulaziz University for Health Sciences, Infectious Diseases, King Abdullah International Medical Research Center, and Infection Prevention and Control, King Abdulaziz Medical City, PO Box 22490, Riyadh 11426, Saudi Arabia. E-mail address: [email protected] (H.H. Balkhy). Conflicts of interest: None to report.
is considered the single most important strategy to reduce the incidence of health care-associated infections.3,4 Additionally, it is a core element for preventing the spread of antimicrobial resistance and reducing colonization of multiresistant microorganisms.5,6 Therefore, it is among the critical indicators for patient safety required by hospitals to be granted accreditation.7 Although the benefits of HH are well known and noncontroversial, the HH compliance in health care settings is still suboptimal, with an average compliance rate of 40%.8 Monitoring HH compliance serves multiple functions: it stimulates HCWs to improve their performance, helps to improve infrastructure design, and works as objective assessment of the
0196-6553/© 2018 Association for Professionals in Infection Control and Epidemiology, Inc. Published by Elsevier Inc. All rights reserved. https://doi.org/10.1016/j.ajic.2018.02.025
A. El-Saed et al. / American Journal of Infection Control 46 (2018) 930-5
quality of care.9,10 Our institution has been conducting HH observations for more than 10 years. We were concerned with the accuracy of the reported high compliance. Although direct human observation of HH practices is considered the gold standard for measuring HH compliance,9,11 its accuracy is challenged by interobserver variability and the Hawthorne effect.9,12 The latter, which is defined as a change in HCW behavior because of the awareness of being observed, is known to overestimate compliance rates.13-15 The influence of Hawthorne effect may be variable in high- or low-compliant hospital locations.16 Covert observation was suggested as a tool to quantify or overcome Hawthorne effect bias.17-19 Few studies tried to quantify the amount of overestimation of HH compliance using the overt or covert methods in different hospital settings, indications, and professional categories.15,20 However, these studies either did not use the standard World Health Organization (WHO) 5 Moment methodology or observed an insufficient number of HH opportunities for appropriate stratification of data. Additionally, such data are absolutely lacking in Saudi Arabia. The objective of the current study was to compare HH compliance using overt or covert methods of direct observation in different professional categories, hospital settings, and HH indications. METHODS Setting The current study was conducted at King Abdulaziz Medical CityRiyadh (KAMC-R), Ministry of National Guard Health Affairs in Saudi Arabia. KAMC-R is a 1,000-bed tertiary care facility that is funded by the government. It provides health care services for about 750,000 Saudi National Guard soldiers, employees, and their families. The care provided ranges from primary and preventive care to tertiary care. At the time of the study, approximately 9,170 HCWs were working for KAMC-R in jobs that involved direct patient care, including approximately 1,670 physicians, 4,660 nurses, and 2,840 other HCWs. KAMC-R has a multisection emergency department (150 beds), 13 different intensive care units (ICUs) (total of 185 beds), and 36 wards covering all other specialties. At the time of the study, the emergency department was serving more than 250,000 visits a year, ICUs were serving 5,000 admissions a year, and wards were serving 27,000. The data about KAMC-R–served populations and HCWs were obtained from the annual census reports for 2013 and 2014. Population The study targeted clinical HCWs (who were directly involved in patient care) in different departments of KAMC-R, including physicians, nurses, and other HCWs. The latter included therapists, technicians, laboratory personnel, emergency medical service personnel, dental personnel, and pharmacists. Nonclinical HCWs not directly involved in patient care such as clerical, dietary, laundry, security, maintenance, and administrative jobs were not included.
931
temporarily hired professionally trained observers during the same period. Study outcome HH compliance was defined as doing either handrubbing (with alcohol-based formulation) or handwashing (with soap and water) during 1 of the WHO 5 Moment HH indications (ie, opportunities): before patient contact, before an aseptic task, after exposure to body fluids, after patient contact, and after contact with patient surroundings.21 More than 1 indication falling into the same opportunity was allowed. Data collection For the overt observation, a standard WHO HH observation form (with space for 32 opportunities) was used in collecting HH compliance per WHO methods.21 The data of more than 1 HCW of different specialty were collected on the same form. Data collection was performed by IPs who were routinely collecting HH data for at least 6 months for the infection control department. For the covert observation, WHO HH observation forms (slightly modified to allow the data of only 1 HCW to be collected on the same form) were used in collecting HH compliance per WHO methods. Observation of the same HCW was continued until the session ended or 20 HH opportunities were observed. Data collection was performed by 2 temporarily hired observers who passed a training test on official WHO HH videos containing standard scenarios as well as real hospital settings. Additionally, validation of HH observation was done by occasional concomitant observation by a senior IP, with almost identical findings (κ > 0.9). The temporarily hired observers doing covert observation wore nursing uniforms but their monitoring task was not revealed to the observed HCWs. In both observation methods, HH observation was done in a series of sessions no more than 30 minutes (mean, 20 ± 10 minutes) each. Data collection from the same unit was continued until at least 200 HH opportunities were observed. Additionally, HH observation was done quietly without attempts to promote HH compliance or provide performance feedback to HCWs. Statistical methods Categorical variables are presented as frequencies and percentages, whereas continuous variables are presented as means ± standard deviation. The χ2 or Fisher exact test, as appropriate, was used to test significant differences in HH compliance between overt and covert observation. Additionally, risk ratio (RR) and odds ratio (OR) for HH compliance using overt compared with covert observations were calculated using standard ways of calculations. Student t test or Mann-Whitney U test, as appropriate, was used to test significant differences in continuous variables between the overt and covert observations. All P values were 2-tailed. A P value < .05 was considered significant. SPSS version 23.0 (IBM-SPSS Inc, Armonk, NY) was used for all statistical analyses. RESULTS
Study design A cross-sectional study design was conducted at KAMC-R between October 2012 and July 2013. The study received all required ethical approvals from King Abdullah International Medical Research Center, Riyadh, Saudi Arabia, before data collection. Overt observation was done by presumably well-recognized infection preventionists (IPs) who were doing their routine HH observation during the study period. Covert observation was done by presumably unrecognized
As shown in Table 1, a total of 15,883 opportunities were observed during 725 observation sessions done using overt observation across 270.9 hours of observations. On the other hand, a total of 7,040 opportunities were observed during 298 sessions done using covert observation across 148.8 hours of observations. For the 2 observation methods, the majority of the observed HCWs were nurses (62.2% and 58.7%). With slight differences between the 2 observation methods, the majority (48.3%) of all opportunities were observed
932
A. El-Saed et al. / American Journal of Infection Control 46 (2018) 930-5
Table 1 Comparisons of the 2 observation methods Overt observation Observation sessions Overall number of sessions Number of sessions by hospital location Emergency department Intensive care units Wards Average duration per session (min) Total duration of sessions (h) Hand hygiene opportunities Overall number of opportunities Number of opportunities by professional category Nurses Physicians Other health care workers Number of opportunities by hospital location Emergency department Intensive care units Wards Hand hygiene indications Overall number of indications Type Before patient contact Before aseptic tasks After exposure to body fluids After patient contact After contact with surroundings Observed health care workers Nurses Physicians Other health care workers Total Observing staff
Covert observation
Total
725 (70.9)
298 (29.1)
150 (20.7) 236 (32.6) 339 (46.8) 22.4 ± 9.4 270.9
69 (23.2) 74 (24.8) 155 (52.0) 29.9 ± 7.7 148.8
219 (21.4) 310 (30.3) 494 (48.3) 24.7 ± 9.6 421.6
15,883 (69.3)
7,040 (30.7)
22,923 (100.0)
9,877 (62.2) 3,994 (25.1) 2,012 (12.7)
4,135 (58.7) 1,358 (19.3) 1,547 (22.0)
14,012 (61.1) 5,352 (23.3) 3,559 (15.5)
4,304 (27.1) 5,061 (31.9) 6,518 (41.0)
1,728 (24.5) 1,933 (27.5) 3,379 (48.0)
6,032 (26.3) 6,994 (30.5) 9,897 (43.2)
15,918 (69.3)
7,069 (30.7)
22,987 (100.0)
5,511 (34.6) 360 (2.3) 299 (1.9) 3,605 (22.6) 6,142 (38.6)
1,899 (26.9) 727 (10.3) 427 (6.0) 1,672 (23.7) 2,343 (33.1)
7,410 (32.2) 1,087 (4.7) 726 (3.2) 5,277 (23.0) 8,485 (36.9)
NA NA NA NA 11
289 (53.7) 119 (22.1) 130 (24.2) 538 (100.0) 2
1,023 (100.0)
NA NA NA NA 13
Values are presented as n (%) or mean ± standard deviation. NA, not available.
Fig 1. Hand hygiene action by observation method. Fig 2. Wearing gloves status (%) in noncompliant opportunities by observation method.
in wards, followed by ICUs (30.3%) and the emergency department (21.4%). With slight differences between the 2 observation methods that does not influence the order, overall, the most common HH indication was “after contact with the patient surroundings” (36.9%), followed by “before patient contact” (32.2%), “after patient contact” (23.0%), “before aseptic tasks” (4.7%), and lastly “after exposure to body fluids” (3.2%). As shown in Figure 1, there was a large and highly significant difference in the HH compliance between the 2 observation methods, being much higher in overt compared with covert observations (87.1% vs 44.9%; P < .001). As shown in Table 1, the RR for the overt compared with the covert observations was 1.94, with even much higher
OR (8.26). The observed difference was mainly due to difference in alcohol-based handrubbing, which represented 92.8% of appropriate actions taken. As shown in Figure 2, wearing gloves with lack of compliance (N = 807 opportunities) was lower in overt compared with covert observations (10.1% vs 15.4%; P < .001). As shown in Table 2, HH compliance was significantly higher in overt compared with covert observations after stratification by professional categories and hospital locations (P < .001 for all). For example, HH compliance in nurses was 89.8% using overt observation compared with 47.0% using covert observation (RR, 1.91; P < .001). Similarly, HH compliance in ICUs was 91.2% using overt
A. El-Saed et al. / American Journal of Infection Control 46 (2018) 930-5
933
Table 2 Hand hygiene compliance by observation method, hospital location, and professional category Overt observation (opportunities) Observed (n)
Compliant (n)
Compliance (%)
15,883
13,828
9,877 3,994 2,012 346 15,537 4,304 5,061 6,518 1,194 1,655 1k395 2k274
Overall Professional category Nurses Physicians Other health care workers Respiratory therapists Nonrespiratory therapists Hospital location Emergency department Intensive care units Ward Medical Surgical Obstetrics Pediatrics
Covert observation (opportunities) P value1
Observed (n)
Compliant (n)
Compliance (%)
P value2
Risk ratio
Odds ratio
87.1
–
7,040
3,161
44.9
–
1.94
8.26
< .001
8,870 3,212 1,746 306 13,522
89.8 80.4 86.8 88.4 87.0
< .001
4,135 1,358 1,547 600 6,440
1,942 567 652 275 2,886
47.0 41.8 42.1 45.8 44.8
< .001
1.91 1.92 2.06 1.93 1.94
9.95 5.73 9.01 9.04 8.26
< .001 < .001 < .001 < .001 < .001
3,610 4,616 5,602 1,078 1,423 1,073 2,028
83.9 91.2 85.9 90.3 86.0 76.9 89.2
< .001
1,728 1,933 3,379 542 873 627 1,337
725 827 1,609 293 446 304 566
42.0 42.8 47.6 54.1 51.1 48.5 42.3
< .001
2.00 2.13 1.80 1.67 1.68 1.59 2.11
7.20 13.87 6.73 7.90 5.87 3.54 11.23
< .001 < .001 < .001 < .001 < .001 < .001 < .001
.440
< .001
.631
< .001
P value3
Differences between groups were evaluated in overt observation using P value1 and in covert observation using P value2. Differences between overt and covert observations were evaluated using P value3. All P values were derived from χ2 or Fisher exact test, as appropriate.
Table 3 Hand hygiene compliance by observation method, professional category, and hand hygiene indication Overt observation (opportunities)
Overall Before patient contact Before aseptic tasks After exposure to body fluids After patient contact After contact with surroundings Nurses Before patient contact Before aseptic tasks After exposure to body fluids After patient contact After contact with surroundings Physicians Before patient contact Before aseptic tasks After exposure to body fluids After patient contact After contact with surroundings Other health care workers Before patient contact Before aseptic tasks After exposure to body fluids After patient contact After contact with surroundings
Observed (n)
Compliant (n)
Compliance (n)
5,511 360 299 3,605 6,142
4,934 340 291 3,227 5,068
89.5 94.4 97.3 89.5 82.5
3,377 285 239 2,254 3,749
3,129 273 232 2,079 3,182
1,379 43 41 899 1,635
1,155 39 41 757 1,223
755 32 19 452 758
650 28 18 391 663
Covert observation (opportunities) Risk ratio
Odds ratio
P value3
< .001
1.75 3.01 2.26 2.35 1.67
8.22 37.21 48.04 13.87 4.81
< .001 < .001 < .001 < .001 < .001
54.1 35.5 50.0 40.0 50.3
< .001
1.71 2.70 1.94 2.31 1.69
10.68 41.27 33.14 17.84 5.55
< .001 < .001 < .001 < .001 < .001
216 10 10 122 210
48.6 16.4 26.3 35.6 44.4
< .001
1.72 5.53 3.80 2.37 1.68
5.44 49.73 9.66 3.72
< .001 < .001 < .001 < .001 < .001
184 35 19 136 280
45.3 23.2 24.1 35.7 52.1
< .001
1.90 3.77 3.93 2.42 1.68
7.47 23.20 56.84 11.55 6.41
< .001 < .001 < .001 < .001 < .001
Observed (n)
Compliant (n)
Compliance (%)
< .001
1,899 727 427 1,672 2,343
968 228 184 637 1,160
51.0 31.4 43.1 38.1 49.5
92.7 95.8 97.1 92.2 84.9
< .001
1,049 515 310 948 1,333
568 183 155 379 670
83.8 90.7 100.0 84.2 74.8
< .001
444 61 38 343 473 406 151 79 381 537
P value1
86.1 87.5% 94.7% 86.5% 87.5%
.788
P value2
Differences between different indications were evaluated in overt observation using P value1 and in covert observation using P value2. Differences between overt and covert observations were evaluated using P value3. All P values were derived from χ2 or Fisher exact test, as appropriate.
observation compared with 42.8% using covert observation (RR, 2.13; P < .001). The difference between the 2 observation methods was more variable in hospital locations than in professional categories. For example, the RRs ranged between 1.59 in obstetric wards and 2.13 in ICUs. On the other hand, the RRs ranged between 1.91 in nurses and 2.06 in other HCWs. In both observation methods, nurses had the highest compliance, followed by other HCWs and then physicians (P < .001 for each). In both observation methods, the emergency department had the lowest compliance (P < .001 for each). On the other hand, ICUs had the highest compliance in overt observation (91.2% vs 85.1%; P < .001), whereas wards had the highest compliance in covert observation (47.6% vs 42.4%; P < .001). As shown in Table 3, HH compliance was significantly higher in overt compared with covert observations after stratification by
HH indications (either overall or professional category-specific; P < .001 for all). For example, HH compliance for “before patient contact” was 89.5% using overt observation compared with 51.0% using covert observation (RR, 1.75; P < .001). The difference between the 2 observation methods was highest with “before aseptic tasks” indication (RR, 3.01) and lowest with “after contact with surroundings” indication (RR, 1.67). Additionally, within each observation method there was great variability with regard to compliance by HH indications. For example, the compliance was highest for “after exposure to body fluids” and lowest for “after contact with surroundings” using overt observation. On the other hand, the compliance was highest for “before patient contact” and lowest for “before aseptic tasks” using covert observation. Largely similar findings were observed after stratification of HH indications by professional category.
934
A. El-Saed et al. / American Journal of Infection Control 46 (2018) 930-5
DISCUSSION The key finding of the current study was the huge difference in HH compliance using 2 different observation methods. We are reporting an overall 94% overestimation of HH compliance rates using overt compared with covert observations, which translates into approximately 42% net difference between both methods. In previous studies, the overt method has been consistently associated with overestimation of HH compliance, which is commonly attributed to the Hawthorne effect.15-17,20,22,23 However, the degree of overestimation in these studies was considerably variable, with a range of 12% to 124% relative overestimation and 7% to 50% net difference between both methods.15-17,20,22,23 The overestimation was even higher in studies that used electronic monitoring to quantify the Hawthorne effect.14,24 This variability may be related to the baseline HH compliance16 and the different and sometimes poor methods used to detect the overestimation in those studies. For example, the majority of the studies that sought to quantify the overestimation using direct observation did not use the standard WHO 5 Moments methodology, which make fair comparisons difficult.15,17,22 Additionally, some of these studies were clearly underpowered20 or even failed to have a true control group.18,19 Although the overestimation of HH compliance in our study was clearly seen in all examined groups, the degree of overestimation tends to be more variable in hospital locations and HH indications. The observed variability probably reflected the variability in overt compliance in different strata of these groups. For example, when we analyzed the data after stratifying the overt compliance as below and above the average (ie, 87.1%), the degree of overestimation was much higher among hospital locations and HH indications that had overt compliance above the study average (data not shown). This was very true with ICUs and “before aseptic tasks” indication, which had >90% overt compliance and >100% overestimation compared with the obstetrics ward and “after contact with surroundings” indication, which had <83% overt compliance and <70% overestimation. Similarly, it has been suggested that the observation of staff compliance has little influence in low-compliant hospital locations but a notable influence in high-compliant hospital locations.16 Although the same findings have not been adequately examined in other studies, reviewing the reported location-based overestimation in these studies showed conflicting findings.15,17,22 The higher overestimation with “before aseptic tasks” HH indication in both nurses and physicians may point to the effect of socially desirable behavior during overt observation. Additionally, when the overestimation was stratified by before-and-after indications (data not shown), the overestimation was significantly higher in before than in after indications (97% vs 92%; P < .001). This may be explained by the perception that patient protection (primarily before indications) is probably more socially desirable behavior than HCW protection (primarily after indications). Our study used unrecognized, temporarily hired professionally trained observers to estimate the real HH compliance during the same period recognized IPs were doing their routine HH observation. Previous studies used different methods to detect the real (covert) HH compliance. For example, several studies used medical students as unrecognized observers.16,17,19,20 Although students are readily available and usually unrecognized by unit staff, they frequently lack knowledge of several invasive procedures and are commonly restricted from some areas.17 A number of studies used the same observers in 2 different periods, 1 with announcement of HH monitoring and the other without announcement.15,23 Obviously, this method ignores the possibility of real differences between periods, requires that the covert period to be before the overt period, and cannot be repeated without some breach of the concealment because the observer is already known. Finally,
electronic monitoring systems have been used to quantify the Hawthorne effect.14,24 However, the inherent limitations of this method can further inflate the Hawthorne effect, with up to 3-fold overestimation.14 Previous studies described the overestimation of HH compliance using overt or covert methods in absolute terms and a few of them additionally calculated the OR of the difference. 15,22 Although we determined the relative overestimation using the risk ratios and ORs, we believe that the OR is not the optimal metric to quantify the overestimation because it tends to further augment the overestimation, a phenomenon that is known to naturally happen when the characteristic under study has a prevalence >10%.25 Therefore, we relied on the risk ratio in our reference to the degree of overestimation throughout this article. Although the current study is reconfirming a huge difference in HH compliance using 2 different methods of direct observation, we still believe that direct HH observation is the gold standard for measuring compliance.9,11 Direct observation is more likely to better assess the timing, indication-specific compliance, and technique of HH action than indirect or automated observation.9 Additionally, it is the monitoring method recognized by WHO and implemented in different health care settings around the world.12,21 Therefore, future work should focus on how to improve the methodology of direct observation to minimize the influence of the Hawthorne effect rather than using indirect or automated observation. It has been suggested that limiting duration of the observation session to approximately 15 minutes can minimize the Hawthorne effect.26 Additionally, frequent and unobtrusive observation may acclimatize staff to the presence of observers.27 Finally, training non-IPs to perform auditing observation in units other than their own and frequently rotating IPs to different hospital areas may help minimize the Hawthorne effect in large hospitals. Although the current study is unique in quantifying the Hawthorne effect in different professional categories, hospital settings, and HH indications using the standard WHO 5 Moments methodology in a large number of observations, we acknowledge a number of limitations. Being a single-center study, the findings of our study should be generalized with caution. Although we used unrecognized, temporarily hired observers to estimate HH compliance covertly, we cannot exclude some breaches of concealment, especially with longer sessions and recruiting multiple HCWs from the same unit. However, with the current methods the influence of this (if any) is believed to be minimal and the huge overestimation reported is by itself a sign of good covert methodology. CONCLUSIONS We are reporting an overall 94% overestimation of HH compliance rates using overt compared with covert observations. Although the overestimation was clearly seen in all examined groups, the degree of overestimation tended to be more variable in hospital locations and HH indications. More work is needed to improve the methodology of direct observation to minimize the influence of the Hawthorne effect. The effectiveness of any suggested methodology change needs to be evaluated in future studies. References 1. World Health Organization. Report on the Burden of Endemic Health CareAssociated Infection Worldwide. A systematic review of the literature. Available from http://apps.who.int/iris/bitstream/handle/10665/80135/978924150 1507_eng.pdf?sequence=1&isAllowed=y; 2011. Accessed February 1, 2015. 2. Weber DJ, Rutala WA, Miller MB, Huslage K, Sickbert-Bennett E. Role of hospital surfaces in the transmission of emerging health care-associated pathogens: norovirus, Clostridium difficile, and Acinetobacter species. Am J Infect Control 2010;38(5 Suppl 1):S25-33.
A. El-Saed et al. / American Journal of Infection Control 46 (2018) 930-5
3. Allegranzi B, Pittet D. Role of hand hygiene in healthcare-associated infection prevention. J Hosp Infect 2009;73:305-15. 4. Pittet D. Compliance with hand disinfection and its impact on hospital-acquired infections. J Hosp Infect 2001;48(Suppl A):S40-6. 5. Vernaz N, Sax H, Pittet D, Bonnabry P, Schrenzel J, Harbarth S. Temporal effects of antibiotic use and hand rub consumption on the incidence of MRSA and Clostridium difficile. J Antimicrob Chemother 2008;62:601-7. 6. Chun HK, Kim KM, Park HR. Effects of hand hygiene education and individual feedback on hand hygiene behaviour, MRSA acquisition rate and MRSA colonization pressure among intensive care unit nurses. Int J Nurs Pract 2015;21:709-15. 7. The Joint Commission: Measuring Hand Hygiene Adherence: Overcoming the Challenges. Available from https://www.jointcommission.org/assets/1/18/ hh_monograph.pdf; 2009. Accessed April 30, 2017. 8. Erasmus V, Daha TJ, Brug H, Richardus JH, Behrendt MD, Vos MC, et al. Systematic review of studies on compliance with hand hygiene guidelines in hospital care. Infect Control Hosp Epidemiol 2010;31:283-94. 9. Boyce JM. Hand hygiene compliance monitoring: current perspectives from the USA. J Hosp Infect 2008;70(Suppl 1):2-7. 10. Rosenthal VD, McCormick RD, Guzman S, Villamayor C, Orellano PW. Effect of education and performance feedback on handwashing: the benefit of administrative support in Argentinean hospitals. Am J Infect Control 2003;31:8592. 11. Jarrin Tejada C, Bearman G. Hand hygiene compliance monitoring: the state of the art. Curr Infect Dis Rep 2015;17:470. 12. Sax H, Allegranzi B, Chraiti MN, Boyce J, Larson E, Pittet D. The World Health Organization hand hygiene observation method. Am J Infect Control 2009;37:827-34. 13. Haessler S. The Hawthorne effect in measurements of hand hygiene compliance: a definite problem, but also an opportunity. BMJ Qual Saf 2014;23:965-7. 14. Srigley JA, Furness CD, Baker GR, Gardam M. Quantification of the Hawthorne effect in hand hygiene compliance monitoring using an electronic monitoring system: a retrospective cohort study. BMJ Qual Saf 2014;23:974-80. 15. Eckmanns T, Bessert J, Behnke M, Gastmeier P, Ruden H. Compliance with antiseptic hand rub use in intensive care units: the Hawthorne effect. Infect Control Hosp Epidemiol 2006;27:931-4. 16. Kohli E, Ptak J, Smith R, Taylor E, Talbot EA, Kirkland KB. Variability in the Hawthorne effect with regard to hand hygiene performance in high- and
17.
18.
19.
20.
21.
22.
23.
24.
25. 26.
27.
935
low-performing inpatient care units. Infect Control Hosp Epidemiol 2009;30:2225. Almaguer-Leyva M, Mendoza-Flores L, Padilla-Orozco M, Sanchez-Hinojosa R, Espinosa-Santacruz D, Camacho-Ortiz A. Integrating medical students as covert observers in the evaluation of hand hygiene compliance. Am J Infect Control 2014;42:937-9. van de Mortel T, Murgo M. An examination of covert observation and solution audit as tools to measure the success of hand hygiene interventions. Am J Infect Control 2006;34:95-9. Wu KS, Chen YS, Lin HS, Hsieh EL, Chen JK, Tsai HC, et al. A nationwide covert observation study using a novel method for hand hygiene compliance in health care. Am J Infect Control 2017;45:240-4. Pan SC, Tien KL, Hung IC, Lin YJ, Sheng WH, Wang MJ, et al. Compliance of health care workers with hand hygiene practices: independent advantages of overt and covert observers. PLoS ONE 2013;8:e53746. World Health Organization: World Health Organization. WHO guidelines for hand hygiene in health care. Geneva, Switzerland: World Health Organization. Available from http://whqlibdoc.who.int/publications/2009/9789241597906_eng.pdf? ua=1; 2009. Accessed April 30, 2017. Dhar S, Tansek R, Toftey EA, Dziekan BA, Chevalier TC, Bohlinger CG, et al. Observer bias in hand hygiene compliance reporting. Infect Control Hosp Epidemiol 2010;31:869-70. Maury E, Moussa N, Lakermi C, Barbut F, Offenstadt G. Compliance of health care workers to hand hygiene: awareness of being observed is important. Intensive Care Med 2006;32:2088-9. Hagel S, Reischke J, Kesselmeier M, Winning J, Gastmeier P, Brunkhorst FM, et al. Quantifying the Hawthorne effect in hand hygiene compliance through comparing direct observation with automated hand hygiene monitoring. Infect Control Hosp Epidemiol 2015;36:957-62. Viera AJ. Odds ratios and risk ratios: what’s the difference and why does it matter? South Med J 2008;101:730-4. Yin J, Reisinger HS, Vander Weg M, Schweizer ML, Jesson A, Morgan DJ, et al. Establishing evidence-based criteria for directly observed hand hygiene compliance monitoring programs: a prospective, multicenter cohort study. Infect Control Hosp Epidemiol 2014;35:1163-8. Gould DJ, Creedon S, Jeanes A, Drey NS, Chudleigh J, Moralejo D. Impact of observing hand hygiene in practice and research: a methodological reconsideration. J Hosp Infect 2017;95:169-74.