- Email: [email protected]
Surgical Safety Checklist Compliance: A Job Done Poorly!
Surgical Safety Checklist Compliance: A Job Done Poorly! Eric A Sparks, MD, Hania Wehbe-Janek, Harry T Papaconstantinou, MD, FACS
PhD,
Rebecca L Johnson,
RN,
W Roy Smythe,
MD, FACS,
The Surgical Safety Checklist (SSC) has been introduced as an effective tool for reducing perioperative mortality and complications. Although reported completion rates are high, objective compliance is not well defined. The purpose of this retrospective analysis is to determine SSC compliance as measured by accuracy and completion, and factors that can affect compliance. STUDY DESIGN: In September 2010, our institution implemented an adaptation of the World Health Organization’s SSC in an effort to improve patient safety and outcomes. A tool was developed for objective evaluation of overall compliance (maximum score 40) that was an aggregate score of completion and accuracy (20 each). Random samples of SSCs were analyzed at specific, predefined, time points throughout the first year after implementation. Procedure start time, operative time, and case complexity were assessed to determine association with compliance. RESULTS: A total of 671 SSCs were analyzed. The participation rate improved from 33% (95 of 285) at week 1 to 94% (249 of 265) at 1 year (p < 0.0001, chi-square test). Mean overall compliance score was 27.7 (5.4 SD) of 40 possible points (69.3% 13.5% of total possible score; n ¼ 671) and did not change over time. Although completion scores were high (16.9 2.7 out of 20 [84.5% 13.6%]), accuracy was poor (10.8 3.4 out of 20 [54.1% 16.9%]). Overall compliance score was significantly associated with case start-time (p < 0.05), and operative time and case complexity showed no association. CONCLUSIONS: Our data indicate that although implementation of an SSC results in a high level of overall participation and completion, accuracy remained poor. Identification of barriers to effective use is needed, as improper checklist use can adversely affect patient safety. (J Am Coll Surg 2013;217:867e873. 2013 by the American College of Surgeons) BACKGROUND:
which has been widely successful in substantially reducing rates of perioperative complications and mortality.3 The celebrated success of this patient-safety tool and its reported ease of implementation have led to rapid adoption of SSCs worldwide.3-7 Although the purpose of checklists and other patient-safety tools is to reduce patient harm and improve outcomes, it has been assumed that these instruments are only effective if used as intended.8,9 Appropriate checklist use and sustainable practice can be a result of effective implementation.9,10 In addition, barriers to implementation are multifactorial and encompass cultural and structural levels, including communication gaps, perceived benefit by the provider, lack of procedural understanding, and perceived ambiguity of the process.11 Although many studies report successful use of SSCs, few have reported on the quality of compliance and accuracy of patient information reviewed during the process.11-13 In September 2010, our hospital started using an SSC in an effort to improve patient safety and outcomes. As
More than a decade has passed since the publication of To Err is Human,1 and this time has brought considerable interest in research and interventions to describe and prevent health careassociated injuries. The WHO launched its “Safe Surgery Saves Lives” campaign in 2008 as a means to prevent unnecessary death and improve outcomes for surgical patients.2 This program resulted in the WHO Surgical Safety Checklist (SSC),
Disclosure Information: Nothing to disclose. Abstract presented at the American College of Surgeons 98th Annual Clinical Congress, Surgical Forum, Chicago, IL, 2012. Received February 19, 2013; Revised June 29, 2013; Accepted July 9, 2013. From the Department of Surgery (Sparks, Johnson, Smythe, Papaconstantinou) and Office of Academic Research Development (Wehbe-Janek), Scott & White Memorial Hospital and Clinic, Texas A&M University System Health Science Center, College of Medicine, Temple, TX. Correspondence address: Harry T Papaconstantinou, MD, FACS, Department of Surgery, Scott & White Memorial Hospital and Clinic, 2401 South 31st St, Temple, TX 76508. email: [email protected]
ª 2013 by the American College of Surgeons Published by Elsevier Inc.
867
ISSN 1072-7515/13/$36.00 http://dx.doi.org/10.1016/j.jamcollsurg.2013.07.393
868
Sparks et al
J Am Coll Surg
Surgical Safety Checklist Compliance
Figure 1. Implementation timeline of the Scott & White Memorial Hospital Surgical Safety Checklist. The details and sequence of events of this study are described completely in the Methods section.
shown in Figure 1, the implementation process included checklist design, focused trial, and educational campaign. Post-implementation process included ongoing monitoring, program reinforcement, and outcomes review. Participation and compliance was identified as an important part of outcomes review as our quality-improvement group assumed that proper checklist use is a requirement to achieve desired safety improvements. Therefore, the purpose of this study was to objectively evaluate the quality of SSC compliance as measured by accuracy and completion and determine factors that can affect compliance.
METHODS Setting Scott & White Memorial Hospital (SWMH) is a 636bed tertiary care center that performed >18,000 operations in our main hospital operating room facility in fiscal year 2011. Our physicians operate as a single multispecialty group practice. At the time of this study, adult and pediatric operative services and operating rooms were contained within SWMH only. The operating rooms all follow the same perioperative practices and use the same personnel. This retrospective study is an observational study for quality improvement in our surgical safety process as it relates to SSCs. Scott & White Memorial Hospital Surgical Safety Checklist Our WHO-adapted SSC was developed using a multidisciplinary team approach with the goal of improving patient safety and surgical outcomes by enhancing communication and standardizing practice within the operating room. The final checklist was composed of 4 sections that parallel perioperative patient care (Appendix 1, online only) and includes check in, sign in, time out, and sign out. The check-in section occurs when the patient is in the preoperative holding unit, and focuses on communication among the surgical team to identify case-specific concerns and needs. The sign-in section occurs on the patient’s arrival to the operating room and highlights specific safety measures and
operative preparation. The time-out section occurs just before incision and includes all elements of the Joint Commissionendorsed time-out process,14 and provides the opportunity for the surgical team to communicate specific equipment and patient concerns. The sign-out section occurs at the end of each procedure and focuses on appropriate documentation of the procedure, specimen, and key postoperative concerns that can be helpful for recovery and management. This section also requires name, signature, date, and time from each member of the surgical team and the checklist is incorporated into the medical record. Focused trials of our SSC were performed by 3 surgeon champions in an effort to make meaningful iterative changes before widespread use. A 3-month multi-medium SWMH-focused SSC education campaign was initiated and championed by departmental and hospital leaders in surgical service areas that focused on why the SSC was important and how the SSC was to be used. This campaign included educational forums, posters, mandatory continuing medical education credit online training modules, and focused frequently asked question email blasts to all surgical providers. On September 1, 2010, SWMH started using the SSC throughout all of its operating rooms. After implementation, there was continued surgical team engagement with structured monitoring and reinforcement. At 1 year, we began a retrospective review of outcomes, which is partially presented in this study. Figure 1 shows a timeline demonstrating the design and implementation process of our SSC. Study design During the 1-year period after SSC implementation, all checklists were collected and saved. Checklist participation was defined by an SSC filled out, regardless of degree of completion or accuracy, for a given case. Surgical Safety Checklist participation was determined for an entire week at week 1 and year 1 post-SSC implementation. A tool was developed for objective evaluation of overall compliance with a maximum compliance score of 40 (Appendix 2, online only). Compliance was an
(249 of 265 cases) at 1 year (p < 0.01). Next, we reviewed a random sample of charts with representation from all surgical specialties during 1 year. A total of 671 SSCs were evaluated during the study period for completion (20 points), accuracy (20 points), and compliance (40 points). All SSCs in our study had an average of 27.7 of 40 points for compliance (69%), and there was no significant change in compliance scores during the study period (Fig. 2). When compliance scores were broken into completion and accuracy components, there was a large disparity between these measures. The average checklist achieved 16.9 of 20 points (85%) for completion vs only 10.8 of 20 points (54%) for accuracy (p < 0.05) (Fig. 3). Next, we determined the number of SSCs that achieved perfect scores for completion, accuracy, and compliance. Of 671 checklists analyzed, 105 (15.6%) received all 20 points for completion and only 1 (0.1%) checklist received all 20 points for accuracy (Table 1). There were no checklists (0%) that received a perfect score, or 40 points, for compliance. Next, individual SSC sections were evaluated to determine barrier areas for our SSC. Table 1 shows that that the time-out section had the highest mean compliance percentage score at 77.8%, and was significantly higher than the check-in (67.5%), sign-in (75.2%), and signout (63.6%) sections (p < 0.05, repeated-measures ANOVA with Tukey post-test). Additionally, when reviewing data for perfect compliance scores, the timeout section was performed perfectly in 60.4% (405 of 671) of checklists, and was significantly higher than all other sections of the SSC, which ranged from 1.2% to 7.1% (p < 0.001, McNemar’s test). Finally, we examined if operation-specific factors affected SSC compliance, including case start time, and
40
30
20
n = 81
n = 100
n = 100
n = 100
n = 671
0
n = 97
10
1
3
6
3
6
9
12
Total
Weeks
RESULTS Overall SWMH SSC participation rates significantly rose from 33% (95 of 285 cases) in week 1 to 94%
869
n = 98
Statistical analysis Data analysis was performed using GraphPad InStat (GraphPad Software, Inc.) statistics software. Quantitative data are expressed as mean standard deviation. Participation data were compared using chi-square test; checklist variables and overall compliance were compared using Student’s t-test or 1-way ANOVA as appropriate; checklist sections and subscores were compared using repeated-measures ANOVA with Tukey’s post-test pairwise multiple comparisons adjustments to compare group means or McNemar’s test as appropriate. Statistical significance was defined as p < 0.05.
Surgical Safety Checklist Compliance
n = 95
aggregate score of completion and accuracy, and each was assigned 20 points. Completion points were scored based on whether or not the checklist element was filled out on the SSC. Accuracy points were scored for elements on the checklist that could be verified as accurate or completed correctly by another part of the patient’s electronic medical record. The purpose of this tool design was to quantify both nonuse and misuse of the checklist. We believed that such a detailed and novel evaluation of checklist compliance would help elucidate specific pitfalls in SSC implementation. We retrospectively reviewed a random sample of SSC at 1-week intervals for SSC compliance, completion, and accuracy. All checklists during the entire year were collected and grouped by week. A statistician developed a sampling scheme to allow a random selection of a subset of up to 100 checklists from each time point. Each selected checklist was then scored. Specific time points analyzed included weeks 1, 3, 6, and 9, as well as months 3, 6, 9, and 12 from date of implementation. Number of checklists scored and collected respectively at each week were 95 of 95 (100%) at week one, 98 of 148 (66.2%) at week three, 97 of 129 (75.2%) at week six, 81 of 81 (100%) at month three, 100 of 186 (53.8%) at month six, 100 of 274 (36.5%) at month nine, and 100 of 249 (40.2%) at month 12. Data acquisition from the checklists and charts were performed by a single resident (ES) and scores were computed from that data by computer algorithm. In an effort to determine barriers to effective SSC use, we evaluated the association of operative characteristics with SSC compliance. Data analyzed included procedure start times, operative times (incision to close), and procedure preparation time (patient into operating room to incision). Surgical service data were also collected for each checklist for analysis.
Sparks et al
Average Score
Vol. 217, No. 5, November 2013
Months
Figure 2. Surgical Safety Checklist compliance scores. Scores are plotted as mean 1 SD error bars. There was no significant difference in compliance scores for time points measured during the study period (1-way ANOVA).
870
Sparks et al
collection of completed checklists contributed to the rise in participation measured in this study. We were unable to assess collection independently. Using our objective evaluation tool on SSC compliance, we found that although completion scores were high (85%), accuracy was poor (54%), resulting in an overall compliance score of 69%, indicating that our surgical teams are not universally using the SSC as intended. The overall compliance scores did not differ over time for the first year, suggesting that we achieved sustainability at the current level of SSC fidelity. We examined potential barriers to effective SSC use and found that case complexity, as determined by procedure preparation time and operative times did not affect SSC compliance scores; however, operative start times before 8:30 AM (ie, first starts of the day) had significantly higher SSC compliance than cases performed after 4 PM (after day shift change). Our data indicate that the high participation rate achieved by our surgical teams is not reflective of compliance or accuracy of SSC use, and suggest barriers to effective use of the SSC exist. It has been suggested that proper SSC use is required to see improved outcomes.8,9 In our study, we have evaluated SSC compliance with an assumption that high compliance scores are required to achieve desired outcomes. Due to limitations of reviewing individual checklists by hand, this study was not powered to evaluate outcomes as a function of checklist compliance. Others have performed similar evaluations looking at completion of checklist elements; however, our study reports total SSC compliance as an aggregate score of both completion and accuracy in an effort to evaluate proper use. In our study, we have found that, despite high participation rate, overall SSC compliance and accuracy were poor. In fact, no SSC scored 100% on compliance, and
Figure 3. Surgical Safety Checklist compliance score with corresponding completion and accuracy components. Data are expressed as a percent of total points possible. Scores for completion were significantly higher than scores for accuracy (p < 0.001, paired Student’s t-test).
case complexity. Checklists from several surgical subspecialties were compared, including general surgery, otolaryngology, neurosurgery, orthopaedics, pediatric surgery, and urology. There was no significant difference in compliance scores between any 2 specialties. As shown in Figure 4A, first cases scored significantly higher mean compliance points when compared with cases starting after 4 PM (28.6 4.8 [n ¼ 183] vs 26.6 6.2 [n ¼ 75]; p < 0.05; ANOVA with Tukey post-test). Case complexity, as determined by procedure preparation time and operative time, had no association with SSC compliance (Fig. 4B and C).
DISCUSSION Our study shows that implementation of our SSC resulted in a high level of participation (94%). It is likely that improvements in both checklist participation and as Table 1.
J Am Coll Surg
Surgical Safety Checklist Compliance
Average and Perfect Scores for Surgical Safety Checklist Total possible points
SSC sections Check in Sign in Time out Sign out Overall SSC Completion Accuracy Compliance
Average score (raw points)
8 8 4 20
5.4 6.1 3.1 12.7
2.1 1.6 1.0 3.7
20 20 40
16.9 2.7 10.8 3.4 27.7 5.4
Mean score (% of total possible score)
No. of checklists with perfect scores
% of checklists with perfect scores
48 8 405 29
7.2 1.2 60.4 4.3
105y 1y 0
15.6 0.1 0
67.5 75.2 77.8 63.6
26.5 20.6 24.4* 18.5
84.5 13.6 54.1 16.9 69.3 13.4
*Mean scores measured by percentage of total possible score demonstrate that the time-out section of the checklist is significantly different from all other sections by individual post-test analysis (p < 0.05, repeated measures ANOVA with Tukey post-test). y There were significantly more checklists reviewed with perfect completion scores (105 of 671) than with perfect accuracy score (1 of 671) p < 0.001 (McNemar’s test). SSC, Surgical Safety Checklist.
Vol. 217, No. 5, November 2013
Sparks et al
Surgical Safety Checklist Compliance
871
Figure 4. Factors associated with Surgical Safety Checklist compliance scores. (A) Operation start times. (B) Operative time. (C) Procedure preparation time with linear regression line, R2, and slope with 95% CI. Compliance scores for first starts (before 8:30 AM) were significantly higher than compliance scores for operations starting after 4 PM (*p < 0.05, ANOVA with Tukey post-test). There was no difference in compliance scores for operative time or procedure preparation time. Of note, case time was analyzed as a categorical variable because we believed these case times were reasonable approximations of “simple,” “intermediate,” and “complicated” cases. Case preparation time as analyzed as a continuous variable because no such natural categorization exists.
although 105 (15.6%) checklists had perfect completion, only 1 (0.1%) was completely accurate. It might be considered that our criteria for evaluating checklist accuracy were stringent, but there does appear to be a clear disparity between checklist completion and accuracy. Our finding is not unique, and is well illustrated by Levy and colleagues,12 whose observational study demonstrated that although institutional compliance rates were reported at 100%, none of the cases had executed all items on the checklist completely. Overall, only 31% (4 of 13) of checklist items were performed in the observed cases. As in our study, these authors do not report their outcomes, but this raises concerns about SSC compliance and the impact SSCs can have on patient outcomes. Two studies have reported performance and completion data with outcomes. First, Haynes and colleagues3 reported that implementation of the WHO SSC resulted in a substantial improvement in the percentage of patients
that had all 6 safety measures of the checklist performed (34.2% vs 56.7%), and this observation was associated with a substantial improvement in overall complications (11% vs 7%) and a 47% reduction in mortality rates (1.5% vs 0.8%); however, in this study, the authors commented that performance of checklist sections could not be verified. In a separate study, van Klei and colleagues4 achieved full completion of SSC in only 39% of patients, but the observed 8.9% decrease in overall mortality rate (3.13% vs 2.85%) was not significant. Although these data might suggest a threshold effect, the authors performed additional analysis by studying actual checklist completion and showed that fully completed SSCs had a significantly lower odds ratio (0.23; 95% CI, 0.160.33]) than partially completed (odds ratio ¼ 1.16; 95% CI, 0.951.41) and noncompleted (odds ratio ¼ 1.57; 95% CI, 1.311.89) checklists. The authors do not report to what degree the partially completed
872
Sparks et al
Surgical Safety Checklist Compliance
checklists were performed. These data might indicate that a threshold effect exists for improved outcomes with SSC. Barriers to SSC implementation and its effective use are multifactorial and encompass cultural and structural factors, including communication gaps, perceived benefit by the provider, lack of procedural understanding, perceived ambiguity of the process, and perceptions of negative impact on operative efficiency.11,15,16 In our study, we examined specific characteristics of the operative cases (start times and case complexity) as potential barriers to effective SSC use. Our data show that procedure operative time and preparation time (Fig. 4B and C), both surrogates of case complexity, had no effect on SSC compliance scores; however, compliance scores were highest for the first case of the day and were considerably higher than SSCs performed after 4 PM (Fig. 4A). Seen in Figure 4A, the cases after 4 PM showed a wider variation in SSC compliance score, indicating that checklist performance fidelity is adversely affected after 4 PM. This might be a direct result of having a fresh surgical team at the beginning of the day, or the fact that active surveillance is being performed more consistently at these times. Alternatively, the significantly lower scores seen after 4 PM might represent poor SSC education or access to support for surgical teams about SSC use for the evening and night shifts. We plan to implement an education and surveillance strategy to accommodate surgical teams outside the day shift. These findings of poor checklist compliance have resulted in an increased awareness of the need to identify barriers related to implementation strategy, team behavior, and checklist design, which might have impacted proper checklist use in our hospital. Recent publications have highlighted the importance of checklist design and effective strategy for implementation to ensure success.2,10 Our SSC design and implementation were executed before these publications, but comparison gives us information about barriers to effective SSC use. First, our checklist design was carried out through a multidisciplinary team that took into consideration institutional needs and culture. Our education program was multifaceted and included mandatory online education modules to explain the “why,” with embedded video examples of SSC to explain the “how,” as recommended by Conley and colleagues.10 However, we also recognize the need to provide real-time or historical institutional examples as feedback to our surgical team providers to enhance sustainability of this patient-safety program. As we perform a critical analysis of our SSC in context to the fidelity of its compliance, we are reassessing whether our current checklist is meeting the needs of our patients, surgical teams, and institution. Checklist items require 2 important characteristics to be effective, and the item checked should be a “great
J Am Coll Surg
danger to the patient if it were missed” and “not adequately checked by other mechanisms.”2 The timeout section of the SSC meets these criteria. In fact, evaluation of the time-out section in our SSC shows it had the highest fidelity of total compliance (60.4%). Others have shown similar results, with the time out having the highest degree of completion at 59.2%.4 We believe these findings might represent physician response to regulatory requirements of the time out by The Joint Commissions14 and its more mature state in the process of patient-safety implementation. We would anticipate that as institutions adopt, maintain, and support SSC use, and use surgical team feedback for meaningful modifications, there will be an improvement in provider “buyin” and effective SSC use. Interpretation of our study must be made in context to the limitations in study design. First, our results are obtained from a retrospective review of completed checklists. Similar to Haynes and colleagues,3 adherence to SSC items, such as introductions and debriefings, were not directly observed. Therefore, it is plausible that the checklists were completed as a check-box exercise rather than a patient-safety instrument. For this reason, we created a compliance assessment tool that included completion (checking the box) and accuracy (entering or marking correct information). We believed this was a more objective evaluation of SSC use. Second, our study does not evaluate or report patient outcomes, which is beyond the scope of the current study. Finally, given the retrospective nature of our study and the quality focus of our research group, we cannot eliminate unintended bias. To minimize potential bias, data acquisition from the checklist and patient’s chart was performed by a single resident (ES) on the research team. In addition to reducing inter-observer error and bias, a single evaluator reduces variability in the quality of reporting and data acquisition, which would improve consistency of results.
CONCLUSIONS Our data indicate that although implementation of an SSC results in high level of participation and overall completion, accuracy was poor, indicating incomplete fidelity of SSC compliance. Identification of barriers to effective use is needed as improper checklist use can adversely affect patient safety. Surgical team resources and education can be factors in decreased SSC compliance seen in operative cases after 4 PM. Our data will be used to establish programs to bridge gaps in our SSC and implementation process to improve compliance and effective use.
Vol. 217, No. 5, November 2013
Author Contributions Study conception and design: Sparks, Wehbe-Janek, Smythe, Papaconstantinou Acquisition of data: Sparks, Johnson Analysis and interpretation of data: Sparks, Wehbe-Janek, Papaconstantinou Drafting of manuscript: Sparks, Wehbe-Janek, Johnson, Papaconstantinou Critical revision: Sparks, Smythe, Papaconstantinou REFERENCES 1. Kohn LT, Corrigan JM, Donaldson MS, et al. To err is human: building a safer health care system. 1999. Available at: http://books.nap.edu/openbook.php?isbn¼0309068371. Accessed January 10, 2013. 2. Weiser TG, Haynes AB, Lashoher A, et al. Perspectives in quality: designing the WHO Surgical Safety Checklist. Int J Qual Health Care 2010;22:365e370. 3. Haynes AB, Weiser TG, Berry WR, et al. A surgical safety checklist to reduce morbidity and mortality in a global population. N Engl J Med 2009;360:491e499. 4. van Klei WA, Hoff RG, van Aarnhem EEHL, et al. Effects of the introduction of the WHO “surgical safety checklist” on inhospital mortality: a cohort study. Ann Surg 2012;255:44e49. 5. Bliss LA, Ross-Richardson CB, Sanzari LJ, et al. Thirty-day outcomes support implementation of a surgical safety checklist. J Am Coll Surg 2012;215:766e776. 6. Hurtado JJ, Jime´nez X, Pen˜alonzo MA, et al. Acceptance of the WHO Surgical Safety Checklist among surgical personnel in hospitals in Guatemala city. BMC Health Serv Res 2012;12:169.
Sparks et al
Surgical Safety Checklist Compliance
873
7. Yuan CT, Walsh D, Tomarken JL, et al. Incorporating the World Health Organization Surgical Safety Checklist into practice at two hospitals in Liberia. Jt Comm J Qual Patient Saf 2012;38:254e260. 8. Vijayasekar C, Steele RJ. The World Health Organization’s Surgical Safety Checklist. Surgeon 2009;7:260e262. 9. Walker IA, Reshamwalla S, Wilson IH. Surgical safety checklists: do they improve outcomes? Br J Anaesth 2012;109: 47e54. 10. Conley DM, Singer SJ, Edmondson L, et al. Effective surgical safety checklist implementation. J Am Coll Surg 2011;212: 873e879. 11. Fourcade A, Blache JL, Grenier C, et al. Barriers to staff adoption of a surgical safety checklist. BMJ Qual Saf 2012;21: 191e197. 12. Levy SM, Senter CE, Hawkins RB, et al. Implementing a surgical checklist: more than just checking a box. Surgery 2012;152:331e336. 13. Vats A, Vincent CA, Nagpal K, et al. Practical challenges of introducing WHO surgical checklist: UK pilot experience. BMJ 2010;340:b5433. 14. Michaels RK, Makary MA, Dahab Y, et al. Achieving the National Quality Forum’s “Never Events”: prevention of wrong site, wrong procedure and wrong patient operations. Ann Surg 2007;245:526e532. 15. Carney BT, West P, Neily J, et al. Differences in nurse and surgeon perceptions of teamwork: implications for use of a briefing checklist in the OR. AORN J 2010;91: 722e729. 16. Awad SS, Fagan SP, Bellows C, et al. Bridging the communication gap in the operating room with medical team training. Am J Surg 2005;190:770e774.
873.e1
Sparks et al
APPENDIX 1.
Surgical Safety Checklist Compliance
J Am Coll Surg
Sparks et al
Vol. 217, No. 5, November 2013
Surgical Safety Checklist Compliance
873.e2
Appendix 2. Scoring chart for Surgical Safety Checklist compliance according to checklist section and completion and accuracy elements SSC Section
Check In
Completion Elements
Section Completion d Completeall items filled out d Partialone or more items left blank d Blankno items filled out
Points
Accuracy Elements
2 1
Section Completion d Completeall items filled out d Partialone or more items left blank d Blankno items filled out
Possible Points
2
0 Estimated Procedure Time Recorded time consistent with either actual case time or posted case time d Within 50% of posted or actual case time d Inaccurate or blank section Implants Must be consistent with implants opened for the case and documented in record d Checked and documented use in case d Unchecked and not indicated d Checked but unable to verify d Inaccurate or blank section VTE Prophylaxis Must be consistent with SCIP guidelines or documented why withheld d Checked and documented given d Unchecked and not indicated d Checked but unable to verity d Inaccurate or blank section
Sign In
Points
2
2 0 2
2 2 1 0 2
2 2 1 0 (Total /8) 2
2 1 0 Allergy Must be consistent with medical record d Allergies documented accurately d Allergies listed but not in medical record d Allergies unchecked but listed in medical record Difficult Airway Must be accurate according to anesthesia documentation system d Checked, difficult airway documented d Unchecked, normal airway documented d Checked, undocumented airway d Unchecked, difficult airway documented d Section left blank Risk of Hypothermia Must be consistent with nursing record use of warming device if operative time >1 hour
2 2 1 0 2
2 2 1 0 0 2
(Continued)
873.e3
Sparks et al
J Am Coll Surg
Surgical Safety Checklist Compliance
Appendix 2. Continued SSC Section
Completion Elements
Points
Accuracy Elements d d d d
Checked, warming apparatus used Unchecked, apparatus not used Checked, unable to verify Inaccurate or section blank
Points
Possible Points
2 2 1 0 (Total /8)
Time Out
Section Completion d Completeall items filled out d Partialone or more items left blank d Blankno items filled out
2 1 0 Antibiotics Must be consistent with medical record and nursing documentation d Checked, given according to SCIP d Unchecked, not given (documented why) d Checked, not given appropriately d Inaccurate or section blank
Sign Out
Section Completion d Completeall items filled out d Partialone or more items left blank d Blankno items filled out Signature Completion d One point awarded for each of the 12 blanks (signature, date, time for surgeon, nurse, and anesthesia provider) filled out properly.
(Total /4) 2 2 1 0 12 12
Signature Each signature must be dated (1 point) and timed (1 point) after case end time d Surgeon signature - date and time d Nurse signature e date and time d Anesthesia provider e date and time Totals
Completion / 20
2 2 1 0
Accuracy /20
SCIP, Surgical Care Improvement Project; VTE, venous thromboembolism.
6
2 2 2 (Total /20) Compliance/ 40
PhD,
Rebecca L Johnson,
RN,
W Roy Smythe,
MD, FACS,
The Surgical Safety Checklist (SSC) has been introduced as an effective tool for reducing perioperative mortality and complications. Although reported completion rates are high, objective compliance is not well defined. The purpose of this retrospective analysis is to determine SSC compliance as measured by accuracy and completion, and factors that can affect compliance. STUDY DESIGN: In September 2010, our institution implemented an adaptation of the World Health Organization’s SSC in an effort to improve patient safety and outcomes. A tool was developed for objective evaluation of overall compliance (maximum score 40) that was an aggregate score of completion and accuracy (20 each). Random samples of SSCs were analyzed at specific, predefined, time points throughout the first year after implementation. Procedure start time, operative time, and case complexity were assessed to determine association with compliance. RESULTS: A total of 671 SSCs were analyzed. The participation rate improved from 33% (95 of 285) at week 1 to 94% (249 of 265) at 1 year (p < 0.0001, chi-square test). Mean overall compliance score was 27.7 (5.4 SD) of 40 possible points (69.3% 13.5% of total possible score; n ¼ 671) and did not change over time. Although completion scores were high (16.9 2.7 out of 20 [84.5% 13.6%]), accuracy was poor (10.8 3.4 out of 20 [54.1% 16.9%]). Overall compliance score was significantly associated with case start-time (p < 0.05), and operative time and case complexity showed no association. CONCLUSIONS: Our data indicate that although implementation of an SSC results in a high level of overall participation and completion, accuracy remained poor. Identification of barriers to effective use is needed, as improper checklist use can adversely affect patient safety. (J Am Coll Surg 2013;217:867e873. 2013 by the American College of Surgeons) BACKGROUND:
which has been widely successful in substantially reducing rates of perioperative complications and mortality.3 The celebrated success of this patient-safety tool and its reported ease of implementation have led to rapid adoption of SSCs worldwide.3-7 Although the purpose of checklists and other patient-safety tools is to reduce patient harm and improve outcomes, it has been assumed that these instruments are only effective if used as intended.8,9 Appropriate checklist use and sustainable practice can be a result of effective implementation.9,10 In addition, barriers to implementation are multifactorial and encompass cultural and structural levels, including communication gaps, perceived benefit by the provider, lack of procedural understanding, and perceived ambiguity of the process.11 Although many studies report successful use of SSCs, few have reported on the quality of compliance and accuracy of patient information reviewed during the process.11-13 In September 2010, our hospital started using an SSC in an effort to improve patient safety and outcomes. As
More than a decade has passed since the publication of To Err is Human,1 and this time has brought considerable interest in research and interventions to describe and prevent health careassociated injuries. The WHO launched its “Safe Surgery Saves Lives” campaign in 2008 as a means to prevent unnecessary death and improve outcomes for surgical patients.2 This program resulted in the WHO Surgical Safety Checklist (SSC),
Disclosure Information: Nothing to disclose. Abstract presented at the American College of Surgeons 98th Annual Clinical Congress, Surgical Forum, Chicago, IL, 2012. Received February 19, 2013; Revised June 29, 2013; Accepted July 9, 2013. From the Department of Surgery (Sparks, Johnson, Smythe, Papaconstantinou) and Office of Academic Research Development (Wehbe-Janek), Scott & White Memorial Hospital and Clinic, Texas A&M University System Health Science Center, College of Medicine, Temple, TX. Correspondence address: Harry T Papaconstantinou, MD, FACS, Department of Surgery, Scott & White Memorial Hospital and Clinic, 2401 South 31st St, Temple, TX 76508. email: [email protected]
ª 2013 by the American College of Surgeons Published by Elsevier Inc.
867
ISSN 1072-7515/13/$36.00 http://dx.doi.org/10.1016/j.jamcollsurg.2013.07.393
868
Sparks et al
J Am Coll Surg
Surgical Safety Checklist Compliance
Figure 1. Implementation timeline of the Scott & White Memorial Hospital Surgical Safety Checklist. The details and sequence of events of this study are described completely in the Methods section.
shown in Figure 1, the implementation process included checklist design, focused trial, and educational campaign. Post-implementation process included ongoing monitoring, program reinforcement, and outcomes review. Participation and compliance was identified as an important part of outcomes review as our quality-improvement group assumed that proper checklist use is a requirement to achieve desired safety improvements. Therefore, the purpose of this study was to objectively evaluate the quality of SSC compliance as measured by accuracy and completion and determine factors that can affect compliance.
METHODS Setting Scott & White Memorial Hospital (SWMH) is a 636bed tertiary care center that performed >18,000 operations in our main hospital operating room facility in fiscal year 2011. Our physicians operate as a single multispecialty group practice. At the time of this study, adult and pediatric operative services and operating rooms were contained within SWMH only. The operating rooms all follow the same perioperative practices and use the same personnel. This retrospective study is an observational study for quality improvement in our surgical safety process as it relates to SSCs. Scott & White Memorial Hospital Surgical Safety Checklist Our WHO-adapted SSC was developed using a multidisciplinary team approach with the goal of improving patient safety and surgical outcomes by enhancing communication and standardizing practice within the operating room. The final checklist was composed of 4 sections that parallel perioperative patient care (Appendix 1, online only) and includes check in, sign in, time out, and sign out. The check-in section occurs when the patient is in the preoperative holding unit, and focuses on communication among the surgical team to identify case-specific concerns and needs. The sign-in section occurs on the patient’s arrival to the operating room and highlights specific safety measures and
operative preparation. The time-out section occurs just before incision and includes all elements of the Joint Commissionendorsed time-out process,14 and provides the opportunity for the surgical team to communicate specific equipment and patient concerns. The sign-out section occurs at the end of each procedure and focuses on appropriate documentation of the procedure, specimen, and key postoperative concerns that can be helpful for recovery and management. This section also requires name, signature, date, and time from each member of the surgical team and the checklist is incorporated into the medical record. Focused trials of our SSC were performed by 3 surgeon champions in an effort to make meaningful iterative changes before widespread use. A 3-month multi-medium SWMH-focused SSC education campaign was initiated and championed by departmental and hospital leaders in surgical service areas that focused on why the SSC was important and how the SSC was to be used. This campaign included educational forums, posters, mandatory continuing medical education credit online training modules, and focused frequently asked question email blasts to all surgical providers. On September 1, 2010, SWMH started using the SSC throughout all of its operating rooms. After implementation, there was continued surgical team engagement with structured monitoring and reinforcement. At 1 year, we began a retrospective review of outcomes, which is partially presented in this study. Figure 1 shows a timeline demonstrating the design and implementation process of our SSC. Study design During the 1-year period after SSC implementation, all checklists were collected and saved. Checklist participation was defined by an SSC filled out, regardless of degree of completion or accuracy, for a given case. Surgical Safety Checklist participation was determined for an entire week at week 1 and year 1 post-SSC implementation. A tool was developed for objective evaluation of overall compliance with a maximum compliance score of 40 (Appendix 2, online only). Compliance was an
(249 of 265 cases) at 1 year (p < 0.01). Next, we reviewed a random sample of charts with representation from all surgical specialties during 1 year. A total of 671 SSCs were evaluated during the study period for completion (20 points), accuracy (20 points), and compliance (40 points). All SSCs in our study had an average of 27.7 of 40 points for compliance (69%), and there was no significant change in compliance scores during the study period (Fig. 2). When compliance scores were broken into completion and accuracy components, there was a large disparity between these measures. The average checklist achieved 16.9 of 20 points (85%) for completion vs only 10.8 of 20 points (54%) for accuracy (p < 0.05) (Fig. 3). Next, we determined the number of SSCs that achieved perfect scores for completion, accuracy, and compliance. Of 671 checklists analyzed, 105 (15.6%) received all 20 points for completion and only 1 (0.1%) checklist received all 20 points for accuracy (Table 1). There were no checklists (0%) that received a perfect score, or 40 points, for compliance. Next, individual SSC sections were evaluated to determine barrier areas for our SSC. Table 1 shows that that the time-out section had the highest mean compliance percentage score at 77.8%, and was significantly higher than the check-in (67.5%), sign-in (75.2%), and signout (63.6%) sections (p < 0.05, repeated-measures ANOVA with Tukey post-test). Additionally, when reviewing data for perfect compliance scores, the timeout section was performed perfectly in 60.4% (405 of 671) of checklists, and was significantly higher than all other sections of the SSC, which ranged from 1.2% to 7.1% (p < 0.001, McNemar’s test). Finally, we examined if operation-specific factors affected SSC compliance, including case start time, and
40
30
20
n = 81
n = 100
n = 100
n = 100
n = 671
0
n = 97
10
1
3
6
3
6
9
12
Total
Weeks
RESULTS Overall SWMH SSC participation rates significantly rose from 33% (95 of 285 cases) in week 1 to 94%
869
n = 98
Statistical analysis Data analysis was performed using GraphPad InStat (GraphPad Software, Inc.) statistics software. Quantitative data are expressed as mean standard deviation. Participation data were compared using chi-square test; checklist variables and overall compliance were compared using Student’s t-test or 1-way ANOVA as appropriate; checklist sections and subscores were compared using repeated-measures ANOVA with Tukey’s post-test pairwise multiple comparisons adjustments to compare group means or McNemar’s test as appropriate. Statistical significance was defined as p < 0.05.
Surgical Safety Checklist Compliance
n = 95
aggregate score of completion and accuracy, and each was assigned 20 points. Completion points were scored based on whether or not the checklist element was filled out on the SSC. Accuracy points were scored for elements on the checklist that could be verified as accurate or completed correctly by another part of the patient’s electronic medical record. The purpose of this tool design was to quantify both nonuse and misuse of the checklist. We believed that such a detailed and novel evaluation of checklist compliance would help elucidate specific pitfalls in SSC implementation. We retrospectively reviewed a random sample of SSC at 1-week intervals for SSC compliance, completion, and accuracy. All checklists during the entire year were collected and grouped by week. A statistician developed a sampling scheme to allow a random selection of a subset of up to 100 checklists from each time point. Each selected checklist was then scored. Specific time points analyzed included weeks 1, 3, 6, and 9, as well as months 3, 6, 9, and 12 from date of implementation. Number of checklists scored and collected respectively at each week were 95 of 95 (100%) at week one, 98 of 148 (66.2%) at week three, 97 of 129 (75.2%) at week six, 81 of 81 (100%) at month three, 100 of 186 (53.8%) at month six, 100 of 274 (36.5%) at month nine, and 100 of 249 (40.2%) at month 12. Data acquisition from the checklists and charts were performed by a single resident (ES) and scores were computed from that data by computer algorithm. In an effort to determine barriers to effective SSC use, we evaluated the association of operative characteristics with SSC compliance. Data analyzed included procedure start times, operative times (incision to close), and procedure preparation time (patient into operating room to incision). Surgical service data were also collected for each checklist for analysis.
Sparks et al
Average Score
Vol. 217, No. 5, November 2013
Months
Figure 2. Surgical Safety Checklist compliance scores. Scores are plotted as mean 1 SD error bars. There was no significant difference in compliance scores for time points measured during the study period (1-way ANOVA).
870
Sparks et al
collection of completed checklists contributed to the rise in participation measured in this study. We were unable to assess collection independently. Using our objective evaluation tool on SSC compliance, we found that although completion scores were high (85%), accuracy was poor (54%), resulting in an overall compliance score of 69%, indicating that our surgical teams are not universally using the SSC as intended. The overall compliance scores did not differ over time for the first year, suggesting that we achieved sustainability at the current level of SSC fidelity. We examined potential barriers to effective SSC use and found that case complexity, as determined by procedure preparation time and operative times did not affect SSC compliance scores; however, operative start times before 8:30 AM (ie, first starts of the day) had significantly higher SSC compliance than cases performed after 4 PM (after day shift change). Our data indicate that the high participation rate achieved by our surgical teams is not reflective of compliance or accuracy of SSC use, and suggest barriers to effective use of the SSC exist. It has been suggested that proper SSC use is required to see improved outcomes.8,9 In our study, we have evaluated SSC compliance with an assumption that high compliance scores are required to achieve desired outcomes. Due to limitations of reviewing individual checklists by hand, this study was not powered to evaluate outcomes as a function of checklist compliance. Others have performed similar evaluations looking at completion of checklist elements; however, our study reports total SSC compliance as an aggregate score of both completion and accuracy in an effort to evaluate proper use. In our study, we have found that, despite high participation rate, overall SSC compliance and accuracy were poor. In fact, no SSC scored 100% on compliance, and
Figure 3. Surgical Safety Checklist compliance score with corresponding completion and accuracy components. Data are expressed as a percent of total points possible. Scores for completion were significantly higher than scores for accuracy (p < 0.001, paired Student’s t-test).
case complexity. Checklists from several surgical subspecialties were compared, including general surgery, otolaryngology, neurosurgery, orthopaedics, pediatric surgery, and urology. There was no significant difference in compliance scores between any 2 specialties. As shown in Figure 4A, first cases scored significantly higher mean compliance points when compared with cases starting after 4 PM (28.6 4.8 [n ¼ 183] vs 26.6 6.2 [n ¼ 75]; p < 0.05; ANOVA with Tukey post-test). Case complexity, as determined by procedure preparation time and operative time, had no association with SSC compliance (Fig. 4B and C).
DISCUSSION Our study shows that implementation of our SSC resulted in a high level of participation (94%). It is likely that improvements in both checklist participation and as Table 1.
J Am Coll Surg
Surgical Safety Checklist Compliance
Average and Perfect Scores for Surgical Safety Checklist Total possible points
SSC sections Check in Sign in Time out Sign out Overall SSC Completion Accuracy Compliance
Average score (raw points)
8 8 4 20
5.4 6.1 3.1 12.7
2.1 1.6 1.0 3.7
20 20 40
16.9 2.7 10.8 3.4 27.7 5.4
Mean score (% of total possible score)
No. of checklists with perfect scores
% of checklists with perfect scores
48 8 405 29
7.2 1.2 60.4 4.3
105y 1y 0
15.6 0.1 0
67.5 75.2 77.8 63.6
26.5 20.6 24.4* 18.5
84.5 13.6 54.1 16.9 69.3 13.4
*Mean scores measured by percentage of total possible score demonstrate that the time-out section of the checklist is significantly different from all other sections by individual post-test analysis (p < 0.05, repeated measures ANOVA with Tukey post-test). y There were significantly more checklists reviewed with perfect completion scores (105 of 671) than with perfect accuracy score (1 of 671) p < 0.001 (McNemar’s test). SSC, Surgical Safety Checklist.
Vol. 217, No. 5, November 2013
Sparks et al
Surgical Safety Checklist Compliance
871
Figure 4. Factors associated with Surgical Safety Checklist compliance scores. (A) Operation start times. (B) Operative time. (C) Procedure preparation time with linear regression line, R2, and slope with 95% CI. Compliance scores for first starts (before 8:30 AM) were significantly higher than compliance scores for operations starting after 4 PM (*p < 0.05, ANOVA with Tukey post-test). There was no difference in compliance scores for operative time or procedure preparation time. Of note, case time was analyzed as a categorical variable because we believed these case times were reasonable approximations of “simple,” “intermediate,” and “complicated” cases. Case preparation time as analyzed as a continuous variable because no such natural categorization exists.
although 105 (15.6%) checklists had perfect completion, only 1 (0.1%) was completely accurate. It might be considered that our criteria for evaluating checklist accuracy were stringent, but there does appear to be a clear disparity between checklist completion and accuracy. Our finding is not unique, and is well illustrated by Levy and colleagues,12 whose observational study demonstrated that although institutional compliance rates were reported at 100%, none of the cases had executed all items on the checklist completely. Overall, only 31% (4 of 13) of checklist items were performed in the observed cases. As in our study, these authors do not report their outcomes, but this raises concerns about SSC compliance and the impact SSCs can have on patient outcomes. Two studies have reported performance and completion data with outcomes. First, Haynes and colleagues3 reported that implementation of the WHO SSC resulted in a substantial improvement in the percentage of patients
that had all 6 safety measures of the checklist performed (34.2% vs 56.7%), and this observation was associated with a substantial improvement in overall complications (11% vs 7%) and a 47% reduction in mortality rates (1.5% vs 0.8%); however, in this study, the authors commented that performance of checklist sections could not be verified. In a separate study, van Klei and colleagues4 achieved full completion of SSC in only 39% of patients, but the observed 8.9% decrease in overall mortality rate (3.13% vs 2.85%) was not significant. Although these data might suggest a threshold effect, the authors performed additional analysis by studying actual checklist completion and showed that fully completed SSCs had a significantly lower odds ratio (0.23; 95% CI, 0.160.33]) than partially completed (odds ratio ¼ 1.16; 95% CI, 0.951.41) and noncompleted (odds ratio ¼ 1.57; 95% CI, 1.311.89) checklists. The authors do not report to what degree the partially completed
872
Sparks et al
Surgical Safety Checklist Compliance
checklists were performed. These data might indicate that a threshold effect exists for improved outcomes with SSC. Barriers to SSC implementation and its effective use are multifactorial and encompass cultural and structural factors, including communication gaps, perceived benefit by the provider, lack of procedural understanding, perceived ambiguity of the process, and perceptions of negative impact on operative efficiency.11,15,16 In our study, we examined specific characteristics of the operative cases (start times and case complexity) as potential barriers to effective SSC use. Our data show that procedure operative time and preparation time (Fig. 4B and C), both surrogates of case complexity, had no effect on SSC compliance scores; however, compliance scores were highest for the first case of the day and were considerably higher than SSCs performed after 4 PM (Fig. 4A). Seen in Figure 4A, the cases after 4 PM showed a wider variation in SSC compliance score, indicating that checklist performance fidelity is adversely affected after 4 PM. This might be a direct result of having a fresh surgical team at the beginning of the day, or the fact that active surveillance is being performed more consistently at these times. Alternatively, the significantly lower scores seen after 4 PM might represent poor SSC education or access to support for surgical teams about SSC use for the evening and night shifts. We plan to implement an education and surveillance strategy to accommodate surgical teams outside the day shift. These findings of poor checklist compliance have resulted in an increased awareness of the need to identify barriers related to implementation strategy, team behavior, and checklist design, which might have impacted proper checklist use in our hospital. Recent publications have highlighted the importance of checklist design and effective strategy for implementation to ensure success.2,10 Our SSC design and implementation were executed before these publications, but comparison gives us information about barriers to effective SSC use. First, our checklist design was carried out through a multidisciplinary team that took into consideration institutional needs and culture. Our education program was multifaceted and included mandatory online education modules to explain the “why,” with embedded video examples of SSC to explain the “how,” as recommended by Conley and colleagues.10 However, we also recognize the need to provide real-time or historical institutional examples as feedback to our surgical team providers to enhance sustainability of this patient-safety program. As we perform a critical analysis of our SSC in context to the fidelity of its compliance, we are reassessing whether our current checklist is meeting the needs of our patients, surgical teams, and institution. Checklist items require 2 important characteristics to be effective, and the item checked should be a “great
J Am Coll Surg
danger to the patient if it were missed” and “not adequately checked by other mechanisms.”2 The timeout section of the SSC meets these criteria. In fact, evaluation of the time-out section in our SSC shows it had the highest fidelity of total compliance (60.4%). Others have shown similar results, with the time out having the highest degree of completion at 59.2%.4 We believe these findings might represent physician response to regulatory requirements of the time out by The Joint Commissions14 and its more mature state in the process of patient-safety implementation. We would anticipate that as institutions adopt, maintain, and support SSC use, and use surgical team feedback for meaningful modifications, there will be an improvement in provider “buyin” and effective SSC use. Interpretation of our study must be made in context to the limitations in study design. First, our results are obtained from a retrospective review of completed checklists. Similar to Haynes and colleagues,3 adherence to SSC items, such as introductions and debriefings, were not directly observed. Therefore, it is plausible that the checklists were completed as a check-box exercise rather than a patient-safety instrument. For this reason, we created a compliance assessment tool that included completion (checking the box) and accuracy (entering or marking correct information). We believed this was a more objective evaluation of SSC use. Second, our study does not evaluate or report patient outcomes, which is beyond the scope of the current study. Finally, given the retrospective nature of our study and the quality focus of our research group, we cannot eliminate unintended bias. To minimize potential bias, data acquisition from the checklist and patient’s chart was performed by a single resident (ES) on the research team. In addition to reducing inter-observer error and bias, a single evaluator reduces variability in the quality of reporting and data acquisition, which would improve consistency of results.
CONCLUSIONS Our data indicate that although implementation of an SSC results in high level of participation and overall completion, accuracy was poor, indicating incomplete fidelity of SSC compliance. Identification of barriers to effective use is needed as improper checklist use can adversely affect patient safety. Surgical team resources and education can be factors in decreased SSC compliance seen in operative cases after 4 PM. Our data will be used to establish programs to bridge gaps in our SSC and implementation process to improve compliance and effective use.
Vol. 217, No. 5, November 2013
Author Contributions Study conception and design: Sparks, Wehbe-Janek, Smythe, Papaconstantinou Acquisition of data: Sparks, Johnson Analysis and interpretation of data: Sparks, Wehbe-Janek, Papaconstantinou Drafting of manuscript: Sparks, Wehbe-Janek, Johnson, Papaconstantinou Critical revision: Sparks, Smythe, Papaconstantinou REFERENCES 1. Kohn LT, Corrigan JM, Donaldson MS, et al. To err is human: building a safer health care system. 1999. Available at: http://books.nap.edu/openbook.php?isbn¼0309068371. Accessed January 10, 2013. 2. Weiser TG, Haynes AB, Lashoher A, et al. Perspectives in quality: designing the WHO Surgical Safety Checklist. Int J Qual Health Care 2010;22:365e370. 3. Haynes AB, Weiser TG, Berry WR, et al. A surgical safety checklist to reduce morbidity and mortality in a global population. N Engl J Med 2009;360:491e499. 4. van Klei WA, Hoff RG, van Aarnhem EEHL, et al. Effects of the introduction of the WHO “surgical safety checklist” on inhospital mortality: a cohort study. Ann Surg 2012;255:44e49. 5. Bliss LA, Ross-Richardson CB, Sanzari LJ, et al. Thirty-day outcomes support implementation of a surgical safety checklist. J Am Coll Surg 2012;215:766e776. 6. Hurtado JJ, Jime´nez X, Pen˜alonzo MA, et al. Acceptance of the WHO Surgical Safety Checklist among surgical personnel in hospitals in Guatemala city. BMC Health Serv Res 2012;12:169.
Sparks et al
Surgical Safety Checklist Compliance
873
7. Yuan CT, Walsh D, Tomarken JL, et al. Incorporating the World Health Organization Surgical Safety Checklist into practice at two hospitals in Liberia. Jt Comm J Qual Patient Saf 2012;38:254e260. 8. Vijayasekar C, Steele RJ. The World Health Organization’s Surgical Safety Checklist. Surgeon 2009;7:260e262. 9. Walker IA, Reshamwalla S, Wilson IH. Surgical safety checklists: do they improve outcomes? Br J Anaesth 2012;109: 47e54. 10. Conley DM, Singer SJ, Edmondson L, et al. Effective surgical safety checklist implementation. J Am Coll Surg 2011;212: 873e879. 11. Fourcade A, Blache JL, Grenier C, et al. Barriers to staff adoption of a surgical safety checklist. BMJ Qual Saf 2012;21: 191e197. 12. Levy SM, Senter CE, Hawkins RB, et al. Implementing a surgical checklist: more than just checking a box. Surgery 2012;152:331e336. 13. Vats A, Vincent CA, Nagpal K, et al. Practical challenges of introducing WHO surgical checklist: UK pilot experience. BMJ 2010;340:b5433. 14. Michaels RK, Makary MA, Dahab Y, et al. Achieving the National Quality Forum’s “Never Events”: prevention of wrong site, wrong procedure and wrong patient operations. Ann Surg 2007;245:526e532. 15. Carney BT, West P, Neily J, et al. Differences in nurse and surgeon perceptions of teamwork: implications for use of a briefing checklist in the OR. AORN J 2010;91: 722e729. 16. Awad SS, Fagan SP, Bellows C, et al. Bridging the communication gap in the operating room with medical team training. Am J Surg 2005;190:770e774.
873.e1
Sparks et al
APPENDIX 1.
Surgical Safety Checklist Compliance
J Am Coll Surg
Sparks et al
Vol. 217, No. 5, November 2013
Surgical Safety Checklist Compliance
873.e2
Appendix 2. Scoring chart for Surgical Safety Checklist compliance according to checklist section and completion and accuracy elements SSC Section
Check In
Completion Elements
Section Completion d Completeall items filled out d Partialone or more items left blank d Blankno items filled out
Points
Accuracy Elements
2 1
Section Completion d Completeall items filled out d Partialone or more items left blank d Blankno items filled out
Possible Points
2
0 Estimated Procedure Time Recorded time consistent with either actual case time or posted case time d Within 50% of posted or actual case time d Inaccurate or blank section Implants Must be consistent with implants opened for the case and documented in record d Checked and documented use in case d Unchecked and not indicated d Checked but unable to verify d Inaccurate or blank section VTE Prophylaxis Must be consistent with SCIP guidelines or documented why withheld d Checked and documented given d Unchecked and not indicated d Checked but unable to verity d Inaccurate or blank section
Sign In
Points
2
2 0 2
2 2 1 0 2
2 2 1 0 (Total /8) 2
2 1 0 Allergy Must be consistent with medical record d Allergies documented accurately d Allergies listed but not in medical record d Allergies unchecked but listed in medical record Difficult Airway Must be accurate according to anesthesia documentation system d Checked, difficult airway documented d Unchecked, normal airway documented d Checked, undocumented airway d Unchecked, difficult airway documented d Section left blank Risk of Hypothermia Must be consistent with nursing record use of warming device if operative time >1 hour
2 2 1 0 2
2 2 1 0 0 2
(Continued)
873.e3
Sparks et al
J Am Coll Surg
Surgical Safety Checklist Compliance
Appendix 2. Continued SSC Section
Completion Elements
Points
Accuracy Elements d d d d
Checked, warming apparatus used Unchecked, apparatus not used Checked, unable to verify Inaccurate or section blank
Points
Possible Points
2 2 1 0 (Total /8)
Time Out
Section Completion d Completeall items filled out d Partialone or more items left blank d Blankno items filled out
2 1 0 Antibiotics Must be consistent with medical record and nursing documentation d Checked, given according to SCIP d Unchecked, not given (documented why) d Checked, not given appropriately d Inaccurate or section blank
Sign Out
Section Completion d Completeall items filled out d Partialone or more items left blank d Blankno items filled out Signature Completion d One point awarded for each of the 12 blanks (signature, date, time for surgeon, nurse, and anesthesia provider) filled out properly.
(Total /4) 2 2 1 0 12 12
Signature Each signature must be dated (1 point) and timed (1 point) after case end time d Surgeon signature - date and time d Nurse signature e date and time d Anesthesia provider e date and time Totals
Completion / 20
2 2 1 0
Accuracy /20
SCIP, Surgical Care Improvement Project; VTE, venous thromboembolism.
6
2 2 2 (Total /20) Compliance/ 40