- Email: [email protected]
A simplified WHO checklist improves compliance and time efficiency for urological surgery
British Journal of Medical and Surgical Urology (2012) 5, 120—127
ORIGINAL ARTICLE
A simplified WHO checklist improves compliance and time efficiency for urological surgery John Henderson ∗, Timothy Fung, Jaimin Bhatt, Amarjit Bdesha Department of Urology, Wycombe General Hospital, Queen Alexandra Road, High Wycombe, Bucks HP11 2TT, UK Received 29 March 2011 ; received in revised form 4 August 2011; accepted 16 August 2011
KEYWORDS WHO; Checklist; Clinical governance
Abstract Objective: A surgical safety checklist has been introduced throughout the UK in response to evidence that its use decreases surgical morbidity and mortality. The adaptation of this has resulted in a lengthy checklist which may be used improperly. We audited the existing and a new simplified checklist with regards to compliance, time-efficiency and relevant safety outcomes. Materials and methods: One hundred operations were observed. Fifty of these cases used the existing 14-question Briefing/Debriefing (BD) and 31-question Sign-in, Timeout and Sign-Out (STS) checklist. The subsequent 50 used a simplified 10-question BD and STS checklist. Percentage compliance, median time taken and relevant safety outcomes were recorded. Results: The median time for the BD questions decreased from 150 to 90 s and the STS questions from 88 to 49 s (p < 0.05). The compliance improved from 68% to 73% (p = 0.49) for BD questions and 53% to 92% (p < 0.05) for STS questions. A clearer phrasing of the antibiotic check question in the revised checklist resulted in no administration of incorrect antibiotics. Conclusion: The nuclear and airline industries have used checklists for many years and observed that long and exhaustive checklists were often used improperly or disregarded completely. We demonstrate that a redesigned, simplified checklist improves time-efficiency and compliance with improved safety outcomes. © 2011 British Association of Urological Surgeons. Published by Elsevier Ltd. All rights reserved.
∗ Corresponding author. Tel.: +01494 526161; fax: +44 01494 426001. E-mail addresses: john [email protected] (J. Henderson), [email protected] (T. Fung), [email protected] (J. Bhatt), [email protected] (A. Bdesha).
Introduction The World Health Organisation (WHO) advocates the introduction of a pre-operative surgical checklist as part of its ‘Safe Surgery Saves Lives’
1875-9742/$ — see front matter © 2011 British Association of Urological Surgeons. Published by Elsevier Ltd. All rights reserved. doi:10.1016/j.bjmsu.2011.08.003
A simplified WHO checklist improves compliance and time efficiency for urological surgery initiative. This has been shown to decrease operative morbidity and in-hospital mortality across a wide range of economic and clinical environments [1]. The use of a checklist may reduce operating room delays [2], communication failures [3] and risk of wrong site surgery [4]. The National Patient Safety Agency (NPSA) issued an alert in January 2009 requiring the implementation of an adapted checklist for all surgical procedures in NHS Trusts in England and Wales by February 2010 [5]. The design of the checklist drew heavily on the experiences of the airline industry whose use of checklists extends to 70 years. It was recognised that the length of the checklist would have to be a compromise between brevity and detail [6]. Concerns were raised during the consultation stage that it ‘would be impractical in short cases and especially during day cases with rapid turnover of patients’ [7]. Atul Gawande, the WHO ‘Safe Surgery Saves Lives’ lead, admits ‘The most intense disagreements flared over what should stay in and what should come out’ [8]. After implementation in the UK, 37% of centres surveyed suggested that ‘not having enough time’ was a barrier to implementation [9]. The issue of prolonged anaesthesia during checklist completion was also raised [10]. After a number of fatal air crashes in the 1980s, the airline industry was forced to analyse its use of checklists. A Northwest Airlines Flight crashed in 1987 with loss of 156 lives after executing a ‘no flaps’ takeoff. The National Transportation Safety Board (NTSB) investigation found that ‘neither pilot recited the items of the taxi checklist’ [11]. Analysis of pilot behaviour showed that with longer checklists, pilots would skip items, group them together and sometimes miss them altogether [12]. There followed a trend towards checklist reduction, incorporating ‘killer’ items only—–those which if missed would have the most serious consequences. An example of a 22-item checklist from 1977 is shown (Fig. 1) with a simplified 9-item checklist from 2009 (Fig. 2). The checklist provided by the NPSA is 28 questions long [13]. Adaptation of the checklist was allowed ‘locally or speciality specifically through local clinical governance arrangements to ensure organisational accountability’ [14]. As suggested by the ‘Patient Safety First’ group, our centre adapted the checklist into separate ‘Briefing and Debriefing (BD)’ and ‘Sign in, Time out and Sign out (STS)’ sections. The former was performed at the beginning and end of the list with 14 questions and the latter for each individual operation with 31 questions. We considered whether a simplified checklist, incorporating only the most important questions,
121
Figure 1 22-Item pre flight checklist from 1977 [courtesy of P. Stevenson].
would result in shorter completion times and higher completion rates.
Methods Observation A total of 100 urological operations with a variable case mix were observed. The first 50 used the existing WHO checklist, and the remaining 50 used a simplified 10-point checklist for each of the BD (Table 1) and STS (Table 2) sections. The operation observed was classified as major (laparoscopic/open organ removal or PCNL), intermediate (endoscopic ureterorenoscopic, bladder, prostate or urethral surgery) or minor (open inguinoscrotal,
Figure 2 9-Item pre flight checklist from 2009 [courtesy of P. Stevenson].
122 Table 1
J. Henderson et al. Original and simplified 10-point Briefing/Debriefing (BD) checklist.
Original checklist
Revised checklist
Briefing (1) Have all team members been introduced by name and role? (2) Is the anaesthetic equipment check complete? (3) Are there any equipment issues for any of the cases?
Briefing (1) Everyone to introduce themselves-name and role (2) Anaesthetic check complete? (3) Any equipment concerns for any cases and are radiographers needed? (4) Any emergency cases to do or anything expected which will disrupt the list? (5) Is the order of the list correct?
(4) Any specific issues with any of the patients that we all need to know about (5) Anything expected during the day which will affect the list? (6) Is the list as published?
(6) Do any of the patients need a pregnancy test? (7) Do the patients have group and save or crossmatched blood if required? (8) Any other issues?
(7) Are radiographers required for the procedure? (8) Do any of the patients require a pregnancy test? (9) Any issues troubling team members? Debriefing (1) What went well today? (2) Did anyone have any concerns above today’s list? (3) Were there any specific equipment issues that need to be addressed before the next list? (4) Is there anything we could do to make the list safer? (5) Is there anything we could do to make the list more productive? Total 14 questions
penile or diagnostic endoscopic surgery). The answer given to each question and whether the question was asked were recorded. The compliance was expressed as a percentage of the checklist questions that were asked, divided by the total checklist questions that should have been asked. Data recording was performed by the same observer throughout the audit. The purpose of the observer was not disclosed to theatre staff.
Checklist revision The results of the pre-revision observations were presented to the departmental meeting with a detailed breakdown of compliance and response for each question on the checklist. The checklist questions were reviewed with all members of the theatre team and the most important questions identified by discussion. The revised checklist was implemented in accordance with local clinical governance policy.
Compliance, timing and safety The time taken for the checklist was recorded from the start of the checklist until the last question was
Debriefing (1) What went well? (2) What could be improved on?
Total 10 questions
asked or when it became apparent that no more questions were forthcoming. Incorrect drug administration, wrong site surgery and any adverse safety outcomes were noted.
Statistical analysis The classification of operations observed before and after checklist simplification was compared between groups using the chi-square test. The median time of the BD and STS checklists were compared between the original and revised checklist with the Mann—Whitney U test. The compliance rate of the BD and STS checklists was compared between the original and revised checklist with Fisher’s exact test (Graphpad Prism version 5.00 for Windows, GraphPad Software, San Diego, CA, USA). Statistical significance was taken as p < 0.05.
Results One hundred urological operations were audited with a varied case mix (Table 3). The difference between the classification of operation observed was not statistically significant (p = 0.30).
A simplified WHO checklist improves compliance and time efficiency for urological surgery Table 2
123
Simplified 10-point STS checklist.
Original checklist
Revised checklist
Sign in (1) Has the patient confirmed his/her identity, site, procedure and consent? (2) Is the surgical site marked? (3) Does the patient have a known allergy? (4) Difficult airway or aspiration risk? (5) Risk of >500 ml blood loss? Pre-operative time out (1) All members of the surgical team confirm the patient’s name? (2) All members confirm the planned procedure and side (3) Has the patient been positioned safely and appropriately? (4) Had appropriate skin preparation without pooling? (5) Diathermy plate applied and appropriate? (6) Surgeon are we prepared for anticipated blood loss? (7) Are there any special equipment requirements or concerns? (8) Are there any critical steps or decision point the team need to know about (what is plan B)? (9) Anaesthetist—–any specific concerns? (10) What is the patient’s ASA score? (11) Do we have all monitoring and specific support necessary? (12) Scrub nurse has sterility been confirmed? (13) Has equipment been set up under laminar flow? (14) Are there any specific equipment concerns? (15) Has the Surgical Site Infection (SSI) bundle been undertaken? (16) Antibiotics within <60 min (17) Hair removal from site? (18) Patient warming? (19) Glycaemic control (20) Is appropriate VTE prophylaxis in place/planned? (21) Is essential imaging displayed? Post-operative Sign-Out (1) Has the name of the procedure been recorded? (2) Has the instrument, swabs and sharps count been confirmed as correct? (3) Are all specimens labelled? (4) Have the key concerns for recovery and post-operative management of this patient been discussed? (5) Have any equipment issues arisen that need to be addressed? Total 31 questions
Compliance The analysis of the existing checklist was performed for 50 cases over 11 operating lists. The 14 BD
Theatre pre-op (1) Read name and hospital number from wristband confirm against consent form and op list (2) What operation? (3) Does the patient have an allergy? (4) What antibiotic and dose has been given? (5) Are TEDs and/or flowtrons on patient? If lateralising procedure; (6) Which side/site is operation? (7) Does that agree with marking and imaging?
Theatre post-op (1) What operation was performed? (2) Is the count correct? (3) Are specimens labelled?
Total 10 questions
checklist questions gave a total of 154 possible questions. The compliance rate for the existing checklist was 68% (105/154) for BD questions. After simplification, analysis of 50 cases over 11 oper-
124
J. Henderson et al.
Table 3 Case mix of audits (chi-square test p = 0.30— –not significant). Pre-modification
Post-modification
Major Intermediate Minor
5 28 17
4 22 24
Total
50
50
ating lists was performed. The 10 BD checklist questions gave a total of 110 possible questions. The compliance for the simplified BD checklist was not significantly different (p = 0.49) at 73% (80/110). For the existing 31-question STS checklist, there were 1550 possible questions for 50 cases. For the simplified checklist 50 cases, each with 10 STS questions, there were 500 possible questions. Thirty-six of these were procedures with no lateralisation, making questions 6 and 7 inapplicable and 25 had no specimen making theatre post-op question 3 inapplicable. The total number of possible questions therefore was 403. The compliance was 53% (823/1550) for STS questions on the original checklist which improved significantly (p < 0.0001) to 92% (369/403) after simplification (Fig. 3). During progression down the longer lists of questions, the compliance decreased with a linear trend (Fig. 4).
Time The median time taken was 150 s for BD and 88 s for the STS (Fig. 5). After the checklist was simplified, this decreased to 90 s and 48 s respectively (p < 0.05).
Compliance Pre and Post Reduction 100
Compliance (%)
p=0.49
p<0.0001
80 60 40 20 0 BD pre
BD post
STS pre
STS post
Domain
Figure 3 Compliance rates before and after checklist reduction with p values.
Safety During the initial audit, the question ‘Antibiotics within <60 minutes?’ was asked 80% (40/50) of the time. Of these patients who were checked, 17% (7/40) received the wrong antibiotic and/or wrong dose. This question was rephrased to ‘What antibiotic and dose has been given?’ The check was completed in 90% (45/50) of cases, of which none received incorrect antibiotic or dose. There were no other adverse safety outcomes.
Discussion and conclusions The evidence that using a surgical checklist improves outcomes is convincing and we support its use. This study demonstrates that compliance to the existing checklist was poor and it took a significant amount of time to ask the questions. The introduction of a simplified checklist improved compliance and decreased the time to completion. The greatest improvement in compliance was seen with the STS section. The number of questions in this section decreased by two-thirds and this was likely to have been a causative factor. The recording of checklist use was carried out by a single observer and so no inter-observer data is presented. The observer was present for all parts of the WHO checklist. The presence of an observer was not disclosed to theatre staff but some may have been aware that they were being observed. The ‘Hawthorne effect’ refers to the increase in performance of people who are aware that they are being observed [15]. Theatre staff were involved in the revision of the checklist, at which point a greater awareness of being observed was likely which may have skewed the Hawthorne effect improving compliance with the simplified checklist. There is a degree of subjectivity in recording the observations and the reliance on a single observer may have introduced a source of error. Knowledge of the study hypothesis was likely and a bias towards supporting the hypothesis was possible. The issue of which questions to include in checklists is important. In the airline industry, reference is made to the ‘shoulder harness’ question [12]. Proponents of an exhaustive checklist would argue that the pilot checking he has fastened his harness is essential for safety. Observers of human factors and behaviour note that fastening the shoulder harness before takeoff is such an integral part of flying, that its inclusion on a checklist increases the chance of the check being ignored or skipped and missing more important checks. A number of
A simplified WHO checklist improves compliance and time efficiency for urological surgery
125
Percentage of Time Question Asked for Pre-operativeTimeout 100% 90% 80% 70% 60% 50% 40% 30% 20% 10% 0% 1
2
3
4
5
6
7
8
9 10 11 12 13 14 15 16 17 18 19 20 21
Question number
Figure 4 Compliance of questions asked during pre-operative timeout.
questions on the checklist were identified which were considered to be ‘shoulder harness’ questions. In discussion with all members of the surgical team, and following local clinical governance procedures, these were removed. The WHO checklist was designed to be applicable worldwide, across a variety of socioeconomic
and clinical environments. The advantage of this uniformity is a consistent format that is well recognised. The disadvantage is that some questions are asked that may be more applicable in one environment than others. An example of this is the question regarding sterility of instruments. In a healthcare environment with limited access to sterilization
Timing Pre and Post Checklist Reduction
Time (seconds)
150
100 p=0.02
50 p<0.0001
st Po ST S
Pr e ST S
Po s D B
B
D
Pr e
t
0
Domain
Figure 5 Median (point) and interquartile range (bars) of checklist timing with p values.
126 equipment, this question would be of paramount importance. However, in the modern operating theatre, the use of sterile instruments is universal and its inclusion detracts from the usefulness of the checklist. In the United Kingdom, patients are anaesthetised in a dedicated ‘anaesthetic room’ whereas in the USA this is often carried out in theatre. The Operating Department Practitioners (ODPs) would carry out a patient identity check before anaesthesia but this would rarely be as part of the checklist as the majority of the theatre team would not be in the anaesthetic room. We encouraged this practice to continue but made the final check in theatre before the commencement of surgery. During the modification of the checklist, trigger points and ‘challenge-response’ techniques were employed. Questions that could be answered or dismissed with a one word answer were rephrased to encourage checking. One example was the question ‘Can all members of the surgical team confirm that this is patient x?’ Team members would often respond ‘yes’ with no checking of wristband or other identification. This question was rephrased to ‘Read name and hospital number from wristband and confirm against consent form and operating list’. References to skin incision and closure were removed in the recognition that a large proportion of urological surgery is carried out endoscopically. The rephrasing of the antibiotic check question resulted in no incorrect antibiotic administration. There were no other adverse safety outcomes observed although the low incidence of these events would make seeing any meaningful change in this small study very unlikely. Despite improvements in compliance and timeefficiency, these fell short of the ideal one hundred percent. In a survey of all UK centres, a lack of engagement (77%) and a view of the checklist as a ‘tick box exercise’ (78%) are cited as challenges to implementation [9]. Our experience supported these findings, with some staff displaying a negative or hostile attitude towards the checklist. This represents a significant challenge, but a focus on training and education may help address this. In the airline industry, Crew Resource Management (CRM) teaching has been used for many years. This focuses on the communication, teamwork and decision making skills required to reduce errors in an organised system. The formal teaching of CRM style skills to theatre staff has been shown to improve non-technical skills and attitudes [16]. Vats et al. report a compliance rate of any use of the checklist of 42—80% during a pilot period [17]. The assumption made was that this is due to unfamiliarity which would improve with time. It may be
J. Henderson et al. that the low compliance was due to the checklist being too long and not appropriate for the environment in which it was implemented. We have shown that a simplified checklist results in improved compliance and time efficiency.
Conclusions The balance between comprehensiveness and practicability was recognised at the planning stage of the WHO checklist. The adaptation by the Patient Safety First group and addition of the BD questions progressively elongated the checklist to a point at which its completion became a challenge during a normal operating list. This study demonstrates that compliance and time efficiency improve with a simplified, shorter checklist. Further research is needed to establish whether this effect is sustained long-term and is reproducible in different healthcare environments.
Conflict of interest statement The authors declare that there are no conflicts of interest.
Acknowledgements The authors gratefully acknowledge figures and background information supplied by Mr P. Stevenson and Mr J. Reynard.
References [1] Haynes AB, Weiser TG, Berry WR, Lipsitz SR, Breizat AS, Dellinger EP, et al. A surgical safety checklist to reduce morbidity and mortality in a global population. N Engl J Med 2009;360:491—9. [2] Nundy S, Mukherjee A, Sexton B, Pronovost PJ, Knight A, Rowen LC, et al. Impact of preoperative briefings on operating room delays. Arch Surg 2008;143(11):1068—72. [3] Lingard L, Regehr G, Orser B, Reznick R, Baker R, Doran D, et al. Evaluation of a preoperative checklist and team briefing among surgeons nurses and anesthesiologists to reduce failures in communication. Arch Surg 2008;143(1):12—7. [4] Makary M, Mukherjee A, Sexton J, Syin D, Goodrich E, Hartmann E, et al. Operating room briefings and wrong-site surgery. J Am Coll Surg 2007;204:236—43. [5] National Patient Safety Agency. Patient safety alert update: WHO surgical safety checklist; January 2009. [6] Weiser TG, Haynes AB, Lashoher A, Dziekan G, Boorman DJ, Berry WR, et al. Perspectives in quality: designing the WHO surgical safety checklist. Int J Qual Health Care 2010;22(5):365—70. [7] Weiser T. Safe surgery saves lives international consultation meeting. Summary of the principle discussion points,
A simplified WHO checklist improves compliance and time efficiency for urological surgery
[8] [9]
[10]
[11]
[12]
http://www.who.int/patientsafety/challenge/safe. surgery/Consultation%20Meeting%20Summary.pdf [accessed 24.05.11]. Gawande A. The checklist manifesto. London: Profile Books; 2010. Patient Safety First. Implementing the surgical safety checklist: the journey so far, http://www.patientsafetyfirst.nhs.uk/ashx/Asset.ashx?path =/Implementing%20the%20Surgical%20Safety%20Checklist%20%20the%20journey%20so%20far%202010.06.21%20FINAL.pdf [accessed 24.05.11]. Sanders RD, Jameson SS. Letter to editor: a surgical safety checklist, http://www.nejm.org/doi/full/10. 1056/NEJMc090417. National Transportation Safety Board. Aircraft accident report: Northwest Airlines McDonnell Douglas DC-9-82; 1988. http://www.ntsb.gov/publictn/1988/AAR8805.htm [accessed 24.05.11]. Degani A. Cockpit checklists: concepts, design, and use. Hum Factors 2003;35(2):28—43.
127
[13] National Patient Safety Agency. WHO surgical safety checklist. Version 1, http://www.nrls.npsa.nhs. uk/EasySiteWeb/getresource. axd?AssetID=61388&type=full&servicetype=Attachment [accessed 24.05.11]. [14] National Patient Safety Agency. WHO surgical safety checklist: supporting information, http://www.nrls.npsa.nhs.uk/EasySiteWeb/getresource. axd?AssetID=60217&type=full&servicetype=Attachment [accessed 24.05.11]. [15] Mayo E. The human problems of an industrial civilization. New York: Macmillan; 1933. [16] McCulloch P, Mishra A, Handa A, Dale T, Hirst G, Catchpole K. The effects of aviation-style non-technical skills training on technical performance and outcome in the operating theatre. Qual Saf Health Care 2009;18:109—15. [17] Vats A, Vincent CA, Nagpal K, Davies RW, Darzi A, Moorthy K. Practical challenges of introducing WHO surgical checklist: UK pilot experience. BMJ 2010;340:133—5.
Available online at www.sciencedirect.com
ORIGINAL ARTICLE
A simplified WHO checklist improves compliance and time efficiency for urological surgery John Henderson ∗, Timothy Fung, Jaimin Bhatt, Amarjit Bdesha Department of Urology, Wycombe General Hospital, Queen Alexandra Road, High Wycombe, Bucks HP11 2TT, UK Received 29 March 2011 ; received in revised form 4 August 2011; accepted 16 August 2011
KEYWORDS WHO; Checklist; Clinical governance
Abstract Objective: A surgical safety checklist has been introduced throughout the UK in response to evidence that its use decreases surgical morbidity and mortality. The adaptation of this has resulted in a lengthy checklist which may be used improperly. We audited the existing and a new simplified checklist with regards to compliance, time-efficiency and relevant safety outcomes. Materials and methods: One hundred operations were observed. Fifty of these cases used the existing 14-question Briefing/Debriefing (BD) and 31-question Sign-in, Timeout and Sign-Out (STS) checklist. The subsequent 50 used a simplified 10-question BD and STS checklist. Percentage compliance, median time taken and relevant safety outcomes were recorded. Results: The median time for the BD questions decreased from 150 to 90 s and the STS questions from 88 to 49 s (p < 0.05). The compliance improved from 68% to 73% (p = 0.49) for BD questions and 53% to 92% (p < 0.05) for STS questions. A clearer phrasing of the antibiotic check question in the revised checklist resulted in no administration of incorrect antibiotics. Conclusion: The nuclear and airline industries have used checklists for many years and observed that long and exhaustive checklists were often used improperly or disregarded completely. We demonstrate that a redesigned, simplified checklist improves time-efficiency and compliance with improved safety outcomes. © 2011 British Association of Urological Surgeons. Published by Elsevier Ltd. All rights reserved.
∗ Corresponding author. Tel.: +01494 526161; fax: +44 01494 426001. E-mail addresses: john [email protected] (J. Henderson), [email protected] (T. Fung), [email protected] (J. Bhatt), [email protected] (A. Bdesha).
Introduction The World Health Organisation (WHO) advocates the introduction of a pre-operative surgical checklist as part of its ‘Safe Surgery Saves Lives’
1875-9742/$ — see front matter © 2011 British Association of Urological Surgeons. Published by Elsevier Ltd. All rights reserved. doi:10.1016/j.bjmsu.2011.08.003
A simplified WHO checklist improves compliance and time efficiency for urological surgery initiative. This has been shown to decrease operative morbidity and in-hospital mortality across a wide range of economic and clinical environments [1]. The use of a checklist may reduce operating room delays [2], communication failures [3] and risk of wrong site surgery [4]. The National Patient Safety Agency (NPSA) issued an alert in January 2009 requiring the implementation of an adapted checklist for all surgical procedures in NHS Trusts in England and Wales by February 2010 [5]. The design of the checklist drew heavily on the experiences of the airline industry whose use of checklists extends to 70 years. It was recognised that the length of the checklist would have to be a compromise between brevity and detail [6]. Concerns were raised during the consultation stage that it ‘would be impractical in short cases and especially during day cases with rapid turnover of patients’ [7]. Atul Gawande, the WHO ‘Safe Surgery Saves Lives’ lead, admits ‘The most intense disagreements flared over what should stay in and what should come out’ [8]. After implementation in the UK, 37% of centres surveyed suggested that ‘not having enough time’ was a barrier to implementation [9]. The issue of prolonged anaesthesia during checklist completion was also raised [10]. After a number of fatal air crashes in the 1980s, the airline industry was forced to analyse its use of checklists. A Northwest Airlines Flight crashed in 1987 with loss of 156 lives after executing a ‘no flaps’ takeoff. The National Transportation Safety Board (NTSB) investigation found that ‘neither pilot recited the items of the taxi checklist’ [11]. Analysis of pilot behaviour showed that with longer checklists, pilots would skip items, group them together and sometimes miss them altogether [12]. There followed a trend towards checklist reduction, incorporating ‘killer’ items only—–those which if missed would have the most serious consequences. An example of a 22-item checklist from 1977 is shown (Fig. 1) with a simplified 9-item checklist from 2009 (Fig. 2). The checklist provided by the NPSA is 28 questions long [13]. Adaptation of the checklist was allowed ‘locally or speciality specifically through local clinical governance arrangements to ensure organisational accountability’ [14]. As suggested by the ‘Patient Safety First’ group, our centre adapted the checklist into separate ‘Briefing and Debriefing (BD)’ and ‘Sign in, Time out and Sign out (STS)’ sections. The former was performed at the beginning and end of the list with 14 questions and the latter for each individual operation with 31 questions. We considered whether a simplified checklist, incorporating only the most important questions,
121
Figure 1 22-Item pre flight checklist from 1977 [courtesy of P. Stevenson].
would result in shorter completion times and higher completion rates.
Methods Observation A total of 100 urological operations with a variable case mix were observed. The first 50 used the existing WHO checklist, and the remaining 50 used a simplified 10-point checklist for each of the BD (Table 1) and STS (Table 2) sections. The operation observed was classified as major (laparoscopic/open organ removal or PCNL), intermediate (endoscopic ureterorenoscopic, bladder, prostate or urethral surgery) or minor (open inguinoscrotal,
Figure 2 9-Item pre flight checklist from 2009 [courtesy of P. Stevenson].
122 Table 1
J. Henderson et al. Original and simplified 10-point Briefing/Debriefing (BD) checklist.
Original checklist
Revised checklist
Briefing (1) Have all team members been introduced by name and role? (2) Is the anaesthetic equipment check complete? (3) Are there any equipment issues for any of the cases?
Briefing (1) Everyone to introduce themselves-name and role (2) Anaesthetic check complete? (3) Any equipment concerns for any cases and are radiographers needed? (4) Any emergency cases to do or anything expected which will disrupt the list? (5) Is the order of the list correct?
(4) Any specific issues with any of the patients that we all need to know about (5) Anything expected during the day which will affect the list? (6) Is the list as published?
(6) Do any of the patients need a pregnancy test? (7) Do the patients have group and save or crossmatched blood if required? (8) Any other issues?
(7) Are radiographers required for the procedure? (8) Do any of the patients require a pregnancy test? (9) Any issues troubling team members? Debriefing (1) What went well today? (2) Did anyone have any concerns above today’s list? (3) Were there any specific equipment issues that need to be addressed before the next list? (4) Is there anything we could do to make the list safer? (5) Is there anything we could do to make the list more productive? Total 14 questions
penile or diagnostic endoscopic surgery). The answer given to each question and whether the question was asked were recorded. The compliance was expressed as a percentage of the checklist questions that were asked, divided by the total checklist questions that should have been asked. Data recording was performed by the same observer throughout the audit. The purpose of the observer was not disclosed to theatre staff.
Checklist revision The results of the pre-revision observations were presented to the departmental meeting with a detailed breakdown of compliance and response for each question on the checklist. The checklist questions were reviewed with all members of the theatre team and the most important questions identified by discussion. The revised checklist was implemented in accordance with local clinical governance policy.
Compliance, timing and safety The time taken for the checklist was recorded from the start of the checklist until the last question was
Debriefing (1) What went well? (2) What could be improved on?
Total 10 questions
asked or when it became apparent that no more questions were forthcoming. Incorrect drug administration, wrong site surgery and any adverse safety outcomes were noted.
Statistical analysis The classification of operations observed before and after checklist simplification was compared between groups using the chi-square test. The median time of the BD and STS checklists were compared between the original and revised checklist with the Mann—Whitney U test. The compliance rate of the BD and STS checklists was compared between the original and revised checklist with Fisher’s exact test (Graphpad Prism version 5.00 for Windows, GraphPad Software, San Diego, CA, USA). Statistical significance was taken as p < 0.05.
Results One hundred urological operations were audited with a varied case mix (Table 3). The difference between the classification of operation observed was not statistically significant (p = 0.30).
A simplified WHO checklist improves compliance and time efficiency for urological surgery Table 2
123
Simplified 10-point STS checklist.
Original checklist
Revised checklist
Sign in (1) Has the patient confirmed his/her identity, site, procedure and consent? (2) Is the surgical site marked? (3) Does the patient have a known allergy? (4) Difficult airway or aspiration risk? (5) Risk of >500 ml blood loss? Pre-operative time out (1) All members of the surgical team confirm the patient’s name? (2) All members confirm the planned procedure and side (3) Has the patient been positioned safely and appropriately? (4) Had appropriate skin preparation without pooling? (5) Diathermy plate applied and appropriate? (6) Surgeon are we prepared for anticipated blood loss? (7) Are there any special equipment requirements or concerns? (8) Are there any critical steps or decision point the team need to know about (what is plan B)? (9) Anaesthetist—–any specific concerns? (10) What is the patient’s ASA score? (11) Do we have all monitoring and specific support necessary? (12) Scrub nurse has sterility been confirmed? (13) Has equipment been set up under laminar flow? (14) Are there any specific equipment concerns? (15) Has the Surgical Site Infection (SSI) bundle been undertaken? (16) Antibiotics within <60 min (17) Hair removal from site? (18) Patient warming? (19) Glycaemic control (20) Is appropriate VTE prophylaxis in place/planned? (21) Is essential imaging displayed? Post-operative Sign-Out (1) Has the name of the procedure been recorded? (2) Has the instrument, swabs and sharps count been confirmed as correct? (3) Are all specimens labelled? (4) Have the key concerns for recovery and post-operative management of this patient been discussed? (5) Have any equipment issues arisen that need to be addressed? Total 31 questions
Compliance The analysis of the existing checklist was performed for 50 cases over 11 operating lists. The 14 BD
Theatre pre-op (1) Read name and hospital number from wristband confirm against consent form and op list (2) What operation? (3) Does the patient have an allergy? (4) What antibiotic and dose has been given? (5) Are TEDs and/or flowtrons on patient? If lateralising procedure; (6) Which side/site is operation? (7) Does that agree with marking and imaging?
Theatre post-op (1) What operation was performed? (2) Is the count correct? (3) Are specimens labelled?
Total 10 questions
checklist questions gave a total of 154 possible questions. The compliance rate for the existing checklist was 68% (105/154) for BD questions. After simplification, analysis of 50 cases over 11 oper-
124
J. Henderson et al.
Table 3 Case mix of audits (chi-square test p = 0.30— –not significant). Pre-modification
Post-modification
Major Intermediate Minor
5 28 17
4 22 24
Total
50
50
ating lists was performed. The 10 BD checklist questions gave a total of 110 possible questions. The compliance for the simplified BD checklist was not significantly different (p = 0.49) at 73% (80/110). For the existing 31-question STS checklist, there were 1550 possible questions for 50 cases. For the simplified checklist 50 cases, each with 10 STS questions, there were 500 possible questions. Thirty-six of these were procedures with no lateralisation, making questions 6 and 7 inapplicable and 25 had no specimen making theatre post-op question 3 inapplicable. The total number of possible questions therefore was 403. The compliance was 53% (823/1550) for STS questions on the original checklist which improved significantly (p < 0.0001) to 92% (369/403) after simplification (Fig. 3). During progression down the longer lists of questions, the compliance decreased with a linear trend (Fig. 4).
Time The median time taken was 150 s for BD and 88 s for the STS (Fig. 5). After the checklist was simplified, this decreased to 90 s and 48 s respectively (p < 0.05).
Compliance Pre and Post Reduction 100
Compliance (%)
p=0.49
p<0.0001
80 60 40 20 0 BD pre
BD post
STS pre
STS post
Domain
Figure 3 Compliance rates before and after checklist reduction with p values.
Safety During the initial audit, the question ‘Antibiotics within <60 minutes?’ was asked 80% (40/50) of the time. Of these patients who were checked, 17% (7/40) received the wrong antibiotic and/or wrong dose. This question was rephrased to ‘What antibiotic and dose has been given?’ The check was completed in 90% (45/50) of cases, of which none received incorrect antibiotic or dose. There were no other adverse safety outcomes.
Discussion and conclusions The evidence that using a surgical checklist improves outcomes is convincing and we support its use. This study demonstrates that compliance to the existing checklist was poor and it took a significant amount of time to ask the questions. The introduction of a simplified checklist improved compliance and decreased the time to completion. The greatest improvement in compliance was seen with the STS section. The number of questions in this section decreased by two-thirds and this was likely to have been a causative factor. The recording of checklist use was carried out by a single observer and so no inter-observer data is presented. The observer was present for all parts of the WHO checklist. The presence of an observer was not disclosed to theatre staff but some may have been aware that they were being observed. The ‘Hawthorne effect’ refers to the increase in performance of people who are aware that they are being observed [15]. Theatre staff were involved in the revision of the checklist, at which point a greater awareness of being observed was likely which may have skewed the Hawthorne effect improving compliance with the simplified checklist. There is a degree of subjectivity in recording the observations and the reliance on a single observer may have introduced a source of error. Knowledge of the study hypothesis was likely and a bias towards supporting the hypothesis was possible. The issue of which questions to include in checklists is important. In the airline industry, reference is made to the ‘shoulder harness’ question [12]. Proponents of an exhaustive checklist would argue that the pilot checking he has fastened his harness is essential for safety. Observers of human factors and behaviour note that fastening the shoulder harness before takeoff is such an integral part of flying, that its inclusion on a checklist increases the chance of the check being ignored or skipped and missing more important checks. A number of
A simplified WHO checklist improves compliance and time efficiency for urological surgery
125
Percentage of Time Question Asked for Pre-operativeTimeout 100% 90% 80% 70% 60% 50% 40% 30% 20% 10% 0% 1
2
3
4
5
6
7
8
9 10 11 12 13 14 15 16 17 18 19 20 21
Question number
Figure 4 Compliance of questions asked during pre-operative timeout.
questions on the checklist were identified which were considered to be ‘shoulder harness’ questions. In discussion with all members of the surgical team, and following local clinical governance procedures, these were removed. The WHO checklist was designed to be applicable worldwide, across a variety of socioeconomic
and clinical environments. The advantage of this uniformity is a consistent format that is well recognised. The disadvantage is that some questions are asked that may be more applicable in one environment than others. An example of this is the question regarding sterility of instruments. In a healthcare environment with limited access to sterilization
Timing Pre and Post Checklist Reduction
Time (seconds)
150
100 p=0.02
50 p<0.0001
st Po ST S
Pr e ST S
Po s D B
B
D
Pr e
t
0
Domain
Figure 5 Median (point) and interquartile range (bars) of checklist timing with p values.
126 equipment, this question would be of paramount importance. However, in the modern operating theatre, the use of sterile instruments is universal and its inclusion detracts from the usefulness of the checklist. In the United Kingdom, patients are anaesthetised in a dedicated ‘anaesthetic room’ whereas in the USA this is often carried out in theatre. The Operating Department Practitioners (ODPs) would carry out a patient identity check before anaesthesia but this would rarely be as part of the checklist as the majority of the theatre team would not be in the anaesthetic room. We encouraged this practice to continue but made the final check in theatre before the commencement of surgery. During the modification of the checklist, trigger points and ‘challenge-response’ techniques were employed. Questions that could be answered or dismissed with a one word answer were rephrased to encourage checking. One example was the question ‘Can all members of the surgical team confirm that this is patient x?’ Team members would often respond ‘yes’ with no checking of wristband or other identification. This question was rephrased to ‘Read name and hospital number from wristband and confirm against consent form and operating list’. References to skin incision and closure were removed in the recognition that a large proportion of urological surgery is carried out endoscopically. The rephrasing of the antibiotic check question resulted in no incorrect antibiotic administration. There were no other adverse safety outcomes observed although the low incidence of these events would make seeing any meaningful change in this small study very unlikely. Despite improvements in compliance and timeefficiency, these fell short of the ideal one hundred percent. In a survey of all UK centres, a lack of engagement (77%) and a view of the checklist as a ‘tick box exercise’ (78%) are cited as challenges to implementation [9]. Our experience supported these findings, with some staff displaying a negative or hostile attitude towards the checklist. This represents a significant challenge, but a focus on training and education may help address this. In the airline industry, Crew Resource Management (CRM) teaching has been used for many years. This focuses on the communication, teamwork and decision making skills required to reduce errors in an organised system. The formal teaching of CRM style skills to theatre staff has been shown to improve non-technical skills and attitudes [16]. Vats et al. report a compliance rate of any use of the checklist of 42—80% during a pilot period [17]. The assumption made was that this is due to unfamiliarity which would improve with time. It may be
J. Henderson et al. that the low compliance was due to the checklist being too long and not appropriate for the environment in which it was implemented. We have shown that a simplified checklist results in improved compliance and time efficiency.
Conclusions The balance between comprehensiveness and practicability was recognised at the planning stage of the WHO checklist. The adaptation by the Patient Safety First group and addition of the BD questions progressively elongated the checklist to a point at which its completion became a challenge during a normal operating list. This study demonstrates that compliance and time efficiency improve with a simplified, shorter checklist. Further research is needed to establish whether this effect is sustained long-term and is reproducible in different healthcare environments.
Conflict of interest statement The authors declare that there are no conflicts of interest.
Acknowledgements The authors gratefully acknowledge figures and background information supplied by Mr P. Stevenson and Mr J. Reynard.
References [1] Haynes AB, Weiser TG, Berry WR, Lipsitz SR, Breizat AS, Dellinger EP, et al. A surgical safety checklist to reduce morbidity and mortality in a global population. N Engl J Med 2009;360:491—9. [2] Nundy S, Mukherjee A, Sexton B, Pronovost PJ, Knight A, Rowen LC, et al. Impact of preoperative briefings on operating room delays. Arch Surg 2008;143(11):1068—72. [3] Lingard L, Regehr G, Orser B, Reznick R, Baker R, Doran D, et al. Evaluation of a preoperative checklist and team briefing among surgeons nurses and anesthesiologists to reduce failures in communication. Arch Surg 2008;143(1):12—7. [4] Makary M, Mukherjee A, Sexton J, Syin D, Goodrich E, Hartmann E, et al. Operating room briefings and wrong-site surgery. J Am Coll Surg 2007;204:236—43. [5] National Patient Safety Agency. Patient safety alert update: WHO surgical safety checklist; January 2009. [6] Weiser TG, Haynes AB, Lashoher A, Dziekan G, Boorman DJ, Berry WR, et al. Perspectives in quality: designing the WHO surgical safety checklist. Int J Qual Health Care 2010;22(5):365—70. [7] Weiser T. Safe surgery saves lives international consultation meeting. Summary of the principle discussion points,
A simplified WHO checklist improves compliance and time efficiency for urological surgery
[8] [9]
[10]
[11]
[12]
http://www.who.int/patientsafety/challenge/safe. surgery/Consultation%20Meeting%20Summary.pdf [accessed 24.05.11]. Gawande A. The checklist manifesto. London: Profile Books; 2010. Patient Safety First. Implementing the surgical safety checklist: the journey so far, http://www.patientsafetyfirst.nhs.uk/ashx/Asset.ashx?path =/Implementing%20the%20Surgical%20Safety%20Checklist%20%20the%20journey%20so%20far%202010.06.21%20FINAL.pdf [accessed 24.05.11]. Sanders RD, Jameson SS. Letter to editor: a surgical safety checklist, http://www.nejm.org/doi/full/10. 1056/NEJMc090417. National Transportation Safety Board. Aircraft accident report: Northwest Airlines McDonnell Douglas DC-9-82; 1988. http://www.ntsb.gov/publictn/1988/AAR8805.htm [accessed 24.05.11]. Degani A. Cockpit checklists: concepts, design, and use. Hum Factors 2003;35(2):28—43.
127
[13] National Patient Safety Agency. WHO surgical safety checklist. Version 1, http://www.nrls.npsa.nhs. uk/EasySiteWeb/getresource. axd?AssetID=61388&type=full&servicetype=Attachment [accessed 24.05.11]. [14] National Patient Safety Agency. WHO surgical safety checklist: supporting information, http://www.nrls.npsa.nhs.uk/EasySiteWeb/getresource. axd?AssetID=60217&type=full&servicetype=Attachment [accessed 24.05.11]. [15] Mayo E. The human problems of an industrial civilization. New York: Macmillan; 1933. [16] McCulloch P, Mishra A, Handa A, Dale T, Hirst G, Catchpole K. The effects of aviation-style non-technical skills training on technical performance and outcome in the operating theatre. Qual Saf Health Care 2009;18:109—15. [17] Vats A, Vincent CA, Nagpal K, Davies RW, Darzi A, Moorthy K. Practical challenges of introducing WHO surgical checklist: UK pilot experience. BMJ 2010;340:133—5.
Available online at www.sciencedirect.com