- Email: [email protected]
Implementation of an Emergency Department Sign-Out Checklist Improves Transfer of Information at Shift Change
The Journal of Emergency Medicine, Vol. 47, No. 5, pp. 580–585, 2014 Copyright Ó 2014 Elsevier Inc. Printed in the USA. All rights reserved 0736-4679/$ - see front matter
http://dx.doi.org/10.1016/j.jemermed.2014.06.017
Education IMPLEMENTATION OF AN EMERGENCY DEPARTMENT SIGN-OUT CHECKLIST IMPROVES TRANSFER OF INFORMATION AT SHIFT CHANGE Nicole M. Dubosh, MD,*† Dylan Carney, MD, MS,‡ Jonathan Fisher, MD, MPH,†* and Carrie D. Tibbles, MD*† *Department of Emergency Medicine, Beth Israel Deaconess Medical Center, Boston, Massachusetts, †Harvard Medical School, Boston, Massachusetts, and ‡Department of Emergency Medicine, University of California at San Francisco, San Francisco, CA Reprint Address: Nicole M. Dubosh, MD, Department of Emergency Medicine, Beth Israel Deaconess Medical Center, One Deaconess Road, W-CC2, Boston, MA 02215
, Abstract—Background: Transitions of care are ubiquitous in the emergency department (ED) and inevitably introduce the opportunity for errors. Few emergency medicine residency programs provide formal training or a standard process for patient handoffs. Checklists have been shown to be effective quality-improvement measures in inpatient settings and may be a feasible method to improve ED handoffs. Objective: To determine if the use of a sign-out checklist improves the accuracy and efficiency of resident sign-out in the ED. Methods: A prospective pre-/postinterventional study of residents rotating in the ED at a tertiary academic medical center. Trained research assistants observed resident sign-out during shift change over a 2-week period and completed a data collection tool to indicate whether or not key components of sign-out occurred and time to sign out each patient. An electronic sign-out checklist was implemented using a multi-faceted educational effort. A 2-week postintervention observation phase was conducted. Proportions, means, and nonparametric comparison tests were calculated using STATA. Results: One hundred fifteen sign-outs were observed prior to checklist implementation and 114 were observed after. Significant improvements were seen in four sign-out components: reporting of history of present illness increased from 81% to 99%, ED course increased from 75% to 86%, likely diagnosis increased from 60% to 77%, and team awareness of plan increased
from 21% to 41%. Use of the repeat-back technique decreased from 13% to 5% after checklist implementation and time to sign-out showed no significant change. Conclusion: Implementation of a checklist improved the transfer of information without increasing time to signout. Ó 2014 Elsevier Inc. , Keywords—sign-out; handoffs; checklist; emergency department; resident education
INTRODUCTION Emergency care is characterized by multiple transitions in patient responsibilities, both as patients leave the emergency department (ED) and as providers change shifts. Patients in an ED setting are often complex, requiring ongoing assessment and multiple interventions. Communication breakdowns among physicians are a commonly recognized source of error and are associated with 61% of sentinel events (1). Patient handoffs are times when this is particularly true, as transitions of care create the potential for critical information to be missed or distorted (2–5). Recently, much attention has been paid to improving the transfer of information during transitions of care of patients in all clinical settings, particularly among residents (6–8). Improving and standardizing transitions of care was a Joint Commission national patient safety goal in 2009 (9). In addition, the Quality Improvement and Patient Safety Section of the American College of Emergency
This project was funded by the 2011 Massachusetts American College of Emergency Physicians (MACEP) Emergency Medicine Resident Grant.
RECEIVED: 22 August 2012; FINAL SUBMISSION RECEIVED: 25 April 2014; ACCEPTED: 30 June 2014 580
ED Sign-Out Checklist
Physicians identified sign-out as a target for patient safety, particularly in the ED setting (10). Although sign-outs happen on a frequent basis in the ED, very few studies have examined how to best standardize and improve this process. Checklists have been shown to be effective in several different clinical settings in terms of decreasing medical errors and morbidity, and may be a feasible option in an ED (11,12). In our study, we examined the impact of introducing a standardized sign-out checklist on resident handoffs. Our objective was to determine if the use of a sign-out checklist improves the accuracy and efficiency of resident signout in the ED as measured by reduced omission of key information, communication behaviors, and time to sign out each patient. MATERIALS AND METHODS We performed a prospective evaluation of a novel sign-out checklist to improve ED-to-ED resident physician handoffs at shift change using a pre-/postintervention design. This was a pilot study conducted from November 7, 2011 to December 15, 2011 at a tertiary academic medical center with an annual ED volume of 53,000 visits and an admission rate of 38%. As part of an initiative to improve physician communication and handoffs at shift change, a standardized electronic checklist involving key elements was developed by the study investigators (Figure 1). The components of the checklist were based on existing literature and previously validated tools regarding safe handoffs in other non-ED settings (13–17). The components of the checklist included history of present
Figure 1. Electronic sign-out checklist for residents rotating in the ED. HPI = history of present illness; ED = emergency department; CT = computed tomography scan; Rx = prescription; DI = discharge instructions; POE = provider order entry.
581 illness, ED course, pending studies, likely disposition, anticipated issues, and algorithms for disposition. Several characteristics specific to our institution, including completion of discharge instructions, admission request to the ED observation unit, and ordering of home medications, were also included in the checklist, as the investigators deemed these elements important for streamlining sign-outs. An effort was undertaken to promote the use of the sign-out checklist, including a verbal announcement during resident didactics, and two separate e-mails sent to all ED residents and residents from other specialties rotating in our ED during the study period. The sign-out checklist was also posted in a print version near resident computers in the ED and made available through the ED electronic patient information system. Prior to the implementation of the sign-out checklist, a small team of hired, ED clinical research assistants (RAs) were trained on the elements and use of the checklist in a structured, nonclinical setting. Training included verbal instruction on coding of the checklist elements and resident communication behaviors using the data collection tool. A brief sign-out demonstration video was also created for the purposes of this study, and the RAs were required to watch this during their training. During a 2-week period, a convenience sample of handoffs involving first and second-year residents was observed by the RAs during the afternoon change of shift. Sign-outs involving medical students, attending physicians, and sign-out to inpatient services were excluded. Given the supervisory role of third-year emergency medicine (EM) residents at our institution, sign-outs involving these residents were also excluded. The RAs used a standardized binary data collection form to record whether or not items on the checklist were completed as residents signed out patients. Communication behaviors including use of the repeat-back technique, requests for additional information, and team awareness of plan (resident, attending, and nurse understood and have communicated about the care plan) were also recorded in a binary fashion. Finally, the length of time for each sign-out as well as the specialty and postgraduate year (PGY) of the resident were documented. The RA was not a member of the clinical team nor did he/she participate in the sign-out or serve in a supervisory role. Residents were aware of the RA’s presence but were not privy to the items on which they were being evaluated. One week after the preintervention observation period, the checklist was electronically linked to the ED electronic tracking system for resident use during sign-out. Residents were made aware of its availability at this time, as described above. A 2week period of observation was conducted in a similar fashion to the preimplementation phase, 1 week after the checklist was implemented. All data were entered into Microsoft Access 2003 (Microsoft Corporation, Redmond, WA) and analyzed using STATA 11.0 (College Station, TX). A c2 analysis was performed to compare proportions and a Wilcoxon rank-sum was used to compare means. The project was submitted to the local Institutional Review Board, who determined that our activities were quality improvement and as such do not require further review under federal guidelines. The procedures followed were in accord with the standards on the use of human subjects at the institution where the study was performed.
582
N. M. Dubosh et al.
Table 1. Baseline Resident Characteristics Year PGY 1 PGY 2 Specialty Internal Medicine Emergency Medicine Other specialty Total
Preimplementation n (%)*
Postimplementation n (%)
66 (61.1%) 42 (38.9%)
42 (36.8%) 72 (63.2%)
34 (29.6%) 63 (54.8%)
59 (51.8%) 52 (45.6%)
18 (15.7%) 115
3 (2.6%) 114
PGY = postgraduate year. * Seven preimplementation sign-outs included missing data points for resident year.
RESULTS One hundred fifteen resident sign-outs were observed during the observation period and 114 were observed after checklist implementation. Of the observed sign-outs, 108 were performed by first-year residents and 114 by second-year residents (Table 1). Seven preimplementation sign-outs were missing data for resident PGY year. Of the specialties represented, 115 of the residents were enrolled in the EM residency program, 93 were internal medicine (IM) residents, and 21 were from other specialties including general surgery, pediatric emergency medicine fellowship, and obstetrics and gynecology. Statistically significant improvements were found in four of the 10 sign-out components after implementation of the sign-out checklist (Table 2). Reporting of the history of present illness increased from 81% to 99%, ED course increased from 75% to 86%, likely diagnosis increased from 60% to 77%, and team awareness of
plan increased from 21% to 41%. The use of the repeat back technique decreased from 13% to 5% after the checklist was implemented. Overall, the proportion of sign-outs in which no checklist items were achieved decreased from 11% to 0%, and the proportion signing out 80% or more of the items increased from 18% to 34%. In both groups, the proportion of sign-outs in which all items were performed was negligible. Additionally, the median length of sign-out did not differ between the two groups. We performed a post hoc subgroup analysis to assess interaction between resident year, specialty, and checklist performance. With the exception of requests for additional information (where first-year resident signouts were more likely to involve requests for additional information), there were no statistically significant differences between sign-outs performed by first-year and second-year residents. When stratified by specialty, residents in non-EM or -IM specialties tended to sign out fewer items than their EM- and IM-trained counterparts. DISCUSSION The results of this study show statistically significant improvements in inclusion of four clinically relevant sign-out components with use of the checklist: history of present illness, ED course, likely diagnosis, and awareness of plan by the entire patient care team. We found nonstatistically significant improvements in inclusion of pending studies, anticipated issues, opportunities for clarification, and requests additional information. Although these elements did not show a statistically significant increase, the first three have particular clinical
Table 2. Percentage of Resident Handoffs Including Key Sign-out Components and Time for Sign-out Prior to and After Checklist Implementation Sign-out Components
Preimplementation (n = 115)
Postimplementation (n = 114)
p-Value (c2)
HPI ED course Likely diagnosis Pending studies Disposition algorithm Anticipated issues All aware of plan Opportunities for clarification Repeat back used Requests additional information Proportion with no items Proportion with >80% of Items Proportion with all items Mean (median) length of sign-out
81% (93/115) 75% (86/115) 60% (68/114) 52% (53/101) 57% (65/115) 43% (47/109) 21% (22/107) 44% (50/114) 13% (15/115) 54% (62/115) 11% (13/115) 18% (21/115) <1% (1/115) 1.39 min (1)*
99% (113/114) 86% (98/114) 77% (85/111) 64% (65/102) 48% (51/107) 45% (48/106) 41% (42/103) 55% (63/114) 5% (6/114) 65% (74/114) 0% (0/115) 34% (39/114) <1% (1/114) 1.42 min (1) s
0.000 (21.1) 0.033 (4.53) 0.007 (7.41) 0.10 (2.64) 0.187 (1.74) 0.750 (0.10) 0.001 (10.1) 0.085 (2.97) 0.041 (4.16) 0.090 (2.87) 0.00 (13.7) 0.006 (7.5) 0.995 (0.00) 0.28 (z = 1.08)*
HPI = history of present illness; ED = emergency department. * Wilcoxon rank-sum test.
ED Sign-Out Checklist
significance for improving patient care. The checklist did not change the time it took residents to sign out. Interestingly, the proportion of residents using 80% of the checklist items after implementation only negligibly increased. This may be due to the fact that residents were not mandated to use the checklist or that they did not find many of the checklist components to be useful. The use of the repeat-back technique paradoxically decreased after the checklist was implemented, whereas the number of requests for additional information showed a nonstatistically significant increase. This suggests there was a shift in the type of communication methods used with a structured sign-out format, however, the reasons are unclear and are the source for future studies. Studies in other specialty settings have looked at resident sign-out processes using similar methodologies. In an observational cohort study of IM interns, Bump et al. coded 124 verbal sign-outs on an inpatient service to characterize sign-out content and organization (15). They identified important sign-out components similar to those in our study, including general hospital course and anticipated issues. Their analysis revealed much variability in organization and content of sign-out, similar to what we saw in the preimplementation phase of our study, and they attributed this to possible lack of education and supervision in sign-out (15). In another study involving IM and surgery residents, investigators created a computerized sign-out system for inpatient services using resident input. The results from their randomized, controlled trial using this system showed a decrease in the number of patients missed on rounds and more time spent providing patient care. The majority of residents reported better quality of sign-out and improved continuity of care (16,17). To our knowledge, there have been no studies to date that quantitatively evaluate a standardized process for patient handoffs in the ED. In a Web-based survey of EM residency directors and pediatric EM fellowship directors, the 90% reported having no uniform written policy regarding sign-out in the ED, and 75% reported having no formal training in sign-out. Additionally, the majority of program directors agreed that this education was valuable and that a standardized sign-out system would improve ED sign-outs (18). In a review article, Dhingra et al. outlined the key components of an effective ED sign-out system (13). They emphasize that sign-out should contain standardized content to ensure that critical details are not lost in transfer, should involve a standardized system such as the use of a checklist, and that specific verbal communication techniques should be stressed (13). Building on these principles, our sign-out checklist and process has all of these elements, which is likely why it was successful in improving the transfer of information among the residents. The 2008 Institute of Medicine report on resident duty hours recommended that residents be taught how to safely
583
transition care from one provider to another (19). Additionally, the Accreditation Council of Graduate Medical Education common program requirements in 2011 highlighted the importance of teaching residents how to safely transition patients to other providers (20). In our ED, we have residents from many different specialties, many of whom are interns, rotating for brief periods of time. Standardization is particularly important in the setting of a teaching hospital where inexperienced providers are developing their clinical and communication skills and are more likely to leave out key pieces of information or future patient care tasks. Limitations In this study, we did not prospectively control for resident specialty or PGY level. It is possible that a resident’s own specialty training influences their priorities in any patient sign-out setting, despite the fact that all residents were observed in ED handoffs. Likewise, residents with more postgraduate clinical experience may have implicitly learned through experience which pieces of information surrounding patient care are essential to mention in handoffs. A second limitation of this study is that not every ED handoff was observed during the study period. We observed only the afternoon shift change and we did not include handoffs in the low acuity zone of our ED. Differences in patient acuity may affect the points that get signed out and communication techniques used by the residents during handoffs. Third, the RAs were not blinded to the study and it is possible this may have added bias to the improvements seen in the postimplementation period. Likewise, although the components of the evaluation were not known to the residents, they were aware that they were being observed and this may have affected their performance. Fourth, we did not measure the penetrance of the educational effort. Our educational effort was widespread but did not reach everybody; despite this limit, use of the checklist was fairly ubiquitous. A more exhaustive effort would likely lead to increased use, so our study underestimates the effect of a checklist. Fifth, this was a pilot study and as such, we chose relatively short pre- and postimplementation observation periods without a formal sample size calculation. This may have underestimated the true effect of the checklist, as there may be a sign-out learning curve with use of the checklist over time. On the contrary, it may have overestimated the effect, as the post implementation period occurred immediately after the introduction of the checklist. Finally, this study does directly measure effects on patient safety or outcomes after handoffs have been completed. Future studies are warranted to investigate how the effectiveness of sign-out may influence time in the ED, bounce-back rate, or adverse outcomes.
584
N. M. Dubosh et al.
CONCLUSION Implementation of a sign-out checklist improved the transfer of information during patient handoffs in the ED and did not increase the overall length of time to sign out. REFERENCES 1. The Joint Commission. Sentinel event data. Root cause by event type. Available at: http://www.jointcommission.org/assets/ 1/18/Root_Causes_Event_Type_2004-2011.pdf. Accessed March 25, 2012. 2. Scoglietti VC, Collier KT, Long EL, et al. After-hours complications: evaluation of the predictive accuracy of resident sign-out. Am Surg 2010;76:682–6. 3. Arora V, Kao J, Lovinger D, et al. Medication discrepancies in resident sign-outs and their potential to harm. J Gen Intern Med 2007; 22:1751–5. 4. Solet DJ, Nrovell JM, Rutan GH, et al. Lost in translation: challenges and opportunities in physician-to physician communication during patient handoffs. Acad Med 2005;80:1094–9. 5. Sutliffe KM, Lewton E, Rosenthal MM. Communication failures: an insidious contributor to medical mishaps. Acad Med 2004;79: 186–94. 6. Arora V, Johnson J, Loviger D, et al. Communication failures in patient signout and suggestions for improvement: a critical incident analysis. Qual Saf Health Care 2005;14:401–7. 7. Horwitz LI, Moin T, Krumholz HM, et al. Consequences of inadequate sign-out for patient care. Arch Intern Med 2008;168:1755–60. 8. Singh H, Thomas EJ, Petersen LA, et al. Medical errors involving trainees: a study of closed malpractice claims from 5 insurers. Arch Intern Med 2007;167:2030–6. 9. Joint Commission on Accreditation of Health Care Organizations. 2009 Patient safety goals. Available at: www.jointcommission. org/. Accessed March 25, 2012.
10. Cheung JS, Kelly JJ, Beach C, et al. Improving handoffs in the emergency department. Ann Emerg Med 2010;55:171–80. 11. Haynes AB, Weiser TG, Berry WR, et al. A surgical safety checklist to reduce morbidity and mortality in a global population. N Engl J Med 2009;360:491–9. 12. Pronovost P, Needham D, Berenholtz S, et al. An intervention to reduce catheter-related bloodstream infections in the ICU. N Engl J Med 2006;355:2725–32. 13. Dhingra KR, Elms A, Hobgood C. Reducing error in the emergency department: a call for standardized sign-out process. Ann Emerg Med 2010;56:637–42. 14. Behara R, Wears RL, Perry SJ, et al. Conceptual framework for the safety of handovers. In: Henriksen K, ed. Advances in patient safety. Rockville, MD: Agency for Healthcare Research and Quality/ Department of Defense; 2005:309–21. 15. Bump GM, Jovin F, Destefano L, et al. Resident sign-out and patient hand-offs: opportunity for improvement. Teach Learn Med 2011; 23:105–11. 16. Van Eaton EG, Horvath KD, Lober WL, et al. Organizing the transfer of patient care information: the development of a computerized resident sign-out system. Surgery 2004;136:5–13. 17. Van Eaton EG, Horvath KD, Lober WB, et al. A randomized, controlled trial evaluating the impact of a computerized rounding a sign-out system on continuity of care and resident work hours. J Am Coll Surg 2005;200:538–45. 18. Sinha M, Shriki J, Salness R, et al. Need for standardized sign-out in the emergency department: a survey of emergency medicine residency directors and pediatric emergency fellowship directors. Acad Emerg Med 2007;14:192–6. 19. Ulmer C, Wolman D, Johns M, et al. Committee on Optimizing Graduate Medical Trainee (Resident) Hours and Work Schedules to Improve Patient Safety, Institute of Medicine. Resident duty hours: enhancing sleep, supervision, and safety. Washington, DC: National Academies Press; 2008. 20. Accreditation Council of Graduate Medical Education. Common program requirements 2011. Available at: http://www.acgme.org/ acwebsite/home/common_program_requirements_07012011.pdf. Accessed March 25, 2012.
ED Sign-Out Checklist
585
ARTICLE SUMMARY 1. Why is this topic important? Transitions of care, which are particularly prevalent in the emergency department (ED) setting, introduce the opportunity for clinical errors. Despite this evidence, the majority of emergency medicine residency programs lack formal training and structure in patient handoffs. 2. What does this study attempt to show? This study evaluates the effectiveness of a structured, computerized sign-out checklist for patient handoffs. 3. What are the key findings? The use of a sign-out checklist improves the transfer of patient information during resident handoffs in the ED without increasing time to sign-out. 4. How is patient care impacted? By improving transitions of care among providers in the ED, there is less opportunity for missed information and error in patient care. Using a structured format for resident handoffs does not significantly increase the time to sign out and thus does not threaten time spent engaging in direct patient care.
http://dx.doi.org/10.1016/j.jemermed.2014.06.017
Education IMPLEMENTATION OF AN EMERGENCY DEPARTMENT SIGN-OUT CHECKLIST IMPROVES TRANSFER OF INFORMATION AT SHIFT CHANGE Nicole M. Dubosh, MD,*† Dylan Carney, MD, MS,‡ Jonathan Fisher, MD, MPH,†* and Carrie D. Tibbles, MD*† *Department of Emergency Medicine, Beth Israel Deaconess Medical Center, Boston, Massachusetts, †Harvard Medical School, Boston, Massachusetts, and ‡Department of Emergency Medicine, University of California at San Francisco, San Francisco, CA Reprint Address: Nicole M. Dubosh, MD, Department of Emergency Medicine, Beth Israel Deaconess Medical Center, One Deaconess Road, W-CC2, Boston, MA 02215
, Abstract—Background: Transitions of care are ubiquitous in the emergency department (ED) and inevitably introduce the opportunity for errors. Few emergency medicine residency programs provide formal training or a standard process for patient handoffs. Checklists have been shown to be effective quality-improvement measures in inpatient settings and may be a feasible method to improve ED handoffs. Objective: To determine if the use of a sign-out checklist improves the accuracy and efficiency of resident sign-out in the ED. Methods: A prospective pre-/postinterventional study of residents rotating in the ED at a tertiary academic medical center. Trained research assistants observed resident sign-out during shift change over a 2-week period and completed a data collection tool to indicate whether or not key components of sign-out occurred and time to sign out each patient. An electronic sign-out checklist was implemented using a multi-faceted educational effort. A 2-week postintervention observation phase was conducted. Proportions, means, and nonparametric comparison tests were calculated using STATA. Results: One hundred fifteen sign-outs were observed prior to checklist implementation and 114 were observed after. Significant improvements were seen in four sign-out components: reporting of history of present illness increased from 81% to 99%, ED course increased from 75% to 86%, likely diagnosis increased from 60% to 77%, and team awareness of plan increased
from 21% to 41%. Use of the repeat-back technique decreased from 13% to 5% after checklist implementation and time to sign-out showed no significant change. Conclusion: Implementation of a checklist improved the transfer of information without increasing time to signout. Ó 2014 Elsevier Inc. , Keywords—sign-out; handoffs; checklist; emergency department; resident education
INTRODUCTION Emergency care is characterized by multiple transitions in patient responsibilities, both as patients leave the emergency department (ED) and as providers change shifts. Patients in an ED setting are often complex, requiring ongoing assessment and multiple interventions. Communication breakdowns among physicians are a commonly recognized source of error and are associated with 61% of sentinel events (1). Patient handoffs are times when this is particularly true, as transitions of care create the potential for critical information to be missed or distorted (2–5). Recently, much attention has been paid to improving the transfer of information during transitions of care of patients in all clinical settings, particularly among residents (6–8). Improving and standardizing transitions of care was a Joint Commission national patient safety goal in 2009 (9). In addition, the Quality Improvement and Patient Safety Section of the American College of Emergency
This project was funded by the 2011 Massachusetts American College of Emergency Physicians (MACEP) Emergency Medicine Resident Grant.
RECEIVED: 22 August 2012; FINAL SUBMISSION RECEIVED: 25 April 2014; ACCEPTED: 30 June 2014 580
ED Sign-Out Checklist
Physicians identified sign-out as a target for patient safety, particularly in the ED setting (10). Although sign-outs happen on a frequent basis in the ED, very few studies have examined how to best standardize and improve this process. Checklists have been shown to be effective in several different clinical settings in terms of decreasing medical errors and morbidity, and may be a feasible option in an ED (11,12). In our study, we examined the impact of introducing a standardized sign-out checklist on resident handoffs. Our objective was to determine if the use of a sign-out checklist improves the accuracy and efficiency of resident signout in the ED as measured by reduced omission of key information, communication behaviors, and time to sign out each patient. MATERIALS AND METHODS We performed a prospective evaluation of a novel sign-out checklist to improve ED-to-ED resident physician handoffs at shift change using a pre-/postintervention design. This was a pilot study conducted from November 7, 2011 to December 15, 2011 at a tertiary academic medical center with an annual ED volume of 53,000 visits and an admission rate of 38%. As part of an initiative to improve physician communication and handoffs at shift change, a standardized electronic checklist involving key elements was developed by the study investigators (Figure 1). The components of the checklist were based on existing literature and previously validated tools regarding safe handoffs in other non-ED settings (13–17). The components of the checklist included history of present
Figure 1. Electronic sign-out checklist for residents rotating in the ED. HPI = history of present illness; ED = emergency department; CT = computed tomography scan; Rx = prescription; DI = discharge instructions; POE = provider order entry.
581 illness, ED course, pending studies, likely disposition, anticipated issues, and algorithms for disposition. Several characteristics specific to our institution, including completion of discharge instructions, admission request to the ED observation unit, and ordering of home medications, were also included in the checklist, as the investigators deemed these elements important for streamlining sign-outs. An effort was undertaken to promote the use of the sign-out checklist, including a verbal announcement during resident didactics, and two separate e-mails sent to all ED residents and residents from other specialties rotating in our ED during the study period. The sign-out checklist was also posted in a print version near resident computers in the ED and made available through the ED electronic patient information system. Prior to the implementation of the sign-out checklist, a small team of hired, ED clinical research assistants (RAs) were trained on the elements and use of the checklist in a structured, nonclinical setting. Training included verbal instruction on coding of the checklist elements and resident communication behaviors using the data collection tool. A brief sign-out demonstration video was also created for the purposes of this study, and the RAs were required to watch this during their training. During a 2-week period, a convenience sample of handoffs involving first and second-year residents was observed by the RAs during the afternoon change of shift. Sign-outs involving medical students, attending physicians, and sign-out to inpatient services were excluded. Given the supervisory role of third-year emergency medicine (EM) residents at our institution, sign-outs involving these residents were also excluded. The RAs used a standardized binary data collection form to record whether or not items on the checklist were completed as residents signed out patients. Communication behaviors including use of the repeat-back technique, requests for additional information, and team awareness of plan (resident, attending, and nurse understood and have communicated about the care plan) were also recorded in a binary fashion. Finally, the length of time for each sign-out as well as the specialty and postgraduate year (PGY) of the resident were documented. The RA was not a member of the clinical team nor did he/she participate in the sign-out or serve in a supervisory role. Residents were aware of the RA’s presence but were not privy to the items on which they were being evaluated. One week after the preintervention observation period, the checklist was electronically linked to the ED electronic tracking system for resident use during sign-out. Residents were made aware of its availability at this time, as described above. A 2week period of observation was conducted in a similar fashion to the preimplementation phase, 1 week after the checklist was implemented. All data were entered into Microsoft Access 2003 (Microsoft Corporation, Redmond, WA) and analyzed using STATA 11.0 (College Station, TX). A c2 analysis was performed to compare proportions and a Wilcoxon rank-sum was used to compare means. The project was submitted to the local Institutional Review Board, who determined that our activities were quality improvement and as such do not require further review under federal guidelines. The procedures followed were in accord with the standards on the use of human subjects at the institution where the study was performed.
582
N. M. Dubosh et al.
Table 1. Baseline Resident Characteristics Year PGY 1 PGY 2 Specialty Internal Medicine Emergency Medicine Other specialty Total
Preimplementation n (%)*
Postimplementation n (%)
66 (61.1%) 42 (38.9%)
42 (36.8%) 72 (63.2%)
34 (29.6%) 63 (54.8%)
59 (51.8%) 52 (45.6%)
18 (15.7%) 115
3 (2.6%) 114
PGY = postgraduate year. * Seven preimplementation sign-outs included missing data points for resident year.
RESULTS One hundred fifteen resident sign-outs were observed during the observation period and 114 were observed after checklist implementation. Of the observed sign-outs, 108 were performed by first-year residents and 114 by second-year residents (Table 1). Seven preimplementation sign-outs were missing data for resident PGY year. Of the specialties represented, 115 of the residents were enrolled in the EM residency program, 93 were internal medicine (IM) residents, and 21 were from other specialties including general surgery, pediatric emergency medicine fellowship, and obstetrics and gynecology. Statistically significant improvements were found in four of the 10 sign-out components after implementation of the sign-out checklist (Table 2). Reporting of the history of present illness increased from 81% to 99%, ED course increased from 75% to 86%, likely diagnosis increased from 60% to 77%, and team awareness of
plan increased from 21% to 41%. The use of the repeat back technique decreased from 13% to 5% after the checklist was implemented. Overall, the proportion of sign-outs in which no checklist items were achieved decreased from 11% to 0%, and the proportion signing out 80% or more of the items increased from 18% to 34%. In both groups, the proportion of sign-outs in which all items were performed was negligible. Additionally, the median length of sign-out did not differ between the two groups. We performed a post hoc subgroup analysis to assess interaction between resident year, specialty, and checklist performance. With the exception of requests for additional information (where first-year resident signouts were more likely to involve requests for additional information), there were no statistically significant differences between sign-outs performed by first-year and second-year residents. When stratified by specialty, residents in non-EM or -IM specialties tended to sign out fewer items than their EM- and IM-trained counterparts. DISCUSSION The results of this study show statistically significant improvements in inclusion of four clinically relevant sign-out components with use of the checklist: history of present illness, ED course, likely diagnosis, and awareness of plan by the entire patient care team. We found nonstatistically significant improvements in inclusion of pending studies, anticipated issues, opportunities for clarification, and requests additional information. Although these elements did not show a statistically significant increase, the first three have particular clinical
Table 2. Percentage of Resident Handoffs Including Key Sign-out Components and Time for Sign-out Prior to and After Checklist Implementation Sign-out Components
Preimplementation (n = 115)
Postimplementation (n = 114)
p-Value (c2)
HPI ED course Likely diagnosis Pending studies Disposition algorithm Anticipated issues All aware of plan Opportunities for clarification Repeat back used Requests additional information Proportion with no items Proportion with >80% of Items Proportion with all items Mean (median) length of sign-out
81% (93/115) 75% (86/115) 60% (68/114) 52% (53/101) 57% (65/115) 43% (47/109) 21% (22/107) 44% (50/114) 13% (15/115) 54% (62/115) 11% (13/115) 18% (21/115) <1% (1/115) 1.39 min (1)*
99% (113/114) 86% (98/114) 77% (85/111) 64% (65/102) 48% (51/107) 45% (48/106) 41% (42/103) 55% (63/114) 5% (6/114) 65% (74/114) 0% (0/115) 34% (39/114) <1% (1/114) 1.42 min (1) s
0.000 (21.1) 0.033 (4.53) 0.007 (7.41) 0.10 (2.64) 0.187 (1.74) 0.750 (0.10) 0.001 (10.1) 0.085 (2.97) 0.041 (4.16) 0.090 (2.87) 0.00 (13.7) 0.006 (7.5) 0.995 (0.00) 0.28 (z = 1.08)*
HPI = history of present illness; ED = emergency department. * Wilcoxon rank-sum test.
ED Sign-Out Checklist
significance for improving patient care. The checklist did not change the time it took residents to sign out. Interestingly, the proportion of residents using 80% of the checklist items after implementation only negligibly increased. This may be due to the fact that residents were not mandated to use the checklist or that they did not find many of the checklist components to be useful. The use of the repeat-back technique paradoxically decreased after the checklist was implemented, whereas the number of requests for additional information showed a nonstatistically significant increase. This suggests there was a shift in the type of communication methods used with a structured sign-out format, however, the reasons are unclear and are the source for future studies. Studies in other specialty settings have looked at resident sign-out processes using similar methodologies. In an observational cohort study of IM interns, Bump et al. coded 124 verbal sign-outs on an inpatient service to characterize sign-out content and organization (15). They identified important sign-out components similar to those in our study, including general hospital course and anticipated issues. Their analysis revealed much variability in organization and content of sign-out, similar to what we saw in the preimplementation phase of our study, and they attributed this to possible lack of education and supervision in sign-out (15). In another study involving IM and surgery residents, investigators created a computerized sign-out system for inpatient services using resident input. The results from their randomized, controlled trial using this system showed a decrease in the number of patients missed on rounds and more time spent providing patient care. The majority of residents reported better quality of sign-out and improved continuity of care (16,17). To our knowledge, there have been no studies to date that quantitatively evaluate a standardized process for patient handoffs in the ED. In a Web-based survey of EM residency directors and pediatric EM fellowship directors, the 90% reported having no uniform written policy regarding sign-out in the ED, and 75% reported having no formal training in sign-out. Additionally, the majority of program directors agreed that this education was valuable and that a standardized sign-out system would improve ED sign-outs (18). In a review article, Dhingra et al. outlined the key components of an effective ED sign-out system (13). They emphasize that sign-out should contain standardized content to ensure that critical details are not lost in transfer, should involve a standardized system such as the use of a checklist, and that specific verbal communication techniques should be stressed (13). Building on these principles, our sign-out checklist and process has all of these elements, which is likely why it was successful in improving the transfer of information among the residents. The 2008 Institute of Medicine report on resident duty hours recommended that residents be taught how to safely
583
transition care from one provider to another (19). Additionally, the Accreditation Council of Graduate Medical Education common program requirements in 2011 highlighted the importance of teaching residents how to safely transition patients to other providers (20). In our ED, we have residents from many different specialties, many of whom are interns, rotating for brief periods of time. Standardization is particularly important in the setting of a teaching hospital where inexperienced providers are developing their clinical and communication skills and are more likely to leave out key pieces of information or future patient care tasks. Limitations In this study, we did not prospectively control for resident specialty or PGY level. It is possible that a resident’s own specialty training influences their priorities in any patient sign-out setting, despite the fact that all residents were observed in ED handoffs. Likewise, residents with more postgraduate clinical experience may have implicitly learned through experience which pieces of information surrounding patient care are essential to mention in handoffs. A second limitation of this study is that not every ED handoff was observed during the study period. We observed only the afternoon shift change and we did not include handoffs in the low acuity zone of our ED. Differences in patient acuity may affect the points that get signed out and communication techniques used by the residents during handoffs. Third, the RAs were not blinded to the study and it is possible this may have added bias to the improvements seen in the postimplementation period. Likewise, although the components of the evaluation were not known to the residents, they were aware that they were being observed and this may have affected their performance. Fourth, we did not measure the penetrance of the educational effort. Our educational effort was widespread but did not reach everybody; despite this limit, use of the checklist was fairly ubiquitous. A more exhaustive effort would likely lead to increased use, so our study underestimates the effect of a checklist. Fifth, this was a pilot study and as such, we chose relatively short pre- and postimplementation observation periods without a formal sample size calculation. This may have underestimated the true effect of the checklist, as there may be a sign-out learning curve with use of the checklist over time. On the contrary, it may have overestimated the effect, as the post implementation period occurred immediately after the introduction of the checklist. Finally, this study does directly measure effects on patient safety or outcomes after handoffs have been completed. Future studies are warranted to investigate how the effectiveness of sign-out may influence time in the ED, bounce-back rate, or adverse outcomes.
584
N. M. Dubosh et al.
CONCLUSION Implementation of a sign-out checklist improved the transfer of information during patient handoffs in the ED and did not increase the overall length of time to sign out. REFERENCES 1. The Joint Commission. Sentinel event data. Root cause by event type. Available at: http://www.jointcommission.org/assets/ 1/18/Root_Causes_Event_Type_2004-2011.pdf. Accessed March 25, 2012. 2. Scoglietti VC, Collier KT, Long EL, et al. After-hours complications: evaluation of the predictive accuracy of resident sign-out. Am Surg 2010;76:682–6. 3. Arora V, Kao J, Lovinger D, et al. Medication discrepancies in resident sign-outs and their potential to harm. J Gen Intern Med 2007; 22:1751–5. 4. Solet DJ, Nrovell JM, Rutan GH, et al. Lost in translation: challenges and opportunities in physician-to physician communication during patient handoffs. Acad Med 2005;80:1094–9. 5. Sutliffe KM, Lewton E, Rosenthal MM. Communication failures: an insidious contributor to medical mishaps. Acad Med 2004;79: 186–94. 6. Arora V, Johnson J, Loviger D, et al. Communication failures in patient signout and suggestions for improvement: a critical incident analysis. Qual Saf Health Care 2005;14:401–7. 7. Horwitz LI, Moin T, Krumholz HM, et al. Consequences of inadequate sign-out for patient care. Arch Intern Med 2008;168:1755–60. 8. Singh H, Thomas EJ, Petersen LA, et al. Medical errors involving trainees: a study of closed malpractice claims from 5 insurers. Arch Intern Med 2007;167:2030–6. 9. Joint Commission on Accreditation of Health Care Organizations. 2009 Patient safety goals. Available at: www.jointcommission. org/. Accessed March 25, 2012.
10. Cheung JS, Kelly JJ, Beach C, et al. Improving handoffs in the emergency department. Ann Emerg Med 2010;55:171–80. 11. Haynes AB, Weiser TG, Berry WR, et al. A surgical safety checklist to reduce morbidity and mortality in a global population. N Engl J Med 2009;360:491–9. 12. Pronovost P, Needham D, Berenholtz S, et al. An intervention to reduce catheter-related bloodstream infections in the ICU. N Engl J Med 2006;355:2725–32. 13. Dhingra KR, Elms A, Hobgood C. Reducing error in the emergency department: a call for standardized sign-out process. Ann Emerg Med 2010;56:637–42. 14. Behara R, Wears RL, Perry SJ, et al. Conceptual framework for the safety of handovers. In: Henriksen K, ed. Advances in patient safety. Rockville, MD: Agency for Healthcare Research and Quality/ Department of Defense; 2005:309–21. 15. Bump GM, Jovin F, Destefano L, et al. Resident sign-out and patient hand-offs: opportunity for improvement. Teach Learn Med 2011; 23:105–11. 16. Van Eaton EG, Horvath KD, Lober WL, et al. Organizing the transfer of patient care information: the development of a computerized resident sign-out system. Surgery 2004;136:5–13. 17. Van Eaton EG, Horvath KD, Lober WB, et al. A randomized, controlled trial evaluating the impact of a computerized rounding a sign-out system on continuity of care and resident work hours. J Am Coll Surg 2005;200:538–45. 18. Sinha M, Shriki J, Salness R, et al. Need for standardized sign-out in the emergency department: a survey of emergency medicine residency directors and pediatric emergency fellowship directors. Acad Emerg Med 2007;14:192–6. 19. Ulmer C, Wolman D, Johns M, et al. Committee on Optimizing Graduate Medical Trainee (Resident) Hours and Work Schedules to Improve Patient Safety, Institute of Medicine. Resident duty hours: enhancing sleep, supervision, and safety. Washington, DC: National Academies Press; 2008. 20. Accreditation Council of Graduate Medical Education. Common program requirements 2011. Available at: http://www.acgme.org/ acwebsite/home/common_program_requirements_07012011.pdf. Accessed March 25, 2012.
ED Sign-Out Checklist
585
ARTICLE SUMMARY 1. Why is this topic important? Transitions of care, which are particularly prevalent in the emergency department (ED) setting, introduce the opportunity for clinical errors. Despite this evidence, the majority of emergency medicine residency programs lack formal training and structure in patient handoffs. 2. What does this study attempt to show? This study evaluates the effectiveness of a structured, computerized sign-out checklist for patient handoffs. 3. What are the key findings? The use of a sign-out checklist improves the transfer of patient information during resident handoffs in the ED without increasing time to sign-out. 4. How is patient care impacted? By improving transitions of care among providers in the ED, there is less opportunity for missed information and error in patient care. Using a structured format for resident handoffs does not significantly increase the time to sign out and thus does not threaten time spent engaging in direct patient care.