Safety Climate and Medical Errors in 62 US Emergency Departments

PATIENT SAFETY/ORIGINAL RESEARCH

Safety Climate and Medical Errors in 62 US Emergency Departments Carlos A. Camargo, Jr, MD, DrPH, Chu-Lin Tsai, MD, ScD, Ashley F. Sullivan, MS, MPH, Paul D. Cleary, PhD, MPH, James A. Gordon, MD, MPA, Edward Guadagnoli, PhD, Rainu Kaushal, MD, MPH, David J. Magid, MD, MPH, Sowmya R. Rao, PhD, David Blumenthal, MD, MPP* From the Department of Emergency Medicine (Camargo, Tsai, Sullivan, Gordon), Mongan Institute for Health Policy (Camargo, Gordon, Blumenthal), and Biostatistics Center (Rao), Massachusetts General Hospital, Harvard Medical School, Boston, MA; the Yale School of Public Health, Yale School of Medicine, New Haven, CT (Cleary); the Department of Health Care Policy, Harvard Medical School, Boston, MA (Guadagnoli); Weill Cornell Medical College and New York–Presbyterian Hospital, New York, NY (Kaushal); the Institute for Health Research, Kaiser Permanente Colorado and the Departments of Emergency Medicine and Preventive Medicine and Biometrics, University of Colorado Denver, Aurora, CO (Magid); and the Office of the National Coordinator for Health Information Technology, Department of Health and Human Services, Washington, DC (Blumenthal).

Study objective: We describe the incidence and types of medical errors in emergency departments (EDs) and assess the validity of a survey instrument that identifies systems factors contributing to errors in EDs. Methods: We conducted the National Emergency Department Safety Study in 62 urban EDs across 20 US states. We reviewed 9,821 medical records of ED patients with one of 3 conditions (myocardial infarction, asthma exacerbation, and joint dislocation) to evaluate medical errors. We also obtained surveys from 3,562 staff randomly selected from each ED; survey data were used to calculate average safety climate scores for each ED. Results: We identified 402 adverse events (incidence rate 4.1 per 100 patient visits; 95% confidence interval [CI] 3.7 to 4.5) and 532 near misses (incidence rate 5.4 per 100 patient visits; 95% CI 5.0 to 5.9). We judged 37% of the adverse events, and all of the near misses, to be preventable (errors); 33% of the near misses were intercepted. In multivariable models, better ED safety climate was not associated with fewer preventable adverse events (incidence rate ratio per 0.2-point increase in ED safety score 0.82; 95% CI 0.57 to 1.16) but was associated with more intercepted near misses (incidence rate ratio 1.79; 95% CI 1.06 to 3.03). We found no association between safety climate and violations of national treatment guidelines. Conclusion: Among the 3 ED conditions studied, medical errors are relatively common, and one third of adverse events are preventable. Improved ED safety climate may increase the likelihood that near misses are intercepted. [Ann Emerg Med. 2012;60:555-563.] Please see page 556 for the Editor’s Capsule Summary of this article. A feedback survey is available with each research article published on the Web at www.annemergmed.com. A podcast for this article is available at www.annemergmed.com. 0196-0644/$-see front matter Copyright © 2012 by the American College of Emergency Physicians. http://dx.doi.org/10.1016/j.annemergmed.2012.02.018

SEE EDITORIAL, P. 564. INTRODUCTION Medical errors are a major cause of morbidity and mortality in the United States. The Institute of Medicine’s report To Err Is Human estimated that adverse events occurred in 2.9% to 3.7% of 33 million hospitalizations and that 44,000 to 98,000 people die each year as a result of *Members of the Emergency Medicine Network and the principal investigators at the 62 participating sites are listed in the Appendix.

Volume , .  : November 

medical errors.1 Previous safety studies have focused on medical errors in the inpatient,2-4 outpatient,5 critical care,6 and long-term care settings.7 Emergency department (ED) care is considered especially prone to medical errors for several reasons, including the fast pace and frequency of complex and life-threatening conditions. Moreover, as documented in another Institute of Medicine report,8 the nation’s EDs face serious challenges, such as crowding, that may increase the likelihood of errors. Although there are data on medication errors in the ED,9 there are sparse data on the overall incidence of medical errors in the ED. A single-ED Annals of Emergency Medicine 555

Safety Climate and Medical Errors

Editor’s Capsule Summary

What is already known on this topic Little is known about how differences in emergency department (ED) safety culture affect the quantity and quality of errors. What question this study addressed The investigators surveyed multiple persons at 62 urban EDs to define their safety culture and performed a structured chart review of 3 tracer conditions to quantify the numbers and kinds of medical errors. What this study adds to our knowledge Although the study failed to find an association between safety culture and the number of preventable errors, it did find one between safety culture and the number of intercepted near misses. How this is relevant to clinical practice This study will not change practice but represents an important step in gaining greater understanding of factors that affect patient safety in the ED.

study interviewed staff during a 7-day period and found that 18% of 1,935 patient visits had self-reported errors.10 A more recent ED interview study showed that 32% of 487 visits had at least 1 “nonideal” care event.11 Traditional approaches to identifying and preventing the causes of errors (eg, root-cause analysis) are often passive and emphasize individual factors.12 Active surveillance of frontline health care providers about systems factors that may cause errors is an innovative strategy for identifying correctable causes of errors.13 Likewise, the overall “ED safety climate”— by which we mean both human factors and measureable attributes of the systems of care, not outward manifestations of safety culture—is a potentially useful proxy for safety. However, the link between ED safety climate and actual medical errors is unknown. We conducted the National Emergency Department Safety Study to address these major gaps in the patient safety literature. The objectives of National Emergency Department Safety Study were to describe the incidence and types of medical errors in EDs and to assess the validity of a survey instrument that directly identifies systems factors thought to contribute to errors in EDs. We hypothesized that better ED safety climate, as measured by our instrument, would be associated with lower incidence of preventable adverse events and higher incidence of intercepted near misses. By contrast, we expected no association between ED safety climate and nonpreventable adverse events or nonintercepted near misses. 556 Annals of Emergency Medicine

Camargo et al

MATERIALS AND METHODS Study Design and Setting Details of the National Emergency Department Safety Study design and data collection have been previously published.14 In brief, it was a multicenter study that sought to characterize both human and systemic factors associated with the occurrence of errors in the ED. The study was coordinated by the Emergency Medicine Network (http://www.emnet-usa.org). We invited EDs affiliated with the network to participate in the study, with additional recruitment through postings on emergency medicine listservs, direct contact with sites, and presentations at national meetings. The institutional review board at all participating hospitals approved the study. In accordance with the human factors and patient safety literature, interviews with key informants, and focus group discussions, we developed a questionnaire to assess systems factors that might contribute to errors in the ED. To refine the draft survey, we administered it to staff in 10 EDs in 2004 to 2005 and then conducted psychometric analyses to select the final set of questions. The details of instrument development were described elsewhere.15 The final instrument consisted of 50 questions about 9 aspects of EDs (subscales): physical environment, staffing, equipment and supplies, teamwork, nursing, culture, triage and monitoring, information coordination and consultation, and inpatient coordination. All questions had a 5-point Likert response scale (never, rarely, sometimes, most of the time, or always), with 1 representing the least desirable response and 5 representing the most desirable response. Subscale scores were derived by averaging across the items, and an overall safety score was derived by averaging across all subscales. The final survey (Appendix E1, available online at http://www.annemergmed.com) was administered to a random sample of 80 ED staff (physicians, nurses, nurse practitioners, or physician assistants who worked an average of 1 or more clinical shifts per week) per ED at 59 EDs in 2005 to 2006. Together with the data collected from the 10 psychometric sites, survey data were available from 69 EDs. The overall response rate was 66%. We excluded 4 EDs with response rates of 45% or less (n⫽99) and an additional 79 individual questionnaires for diverse reasons (eg, worked ⬍3 months in that ED, had answers to ⬍80% of survey items, social worker). As a result, the perceived safety scores were derived from 3,562 respondents in 65 EDs, and these scores were averaged at the ED level to represent ED safety climate. We focused on 3 ED conditions (myocardial infarction, asthma exacerbation, and joint dislocation involving procedural sedation) for the following reasons: (1) they are relatively common ED conditions; (2) national guidelines exist for their care16-18; (3) they are diverse, including surgical and medical problems affecting both men and women; and (4) rigorous guidelines and compliance with them have been reliably assessed in previous research,19,20 including documentation of errors in the implementation of guideline-based care. Volume , .  : November 

Camargo et al We reviewed medical records of patients presenting with the 3 conditions to estimate the incidence of adverse events and near misses. Using International Classification of Diseases, Ninth Revision, Clinical Modification21 codes from hospital administrative records, sites identified all charts with a primary ED or hospital discharge diagnosis of the 3 conditions during a 12-month period in 2003 to 2006. On-site trained research personnel used a standardized form to abstract data from 70 randomly selected ED visits for each condition. Sites with fewer than 70 charts reviewed all eligible charts. The inclusion and exclusion criteria for each condition are available as Table E1 (available online at http://www.annemergmed.com). For each ED, medical records reviewed were selected from a period before the administration of the survey instrument to avoid a Hawthorne effect. Medical errors were identified with a 2-step process. The error identification and assessment process was not blinded to outcomes. The initial screening method and criteria were adapted from previous studies2-4,22 and comprised 18 questions designed to screen charts for adverse events or near misses (form available as Appendix E2, available online at http://www.annemergmed.com). If a chart screened positive, it was independently reviewed by an off-site pair of physician reviewers, including at least 1 attending emergency physician. All reviewers received standardized training, which included the completion of practice charts for each condition and individual feedback from study investigators. The causative factor and outcome did not have to occur in the same sampled ED visit (eg, the study design included review of the inpatient discharge summary); moreover, all patients with one of the 3 diagnoses who had a return visit to the ED within 48 hours underwent physician review. Reviewers confirmed or rejected whether the screen-positive items represented true adverse events or near misses and then reevaluated the entire deidentified chart for the occurrence of any other adverse events or near misses. Reviewers described and coded the type of event and its preventability. Following previously used methodology,23,24 reviewers also determined effect on a 4-point scale: significant, serious, life threatening, or fatal. After discussion among the 2 reviewers, any unresolved discrepancies were adjudicated by a third physician reviewer. The detailed chart flow is illustrated in Figure 1. The final sample consisted of 9,821 charts from 62 EDs in 20 US states. We determined interrater reliability by a random sampling of 100 screen-positive charts (126 events), with the associated reviewers divided into 2 panels. Preconsensus interrater reliability between the reviewer panels was calculated with the ␬ statistic.25 As expected, interrater agreement was fair for the classification of events as adverse events, near misses, or neither (␬ 0.34); and fair for both the preventability of adverse events (␬ 0.38) and whether or not a near miss was intercepted (␬ 0.36). Agreement between reviewers was moderate for the actual effect of adverse events (␬ 0.46) and very good for the potential effect of near misses (␬ 0.83). Additional details are available as Table E2 (available online at http://www.annemergmed.com). After discussion among the 2 reviewers, only 3% (53) of the 1,560 flagged charts required adjudication by a third physician. Volume , .  : November 

Safety Climate and Medical Errors

Figure 1. Patient flow. MI, myocardial infarction; AE, adverse event; NM, near miss.

Outcome Measures According to the Institute of Medicine report1 and work by Reason,26,27 we used the following definitions for our study: 1. Medical error: A failure of a planned action to be completed as intended or the use of a wrong plan to achieve an aim. 2. Adverse event: An injury (pathologic alteration in condition) that might have resulted from medical management (interventions or lack thereof) in the ED. Injuries resulting solely from an underlying disease process, rather than medical management, are not adverse events. a. Nonpreventable adverse event: An unavoidable injury resulting from appropriate medical management. Nonpreventable adverse events are not medical errors. b. Preventable adverse event: An injury resulting from a medical error. 3. Near miss: A medical error with the potential to cause injury but that does not for one of 2 reasons: a. Intercepted near miss: A potentially harmful error that is intercepted before reaching the patient. b. Nonintercepted near miss: A potentially harmful error that unexpectedly does no detectable harm because of patient characteristics or chance, despite reaching the patient. The interrelationships between these outcome measures are illustrated by the Venn diagram in Figure E1 (available online at http://www.annemergmed.com). Examples are provided in Appendix E3 (available online at http://www.annemergmed. com). For each of the 3 conditions, we also defined serious guideline violations according to the literature and in the consensus view of the National Emergency Department Safety Study leadership and Emergency Medicine Network Steering Committee (Table E3, available online at http://www. annemergmed.com). We tested the hypothesis that better ED Annals of Emergency Medicine 557

Safety Climate and Medical Errors safety climate is associated with lower incidence of serious guideline violations. From each chart, we abstracted baseline patient characteristics, medical history, ED presenting symptoms, ED course and management, ED provider type, and ED disposition. We also contacted a key informant at each site (eg, ED medical director, ED nurse manager) to collect basic data on ED characteristics, such as annual visit volume, annual number of visits for the 3 conditions, and number of ED beds. Geographic regions (Northeast, South, Midwest, and West) were defined according to Census Bureau boundaries.28 Primary Data Analyses All analyses were performed with Stata (version 10.0; StataCorp, College Station, TX). Summary statistics at the patient and ED levels are presented as proportions (with 95% confidence intervals [CIs]), means (with SDs), or medians (with interquartile ranges [IQRs]), as appropriate. Bivariate associations were examined with Student’s t test, Wilcoxon rank-sum test, and ␹2 test, as appropriate. Because the number of outcomes per patient followed a Poisson distribution, multivariable Poisson regression models were used to assess the associations between ED safety scores (overall and subscales) and outcome measures. Because subscale-specific safety scores are strongly correlated, separate regression models were fit with 1 subscale score at a time. To account for the potential effects of clustering of patients within EDs, all models were fit with generalized estimating equations to obtain robust estimates of standard errors.29 Model variables were selected a priori,30 according to review of the medical literature,2,3,31,32 or from variables associated with the outcome measure at P⬍.10 in univariable analyses with change-in-estimate methods.33 The multivariable models include age, sex, 3 conditions, region, affiliation with an emergency medicine residency program, number of beds in the ED, and calendar year. To determine the functional form used for continuous predictors (ie, ED safety scores), we grouped the predictor into tertiles or quartiles and determined whether the log incidence rate increased or decreased linearly.34 The dose-response relationship appeared linear for the overall summary score and most of the subscale scores, and the incidence rate ratios were reported per SD (ie, 0.2-point) increase in ED safety score. All incidence rate ratios are presented with 95% CIs. Sensitivity Analyses To assess the effect of different modeling approaches, we repeated the main analysis with a 2-level random-effects35 model to account for potential clustering of patients within EDs. We also fit a zero-inflated Poisson regression model to address potential excess zeros in our data.36 Next, because ED clinicians’ perceptions of ED safety climate may differ according to length of employment, we refit the models with safety climate perceived by ED staff who had worked in the ED for fewer than 5 versus greater than or equal to 5 years. 558 Annals of Emergency Medicine

Camargo et al Table 1. Characteristics and safety climate of the 62 EDs. Characteristics

Result

ED characteristics Number of ED visits per year, median (IQR) 56,672 (43,000–75,000) Number of ED beds, median (IQR) 40 (27–50) Number of ED visits for acute MI per year, 281 (115–454) median (IQR) Number of ED visits for acute asthma per 1,015 (511–1,767) year, median (IQR) Number of ED visits for joint dislocation per 150 (84–200) year, median (IQR) Residency affiliated, % 77 Urban location, % 100 Census region, % Northeast 45 Midwest 24 South 11 West 19 Safety climate (scored as 1 low, 5 high) Overall summary score, mean (SD) 3.5 (0.2) Subscales, mean (SD) Physical environment 3.2 (0.3) Staffing 3.2 (0.2) Equipment and supplies 4.0 (0.2) Teamwork 3.8 (0.1) Nursing 3.7 (0.2) Culture 3.8 (0.2) Triage and monitoring 3.8 (0.2) Information coordination and consultation 3.4 (0.2) Inpatient coordination 2.4 (0.4)

RESULTS The 62 study EDs were located throughout the country, had large annual visit volumes, and cared for many patients with each of the 3 conditions (Table 1). All of the EDs were urban and most (77%) were affiliated with an emergency medicine residency program. The overall ED clinicians’ perceptions of ED safety climate were represented by a mean safety score of 3.5 on a scale of 1 to 5. The ratings appeared lower for 4 subscales: physical environment, staffing, information coordination and consultation, and inpatient coordination. The final sample comprised 9,821 patients who presented between 2003 and 2006, with 89% of the visits made in 2004. The median age of the patients was 46 years, and 52% were men. The median number of medical records reviewed per ED was 168 (IQR 144 to 186) and consisted of 70 (IQR 61 to 70) myocardial infarction visits, 69 (IQR 64 to 70) asthma visits, and 34 (IQR 19 to 48) joint dislocation visits involving procedural sedation. Table 2 shows the distribution and incidence rates of adverse events and near misses. Overall, 647 (7%) patients had either adverse events or near misses, with a total of 934 outcomes (incidence rate 9.5 per 100 patient visits). The incidence of adverse events/ near misses was highest in myocardial infarction, followed by dislocation and then asthma (incidence rate 17.3, 7.7, and 3.0 per 100 patient visits, respectively; P⬍.001). At the ED level, there was a significant positive correlation between the Volume , .  : November 

Camargo et al

Safety Climate and Medical Errors

Table 2. Distribution and incidence rates of adverse events and near misses.

Variable Overall adverse events or near misses By condition Acute myocardial infarction Acute asthma Joint dislocation involving procedural sedation By type of event Adverse events Total number Preventable Nonpreventable Effect Significant Serious Life threatening Fatal Near misses Total number Intercepted Nonintercepted Potential effect Significant Serious Life threatening Fatal

Number of Events, No. (%)

Incidence Rate (95% CI), No./100 Patient Visits

934

9.5 (8.9–10.1)

658 (70) 117 (13) 159 (17)

17.3 (16.0–18.7) 3.0 (0.4–3.5) 7.7 (6.6–9.0)

402 149 (37) 253 (63)

4.1 (3.7–4.5) 1.5 (1.3–1.8) 2.6 (2.3–2.9)

102 (25) 152 (38) 128 (32) 20 (5)

1.0 (0.8–1.3) 1.5 (1.3–1.8) 1.3 (1.1–1.5) 0.2 (0.1–0.3)

532 178 (33) 354 (67)

5.4 (5.0–5.9) 1.8 (1.6–2.1) 3.6 (3.2–4.0)

154 (29) 234 (44) 125 (23) 19 (4)

1.6 (1.3–1.8) 2.4 (2.1–2.7) 1.3 (1.1–1.5) 0.2 (0.1–0.3)

incidence of adverse events and the incidence of near misses (r⫽0.41; P⬍.001). There were 402 adverse events (incidence rate 4.1 per 100 patient visits) and 532 near misses (incidence rate 5.4 per 100 patient visits). Physician reviewers deemed 37% of the adverse events preventable (incidence rate 1.5 per 100 patient-visits), with 25% of the all adverse events deemed significant, 38% serious, 32% life threatening, and 5% fatal. The preventability of adverse events was higher in more severe injury categories (18%, 34%, 58%, and 45%, respectively; P⬍.001). Of the near misses, one third were intercepted. The distribution of impact levels among near misses was similar to that of adverse events. The associations between ED safety climate and main outcome measures are shown in Table 3 and Figure 2. In general, better ED safety climate, as measured by higher safety scores, was associated with lower incidence of preventable adverse events (incidence rate ratio ⬍1); however, the association was not statistically significant. By contrast, better ED safety climate was statistically significantly associated with higher incidence of intercepted near misses (incidence rate ratio per 0.2-point increase in ED safety score 1.79; 95% CI 1.06 to 3.03). The associations were particularly strong for some aspects of ED safety climate: staffing, nursing, culture, and triage and monitoring. As expected, there were no associations between ED safety climate and nonpreventable adverse events (incidence rate ratio 0.93; 95% CI 0.76 to 1.14) or nonintercepted near Volume , .  : November 

misses (incidence rate ratio 1.08; 95% CI 0.78 to 1.49). Moreover, there was no association between ED safety climate and serious guideline violations across all 3 conditions (overall incidence rate ratio 1.03; 95% CI 0.96 to 1.10); full results are available as Table E4 (available online at http://www. annemergmed.com). Sensitivity Analyses The association between better ED safety climate and higher incidence of intercepted near misses remained significant in a random-effects model (incidence rate ratio per 0.2-point increase in ED safety score 1.89; 95% CI 1.03 to 3.47) or a zero-inflated Poisson regression model (incidence rate ratio 1.96; 95% CI 1.24 to 3.08). Moreover, perceptions of better ED safety climate among junior ED staff (who had worked in the ED for ⬍5 years) were associated with intercepted near misses (incidence rate ratio 1.73; 95% CI 1.11 to 2.70), whereas senior staff’s perceptions were not (incidence rate ratio 1.36; 95% CI 0.84 to 2.17). The associations between ED safety climate and preventable adverse events remained statistically nonsignificant in these sensitivity analyses.

LIMITATIONS Our study has several potential limitations. First, because most participating EDs are urban academic centers, the results may not be generalizable to other settings. Second, we did not sample all ED visits, so we are unable to estimate the incidence of errors for all ED conditions. Third, although the reliability of initial physician judgments was similar to that of previous studies, in which ␬ statistics have ranged from 0.2 to 0.6,3,4 only 3% of charts produced rater disagreement that required adjudication by a third reviewer. Nevertheless, our findings should be interpreted in the context of potential measurement errors. Also, the correlation between high rates of preventability and more severe injuries suggests a potential outcome bias in classification of errors.37,38 Fourth, the study design cannot rule out the possibility that EDs with high safety scores were more likely to document intercepted near misses compared with being more likely to intercept problems with care. As with all observational studies, unmeasured factors (eg, ED crowding) might help explain the observed associations. Fifth, the subscale analyses are subject to multiple comparison problems and should be interpreted with caution. Sixth, despite the relatively large numbers (3,562 staff and 9,821 medical records), the statistical power for the primary outcomes was somewhat limited. For example, a post hoc power analysis, with ␣⫽.05, revealed that the current study had 50% power to detect a 0.82fold decreased risk of preventable adverse events and greater than 99% power to detect a 1.79-fold increased risk of intercepted near misses. Holding other factors constant, it would require 130,000 charts to achieve 80% power for preventable adverse events. Nevertheless, the observed associations were consistent with our study hypotheses, with a nonsignificant reduction in preventable adverse events and a Annals of Emergency Medicine 559

Safety Climate and Medical Errors

Camargo et al

Table 3. Multivariable associations between ED safety climate and outcome measures. Multivariable-Adjusted Incidence Rate Ratio (95% CI)* Safety Climate Overall summary score Subscales Physical environment Staffing Equipment and supplies Teamwork Nursing Culture Triage and monitoring Information coordination and consultation Inpatient coordination

Adverse Event (Overall)

Preventable Adverse Event

Nonintercepted Near Miss

Intercepted Near Miss

0.89 (0.74–1.06)

0.82 (0.57–1.16)

1.08 (0.78–1.49)

1.79 (1.06–3.03)

0.92 (0.82–1.03) 0.99 (0.87–1.12) 0.96 (0.83–1.10) 0.92 (0.74–1.15) 1.05 (0.89–1.23) 0.92 (0.79–1.07) 0.87 (0.71–1.06) 0.98 (0.86–1.12)

0.85 (0.68–1.06) 0.96 (0.76–1.21) 0.85 (0.65–1.10) 1.06 (0.70–1.63) 0.93 (0.68–1.29) 0.98 (0.76–1.27) 0.84 (0.61–1.16) 0.82 (0.62–1.08)

0.89 (0.66–1.21) 1.15 (0.88–1.50) 1.09 (0.82–1.46) 1.42 (0.97–2.09) 1.11 (0.81–1.52) 1.24 (0.95–1.60) 0.82 (0.51–1.33) 1.04 (0.81–1.33)

1.17 (0.85–1.61) 1.54 (1.08–2.20) 1.41 (0.91–2.19) 2.09 (1.00–4.38) 1.89 (1.17–3.07) 1.69 (1.18–2.42) 1.74 (1.02–2.98) 1.32 (0.95–1.84)

0.93 (0.88–0.99)

0.93 (0.81–1.07)

1.04 (0.89–1.23)

1.17 (0.97–1.40)

*A multivariable Poisson regression model was fit with generalized estimating equations accounting for clustering of patients within EDs. The model adjusted for age, sex, 3 conditions (myocardial infarction, dislocation, and asthma), 4 regions (Northeast, Midwest, South, and West), calendar year, affiliation with an emergency medicine residency program, and number of beds in the ED. The incidence rate ratios correspond to a 0.2-point increase in safety score.

Figure 2. Multivariable associations between ED safety climate and outcome measures. Circles represent the point estimates of the multivariable-adjusted incidence rate ratios per 0.2-point increase in ED safety score; whiskers represent the associated 95% CIs. The y axis is plotted on a log scale. The number of events for each category is as follows: preventable AE n⫽149; nonpreventable AE n⫽253; nonintercepted NM n⫽354; intercepted NM n⫽178.

significant increase in intercepted near misses. “Significant” associations may not be clinically relevant, whereas “nonsignificant” associations may be underpowered true associations. Inclusion of EDs that did not respond to our survey could have strengthened or weakened our results because we did not collect data on nonresponders.

DISCUSSION In this study of 9,821 patients presenting to 62 EDs with 3 common conditions, the incidence of adverse events was 4.1%, of which 37% were deemed preventable. Although our measure 560 Annals of Emergency Medicine

of ED safety climate was not associated with a statistically significant decrease in preventable adverse events, it was associated with a higher likelihood of intercepted near misses. Last, there was no association between ED safety climate and serious violations of national treatment guidelines. The National Emergency Department Safety Study demonstrates that preventable adverse events and near misses are relatively common in EDs. With nearly 120 million ED visits each year,39 the observed frequency of ED errors suggests that significant numbers of ED patients are affected by medical errors. The incidence of total adverse events in the ED appeared similar to that in the inpatient setting, as reported by the Harvard Medical Practice Study (3.7%; 95% CI 3.2% to 4.2%).2,3 Because ED visits are typically measured in hours and hospitalizations in days, the ED incidence may indeed be higher per exposed time. We also found that only 37% of the adverse events in the ED were preventable, in contrast to 53% to 70% estimates from previous studies.3,4 These differences may result from changes in care over time, the use of 3 specific conditions (compared with a random sample of all conditions), lack of emergency physician reviewers in previous studies, or different definitions of preventability across studies. In psychometrics, an instrument demonstrates validity if it correlates with other related clinical parameters (convergent) and does not correlate with unrelated or dissimilar ones (divergent).40 Our survey instrument demonstrates both convergent (ie, intercepted near misses) and divergent (ie, nonpreventable adverse events [nonerrors] and nonintercepted near misses [random events]) validity. The validity of the survey may stem from the theory that safety climate is determined by the internalized values and beliefs of ED staff, which ultimately influence their behaviors.41 For example, nurses completed most of our surveys.15 Compared with physicians, nurses might be more aware of ED safety climate and more likely to intercept (or document intercepted) near misses. To our knowledge, our study also provides the first quantitative evidence of the link Volume , .  : November 

Camargo et al between ED safety climate and patient safety outcomes. If an ED improves its safety climate by 0.2 points (1 SD), it is possible that 80% more near misses could be intercepted by this safer ED system. Consistent with our findings, previous studies of hospitalized patients support the relation of safety climate to fewer reported medication errors,42 fewer incident reports,43 and lower incidence of patient safety indicators.44 The situation is more complex for nonintercepted near misses, in which there are many different forces at play. For example, one could argue that an ED with higher safety scores would have more near misses because of greater documentation of medical errors with the potential to cause injury. On the other hand, an ED with higher safety scores would tend to have more intercepted near misses because of system design changes meant to capture these events; in this scenario, this would tend to drive nonintercepted near misses down. After much discussion, the consensus of the National Emergency Department Safety Study investigators was that we would find no association between safety climate and nonintercepted near misses, which was what we found. Our study identified systems factors that could be modified to reduce errors, including staffing, nursing, safety culture, and triage and monitoring. As documented in the Institute of Medicine report,8 the nation’s urban EDs are crowded and “at the breaking point.” As evidence of crowding-related errors increases,45,46 proper staffing levels are critically important in meeting patient care demand and preventing errors in the ED. Unfortunately, the nationwide nursing shortage further exacerbates the problem. Lower nurse staffing levels have been linked with poorer patient outcomes, such as inpatient mortality, in acute care settings.47 Our data suggest that, as frontline providers of care, nurses are in key positions to intercept a medical error before it affects a patient. Finally, among these systems factors, safety culture is also malleable to targeted interventions. EDs might adopt the approach of a patient safety program for ICUs, which involved conducting a culture survey, identifying staff safety concerns, and implementing improvements; the program successfully reduced medication errors and length of stay.48 We did not find an association between safety climate and serious guideline violations. The factors affecting safety appear to be distinct from those affecting some other aspects of quality, such as adherence to promulgated guidelines. A potential explanation might be that guideline compliance is more determined by physicians than nurses who may be more aware of safety climate. Alternatively, guideline compliance may reflect translating evidence into care more than preventing rare events. Studies of administrative data also found little relationship between patient safety indicators and measures of health care quality.49,50 Although the Institute of Medicine report51 defines safety as one of the 6 aims for quality improvement (safe, effective, patientcentered, timely, efficient, and equitable), our data suggest Volume , .  : November 

Safety Climate and Medical Errors that the creation or development of safer EDs requires targeted error-reduction initiatives. In summary, medical errors are relatively common in the ED, as in other clinical settings. Among the 3 conditions studied, more than one third of adverse events were preventable and there were many documented near misses. Assuming no bias and a correct model of safety climate’s effect on error, our data suggest that improved ED safety climate may increase the likelihood that near misses will be intercepted but has no association with serious guideline violations. Although the National Emergency Department Safety Study survey instrument requires further validation efforts, our data suggest that it may help to prospectively identify patient safety concerns in individual EDs for further interventions. The authors acknowledge the participating site investigators for their ongoing dedication to emergency medicine and patient safety research and numerous Emergency Medicine Network staff for their important contributions throughout the study. Supervising editor: David L. Schriger, MD, MPH Author contributions: CAC, PDC, JAG, EG, RK, DJM, SRR, and DB conceived and designed the study. DB obtained research funding. CAC, AFS, JAG, DJM, and DB collected the data. CAC and C-LT analyzed the data and drafted the article. All authors contributed substantially to article revision. CAC supervised the study and takes responsibility for the paper as a whole. Funding and support: By Annals policy, all authors are required to disclose any and all commercial, financial, and other relationships in any way related to the subject of this article as per ICMJE conflict of interest guidelines (see www.icmje.org). The authors have stated that no such relationships exist. This study was supported by grant R01 HS013099 from the Agency for Healthcare Research and Quality (Rockville, MD). Publication dates: Received for publication May 3, 2011. Revisions received August 8, 2011; January 11, 2012; February 2, 2012; and February 8, 2012. Accepted for publication February 13, 2012. Address for correspondence: Carlos A. Camargo, Jr, MD, DrPH, E-mail [email protected]. REFERENCES 1. Kohn LT, Corrigan J, Donaldson MS; Institute of Medicine; Committee on Quality of Health Care in America. To Err Is Human: Building a Safer Health System. Washington, DC: National Academy Press; 2000. 2. Brennan TA, Leape LL, Laird NM, et al. Incidence of adverse events and negligence in hospitalized patients. Results of the Harvard Medical Practice Study I. N Engl J Med. 1991;324:370-376. 3. Leape LL, Brennan TA, Laird N, et al. The nature of adverse events in hospitalized patients. Results of the Harvard Medical Practice Study II. N Engl J Med. 1991;324:377-384. 4. Thomas EJ, Studdert DM, Burstin HR, et al. Incidence and types of adverse events and negligent care in Utah and Colorado. Med Care. 2000;38:261-271.

Annals of Emergency Medicine 561

Safety Climate and Medical Errors 5. Gandhi TK, Weingart SN, Borus J, et al. Adverse drug events in ambulatory care. N Engl J Med. 2003;348:1556-1564. 6. Rothschild JM, Landrigan CP, Cronin JW, et al. The Critical Care Safety Study: the incidence and nature of adverse events and serious medical errors in intensive care. Crit Care Med. 2005;33: 1694-1700. 7. Gurwitz JH, Field TS, Avorn J, et al. Incidence and preventability of adverse drug events in nursing homes. Am J Med. 2000;109:87-94. 8. Institute of Medicine; Committee on the Future of Emergency Care in the US Health System. The Future of Emergency Care in the United States Health System. Washington, DC: National Academy Press; 2006. 9. Hafner JW Jr, Belknap SM, Squillante MD, et al. Adverse drug events in emergency department patients. Ann Emerg Med. 2002; 39:258-267. 10. Fordyce J, Blank FS, Pekow P, et al. Errors in a busy emergency department. Ann Emerg Med. 2003;42:324-333. 11. Hall KK, Schenkel SM, Hirshon JM, et al. Incidence and types of non-ideal care events in an emergency department. Qual Saf Health Care. 2010;19(suppl 3):i20-25. 12. Leape LL, Berwick DM. Five years after To Err Is Human: what have we learned? JAMA. 2005;293:2384-2390. 13. Colla JB, Bracken AC, Kinney LM, et al. Measuring patient safety climate: a review of surveys. Qual Saf Health Care. 2005;14:364-366. 14. Sullivan AF, Camargo CA Jr, Cleary PD, et al. The National Emergency Department Safety Study: study rationale and design. Acad Emerg Med. 2007;14:1182-1189. 15. Magid DJ, Sullivan AF, Cleary PD, et al. The safety of emergency care systems: results of a survey of clinicians in 65 US emergency departments. Ann Emerg Med. 2009;53: 715-723.e1. 16. National Asthma Education and Prevention Program. Expert Panel Report 3 (EPR-3): guidelines for the diagnosis and management of asthma—summary report 2007. J Allergy Clin Immunol. 2007;120(5 Suppl):S94-138. Available at: http://www.nhlbi.nih.gov/guidelines/ asthma/asthgdln.htm. Accessed April 9, 2011. 17. Godwin SA, Caro DA, Wolf SJ, et al. Clinical policy: procedural sedation and analgesia in the emergency department. Ann Emerg Med. 2005;45:177-196. 18. Antman EM, Anbe DT, Armstrong PW, et al. ACC/AHA guidelines for the management of patients with ST-elevation myocardial infarction— executive summary: a report of the American College of Cardiology/American Heart Association Task Force on Practice Guidelines (Writing Committee to Revise the 1999 Guidelines for the Management of Patients With Acute Myocardial Infarction). Circulation. 2004;110:588-636. 19. Tsai CL, Magid DJ, Sullivan AF, et al. Quality of care for acute myocardial infarction in 58 US emergency departments. Acad Emerg Med. 2010;17:940-950. 20. Tsai CL, Sullivan AF, Gordon JA, et al. Quality of care for acute asthma in 63 US emergency departments. J Allergy Clin Immunol. 2009;123:354-361. 21. Department of Health and Human Services. The International Classification of Diseases, Ninth Revision, Clinical Modification: ICD-9CM. 5th ed. Vol 1. Washington, DC: Government Printing Office; 1994. 22. Thomas EJ, Studdert DM, Newhouse JP, et al. Costs of medical injuries in Utah and Colorado. Inquiry. 1999;36:255-264. 23. Bates DW, Cullen DJ, Laird N, et al. Incidence of adverse drug events and potential adverse drug events. Implications for prevention. ADE Prevention Study Group. JAMA. 1995;274:29-34. 24. Kaushal R, Bates DW, Landrigan C, et al. Medication errors and adverse drug events in pediatric inpatients. JAMA. 2001;285: 2114-2120.

562 Annals of Emergency Medicine

Camargo et al 25. Altman DG. Practical Statistics for Medical Research. London, England: Chapman & Hall; 1991. 26. Reason J. Human error: models and management. BMJ. 2000; 320:768-770. 27. Reason JT. Human Error. Cambridge, England: Cambridge University Press; 1990. 28. United States Census Bureau. Census regions and divisions of the United States. Available at: http://www.census. gov/geo/www/us_regdiv.pdf. Accessed April 9, 2011. 29. Liang KY, Zeger SL. Longitudinal data analysis using generalized linear models. Biometrika. 1986;73:13-22. 30. Tsai CL, Camargo CA Jr. Methodological considerations, such as directed acyclic graphs, for studying “acute on chronic” disease epidemiology: chronic obstructive pulmonary disease example. J Clin Epidemiol. 2009;62:982-990. 31. Krumholz HM, Wang Y, Mattera JA, et al. An administrative claims model suitable for profiling hospital performance based on 30-day mortality rates among patients with an acute myocardial infarction. Circulation. 2006;113:1683-1692. 32. Tsai CL, Clark S, Sullivan AF, et al. Development and validation of a risk-adjustment tool in acute asthma. Health Serv Res. 2009; 44:1701-1717. 33. Greenland S. Modeling and variable selection in epidemiologic analysis. Am J Public Health. 1989;79:340-349. 34. Greenland S. Dose-response and trend analysis in epidemiology: alternatives to categorical analysis. Epidemiology. 1995;6:356-365. 35. Austin PC, Goel V, van Walraven C. An introduction to multilevel regression models. Can J Public Health. 2001;92: 150-154. 36. Afifi AA, Kotlerman JB, Ettner SL, et al. Methods for improving regression analysis for skewed continuous or counted responses. Annu Rev Public Health. 2007;28:95-111. 37. Caplan RA, Posner KL, Cheney FW. Effect of outcome on physician judgments of appropriateness of care. JAMA. 1991; 265:1957-1960. 38. Gupta M, Schriger DL, Tabas JA. The presence of outcome bias in emergency physician retrospective judgments of the quality of care. Ann Emerg Med. 2011;57:323-328.e9. 39. Niska R, Bhuiya F, Xu J. National Hospital Ambulatory Medical Care Survey: 2007 emergency department summary. National Health Statistics Reports. Hyattsville, MD: National Center for Health Statistics; 2010. No. 26. Available at: http://www.cdc.gov/nchs/data/nhsr/nhsr026.pdf. Accessed August 28, 2012. 40. Streiner DL, Norman GR. Health Measurement Scales: A Practical Guide to Their Development and Use. 3rd ed. New York, NY: Oxford University Press; 2003. 41. Neal A, Griffin MA. A study of the lagged relationships among safety climate, safety motivation, safety behavior, and accidents at the individual and group levels. J Appl Psychol. 2006;91:946-953. 42. Vogus TJ, Sutcliffe KM. The impact of safety organizing, trusted leadership, and care pathways on reported medication errors in hospital nursing units. Med Care. 2007;45:997-1002. 43. Weingart SN, Farbstein K, Davis RB, et al. Using a multihospital survey to examine the safety culture. Jt Comm J Qual Saf. 2004; 30:125-132. 44. Singer S, Lin S, Falwell A, et al. Relationship of safety climate and safety performance in hospitals. Health Serv Res. 2009;44: 399-421. 45. Weissman JS, Rothschild JM, Bendavid E, et al. Hospital workload and adverse events. Med Care. 2007;45:448-455. 46. Liu SW, Thomas SH, Gordon JA, et al. A pilot study examining undesirable events among emergency department– boarded

Volume , .  : November 

Camargo et al

47.

48.

49.

50.

51.

patients awaiting inpatient beds. Ann Emerg Med. 2009;54:381-385. Clarke SP. Nurse staffing in acute care settings: research perspectives and practice implications. Jt Comm J Qual Patient Saf. 2007;33:30-44. Pronovost PJ, Weast B, Bishop K, et al. Senior executive adopt-awork unit: a model for safety improvement. Jt Comm J Qual Saf. 2004;30:59-68. Isaac T, Jha AK. Are patient safety indicators related to widely used measures of hospital quality? J Gen Intern Med. 2008;23: 1373-1378. Miller MR, Pronovost P, Donithan M, et al. Relationship between performance measurement and accreditation: implications for quality of care and patient safety. Am J Med Qual. 2005;20:239252. Institute of Medicine. Committee on Quality of Health Care in America. Crossing the Quality Chasm: A New Health System for the 21st Century. Washington, DC: National Academy Press; 2001.

APPENDIX Emergency Medicine Network Steering Committee: Carlos A. Camargo, Jr, MD, DrPH (Chair); Sunday Clark, ScD; Adit A. Ginde, MD, MPH; Robert A. Lowe, MD, MPH; Jonathan M. Mansbach, MD; Ashley F. Sullivan, MPH, MS; and Scott T. Wilber, MD, MPH. Emergency Medicine Network Coordinating Center: Carlos A. Camargo, Jr, MD, DrPH (Director); Erica Eagan; Janice A. Espinola, MPH; Tate Forgey, MA; Kathryn M. Niro; Susan A. Rudders, MD; Anne P. Steptoe; Ashley F. Sullivan, MS, MPH; Chu-Lin Tsai, MD, ScD; and Milo F. Vassallo, MD, PhD (all at Massachusetts General Hospital, Boston). Principal Investigators at the 62 Participating Sites: J.F. Acosta (St. Barnabas Hospital, Bronx, NY); H.J. Alter (Highland General Hospital, Oakland, CA); J.M. Basior (Buffalo General Hospital, Buffalo, NY); R.S. Benenson (York Hospital, York, PA); S.L. Bernstein (Montefiore Medical Center, Bronx, NY); W.F. Bond (Lehigh Valley Hospital, Allentown, PA); C.B. Cairns (Duke University Medical Center, Durham, NC); J.M. Caterino (Ohio State University Hospital and Ohio State University Hospital East, Columbus, OH); R. Coleman and E.A. Hooker, II (University of Louisville Hospital, Louisville, KY); R.K. Cydulka (MetroHealth Medical Center, Cleveland, OH); L.C. Degutis (Yale–New Haven Medical Center, New Haven, CT); D.B. Diercks (UC Davis Medical Center, Sacramento, CA); S.K. Epstein (Beth Israel Deaconess Medical Center, Boston, MA); R.J. Fairbanks (University of Rochester Medical Center, Rochester, NY); J.A. Feldman (Boston Medical Center, Boston, MA); G.M. Gaddis (Saint Luke’s Hospital of Kansas City, Kansas City, MO);

Volume , .  : November 

Safety Climate and Medical Errors T.J. Gaeta (New York Methodist Hospital, Brooklyn, NY); R.M. Goldberg (Providence Saint Joseph Medical Center, Burbank, CA); R.O. Gray (Hennepin County Medical Center, Minneapolis, MN); M. Griffin (Sinai-Grace Hospital, Detroit, MI); J.W. Hafner, Jr (OSF Saint Francis Medical Center, Peoria, IL); B.J. Hahn (Staten Island University Hospital, Staten Island, NY); F.P. Harchelroad, Jr (Allegheny General Hospital, Pittsburgh, PA); J.S. Haukoos (Denver Health Medical Center, Denver, CO); D.S. Huckins (Newton-Wellesley Hospital, Newton, MA); M. Huston (Franklin Square Hospital Center, Baltimore, MD); D. Johnson (St. Vincent Mercy Medical Center, Toledo, OH); K.A. Jones (Detroit Receiving Hospital, Detroit, MI); S. Key (Cape Canaveral Hospital, Cocoa Beach, FL); H. Kilgannon (Cooper University Hospital, Camden, NJ); E. Lashley (Maimonides Medical Center, Brooklyn, NY); M.J. Leber (Brooklyn Hospital Center, Brooklyn, NY); S. Liu (Massachusetts General Hospital, Boston, MA); L.F. Lobon (Caritas St. Elizabeth’s Medical Center, Boston, MA); B.L. Lopez (Thomas Jefferson University Hospital, Philadelphia, PA); F. LoVecchio (Maricopa Medical Center, Phoenix, AZ); E.L. Lynch (Loma Linda University Medical Center and Children’s Hospital, Loma Linda, CA); D.J. Magid (Saint Joseph Hospital, Denver, CO); J.S. Myslinski (Palmetto Richland Memorial Hospital, Columbia, SC); N.W. Naviaux (University of Colorado Hospital, Aurora, CO); D.J. Pallin (Brigham and Women’s Hospital, Boston, MA); A. Papa (Doylestown Hospital, Doylestown, PA); S.K. Polevoi (UCSF Medical Center, San Francisco, CA); B.D. Probst (Loyola University Medical Center, Maywood, IL); M.S. Radeos (Lincoln Medical Center, Bronx, NY); G.R. Ramalanjaona (Saint Michael’s Medical Center, Newark, NJ); T.J. Reeder (Pitt County Memorial Hospital, Greenville, NC); S. Reingold (Advocate Christ Medical Center, Oak Lawn, IL); D.J. Robinson (Memorial Hermann Hospital, Houston, TX); A. Sacchetti (Our Lady of Lourdes Medical Center, Camden, NJ); S. Sallustio (Elmhurst Hospital Center, Elmhurst, NY); M. Sigal (Salem Hospital, Salem, MA); H.A. Smithline (Baystate Medical Center, Springfield, MA); B.K. Snyder (UCSD Medical Center–Hillcrest, San Diego, CA); L.A. Starke (Kettering Medical Center, Kettering, OH); E. Thallner (Cleveland Clinic Hospital, Cleveland, OH); K.H. Todd (Beth Israel Medical Center, New York, NY); T.P. Tran (University of Nebraska Medical Center, Omaha, NE); D.R. Vinson (Kaiser Permanente Roseville Medical Center, Roseville, CA); E. Wang (Stanford University Medical Center, Stanford, CA); and S.T. Wilber (Akron City Hospital, Akron, OH).

Annals of Emergency Medicine 563

Safety Climate and Medical Errors

Camargo et al

APPENDIX E1. The National Emergency Department Safety Study Survey.

National Emergency Department Safety Study Funded by the Agency for Healthcare Research and Quality

Dear Colleague: We would like your help with a major study of quality and safety in emergency departments (EDs). As part of this federally funded study, we are surveying clinicians like yourself who work in EDs about factors that might affect the quality of care in your ED. The study, entitled the National Emergency Department Safety Study (NEDSS), is being conducted in 85 EDs around the country. The first step is this survey, which asks about working conditions and care in EDs. The second step involves chart reviews of patients presenting with acute myocardial infarction, asthma and dislocations. Without your response to this survey, however, the study cannot succeed. The survey takes about 15 minutes to complete, and results will be completely confidential. In fact, your name will not appear on the survey and no individual survey responses will be shared with anyone in your ED. Only aggregate responses will be shared and published. Participation is completely voluntary. Your decision whether or not to participate will not affect your employment. If you do not want to answer a question, feel free to skip it. We know how busy you are, and how many requests of this type you receive. We hope you will take the time to complete this survey, however, because we are confident that the resulting data will help inform efforts to improve the quality and safety of ED care. Thank you for your assistance. Best wishes,

David Blumenthal MD, MPP Principal Investigator Director, Institute for Health Policy

Carlos Camargo, MD, DrPH Co-investigator Director, Emergency Medicine Network

1

563.e1 Annals of Emergency Medicine

# __ __ __ __ __ __ __ __

Volume , .  : November 

Camargo et al

Safety Climate and Medical Errors

INSTRUCTIONS • If you work in more than one ED, please answer the survey questions based on your experience at the site where you received this survey. • If you have no idea how to answer a question, please provide your best guess. Your responses are completely confidential. A.

Are you an ED attending, fellow, intern, nurse practitioner, LPN, physician assistant, RN, nursing assistant or equivalent, or resident who works at least one shift a week? 1

Yes

0

No

DO NOT CONTINUE. Please return the questionnaire in the envelope provided, and we will remove your survey ID number from our list. Thank you.

B. Within the last 6 months, has your work been predominantly in “acute” or “emergent” areas of the ED or the “fast track” or “urgent care” areas of the ED? 1

“acute” or “emergent “

2

“fast track” or “urgent care”

3

both “acute” / “emergent” AND “fast track” / “urgent care”

A. PHYSICAL EQUIPMENT AND CHARACTERISTICS OF THE ED A1.

Please indicate how often the following things occur in your ED. Check one.

Never

Rarely

Sometimes

Most of the time

Always

a. There is sufficient space in the ED for the delivery of care

1

2

3

4

5

b. Monitoring devices (e.g., pulse oximeter, vital sign monitor, or cardiac monitor) function in the ED

1

2

3

4

5

c. Sick patients receive care in the hallway

1

2

3

4

5

d. Functioning routine physical exam equipment (e.g., otoscope, lights for a gynecological exam, ophthalmoscope, manual blood pressure cuff) is available at the patient’s bedside

1

2

3

4

5

e. There is a clear way to identify patient location in the ED (e.g., room 7, hallway bed G)

1

2

3

4

5

f. Urgently needed medications (e.g., albuterol, dopamine) are stocked in the ED

1

2

3

4

5

g. All patient beds can be easily observed by ED staff

1

2

3

4

5

h. Stretchers are available

1

2

3

4

5

i. The number of computers in the ED is adequate

1

2

3

4

5

j. Patients are cared for in more than one bed space during their ED stay

1

2

3

4

5

1

2

3

4

5

k. ED personnel feel physically safe while working in the ED

Volume , .  : November 

Annals of Emergency Medicine 563.e2

Safety Climate and Medical Errors

Camargo et al

B. STAFFING B1. Please indicate how often the following things occur in your ED. Check one. Never

a. Supervisors intervene to manage incompetent staff

Rarely

Sometimes

Most of the time

Always

1

2

3

4

5

1

2

3

4

5

1

2

3

4

5

1

2

3

4

5

1

2

3

4

5

1

2

3

4

5

1

2

3

4

5

1

2

3

4

5

1

2

3

4

5

1

2

3

4

5

1

2

3

4

5

1

2

3

4

5

1

2

3

4

5

1

2

3

4

5

o. It is easy to find the nurse caring for a patient

1

2

3

4

5

p. Nurses have support from ancillary personnel (e.g., techs) when needed

1

2

3

4

5

b. Attendings review the care provided for every patient by a resident before the patient leaves the ED c. Doctors and nurses work well together in the ED d. Physician staffing is sufficient to handle the patient care load during busy periods e. It is easy to find the physician caring for a patient f. Important clinical information is lost in handoffs between physicians at the change of shift g. Interruptions disrupt the ability of staff to provide care h. The number of patients cared for exceeds the capacity of the ED to provide safe care i. There is good communication of patient care plans between ED physicians and ED nurses j. Important clinical information is lost in handoffs between nurses at the change of shift k. Nurses feel comfortable questioning orders l. Triage nurses are well trained in emergency assessment m. Nurse staffing is sufficient to handle the patient care load during busy periods n. New nurses are well monitored by nurses with more experience

C. ORGANIZATIONAL FACTORS IN THE ED C1.

Please indicate how often the following things occur in your ED. Check one. Never

a. Individuals are blamed when safety problems occur b. ED administrators are more concerned with getting work done than with patient safety c. Hospital administrators support improvement in patient safety in the ED d. Mistakes that hurt patients are reported to supervisors

Rarely

Sometimes

Most of the time

Always

1

2

3

4

5

1

2

3

4

5

1

2

3

4

5

1

2

3

4

5

e. In providing clinical care, patient safety is a top priority in the ED

1

2

3

4

5

f. Efforts are underway to improve patient safety

1

2

3

4

5

1

2

3

4

5

1

2

3

4

5

g. Staff feel comfortable raising concerns about the safety of patient clinical care with ED supervisors h. ED leadership takes action to improve safety of clinical care for patients in the ED

563.e3 Annals of Emergency Medicine

Volume , .  : November 

Camargo et al

Safety Climate and Medical Errors

D. COORDINATION OF CARE IN THE ED D1.

Please indicate how often the following things occur in your ED. Check one. Never

a. A pharmacist is available to answer my questions within 5 minutes b. Patients are transferred to an inpatient bed in less than 1 hour after bed is requested by the ED c. Patients triaged back to the waiting room are monitored as often as necessary for their clinical condition d. Patient triage works well in my ED e. When treatment begins, ED staff know the patient’s current allergies f. It is difficult to gain access to a patient’s past medical record when needed g. Vital signs of acutely ill patients are monitored as often as necessary for their clinical condition h. Emergent CT scans are completed within a half hour i. Specialty consultation for critically ill patients arrives within 30 minutes of being contacted j. A radiology attending is available within 30 minutes for test interpretation at the request of ED staff k. Specialty consultants for critically ill patients can be contacted in 15 minutes l. When treatment begins, ED staff know the patient’s current medications m. It is difficult to locate an ED chart when needed n. Protocols are used for complex medication administration such as thrombolytic therapy o. ED patients requiring admission to an ICU are transferred within 1 hour of ordering a bed p. Stat medications are administered within 15 minutes of ordering q. Translators are available within an hour of request

Sometimes

Rarely

Most of the time

Always

1

2

3

4

5

1

2

3

4

5

1

2

3

4

5

1

2

3

4

5

1

2

3

4

5

1

2

3

4

5

1

2

3

4

5

1

2

3

4

5

1

2

3

4

5

1

2

3

4

5

1

2

3

4

5

1

2

3

4

5

1

2

3

4

5

1

2

3

4

5

1

2

3

4

5

1

2

3

4

5

1

2

3

4

5

E. CONDITION-SPECIFIC QUESTIONS The following statements refer to specific issues related to the treatment of AMI, dislocations, and asthma in the ED.

E1. Acute Myocardial Infarction How often do you see patients with a suspected AMI? 1 Rarely 2

Sometimes

3

Frequently

Please indicate how often the following things occur in your ED. 0%

a. The first EKG is done within 10 minutes b. A chest pain checklist (e.g., template, guideline) is used to document care

Volume , .  : November 

19%

1019%

2029%

3039%

4049%

5059%

6069%

7079%

8089%

90100%

1

2

3

4

5

6

7

8

9

10

11

1

2

3

4

5

6

7

8

9

10

11

Annals of Emergency Medicine 563.e4

Safety Climate and Medical Errors

Camargo et al

E1. Acute Myocardial Infarction (continued) 0%

c. ED patients with symptoms of a possible AMI who have no contraindications and who have not already taken an aspirin or other antiplatelet agent that day are given an aspirin or other anti-platelet agent d. ED patients with an AMI who have no contraindications are given a betablocker

19%

1019%

2029%

3039%

4049%

5059%

6069%

7079%

8089%

90100%

1

2

3

4

5

6

7

8

9

10

11

1

2

3

4

5

6

7

8

9

10

11

6

7

8

9

10

11

Does your ED ever use fibrinolytic therapy for reperfusion of AMI patients? 1 0

Yes No

answer question e below skip to question f

e. When fibrinolytic therapy is administered to an AMI patient, it is given within 30 minutes of ED arrival

1

2

3

4

5

Does your ED ever use primary PCI (percutaneous coronary intervention) for reperfusion of AMI patients? 1 0

f.

Yes No

answer question f below skip to section E2 (asthma)

When PCI is performed in an AMI patient, the patient is transferred to the cath lab within 60 minutes of ED arrival

1

2

3

4

5

6

7

8

9

10

11

1019%

2029%

3039%

4049%

5059%

6069%

7079%

8089%

90100%

E2. Asthma How often do you see patients with asthma? 1 Rarely 2

Sometimes

3

Frequently

Please indicate how often the following things occur in your ED.

0%

a. ED patients with an asthma exacerbation have an objective measure of pulmonary function (peak flow or FEV1) checked within 30 minutes of ED arrival b. ED patients with an asthma exacerbation are given their first albuterol treatment within 15 minutes of ED arrival c. ED patients with asthma exacerbation are given systemic corticosteroids d. Among those patients who get systemic corticosteroids, it is given within 75 minutes of ED arrival

563.e5 Annals of Emergency Medicine

19%

1

2

3

4

5

6

7

8

9

10

11

1

2

3

4

5

6

7

8

9

10

11

1

2

3

4

5

6

7

8

9

10

11

1

2

3

4

5

6

7

8

9

10

11

Volume , .  : November 

Camargo et al

Safety Climate and Medical Errors

E2. Asthma (continued) 0%

e. ED patients with an asthma exacerbation have an objective measure of pulmonary function (peak flow or FEV1) checked before discharge from the ED (i.e., after receiving treatment) f. ED patients with an asthma exacerbation who are sent home are discharged on inhaled corticosteroids . g. ED patients with an asthma exacerbation are treated with antibiotics

19%

1019%

2029%

3039%

4049%

5059%

6069%

7079%

8089%

90100%

1

2

3

4

5

6

7

8

9

10

11

1

2

3

4

5

6

7

8

9

10

11

1

2

3

4

5

6

7

8

9

10

11

E3. Dislocations How often do you see patients with a dislocation? 1 Rarely 2

Sometimes

3

Frequently

Please indicate how often the following things occur in your ED.

0%

a. ED patients with shoulder, elbow, hip, knee, or ankle dislocations who have no contraindications to analgesics are given pain medication (including ibuprofen, acetaminophen) within 30 minutes of ED arrival

1

19%

1019%

2029%

3039%

4049%

5059%

6069%

7079%

8089%

90100%

3

4

5

6

7

8

9

10

11

2

Among patients undergoing procedural sedation for a dislocation: 0%

b. Patients are administered supplemental oxygen prior to the procedure c. Patients have an ASA classification documented on the chart prior to the procedure d. Patients have vital signs documented on the chart every 15 minutes during the procedure e. Nurses caring for a patient are free of other responsibilities both during the procedure and for 30 minutes after the procedure

19%

1019%

2029%

3039%

4049%

5059%

6069%

7079%

8089%

90100%

1

2

3

4

5

6

7

8

9

10

11

1

2

3

4

5

6

7

8

9

10

11

1

2

3

4

5

6

7

8

9

10

11

1

2

3

4

5

6

7

8

9

10

11

6

Volume , .  : November 

Annals of Emergency Medicine 563.e6

Safety Climate and Medical Errors

Camargo et al

F. BACKGROUND INFORMATION This background information will help in the analysis of the survey results. Your answers will be kept CONFIDENTIAL. F1.

How long have you worked in this hospital’s ED? YEARS

F6.

MARK ALL THAT APPLY

OR

If < 1 year

MONTHS

F7. F2.

Please list your degrees.

Typically, how many hours per week do you work in the ED?

1

Nursing degree

2

Master’s degree

3

MD / DO

Please indicate your age. MARK ONE

HOURS

F3. What day of the week and time was your last ED shift? Day: _________________________

1

18 - 29

2

30 - 39

3

40 - 49

4

50 - 59

5

60 - 69

6

70+

Start time: __ __: __ __ a.m. / p.m. [please circle] Stop time: __ __: __ __ a.m. / p.m. [please circle]

F8.

Are you of Latino or Hispanic origin? MARK ONE

F4.

What is your position in this hospital?

1

Yes

MARK ONE

0

No

1

Attending / Staff Physician

2

Resident

3

Nurse Practitioner

4

Nurse

5

Physician Assistant

6

Other (Please Specify)

a. If you are an RN, are you CEN certified?

F9.

Please indicate your race. CHECK ALL THAT APPLY 1

Black or African American

2

Asian

3

American Indian/Alaska Native

4

Native Hawaiian or Other Pacific Islander

5

White

6

Other (Please Specify)

MARK ONE 1

Yes

0

No

b. If you are a physician, are you Board certified in Emergency Medicine? MARK ONE

F5.

1

Yes

0

No

Please indicate your sex below. MARK ONE 1

Female

2

Male

563.e7 Annals of Emergency Medicine

Thank you for taking the time to complete this important survey.

RETURN INSTRUCTIONS Please return your completed questionnaire in the postage-paid envelope provided. If you’ve misplaced the envelope, please send your questionnaire to: Attn: Ashley Sullivan EMNet Coordinating Center Massachusetts General Hospital 326 Cambridge Street, Suite 410 Boston, MA 02114

Volume , .  : November 

Camargo et al

Safety Climate and Medical Errors

Table E1. Exclusion criteria for chart review of three conditions. Acute Myocardial Infarction ● ● ● ●

Age ⱖ 90 years Transferred from another acute care facility Cardiac arrest prior to or within 15 minutes of ED arrival Cardiac enzymes not elevated within 24 hours or ED arrival

Acute Asthma ● ● ● ●

Age ⱕ 13 or ⱖ 55 years History of chronic obstructive pulmonary disease or emphysema No history of asthma before index visit ED visit not prompted, in large part, by asthma exacerbation

Joint Dislocation ● ● ● ● ●

Age ⱕ 13 or ⱖ 90 years Acromioclavicular shoulder dislocation No dislocated joint of interest No joint relocation procedure No intravenous or intramuscular sedative or anesthetic administered

ED, emergency department.

Volume , .  : November 

Annals of Emergency Medicine 563.e8

Safety Climate and Medical Errors

Camargo et al

APPENDIX E2. The National Emergency Department Safety Study Adverse Event Form.

National Emergency Department Safety Study

Adverse Event Form

1. Abstractor Initials

__ __ __

2. Site ID

__ __ __

3. Patient ID

__ __ __

A. Adverse Events A1. The patient returned to this ED for re-evaluation or was admitted to this hospital within 48 hours after treatment at this ED. Yes No A2. The patient died in the ED. Yes

No

A3. The patient had a cardiac or respiratory arrest in the ED. Yes

No

A4. There was an injury or complication related to a major procedure conducted in the ED (e.g. central line placement, temporary pacer placement, intubation, thoracentesis, chest tube placement). [Check ED chart and hospital discharge summary.] Yes

No

Not Documented

No major procedure

A4a. If yes, please describe the injury or complication:

A5. There was an injury or complication related to any non-major procedure (e.g. peripheral IV) in the ED. Yes

No

Not Documented

No non-major procedure

A5a. If yes, please describe the injury or complication:

Revised 2/05

563.e9 Annals of Emergency Medicine

Volume , .  : November 

Camargo et al

Safety Climate and Medical Errors

A6. The patient stayed in the ED for longer than 2 hours post intubation. Yes

No

Not Documented

Not Intubated

A7. The patient had a surgical airway procedure (e.g. cricothyroidotomy) performed in the ED for intubation. Yes

No or Not Documented

Not Intubated

A8. At this ED visit, the patient was diagnosed as having an open fracture or a bacterial infection (e.g. pneumonia) and did NOT receive antibiotics in the ED. Yes

No

No Open Fracture or Bacterial Infection

A9. At this ED visit, the patient was diagnosed as having an open fracture or a bacterial infection (e.g. pneumonia), and did NOT receive a prescription for antibiotics at the time of discharge from the ED. Yes No

N/A (e.g. No Open Fracture or Bacterial Infection OR Not Discharged)

A10. The patient had a delay in blood transfusion for greater than 2 hours after ordering. Yes

No or Not Documented

Did Not Require Transfusion

A11. There was a critical lab value in the ED (see list in manual). Yes

No

A12. Patient had a central line, excluding femoral lines, WITHOUT a post procedure chest x-ray to check for placement in the ED. Yes

No

No central line

A13. Patient was admitted to the hospital with a diagnosis of asthma or a MI WITHOUT having a chest x-ray in the ED. Yes

No

No diagnosis of asthma or MI

Not admitted

A14. There was a complication due to a delay in treatment such as surgery, coronary re-perfusion or antibiotics. [Check ED chart and hospital discharge summary.] Yes Site ID __ __ __ Patient ID __ __ __

Volume , .  : November 

No

Not Documented 2

Annals of Emergency Medicine 563.e10

Safety Climate and Medical Errors

Camargo et al

A15. The patient had a possible adverse drug event. Yes

No

A15a. If yes, please describe each possible adverse drug event:

A16. The patient had at least one other undesirable outcome. Yes

No

A16a. If yes, please describe each additional undesirable outcome:

Please continue to Section B (Medication Event Form).

Site ID __ __ __ Patient ID __ __ __

563.e11 Annals of Emergency Medicine

Volume , .  : November 

Camargo et al

Safety Climate and Medical Errors National Emergency Department Safety Study

Medication Event Form

1. Abstractor Initials

__ __ __

2. Site ID

__ __ __

3. Patient ID

__ __ __

Form B should be filled out for EACH medication event or error, including any medication error associated with an adverse drug event previously identified on Form A. Form B should also be filled out for any adverse drug event (including those identified on Form A) whether or not the adverse drug event was caused by a medication error. This is form # ___ of ___ .

B1. There was a medication error (defined as an error in the process of ordering, transcribing, dispensing, administering or monitoring a medication).

___ 1 Yes 2 No

If yes: B1a. There was significant potential for injury from this medication error, but no injury occurred (i.e. a near-miss) (e.g. amoxicillin ordered in a patient with a known allergy who is given the medication but did not develop a reaction OR amoxicillin ordered in a patient with a known allergy who did not receive the medication because the nurse intercepted the incorrect order).

___ 1 Yes 2 No

B1b. There was actual harm or injury from this medication error (i.e. a preventable adverse drug event).

___ 1 Yes 2 No

B2. There was actual harm or injury to a patient but no identifiable error (i.e. a non-preventable adverse drug event).

___ 1 Yes 2 No

If you answered “NO” to BOTH B1 and B2, STOP HERE. Otherwise proceed to the next question.

Site ID __ __ __ Patient ID __ __ __

Volume , .  : November 

Annals of Emergency Medicine 563.e12

Safety Climate and Medical Errors

Camargo et al

B3. Narrative description of event _____________________________________________________________________________________ _____________________________________________________________________________________ _____________________________________________________________________________________ B4. Name of drug

________________________

B5. Category of drug

___ ___

Cardiovascular 1. Beta-blockers 2. Anti-arrhythmic 3. Nitrates 4. IV Vasodilator 5. IV Vasoconstrictor / pressor 6. Inotrope 7. Diuretic 8. Digitalis 9. Calcium channel blocker 10. Other antihypertensive agent 11. Anti-lipemic agent 12. GIIb/IIIa inhibitor 13. Heparin 14. Other anticoagulant 15. Thrombolytic agent

Central Nervous System (CNS)/Pain/anxiety 16. Non-narcotic analgesic 17. Narcotics analgesic 18. Muscle relaxant 19. Sedative, hypnotic 20. Intravenous anesthetic 21. Anti-seizure

Gastrointestinal 29. TPN 30. H2 blocker 31. Other GI agent 32. Proton pump inhibitor 33. Other gastrointestinal agent

Other CNS agents Infectious Disease 22. Antiviral 23. Antifungal 24. Antibiotic

34. 35. 36. 37. 38. 39. 40. 41.

Intravenous Treatment 25. IV Fluids 26. Electrolyte concentration 27. Blood products (RBC,

platelets, FFP) 28. Colloids (albumin,

Other Categories Anti-tumor Diabetes Anti-depressant Anti-psychotic Anti-asthmatic Immunosuppressants Steroids Diagnostic agent (e.g. contrast dye) 42. Other _______________ 43. Antihistamine

hetastarch)

______________________

B6. Dose of drug ordered B7. Route of drug

____ 1 PO 2 IV 3 Inhalation 4 PG/PJ/feeding tube 5 Epidural

B8. Was this a verbal order?

____

6 Topical 7 Subcutaneous 8 Suppository 9 Intramuscular 10 Other ____________________

1 Yes 2 No

Site ID __ __ __ Patient ID __ __ __

563.e13 Annals of Emergency Medicine

Volume , .  : November 

Camargo et al

Safety Climate and Medical Errors

B9. Type of event (may list more than one) ___ ___

___ ___

Wrong patient Wrong drug Drug-drug interaction Dose error a. Overdose b. Underdose c. Dose omitted d. Dose unit omitted e. Incorrect dose form f. Dose illegible 5. Route error a. Route incorrect b. Route omitted c. Route illegible 1. 2. 3. 4.

___ ___

___ ___

___ ___ 9. Date Error a. Date illegible b. Date wrong c. No date 10. Time omitted 11. Inappropriate use of abbreviation 12. Illegible order 13. There was no identifiable error 14. Other, Specify ______________________

Frequency error a. Frequency incorrect b. Frequency omitted c. Frequency illegible 7. Strength Error a. Strength omitted b. Strength incorrect c. Strength without units d. Strength illegible 8. Allergy a. Not documented b. Documented, but ordered c. Allergy illegible 6.

______________________

1

2

3

4

5

6

Physician order

Nurse transcription

Pharm disp

Nurse disp

Nurse admin

Nurse monitoring

____ 1 Physician order

B10. At what level in the above process did the primary failure occur?

2 Nurse transcription 3 Pharmacy dispensing 4 Nurse dispensing 5 Nurse administration 6 Nurse monitoring 7 There was no identifiable failure or error 8 Can’t tell

B10a. At what level did additional failures occur? No additional failures Additional failures:

____

____

____

____

Site ID __ __ __ Patient ID __ __ __

Volume , .  : November 

Annals of Emergency Medicine 563.e14

Safety Climate and Medical Errors B11. For adverse drug events (i.e. answered “YES” to B1b OR B2), what was the system involved?

Camargo et al

___

1 Not an adverse drug event 2 Bleeding 3 CNS 4 Allergic/cutaneous 5 Metabolic 6 Cardiovascular 7 Gastrointestinal 8 Renal 9 Respiratory 10 Marrow depression 11 Other ________________

B12. Any additional medication events during this ED visit? No additional events (STOP) One or more additional medication events (Complete Section B for the next medication event. Please record the # of forms created for this patient in appropriate spot on page 4.)

Site ID __ __ __ Patient ID __ __ __

563.e15 Annals of Emergency Medicine

Volume , .  : November 

Camargo et al

Safety Climate and Medical Errors

Table E2. Reliability of study outcomes. Abbreviations: AE, adverse event; and NM, near miss.

Table 1.1. Agreement on the presence of an adverse event among a random sample of charts screening positive for possible AE or NM. Reviewer Panel A

Reviewer Panel B AE absent

AE present

Total

88 11 99

7 20 27

95 31 126

AE absent AE present Total Kappa statistic ⫽ 0.60

Table 1.2. Agreement on the presence of an adverse event or a near miss among a random sample of charts screening positive for possible AE or NM. Reviewer Panel A

Neither AE NM Total

Reviewer Panel B Neither

AE

NM

Total

48 9 17 74

6 20 1 27

14 2 9 25

68 31 27 126

Kappa statistic ⫽ 0.34

Table 2. Agreement on the preventability of an adverse event among a random sample of charts screening positive for possible AE or NM Reviewer Panel A

Preventable AE absent Preventable AE present Total

Reviewer Panel B Preventable AE absent

Preventable AE present

Total

118 2 120

4 2 6

112 4 126

Kappa statistic ⫽ 0.38

Volume , .  : November 

Annals of Emergency Medicine 563.e16

Safety Climate and Medical Errors

Camargo et al

Table 3. Agreement on whether or not a near miss is intercepted among a random sample of charts screening positive for possible AE or NM Reviewer Panel A

Reviewer Panel B Intercepted NM absent

Intercepted NM present

Total

107 9 116

5 5 10

112 14 126

Intercepted NM absent Intercepted NM presen Total Kappa statistic ⫽ 0.36

Table 4. Agreement on the impact level of an adverse event. Reviewer Panel A

Reviewer Panel B Significant

Serious

Life threatening

Total

6 1 1 8

3 3 0 6

1 1 4 6

10 5 5 20

Significant Serious Life threatening Total Kappa statistic ⫽ 0.46

Table 5. Agreement on the potential impact level of a near miss. Reviewer Panel A

Reviewer Panel B Significant

Serious

Life threatening

Total

3 0 0 3

0 3 1 4

0 0 2 2

3 3 3 9

Significant Serious Life threatening Total Kappa statistic ⫽ 0.83

Medical Errors

Adverse Events

Near Misses

Non-Preventable AE

Preventable AE Figure E1. A Venn diagram illustrating the inter-relationships between the outcome measures in this study.

563.e17 Annals of Emergency Medicine

Volume , .  : November 

Camargo et al

Safety Climate and Medical Errors

Appendix E3. Examples of adverse events and near misses. Preventable AE: An injury resulting from a medical error. Teaching example: Patient with documented allergy to penicillin given dicloxacillin ¡ developed hives. Examples: 1. Initial ECG showed STEMI but no mention in physician notes; cardiac treatment delayed for several hours ¡ cardiac arrest, death. 2. Nitroglycerin given to patient who reported recent use of sildenafil ¡ severe hypotension for 2 hours, required IV fluids. 3. Patient given both beta-blocker and calcium-channel blocker ¡ severe hypotension, required IV fluids. 4. Third dose of metoprolol given to an already bradycardic patient ¡ worsened bradycardia & hypotension, required IV atropine. 5. Patient given heparin, reteplase, eptifibatide and enoxaparin at same time ¡ developed upper GI bleed, with major drop in hematocrit. 6. Hyperglycemia prompted multiple insulin doses and initiation of insulin drip ¡ developed hypoglycemia, required IV D50 solution. 7. Patient given rapid transfusion of packed RBCs to correct minor anemia ¡ onset of acute heart failure, required intubation. 8. ETT placed but had negative C02 test 15 minutes later ¡ new ETT placed ⬃20 minutes after initial ETT placement. 9. Not given any corticosteroids for serious asthma exacerbation ¡ returned to ED within 24 hours for worsened exacerbation, hospitalized. 10. Patient given large doses of midazolam and fentanyl at same time ¡ became hypoxic & obtunded; required flumazenil and naloxone. Non-preventable AE: An unavoidable injury resulting from appropriate medical management. Non-preventable adverse events are not medical errors. Teaching example: Patient with AMI and history of a thromboembolic stroke 14 months ago given tPA ¡ developed intra-cranial hemorrhage. Examples: 1. Given IV nitroglycerin (normal dose, no confounding factors) ¡ hypotension. 2. Given IV heparin (normal dose, no confounding factors) ¡ thrombocytopenia. 3. Given tPA (normal dose, no confounding factors), then required emergent femoral line ¡ developed groin hematoma. 4. Given morphine (normal dose, no confounding factors) ¡ nausea, dry heaves. 5. Peripheral IV placed and working well (no confounding factors) ¡ IV infiltrate with associated pain. 6. Given CT contrast dye (no known allergy) ¡ developed acute urticaria. 7. Given IV levofloxacin (no known allergy) ¡ develop acute urticaria, pruritus. 8. Given IV corticosteroids (normal dose, no confounding factors) ¡ developed intractable vomiting, admitted. 9. Given fentanyl (normal dose, no confounding factors) ¡ hypoxia, required naloxone. 10. Given procedural sedation (no known allergies) ¡ developed acute urticaria. Intercepted near miss: A potentially harmful error that is intercepted before reaching the patient. Teaching example: Percodan (oxycodone ⫹ aspirin) ordered for patient with known aspirin allergy ¡ intercepted, mediciation not given. Examples: 1. Nitroglycerin paste ordered twice (unintentional) ¡ intercepted, second ordered deleted. 2. Nitroglycerin ordered for patient on sildenafil ¡ intercepted, nitroglycerin not given. 3. IV nitroglycerin ordered without BP “hold” parameters ¡ intercepted, hold parameters added before initiation. 4. Beta-blocker ordered for patient with severe bradycardia ¡ intercepted, beta-blocker not given. 5. Beta-blocker ordered in patient with cocaine-induced chest pain ¡ intercepted, beta-blocker not given. 6. IV metoprolol 0.5 mg ordered ¡ intercepted, 5 mg given. 7. Albuterol nebulizer ordered 5 mg q3 ¡ intercepted, order corrected (q20 minutes ⫻ 3). 8. Penicillin ordered for penicillin-allergic patient ¡ intercepted, penicillin not given. 9. MSO4 written instead of morphine sulfate ¡ intercepted (⬙not magnesium⬙), correct medication given. 10. Fentanyl dose written as mg (not mcg) ¡ intercepted, correct dose given. Non-intercepted near miss: A potentially harmful error that unexpectedly does no detectable harm, due to patient characteristics or chance, despite reaching the patient. Teaching example: Serum potassium 6.0 recorded and not rechecked nor treated in the ED ¡ no apparent harm. Examples: 1. IV nitroglycerin ordered without BP “hold” parameters ¡ given without apparent harm. 2. Sublingual nitroglycerin ordered for patient on sildenafil ¡ given without apparent harm. 3. Aspirin ordered in patient with history of aspirin allergy ¡ given without apparent harm. 4. Aspirin 81 mg ⫻ 4 ordered ¡ given only 81 mg without apparent harm. 5. Beta-blocker ordered in patient with cocaine-induced chest pain ¡ given without apparent harm. 6. Heparin 4000 unit bolus ordered ¡ given 8000 unit bolus without apparent harm. 7. IV D50 ordered in patient with blood glucose 703 mg/dl ¡ given without apparent harm. 8. Patient with markedly elevated blood glucose was not given insulin ¡ no apparent harm. 9. Zosyn (piperacillin ⫹ tazobactam) ordered in patient with history of penicillin allergy ¡ given without apparent harm. 10. Patient on digoxin given gatifloxacin (which may increase digoxin level) ¡ no change in digoxin level. Abbreviations: ECG, electrocardiogram; STEMI, ST-elevation myocardial infarction; IV, intravenous; GI, gastrointestinal; RBC, red blood cell; ETT, endotracheal tube; ED, emergency department; tPA, tissue plasminogen activator; CT, computerized tomography; BP, blood pressure.

Volume , .  : November 

Annals of Emergency Medicine 563.e18

Safety Climate and Medical Errors

Camargo et al

Table E3. Condition-specific criteria for serious guidelines violations. Measure AMI Timely ECG

Administration of aspirin/ anti-platelet agent

Administration of betablockers

Timely thrombolytic therapy Timely PCI therapy

Asthma Initial assessment of airflow limitation

Post-treatment assessment of airflow limitation

Beta-agonist administration Corticosteroid administration

Timely corticosteroids

Patient Population

Violation Criteria

Proportion of Eligible Patients Who Have Serious Guideline Violation

All patients evaluated in ED who present with symptoms suggestive of MI (e.g. chest, jaw, neck, shoulder or arm pain) All patients who present with symptoms suggestive of MI, have a confirmed MI, or those with admitting diagnosis of AMI, ACS, r/o AMI/ACS. Exclusion: aspirin contraindications (active bleeding, aspiring allergy, or other reasons documented by the ED provider) Patients with positive cardiac enzymes in the ED, or new ST-segment elevation on an ED ECG Exclusion: beta-blocker contraindications (beta-blocker allergy, bradycardia, shock, history of asthma, heart failure, or other reasons documented by the ED provider) All AMI patients evaluated in the ED who are given thrombolytic All AMI patients evaluated in the ED who are sent for primary angioplasty or are transferred to a PCI-capable institutions for PCI

Door-to-ECG time ⬎ 15 minutes

44% (1,280/2,882)

Aspirin/anti-platelet agent not given in the ED

16% (576/3,574)

Beta-blockers not given in the ED

46% (900/1,941)

Door-to-needle time ⬎ 45 minutes Door-to-ED disposition time ⬎ 60 minutes

48% (48/99)

PEF not measured within 30 minutes of arrival

75% (2,774/3,697)

A post-treatment PEF not checked within 30–90 min of first beta-agonist treatment

78% (2,926/3,760)

Inhaled beta-agonist not given within 15 minutes of arrival Systemic corticosteroids not given in ED

75% (2,783/3,730)

Systemic corticosteroids not given within 75 minutes of ED arrival

37% (575/1,534)

All patients presenting to the ED with an asthma exacerbation Exclusion: respiratory extremis (oxygen saturation ⬍90% or RR ⱖ30) All patients presenting to the ED with an asthma exacerbation. Exclusion: respiratory extremis (oxygen saturation ⬍90% or RR ⱖ30) All patients presenting to the ED with an asthma exacerbation Patients presenting to the ED with a moderate-to-severe asthma exacerbation, which is defined as any of the following criteria: 1. On oral corticosteroids at time of ED visit 2. Admitted to hospital, ICU, or observation unit 3. PEF ⬍300 for women, ⬍400 for men Exclusion: the mildest exacerbations (oxygen saturation ⫽ 100% and RR ⬍18) ED asthma patients who meet criteria of administration of systemic corticosteroids (above) and are given systemic corticosteroids

53% (276/517)

22% (463/2,072)

(Continued)

563.e19 Annals of Emergency Medicine

Volume , .  : November 

Camargo et al

Safety Climate and Medical Errors

Table E3. Condition-specific criteria for serious guidelines violations. Measure Antibiotics in ED

Oral corticosteroids at discharge

Oral antibiotics at discharge Dislocation Pain assessment Pain medication

Patient Population All patients presenting to the ED with an asthma exacerbation. Exclusion: infections that are generally of bacterial origin Asthma patients who meet criteria to receive systemic corticosteroids in the ED (see above) who are discharged home Asthma patients who meet criteria to receive antibiotics in the ED (see above) and are discharged home All patients presenting to the ED with a dislocation Dislocation patients reporting severe level of pain (⬎⫽7/10 or ⬎⫽4/5) and above All patients presenting to the ED with a dislocation

Assessment of neurovascular status in the limb of a dislocated joint Assessment of vital signs Patients with a dislocation who receive in patients receiving procedural sedation (IV or IM procedural sedation sedative or anesthetic agent) Successful reduction Patients with a dislocation who receive procedural sedation (IV or IM sedative or anesthetic agent) and are sent home

Proportion of Eligible Patients Who Have Serious Guideline Violation

Violation Criteria Antibiotics given in ED

7% (296/3,963)

Oral corticosteroids not given at discharge

34% (452/1,338)

Oral antibiotics given at discharge

17% (529/3,080)

No assessment of pain documented in the chart No administration of any pain medication during the ED stay Neither neurological nor vascular status documented during the ED stay

10% (206/2,059)

Full vitals not documented after IV/IM sedative or anesthetic agents given Unsuccessful reduction of joint (shoulder, elbow, hip, knee, ankle)

11% (218/2,059)

13% (219/1,671)

2% (45/2,059)

3% (61/1,771)

Abbreviations: AMI, acute myocardial infarction; ECG, electrocardiogram; ED, emergency department; ACS, acute coronary syndrome; UA, unstable angina; PCI, percutaneous coronary intervention; PEF, peak expiratory flow; RR, respiratory rate; ICU, intensive care unit; IV, intravenous; and IM, intramuscular.

Table E4. Associations between emergency department safety scores and serious guideline violations. Safety Score

Serious Guideline Violation Overall Unadjusted IRR

Overall summary score Subscales Physical environment Equipment and supplies Teamwork Staffing Nursing Culture Information coordination and consultation Inpatient coordination Triage and monitoring

Acute Myocardial Infarction Acute Asthma Joint Dislocation Multivariable-Adjusted IRR (95% Confidence Interval)*

1.03 (0.96⫺1.10)

1.03 (0.97⫺1.08)

1.00 (0.96⫺1.04)

1.10 (0.95⫺1.27)

1.01 (0.97⫺1.04) 1.05 (0.99⫺1.10) 1.02 (0.97⫺1.08) 1.04 (0.99⫺1.08) 1.01 (0.95⫺1.06) 1.04 (0.99⫺1.09) 1.00 (0.96⫺1.04)

1.02 (0.99⫺1.06) 1.03 (0.99⫺1.08) 1.00 (0.91⫺1.09) 1.02 (0.97⫺1.07) 0.99 (0.93⫺1.05) 1.02 (0.97⫺1.07) 0.98 (0.93⫺1.02)

0.99 (0.97⫺1.01) 1.03 (1.00⫺1.06) 0.98 (0.94⫺1.02) 1.01 (0.98⫺1.04) 1.00 (0.96⫺1.04) 1.02 (0.98⫺1.06) 1.02 (0.99⫺1.06)

1.09 (1.002⫺1.18) 1.09 (0.97⫺1.21) 1.13 (0.95⫺1.35) 1.11 (1.00⫺1.24) 1.04 (0.92⫺1.17) 1.02 (0.88⫺1.19) 1.04 (0.93⫺1.16)

1.00 (0.98⫺1.02) 1.03 (0.97⫺1.09)

1.02 (0.99⫺1.04) 1.02 (0.96⫺1.08)

0.99 (0.98⫺1.00) 0.99 (0.95⫺1.03)

1.00 (0.94⫺1.08) 1.13 (0.98⫺1.31)

Bold number indicates P ⬍ 0.05. Abbreviation: IRR, incidence rate ratio. *A multivariable Poisson regression model was fit with generalized estimating equations accounting for clustering of patients within emergency departments. The incidence rate ratios correspond to a 0.2-point increase in the safety score. The model for myocardial infarction adjusted for age, sex, race/ethnicity, prior myocardial infarction, rales ⬎50% of lung fields at presentation, presence of bradycardia, calendar year, and several emergency department characteristics (number of visits for myocardial infarction per year, region, and affiliation with an emergency medicine residency program). The model for asthma adjusted for age, sex, initial respiratory rate, initial oxygen saturation, initial peak expiratory flow, calendar year, and several emergency department characteristics (number of beds in the ED, annual visit volume, region, and affiliation with an emergency medicine residency program). The model for dislocation adjusted for age, sex, severe pain at presentation (7/10 or higher), location of the dislocated joint, calendar year, and several emergency department characteristics (number of beds in the ED, region, and affiliation with an emergency medicine residency program).

Volume , .  : November 

Annals of Emergency Medicine 563.e20