- Email: [email protected]
Reducing Medication Errors in Hospitals: A Peer Review Organization Collaboration
THE JOINT COMMISSION Decreases in reported medication errors were consistent with improvements in the reliability of medication systems in participating organizations. PATIENT SAFETY
Reducing Medication Errors in Hospitals: A Peer Review Organization Collaboration MICHAEL P. SILVER, MPH JULI A. ANTONOW, MD, MHA
n summer 1997 HealthInsight, the Medicare Peer Review Organization (PRO) for Nevada and Utah, initiated a project to improve patient safety in Utah hospitals. The project was motivated by recent studies of the incidence, cost, causes, and nature of adverse drug events (ADEs) in hospitals. Hospitalized patients face an unnecessarily high risk of iatrogenic injury resulting from medication errors (or preventable ADEs).1–3 Project development was guided by published analyses of the underlying causes
I
of medication errors. These analyses4–6 suggest that many medication errors stem from basic ergonomic flaws in medication systems and the hospital safety environment.7 The majority of these errors are caused by complex systems involving potentially hazardous materials, with poorly standardized procedures for the use of those materials and for communication between operators within the systems. These medication systems challenge the capacity of their users through exceptional demands on memory, attention,
Michael P. Silver, MPH, is Health Care Analyst, HealthInsight, Salt Lake City, Utah, and Juli A. Antonow, MD, MHA, is Medical Director, Utah, HealthInsight. The analyses on which this article is based were performed under contract no. 500-96-P604, “Utilization and Quality Control Peer Review Organization for the State of Utah,” sponsored by the Health Care Financing Administration (HCFA), Department of Health and Human Services. The content of this article does not necessarily reflect the views or policies of the Department of Health and Human Services, nor does mention of trade names, commercial products, or organizations imply endorsement by the U.S. Government. The authors assume full responsibility for the accuracy and completeness of
the ideas presented. This article is a direct result of the Health Care Quality Improvement Program initiated by HCFA, which has encouraged identification of QI projects derived from analysis of patterns of care, and therefore, required no special funding on the part of this contractor. Ideas and contributions to the authors concerning experience in engaging with issues presented are welcomed. The authors thank David C. Classen, MS, MD, for his contributions to the project, as well as all the hospital teams that participated in the collaborative. Please address requests for reprints to Michael P. Silver, HealthInsight, 348 East 4500 South, Suite 300, Salt Lake City, UT 84107; phone 801/892-0155; fax 801/892-0160; e-mail [email protected].
Copyright © 2000 by the Joint Commission on Accreditation of Healthcare Organizations
332
JUNE 2000 JOURNAL Copyright 2000 Joint Commission on Accreditation of Healthcare Organizations
JOURNAL
ON
QUALITY IMPROVEMENT
Article-at-a-Glance Background: Motivated by published reports of the incidence, costs, causes, and nature of adverse drug events (ADEs) in hospitalized patients, in 1997 the Medicare peer review organization for Nevada and Utah initiated a voluntary project of medication error reduction for Utah hospitals. Methods: Through project activities, hospital teams were encouraged to make changes to their medication processes based on direct evaluation of medication systems characteristics, informed by ergonomic principles and published studies of medication errors. Assessment of project effects included an evaluation of the changes implemented and results from an anonymous medication errors survey of clinical staff from participating organizations. Results: Thirteen of the 39 acute care hospitals in Utah participated in 1997–1998 in the collabora-
cognition, vigilance, and stamina. The project was designed to engage Utah hospitals in a collaborative effort to modify their medication systems to reduce the incidence of medication errors and preventable ADEs. The PRO’s objective was that participating hospitals would make substantive improvements to their medication systems which would result in at least a 25% reduction in medication error rates in those organizations.
Methods The authors and other PRO staff adapted the Institute for Healthcare Improvement’s (IHI’s; Boston) Breakthrough Series model for this project. In September 1997, using mail, telephone, and personal contacts, we recruited the hospital teams; team members included quality improvement (QI), pharmacy, and medical staff. Hospitals were required to register their teams and obtain formal letters of support from their administration. Seminars. Hospital teams participated in a series of four seminars (beginning in October 1997 and ending in May 1998) that included presentations by subject experts, QI training, structured team exercises, and sharing experiences in making changes to their medication systems. QI facilitators. Between seminars, PRO QI facilitators worked with hospital teams to help focus efforts on key processes in their medication systems,
Team recruitment.
VOLUME
26
NUMBER
tive project. Participants reported substantive medication system changes that were expected to result in improved patient safety. Baseline and follow-up survey data were available for 8 of the participating hospitals. Analysis of 560 responses showed a 26.9% decrease in overall error frequency, a 12.5% increase in error detection and prevention, and a 24.1% increase in formal written reporting of errors that reached the patient. Conclusions: This project demonstrated community interest in a proactive and collaborative approach to improving patient safety. The improvement efforts were substantive and sustainable. Survey results suggest that the changes implemented in participating organizations may have reduced medication errors and improved capacity for error detection and prevention.
evaluate those processes for error-producing potential, redesign systems where weaknesses were found, implement the changes selected, and monitor the effects of the changes. The key processes targeted included those related to allergy, patient, and drug information, medications with high error potential, and potentially lethal medications. Project activities. Hospitals’ formal error reporting systems (written incident reports) are generally considered to lack sufficient completeness and reliability for identifying weaknesses in medication systems.8,9 As part of a sustained program for enhancing patient safety, project activities included efforts to improve error reporting. More importantly, participants were asked to identify key areas for initial improvement efforts on the basis of available studies of medication errors and ADEs. Participants were encouraged to evaluate their medication systems’ error-producing potential and to select strategies for systems redesign on the basis of the literature on human factors in nonmedical settings.10 Evaluation. Hospital activities, including systems evaluation, redesign strategies, implementation efforts, and training/educational initiatives, were reported by all teams at the project seminars—and by selected teams at a community forum on health care quality that was held four months following the final seminar. During visits to hospitals between seminars, PRO staff could observe the changes that had been
6
Copyright 2000 Joint Commission on Accreditation of Healthcare Organizations
333
THE JOINT COMMISSION Table 1. Medication Errors Survey: Excerpt I. Medication Ordering Section
This section relates to the most recent medication ordering error you have observed. Medication ordering errors might include (but are not limited to): ordering medications for which the patient has a known allergy, ordering the wrong dose, strength, or route of administration, etc. Remember, for this survey it does not matter if the medication was actually administered, just that there was an error in medication ordering.
How long ago was the most recent medication ordering error you observed while working at this hospital? CHECK ONE
1. 2. 3. 4.
___ ___ ___ ___
less than one week less than two weeks less than one month less than three months
Was the error prevented (ie, was the error discovered before the medication was administered)? CHECK ONE
1. 2. 3. 4.
___ ___ ___ ___
Yes, by me Yes, by someone else No; if no, was the patient injured ( ___ Yes ___ No ___ Unknown) Unknown
Was an event/incident/occurrence report filled out? CHECK ONE
1. ___ Yes 2. ___ No 3. ___ Unknown
implemented. The PRO staff drew on the teams’ reports; their own on-site observations; useful strategies identified on the basis of the ADE and medication error, ergonomics, and safety engineering literatures11–13; and a medication error survey, as described below, in evaluating the success of the hospital’s efforts. Medication errors survey and analysis of results. The PRO developed a medication errors sur-
vey, using a taxonomy described by Leape et al,5 to classify errors according to their stage in the medication process, describe the nature of the error, identify contributing factors, measure the frequency of errors discovered and prevented from reaching the patient, and measure the frequency of errors not reported through formal written incident reports. The survey (see Table 1, above, for an excerpt) asked respondents to describe the most recently observed medication error in each of four stages in the medication process: 1. Ordering; 2. Transcription and verification; 3. Dispensing and delivery; and 4. Administration. The survey used a brief format and anonymous administration to overcome recognized barriers to medication error reporting.9 Survey data were intended to be used to help teams focus their efforts on key areas early in the project, to provide them with a compelling baseline as an aid in implementing sys-
334
5. ___ less than six months 6. ___ more than six months 7. ___ never (skip to next section)
tem redesign strategies, and to help them evaluate project effects. Clinical staff (nurses, pharmacists, and physicians) at participating hospitals were surveyed by their project teams in fall 1997 for baseline data, with follow-up measurement in fall and early winter 1998. PRO staff processed and analyzed the responses. Statistical analysis of the survey data focused on three outcome variables: error frequency, detection and prevention, and formal reporting. We used responses indicating the observation of an error during the past week as the measure of error frequency. The measures of error detection and prevention and formal reporting were based on survey respondents’ descriptions of the errors they had most recently observed, regardless of when they occurred. Each respondent contributed up to four observations for each outcome variable to the analysis sample, one for each of the stages in the process from medication ordering to administration. Respondents contributed fewer than four observations if they left a question blank, or, for the error detection and prevention and formal reporting variables only, when these questions were skipped because the respondent indicated that he or she had never observed an error in a particular stage in the process. We performed the analysis with use of the logistic regression function in the Stata® statistical software package.14 For each outcome variable, improvement was estimated using an indicator variable coded as 0
JUNE 2000 JOURNAL Copyright 2000 Joint Commission on Accreditation of Healthcare Organizations
JOURNAL
ON
QUALITY IMPROVEMENT
for observations from the baseline, and 1 for observations from the follow-up period. The logistic regression model also included indicators of the respondent’s profession (nurse, pharmacist, and all others), indicators of the stage in medication ordering to administration process in which the error was observed, and dummy variables representing the respondent’s hospital. Because the multiple observations contributed by respondents could not be assumed to be independent, robust variance-estimation methods involving clustering observations by respondent were used to compute confidence intervals and tests of statistical significance. We performed a descriptive analysis using the error frequency outcome variable by disaggregating observations into separate stages in the ordering to administration process. The regression model for this analysis was the same as that described above, except that indicators of the stage in the process in which the error occurred were no longer required. Finally, we performed a post hoc analysis of the error reporting outcome variable using only errors that reached the patient; two-tailed p values are reported.
tion access. Improvements in this area were expected to reduce the number of missed, duplicated, delayed, and mistimed medication administrations. MAR redesign was thought to be an appropriate initial target for an ongoing effort to improve patient safety. Nurses were thought to be key to the long-term success of the effort. By beginning with MAR redesign, hospital teams demonstrated their commitment to a proactive, nonblaming, and nonpunitive approach that addressed a stated concern of nurses in these organizations. Eight hospitals initiated efforts aimed at improving access to allergy information. This area was identified early in project development as high leverage—an area in which many errors with serious consequences occur. Improvement strategies generally involved the use of reminders or automated pharmacy checking systems that would reduce demands on clinicians’ vigilance, attention, and memory capacity. Six hospitals included efforts to improve access to drug information, mostly focusing on information needs of nursing staff. Two hospitals initiated a training effort that included improved access to antidote information.
Results
Standardizing and simplifying medication procedures. Eight participating hospitals initiated efforts
Participants Of 39 community hospitals in Utah, 13 (accounting for approximately 73,266 [35%] of all 200,870 nonfederal acute care discharges in the state) in 1996 agreed to participate in the project. Participating hospitals ranged from rural hospitals with 20 licensed beds to urban teaching hospitals with more than 400 licensed beds. All integrated health care delivery systems that operate hospitals in Utah were represented.
aimed at simplifying and standardizing procedures. These efforts generally targeted high-risk and/or higherror-potential medications, including anticoagulants, chemotherapy drugs, and pain medications. Hospital teams reported substantive changes to these processes. These changes were expected to reduce hazards throughout the medication ordering to administration process by guiding operators toward correct actions, reducing variation in the use of materials, reducing demands on memory, and improving communication.
Hospital Team Activities Improvement strategies adopted and successfully implemented by hospital teams included improving information access, standardizing and simplifying medication procedures, restricting physical access to potentially lethal drugs, and educating clinical staff about medications. Hospital teams also reported efforts to increase pharmacist involvement in patient education and interaction with patient care teams. (See Sidebar 1, p 334, and Sidebar 2, p 335.) Improving information access. Five hospital teams redesigned the layout of their medication administration records (MARs) to improve informa-
VOLUME
26
NUMBER
Restricting physical access to potentially lethal drugs. Seven hospitals adopted the strategy of limiting
access to potentially lethal drugs. This strategy is intended to reduce the likelihood of rare but catastrophic events stemming from simple slips and substitution errors. The specific drugs targeted included chemotherapy drugs, insulin, and concentrated solutions of KCl (potassium chloride), NaCl (sodium chloride), and MgSO4 (magnesium sulfate). Educating clinical staff about medications. Five hospitals instituted comprehensive educational programs primarily for nursing staff, which were designed to assess knowledge deficiencies, increase drug knowledge, and increase awareness of the potential for error.
6
Copyright 2000 Joint Commission on Accreditation of Healthcare Organizations
335
THE JOINT COMMISSION Sidebar 1. Case Study 1: A 120-Bed Urban Acute Care Hospital A project team was formed, consisting of the pharmacy director, the nursing director of medicine and surgery, a physician, the chief clinical officer, and a patient care nurse. The team set an agenda for improvement based on a model of identifying and correcting latent failures* that can lead to adverse drug events (ADEs). The team’s review of its hospital’s systems, informed by input from the speakers at project seminars and review of the medication errors literature, led to the identification of processes with high-error-producing potential. The initial agenda for improvement included four activities. Patient Assessment on Admission In evaluating the admission process, and in drawing on their direct experiences, team members identified a potential for ADEs arising from interactions between medications prescribed in the hospital and those the patient was taking on admission. Information about continuation of home medications was not effectively transferred to the pharmacists. In response, the pharmacists developed a screen for potential problem medication (eg, anticoagulants, gentamycin, lithium, multiple medications) on admission, which was programmed into the computerized nursing admission history. The system notifies the pharmacy whenever a patient meeting any screening criterion is admitted, and a pharmacist reviews medications with the patient and contacts the attending physician if necessary. Patient Discharge Evaluation The team also identified insufficient or ineffective patient education regarding discharge medications as a potential source of preventable ADEs. It developed a new process similar to the assessment of home medications according to which certain discharge medications would trigger pharmacy, nursing, and/or dietary consultation. Drug monographs were acquired or developed to provide patients with useful drug information after discharge. * Reason J: Human Error. New York: Cambridge University Press, 1990.
Among other initiatives reported, several were particularly noteworthy. Two hospitals implemented a policy to educate patients about their medications; coherent patients were educated as to what drugs they should expect to be given and when they should expect to receive them. One hospital added discharge patient education by pharmacists for patients on specific medications, and pharmacist review for Other
336
initiatives.
Medication Error Prevention After reviewing the current medication error literature and the results from the baseline medication errors survey, the team initiated a variety of activities: Instituting a pharmacology exam program for nursing staff to identify areas in which additional training was required; Designing “skills labs” specific to medication administration for high-risk or problem-prone medications; Changing vendors for patient-controlled analgesia (PCA) to create a standard within the hospital; Revising and updating protocols for PCA pumps, anticoagulants, and medications used in the intensive care unit, including those used for conscious sedation; Improving the layout of information on the medication administration record (MAR); Developing a MAR for contrast media and medications used for radiologic procedures; and Instituting a program of ongoing training in medication error prevention for pharmacists and nurses. Anesthesia Kit Management Because of reports of problems with the anesthesia kit contents, the team instituted changes to improve control of and accountability for anesthesia medications. The team developed specifications for a standard anesthesia kit; processes for tracking kits, for tracking used and wasted medication, and for refreshing kits after use; and standards for secure storage of kits. This team reported a substantial (fivefold) increase in pharmacy clinical interventions, greater pharmacist involvement in discharge consultations, increased awareness of preventable ADEs, and better control of and accountability for anesthesia medications. At the end of the collaborative, it had developed plans for expanding the training efforts and improving medication ordering processes (eg, through electronic ordering systems or protocols with prewritten orders).
patients receiving multiple or selected medications at admission. One team performed an ergonomic survey of the pharmacy, reducing error potential through separation of look-alike and sound-alike drugs and changing vendors for medications with dangerous packaging. Project evaluation indicates that these efforts have been sustained in many participating hospitals, and that, in some, the teams have evolved into standing committees.
JUNE 2000 JOURNAL Copyright 2000 Joint Commission on Accreditation of Healthcare Organizations
JOURNAL
ON
QUALITY IMPROVEMENT
Sidebar 2. Case Study 2: A 40-Bed Rural Acute Care Hospital A project team was formed, consisting of a pharmacist, two patient care nurses from the medicine/surgery department, an emergency medicine physician, and the hospital’s quality resource manager. The team used the results from its medication errors survey and input from the speakers at project seminars to identify processes with high-error-producing potential. The initial agenda for improvement included six activities. Medication Administration Record (MAR) Revision Before the Peer Review Organization (PRO) project began, a new pharmacy information system had been installed at this hospital and at four others participating in the project. The system was implemented with limited input from nursing staff, who subsequently expressed concerns about the format of the MAR generated by the new system (eg, information on the administration route and special comments were easy to miss, and it was difficult to tell whether the full daily course of medications had been given). Nurses and pharmacists from the five hospitals collaborated to redesign the MAR format, standardize medication administration times, and submit a uniform request to reprogram the pharmacy information system. The team reported positive feedback from nursing staff on the implementation of the redesigned MAR, an early success that helped in overcoming some resistance it later encountered in implementing other changes. In association with the MAR revision, the pharmacy at this hospital began attaching reminders to medications with special instructions for use or high-error potential (eg, if the full dose is two tablets). Adverse Event Reporting The team identified the process and forms used to document medication errors and adverse events as a barrier to reporting. The team developed a shorter form and a policy that gave the user the option of anonymous submission. These changes were adopted by the hospital late in the project. Limiting Access to Potentially Lethal Drugs High-hazard medications (KCl, concentrated NaCl, and MgSO4) were removed from floor stocks. Because the
Medication Errors Survey of Clinical Staff Of the 13 hospitals participating in the project, 12 administered a baseline medication errors survey. One of these hospitals, however, administered the survey in a manner that did not preserve the anonymity of the respondent; results from this hospital were
VOLUME
26
NUMBER
pharmacy is not open 24 hours at this hospital, nursing staff were educated on procedures for obtaining these medications (from pharmacists on call) after hours. The team was concerned that nurses might resist this change, and adopted spot-check monitoring to ensure that these medications did not reappear in floor stocks. Medication Orders The team instituted a program of education and feedback for reliable communication of medication orders. Recommendations were given to all clinical staff on avoiding abbreviations, using metric units, using a leading zero and avoiding a trailing zero, and repeating verbal orders for verification. The pharmacy tracked exceptions and illegible orders and provided feedback on progress and problems identified to reinforce the recommendations. Allergy Information The team reviewed its allergy information tracking systems and procedures, and identified a system defect that allowed violation of the standard procedure of recording allergy information at admission. A “known allergies” field did not require entry to continue processing and was sometimes left blank. The system was modified to require completion of this field. Nursing staff were educated on the change and the correct use of this field when known allergies could not be determined. Staff Training The team obtained self-guided (videotaped) educational materials on medication errors. At the end of the collaborative, the team reported staff satisfaction with the changes, no reappearance of the high-hazard medications in floor stocks, continued monitoring of medication ordering recommendations, and almostcomplete participation in the self-guided educational program. The team’s agenda for further improvement included a program of pharmacist review of medications, development and/or adoption of protocols for high-error-potential medications, and expanded staff education.
excluded from further analysis. At follow-up, 3 participating hospitals declined to resurvey. Analysis was restricted to the 8 hospitals with both valid baseline and follow-up samples. Survey administration methods varied by facility and by profession of respondent. Nurses usually received the surveys at regular meetings
6
Copyright 2000 Joint Commission on Accreditation of Healthcare Organizations
337
THE JOINT COMMISSION or from supervisors. Distribution in an internal mailbox was used for a small number of nurses but was used almost exclusively for physicians and pharmacists. Some teams collected the surveys at the same meeting in which they distributed them, although centralized anonymous collection was more common. Precise survey response rates could not be determined because participating hospital teams did not maintain detailed logs of survey activity. Using notes from team survey administration activities, response rates were estimated at 80%–90% for all pharmacists and for the nurses with surveys administered and collected during the same meeting. The response rate was estimated at 40%–50% for nurses who received surveys during meetings and were asked to complete and return them at a later time, and at 20%–30% for physicians and others who received the survey through internal mailboxes. A total of 560 completed surveys (341 baseline and 219 follow-up) were available for analysis. The majority (396; 70.7%) of respondents were nurses, with the remainder being pharmacists (66; 11.8%), physicians (46; 8.2%), and other clinical professions or not specified by the respondent (52; 9.3%). Because physicians were not well represented in the follow-up survey sample, contributing only 4 responses, we reanalyzed the data excluding these responses. No material differences from the reported results were observed. Comparisons of baseline and follow-up rates for the outcome variables are shown in Table 2 (p 337). A statistically significant 26.9% reduction in the error frequency outcome variable was observed; descriptive analyses showed error frequency reductions ranging from 19.4% for administration errors to 35.7% for transcription and verification errors. A statistically significant 12.5% increase was observed in error discovery and prevention. An 11.4% increase in overall error reporting was observed, which was not statistically significant; however, the post hoc analysis showed a statistically significant 24.1% increase in reporting of errors that reached the patient.
Discussion Many of the error-reduction strategies selected by project teams overlap with recommendations from the 1999 Institute of Medicine report on error in medicine.15 Our observations suggest one method by which
338
external organizations, such as PROs, can facilitate the successful implementation of such recommendations. The findings from this study are, however, somewhat limited in that hospital team reports to the collaborative concentrated almost exclusively on strategies that were successfully implemented and in that long-term follow-up was lacking. Future collaborative efforts should actively seek out information with respect to extent of implementation and success of all attempted initiatives, including those abandoned or unsuccessful, and make provisions for extended observation of program effects. The medication errors survey results must be interpreted with caution. Because no reliable error or ADE databases were available, the survey itself could not be validated against external criteria. While hospital teams were advised to use administration and collection methods that would maximize response and ensure, to the extent possible, the validity of pre–post comparisons, this advice was generally not followed. Incomplete participation in the pre–post survey by all organizations and the low response rates in the organizations that did participate may have introduced self-selection biases. Various forms of respondent recall bias may have influenced the results: Respondents may have passed over more recent, less serious errors to describe older, more serious errors, or respondents may have become more aware of errors as a result of project activities. There was no control condition; the possibility of measurement artifacts cannot be eliminated (although the follow-up survey took place approximately six months after the end of the active collaborative). Because only a cross-sectional baseline assessment was used, it is possible that the changes observed were part of a larger trend. Finally, the survey cannot capture errors that were never detected. Future attempts to measure error frequency, detection and prevention, and reporting using survey methods should be able to overcome some, but not all, of the limitations noted through more direct control of survey administration. With the limitations described in mind, we note that the survey results were consistent with breakthrough improvements in the reliability of medication systems in participating organizations. Because the survey is designed to measure all errors observed throughout the medication ordering to administration process (and not only those targeted by the activities described),
JUNE 2000 JOURNAL Copyright 2000 Joint Commission on Accreditation of Healthcare Organizations
JOURNAL
ON
QUALITY IMPROVEMENT
Table 2. Medication Errors Survey of Clinical Staff: Comparisons of Baseline and Follow-Up Results Baseline
Follow-up
Improvement**
percentage percentage OR (no. of responses)(no. of responses) (95% CI)
Outcome*
Respondents indicating the observation of an error during the previous week: all errors† medication ordering errors‡ transcription and verification errors‡ dispensing and delivery errors‡ medication administration errors‡ Respondents indicating that the error most recently observed was discovered and prevented from reaching the patient† Respondents indicating that the error most recently observed was reported all errors† errors that reached the patient§
p value
20.8% (1,305) 23.0% (335) 24.4% (328) 20.2% (326) 15.5% (316)
15.2% (833) 16.4% (213) 15.7% (210) 16.2% (210) 12.5% (200)
0.62 (0.44 to 0.87) 0.63 (0.38 to 1.04) 0.47 (0.29 to 0.77) 0.68 (0.41 to 1.12) 0.73 (0.42 to 1.28)
0.006
51.2% (934)
57.6% (585)
1.32 (1.02 to 1.70)
0.032
29.0% (954) 43.9% (478)
32.3% (589) 54.5% (254)
1.16 (0.86 to 1.57) 1.67 (1.11 to 2.50)
0.337
0.068 0.003 0.127 0.278
0.013
* outcome variable, descriptive analysis, and post hoc analysis. †
outcome variable
‡
descriptive analysis
§
post hoc analysis
** From logistic regression analysis. The dependent variable is listed in the Outcome column. Improvement compares follow-up to baseline proportions; other variables included in the model are indicators of the respondent’s profession, respondent’s hospital, and, for dependent variables combining errors from the four stages of the medication process (outcome variables and post hoc analysis), the stage in the medication ordering to administration process in which the error occurred. OR = odds ratio, CI = confidence interval.
these improvements may appear to be disproportionate to the actions taken by participating teams. This observation may raise an additional concern about the validity of the survey results. The small number of hospitals for which pre–post data were available precluded meaningful analysis of associations between activities undertaken and improvement as measured by survey. An alternate explanation is that the activities undertaken had effects beyond their specific targets. The hospital teams described intervention strategies that were broad in scope that had the potential to affect the knowledge, attitudes, beliefs, and behaviors of a substantial portion of the clinical staff. Participation in the survey may have heightened clinical staff members’ awareness of errors. The most important change produced by project activ-
VOLUME
26
NUMBER
ities may have been improvement in the effectiveness of the safety cultures in participating hospitals. The observation of increased reporting of errors that reached the patient is consistent with this explanation. Enhancing the hospital’s safety culture was not a stated objective for this project; future medication error reduction program designers may want to consider its improvement as an explicit aim and expected outcome of their efforts.16 It may be possible to expand the range of topics on which clinical staff are surveyed (for example, perceptions of organizational response to and tolerance of error, individual attitudes toward limitations in human performance, knowledge of and compliance with policies) to include indicators of progress toward this objective.17
6
Copyright 2000 Joint Commission on Accreditation of Healthcare Organizations
339
THE JOINT COMMISSION Conclusions This voluntary project was initiated by the PRO because of national data, and not because of any highprofile incidents in local hospitals. The PRO and hospital collaborators had long recognized medication errors as a significant threat to patient safety. This project demonstrated interest in a proactive, nonpunitive, and collaborative approach to reducing this threat. The level and quality of activity we observed demon-
strated that this interest could be channeled, through active QI facilitation, into structured medication systems assessment and redesign, and that hospital teams could be empowered to implement substantive changes in their medication systems. Our observations indicate that participants increased their capacity for this task through project activities and may have improved patient safety by significantly reducing medication errors. J
References 1. Classen DC, et al: Adverse drug events in hospitalized patients: Excess length of stay, extra costs, and attributable mortality. JAMA 277:301–306, 1997. 2. Bates DW, et al, for the ADE Prevention Study Group: Incidence of adverse drug events and potential adverse drug events. JAMA 274: 29–34, 1995. 3. Bates DW, et al, for the Adverse Drug Events Prevention Study Group: The costs of adverse drug events in hospitalized patients. JAMA 277:307–311, 1997. 4. Leape LL, et al: The nature of adverse events in hospitalized
340
patients: Results of the Harvard Medical Practice Study II. New Engl J Med 324:377–384, 1991. 5. Leape LL, et al, for the ADE Prevention Study Group: Systems analysis of adverse drug events. JAMA 274:35–43, 1995. 6. Lesar TS, Lomaestro BM, Pohl H: Medication-prescribing errors in a teaching hospital. Arch Intern Med 157:1569–1576, 1997. 7. Allnutt MF: Human factors in accidents. Br J Anaesth 59:856–864, 1987. 8. Leape LL: Error in medicine. JAMA 272:1851–1857, 1994. 9. Classen DC, et al: Computer-
ized surveillance of adverse drug events in hospital patients. JAMA 266:2847–2851, 1991. 10. Reason J: Human Error. New York: Cambridge University Press, 1990. 11. Proctor RW, Van Zandt T: Human Factors in Simple and Complex Systems. Boston: Allyn & Bacon, 1994. 12. Sanders MS, McCormack EJ: Human Factors in Engineering and Design 7th ed. New York: McGraw-Hill, 1993. 13. Petersen D: Human Error Reduction and Safety Management 3rd ed. New York: Van Nostrand
Reinhold, 1996. 14. Stata Corporation: Stata Statistical Software: Release 5.0 (Volume 2). College Station, TX, 1997, pp 343–367. 15. Institute of Medicine: To Err Is Human: Building a Safer Health System (Kohn L, Corrigan J, Donaldson M, eds). Washington, DC: National Academy Press, 2000. 16. Reason J: Managing the Risks of Organizational Accidents. Brookfield, VT: Ashgate, 1997. 17. Helmreich RL, Merritt AC: Culture at Work in Aviation and Medicine. Brookfield, VT: Ashgate, 1998.
JUNE 2000 JOURNAL Copyright 2000 Joint Commission on Accreditation of Healthcare Organizations
Reducing Medication Errors in Hospitals: A Peer Review Organization Collaboration MICHAEL P. SILVER, MPH JULI A. ANTONOW, MD, MHA
n summer 1997 HealthInsight, the Medicare Peer Review Organization (PRO) for Nevada and Utah, initiated a project to improve patient safety in Utah hospitals. The project was motivated by recent studies of the incidence, cost, causes, and nature of adverse drug events (ADEs) in hospitals. Hospitalized patients face an unnecessarily high risk of iatrogenic injury resulting from medication errors (or preventable ADEs).1–3 Project development was guided by published analyses of the underlying causes
I
of medication errors. These analyses4–6 suggest that many medication errors stem from basic ergonomic flaws in medication systems and the hospital safety environment.7 The majority of these errors are caused by complex systems involving potentially hazardous materials, with poorly standardized procedures for the use of those materials and for communication between operators within the systems. These medication systems challenge the capacity of their users through exceptional demands on memory, attention,
Michael P. Silver, MPH, is Health Care Analyst, HealthInsight, Salt Lake City, Utah, and Juli A. Antonow, MD, MHA, is Medical Director, Utah, HealthInsight. The analyses on which this article is based were performed under contract no. 500-96-P604, “Utilization and Quality Control Peer Review Organization for the State of Utah,” sponsored by the Health Care Financing Administration (HCFA), Department of Health and Human Services. The content of this article does not necessarily reflect the views or policies of the Department of Health and Human Services, nor does mention of trade names, commercial products, or organizations imply endorsement by the U.S. Government. The authors assume full responsibility for the accuracy and completeness of
the ideas presented. This article is a direct result of the Health Care Quality Improvement Program initiated by HCFA, which has encouraged identification of QI projects derived from analysis of patterns of care, and therefore, required no special funding on the part of this contractor. Ideas and contributions to the authors concerning experience in engaging with issues presented are welcomed. The authors thank David C. Classen, MS, MD, for his contributions to the project, as well as all the hospital teams that participated in the collaborative. Please address requests for reprints to Michael P. Silver, HealthInsight, 348 East 4500 South, Suite 300, Salt Lake City, UT 84107; phone 801/892-0155; fax 801/892-0160; e-mail [email protected].
Copyright © 2000 by the Joint Commission on Accreditation of Healthcare Organizations
332
JUNE 2000 JOURNAL Copyright 2000 Joint Commission on Accreditation of Healthcare Organizations
JOURNAL
ON
QUALITY IMPROVEMENT
Article-at-a-Glance Background: Motivated by published reports of the incidence, costs, causes, and nature of adverse drug events (ADEs) in hospitalized patients, in 1997 the Medicare peer review organization for Nevada and Utah initiated a voluntary project of medication error reduction for Utah hospitals. Methods: Through project activities, hospital teams were encouraged to make changes to their medication processes based on direct evaluation of medication systems characteristics, informed by ergonomic principles and published studies of medication errors. Assessment of project effects included an evaluation of the changes implemented and results from an anonymous medication errors survey of clinical staff from participating organizations. Results: Thirteen of the 39 acute care hospitals in Utah participated in 1997–1998 in the collabora-
cognition, vigilance, and stamina. The project was designed to engage Utah hospitals in a collaborative effort to modify their medication systems to reduce the incidence of medication errors and preventable ADEs. The PRO’s objective was that participating hospitals would make substantive improvements to their medication systems which would result in at least a 25% reduction in medication error rates in those organizations.
Methods The authors and other PRO staff adapted the Institute for Healthcare Improvement’s (IHI’s; Boston) Breakthrough Series model for this project. In September 1997, using mail, telephone, and personal contacts, we recruited the hospital teams; team members included quality improvement (QI), pharmacy, and medical staff. Hospitals were required to register their teams and obtain formal letters of support from their administration. Seminars. Hospital teams participated in a series of four seminars (beginning in October 1997 and ending in May 1998) that included presentations by subject experts, QI training, structured team exercises, and sharing experiences in making changes to their medication systems. QI facilitators. Between seminars, PRO QI facilitators worked with hospital teams to help focus efforts on key processes in their medication systems,
Team recruitment.
VOLUME
26
NUMBER
tive project. Participants reported substantive medication system changes that were expected to result in improved patient safety. Baseline and follow-up survey data were available for 8 of the participating hospitals. Analysis of 560 responses showed a 26.9% decrease in overall error frequency, a 12.5% increase in error detection and prevention, and a 24.1% increase in formal written reporting of errors that reached the patient. Conclusions: This project demonstrated community interest in a proactive and collaborative approach to improving patient safety. The improvement efforts were substantive and sustainable. Survey results suggest that the changes implemented in participating organizations may have reduced medication errors and improved capacity for error detection and prevention.
evaluate those processes for error-producing potential, redesign systems where weaknesses were found, implement the changes selected, and monitor the effects of the changes. The key processes targeted included those related to allergy, patient, and drug information, medications with high error potential, and potentially lethal medications. Project activities. Hospitals’ formal error reporting systems (written incident reports) are generally considered to lack sufficient completeness and reliability for identifying weaknesses in medication systems.8,9 As part of a sustained program for enhancing patient safety, project activities included efforts to improve error reporting. More importantly, participants were asked to identify key areas for initial improvement efforts on the basis of available studies of medication errors and ADEs. Participants were encouraged to evaluate their medication systems’ error-producing potential and to select strategies for systems redesign on the basis of the literature on human factors in nonmedical settings.10 Evaluation. Hospital activities, including systems evaluation, redesign strategies, implementation efforts, and training/educational initiatives, were reported by all teams at the project seminars—and by selected teams at a community forum on health care quality that was held four months following the final seminar. During visits to hospitals between seminars, PRO staff could observe the changes that had been
6
Copyright 2000 Joint Commission on Accreditation of Healthcare Organizations
333
THE JOINT COMMISSION Table 1. Medication Errors Survey: Excerpt I. Medication Ordering Section
This section relates to the most recent medication ordering error you have observed. Medication ordering errors might include (but are not limited to): ordering medications for which the patient has a known allergy, ordering the wrong dose, strength, or route of administration, etc. Remember, for this survey it does not matter if the medication was actually administered, just that there was an error in medication ordering.
How long ago was the most recent medication ordering error you observed while working at this hospital? CHECK ONE
1. 2. 3. 4.
___ ___ ___ ___
less than one week less than two weeks less than one month less than three months
Was the error prevented (ie, was the error discovered before the medication was administered)? CHECK ONE
1. 2. 3. 4.
___ ___ ___ ___
Yes, by me Yes, by someone else No; if no, was the patient injured ( ___ Yes ___ No ___ Unknown) Unknown
Was an event/incident/occurrence report filled out? CHECK ONE
1. ___ Yes 2. ___ No 3. ___ Unknown
implemented. The PRO staff drew on the teams’ reports; their own on-site observations; useful strategies identified on the basis of the ADE and medication error, ergonomics, and safety engineering literatures11–13; and a medication error survey, as described below, in evaluating the success of the hospital’s efforts. Medication errors survey and analysis of results. The PRO developed a medication errors sur-
vey, using a taxonomy described by Leape et al,5 to classify errors according to their stage in the medication process, describe the nature of the error, identify contributing factors, measure the frequency of errors discovered and prevented from reaching the patient, and measure the frequency of errors not reported through formal written incident reports. The survey (see Table 1, above, for an excerpt) asked respondents to describe the most recently observed medication error in each of four stages in the medication process: 1. Ordering; 2. Transcription and verification; 3. Dispensing and delivery; and 4. Administration. The survey used a brief format and anonymous administration to overcome recognized barriers to medication error reporting.9 Survey data were intended to be used to help teams focus their efforts on key areas early in the project, to provide them with a compelling baseline as an aid in implementing sys-
334
5. ___ less than six months 6. ___ more than six months 7. ___ never (skip to next section)
tem redesign strategies, and to help them evaluate project effects. Clinical staff (nurses, pharmacists, and physicians) at participating hospitals were surveyed by their project teams in fall 1997 for baseline data, with follow-up measurement in fall and early winter 1998. PRO staff processed and analyzed the responses. Statistical analysis of the survey data focused on three outcome variables: error frequency, detection and prevention, and formal reporting. We used responses indicating the observation of an error during the past week as the measure of error frequency. The measures of error detection and prevention and formal reporting were based on survey respondents’ descriptions of the errors they had most recently observed, regardless of when they occurred. Each respondent contributed up to four observations for each outcome variable to the analysis sample, one for each of the stages in the process from medication ordering to administration. Respondents contributed fewer than four observations if they left a question blank, or, for the error detection and prevention and formal reporting variables only, when these questions were skipped because the respondent indicated that he or she had never observed an error in a particular stage in the process. We performed the analysis with use of the logistic regression function in the Stata® statistical software package.14 For each outcome variable, improvement was estimated using an indicator variable coded as 0
JUNE 2000 JOURNAL Copyright 2000 Joint Commission on Accreditation of Healthcare Organizations
JOURNAL
ON
QUALITY IMPROVEMENT
for observations from the baseline, and 1 for observations from the follow-up period. The logistic regression model also included indicators of the respondent’s profession (nurse, pharmacist, and all others), indicators of the stage in medication ordering to administration process in which the error was observed, and dummy variables representing the respondent’s hospital. Because the multiple observations contributed by respondents could not be assumed to be independent, robust variance-estimation methods involving clustering observations by respondent were used to compute confidence intervals and tests of statistical significance. We performed a descriptive analysis using the error frequency outcome variable by disaggregating observations into separate stages in the ordering to administration process. The regression model for this analysis was the same as that described above, except that indicators of the stage in the process in which the error occurred were no longer required. Finally, we performed a post hoc analysis of the error reporting outcome variable using only errors that reached the patient; two-tailed p values are reported.
tion access. Improvements in this area were expected to reduce the number of missed, duplicated, delayed, and mistimed medication administrations. MAR redesign was thought to be an appropriate initial target for an ongoing effort to improve patient safety. Nurses were thought to be key to the long-term success of the effort. By beginning with MAR redesign, hospital teams demonstrated their commitment to a proactive, nonblaming, and nonpunitive approach that addressed a stated concern of nurses in these organizations. Eight hospitals initiated efforts aimed at improving access to allergy information. This area was identified early in project development as high leverage—an area in which many errors with serious consequences occur. Improvement strategies generally involved the use of reminders or automated pharmacy checking systems that would reduce demands on clinicians’ vigilance, attention, and memory capacity. Six hospitals included efforts to improve access to drug information, mostly focusing on information needs of nursing staff. Two hospitals initiated a training effort that included improved access to antidote information.
Results
Standardizing and simplifying medication procedures. Eight participating hospitals initiated efforts
Participants Of 39 community hospitals in Utah, 13 (accounting for approximately 73,266 [35%] of all 200,870 nonfederal acute care discharges in the state) in 1996 agreed to participate in the project. Participating hospitals ranged from rural hospitals with 20 licensed beds to urban teaching hospitals with more than 400 licensed beds. All integrated health care delivery systems that operate hospitals in Utah were represented.
aimed at simplifying and standardizing procedures. These efforts generally targeted high-risk and/or higherror-potential medications, including anticoagulants, chemotherapy drugs, and pain medications. Hospital teams reported substantive changes to these processes. These changes were expected to reduce hazards throughout the medication ordering to administration process by guiding operators toward correct actions, reducing variation in the use of materials, reducing demands on memory, and improving communication.
Hospital Team Activities Improvement strategies adopted and successfully implemented by hospital teams included improving information access, standardizing and simplifying medication procedures, restricting physical access to potentially lethal drugs, and educating clinical staff about medications. Hospital teams also reported efforts to increase pharmacist involvement in patient education and interaction with patient care teams. (See Sidebar 1, p 334, and Sidebar 2, p 335.) Improving information access. Five hospital teams redesigned the layout of their medication administration records (MARs) to improve informa-
VOLUME
26
NUMBER
Restricting physical access to potentially lethal drugs. Seven hospitals adopted the strategy of limiting
access to potentially lethal drugs. This strategy is intended to reduce the likelihood of rare but catastrophic events stemming from simple slips and substitution errors. The specific drugs targeted included chemotherapy drugs, insulin, and concentrated solutions of KCl (potassium chloride), NaCl (sodium chloride), and MgSO4 (magnesium sulfate). Educating clinical staff about medications. Five hospitals instituted comprehensive educational programs primarily for nursing staff, which were designed to assess knowledge deficiencies, increase drug knowledge, and increase awareness of the potential for error.
6
Copyright 2000 Joint Commission on Accreditation of Healthcare Organizations
335
THE JOINT COMMISSION Sidebar 1. Case Study 1: A 120-Bed Urban Acute Care Hospital A project team was formed, consisting of the pharmacy director, the nursing director of medicine and surgery, a physician, the chief clinical officer, and a patient care nurse. The team set an agenda for improvement based on a model of identifying and correcting latent failures* that can lead to adverse drug events (ADEs). The team’s review of its hospital’s systems, informed by input from the speakers at project seminars and review of the medication errors literature, led to the identification of processes with high-error-producing potential. The initial agenda for improvement included four activities. Patient Assessment on Admission In evaluating the admission process, and in drawing on their direct experiences, team members identified a potential for ADEs arising from interactions between medications prescribed in the hospital and those the patient was taking on admission. Information about continuation of home medications was not effectively transferred to the pharmacists. In response, the pharmacists developed a screen for potential problem medication (eg, anticoagulants, gentamycin, lithium, multiple medications) on admission, which was programmed into the computerized nursing admission history. The system notifies the pharmacy whenever a patient meeting any screening criterion is admitted, and a pharmacist reviews medications with the patient and contacts the attending physician if necessary. Patient Discharge Evaluation The team also identified insufficient or ineffective patient education regarding discharge medications as a potential source of preventable ADEs. It developed a new process similar to the assessment of home medications according to which certain discharge medications would trigger pharmacy, nursing, and/or dietary consultation. Drug monographs were acquired or developed to provide patients with useful drug information after discharge. * Reason J: Human Error. New York: Cambridge University Press, 1990.
Among other initiatives reported, several were particularly noteworthy. Two hospitals implemented a policy to educate patients about their medications; coherent patients were educated as to what drugs they should expect to be given and when they should expect to receive them. One hospital added discharge patient education by pharmacists for patients on specific medications, and pharmacist review for Other
336
initiatives.
Medication Error Prevention After reviewing the current medication error literature and the results from the baseline medication errors survey, the team initiated a variety of activities: Instituting a pharmacology exam program for nursing staff to identify areas in which additional training was required; Designing “skills labs” specific to medication administration for high-risk or problem-prone medications; Changing vendors for patient-controlled analgesia (PCA) to create a standard within the hospital; Revising and updating protocols for PCA pumps, anticoagulants, and medications used in the intensive care unit, including those used for conscious sedation; Improving the layout of information on the medication administration record (MAR); Developing a MAR for contrast media and medications used for radiologic procedures; and Instituting a program of ongoing training in medication error prevention for pharmacists and nurses. Anesthesia Kit Management Because of reports of problems with the anesthesia kit contents, the team instituted changes to improve control of and accountability for anesthesia medications. The team developed specifications for a standard anesthesia kit; processes for tracking kits, for tracking used and wasted medication, and for refreshing kits after use; and standards for secure storage of kits. This team reported a substantial (fivefold) increase in pharmacy clinical interventions, greater pharmacist involvement in discharge consultations, increased awareness of preventable ADEs, and better control of and accountability for anesthesia medications. At the end of the collaborative, it had developed plans for expanding the training efforts and improving medication ordering processes (eg, through electronic ordering systems or protocols with prewritten orders).
patients receiving multiple or selected medications at admission. One team performed an ergonomic survey of the pharmacy, reducing error potential through separation of look-alike and sound-alike drugs and changing vendors for medications with dangerous packaging. Project evaluation indicates that these efforts have been sustained in many participating hospitals, and that, in some, the teams have evolved into standing committees.
JUNE 2000 JOURNAL Copyright 2000 Joint Commission on Accreditation of Healthcare Organizations
JOURNAL
ON
QUALITY IMPROVEMENT
Sidebar 2. Case Study 2: A 40-Bed Rural Acute Care Hospital A project team was formed, consisting of a pharmacist, two patient care nurses from the medicine/surgery department, an emergency medicine physician, and the hospital’s quality resource manager. The team used the results from its medication errors survey and input from the speakers at project seminars to identify processes with high-error-producing potential. The initial agenda for improvement included six activities. Medication Administration Record (MAR) Revision Before the Peer Review Organization (PRO) project began, a new pharmacy information system had been installed at this hospital and at four others participating in the project. The system was implemented with limited input from nursing staff, who subsequently expressed concerns about the format of the MAR generated by the new system (eg, information on the administration route and special comments were easy to miss, and it was difficult to tell whether the full daily course of medications had been given). Nurses and pharmacists from the five hospitals collaborated to redesign the MAR format, standardize medication administration times, and submit a uniform request to reprogram the pharmacy information system. The team reported positive feedback from nursing staff on the implementation of the redesigned MAR, an early success that helped in overcoming some resistance it later encountered in implementing other changes. In association with the MAR revision, the pharmacy at this hospital began attaching reminders to medications with special instructions for use or high-error potential (eg, if the full dose is two tablets). Adverse Event Reporting The team identified the process and forms used to document medication errors and adverse events as a barrier to reporting. The team developed a shorter form and a policy that gave the user the option of anonymous submission. These changes were adopted by the hospital late in the project. Limiting Access to Potentially Lethal Drugs High-hazard medications (KCl, concentrated NaCl, and MgSO4) were removed from floor stocks. Because the
Medication Errors Survey of Clinical Staff Of the 13 hospitals participating in the project, 12 administered a baseline medication errors survey. One of these hospitals, however, administered the survey in a manner that did not preserve the anonymity of the respondent; results from this hospital were
VOLUME
26
NUMBER
pharmacy is not open 24 hours at this hospital, nursing staff were educated on procedures for obtaining these medications (from pharmacists on call) after hours. The team was concerned that nurses might resist this change, and adopted spot-check monitoring to ensure that these medications did not reappear in floor stocks. Medication Orders The team instituted a program of education and feedback for reliable communication of medication orders. Recommendations were given to all clinical staff on avoiding abbreviations, using metric units, using a leading zero and avoiding a trailing zero, and repeating verbal orders for verification. The pharmacy tracked exceptions and illegible orders and provided feedback on progress and problems identified to reinforce the recommendations. Allergy Information The team reviewed its allergy information tracking systems and procedures, and identified a system defect that allowed violation of the standard procedure of recording allergy information at admission. A “known allergies” field did not require entry to continue processing and was sometimes left blank. The system was modified to require completion of this field. Nursing staff were educated on the change and the correct use of this field when known allergies could not be determined. Staff Training The team obtained self-guided (videotaped) educational materials on medication errors. At the end of the collaborative, the team reported staff satisfaction with the changes, no reappearance of the high-hazard medications in floor stocks, continued monitoring of medication ordering recommendations, and almostcomplete participation in the self-guided educational program. The team’s agenda for further improvement included a program of pharmacist review of medications, development and/or adoption of protocols for high-error-potential medications, and expanded staff education.
excluded from further analysis. At follow-up, 3 participating hospitals declined to resurvey. Analysis was restricted to the 8 hospitals with both valid baseline and follow-up samples. Survey administration methods varied by facility and by profession of respondent. Nurses usually received the surveys at regular meetings
6
Copyright 2000 Joint Commission on Accreditation of Healthcare Organizations
337
THE JOINT COMMISSION or from supervisors. Distribution in an internal mailbox was used for a small number of nurses but was used almost exclusively for physicians and pharmacists. Some teams collected the surveys at the same meeting in which they distributed them, although centralized anonymous collection was more common. Precise survey response rates could not be determined because participating hospital teams did not maintain detailed logs of survey activity. Using notes from team survey administration activities, response rates were estimated at 80%–90% for all pharmacists and for the nurses with surveys administered and collected during the same meeting. The response rate was estimated at 40%–50% for nurses who received surveys during meetings and were asked to complete and return them at a later time, and at 20%–30% for physicians and others who received the survey through internal mailboxes. A total of 560 completed surveys (341 baseline and 219 follow-up) were available for analysis. The majority (396; 70.7%) of respondents were nurses, with the remainder being pharmacists (66; 11.8%), physicians (46; 8.2%), and other clinical professions or not specified by the respondent (52; 9.3%). Because physicians were not well represented in the follow-up survey sample, contributing only 4 responses, we reanalyzed the data excluding these responses. No material differences from the reported results were observed. Comparisons of baseline and follow-up rates for the outcome variables are shown in Table 2 (p 337). A statistically significant 26.9% reduction in the error frequency outcome variable was observed; descriptive analyses showed error frequency reductions ranging from 19.4% for administration errors to 35.7% for transcription and verification errors. A statistically significant 12.5% increase was observed in error discovery and prevention. An 11.4% increase in overall error reporting was observed, which was not statistically significant; however, the post hoc analysis showed a statistically significant 24.1% increase in reporting of errors that reached the patient.
Discussion Many of the error-reduction strategies selected by project teams overlap with recommendations from the 1999 Institute of Medicine report on error in medicine.15 Our observations suggest one method by which
338
external organizations, such as PROs, can facilitate the successful implementation of such recommendations. The findings from this study are, however, somewhat limited in that hospital team reports to the collaborative concentrated almost exclusively on strategies that were successfully implemented and in that long-term follow-up was lacking. Future collaborative efforts should actively seek out information with respect to extent of implementation and success of all attempted initiatives, including those abandoned or unsuccessful, and make provisions for extended observation of program effects. The medication errors survey results must be interpreted with caution. Because no reliable error or ADE databases were available, the survey itself could not be validated against external criteria. While hospital teams were advised to use administration and collection methods that would maximize response and ensure, to the extent possible, the validity of pre–post comparisons, this advice was generally not followed. Incomplete participation in the pre–post survey by all organizations and the low response rates in the organizations that did participate may have introduced self-selection biases. Various forms of respondent recall bias may have influenced the results: Respondents may have passed over more recent, less serious errors to describe older, more serious errors, or respondents may have become more aware of errors as a result of project activities. There was no control condition; the possibility of measurement artifacts cannot be eliminated (although the follow-up survey took place approximately six months after the end of the active collaborative). Because only a cross-sectional baseline assessment was used, it is possible that the changes observed were part of a larger trend. Finally, the survey cannot capture errors that were never detected. Future attempts to measure error frequency, detection and prevention, and reporting using survey methods should be able to overcome some, but not all, of the limitations noted through more direct control of survey administration. With the limitations described in mind, we note that the survey results were consistent with breakthrough improvements in the reliability of medication systems in participating organizations. Because the survey is designed to measure all errors observed throughout the medication ordering to administration process (and not only those targeted by the activities described),
JUNE 2000 JOURNAL Copyright 2000 Joint Commission on Accreditation of Healthcare Organizations
JOURNAL
ON
QUALITY IMPROVEMENT
Table 2. Medication Errors Survey of Clinical Staff: Comparisons of Baseline and Follow-Up Results Baseline
Follow-up
Improvement**
percentage percentage OR (no. of responses)(no. of responses) (95% CI)
Outcome*
Respondents indicating the observation of an error during the previous week: all errors† medication ordering errors‡ transcription and verification errors‡ dispensing and delivery errors‡ medication administration errors‡ Respondents indicating that the error most recently observed was discovered and prevented from reaching the patient† Respondents indicating that the error most recently observed was reported all errors† errors that reached the patient§
p value
20.8% (1,305) 23.0% (335) 24.4% (328) 20.2% (326) 15.5% (316)
15.2% (833) 16.4% (213) 15.7% (210) 16.2% (210) 12.5% (200)
0.62 (0.44 to 0.87) 0.63 (0.38 to 1.04) 0.47 (0.29 to 0.77) 0.68 (0.41 to 1.12) 0.73 (0.42 to 1.28)
0.006
51.2% (934)
57.6% (585)
1.32 (1.02 to 1.70)
0.032
29.0% (954) 43.9% (478)
32.3% (589) 54.5% (254)
1.16 (0.86 to 1.57) 1.67 (1.11 to 2.50)
0.337
0.068 0.003 0.127 0.278
0.013
* outcome variable, descriptive analysis, and post hoc analysis. †
outcome variable
‡
descriptive analysis
§
post hoc analysis
** From logistic regression analysis. The dependent variable is listed in the Outcome column. Improvement compares follow-up to baseline proportions; other variables included in the model are indicators of the respondent’s profession, respondent’s hospital, and, for dependent variables combining errors from the four stages of the medication process (outcome variables and post hoc analysis), the stage in the medication ordering to administration process in which the error occurred. OR = odds ratio, CI = confidence interval.
these improvements may appear to be disproportionate to the actions taken by participating teams. This observation may raise an additional concern about the validity of the survey results. The small number of hospitals for which pre–post data were available precluded meaningful analysis of associations between activities undertaken and improvement as measured by survey. An alternate explanation is that the activities undertaken had effects beyond their specific targets. The hospital teams described intervention strategies that were broad in scope that had the potential to affect the knowledge, attitudes, beliefs, and behaviors of a substantial portion of the clinical staff. Participation in the survey may have heightened clinical staff members’ awareness of errors. The most important change produced by project activ-
VOLUME
26
NUMBER
ities may have been improvement in the effectiveness of the safety cultures in participating hospitals. The observation of increased reporting of errors that reached the patient is consistent with this explanation. Enhancing the hospital’s safety culture was not a stated objective for this project; future medication error reduction program designers may want to consider its improvement as an explicit aim and expected outcome of their efforts.16 It may be possible to expand the range of topics on which clinical staff are surveyed (for example, perceptions of organizational response to and tolerance of error, individual attitudes toward limitations in human performance, knowledge of and compliance with policies) to include indicators of progress toward this objective.17
6
Copyright 2000 Joint Commission on Accreditation of Healthcare Organizations
339
THE JOINT COMMISSION Conclusions This voluntary project was initiated by the PRO because of national data, and not because of any highprofile incidents in local hospitals. The PRO and hospital collaborators had long recognized medication errors as a significant threat to patient safety. This project demonstrated interest in a proactive, nonpunitive, and collaborative approach to reducing this threat. The level and quality of activity we observed demon-
strated that this interest could be channeled, through active QI facilitation, into structured medication systems assessment and redesign, and that hospital teams could be empowered to implement substantive changes in their medication systems. Our observations indicate that participants increased their capacity for this task through project activities and may have improved patient safety by significantly reducing medication errors. J
References 1. Classen DC, et al: Adverse drug events in hospitalized patients: Excess length of stay, extra costs, and attributable mortality. JAMA 277:301–306, 1997. 2. Bates DW, et al, for the ADE Prevention Study Group: Incidence of adverse drug events and potential adverse drug events. JAMA 274: 29–34, 1995. 3. Bates DW, et al, for the Adverse Drug Events Prevention Study Group: The costs of adverse drug events in hospitalized patients. JAMA 277:307–311, 1997. 4. Leape LL, et al: The nature of adverse events in hospitalized
340
patients: Results of the Harvard Medical Practice Study II. New Engl J Med 324:377–384, 1991. 5. Leape LL, et al, for the ADE Prevention Study Group: Systems analysis of adverse drug events. JAMA 274:35–43, 1995. 6. Lesar TS, Lomaestro BM, Pohl H: Medication-prescribing errors in a teaching hospital. Arch Intern Med 157:1569–1576, 1997. 7. Allnutt MF: Human factors in accidents. Br J Anaesth 59:856–864, 1987. 8. Leape LL: Error in medicine. JAMA 272:1851–1857, 1994. 9. Classen DC, et al: Computer-
ized surveillance of adverse drug events in hospital patients. JAMA 266:2847–2851, 1991. 10. Reason J: Human Error. New York: Cambridge University Press, 1990. 11. Proctor RW, Van Zandt T: Human Factors in Simple and Complex Systems. Boston: Allyn & Bacon, 1994. 12. Sanders MS, McCormack EJ: Human Factors in Engineering and Design 7th ed. New York: McGraw-Hill, 1993. 13. Petersen D: Human Error Reduction and Safety Management 3rd ed. New York: Van Nostrand
Reinhold, 1996. 14. Stata Corporation: Stata Statistical Software: Release 5.0 (Volume 2). College Station, TX, 1997, pp 343–367. 15. Institute of Medicine: To Err Is Human: Building a Safer Health System (Kohn L, Corrigan J, Donaldson M, eds). Washington, DC: National Academy Press, 2000. 16. Reason J: Managing the Risks of Organizational Accidents. Brookfield, VT: Ashgate, 1997. 17. Helmreich RL, Merritt AC: Culture at Work in Aviation and Medicine. Brookfield, VT: Ashgate, 1998.
JUNE 2000 JOURNAL Copyright 2000 Joint Commission on Accreditation of Healthcare Organizations