- Email: [email protected]
Ambulatory Care Visits for Treating Adverse Drug Effects in the United States, 1995–2001
Medication Safety
Ambulatory Care Visits for Treating Adverse Drug Effects in the United States, 1995–2001 dverse drug events (ADEs) are a well recognized concern for patients, health professionals, and policy makers, but the magnitude of this problem in the United States is largely unknown.1 Efforts to assess ADEs have focused on selected settings or institutions such as hospitals,2–5 nursing homes,3,4,6,7 and outpatient settings.8–10 Although ADEs are generally defined as injuries resulting from the use of medications, researchers vary substantially in their approaches to defining, detecting, and classifying ADEs and in determining their severity and preventability. For example, a recent study by Gandhi et al.10 used chart reviews and a survey of outpatients at four adult primary care practices in Boston. They found that 25% of patients had ADEs and that 13% of the events were considered serious. In contrast, an earlier study by Gurwitz et al.9 investigated a cohort of Medicare patients cared for by a multispecialty group practice where ADEs were detected using a combination of provider report, chart review, automatic free-text electronic clinic notes, and administrative incidence reports. They found an overall ADE rate of 5.01 per 100 personyears, of which 27.6% were deemed preventable and 38.0% serious. Results from such studies vary substantially in part because of differences in defining and detecting ADEs. With data from one or a few institutions, such studies were able to improve our understanding of ADEs and potential preventive strategies, but they offer little insight into the nationwide prevalence of this problem or resulting need for health care.
A
372
July 2005
Chunliu Zhan, M.D., Ph.D. Irma Arispe, Ph.D. Edward Kelley, Ph.D. Tina Ding, M.S. Catharine W. Burt, Ed.D. Judith Shinogle, Ph.D. Daniel Stryer, M.D.
Article-at-a-Glance Background: Adverse drug events (ADEs) are a wellrecognized patient safety concern, but their magnitude is unknown. Ambulatory visits for treating adverse drug effects (VADEs) as recorded in national surveys offer an alternative way to estimate the national prevalence of ADEs because each VADE indicates that an ADE occurred and was serious enough to require care. Methods: A nationally representative sample of visits to physician offices, hospital outpatient departments, and emergency departments was analyzed. VADEs were identified as the first-listed cause of injury. Results: In 2001, there were 4.3 million VADEs in the United States, averaging 15 visits per 1,000 population. VADE rates at physician offices, hospital outpatient departments, and hospital emergency departments were at 3.7, 3.4, and 7.3 per 1,000 visits, respectively. There was an upward trend in the total number of VADEs from 1995 to 2001 (p < .05), but the increases in VADEs per 1,000 visits and per 1,000 population were not statistically significant. VADEs were lower in children younger than 15 and higher in the elderly aged 65–74 than in adults aged 25–44 (p < .01) and were more frequent in females than in males (p < .05). Discussion: Although methodologically conservative, the study suggests that ADEs are a significant threat to patient safety in the United States.
Volume 31 Number 7
Copyright 2005 Joint Commission on Accreditation of Healthcare Organizations
There are other potential data sources for national ADE assessment, such as MedWatch, an ADE reporting system at the Food and Drug Administration (FDA), and MEDMARX, a voluntary reporting system of medication errors maintained by the United States Pharmacopeia (USP).11,12 However, the usefulness of such data is limited because they are based on voluntary report. For example, the FDA’s voluntary ADE reporting covers about 1% to 10% of the actual ADEs.13 In addition, such data are usually collected without a common taxonomy and without proper denominators for prevalence assessment.12 Another potential data source is existing electronic medical records (EMRs). With the development and refinement of automatic triggers,14 automatic surveillance systems,15 data mining techniques, and free text searching,16 EMRs hold new promises for ADE assessment; however, nationwide use of such systems is in the distant future.12 In the absence of valid national data, interest is growing in the use of the International Classification of Diseases, 9th Revision, Clinical Modification (ICD-9-CM)17 codes recorded in administrative data to assess adverse events, including ADEs, in hospitals, nursing homes, and ambulatory settings.12,18–20 A number of studies have been done and some are underway to assess the validity of ICD-9-CM codes in identifying ADEs.21,22 Despite some well-known limitations, this approach is destined to be an integral part of any future EMR system, and much refinement in defining and applying this approach is expected in the near future.12 This study explored the use of existing national surveys and ICD-9-CM codes for ambulatory visits for adverse drug effects (VADEs) to assess national prevalence of ADEs. Specifically, this study used the National Ambulatory Medical Care Survey (NAMCS) and the National Hospital Ambulatory Medical Care Survey (NHAMCS) to examine ambulatory visits to physician offices, hospital outpatient departments, and emergency departments, where VADEs were coded as the cause of injury. Since each of visits for treating VADEs indicates that an ADE had occurred and was serious enough to require care, these visits recorded in the national surveys offer an alternative way to estimate national prevalence of ADEs and the burden of ADEs to the health care system.
July 2005
Methods Data Sources This study used NAMCS and NHAMCS public use data from 1995 to 2001, from the Centers for Disease Control and Prevention’s (CDC) National Center for Health Statistics (NCHS).23 NAMCS is a national probability sample survey of visits to nonfederally employed, officebased physicians who are primarily engaged in direct patient care. The survey uses a three-stage probability design with samples of geographically defined areas, physician practices within these areas, and patient visits within physician practices. Each physician is randomly assigned to a 1-week reporting period. During this period, data from a systematic random sample of visits are recorded by the physician or office staff on an encounter form provided for that purpose. Data are obtained on patients’ symptoms, physicians’ diagnoses, medications and services provided, and planned future treatment. NHAMCS is a similar national probability sample survey that collects data on ambulatory care services provided by hospital outpatient and emergency departments of general nonfederal and short-stay hospitals. The survey uses a four-stage probability design with samples of geographically defined areas, hospitals within these areas, clinics within hospitals, and patient visits within clinics. Data items similar to NAMCS are collected from a random sample of patient visits during a randomly assigned four-week reporting period. Patient records in both databases are assigned year-based weights for calculating annual, national estimates. The sample sizes each year from 1995–2001 ranged from 21,000 to 37,000 visits to physician office visits, 27,000 to 37,000 visits to hospital outpatient departments, and 21,000 to 34,000 visits to hospital emergency departments, which included visits resulting in admission to the hospital. The total numbers of VADEs identified from the samples ranged from 125 to 248 each year. The United States population estimates from 1995 to 2001, which were used to calculate VADEs per 1,000 population, were derived from U.S. Census estimates for the civilian noninstitutionalized population and were obtained from NCHS Public Use File documentation.23
Volume 31 Number 7
Copyright 2005 Joint Commission on Accreditation of Healthcare Organizations
373
Identifying Ambulatory Visits for Treating VADEs This study used ICD-9-CM codes for External Causes and Injury (E-codes), E930-E947, to identify VADEs. Among the E-codes of adverse effects of drugs, we excluded adverse effects due to bacterial vaccine (E948) and other vaccine and biological substances (E949) and excluded adverse reactions to heroin (E9350) and methadone (E9351). NCHS determines whether a visit is related to injury and poisoning and, if so, assigns Ecodes based on the verbatim description of the causes of injuries recorded on visit records. A 10% quality control sample of survey records was independently coded as part of the quality assurance procedure, which suggests that the coding errors ranged between 0.8 to 2% of records for E-codes. E-codes have been used in previous studies examining VADEs. Aparasu and colleagues24–26 used E930-E947 recoded in NAMCS and NHAMCS data to study ambulatory care visits for adverse effects of medications. Burt27 used E930-E947 in NHAMCS to study VADE-related visits to emergency departments but excluded adverse reactions to heroin (E9350) and methadone (E9351). Researchers at the Utah Department of Health, HealthInsight (the Utah Quality Improvement Organization), and the University of Utah have been examining the utility of ICD-9-CM codes in hospital administrative data to identify ADEs in hospitalization.21,22 In addition to the E-codes mentioned above, they included clinical diagnosis codes for drug psychoses (292), dermatitis (6923, 6929, 6930, 6938, 6939), drug reactions and intoxications to the newborns (76072, 76074, 7635, 7794), rash and spontaneous ecchymoses (7821, 7827), and intentional and unintentional poisoning (960-969, 9090, 970-979, E850-E858). The validity of these codes in identifying ADEs is still under investigation. In general, the validity varies by individual codes, depends on coder’s understanding of the codes, and depends on data collection methods, and the validity could be improved by clearly and uniformly defining what should be coded in E-codes and by improving data collection process so what should be coded in E-codes are coded so. Our analysis focused on E930-E947 that identify VADEs most likely to be attributable to inappropriate use or misuse of medicines, in other words, the events with patient safety implications.
374
July 2005
Our analysis of the E-codes was restricted to the first of the three listed causes of injury to ensure that the primary reason for the flagged visit was to treat VADEs.
Statistical Analysis Survey software28 was used to incorporate features of the survey designs in producing national estimates of VADEs and standard errors for each of the seven years (1995–2001) and three care settings (physician office, hospital outpatient department, and hospital emergency department). The distributions of VADEs by patient characteristics in 2001 were tabulated. Patient complaints and drug classes associated with the VADEs were tabulated based on the pooled data from 1995–2001. Differences in VADE rates between patient groups were tested using t-test and analysis of variance with the Bonferroni correction for multiple comparison. Statistical significance of trends was estimated from least-square regressions of VADE estimates on time (year), weighted by the inverse of the standard deviations of year-specific estimates. P values < 0.05 were considered statistically significant.
Results Table 1 (page 375) shows the national estimates of VADEs to physician offices, hospital outpatient departments, and emergency rooms from 1995–2001. The VADE rates were 2.5–3.7 per 1,000 physician office visits, 1.8–3.4 per 1,000 hospital outpatient department visits, and 5.7–7.3 per 1,000 hospital emergency department visits. Of all the VADEs in 2001, 74% were made to physician offices, 20% to hospital emergency departments, and 6% to hospital outpatient departments. Overall, 11–15 VADEs were made per 1,000 US population each year. The total numbers of VADEs in the US increased from 2.9 million in 1995 to 4.3 million in 2001. Weighted least square regressions indicate that the increase in the total number of VADEs in the United States from 1995 to 2001 was statistically significant (p < .05), but the trends in VADEs per 1,000 visits and per 1,000 population were not statistically significant (p > .05). Table 2 (page 376) shows the distribution of VADEs across population characteristics. The VADE rates per 1,000 ambulatory visits and per 1,000 population were lower in children (0–15 years old) and higher for the
Volume 31 Number 7
Copyright 2005 Joint Commission on Accreditation of Healthcare Organizations
Table 1. National Estimates of Ambulatory Care Visits due to Adverse Drug Effects (VADEs) in the United States, 1995–2001* All Ambulatory Visits Physician Office Visits Total Number of VADEs (in 1,000s) VADEs per 1,000 visits VADEs per 1,000 population per year Hospital Outpatient Department Visits Total Number of VADEs (in 1,000s) VADEs per 1,000 visits VADEs per 1,000 population per year Hospital Emergency Department Visits Total Number of VADEs (in 1,000s) VADEs per 1,000 visits VADEs per 1,000 population per year Total Total Number of VADEs (in 1,000s) VADEs per 1,000 visits VADEs per 1,000 population per year
1995
1996
1997
1998
1999
2000
2001
2016 (293) 2395 (351) 1983 (320) 2358 (446) 2125 (396) 2673 (494) 3237 (522) 2.9 (0.4) 3.3 (0.4) 2.5 (0.4) 2.8 (0.5) 2.8 (0.5) 3.2 (0.6) 3.7 (0.6) 7.7 (1.12) 9.1 (1.3) 7.4 (1.0) 8.7 (1.6) 7.8 (1.5) 9.7 (1.8) 11.5 (1.8) 131 (29) 1.9 (0.4) 0.5 (0.1)
120 (28) 1.8 (0.4) 0.5 (0.1)
137 (35) 1.8 (0.4) 0.5 (0.1)
174 (35) 2.3 (0.4) 0.7 (0.1)
258 (49) 3.0 (0.5) 1.0 (0.2)
281 (70) 3.4 (0.8) 1.0 (0.3)
285 (57) 3.4 (0.6) 1.0 (0.2)
709 (79) 7.2 (0. 8) 2.7 (0.3)
511 (59) 5.7 (0.6) 1.9 (0.2)
594 (76) 6.3 (0.8) 2.2 (0.3)
587 (71) 5.8 (0.6) 2.2 (0.3)
687 (87) 6.7 (0.8) 2.5 (0.3)
594 (65) 5.5 (0.5) 2.2 (0.2)
790 (72) 7.3 (0.6) 2.8 (0.3)
2856 (311) 3027 (372) 2713 (322) 3120 (448) 3070 (418) 3547 (491) 4311 (535) 3.3 (0.3) 3.4 (0.4) 2.8 (0.3) 3.1 (0.4) 3.3 (0.4) 3.5 (0.5) 4.0 (0.5) 10.9 (1.2) 11.4 (1.4) 10.2 (1.2) 11.6 (1.7) 11.3 (1.5) 12.9 (1.8) 15.3 (1.9)
* Standard error in parentheses. Visits per 1,000 population were obtained by dividing the total number of visits by U.S. Bureau of the Census estimates of the civilian noninstititutional population as of July 1, for 1995–2001.
elderly (65–74 years old) than in adults aged 25–44 years (p < .01) and were more frequent in females than in males (p <.05). Variations across racial groups and regions were not statistically significant. Table 3 (page 377) shows the medications associated with VADEs by therapeutic class. Therapeutic class was identified in about 70% of the VADEs. Four classes of drugs (antibiotics and other anti-infectives; hormones and synthetic substitutes; analgesics, antipyretics, and antirheumatics; and agents primarily affecting the cardiovascular system) were most frequently implicated, accounting for almost half of the VADEs. Patients who sought ambulatory care for treating VADEs most frequently suffered from dermatological symptoms such as skin rash (11%), followed by gastrointestinal symptoms such as nausea, vomiting, and abdominal pains (8%).* We also examined other ADE-related codes to assess whether including them would change the estimates substantially. Few cases of adverse effects
* The frequency distribution of patient symptoms is available by e-mail request to Dr. Zhan.
July 2005
due to bacterial vaccine (E948) and other vaccine and biological substances (E949) and adverse reactions to heroin (E9350), methadone (E9351), and clinical and poisoning codes were found in the data. Therefore, inclusion of other ADE-related codes in addition to E930–E947 would not have substantially altered the findings.
Discussion This study used existing national surveys and ICD-9-CM E-codes to assess the magnitude of ADEs in the United States. Our data show that about 1 ambulatory care visit is made per 100 population each year to treat VADEs, totaling 3–4 million VADEs a year in the United States. Although the VADE rates in ambulatory care did not increase significantly from 1995 to 2001, our data did suggest an upward trend in the total number of VADEs in the nation because of increases in the population. Currently there is no common approach to defining and identifying ADEs, and there are no national or largescale data for assessing national prevalence of ADEs. Our approach provides a patient-centered, utilizationbased definition of ADEs, where only VADEs that are
Volume 31 Number 7
Copyright 2005 Joint Commission on Accreditation of Healthcare Organizations
375
Table 2. Ambulatory Care Visits due to Adverse Drug Effects by Patient Demographics, 2001*
may be subject to abstraction errors. The extent of these errors cannot be ascertained without additional reliability studies. Patient Total Number of Visits per Visits per 1,000 Second, some VADEs might not be correctDemograpics Visits (in 1,000s) 1,000 Visits Population Per Year ly identified and reported by the survey Age Group (years) respondents. For example, reporting physi0–15 244 (80) 1.3 (0.4) 4.1 (1.3) cians may not attribute a patient’s visit to 15–24 428 (136) 4.6 (0.2) 11.0 (3.5) an adverse effect, or the external cause of 25–44 783 (187) 3.1 (0.7) 9.4 (2.3) injury section of the data form may not 45–64 1286 (279) 4.6 (0.1) 20.1 (4.4) record the adverse effect. In this case, an E65–74 1027 (270) 8.1 (0.2) 56.8 (14.8) code would not be assigned. Such errors in 542 (164) 4.1 (0.1) 34.7 (10.5) <75 general result in underestimation of Sex VADEs. Third, there is a concern that a corFemale 2858 (406) 4.6 (0.1) 20.9 (2.9) rectly coded E-code may not always indiMale 1453 (317) 3.3 (0.1) 10.1 (2.2) cate an ADE. Fourth, there is little temporal Race information and clinical content in ICD-9White 3648 (503) 4.0 (0.6) 16.1 (2.2) CM codes. As a result, E-code-based ADE Black 525 (153) 5.0 (0.2) 15.0 (4.4) studies offer little insight into the harm, Other 139 (82) 3.2 (0.2) 7.6 (4.5) severity, preventability, and causes of the Region ADEs and provide little guidance in the Northeast 726 (205) 3.0 (0.1) 13.7 (3.9) Midwest 1028 (219) 4.4 (0.1) 16.1 (3.4) development of preventive strategies. South 1867 (409) 5.3 (0.1) 18.7 (4.1) Finally, some variation in VADEs is rooted West 690 (168) 2.8 (0.1) 10.9 (2.6) in patient tolerance of drugs, patient inclination in seeking care, and availability of * Standard error in parentheses. care, which may lead to biases in comparative analysis. It must also be noted that the estimates we provide are for visits for treating VADEs, serious enough to prompt a patient to seek ambulatory not ADEs. ADEs that were not treated at ambulatory care and are diagnosed and documented by a health care care settings were not counted. professional are considered an ADE. This approach to Consistent with previous findings,2–4,7–10 this study sugassessing ADEs offers some advantages. First, this definition eliminates arbitrariness in defining and detecting gests that ADEs are a significant safety concern in ADEs. In combination with the standardized methods for United States health care. With the increasing demand defining and collecting data in NAMCS and NHAMCS, for assessment data at the national level,1 the national this approach provides objective estimates and facilisurveys and E-codes appear to be a viable alternative to tates objective comparisons of ADE rates over time, provide national estimates of ADE prevalence. They across regions, and across care settings. Second, this offer a potential measure for the annual National approach relies on existing national representative surHealthcare Quality Report,29 for which no ADE measure veys and is therefore a cost-effective way to develop is currently available. J national estimates of ADEs. The authors acknowledge comments and suggestions made by Barry Friedman, Ph.D., Marge Keyes, M.S., and Joanna Jiang, Ph.D., Agency Several limitations should be noted. First, our estifor Healthcare Research and Quality, and Xu Wu, Ph.D., and Paul Hougland, Ph.D., Utah Department of Health. This article was a result mates can suffer from errors in the survey data. As indiof an intramural research project conducted at the Agency for cated in the methods section, the NAMCS and NHAMCS Healthcare Research and Quality (AHRQ). No external funding was used. The authors of this article are responsible for its content. capture adverse effects data using diagnosis and verbaNo statement in this article should be construed as an official tim text on external causes of injury. Although survey position of the AHRQ or the U.S. Department of Health and Human Services. coding errors are < 1%, it is possible that the survey
376
July 2005
Volume 31 Number 7
Copyright 2005 Joint Commission on Accreditation of Healthcare Organizations
Table 3. Medications Responsible for Adverse Drug Effects, Pooled 1995–2001 Data (%)* Drug Implicated (E-code) Antibiotics and other anti-infectives (E930-E931) Hormones and synthetic substitutes (E932) Primarily systemic agents (E933) Agents primarily affecting blood constituents (E934) Analgesics, antipyretics, and antirheumatics (E935) Anticonvulsants and anti-Parkinsonism drugs (E936) Sedatives and hypnotics (E937) Other central nervous system depressants, stimulants, nervous system agents (E938, E940, E941) Psychotropic agents (E939) Agents primarily affecting the cardiovascular system (E942) Agents primarily affecting the gastrointestinal system (E943) Water, mineral, and uric acid metabolism drugs (E944) Agents primarily acting on smooth/skeletal muscles and the respiratory system (E945) Agents primarily affecting skin/mucous membrance and ophthalmological, otorhinolarygological, and dental drugs (E946) Other and unspecified drugs and medical substances (E947) Total
Physician Office 11.0 12.7 1.9 1.3 5.7 2.1 0.6
Hospital OPD 16.8 16.0 7.9 3.2 6.3 2.2 0.6
Hospital ED 23.0 6.9 3.6 1.6 11.1 2.7 1.2
0.8
1.3
1.8
1.3
5.7 10.5 0.8 1.1
4.1 5.4 0.6 0.9
6.1 6.7 1.1 1.3
5.4 7.6 0.9 1.2
0.9
1.1
1.6
1.9
4.0
1.7
3.2
3.1
40.9 100.0
31.8 100.0
28.2 100.0
33.0 100.0
Total 18.5 10.7 3.9 1.9 7.9 2.3 0.9
* OPD, outpatient department; ED, emergency department.
Chunliu Zhan, M.D., Ph.D., is Staff Service Fellow, Intramural, National Healthcare Quality Report, Patient Safety, Agency for Healthcare Research and Quality (AHRQ), Department of Health and Human Services, Rockville, Maryland. Irma E. Arispe, Ph.D., is Associate Director for Science, Division of Health Care Statistics, National Center for Health Statistics (NCHS), Centers for Disease Control and Prevention. Edward Kelley, Ph.D., is Director, National Healthcare Quality Report, AHRQ. Tina Ding, M.S., is Research Analyst, Social Science and System, Inc., Silver Spring, Maryland. Catharine W. Burt, Ed.D., is Chief, Ambulatory Care Statistics Branch, Division of Health Care Statistics. Judith Shinogle, Ph.D., is NCHS/AcademyHealth Health Policy Fellow, Division of Health Care Statistics. Daniel Stryer, M.D. (deceased), was Director, Center for Quality Improvement and Patient Safety, AHRQ. Please address reprint requests to Chunliu Zhan, M.D., Ph.D., [email protected].
References 1. General Accounting Office (GAO): Adverse Drug Events: Substantial Problem but Magnitude Uncertain. GAO: Washington, D.C., 2000. 2. Classen D., et al.: Computerized surveillance of adverse drug events in hospital patients. JAMA 266:2847–2851, Nov. 27, 1991. 3. Bates D.W., et al.: Incidence of adverse drug events and potential adverse drug events: Implications for prevention. ADE Prevention Study Group. JAMA 274:29–34, Jul. 5, 1995. 4. Bates D.W., et al.: The costs of adverse drug events in hospitalized patients. Adverse Drug Events Prevention Study Group. JAMA 277:307–311, Jan. 22, 1997. 5. Raschetti R., et al.: Suspected adverse drug events requiring emergency department visits or hospital admissions. Eur J Clin Pharmacol 54:959–963, Feb. 1999. 6. Monette J., Gurwitz J.H., Avorn J.: Epidemiology of adverse drug events in the nursing home setting. Drugs Aging 7:203–211, Sep. 1995. 7. Gurwitz J.H., et al.: Incidence and preventability of adverse drug events in nursing homes. Am J Med 109:87–94, Aug. 1, 2000.
continued
July 2005
Volume 31 Number 7
Copyright 2005 Joint Commission on Accreditation of Healthcare Organizations
377
References, continued 8. Honigman B., et al.: Using computerized data to identify adverse drug events in outpatients. J Am Med Inform Assoc 8:254–266, May–Jun., 2001. 9. Gurwitz J.H., et al.: Incidence and preventability of adverse drug events among older persons in the ambulatory setting. JAMA 289:1107–1116, May 5, 2003. 10. Gandhi T.K., et al.: Adverse drug events in ambulatory care. N Engl J Med 348:1556–1564, Apr. 17, 2003. 11. Hicks R., Cousins D., Williams R.: Summary of Information Submitted to MEDMARX in the Year 2002. Rockville, MD: United States Pharmacopeia Center for the Advancement of Patient Safety, 2003. 12. Institute of Medicine: Patient Safety: Achieving a New Standard for Care. Washington, D.C.: National Academy Press; 2004. 13. Center for Drug Evaluation and Research, Food and Drug Administration: Annual Adverse Drug Experience Report: 1996. http://www.fda.gov/cder/dpe/annrep96/index.htm (last accessed May 11, 2005). 14. Rozich J., Haraden C., Resar R.: Adverse drug trigger tool: A practical methodology for measuring medication-related harm. Qual Saf Health Care 12:194–200, Jun 2003. 15. Honigman B., et al.: A computerized method for identifying incidences associated with adverse drug events in outpatients. Int J Med Inf 6:21–32, Apr. 2001. 16. Bates D.W., et al.: Asking residents about adverse events in a computer dialogue: How accurate are they? Jt Comm J Qual Improv 24:197–202, Apr. 1998. 17. U.S. Department of Health and Human Services (DHHS): International Classification of Diseases, Ninth Revision, Clinical Modification. Washington, D.C.: DHHS, 1997. 18. Iezzoni L.I., et al.: A method for screening the quality of hospital care using administrative data: preliminary validation results. QRB Qual Rev Bull 18:361–371, Nov. 1992.
378
July 2005
19. University of California at San Francisco, Stanford Evidence-Based Practice Center: Evidence Report for Measures of Patient Safety Based on Hospital Administrative Data: The Patient Safety Indicators. Rockville, MD: Agency for Healthcare Research and Quality, 2002. 20. Zhan C., Miller M.: Administrative data-based patient safety research: A critical review. Qual Saf Health Care 12(suppl II):ii58–ii63, Dec. 2003. 21. Utah Department of Health/Utah Hospitals and Health Systems Association: Utah Patient Safety Update, 1(3):1–7, 2003. 22. Xu W., Hougland P.: Using ICD-9-CM and ICD-10 data to improve patient safety in Utah. Paper presented at the AcademyHealth Annual Research Meeting, Nashville, TN, Jun. 28, 2003. 23. National Center for Health Statistics: Ambulatory Health Care Data. http://www.cdc.gov/nchs/about/major/ahcd/ahcd1.htm (last accessed May 11, 2005). 24. Aparasu R.R., Helgeland D.: Utilization of ambulatory care services caused by adverse effects of medications in the United States. Manag Care Interface 13:70–75, Apr. 2000. 25. Aparasu R.R., Helgeland D.L.: Visits to hospital outpatient departments in the United States due to adverse effects of medications. Hosp Pharm 35:825–831, Aug. 2000. 26. Aparasu R.R.: Visits to office-based physicians in the United States for medication-related morbidity. J Am Pharm Assoc 39:332–337, May–Jun. 1999. 27. Burt C.: Emergency health care encounters for adverse effects of medical treatment. Manag Care Interface 14:39–42, Dec. 2001. 28. Research Triangle Institute: SUDAAN 8.0.2. (computer software). Research Triangle Park, NC: Research Triangle Institute, 2003. 29. Institute of Medicine: Envisioning the National Health Care Quality Report. Washington, D.C.: National Academy Press; 2001.
Volume 31 Number 7
Copyright 2005 Joint Commission on Accreditation of Healthcare Organizations
Ambulatory Care Visits for Treating Adverse Drug Effects in the United States, 1995–2001 dverse drug events (ADEs) are a well recognized concern for patients, health professionals, and policy makers, but the magnitude of this problem in the United States is largely unknown.1 Efforts to assess ADEs have focused on selected settings or institutions such as hospitals,2–5 nursing homes,3,4,6,7 and outpatient settings.8–10 Although ADEs are generally defined as injuries resulting from the use of medications, researchers vary substantially in their approaches to defining, detecting, and classifying ADEs and in determining their severity and preventability. For example, a recent study by Gandhi et al.10 used chart reviews and a survey of outpatients at four adult primary care practices in Boston. They found that 25% of patients had ADEs and that 13% of the events were considered serious. In contrast, an earlier study by Gurwitz et al.9 investigated a cohort of Medicare patients cared for by a multispecialty group practice where ADEs were detected using a combination of provider report, chart review, automatic free-text electronic clinic notes, and administrative incidence reports. They found an overall ADE rate of 5.01 per 100 personyears, of which 27.6% were deemed preventable and 38.0% serious. Results from such studies vary substantially in part because of differences in defining and detecting ADEs. With data from one or a few institutions, such studies were able to improve our understanding of ADEs and potential preventive strategies, but they offer little insight into the nationwide prevalence of this problem or resulting need for health care.
A
372
July 2005
Chunliu Zhan, M.D., Ph.D. Irma Arispe, Ph.D. Edward Kelley, Ph.D. Tina Ding, M.S. Catharine W. Burt, Ed.D. Judith Shinogle, Ph.D. Daniel Stryer, M.D.
Article-at-a-Glance Background: Adverse drug events (ADEs) are a wellrecognized patient safety concern, but their magnitude is unknown. Ambulatory visits for treating adverse drug effects (VADEs) as recorded in national surveys offer an alternative way to estimate the national prevalence of ADEs because each VADE indicates that an ADE occurred and was serious enough to require care. Methods: A nationally representative sample of visits to physician offices, hospital outpatient departments, and emergency departments was analyzed. VADEs were identified as the first-listed cause of injury. Results: In 2001, there were 4.3 million VADEs in the United States, averaging 15 visits per 1,000 population. VADE rates at physician offices, hospital outpatient departments, and hospital emergency departments were at 3.7, 3.4, and 7.3 per 1,000 visits, respectively. There was an upward trend in the total number of VADEs from 1995 to 2001 (p < .05), but the increases in VADEs per 1,000 visits and per 1,000 population were not statistically significant. VADEs were lower in children younger than 15 and higher in the elderly aged 65–74 than in adults aged 25–44 (p < .01) and were more frequent in females than in males (p < .05). Discussion: Although methodologically conservative, the study suggests that ADEs are a significant threat to patient safety in the United States.
Volume 31 Number 7
Copyright 2005 Joint Commission on Accreditation of Healthcare Organizations
There are other potential data sources for national ADE assessment, such as MedWatch, an ADE reporting system at the Food and Drug Administration (FDA), and MEDMARX, a voluntary reporting system of medication errors maintained by the United States Pharmacopeia (USP).11,12 However, the usefulness of such data is limited because they are based on voluntary report. For example, the FDA’s voluntary ADE reporting covers about 1% to 10% of the actual ADEs.13 In addition, such data are usually collected without a common taxonomy and without proper denominators for prevalence assessment.12 Another potential data source is existing electronic medical records (EMRs). With the development and refinement of automatic triggers,14 automatic surveillance systems,15 data mining techniques, and free text searching,16 EMRs hold new promises for ADE assessment; however, nationwide use of such systems is in the distant future.12 In the absence of valid national data, interest is growing in the use of the International Classification of Diseases, 9th Revision, Clinical Modification (ICD-9-CM)17 codes recorded in administrative data to assess adverse events, including ADEs, in hospitals, nursing homes, and ambulatory settings.12,18–20 A number of studies have been done and some are underway to assess the validity of ICD-9-CM codes in identifying ADEs.21,22 Despite some well-known limitations, this approach is destined to be an integral part of any future EMR system, and much refinement in defining and applying this approach is expected in the near future.12 This study explored the use of existing national surveys and ICD-9-CM codes for ambulatory visits for adverse drug effects (VADEs) to assess national prevalence of ADEs. Specifically, this study used the National Ambulatory Medical Care Survey (NAMCS) and the National Hospital Ambulatory Medical Care Survey (NHAMCS) to examine ambulatory visits to physician offices, hospital outpatient departments, and emergency departments, where VADEs were coded as the cause of injury. Since each of visits for treating VADEs indicates that an ADE had occurred and was serious enough to require care, these visits recorded in the national surveys offer an alternative way to estimate national prevalence of ADEs and the burden of ADEs to the health care system.
July 2005
Methods Data Sources This study used NAMCS and NHAMCS public use data from 1995 to 2001, from the Centers for Disease Control and Prevention’s (CDC) National Center for Health Statistics (NCHS).23 NAMCS is a national probability sample survey of visits to nonfederally employed, officebased physicians who are primarily engaged in direct patient care. The survey uses a three-stage probability design with samples of geographically defined areas, physician practices within these areas, and patient visits within physician practices. Each physician is randomly assigned to a 1-week reporting period. During this period, data from a systematic random sample of visits are recorded by the physician or office staff on an encounter form provided for that purpose. Data are obtained on patients’ symptoms, physicians’ diagnoses, medications and services provided, and planned future treatment. NHAMCS is a similar national probability sample survey that collects data on ambulatory care services provided by hospital outpatient and emergency departments of general nonfederal and short-stay hospitals. The survey uses a four-stage probability design with samples of geographically defined areas, hospitals within these areas, clinics within hospitals, and patient visits within clinics. Data items similar to NAMCS are collected from a random sample of patient visits during a randomly assigned four-week reporting period. Patient records in both databases are assigned year-based weights for calculating annual, national estimates. The sample sizes each year from 1995–2001 ranged from 21,000 to 37,000 visits to physician office visits, 27,000 to 37,000 visits to hospital outpatient departments, and 21,000 to 34,000 visits to hospital emergency departments, which included visits resulting in admission to the hospital. The total numbers of VADEs identified from the samples ranged from 125 to 248 each year. The United States population estimates from 1995 to 2001, which were used to calculate VADEs per 1,000 population, were derived from U.S. Census estimates for the civilian noninstitutionalized population and were obtained from NCHS Public Use File documentation.23
Volume 31 Number 7
Copyright 2005 Joint Commission on Accreditation of Healthcare Organizations
373
Identifying Ambulatory Visits for Treating VADEs This study used ICD-9-CM codes for External Causes and Injury (E-codes), E930-E947, to identify VADEs. Among the E-codes of adverse effects of drugs, we excluded adverse effects due to bacterial vaccine (E948) and other vaccine and biological substances (E949) and excluded adverse reactions to heroin (E9350) and methadone (E9351). NCHS determines whether a visit is related to injury and poisoning and, if so, assigns Ecodes based on the verbatim description of the causes of injuries recorded on visit records. A 10% quality control sample of survey records was independently coded as part of the quality assurance procedure, which suggests that the coding errors ranged between 0.8 to 2% of records for E-codes. E-codes have been used in previous studies examining VADEs. Aparasu and colleagues24–26 used E930-E947 recoded in NAMCS and NHAMCS data to study ambulatory care visits for adverse effects of medications. Burt27 used E930-E947 in NHAMCS to study VADE-related visits to emergency departments but excluded adverse reactions to heroin (E9350) and methadone (E9351). Researchers at the Utah Department of Health, HealthInsight (the Utah Quality Improvement Organization), and the University of Utah have been examining the utility of ICD-9-CM codes in hospital administrative data to identify ADEs in hospitalization.21,22 In addition to the E-codes mentioned above, they included clinical diagnosis codes for drug psychoses (292), dermatitis (6923, 6929, 6930, 6938, 6939), drug reactions and intoxications to the newborns (76072, 76074, 7635, 7794), rash and spontaneous ecchymoses (7821, 7827), and intentional and unintentional poisoning (960-969, 9090, 970-979, E850-E858). The validity of these codes in identifying ADEs is still under investigation. In general, the validity varies by individual codes, depends on coder’s understanding of the codes, and depends on data collection methods, and the validity could be improved by clearly and uniformly defining what should be coded in E-codes and by improving data collection process so what should be coded in E-codes are coded so. Our analysis focused on E930-E947 that identify VADEs most likely to be attributable to inappropriate use or misuse of medicines, in other words, the events with patient safety implications.
374
July 2005
Our analysis of the E-codes was restricted to the first of the three listed causes of injury to ensure that the primary reason for the flagged visit was to treat VADEs.
Statistical Analysis Survey software28 was used to incorporate features of the survey designs in producing national estimates of VADEs and standard errors for each of the seven years (1995–2001) and three care settings (physician office, hospital outpatient department, and hospital emergency department). The distributions of VADEs by patient characteristics in 2001 were tabulated. Patient complaints and drug classes associated with the VADEs were tabulated based on the pooled data from 1995–2001. Differences in VADE rates between patient groups were tested using t-test and analysis of variance with the Bonferroni correction for multiple comparison. Statistical significance of trends was estimated from least-square regressions of VADE estimates on time (year), weighted by the inverse of the standard deviations of year-specific estimates. P values < 0.05 were considered statistically significant.
Results Table 1 (page 375) shows the national estimates of VADEs to physician offices, hospital outpatient departments, and emergency rooms from 1995–2001. The VADE rates were 2.5–3.7 per 1,000 physician office visits, 1.8–3.4 per 1,000 hospital outpatient department visits, and 5.7–7.3 per 1,000 hospital emergency department visits. Of all the VADEs in 2001, 74% were made to physician offices, 20% to hospital emergency departments, and 6% to hospital outpatient departments. Overall, 11–15 VADEs were made per 1,000 US population each year. The total numbers of VADEs in the US increased from 2.9 million in 1995 to 4.3 million in 2001. Weighted least square regressions indicate that the increase in the total number of VADEs in the United States from 1995 to 2001 was statistically significant (p < .05), but the trends in VADEs per 1,000 visits and per 1,000 population were not statistically significant (p > .05). Table 2 (page 376) shows the distribution of VADEs across population characteristics. The VADE rates per 1,000 ambulatory visits and per 1,000 population were lower in children (0–15 years old) and higher for the
Volume 31 Number 7
Copyright 2005 Joint Commission on Accreditation of Healthcare Organizations
Table 1. National Estimates of Ambulatory Care Visits due to Adverse Drug Effects (VADEs) in the United States, 1995–2001* All Ambulatory Visits Physician Office Visits Total Number of VADEs (in 1,000s) VADEs per 1,000 visits VADEs per 1,000 population per year Hospital Outpatient Department Visits Total Number of VADEs (in 1,000s) VADEs per 1,000 visits VADEs per 1,000 population per year Hospital Emergency Department Visits Total Number of VADEs (in 1,000s) VADEs per 1,000 visits VADEs per 1,000 population per year Total Total Number of VADEs (in 1,000s) VADEs per 1,000 visits VADEs per 1,000 population per year
1995
1996
1997
1998
1999
2000
2001
2016 (293) 2395 (351) 1983 (320) 2358 (446) 2125 (396) 2673 (494) 3237 (522) 2.9 (0.4) 3.3 (0.4) 2.5 (0.4) 2.8 (0.5) 2.8 (0.5) 3.2 (0.6) 3.7 (0.6) 7.7 (1.12) 9.1 (1.3) 7.4 (1.0) 8.7 (1.6) 7.8 (1.5) 9.7 (1.8) 11.5 (1.8) 131 (29) 1.9 (0.4) 0.5 (0.1)
120 (28) 1.8 (0.4) 0.5 (0.1)
137 (35) 1.8 (0.4) 0.5 (0.1)
174 (35) 2.3 (0.4) 0.7 (0.1)
258 (49) 3.0 (0.5) 1.0 (0.2)
281 (70) 3.4 (0.8) 1.0 (0.3)
285 (57) 3.4 (0.6) 1.0 (0.2)
709 (79) 7.2 (0. 8) 2.7 (0.3)
511 (59) 5.7 (0.6) 1.9 (0.2)
594 (76) 6.3 (0.8) 2.2 (0.3)
587 (71) 5.8 (0.6) 2.2 (0.3)
687 (87) 6.7 (0.8) 2.5 (0.3)
594 (65) 5.5 (0.5) 2.2 (0.2)
790 (72) 7.3 (0.6) 2.8 (0.3)
2856 (311) 3027 (372) 2713 (322) 3120 (448) 3070 (418) 3547 (491) 4311 (535) 3.3 (0.3) 3.4 (0.4) 2.8 (0.3) 3.1 (0.4) 3.3 (0.4) 3.5 (0.5) 4.0 (0.5) 10.9 (1.2) 11.4 (1.4) 10.2 (1.2) 11.6 (1.7) 11.3 (1.5) 12.9 (1.8) 15.3 (1.9)
* Standard error in parentheses. Visits per 1,000 population were obtained by dividing the total number of visits by U.S. Bureau of the Census estimates of the civilian noninstititutional population as of July 1, for 1995–2001.
elderly (65–74 years old) than in adults aged 25–44 years (p < .01) and were more frequent in females than in males (p <.05). Variations across racial groups and regions were not statistically significant. Table 3 (page 377) shows the medications associated with VADEs by therapeutic class. Therapeutic class was identified in about 70% of the VADEs. Four classes of drugs (antibiotics and other anti-infectives; hormones and synthetic substitutes; analgesics, antipyretics, and antirheumatics; and agents primarily affecting the cardiovascular system) were most frequently implicated, accounting for almost half of the VADEs. Patients who sought ambulatory care for treating VADEs most frequently suffered from dermatological symptoms such as skin rash (11%), followed by gastrointestinal symptoms such as nausea, vomiting, and abdominal pains (8%).* We also examined other ADE-related codes to assess whether including them would change the estimates substantially. Few cases of adverse effects
* The frequency distribution of patient symptoms is available by e-mail request to Dr. Zhan.
July 2005
due to bacterial vaccine (E948) and other vaccine and biological substances (E949) and adverse reactions to heroin (E9350), methadone (E9351), and clinical and poisoning codes were found in the data. Therefore, inclusion of other ADE-related codes in addition to E930–E947 would not have substantially altered the findings.
Discussion This study used existing national surveys and ICD-9-CM E-codes to assess the magnitude of ADEs in the United States. Our data show that about 1 ambulatory care visit is made per 100 population each year to treat VADEs, totaling 3–4 million VADEs a year in the United States. Although the VADE rates in ambulatory care did not increase significantly from 1995 to 2001, our data did suggest an upward trend in the total number of VADEs in the nation because of increases in the population. Currently there is no common approach to defining and identifying ADEs, and there are no national or largescale data for assessing national prevalence of ADEs. Our approach provides a patient-centered, utilizationbased definition of ADEs, where only VADEs that are
Volume 31 Number 7
Copyright 2005 Joint Commission on Accreditation of Healthcare Organizations
375
Table 2. Ambulatory Care Visits due to Adverse Drug Effects by Patient Demographics, 2001*
may be subject to abstraction errors. The extent of these errors cannot be ascertained without additional reliability studies. Patient Total Number of Visits per Visits per 1,000 Second, some VADEs might not be correctDemograpics Visits (in 1,000s) 1,000 Visits Population Per Year ly identified and reported by the survey Age Group (years) respondents. For example, reporting physi0–15 244 (80) 1.3 (0.4) 4.1 (1.3) cians may not attribute a patient’s visit to 15–24 428 (136) 4.6 (0.2) 11.0 (3.5) an adverse effect, or the external cause of 25–44 783 (187) 3.1 (0.7) 9.4 (2.3) injury section of the data form may not 45–64 1286 (279) 4.6 (0.1) 20.1 (4.4) record the adverse effect. In this case, an E65–74 1027 (270) 8.1 (0.2) 56.8 (14.8) code would not be assigned. Such errors in 542 (164) 4.1 (0.1) 34.7 (10.5) <75 general result in underestimation of Sex VADEs. Third, there is a concern that a corFemale 2858 (406) 4.6 (0.1) 20.9 (2.9) rectly coded E-code may not always indiMale 1453 (317) 3.3 (0.1) 10.1 (2.2) cate an ADE. Fourth, there is little temporal Race information and clinical content in ICD-9White 3648 (503) 4.0 (0.6) 16.1 (2.2) CM codes. As a result, E-code-based ADE Black 525 (153) 5.0 (0.2) 15.0 (4.4) studies offer little insight into the harm, Other 139 (82) 3.2 (0.2) 7.6 (4.5) severity, preventability, and causes of the Region ADEs and provide little guidance in the Northeast 726 (205) 3.0 (0.1) 13.7 (3.9) Midwest 1028 (219) 4.4 (0.1) 16.1 (3.4) development of preventive strategies. South 1867 (409) 5.3 (0.1) 18.7 (4.1) Finally, some variation in VADEs is rooted West 690 (168) 2.8 (0.1) 10.9 (2.6) in patient tolerance of drugs, patient inclination in seeking care, and availability of * Standard error in parentheses. care, which may lead to biases in comparative analysis. It must also be noted that the estimates we provide are for visits for treating VADEs, serious enough to prompt a patient to seek ambulatory not ADEs. ADEs that were not treated at ambulatory care and are diagnosed and documented by a health care care settings were not counted. professional are considered an ADE. This approach to Consistent with previous findings,2–4,7–10 this study sugassessing ADEs offers some advantages. First, this definition eliminates arbitrariness in defining and detecting gests that ADEs are a significant safety concern in ADEs. In combination with the standardized methods for United States health care. With the increasing demand defining and collecting data in NAMCS and NHAMCS, for assessment data at the national level,1 the national this approach provides objective estimates and facilisurveys and E-codes appear to be a viable alternative to tates objective comparisons of ADE rates over time, provide national estimates of ADE prevalence. They across regions, and across care settings. Second, this offer a potential measure for the annual National approach relies on existing national representative surHealthcare Quality Report,29 for which no ADE measure veys and is therefore a cost-effective way to develop is currently available. J national estimates of ADEs. The authors acknowledge comments and suggestions made by Barry Friedman, Ph.D., Marge Keyes, M.S., and Joanna Jiang, Ph.D., Agency Several limitations should be noted. First, our estifor Healthcare Research and Quality, and Xu Wu, Ph.D., and Paul Hougland, Ph.D., Utah Department of Health. This article was a result mates can suffer from errors in the survey data. As indiof an intramural research project conducted at the Agency for cated in the methods section, the NAMCS and NHAMCS Healthcare Research and Quality (AHRQ). No external funding was used. The authors of this article are responsible for its content. capture adverse effects data using diagnosis and verbaNo statement in this article should be construed as an official tim text on external causes of injury. Although survey position of the AHRQ or the U.S. Department of Health and Human Services. coding errors are < 1%, it is possible that the survey
376
July 2005
Volume 31 Number 7
Copyright 2005 Joint Commission on Accreditation of Healthcare Organizations
Table 3. Medications Responsible for Adverse Drug Effects, Pooled 1995–2001 Data (%)* Drug Implicated (E-code) Antibiotics and other anti-infectives (E930-E931) Hormones and synthetic substitutes (E932) Primarily systemic agents (E933) Agents primarily affecting blood constituents (E934) Analgesics, antipyretics, and antirheumatics (E935) Anticonvulsants and anti-Parkinsonism drugs (E936) Sedatives and hypnotics (E937) Other central nervous system depressants, stimulants, nervous system agents (E938, E940, E941) Psychotropic agents (E939) Agents primarily affecting the cardiovascular system (E942) Agents primarily affecting the gastrointestinal system (E943) Water, mineral, and uric acid metabolism drugs (E944) Agents primarily acting on smooth/skeletal muscles and the respiratory system (E945) Agents primarily affecting skin/mucous membrance and ophthalmological, otorhinolarygological, and dental drugs (E946) Other and unspecified drugs and medical substances (E947) Total
Physician Office 11.0 12.7 1.9 1.3 5.7 2.1 0.6
Hospital OPD 16.8 16.0 7.9 3.2 6.3 2.2 0.6
Hospital ED 23.0 6.9 3.6 1.6 11.1 2.7 1.2
0.8
1.3
1.8
1.3
5.7 10.5 0.8 1.1
4.1 5.4 0.6 0.9
6.1 6.7 1.1 1.3
5.4 7.6 0.9 1.2
0.9
1.1
1.6
1.9
4.0
1.7
3.2
3.1
40.9 100.0
31.8 100.0
28.2 100.0
33.0 100.0
Total 18.5 10.7 3.9 1.9 7.9 2.3 0.9
* OPD, outpatient department; ED, emergency department.
Chunliu Zhan, M.D., Ph.D., is Staff Service Fellow, Intramural, National Healthcare Quality Report, Patient Safety, Agency for Healthcare Research and Quality (AHRQ), Department of Health and Human Services, Rockville, Maryland. Irma E. Arispe, Ph.D., is Associate Director for Science, Division of Health Care Statistics, National Center for Health Statistics (NCHS), Centers for Disease Control and Prevention. Edward Kelley, Ph.D., is Director, National Healthcare Quality Report, AHRQ. Tina Ding, M.S., is Research Analyst, Social Science and System, Inc., Silver Spring, Maryland. Catharine W. Burt, Ed.D., is Chief, Ambulatory Care Statistics Branch, Division of Health Care Statistics. Judith Shinogle, Ph.D., is NCHS/AcademyHealth Health Policy Fellow, Division of Health Care Statistics. Daniel Stryer, M.D. (deceased), was Director, Center for Quality Improvement and Patient Safety, AHRQ. Please address reprint requests to Chunliu Zhan, M.D., Ph.D., [email protected].
References 1. General Accounting Office (GAO): Adverse Drug Events: Substantial Problem but Magnitude Uncertain. GAO: Washington, D.C., 2000. 2. Classen D., et al.: Computerized surveillance of adverse drug events in hospital patients. JAMA 266:2847–2851, Nov. 27, 1991. 3. Bates D.W., et al.: Incidence of adverse drug events and potential adverse drug events: Implications for prevention. ADE Prevention Study Group. JAMA 274:29–34, Jul. 5, 1995. 4. Bates D.W., et al.: The costs of adverse drug events in hospitalized patients. Adverse Drug Events Prevention Study Group. JAMA 277:307–311, Jan. 22, 1997. 5. Raschetti R., et al.: Suspected adverse drug events requiring emergency department visits or hospital admissions. Eur J Clin Pharmacol 54:959–963, Feb. 1999. 6. Monette J., Gurwitz J.H., Avorn J.: Epidemiology of adverse drug events in the nursing home setting. Drugs Aging 7:203–211, Sep. 1995. 7. Gurwitz J.H., et al.: Incidence and preventability of adverse drug events in nursing homes. Am J Med 109:87–94, Aug. 1, 2000.
continued
July 2005
Volume 31 Number 7
Copyright 2005 Joint Commission on Accreditation of Healthcare Organizations
377
References, continued 8. Honigman B., et al.: Using computerized data to identify adverse drug events in outpatients. J Am Med Inform Assoc 8:254–266, May–Jun., 2001. 9. Gurwitz J.H., et al.: Incidence and preventability of adverse drug events among older persons in the ambulatory setting. JAMA 289:1107–1116, May 5, 2003. 10. Gandhi T.K., et al.: Adverse drug events in ambulatory care. N Engl J Med 348:1556–1564, Apr. 17, 2003. 11. Hicks R., Cousins D., Williams R.: Summary of Information Submitted to MEDMARX in the Year 2002. Rockville, MD: United States Pharmacopeia Center for the Advancement of Patient Safety, 2003. 12. Institute of Medicine: Patient Safety: Achieving a New Standard for Care. Washington, D.C.: National Academy Press; 2004. 13. Center for Drug Evaluation and Research, Food and Drug Administration: Annual Adverse Drug Experience Report: 1996. http://www.fda.gov/cder/dpe/annrep96/index.htm (last accessed May 11, 2005). 14. Rozich J., Haraden C., Resar R.: Adverse drug trigger tool: A practical methodology for measuring medication-related harm. Qual Saf Health Care 12:194–200, Jun 2003. 15. Honigman B., et al.: A computerized method for identifying incidences associated with adverse drug events in outpatients. Int J Med Inf 6:21–32, Apr. 2001. 16. Bates D.W., et al.: Asking residents about adverse events in a computer dialogue: How accurate are they? Jt Comm J Qual Improv 24:197–202, Apr. 1998. 17. U.S. Department of Health and Human Services (DHHS): International Classification of Diseases, Ninth Revision, Clinical Modification. Washington, D.C.: DHHS, 1997. 18. Iezzoni L.I., et al.: A method for screening the quality of hospital care using administrative data: preliminary validation results. QRB Qual Rev Bull 18:361–371, Nov. 1992.
378
July 2005
19. University of California at San Francisco, Stanford Evidence-Based Practice Center: Evidence Report for Measures of Patient Safety Based on Hospital Administrative Data: The Patient Safety Indicators. Rockville, MD: Agency for Healthcare Research and Quality, 2002. 20. Zhan C., Miller M.: Administrative data-based patient safety research: A critical review. Qual Saf Health Care 12(suppl II):ii58–ii63, Dec. 2003. 21. Utah Department of Health/Utah Hospitals and Health Systems Association: Utah Patient Safety Update, 1(3):1–7, 2003. 22. Xu W., Hougland P.: Using ICD-9-CM and ICD-10 data to improve patient safety in Utah. Paper presented at the AcademyHealth Annual Research Meeting, Nashville, TN, Jun. 28, 2003. 23. National Center for Health Statistics: Ambulatory Health Care Data. http://www.cdc.gov/nchs/about/major/ahcd/ahcd1.htm (last accessed May 11, 2005). 24. Aparasu R.R., Helgeland D.: Utilization of ambulatory care services caused by adverse effects of medications in the United States. Manag Care Interface 13:70–75, Apr. 2000. 25. Aparasu R.R., Helgeland D.L.: Visits to hospital outpatient departments in the United States due to adverse effects of medications. Hosp Pharm 35:825–831, Aug. 2000. 26. Aparasu R.R.: Visits to office-based physicians in the United States for medication-related morbidity. J Am Pharm Assoc 39:332–337, May–Jun. 1999. 27. Burt C.: Emergency health care encounters for adverse effects of medical treatment. Manag Care Interface 14:39–42, Dec. 2001. 28. Research Triangle Institute: SUDAAN 8.0.2. (computer software). Research Triangle Park, NC: Research Triangle Institute, 2003. 29. Institute of Medicine: Envisioning the National Health Care Quality Report. Washington, D.C.: National Academy Press; 2001.
Volume 31 Number 7
Copyright 2005 Joint Commission on Accreditation of Healthcare Organizations