- Email: [email protected]
Adverse drug events and near misses: who’s counting?
Adverse Drug Events and Near Misses: Who’s Counting? Albert W. Wu, MD, MPH
I
t is no secret that drugs can harm as well as benefit a patient (1–3). In the Harvard Medical Practice Study, an intensive review of over 30,000 hospitalizations in New York State in the mid 1980s, drugs were the leading cause of medical injury, accounting for about one fifth of morbidity and mortality (4). Two terms describing these harms are in common use: the older of the two, adverse drug reaction (ADR), is defined as “any response to a drug which is noxious, unintended, and which occurs at doses normally used in man for the prophylaxis, diagnosis, or therapy of disease” (5) and includes only the appropriate use of drugs. The second, adverse drug event (ADE), can be defined as injury resulting from the use of a drug and includes medical errors. These definitions serve many purposes, including as outcomes in studies of medications, indicators for surveillance of drug complications, and sentinels in quality assurance. The Joint Commission on Accreditation of Healthcare Organizations (6) requires that hospitals have written procedures for recording and reporting ADRs. The Food and Drug Administration maintains that a concurrent surveillance system, driven by spontaneous, voluntary reporting of events by physicians, nurses, and pharmacists, is the best method of documenting ADRs. However, reporting has always been problematic (7). Estimates of the incidence of adverse drug reactions in hospitals have ranged widely, depending on the definition of events and the rigor of pursuit. A study in Boston hospitals suggested an incidence of 6.5 ADEs per 100 hospital admissions, which can be extrapolated to 1,900 ADEs per hospital per year, 1% of which are fatal (8). Although the precise incidence rates are arguable, it is widely recognized that older patients are highly vulnerable to ADEs (9 –11). Contributors include greater physiologic susceptibility, less functional reserve, and problems with recall; those with chronic illness may be under the care of multiple physicians and have more than one pharmacy filling many prescriptions, increasing the likelihood of inappropriate prescribing and drug-to-drug interactions (12). These problems are magnified by the disruptions that accompany acute illness, hospitalization, and return to the community (13–15). Am J Med. 2000;109:166 –168. From the Departments of Health Policy and Management, and Epidemiology, School of Hygiene and Public Health, and the Department of Medicine, School of Medicine, The Johns Hopkins University, Baltimore, Maryland. Requests for reprints should be addressed to Albert W. Wu, MD, MPH, Health Services Research and Development Center, 624 North Broadway, Baltimore, Maryland 21205. 166
䉷2000 by Excerpta Medica, Inc. All rights reserved.
This issue of the Journal contains two articles that shed light on the epidemiology of ADEs and point out problems in reporting. In the first, Chyka (16) demonstrates a marked disparity between two national estimates of the number of deaths from ADRs. Tabulations of death certificates suggested 206 deaths in 1995 resulting from ADRs, whereas data from the Food and Drug Administration’s postmarketing surveillance system (MEDWatch) suggested 6,894 such fatalities. Which of these figures is correct? Probably neither, because both estimates are susceptible to underreporting by health professionals, who contributed only one of eight cases to MEDWatch. The author notes important disincentives to submitting reports, including fear of litigation, inadequate time to report, and lack of knowledge about how to report; he identifies this as a major problem with the current spontaneous reporting system. Gurwitz et al (17) studied ADEs and near misses (medication errors that did not result in adverse outcomes) in residents of 18 nursing homes in Massachusetts. Although others have broached the subject (18,19), this is the first report to provide true incidence data in a nursing home population. The team applied their own rigorous and previously tested methods for detection, including a structured review process and reports from nursing staff. The most important findings were that ADEs were common, and half were preventable, including the majority of severe outcomes. Scaling up their results to all US nursing homes, they estimate that approximately 24 ADEs and 8 potential ADEs should be identifiable each year in an average facility. This includes approximately 1,500 fatal and 18,000 life-threatening ADEs per year, 80% of which could be prevented. Perhaps equally important for this population, nearly 25,000 ADEs are disabling, with over 5,000 resulting in permanent disability. The surprise was how few ADEs they actually found, amounting to less than one ADE per resident-year. Examination of the pattern of ADEs leads to the suspicion of undercounting, because the errors found most frequently tended to be those most convenient to measure. Most events occurred at the stages of drug ordering (wrong dose or established drug– drug interaction) or monitoring (failing to monitor drug levels, or to act on abnormal findings). Psychoactive medications and anticoagulants were most commonly implicated. Errors less likely to be documented in the medical record, such as errors in administration, omissions, or wrong choice of drug, were also found less frequently. It is also surprising that the study tallied fewer near 0002-9343/00/$–see front matter PII S0002-9343(00)00509-X
Adverse Drug Events and Near Misses: Who’s Counting/Wu
misses than actual ADEs. Common sense suggests that because most errors do not result in bad outcomes, there should be more near misses than completed ADEs. The fact that nearly 80% of the recorded near misses involved anticoagulants suggests that this was simply because they were easier to detect (INR ⱖ 4) than others in the universe of possible errors. Reporting by nursing staff failed to fill this gap. Despite encouragement, only 17% of ADEs and only 4% (seven total) of near misses were identified by this mechanism. Nurses are not always aware of medication errors, but many of the errors they see are unlikely to be caught by other mechanisms. It is increasingly clear that underreporting of ADEs is a major problem (20). To reduce ADEs and iatrogenic injury, an important goal is to increase reporting of ADEs and medical errors. Individual institutions need better epidemiologic data on their own ADEs and near misses to provide grist for quality-improvement mills and improved management. Aggregated data are needed to make evident patterns that would otherwise not be apparent. None of the existing methods for identifying ADEs is sufficient on its own. Chart review is effective for research but is too costly for routine clinical use and still misses many errors (21). Computerized event monitoring that searches for adverse events that may have occurred offers a promising and efficient way of obtaining this information (22,23). Self-reports by physicians, nurses, and pharmacists are incomplete but can provide unique information. In particular, near misses may be known only to the physician or nurse (24). How can we encourage voluntary reporting? Every physician and nurse has been party to a medication error or ADE, but most have never reported one. One important barrier is that most are unaware of the procedures to do so (25). MEDWatch (26), established in 1993, is not a familiar term to physicians. The United States Pharmacopeia has developed MedMARx, a nationwide, anonymous, internet-accessible service for hospitals to report preventable ADEs (27), but there is little awareness of its existence. A second barrier is reluctance to report ADEs and mistakes. How do you motivate professionals to do something that is at best inconvenient and at worst painful and self-incriminating? In the current environment, even the best-intentioned physician may be reluctant to be in the vanguard of error reporting. What is needed is an environment where providers can feel safe and motivated to report mistakes (28). The recent report from the US Institute of Medicine (29) stresses that health-care organizations must create an environment in which patient safety is a top priority. To do so, organizations must be transformed to see medical systems and processes as the critical sources of most error, not individuals. An approach that emphasizes individual culpability is incompatible with one that promotes safety. Pressures for un-
restricted public reporting on individual providers about medical errors are in direct conflict with these goals (30). The solution requires promotion of voluntary disclosure of mistakes without penalty. What can be done to make nursing homes safer? Like other health-care organizations, nursing homes need to design systems geared to preventing, detecting, and minimizing the likelihood of error. Efforts will be needed at both the system and human levels. There are many system-level interventions that have been effective at reducing error rates in other industries and that could be applied (31). Enhanced surveillance and reporting systems for ADEs are essential. Computerized order-entry systems could be a crucial component of safer drug ordering and longer-term monitoring. Nursing homes will also need to determine whether nurse and physician staffing are adequate to allow for adequate monitoring of patients, reporting of ADEs, and application of automated systems. Until ADEs and drug errors are reported and tracked down routinely and then used to improve care, harm from drugs will continue often to outweigh the good.
REFERENCES 1. Cluff LE, Seidl LG, Smith JW. Studies on the epidemiology of adverse drug reactions. Ann Intern Med. 1966;65:629 – 633. 2. Tally RB, Laventurier MF. Drug-induced illness. JAMA. 1974:229: 1043–1048. 3. Jick H. Drugs—remarkably nontoxic. NEJM. 1974:291:824 – 828. 4. Leape LL, Brennan TA, Laird N, et al. The nature of adverse events in hospitalized patients: results from the Harvard Medical Practice Study. NEJM. 1991;324:377–384. 5. World Health Organization. International drug monitoring. The role of the hospital. WHO Tech Rep Ser 1969;426:5–24. 6. Accreditation manual for hospitals. Chicago: Joint Commission on Accreditation of Healthcare Organizations, 1991. 7. Karch FE, Lasagna L. Adverse drug reactions: a critical review. JAMA. 1975:234:1236 –1241. 8. Bates DW, Cullen DJ, Laird N, et al. Incidence of adverse drug event and potential adverse drug events. Implications for prevention. JAMA. 1995;274:29 –34. 9. Williamson J, Chopin JM. Adverse reactions to prescribed drugs in the elderly: a multicentre investigation. Age Ageing 1980;9:73– 80. 10. Einarson TR. Drug-related hospital admissions. Ann Pharmacother. 1993;27:832– 840. 11. Hanlon JT, Schmader KE, Koronkowski MJ, et al. Adverse drug events in high risk older outpatients. J Am Geriatr Soc. 1997;45:945– 948. 12. Col N, Fanale JE, Kronholm P. The role of medication noncompliance and adverse drug reactions in hospitalizations of the elderly. Arch Intern Med. 1990;150:841– 845. 13. Willcox SM, Himmelstein DU, Woolhandler S. Inappropriate drug prescribing for the community-dwelling elderly. JAMA. 1994; 272: 292–296. 14. Gray SL, Mahoney JE, Blough DK. Adverse drug events in elderly patients receiving home health services following hospital discharge. Ann Pharmacother. 1999;33:1147–1153. 15. Beers MH, Slikowski J, Brooks J. Compliance with medication orders among the elderly after hospital discharge. Hosp Formul. 1992; 27:720 –724.
August 1, 2000
THE AMERICAN JOURNAL OF MEDICINE威
Volume 109 167
Adverse Drug Events and Near Misses: Who’s Counting/Wu 16. Chyka PA. How many deaths occur annually from adverse drug reactions in the United States? Am J Med. 2000;109:122–130. 17. Gurwitz JH, Field TS, Avorn J, et al. Am J Med. 2000;109:87–94. 18. Gerety MB, Cornell JE, Plichta DT, Eimer M. Adverse events related to drugs and drug withdrawal in nursing home residents. J Am Geriatr Soc. 1993;41:1326 –1332. 19. Monette J, Gurwitz JH, Avorn J. Epidemiology of adverse drug events in the nursing home setting. Drugs Aging. 1995;7:203–211. 20. US General Accounting Office. Adverse drug events. The magnitude of health risk is uncertain because of limited incidence data. Washington, DC: US General Accounting Office, 2000. Publication GAO/HEHS-00-21. 21. Bates DW, Gawande AA. Error in medicine: what have we learned? Ann Intern Med. 2000;132:763–767. 22. Classen DC, Pestotnik SL, Evans RS, Burke JP. Computerized surveillance of adverse drug events in hospital patients. JAMA. 1991; 266:2847–2851. 23. Jha AK, Kuperman GJ, Teich JM, et al. Identifying adverse drug events: development of a computer-based monitor and comparison with chart review and stimulated voluntary report. J Am Med Inform Assoc 1998;5:305–314.
168
August 1, 2000
THE AMERICAN JOURNAL OF MEDICINE威
24. van den Bemt PM, Egberts AC, Lenderink AW, et al. Adverse drug events in hospitalized patients. A comparison of doctors, nurses and patients as sources of reports. Eur J Clin Pharmacol. 1999;55: 155–158. 25. Figueiras A, Tato F, Fontainas J, Gestal-Otero JJ. Influence of physicians’ attitudes on reporting adverse drug events: a case control study. Med Care. 1999;37:809 – 814. 26. Kessler DA. Introducing MEDWatch: a new approach to reporting medication and device adverse effects and product problems. JAMA 1993;269:2765–2768. 27. Cousins DD. Developing a uniform reporting system for preventable adverse drug events. Clin Ther. 1998;20(suppl C):C45–C58. 28. Steinhauer J. New York lawmakers vote to post doctors’ missteps on the internet. New York Times, June 24, 2000. 29. Wu AW. Handling hospital errors: is disclosure the best defense? Ann Intern Med. 1999;131:970 –972. 30. Corrigan J, Kohn LT, Donaldson MS, eds. To Err Is Human: Building a Safer Health System. Washington, DC: National Academy Press, 1999. 31. Leape LL, Kabcenell AI, Berwick DM. Reducing Adverse Drug Events. Boston: Institute for Healthcare Improvement; 1998.
Volume 109
I
t is no secret that drugs can harm as well as benefit a patient (1–3). In the Harvard Medical Practice Study, an intensive review of over 30,000 hospitalizations in New York State in the mid 1980s, drugs were the leading cause of medical injury, accounting for about one fifth of morbidity and mortality (4). Two terms describing these harms are in common use: the older of the two, adverse drug reaction (ADR), is defined as “any response to a drug which is noxious, unintended, and which occurs at doses normally used in man for the prophylaxis, diagnosis, or therapy of disease” (5) and includes only the appropriate use of drugs. The second, adverse drug event (ADE), can be defined as injury resulting from the use of a drug and includes medical errors. These definitions serve many purposes, including as outcomes in studies of medications, indicators for surveillance of drug complications, and sentinels in quality assurance. The Joint Commission on Accreditation of Healthcare Organizations (6) requires that hospitals have written procedures for recording and reporting ADRs. The Food and Drug Administration maintains that a concurrent surveillance system, driven by spontaneous, voluntary reporting of events by physicians, nurses, and pharmacists, is the best method of documenting ADRs. However, reporting has always been problematic (7). Estimates of the incidence of adverse drug reactions in hospitals have ranged widely, depending on the definition of events and the rigor of pursuit. A study in Boston hospitals suggested an incidence of 6.5 ADEs per 100 hospital admissions, which can be extrapolated to 1,900 ADEs per hospital per year, 1% of which are fatal (8). Although the precise incidence rates are arguable, it is widely recognized that older patients are highly vulnerable to ADEs (9 –11). Contributors include greater physiologic susceptibility, less functional reserve, and problems with recall; those with chronic illness may be under the care of multiple physicians and have more than one pharmacy filling many prescriptions, increasing the likelihood of inappropriate prescribing and drug-to-drug interactions (12). These problems are magnified by the disruptions that accompany acute illness, hospitalization, and return to the community (13–15). Am J Med. 2000;109:166 –168. From the Departments of Health Policy and Management, and Epidemiology, School of Hygiene and Public Health, and the Department of Medicine, School of Medicine, The Johns Hopkins University, Baltimore, Maryland. Requests for reprints should be addressed to Albert W. Wu, MD, MPH, Health Services Research and Development Center, 624 North Broadway, Baltimore, Maryland 21205. 166
䉷2000 by Excerpta Medica, Inc. All rights reserved.
This issue of the Journal contains two articles that shed light on the epidemiology of ADEs and point out problems in reporting. In the first, Chyka (16) demonstrates a marked disparity between two national estimates of the number of deaths from ADRs. Tabulations of death certificates suggested 206 deaths in 1995 resulting from ADRs, whereas data from the Food and Drug Administration’s postmarketing surveillance system (MEDWatch) suggested 6,894 such fatalities. Which of these figures is correct? Probably neither, because both estimates are susceptible to underreporting by health professionals, who contributed only one of eight cases to MEDWatch. The author notes important disincentives to submitting reports, including fear of litigation, inadequate time to report, and lack of knowledge about how to report; he identifies this as a major problem with the current spontaneous reporting system. Gurwitz et al (17) studied ADEs and near misses (medication errors that did not result in adverse outcomes) in residents of 18 nursing homes in Massachusetts. Although others have broached the subject (18,19), this is the first report to provide true incidence data in a nursing home population. The team applied their own rigorous and previously tested methods for detection, including a structured review process and reports from nursing staff. The most important findings were that ADEs were common, and half were preventable, including the majority of severe outcomes. Scaling up their results to all US nursing homes, they estimate that approximately 24 ADEs and 8 potential ADEs should be identifiable each year in an average facility. This includes approximately 1,500 fatal and 18,000 life-threatening ADEs per year, 80% of which could be prevented. Perhaps equally important for this population, nearly 25,000 ADEs are disabling, with over 5,000 resulting in permanent disability. The surprise was how few ADEs they actually found, amounting to less than one ADE per resident-year. Examination of the pattern of ADEs leads to the suspicion of undercounting, because the errors found most frequently tended to be those most convenient to measure. Most events occurred at the stages of drug ordering (wrong dose or established drug– drug interaction) or monitoring (failing to monitor drug levels, or to act on abnormal findings). Psychoactive medications and anticoagulants were most commonly implicated. Errors less likely to be documented in the medical record, such as errors in administration, omissions, or wrong choice of drug, were also found less frequently. It is also surprising that the study tallied fewer near 0002-9343/00/$–see front matter PII S0002-9343(00)00509-X
Adverse Drug Events and Near Misses: Who’s Counting/Wu
misses than actual ADEs. Common sense suggests that because most errors do not result in bad outcomes, there should be more near misses than completed ADEs. The fact that nearly 80% of the recorded near misses involved anticoagulants suggests that this was simply because they were easier to detect (INR ⱖ 4) than others in the universe of possible errors. Reporting by nursing staff failed to fill this gap. Despite encouragement, only 17% of ADEs and only 4% (seven total) of near misses were identified by this mechanism. Nurses are not always aware of medication errors, but many of the errors they see are unlikely to be caught by other mechanisms. It is increasingly clear that underreporting of ADEs is a major problem (20). To reduce ADEs and iatrogenic injury, an important goal is to increase reporting of ADEs and medical errors. Individual institutions need better epidemiologic data on their own ADEs and near misses to provide grist for quality-improvement mills and improved management. Aggregated data are needed to make evident patterns that would otherwise not be apparent. None of the existing methods for identifying ADEs is sufficient on its own. Chart review is effective for research but is too costly for routine clinical use and still misses many errors (21). Computerized event monitoring that searches for adverse events that may have occurred offers a promising and efficient way of obtaining this information (22,23). Self-reports by physicians, nurses, and pharmacists are incomplete but can provide unique information. In particular, near misses may be known only to the physician or nurse (24). How can we encourage voluntary reporting? Every physician and nurse has been party to a medication error or ADE, but most have never reported one. One important barrier is that most are unaware of the procedures to do so (25). MEDWatch (26), established in 1993, is not a familiar term to physicians. The United States Pharmacopeia has developed MedMARx, a nationwide, anonymous, internet-accessible service for hospitals to report preventable ADEs (27), but there is little awareness of its existence. A second barrier is reluctance to report ADEs and mistakes. How do you motivate professionals to do something that is at best inconvenient and at worst painful and self-incriminating? In the current environment, even the best-intentioned physician may be reluctant to be in the vanguard of error reporting. What is needed is an environment where providers can feel safe and motivated to report mistakes (28). The recent report from the US Institute of Medicine (29) stresses that health-care organizations must create an environment in which patient safety is a top priority. To do so, organizations must be transformed to see medical systems and processes as the critical sources of most error, not individuals. An approach that emphasizes individual culpability is incompatible with one that promotes safety. Pressures for un-
restricted public reporting on individual providers about medical errors are in direct conflict with these goals (30). The solution requires promotion of voluntary disclosure of mistakes without penalty. What can be done to make nursing homes safer? Like other health-care organizations, nursing homes need to design systems geared to preventing, detecting, and minimizing the likelihood of error. Efforts will be needed at both the system and human levels. There are many system-level interventions that have been effective at reducing error rates in other industries and that could be applied (31). Enhanced surveillance and reporting systems for ADEs are essential. Computerized order-entry systems could be a crucial component of safer drug ordering and longer-term monitoring. Nursing homes will also need to determine whether nurse and physician staffing are adequate to allow for adequate monitoring of patients, reporting of ADEs, and application of automated systems. Until ADEs and drug errors are reported and tracked down routinely and then used to improve care, harm from drugs will continue often to outweigh the good.
REFERENCES 1. Cluff LE, Seidl LG, Smith JW. Studies on the epidemiology of adverse drug reactions. Ann Intern Med. 1966;65:629 – 633. 2. Tally RB, Laventurier MF. Drug-induced illness. JAMA. 1974:229: 1043–1048. 3. Jick H. Drugs—remarkably nontoxic. NEJM. 1974:291:824 – 828. 4. Leape LL, Brennan TA, Laird N, et al. The nature of adverse events in hospitalized patients: results from the Harvard Medical Practice Study. NEJM. 1991;324:377–384. 5. World Health Organization. International drug monitoring. The role of the hospital. WHO Tech Rep Ser 1969;426:5–24. 6. Accreditation manual for hospitals. Chicago: Joint Commission on Accreditation of Healthcare Organizations, 1991. 7. Karch FE, Lasagna L. Adverse drug reactions: a critical review. JAMA. 1975:234:1236 –1241. 8. Bates DW, Cullen DJ, Laird N, et al. Incidence of adverse drug event and potential adverse drug events. Implications for prevention. JAMA. 1995;274:29 –34. 9. Williamson J, Chopin JM. Adverse reactions to prescribed drugs in the elderly: a multicentre investigation. Age Ageing 1980;9:73– 80. 10. Einarson TR. Drug-related hospital admissions. Ann Pharmacother. 1993;27:832– 840. 11. Hanlon JT, Schmader KE, Koronkowski MJ, et al. Adverse drug events in high risk older outpatients. J Am Geriatr Soc. 1997;45:945– 948. 12. Col N, Fanale JE, Kronholm P. The role of medication noncompliance and adverse drug reactions in hospitalizations of the elderly. Arch Intern Med. 1990;150:841– 845. 13. Willcox SM, Himmelstein DU, Woolhandler S. Inappropriate drug prescribing for the community-dwelling elderly. JAMA. 1994; 272: 292–296. 14. Gray SL, Mahoney JE, Blough DK. Adverse drug events in elderly patients receiving home health services following hospital discharge. Ann Pharmacother. 1999;33:1147–1153. 15. Beers MH, Slikowski J, Brooks J. Compliance with medication orders among the elderly after hospital discharge. Hosp Formul. 1992; 27:720 –724.
August 1, 2000
THE AMERICAN JOURNAL OF MEDICINE威
Volume 109 167
Adverse Drug Events and Near Misses: Who’s Counting/Wu 16. Chyka PA. How many deaths occur annually from adverse drug reactions in the United States? Am J Med. 2000;109:122–130. 17. Gurwitz JH, Field TS, Avorn J, et al. Am J Med. 2000;109:87–94. 18. Gerety MB, Cornell JE, Plichta DT, Eimer M. Adverse events related to drugs and drug withdrawal in nursing home residents. J Am Geriatr Soc. 1993;41:1326 –1332. 19. Monette J, Gurwitz JH, Avorn J. Epidemiology of adverse drug events in the nursing home setting. Drugs Aging. 1995;7:203–211. 20. US General Accounting Office. Adverse drug events. The magnitude of health risk is uncertain because of limited incidence data. Washington, DC: US General Accounting Office, 2000. Publication GAO/HEHS-00-21. 21. Bates DW, Gawande AA. Error in medicine: what have we learned? Ann Intern Med. 2000;132:763–767. 22. Classen DC, Pestotnik SL, Evans RS, Burke JP. Computerized surveillance of adverse drug events in hospital patients. JAMA. 1991; 266:2847–2851. 23. Jha AK, Kuperman GJ, Teich JM, et al. Identifying adverse drug events: development of a computer-based monitor and comparison with chart review and stimulated voluntary report. J Am Med Inform Assoc 1998;5:305–314.
168
August 1, 2000
THE AMERICAN JOURNAL OF MEDICINE威
24. van den Bemt PM, Egberts AC, Lenderink AW, et al. Adverse drug events in hospitalized patients. A comparison of doctors, nurses and patients as sources of reports. Eur J Clin Pharmacol. 1999;55: 155–158. 25. Figueiras A, Tato F, Fontainas J, Gestal-Otero JJ. Influence of physicians’ attitudes on reporting adverse drug events: a case control study. Med Care. 1999;37:809 – 814. 26. Kessler DA. Introducing MEDWatch: a new approach to reporting medication and device adverse effects and product problems. JAMA 1993;269:2765–2768. 27. Cousins DD. Developing a uniform reporting system for preventable adverse drug events. Clin Ther. 1998;20(suppl C):C45–C58. 28. Steinhauer J. New York lawmakers vote to post doctors’ missteps on the internet. New York Times, June 24, 2000. 29. Wu AW. Handling hospital errors: is disclosure the best defense? Ann Intern Med. 1999;131:970 –972. 30. Corrigan J, Kohn LT, Donaldson MS, eds. To Err Is Human: Building a Safer Health System. Washington, DC: National Academy Press, 1999. 31. Leape LL, Kabcenell AI, Berwick DM. Reducing Adverse Drug Events. Boston: Institute for Healthcare Improvement; 1998.
Volume 109