- Email: [email protected]
Measuring quality in pediatric ophthalmology
Measuring Quality in Pediatric Ophthalmology Robert W. Enzenauer, MD, MPH Pediatric ophthalmologists should respond to quality expectations inherent in all disciplines of medicine. The specialists must be able to collect data to document and prove the quality of their work if it anticipates continued reimbursement for services. Quality of medical practice is assessed by outcomes research, which is different from clinical research. All specialists are systemically measuring individual and group outcomes across the spectrum of American health care systems. Pediatric ophthalmologists, both in private practice and at academic centers, must concentrate their research efforts on the assessment of cost, quality, utilization, and patient-centered health-related quality of life for the most common pathologies in pediatric ophthalmology practice. (J AAPOS 1999;3:283-8)
ediatric ophthalmologists must respond to current expectations in health care to produce outcomes data. The specialists must document and prove the quality of their work. There are 7 stages of coping in response to such an assertion: (1) Why measure quality? (2) What is outcomes research, anyway? (3) Whose idea was quality measurement, and how do you do it? (4) How has quality been measured in various specialties? (5) How has quality been measured in ophthalmology? (6) Where does pediatric ophthalmology stand? and (7) Where should pediatric ophthalmology go?
P
1. WHY MEASURE QUALITY? Quality of providers is no longer assessed by traditional documentation of licensure, training, or board certification. Health care organizations have begun to use results, or “outcomes,” to measure the quality of providers and institutions.1 Outcomes information makes accountability possible, and it allows those who pay for health care— employers, third-party insurers, and patients—to demand that accountability.1 Poor quality is expensive. Poor quality can result in readmission, reoperation, or re-treatment, whereas good quality can result in optimal outcome from the first patient encounter. Therefore, efforts to control cost have put
health care in the business of quality. These cost-containment strategies have driven the development of practice guidelines in an effort to standardize and optimize care. The effect of that care can then be measured in a standardized fashion.2 Physicians need to look at the interest in overall quality as an opportunity to determine the best way to take care of patients. Using these tools, providers can both prove and improve the quality of the care provided. Physician participation in the development of the guidelines and selection of outcomes to be measured will also ensure that the guidelines are realistic and relevant. Well-designed outcomes studies can debunk wellaccepted preconceptions such as the relative expense of medical teaching hospitals and subspecialty care.3-4 A recent article in the Journal of the American Medical Association documented that teaching hospital performance as assessed by 2 commonly used indicators, risk-adjusted mortality and length of stay, was better for major teaching hospitals than for minor teaching hospitals and nonteaching hospitals.3 Another study by Snow et al4 documented that pediatric urology subspecialists offer lower hospital costs, fewer preoperative studies, shorter operating room times, shorter hospital stays, and lower complication rates compared with general urologists.
2. WHAT IS OUTCOMES RESEARCH? From the Department of Ophthalmology, Chattanooga Unit, University of Tennessee College of Medicine, Chattanooga, Tennessee. Presented at the Outcomes Symposia at the 24th Annual Meeting of the American Association for Pediatric Ophthalmology and Strabismus, Palm Springs, California, April 7, 1998. The author has no financial interest in any product, device, software, other features. Submitted March 25, 1998. Revision accepted May 19, 1999. Reprint requests: Robert W. Enzenauer, MD, MPH, Associate Professor, Department of Opthalmology, UTCOM-Chattanooga Unit, 975 East Third St, Chattanooga, TN 37403 (e-mail: [email protected]). Copyright © 1999 by the American Association for Pediatric Ophthalmology and Strabismus. 1091-8531/99 $8.00 + 0 75/1/100975
Journal of AAPOS
Outcomes research has been defined as the assessment of the effect of a given product, procedure, or medical technology on health and/or cost outcomes.1 Although there is no universal definition of outcomes of care, it usually includes some assessment of cost, quality, utilization, and patient-centered health-related quality of life. Outcomes research expands the concept of quality beyond the traditional considerations of disease-specific morbidity and mortality. This expanded interpretation of quality can confuse the reader of quality literature.
October 1999
283
284 Enzenauer
Journal of AAPOS Volume 3 Number 5 October 1999
cal trials and population-based studies, and to perform multivariate analysis of potential risk factors that may influence outcomes.7-8
3. WHOSE IDEA WAS QUALITY MEASUREMENT, AND HOW IS IT DONE?
FIG 1. Data plotted in time sequence shows the elements of a control chart (ie, UCL, LCL, and process average) and shows how the data distribution is in a bell-shaped curve on the right side of the figure.
Cost outcomes are measured by analyzing total dollars per case, episode, procedure, or encounter or by calculating costs of care on the basis of each member per month. Quality outcomes are measured by analyzing selected mortality, morbidity, or preventive care indicators. Utilization outcomes can be measured by assessing length of stay and resource consumption or by measuring the frequency of admissions, procedures, or encounters per 1000 population. Patient-centered health-related quality of life outcomes can be measured by assessment of perceptual (patient satisfaction) and functional (quality of life) benefits from the patient’s perspective as the result of the health care intervention. Traditional clinical research is conducted to determine the safety and efficacy of a treatment under controlled study conditions, whereas outcomes research is concerned with the effectiveness and efficiency of the treatment under reallife conditions.5-6 Problems generated by our traditional clinical research include a multitude of pharmacologic agents all used for the same indication, with little evidence of their relative efficacy because of the tradition of testing against placebo. Insurers are often most interested in outcomes research data that are rare, the relative cost-effectiveness of different treatments, or individual providers, or different categories of providers. Outcomes research can be performed prospectively as stand-alone studies or as secondary objectives during controlled clinical trials that are designed for safety and efficacy.5 It can also be conducted retrospectively on clinical data drawn from previous trials or as a review of claims data.6 Outcomes studies do not necessarily require the rigid patient selection criteria and protocol-imposed testing found in traditional clinical trials, because they often reflect routine clinically practiced patterns.6 Outcomes research based on insurance claims databases offers new and unique opportunities to confirm clinical hypotheses regarding new outcomes of care, to detect potential advantages or disadvantages of emerging technologies of care, to study infrequent events that cannot be addressed by clini-
Walter A. Shewhart did groundbreaking work on “quality control” in the early 1920s with his student W. Edward Deming. Shewhart was the first to understand and measure the variability of a product’s dimensions and attributes, so that one could seek out the causes of variation beyond acceptable limits.9 Attempts to measure outcomes and monitor quality have at various times been termed quality assurance, quality improvement (QI), quality management, and finally, total quality management (TQM), as coined by the Department of Defense.9 The single tool most associated with Deming and his statistical process control approach is the control chart. A control chart is a graphic display of measurements of an important product, process variable, or outcome. The most important distinction is that it is a dynamic display that allows visualization of how data vary over time (Figure 1). The chart makes it easy to see whether measurements fall within the statistical defined upper control limits (UCLs) and lower control limits (LCLs) of such a distribution. The issue at the heart of quality control is understanding and controlling variability. QI only comes about when people use this information to identify the source of the variability and take action to remove that variability. Changes are then made so that variation is truly random and is kept within certain specified limits (UCL and LCL). Finally, one works to narrow those limits to reduce variability further and further. For this to occur, the “process” itself must be improved. This cycle of improvement is based on the Shewart cycle (Plan-Do-Act-Check) and has been adapted to the health care industry.10-12 The real importance of TQM is the psychologic benefit of being able to control one’s work and its outcomes.9 The use of TQM tools shows people that they can be causes, that they can control, and that they can determine how things turn out. In the traditional approach to quality assurance in medicine, action will be taken only when the process average exceeds a predetermined threshold. When the threshold is exceeded, there is finger-pointing and faultfinding. When the process average is below the threshold, there is complacency.13 In the QI approach, it is recognized that the entire output of the process provides the basis for action, and not just those occurrences that are deemed unacceptable because they exceeded a specific threshold. The application and acceptance of a QI perspective in the health care industry require a fundamental shift in the way we view the world in which we work.13
Journal of AAPOS Volume 3 Number 5 October 1999
4. HOW HAS QUALITY BEEN MEASURED IN MEDICINE? Medical and surgical quality was first addressed systematically by Dr Ernest A. Codman (1869-1940),14 a controversial surgeon and a founding member of the American College of Surgeons. In 1900, he proposed the “end-result system” in which a classification scheme for error and adverse outcomes was developed, and advocated that “. . . every hospital should follow every patient it treats long enough to determine whether or not the treatment has been successful, and then to inquire ‘If not, why not?’ with a view of preventing similar failures in the future.”15 Ninety years later, cost-containment strategies have driven physicians and health care organizations to evaluate more systematically the various outcomes of cost, quality, utilization, and patient-centered health-related quality of life. Comparisons of cost and the health outcomes at health maintenance organizations with those from previously traditional fee-for-service cares are controversial.16 Interest in better measuring quality and utilization has prompted the development of practice guidelines to make it easier to evaluate outcomes achievement in these areas. Practice guidelines are generally derived from consensus statements, scientific studies, and clinical experience.17 Practice guidelines should allow for a more standardized practice of medicine and reduce geographic variations in practice style.18 The American Academy of Pediatrics has launched the Ambulatory Care Quality Improvement Program to enable individual practitioners to improve office performance and demonstrate adherence to national guidelines and recommend health outcomes. Quality and utilization outcomes of treatment for common conditions such as hypertension, diabetes, and myocardial infarction are being systematically addressed in the Medical Outcomes Study.19 Like other facets of outcomes management, the measurement of patient-centered health-related quality of life is being developed.20-21 Health services research has led to the development of broader measures of health to determine if functioning can be improved by an intervention, using such functional statues measures as the Sickness Impact Profile, the Index of Activities of Daily Living, and the General Well-Being Schedule.22 No group of physicians was awakened to the full magnitude of outcomes assessment more than the cardiac surgical community in 1987. The Health Care Financing Administration, under the auspices of the US Department of Health and Human Services, published the Medicare Hospital Mortality Information. This publication gives consumers, patients, and payers a direct comparison of mortality statistics for various hospitals performing cardiac surgery. The report cards are here to stay.23 According to a recent study in 1994, closely rating hospitals can improve the quality of health care.24 The
Enzenauer
285
Missouri Department of Health released report cards rating the performance of all hospitals in the state that provided obstetric care. Borrowing from the state motto, the department entitled the reports “Show me: Buyer’s Guide to Obstetrical Services.”24
5. HOW HAS QUALITY BEEN MEASURED IN OPHTHALMOLOGY? Ophthalmologists have done reasonably well in developing practice guidelines, which is a logical first step toward a systematic examination of ophthalmology outcomes.25 The American Academy of Ophthalmology (AAO) has preferred practice guidelines for cataracts, glaucoma, bacterial keratitis, corneal opacification, and low to moderate refractive errors, in addition to 3 major pediatric ophthalmology subjects: amblyopia, esotropia, and the comprehensive pediatric eye evaluation. The Department of Health and Human Services has released guidelines for patients with cataracts that are consistent with the AAO’s preferred practice guidelines.26-28 Also, the National Eyecare Outcomes Network, from the AAO, has an outcomes database software for cataract surgery, glaucoma treatment, and diabetic retinopathy. Ophthalmologists have recently recognized the need for outcomes studies in terms other than visual acuity. Several groups of ophthalmologists are working to evaluate the quality of ophthalmology care in the new health care environment.29-31 An important visual function score (VF-14) has been proposed to better measure the benefit of cataract surgery.32-34 Additionally, the Activities of Daily Vision Scale and the Medical Outcomes Study 36item short form (SF-36) have been used to evaluate improved visual function and attenuation of declines in health-related quality of life after cataract extraction.35 Other ophthalmology outcomes studies have shown that different surgical technique and surgeon-patient characteristics do not affect clinical outcomes.36 The outcomes of cataract surgery by residents at a public county hospital and in a military setting have been determined.37-38 Various investigations have included the improvement in quality of life after cataract surgery in the second eye of patients with bilateral cataracts, the relationship between visual acuity and actual functioning and well-being among patients with diabetes, the impact of vitreoretinal surgery on bilateral visual function, the outcomes of trabeculectomy for primary open-angle glaucoma, and the relationship between patients’ preoperative expectations regarding the outcome of cataract surgery and actual postoperative experience.39-43
6. WHERE DOES PEDIATRIC OPHTHALMOLOGY STAND? As a group, pediatric ophthalmologists have been less successful in reporting outcomes for pediatric ophthalmologic
286 Enzenauer
Journal of AAPOS Volume 3 Number 5 October 1999
A
B
C
D FIG 2. Results of 3 fictitious strabismus surgeons. The histograms for each of the first 3 control charts are identical for each surgeon, showing how misleading the statistics of mean and SD can be. The histogram in D has one half the variability as those in A, B, and C.
Journal of AAPOS Volume 3 Number 5 October 1999
Enzenauer
287
procedures and treatments. Until recently, most published research concentrated on the postoperative strabismic angle, just as earlier ophthalmology studies concentrated on visual acuity.44 However, the value of strabismus surgery as reconstructive surgery is being increasingly addressed.45-47 It has been demonstrated that surgical realignment of longstanding strabismus in adults can result in binocular vision in 80% of adults and stereopsis in as many as 50% of cases.48-50 In addition to primary position alignment, improved binocular single vision can be expected in many cases after correction of long-standing strabismus.51,52 Improved psychosocial functioning has been shown after surgical correction of strabismus, both in children and adults.53-55 In addition, a recent study has supported the premise that resident strabismus surgery is as successful as attending surgery.56-57
tical histograms on the side of the control chart. However, when the same data are viewed over time on a control chart, 1 physician (Surgeon A) appears to be getting better (Figure 2, A), 1 physician (Surgeon B) appears to be getting worse (Figure 2, B), and 1 physician (Surgeon C) appears to have “randomly” variable results that are predictable within certain limits (Figure 2, C, before). Surgeon C changed something in his process of performing strabismus surgery, which brought about less random variation (as evidenced by the smaller difference between the UCL and the LCL) and less SD, as represented on the histogram (Figure 2, D, after). When control charts are used, it is up to the individuals involved to determine the sources of variability. Control charts could be equally applicable for tracking the results of amblyopia treatment, convergence training, pediatric cataract surgery, and operating room time.
7. WHERE SHOULD PEDIATRIC OPHTHALMOLOGY GO?
CONCLUSION
Future research must systematically investigate the outcomes of the care of common pediatric eye conditions. For example, researchers for a prospective horizontal strabismus study, similar to Snow’s pediatric urology study, could investigate cost (provider charges, facility charges, and operating room time), quality (postoperative complications, ultimate alignment, need for second operation), utilization (the number of preoperative and postoperative visits), and patient-centered health-related quality of life (incorporating some form of child health quality of life questionnaire).58 Such a study could help differentiate the care rendered by pediatric ophthalmology subspecialists compared with their colleagues in general ophthalmology. A challenge for ophthalmologists is to define what is core surgery for the general ophthalmologist and what cases should most appropriately be referred to those subspecialists trained to manage the more complicated cases. Any new studies should incorporate mechanisms to evaluate patient-centered health-related quality of life of pediatric eye care provided by the various eyecare providers. The evaluation of quality in pediatric ophthalmology is particularly suitable for the application of technologic tools for quality measurement in health care.59-61 Software is currently available to evaluate clinical outcomes data using statistical control charts.62,63 Control charts are appropriate for surgical procedures like strabismus repair. They can graphically present data over time more accurately than static tabular information such as mean and SD.64 Applications of statistical process control to pediatric ophthalmology are illustrated in Figure 2. The control charts in Figure 2 demonstrate how a dynamic display allows visual representation of variation over time. The 4 control charts in Figure 2 represent the results of 3 fictitious strabismus surgeons, each with “exactly” the same means and SDs after esotropia surgery on 50 patients (mean, about 4 PD esotropia; SD, about 2.5 PD), as represented by iden-
Pediatric ophthalmology can build on the QI foundation established by industry, by other medical specialties, and by ophthalmology in general as it seeks to prove and improve the quality of care it delivers. Managed care may have stimulated the interest in the process of QI, but it is up to physicians to maintain the focus of where it belongs, which is on the actual quality of the care provided.65,66 References 1. The Zitter Group. Outcomes report: how purchasers use outcomes and quality information to select providers. Nutley (NJ): Roche laboratories, 1994. 2. Schroeder SA. Outcomes assessment 70 years later: are we ready? N Engl J Med 1987;269:794-6. 3. Rosenthal GE, Harper DL, Quinn LM, Cooper GS. Severityadjusted mortality and length of stay in teaching and nonteaching hospital: results of a regional study. JAMA 1997;278:485-90. 4. Snow BW, Cartwright PC, Young MD. Does surgical subspecialization in pediatrics provide high-quality cost-effective patient care? Pediatrics 1996;97:14-7. 5. Bootman JL, Larson LN, McGhan WF, Townsend RJ. Pharmacoeconomic research and clinical trials: concepts and issues. DICP 1989;23:693-7. 6. Medd BH. Medical director’s page. Outcomes research: what is it? [Roche advertisement]. JAMA 1995;274:opposite page 365. 7. Javitt JC, Vitale S, Camer JK, Street DA, Krakauer H, McBean AM, et al. National outcomes of cataract extraction: endophthalmitis following inpatient surgery. Arch Ophthalmol 1991;109:1085-9. 8. Javitt JC. Outcomes of eye care from Medicare data [editorial]. Arch Ophthalmol 1991;109:1079-80. 9. Sashkin M, Kiser KJ. Putting total quality management to work. San Francisco: Berrett-Koehler Publishers, Inc; 1993. 10. Creech B. The five pillar of TQM: how to make total quality management work for you. New York: Penguin Books; 1994. 11. Walton M. The DEMING management method. New York: Berkley Publishing Group; 1986. 12. Walton M. Deming management at work. New York: Berkley Publishing Group; 1990. 13. Carey RG, Lloyd RC. Quality with confidence in healthcare. New York: Quality Resources; 1995. 14. Codman EA. The shoulder. Boston Thomas Todd Co; 1934. 15. Reverby S. Stealing the golden eggs: Ernest Amory Codman and the science and management of medicine. Bull Hist Med 1981;55:145-71.
288 Enzenauer 16. Ware JE Jr, Brook RH, Rogers WH, Keeler EB, Davies AR, Sherbourne CD, et al. Comparison of health outcomes at a health maintenance organization with those of fee-for-service care. Lancet 1986;1:1017-22. 17. Hayward RSA, Wilson MC, Tunis SR, Bass EB, Guyatt G. Users’ guides to the medical literature, VIII: how to use clinical practice guidelines, A: are the recommendations valid? JAMA 1995;274:570-4. 18. Wennberg JE: The paradox of appropriate care? JAMA 1987;258:2568-9. 19. Greenfield S, Rogers W, Mangotich M, Carney MF, Tarlov AR. Outcomes of patients with hypertension and non-insulin-dependent diabetes mellitus treated by difference systems and specialties. JAMA 1995;274:1436-44. 20. Rubin HR, Gandek B, Rogers WH, Kosinski M, McHorney CA, Ware JE Jr. Patients’ ratings of outpatient visits in difference practice settings: results from the Medical Outcomes Study. JAMA 1993;270:835-40. 21. Ellwood PM. Shattuck lecture—outcomes management: a technology of patient experience [special report]. N Engl J Med 1988;318:1549-56. 22. Salive MR, Mayfield JA, Weissman NW. Patients outcomes research teams and the agency for health care policy and research. Health Serv Res 1990;7:432-4. 23. O’Connor GT, Plume SK, Olmstead EM, Coffin LH, Morton JR, Maloney CT, et al. A regional prospective study of in-hospital mortality associated with coronary artery bypass grafting. The Northern New England Disease Study Group. JAMA 1991;266:2110-8. 24. Longo DR, Land G, Schramm W, Fraas J, Hoskins B, Howell V. Consumer reports in health care: do they make a difference in patient care? JAMA 1997;2781579-84. 25. Hoskins HD. AAO executive vice president address. The Annual Meeting of the American Academy of Ophthalmology; 1995 October; Atlanta, Georgia. San Francisco: The Academy; 1995. 26. O’Day DM. A new guideline for patients with cataract. Arch Ophthalmol 1993;111:317-8. 27. Foreman J. DHHS cataract guidelines released. Arch Ophthalmol 1993;111:591. 28. Lee PP. Guidelines: cataract surgery and beyond [editorial]. Arch Ophthalmol 1993;111:597-8. 29. Lee PP, Schachat AP. Evaluating quality of care in the new health care environment. Arch Ophthalmol 1995;113:149-52. 30. Lee PP, Javitt JC. Measuring the benefit and value of services. Arch Ophthalmol 1994;112. 31. Lichter PR. The relative value of quality care [editorial]. Ophthalmology 1991;98:1151-2. 32. Steinberg EP, Tielsch JM, Schein OD, Javitt JC, Sharkey P, Cassard SD, et al. National study of cataract surgery outcomes: variation in 4-month postoperative outcomes as reflected in multiple outcome measures. Ophthalmology 1994;101:1131-41. 33. Steinberg EP, Tielsch JM, Schein OD, Javitt JC, Sharkey P, Cassard SD, et al. The VF-14: an index of functional impairment in patients with cataract. Arch Ophthalmol 1994;112:630-8. 34. Cassard SD, Patrick DL, Damiano AM, Legro MW, Tielsch JM, Diener-West M, et al. Reproducibility and responsiveness of the VF14: an index of functional impairment in patients with cataracts. Arch Ophthalmol 1995;113:1508-13. 35. Mangione CM, Phillips RS, Lawrence MG, Seddom JM, Orav EJ, Goldman L. Improved visual function and attenuation of declines in health-related quality of life after cataract extraction. Arch Ophthalmol 1994;112:1419-25. 36. Schein OD, Steinberg EP, Javitt JC, Cassard SD, Tielsch JM, Steinwachs DM, et al. Variation in cataract surgery practice and clinical outcomes. Ophthalmology 1994;101:1142-52. 37. Smith JH, Seiff SR. Outcomes of cataract surgery by residents at a public county hospital. Am J Ophthalmol 1997;123:448-54. 38. Green RP, Peters DR. Visual results after cataract surgery by military resident surgeon. Ophthalmic Surg 1988;19:244-8. 39. Javitt JC, Steinberg EP, Sharkey P, Schein OD, Tielsch JM, Diener M, et al. Cataract surgery in one eye or both: a billion dollar per year issue. Ophthalmology 1995;102:1583-93.
Journal of AAPOS Volume 3 Number 5 October 1999 40. Lee PP, Whitcup SM, Hays RD, Spritzer K, Javitt J. The relationship between visual acuity and functioning and well-being among diabetics. Qual Life Res 1995;4:319-23. 41. Scott IU, Smiddy WE, Merikansky A, Feuer W. Vitreoretinal surgery outcomes: impact on bilateral visual function. Ophthalmology 1997;104:1041-8. 42. Nouri-Mahdavi K, Brigatti L, Weitzman M, Caprioli J. Outcomes of trabeculectomy for primary open-angle glaucoma. Ophthalmology 1995;102:1760-9. 43. Tielsch JM, Steinberg EP, Cassard SD, Schein OD, Javitt JC, Legro MW, et al. Preoperative functional expectations and postoperative outcomes among patients undergoing first eye cataract surgery. Arch Ophthalmol 1995;113:1312-8. 44. Kushner BJ, Fisher M. Is alignment within 8 prism diopters of orthotropia a successful outcome for infantile esotropia surgery? Arch Ophthalmol 1996;114:176-80. 45. Enzenauer RW. Strabismus repair is not “cosmetic” [letter]. J Pediatr Ophthalmol Strabismus 1994;31:67. 46. Metz HS. Strabismus surgery is reconstructive surgery [editorial]. J Pediatr Ophthalmol Strabismus 1988;25:263. 47. Helveston EM. The value of strabismus surgery. Ophthalmic Surg 1990;21:311-7. 48. Kushner BJ, Morton GV. Postoperative binocularity in adults with long-standing strabismus. Ophthalmology 1992;99:316-9. 49. Morris RJ, Scott WE, Dickey CF. Fusion after surgical alignment of long-standing strabismus in adults. Ophthalmology 1993;100:135-8. 50. Ball A, Drummond GT, Pearce WG. Unexpected stereoacuity following surgical correction of long-standing horizontal strabismus. Can J Ophthalmol 1993;28:217-20. 51. Wortham EV, Greenwald MJ. Expanded binocular peripheral visual fields following surgery for esotropia. J Pediatr Ophthalmol Strabismus 1989;26:109-12. 52. Kushner BJ. Binocular field expansion in adults after surgery for esotropia. Arch Ophthalmol 1994;112:639-43. 53. Satterfield D, Keltner JL, Morrison TL. Psychosocial aspects of strabismus surgery. Arch Ophthalmol 1993;111:1100-5. 54. Keltner JL. Strabismus surgery in adults: functional and psychosocial implications [editorial]. Arch Ophthalmol 1994:112:599-600. 55. Mruthyunjaya P, Simon JW, Pickering JD, Lininger LL. Subjective and objective outcomes in strabismus surgery in children. J Pediatr Ophthalmol Strabismus 1996;33:167-70. 56. Wisnicki HJ, Repka MX, Raab E, Hamad GG, Kirsch D, Nath A, et al. A comparison of the success rates of resident and attending strabismus surgery. J Pediatr Ophthalmol Strabismus 1993;30:118-21. 57. Crouch ER Jr. A comparison of the success rates of resident and attending strabismus surgery [discussion]. J Pediatr Ophthalmol Strabismus 1993:30:157-8. 58. Langraf JM, Abetz R, Ware JE Jr. Child health questionnaire (CHQ): a user’s manual. Boston: Health Institute, New England Medical Center; 1996. 59. Berwick DM. Toward an applied technology for quality measurement in health care. Med Decis Making 1988;8:253-8. 60. Brook R. Health care ratings [transcript]. NPR (National Public Radio) Morning Edition Report. November 19, 1997. 61. Greenfield S. The state of outcome research: are we on target? N Engl J Med 1989;320:1142-3. 62. Coleman RL. Statistical approaches to outcomes assessment. Best Pract Benchmarking Healthc 1996;1:242-9. 63. Kahn MG, Barley JC, Steib SA, Fraser VJ, Dunagan WC. Statistical process control methods for expert system performance monitoring. J Am Med Inform Assoc 1996;3:258-69. 64. Shahian DM, Williams WA, Svensson LG, Restuccia JD, D’Agostino RS. Applications of statistical quality control to cardiac surgery. Ann Thorac Surg 1996;62:1351-8, discussion 1358-9. 65. Cleary PD, Edgman-Livitan S. Health care quality: incorporating consumer perspectives. JAMA 1997;278:1608-12. 66. Brook RH. Managed care is not the problem, quality is [commentary]. JAMA 1997;278:1612-4.
ediatric ophthalmologists must respond to current expectations in health care to produce outcomes data. The specialists must document and prove the quality of their work. There are 7 stages of coping in response to such an assertion: (1) Why measure quality? (2) What is outcomes research, anyway? (3) Whose idea was quality measurement, and how do you do it? (4) How has quality been measured in various specialties? (5) How has quality been measured in ophthalmology? (6) Where does pediatric ophthalmology stand? and (7) Where should pediatric ophthalmology go?
P
1. WHY MEASURE QUALITY? Quality of providers is no longer assessed by traditional documentation of licensure, training, or board certification. Health care organizations have begun to use results, or “outcomes,” to measure the quality of providers and institutions.1 Outcomes information makes accountability possible, and it allows those who pay for health care— employers, third-party insurers, and patients—to demand that accountability.1 Poor quality is expensive. Poor quality can result in readmission, reoperation, or re-treatment, whereas good quality can result in optimal outcome from the first patient encounter. Therefore, efforts to control cost have put
health care in the business of quality. These cost-containment strategies have driven the development of practice guidelines in an effort to standardize and optimize care. The effect of that care can then be measured in a standardized fashion.2 Physicians need to look at the interest in overall quality as an opportunity to determine the best way to take care of patients. Using these tools, providers can both prove and improve the quality of the care provided. Physician participation in the development of the guidelines and selection of outcomes to be measured will also ensure that the guidelines are realistic and relevant. Well-designed outcomes studies can debunk wellaccepted preconceptions such as the relative expense of medical teaching hospitals and subspecialty care.3-4 A recent article in the Journal of the American Medical Association documented that teaching hospital performance as assessed by 2 commonly used indicators, risk-adjusted mortality and length of stay, was better for major teaching hospitals than for minor teaching hospitals and nonteaching hospitals.3 Another study by Snow et al4 documented that pediatric urology subspecialists offer lower hospital costs, fewer preoperative studies, shorter operating room times, shorter hospital stays, and lower complication rates compared with general urologists.
2. WHAT IS OUTCOMES RESEARCH? From the Department of Ophthalmology, Chattanooga Unit, University of Tennessee College of Medicine, Chattanooga, Tennessee. Presented at the Outcomes Symposia at the 24th Annual Meeting of the American Association for Pediatric Ophthalmology and Strabismus, Palm Springs, California, April 7, 1998. The author has no financial interest in any product, device, software, other features. Submitted March 25, 1998. Revision accepted May 19, 1999. Reprint requests: Robert W. Enzenauer, MD, MPH, Associate Professor, Department of Opthalmology, UTCOM-Chattanooga Unit, 975 East Third St, Chattanooga, TN 37403 (e-mail: [email protected]). Copyright © 1999 by the American Association for Pediatric Ophthalmology and Strabismus. 1091-8531/99 $8.00 + 0 75/1/100975
Journal of AAPOS
Outcomes research has been defined as the assessment of the effect of a given product, procedure, or medical technology on health and/or cost outcomes.1 Although there is no universal definition of outcomes of care, it usually includes some assessment of cost, quality, utilization, and patient-centered health-related quality of life. Outcomes research expands the concept of quality beyond the traditional considerations of disease-specific morbidity and mortality. This expanded interpretation of quality can confuse the reader of quality literature.
October 1999
283
284 Enzenauer
Journal of AAPOS Volume 3 Number 5 October 1999
cal trials and population-based studies, and to perform multivariate analysis of potential risk factors that may influence outcomes.7-8
3. WHOSE IDEA WAS QUALITY MEASUREMENT, AND HOW IS IT DONE?
FIG 1. Data plotted in time sequence shows the elements of a control chart (ie, UCL, LCL, and process average) and shows how the data distribution is in a bell-shaped curve on the right side of the figure.
Cost outcomes are measured by analyzing total dollars per case, episode, procedure, or encounter or by calculating costs of care on the basis of each member per month. Quality outcomes are measured by analyzing selected mortality, morbidity, or preventive care indicators. Utilization outcomes can be measured by assessing length of stay and resource consumption or by measuring the frequency of admissions, procedures, or encounters per 1000 population. Patient-centered health-related quality of life outcomes can be measured by assessment of perceptual (patient satisfaction) and functional (quality of life) benefits from the patient’s perspective as the result of the health care intervention. Traditional clinical research is conducted to determine the safety and efficacy of a treatment under controlled study conditions, whereas outcomes research is concerned with the effectiveness and efficiency of the treatment under reallife conditions.5-6 Problems generated by our traditional clinical research include a multitude of pharmacologic agents all used for the same indication, with little evidence of their relative efficacy because of the tradition of testing against placebo. Insurers are often most interested in outcomes research data that are rare, the relative cost-effectiveness of different treatments, or individual providers, or different categories of providers. Outcomes research can be performed prospectively as stand-alone studies or as secondary objectives during controlled clinical trials that are designed for safety and efficacy.5 It can also be conducted retrospectively on clinical data drawn from previous trials or as a review of claims data.6 Outcomes studies do not necessarily require the rigid patient selection criteria and protocol-imposed testing found in traditional clinical trials, because they often reflect routine clinically practiced patterns.6 Outcomes research based on insurance claims databases offers new and unique opportunities to confirm clinical hypotheses regarding new outcomes of care, to detect potential advantages or disadvantages of emerging technologies of care, to study infrequent events that cannot be addressed by clini-
Walter A. Shewhart did groundbreaking work on “quality control” in the early 1920s with his student W. Edward Deming. Shewhart was the first to understand and measure the variability of a product’s dimensions and attributes, so that one could seek out the causes of variation beyond acceptable limits.9 Attempts to measure outcomes and monitor quality have at various times been termed quality assurance, quality improvement (QI), quality management, and finally, total quality management (TQM), as coined by the Department of Defense.9 The single tool most associated with Deming and his statistical process control approach is the control chart. A control chart is a graphic display of measurements of an important product, process variable, or outcome. The most important distinction is that it is a dynamic display that allows visualization of how data vary over time (Figure 1). The chart makes it easy to see whether measurements fall within the statistical defined upper control limits (UCLs) and lower control limits (LCLs) of such a distribution. The issue at the heart of quality control is understanding and controlling variability. QI only comes about when people use this information to identify the source of the variability and take action to remove that variability. Changes are then made so that variation is truly random and is kept within certain specified limits (UCL and LCL). Finally, one works to narrow those limits to reduce variability further and further. For this to occur, the “process” itself must be improved. This cycle of improvement is based on the Shewart cycle (Plan-Do-Act-Check) and has been adapted to the health care industry.10-12 The real importance of TQM is the psychologic benefit of being able to control one’s work and its outcomes.9 The use of TQM tools shows people that they can be causes, that they can control, and that they can determine how things turn out. In the traditional approach to quality assurance in medicine, action will be taken only when the process average exceeds a predetermined threshold. When the threshold is exceeded, there is finger-pointing and faultfinding. When the process average is below the threshold, there is complacency.13 In the QI approach, it is recognized that the entire output of the process provides the basis for action, and not just those occurrences that are deemed unacceptable because they exceeded a specific threshold. The application and acceptance of a QI perspective in the health care industry require a fundamental shift in the way we view the world in which we work.13
Journal of AAPOS Volume 3 Number 5 October 1999
4. HOW HAS QUALITY BEEN MEASURED IN MEDICINE? Medical and surgical quality was first addressed systematically by Dr Ernest A. Codman (1869-1940),14 a controversial surgeon and a founding member of the American College of Surgeons. In 1900, he proposed the “end-result system” in which a classification scheme for error and adverse outcomes was developed, and advocated that “. . . every hospital should follow every patient it treats long enough to determine whether or not the treatment has been successful, and then to inquire ‘If not, why not?’ with a view of preventing similar failures in the future.”15 Ninety years later, cost-containment strategies have driven physicians and health care organizations to evaluate more systematically the various outcomes of cost, quality, utilization, and patient-centered health-related quality of life. Comparisons of cost and the health outcomes at health maintenance organizations with those from previously traditional fee-for-service cares are controversial.16 Interest in better measuring quality and utilization has prompted the development of practice guidelines to make it easier to evaluate outcomes achievement in these areas. Practice guidelines are generally derived from consensus statements, scientific studies, and clinical experience.17 Practice guidelines should allow for a more standardized practice of medicine and reduce geographic variations in practice style.18 The American Academy of Pediatrics has launched the Ambulatory Care Quality Improvement Program to enable individual practitioners to improve office performance and demonstrate adherence to national guidelines and recommend health outcomes. Quality and utilization outcomes of treatment for common conditions such as hypertension, diabetes, and myocardial infarction are being systematically addressed in the Medical Outcomes Study.19 Like other facets of outcomes management, the measurement of patient-centered health-related quality of life is being developed.20-21 Health services research has led to the development of broader measures of health to determine if functioning can be improved by an intervention, using such functional statues measures as the Sickness Impact Profile, the Index of Activities of Daily Living, and the General Well-Being Schedule.22 No group of physicians was awakened to the full magnitude of outcomes assessment more than the cardiac surgical community in 1987. The Health Care Financing Administration, under the auspices of the US Department of Health and Human Services, published the Medicare Hospital Mortality Information. This publication gives consumers, patients, and payers a direct comparison of mortality statistics for various hospitals performing cardiac surgery. The report cards are here to stay.23 According to a recent study in 1994, closely rating hospitals can improve the quality of health care.24 The
Enzenauer
285
Missouri Department of Health released report cards rating the performance of all hospitals in the state that provided obstetric care. Borrowing from the state motto, the department entitled the reports “Show me: Buyer’s Guide to Obstetrical Services.”24
5. HOW HAS QUALITY BEEN MEASURED IN OPHTHALMOLOGY? Ophthalmologists have done reasonably well in developing practice guidelines, which is a logical first step toward a systematic examination of ophthalmology outcomes.25 The American Academy of Ophthalmology (AAO) has preferred practice guidelines for cataracts, glaucoma, bacterial keratitis, corneal opacification, and low to moderate refractive errors, in addition to 3 major pediatric ophthalmology subjects: amblyopia, esotropia, and the comprehensive pediatric eye evaluation. The Department of Health and Human Services has released guidelines for patients with cataracts that are consistent with the AAO’s preferred practice guidelines.26-28 Also, the National Eyecare Outcomes Network, from the AAO, has an outcomes database software for cataract surgery, glaucoma treatment, and diabetic retinopathy. Ophthalmologists have recently recognized the need for outcomes studies in terms other than visual acuity. Several groups of ophthalmologists are working to evaluate the quality of ophthalmology care in the new health care environment.29-31 An important visual function score (VF-14) has been proposed to better measure the benefit of cataract surgery.32-34 Additionally, the Activities of Daily Vision Scale and the Medical Outcomes Study 36item short form (SF-36) have been used to evaluate improved visual function and attenuation of declines in health-related quality of life after cataract extraction.35 Other ophthalmology outcomes studies have shown that different surgical technique and surgeon-patient characteristics do not affect clinical outcomes.36 The outcomes of cataract surgery by residents at a public county hospital and in a military setting have been determined.37-38 Various investigations have included the improvement in quality of life after cataract surgery in the second eye of patients with bilateral cataracts, the relationship between visual acuity and actual functioning and well-being among patients with diabetes, the impact of vitreoretinal surgery on bilateral visual function, the outcomes of trabeculectomy for primary open-angle glaucoma, and the relationship between patients’ preoperative expectations regarding the outcome of cataract surgery and actual postoperative experience.39-43
6. WHERE DOES PEDIATRIC OPHTHALMOLOGY STAND? As a group, pediatric ophthalmologists have been less successful in reporting outcomes for pediatric ophthalmologic
286 Enzenauer
Journal of AAPOS Volume 3 Number 5 October 1999
A
B
C
D FIG 2. Results of 3 fictitious strabismus surgeons. The histograms for each of the first 3 control charts are identical for each surgeon, showing how misleading the statistics of mean and SD can be. The histogram in D has one half the variability as those in A, B, and C.
Journal of AAPOS Volume 3 Number 5 October 1999
Enzenauer
287
procedures and treatments. Until recently, most published research concentrated on the postoperative strabismic angle, just as earlier ophthalmology studies concentrated on visual acuity.44 However, the value of strabismus surgery as reconstructive surgery is being increasingly addressed.45-47 It has been demonstrated that surgical realignment of longstanding strabismus in adults can result in binocular vision in 80% of adults and stereopsis in as many as 50% of cases.48-50 In addition to primary position alignment, improved binocular single vision can be expected in many cases after correction of long-standing strabismus.51,52 Improved psychosocial functioning has been shown after surgical correction of strabismus, both in children and adults.53-55 In addition, a recent study has supported the premise that resident strabismus surgery is as successful as attending surgery.56-57
tical histograms on the side of the control chart. However, when the same data are viewed over time on a control chart, 1 physician (Surgeon A) appears to be getting better (Figure 2, A), 1 physician (Surgeon B) appears to be getting worse (Figure 2, B), and 1 physician (Surgeon C) appears to have “randomly” variable results that are predictable within certain limits (Figure 2, C, before). Surgeon C changed something in his process of performing strabismus surgery, which brought about less random variation (as evidenced by the smaller difference between the UCL and the LCL) and less SD, as represented on the histogram (Figure 2, D, after). When control charts are used, it is up to the individuals involved to determine the sources of variability. Control charts could be equally applicable for tracking the results of amblyopia treatment, convergence training, pediatric cataract surgery, and operating room time.
7. WHERE SHOULD PEDIATRIC OPHTHALMOLOGY GO?
CONCLUSION
Future research must systematically investigate the outcomes of the care of common pediatric eye conditions. For example, researchers for a prospective horizontal strabismus study, similar to Snow’s pediatric urology study, could investigate cost (provider charges, facility charges, and operating room time), quality (postoperative complications, ultimate alignment, need for second operation), utilization (the number of preoperative and postoperative visits), and patient-centered health-related quality of life (incorporating some form of child health quality of life questionnaire).58 Such a study could help differentiate the care rendered by pediatric ophthalmology subspecialists compared with their colleagues in general ophthalmology. A challenge for ophthalmologists is to define what is core surgery for the general ophthalmologist and what cases should most appropriately be referred to those subspecialists trained to manage the more complicated cases. Any new studies should incorporate mechanisms to evaluate patient-centered health-related quality of life of pediatric eye care provided by the various eyecare providers. The evaluation of quality in pediatric ophthalmology is particularly suitable for the application of technologic tools for quality measurement in health care.59-61 Software is currently available to evaluate clinical outcomes data using statistical control charts.62,63 Control charts are appropriate for surgical procedures like strabismus repair. They can graphically present data over time more accurately than static tabular information such as mean and SD.64 Applications of statistical process control to pediatric ophthalmology are illustrated in Figure 2. The control charts in Figure 2 demonstrate how a dynamic display allows visual representation of variation over time. The 4 control charts in Figure 2 represent the results of 3 fictitious strabismus surgeons, each with “exactly” the same means and SDs after esotropia surgery on 50 patients (mean, about 4 PD esotropia; SD, about 2.5 PD), as represented by iden-
Pediatric ophthalmology can build on the QI foundation established by industry, by other medical specialties, and by ophthalmology in general as it seeks to prove and improve the quality of care it delivers. Managed care may have stimulated the interest in the process of QI, but it is up to physicians to maintain the focus of where it belongs, which is on the actual quality of the care provided.65,66 References 1. The Zitter Group. Outcomes report: how purchasers use outcomes and quality information to select providers. Nutley (NJ): Roche laboratories, 1994. 2. Schroeder SA. Outcomes assessment 70 years later: are we ready? N Engl J Med 1987;269:794-6. 3. Rosenthal GE, Harper DL, Quinn LM, Cooper GS. Severityadjusted mortality and length of stay in teaching and nonteaching hospital: results of a regional study. JAMA 1997;278:485-90. 4. Snow BW, Cartwright PC, Young MD. Does surgical subspecialization in pediatrics provide high-quality cost-effective patient care? Pediatrics 1996;97:14-7. 5. Bootman JL, Larson LN, McGhan WF, Townsend RJ. Pharmacoeconomic research and clinical trials: concepts and issues. DICP 1989;23:693-7. 6. Medd BH. Medical director’s page. Outcomes research: what is it? [Roche advertisement]. JAMA 1995;274:opposite page 365. 7. Javitt JC, Vitale S, Camer JK, Street DA, Krakauer H, McBean AM, et al. National outcomes of cataract extraction: endophthalmitis following inpatient surgery. Arch Ophthalmol 1991;109:1085-9. 8. Javitt JC. Outcomes of eye care from Medicare data [editorial]. Arch Ophthalmol 1991;109:1079-80. 9. Sashkin M, Kiser KJ. Putting total quality management to work. San Francisco: Berrett-Koehler Publishers, Inc; 1993. 10. Creech B. The five pillar of TQM: how to make total quality management work for you. New York: Penguin Books; 1994. 11. Walton M. The DEMING management method. New York: Berkley Publishing Group; 1986. 12. Walton M. Deming management at work. New York: Berkley Publishing Group; 1990. 13. Carey RG, Lloyd RC. Quality with confidence in healthcare. New York: Quality Resources; 1995. 14. Codman EA. The shoulder. Boston Thomas Todd Co; 1934. 15. Reverby S. Stealing the golden eggs: Ernest Amory Codman and the science and management of medicine. Bull Hist Med 1981;55:145-71.
288 Enzenauer 16. Ware JE Jr, Brook RH, Rogers WH, Keeler EB, Davies AR, Sherbourne CD, et al. Comparison of health outcomes at a health maintenance organization with those of fee-for-service care. Lancet 1986;1:1017-22. 17. Hayward RSA, Wilson MC, Tunis SR, Bass EB, Guyatt G. Users’ guides to the medical literature, VIII: how to use clinical practice guidelines, A: are the recommendations valid? JAMA 1995;274:570-4. 18. Wennberg JE: The paradox of appropriate care? JAMA 1987;258:2568-9. 19. Greenfield S, Rogers W, Mangotich M, Carney MF, Tarlov AR. Outcomes of patients with hypertension and non-insulin-dependent diabetes mellitus treated by difference systems and specialties. JAMA 1995;274:1436-44. 20. Rubin HR, Gandek B, Rogers WH, Kosinski M, McHorney CA, Ware JE Jr. Patients’ ratings of outpatient visits in difference practice settings: results from the Medical Outcomes Study. JAMA 1993;270:835-40. 21. Ellwood PM. Shattuck lecture—outcomes management: a technology of patient experience [special report]. N Engl J Med 1988;318:1549-56. 22. Salive MR, Mayfield JA, Weissman NW. Patients outcomes research teams and the agency for health care policy and research. Health Serv Res 1990;7:432-4. 23. O’Connor GT, Plume SK, Olmstead EM, Coffin LH, Morton JR, Maloney CT, et al. A regional prospective study of in-hospital mortality associated with coronary artery bypass grafting. The Northern New England Disease Study Group. JAMA 1991;266:2110-8. 24. Longo DR, Land G, Schramm W, Fraas J, Hoskins B, Howell V. Consumer reports in health care: do they make a difference in patient care? JAMA 1997;2781579-84. 25. Hoskins HD. AAO executive vice president address. The Annual Meeting of the American Academy of Ophthalmology; 1995 October; Atlanta, Georgia. San Francisco: The Academy; 1995. 26. O’Day DM. A new guideline for patients with cataract. Arch Ophthalmol 1993;111:317-8. 27. Foreman J. DHHS cataract guidelines released. Arch Ophthalmol 1993;111:591. 28. Lee PP. Guidelines: cataract surgery and beyond [editorial]. Arch Ophthalmol 1993;111:597-8. 29. Lee PP, Schachat AP. Evaluating quality of care in the new health care environment. Arch Ophthalmol 1995;113:149-52. 30. Lee PP, Javitt JC. Measuring the benefit and value of services. Arch Ophthalmol 1994;112. 31. Lichter PR. The relative value of quality care [editorial]. Ophthalmology 1991;98:1151-2. 32. Steinberg EP, Tielsch JM, Schein OD, Javitt JC, Sharkey P, Cassard SD, et al. National study of cataract surgery outcomes: variation in 4-month postoperative outcomes as reflected in multiple outcome measures. Ophthalmology 1994;101:1131-41. 33. Steinberg EP, Tielsch JM, Schein OD, Javitt JC, Sharkey P, Cassard SD, et al. The VF-14: an index of functional impairment in patients with cataract. Arch Ophthalmol 1994;112:630-8. 34. Cassard SD, Patrick DL, Damiano AM, Legro MW, Tielsch JM, Diener-West M, et al. Reproducibility and responsiveness of the VF14: an index of functional impairment in patients with cataracts. Arch Ophthalmol 1995;113:1508-13. 35. Mangione CM, Phillips RS, Lawrence MG, Seddom JM, Orav EJ, Goldman L. Improved visual function and attenuation of declines in health-related quality of life after cataract extraction. Arch Ophthalmol 1994;112:1419-25. 36. Schein OD, Steinberg EP, Javitt JC, Cassard SD, Tielsch JM, Steinwachs DM, et al. Variation in cataract surgery practice and clinical outcomes. Ophthalmology 1994;101:1142-52. 37. Smith JH, Seiff SR. Outcomes of cataract surgery by residents at a public county hospital. Am J Ophthalmol 1997;123:448-54. 38. Green RP, Peters DR. Visual results after cataract surgery by military resident surgeon. Ophthalmic Surg 1988;19:244-8. 39. Javitt JC, Steinberg EP, Sharkey P, Schein OD, Tielsch JM, Diener M, et al. Cataract surgery in one eye or both: a billion dollar per year issue. Ophthalmology 1995;102:1583-93.
Journal of AAPOS Volume 3 Number 5 October 1999 40. Lee PP, Whitcup SM, Hays RD, Spritzer K, Javitt J. The relationship between visual acuity and functioning and well-being among diabetics. Qual Life Res 1995;4:319-23. 41. Scott IU, Smiddy WE, Merikansky A, Feuer W. Vitreoretinal surgery outcomes: impact on bilateral visual function. Ophthalmology 1997;104:1041-8. 42. Nouri-Mahdavi K, Brigatti L, Weitzman M, Caprioli J. Outcomes of trabeculectomy for primary open-angle glaucoma. Ophthalmology 1995;102:1760-9. 43. Tielsch JM, Steinberg EP, Cassard SD, Schein OD, Javitt JC, Legro MW, et al. Preoperative functional expectations and postoperative outcomes among patients undergoing first eye cataract surgery. Arch Ophthalmol 1995;113:1312-8. 44. Kushner BJ, Fisher M. Is alignment within 8 prism diopters of orthotropia a successful outcome for infantile esotropia surgery? Arch Ophthalmol 1996;114:176-80. 45. Enzenauer RW. Strabismus repair is not “cosmetic” [letter]. J Pediatr Ophthalmol Strabismus 1994;31:67. 46. Metz HS. Strabismus surgery is reconstructive surgery [editorial]. J Pediatr Ophthalmol Strabismus 1988;25:263. 47. Helveston EM. The value of strabismus surgery. Ophthalmic Surg 1990;21:311-7. 48. Kushner BJ, Morton GV. Postoperative binocularity in adults with long-standing strabismus. Ophthalmology 1992;99:316-9. 49. Morris RJ, Scott WE, Dickey CF. Fusion after surgical alignment of long-standing strabismus in adults. Ophthalmology 1993;100:135-8. 50. Ball A, Drummond GT, Pearce WG. Unexpected stereoacuity following surgical correction of long-standing horizontal strabismus. Can J Ophthalmol 1993;28:217-20. 51. Wortham EV, Greenwald MJ. Expanded binocular peripheral visual fields following surgery for esotropia. J Pediatr Ophthalmol Strabismus 1989;26:109-12. 52. Kushner BJ. Binocular field expansion in adults after surgery for esotropia. Arch Ophthalmol 1994;112:639-43. 53. Satterfield D, Keltner JL, Morrison TL. Psychosocial aspects of strabismus surgery. Arch Ophthalmol 1993;111:1100-5. 54. Keltner JL. Strabismus surgery in adults: functional and psychosocial implications [editorial]. Arch Ophthalmol 1994:112:599-600. 55. Mruthyunjaya P, Simon JW, Pickering JD, Lininger LL. Subjective and objective outcomes in strabismus surgery in children. J Pediatr Ophthalmol Strabismus 1996;33:167-70. 56. Wisnicki HJ, Repka MX, Raab E, Hamad GG, Kirsch D, Nath A, et al. A comparison of the success rates of resident and attending strabismus surgery. J Pediatr Ophthalmol Strabismus 1993;30:118-21. 57. Crouch ER Jr. A comparison of the success rates of resident and attending strabismus surgery [discussion]. J Pediatr Ophthalmol Strabismus 1993:30:157-8. 58. Langraf JM, Abetz R, Ware JE Jr. Child health questionnaire (CHQ): a user’s manual. Boston: Health Institute, New England Medical Center; 1996. 59. Berwick DM. Toward an applied technology for quality measurement in health care. Med Decis Making 1988;8:253-8. 60. Brook R. Health care ratings [transcript]. NPR (National Public Radio) Morning Edition Report. November 19, 1997. 61. Greenfield S. The state of outcome research: are we on target? N Engl J Med 1989;320:1142-3. 62. Coleman RL. Statistical approaches to outcomes assessment. Best Pract Benchmarking Healthc 1996;1:242-9. 63. Kahn MG, Barley JC, Steib SA, Fraser VJ, Dunagan WC. Statistical process control methods for expert system performance monitoring. J Am Med Inform Assoc 1996;3:258-69. 64. Shahian DM, Williams WA, Svensson LG, Restuccia JD, D’Agostino RS. Applications of statistical quality control to cardiac surgery. Ann Thorac Surg 1996;62:1351-8, discussion 1358-9. 65. Cleary PD, Edgman-Livitan S. Health care quality: incorporating consumer perspectives. JAMA 1997;278:1608-12. 66. Brook RH. Managed care is not the problem, quality is [commentary]. JAMA 1997;278:1612-4.