- Email: [email protected]
Excellence in endoscopy: toward practical metrics
REVIEW ARTICLE
Excellence in endoscopy: toward practical metrics Peter B. Cotton, MD, FRCP, FRCS, Robert H. Hawes, MD, Alan Barkun, MD, Gregory G. Ginsberg, MD, Stephan Amman, MD, Jonathan Cohen, MD, Jeffrey Ponsky, MD, Douglas K. Rex, MD, Drew Schembre, MD, C. Mel Wilcox, MD
There is increasing interest and literature on improving and measuring the quality of endoscopy performance. We present the results of a workshop designed to define quality metrics that can be used now. Collection of structured data on a voluntary basis will stimulate improvement in endoscopic training, practice, and credentialing. Patients and practitioners will benefit. It is the right thing to do. Over the last 4 decades, digestive endoscopy has evolved from an occasional diversion for a few enthusiasts into a fundamental tool of gastroenterology. Endoscopy has become enormously popular throughout the world because of its tremendous and proven value in diagnosis and treatment. The problem is that the benefits are maximized only when procedures are performed at an optimal level of quality, which is not always the case. Technical failures and serious complications can occur in the best of hands but are more likely when procedures are performed by endoscopists with inadequate training and experience. Practitioners, patients, and payers should all be interested in enhancing the quality of endoscopic services. The professional organizations associated with endoscopy and their leaders have increasingly embraced the quality improvement paradigm that is advancing through medicine, as it has through manufacturing industry.1-4 The American Society for Gastrointestinal Endoscopy (ASGE), and the American College of Gastroenterology particularly have produced helpful reports and guidelines.5,6 In Britain, the Joint Advisory Group that represents all of the interested parties7 has been active in this area, as have other organizations around the world. The problem is that these thoughtful conclusions and well-meaning documents have had little impact so far in the real world. Quality is discussed but is not measured to any extent. Seldom do hospital privileging bodies follow published guidelines for credentialing, which include the use of proctored evaluation in some circumstances.8 We need to agree on what constitutes ‘‘excellence’’ and then develop appropriate metrics and systems to collect
the data. This will bring objectivity to training and will stimulate improvements in practice. Patients will benefit. These considerations led to the development of a study meeting entitled ‘‘Excellence in Endoscopy,’’ which was held in Albuquerque, New Mexico in October 2004.
Measuring and improving the quality of endoscopic performance will undoubtedly benefit patients and practitioners. It is the right thing to do.
METHODS Twenty leading North American gastroenterologists and selected guests were invited to consider 3 related topics. 1. What are the metrics of excellence in endoscopy? 2. How can the data be collected, analyzed, and used? 3. How can excellence be trained and enhanced? There were 3 team leaders, Robert Hawes, Alan Barkun, and Gregory Ginsberg. Each selected a team of 6 to 8 participants about 6 months before the meeting to focus on specific aspects of their allocated topic. They were tasked to review the relevant literature, circulate drafts, and then to present their recommendations in plenary session at the meeting, which lasted 2 days. After the workshop, the team submitted updated recommendations for inclusion in the final document. This report covers the first section of the meeting, i.e., developing the metrics of endoscopic excellence. Reports of the other sections will follow. The meeting was supported by Olympus America Inc., Medical Systems Group, Melville, NY.
What is endoscopic excellence?
Copyright ª 2006 by the American Society for Gastrointestinal Endoscopy 0016-5107/$32.00 doi:10.1016/j.gie.2005.04.048
We cannot start to measure excellence unless we can recognize it and define it. Therefore, what constitutes an excellent endoscopic experience? Society expects that diagnostic and therapeutic procedures will be appropriate (indicated) and that they will be
286 GASTROINTESTINAL ENDOSCOPY Volume 63, No. 2 : 2006
www.giejournal.org
Cotton et al
Practical endoscopic metrics
performed expeditiously, skillfully, successfully, safely, and comfortably.9 These expectations can be expanded to make a list of desirable characteristics for all types of endoscopic procedures. These include cognitive and technical skills of endoscopists but also appropriate behavior, environment, and staff. Elements of good endoscopy performance include the following: d Correct indications, adherence to published guidelines d Appropriate environment, support team, and behavior d Well prepared and informed patients d Strategies to minimize risk, including patient preparation and monitoring d Appropriate use of medications, including sedation/ analgesia when used d Correct selection of equipment d Comfortable intubation d Complete survey of the target organ(s) d Recognition of all abnormalities (and photographic documentation) d Appropriate tissue sampling (adherence to published guidelines) d Application of indicated therapy d Avoiding, recognizing, and managing complications d Reasonable duration d Smooth recovery, explanation, and discharge d Detailed and clear recommendations and follow-up plans d Integrated pathology results and communications d Complete documentation (and billing) d Positive feedback from patients Some of these elements apply to all procedures, whereas others (e.g., indications, precautions, protocol guidelines, therapies, outcomes) are specific to the individual type (e.g., upper endoscopy, colonoscopy, ERCP, EUS, capsule). Many articles have explored these quality issues10-12 and have provided similar lists of elements that seem to be important. The ASGE has published quality indicators specifically for upper endoscopy, colonoscopy, ERCP, and EUS.3,13,14 By far, the most comprehensive report to date comes from the prestigious U.S. multisociety task force on ‘‘Quality in technical performance of colonoscopy, and the continuous quality improvement process for colonoscopy.’’15 This document discussed the key issues (and questions needing more research) in 7 topic areas: indications and intervals (guideline adherence), precautions, insertion, withdrawal, biopsy and polypectomy, complications, and pathology interface. For each of these areas, they recommended up to 6 quality-improvement targets. Other groups and organizations have discussed quality issues in more general terms and in the context of credentialing. Thus, the ASGE has published guidelines for credentialing physicians and granting privileges in upper endoscopy, ERCP, and EUS.16-21 The recent report from the National Institutes of Health ‘‘state of the science’’
conference on ERCP made many comments on quality issues.22 These documents speak about adequate training, numbers of procedures, and desirable qualities in endoscopists but include no objective measures of performance, for the obvious reason that there has been no agreement for measures. This is the main rationale for this workshop and discussion.
www.giejournal.org
Volume 63, No. 2 : 2006 GASTROINTESTINAL ENDOSCOPY 287
How to recognize, predict, and measure excellence? There are some factors that make a good experience more likely. Good endoscopic training and extensive experience do not guarantee quality practice, but they certainly make it more likely. Thus, documentation of these and related elements should be a part of any assessment of endoscopists. Appropriate metrics could include the following: d Specialty training and certification (place and dates) d Training and maintenance of competence in life support and sedation d Evidence for continuing education in endoscopy d For each group of procedures, lifetime numbers, total in the last year, and spectrum of practice The proof of quality comes from documentation of performance. There is no substitute for collecting relevant data.23 The ASGE has recommended that endoscopists should collect data prospectively on their endoscopic practice and performance.3 This translates into ‘‘endoscopy report cards.’’24
Report cards for endoscopy The report card is simply an extension of the log books used by trainees to record their performance. It cannot possibly include all of the data elements that have been listed above and in various well-meaning publications. Items should be selected based on ease of data collection and by assumed relative importance. Some items are easily recorded and appear in most reports already (e.g., procedure indication, anatomical extent, duration, diagnosis, immediate unplanned events). Other items are more subjective (e.g., lesion interpretation) or more difficult to record (e.g., delayed complications, endoscopist-specific patient satisfaction). Some items would appear to be more important markers of quality than others. For ERCP, selective cannulation performance and complication rates are obvious key parameters. For colonoscopy, cecal intubation rates and minimum withdrawal times appear to be particularly relevant.15 The U.S. multisociety task force on colonoscopy suggested 25 quality improvement targets (each with goals).15 While such a comprehensive assessment would be ideal, it will be easier to start the process with simpler tools. Once the system is established, the parameters will be refined and expanded. Our ‘‘starter recommendations’’ are given in Appendix 1.
Practical endoscopic metrics
Why should we bother and how? It is likely that patients and payers will increasingly seek help from those with documented expertise and will avoid others. Keeping a report card will provide a competitive advantage. It also is a crucial tool in the evolving use of ‘‘pay for performance’’ in the United States. Those driving that agenda need professional help in determining meaningful quality measures. How to collect, analyze, and use these data was the topic for the second section of our workshop and will be reported separately. The increasing use of electronic endoscopy reporting systems will make this process easy, even automatic.
Endoscopy proctoring, examinations, and diplomas Knowledge of an endoscopist’s training, experience, and some report-card data can provide important data on the quality of his or her practice. But this may not be sufficient, especially when there may be questions about those elements that are more difficult to document precisely, such as the extent of knowledge, skill in lesion recognition, judgment in difficult treatment decisions, and teamwork. We may need additional information from proctoring, as recommended by ASGE.18,19 Another possibility is a formal examination. This would be resource intensive and not without controversy, but examinations are the recognized method for assuring a reasonable level of knowledge and performance in many other fields and usually lead to a diploma. It initially could be piloted for a lower volume, higher-risk procedure such as ERCP. The diploma would be based on data from report cards, an examination of core knowledge, observation of a few cases, and, possibly, some work on simulators. This exercise would require agreement on how to ‘‘score’’ the more subjective elements of a procedure. Unless and until the official agencies (such as the American Board of Internal Medicine [ABIM]) embrace this concept, the ERCP diploma would have a significant impact only if a substantial number of practitioners volunteer to participate.
Cotton et al
specific for endoscopy units in the United States, and the agencies that accredit them do not have comprehensive guidelines. Hitherto, inspections have focused on the existence (and compliance with) guidelines for sedation and infection control. These are important, but there are other elements.
Metrics of quality in endoscopy units It is not difficult to list features of endoscopy units that may impact the quality of the procedures being performed in them. 1. Facilities and equipment (and maintenance) 2. Volumes and range of procedures performed 3. Method for credentialing doctors 4. Staffing levels, with appropriate training and certification 5. Leadership and cohesion (retention) 6. Documented policies and compliance systems d Sedation and monitoring d Cleaning and disinfection d Methods for risk minimization 7. Regularly collected safety data d Infection rates, disinfection surveillance d Sedation issues, intubations, prolonged recovery d Accidents d Unplanned admissions 8. Communications d No-show rates d Pathology follow-up system d Surveillance returns (patients with polyps, stents) d Referring physicians 9. Patient feedback d Complaint management d Formal satisfaction surveys An ‘‘endoscopy unit report card’’ could be developed by picking a selection of these criteria, as shown in the Appendix. As part of the endoscopy modernization process in Britain, a ‘‘global rating scale’’ was completed by most of the endoscopy units.25 This was a qualitative self-rating survey dealing mainly with processes instead of numbers and yielded important comparative information. It will be updated.
The excellence of endoscopy units Patients are concerned about their safety, comfort, and dignity during endoscopy, and the efficiency of the process. While endoscopists have a responsibility for these elements and can influence the way the rest of the team functions, there are important quality elements of the endoscopy unit and the staff that can be considered separately. Endoscopists (however talented) cannot work without good facilities, equipment, and a well-trained and motivated staff. While most health care facilities pay some attention to ‘‘quality improvement,’’ the extent to which this percolates down to the endoscopy unit is variable. There is no national quality improvement program 288 GASTROINTESTINAL ENDOSCOPY Volume 63, No. 2 : 2006
What performance level is good enough: benchmarking Endoscopists and units vary in their levels of performance. Not all patients can be managed by the superexperts. The issue then is who decides what constitutes acceptable performance and what that should be. The U.S. multisociety task force for colonoscopy suggested goals (e.g., 95% cecal intubation rates in screening examinations, and 100% compliance with coagulation issues). Other professional bodies have also made similar recommendations (e.g., 90% selective biliary cannulation rate in www.giejournal.org
Cotton et al
ERCP). But, we currently have no idea how much variation really exists in the broad endoscopic community. Aggregated data on cecal intubation rates from acknowledged experts in prospective studies15 paint a much more favorable picture than those from more general audits of broader community practice.26,27 The true picture can be documented only if a large and representative sample of endoscopists and units share and compare data from similar report cards. Benchmarking is the practice of mapping the performance of an individual (endoscopist or unit) against that from a broad spectrum of ‘‘competitors.’’ There are many practical and political issues involved in attempting to aggregate and analyze such data, not least because, by definition, 50% of the doctors and units will be shown to be performing below the mean. These issues were the subject of the second part of the workshop and will be reported separately. Ultimately, informed consumers should be able to judge what is good enough for them. Increasing publicity about variations in performance (e.g., in colonoscopy completion rates), if linked to specific report card data, will drive this agenda. Practitioners with poor outcomes or those who choose not to collect data will be disadvantaged.
Practical endoscopic metrics
DISCLOSURE The workshop was supported by an educational grant from Olympus America Inc.
REFERENCES
We recommend that all endoscopists who are comfortable with their performance should begin to collect data for individual report cards, by using simple data points as shown in Appendix 1. The effort need not be overambitious or burdensome. Let us get something started. The fact that some endoscopists will be reluctant to document and advertise their poor performance should not stop us from doing the right things, right? We should wear our data plainly and proudly as badges of quality.
1. Berwick DM. Quality comes home. Ann Intern Med 1996;125:839-43. 2. Chassin MR. Improving the quality of care. N Engl J Med 1996;335: 1060-3. 3. Quality and outcomes assessment in gastrointestinal endoscopy. American Society for Gastrointestinal Endoscopy. Gastrointest Endosc 2000;52:827-30. 4. Johanson JF. Continuous quality improvement in the ambulatory endoscopy center. Gastrointest Endosc Clin N Am 2002;12:351-65. 5. American Society for Gastrointestinal Endoscopy. ASGE Web site. Available from URL: http://www.asge.org. 6. American College of Gastroenterology. ACG Web site. Available from URL: http://www.acg.gi.org. 7. Joint Advisory Group on Gastrointestinal Endoscopy. JAG Web site. Available from URL: http://www.thejag.org.uk. 8. ASGE guidelines for clinical application. Proctoring for hospital endoscopy privileges. American Society for Gastrointestinal Endoscopy. Gastrointest Endosc 1999;50:901-5. 9. Johanson JF, Overholt BF, Frakes JT. Characteristics of best gastroenterology practices. Am J Gastroenterol 1999;94:2519-30. 10. Minoli G, Meucci G, Prada A, Terruzzi V, Bortoli A, Gullotta R, et al. Quality assurance and colonoscopy. Endoscopy 1999;31:522-7. 11. Chak A, Cooper GS. Procedure-specific outcomes assessment for endoscopic ultrasonography. Gastrointest Endosc Clin N Am 1999;9: 649-56. 12. Clinical competence in diagnostic esophagogastroduodenoscopy. Health and Public Policy Committees, American College of Physicians. Ann Intern Med 1987;107:937-9. 13. Johanson JF, Cooper G, Eisen GM, Freeman M, Goldstein JL, Jensen DM, et al. Quality assessment of ERCP. Gastrointest Endosc 2002;56: 165-9. 14. Johanson JR, Cooper G, Eisen GM, Freeman M, Goldstein JL, Jensen DM, et al. Quality assessment of endoscopic ultrasound. Gastrointest Endosc 2002;55:798-801. 15. Rex DK, Bond JH, Winawer S, Levin TR, Burt RW, Johnson DA, et al. Quality in the technical performance of colonoscopy and the continuous quality improvement process for colonoscopy: recommendations of the U.S. Multi-Society Task Force on Colorectal Cancer. Am J Gastroenterol 2002;97:1296-305. 16. Principles of training in gastrointestinal endoscopy. From the American Society for Gastrointestinal Endoscopy. Gastrointest Endosc 1999;49:845-50. 17. American Society for Gastrointestinal Endoscopy. Appropriate use of gastrointestinal endoscopy. Gastrointest Endosc 2000;52:831-7. 18. Methods of granting hospital privileges to perform gastrointestinal endoscopy. American Society for Gastrointestinal Endoscopy Standard of Training and Practice Committee. Gastrointest Endosc 2002;55:780-3. 19. Guidelines for credentialing and granting privileges for gastrointestinal endoscopy. American Society for Gastrointestinal Endoscopy. Gastrointest Endosc 1998;48:679-82. 20. ASGE Guidelines for clinical application. Methods for privileging for new technology in gastrointestinal endoscopy. American Society for Gastrointestinal Endoscopy. Gastrointest Endosc 1999;50:899-900. 21. Guidelines for credentialing and granting privileges for endoscopic ultrasound. Gastrointest Endosc 2001;54:811-4. 22. Cohen S, Bacon BR, Berlin JA, Fleischer D, Hecht GA, Loehrer PJ, et al. National Institutes of Health State-of-the-Science Conference Statement; ERCP for diagnosis and therapy, January 14-16, 2002. Gastrointest Endosc 2002;56:803-9.
www.giejournal.org
Volume 63, No. 2 : 2006 GASTROINTESTINAL ENDOSCOPY 289
DISCUSSION No one involved in endoscopy doubts the importance of ensuring the highest possible quality of our processes and procedures. Many patients assume that any doctor who offers a procedure is competent to do it and that all facilities are equally safe (although some may look less appealing). The very simplicity of endoscopy as a ‘‘walk-in, walk-out’’ procedure can lull patients and practitioners alike into a sense of false security. Bad things can and do happen. Gastroenterologists often assume and argue that their procedures are more successful and safer than those performed by others with less focused training, but there is little proof. Our profession must work harder to develop and use methods for documenting performance, such as the report card. It will pay huge dividends eventually. We further call on institutions to incorporate these objective data into their processes for credentialing and recredentialing.
CONCLUSIONS
Practical endoscopic metrics 23. Nelson EC, Splaine ME, Batalden PB, Plume SK. Building measurement and data collection into medical practice. Ann Intern Med 1998;128:460-6. 24. Cotton PB. How many times have you done this procedure, doctor? Am J Gastroenterol 2002;97:522-3. 25. Thuraisingam AI, Tailor V, Johnston D, Allan I, Valori RM, Malpas P, et al. Results of an endoscopy unit global rating scale (GRS) in South Carolina and the United Kingdom (UK). Gastrointest Endosc 2005;61:AB123. 26. Bowles CJ, Leicester R, Romaya C, Swarbrick E, Williams CB, Epstein O. A prospective study of colonoscopy practice in the UK today; are we adequately prepared for national colorectal cancer screening tomorrow? Gut 2004;53:277-83. 27. Cotton PB, Connor P, McGee D, Jowell P, Nickl N, Schutz S, et al. Colonoscopy: practice variation among 69 hospital-based endoscopists. Gastrointest Endosc 2003;57:352-7.
APPENDIX 1.
Cotton et al
Current affiliations: Medical University of South Carolina, Charleston, South Carolina, USA; McGill University Health Center, Montreal, Canada; University of Pennsylvania Health System, Philadelphia, Pennsylvania, USA; North Mississippi Medical Center, Tupelo, Mississippi, USA; Concorde Medical Group, PLLC, New York, New York, USA; Cleveland Clinic Foundation, Cleveland, Ohio, USA; Indiana University School of Medicine, Indianapolis, Indiana, USA; Virginia Mason Clinic, Seattle, Washington, USA; University of Alabama School of Medicine, Birmingham, Alabama, USA. Reprint requests: Peter B. Cotton, MD, MRCP, Medical University of South Carolina, Digestive Disease Center, PO Box 250327, Suite 210 Clinical Sciences Bldg., 96 Jonathan Lucas St., Charleston, SC 29425.
d d
Report cards for endoscopists; potential metrics Colonoscopy 1. Specialty training, nature, and date 2. Board certification, nature, and date 3. Life-support certification 4. Lifetime numbers of colonoscopy 5. In last calendar year (and %) d Total numbers of colonoscopies d Spectrum of practice (% screening, % therapy) d Meet ASGE indication guidelines d Cecum reached and documented d Average total procedure time d Withdrawal time (% !8 minutes) d Adenoma yield in patients undergoing first colonoscopy d Adenoma yield in polyp surveillance d Polyps retrieved for pathology d Colonic perforation d Endoscopist-specific patient satisfaction data (ASGE scale*) Upper endoscopy 1. Specialty training, nature, and date 2. Board certification, nature, and date 3. Life support certification 4. Lifetime numbers of upper endoscopy 5. In last calendar year (and %) d Number of upper endoscopies d Spectrum of practice (% reflux related, % therapy) d Meet ASGE indication guidelines d Examinations to the descending duodenum d Number of biopsy specimens in Barrett’s esophagus d Dyspeptics tested for Helicobacter pylori d Immediate (apparent) technical success in hemostasis d Complications d Esophageal perforation (separate diagnostic and therapeutic procedures) 290 GASTROINTESTINAL ENDOSCOPY Volume 63, No. 2 : 2006
Infection rates after PEG Endoscopist-specific patient satisfaction data (ASGE scale*)
ERCP 1. Specialty training, nature and date 2. Board certification, nature, and date 3. Life-support certification 4. Lifetime numbers of ERCP 5. Data on procedures in last calendar year (and %) d Meet ASGE indication guidelines d Spectrum of cases by grade of difficulty (see reference 13) d Deep biliary cannulation (biliary indications) d Minor papilla cannulation in divisum d Stent insertion in obstructive jaundice d Stone extraction success d Failed procedure requiring further intervention d Biliary access pre-cuts d Complications total d Pancreatitis d Perforation d Procedure duration d Fluoroscopy duration d Endoscopist-specific patient satisfaction data (ASGE scale*) EUS 1. Specialty training, nature, and date 2. Board certification, nature, and date 3. Life-support certification 4. Lifetime numbers of EUS 5. Data on procedures in last calendar year d Total cases d Spectrum of practice d Total upper GI d Total pancreato-biliary d Total rectal d Total pulmonary d Meet ASGE indication guidelines d Reached the target region and visualized target organ(s) www.giejournal.org
Cotton et al
Practical endoscopic metrics
Documentation of stations (transducer positions) relevant to the indication d Documentation of lesions and interventions d Staging accuracy, with surgical confirmation d Complications requiring hospital treatment d Endoscopist-specific patient satisfaction data (ASGE scale*) Report cards for endoscopy units, potential metrics 1. Years unit existed 2. Nature: hospital, freestanding endoscopy clinic, or office 3. Accreditation agency (and most recent rating) 4. Name of medical director 5. Name of nurse manager 6. Volumes last calendar year (uppers, colons, ERCP, EUS) 7. Number of procedure rooms and patient bays
8. Total number trained nursing staff (levels) 9. Written policies for d Sedation and monitoring d Cleaning and disinfection d Risk reduction strategies d Patient recall for surveillance d Tracking pathology results 10. Safety data (%) d Infection rates d Unplanned intubations d Unplanned admissions 11. Communications and feedback d No-show rates (%) d Patient satisfaction monthly data (ASGE scale*) ASGE, American Society for Gastrointestinal Endoscopy. *ASGE patient satisfaction instrument (reference 3).
www.giejournal.org
Volume 63, No. 2 : 2006 GASTROINTESTINAL ENDOSCOPY 291
d
Excellence in endoscopy: toward practical metrics Peter B. Cotton, MD, FRCP, FRCS, Robert H. Hawes, MD, Alan Barkun, MD, Gregory G. Ginsberg, MD, Stephan Amman, MD, Jonathan Cohen, MD, Jeffrey Ponsky, MD, Douglas K. Rex, MD, Drew Schembre, MD, C. Mel Wilcox, MD
There is increasing interest and literature on improving and measuring the quality of endoscopy performance. We present the results of a workshop designed to define quality metrics that can be used now. Collection of structured data on a voluntary basis will stimulate improvement in endoscopic training, practice, and credentialing. Patients and practitioners will benefit. It is the right thing to do. Over the last 4 decades, digestive endoscopy has evolved from an occasional diversion for a few enthusiasts into a fundamental tool of gastroenterology. Endoscopy has become enormously popular throughout the world because of its tremendous and proven value in diagnosis and treatment. The problem is that the benefits are maximized only when procedures are performed at an optimal level of quality, which is not always the case. Technical failures and serious complications can occur in the best of hands but are more likely when procedures are performed by endoscopists with inadequate training and experience. Practitioners, patients, and payers should all be interested in enhancing the quality of endoscopic services. The professional organizations associated with endoscopy and their leaders have increasingly embraced the quality improvement paradigm that is advancing through medicine, as it has through manufacturing industry.1-4 The American Society for Gastrointestinal Endoscopy (ASGE), and the American College of Gastroenterology particularly have produced helpful reports and guidelines.5,6 In Britain, the Joint Advisory Group that represents all of the interested parties7 has been active in this area, as have other organizations around the world. The problem is that these thoughtful conclusions and well-meaning documents have had little impact so far in the real world. Quality is discussed but is not measured to any extent. Seldom do hospital privileging bodies follow published guidelines for credentialing, which include the use of proctored evaluation in some circumstances.8 We need to agree on what constitutes ‘‘excellence’’ and then develop appropriate metrics and systems to collect
the data. This will bring objectivity to training and will stimulate improvements in practice. Patients will benefit. These considerations led to the development of a study meeting entitled ‘‘Excellence in Endoscopy,’’ which was held in Albuquerque, New Mexico in October 2004.
Measuring and improving the quality of endoscopic performance will undoubtedly benefit patients and practitioners. It is the right thing to do.
METHODS Twenty leading North American gastroenterologists and selected guests were invited to consider 3 related topics. 1. What are the metrics of excellence in endoscopy? 2. How can the data be collected, analyzed, and used? 3. How can excellence be trained and enhanced? There were 3 team leaders, Robert Hawes, Alan Barkun, and Gregory Ginsberg. Each selected a team of 6 to 8 participants about 6 months before the meeting to focus on specific aspects of their allocated topic. They were tasked to review the relevant literature, circulate drafts, and then to present their recommendations in plenary session at the meeting, which lasted 2 days. After the workshop, the team submitted updated recommendations for inclusion in the final document. This report covers the first section of the meeting, i.e., developing the metrics of endoscopic excellence. Reports of the other sections will follow. The meeting was supported by Olympus America Inc., Medical Systems Group, Melville, NY.
What is endoscopic excellence?
Copyright ª 2006 by the American Society for Gastrointestinal Endoscopy 0016-5107/$32.00 doi:10.1016/j.gie.2005.04.048
We cannot start to measure excellence unless we can recognize it and define it. Therefore, what constitutes an excellent endoscopic experience? Society expects that diagnostic and therapeutic procedures will be appropriate (indicated) and that they will be
286 GASTROINTESTINAL ENDOSCOPY Volume 63, No. 2 : 2006
www.giejournal.org
Cotton et al
Practical endoscopic metrics
performed expeditiously, skillfully, successfully, safely, and comfortably.9 These expectations can be expanded to make a list of desirable characteristics for all types of endoscopic procedures. These include cognitive and technical skills of endoscopists but also appropriate behavior, environment, and staff. Elements of good endoscopy performance include the following: d Correct indications, adherence to published guidelines d Appropriate environment, support team, and behavior d Well prepared and informed patients d Strategies to minimize risk, including patient preparation and monitoring d Appropriate use of medications, including sedation/ analgesia when used d Correct selection of equipment d Comfortable intubation d Complete survey of the target organ(s) d Recognition of all abnormalities (and photographic documentation) d Appropriate tissue sampling (adherence to published guidelines) d Application of indicated therapy d Avoiding, recognizing, and managing complications d Reasonable duration d Smooth recovery, explanation, and discharge d Detailed and clear recommendations and follow-up plans d Integrated pathology results and communications d Complete documentation (and billing) d Positive feedback from patients Some of these elements apply to all procedures, whereas others (e.g., indications, precautions, protocol guidelines, therapies, outcomes) are specific to the individual type (e.g., upper endoscopy, colonoscopy, ERCP, EUS, capsule). Many articles have explored these quality issues10-12 and have provided similar lists of elements that seem to be important. The ASGE has published quality indicators specifically for upper endoscopy, colonoscopy, ERCP, and EUS.3,13,14 By far, the most comprehensive report to date comes from the prestigious U.S. multisociety task force on ‘‘Quality in technical performance of colonoscopy, and the continuous quality improvement process for colonoscopy.’’15 This document discussed the key issues (and questions needing more research) in 7 topic areas: indications and intervals (guideline adherence), precautions, insertion, withdrawal, biopsy and polypectomy, complications, and pathology interface. For each of these areas, they recommended up to 6 quality-improvement targets. Other groups and organizations have discussed quality issues in more general terms and in the context of credentialing. Thus, the ASGE has published guidelines for credentialing physicians and granting privileges in upper endoscopy, ERCP, and EUS.16-21 The recent report from the National Institutes of Health ‘‘state of the science’’
conference on ERCP made many comments on quality issues.22 These documents speak about adequate training, numbers of procedures, and desirable qualities in endoscopists but include no objective measures of performance, for the obvious reason that there has been no agreement for measures. This is the main rationale for this workshop and discussion.
www.giejournal.org
Volume 63, No. 2 : 2006 GASTROINTESTINAL ENDOSCOPY 287
How to recognize, predict, and measure excellence? There are some factors that make a good experience more likely. Good endoscopic training and extensive experience do not guarantee quality practice, but they certainly make it more likely. Thus, documentation of these and related elements should be a part of any assessment of endoscopists. Appropriate metrics could include the following: d Specialty training and certification (place and dates) d Training and maintenance of competence in life support and sedation d Evidence for continuing education in endoscopy d For each group of procedures, lifetime numbers, total in the last year, and spectrum of practice The proof of quality comes from documentation of performance. There is no substitute for collecting relevant data.23 The ASGE has recommended that endoscopists should collect data prospectively on their endoscopic practice and performance.3 This translates into ‘‘endoscopy report cards.’’24
Report cards for endoscopy The report card is simply an extension of the log books used by trainees to record their performance. It cannot possibly include all of the data elements that have been listed above and in various well-meaning publications. Items should be selected based on ease of data collection and by assumed relative importance. Some items are easily recorded and appear in most reports already (e.g., procedure indication, anatomical extent, duration, diagnosis, immediate unplanned events). Other items are more subjective (e.g., lesion interpretation) or more difficult to record (e.g., delayed complications, endoscopist-specific patient satisfaction). Some items would appear to be more important markers of quality than others. For ERCP, selective cannulation performance and complication rates are obvious key parameters. For colonoscopy, cecal intubation rates and minimum withdrawal times appear to be particularly relevant.15 The U.S. multisociety task force on colonoscopy suggested 25 quality improvement targets (each with goals).15 While such a comprehensive assessment would be ideal, it will be easier to start the process with simpler tools. Once the system is established, the parameters will be refined and expanded. Our ‘‘starter recommendations’’ are given in Appendix 1.
Practical endoscopic metrics
Why should we bother and how? It is likely that patients and payers will increasingly seek help from those with documented expertise and will avoid others. Keeping a report card will provide a competitive advantage. It also is a crucial tool in the evolving use of ‘‘pay for performance’’ in the United States. Those driving that agenda need professional help in determining meaningful quality measures. How to collect, analyze, and use these data was the topic for the second section of our workshop and will be reported separately. The increasing use of electronic endoscopy reporting systems will make this process easy, even automatic.
Endoscopy proctoring, examinations, and diplomas Knowledge of an endoscopist’s training, experience, and some report-card data can provide important data on the quality of his or her practice. But this may not be sufficient, especially when there may be questions about those elements that are more difficult to document precisely, such as the extent of knowledge, skill in lesion recognition, judgment in difficult treatment decisions, and teamwork. We may need additional information from proctoring, as recommended by ASGE.18,19 Another possibility is a formal examination. This would be resource intensive and not without controversy, but examinations are the recognized method for assuring a reasonable level of knowledge and performance in many other fields and usually lead to a diploma. It initially could be piloted for a lower volume, higher-risk procedure such as ERCP. The diploma would be based on data from report cards, an examination of core knowledge, observation of a few cases, and, possibly, some work on simulators. This exercise would require agreement on how to ‘‘score’’ the more subjective elements of a procedure. Unless and until the official agencies (such as the American Board of Internal Medicine [ABIM]) embrace this concept, the ERCP diploma would have a significant impact only if a substantial number of practitioners volunteer to participate.
Cotton et al
specific for endoscopy units in the United States, and the agencies that accredit them do not have comprehensive guidelines. Hitherto, inspections have focused on the existence (and compliance with) guidelines for sedation and infection control. These are important, but there are other elements.
Metrics of quality in endoscopy units It is not difficult to list features of endoscopy units that may impact the quality of the procedures being performed in them. 1. Facilities and equipment (and maintenance) 2. Volumes and range of procedures performed 3. Method for credentialing doctors 4. Staffing levels, with appropriate training and certification 5. Leadership and cohesion (retention) 6. Documented policies and compliance systems d Sedation and monitoring d Cleaning and disinfection d Methods for risk minimization 7. Regularly collected safety data d Infection rates, disinfection surveillance d Sedation issues, intubations, prolonged recovery d Accidents d Unplanned admissions 8. Communications d No-show rates d Pathology follow-up system d Surveillance returns (patients with polyps, stents) d Referring physicians 9. Patient feedback d Complaint management d Formal satisfaction surveys An ‘‘endoscopy unit report card’’ could be developed by picking a selection of these criteria, as shown in the Appendix. As part of the endoscopy modernization process in Britain, a ‘‘global rating scale’’ was completed by most of the endoscopy units.25 This was a qualitative self-rating survey dealing mainly with processes instead of numbers and yielded important comparative information. It will be updated.
The excellence of endoscopy units Patients are concerned about their safety, comfort, and dignity during endoscopy, and the efficiency of the process. While endoscopists have a responsibility for these elements and can influence the way the rest of the team functions, there are important quality elements of the endoscopy unit and the staff that can be considered separately. Endoscopists (however talented) cannot work without good facilities, equipment, and a well-trained and motivated staff. While most health care facilities pay some attention to ‘‘quality improvement,’’ the extent to which this percolates down to the endoscopy unit is variable. There is no national quality improvement program 288 GASTROINTESTINAL ENDOSCOPY Volume 63, No. 2 : 2006
What performance level is good enough: benchmarking Endoscopists and units vary in their levels of performance. Not all patients can be managed by the superexperts. The issue then is who decides what constitutes acceptable performance and what that should be. The U.S. multisociety task force for colonoscopy suggested goals (e.g., 95% cecal intubation rates in screening examinations, and 100% compliance with coagulation issues). Other professional bodies have also made similar recommendations (e.g., 90% selective biliary cannulation rate in www.giejournal.org
Cotton et al
ERCP). But, we currently have no idea how much variation really exists in the broad endoscopic community. Aggregated data on cecal intubation rates from acknowledged experts in prospective studies15 paint a much more favorable picture than those from more general audits of broader community practice.26,27 The true picture can be documented only if a large and representative sample of endoscopists and units share and compare data from similar report cards. Benchmarking is the practice of mapping the performance of an individual (endoscopist or unit) against that from a broad spectrum of ‘‘competitors.’’ There are many practical and political issues involved in attempting to aggregate and analyze such data, not least because, by definition, 50% of the doctors and units will be shown to be performing below the mean. These issues were the subject of the second part of the workshop and will be reported separately. Ultimately, informed consumers should be able to judge what is good enough for them. Increasing publicity about variations in performance (e.g., in colonoscopy completion rates), if linked to specific report card data, will drive this agenda. Practitioners with poor outcomes or those who choose not to collect data will be disadvantaged.
Practical endoscopic metrics
DISCLOSURE The workshop was supported by an educational grant from Olympus America Inc.
REFERENCES
We recommend that all endoscopists who are comfortable with their performance should begin to collect data for individual report cards, by using simple data points as shown in Appendix 1. The effort need not be overambitious or burdensome. Let us get something started. The fact that some endoscopists will be reluctant to document and advertise their poor performance should not stop us from doing the right things, right? We should wear our data plainly and proudly as badges of quality.
1. Berwick DM. Quality comes home. Ann Intern Med 1996;125:839-43. 2. Chassin MR. Improving the quality of care. N Engl J Med 1996;335: 1060-3. 3. Quality and outcomes assessment in gastrointestinal endoscopy. American Society for Gastrointestinal Endoscopy. Gastrointest Endosc 2000;52:827-30. 4. Johanson JF. Continuous quality improvement in the ambulatory endoscopy center. Gastrointest Endosc Clin N Am 2002;12:351-65. 5. American Society for Gastrointestinal Endoscopy. ASGE Web site. Available from URL: http://www.asge.org. 6. American College of Gastroenterology. ACG Web site. Available from URL: http://www.acg.gi.org. 7. Joint Advisory Group on Gastrointestinal Endoscopy. JAG Web site. Available from URL: http://www.thejag.org.uk. 8. ASGE guidelines for clinical application. Proctoring for hospital endoscopy privileges. American Society for Gastrointestinal Endoscopy. Gastrointest Endosc 1999;50:901-5. 9. Johanson JF, Overholt BF, Frakes JT. Characteristics of best gastroenterology practices. Am J Gastroenterol 1999;94:2519-30. 10. Minoli G, Meucci G, Prada A, Terruzzi V, Bortoli A, Gullotta R, et al. Quality assurance and colonoscopy. Endoscopy 1999;31:522-7. 11. Chak A, Cooper GS. Procedure-specific outcomes assessment for endoscopic ultrasonography. Gastrointest Endosc Clin N Am 1999;9: 649-56. 12. Clinical competence in diagnostic esophagogastroduodenoscopy. Health and Public Policy Committees, American College of Physicians. Ann Intern Med 1987;107:937-9. 13. Johanson JF, Cooper G, Eisen GM, Freeman M, Goldstein JL, Jensen DM, et al. Quality assessment of ERCP. Gastrointest Endosc 2002;56: 165-9. 14. Johanson JR, Cooper G, Eisen GM, Freeman M, Goldstein JL, Jensen DM, et al. Quality assessment of endoscopic ultrasound. Gastrointest Endosc 2002;55:798-801. 15. Rex DK, Bond JH, Winawer S, Levin TR, Burt RW, Johnson DA, et al. Quality in the technical performance of colonoscopy and the continuous quality improvement process for colonoscopy: recommendations of the U.S. Multi-Society Task Force on Colorectal Cancer. Am J Gastroenterol 2002;97:1296-305. 16. Principles of training in gastrointestinal endoscopy. From the American Society for Gastrointestinal Endoscopy. Gastrointest Endosc 1999;49:845-50. 17. American Society for Gastrointestinal Endoscopy. Appropriate use of gastrointestinal endoscopy. Gastrointest Endosc 2000;52:831-7. 18. Methods of granting hospital privileges to perform gastrointestinal endoscopy. American Society for Gastrointestinal Endoscopy Standard of Training and Practice Committee. Gastrointest Endosc 2002;55:780-3. 19. Guidelines for credentialing and granting privileges for gastrointestinal endoscopy. American Society for Gastrointestinal Endoscopy. Gastrointest Endosc 1998;48:679-82. 20. ASGE Guidelines for clinical application. Methods for privileging for new technology in gastrointestinal endoscopy. American Society for Gastrointestinal Endoscopy. Gastrointest Endosc 1999;50:899-900. 21. Guidelines for credentialing and granting privileges for endoscopic ultrasound. Gastrointest Endosc 2001;54:811-4. 22. Cohen S, Bacon BR, Berlin JA, Fleischer D, Hecht GA, Loehrer PJ, et al. National Institutes of Health State-of-the-Science Conference Statement; ERCP for diagnosis and therapy, January 14-16, 2002. Gastrointest Endosc 2002;56:803-9.
www.giejournal.org
Volume 63, No. 2 : 2006 GASTROINTESTINAL ENDOSCOPY 289
DISCUSSION No one involved in endoscopy doubts the importance of ensuring the highest possible quality of our processes and procedures. Many patients assume that any doctor who offers a procedure is competent to do it and that all facilities are equally safe (although some may look less appealing). The very simplicity of endoscopy as a ‘‘walk-in, walk-out’’ procedure can lull patients and practitioners alike into a sense of false security. Bad things can and do happen. Gastroenterologists often assume and argue that their procedures are more successful and safer than those performed by others with less focused training, but there is little proof. Our profession must work harder to develop and use methods for documenting performance, such as the report card. It will pay huge dividends eventually. We further call on institutions to incorporate these objective data into their processes for credentialing and recredentialing.
CONCLUSIONS
Practical endoscopic metrics 23. Nelson EC, Splaine ME, Batalden PB, Plume SK. Building measurement and data collection into medical practice. Ann Intern Med 1998;128:460-6. 24. Cotton PB. How many times have you done this procedure, doctor? Am J Gastroenterol 2002;97:522-3. 25. Thuraisingam AI, Tailor V, Johnston D, Allan I, Valori RM, Malpas P, et al. Results of an endoscopy unit global rating scale (GRS) in South Carolina and the United Kingdom (UK). Gastrointest Endosc 2005;61:AB123. 26. Bowles CJ, Leicester R, Romaya C, Swarbrick E, Williams CB, Epstein O. A prospective study of colonoscopy practice in the UK today; are we adequately prepared for national colorectal cancer screening tomorrow? Gut 2004;53:277-83. 27. Cotton PB, Connor P, McGee D, Jowell P, Nickl N, Schutz S, et al. Colonoscopy: practice variation among 69 hospital-based endoscopists. Gastrointest Endosc 2003;57:352-7.
APPENDIX 1.
Cotton et al
Current affiliations: Medical University of South Carolina, Charleston, South Carolina, USA; McGill University Health Center, Montreal, Canada; University of Pennsylvania Health System, Philadelphia, Pennsylvania, USA; North Mississippi Medical Center, Tupelo, Mississippi, USA; Concorde Medical Group, PLLC, New York, New York, USA; Cleveland Clinic Foundation, Cleveland, Ohio, USA; Indiana University School of Medicine, Indianapolis, Indiana, USA; Virginia Mason Clinic, Seattle, Washington, USA; University of Alabama School of Medicine, Birmingham, Alabama, USA. Reprint requests: Peter B. Cotton, MD, MRCP, Medical University of South Carolina, Digestive Disease Center, PO Box 250327, Suite 210 Clinical Sciences Bldg., 96 Jonathan Lucas St., Charleston, SC 29425.
d d
Report cards for endoscopists; potential metrics Colonoscopy 1. Specialty training, nature, and date 2. Board certification, nature, and date 3. Life-support certification 4. Lifetime numbers of colonoscopy 5. In last calendar year (and %) d Total numbers of colonoscopies d Spectrum of practice (% screening, % therapy) d Meet ASGE indication guidelines d Cecum reached and documented d Average total procedure time d Withdrawal time (% !8 minutes) d Adenoma yield in patients undergoing first colonoscopy d Adenoma yield in polyp surveillance d Polyps retrieved for pathology d Colonic perforation d Endoscopist-specific patient satisfaction data (ASGE scale*) Upper endoscopy 1. Specialty training, nature, and date 2. Board certification, nature, and date 3. Life support certification 4. Lifetime numbers of upper endoscopy 5. In last calendar year (and %) d Number of upper endoscopies d Spectrum of practice (% reflux related, % therapy) d Meet ASGE indication guidelines d Examinations to the descending duodenum d Number of biopsy specimens in Barrett’s esophagus d Dyspeptics tested for Helicobacter pylori d Immediate (apparent) technical success in hemostasis d Complications d Esophageal perforation (separate diagnostic and therapeutic procedures) 290 GASTROINTESTINAL ENDOSCOPY Volume 63, No. 2 : 2006
Infection rates after PEG Endoscopist-specific patient satisfaction data (ASGE scale*)
ERCP 1. Specialty training, nature and date 2. Board certification, nature, and date 3. Life-support certification 4. Lifetime numbers of ERCP 5. Data on procedures in last calendar year (and %) d Meet ASGE indication guidelines d Spectrum of cases by grade of difficulty (see reference 13) d Deep biliary cannulation (biliary indications) d Minor papilla cannulation in divisum d Stent insertion in obstructive jaundice d Stone extraction success d Failed procedure requiring further intervention d Biliary access pre-cuts d Complications total d Pancreatitis d Perforation d Procedure duration d Fluoroscopy duration d Endoscopist-specific patient satisfaction data (ASGE scale*) EUS 1. Specialty training, nature, and date 2. Board certification, nature, and date 3. Life-support certification 4. Lifetime numbers of EUS 5. Data on procedures in last calendar year d Total cases d Spectrum of practice d Total upper GI d Total pancreato-biliary d Total rectal d Total pulmonary d Meet ASGE indication guidelines d Reached the target region and visualized target organ(s) www.giejournal.org
Cotton et al
Practical endoscopic metrics
Documentation of stations (transducer positions) relevant to the indication d Documentation of lesions and interventions d Staging accuracy, with surgical confirmation d Complications requiring hospital treatment d Endoscopist-specific patient satisfaction data (ASGE scale*) Report cards for endoscopy units, potential metrics 1. Years unit existed 2. Nature: hospital, freestanding endoscopy clinic, or office 3. Accreditation agency (and most recent rating) 4. Name of medical director 5. Name of nurse manager 6. Volumes last calendar year (uppers, colons, ERCP, EUS) 7. Number of procedure rooms and patient bays
8. Total number trained nursing staff (levels) 9. Written policies for d Sedation and monitoring d Cleaning and disinfection d Risk reduction strategies d Patient recall for surveillance d Tracking pathology results 10. Safety data (%) d Infection rates d Unplanned intubations d Unplanned admissions 11. Communications and feedback d No-show rates (%) d Patient satisfaction monthly data (ASGE scale*) ASGE, American Society for Gastrointestinal Endoscopy. *ASGE patient satisfaction instrument (reference 3).
www.giejournal.org
Volume 63, No. 2 : 2006 GASTROINTESTINAL ENDOSCOPY 291
d