Training to competency in colonoscopy: assessing and defining competency standards

ORIGINAL ARTICLE: Clinical Endoscopy

Training to competency in colonoscopy: assessing and defining competency standards Robert E. Sedlack, MD, MHPE Rochester, Minnesota, USA

Background: How to define competency in colonoscopy, how to assess it, and how much training is required are questions that experts in endoscopy have grappled with since the advent of the procedure. Objective: To describe methods to assess core endoscopy skills in trainees and learning curves for these parameters and to define competency thresholds for these skills. Design: A prospective descriptive assessment of trainee colonoscopy performance. Setting: Mayo Clinic, Rochester, Minnesota. Subjects: Gastroenterology fellows undergoing endoscopy training. Intervention: From July 2007 through June 2010, fellows’ core cognitive and motor colonoscopy skills were assessed by using the Mayo Colonoscopy Skills Assessment Tool (MCSAT). Main Outcome Measurements: Average MCSAT item scores and learning curves are described. Minimal competence thresholds for each MSCAT item are established by using the contrasting groups method. Results: Forty-one GI fellows performed 6635 colonoscopies; 4103 procedures (62%) were assessed by using the MCSAT. Average scores of 3.5 set the competency bar for each of the core skills and were reached by 275 procedures on average. Independent cecal intubation rates of 85% and cecal intubation times of 16 minutes or less were also achieved at 275 procedures on average. Limitations: Limited to a single center. Conclusions: Learning curves for core colonoscopy skills are described. MCSAT scores of 3.5, cecal intubation rates of 85%, and intubation times of less than 16 minutes are recommended as minimal competency criteria. It takes on average 275 procedures to achieve competence in colonoscopy. This is more than previous gastroenterology training recommendations and far more than current training requirements in other specialties. (Gastrointest Endosc 2011;74:355-66.)

Approximately 1400 gastroenterology fellows and general surgery residents graduate from U.S. training programs annually.1 Each of these training tracts has a common goal of ensuring that the core motor and cognitive skills necessary to successfully and competently perform colonoscopy are acquired. However, training requirements for each pathway are markedly different, with drastically different expectations of procedure volumes and

training endpoints. In addition, one fourth of all family practice residency programs provide colonoscopy training, and a few thousand family practitioners are privileged to perform colonoscopy based on completion of alternate training pathways.2 Ensuring that competency is achieved by all of these different training methods is generally quite difficult. These marked differences in training expectations are borne out of the fact that the factors that define com-

Abbreviations: MCC, minimal competency criteria; MCSAT, Mayo Colonoscopy Skills Assessment Tool.

0016-5107/$36.00 doi:10.1016/j.gie.2011.02.019

DISCLOSURE: The author disclosed no financial relationships relevant to this publication. Funding for the development and implementation of the MCSAT form was provided by an unrestricted educational grant from the Mayo Clinic, Rochester, Minnesota.

Received October 22, 2010. Accepted February 22, 2011.

Copyright © 2011 by the American Society for Gastrointestinal Endoscopy

www.giejournal.org

Current affiliation: Mayo Clinic, Rochester, Minnesota, USA. Reprint requests: Robert E. Sedlack, MD, MHPE, Mayo Clinic, 200 First Street SW, Mayo 9-E, Rochester, MN 55905.

Volume 74, No. 2 : 2011 GASTROINTESTINAL ENDOSCOPY 355

Training to competency in colonoscopy

petence in colonoscopy and how to assess them are poorly understood and quite subjective. As such, there is a critical need for a better understanding of how we define competence and a means to objectively assess this. Experts in the gastroenterology community have long sought a means to accurately assess individual endoscopy skills.3,4 At the Mayo Clinic, the previously validated Mayo Colonoscopy Skills Assessment Tool (MCSAT) was developed by breaking down the procedure into its core motor and cognitive skill elements as a means to objectively assess these skills during live procedures.5 In this study, the goal was to define the average learning curves of colonoscopy training for each of these core skills as well as for overall cognitive and motor competence. In addition, MCSAT score benchmarks are established for each of these skills to define the minimum standards for competence. Finally, the average procedure volumes required to achieve these minimal competency criteria (MCC) are described. These results should aid training programs, professional societies, and credentialing bodies in the development of more uniform, objective, and ideal training standards in colonoscopy to replace the current patchwork of guidelines that exist between specialties.

METHODS Study design This is a descriptive report of the prospective assessment of trainee skills in colonoscopy by using the validated MCSAT survey. This study was reviewed and obtained a waiver from the local institutional review board and required only verbal consent from each trainee, which was obtained in accordance with 45 C.F.R. 46.117 (c) (2).

Setting This study was conducted in the Division of Gastroenterology and Hepatology at the Mayo Clinic, Rochester, Minnesota, over a 3-year period from July 2007 through June 2010. The goal of this study was to focus on assessing minimal procedural skills required to be competent to perform routine colonoscopic examinations. As such, the assessment was limited to colonoscopies performed at the institution’s routine outpatient procedure suites. Procedures performed in therapeutic or “complex” endoscopy suites were excluded from this analysis because of the fact that these procedures tend to be longer, more difficult, and more likely to require therapeutic intervention than a typical routine examination and hence require a higher skill set than that for routine colonoscopy. Data from these complex cases performed by fellows were still recorded in the database, simply to allow tracking of the sequential order of cases performed by a trainee so that each MCSAT result could be correctly identified by the order in which it was performed. 356 GASTROINTESTINAL ENDOSCOPY Volume 74, No. 2 : 2011

Sedlack

Take-home Message ●

●

The learning curves of the core motor and cognitive skills required to perform colonoscopy are described and the minimal competency criteria for these skills are defined. The average number of procedures required to achieve these minimal competency thresholds are identified. These training volumes are much more than current training guidelines recommend.

Participants The participants included all gastroenterology fellows involved in colonoscopy training during the 3-year study period. Because training classes graduate and new fellow classes begin with each year, this assessment was carried out with classes on a rotating basis. Each year there were approximately 8 fellows per class (first-, second-, and third-year trainees) for a total of approximately 24 trainees at any given time. At all times, even with fellows beginning their final year of training at the start of this study, records were available as part of the same institutional endoscopy database that allowed their procedure data to be accurately linked to the total number of procedure experience they have to ensure that MCSAT form results were associated with the correct procedure number.

Intervention Teaching staff were given the task of assessing each fellow’s cognitive and motor skills by using the MCSAT during each colonoscopy performed. The MCSAT form consists of 14 survey items (Online Appendix 1, available at www.giejournal.org). Five of these items assess the following core motor skills: (1) the degree of the fellow’s hands-on participation (hands-on participation), (2) safe colonoscope advancement technique (safe colonoscope advancement), (3) effective loop reduction maneuvers (loop reduction), (4) the adequacy of mucosal visualization during withdrawal (mucosal visualization), and (5) the ability to perform a therapeutic maneuver (ie, biopsy, polypectomy) if indicated (therapeutic maneuver). Each of these skills was assessed by using a 4-point grading scale (1, novice; 2, intermediate; 3, advanced; 4, superior/ competent to perform skill independently). For each item score, an objective set of performance criteria was included on the MCSAT form to allow reproducible and objective scoring5 (Online Appendix 1). In addition to these individual motor metrics, an overall hands-on competence assessment score was also given by using the same 4-point scoring system outlined previously. Finally, the depth of independent colonoscope advancement was recorded by using a different categorical scale (1, rectum; 2, sigmoid; 3, splenic flexure; 4, hepatic flexure; 5, cecum with no attempt at ileal intubation; 6, cecum with unsuccessful attempts at ileal intubation; 7, cecum with successwww.giejournal.org

Sedlack

ful ileal intubation, and 9, other surgical anatomic endpoint reached). This allowed calculations of average independent cecal and ileal intubation rates. Successful cecal intubation was counted if the fellow received a score of 5, 6, or 7. If staff members provided any hands-on assistance, the location of the last landmark reached by the fellow before assistance was recorded as the depth of independent insertion even if the fellow completed the examination after receiving help. Cecal intubation and withdrawal times were also recorded to allow assessment of average cecal intubation and withdrawal times. The MCSAT also assesses fellows’ cognitive skills in the following 6 parameters: 1, knowledge of procedure indication and relevant medical issues (indications/Issues); 2, appropriate titration of initial sedation (initial sedation); 3, appropriate monitoring and management of patient discomfort during the procedure (pain management); 4, awareness of colonoscope location at all times by landmark recognition (landmark recognition); 5, ability to independently recognize abnormal findings (pathology identification); and 6, appropriate selection of therapeutic tools and their settings (eg, snare, hot biopsy, power settings) if indicated (tool selection/settings). Each of these metrics used the same 4-point scale as used by the core motor skills described previously. In addition to these individual core cognitive skills, a parameter measuring the overall cognitive competence for the procedure was also included. This overall competence score along with the overall hands-on scores act as the standards with which the other individual core cognitive and motor parameters are respectively compared. MCSAT scores are entered by the staff directly into the institution’s endoscopy database, becoming part of the procedure record, thus linking them to all the other procedure-specific data (ie, procedure date, start times, time cecum was reached, completion time, medications given, pathologic findings, therapeutic interventions performed). The recorded dates and times of each procedure allow sequential tracking of individual fellow data, so scores from any point of an individual’s training can be compared with another (ie, the 150th procedure scores can be compared with the 300th colonoscopy). This provides a means for establishing average learning curves for each MCSAT parameter. Access to the MCSAT survey in the database is password protected, allowing only supervising staff access to this area of the procedure record. Additionally, scores from a specific staff or a specific procedure were never shared with fellows to ensure that staff members felt free to grade as openly as possible.

Analysis MCSAT scores and times for each procedure were prospectively collected. These data were grouped based on the order of performance. To minimize the effect of a single outlying or missing data point (eg, the patient for a fellow’s 150th procedure had unusually difficult anatomy www.giejournal.org

Training to competency in colonoscopy

resulting in failure to reach the cecum or the staff did not complete a form for this particular procedure), average scores were used that were composed of all procedures in the individual category. The first category of scores (first 10) contained data from the first 10 colonoscopies performed by each fellow. The remaining groups were formed at sequential steps of 25 procedures (eg, 25, 50, 75) to the 400th procedure group. Each stage comprises the 10 procedures before and subsequent to the category label (ie, an individual’s 25th procedure score is an average of MCSAT scores for their 16th to 35th procedures). An average of these individuals’ scores is calculated for each MCSAT parameter at each step of training to establish normal learning curves. There are various methods for establishing MCC, or competency thresholds, for these MCSAT learning curves (eg, Angoff, Ebel Hofstee, Borderline Group, and contrasting groups method).6 There is no criterion standard method. Each is designed to define different pass/ fail thresholds depending on the researcher’s or educator’s needs. With the goal being to compare scores from the core skills among the fellows who have been deemed to have achieved overall competence with those who have not (ie, defining the pass/fail cutoff of 1 independent variable [core skill score] with that of an independent assessment of overall competence), the contrasting groups method is specifically designed to accomplish this type of score discrimination.7 With this method, it is necessary to first define the independent assessment of overall competence by splitting the motor and cognitive data from each individual procedure into 2 contrasting groups: competent and noncompetent. To accomplish this, the final 2 items on the MCSAT assessed the performance for the particular procedure as competent or not. These 2 items (overall cognitive and overall motor scores) were created as part of this MCSAT form specifically to act as anchors for their respective core skills. Scores in these 2 parameters of 1, 2, or 3 indicate stages of skill progressing toward, but not yet achieving, competence and form the noncompetent group. Scores of 4, by definition, form the competent group (ie, able to perform the cognitive or motor tasks of colonoscopy independently). For example, if a procedure received a score of 4 on the overall motor competence parameter, each of the individual core motor skills from that procedure, regardless of the score, would belong to the competent group. If in the next procedure, the faculty grades the overall motor competence with a 3, all of the individual core motor parameter scores from that procedure would belong to the noncompetent group (even if 1 of these individual core skills received a score of 4). The score distribution for all data from a specific core skill (eg, mucosal visualization) is then plotted for the competent and noncompetent groups as separate curves (Fig. 1). A vertical line drawn through the point of intersection of these 2 Volume 74, No. 2 : 2011 GASTROINTESTINAL ENDOSCOPY 357

Training to competency in colonoscopy

Sedlack

Figure 1. Contrasting groups intercept graph. An example of the contrasting groups intercept graph is shown for the core skill of loop reduction. A normal fitted distribution density curve is plotted for the noncompetent (dashed line) group against the fitted curve for the competent (solid line) group. Where the intercept of these 2 curves projects onto the x axis represents the minimal average passing score required for this particular parameter (3.40). The density (y-axis) is expressed as a fraction of the total scores for each group. (When one uses probability density functions to estimate the curves, the top of the competent group curve may exceed 1.0 when data variance is small, as is seen in the competent group.)

curves projecting onto the x axis indicates the pass/fail score for that particular core skill or the MCC score that a fellow should reach to be deemed minimally competent in that particular skill. The average number of procedures required to reach this competency threshold for each skill is then described. This number is based on the point in training at which a skill’s learning curve first reaches, and stays above, its respective MCC score. Independent cecal intubation rates are calculated by categorizing the results of the depth of independent insertion parameter to 1 of 2 possibilities: either failed to reach the cecum (depth score of 1, 2, 3, or 4) or successfully reached the cecum (score of 5, 6, or 7). Procedures with scores of 9 are excluded because this score characterizes procedures in which altered surgical anatomy is encountered as an endpoint (such as an ileocolonic anastomosis). From these pass/fail data, the average proportion of examinations in which the cecum was reached independently will be described at each stage of training. Finally, the average time required by trainees to reach the cecum will be described at the same stages of training and the MCC determined by using a contrasting groups analysis. For this analysis, cecal intubation times are corrected by examining data only for cases in which the 358 GASTROINTESTINAL ENDOSCOPY Volume 74, No. 2 : 2011

cecum was reached independently. This is done to eliminate any influence that staff hands-on assistance has on the recorded intubation time.

Other analyses Average cecal intubation rates, cecal intubation times, and overall cognitive and hands-on skills scores are compared for trainees at 50, 150, 275, and at 400 procedures (by using a 2-tailed t test with P values ⬍.05) to determine whether there is any significant difference in abilities among these different stages of training. JMP 8.0 statistical software (SAS Institute Inc, Cary, NC) was used for all analyses.

RESULTS Participants From July 2007 through June 2010, 41 gastroenterology fellows performed 10,071 colonoscopies. Of these, 6635 colonoscopies (66%) were performed in the routine outpatient endoscopy suites, whereas the remainder were performed in the complex endoscopy suites at the institution. Of these routine procedures, 4130 (62%) had MCSAT evaluations completed by 58 different staff endoscopists. With graduation of classes and enrollment of others during www.giejournal.org

Sedlack

Training to competency in colonoscopy

TABLE 1. Individual motor skill scores

No. of procedures First 10

Hands-on participation, average score (no., 95% CI)

Safe endoscope advancement, average score (no., 95% CI)

Loop reduction, average score (no., 95% CI)

Mucosal visualization, average score (no., 95% CI)

Therapeutic maneuver, average score (no., 95% CI)

Average of combined individual motor scores, average score (no., 95% CI)

Overall hands-on competence, average score (no., 95% CI)

1.71 (140, 1.59-1.84)

2.12 (139, 2.00-2.25)

1.83 (135, 1.70-1.96)

2.10 (129, 1.96-2.23)

2.60 (44, 2.33-2.86)

1.97 (143, 1.86-2.08)

1.43 (140, 1.33-1.53)

25

2.34 (288, 2.23-2.44)

2.60 (289, 2.50-2.68)

2.31 (281, 2.21-2.41)

2.70 (265, 2.60-2.79)

3.06 (100, 2.88-3.24)

2.50 (293, 2.41-2.58)

2.03 (288, 1.94-2.13)

50

2.36 (287, 2.26-2.46)

2.62 (290, 2.53-2.71)

2.43 (288, 2.34-2.53)

2.73 (280, 2.64-2.82)

2.96 (109, 2.78-3.15)

2.55 (293, 2.47-2.64)

2.18 (292, 2.09-2.27)

75

2.73 (303, 2.63-2.83)

2.91 (308, 2.82-2.99)

2.79 (304, 2.69-2.88)

3.06 (289, 2.97-3.15)

3.15 (123, 3.00-3.29)

2.87 (308, 2.79-2.95)

2.55 (302, 2.46-2.63)

100

2.98 (242, 2.88-3.08)

3.16 (242, 3.08-3.25)

2.96 (234, 2.86-3.07)

3.23 (234, 3.14-3.31)

3.41 (103, 3.23-3.58)

3.08 (246, 3.01-3.18)

2.78 (238, 2.68-2.87)

125

3.09 (232, 2.99-3.19)

3.30 (235, 3.21-3.39)

3.15 (236, 3.06-3.25)

3.39 (227, 3.30-3.48)

3.49 (100, 3.36-3.62)

3.26 (237, 3.18-3.33)

2.99 (232, 2.89-3.08)

150

3.25 (153, 3.11-3.39)

3.30 (154, 3.18-3.41)

3.16 (152, 3.04-3.29)

3.37 (147, 3.27-3.48)

3.47 (62, 3.30-3.63)

3.28 (155, 3.18-3.38)

3.02 (150, 2.90-3.14)

175

3.44 (139, 3.31-3.56)

3.51 (139, 3.41-3.61)

3.42 (139, 3.31-3.53)

3.54 (136, 3.43-3.64)

3.68 (59, 3.51-3.84)

3.48 (139, 3.40-3.57)

3.27 (135, 3.15-3.38)

200

3.39 (174, 3.28-3.5)

3.46 (179, 3.37-3.55)

3.34 (176, 3.24-3.44)

3.52 (174, 3.43-3.61)

3.55 (86, 3.39-3.70)

3.43 (180, 3.35-3.51)

3.34 (176, 3.23-3.44)

225

3.26 (130, 3.12-3.4)

3.46 (131, 3.35-3.57)

3.30 (131, 3.18-3.41)

3.61 (127, 3.51-3.70)

3.64 (58, 3.47-3.81)

3.41 (134, 3.32-3.50)

3.25 (131, 3.12-3.38)

250

3.43 (136, 3.32-3.55)

3.60 (138, 3.51-3.69)

3.50 (137, 3.40-3.59)

3.58 (134, 3.50-3.68)

3.74 (53, 3.57-3.90)

3.53 (138, 3.45-3.62)

3.46 (137, 3.36-3.56)

275

3.50 (161, 3.39-3.61)

3.62 (159, 3.53-3.72)

3.52 (161, 3.41-3.62)

3.70 (156, 3.62-3.78)

3.66 (83, 3.51-3.81)

3.60 (163, 3.52-3.68)

3.50 (159, 3.39-3.61)

300

3.81 (129, 3.73-3.88)

3.82 (130, 3.75-3.88)

3.76 (130, 3.68-3.84)

3.87 (129, 3.81-3.93)

3.86 (57, 3.74-3.98)

3.81 (130, 3.78-3.87)

3.81 (128, 3.74-3.88)

325

3.64 (183, 3.55-3.74)

3.73 (184, 3.66-3.80)

3.65 (184, 3.56-3.73)

3.74 (179, 3.67-3.81)

3.83 (84, 3.73-3.93)

3.69 (186, 3.62-3.76)

3.70 (183, 3.63-3.78)

350

3.62 (140, 3.51-3.74)

3.71 (140, 3.62-3.80)

3.68 (137, 3.59-3.77)

3.74 (137, 3.67-3.82)

3.71 (56, 3.56-3.87)

3.69 (140, 3.60-3.77)

3.64 (133, 3.54-3.74)

375

3.65 (137, 3.55-3.74)

3.73 (138, 3.65-3.81)

3.68 (140, 3.59-3.77)

3.77 (135, 3.69-3.85)

3.81 (68, 3.68-3.94)

3.71 (140, 3.64-3.78)

3.74 (138, 3.66-3.82)

400

3.68 (31, 3.50-3.85)

3.65 (31, 3.47-3.82)

3.63 (30, 3.43-3.84)

3.76 (29, 3.59-3.92)

3.84 (19, 3.66-4.00)

3.69 (31, 3.53-3.85)

3.87 (30, 3.74-4.00)

The average Mayo Colonoscopy Skills Assessment Tool score for each of the core motor skill parameters is shown at sequential steps of every 25 procedures. These averages are based on a 1- to 4-point scoring scale where 1 to 3 represents progression toward competence and 4 represents the achievement of competence. Minimal competence criteria are defined as an average score of 3.5 or greater. Shaded cells indicate that the average score for this parameter has reached and is subsequently maintained above the minimal competency threshold score of 3.5.

training, there were 5 distinct training classes participating in this study. At any given time, there were generally 24 fellows in the training program (approximately 8 per class); however, 1 class had 9 trainees, giving us a total of 41 participants. At the end of the study period, we had examined 3 classes of first-year, 3 classes of second-year, and 3 classes of third-year fellows, so all stages of training were represented approximately equally (25 first-year, 25 second-year, and 24 third-year trainees). Because this study was conducted over a 3-year period, many of these 41 fellows were part of this study for multiple years, so are included in more than 1 class. All participants had no previous endoscopy experience before beginning fellowship. All 24 fellows who had completed their fellowship by the end of study period had completed an average of 399 colonoscopies (95% CI, 365-433).

Descriptive data The average score (number, 95% CI) for each of the MCSAT’s individual motor parameters, the average of these scores, and the overall motor scores are shown at intervals of every 25 procedures (Table 1). Average cognitive parameter scores for each stage are similarly shown (Table 2). In Tables 1 and 2, the shaded areas represent the www.giejournal.org

points at which the average scores surpass the MCC as described later. By 275 procedures, all averages surpass the threshold. By 300 procedures, even the low end of the 95% confidence interval for each metric passes this mark. It should be noted that at 400 procedures, this low end of the 95% confidence interval does dip below the threshold in a select few parameters primarily because of the low number of evaluation forms completed at this stage of training as a result of fellows being primarily engaged in training in the therapeutic endoscopy suites where the forms are not included in this analysis. The majority of each fellow’s scores were still above the threshold at this point. The overall cognitive and hands-on competence scores are plotted against the number of procedures to demonstrate the learning curves (Fig. 2). Finally, the independent cecal intubation rates and cecal intubation times are separately plotted at each stage of training (Figs. 3 and 4, respectively). In each of these tables and figures, the number represents the number of procedures that were performed in Mayo’s routine endoscopy areas and had evaluation forms completed by the supervising staff. Volume 74, No. 2 : 2011 GASTROINTESTINAL ENDOSCOPY 359

Training to competency in colonoscopy

Sedlack

TABLE 2. Individual cognitive skill scores

Indications/issues, average score (no., 95% CI)

Initial sedation, average score (no., 95% CI)

Pain management, average score (no., 95% CI)

Pathology recognition, average score (no., 95% CI)

Landmark recognition, average score (no., 95% CI)

First 10 3.05 (149, 2.90-3.21) 2.71 (143, 2.60-2.84) 2.38 (137, 2.23-2.53) 2.21 (135, 2.06-2.37)

Average combined Overall cognitive individual competence, cognitive scores, Tool selection/ average score average score settings, average (no., 95% CI) (no., 95% CI) score (no., 95% CI)

2.14 (79, 1.96-2.32)

2.76 (41, 2.47-3.05)

2.57 (150, 2.45-2.69) 1.83 (138, 1.71-1.96)

25

3.23 (292, 3.13-3.33) 2.92 (292, 2.83-3.02) 2.63 (283, 2.54-2.72) 2.62 (275, 2.52-2.72) 2.49 (162, 2.37-2.62)

3.12 (94, 2.94-3.29)

2.83 (297, 2.75-2.90) 2.26 (284, 2.17-2.36)

50

3.22 (291, 3.12-3.32) 2.92 (283, 2.82-3.01) 2.68 (284, 2.59-2.77) 2.73 (272, 2.64-2.82) 2.69 (192, 2.57-2.80) 3.19 (108, 3.02-3.36) 2.88 (295, 2.81-2.96) 2.41 (285, 2.32-2.50)

75

3.46 (309, 3.39-3.53) 3.20 (308, 3.12-3.29) 3.03 (300, 2.95-3.11) 3.10 (287, 3.01-3.18) 3.00 (192, 2.88-3.12) 3.42 (119, 3.29-3.55) 3.19 (309, 3.12-3.25) 2.81 (302, 2.72-2.90)

100

3.51 (245, 3.42-3.59) 3.28 (242, 3.19-3.36) 3.14 (236, 3.05-3.24) 3.25 (233, 3.17-3.34) 3.19 (161, 3.08-3.29) 3.60 (103, 3.47-3.74) 3.30 (246, 3.23-3.38) 2.90 (236, 2.91-2.99)

125

3.56 (239, 3.49-3.64) 3.38 (236, 3.29-3.46) 3.31 (232, 3.22-3.39) 3.36 (232, 3.27-3.45) 3.29 (153, 3.19-3.40)

150

3.64 (151, 3.56-3.73) 3.45 (152, 3.36-3.55) 3.27 (151, 3.17-3.38) 3.44 (148, 3.34-3.54)

175

3.76 (139, 3.68-3.84) 3.60 (138, 3.50-3.70) 3.42 (139, 3.31-3.54) 3.64 (137, 3.54-3.73)

200

3.63 (179, 3.55-3.71) 3.53 (175, 3.44-3.61) 3.41 (176, 3.32-3.50) 3.59 (171, 3.51-3.67) 3.52 (127, 3.41-3.63) 3.72 (082, 3.60-3.84) 3.54 (180, 3.47-3.61) 3.43 (178, 3.34-3.53)

225

3.69 (134, 3.61-3.78) 3.54 (132, 3.43-3.64) 3.42 (130, 3.31-3.52) 3.64 (127, 3.55-3.73)

3.52 (96, 3.41-3.63)

3.75 (57, 3.59-3.92)

3.57 (135, 3.49-3.64) 3.33 (130, 3.22-3.44)

250

3.71 (139, 3.64-3.79) 3.69 (137, 3.61-3.78) 3.50 (136, 3.39-3.61) 3.63 (136, 3.54-3.71) 3.56 (102, 3.45-3.67)

3.90 (51, 3.82-3.99)

3.64 (140, 3.57-3.71) 3.51 (135, 3.41-3.61)

275

3.83 (164, 3.77-389) 3.64 (163, 3.56-3.73) 3.50 (157, 3.34-3.57) 3.74 (154, 3.66-3.82) 3.61 (109, 3.49-3.72)

3.61 (80, 3.45-3.78)

3.66 (163, 3.60-3.73) 3.57 (160, 3.47-3.67)

300

3.92 (130, 3.87-3.96) 3.83 (128, 3.76-3.89) 3.74 (129, 3.66-3.82) 3.88 (126, 3.82-3.94)

3.88 (88, 3.80-3.95)

3.96 (55, 3.91-4.00)

3.85 (130, 3.80-3.89) 3.91 (128, 3.86-3.97)

325

3.83 (187, 3.77-3.89) 3.72 (184, 3.65-3.79) 3.67 (181, 3.59-3.74) 3.78 (182, 3.71-3.85) 3.68 (140, 3.59-3.77)

3.90 (81, 3.83-3.98)

3.75 (187, 3.69-3.80) 3.79 (183, 3.73-3.85)

350

3.83 (142, 3.77-3.89) 3.70 (140, 3.62-3.78) 3.61 (137, 3.51-3.71) 3.84 (136, 3.77-3.90) 3.68 (100, 3.58-3.78)

3.75 (55, 3.56-3.93)

3.74 (142, 3.67-3.80) 3.73 (134, 3.65-3.81)

375

3.90 (139, 3.85-3.95) 3.79 (138, 3.72-3.86) 3.66 (140, 3.57-3.75) 3.87 (138, 3.81-3.93) 3.79 (104, 3.71-3.87)

3.85 (65, 3.74-3.96)

3.80 (140, 3.75-3.85) 3.84 (139, 3.78-3.90)

400

3.74 (31, 3.58-3.91)

3.94 (18, 3.82-4.00)

3.70 (31, 3.55-3.86)

3.70 (30, 3.53-3.87)

3.61 (28, 3.41-3.80)

3.73 (30, 3.57-3.90)

3.55 (94, 3.41-3.70)

3.40 (240, 3.33-3.47) 3.24 (231, 3.16-3.32)

3.32 (90, 3.19-3.45)

3.64 (61, 3.49-3.79)

3.45 (155, 3.38-3.53) 3.20 (153, 3.09-3.32)

3.51 (96, 3.39-3.63)

3.75 (057, 3.61-3.90) 3.60 (139, 3.52-3.67) 3.42 (130, 3.30-3.53)

3.63 (27, 3.43-3.82)

3.90 (30, 3.79-4.00)

The average Mayo Colonoscopy Skills Assessment Tool score for each of the core cognitive skill parameters is shown at sequential steps of every 25 procedures. These averages are based on a 1- to 4-point scoring scale where 1 to 3 represents progression toward competence and 4 represents the achievement of competence. Minimal competence criteria are defined as an average score of 3.5 or greater. Shaded cells indicate the average score for this parameter was reached and is subsequently maintained above the minimal competency threshold score of 3.5.

Figure 2. Overall skills learning curves. The learning curve of average MCSAT scores for overall motor (solid line) and overall cognitive (dashed line) skills are shown. Average scores are labeled with error bars showing the 95% confidence interval. By 250 procedures, cognitive skills have achieved the 3.5 MCC goals, with motor scores reaching this by 275 procedures. By 300 procedures, all scores and the 95% confidence interval are above this threshold.

Outcome data The MCC score required for each of the core parameters was calculated by using the contrasting groups’ method of standard setting. An example of the intersecting distribu360 GASTROINTESTINAL ENDOSCOPY Volume 74, No. 2 : 2011

tion curves for the parameter of loop reduction is shown (Fig. 1). In this figure, the dashed line shows the distribution of loop reduction scores for individuals who received a score of 1, 2, or 3 (ie, not competent) on the overall www.giejournal.org

Sedlack

Training to competency in colonoscopy

Figure 3. Cecal intubation rate learning curves. The learning curve for average cecal intubation rates is shown. (Average score is labeled. Error bars represent the 95% confidence interval). Independent cecal intubation rates reach the MCC of 85% at 250 to 275 procedures.

Figure 4. Cecal intubation time learning curves. The learning curve for average cecal intubation times is shown. (Average score is labeled. Error bars represent the 95% confidence interval). Independent cecal intubation times reach the MCC of less than 16 minutes at 275 procedures.

motor competence score. The solid line shows the loop reduction score distribution curve for trainees deemed competent to operate independently (ie, overall motor competence score of 4 for that procedure). As is demonstrated by the dashed line at the far right of this figure, it is possible for some procedures to achieve a score of 4 for this core skill yet be part of the noncompetent curve because of receiving a score of 1, 2, or 3 for the overall motor competence score. The reverse is also true, resulting in the competent curve (solid line) trailing off to the left. These curves represent a best statistical fit of the data distribution using probability density functions that, by www.giejournal.org

definition, it is possible for the curve to marginally exceed a value of 1.0 when data variance is small (as is seen in competent curve where only scores of 3 and 4 were recorded).8 The point along the x axis where these 2 curves intercept establishes the minimal competency score for this loop reduction parameter. For this parameter, the intersection correlates to a loop reduction score of 3.40. The MCC scores for each of the individual core skills is calculated and shown by using this same method (Table 3). These range from average scores of 3.35 for mucosal visualization to 3.5 for initial sedation; however, most are in the range of 3.4 to 3.5. Volume 74, No. 2 : 2011 GASTROINTESTINAL ENDOSCOPY 361

Training to competency in colonoscopy

Sedlack

TABLE 3. Minimal competence scores Contrasting group minimal passing score

Average no. of procedures to reach contrasting group threshold (95% CI)

Average no. of procedures to reach score of 3.5 (95% CI)

Hands-on participation

3.38

250 (150-275)

275 (250-300)

Safe colonoscope advancement

3.43

175 (175-250)

250 (250-250)

Loop reduction

3.40

250 (175-250)

250 (250-300)

Mucosal visualization

3.35

125 (125-175)

175 (175-225)

Therapeutic maneuver

3.41

100 (100-225)

175 (100-250)

Average of combined individual motor scores

3.48

250 (175-275)

250 (175-275)

Overall hands-on competence

3.50

275 (250-300)

275 (250-300)

Indications/issues

3.43

75 (75-125)

100 (75-150)

Initial sedation

3.50

175 (150-250)

175 (150-250)

Pain management

3.47

250 (175-300)

250 (175-300)

Landmark recognition

3.43

150 (125-175)

175 (150-175)

Pathology recognition

3.47

175 (175-275)

175 (175-300)

Tool selection/settings

3.46

100 (75-150)

100 (75-175)

Average of combined individual cognitive scores

3.50

175 (150-250)

175 (150-250)

Overall cognitive competence

3.50

250 (250-300)

250 (250-300)

Motor scores

Cognitive scores

With the contrasting groups method, the minimal score required to be deemed competent is shown in column 1 for each parameter. The number of procedures required by the average fellow to first surpass (and remain above) these contrasting groups threshold scores are shown in the second column along with the range of procedure numbers required based on the upper and lower confidence intervals shown in parentheses. Procedure numbers required to achieve and maintain a score of 3.5 or greater in each parameter is also shown. A total of 275 procedures are required for the average performance scores to meet all these limits, and by 300 procedures, all scores (including their 95% CIs) surpass the minimum competence threshold of 3.5.

The stage of training at which the respective MCC score is first reached and remains consistently above this threshold is also shown in Table 3. In parameters such as knowledge of the indication and medical issues, these minimal competency thresholds are reached as early as 75 procedures. Most, however, are not reached until approximately 250 to 275 procedures. These numbers represent the stage of training that the average trainee requires to reach this threshold. The 95% confidence intervals in this table represent the number of procedures at which the top 5% of scores reach the threshold and when 95% of scores surpassed the MCC threshold. As stated previously, by 3.5, all parameters were at, or exceeded, the MCC threshold. To simplify scoring but still ensure that all parameters fall within the competency criteria established, using a conservative score of 3.5 for all the MCC parameters is recommended. The number of procedures required to reach an average score of 3.5 for 362 GASTROINTESTINAL ENDOSCOPY Volume 74, No. 2 : 2011

each parameter is also shown in Table 3 with similar confidence intervals. Examination of the contrasting groups results for time to reach the cecum reveals the intercept of the competent and noncompetent curves to be at 16.38 minutes. Applying this cutoff to the learning curves seen in Figure 4, this cutoff is achieved consistently starting at 250 procedures. By conservatively rounding this down to 16 minutes, this MCC threshold is reached on average by 275 procedures.

Other analyses The mean scores for the 50th, 150th, 275th, and 400th procedure stages are compared for cecal intubation rates, intubation times, overall hands-on, and overall cognitive competence (Table 4). There is a measurable difference (P ⬍ .05) for each of these parameter scores among the 50-, 150-, and 275-procedure groups. As expected, learning curves continue upward when measured at the 400th procedure, but this improvement only reaches statistical www.giejournal.org

Sedlack

Training to competency in colonoscopy

TABLE 4. Performance at various recommended training durations No. of procedures 50

150

275

400

P value

Cecal intubation rates

51.0% (279, 0.50)

76.0% (150, 0.43)

85.0% (152, 0.36)

90.0%* (29, 0.31)

⬍.05

Cecal intubation times

20.9 (454, 9.55)

17.5 (255, 8.38)

15.4 (201, 9.73)

14.7* (59, 9.48)

⬍.05

Overall hands-on competence

2.18 (292, 0.75)

3.02 (150, 0.74)

3.81 (128, 0.39)

3.87* (30, 0.95)

⬍.05

Overall cognitive competence

2.41 (285, 0.75)

3.20 (153, 0.72)

3.57 (160, 0.64)

3.90 (30, 0.31)

⬍.05

Shown is a comparison of average performance scores (no., standard deviation) at 3 suggested levels of training experience and beyond using a 2-tailed t test (P ⬍ .05). General surgery training guidelines suggest 50 procedures be completed for graduation, gastroenterology fellowships require 140 procedures before competency can be assessed, and this study suggests 275 procedures are required to achieve defined competency benchmarks. There was significant improvement in the scores among each of the first 3 stages of the learning curve (P ⬍ .05). There was continued improvement between 275 and 400 procedures, but the differences does not reach significance for 3 of the 4 parameters examined, suggesting scores have begun to plateau between these 2 stages of training. *Not significant (NS) for the 400-procedure group.

TABLE 5. Summary of minimal competency criteria for colonoscopy Parameter Independent cecal intubation rate Cecal intubation time Average MCSAT score for each skill

MCC ⱖ85% ⬎16 min ⱖ3.5

These numbers represent the recommendations for minimal colonoscopy skills that are required to be deemed minimally competent to operate independently. Average performance scores taken at periodic training intervals should meet or surpass these thresholds and scores maintained above these limits to be deemed competent. On average, all these limits are achieved by experience performing approximately 275 colonoscopies. MCC, Minimal competency criteria; MCSAT, Mayo Colonoscopy Skills Assessment Tool.

significance in 1 of the 4 parameters examined (overall cognitive competence) compared with the 275-procedure score.

DISCUSSION Professional gastroenterology societies recommend that fellows perform a minimum of 140 colonoscopies before competency can be assessed, whereas general surgery and family practice residency guidelines require only 50 procedures during training.2,9,10 Alternate privileging pathways have also been supported.11,12 Recent data suggest that even the most rigorous of these training recommendations (140 procedures) is likely insufficient to achieve competence in trainees, instead placing the competency bar anywhere from 175 to 500 procedures.13-15 However, each of these reports is based on limited surrogate competency endpoints such as cecal intubation rates www.giejournal.org

and cecal intubation times. Defining more direct competency endpoints has proven difficult primarily because of the lack of any objective means to assess the core skills of the procedure and is compounded by the substantial variation in the literature on where to set the bar. As a result, it is clear how such differences in training expectations between GI and other training programs can exist. Instead, a set of objective standards is needed to ensure that all physicians being trained and credentialed to perform colonoscopy, regardless of their specialty or method of training, are held to the same standards of quality and competency for the sake of patient safety. The MCSAT was created and has been validated specifically to define the spectrum of core cognitive and motor skills necessary to perform colonoscopy and provide an objective means to measure these.5 The data collected with the MCSAT over this 3-year study period demonstrate the normal learning curves for each of the skills on the MCSAT (Tables 1 and 2, respectively). As the data demonstrate, there is a steep initial learning curve, but these scores begin to plateau after 300 procedures. However, in the absence of a set of defined minimal competency thresholds, these curves tell us little except that a significant amount of learning continues well past current training recommendations of 140 procedures. By using the contrasting groups standard-setting process, MCC are defined (Table 3). The results suggest that average scores ranging from 3.35 to 3.5 are required (on a 4-point scale) to achieve minimal competence in the individual core skills. To simplify the assessment process for practical application yet ensure all parameter scores fall at or above their minimal competency cutoff, it is suggested that a conservative MCSAT item average score of 3.5 be used as the MCC for each core skill (Table 5). In contrast to these individual core skills, the scores of overall cognitive and overall hands-on skills define the Volume 74, No. 2 : 2011 GASTROINTESTINAL ENDOSCOPY 363

Training to competency in colonoscopy

contrasting groups by which the other individual core skills scores are plotted and hence cannot be plotted against themselves in a similar fashion. Instead, for this pair of overall competency parameters, a minimal threshold score of 3.5 is suggested, based in part on the information provided previously and the logical fact that it is above this average score at which a majority of the scores in each of these parameters are graded as competent (ie, at least half or more of the scores must be 4 to achieve and average of ⱖ3.5) to operate independently. Applying these MCC thresholds to the average learning curves, most core skills require 175 to 275 colonoscopies to reach and maintain scores above this threshold. The more basic skills, such as understanding the procedure indications and medical issues applicable to a given procedure and selecting tools such as a cold biopsy cable versus a snare, are reached as early as 100 procedures. By 275 procedures, however, all of the parameters, including the overall cognitive and hands-on competence, have reached the minimal competency threshold, suggesting that the average trainee requires this many colonoscopies to achieve minimal competence in all the required skills. This is not to say that some individuals cannot achieve competence sooner or require additional training to reach these marks. To examine how these benchmarks compare with traditional surrogate competency markers of cecal intubation rates and times, it is shown that at 275 procedures, fellows achieve average cecal intubation rates of 85% and intubation times of less than 16 minutes. These correlate well with the thresholds used by other authors. Previously, professional recommendations have identified cecal intubation rates of 80% to 90% as endpoints for competence.9 These numbers were based originally on data presented by Cass et al16 in the early 1990s when they reported that experienced endoscopists generally achieve a 94% cecal intubation rate, whereas trainees achieve an approximately 84% rate. Chak et al17 reported similar numbers for the same 2 experience levels. Based on these data, however, both reports subjectively selected to set the minimal competence bar at 90% without further formal testing of the accuracy of this benchmark. As a result, 90% has since generally been quoted as the cutoff for competency. Clearly one would expect the number to rest somewhere between 84% and 90%. Based on the data presented here, an 85% independent cecal intubation rate is more directly associated with the point at which staff independently deem fellows to have achieved minimal competence. Like the other MCC, this benchmark of 85% is achieved at 250 to 275 procedures by the average gastroenterology trainee. Published cecal intubation time benchmarks for minimal competency have ranged from 15 to 20 minutes.13,17 By using the contrasting groups method, a cecal intubation time of 16.38 minutes is demonstrated to be the threshold between those procedures in which trainees are deemed competent from those that are not. After simplifying this 364 GASTROINTESTINAL ENDOSCOPY Volume 74, No. 2 : 2011

Sedlack

metric by rounding this down to a slightly more conservative cutoff of 16 minutes or less, this threshold occurs at 275 procedures. This result supports these previously published estimates and coincides with the other MCC data thresholds. A separate examination of the MCSAT scores at different stages of training confirms that there is a marked difference in average scores among trainees with 50, 150, and 275 procedures. Overall competence scores, cecal intubation rates, and times obtained at this 275 stage of training show a significant improvement over the scores of the other 2 groups, suggesting that there is a measurable difference in skill as a result of this additional training. By 400 procedures, only 1 of these metrics has any measurable difference with the 275-procedure group. These comparisons suggest that the learning curve is still quite steep up to 275 procedures yet begins to level off beyond this point. It is important to note that none of the core skills were mastered as early as 50 procedures, not even by the most advanced trainees, suggesting that the current volumes that surgical and family practice residents receive is not sufficient to allow competent independent performance of colonoscopy, and such exposure should not be used as grounds for privileging in this procedure. Even achieving 275 procedures is not intended to imply competence or to replace objective skills assessment; rather, it simply is a point at which the average trainee is likely to meet the MCC benchmarks on independent objective competency assessment. This study was not specifically designed to examine the percentage of fellows reaching these thresholds at each stage of training; however, the data do yield some useful information based on our experience with this group of trainees. Assessing each of the participants’ scores, the percentage of fellows who have achieved competence (those who consistently score above the MCC) is calculated at each stage of training (Fig. 5). We find that no fellow achieved competency before 200 procedures, yet by 275 procedures, 50% have surpassed these marks. It is not until approximately 400 procedures that the competence rate for this group of trainees is 100%. A larger, multicentered trial designed to examine this aspect specifically would be needed to provide a more precise competency curve. Some limitations of this study are observed. First is the fact that this study is limited to a single institution, which makes the generalizability of the learning curve data difficult. Despite this, it is reassuring that the competency endpoints for cecal intubation rates and times are within the range of previously published data from other institutions. One exception to this is attributed to the fact that it is standard practice at the Mayo Clinic for each fellow to perform approximately 20 computer-simulated colonoscopy procedures before beginning patient-based training. Previous reports have shown that any benefit of this early simulation training curriculum is brief.18 As such, the data www.giejournal.org

Sedlack

Training to competency in colonoscopy

Figure 5. This graph shows the percentage of fellows achieving competence defined by consistently scoring greater than the MCC defined in Table 5.

collected during the first 10 procedures may be higher than might be seen in fellows who do not have simulation training, but at all other stages it is expected to be unchanged regardless of the addition of simulation training. A large multicenter trial using the MCSAT form would be recommended to prove the generalizability of these learning curves and competency benchmarks. Second, staff were not blinded to the fellows’ year of training, which could lead to some scoring bias. Despite this, staff had no knowledge of how many procedures that the fellow had performed at any given point. A future multicentered trial could control for this possible bias, although this would likely require that fellow performance evaluations be completed by reviewers outside the training program to ensure reviewers are not familiar with the trainees. This would add considerable time and expense to such a project. Third, as is inherent to educational survey data like these, staff compliance of 62% with completing the MCSAT form does add a potential bias to the results. By limiting the use of the MCSAT form to periodic spot checks of skill, it is anticipated that compliance would be higher than is seen in this 3-year study in which continuous assessment was required to establish learning curves. Another potential factor is the effect of an individual staff member’s teaching practices (ie, how quickly he or she takes the colonoscope away from the trainee, sedation tolerances). In this study, we had more than 50 different teaching staff, providing a likely broad variety of teaching tolerances, suggesting some element of generalizability from the standpoint of who is completing the MCSAT form. Finally, these learning curve data are valid distinctly for gastroenterology fellows www.giejournal.org

as graded by gastroenterology staff. Presumably these endpoints would extrapolate to general surgeon and family practice trainees as well; however, a formal study of this is needed. With such a study, a strong emphasis should be placed on having the MCSAT form completed only by very experienced teaching endoscopists (ie, an experienced staff gastroenterologist grading general surgery or family practice residents) to ensure that the same standards are used in assessing skill. The completion of the MCSAT form by someone of lesser experience may inflate the scores of the trainee, suggesting that learning curves are faster than they really are and that the competency threshold is met despite the trainee not fully acquiring the requisite skills. Potential uses of this form and learning curves outside of those defined here are numerous. It is clear that fellows acquire skills at different rates, but it is also not uncommon for some to significantly struggle with endoscopy skills. The use of this form and learning curves established here can help identify fellows who excel at endoscopy or more importantly identifying those falling off the learning curves much earlier and allow for more timely intervention/ remediation. In the event that a fellow still cannot adequately acquire the requisite skills, the data from a continuous assessment process such as this can provide evidence supporting the potential need for an unfortunate removal of a fellow from training. Another potential use could be using MCSAT assessment for credentialing of new staff. An institution could use the form to assess a new endoscopist’s abilities before granting colonoscopy privileges. Alternatively, it could be used in assessing existing staff who have not performed endoscopy for extended Volume 74, No. 2 : 2011 GASTROINTESTINAL ENDOSCOPY 365

Training to competency in colonoscopy

periods of time or who have small practice volumes that might not allow adequate maintenance of skills. Having these individuals undergo a period during which they are precepted by experienced staff and their skills assessed by using the MCSAT form could identify those who are ready to operate independently and those who may need a longer period in the precepted environment before privileges are granted. This ability to assess the core skills also opens up other research opportunities such as answering the question of how many procedures are required annually to maintain minimal endoscopic skills or the ability to determine what educational interventions might accelerate the motor or cognitive learning curves in trainees. The MCSAT provides a means to ensure that physicians have the necessary skills to perform colonoscopy. Not surprisingly, this requires approximately 275 procedures of training for the average trainee to acquire the core skills and achieve the MCC goals established here. As many endoscopy educators have long suspected and more recent publications have suggested, this exceeds current gastroenterology training recommendations and far surpasses training requirements of alternate endoscopy training routes. To ensure patient safety, all physicians seeking to gain requisite skills to be privileged in colonoscopy, regardless of their specialty, should be held to a common standard of competency metrics to ensure patient safety and provide the highest quality of care. Similarly, all professional societies, regulating bodies, and training programs alike should ensure not only that adequate procedure volumes are available to accomplish this but that ongoing assessment of a trainee’s core skills be recorded to ensure these benchmarks are being met.

Sedlack

2.

3.

4. 5.

6.

7.

8.

9. 10.

11. 12.

13.

14.

ACKNOWLEDGMENTS 15.

A special thanks to all the colonoscopy teaching staff at the Mayo Clinic for their tireless effort in completing the MCSAT forms during this study period and beyond. Without their efforts, this study would not have been possible.

16. 17.

REFERENCES 18. 1. Accreditation Council for Graduate Medical Education. 2009-2010 Data Resource Book. ACGME Web site. Available at: http://www.acgme.org/

366 GASTROINTESTINAL ENDOSCOPY Volume 74, No. 2 : 2011

acWebsite/dataBook/2009-2010_ACGME_Data_Resource_Book.pdf. Accessed October 18, 2010. American Academy of Family Practitioners Web site. AAFP Colonoscopy Position Paper. Available at: http://www.aafp.org/online/en/home/ policy/policies/c/colonoscopypositionpaper.html. Accessed October 18, 2010. Cass OW. Training to competence in gastrointestinal endoscopy: a plea for continuous measuring of objective end points. Endoscopy 1999;31: 702-6. Bjorkman DJ, Popp JW. Measuring the quality of endoscopy. Gastrointest Endosc 2006;63:S1-2. Sedlack RE. The Mayo Colonoscopy Skills Assessment Tool: validation of a unique instrument to assess colonoscopy skills in trainees. Gastrointest Endosc 2010;72:1125-33. Yudkowsky R, Downing SM, Tekian A. Standard setting. In: Downing SM, Yudkowsky R, editors. Assessment in health professions education. New York (NY): Routledge; 2009. p. 119-48. Downing SM, Tekian A, Yudkowsky R. Procedures for establishing defensible absolute passing scores on performance examinations in health professions education. Teaching Learning Med 2006;18: 50-7. National Institute of Standards and Technology. What is a probability distribution. In: Engineering statistics handbook. Available at: http:// www.itl.nist.gov/div898/handbook/eda/section3/eda361.htm. Accessed February 16, 2011. ASGE Standards of Training Committees. Principles of Training in Gastrointestinal Endoscopy. Gastrointest Endosc 1999;49:845-50. ACGME Web Site. Surgery Policy Information. Available at: http:// www.acgme.org/acWebsite/RRC_440/440_policyArchive.asp. Accessed October 18, 2010. Joint Statement. Principles of privileging and credentialing for endoscopy and colonoscopy. Gastrointest Endosc 2002;55:145-8. ASGE Standards of Practice Committee.Methods of granting hospital privileges to perform gastrointestinal endoscopy. Gastrointest Endosc 2002;55:780-3. Spier BJ, Benson M, Pfau PR, et al. Colonoscopy training in gastroenterology fellowships: determining competence. Gastrointest Endosc 2010; 71:319-24. Chung JI, Kim N, Um MS, et al. Learning curves for colonoscopy: a prospective evaluation of gastroenterology fellows at a single center. Gut Liver 2010;4:31-5. Suck-Ho L, Chun, IK, Kim SJ, et al. An adequate level of training for technical competence in screening and diagnostic colonoscopy: a prospective multicenter evaluation of the learning curve. Gastrointest Endosc 2008;67:683-9. Cass OW, Freeman ML, Peine CJ, et al. Objective evaluation of endoscopy skills during training. Ann Intern Med 1993;118:40-4. Chak A, Cooper GS, Blades EW. Prospective assessment of colonoscopic intubation skills in trainees. Gastrointest Endosc 1996;44:54-7. Sedlack RE, Kolars JC. Computer simulator training enhances the competency of GI fellows at colonoscopy: results of a pilot study. Am J Gastroenterol 2004;99:33-7.

www.giejournal.org

Sedlack

Training to competency in colonoscopy

Appendix 1 Mayo Colonoscopy Skills Assessment Tool Date: Fellow: Staff: Time of Intubation: Time of Maximal Insertion Extent: Time of Extubation: Pre-Procedure: 1.

Fellow’s knowledge of the Indication & Pertinent Medical Issues (INR, Vitals, Allergies, PMH etc.) □ NA- fellow observed; □ 1-Novice (Poor knowledge of patient’s issue, or started sedating without knowing the indication); □ 2-Intermediate (Missed an Important element, i.e. Allergies, GI Surgical History or INR in pt on Coumadin); □ 3-Advanced (Missed minor elements); □ 4-Superior(Appropriate knowledge and integration of patient information). 2. Use of Initial Sedation during this procedure: □ N/A- Fellow Observed, □ 1-Novice (Little attention to titration or sedation adequacy before beginning procedure). □ 2-Intermediate (Initial titration was somewhat too rapid or slow, or too light/heavy). □ 3-Advanced (Initial sedation was appropriate.) □ 4-Superior (Fellow independently monitors vitals, Titration to achieve ideal sedation before beginning procedure.) Procedure Skills: 3. Fellows participation in this procedure □ N/A - Observed only, □ 1-Novice – (Performed with significant hands-on assistance) □ 2-Intermediate – (Performed with significant coaching or limited hands-on assistance) □ 3-Advanced (Performed Independently with limited coaching), □ 4-Superior (Performed Independently without coaching). 4. What is the farthest landmark the fellow reached without any hands-on assistance: □ N/A - fellow observed only or Procedure terminated before completion. □ 1-Rectum, □ 2-Sigmoid, □ 3-Splenic flexure, □ 4-hepatic flexure, □ 5-Cecum No TI attempt (completed cecal intubation without hands-on assistance and no attempt at TI) □ 5-Cecum Failed TI attempt (completed cecal intubation without hands-on assistance and Failed attempt at TI) □ 6-Terminal Ileum (Successful intubation of TI) □ 9- Other-Post surgical anatomy encountered, fellow reached maximal intubation. 5. Safe Endoscope Advancement techniques: □ N/A, Fellow observed; □ 1-Novice (Pushes blindly or Against fixed resistance/requires significant hands-on assistance); □ 2-Intermediate (Slow advancement/Repeated red-out/needs considerable coaching); □ 3-Advanced (Able to keep lumen in center and advance at a reasonable pace, limited coaching); □ 4-Superior (Safe technique, expedient independent advancement without the need for coaching) 6. Loop Reduction Techniques (Pull-back, External pressure, Patient Position Change) □ N/A, Fellow observed; □ 1-Novice (Unable to reduce/avoid loops without hands-on assistance); □ 2-Intermediate (Needs considerable coaching on when or how to perform loop reduction maneuvers); □ 3-Advanced (Able to reduce/avoid loops with limited coaching); □ 4- Superior.(without coaching, uses appropriate ext. pressure/position changes/loop reduction techniques). 7. Monitoring and management of patient Discomfort during this procedure: □ N/A Fellow observed; □ 1-Novice (Does not quickly recognize patient discomfort or requires repeated staff prompting to act); □ 2-Intermediate (Recognizes pain but does not address loop or sedation problems in a timely manner); www.giejournal.org

Volume 74, No. 2 : 2011

GASTROINTESTINAL ENDOSCOPY

366.e1

Training to competency in colonoscopy

8.

9.

10.

11.

12.

13.

Sedlack

□ 3-Advanced (Adequate recognition and correction measures); □ 4-Superior (Proactive assessment and management. i.e. intermittently talks to patient to assess sedation and comfort). Landmark Recognition/Localization of Instrument □ NA/Not Assessed □ 1-Novice (Generally unable to recognize most landmarks); □ 2-Intermediate (Recognizes some landmarks but generally poor perception of Instrument/pathology location) □ 3-Advanced (Recognizes major landmarks; Can broadly localize instrument/pathology based on landmarks); □ 4-Superior.(Able to recognize landmarks and has clear idea of instrument/pathology location in relation to landmarks) Adequately visualized mucosa during withdrawal: □ NA/Fellow Observed Withdrawal; □ 1-Novice (red out much of the time, does not visualize significant portions of the mucosa or requires assistance); □ 2-Intermediate (Able to Visualize much of the mucosa but requires direction to re-inspect missed areas) □ 3-Advanced (Able to adequately visualize most of the mucosa without coaching); □ 4-Superior (Good visualization around difficult corners and folds and good use of suction/cleaning techniques.) Pathology identification/interpretation: □ N/A, Study was normal; □ 1-Novice - Poor recognition of abnormalities (Misses or cannot ID significant pathology); □ 2-Intermediate - Recognize abnormal findings but cannot interpret (“ erythema”). □ 3-Advanced - Recognizes abnormalities and correctly interprets (“colitis”); □ 4-Superior- competent Identification and assessment (“Mild chronic appearing colitis in a pattern suggestive of UC”). Interventions Performed by fellow: CHECK ALL THAT APPLY □ N/A – Fellow did not perform any interventions (go to question 12) □ Biopsy □ Snare polypectomy □ Complex Polypectomy □ Other ___________ 11a. Therapeutic tool/cautery setting selection □ 1-Novice - Unsure of the possible tool(s) indicated for pathology. □ 2-Intermediate - Able to identify possible appropriate tool choices but not sure which would be ideal □ 3-Advanced - Independently selects the correct tool yet needs coaching on settings □ 4-Superior - Independently identifies correct tool and settings as applicable. 11b. What was the fellows participation in the therapeutic maneuver □ 1-Novice - Performed with significant hands-on assistance, □ 2-Intermediate - Performed with minor hands-on assistance or significant coaching, □ 3-Advanced - Performed Independently with minor coaching, □ 4-Superior - Performed Independently without coaching. Overall Assessment: The fellows Hands-on skills are equivalent to those of a: □ 1-Novice (Learning basic scope advancement; requires significant assistance and coaching); □ 2-Intermediate (Acquired basic motor skills but still requires limited hands-on assistance and/or significant coaching) □ 3-Advanced (Able to perform independently with limited coaching and/or requires additional time to complete) □ 4-Superior (Competent to perform routine colonoscopy independently) The fellow’s Cognitive Skills (Situational Awareness (SA)/Abnormality interpretation/decision making skills) are: □ 1-Novice (Needs significant prompting, correction or basic instruction by staff) □ 2-Intermediate (Needs intermittent coaching or correction by staff) □ 3-Advanced (Fellow has good SA, and interpretation/decision making skills) □ 4-Superior (Competent to make decisions and interpretations independently)

366.e2 GASTROINTESTINAL ENDOSCOPY Volume 74, No. 2 : 2011

www.giejournal.org