- Email: [email protected]
Training resident physicians in fiberoptic sigmoidoscopy. How many supervised examinations are required to achieve competence?
SPECIAL ARTICLES
Training Resident Physicians in Fiberoptic Sigmoidoscopy How Many Supervised Examinations Are Required to Achieve Competence?
ROBERT HAWES, M.D. GLEN A. LEHMAN, M.D. JOHN HAST, M.D. KATHERINE W. O’CONNOR, M.D. DAVID W. CRABB, M.D. ALEC LUI, M.D. PHILIP A. CHRISTIANSEN, M.D. Indianapolis,
Indiana
Twenty-five resident physicians performed 495 fiberoptic sigmoidoscopic examinations that were graded for overall skill according to a six-point competence scale. In general, 24 to 30 examinations were required to become competent at fiberoptic sigmoidoscopy. Trainees with prior rigid sigmoidoscopy experience achieved competence more quickly than those with no prior rigid sigmoktoscopy experience. As experience increased, unassisted insertion distance and luminal visualization increased, insertion time and assisted time decreased, and management scores and percent correct diagnoses improved. Trainees detected 93 to 100 percent of polyps and cancers viewed by the experienced sigmoidoscopist once competence was achieved. These data indicate that programs for training primary care physicians in fiberoptic sigmoktoscopy are feasible, help define the number of examinations required to become competent, and indicate that such trainees should be effective in cancer screening. The superiority of fiberoptic sigmoidoscopy over rigid sigmoidoscopy in screening for colorectal disease has been clearly demonstrated [1,2]. One obstacle encountered in effecting the use of fiberoptic sigmoidoscopy is related to difficulty in providing training for primary care physicians in its use [3]. Training programs for internal medicine, family practice, and general surgery are promoting instruction in fiberoptic sigmoidoscopy. Therefore it is important to quantitate the training required to become competent at this procedure. It was toward this end that this study was undertaken.
METHODS
From the Division of Gastroenterology, Department of Medicine, Indiana University School of Medicine, Indianapolis, Indiana. Requests for reprints should be addressed to Dr. Glen A. Lehman, Regenstrief Health Center, Room 614B, 1001 West Tenth Street, Indianapolis, Indiana 46202. Manuscript accepted March 7, 1985.
A total of 25 residents in general medicine, surgery, and family practice participated in the fiberoptic sigmoidoscopy training program during one- to two-month rotations on the gastroenterology medicine service. The trainees had variable rigid sigmoidoscopy experience but no previous experience with fiberoptic sigmoidoscopy. Each trainee received baseline training consisting of written materials on fiberoptic sigmoidoscopic instrumentation, technique, complications, and diagnostic capabilities. Each resident was given individual instruction about the instrument by an experienced endoscopist. Additionally, all were required to spend at least two hours practicing on a rubber colon model (Olympus CM-l) and viewed several examinations via a teaching attachment. Following this, they began performing examinations on patients under the supervision of an experienced endoscopist. Each examination was begun by allowing the trainee to perform as much of a complete examination as possible within set limits of time, discomfort, and safety. Trainees were allowed 10 minutes for full
March
1986
The American
Journal
of Medicine
Volume
80
465
FIBEROPTIC
SIGMOIDOSCOPY
TABLE I Not 1. 2. 3.
TRAINING-HAWES
ET AL
Overall Competence Score
major and minor categories. True-positive, false-positive, and false-negative diagnoses were recorded. The treatment plan was scored according to the following: 1 = clearly correct; 2 = reasonable choice but not totally state-of-theart management; 3 = clearly suboptimal; 4 = wrong. Fiberoptic sigmoidoscopic examinations were performed in patients referred to the gastroenterology service for standard indications. Most examinations were performed with Olympus TCF-1S and lTS2 60 cm sigmoidoscopes. No shorter instruments were used. A one-way analysis of variance model was used to compare means for more than one group, and if a significant difference among the groups was observed at the 0.05 level, a Duncan’s multiple-range test was used to make all the pairwise comparisons. A two-sample t test, two-tailed, was used to compare means from two independent groups. A corrected chi-square test, or a two-tailed Fisher’s exact test when necessary, was used to compare percentages from two independent groups. All test statistics were calculated using the Statistical Analysis System computer program [4].
Competent Totally unskilled and possibly dangerous to patients. A complete novice Knows few basics but examinations slow and clearly incomplete Modest facility in instrument manipulation but lacks firm control of examinations (technical or interpretive)
Competent 4. 5. 6.
Good technical and interpretive skills with only occasional errors Skill comparable to average first-year gastrointestinal fellow with six to 12 months of endoscopy experience Skill comparable to average second-year gastrointestinal fellow or staff
The above overall individual fiberoptic above were judged tions.
competence scale was used to grade the sigmoidoscopy examinations. Grades 4 and competent to perform unsupervised examina-
RESULTS
insertion and 15 minutes for the total examination. The unassisted (Le., solo) portion of the examination was terminated prematurely if reasonable progress was not being attained, excessive discomfort occurred, or the staff believed that patient safety might be compromised. The staff viewed through the teaching attachment but gave no verbal or mechanical assistance with the exception of the first IO examinations, when advancing the scope under trainee command was permitted if progress was slow. During the solo effort, the trainees’ performance was evaluated and recorded for distance penetrated, insertion and withdrawal time, quadrants viewed on withdrawal, diagnoses, and patient management based on the fiberoptic sigmoidoscopic findings. If the trainee failed to perform a complete examination during the solo effort, the staff then actively assisted to assure an adequate examination of the patient. Only the solo effort was graded. In an effort to quantitate the percent of mucosal surface viewed, each IO cm of colon during withdrawal was divided into four quadrants. As the sigmoidoscope was withdrawn, the trainee was graded as to whether each of four quadrants in each 10 cm of withdrawal had been adequately visualized. The number of adequately viewed quadrants was divided by the number of total quadrants and expressed as a percentage. A six-point competence scale was developed (Table I). For each supervised examination, the trainee was assigned a competency score reflecting the skill demonstrated on that examination. Examinations scored as 1 to 3 represented insufficient skill to warrant a rating of competent. A score of 4 or greater was judged to represent sufficient skill to be rated as competent and to perform fiberoptic sigmoidoscopy independently. Data were tallied separately for persons with rigid\ sigmoidoscopy experience of more than and less than 15 examinations. At the end of each examination, the trainee and staff independently listed the diagnoses and treatment plan based on that examination. Diagnoses were grouped into
466
March
1966
The American
Journal
of Medicine
Volume
Twenty-five residents performed 662 fiberoptic sigmoidoscopic examinations, of which 495 were graded for competence and are summarized in this report. Ungraded examinations occurred if the schedule was too busy to permit the trainee “solo” examining time or if no staff participating in this study was available for grading. Competence Score versus Experience. Figure 1 displays the competence grade assigned versus the exam number for the 495 examinations tallied. As experience increased, skill increased as reflected in the competence score. The initial 10 examinations were largely graded 3 or less (92.3 percent). Examinations IO to 25 were nearly equally divided between grades 3 and 4. After 25 examinations, 82 percent were grade 4 or above (i.e., competent). Table II shows the mean values of various objective parameters for each group of three examinations. With increasing experience, unassisted insertion distance increased and assisted time decreased.
Prior versus No Prior Rigid Sigmoidoscopy Experience. Figure 2 shows the percent of competent examinations versus the examination number for trainees with and without prior rigid sigmoidoscopy experience. Persons with prior rigid sigmoidoscopy experience performed 85 to 90 percent of their examinations competently after 24 examinations, whereas persons without prior rigid sigmoidoscopy experience required 30 examinations to achieve the same skill level. Competence Score versus Objective Parameters. The competence score was admittedly a subjective summary grade given at the end of each examination. Table Ill shows the objective parameters that corresponded to each competence grade given. The unassisted distances of insertion for competence scores 2, 3, and 4 were 27,
60
FIBEROPTIC
Figure 1. Cumulative experietke of 495 graded fiberoptic sigmoidoscopic examinations as performed by 25 resident trainees. Each solid circle represents the competence score (see Table I) given the trainee for that individual examination.
t
II
Cumulative
Experience
10
15
25
30
35
40
of Exams
of All 25 Trainees Insertion Time Unassisted (minutes)
Withdrawal Time Assisted (minutes)
Withdrawal Time (minutes)
l-3 4-6 7-9 10-12 13-15 16-18 19-21 22-24 25-27 28-30
27.0 35.3 38.4 44.1 40.0 43.1 45.4 48.7 45.0 40.1
4.9 5.4 5.8 6.2 5.7 5.9 5.6 6.5 6.2 6.3
2.2 1.9 1.4 1.0 1.3 0.5 0.9 0.7 0.7 0.6
5.8 6.3 5.8 6.9 6.0 6.4 6.2 6.6 5.4 5.3
by threes
ET AL
Lesion Recognition. Cancer, polyps, strictures, inflammatory bowel disease, and diverticulitis were categorized as major lesions. All other lesions such as hemorrhoids, uninflamed divetticula, and melanosis coli were classed as minor lesions. Table V compares each competence group in terms of their accuracy in identifying major and minor lesions. Except for competence group 1, which was probably not representative due to small sample size, diagnostic accuracy increased as competence scores increased. Table VI displays similar comparisons for polyp/cancer recognition. There were a total of 28 false-positive diagnoses, 13 major and 15 minor. The total false-positive rate was 3.8 percent with a major false-positive rate of 1.8 percent.
Distance Unassisted (cm)
are grouped
20 Number
Examinations Grouped by Threes
Examinations
TRAINING-HAWES
l”“l““l~“~l”“l”“l”“l”“l 5
39, and 49 cm, respectively. Each insertion distance was statistically different from the next (p <0.05). Similarly, the total assisted times were 6.5, 2.6, and 0.9 minutes for grades 2 through 4, respectively. Each mean value was again statistically significantly different from the others. Patient management scores were significantly better for those examinations graded as competent. Visualization of the mucosal surface as reflected in the quadrant score improved with increasing competence score. Overall Competence. At the end of each one- to twomonth training period, the principal instructors (G.A.L. and R.H.) reached a consensus as to the overall competence of each trainee, i.e., qualified to perform unsupervised screening fiberoptic sigmoidoscopy (Table IV).
TABLE
SIGMOIDOSCOPY
and each
column
represents
the average
March
1986
for the group
The American
Total Time Assisted (minutes) 5.5 5.1 3.6 3.2 3.2 2.0 2.4 1.7 1.2 1.3 for each
Journal
Diagnoses (percent correct)
Quadrant (percent viewed)
88.4 89.7 85.8 80.6 89.2 91.5 86.6 86.3 93.1 95.8
91.6 93.0 93.0 90.7 92.4 93.0 93.3 92.5 94.6 95.5
parameter
of Medicine
listed.
Volume
80
467
FIBEROPTIC
SIGMOIDOSCOPY
TRAINING-HAWES
ET AL
TABLE k-
IV
Percentage of Trainees Achieving Overall Competence versus Total Number of Examinations Performed
EO-
Total Number of Examinations Performed by Each Trainee 16-20 21-25 26-30
Lz L 4
60-
115
8 5
Number of trainees competent Number of trainees not competent Percent competent
40-
: ffn
20-
>30
0
0
5
4
8
2
2
2
1
1
0
0
71
80
89
o-
I
(p<.05) I
l-5
6-10
(NS)
(p<.Ol) I 11-15 PATIENT
(p-Z.05) I
(NS) I
(NS) 1
16-20
21-25
26-30
NUMBER
Figure 2. Percent of competent examinations versus examination number for trainees with (triangles) and without (solid circles) prior rigid sigmoidoscopy experience. Open circles represent all trainees combined. For each group of five examinations, the symbol represents the percent of examinations grade 4 (competent) or above. The p values reflect the differences in percent competent scores between groups with and without prior rigid sigmoidoscopy experience.
TABLE V
Competence Percentage
Score Group versus of Correct Diagnoses Competence 2 3
1 Number of major diagnoses* Percent major diagnoses correct Number of minor diagnoses Percent minor diagnoses correct
Score 4
5
6
32
70
61
5
100
78
81
92
100
3
56
130
113
5
100
71
78
81
100
Normal examinations excluded. This table displays the total number of major and minor diagnoses observed by the experienced endoscopist during the 495 fiberoptic sigmoidoscopic examinations. The percent of correct diagnoses by the trainees is compared with the competence score for individual fiberoptic sigmoidoscopic examinations. l
TABLE
Ill
Obiective
Objective Parameters (mean versus Competence Scores
values)
Competence Score* 4 2 3
Parameter
Insertion distanceunassisted (cm) Total insertion distance (cm) Insertion time-assisted (minutes) Total time-unassisted (minutes) Total time-assisted (minutes) Management score Diagnoses (percent correct) Quadrant score (percent viewed)
Analysis of Variance 1~ value)
277 46t
39r 46t
499 507
0.001 0.134
2.at
0.9$
0.43
0.001
6.57
9.5%
12.55
0.001
6.5t 1.37 86tl
2.6t 1.3t 831
0.95 1.1t 927
0.001 0.011 0.039
87t
93t
94f
0.010
TABLE VI
Competence Polyp/Cancer 1
Polyps Polyps Polyps Cancer
Grades 1 and 5 scores were eliminated from table due to small sample size. t,t,* Mean values with the same superscript are not significantly different from one another at the p = 0.05 level, Duncan’s multiple-comparisons results. Analysis of variance simultaneously compares the three mean values for each parameter.
13 mm 4-9 mm Ll cm
5” (100) 0 0 0
Score versus Accuracy Detection 2
Competence 3
13 (77) 11 (82) l(lO0) 0
28 (75) 21 (86) 7(86) 4(100)
in
Score 4 27 (93) 9 (100) 11 (100) 3(100)
5 0 0 l(lO0) 0
l
466
March
1986
The American
Journal
of Medicine
Volume
This table lists the number’ of polyps (grouped according to sizes) and cancers detected by the experienced endoscopist (instructor). Numbers in parentheses are the percent of lesions in each subgroup correctly diagnosed by trainees compared with the competence score given for individual fiberoptic sigmoidoscopic examinations.
80
FIBEROPTIC
Twelve of 13 of the major false-positive diagnoses occurred in examinations graded 1 to 3. Only one major false-positive diagnosis occurred in an examination graded as competent, and this was a polyp of 3 mm or less. Sixty-six percent of minor false-positive diagnoses occurred on examinations graded 1 to 3, with 33 percent occurring in examinations graded as competent [4]. There were a total of 196 normal examinations. COMMENTS
Our data give general guidelines as to the number of supervised fiberoptic sigmoidoscopic examinations required to gain competence. Despite intensive baseline training, it was noted (Figure 2) that only 19 percent of the initial 10 examinations were graded as competent even in persons with prior rigid sigmoidoscopy experience. This would indicate that one- to two-day seminars on fiberoptic sigmoidoscopy are generally inadequate to achieve competence in this procedure. For persons with prior rigid sigmoidoscopy experience, 24 to 27 examinations were required before 90 percent of examinations were considered competent. For persons without prior rigid sigmoidoscopy experience, 30 to 33 of examinations were required before 85 to 90 percent of examinations were graded as competent. Training program directors and hospital credentialing committees should find such data helpful. Extrapolation of our data to primary care physicians with extensive rigid sigmoidoscopy experience or surgeons with extensive experience handling tissue needs further study. Although competence at any skill is ultimately a subjective evaluation dependent on the standards established by the evaluator, we attempted to correlate competence scores with objective parameters. Table Ill shows that the mean values for many parameters such as insertion distance and unassisted examination time were statistically different for each competence score. Such data help to validate the evaluators’ subjective summary competence
TABLE
VII
Objective
SIGMOIDOSCOPY
TRAINING-HAWES
ET AL
Comparison of Objective Parameters Competent and Noncompetent Examinations Competence l-3
Parameter
Insertion distanceunassisted (cm) Total insertion distance (cm) Insertion time-unassisted (minutes) Insertion time-assisted (minutes) Withdrawal time (minutes) Total time-unassisted (minutes) Total time-assisted (minutes) Management score Diagnoses correct Quadrant score (percent observed)
Score 4-5
for
Group Comparison (p Value”)
34.5
49.5
0.001
46.7
50.9
0.018
5.5
6.0
0.142
1.6
0.4
0.001
5.8 8.4
6.8 12.4
0.019 0.001
4.2
0.9
0.001
1.3 85.0 91.5
1.1 92.4 94.3
0.006 0.010 0.070
Duncan’s multiple-range test. The mean values of the various objective parameters listed in the left column are compared for the fiberoptic sigmoidoscopic examinations graded 1, 2, or 3 (i.e, not competent) and those graded 4 or 5 (i.e., competent). l
score. Additionally, Table VII further compares the results of all competent versus all incompetent scored examinations. Differences were highly significant for all parameters listed except unassisted insertion time (p = 0.14) and the quadrant score (p = 0.07), which approached statistical significance. Further studies are needed to determine whether competence in fiberoptic sigmoidoscopy can be defined solely by the use of an objective checklist. Previous studies vary as to when competence is achieved. Pintauro et al [5] reported that only nine examinations were required to achieve competence among
r Figure 3. The left side of the figure represents possible definitions of corn petence. The stars represent trainees who met that criterion and at what point (number of examinations performed) in their training they achieved it. The far right of the figure represents the percent of our 25 trainees failing to achieve the hypothetic definition. Trainees who performed a total of less than 22 examinations were eliminated from the calculations.
March
1988
The American
Journal
of Medicine
Volume
80
489
FIBEROPTIC
SIGMOIDOSCOPY
TRAINING-HAWES
ET AL
medical trainees, although limited information was given. McCray [6] reported that after an average of 50 examinations with a 60 cm instrument, four of five persons with prior rigid sigmoidoscopy experience attained competence. Baskin et al [7] stated that 25 one-on-one supervised examinations were a minimum to attain competence with 60 cm instruments. The prior rigid sigmoidoscopy experience of these trainees was not stated. Schapiro et al [8] reported fiberoptic sigmoidoscopy training using 30 cm instruments. Only five to 10 examinations were required to achieve adequate training. Such data indicate that 30 cm instruments are easier to use than 60 cm ones. Despite the achievements of our trainees at examination 25 and beyond, the skill demonstrated on occasional examinations was still considered incompetent. This raises the question of the level of skill at which a physician is truly considered competent overall to perform screening fiberoptic sigmoidoscopy. We lack sufficient data on experience beyond 30 examinations to determine if these occasional suboptimal examinations continue. Hopefully a newly trained endoscopist will recognize these insufficient examinations and ask for assistance. The two principal instructors judged each of our trainees’ overall competence at the end of their training course. Eighty percent of trainees who performed 26 or more examinations and 89 percent of trainees who performed greater than 30
examinations were judged to be sufficiently skillful to perform independent screening flexible sigmoidoscopic examinations (Table IV). No specific formula was used to arrive at an overall competence grade for our series. Figure 3 presents a number of potential definitions for competence and shows how our group performed in meeting these criteria. The competence criteria chosen by the evaluator will obviously affect the number of examinations required to reach competence. Our competence scale extended to skill equivalent to that in a six-month junior gastrointestinal fellow (grade 5) and staff gastroenterologists (grade 6). Only 2 percent of examinations were grade 5 and none were grade 6. Such data indicate that consultant level skills in endoscopy require more extensive training. Our data apply to persons performing screening fiberoptic sigmoidoscopy and cannot be translated to the performance of colonoscopy. Because fiberoptic sigmoidoscopy by primary care physicians is promoted especially for cancer screening, our data regarding recognition of lesions are important. In our group, seven of seven cancers, 84 percent of polyps 3 mm or less, and 90 percent of polyps 4 mm or more were recognized. In those examinations with a competence of 4 or greater, 93 percent of polyps 3 mm or less and 100 percent of polyps 4 mm or more were recognized. These data indicate that such trainees should be effective in cancer screening.
REFERENCES 1.
2.
3.
4.
470
Winnan G, Berci G, Panish J, Talbot TM, Overholt BF, McCallum RW: Superiority of the flexible to the rigid sigmoidoscope in routine proctosigmoidoscopy. N Engl J Med 1980; 302: 1011-1012. Bohlman TW, Katon RM, Lipshutz GR, McCool MF, Smith FW, Melnyk CS: Fiberoptic pansigmoidoscopy. An evaluation and comparison with rigid sigmoidoscopy. Gastroenterology 1977; 72: 644-649. Schapiro M: Flexible fiberoptic sigmoidoscopy-the long and short of it (editorial). Gastrointest Endosc 1984; 30: 114. SAS, Statistical Analysis System. Cary, North Carolina: SAS Institute Inc., release 82.3.
March
1988
The American
Journal
of Medicine
Volume
5.
6.
7. ’ 8.
80
Pintauro WM, Wolfman MR, Floch MH: The training of medical residents in flexible sigmoidoscopy (abstr). Gastrointest Endosc 1980; 26: 74. McCray RS: A fiberoptic sigmoidoscopy training program for cancer screening physicians (abstr). Gastrointest Endosc 1981; 27: 137. Baskin WN, Greenlaw RL, Frakes JT, Vidican DE, Lewan RB: Flexible sigmoidoscopy training for primary care physicians (abstr). Gastrointest Endosc 1984; 30: 141. Schapiro M, Auslander MO, Getzug SJ, Klasky I: Flexible fiberoptic sigmoidoscopy training-of non-endoscopic physicians in the community hospital (abstr). Gastrointest Endose 1983; 29: 186.
Training Resident Physicians in Fiberoptic Sigmoidoscopy How Many Supervised Examinations Are Required to Achieve Competence?
ROBERT HAWES, M.D. GLEN A. LEHMAN, M.D. JOHN HAST, M.D. KATHERINE W. O’CONNOR, M.D. DAVID W. CRABB, M.D. ALEC LUI, M.D. PHILIP A. CHRISTIANSEN, M.D. Indianapolis,
Indiana
Twenty-five resident physicians performed 495 fiberoptic sigmoidoscopic examinations that were graded for overall skill according to a six-point competence scale. In general, 24 to 30 examinations were required to become competent at fiberoptic sigmoidoscopy. Trainees with prior rigid sigmoidoscopy experience achieved competence more quickly than those with no prior rigid sigmoktoscopy experience. As experience increased, unassisted insertion distance and luminal visualization increased, insertion time and assisted time decreased, and management scores and percent correct diagnoses improved. Trainees detected 93 to 100 percent of polyps and cancers viewed by the experienced sigmoidoscopist once competence was achieved. These data indicate that programs for training primary care physicians in fiberoptic sigmoktoscopy are feasible, help define the number of examinations required to become competent, and indicate that such trainees should be effective in cancer screening. The superiority of fiberoptic sigmoidoscopy over rigid sigmoidoscopy in screening for colorectal disease has been clearly demonstrated [1,2]. One obstacle encountered in effecting the use of fiberoptic sigmoidoscopy is related to difficulty in providing training for primary care physicians in its use [3]. Training programs for internal medicine, family practice, and general surgery are promoting instruction in fiberoptic sigmoidoscopy. Therefore it is important to quantitate the training required to become competent at this procedure. It was toward this end that this study was undertaken.
METHODS
From the Division of Gastroenterology, Department of Medicine, Indiana University School of Medicine, Indianapolis, Indiana. Requests for reprints should be addressed to Dr. Glen A. Lehman, Regenstrief Health Center, Room 614B, 1001 West Tenth Street, Indianapolis, Indiana 46202. Manuscript accepted March 7, 1985.
A total of 25 residents in general medicine, surgery, and family practice participated in the fiberoptic sigmoidoscopy training program during one- to two-month rotations on the gastroenterology medicine service. The trainees had variable rigid sigmoidoscopy experience but no previous experience with fiberoptic sigmoidoscopy. Each trainee received baseline training consisting of written materials on fiberoptic sigmoidoscopic instrumentation, technique, complications, and diagnostic capabilities. Each resident was given individual instruction about the instrument by an experienced endoscopist. Additionally, all were required to spend at least two hours practicing on a rubber colon model (Olympus CM-l) and viewed several examinations via a teaching attachment. Following this, they began performing examinations on patients under the supervision of an experienced endoscopist. Each examination was begun by allowing the trainee to perform as much of a complete examination as possible within set limits of time, discomfort, and safety. Trainees were allowed 10 minutes for full
March
1986
The American
Journal
of Medicine
Volume
80
465
FIBEROPTIC
SIGMOIDOSCOPY
TABLE I Not 1. 2. 3.
TRAINING-HAWES
ET AL
Overall Competence Score
major and minor categories. True-positive, false-positive, and false-negative diagnoses were recorded. The treatment plan was scored according to the following: 1 = clearly correct; 2 = reasonable choice but not totally state-of-theart management; 3 = clearly suboptimal; 4 = wrong. Fiberoptic sigmoidoscopic examinations were performed in patients referred to the gastroenterology service for standard indications. Most examinations were performed with Olympus TCF-1S and lTS2 60 cm sigmoidoscopes. No shorter instruments were used. A one-way analysis of variance model was used to compare means for more than one group, and if a significant difference among the groups was observed at the 0.05 level, a Duncan’s multiple-range test was used to make all the pairwise comparisons. A two-sample t test, two-tailed, was used to compare means from two independent groups. A corrected chi-square test, or a two-tailed Fisher’s exact test when necessary, was used to compare percentages from two independent groups. All test statistics were calculated using the Statistical Analysis System computer program [4].
Competent Totally unskilled and possibly dangerous to patients. A complete novice Knows few basics but examinations slow and clearly incomplete Modest facility in instrument manipulation but lacks firm control of examinations (technical or interpretive)
Competent 4. 5. 6.
Good technical and interpretive skills with only occasional errors Skill comparable to average first-year gastrointestinal fellow with six to 12 months of endoscopy experience Skill comparable to average second-year gastrointestinal fellow or staff
The above overall individual fiberoptic above were judged tions.
competence scale was used to grade the sigmoidoscopy examinations. Grades 4 and competent to perform unsupervised examina-
RESULTS
insertion and 15 minutes for the total examination. The unassisted (Le., solo) portion of the examination was terminated prematurely if reasonable progress was not being attained, excessive discomfort occurred, or the staff believed that patient safety might be compromised. The staff viewed through the teaching attachment but gave no verbal or mechanical assistance with the exception of the first IO examinations, when advancing the scope under trainee command was permitted if progress was slow. During the solo effort, the trainees’ performance was evaluated and recorded for distance penetrated, insertion and withdrawal time, quadrants viewed on withdrawal, diagnoses, and patient management based on the fiberoptic sigmoidoscopic findings. If the trainee failed to perform a complete examination during the solo effort, the staff then actively assisted to assure an adequate examination of the patient. Only the solo effort was graded. In an effort to quantitate the percent of mucosal surface viewed, each IO cm of colon during withdrawal was divided into four quadrants. As the sigmoidoscope was withdrawn, the trainee was graded as to whether each of four quadrants in each 10 cm of withdrawal had been adequately visualized. The number of adequately viewed quadrants was divided by the number of total quadrants and expressed as a percentage. A six-point competence scale was developed (Table I). For each supervised examination, the trainee was assigned a competency score reflecting the skill demonstrated on that examination. Examinations scored as 1 to 3 represented insufficient skill to warrant a rating of competent. A score of 4 or greater was judged to represent sufficient skill to be rated as competent and to perform fiberoptic sigmoidoscopy independently. Data were tallied separately for persons with rigid\ sigmoidoscopy experience of more than and less than 15 examinations. At the end of each examination, the trainee and staff independently listed the diagnoses and treatment plan based on that examination. Diagnoses were grouped into
466
March
1966
The American
Journal
of Medicine
Volume
Twenty-five residents performed 662 fiberoptic sigmoidoscopic examinations, of which 495 were graded for competence and are summarized in this report. Ungraded examinations occurred if the schedule was too busy to permit the trainee “solo” examining time or if no staff participating in this study was available for grading. Competence Score versus Experience. Figure 1 displays the competence grade assigned versus the exam number for the 495 examinations tallied. As experience increased, skill increased as reflected in the competence score. The initial 10 examinations were largely graded 3 or less (92.3 percent). Examinations IO to 25 were nearly equally divided between grades 3 and 4. After 25 examinations, 82 percent were grade 4 or above (i.e., competent). Table II shows the mean values of various objective parameters for each group of three examinations. With increasing experience, unassisted insertion distance increased and assisted time decreased.
Prior versus No Prior Rigid Sigmoidoscopy Experience. Figure 2 shows the percent of competent examinations versus the examination number for trainees with and without prior rigid sigmoidoscopy experience. Persons with prior rigid sigmoidoscopy experience performed 85 to 90 percent of their examinations competently after 24 examinations, whereas persons without prior rigid sigmoidoscopy experience required 30 examinations to achieve the same skill level. Competence Score versus Objective Parameters. The competence score was admittedly a subjective summary grade given at the end of each examination. Table Ill shows the objective parameters that corresponded to each competence grade given. The unassisted distances of insertion for competence scores 2, 3, and 4 were 27,
60
FIBEROPTIC
Figure 1. Cumulative experietke of 495 graded fiberoptic sigmoidoscopic examinations as performed by 25 resident trainees. Each solid circle represents the competence score (see Table I) given the trainee for that individual examination.
t
II
Cumulative
Experience
10
15
25
30
35
40
of Exams
of All 25 Trainees Insertion Time Unassisted (minutes)
Withdrawal Time Assisted (minutes)
Withdrawal Time (minutes)
l-3 4-6 7-9 10-12 13-15 16-18 19-21 22-24 25-27 28-30
27.0 35.3 38.4 44.1 40.0 43.1 45.4 48.7 45.0 40.1
4.9 5.4 5.8 6.2 5.7 5.9 5.6 6.5 6.2 6.3
2.2 1.9 1.4 1.0 1.3 0.5 0.9 0.7 0.7 0.6
5.8 6.3 5.8 6.9 6.0 6.4 6.2 6.6 5.4 5.3
by threes
ET AL
Lesion Recognition. Cancer, polyps, strictures, inflammatory bowel disease, and diverticulitis were categorized as major lesions. All other lesions such as hemorrhoids, uninflamed divetticula, and melanosis coli were classed as minor lesions. Table V compares each competence group in terms of their accuracy in identifying major and minor lesions. Except for competence group 1, which was probably not representative due to small sample size, diagnostic accuracy increased as competence scores increased. Table VI displays similar comparisons for polyp/cancer recognition. There were a total of 28 false-positive diagnoses, 13 major and 15 minor. The total false-positive rate was 3.8 percent with a major false-positive rate of 1.8 percent.
Distance Unassisted (cm)
are grouped
20 Number
Examinations Grouped by Threes
Examinations
TRAINING-HAWES
l”“l““l~“~l”“l”“l”“l”“l 5
39, and 49 cm, respectively. Each insertion distance was statistically different from the next (p <0.05). Similarly, the total assisted times were 6.5, 2.6, and 0.9 minutes for grades 2 through 4, respectively. Each mean value was again statistically significantly different from the others. Patient management scores were significantly better for those examinations graded as competent. Visualization of the mucosal surface as reflected in the quadrant score improved with increasing competence score. Overall Competence. At the end of each one- to twomonth training period, the principal instructors (G.A.L. and R.H.) reached a consensus as to the overall competence of each trainee, i.e., qualified to perform unsupervised screening fiberoptic sigmoidoscopy (Table IV).
TABLE
SIGMOIDOSCOPY
and each
column
represents
the average
March
1986
for the group
The American
Total Time Assisted (minutes) 5.5 5.1 3.6 3.2 3.2 2.0 2.4 1.7 1.2 1.3 for each
Journal
Diagnoses (percent correct)
Quadrant (percent viewed)
88.4 89.7 85.8 80.6 89.2 91.5 86.6 86.3 93.1 95.8
91.6 93.0 93.0 90.7 92.4 93.0 93.3 92.5 94.6 95.5
parameter
of Medicine
listed.
Volume
80
467
FIBEROPTIC
SIGMOIDOSCOPY
TRAINING-HAWES
ET AL
TABLE k-
IV
Percentage of Trainees Achieving Overall Competence versus Total Number of Examinations Performed
EO-
Total Number of Examinations Performed by Each Trainee 16-20 21-25 26-30
Lz L 4
60-
115
8 5
Number of trainees competent Number of trainees not competent Percent competent
40-
: ffn
20-
>30
0
0
5
4
8
2
2
2
1
1
0
0
71
80
89
o-
I
(p<.05) I
l-5
6-10
(NS)
(p<.Ol) I 11-15 PATIENT
(p-Z.05) I
(NS) I
(NS) 1
16-20
21-25
26-30
NUMBER
Figure 2. Percent of competent examinations versus examination number for trainees with (triangles) and without (solid circles) prior rigid sigmoidoscopy experience. Open circles represent all trainees combined. For each group of five examinations, the symbol represents the percent of examinations grade 4 (competent) or above. The p values reflect the differences in percent competent scores between groups with and without prior rigid sigmoidoscopy experience.
TABLE V
Competence Percentage
Score Group versus of Correct Diagnoses Competence 2 3
1 Number of major diagnoses* Percent major diagnoses correct Number of minor diagnoses Percent minor diagnoses correct
Score 4
5
6
32
70
61
5
100
78
81
92
100
3
56
130
113
5
100
71
78
81
100
Normal examinations excluded. This table displays the total number of major and minor diagnoses observed by the experienced endoscopist during the 495 fiberoptic sigmoidoscopic examinations. The percent of correct diagnoses by the trainees is compared with the competence score for individual fiberoptic sigmoidoscopic examinations. l
TABLE
Ill
Obiective
Objective Parameters (mean versus Competence Scores
values)
Competence Score* 4 2 3
Parameter
Insertion distanceunassisted (cm) Total insertion distance (cm) Insertion time-assisted (minutes) Total time-unassisted (minutes) Total time-assisted (minutes) Management score Diagnoses (percent correct) Quadrant score (percent viewed)
Analysis of Variance 1~ value)
277 46t
39r 46t
499 507
0.001 0.134
2.at
0.9$
0.43
0.001
6.57
9.5%
12.55
0.001
6.5t 1.37 86tl
2.6t 1.3t 831
0.95 1.1t 927
0.001 0.011 0.039
87t
93t
94f
0.010
TABLE VI
Competence Polyp/Cancer 1
Polyps Polyps Polyps Cancer
Grades 1 and 5 scores were eliminated from table due to small sample size. t,t,* Mean values with the same superscript are not significantly different from one another at the p = 0.05 level, Duncan’s multiple-comparisons results. Analysis of variance simultaneously compares the three mean values for each parameter.
13 mm 4-9 mm Ll cm
5” (100) 0 0 0
Score versus Accuracy Detection 2
Competence 3
13 (77) 11 (82) l(lO0) 0
28 (75) 21 (86) 7(86) 4(100)
in
Score 4 27 (93) 9 (100) 11 (100) 3(100)
5 0 0 l(lO0) 0
l
466
March
1986
The American
Journal
of Medicine
Volume
This table lists the number’ of polyps (grouped according to sizes) and cancers detected by the experienced endoscopist (instructor). Numbers in parentheses are the percent of lesions in each subgroup correctly diagnosed by trainees compared with the competence score given for individual fiberoptic sigmoidoscopic examinations.
80
FIBEROPTIC
Twelve of 13 of the major false-positive diagnoses occurred in examinations graded 1 to 3. Only one major false-positive diagnosis occurred in an examination graded as competent, and this was a polyp of 3 mm or less. Sixty-six percent of minor false-positive diagnoses occurred on examinations graded 1 to 3, with 33 percent occurring in examinations graded as competent [4]. There were a total of 196 normal examinations. COMMENTS
Our data give general guidelines as to the number of supervised fiberoptic sigmoidoscopic examinations required to gain competence. Despite intensive baseline training, it was noted (Figure 2) that only 19 percent of the initial 10 examinations were graded as competent even in persons with prior rigid sigmoidoscopy experience. This would indicate that one- to two-day seminars on fiberoptic sigmoidoscopy are generally inadequate to achieve competence in this procedure. For persons with prior rigid sigmoidoscopy experience, 24 to 27 examinations were required before 90 percent of examinations were considered competent. For persons without prior rigid sigmoidoscopy experience, 30 to 33 of examinations were required before 85 to 90 percent of examinations were graded as competent. Training program directors and hospital credentialing committees should find such data helpful. Extrapolation of our data to primary care physicians with extensive rigid sigmoidoscopy experience or surgeons with extensive experience handling tissue needs further study. Although competence at any skill is ultimately a subjective evaluation dependent on the standards established by the evaluator, we attempted to correlate competence scores with objective parameters. Table Ill shows that the mean values for many parameters such as insertion distance and unassisted examination time were statistically different for each competence score. Such data help to validate the evaluators’ subjective summary competence
TABLE
VII
Objective
SIGMOIDOSCOPY
TRAINING-HAWES
ET AL
Comparison of Objective Parameters Competent and Noncompetent Examinations Competence l-3
Parameter
Insertion distanceunassisted (cm) Total insertion distance (cm) Insertion time-unassisted (minutes) Insertion time-assisted (minutes) Withdrawal time (minutes) Total time-unassisted (minutes) Total time-assisted (minutes) Management score Diagnoses correct Quadrant score (percent observed)
Score 4-5
for
Group Comparison (p Value”)
34.5
49.5
0.001
46.7
50.9
0.018
5.5
6.0
0.142
1.6
0.4
0.001
5.8 8.4
6.8 12.4
0.019 0.001
4.2
0.9
0.001
1.3 85.0 91.5
1.1 92.4 94.3
0.006 0.010 0.070
Duncan’s multiple-range test. The mean values of the various objective parameters listed in the left column are compared for the fiberoptic sigmoidoscopic examinations graded 1, 2, or 3 (i.e, not competent) and those graded 4 or 5 (i.e., competent). l
score. Additionally, Table VII further compares the results of all competent versus all incompetent scored examinations. Differences were highly significant for all parameters listed except unassisted insertion time (p = 0.14) and the quadrant score (p = 0.07), which approached statistical significance. Further studies are needed to determine whether competence in fiberoptic sigmoidoscopy can be defined solely by the use of an objective checklist. Previous studies vary as to when competence is achieved. Pintauro et al [5] reported that only nine examinations were required to achieve competence among
r Figure 3. The left side of the figure represents possible definitions of corn petence. The stars represent trainees who met that criterion and at what point (number of examinations performed) in their training they achieved it. The far right of the figure represents the percent of our 25 trainees failing to achieve the hypothetic definition. Trainees who performed a total of less than 22 examinations were eliminated from the calculations.
March
1988
The American
Journal
of Medicine
Volume
80
489
FIBEROPTIC
SIGMOIDOSCOPY
TRAINING-HAWES
ET AL
medical trainees, although limited information was given. McCray [6] reported that after an average of 50 examinations with a 60 cm instrument, four of five persons with prior rigid sigmoidoscopy experience attained competence. Baskin et al [7] stated that 25 one-on-one supervised examinations were a minimum to attain competence with 60 cm instruments. The prior rigid sigmoidoscopy experience of these trainees was not stated. Schapiro et al [8] reported fiberoptic sigmoidoscopy training using 30 cm instruments. Only five to 10 examinations were required to achieve adequate training. Such data indicate that 30 cm instruments are easier to use than 60 cm ones. Despite the achievements of our trainees at examination 25 and beyond, the skill demonstrated on occasional examinations was still considered incompetent. This raises the question of the level of skill at which a physician is truly considered competent overall to perform screening fiberoptic sigmoidoscopy. We lack sufficient data on experience beyond 30 examinations to determine if these occasional suboptimal examinations continue. Hopefully a newly trained endoscopist will recognize these insufficient examinations and ask for assistance. The two principal instructors judged each of our trainees’ overall competence at the end of their training course. Eighty percent of trainees who performed 26 or more examinations and 89 percent of trainees who performed greater than 30
examinations were judged to be sufficiently skillful to perform independent screening flexible sigmoidoscopic examinations (Table IV). No specific formula was used to arrive at an overall competence grade for our series. Figure 3 presents a number of potential definitions for competence and shows how our group performed in meeting these criteria. The competence criteria chosen by the evaluator will obviously affect the number of examinations required to reach competence. Our competence scale extended to skill equivalent to that in a six-month junior gastrointestinal fellow (grade 5) and staff gastroenterologists (grade 6). Only 2 percent of examinations were grade 5 and none were grade 6. Such data indicate that consultant level skills in endoscopy require more extensive training. Our data apply to persons performing screening fiberoptic sigmoidoscopy and cannot be translated to the performance of colonoscopy. Because fiberoptic sigmoidoscopy by primary care physicians is promoted especially for cancer screening, our data regarding recognition of lesions are important. In our group, seven of seven cancers, 84 percent of polyps 3 mm or less, and 90 percent of polyps 4 mm or more were recognized. In those examinations with a competence of 4 or greater, 93 percent of polyps 3 mm or less and 100 percent of polyps 4 mm or more were recognized. These data indicate that such trainees should be effective in cancer screening.
REFERENCES 1.
2.
3.
4.
470
Winnan G, Berci G, Panish J, Talbot TM, Overholt BF, McCallum RW: Superiority of the flexible to the rigid sigmoidoscope in routine proctosigmoidoscopy. N Engl J Med 1980; 302: 1011-1012. Bohlman TW, Katon RM, Lipshutz GR, McCool MF, Smith FW, Melnyk CS: Fiberoptic pansigmoidoscopy. An evaluation and comparison with rigid sigmoidoscopy. Gastroenterology 1977; 72: 644-649. Schapiro M: Flexible fiberoptic sigmoidoscopy-the long and short of it (editorial). Gastrointest Endosc 1984; 30: 114. SAS, Statistical Analysis System. Cary, North Carolina: SAS Institute Inc., release 82.3.
March
1988
The American
Journal
of Medicine
Volume
5.
6.
7. ’ 8.
80
Pintauro WM, Wolfman MR, Floch MH: The training of medical residents in flexible sigmoidoscopy (abstr). Gastrointest Endosc 1980; 26: 74. McCray RS: A fiberoptic sigmoidoscopy training program for cancer screening physicians (abstr). Gastrointest Endosc 1981; 27: 137. Baskin WN, Greenlaw RL, Frakes JT, Vidican DE, Lewan RB: Flexible sigmoidoscopy training for primary care physicians (abstr). Gastrointest Endosc 1984; 30: 141. Schapiro M, Auslander MO, Getzug SJ, Klasky I: Flexible fiberoptic sigmoidoscopy training-of non-endoscopic physicians in the community hospital (abstr). Gastrointest Endose 1983; 29: 186.