- Email: [email protected]
Learning curve for EUS in gastric cancer T staging by using cumulative sum analysis
ORIGINAL ARTICLE: Clinical Endoscopy
Learning curve for EUS in gastric cancer T staging by using cumulative sum analysis Chan Hyuk Park, MD, Jun Chul Park, MD, Eun Hye Kim, MD, Da Hyun Jung, MD, Hyunsoo Chung, MD, Sung Kwan Shin, MD, Sang Kil Lee, MD, PhD, Yong Chan Lee, MD, PhD Seoul, Republic of Korea
Background: EUS is an operator-dependent procedure and requires more technical and cognitive skills than a routine endoscopic procedure. The learning curve for the staging of gastric cancer, however, has not been evaluated. Objective: To evaluate the threshold number of EUS examinations for gastric cancer T staging. Design: Retrospective study. Setting: University-affiliated tertiary care hospital in the Republic of Korea. Patients: Four trainees with no previous EUS experience. Intervention: Analyzing performance of EUS trainees in gastric cancer T staging by using cumulative sum (CUSUM) analysis. Main Outcome Measurements: CUSUM plot and a minimal number of procedures for reaching a plateau. Results: A total of 553 initial EUS examinations for treatment-naïve gastric cancers, performed by trainees, were enrolled in the study. The final T stage was determined by experts by using EUS in 332 gastric cancer cases, whereas the T stage of the other 221 lesions was determined by trainees by using EUS. The accuracy of EUS examinations performed by trainees and experts was 72.6% and 84.3%, respectively. The number of EUS examinations required to reach the first plateau in each trainee was 20, 41, 60, and 65. Limitations: Retrospective study with a relatively small number of trainees. Conclusion: The CUSUM scores of all of 4 trainees in the study reached a plateau by the 65th examination. (Gastrointest Endosc 2015;81:898-905.)
Despite its decreasing incidence in Western countries, gastric cancer remains the second leading cause of cancer death worldwide.1,2 Although radical gastrectomy with lymph node dissection is the mainstay of curative treatment for gastric cancer, endoscopic resection is a feasible treatment option for early gastric cancers that fulfill absolute or expanded indication criteria for endoscopic
submucosal dissection.3,4 In addition, neoadjuvant chemotherapy with or without radiotherapy can be used for controlling microscopic metastasis and/or downstaging/ downsizing the tumor, thereby enhancing surgical curability.3 Selection of the most appropriate treatment method for gastric cancer can be achieved through reliable clinical cancer staging.
Abbreviations: ASGE, American Society for Gastrointestinal Endoscopy; AJCC/UICC, American Joint Cancer Committee/Union Internationale Contre le Cancer; CI, confidence interval; CUSUM, cumulative sum; OR, odds ratio.
Current affiliations: Division of Gastroenterology, Department of Internal Medicine, Severance Hospital, Institute of Gastroenterology, Yonsei University College of Medicine, Seoul, Republic of Korea.
DISCLOSURE: All authors have disclosed no financial relationships relevant to this article. Copyright ª 2015 by the American Society for Gastrointestinal Endoscopy 0016-5107/$36.00 http://dx.doi.org/10.1016/j.gie.2014.08.024
Reprint requests: Jun Chul Park, MD, Division of Gastroenterology, Department of Internal Medicine, Severance Hospital, Institute of Gastroenterology, Yonsei University College of Medicine, 50 Yonsei-ro, Seodaemun-gu, Seoul 120-752, Republic of Korea. If you would like to chat with an author of this article, you may contact Dr Park at [email protected].
Received May 24, 2014. Accepted August 21, 2014.
898 GASTROINTESTINAL ENDOSCOPY Volume 81, No. 4 : 2015
www.giejournal.org
Park et al
One of the most studied tools for locoregional staging of gastric cancer is EUS.5,6 EUS has been recommended for preoperative locoregional staging of gastric cancer by several groups, including the National Comprehensive Cancer Network.7,8 EUS allows evaluation of individual layers of the gastric wall as well as identification of enlarged regional lymph nodes.5,6 However, it is an operatordependent procedure and requires more technical and cognitive skills than a routine endoscopic procedure.9,10 The American Society for Gastrointestinal Endoscopy (ASGE) suggests minimum numbers of EUS procedures according to lesion site.9 In cases of mucosal tumors, including esophageal cancer, gastric cancer, and rectal cancer, the suggested threshold number of procedures is 75. This number, however, was determined based on 2 studies on the learning curve for staging esophageal cancer rather than gastric cancer.11,12 No study has been performed on the learning curve for staging gastric cancer. We aimed to evaluate the learning curve for EUS in gastric cancer T staging to suggest the threshold number of procedures for gastric cancer.
METHODS Patients
Learning curve for EUS in gastric cancer T staging
the third layer. The muscularis propria layer was visualized as the fourth layer, and the fifth layer included the serosa and subserosa. Assessment of tumor depth by EUS was made in accordance with the seventh edition of the American Joint Cancer Committee/Union Internationale Contre le Cancer (AJCC/UICC) TNM classification.13 When the T stage was determined by using EUS by both trainees and experts, the T stage determined by the expert was recorded as the final T stage in the electronic database of our institution, whereas the T stage determined by a trainee was recorded as the final T stage if the expert did not determine the T stage.
Accuracy of T staging by using EUS When endoscopic resection was indicated by preoperative staging,3,4 EMR or endoscopic submucosal dissection was performed. Patients who were candidates for curative treatments but not endoscopic resection or had residual disease after endoscopic resection underwent radical gastrectomy with lymph node dissection.3 In patients who underwent endoscopic resection or radical gastrectomy, the accuracy of the T stage obtained by EUS was assessed based on the results of pathologic examinations.
Statistical analysis
All procedures were performed with patients under sedation with midazolam or propofol. Before EUS, conventional endoscopy was performed to obtain general information about the stomach and remove food residue and mucus. EUS was then performed by using a radial array echoendoscope (GF-UE260; Olympus Optical Co Ltd, Tokyo, Japan). Deaerated water was instilled to improve transmission of the US beam. Acoustic coupling with the gastric wall was obtained by instilling 300 to 800 mL of deaerated water into the gastric cavity. The ultrasonic aspect of tumors and their contiguous structures was assessed by moving the endoscope tip along the entire stomach. After observing tumor characteristics, including size and depth of invasion, regional lymph nodes were assessed. Local tumor infiltration was determined by using the 5-layer structure of the gastric wall. Briefly, the mucosal layer was visualized as a combination of the first and second layers, and the submucosal layer corresponded to
Analysis of variance and the c2 test were used for comparing baseline patient and lesion characteristics of groups classified by each trainee who performed EUS. Next, the diagnostic performance of trainees and experts was assessed based on the results of EUS and pathologic examinations. Because distribution of pathologic T stages has a great effect on the accuracy of T staging by using EUS,6 the accuracy of EUS T staging by experts was adjusted by standardization according to the distribution of pathologic T stages. Cumulative sum (CUSUM) analysis was used to assess the learning curve for determining the T stage by using EUS for each trainee. This is a method of continuously assessing the performance of an individual or process against a predetermined standard to detect adverse trends and allow early intervention.14-17 In the CUSUM plot, each case was plotted in sequence along the x-axis. When a success occurred, a constant of S was subtracted from the CUSUM score. When a failure occurred, a constant of ‘1 S’ was added to the CUSUM score. Thus, a positive trend in the CUSUM plot indicated failure, whereas a negative trend indicated success. A constant of S was based on a prespecified acceptable failure rate (p0, level of inherent error if the procedure is carried out correctly) and unacceptable failure rate (p1, where p1 p0 represents the maximum acceptable level of human error). Generally, a p0 value is set by minimum acceptable criteria for assessing the competency.18 Unfortunately, however, there have been no definitive criteria for assessing the competency of EUS. Although a previous meta-analysis showed that the pooled estimate of accuracy of overall T staging was
www.giejournal.org
Volume 81, No. 4 : 2015 GASTROINTESTINAL ENDOSCOPY 899
We retrospectively reviewed the clinical records of patients who underwent EUS for gastric cancer performed by trainees at Severance Hospital, Seoul, Republic of Korea, between March 2011 and February 2012. EUS examinations that were not the first in patients were excluded from the study. In addition, EUS examinations performed after chemotherapy, radiotherapy, or endoscopic resection were excluded. As a result, initial EUS examinations for treatment-naïve gastric cancer performed by trainees were included in the study.
EUS procedure
Learning curve for EUS in gastric cancer T staging
75.0%, the accuracy varied vastly across the included studies (range 56.9%-87.7%).6 In addition, significant heterogeneity was identified in the meta-analysis (I2 Z 89.5%). In our study, therefore, the p0 value was defined as adjusted accuracy of the T stage determined by experts by using EUS at our institution instead of the results of previous studies. Next, a p1 value was set to twice p0 as in previous studies.14,19-22 Typical values for type I and type II errors (a and b, respectively) are 10%, and unlike power calculations for clinical trials in the context of performance monitoring, they are typically set to be equal.23 In addition, decision interval (h0) on the CUSUM plot was calculated based on the variables mentioned previously, including acceptable and unacceptable failure rates and type I and type II errors.14 When the CUSUM score decreased below the current decision level, the CUSUM score was replaced by the current decision level. If the CUSUM score exceeded decision interval, in contrast, it indicated an unacceptable performance, and the CUSUM monitoring was then restarted at the next decision interval. The number of EUS examinations and decision interval required for reaching a plateau in each trainee were identified by upward CUSUM plots.16,17 The formulas for S and h0 are as follows14: S Z Q=ðP þ QÞ; where P Z ln ðp1 =p0 Þ and Q Z ln ½ð1 p0 Þ=ð1 p1 Þ h0 Z b=ðP þ QÞ; where b Z ln ½ð1 bÞ=a Finally, logistic regression analysis was used for identifying factors associated with accurate T staging of gastric cancer by using EUS. Variables with significance of 0.2 or less on univariate analysis were included in a logistic regression model. All statistical procedures were conducted by using the statistical software SPSS for Windows, version 18.0 (IBM Inc, Chicago, Ill) with the exception of the CUSUM plots, which were performed in Excel 2010 (Microsoft, Redmond, Wash).
RESULTS Four trainees performed EUS in the study. All trainees had completed a standard 1-year gastroenterology fellowship and had performed 1000 or more diagnostic upper endoscopies. None of the trainees had previous experience or training in EUS. In addition, all of the experts involved in the study had at least 5 years of experience with EUS before March 2011. As shown in Figure 1, of the 574 EUS examinations for gastric cancer performed by trainees, 9 were excluded because they were not the first examination performed in the patient. In addition, 12 examinations performed after chemotherapy, radiotherapy, or endoscopic resection were excluded. Of the remaining 553 examinations, 332 were confirmed by experts. The other 221 examinations were performed by trainees without expert confirmations. In addition, 299 of 900 GASTROINTESTINAL ENDOSCOPY Volume 81, No. 4 : 2015
Park et al
332 lesions, which had been examined by using EUS by both trainees and experts, and 208 of 221 lesions, which had been examined by using EUS by trainees only, were treated by either endoscopic resection or surgical resection; thereafter, pathologic stages were determined for these lesions by pathologic examinations.
Baseline characteristics Patient and lesion characteristics are shown in Table 1. The number of lesions examined by EUS was 160, 153, 85, and 155 for trainees 1, 2, 3, and 4, respectively. Mean age and sex distribution did not differ among the 4 groups classified based on the individuals performing EUS. Although a difference in lesion location was identified, lesion size and pathologic T stages did not differ among the 4 groups. In addition, there was no adverse event related to EUS.
Diagnostic performance of EUS Table 2 shows the diagnostic performance of EUS performed by trainees without experts’ confirmations. The overall accuracy of T staging by trainees by using EUS was 72.6%. Diagnostic performance of EUS performed by trainees with expert confirmation is shown in Table 3. The overall accuracy of T staging by experts by using EUS was 84.3%. As mentioned in the Methods section, the accuracy of EUS T staging by experts was adjusted by standardization according to the distribution of pathologic T stages. The ratio of pathologic T stages in the trainee group was 160:21:17:10 (Table 2), whereas that in the expert group was 254:15:11:19 (Table 3). In cases of pT1 stage, the numbers of examinations in the expert group were multiplied by 0.906, which was calculated by the following formula: ½ðNumber of examinations of pT1 stage in trainee groupÞ= ðTotal number of examinations in trainee groupÞ= ½ðNumber of examinations of pT1 stage in expert groupÞ= ðTotal number of examinations in expert groupÞ Z½ð160=208Þ=ð254=299ÞZ0:906 The calculations for examinations of pT2, pT3, and pT4 stages were conducted along the same lines. After standardization according to the distribution of pathologic T stages, the adjusted overall accuracy of T staging by experts by using EUS was 79.6% (Supplemental Table 1, available online at www.giejournal.org).
CUSUM plots Because the adjusted overall accuracy of T staging by experts by using EUS was 79.6%, acceptable and unacceptable failure rates were set at 0.204 and 0.408, respectively. Then, S and h0 were calculated based on acceptable and unacceptable failure rates and type I and type II errors by using the formulas in the Methods. As a result, S and h0 values were 0.30 and 2.22, respectively. The CUSUM scores were then www.giejournal.org
Park et al
Learning curve for EUS in gastric cancer T staging
EUS examinations for gastric cancer performed by trainee 574 lesions (567 patients) Exclusion
Prior EUS history: 9 lesions (9 patients) Prior CCRT or chemotherapy history: 11 lesions (11 patients) Prior endoscopic resection history: 1 lesion (1 patient)
Initial EUS examinations f e gastric cancer performed by trainee 553 lesions (546 patients)
Staging after expert´s confirmation 332 lesions (329 patients)
With pathologic confirmation 299 lesions (297 patients)
Trainee´s staging only 221 lesions (217 patients)
Without pathologic confirmation 33 lesions (32 patients)
With pathologic confirmation 208 lesions (204 patients)
Without pathologic confirmation 13 lesions (13 patients)
Figure 1. Flow diagram of enrolled patients. CCRT, concurrent chemoradiotherapy.
TABLE 1. Baseline patient and lesion characteristics Trainees who performed EUS for gastric cancer Variable
Trainee 2
Trainee 3
No. of patients
158
152
84
152
546
No. of lesions
160
153
85
155
553
59.4 11.3
59.1 10.9
59.6 11.1
59.1 11.2
Age, y, mean SD
Trainee 4
P value
Trainee 1
Sex, no. (%)
.926
Total
59.2 11.1
.599
Male
107 (67.7)
106 (69.7)
60 (71.4)
97 (63.8)
370 (67.8)
Female
51 (32.3)
46 (30.3)
24 (28.6)
55 (36.2)
176 (32.2)
Location, no. (%)
.020
Upper third
11 (6.9)
22 (14.4)
16 (18.8)
25 (16.1)
201 (12.6)
Middle third
41 (25.6)
43 (28.1)
28 (32.9)
51 (32.9)
518 (32.5)
Lower third
108 (67.5)
88 (57.5)
41 (48.2)
79 (51.0)
873 (54.8)
21.6 16.1
25.3 20.3
20.9 15.8
20.4 11.5
Size, mm, mean SD T stage, no. (%)*
.068
20.7 15.7
.194
pT1
118 (80.3)
122 (85.9)
66 (86.8)
108 (76.1)
414 (81.7)
pT2
11 (7.5)
7 (4.9)
3 (3.9)
15 (10.6)
36 (7.1)
pT3
10 (6.8)
5 (3.5)
6 (7.9)
7 (4.9)
28 (5.5)
pT4
8 (5.4)
8 (5.6)
1 (1.3)
12 (8.5)
29 (5.7)
SD, Standard deviation. *Only lesions confirmed by pathologic examination were included.
www.giejournal.org
Volume 81, No. 4 : 2015 GASTROINTESTINAL ENDOSCOPY 901
Learning curve for EUS in gastric cancer T staging
Park et al
TABLE 2. Diagnostic performance of EUS performed by trainees without experts’ confirmation in patients who underwent endoscopic or surgical resection* EUS staging, no. (%) cT1
cT2
cT3
cT4
pT1
Pathologic staging
136 (85.0)
19 (11.9)
4 (2.5)
1 (0.6)
160
84.1
pT2
5 (23.8)
9 (42.9)
6 (28.6)
1 (4.8)
21
81.3
pT3
3 (17.6)
4 (23.5)
3 (17.6)
7 (41.2)
17
87.5
pT4
1 (10.0)
4 (40.0)
2 (20.0)
3 (30.0)
10
92.3
145
36
15
12
208
72.6
Overall
Overall
Accuracy, %
*Only lesions confirmed by pathologic examination were included.
TABLE 3. Diagnostic performance of EUS performed by trainees with experts’ confirmation in patients who underwent endoscopic or surgical resection* EUS staging, no. (%) cT1
cT2
cT3
cT4
pT1
Pathologic staging
243 (95.7)
9 (3.5)
2 (0.8)
0 (0.0)
254
91.6
pT2
10 (66.7)
3 (20.0)
2 (13.3)
0 (0.0)
15
90.6
pT3
2 (18.2)
1 (9.1)
5 (45.5)
3 (27.3)
11
93.3
pT4
2 (10.5)
6 (31.6)
10 (52.6)
1 (5.3)
19
93.0
257
19
19
4
299
84.3
Overall
Overall
Accuracy, %
*Only lesions confirmed by pathologic examination were included.
calculated based on S and 1 S according to the success of accurate T staging (Fig. 2). For trainee 1, the CUSUM score crossed the first decision interval after the 10th examination and reached a plateau after examination 20 without further crossing of the decision level. For trainees 2 and 3, the CUSUM score crossed the first decision interval after examinations 39 and 60, respectively. The scores then reached a plateau after examinations 41 and 60, respectively, without further crossing of the decision level. Finally, for trainee 4, the CUSUM score first crossed the decision interval after examination 18 and again after examination 54. Although the score reached a plateau after examination 65, the third decision interval was crossed after examination 145.
pT4 stages (OR 0.074; 95% CI, 0.031–0.176; OR 0.088; 95% CI, 0.032–0.238; and OR 0.031; 95% CI, 0.009–0.104, respectively) compared with pT1 stage; and EUS performed by trainees (OR 0.539; 95% CI, 0.295–0.988) were factors independently associated with accurate T staging.
DISCUSSION
Table 4 shows the factors associated with accurate T staging of gastric cancer. On multivariate analysis, a size of 20 to 30 mm and greater than 30 mm (odds ratio [OR] 0.364; 95% confidence interval [CI], 0.186–0.713 and OR 0.135; 95% CI, 0.059–0.310, respectively) compared with a size of 20 mm or less; pT2, pT3, and
Although EUS is useful for locoregional staging of gastric cancer,5,6 training in EUS requires significant time and effort to attain competency. ASGE guidelines suggest that 75 examinations are needed before competency in EUS for mucosal cancer, including esophageal, gastric, and rectal cancer, can be determined. This minimum number of EUS examinations was suggested based on the 2 previous studies performed on the learning curve for EUS in esophageal cancer.11,12 However, only 1 trainee was included in each of the 2 studies. Moreover, approaching the lesion with an endoscope transducer may be more difficult in gastric cancer than in esophageal cancer. Therefore, analysis of the learning curve for EUS in gastric cancer is needed to determine a reasonable number of examinations for training.
902 GASTROINTESTINAL ENDOSCOPY Volume 81, No. 4 : 2015
www.giejournal.org
Associated factors for accurate T staging of gastric cancer
Park et al
Learning curve for EUS in gastric cancer T staging
CUSUM
11.10
Trainee 3 Reaching a plateau after examination 60
8.88
6.66
Trainee 1 Reaching a plateau after examination 20
Trainee 4 Reaching a plateau after examination 65
Trainee 2 Reaching a plateau after examination 41
4.44
2.22
1 6 11 16 21 26 31 36 41 46 51 56 61 66 71 76 81 86 91 96 10 1 10 6 11 1 11 6 12 1 12 6 13 1 13 6 14 1 14 6 15 1 15 6 16 1 16 6
0.00
Number of EUS examinations
Figure 2. Upward CUSUM plots of 4 EUS trainees. CUSUM, cumulative sum.
To our knowledge, our study is the first to demonstrate the learning curve for EUS staging of gastric cancer. All of the learning curves reached a plateau within 65 examinations. This finding supports the suggestion in the ASGE guidelines that a threshold number of 75 may be reasonable. This threshold number is relatively small compared with the minimum number of EUS examinations needed for evaluation of the pancreas.10 The accuracy of T staging by using EUS, however, was not high, even in experts. In our study, the overall accuracy of T staging by trainees and experts by using EUS was 72.6% and 84.3%, respectively. Previous studies also reported that overall accuracy was approximately 70% to 80%.24-26 These results indicate that accurate tumor staging by EUS is challenging, although the required number of EUS examinations for T staging in gastric cancer for achieving competency is relatively small. This was reflected in the learning curve of trainee 4 in our study. Although trainee 4 seemed to achieve competency after 65 examinations, his performance deteriorated at examination 145. Continuous performance monitoring may be needed even after achieving competency. CUSUM analysis would be helpful for self-monitoring and continuous quality improvement in trainees.15,27 In contrast to esophageal cancer, gastric cancer requires relatively complex endoscope control during EUS according to the tumor’s location. Before initiation of this study, we expected lesion location to affect the accuracy of T staging by using EUS. However, it was not a factor associated with accurate T staging of gastric cancer. On the contrary, lesion size was a factor associated accurate T staging. Accuracy of T staging in large lesions (20–30 mm and O30 mm) decreased compared with those in smaller
lesions (%20 mm). This may be because EUS only obtains cross-sectional views of lesions. A more careful and broad scanning of the transducer would be needed for accurate T staging of large lesions. In addition, advanced T stage was a poor predictive factor for accurate T staging by using EUS. This finding was inconsistent with that of a meta-analysis that concluded that EUS may be most useful for staging gastric cancers with greater tumor involvement (T3 and T4).6 The discrepancy may be attributable to the criteria for T staging. Most studies included in the meta-analysis assessed T stages according to the sixth edition of the AJCC/UICC TNM classification, whereas our study used the seventh edition of the AJCC/ UICC TNM classification. Under the current TNM classification, subserosal penetration and serosa invasion are classified as T3 and T4, respectively. However, it is difficult to differentiate between tumors that penetrate the subserosal connective tissue and those that invade the serosa. Excluded lesions that were inoperable could be another reason for the low accuracy rate in advanced cancers. Although this is the first study to assess the learning curve for EUS in gastric cancer T staging, the number of trainees was too small to provide a definitive threshold number of EUS examinations. Our results, therefore, may be considered only preliminary. Our CUSUM analysis, however, was shown to be useful for assessing continuous quality assurance. CUSUM analysis is a method that can monitor continuously a production process and detect subtle deviations from a preset defined level of achievement.14 In the cases in our study, the deterioration of performance of trainee 4 was identified on a CUSUM plot. If the deterioration of performance was
www.giejournal.org
Volume 81, No. 4 : 2015 GASTROINTESTINAL ENDOSCOPY 903
Learning curve for EUS in gastric cancer T staging
Park et al
TABLE 4. Associated factors for accurate T staging of gastric cancer
Variable
EUS exams
Accurate T staging, no. (%)
Age, y
Univariate analysis P value
Multivariate analysis OR
95% CI
P value
0.834-2.875
.166
.261
R60
250
193 (77.2)
!60
251
204 (81.3)
Sex
.021
Male
341
280 (82.1)
1.548
Female
160
117 (73.1)
1.000
Location
.115
Upper third
60
47 (78.3)
1.203
0.467-3.100
.701
Middle third
146
108 (74.0)
0.650
0.336-1.259
.201
Lower third
301
248 (82.4)
1.000 !.001
Size, mm %20
329
304 (92.4)
1.000
20-30
114
80 (70.2)
0.364
0.186-0.713
.003
O30
64
19 (29.7)
0.135
0.059-0.310
!.001
!.001
T stage pT1
414
379 (91.5)
1.000
pT2
36
12 (33.3)
0.074
0.031-0.176
!.001
pT3
28
8 (28.6)
0.088
0.032-0.238
!.001
pT4
29
4 (13.8)
0.031
0.009-0.104
!.001
0.295-0.988
.045
EUS
.001
By expert
299
252 (84.3)
1.000
By trainee
208
151 (72.6)
0.539
OR, Odds ratio; CI, confidence interval.
detected during CUSUM monitoring, re-education could be provided; it would then help to prevent further deterioration. Our study has several other limitations. This was a retrospective study; therefore, results of staging by trainees were unknown when the EUS examinations were confirmed by experts. In addition, our study focused on gastric cancer T staging rather than N staging. To accurately determine the minimum number of examinations for EUS staging, analysis of the learning curve for N staging is also necessary. A prospective study on the learning curve for both T and N staging with more trainees and patients would help provide a definite conclusion. Despite the preceding limitations, our data form the basis of a system for recommending the minimum number of EUS examinations needed for achieving competency. The CUSUM scores of all of 4 trainees in the study reached
a plateau by the 65th examination. A threshold number of 75, as suggested by ASGE guidelines, may be reasonable for achieving competency in gastric cancer T staging by using EUS.
904 GASTROINTESTINAL ENDOSCOPY Volume 81, No. 4 : 2015
www.giejournal.org
REFERENCES 1. Crew KD, Neugut AI. Epidemiology of gastric cancer. World J Gastroenterol 2006;12:354-62. 2. Shah MA, Kelsen DP. Gastric cancer: a primer on the epidemiology and biology of the disease and an overview of the medical management of advanced disease. J Natl Compr Canc Netw 2010;8:437-47. 3. Japanese Gastric Cancer Association. Japanese gastric cancer treatment guidelines 2010 (ver. 3). Gastric Cancer 2011;14:113-23. 4. Gotoda T, Yamamoto H, Soetikno RM. Endoscopic submucosal dissection of early gastric cancer. J Gastroenterol 2006;41:929-42. 5. Mocellin S, Marchet A, Nitti D. EUS for the staging of gastric cancer: a meta-analysis. Gastrointest Endosc 2011;73:1122-34.
Park et al
Learning curve for EUS in gastric cancer T staging
6. Cardoso R, Coburn N, Seevaratnam R, et al. A systematic review and meta-analysis of the utility of EUS for preoperative staging for gastric cancer. Gastric Cancer 2012;15(Suppl 1):S19-26. 7. Ajani JA, Bentrem DJ, Besh S. Gastric cancer, version 2.2013: featured updates to the NCCN Guidelines. J Natl Compr Canc Netw 2013;11: 531-46. 8. Maluf Filho F, Dotti CM, Halwan B, et al. An evidence-based consensus statement on the role and application of endosonography in clinical practice. Endoscopy 2009;41:979-87. 9. Eisen GM, Dominitz JA, Faigel DO, et al. Guidelines for credentialing and granting privileges for endoscopic ultrasound. Gastrointest Endosc 2001;54:811-4. 10. Wani S, Cotà GA, Keswani R, et al. Learning curves for EUS by using cumulative sum analysis: implications for American Society for Gastrointestinal Endoscopy recommendations for training. Gastrointest Endosc 2013;77:558-65. 11. Fockens P, Van den Brande JH, van Dullemen HM, et al. Endosonographic T-staging of esophageal carcinoma: a learning curve. Gastrointest Endosc 1996;44:58-62. 12. Schlick T, Heintz A, Junginger T. The examiner’s learning effect and its influence on the quality of endoscopic ultrasonography in carcinoma of the esophagus and gastric cardia. Surg Endosc 1999;13: 894-8. 13. Greene FL; American Joint Committee on Cancer; American Cancer Society. AJCC cancer staging manual. New York (NY): Springer-Verlag; 2010. 14. Bolsin S, Colson M. The use of the CUSUM technique in the assessment of trainee competence in new procedures. Int J Qual Health Care 2000;12:433-8. 15. Grunkemeier GL, Jin R, Wu Y. Cumulative sum curves and their prediction limits. Ann Thorac Surg 2009;87:361-4. 16. Jeong O, Ryu SY, Choi WY, et al. Risk factors and learning curve associated with postoperative morbidity of laparoscopic total gastrectomy for gastric carcinoma. Ann Surg Oncol 2014;21:2994-3001.
17. Cho SY, Choo MS, Jung JH, et al. Cumulative sum analysis for experiences of a single-session retrograde intrarenal stone surgery and analysis of predictors for stone-free status. PLoS One 2014;9:e84878-e. 18. Ward ST, Mohammed MA, Walt R, et al. An analysis of the learning curve to achieve competency at colonoscopy using the JETS database. Gut. Epub 2014 Jan 27. 19. Williams SM, Parry BR, Schlup MM. Quality control: an application of the CUSUM. BMJ 1992;304:1359-61. 20. Lee YK, Ha YC, Hwang DS, et al. Learning curve of basic hip arthroscopy technique: CUSUM analysis. Knee Surg Sports Traumatol Arthrosc 2013;21:1940-4. 21. Kheir F, Alokla K, Myers L, et al. Endobronchial ultrasoundtransbronchial needle aspiration of mediastinal and hilar lymphadenopathy learning curve. Am J Ther. Epub 2014 Mar 10. 22. Je S, Cho Y, Choi HJ, et al. An application of the learning curvecumulative summation test to evaluate training for endotracheal intubation in emergency medicine. Emerg Med J. Epub 2013 Oct 23. 23. Noyez L. Control charts, Cusum techniques and funnel plots. A review of methods for monitoring performance in healthcare. Interact Cardiovasc Thorac Surg 2009;9:494-9. 24. Feng X, Wang W, Luo G, et al. Comparison of endoscopic ultrasonography and multislice spiral computed tomography for the preoperative staging of gastric cancer - results of a single institution study of 610 Chinese patients. PLoS One 2013;8:e78846-e. 25. Hwang SW, Lee DH, Lee SH, et al. Preoperative staging of gastric cancer by endoscopic ultrasonography and multidetector-row computed tomography. J Gastroenterol Hepatol 2010;25:512-8. 26. Repiso A, Gómez-Rodríguez R, López Pardo R, et al. Usefulness of endoscopic ultrasonography in preoperative gastric cancer staging: diagnostic yield and therapeutic impact. Rev Esp Enferm Dig 2010;102:413-20. 27. Sibanda T, Sibanda N. The CUSUM chart method as a tool for continuous monitoring of clinical outcomes using routinely collected data. BMC Med Res Methodol 2007;7:46.
www.giejournal.org
Volume 81, No. 4 : 2015 GASTROINTESTINAL ENDOSCOPY 905
Learning curve for EUS in gastric cancer T staging
Park et al
APPENDIX
SUPPLEMENTAL TABLE 1. Diagnostic performance of EUS performed by trainees with expert confirmation in patients who underwent endoscopic or surgical resection after pathologic T stage standardization* EUS staging, no. (%) Pathologic staging
cT1
cT2
cT3
cT4
pT1
220.0 (95.7)
8.1 (3.5)
1.8 (0.8)
0.0 (0.0)
230.0
87.9
pT2
20.1 (66.7)
6.0 (20.0)
4.0 (13.3)
0.0 (0.0)
30.2
86.9
pT3
4.4 (18.2)
2.2 (9.1)
11.1 (45.5)
6.7 (27.3)
24.4
91.1
pT4
1.5 (10.5)
4.5 (31.6)
7.6 (52.6)
0.8 (5.3)
14.4
93.2
246.1
20.9
24.5
7.4
299.0
79.6
Overall
Overall
Accuracy, %
*Only lesions confirmed by pathologic examination were included.
905.e1 GASTROINTESTINAL ENDOSCOPY Volume 81, No. 4 : 2015
www.giejournal.org
Learning curve for EUS in gastric cancer T staging by using cumulative sum analysis Chan Hyuk Park, MD, Jun Chul Park, MD, Eun Hye Kim, MD, Da Hyun Jung, MD, Hyunsoo Chung, MD, Sung Kwan Shin, MD, Sang Kil Lee, MD, PhD, Yong Chan Lee, MD, PhD Seoul, Republic of Korea
Background: EUS is an operator-dependent procedure and requires more technical and cognitive skills than a routine endoscopic procedure. The learning curve for the staging of gastric cancer, however, has not been evaluated. Objective: To evaluate the threshold number of EUS examinations for gastric cancer T staging. Design: Retrospective study. Setting: University-affiliated tertiary care hospital in the Republic of Korea. Patients: Four trainees with no previous EUS experience. Intervention: Analyzing performance of EUS trainees in gastric cancer T staging by using cumulative sum (CUSUM) analysis. Main Outcome Measurements: CUSUM plot and a minimal number of procedures for reaching a plateau. Results: A total of 553 initial EUS examinations for treatment-naïve gastric cancers, performed by trainees, were enrolled in the study. The final T stage was determined by experts by using EUS in 332 gastric cancer cases, whereas the T stage of the other 221 lesions was determined by trainees by using EUS. The accuracy of EUS examinations performed by trainees and experts was 72.6% and 84.3%, respectively. The number of EUS examinations required to reach the first plateau in each trainee was 20, 41, 60, and 65. Limitations: Retrospective study with a relatively small number of trainees. Conclusion: The CUSUM scores of all of 4 trainees in the study reached a plateau by the 65th examination. (Gastrointest Endosc 2015;81:898-905.)
Despite its decreasing incidence in Western countries, gastric cancer remains the second leading cause of cancer death worldwide.1,2 Although radical gastrectomy with lymph node dissection is the mainstay of curative treatment for gastric cancer, endoscopic resection is a feasible treatment option for early gastric cancers that fulfill absolute or expanded indication criteria for endoscopic
submucosal dissection.3,4 In addition, neoadjuvant chemotherapy with or without radiotherapy can be used for controlling microscopic metastasis and/or downstaging/ downsizing the tumor, thereby enhancing surgical curability.3 Selection of the most appropriate treatment method for gastric cancer can be achieved through reliable clinical cancer staging.
Abbreviations: ASGE, American Society for Gastrointestinal Endoscopy; AJCC/UICC, American Joint Cancer Committee/Union Internationale Contre le Cancer; CI, confidence interval; CUSUM, cumulative sum; OR, odds ratio.
Current affiliations: Division of Gastroenterology, Department of Internal Medicine, Severance Hospital, Institute of Gastroenterology, Yonsei University College of Medicine, Seoul, Republic of Korea.
DISCLOSURE: All authors have disclosed no financial relationships relevant to this article. Copyright ª 2015 by the American Society for Gastrointestinal Endoscopy 0016-5107/$36.00 http://dx.doi.org/10.1016/j.gie.2014.08.024
Reprint requests: Jun Chul Park, MD, Division of Gastroenterology, Department of Internal Medicine, Severance Hospital, Institute of Gastroenterology, Yonsei University College of Medicine, 50 Yonsei-ro, Seodaemun-gu, Seoul 120-752, Republic of Korea. If you would like to chat with an author of this article, you may contact Dr Park at [email protected].
Received May 24, 2014. Accepted August 21, 2014.
898 GASTROINTESTINAL ENDOSCOPY Volume 81, No. 4 : 2015
www.giejournal.org
Park et al
One of the most studied tools for locoregional staging of gastric cancer is EUS.5,6 EUS has been recommended for preoperative locoregional staging of gastric cancer by several groups, including the National Comprehensive Cancer Network.7,8 EUS allows evaluation of individual layers of the gastric wall as well as identification of enlarged regional lymph nodes.5,6 However, it is an operatordependent procedure and requires more technical and cognitive skills than a routine endoscopic procedure.9,10 The American Society for Gastrointestinal Endoscopy (ASGE) suggests minimum numbers of EUS procedures according to lesion site.9 In cases of mucosal tumors, including esophageal cancer, gastric cancer, and rectal cancer, the suggested threshold number of procedures is 75. This number, however, was determined based on 2 studies on the learning curve for staging esophageal cancer rather than gastric cancer.11,12 No study has been performed on the learning curve for staging gastric cancer. We aimed to evaluate the learning curve for EUS in gastric cancer T staging to suggest the threshold number of procedures for gastric cancer.
METHODS Patients
Learning curve for EUS in gastric cancer T staging
the third layer. The muscularis propria layer was visualized as the fourth layer, and the fifth layer included the serosa and subserosa. Assessment of tumor depth by EUS was made in accordance with the seventh edition of the American Joint Cancer Committee/Union Internationale Contre le Cancer (AJCC/UICC) TNM classification.13 When the T stage was determined by using EUS by both trainees and experts, the T stage determined by the expert was recorded as the final T stage in the electronic database of our institution, whereas the T stage determined by a trainee was recorded as the final T stage if the expert did not determine the T stage.
Accuracy of T staging by using EUS When endoscopic resection was indicated by preoperative staging,3,4 EMR or endoscopic submucosal dissection was performed. Patients who were candidates for curative treatments but not endoscopic resection or had residual disease after endoscopic resection underwent radical gastrectomy with lymph node dissection.3 In patients who underwent endoscopic resection or radical gastrectomy, the accuracy of the T stage obtained by EUS was assessed based on the results of pathologic examinations.
Statistical analysis
All procedures were performed with patients under sedation with midazolam or propofol. Before EUS, conventional endoscopy was performed to obtain general information about the stomach and remove food residue and mucus. EUS was then performed by using a radial array echoendoscope (GF-UE260; Olympus Optical Co Ltd, Tokyo, Japan). Deaerated water was instilled to improve transmission of the US beam. Acoustic coupling with the gastric wall was obtained by instilling 300 to 800 mL of deaerated water into the gastric cavity. The ultrasonic aspect of tumors and their contiguous structures was assessed by moving the endoscope tip along the entire stomach. After observing tumor characteristics, including size and depth of invasion, regional lymph nodes were assessed. Local tumor infiltration was determined by using the 5-layer structure of the gastric wall. Briefly, the mucosal layer was visualized as a combination of the first and second layers, and the submucosal layer corresponded to
Analysis of variance and the c2 test were used for comparing baseline patient and lesion characteristics of groups classified by each trainee who performed EUS. Next, the diagnostic performance of trainees and experts was assessed based on the results of EUS and pathologic examinations. Because distribution of pathologic T stages has a great effect on the accuracy of T staging by using EUS,6 the accuracy of EUS T staging by experts was adjusted by standardization according to the distribution of pathologic T stages. Cumulative sum (CUSUM) analysis was used to assess the learning curve for determining the T stage by using EUS for each trainee. This is a method of continuously assessing the performance of an individual or process against a predetermined standard to detect adverse trends and allow early intervention.14-17 In the CUSUM plot, each case was plotted in sequence along the x-axis. When a success occurred, a constant of S was subtracted from the CUSUM score. When a failure occurred, a constant of ‘1 S’ was added to the CUSUM score. Thus, a positive trend in the CUSUM plot indicated failure, whereas a negative trend indicated success. A constant of S was based on a prespecified acceptable failure rate (p0, level of inherent error if the procedure is carried out correctly) and unacceptable failure rate (p1, where p1 p0 represents the maximum acceptable level of human error). Generally, a p0 value is set by minimum acceptable criteria for assessing the competency.18 Unfortunately, however, there have been no definitive criteria for assessing the competency of EUS. Although a previous meta-analysis showed that the pooled estimate of accuracy of overall T staging was
www.giejournal.org
Volume 81, No. 4 : 2015 GASTROINTESTINAL ENDOSCOPY 899
We retrospectively reviewed the clinical records of patients who underwent EUS for gastric cancer performed by trainees at Severance Hospital, Seoul, Republic of Korea, between March 2011 and February 2012. EUS examinations that were not the first in patients were excluded from the study. In addition, EUS examinations performed after chemotherapy, radiotherapy, or endoscopic resection were excluded. As a result, initial EUS examinations for treatment-naïve gastric cancer performed by trainees were included in the study.
EUS procedure
Learning curve for EUS in gastric cancer T staging
75.0%, the accuracy varied vastly across the included studies (range 56.9%-87.7%).6 In addition, significant heterogeneity was identified in the meta-analysis (I2 Z 89.5%). In our study, therefore, the p0 value was defined as adjusted accuracy of the T stage determined by experts by using EUS at our institution instead of the results of previous studies. Next, a p1 value was set to twice p0 as in previous studies.14,19-22 Typical values for type I and type II errors (a and b, respectively) are 10%, and unlike power calculations for clinical trials in the context of performance monitoring, they are typically set to be equal.23 In addition, decision interval (h0) on the CUSUM plot was calculated based on the variables mentioned previously, including acceptable and unacceptable failure rates and type I and type II errors.14 When the CUSUM score decreased below the current decision level, the CUSUM score was replaced by the current decision level. If the CUSUM score exceeded decision interval, in contrast, it indicated an unacceptable performance, and the CUSUM monitoring was then restarted at the next decision interval. The number of EUS examinations and decision interval required for reaching a plateau in each trainee were identified by upward CUSUM plots.16,17 The formulas for S and h0 are as follows14: S Z Q=ðP þ QÞ; where P Z ln ðp1 =p0 Þ and Q Z ln ½ð1 p0 Þ=ð1 p1 Þ h0 Z b=ðP þ QÞ; where b Z ln ½ð1 bÞ=a Finally, logistic regression analysis was used for identifying factors associated with accurate T staging of gastric cancer by using EUS. Variables with significance of 0.2 or less on univariate analysis were included in a logistic regression model. All statistical procedures were conducted by using the statistical software SPSS for Windows, version 18.0 (IBM Inc, Chicago, Ill) with the exception of the CUSUM plots, which were performed in Excel 2010 (Microsoft, Redmond, Wash).
RESULTS Four trainees performed EUS in the study. All trainees had completed a standard 1-year gastroenterology fellowship and had performed 1000 or more diagnostic upper endoscopies. None of the trainees had previous experience or training in EUS. In addition, all of the experts involved in the study had at least 5 years of experience with EUS before March 2011. As shown in Figure 1, of the 574 EUS examinations for gastric cancer performed by trainees, 9 were excluded because they were not the first examination performed in the patient. In addition, 12 examinations performed after chemotherapy, radiotherapy, or endoscopic resection were excluded. Of the remaining 553 examinations, 332 were confirmed by experts. The other 221 examinations were performed by trainees without expert confirmations. In addition, 299 of 900 GASTROINTESTINAL ENDOSCOPY Volume 81, No. 4 : 2015
Park et al
332 lesions, which had been examined by using EUS by both trainees and experts, and 208 of 221 lesions, which had been examined by using EUS by trainees only, were treated by either endoscopic resection or surgical resection; thereafter, pathologic stages were determined for these lesions by pathologic examinations.
Baseline characteristics Patient and lesion characteristics are shown in Table 1. The number of lesions examined by EUS was 160, 153, 85, and 155 for trainees 1, 2, 3, and 4, respectively. Mean age and sex distribution did not differ among the 4 groups classified based on the individuals performing EUS. Although a difference in lesion location was identified, lesion size and pathologic T stages did not differ among the 4 groups. In addition, there was no adverse event related to EUS.
Diagnostic performance of EUS Table 2 shows the diagnostic performance of EUS performed by trainees without experts’ confirmations. The overall accuracy of T staging by trainees by using EUS was 72.6%. Diagnostic performance of EUS performed by trainees with expert confirmation is shown in Table 3. The overall accuracy of T staging by experts by using EUS was 84.3%. As mentioned in the Methods section, the accuracy of EUS T staging by experts was adjusted by standardization according to the distribution of pathologic T stages. The ratio of pathologic T stages in the trainee group was 160:21:17:10 (Table 2), whereas that in the expert group was 254:15:11:19 (Table 3). In cases of pT1 stage, the numbers of examinations in the expert group were multiplied by 0.906, which was calculated by the following formula: ½ðNumber of examinations of pT1 stage in trainee groupÞ= ðTotal number of examinations in trainee groupÞ= ½ðNumber of examinations of pT1 stage in expert groupÞ= ðTotal number of examinations in expert groupÞ Z½ð160=208Þ=ð254=299ÞZ0:906 The calculations for examinations of pT2, pT3, and pT4 stages were conducted along the same lines. After standardization according to the distribution of pathologic T stages, the adjusted overall accuracy of T staging by experts by using EUS was 79.6% (Supplemental Table 1, available online at www.giejournal.org).
CUSUM plots Because the adjusted overall accuracy of T staging by experts by using EUS was 79.6%, acceptable and unacceptable failure rates were set at 0.204 and 0.408, respectively. Then, S and h0 were calculated based on acceptable and unacceptable failure rates and type I and type II errors by using the formulas in the Methods. As a result, S and h0 values were 0.30 and 2.22, respectively. The CUSUM scores were then www.giejournal.org
Park et al
Learning curve for EUS in gastric cancer T staging
EUS examinations for gastric cancer performed by trainee 574 lesions (567 patients) Exclusion
Prior EUS history: 9 lesions (9 patients) Prior CCRT or chemotherapy history: 11 lesions (11 patients) Prior endoscopic resection history: 1 lesion (1 patient)
Initial EUS examinations f e gastric cancer performed by trainee 553 lesions (546 patients)
Staging after expert´s confirmation 332 lesions (329 patients)
With pathologic confirmation 299 lesions (297 patients)
Trainee´s staging only 221 lesions (217 patients)
Without pathologic confirmation 33 lesions (32 patients)
With pathologic confirmation 208 lesions (204 patients)
Without pathologic confirmation 13 lesions (13 patients)
Figure 1. Flow diagram of enrolled patients. CCRT, concurrent chemoradiotherapy.
TABLE 1. Baseline patient and lesion characteristics Trainees who performed EUS for gastric cancer Variable
Trainee 2
Trainee 3
No. of patients
158
152
84
152
546
No. of lesions
160
153
85
155
553
59.4 11.3
59.1 10.9
59.6 11.1
59.1 11.2
Age, y, mean SD
Trainee 4
P value
Trainee 1
Sex, no. (%)
.926
Total
59.2 11.1
.599
Male
107 (67.7)
106 (69.7)
60 (71.4)
97 (63.8)
370 (67.8)
Female
51 (32.3)
46 (30.3)
24 (28.6)
55 (36.2)
176 (32.2)
Location, no. (%)
.020
Upper third
11 (6.9)
22 (14.4)
16 (18.8)
25 (16.1)
201 (12.6)
Middle third
41 (25.6)
43 (28.1)
28 (32.9)
51 (32.9)
518 (32.5)
Lower third
108 (67.5)
88 (57.5)
41 (48.2)
79 (51.0)
873 (54.8)
21.6 16.1
25.3 20.3
20.9 15.8
20.4 11.5
Size, mm, mean SD T stage, no. (%)*
.068
20.7 15.7
.194
pT1
118 (80.3)
122 (85.9)
66 (86.8)
108 (76.1)
414 (81.7)
pT2
11 (7.5)
7 (4.9)
3 (3.9)
15 (10.6)
36 (7.1)
pT3
10 (6.8)
5 (3.5)
6 (7.9)
7 (4.9)
28 (5.5)
pT4
8 (5.4)
8 (5.6)
1 (1.3)
12 (8.5)
29 (5.7)
SD, Standard deviation. *Only lesions confirmed by pathologic examination were included.
www.giejournal.org
Volume 81, No. 4 : 2015 GASTROINTESTINAL ENDOSCOPY 901
Learning curve for EUS in gastric cancer T staging
Park et al
TABLE 2. Diagnostic performance of EUS performed by trainees without experts’ confirmation in patients who underwent endoscopic or surgical resection* EUS staging, no. (%) cT1
cT2
cT3
cT4
pT1
Pathologic staging
136 (85.0)
19 (11.9)
4 (2.5)
1 (0.6)
160
84.1
pT2
5 (23.8)
9 (42.9)
6 (28.6)
1 (4.8)
21
81.3
pT3
3 (17.6)
4 (23.5)
3 (17.6)
7 (41.2)
17
87.5
pT4
1 (10.0)
4 (40.0)
2 (20.0)
3 (30.0)
10
92.3
145
36
15
12
208
72.6
Overall
Overall
Accuracy, %
*Only lesions confirmed by pathologic examination were included.
TABLE 3. Diagnostic performance of EUS performed by trainees with experts’ confirmation in patients who underwent endoscopic or surgical resection* EUS staging, no. (%) cT1
cT2
cT3
cT4
pT1
Pathologic staging
243 (95.7)
9 (3.5)
2 (0.8)
0 (0.0)
254
91.6
pT2
10 (66.7)
3 (20.0)
2 (13.3)
0 (0.0)
15
90.6
pT3
2 (18.2)
1 (9.1)
5 (45.5)
3 (27.3)
11
93.3
pT4
2 (10.5)
6 (31.6)
10 (52.6)
1 (5.3)
19
93.0
257
19
19
4
299
84.3
Overall
Overall
Accuracy, %
*Only lesions confirmed by pathologic examination were included.
calculated based on S and 1 S according to the success of accurate T staging (Fig. 2). For trainee 1, the CUSUM score crossed the first decision interval after the 10th examination and reached a plateau after examination 20 without further crossing of the decision level. For trainees 2 and 3, the CUSUM score crossed the first decision interval after examinations 39 and 60, respectively. The scores then reached a plateau after examinations 41 and 60, respectively, without further crossing of the decision level. Finally, for trainee 4, the CUSUM score first crossed the decision interval after examination 18 and again after examination 54. Although the score reached a plateau after examination 65, the third decision interval was crossed after examination 145.
pT4 stages (OR 0.074; 95% CI, 0.031–0.176; OR 0.088; 95% CI, 0.032–0.238; and OR 0.031; 95% CI, 0.009–0.104, respectively) compared with pT1 stage; and EUS performed by trainees (OR 0.539; 95% CI, 0.295–0.988) were factors independently associated with accurate T staging.
DISCUSSION
Table 4 shows the factors associated with accurate T staging of gastric cancer. On multivariate analysis, a size of 20 to 30 mm and greater than 30 mm (odds ratio [OR] 0.364; 95% confidence interval [CI], 0.186–0.713 and OR 0.135; 95% CI, 0.059–0.310, respectively) compared with a size of 20 mm or less; pT2, pT3, and
Although EUS is useful for locoregional staging of gastric cancer,5,6 training in EUS requires significant time and effort to attain competency. ASGE guidelines suggest that 75 examinations are needed before competency in EUS for mucosal cancer, including esophageal, gastric, and rectal cancer, can be determined. This minimum number of EUS examinations was suggested based on the 2 previous studies performed on the learning curve for EUS in esophageal cancer.11,12 However, only 1 trainee was included in each of the 2 studies. Moreover, approaching the lesion with an endoscope transducer may be more difficult in gastric cancer than in esophageal cancer. Therefore, analysis of the learning curve for EUS in gastric cancer is needed to determine a reasonable number of examinations for training.
902 GASTROINTESTINAL ENDOSCOPY Volume 81, No. 4 : 2015
www.giejournal.org
Associated factors for accurate T staging of gastric cancer
Park et al
Learning curve for EUS in gastric cancer T staging
CUSUM
11.10
Trainee 3 Reaching a plateau after examination 60
8.88
6.66
Trainee 1 Reaching a plateau after examination 20
Trainee 4 Reaching a plateau after examination 65
Trainee 2 Reaching a plateau after examination 41
4.44
2.22
1 6 11 16 21 26 31 36 41 46 51 56 61 66 71 76 81 86 91 96 10 1 10 6 11 1 11 6 12 1 12 6 13 1 13 6 14 1 14 6 15 1 15 6 16 1 16 6
0.00
Number of EUS examinations
Figure 2. Upward CUSUM plots of 4 EUS trainees. CUSUM, cumulative sum.
To our knowledge, our study is the first to demonstrate the learning curve for EUS staging of gastric cancer. All of the learning curves reached a plateau within 65 examinations. This finding supports the suggestion in the ASGE guidelines that a threshold number of 75 may be reasonable. This threshold number is relatively small compared with the minimum number of EUS examinations needed for evaluation of the pancreas.10 The accuracy of T staging by using EUS, however, was not high, even in experts. In our study, the overall accuracy of T staging by trainees and experts by using EUS was 72.6% and 84.3%, respectively. Previous studies also reported that overall accuracy was approximately 70% to 80%.24-26 These results indicate that accurate tumor staging by EUS is challenging, although the required number of EUS examinations for T staging in gastric cancer for achieving competency is relatively small. This was reflected in the learning curve of trainee 4 in our study. Although trainee 4 seemed to achieve competency after 65 examinations, his performance deteriorated at examination 145. Continuous performance monitoring may be needed even after achieving competency. CUSUM analysis would be helpful for self-monitoring and continuous quality improvement in trainees.15,27 In contrast to esophageal cancer, gastric cancer requires relatively complex endoscope control during EUS according to the tumor’s location. Before initiation of this study, we expected lesion location to affect the accuracy of T staging by using EUS. However, it was not a factor associated with accurate T staging of gastric cancer. On the contrary, lesion size was a factor associated accurate T staging. Accuracy of T staging in large lesions (20–30 mm and O30 mm) decreased compared with those in smaller
lesions (%20 mm). This may be because EUS only obtains cross-sectional views of lesions. A more careful and broad scanning of the transducer would be needed for accurate T staging of large lesions. In addition, advanced T stage was a poor predictive factor for accurate T staging by using EUS. This finding was inconsistent with that of a meta-analysis that concluded that EUS may be most useful for staging gastric cancers with greater tumor involvement (T3 and T4).6 The discrepancy may be attributable to the criteria for T staging. Most studies included in the meta-analysis assessed T stages according to the sixth edition of the AJCC/UICC TNM classification, whereas our study used the seventh edition of the AJCC/ UICC TNM classification. Under the current TNM classification, subserosal penetration and serosa invasion are classified as T3 and T4, respectively. However, it is difficult to differentiate between tumors that penetrate the subserosal connective tissue and those that invade the serosa. Excluded lesions that were inoperable could be another reason for the low accuracy rate in advanced cancers. Although this is the first study to assess the learning curve for EUS in gastric cancer T staging, the number of trainees was too small to provide a definitive threshold number of EUS examinations. Our results, therefore, may be considered only preliminary. Our CUSUM analysis, however, was shown to be useful for assessing continuous quality assurance. CUSUM analysis is a method that can monitor continuously a production process and detect subtle deviations from a preset defined level of achievement.14 In the cases in our study, the deterioration of performance of trainee 4 was identified on a CUSUM plot. If the deterioration of performance was
www.giejournal.org
Volume 81, No. 4 : 2015 GASTROINTESTINAL ENDOSCOPY 903
Learning curve for EUS in gastric cancer T staging
Park et al
TABLE 4. Associated factors for accurate T staging of gastric cancer
Variable
EUS exams
Accurate T staging, no. (%)
Age, y
Univariate analysis P value
Multivariate analysis OR
95% CI
P value
0.834-2.875
.166
.261
R60
250
193 (77.2)
!60
251
204 (81.3)
Sex
.021
Male
341
280 (82.1)
1.548
Female
160
117 (73.1)
1.000
Location
.115
Upper third
60
47 (78.3)
1.203
0.467-3.100
.701
Middle third
146
108 (74.0)
0.650
0.336-1.259
.201
Lower third
301
248 (82.4)
1.000 !.001
Size, mm %20
329
304 (92.4)
1.000
20-30
114
80 (70.2)
0.364
0.186-0.713
.003
O30
64
19 (29.7)
0.135
0.059-0.310
!.001
!.001
T stage pT1
414
379 (91.5)
1.000
pT2
36
12 (33.3)
0.074
0.031-0.176
!.001
pT3
28
8 (28.6)
0.088
0.032-0.238
!.001
pT4
29
4 (13.8)
0.031
0.009-0.104
!.001
0.295-0.988
.045
EUS
.001
By expert
299
252 (84.3)
1.000
By trainee
208
151 (72.6)
0.539
OR, Odds ratio; CI, confidence interval.
detected during CUSUM monitoring, re-education could be provided; it would then help to prevent further deterioration. Our study has several other limitations. This was a retrospective study; therefore, results of staging by trainees were unknown when the EUS examinations were confirmed by experts. In addition, our study focused on gastric cancer T staging rather than N staging. To accurately determine the minimum number of examinations for EUS staging, analysis of the learning curve for N staging is also necessary. A prospective study on the learning curve for both T and N staging with more trainees and patients would help provide a definite conclusion. Despite the preceding limitations, our data form the basis of a system for recommending the minimum number of EUS examinations needed for achieving competency. The CUSUM scores of all of 4 trainees in the study reached
a plateau by the 65th examination. A threshold number of 75, as suggested by ASGE guidelines, may be reasonable for achieving competency in gastric cancer T staging by using EUS.
904 GASTROINTESTINAL ENDOSCOPY Volume 81, No. 4 : 2015
www.giejournal.org
REFERENCES 1. Crew KD, Neugut AI. Epidemiology of gastric cancer. World J Gastroenterol 2006;12:354-62. 2. Shah MA, Kelsen DP. Gastric cancer: a primer on the epidemiology and biology of the disease and an overview of the medical management of advanced disease. J Natl Compr Canc Netw 2010;8:437-47. 3. Japanese Gastric Cancer Association. Japanese gastric cancer treatment guidelines 2010 (ver. 3). Gastric Cancer 2011;14:113-23. 4. Gotoda T, Yamamoto H, Soetikno RM. Endoscopic submucosal dissection of early gastric cancer. J Gastroenterol 2006;41:929-42. 5. Mocellin S, Marchet A, Nitti D. EUS for the staging of gastric cancer: a meta-analysis. Gastrointest Endosc 2011;73:1122-34.
Park et al
Learning curve for EUS in gastric cancer T staging
6. Cardoso R, Coburn N, Seevaratnam R, et al. A systematic review and meta-analysis of the utility of EUS for preoperative staging for gastric cancer. Gastric Cancer 2012;15(Suppl 1):S19-26. 7. Ajani JA, Bentrem DJ, Besh S. Gastric cancer, version 2.2013: featured updates to the NCCN Guidelines. J Natl Compr Canc Netw 2013;11: 531-46. 8. Maluf Filho F, Dotti CM, Halwan B, et al. An evidence-based consensus statement on the role and application of endosonography in clinical practice. Endoscopy 2009;41:979-87. 9. Eisen GM, Dominitz JA, Faigel DO, et al. Guidelines for credentialing and granting privileges for endoscopic ultrasound. Gastrointest Endosc 2001;54:811-4. 10. Wani S, Cotà GA, Keswani R, et al. Learning curves for EUS by using cumulative sum analysis: implications for American Society for Gastrointestinal Endoscopy recommendations for training. Gastrointest Endosc 2013;77:558-65. 11. Fockens P, Van den Brande JH, van Dullemen HM, et al. Endosonographic T-staging of esophageal carcinoma: a learning curve. Gastrointest Endosc 1996;44:58-62. 12. Schlick T, Heintz A, Junginger T. The examiner’s learning effect and its influence on the quality of endoscopic ultrasonography in carcinoma of the esophagus and gastric cardia. Surg Endosc 1999;13: 894-8. 13. Greene FL; American Joint Committee on Cancer; American Cancer Society. AJCC cancer staging manual. New York (NY): Springer-Verlag; 2010. 14. Bolsin S, Colson M. The use of the CUSUM technique in the assessment of trainee competence in new procedures. Int J Qual Health Care 2000;12:433-8. 15. Grunkemeier GL, Jin R, Wu Y. Cumulative sum curves and their prediction limits. Ann Thorac Surg 2009;87:361-4. 16. Jeong O, Ryu SY, Choi WY, et al. Risk factors and learning curve associated with postoperative morbidity of laparoscopic total gastrectomy for gastric carcinoma. Ann Surg Oncol 2014;21:2994-3001.
17. Cho SY, Choo MS, Jung JH, et al. Cumulative sum analysis for experiences of a single-session retrograde intrarenal stone surgery and analysis of predictors for stone-free status. PLoS One 2014;9:e84878-e. 18. Ward ST, Mohammed MA, Walt R, et al. An analysis of the learning curve to achieve competency at colonoscopy using the JETS database. Gut. Epub 2014 Jan 27. 19. Williams SM, Parry BR, Schlup MM. Quality control: an application of the CUSUM. BMJ 1992;304:1359-61. 20. Lee YK, Ha YC, Hwang DS, et al. Learning curve of basic hip arthroscopy technique: CUSUM analysis. Knee Surg Sports Traumatol Arthrosc 2013;21:1940-4. 21. Kheir F, Alokla K, Myers L, et al. Endobronchial ultrasoundtransbronchial needle aspiration of mediastinal and hilar lymphadenopathy learning curve. Am J Ther. Epub 2014 Mar 10. 22. Je S, Cho Y, Choi HJ, et al. An application of the learning curvecumulative summation test to evaluate training for endotracheal intubation in emergency medicine. Emerg Med J. Epub 2013 Oct 23. 23. Noyez L. Control charts, Cusum techniques and funnel plots. A review of methods for monitoring performance in healthcare. Interact Cardiovasc Thorac Surg 2009;9:494-9. 24. Feng X, Wang W, Luo G, et al. Comparison of endoscopic ultrasonography and multislice spiral computed tomography for the preoperative staging of gastric cancer - results of a single institution study of 610 Chinese patients. PLoS One 2013;8:e78846-e. 25. Hwang SW, Lee DH, Lee SH, et al. Preoperative staging of gastric cancer by endoscopic ultrasonography and multidetector-row computed tomography. J Gastroenterol Hepatol 2010;25:512-8. 26. Repiso A, Gómez-Rodríguez R, López Pardo R, et al. Usefulness of endoscopic ultrasonography in preoperative gastric cancer staging: diagnostic yield and therapeutic impact. Rev Esp Enferm Dig 2010;102:413-20. 27. Sibanda T, Sibanda N. The CUSUM chart method as a tool for continuous monitoring of clinical outcomes using routinely collected data. BMC Med Res Methodol 2007;7:46.
www.giejournal.org
Volume 81, No. 4 : 2015 GASTROINTESTINAL ENDOSCOPY 905
Learning curve for EUS in gastric cancer T staging
Park et al
APPENDIX
SUPPLEMENTAL TABLE 1. Diagnostic performance of EUS performed by trainees with expert confirmation in patients who underwent endoscopic or surgical resection after pathologic T stage standardization* EUS staging, no. (%) Pathologic staging
cT1
cT2
cT3
cT4
pT1
220.0 (95.7)
8.1 (3.5)
1.8 (0.8)
0.0 (0.0)
230.0
87.9
pT2
20.1 (66.7)
6.0 (20.0)
4.0 (13.3)
0.0 (0.0)
30.2
86.9
pT3
4.4 (18.2)
2.2 (9.1)
11.1 (45.5)
6.7 (27.3)
24.4
91.1
pT4
1.5 (10.5)
4.5 (31.6)
7.6 (52.6)
0.8 (5.3)
14.4
93.2
246.1
20.9
24.5
7.4
299.0
79.6
Overall
Overall
Accuracy, %
*Only lesions confirmed by pathologic examination were included.
905.e1 GASTROINTESTINAL ENDOSCOPY Volume 81, No. 4 : 2015
www.giejournal.org