- Email: [email protected]
Su1347 Predicting Endoscopist Technical Skill Level Utilizing Kinematic Data
Abstracts
Spearman correlation coefficient. Results: Adenomatous polyps were detected in 4/14 (29%) of the collected colonoscopy video samples. There were large differences in the withdrawal times during which no polyps were removed (range 4 to 12 minutes). The median quality rating from the automated system and the reviewers was 3.45 (interquartile range [IQR], 3.1-3.68) and 2.67 (IQR, 2.33-3) respectively for all colonoscopy video samples. The automated rating revealed a strong correlation with the reviewer’s rating ( coefficient⫽ 0.65, p ⫽ 0.01). Conclusions: In this novel, preliminary study, the results from the automated system strongly agreed with the endoscopists’ quality assessments. This tool could facilitate real-time colonoscopy quality feedback for clinical practice, with easily accessible, in depth quality metrics for individual patients. Further study is required in order to better define blurriness and velocity thresholds and to validate this approach.
Su1347 Predicting Endoscopist Technical Skill Level Utilizing Kinematic Data Keith L. Obstein*1, Rowena Ong2, James C. Slaughter3, Robert L. Galloway2 1 Division of Gastroenterology, Vanderbilt University Medical Center, Nashville, TN; 2Biomedical Engineering, Vanderbilt University, Nashville, TN; 3Biostatistics, Vanderbilt University School of Medicine, Nashville, TN Adenoma Detection Rate comparing phase 2 and phase 3
Adenomas per patient comparing phases 2 and 3.
with good quality metrics. Many colon segments may have low diagnostic yield caused by poor visualization of colonic surface and transiently rapid withdrawal, which are not captured by total withdrawal time. A novel real-time colonoscopy video quality indication system was developed for evaluating the adequacy of image clarity and withdrawal velocity. The aim of the present pilot study was to comparatively validate the performance of this new colonoscopy quality assessment system. Methods: The software-based system employs a novel image processing algorithm which detects the levels of image clarity and withdrawal velocity in a real-time fashion from live video signal. Threshold levels of image blurriness and the withdrawal velocity below which the visualization could be considered adequate have initially been determined arbitrarily by review of sample colonoscopy videos by two experienced endoscopists. Subsequently, an overall colonoscopy quality rating was computed based on the percentage of the withdrawal time with adequate visualization (scored 1-5; 1 when the percentage was 1-20%, 2 when the percentage was 21-40%, etc.). In order to test the proposed velocity and blurriness thresholds, 14 screening colonoscopy withdrawal videos from a specialized ambulatory colon cancer screening center were collected, automatically processed and rated. Three experienced endoscopists reviewed the collected videos in a blinded fashion and rated the overall quality of each (scored 1-5; 1-poor, 3-average, 5-excellent) based on 2 major aspects: image quality and withdrawal velocity. The automated quality ratings were compared to the averaged endoscopist quality ratings using
Background: Colonoscopy (CLS) requires training and experience to ensure accuracy and safety. There is no objective, validated process to determine when an endoscopist has attained technical competence. Kinematic data describing movement of the colonoscope has been demonstrated to be an objective and quantitative metric for assessing technical skill level. We have developed a novel system and model for predicting technical skill level using kinematic data. Aim To utilize kinematic analysis of CLS to predict endoscopist technical skill level. Methods: 5 attending endoscopists with ⬎1500 colonoscopies and 7 GI fellows with ⬍1000 colonoscopies were selected from a tertiary care hospital to perform 3 colonoscopies on a single standardized colon model with a variable stiffness adult colonoscope. Seven electromagnetic sensors were embedded along the length of a catheter and placed into the therapeutic channel of the colonoscope. Kinematic data was captured beginning with initial insertion and ended with terminal ileum (TI) intubation. Measured parameters included time, path length (PL), flex, velocity, acceleration, jerk, tip angulation, angular velocity, rotation, and curvature. A univariable logistic regression model was used and a c-index (cidx) was calculated. Based on the cidx, an equation was developed to predict technical skill level. The equation was then validated in a separate test group of 2 attendings and 3 fellows. Results: All endoscopists reached the TI of the standard colon model. Attendings reached the TI more quickly than fellows (150 [145,151] s v. 232 [209,371] s; p⫽0.01), with reduced path length (1.7 [1.1,1.9] m v. 3.4 [2.3,5] m, p⫽0.01), and less curvature (0.012 [0.011,0.018] v. 0.024 [0.022,0.034], p⬍0.01). Two covariates had significantly higher cidx than other covariates (PL 4: 0.95 [0.87,0.99] and absolute sum angle 4: 0.96 [0.88,0.99]) and when combined as predictors of attending status the cidx remained at 0.96 [0.90,0.99]. The equation, P(fellow)⫽1/(1⫹exp(-X)); X⫽6.98⫹0.00147*sumangle4⫹0.00154*PL4, was developed where a negative value is more likely to be an attending. This equation accurately predicted attending status 84% of the time when utilized in the test group. Conclusions: Kinematic data successfully differentiated the technical skill level of endoscopists (attending v. fellow). Time, PL, and curvature, which were found to be significant in prior independent studies, remained significant. Kinematic data effectively predicted technical skill level in an objective, quantitative fashion. As governing bodies increase their focus on quantification of competence and quality in medical applications, kinematic data may serve as a method of measurement in colonoscopy. Ongoing studies are focused on refining characteristic profiles, differentiating performance algorithms, and enhancing the kinematic data collection system.
Su1348 Does Reporting Semi-Annual Adenoma Detection Rate Increase Adenoma Detection Rate Over Time? Stacy B. Menees*, Grace H. Elta University of Michigan, Ann Arbor, MI Background: Adenoma detection rate (ADR) is a colonoscopy quality measure advocated by the Centers for Medicare & Medicaid (CMS). By 2015, ADR is proposed to be incorporated into a Physician Quality Index formula that will positively or negatively impact colonoscopy reimbursement. Therefore, further study of adenoma detection rate is needed. The aim of our study was to assess the impact of biannual reports to gastroenterologists’(GIs) regarding their ADR rate compared to their peers. Methods: Blinded observational study of the institution of semi-annual audit of GIs’ ADR over an 18 month period at a Tertiary Care center. Inclusion criteria: GIs who perform screening/surveillance colonoscopy. Exclusion criteria: Less than 40 colonoscopies in 6 months and
AB301 GASTROINTESTINAL ENDOSCOPY Volume 75, No. 4S : 2012
www.giejournal.org
Spearman correlation coefficient. Results: Adenomatous polyps were detected in 4/14 (29%) of the collected colonoscopy video samples. There were large differences in the withdrawal times during which no polyps were removed (range 4 to 12 minutes). The median quality rating from the automated system and the reviewers was 3.45 (interquartile range [IQR], 3.1-3.68) and 2.67 (IQR, 2.33-3) respectively for all colonoscopy video samples. The automated rating revealed a strong correlation with the reviewer’s rating ( coefficient⫽ 0.65, p ⫽ 0.01). Conclusions: In this novel, preliminary study, the results from the automated system strongly agreed with the endoscopists’ quality assessments. This tool could facilitate real-time colonoscopy quality feedback for clinical practice, with easily accessible, in depth quality metrics for individual patients. Further study is required in order to better define blurriness and velocity thresholds and to validate this approach.
Su1347 Predicting Endoscopist Technical Skill Level Utilizing Kinematic Data Keith L. Obstein*1, Rowena Ong2, James C. Slaughter3, Robert L. Galloway2 1 Division of Gastroenterology, Vanderbilt University Medical Center, Nashville, TN; 2Biomedical Engineering, Vanderbilt University, Nashville, TN; 3Biostatistics, Vanderbilt University School of Medicine, Nashville, TN Adenoma Detection Rate comparing phase 2 and phase 3
Adenomas per patient comparing phases 2 and 3.
with good quality metrics. Many colon segments may have low diagnostic yield caused by poor visualization of colonic surface and transiently rapid withdrawal, which are not captured by total withdrawal time. A novel real-time colonoscopy video quality indication system was developed for evaluating the adequacy of image clarity and withdrawal velocity. The aim of the present pilot study was to comparatively validate the performance of this new colonoscopy quality assessment system. Methods: The software-based system employs a novel image processing algorithm which detects the levels of image clarity and withdrawal velocity in a real-time fashion from live video signal. Threshold levels of image blurriness and the withdrawal velocity below which the visualization could be considered adequate have initially been determined arbitrarily by review of sample colonoscopy videos by two experienced endoscopists. Subsequently, an overall colonoscopy quality rating was computed based on the percentage of the withdrawal time with adequate visualization (scored 1-5; 1 when the percentage was 1-20%, 2 when the percentage was 21-40%, etc.). In order to test the proposed velocity and blurriness thresholds, 14 screening colonoscopy withdrawal videos from a specialized ambulatory colon cancer screening center were collected, automatically processed and rated. Three experienced endoscopists reviewed the collected videos in a blinded fashion and rated the overall quality of each (scored 1-5; 1-poor, 3-average, 5-excellent) based on 2 major aspects: image quality and withdrawal velocity. The automated quality ratings were compared to the averaged endoscopist quality ratings using
Background: Colonoscopy (CLS) requires training and experience to ensure accuracy and safety. There is no objective, validated process to determine when an endoscopist has attained technical competence. Kinematic data describing movement of the colonoscope has been demonstrated to be an objective and quantitative metric for assessing technical skill level. We have developed a novel system and model for predicting technical skill level using kinematic data. Aim To utilize kinematic analysis of CLS to predict endoscopist technical skill level. Methods: 5 attending endoscopists with ⬎1500 colonoscopies and 7 GI fellows with ⬍1000 colonoscopies were selected from a tertiary care hospital to perform 3 colonoscopies on a single standardized colon model with a variable stiffness adult colonoscope. Seven electromagnetic sensors were embedded along the length of a catheter and placed into the therapeutic channel of the colonoscope. Kinematic data was captured beginning with initial insertion and ended with terminal ileum (TI) intubation. Measured parameters included time, path length (PL), flex, velocity, acceleration, jerk, tip angulation, angular velocity, rotation, and curvature. A univariable logistic regression model was used and a c-index (cidx) was calculated. Based on the cidx, an equation was developed to predict technical skill level. The equation was then validated in a separate test group of 2 attendings and 3 fellows. Results: All endoscopists reached the TI of the standard colon model. Attendings reached the TI more quickly than fellows (150 [145,151] s v. 232 [209,371] s; p⫽0.01), with reduced path length (1.7 [1.1,1.9] m v. 3.4 [2.3,5] m, p⫽0.01), and less curvature (0.012 [0.011,0.018] v. 0.024 [0.022,0.034], p⬍0.01). Two covariates had significantly higher cidx than other covariates (PL 4: 0.95 [0.87,0.99] and absolute sum angle 4: 0.96 [0.88,0.99]) and when combined as predictors of attending status the cidx remained at 0.96 [0.90,0.99]. The equation, P(fellow)⫽1/(1⫹exp(-X)); X⫽6.98⫹0.00147*sumangle4⫹0.00154*PL4, was developed where a negative value is more likely to be an attending. This equation accurately predicted attending status 84% of the time when utilized in the test group. Conclusions: Kinematic data successfully differentiated the technical skill level of endoscopists (attending v. fellow). Time, PL, and curvature, which were found to be significant in prior independent studies, remained significant. Kinematic data effectively predicted technical skill level in an objective, quantitative fashion. As governing bodies increase their focus on quantification of competence and quality in medical applications, kinematic data may serve as a method of measurement in colonoscopy. Ongoing studies are focused on refining characteristic profiles, differentiating performance algorithms, and enhancing the kinematic data collection system.
Su1348 Does Reporting Semi-Annual Adenoma Detection Rate Increase Adenoma Detection Rate Over Time? Stacy B. Menees*, Grace H. Elta University of Michigan, Ann Arbor, MI Background: Adenoma detection rate (ADR) is a colonoscopy quality measure advocated by the Centers for Medicare & Medicaid (CMS). By 2015, ADR is proposed to be incorporated into a Physician Quality Index formula that will positively or negatively impact colonoscopy reimbursement. Therefore, further study of adenoma detection rate is needed. The aim of our study was to assess the impact of biannual reports to gastroenterologists’(GIs) regarding their ADR rate compared to their peers. Methods: Blinded observational study of the institution of semi-annual audit of GIs’ ADR over an 18 month period at a Tertiary Care center. Inclusion criteria: GIs who perform screening/surveillance colonoscopy. Exclusion criteria: Less than 40 colonoscopies in 6 months and
AB301 GASTROINTESTINAL ENDOSCOPY Volume 75, No. 4S : 2012
www.giejournal.org