- Email: [email protected]
364 Assessment of surgical competency for robot-assisted radical prostatectomy: Development and validation of Prostatectomy Assessment and Competency Evaluation (PACE)
364
Assessment of surgical competency for robot-assisted radical prostatectomy: Development and validation of Prostatectomy Assessment and Competency Evaluation (PACE) Eur Urol Suppl 2016;15(3);e364
Print! Print!
Ghani K.R. 1 , Aly A. 2 , Peabody J. 3 , Lane B. 4 , Sarle R.5 , Abaza R.6 , Montgomery J. 1 , Hu J. 7 , Eun D.8 , Fumo M. 9 , Comstock B. 10 , Linsell S. 1 , Miller D.C.1 , Guru K. 2 1 University
of Michigan, Dept. of Urology, Ann Arbor, United States of America, 2 Roswell Park Cancer Center, Dept. of Urology, Buffalo,
United States of America, 3 Henry Ford Hospital, Vattikuti Urology Institute, Detroit, United States of America, 4 Spectrum Health, Dept. of Urology, Grand Rapids, United States of America, 5 Michigan Institute Urology, Dept. of Urology, Dearborn, United States of America, 6 OhioHealth,
Dept. of Urology, Columbus, United States of America, 7 Cornell University, Dept. of Urology, New Tork, United States of
America, 8 Temple University, Dept. of Urology, Philadelphia, United States of America, 9 Rockford Urological Associates, Dept. of Urology, Rockford, United States of America, 10 University of Washington, Dept. of Biostatistics, Seattle, United States of America INTRODUCTION & OBJECTIVES: With the widespread adoption of robot-assisted surgery, it is vital to ensure skill acquisition and maintenance of competency aligns with best surgical outcomes and patient safety. We aimed to develop and validate a scoring tool for robot-assisted radical prostatectomy (RARP) - Prostatectomy Assessment and Competence Evaluation (PACE) - that objectively measures surgical performance during RARP in qualified surgeons. MATERIAL & METHODS: A multi-institutional study was conducted in two phases using surgeons from the Michigan Urological Surgery Improvement Collaborative (MUSIC), which is a consortium of 42 diverse urology practices in the state of Michigan. The first phase was development and content validation of PACE by a panel of 10 experienced robotic surgeons who deconstructed the critical steps of RARP into 7 key domains utilizing the Delphi methodology. Anchor description for poor and ideal level of performance was assigned Likert scores 1 and 5, respectively. Content validation index (CVI) was used to validate the scoring system and report consensus in phase 1. The second phase assessed reliability through assessment of de-identified RARP videos from 10 attending surgeons within MUSIC. Video clips of the seven key steps for each procedure were placed on a web-based system and rated in a blinded manner by 23 robotic surgeons. Each surgical step was reviewed by at least 4 expert reviewers using a fully crossed design. A weighted average was used to compare scores between surgeons. Inter-rater reliability was established by determining the intra-class correlation (ICC). RESULTS: CVI: The expert panel reached consensus after 3 rounds on all aspects, which included language, relevance of skills assessed, and concordance between the language used and the skill assessed. CVI >0.75 was achieved in 56 statements in the first round, 31 statements in the second, and consensus on the 3 remaining statements after the third. PACE: The seven domains of PACE are bladder drop, preparation of the prostate, bladder neck dissection, dissection of the seminal vesicles and posterior plane, preparation of the neurovascular bundle, apical dissection, and anastomosis. Mean evaluation scores for the ten surgeons ranged from 3.34 to 4.39. Reliability: In this cohort without trainee surgeon performances, ICC values were stable. ICC was strongest (>0.3) for dissection of the bladder neck, dissection of the seminal vesicles, and anastomosis, followed by bladder drop, and neurovascular bundle preparation (0.2 to 0.3). CONCLUSIONS: We describe the first procedure-specific scoring system for RARP validated using video performances from qualified robotic surgeons. PACE allows for evaluation of surgical competency of RARP in qualified surgeons, and may have a role in the assessment of surgical performance. Further work is needed to determine if surgeon scores on specific domains predicts patient outcomes.
Assessment of surgical competency for robot-assisted radical prostatectomy: Development and validation of Prostatectomy Assessment and Competency Evaluation (PACE) Eur Urol Suppl 2016;15(3);e364
Print! Print!
Ghani K.R. 1 , Aly A. 2 , Peabody J. 3 , Lane B. 4 , Sarle R.5 , Abaza R.6 , Montgomery J. 1 , Hu J. 7 , Eun D.8 , Fumo M. 9 , Comstock B. 10 , Linsell S. 1 , Miller D.C.1 , Guru K. 2 1 University
of Michigan, Dept. of Urology, Ann Arbor, United States of America, 2 Roswell Park Cancer Center, Dept. of Urology, Buffalo,
United States of America, 3 Henry Ford Hospital, Vattikuti Urology Institute, Detroit, United States of America, 4 Spectrum Health, Dept. of Urology, Grand Rapids, United States of America, 5 Michigan Institute Urology, Dept. of Urology, Dearborn, United States of America, 6 OhioHealth,
Dept. of Urology, Columbus, United States of America, 7 Cornell University, Dept. of Urology, New Tork, United States of
America, 8 Temple University, Dept. of Urology, Philadelphia, United States of America, 9 Rockford Urological Associates, Dept. of Urology, Rockford, United States of America, 10 University of Washington, Dept. of Biostatistics, Seattle, United States of America INTRODUCTION & OBJECTIVES: With the widespread adoption of robot-assisted surgery, it is vital to ensure skill acquisition and maintenance of competency aligns with best surgical outcomes and patient safety. We aimed to develop and validate a scoring tool for robot-assisted radical prostatectomy (RARP) - Prostatectomy Assessment and Competence Evaluation (PACE) - that objectively measures surgical performance during RARP in qualified surgeons. MATERIAL & METHODS: A multi-institutional study was conducted in two phases using surgeons from the Michigan Urological Surgery Improvement Collaborative (MUSIC), which is a consortium of 42 diverse urology practices in the state of Michigan. The first phase was development and content validation of PACE by a panel of 10 experienced robotic surgeons who deconstructed the critical steps of RARP into 7 key domains utilizing the Delphi methodology. Anchor description for poor and ideal level of performance was assigned Likert scores 1 and 5, respectively. Content validation index (CVI) was used to validate the scoring system and report consensus in phase 1. The second phase assessed reliability through assessment of de-identified RARP videos from 10 attending surgeons within MUSIC. Video clips of the seven key steps for each procedure were placed on a web-based system and rated in a blinded manner by 23 robotic surgeons. Each surgical step was reviewed by at least 4 expert reviewers using a fully crossed design. A weighted average was used to compare scores between surgeons. Inter-rater reliability was established by determining the intra-class correlation (ICC). RESULTS: CVI: The expert panel reached consensus after 3 rounds on all aspects, which included language, relevance of skills assessed, and concordance between the language used and the skill assessed. CVI >0.75 was achieved in 56 statements in the first round, 31 statements in the second, and consensus on the 3 remaining statements after the third. PACE: The seven domains of PACE are bladder drop, preparation of the prostate, bladder neck dissection, dissection of the seminal vesicles and posterior plane, preparation of the neurovascular bundle, apical dissection, and anastomosis. Mean evaluation scores for the ten surgeons ranged from 3.34 to 4.39. Reliability: In this cohort without trainee surgeon performances, ICC values were stable. ICC was strongest (>0.3) for dissection of the bladder neck, dissection of the seminal vesicles, and anastomosis, followed by bladder drop, and neurovascular bundle preparation (0.2 to 0.3). CONCLUSIONS: We describe the first procedure-specific scoring system for RARP validated using video performances from qualified robotic surgeons. PACE allows for evaluation of surgical competency of RARP in qualified surgeons, and may have a role in the assessment of surgical performance. Further work is needed to determine if surgeon scores on specific domains predicts patient outcomes.