- Email: [email protected]
Development and evaluation of a simulation-based continuing medical education course: beyond lectures and credit hours
Accepted Manuscript Development and evaluation of a simulation-based CME course: beyond lectures and credit-hours Carla M. Pugh, M.D., Ph.D., Fahd O. Arafat, M.D., Calvin Kwan, B.S., Elaine R. Cohen, M.Ed, Yo Kurashima, M.D., Melina C. Vassiliou, M.D., M.Ed, Gerald M. Fried, M.D., Shannon DiMarco PII:
S0002-9610(15)00439-0
DOI:
10.1016/j.amjsurg.2015.05.034
Reference:
AJS 11628
To appear in:
The American Journal of Surgery
Received Date: 18 February 2015 Revised Date:
15 May 2015
Please cite this article as: Pugh CM, Arafat FO, Kwan C, Cohen ER, Kurashima Y, Vassiliou MC, Fried GM, DiMarco S, Development and evaluation of a simulation-based CME course: beyond lectures and credit-hours, The American Journal of Surgery (2015), doi: 10.1016/j.amjsurg.2015.05.034. This is a PDF file of an unedited manuscript that has been accepted for publication. As a service to our customers we are providing this early version of the manuscript. The manuscript will undergo copyediting, typesetting, and review of the resulting proof before it is published in its final form. Please note that during the production process errors may be discovered which could affect the content, and all legal disclaimers that apply to the journal pertain.
ACCEPTED MANUSCRIPT
Development and Evaluation of a Simulation-Based CME course: Beyond Lectures and Credit-hours Carla M. Pugh, MD, PhD1, Fahd O. Arafat, MD2, Calvin Kwan, BS1, Elaine R. Cohen, MEd1, Yo Kurashima, MD3, Melina C. Vassiliou, MD, MEd3, Gerald M. Fried, MD3 1
EP
TE D
M AN U
SC
RI PT
University of Wisconsin School of Medicine and Public Health, Department of Surgery 2 Northwestern University, Department of Surgery, Chicago, IL 3 The Department of Surgery, Steinberg-Bernstein Centre for Minimally Invasive Surgery & Innovation, and The Arnold and Blema Steinberg Medical Simulation Centre, McGill University, Montréal, QC, Canada
AC C
Funding: Department of Defense USAMRMC Grant # W81XWH0710190
Corresponding Author: Carla Pugh, MD, PhD Vice-Chair of Education and Patient Safety Department of Surgery H4/785B Clinical Science Center 600 Highland Avenue, Madison WI 53792 Email: [email protected] Phone (608)-265-9574 Fax (608)-252-0936
ACCEPTED MANUSCRIPT
ABSTRACT
RI PT
Background: The aim of our study was to modify our previously developed LVH simulator to increase difficulty and then reassess validity and feasibility for using the simulator in a newly developed simulation-based Continuing Medical Education (CME) course.
M AN U
SC
Methods: Participants (N=30) were practicing surgeons who signed up for a hands-on postgraduate laparoscopic hernia course. A laparoscopic ventral hernia (LVH) simulator, with prior validity evidence, was modified for the course to increase difficulty. Participants completed one of three variations in hernia anatomy: incarcerated omentum, incarcerated bowel, and diffuse adhesions. During the procedure, course faculty and peer observers rated surgeon performance using GOALS-IH and GOALS rating scales with prior validity evidence. Rating scale reliability was reassessed for internal consistency. Peer and faculty raters’ scores were compared. In addition, quality and completeness of the hernia repair were rated.
TE D
Results: Internal consistency on the general skills performance (peer α=.96, faculty α=.94) and procedure specific performance (peer α=.91, faculty α=.88) scores were high. Peers were more lenient than faculty raters on all LVH items in both the procedure specific skills and general skills ratings. Overall, participants scored poorly on the quality and completeness of their hernia repairs (M=3.90/16 SD=2.72), suggesting a mis-match between course attendees and hernia difficulty and identifying a learning need.
EP
Conclusion: Simulation-based CME courses provide hands-on experiences that can positively affect clinical practice. While our data appears to show a significant mis-match between clinical skill and simulator difficulty, these findings also underscore significant learning needs in the surgical community.
AC C
Keywords: Simulation, Validity, Training, CME, Evaluation, Surgery, Laparoscopy
ACCEPTED MANUSCRIPT 1
BACKGROUND A commitment to continuous learning and practice improvement is essential in everyday
RI PT
clinical practice. However, the traditional implementation of continuing medical education allows course attendance and course credits to overshadow measured achievement and internal motivation for excellence. Continuing Medical Education is part of a rapidly changing system of professional development that includes assessment, remediation and reassessment. Although jurisdictional
SC
requirements may vary from state to state, most simply require a number of hours of Continuing Medical Education to be completed annually to satisfy licensure needs.1 Currently, the larger focus
M AN U
for Continuing Medical Education is the hours needed to maintain licensure and certification. However, the motivation behind the original development of Continuing Medical Education programs was to inspire lifelong learning. With the increasing emphasis on quality in healthcare, Continuing Medical Education has great potential to move from a basic focus on maintenance of
TE D
licensure to improving quality in clinical practice and ensuring ongoing physician competence. While simulation has been widely accepted as a training and assessment modality in Graduate Medical Education, use in Continuing Medical Education is not as common. In addition,
EP
the external drivers for objective assessment of clinical skill during residency training are on the rise. Many of the residency review programs are requiring documentation of annual evaluations of skill in
AC C
a hands-on setting, away from direct patient care.2 These evaluations include assessments of knowledge, skills and attitudes. However, following completion of training, these types of assessments are sparse. The exception is the newly implemented Maintenance of Certification program in the United States.3 These programs, as defined by specialty board organizations, are bringing focus to the need for ongoing assessment after residency training. In the medical literature, there is a paucity of research on the use of simulation-based technology in Continuing Medical Education courses.4,5 One study used a proficiency-based
1
ACCEPTED MANUSCRIPT 2 curriculum as part of a Continuing Medical Education course. This group demonstrated the feasibility of a half-day Continuing Medical Education course to improve performance in laparoscopic suturing.6 The American Board of Anesthesia has also made some headway in
RI PT
developing simulation-based assessments for maintenance of certification. Overall, research in this field is largely based on self-report satisfaction data and lacks any standardized performance evaluation.7
SC
The American College of Surgeons supports the use of simulation-based surgical education to enhance patient safety, meet the requirements for Maintenance of Certification and address the
M AN U
core competencies that all surgeons and trainees are required to achieve.8 Despite the American College of Surgeons support, development, implementation and evaluation of simulation-based Continuing Medical Education courses are lacking. In order to achieve these Global Operative Assessment of Laparoscopic Skills, valid and reliable measures of performance are necessary. Our prior work using the Laparoscopic Ventral Hernia simulator revealed the importance of
TE D
intra-operative decision-making for this procedure. This work underscored the need for decisionbased metrics in addition to those used to assess technical skills.9 The aim of our current study was to modify our previous Laparoscopic Ventral Hernia simulator to increase difficulty and then
EP
reassess validity and feasibility for use in a Continuing Medical Education course. Specifically, we
AC C
sought to assess the following: (1) the validity and reliability of a previously developed procedurespecific (Global Operative Assessment of Laparoscopic Skills-Incisional Hernia) rating scale and global (Generic Global Operative Assessment of Laparoscopic Skills ) rating scale largely used for Graduate Medical Education and (2) to document validity support for simulated Laparoscopic Ventral Hernia scenarios of increasing difficulty.
METHODS Setting and Participants
2
ACCEPTED MANUSCRIPT 3 This study was performed at the 97th Clinical Congress in San Francisco, California in 2011. The Clinical Congress is designed to provide individuals with a wide range of learning opportunities, activities, and experiences that will match their educational and professional development needs.
RI PT
Practicing surgeons attending the conference, who signed up for the hernia course, served as
participants. Minimally Invasive Surgery trained surgeons served as faculty raters. Participant data were collected over a one day period. The Northwestern University Institutional Review Board
SC
approved the study and all participants provided informed consent. Protocol
M AN U
This was an evaluation study to assess the feasibility of creating a simulation-based Continuing Medical Education course for practicing surgeons performing Laparoscopic Ventral Hernia repairs. Course objectives included demonstrating (1) proper port placement strategies; (2) efficient and strategic adhesiolysis; and (3) effective mesh management. The course was limited to thirty participants. Before beginning, participants were randomly
TE D
placed into ten groups of three. Five groups worked simultaneously during the first half of the course. The second cohort completed the task during the second half of the course. Each group was assigned to one of five faculty raters. While one participant performed a Laparoscopic Ventral
EP
Hernia repair on one of three variations in hernia anatomy (incarcerated omentum, incarcerated
AC C
bowel, and diffuse adhesions), another participant navigated the camera and the other participant served as a peer-rater. Participants rotated until they played all three roles and all three variations of the hernia repairs were performed (Figure 1). Individual hernia skins were retained for post-use evaluation.
Each participant was allotted 30 minutes to complete the repair. Throughout the simulated procedure, the faculty rater and participant serving as the peer-rater completed the Global Operative Assessment of Laparoscopic Skills –Incisional Hernia module and Global Operative Assessment of Laparoscopic Skills rating scale assessments to evaluate performance. After the conference, a
3
ACCEPTED MANUSCRIPT 4 blinded Minimally Invasive Surgery-trained faculty member rated the quality and level of completeness of the hernia repairs based on the skin used by each participant. Materials
RI PT
The Laparoscopic Ventral Hernia simulator consists of a plastic box-trainer base covered by a fabric skin that contains a ventral hernia defect, Figure 2. The box-trainer is intended to mimic the abdominal cavity extending from the diaphragm to the upper plane of the pelvic cavity. The
SC
abdominal cavity is layered with various simulated organs to mimic the gross anatomy of the
abdominal cavity, including the peritoneum, mesentery, and bowel. In our first study, participants
M AN U
rated the model as having similar fidelity and utility to an animal model for hernia repair.9 While the base configuration of the Laparoscopic Ventral Hernia simulator was used and showed evidence of validity in another study,9 for this project, we modified the pre-existing simulator to include 2 additional variations in hernia adhesions (Figure 3). The incarcerated omentum configuration had been previously used in a course evaluating chief residents. 8 This hernia
TE D
type was fabricated using loosely woven cotton with simulated blood vessels. The omentum was attached to the hernia sac using fabric glue. The two additional variations included diffuse adhesions and incarcerated bowel. The diffuse adhesion simulator was fabricated using self-adherent
EP
translucent material to simulate the adhesions. This material was placed over the anterior abdominal
AC C
peritoneum and omentum within the hernia sac, thus creating a combination of filmy and dense adhesions. Lastly, the incarcerated bowel hernia type was fabricated using polar fleece fabric for the bowel and mesentery. The bowel was attached to the hernia sac using Krazy Glue™, which resulted in a dense, firm adhesion to the hernia sac. While the hernia adhesion types varied, the size and location of the hernia defect was standard across all users. In addition, once the materials and fabrication procedures were finalized, the models for this study were mass produced by a commercial vendor, Busy Bee Enterprises, (Yorba Linda, CA).
4
ACCEPTED MANUSCRIPT 5 Based on our prior experience with the omentum adhesion model, we became aware that the difficulty of this type of adhesion could be increased by modifying the type of glue or adhesive used to attach the omentum inside the hernia defect. In essence, double sided tape allowed for an easy
RI PT
adhesiolysis, while Krazy Glue™ made for a more a more difficult adhesiolysis. Despite the use of Krazy Glue™ in this model, we hypothesized that the omentum variation would be less difficult than the incarcerated bowel model. Likewise, we hypothesized that the incarcerated bowel model would
SC
be less difficult than the diffuse adhesion model. The evidence that we sought in this project to assess our hypothesis was the amount of time spent during the adhesiolysis and the performance
M AN U
ratings on the Global Operative Assessment of Laparoscopic Skills –Incisional Hernia module and Global Operative Assessment of Laparoscopic Skills rating scale assessments. Each simulator was prepared as if the patient was prepped, draped, and pre-insufflated. Participants were given a standard laparoscopic instrument set that included a 30° rigid laparoscope (Karl Storz, El Segundo, CA), VersaStep trocars (Covidien, Norwalk, CT), open abdomen needle
TE D
drivers and hemostats, Maryland dissectors (Karl Storz), Endo Shears (Covidien), ProTack (Covidien), Endo Graspers (Covidien), Carter-Thomason suture passer (Inlet Medical, Eden Prairie, MN), and Parietex mesh (Covidien). Operative tasks included port placement, dissection of the
AC C
Outcome Measures
EP
adhesions, mesh preparation and fixation for completion of the hernia repair.
Faculty and peer-raters assessed the hernia repairs using two rating scales with prior validity
evidence, The Global Operative Assessment of Laparoscopic Skills-Incisional Hernia is a 7-item global rating scale developed by experts to evaluate the steps of LIHR (placement of trocars, adhesiolysis, estimation of mesh size and shape, mesh orientation and positioning, mesh fixation, knowledge and autonomy in use of instruments, overall competence), each rated on a 5-point Likert scale.9 The Generic Global Operative Assessment of Laparoscopic Skills checklist was also used.10 This checklist evaluates general laparoscopic performance in five domains (depth perception,
5
ACCEPTED MANUSCRIPT 6 bimanual dexterity, efficiency, tissue handling, and autonomy).These items are also rated on a 5point Likert scale: 1=poor, 3=average, 5=excellent, with descriptors to anchor ratings of 1, 3 and 5. We also sought to assess validity support for the Global Operative Assessment of Laparoscopic
RI PT
Skills ratings. To support ssment of Laparoscopic Skillsopic Skills (ioning, mesh fixation, knowledge and autonomy in use of instruments, overall competence), each rated on a 5-point Likert scaler raters. Participants’ Laparoscopic Ventral Hernia simulator skins were analyzed and individually
SC
graded in four major areas; suturing, tacking, ports and mesh placement. For suturing, the score was based on the number of sutures placed and tied, and on the location of suture placement with respect
M AN U
to the edge of the mesh (maximum 6 points possible). For tacking, the score was based on the number of quadrants completed and the location of tack placement with respect to the edge of the mesh (maximum 5 points possible). For port placement, participants were scored on the absolute number of ports (<3 ports=0, ≥3 = 1) and whether or not there was triangulation (yes=1, no=0), (maximum 2 points). For mesh placement, the score was based on whether or not the mesh was
TE D
skewed or flat with respect to the hernia defect (2 points). Finally, participants were graded on whether or not they cut the peritoneum during the lysis of adhesions (1 point). Participants could
Data Analysis
EP
score up to 16 points depending on the quality and completeness of their hernia repair.
AC C
Global Operative Assessment of Laparoscopic Skills –Incisional Hernia module and Global Operative Assessment of Laparoscopic Skills rating scale reliability was assessed for internal consistency using Cronbach’s alpha. Peer and faculty raters’ item scores for each scale were assessed separately with their own reliability estimate. Peer and faculty raters’ scores were compared using a paired samples t-test. Correlations between peer and faculty scores were also assessed. Hernia repair scores were averaged and later stratified based on adhesion type. Stratification results were used to understand the effects of simulator difficulty on hernia repair scores. Data analysis was performed using SPSS version 20 (IBM Corp, Armonk, NY).
6
ACCEPTED MANUSCRIPT 7 RESULTS Internal consistency on the general skills performance- procedure specific performanceGlobal Operative Assessment of Laparoscopic Skills-IH (peer α=.91, faculty α=.88) and Global
RI PT
Operative Assessment of Laparoscopic Skills (peer α=.96, faculty α=.94) were very high. Peers were more lenient than faculty raters on all items in both the procedure specific skills and general
procedure skills ratings; correlations were generally low between the peer and faculty scores with the
SC
exception of a few moderate correlations (Table 1).
Overall, out of a possible 16 points, participants scored poorly on the quality and
M AN U
completeness of their hernia repairs (M=3.90 SD=2.72). Key performance areas where the majority of participants lost points are as follows: (1) 70% of participants failed to triangulate their ports; (2) 39% of participants spent most of the time dissecting adhesions and never progressed to the mesh preparation step in the allotted time. For Global Operative Assessment of Laparoscopic Skills grading and participant evaluation purposes, these persons were allowed to execute the last three
TE D
steps of the procedure at the end of the learning experience. Of those who progressed to the mesh preparation step in the allotted time, 43% failed to conduct proper planning for the mesh landing zone (Figure 4). Participants repairing the diffuse
EP
adhesions had the most difficult time completing the task (56% failed to reach the mesh preparation
AC C
step). When comparing scores between hernia repair types, participants scored 4.21(SD=3.18) on the incarcerated omentum, 3.81 (SD=2.77) on the incarcerated bowel, and 3.66 (SD=2.38) on the diffuse adhesions. While there is a trend towards lower scores with increased adhesion complexity, this was not statistically significant. DISCUSSION Simulation-based Continuing Medical Education courses have great potential to change the traditional lecture based implementation of continuing education for practicing physicians. The purpose of this study was to modify our previous Laparoscopic Ventral Hernia simulator to increase
7
ACCEPTED MANUSCRIPT 8 difficulty and reassess validity and feasibility of a newly developed Continuing Medical Education course for practicing surgeons. In this study, we used our previously developed Laparoscopic Ventral Hernia simulator and two rating scales (Global Operative Assessment of Laparoscopic Skills –
RI PT
Incisional Hernia module and Global Operative Assessment of Laparoscopic Skills), both of which have shown validity evidence in prior work. As part of our study, we reassessed the validity and reliability of the Global Operative Assessment of Laparoscopic Skills –Incisional Hernia module and
SC
Global Operative Assessment of Laparoscopic Skills rating scales for a Continuing Medical
Education course as prior use has largely focused on residents.10,11 Our results show that these rating
M AN U
scales can serve as valid and reliable measures for Continuing Medical Education courses. Faculty and peer raters were able to use the previous surveys for Continuing Medical Education purposes, which provides validity support for use of the Global Operative Assessment of Laparoscopic Skills ratings scales in a different setting and with a variety of simulation scenarios. Additional validity support was evidenced by the more stringent ratings by faculty raters on the rating scale assessments.
TE D
Lastly, despite the differences in context and raters, the Global Operative Assessment of Laparoscopic Skills rating scales maintained reliability levels as previously reported.10,11 These findings may help facilitate the integration of simulation into clinical curricula that include
EP
certification and recertification4 without the need for development of new rating scales. The
AC C
differences in peer and faculty evaluation warrant further investigation whether these reflect undue rater stringency or lack of insight among the participants about what constitutes good quality performance. This has implications for self-assessment and for feedback. Most of our faculty raters were Minimally Invasive Surgery-trained surgeons with extensive
experience in advanced laparoscopic procedures and in training residents and fellows in laparoscopic hernia surgery. This may have created high expectations for performance of practicing surgeons and some bias towards lower ratings. For the peer evaluators, there is the possibility that they were not sufficiently knowledgeable about what constitutes a high-quality repair for these more difficult
8
ACCEPTED MANUSCRIPT 9 simulated scenarios. Laparoscopic adhesiolysis requires good planning and a set of psychomotor skills and visual perception that can add significant difficulty to a simple hernia repair. This makes a procedure with multiple steps and a variety of surgical instruments and materials sufficiently
RI PT
complex. Future work is needed to understand whether future courses should incorporate more gradual increases in the level of difficulty and to better understand whether raters’ expectations were excessively high for the learners involved in such courses.
SC
Our original Laparoscopic Ventral Hernia simulator was shown to have validity evidence in a prior study that used the simulator for 4th and 5th year surgical residents.9 For the current project, we
M AN U
wanted to increase the difficulty of the scenarios with the assumption that the Continuing Medical Education course attendees would have more laparoscopic experience. In our evaluation of simulation difficulty, we noted that the surgeons who had the diffuse type of adhesions spent more time and were at greater risk of not finishing the procedure. While the simulation was more difficult, it is duly noted that the surgeons spent extra time carefully dissecting the adhesions. As
TE D
such, future use of this model should allow for more time to complete the entire procedure or have course objectives that are less dependent on task completion rates. In addition, future choice of the adhesion variations should be based on better assessment of the experience level of course attendees.
EP
Despite the mismatch between simulator difficulty and course participant skill, the majority of course
AC C
participants reported a high level of satisfaction with this course. Moreover, while our data appears to show a significant mismatch between clinical skill and simulator difficulty, these findings also underscore significant learning needs in the surgical community. The implications of our study relate to the potential for simulation to be used for basic and
advanced Continuing Medical Education courses. The first step in course development is highly dependent on learner characteristics. Once this is determined, the learning objectives can be more clearly aligned. Use of off the shelf and locally fabricated simulations can greatly facilitate achievement of a wide variety of learning objectives. While we are in the beginning phases of
9
ACCEPTED MANUSCRIPT 10 exploring all of the possibilities, there is clear support for use of simulation as a teaching and assessment modality for practicing surgeons. Our evaluation of this simulation-based Continuing Medical Education course provided useful feedback regarding use of rating scales, fabrication of
RI PT
operative scenarios and the range of skill levels in a small cohort of practicing surgeons. This study added validity evidence to the current simulation-based assessment by showing that simulator
difficulty can be modified with sufficient realism for different skill levels. In addition, the Global
SC
Operative Assessment of Laparoscopic Skills –Incisional Hernia module and Global Operative
Assessment of Laparoscopic Skills rating scales both with prior validity evidence, were adopted for
M AN U
use in a Continuing Medical Education course. Lastly, the rating scale results confirmed simulator difficulty.
Continuing Medical Education has resurfaced in its level of importance, as Maintenance of Certification has become a new requirement for most clinical specialties. Maintenance of Certification involves four specific requirement areas designed to assess physician competencies.
TE D
These include maintenance of an unrestricted license, hospital privileges and satisfactory references; continuing education and periodic self-assessment; satisfactory performance on a secure, standardized examination and participation in outcome registries and quality improvement
EP
programs.12 If designed correctly, simulation-based scenarios can be implemented as a form of
AC C
continuing education, self-assessment and standardized examination. This study has several limitations. First, it was a small group of surgeons at a single course.
However, the multiple sources of evaluation helped to facilitate an objective course evaluation as well as performance assessment. Simulator design had an effect on completion times. The design of adhesions involved the exploration and investigation of a number of different adhesives. The one chosen as our final product turned out to be quite strong and more difficult to dissect. We did not assess the level of difficulty of the newly developed adhesions prior to the session. As some adhesions were quite difficult, additional time may have allowed more users to complete the hernia
10
ACCEPTED MANUSCRIPT 11 repair. More time might have yielded a more complete assessment of each user’s ability to perform a complete hernia repair. As case difficulty affects performance, specific measures (beyond previous number of cases and years in practice) should be included in course planning and possibly judged
RI PT
based on a structured, detailed pre-assessment of the learner and learner needs. In addition, 39% of participants spent most of the time dissecting adhesions and failed to progress to the mesh
preparation step in the allotted time. For Global Operative Assessment of Laparoscopic Skills
SC
grading and participant evaluation purposes, these persons were allowed to execute the last three steps of the procedure at the end of the learning experience. As a result, the last three procedure
M AN U
specific items include peer and faculty ratings under “second chance” circumstances and fall out of the original testing and evaluation time. Despite this nuance, the trend in faculty and peer rating remained the same.
Overall the course appeared to be a success. Course instructors and participants were engaged in the tasks and there were high level discussions regarding operative approaches and choices. Our
TE D
findings show promise for the use of previously developed Graduate Medical Education assessments in Continuing Medical Education courses. In addition, we were able to easily modify a previously developed simulator to achieve significant variations in scenario difficulty. While the most effective
EP
delivery of advanced Continuing Medical Education courses for surgeons is still unknown, it is clear
AC C
that there is a need to promote continued development and evaluation of a variety of learning and assessment modalities and approaches. As the use of simulation continues to increase, we predict this will change the focus and delivery of Continuing Medical Education courses in the future. Achieving this goal will help to move Continuing Medical Education beyond a time and credit-based requirement to an experience that motivates ongoing physician competence and improves quality in clinical practice.
11
ACCEPTED MANUSCRIPT 12
REFERENCES
RI PT
1) Miller SH, Thompson JN, Mazmanian PE, Aparicio A, Davis DA, et al. Continuing medical education, professional development, and requirements for medical licensure: a white paper of the conjoint committee on continuing medical education. J Contin Educ Health Prof.
SC
2008;28(2): 95–98.
2) Accreditation Council for Graduate Medical Education (ACGRADUATE MEDICAL
Education.org/acGraduate Medical
M AN U
EDUCATION) Program Requirements for Surgery. https://www.acGraduate Medical
Educationweb/tabid/150/ProgramandInstitutionalAccreditation/SurgicalSpecialties/Surgery.a spx. Accessed April 9, 2015
3) Spivey BE. Continuing medical education in the United States: why it needs reform and how
TE D
we propose to accomplish it. J Contin Educ Health Prof. 2005 Summer;25(3):134-43. 4) Scott DJ. Patient safety, competency, and the future of surgical simulation. Simul Healthc. 2006 Fall;1(3):164-70.
EP
5) McGaghie WC, Siddall VJ, Mazmanian PE, Myers J. American College of Chest Physicians
AC C
Health and Science Policy Committee. Lessons for continuing medical education from simulation research in undergraduate and graduate medical education: effectiveness of continuing medical education: American College of Chest Physicians Evidence-Based Educational Guidelines. Chest. 2009 Mar;135(3 Suppl):62S-68S.
6) Stefanidis D, Sierra R, Korndorffer JR Jr, Dunne JB, Markley S, Touchard CL, et al. Intensive continuing medical education course training on simulators results in proficiency for laparoscopic suturing. Am J Surg. 2006 Jan;191(1):23-7.
12
ACCEPTED MANUSCRIPT 13 7) McIvor W, Burden A, Weinger MB, Steadman R. Simulation for maintenance of certification in anesthesiology: the first two years. J Contin Educ Health Prof. 2012 Fall;32(4):236-42. 8) Sachdeva AK, Pellegrini CA, Johnson KA. Support for simulation-based surgical education
RI PT
through American College of Surgeons--accredited education institutes. World J Surg. 2008 Feb;32(2):196-207.
9) Pugh C, Plachta S, Auyang E, Pryor A, Hungness E. Outcome measures for surgical
SC
simulators: is the focus on technical skills the best approach? Surgery. 2010 May;147(5):64654.
M AN U
10) Ghaderi I, Vaillancourt M, Sroka G, Kaneva PA, Vassiliou MC, Choy I, et al. Evaluation of surgical performance during laparoscopic incisional hernia repair: a multicenter study. Surg Endosc. 2011 Aug;25(8):2555-63
11) Vassiliou MC, Feldman LS, Andrew CG, Bergman S, Leffondré K, Stanbridge D, et al. A
Jul;190(1):107-13.
TE D
global assessment tool for evaluation of intraoperative laparoscopic skills. Am J Surg. 2005
12) The American Board of Surgery. Maintenance of Certification (MOC) Overview.
AC C
EP
http://www.absurgery.org/default.jsp?exam-moc. Accessed March 15, 2013.
13
ACCEPTED MANUSCRIPT
AC C
EP
TE D
M AN U
SC
RI PT
14
14
ACCEPTED MANUSCRIPT 15
Figure Legend Figure 1. Station role rotation of participants
RI PT
Figure 2. Participant repairing hernia defect using LVH box-trainer
Figure 4. Skewed mesh with poor suture placement
AC C
EP
TE D
M AN U
Figure 5. Comparison of peer and faculty ratings
SC
Figure 3. Three variations of hernia repairs: incarcerated omentum, incarcerated bowel, and diffuse adhesions
15
Faculty Rater
P2 Performer
Incarcerated Bowel
TE D
P2 Camera Navigator
EP
P3 Camera Navigator
M AN U
P3 Peer Rater
Incarcerated Omentum
P3 Performer
P1 Performer
AC C
P1 Peer Rater
Faculty Rater
SC
Faculty Rater
RI PT
ACCEPTED MANUSCRIPT
P2 Peer Rater
Diffuse Adhesion P1 Camera Navigator
AC C
EP
TE D
M AN U
SC
RI PT
ACCEPTED MANUSCRIPT
AC C
EP
TE D
M AN U
SC
RI PT
ACCEPTED MANUSCRIPT
AC C
EP
TE D
M AN U
SC
RI PT
ACCEPTED MANUSCRIPT
ACCEPTED MANUSCRIPT
*
†
n = 22, 19
n = 23, 23
n = 23, 23
Placement of trocars
Adhesiolysis
Has a very smooth, systemic approach to adhesiolysis
n = 22, 20
n = 22, 23
n = 23, 23
*
n = 23, 23
n = 22, 22
*
n = 22, 23
*
n = 22, 23
Strategically Moved camera Appeared to Estimation of Introduction, Fixation of the Knowledge and Overall added to different port exercise caution size and shape orientation and mesh autonomy in the competence additional ports sites to facilitate during of mesh positioning of use of to facilitate adhesiolysis adhesiolysis mesh instruments adhesiolysis
M AN U
n = 23, 23
*
SC
†
RI PT
Procedure Specific Skills
†
General Procedure Skills †
TE D
5
*
†
3
2
1
0
*
EP
4
*
Peer Mean (n=23)
AC C
5 4 3 2 1 0
*
n = 18, 23
n = 18, 23
Depth perception Bimanual dexterity
Faculty Mean (n=23)
n = 18, 23
n = 18, 23
n = 18, 23
Efficiency
Tissue handling
Autonomy
Peer Mean (n=23) Faculty Mean (n=23)
ACCEPTED MANUSCRIPT
Management Plan Investigator: Carla Pugh Entity: Medical Teaching Systems, Inc
RI PT
1. Publications and Presentations. Your relationship with Medical Teaching Systems, Inc may not restrict publication or presentation, although publication may be delayed for the purpose of pre-publication review for a period consistent with UW-Madison policies.
SC
You, your students, fellows, trainees, and other research workers whom you supervise in the course of your research must disclose the relationship with Medical Teaching Systems, Inc in publications and academic presentations. For researchers in the biomedical sciences, disclosure in publications should conform to recent uniform disclosure guidelines published by a group of editors of major medical journals (Davidoff et al. JAMA 286: 1232-1234, 2001). (See Appendix A: Disclosing in publications.)
M AN U
2. Interaction with Students and Staff. The UW-Madison has a duty to ensure that the entrepreneurial activities of its faculty and staff do not have a negative impact on students or research staff, especially on the academic progress of students. To fulfill this obligation, the COI Committee requires that individuals with potential conflicts of interest inform others who may be impacted by the potential conflicts.
Includes a written summary of the information for each student or staff member Provides documentation that this process has occurred to the COI Committee within 45 days of receipt of this management plan Provides any individuals who subsequently join the group comparable information in a timely manner Updates documentation to the COI Committee at least once a year
AC C
o o
EP
TE D
You should provide information on potential conflicts of interest to all students, fellows, trainees, and other research workers whom you supervise in the course of your research (hereafter students and staff). The information should include explanations of: a) your relationship with Medical Teaching Systems, Inc, and b) the right of students and staff to bring concerns about the effect of your relationship with Medical Teaching Systems, Inc on their work, studies, or progress towards degree to your dean, director, his or her designee, or the COI Committee. The process for providing information should meet the following criteria:
o o
See "Appendix B: Informing students and staff of potential conflicts of interest" for additional guidance. You must also notify all your co-investigators on federal grants of potential conflicts of interest.
ACCEPTED MANUSCRIPT
RI PT
Your relationship with Medical Teaching Systems, Inc may not place restrictions on the ability of your students and staff to receive, analyze, or interpret data. In addition, students may not participate in research sponsored by Medical Teaching Systems, Inc, if the terms and conditions of participation would prevent or inhibit them from meeting applicable UW-Madison degree requirements, such as completion and public defense of a thesis or dissertation. Annually, you must notify the COI Committee and your dean or director in writing of any students and staff involved in the activities of Medical Teaching Systems, Inc. Such involvement may require modification of this management plan.
SC
The COI Committee recommends that all involvement of students and staff with Medical Teaching Systems, Inc be conducted under formal University agreements, such as sponsored research agreements or appointments approved by your dean or director's office.
M AN U
You must direct any of your students and staff with significant financial interests in Medical Teaching Systems, Inc to make an annual report of outside activities using the on-line process (https://www.gradsch.wisc.edu/disclose), if they have not already done so and regardless of whether they would normally be required to make such a report. Any of your students and staff who independently have a reportable significant financial interest in Medical Teaching Systems, Inc will be reviewed by the COI Committee and may be issued their own management plans.
TE D
3. Human Subjects Research. Absent written approval from the COI Committee, you cannot serve as principal investigator (PI), co-investigator (co-PI), or key personnel for a human subjects protocol reviewed by a UW-Madison IRB if Medical Teaching Systems, Inc: a) sponsors the study, or b) owns or licenses a technology tested in the study.
EP
Likewise, you cannot recruit potential subjects for a human subjects protocol, if Medical Teaching Systems, Inc: a) sponsors the study, or b) owns or licenses a technology tested in the study.
AC C
You may refer potential subjects to an independent third party for study information and possible enrollment in human subjects protocols. The COI Committee may grant you an exception to restrictions on participation in human subjects research. The COI Committee only grants exceptions for specific protocols. The exception process depends on the risk level of the proposed research. If a UWMadison IRB determines that a study poses no greater than minimal risks to subjects, investigators will normally be granted an exception to this prohibition without COI Committee review, unless a UW-Madison IRB determines that the investigator must apply to the COI Committee for an exception. For all other protocols, individuals must apply to the COI Committee for exceptions in writing. For instructions on applying for an exception, see the Graduate School's COI Web pages
ACCEPTED MANUSCRIPT
(http://grad.wisc.edu/respolcomp/coioar/).
RI PT
The COI Committee only grants exceptions for protocols that pose greater than minimal risks to subjects in very rare circumstances. For example, the COI Committee might grant an exception if an individual is uniquely qualified by virtue of expertise or experience to participate in the research and the research could not be conducted as safely or effectively without the individual.
SC
If the COI Committee approves an exception, the COI Committee may require modifications to the human subjects protocol or this management plan. Required modifications could include disclosure of financial interests to participants, disclosure of financial interests to research collaborators, additional protocol monitoring, restriction of your roles in the study, or other management, as determined by the committee.
o
The payment reasonably relates to costs incurred, as specified in research agreements between the sponsor and the UW-Madison. The payment reflects the fair market value of services performed. The payment is commensurate with the efforts of the investigator(s) performing the research.
EP
o o
TE D
M AN U
You may not receive payments from University accounts or directly or indirectly from Medical Teaching Systems, Inc for particular research results or for research outcomes related to human subject protocols conducted at or through the UW-Madison. Further, you, or your immediate family, may not receive any personal incentives from University accounts or directly or indirectly from business entities, such as recruitment incentives, performance incentives, fellowships, or other research support, except through an agreement entered into by the University for a sponsored human subjects study. The UWMadison only permits payments for subject enrollment, or for the referral of potential subjects to human subjects studies, when all of the following are present:
AC C
4. Research Support from Medical Teaching Systems, Inc. Before accepting any research support (e.g. grants, contracts, unrestricted gifts, or materials) from Medical Teaching Systems, Inc, you must specifically disclose details of the award, including the scope of the work and any award conditions, to your dean's office for approval. The written disclosure details must accompany an Extramural Support Transmittal Form and be routed from the department chair to the dean or director's office for approval. You must notify the COI Committee of any award approved by the dean's office, so that it may be noted in the committee's records. If support from Medical Teaching Systems, Inc is expected to be $250,000 or more in any two year period, additional clearance may be required. You should contact your dean or director's office regarding this possibility well in advance of the award.
ACCEPTED MANUSCRIPT
5. Reporting of Outside Activities. As a reminder, you are required to submit an annual report of outside activities each spring using the on-line process. In addition, if you have major changes in your outside activities between annual reports, you must update your report. You may access your OAR at any time and make the necessary changes. (http://www.gradsch.wisc.edu/disclose/index.html).
RI PT
6. Outside Activities Agreement. A departmental agreement on the appropriate level of outside activities in light of your faculty appointment needs to be established.
M AN U
SC
7. Use of University Facilities and Services. Any activity involving the use of University facilities or services for the benefit of Medical Teaching Systems, Inc must be conducted in accordance with all relevant University policies pertaining to the use of University facilities. If you wish to use University facilities for the benefit of Medical Teaching Systems, Inc, you must make arrangements through your department chair and your dean. The COI Committee recommends that you have a written facilities use agreement before the activity begins. Links to relevant resources are provided on the Graduate School's Conflict of Interest Web Site (http://www.grad.wisc.edu/research/policyrp/coi/2.facilitiesuse.html).
TE D
8. Purchases. You may not be directly involved in making decisions involving the purchase of items from Medical Teaching Systems, Inc using funding under your control. Any such contractual arrangements are to be delegated to an impartial party, who is not under your supervision or control, such as your department chair or someone designated by your chair. If you are a department chair, your dean should be asked to designate someone on your behalf.
AC C
EP
9. Annual Review. You must meet annually with your department chair, center director, or, if you are the department chair or center director, with your dean or director, or his or her designee, to review information related to your relationship with Medical Teaching Systems, Inc, its influence on your University activities, and compliance with the terms of this management plan. The assessment will then be forwarded to the COI Committee and utilized in its annual review of the relationship between your University research and Medical Teaching Systems, Inc.
Appendix A: Disclosing in publications The COI Committee provides the following examples to guide investigators disclosing their significant financial interests in publications and presentations. Investigators may use alternative approaches that meet the requirements laid out in the body of this management plan. • • •
Dr. A has an ownership interest in Company 1, which has licensed the technology reported in this publication. The research reported was supported by funding provided by Company 1, Company 2, and Company 3, with which Professor B has significant financial interests. Name [A member of Name's family] owns stock in [has stock options with] Company 1.
ACCEPTED MANUSCRIPT
Additional information on disclosing potential conflicts of interest in biomedical research can be found in: •
RI PT
•
Davidoff F, DeAngelis C, Drazen J, et al. Sponsorship, authorship, and accountability. JAMA. 2001;286:1232-1234. DeAngelis CD, Fontanarosa PB, Flanagin A. Reporting financial conflicts of interest and relationships between investigators and research sponsors. JAMA. 2001;286:89-91.
Appendix B: Informing students and staff of potential conflicts of interest
SC
The COI Committee provides the following guidance to help investigators communicate the details of conflict of interest management plans. Investigators may use alternative approaches, if they meet the requirements laid out in the body of this management plan.
M AN U
One effective way to communicate the details of conflict of interest management plans is to hold a meeting to explain the conflict situation, review the provisions of the management plans, and allow participants to ask questions. During this meeting, participants would receive two copies of a written summary signed by the investigator. They would sign and date one copy and return it to the investigator. The investigator would provide the COI Committee a single copy of the written summary and a list of all individuals who received and signed the summary. After an initial meeting, the investigator would speak individually with anyone new who should receive information. The list of individuals who have received information must be updated annually for the COI Committee.
EP
Date Description of the investigator's involvement with the company Description of the purpose of the company Description of the relationship between the investigator's professional work (research) at the UW-Madison and the company 5. Description of any relationships between the UW-Madison and the company, such as sponsored research agreements, facilities use agreements, etc. 6. Description of any restrictions placed on the design, conduct, and reporting of research by the company 7. Description of the ownership of any intellectual property resulting from research connected to the company 8. Impartial contacts for students and staff (the investigator's dean or director's designee AND a representative of the COI Committee. See list below.) 9. Investigator's signature 10. Statement of acknowledgement to be signed and dated by the recipient
AC C
1. 2. 3. 4.
TE D
The written summary would be printed on department letterhead and include the following (See example letter below.):
ACCEPTED MANUSCRIPT
Student and Staff contacts: Conflict of Interest Committee: Kelly Ullrick (890-1613 or [email protected])
RI PT
Email Address [email protected] [email protected] [email protected] [email protected] [email protected] [email protected] [email protected] [email protected] [email protected] [email protected]
SC
Dean or Director's Designee Cheryl Deering Phil MIller Beth Walsh Deanna Dietrich Nicholas Novak Bethany Pluymers Kurt Stephenson Steve Harsy Sharon Vetter Dale Bjorling
AC C
EP
TE D
M AN U
College or School Agricultural and Life Sciences Business Education Engineering Graduate School Law School Letters and Science Medical School Pharmacy Veterinary Medicine
ACCEPTED MANUSCRIPT
Example letter [Letterhead]
To All Personnel in the Laboratory of [PI Name],
RI PT
[Date]
SC
This letter is to inform you that I am [List position in company.] of a company called [Name of Company], devoted to research in the areas of [Add text explaining the purpose of the company, your involvement with it, and any connections to the lab.]. My relationship does not carry with it any restrictions on publication, and any associated intellectual property will be disclosed and processed according to UW-Madison policy.
M AN U
The purpose of this letter is to inform you of this potential conflict of interest, and, if you feel that at any time your academic pursuits and freedoms are compromised by this relationship, that you may contact [Provide contact information for the appropriate dean or director's designee listed above.] or the Conflict of Interest Committee via Kelly Ullrick (890-1613 or [email protected]). Sincerely yours, [Signature]
AC C
Date:
EP
Name: [of recipient]
TE D
I acknowledge receipt of the above information on a potential conflict of interest. I understand that I may address questions about this matter to my dean or director's designee or to the Conflict of Interest Committee.
ACCEPTED MANUSCRIPT
Investigator Concurrence Please indicate your agreement with the appropriate option below and click "Save"
No Response Saved
AC C
EP
TE D
M AN U
SC
Yes
RI PT
I agree to the terms and conditions of this management plan:
S0002-9610(15)00439-0
DOI:
10.1016/j.amjsurg.2015.05.034
Reference:
AJS 11628
To appear in:
The American Journal of Surgery
Received Date: 18 February 2015 Revised Date:
15 May 2015
Please cite this article as: Pugh CM, Arafat FO, Kwan C, Cohen ER, Kurashima Y, Vassiliou MC, Fried GM, DiMarco S, Development and evaluation of a simulation-based CME course: beyond lectures and credit-hours, The American Journal of Surgery (2015), doi: 10.1016/j.amjsurg.2015.05.034. This is a PDF file of an unedited manuscript that has been accepted for publication. As a service to our customers we are providing this early version of the manuscript. The manuscript will undergo copyediting, typesetting, and review of the resulting proof before it is published in its final form. Please note that during the production process errors may be discovered which could affect the content, and all legal disclaimers that apply to the journal pertain.
ACCEPTED MANUSCRIPT
Development and Evaluation of a Simulation-Based CME course: Beyond Lectures and Credit-hours Carla M. Pugh, MD, PhD1, Fahd O. Arafat, MD2, Calvin Kwan, BS1, Elaine R. Cohen, MEd1, Yo Kurashima, MD3, Melina C. Vassiliou, MD, MEd3, Gerald M. Fried, MD3 1
EP
TE D
M AN U
SC
RI PT
University of Wisconsin School of Medicine and Public Health, Department of Surgery 2 Northwestern University, Department of Surgery, Chicago, IL 3 The Department of Surgery, Steinberg-Bernstein Centre for Minimally Invasive Surgery & Innovation, and The Arnold and Blema Steinberg Medical Simulation Centre, McGill University, Montréal, QC, Canada
AC C
Funding: Department of Defense USAMRMC Grant # W81XWH0710190
Corresponding Author: Carla Pugh, MD, PhD Vice-Chair of Education and Patient Safety Department of Surgery H4/785B Clinical Science Center 600 Highland Avenue, Madison WI 53792 Email: [email protected] Phone (608)-265-9574 Fax (608)-252-0936
ACCEPTED MANUSCRIPT
ABSTRACT
RI PT
Background: The aim of our study was to modify our previously developed LVH simulator to increase difficulty and then reassess validity and feasibility for using the simulator in a newly developed simulation-based Continuing Medical Education (CME) course.
M AN U
SC
Methods: Participants (N=30) were practicing surgeons who signed up for a hands-on postgraduate laparoscopic hernia course. A laparoscopic ventral hernia (LVH) simulator, with prior validity evidence, was modified for the course to increase difficulty. Participants completed one of three variations in hernia anatomy: incarcerated omentum, incarcerated bowel, and diffuse adhesions. During the procedure, course faculty and peer observers rated surgeon performance using GOALS-IH and GOALS rating scales with prior validity evidence. Rating scale reliability was reassessed for internal consistency. Peer and faculty raters’ scores were compared. In addition, quality and completeness of the hernia repair were rated.
TE D
Results: Internal consistency on the general skills performance (peer α=.96, faculty α=.94) and procedure specific performance (peer α=.91, faculty α=.88) scores were high. Peers were more lenient than faculty raters on all LVH items in both the procedure specific skills and general skills ratings. Overall, participants scored poorly on the quality and completeness of their hernia repairs (M=3.90/16 SD=2.72), suggesting a mis-match between course attendees and hernia difficulty and identifying a learning need.
EP
Conclusion: Simulation-based CME courses provide hands-on experiences that can positively affect clinical practice. While our data appears to show a significant mis-match between clinical skill and simulator difficulty, these findings also underscore significant learning needs in the surgical community.
AC C
Keywords: Simulation, Validity, Training, CME, Evaluation, Surgery, Laparoscopy
ACCEPTED MANUSCRIPT 1
BACKGROUND A commitment to continuous learning and practice improvement is essential in everyday
RI PT
clinical practice. However, the traditional implementation of continuing medical education allows course attendance and course credits to overshadow measured achievement and internal motivation for excellence. Continuing Medical Education is part of a rapidly changing system of professional development that includes assessment, remediation and reassessment. Although jurisdictional
SC
requirements may vary from state to state, most simply require a number of hours of Continuing Medical Education to be completed annually to satisfy licensure needs.1 Currently, the larger focus
M AN U
for Continuing Medical Education is the hours needed to maintain licensure and certification. However, the motivation behind the original development of Continuing Medical Education programs was to inspire lifelong learning. With the increasing emphasis on quality in healthcare, Continuing Medical Education has great potential to move from a basic focus on maintenance of
TE D
licensure to improving quality in clinical practice and ensuring ongoing physician competence. While simulation has been widely accepted as a training and assessment modality in Graduate Medical Education, use in Continuing Medical Education is not as common. In addition,
EP
the external drivers for objective assessment of clinical skill during residency training are on the rise. Many of the residency review programs are requiring documentation of annual evaluations of skill in
AC C
a hands-on setting, away from direct patient care.2 These evaluations include assessments of knowledge, skills and attitudes. However, following completion of training, these types of assessments are sparse. The exception is the newly implemented Maintenance of Certification program in the United States.3 These programs, as defined by specialty board organizations, are bringing focus to the need for ongoing assessment after residency training. In the medical literature, there is a paucity of research on the use of simulation-based technology in Continuing Medical Education courses.4,5 One study used a proficiency-based
1
ACCEPTED MANUSCRIPT 2 curriculum as part of a Continuing Medical Education course. This group demonstrated the feasibility of a half-day Continuing Medical Education course to improve performance in laparoscopic suturing.6 The American Board of Anesthesia has also made some headway in
RI PT
developing simulation-based assessments for maintenance of certification. Overall, research in this field is largely based on self-report satisfaction data and lacks any standardized performance evaluation.7
SC
The American College of Surgeons supports the use of simulation-based surgical education to enhance patient safety, meet the requirements for Maintenance of Certification and address the
M AN U
core competencies that all surgeons and trainees are required to achieve.8 Despite the American College of Surgeons support, development, implementation and evaluation of simulation-based Continuing Medical Education courses are lacking. In order to achieve these Global Operative Assessment of Laparoscopic Skills, valid and reliable measures of performance are necessary. Our prior work using the Laparoscopic Ventral Hernia simulator revealed the importance of
TE D
intra-operative decision-making for this procedure. This work underscored the need for decisionbased metrics in addition to those used to assess technical skills.9 The aim of our current study was to modify our previous Laparoscopic Ventral Hernia simulator to increase difficulty and then
EP
reassess validity and feasibility for use in a Continuing Medical Education course. Specifically, we
AC C
sought to assess the following: (1) the validity and reliability of a previously developed procedurespecific (Global Operative Assessment of Laparoscopic Skills-Incisional Hernia) rating scale and global (Generic Global Operative Assessment of Laparoscopic Skills ) rating scale largely used for Graduate Medical Education and (2) to document validity support for simulated Laparoscopic Ventral Hernia scenarios of increasing difficulty.
METHODS Setting and Participants
2
ACCEPTED MANUSCRIPT 3 This study was performed at the 97th Clinical Congress in San Francisco, California in 2011. The Clinical Congress is designed to provide individuals with a wide range of learning opportunities, activities, and experiences that will match their educational and professional development needs.
RI PT
Practicing surgeons attending the conference, who signed up for the hernia course, served as
participants. Minimally Invasive Surgery trained surgeons served as faculty raters. Participant data were collected over a one day period. The Northwestern University Institutional Review Board
SC
approved the study and all participants provided informed consent. Protocol
M AN U
This was an evaluation study to assess the feasibility of creating a simulation-based Continuing Medical Education course for practicing surgeons performing Laparoscopic Ventral Hernia repairs. Course objectives included demonstrating (1) proper port placement strategies; (2) efficient and strategic adhesiolysis; and (3) effective mesh management. The course was limited to thirty participants. Before beginning, participants were randomly
TE D
placed into ten groups of three. Five groups worked simultaneously during the first half of the course. The second cohort completed the task during the second half of the course. Each group was assigned to one of five faculty raters. While one participant performed a Laparoscopic Ventral
EP
Hernia repair on one of three variations in hernia anatomy (incarcerated omentum, incarcerated
AC C
bowel, and diffuse adhesions), another participant navigated the camera and the other participant served as a peer-rater. Participants rotated until they played all three roles and all three variations of the hernia repairs were performed (Figure 1). Individual hernia skins were retained for post-use evaluation.
Each participant was allotted 30 minutes to complete the repair. Throughout the simulated procedure, the faculty rater and participant serving as the peer-rater completed the Global Operative Assessment of Laparoscopic Skills –Incisional Hernia module and Global Operative Assessment of Laparoscopic Skills rating scale assessments to evaluate performance. After the conference, a
3
ACCEPTED MANUSCRIPT 4 blinded Minimally Invasive Surgery-trained faculty member rated the quality and level of completeness of the hernia repairs based on the skin used by each participant. Materials
RI PT
The Laparoscopic Ventral Hernia simulator consists of a plastic box-trainer base covered by a fabric skin that contains a ventral hernia defect, Figure 2. The box-trainer is intended to mimic the abdominal cavity extending from the diaphragm to the upper plane of the pelvic cavity. The
SC
abdominal cavity is layered with various simulated organs to mimic the gross anatomy of the
abdominal cavity, including the peritoneum, mesentery, and bowel. In our first study, participants
M AN U
rated the model as having similar fidelity and utility to an animal model for hernia repair.9 While the base configuration of the Laparoscopic Ventral Hernia simulator was used and showed evidence of validity in another study,9 for this project, we modified the pre-existing simulator to include 2 additional variations in hernia adhesions (Figure 3). The incarcerated omentum configuration had been previously used in a course evaluating chief residents. 8 This hernia
TE D
type was fabricated using loosely woven cotton with simulated blood vessels. The omentum was attached to the hernia sac using fabric glue. The two additional variations included diffuse adhesions and incarcerated bowel. The diffuse adhesion simulator was fabricated using self-adherent
EP
translucent material to simulate the adhesions. This material was placed over the anterior abdominal
AC C
peritoneum and omentum within the hernia sac, thus creating a combination of filmy and dense adhesions. Lastly, the incarcerated bowel hernia type was fabricated using polar fleece fabric for the bowel and mesentery. The bowel was attached to the hernia sac using Krazy Glue™, which resulted in a dense, firm adhesion to the hernia sac. While the hernia adhesion types varied, the size and location of the hernia defect was standard across all users. In addition, once the materials and fabrication procedures were finalized, the models for this study were mass produced by a commercial vendor, Busy Bee Enterprises, (Yorba Linda, CA).
4
ACCEPTED MANUSCRIPT 5 Based on our prior experience with the omentum adhesion model, we became aware that the difficulty of this type of adhesion could be increased by modifying the type of glue or adhesive used to attach the omentum inside the hernia defect. In essence, double sided tape allowed for an easy
RI PT
adhesiolysis, while Krazy Glue™ made for a more a more difficult adhesiolysis. Despite the use of Krazy Glue™ in this model, we hypothesized that the omentum variation would be less difficult than the incarcerated bowel model. Likewise, we hypothesized that the incarcerated bowel model would
SC
be less difficult than the diffuse adhesion model. The evidence that we sought in this project to assess our hypothesis was the amount of time spent during the adhesiolysis and the performance
M AN U
ratings on the Global Operative Assessment of Laparoscopic Skills –Incisional Hernia module and Global Operative Assessment of Laparoscopic Skills rating scale assessments. Each simulator was prepared as if the patient was prepped, draped, and pre-insufflated. Participants were given a standard laparoscopic instrument set that included a 30° rigid laparoscope (Karl Storz, El Segundo, CA), VersaStep trocars (Covidien, Norwalk, CT), open abdomen needle
TE D
drivers and hemostats, Maryland dissectors (Karl Storz), Endo Shears (Covidien), ProTack (Covidien), Endo Graspers (Covidien), Carter-Thomason suture passer (Inlet Medical, Eden Prairie, MN), and Parietex mesh (Covidien). Operative tasks included port placement, dissection of the
AC C
Outcome Measures
EP
adhesions, mesh preparation and fixation for completion of the hernia repair.
Faculty and peer-raters assessed the hernia repairs using two rating scales with prior validity
evidence, The Global Operative Assessment of Laparoscopic Skills-Incisional Hernia is a 7-item global rating scale developed by experts to evaluate the steps of LIHR (placement of trocars, adhesiolysis, estimation of mesh size and shape, mesh orientation and positioning, mesh fixation, knowledge and autonomy in use of instruments, overall competence), each rated on a 5-point Likert scale.9 The Generic Global Operative Assessment of Laparoscopic Skills checklist was also used.10 This checklist evaluates general laparoscopic performance in five domains (depth perception,
5
ACCEPTED MANUSCRIPT 6 bimanual dexterity, efficiency, tissue handling, and autonomy).These items are also rated on a 5point Likert scale: 1=poor, 3=average, 5=excellent, with descriptors to anchor ratings of 1, 3 and 5. We also sought to assess validity support for the Global Operative Assessment of Laparoscopic
RI PT
Skills ratings. To support ssment of Laparoscopic Skillsopic Skills (ioning, mesh fixation, knowledge and autonomy in use of instruments, overall competence), each rated on a 5-point Likert scaler raters. Participants’ Laparoscopic Ventral Hernia simulator skins were analyzed and individually
SC
graded in four major areas; suturing, tacking, ports and mesh placement. For suturing, the score was based on the number of sutures placed and tied, and on the location of suture placement with respect
M AN U
to the edge of the mesh (maximum 6 points possible). For tacking, the score was based on the number of quadrants completed and the location of tack placement with respect to the edge of the mesh (maximum 5 points possible). For port placement, participants were scored on the absolute number of ports (<3 ports=0, ≥3 = 1) and whether or not there was triangulation (yes=1, no=0), (maximum 2 points). For mesh placement, the score was based on whether or not the mesh was
TE D
skewed or flat with respect to the hernia defect (2 points). Finally, participants were graded on whether or not they cut the peritoneum during the lysis of adhesions (1 point). Participants could
Data Analysis
EP
score up to 16 points depending on the quality and completeness of their hernia repair.
AC C
Global Operative Assessment of Laparoscopic Skills –Incisional Hernia module and Global Operative Assessment of Laparoscopic Skills rating scale reliability was assessed for internal consistency using Cronbach’s alpha. Peer and faculty raters’ item scores for each scale were assessed separately with their own reliability estimate. Peer and faculty raters’ scores were compared using a paired samples t-test. Correlations between peer and faculty scores were also assessed. Hernia repair scores were averaged and later stratified based on adhesion type. Stratification results were used to understand the effects of simulator difficulty on hernia repair scores. Data analysis was performed using SPSS version 20 (IBM Corp, Armonk, NY).
6
ACCEPTED MANUSCRIPT 7 RESULTS Internal consistency on the general skills performance- procedure specific performanceGlobal Operative Assessment of Laparoscopic Skills-IH (peer α=.91, faculty α=.88) and Global
RI PT
Operative Assessment of Laparoscopic Skills (peer α=.96, faculty α=.94) were very high. Peers were more lenient than faculty raters on all items in both the procedure specific skills and general
procedure skills ratings; correlations were generally low between the peer and faculty scores with the
SC
exception of a few moderate correlations (Table 1).
Overall, out of a possible 16 points, participants scored poorly on the quality and
M AN U
completeness of their hernia repairs (M=3.90 SD=2.72). Key performance areas where the majority of participants lost points are as follows: (1) 70% of participants failed to triangulate their ports; (2) 39% of participants spent most of the time dissecting adhesions and never progressed to the mesh preparation step in the allotted time. For Global Operative Assessment of Laparoscopic Skills grading and participant evaluation purposes, these persons were allowed to execute the last three
TE D
steps of the procedure at the end of the learning experience. Of those who progressed to the mesh preparation step in the allotted time, 43% failed to conduct proper planning for the mesh landing zone (Figure 4). Participants repairing the diffuse
EP
adhesions had the most difficult time completing the task (56% failed to reach the mesh preparation
AC C
step). When comparing scores between hernia repair types, participants scored 4.21(SD=3.18) on the incarcerated omentum, 3.81 (SD=2.77) on the incarcerated bowel, and 3.66 (SD=2.38) on the diffuse adhesions. While there is a trend towards lower scores with increased adhesion complexity, this was not statistically significant. DISCUSSION Simulation-based Continuing Medical Education courses have great potential to change the traditional lecture based implementation of continuing education for practicing physicians. The purpose of this study was to modify our previous Laparoscopic Ventral Hernia simulator to increase
7
ACCEPTED MANUSCRIPT 8 difficulty and reassess validity and feasibility of a newly developed Continuing Medical Education course for practicing surgeons. In this study, we used our previously developed Laparoscopic Ventral Hernia simulator and two rating scales (Global Operative Assessment of Laparoscopic Skills –
RI PT
Incisional Hernia module and Global Operative Assessment of Laparoscopic Skills), both of which have shown validity evidence in prior work. As part of our study, we reassessed the validity and reliability of the Global Operative Assessment of Laparoscopic Skills –Incisional Hernia module and
SC
Global Operative Assessment of Laparoscopic Skills rating scales for a Continuing Medical
Education course as prior use has largely focused on residents.10,11 Our results show that these rating
M AN U
scales can serve as valid and reliable measures for Continuing Medical Education courses. Faculty and peer raters were able to use the previous surveys for Continuing Medical Education purposes, which provides validity support for use of the Global Operative Assessment of Laparoscopic Skills ratings scales in a different setting and with a variety of simulation scenarios. Additional validity support was evidenced by the more stringent ratings by faculty raters on the rating scale assessments.
TE D
Lastly, despite the differences in context and raters, the Global Operative Assessment of Laparoscopic Skills rating scales maintained reliability levels as previously reported.10,11 These findings may help facilitate the integration of simulation into clinical curricula that include
EP
certification and recertification4 without the need for development of new rating scales. The
AC C
differences in peer and faculty evaluation warrant further investigation whether these reflect undue rater stringency or lack of insight among the participants about what constitutes good quality performance. This has implications for self-assessment and for feedback. Most of our faculty raters were Minimally Invasive Surgery-trained surgeons with extensive
experience in advanced laparoscopic procedures and in training residents and fellows in laparoscopic hernia surgery. This may have created high expectations for performance of practicing surgeons and some bias towards lower ratings. For the peer evaluators, there is the possibility that they were not sufficiently knowledgeable about what constitutes a high-quality repair for these more difficult
8
ACCEPTED MANUSCRIPT 9 simulated scenarios. Laparoscopic adhesiolysis requires good planning and a set of psychomotor skills and visual perception that can add significant difficulty to a simple hernia repair. This makes a procedure with multiple steps and a variety of surgical instruments and materials sufficiently
RI PT
complex. Future work is needed to understand whether future courses should incorporate more gradual increases in the level of difficulty and to better understand whether raters’ expectations were excessively high for the learners involved in such courses.
SC
Our original Laparoscopic Ventral Hernia simulator was shown to have validity evidence in a prior study that used the simulator for 4th and 5th year surgical residents.9 For the current project, we
M AN U
wanted to increase the difficulty of the scenarios with the assumption that the Continuing Medical Education course attendees would have more laparoscopic experience. In our evaluation of simulation difficulty, we noted that the surgeons who had the diffuse type of adhesions spent more time and were at greater risk of not finishing the procedure. While the simulation was more difficult, it is duly noted that the surgeons spent extra time carefully dissecting the adhesions. As
TE D
such, future use of this model should allow for more time to complete the entire procedure or have course objectives that are less dependent on task completion rates. In addition, future choice of the adhesion variations should be based on better assessment of the experience level of course attendees.
EP
Despite the mismatch between simulator difficulty and course participant skill, the majority of course
AC C
participants reported a high level of satisfaction with this course. Moreover, while our data appears to show a significant mismatch between clinical skill and simulator difficulty, these findings also underscore significant learning needs in the surgical community. The implications of our study relate to the potential for simulation to be used for basic and
advanced Continuing Medical Education courses. The first step in course development is highly dependent on learner characteristics. Once this is determined, the learning objectives can be more clearly aligned. Use of off the shelf and locally fabricated simulations can greatly facilitate achievement of a wide variety of learning objectives. While we are in the beginning phases of
9
ACCEPTED MANUSCRIPT 10 exploring all of the possibilities, there is clear support for use of simulation as a teaching and assessment modality for practicing surgeons. Our evaluation of this simulation-based Continuing Medical Education course provided useful feedback regarding use of rating scales, fabrication of
RI PT
operative scenarios and the range of skill levels in a small cohort of practicing surgeons. This study added validity evidence to the current simulation-based assessment by showing that simulator
difficulty can be modified with sufficient realism for different skill levels. In addition, the Global
SC
Operative Assessment of Laparoscopic Skills –Incisional Hernia module and Global Operative
Assessment of Laparoscopic Skills rating scales both with prior validity evidence, were adopted for
M AN U
use in a Continuing Medical Education course. Lastly, the rating scale results confirmed simulator difficulty.
Continuing Medical Education has resurfaced in its level of importance, as Maintenance of Certification has become a new requirement for most clinical specialties. Maintenance of Certification involves four specific requirement areas designed to assess physician competencies.
TE D
These include maintenance of an unrestricted license, hospital privileges and satisfactory references; continuing education and periodic self-assessment; satisfactory performance on a secure, standardized examination and participation in outcome registries and quality improvement
EP
programs.12 If designed correctly, simulation-based scenarios can be implemented as a form of
AC C
continuing education, self-assessment and standardized examination. This study has several limitations. First, it was a small group of surgeons at a single course.
However, the multiple sources of evaluation helped to facilitate an objective course evaluation as well as performance assessment. Simulator design had an effect on completion times. The design of adhesions involved the exploration and investigation of a number of different adhesives. The one chosen as our final product turned out to be quite strong and more difficult to dissect. We did not assess the level of difficulty of the newly developed adhesions prior to the session. As some adhesions were quite difficult, additional time may have allowed more users to complete the hernia
10
ACCEPTED MANUSCRIPT 11 repair. More time might have yielded a more complete assessment of each user’s ability to perform a complete hernia repair. As case difficulty affects performance, specific measures (beyond previous number of cases and years in practice) should be included in course planning and possibly judged
RI PT
based on a structured, detailed pre-assessment of the learner and learner needs. In addition, 39% of participants spent most of the time dissecting adhesions and failed to progress to the mesh
preparation step in the allotted time. For Global Operative Assessment of Laparoscopic Skills
SC
grading and participant evaluation purposes, these persons were allowed to execute the last three steps of the procedure at the end of the learning experience. As a result, the last three procedure
M AN U
specific items include peer and faculty ratings under “second chance” circumstances and fall out of the original testing and evaluation time. Despite this nuance, the trend in faculty and peer rating remained the same.
Overall the course appeared to be a success. Course instructors and participants were engaged in the tasks and there were high level discussions regarding operative approaches and choices. Our
TE D
findings show promise for the use of previously developed Graduate Medical Education assessments in Continuing Medical Education courses. In addition, we were able to easily modify a previously developed simulator to achieve significant variations in scenario difficulty. While the most effective
EP
delivery of advanced Continuing Medical Education courses for surgeons is still unknown, it is clear
AC C
that there is a need to promote continued development and evaluation of a variety of learning and assessment modalities and approaches. As the use of simulation continues to increase, we predict this will change the focus and delivery of Continuing Medical Education courses in the future. Achieving this goal will help to move Continuing Medical Education beyond a time and credit-based requirement to an experience that motivates ongoing physician competence and improves quality in clinical practice.
11
ACCEPTED MANUSCRIPT 12
REFERENCES
RI PT
1) Miller SH, Thompson JN, Mazmanian PE, Aparicio A, Davis DA, et al. Continuing medical education, professional development, and requirements for medical licensure: a white paper of the conjoint committee on continuing medical education. J Contin Educ Health Prof.
SC
2008;28(2): 95–98.
2) Accreditation Council for Graduate Medical Education (ACGRADUATE MEDICAL
Education.org/acGraduate Medical
M AN U
EDUCATION) Program Requirements for Surgery. https://www.acGraduate Medical
Educationweb/tabid/150/ProgramandInstitutionalAccreditation/SurgicalSpecialties/Surgery.a spx. Accessed April 9, 2015
3) Spivey BE. Continuing medical education in the United States: why it needs reform and how
TE D
we propose to accomplish it. J Contin Educ Health Prof. 2005 Summer;25(3):134-43. 4) Scott DJ. Patient safety, competency, and the future of surgical simulation. Simul Healthc. 2006 Fall;1(3):164-70.
EP
5) McGaghie WC, Siddall VJ, Mazmanian PE, Myers J. American College of Chest Physicians
AC C
Health and Science Policy Committee. Lessons for continuing medical education from simulation research in undergraduate and graduate medical education: effectiveness of continuing medical education: American College of Chest Physicians Evidence-Based Educational Guidelines. Chest. 2009 Mar;135(3 Suppl):62S-68S.
6) Stefanidis D, Sierra R, Korndorffer JR Jr, Dunne JB, Markley S, Touchard CL, et al. Intensive continuing medical education course training on simulators results in proficiency for laparoscopic suturing. Am J Surg. 2006 Jan;191(1):23-7.
12
ACCEPTED MANUSCRIPT 13 7) McIvor W, Burden A, Weinger MB, Steadman R. Simulation for maintenance of certification in anesthesiology: the first two years. J Contin Educ Health Prof. 2012 Fall;32(4):236-42. 8) Sachdeva AK, Pellegrini CA, Johnson KA. Support for simulation-based surgical education
RI PT
through American College of Surgeons--accredited education institutes. World J Surg. 2008 Feb;32(2):196-207.
9) Pugh C, Plachta S, Auyang E, Pryor A, Hungness E. Outcome measures for surgical
SC
simulators: is the focus on technical skills the best approach? Surgery. 2010 May;147(5):64654.
M AN U
10) Ghaderi I, Vaillancourt M, Sroka G, Kaneva PA, Vassiliou MC, Choy I, et al. Evaluation of surgical performance during laparoscopic incisional hernia repair: a multicenter study. Surg Endosc. 2011 Aug;25(8):2555-63
11) Vassiliou MC, Feldman LS, Andrew CG, Bergman S, Leffondré K, Stanbridge D, et al. A
Jul;190(1):107-13.
TE D
global assessment tool for evaluation of intraoperative laparoscopic skills. Am J Surg. 2005
12) The American Board of Surgery. Maintenance of Certification (MOC) Overview.
AC C
EP
http://www.absurgery.org/default.jsp?exam-moc. Accessed March 15, 2013.
13
ACCEPTED MANUSCRIPT
AC C
EP
TE D
M AN U
SC
RI PT
14
14
ACCEPTED MANUSCRIPT 15
Figure Legend Figure 1. Station role rotation of participants
RI PT
Figure 2. Participant repairing hernia defect using LVH box-trainer
Figure 4. Skewed mesh with poor suture placement
AC C
EP
TE D
M AN U
Figure 5. Comparison of peer and faculty ratings
SC
Figure 3. Three variations of hernia repairs: incarcerated omentum, incarcerated bowel, and diffuse adhesions
15
Faculty Rater
P2 Performer
Incarcerated Bowel
TE D
P2 Camera Navigator
EP
P3 Camera Navigator
M AN U
P3 Peer Rater
Incarcerated Omentum
P3 Performer
P1 Performer
AC C
P1 Peer Rater
Faculty Rater
SC
Faculty Rater
RI PT
ACCEPTED MANUSCRIPT
P2 Peer Rater
Diffuse Adhesion P1 Camera Navigator
AC C
EP
TE D
M AN U
SC
RI PT
ACCEPTED MANUSCRIPT
AC C
EP
TE D
M AN U
SC
RI PT
ACCEPTED MANUSCRIPT
AC C
EP
TE D
M AN U
SC
RI PT
ACCEPTED MANUSCRIPT
ACCEPTED MANUSCRIPT
*
†
n = 22, 19
n = 23, 23
n = 23, 23
Placement of trocars
Adhesiolysis
Has a very smooth, systemic approach to adhesiolysis
n = 22, 20
n = 22, 23
n = 23, 23
*
n = 23, 23
n = 22, 22
*
n = 22, 23
*
n = 22, 23
Strategically Moved camera Appeared to Estimation of Introduction, Fixation of the Knowledge and Overall added to different port exercise caution size and shape orientation and mesh autonomy in the competence additional ports sites to facilitate during of mesh positioning of use of to facilitate adhesiolysis adhesiolysis mesh instruments adhesiolysis
M AN U
n = 23, 23
*
SC
†
RI PT
Procedure Specific Skills
†
General Procedure Skills †
TE D
5
*
†
3
2
1
0
*
EP
4
*
Peer Mean (n=23)
AC C
5 4 3 2 1 0
*
n = 18, 23
n = 18, 23
Depth perception Bimanual dexterity
Faculty Mean (n=23)
n = 18, 23
n = 18, 23
n = 18, 23
Efficiency
Tissue handling
Autonomy
Peer Mean (n=23) Faculty Mean (n=23)
ACCEPTED MANUSCRIPT
Management Plan Investigator: Carla Pugh Entity: Medical Teaching Systems, Inc
RI PT
1. Publications and Presentations. Your relationship with Medical Teaching Systems, Inc may not restrict publication or presentation, although publication may be delayed for the purpose of pre-publication review for a period consistent with UW-Madison policies.
SC
You, your students, fellows, trainees, and other research workers whom you supervise in the course of your research must disclose the relationship with Medical Teaching Systems, Inc in publications and academic presentations. For researchers in the biomedical sciences, disclosure in publications should conform to recent uniform disclosure guidelines published by a group of editors of major medical journals (Davidoff et al. JAMA 286: 1232-1234, 2001). (See Appendix A: Disclosing in publications.)
M AN U
2. Interaction with Students and Staff. The UW-Madison has a duty to ensure that the entrepreneurial activities of its faculty and staff do not have a negative impact on students or research staff, especially on the academic progress of students. To fulfill this obligation, the COI Committee requires that individuals with potential conflicts of interest inform others who may be impacted by the potential conflicts.
Includes a written summary of the information for each student or staff member Provides documentation that this process has occurred to the COI Committee within 45 days of receipt of this management plan Provides any individuals who subsequently join the group comparable information in a timely manner Updates documentation to the COI Committee at least once a year
AC C
o o
EP
TE D
You should provide information on potential conflicts of interest to all students, fellows, trainees, and other research workers whom you supervise in the course of your research (hereafter students and staff). The information should include explanations of: a) your relationship with Medical Teaching Systems, Inc, and b) the right of students and staff to bring concerns about the effect of your relationship with Medical Teaching Systems, Inc on their work, studies, or progress towards degree to your dean, director, his or her designee, or the COI Committee. The process for providing information should meet the following criteria:
o o
See "Appendix B: Informing students and staff of potential conflicts of interest" for additional guidance. You must also notify all your co-investigators on federal grants of potential conflicts of interest.
ACCEPTED MANUSCRIPT
RI PT
Your relationship with Medical Teaching Systems, Inc may not place restrictions on the ability of your students and staff to receive, analyze, or interpret data. In addition, students may not participate in research sponsored by Medical Teaching Systems, Inc, if the terms and conditions of participation would prevent or inhibit them from meeting applicable UW-Madison degree requirements, such as completion and public defense of a thesis or dissertation. Annually, you must notify the COI Committee and your dean or director in writing of any students and staff involved in the activities of Medical Teaching Systems, Inc. Such involvement may require modification of this management plan.
SC
The COI Committee recommends that all involvement of students and staff with Medical Teaching Systems, Inc be conducted under formal University agreements, such as sponsored research agreements or appointments approved by your dean or director's office.
M AN U
You must direct any of your students and staff with significant financial interests in Medical Teaching Systems, Inc to make an annual report of outside activities using the on-line process (https://www.gradsch.wisc.edu/disclose), if they have not already done so and regardless of whether they would normally be required to make such a report. Any of your students and staff who independently have a reportable significant financial interest in Medical Teaching Systems, Inc will be reviewed by the COI Committee and may be issued their own management plans.
TE D
3. Human Subjects Research. Absent written approval from the COI Committee, you cannot serve as principal investigator (PI), co-investigator (co-PI), or key personnel for a human subjects protocol reviewed by a UW-Madison IRB if Medical Teaching Systems, Inc: a) sponsors the study, or b) owns or licenses a technology tested in the study.
EP
Likewise, you cannot recruit potential subjects for a human subjects protocol, if Medical Teaching Systems, Inc: a) sponsors the study, or b) owns or licenses a technology tested in the study.
AC C
You may refer potential subjects to an independent third party for study information and possible enrollment in human subjects protocols. The COI Committee may grant you an exception to restrictions on participation in human subjects research. The COI Committee only grants exceptions for specific protocols. The exception process depends on the risk level of the proposed research. If a UWMadison IRB determines that a study poses no greater than minimal risks to subjects, investigators will normally be granted an exception to this prohibition without COI Committee review, unless a UW-Madison IRB determines that the investigator must apply to the COI Committee for an exception. For all other protocols, individuals must apply to the COI Committee for exceptions in writing. For instructions on applying for an exception, see the Graduate School's COI Web pages
ACCEPTED MANUSCRIPT
(http://grad.wisc.edu/respolcomp/coioar/).
RI PT
The COI Committee only grants exceptions for protocols that pose greater than minimal risks to subjects in very rare circumstances. For example, the COI Committee might grant an exception if an individual is uniquely qualified by virtue of expertise or experience to participate in the research and the research could not be conducted as safely or effectively without the individual.
SC
If the COI Committee approves an exception, the COI Committee may require modifications to the human subjects protocol or this management plan. Required modifications could include disclosure of financial interests to participants, disclosure of financial interests to research collaborators, additional protocol monitoring, restriction of your roles in the study, or other management, as determined by the committee.
o
The payment reasonably relates to costs incurred, as specified in research agreements between the sponsor and the UW-Madison. The payment reflects the fair market value of services performed. The payment is commensurate with the efforts of the investigator(s) performing the research.
EP
o o
TE D
M AN U
You may not receive payments from University accounts or directly or indirectly from Medical Teaching Systems, Inc for particular research results or for research outcomes related to human subject protocols conducted at or through the UW-Madison. Further, you, or your immediate family, may not receive any personal incentives from University accounts or directly or indirectly from business entities, such as recruitment incentives, performance incentives, fellowships, or other research support, except through an agreement entered into by the University for a sponsored human subjects study. The UWMadison only permits payments for subject enrollment, or for the referral of potential subjects to human subjects studies, when all of the following are present:
AC C
4. Research Support from Medical Teaching Systems, Inc. Before accepting any research support (e.g. grants, contracts, unrestricted gifts, or materials) from Medical Teaching Systems, Inc, you must specifically disclose details of the award, including the scope of the work and any award conditions, to your dean's office for approval. The written disclosure details must accompany an Extramural Support Transmittal Form and be routed from the department chair to the dean or director's office for approval. You must notify the COI Committee of any award approved by the dean's office, so that it may be noted in the committee's records. If support from Medical Teaching Systems, Inc is expected to be $250,000 or more in any two year period, additional clearance may be required. You should contact your dean or director's office regarding this possibility well in advance of the award.
ACCEPTED MANUSCRIPT
5. Reporting of Outside Activities. As a reminder, you are required to submit an annual report of outside activities each spring using the on-line process. In addition, if you have major changes in your outside activities between annual reports, you must update your report. You may access your OAR at any time and make the necessary changes. (http://www.gradsch.wisc.edu/disclose/index.html).
RI PT
6. Outside Activities Agreement. A departmental agreement on the appropriate level of outside activities in light of your faculty appointment needs to be established.
M AN U
SC
7. Use of University Facilities and Services. Any activity involving the use of University facilities or services for the benefit of Medical Teaching Systems, Inc must be conducted in accordance with all relevant University policies pertaining to the use of University facilities. If you wish to use University facilities for the benefit of Medical Teaching Systems, Inc, you must make arrangements through your department chair and your dean. The COI Committee recommends that you have a written facilities use agreement before the activity begins. Links to relevant resources are provided on the Graduate School's Conflict of Interest Web Site (http://www.grad.wisc.edu/research/policyrp/coi/2.facilitiesuse.html).
TE D
8. Purchases. You may not be directly involved in making decisions involving the purchase of items from Medical Teaching Systems, Inc using funding under your control. Any such contractual arrangements are to be delegated to an impartial party, who is not under your supervision or control, such as your department chair or someone designated by your chair. If you are a department chair, your dean should be asked to designate someone on your behalf.
AC C
EP
9. Annual Review. You must meet annually with your department chair, center director, or, if you are the department chair or center director, with your dean or director, or his or her designee, to review information related to your relationship with Medical Teaching Systems, Inc, its influence on your University activities, and compliance with the terms of this management plan. The assessment will then be forwarded to the COI Committee and utilized in its annual review of the relationship between your University research and Medical Teaching Systems, Inc.
Appendix A: Disclosing in publications The COI Committee provides the following examples to guide investigators disclosing their significant financial interests in publications and presentations. Investigators may use alternative approaches that meet the requirements laid out in the body of this management plan. • • •
Dr. A has an ownership interest in Company 1, which has licensed the technology reported in this publication. The research reported was supported by funding provided by Company 1, Company 2, and Company 3, with which Professor B has significant financial interests. Name [A member of Name's family] owns stock in [has stock options with] Company 1.
ACCEPTED MANUSCRIPT
Additional information on disclosing potential conflicts of interest in biomedical research can be found in: •
RI PT
•
Davidoff F, DeAngelis C, Drazen J, et al. Sponsorship, authorship, and accountability. JAMA. 2001;286:1232-1234. DeAngelis CD, Fontanarosa PB, Flanagin A. Reporting financial conflicts of interest and relationships between investigators and research sponsors. JAMA. 2001;286:89-91.
Appendix B: Informing students and staff of potential conflicts of interest
SC
The COI Committee provides the following guidance to help investigators communicate the details of conflict of interest management plans. Investigators may use alternative approaches, if they meet the requirements laid out in the body of this management plan.
M AN U
One effective way to communicate the details of conflict of interest management plans is to hold a meeting to explain the conflict situation, review the provisions of the management plans, and allow participants to ask questions. During this meeting, participants would receive two copies of a written summary signed by the investigator. They would sign and date one copy and return it to the investigator. The investigator would provide the COI Committee a single copy of the written summary and a list of all individuals who received and signed the summary. After an initial meeting, the investigator would speak individually with anyone new who should receive information. The list of individuals who have received information must be updated annually for the COI Committee.
EP
Date Description of the investigator's involvement with the company Description of the purpose of the company Description of the relationship between the investigator's professional work (research) at the UW-Madison and the company 5. Description of any relationships between the UW-Madison and the company, such as sponsored research agreements, facilities use agreements, etc. 6. Description of any restrictions placed on the design, conduct, and reporting of research by the company 7. Description of the ownership of any intellectual property resulting from research connected to the company 8. Impartial contacts for students and staff (the investigator's dean or director's designee AND a representative of the COI Committee. See list below.) 9. Investigator's signature 10. Statement of acknowledgement to be signed and dated by the recipient
AC C
1. 2. 3. 4.
TE D
The written summary would be printed on department letterhead and include the following (See example letter below.):
ACCEPTED MANUSCRIPT
Student and Staff contacts: Conflict of Interest Committee: Kelly Ullrick (890-1613 or [email protected])
RI PT
Email Address [email protected] [email protected] [email protected] [email protected] [email protected] [email protected] [email protected] [email protected] [email protected] [email protected]
SC
Dean or Director's Designee Cheryl Deering Phil MIller Beth Walsh Deanna Dietrich Nicholas Novak Bethany Pluymers Kurt Stephenson Steve Harsy Sharon Vetter Dale Bjorling
AC C
EP
TE D
M AN U
College or School Agricultural and Life Sciences Business Education Engineering Graduate School Law School Letters and Science Medical School Pharmacy Veterinary Medicine
ACCEPTED MANUSCRIPT
Example letter [Letterhead]
To All Personnel in the Laboratory of [PI Name],
RI PT
[Date]
SC
This letter is to inform you that I am [List position in company.] of a company called [Name of Company], devoted to research in the areas of [Add text explaining the purpose of the company, your involvement with it, and any connections to the lab.]. My relationship does not carry with it any restrictions on publication, and any associated intellectual property will be disclosed and processed according to UW-Madison policy.
M AN U
The purpose of this letter is to inform you of this potential conflict of interest, and, if you feel that at any time your academic pursuits and freedoms are compromised by this relationship, that you may contact [Provide contact information for the appropriate dean or director's designee listed above.] or the Conflict of Interest Committee via Kelly Ullrick (890-1613 or [email protected]). Sincerely yours, [Signature]
AC C
Date:
EP
Name: [of recipient]
TE D
I acknowledge receipt of the above information on a potential conflict of interest. I understand that I may address questions about this matter to my dean or director's designee or to the Conflict of Interest Committee.
ACCEPTED MANUSCRIPT
Investigator Concurrence Please indicate your agreement with the appropriate option below and click "Save"
No Response Saved
AC C
EP
TE D
M AN U
SC
Yes
RI PT
I agree to the terms and conditions of this management plan: