- Email: [email protected]
Increasing faculty participation in resident education and providing cost-effective self-assessment module credit to faculty through resident-generated didactics
Increasing faculty participation in resident education and providing costeffective self-assessment module credit to faculty through resident generated didactics Hyun Kim, Theresa M. Malatesta, Pramila R. Anne ’, John McAna, Voichita Bar Ad, Adam P. Dicker, Robert B. Den PII: DOI: Reference:
S1879-8500(16)30300-9 doi: 10.1016/j.prro.2016.12.003 PRRO 713
To appear in:
Practical Radiation Oncology
Received date: Revised date: Accepted date:
11 November 2016 7 December 2016 12 December 2016
Please cite this article as: Kim Hyun, Malatesta Theresa M., Anne ’ Pramila R., McAna John, Ad Voichita Bar, Dicker Adam P., Den Robert B., Increasing faculty participation in resident education and providing cost-effective self-assessment module credit to faculty through resident generated didactics, Practical Radiation Oncology (2016), doi: 10.1016/j.prro.2016.12.003
This is a PDF file of an unedited manuscript that has been accepted for publication. As a service to our customers we are providing this early version of the manuscript. The manuscript will undergo copyediting, typesetting, and review of the resulting proof before it is published in its final form. Please note that during the production process errors may be discovered which could affect the content, and all legal disclaimers that apply to the journal pertain.
ACCEPTED MANUSCRIPT Increasing faculty participation in resident education and providing costeffective self-assessment module credit to faculty through resident generated didactics
RI P
T
Hyun Kim, MD1; Theresa M. Malatesta1; Pramila R. Anne’, MD1; John McAna, PhD2; Voichita Bar Ad, MD1; Adam P. Dicker, MD, PhD1 and Robert B. Den, MD1.
PT
ED
MA
NU
SC
1 Department of Radiation Oncology, Sidney Kimmel Medical College at Thomas Jefferson University, Philadelphia, Pennsylvania 2 Jefferson College of Population Health, Thomas Jefferson University, Philadelphia, Pennsylvania
AC
CE
Running title: Resident generated SAM CME lectures
Corresponding author: Robert Den, MD Department of Radiation Oncology Sidney Kimmel Cancer Center at Thomas Jefferson University 111 South 11th Street, Philadelphia, PA 19107 Email: [email protected] Tel: (215) 503-1197 Fax: (215) 955-0412 Conflicts of interest: None
ACCEPTED MANUSCRIPT Abstract Purpose/Objective(s): Board certified radiation oncologists and medical
T
physicists are required to earn self-assessment module (SAM) continuing
RI P
medical education (CME) credit, which may require travel costs or usage fees. Data indicate that faculty participation in resident teaching activities is beneficial
SC
to resident education. Our hypothesis was that providing the opportunity to earn
NU
SAM credit in resident didactics would increase faculty participation in and improve resident education.
MA
Materials/Methods: SAM applications, comprised of CME certified category 1 resident didactic lectures and faculty-generated questions with respective
ED
answers, rationales and references, were submitted to the American Board of
PT
Radiology for formal review. Surveys were distributed to assess main academic campus physician (MCP), affiliate campus physician (ACP), physicist and
CE
radiation oncology resident impressions regarding the quality of the lectures.
AC
Survey responses were designed in Likert-scale format. Sign-test was performed with p<0.05 considered statistically different from neutral. Results: First submission SAM approval was obtained for 9 of 9 lectures to date. A total of 52 SAM credits have been awarded to 4 physicists and 7 attending physicians. MCP and ACP attendance increased from 20% and 0%, respectively, over the 12 months preceding CME/SAM lectures, to 55.6% and 20%, respectively. Survey results indicated that the change to SAM lectures increased the quality of resident lectures (p=.001), attending physician participation in resident education (p<.0001), physicist involvement in medical resident education
ACCEPTED MANUSCRIPT (p=.0006) and faculty motivation to attend resident didactics (p=.004). Residents reported an increased amount of time required to prepare lectures (p=.008).
T
Conclusions: We are the first department, to our knowledge, to offer SAM credit
RI P
to clinical faculty for participation in resident generated didactics. Offering SAM credit at resident lectures is a cost effective alternative to purchasing SAM
SC
resources, increases faculty attendance and may improve the quality of radiation
AC
CE
PT
ED
MA
NU
oncology resident education.
ACCEPTED MANUSCRIPT Introduction Physicians in the United States are required to participate in continuing medical
T
education (CME), a specified portion of which must include self-assessment
RI P
module (SAM) activities1. Self-assessment and lifelong learning are associated with the quality of patient care and comprise one of four core components of the
SC
Maintenance of Certification (MOC) program2. Given the numerous other
NU
competing administrative and patient care responsibilities, meeting the SAM requirements can be challenging and perceived as an additional burden,
MA
especially without clear evidence that MOC certification has a definitive link to improved clinical outcomes3. Further, the efficacy of this process is contingent on
ED
the physician to identify personal educational needs and choose the appropriate
PT
resources to meet them. Yet there are multiple reports indicating a poor
CE
correlation between physician self- and external assessments4–7.
AC
A significant component of resident education is didactic lectures, a portion of which may be prepared and presented by the residents, with feedback from a faculty mentor. Faculty involvement in resident teaching activities is beneficial to resident education and various strategies have been implemented to increase faculty participation8–10. A prior study demonstrated that offering CME credit at emergency medicine resident lectures increased faculty attendance10. Data regarding faculty attendance at radiation oncology resident lectures or MOC compliance in radiation oncology are lacking. Here we report our experience awarding CME and SAM credit to faculty at resident didactic sessions. We
ACCEPTED MANUSCRIPT hypothesized that providing the opportunity to earn CME and SAM credit at no cost during the workday would increase faculty participation and improve the
RI P
T
quality of resident didactics.
Methods
SC
We requested and received permission from our institution’s Office of CME to
NU
offer CME credit for attendance at faculty mentored resident didactic lectures. For each lecture we submitted the title, date, time, location and name of
MA
speakers on the CME office website. Faculty members and presenters submitted a conflict of interest form to our institution Office of CME as well. SAM
ED
applications, comprised of CME certified category 1 resident didactic lectures,
PT
educational objectives and 5 faculty-generated questions with respective answers, rationales and references, were submitted to the American Board of
CE
Radiology (ABR) for formal review 6 weeks prior to the tentative lecture date
AC
(required components of application and detailed instructions are available at https://www.theabr.org/moc-gen-sam-information). Third and fourth year radiation oncology residents were responsible for generating PowerPoint presentations for faculty review prior to SAM application submission. Faculty advisors with expertise in a respective disease site would review the lecture for content and generate the 5 referenced questions, answers and rationale. One faculty member with experience in board examination question writing reviewed the faculty mentor’s self-assessment questions for each lecture. One SAM credit was awarded to faculty who attended the entire lecture (1), signed in, texted the
ACCEPTED MANUSCRIPT conference code to our CME office and scored at least 80% on the SAM questions (at least 4 of 5 questions correct). Examples of topics covered include
T
esophagus cancer, cervical cancer, oral cavity cancer, pancreas cancer,
RI P
ependymoma and neuroblastoma. Attendance from the previous 12 months was collected from the records maintained by our residency program coordinator
NU
SC
based on sign-in sheets.
Surveys were distributed to assess main academic campus physician (MCP),
MA
affiliate campus physician (ACP), physicist and radiation oncology resident impressions regarding the quality of the lectures. Survey responses were
ED
designed in Likert-scale format. Sign-test was performed with p<0.05 considered
CE
Results
PT
statistically different from neutral.
AC
First submission SAM approval was obtained for 9 of 9 lectures to date. SAM approval was usually received 4-5 weeks after submission. Fifty-two SAM credits have been awarded to 4 physicists and 7 attending physicians. MCP attendance increased from a mean of 20% at each lecture during the 12 months preceding SAM lectures to 55.6% at each SAM lecture. ACPs did not participate in resident didactics prior to SAM lectures. The mean ACP attendance at SAM credit didactics was 20%. Nine physicists (75%), 8 residents (100%), 9 MCPs (100%) and 5 ACPs (100%) responded to the survey (Table 1). Eighty-eight percent of MCPs and 60% of ACPs prefer this resident didactic method of receiving SAM
ACCEPTED MANUSCRIPT credit compared to other options of receiving SAM credit. The main reason for ACPs not preferring this method of SAM credit included not being physically
T
present for the lecture (webinar based participation). Two of the ACPs (40%) and
RI P
three physicists (33%) did not attend resident lectures prior to SAM credit being offered and were unable to comment whether or not the quality of lectures
SC
improved (Table 1). Residents generally asserted that the amount of faculty
NU
feedback for prepared didactics and resident learning during preparation for the lecture were similar prior to and after SAM lecture initiation. They also reported
MA
spending more time in preparing their didactic lectures.
ED
Discussion
PT
We are the first department, to our knowledge, to offer SAM credit to clinical faculty for participation in resident generated didactics. This work demonstrates
CE
that offering SAM/CME credit at resident generated didactics increases faculty
AC
attendance and may improve the quality of resident education. Departments in any medical specialty may emulate this model.
Self-assessment continuing medical education (SA-CME) journal articles and SAM lectures are available online for purchase at prices ranging from $25-30 to $55-10511. These fees may be in addition to professional membership fees charged by the organization. Board certified radiation oncologists must have at least 25 SA-CME or SAM credits out of 75 CME credits in each three-year period. This can become a significant financial and time demand on physicians.
ACCEPTED MANUSCRIPT With SAM credit integrated with resident education, physicians can comply with MOC requirements during work hours while also contributing to resident
T
education by participating in lectures. Approaches such as the one presented in
RI P
this manuscript can help physicians avoid paying education fees in addition to
SC
the organizational fees required for professional society membership.
NU
We and other authors have published different approaches to meet the MOC requirements in cooperation with resident education efforts. At our institution we
MA
combined an attending physician practice quality improvement project with a resident quality improvement project so both MOC and ACGME requirements
ED
were fulfilled12. Others have established their department to become a portfolio
PT
sponsor so that they could continually offer MOC part 4 credit to faculty that
CE
supervise resident quality improvement projects13.
AC
Whereas initially the department must submit each lecture to the ABR as an application for SAM approval, after 9 of 10 first submission approvals the institution will apply for SAM deemed status, which allows the organization to develop and deliver SAMs without approval for each activity14. Further, physicians will be exposed to the entire resident curriculum each year, which may identify physician knowledge gaps more frequently than self-assessment4. Although the faculty attendance did increase, it did not reach 100%. The most common reasons for not attending conference included clinic duties or being away at a conference.
ACCEPTED MANUSCRIPT
From this process the authors learned that SAM lectures applications can receive
T
approval if they are carefully prepared, submitted on time and include appropriate
RI P
self-assessment questions. Our CME office indicated that the question quality is often the reason applications are returned or rejected. We attribute our success
SC
in part to the attending physician who reviewed our self-assessment questions
NU
(PRA), she having experience in writing questions for the radiation oncology written certification exam. As we have not had any applications rejected from
MA
initial review we are unable to comment on areas that we improved or changed to
ED
have success.
PT
Future directions include obtaining SAM deemed status. As only third and fourth year residents were arbitrarily determined to have sufficient clinical experience to
AC
CE
generate SAM lectures, not all resident lectures will become SAM lectures.
It is interesting to note that residents did not feel that faculty input into lecture preparation changed with SAM didactics. This may be due to having sufficient feedback from the faculty even prior to SAM lectures or sufficient quality of resident didactics so that significant feedback was not necessary (possibly due to the increased time put into lecture preparation). Given the small sample size of this report it may simply be due to a lack of statistical power as well. Unchanged learning during lecture preparation may be due to similar factors.
ACCEPTED MANUSCRIPT Important to note is that residents reported increased time preparing SAM lectures. In discussion with the residents this was likely due to multiple factors.
T
Residents were anxious to not be the first resident not to receive SAM approval
RI P
from the ABR, thus ensured presentations were well prepared. All material was referenced and figures in the presentation were either referenced (as per ABR
SC
instructions) or generated by the resident. Once SAM deemed status is achieved
NU
the amount of time a resident invests in a lecture may return to baseline. There were not any other reported “negative” implications for instituting SAM didactics;
MA
the overall impression of the residents was favorable, especially citing the increased medical and physics faculty attendance as positive outcomes of SAM
PT
ED
lectures.
Limitations of this study include that it is a single institution experience and that
CE
the survey responses are a surrogate for actual improvement in resident
AC
education. It is difficult to assess if the quality of resident education improved with this intervention. Prior data indicate that resident participation in self-assessment activities increases initial board certification success15. Future studies may evaluate institutional board certification pass rate before and after implementing resident lectures with SAM credit. However, we emphasize that the import of this study is not whether or not the education was improved but that it describes a methodology to combine resident education with faculty maintenance of certification requirements. We propose that the strength of this work is that no other institution has published their success in integrating SAM with resident
ACCEPTED MANUSCRIPT generated lectures. The benefits of this combination are briefly assessed through our single institution survey but require additional studies. Further follow-up is
T
required to see if the SAM component of lectures continues to maintain high
RI P
faculty attendance at lectures. Finally, this initiative primarily benefits radiation oncologists at academic / teaching hospitals where residents rotate and private
NU
SC
practice physicians may not benefit from this model.
Although the ABR does not charge fees to review SAM didactic applications,
MA
there was an institutional CME administrative fee of approximately $1,000 for a 3 year period. This was a flat fee independent of the number of SAM credits
ED
awarded or faculty recipients. Other CME offices and even professional societies
PT
may charge fees, which may deter other institutions from pursuing this process. While the exact difference in costs between the resident SAM didactics and
CE
individuals purchasing SAM materials is difficult to project, it is important to note
AC
that it does save the individual physician from out of pocket or personal education/research fund expenses. Importantly, if too many departments pursue SAM approval this may initially saturate the resources available to approve the SAM applications. However, if these institutions can eventually obtain SAM deemed status then they could independently administer SAM CME credit to their faculty without drawing on the society’s resources.
In an era of competing priorities, integrating SAMs with resident generated didactics may increase faculty participation, encourage MOC compliance,
ACCEPTED MANUSCRIPT eliminate SAM associated costs and improve the quality of resident education. Although additional studies are needed to clarify the impact of resident generated
T
SAM didactics on MOC compliance and resident education, this work establishes
RI P
the feasibility of a department producing resident generated SAM lectures that
AC
CE
PT
ED
MA
NU
SC
can be presented on site during the work day at no cost to its faculty.
ACCEPTED MANUSCRIPT References
AC
CE
PT
ED
MA
NU
SC
RI P
T
1. Davis NL, Willis CE. A new metric for continuing medical education credit. J Contin Educ Health Prof. 2004;24(3):139-144. doi:10.1002/chp.1340240304. 2. Board of Directors of the American Board of Medical Specialties. Standards for the ABMS Program for Maintenance of Certification (MOC). January 2014. 3. Sharp LK, Bashook PG, Lipsky MS, Horowitz SD, Miller SH. Specialty board certification and clinical outcomes: the missing link. Acad Med J Assoc Am Med Coll. 2002;77(6):534-542. 4. Davis DA, Mazmanian PE, Fordis M, Van Harrison R, Thorpe KE, Perrier L. Accuracy of physician self-assessment compared with observed measures of competence: a systematic review. JAMA. 2006;296(9):1094-1102. doi:10.1001/jama.296.9.1094. 5. Eva KW, Regehr G. Self-assessment in the health professions: a reformulation and research agenda. Acad Med J Assoc Am Med Coll. 2005;80(10 Suppl):S46-54. 6. Sibley JC, Sackett DL, Neufeld V, Gerrard B, Rudnick KV, Fraser W. A randomized trial of continuing medical education. N Engl J Med. 1982;306(9):511-515. doi:10.1056/NEJM198203043060904. 7. Gordon MJ. A review of the validity and accuracy of self-assessments in health professions training. Acad Med J Assoc Am Med Coll. 1991;66(12):762-769. 8. Rosenblum ND, Nagler J, Lovejoy FH, Hafler JP. The pedagogic characteristics of a clinical conference for senior residents and faculty. Arch Pediatr Adolesc Med. 1995;149(9):1023-1028. 9. Joyner BD, Nicholson C, Seidel K. Medical Knowledge: The Importance of Faculty Involvement and Curriculum in Graduate Medical Education. J Urol. 2006;175(5):1843-1846. doi:10.1016/S0022-5347(05)00981-X. 10. Lefebvre CW, Hiestand B, Bond MC, et al. Increasing Faculty Attendance at Emergency Medicine Resident Conferences: Does CME Credit Make a Difference? J Grad Med Educ. 2013;5(1):41-45. doi:10.4300/JGME-D-12-00030.1. 11. American Society for Radiation Oncology. ASTRO Educational Resources. https://www.astro.org/Educational-Resources/Index.aspx. 12. Kim H, Malatesta TM, Simone NL, et al. A single activity with a practice quality improvement project for faculty and a quality improvement project for residents. Pract Radiat Oncol. 2016;6(2):114-118. doi:10.1016/j.prro.2015.10.014. 13. Rosenbluth G, Tabas JA, Baron RB. What’s in It for Me? Maintenance of Certification as an Incentive for Faculty Supervision of Resident Quality Improvement Projects. Acad Med J Assoc Am Med Coll. 2016;91(1):56-59. doi:10.1097/ACM.0000000000000797. 14. The American Board of Radiology. Maintenance of Certification - SAM Information. http://www.theabr.org/moc-gen-sam-information. Accessed March 17, 2016. 15. Peterson LE, Blackburn B, King MR. Completing self-assessment modules during residency is associated with better certification exam results. Fam Med. 2014;46(8):597602.
ACCEPTED MANUSCRIPT
Table 1. Survey results (percent)
25
I
N
D
88
13
0
.29 25
0
.29 38 .73
100
0
88
D
60
0
0
0
80
13
0
.73
25
0
67
0
80
.18
.25 20
33
20
22
0
.001
<.0001
0
89
20
11
0
.0006
0
.004
0
.18
.039 0
78
22 .18
0
67
33 .51
0
.33
.039
100
0
0
.008
75
.008 25 .29
0
.29
MCP, main campus physician; ACP, affiliate campus physician; I, increased; N, no change; D, decreased. Sum of percentages may not equal 100 due to rounding and participant decline to response.
AC
0
Aggregate p-value
.69
.38
.38 13
44
.38
80
88
Physicist (n=9) I N D
.38
.039
MA 63
N
.004
75
38
0
NU
63
I
.07
SC
75
PT
CE
0
ACP (n=5)
T
75
ED
the quality/educational value of medical resident didactic sessions? p-value the attending physician attendance/participation in medical resident didactic sessions? p-value the physicist attendance/participation in medical resident didactic sessions? p-value your motivation to attend medical resident didactic sessions? p-value your engagement in main campus activities/resident education? p-value attending physician (advisor) input into medical resident-prepared lectures? p-value affected medical resident hours dedicated to preparing didactic sessions? p-value learning during the preparation of each lecture? p-value
MCP (n=9)
RI P
Resident (n=8) I N D
How has the implementation of CME/SAM certified lectures affected:
S1879-8500(16)30300-9 doi: 10.1016/j.prro.2016.12.003 PRRO 713
To appear in:
Practical Radiation Oncology
Received date: Revised date: Accepted date:
11 November 2016 7 December 2016 12 December 2016
Please cite this article as: Kim Hyun, Malatesta Theresa M., Anne ’ Pramila R., McAna John, Ad Voichita Bar, Dicker Adam P., Den Robert B., Increasing faculty participation in resident education and providing cost-effective self-assessment module credit to faculty through resident generated didactics, Practical Radiation Oncology (2016), doi: 10.1016/j.prro.2016.12.003
This is a PDF file of an unedited manuscript that has been accepted for publication. As a service to our customers we are providing this early version of the manuscript. The manuscript will undergo copyediting, typesetting, and review of the resulting proof before it is published in its final form. Please note that during the production process errors may be discovered which could affect the content, and all legal disclaimers that apply to the journal pertain.
ACCEPTED MANUSCRIPT Increasing faculty participation in resident education and providing costeffective self-assessment module credit to faculty through resident generated didactics
RI P
T
Hyun Kim, MD1; Theresa M. Malatesta1; Pramila R. Anne’, MD1; John McAna, PhD2; Voichita Bar Ad, MD1; Adam P. Dicker, MD, PhD1 and Robert B. Den, MD1.
PT
ED
MA
NU
SC
1 Department of Radiation Oncology, Sidney Kimmel Medical College at Thomas Jefferson University, Philadelphia, Pennsylvania 2 Jefferson College of Population Health, Thomas Jefferson University, Philadelphia, Pennsylvania
AC
CE
Running title: Resident generated SAM CME lectures
Corresponding author: Robert Den, MD Department of Radiation Oncology Sidney Kimmel Cancer Center at Thomas Jefferson University 111 South 11th Street, Philadelphia, PA 19107 Email: [email protected] Tel: (215) 503-1197 Fax: (215) 955-0412 Conflicts of interest: None
ACCEPTED MANUSCRIPT Abstract Purpose/Objective(s): Board certified radiation oncologists and medical
T
physicists are required to earn self-assessment module (SAM) continuing
RI P
medical education (CME) credit, which may require travel costs or usage fees. Data indicate that faculty participation in resident teaching activities is beneficial
SC
to resident education. Our hypothesis was that providing the opportunity to earn
NU
SAM credit in resident didactics would increase faculty participation in and improve resident education.
MA
Materials/Methods: SAM applications, comprised of CME certified category 1 resident didactic lectures and faculty-generated questions with respective
ED
answers, rationales and references, were submitted to the American Board of
PT
Radiology for formal review. Surveys were distributed to assess main academic campus physician (MCP), affiliate campus physician (ACP), physicist and
CE
radiation oncology resident impressions regarding the quality of the lectures.
AC
Survey responses were designed in Likert-scale format. Sign-test was performed with p<0.05 considered statistically different from neutral. Results: First submission SAM approval was obtained for 9 of 9 lectures to date. A total of 52 SAM credits have been awarded to 4 physicists and 7 attending physicians. MCP and ACP attendance increased from 20% and 0%, respectively, over the 12 months preceding CME/SAM lectures, to 55.6% and 20%, respectively. Survey results indicated that the change to SAM lectures increased the quality of resident lectures (p=.001), attending physician participation in resident education (p<.0001), physicist involvement in medical resident education
ACCEPTED MANUSCRIPT (p=.0006) and faculty motivation to attend resident didactics (p=.004). Residents reported an increased amount of time required to prepare lectures (p=.008).
T
Conclusions: We are the first department, to our knowledge, to offer SAM credit
RI P
to clinical faculty for participation in resident generated didactics. Offering SAM credit at resident lectures is a cost effective alternative to purchasing SAM
SC
resources, increases faculty attendance and may improve the quality of radiation
AC
CE
PT
ED
MA
NU
oncology resident education.
ACCEPTED MANUSCRIPT Introduction Physicians in the United States are required to participate in continuing medical
T
education (CME), a specified portion of which must include self-assessment
RI P
module (SAM) activities1. Self-assessment and lifelong learning are associated with the quality of patient care and comprise one of four core components of the
SC
Maintenance of Certification (MOC) program2. Given the numerous other
NU
competing administrative and patient care responsibilities, meeting the SAM requirements can be challenging and perceived as an additional burden,
MA
especially without clear evidence that MOC certification has a definitive link to improved clinical outcomes3. Further, the efficacy of this process is contingent on
ED
the physician to identify personal educational needs and choose the appropriate
PT
resources to meet them. Yet there are multiple reports indicating a poor
CE
correlation between physician self- and external assessments4–7.
AC
A significant component of resident education is didactic lectures, a portion of which may be prepared and presented by the residents, with feedback from a faculty mentor. Faculty involvement in resident teaching activities is beneficial to resident education and various strategies have been implemented to increase faculty participation8–10. A prior study demonstrated that offering CME credit at emergency medicine resident lectures increased faculty attendance10. Data regarding faculty attendance at radiation oncology resident lectures or MOC compliance in radiation oncology are lacking. Here we report our experience awarding CME and SAM credit to faculty at resident didactic sessions. We
ACCEPTED MANUSCRIPT hypothesized that providing the opportunity to earn CME and SAM credit at no cost during the workday would increase faculty participation and improve the
RI P
T
quality of resident didactics.
Methods
SC
We requested and received permission from our institution’s Office of CME to
NU
offer CME credit for attendance at faculty mentored resident didactic lectures. For each lecture we submitted the title, date, time, location and name of
MA
speakers on the CME office website. Faculty members and presenters submitted a conflict of interest form to our institution Office of CME as well. SAM
ED
applications, comprised of CME certified category 1 resident didactic lectures,
PT
educational objectives and 5 faculty-generated questions with respective answers, rationales and references, were submitted to the American Board of
CE
Radiology (ABR) for formal review 6 weeks prior to the tentative lecture date
AC
(required components of application and detailed instructions are available at https://www.theabr.org/moc-gen-sam-information). Third and fourth year radiation oncology residents were responsible for generating PowerPoint presentations for faculty review prior to SAM application submission. Faculty advisors with expertise in a respective disease site would review the lecture for content and generate the 5 referenced questions, answers and rationale. One faculty member with experience in board examination question writing reviewed the faculty mentor’s self-assessment questions for each lecture. One SAM credit was awarded to faculty who attended the entire lecture (1), signed in, texted the
ACCEPTED MANUSCRIPT conference code to our CME office and scored at least 80% on the SAM questions (at least 4 of 5 questions correct). Examples of topics covered include
T
esophagus cancer, cervical cancer, oral cavity cancer, pancreas cancer,
RI P
ependymoma and neuroblastoma. Attendance from the previous 12 months was collected from the records maintained by our residency program coordinator
NU
SC
based on sign-in sheets.
Surveys were distributed to assess main academic campus physician (MCP),
MA
affiliate campus physician (ACP), physicist and radiation oncology resident impressions regarding the quality of the lectures. Survey responses were
ED
designed in Likert-scale format. Sign-test was performed with p<0.05 considered
CE
Results
PT
statistically different from neutral.
AC
First submission SAM approval was obtained for 9 of 9 lectures to date. SAM approval was usually received 4-5 weeks after submission. Fifty-two SAM credits have been awarded to 4 physicists and 7 attending physicians. MCP attendance increased from a mean of 20% at each lecture during the 12 months preceding SAM lectures to 55.6% at each SAM lecture. ACPs did not participate in resident didactics prior to SAM lectures. The mean ACP attendance at SAM credit didactics was 20%. Nine physicists (75%), 8 residents (100%), 9 MCPs (100%) and 5 ACPs (100%) responded to the survey (Table 1). Eighty-eight percent of MCPs and 60% of ACPs prefer this resident didactic method of receiving SAM
ACCEPTED MANUSCRIPT credit compared to other options of receiving SAM credit. The main reason for ACPs not preferring this method of SAM credit included not being physically
T
present for the lecture (webinar based participation). Two of the ACPs (40%) and
RI P
three physicists (33%) did not attend resident lectures prior to SAM credit being offered and were unable to comment whether or not the quality of lectures
SC
improved (Table 1). Residents generally asserted that the amount of faculty
NU
feedback for prepared didactics and resident learning during preparation for the lecture were similar prior to and after SAM lecture initiation. They also reported
MA
spending more time in preparing their didactic lectures.
ED
Discussion
PT
We are the first department, to our knowledge, to offer SAM credit to clinical faculty for participation in resident generated didactics. This work demonstrates
CE
that offering SAM/CME credit at resident generated didactics increases faculty
AC
attendance and may improve the quality of resident education. Departments in any medical specialty may emulate this model.
Self-assessment continuing medical education (SA-CME) journal articles and SAM lectures are available online for purchase at prices ranging from $25-30 to $55-10511. These fees may be in addition to professional membership fees charged by the organization. Board certified radiation oncologists must have at least 25 SA-CME or SAM credits out of 75 CME credits in each three-year period. This can become a significant financial and time demand on physicians.
ACCEPTED MANUSCRIPT With SAM credit integrated with resident education, physicians can comply with MOC requirements during work hours while also contributing to resident
T
education by participating in lectures. Approaches such as the one presented in
RI P
this manuscript can help physicians avoid paying education fees in addition to
SC
the organizational fees required for professional society membership.
NU
We and other authors have published different approaches to meet the MOC requirements in cooperation with resident education efforts. At our institution we
MA
combined an attending physician practice quality improvement project with a resident quality improvement project so both MOC and ACGME requirements
ED
were fulfilled12. Others have established their department to become a portfolio
PT
sponsor so that they could continually offer MOC part 4 credit to faculty that
CE
supervise resident quality improvement projects13.
AC
Whereas initially the department must submit each lecture to the ABR as an application for SAM approval, after 9 of 10 first submission approvals the institution will apply for SAM deemed status, which allows the organization to develop and deliver SAMs without approval for each activity14. Further, physicians will be exposed to the entire resident curriculum each year, which may identify physician knowledge gaps more frequently than self-assessment4. Although the faculty attendance did increase, it did not reach 100%. The most common reasons for not attending conference included clinic duties or being away at a conference.
ACCEPTED MANUSCRIPT
From this process the authors learned that SAM lectures applications can receive
T
approval if they are carefully prepared, submitted on time and include appropriate
RI P
self-assessment questions. Our CME office indicated that the question quality is often the reason applications are returned or rejected. We attribute our success
SC
in part to the attending physician who reviewed our self-assessment questions
NU
(PRA), she having experience in writing questions for the radiation oncology written certification exam. As we have not had any applications rejected from
MA
initial review we are unable to comment on areas that we improved or changed to
ED
have success.
PT
Future directions include obtaining SAM deemed status. As only third and fourth year residents were arbitrarily determined to have sufficient clinical experience to
AC
CE
generate SAM lectures, not all resident lectures will become SAM lectures.
It is interesting to note that residents did not feel that faculty input into lecture preparation changed with SAM didactics. This may be due to having sufficient feedback from the faculty even prior to SAM lectures or sufficient quality of resident didactics so that significant feedback was not necessary (possibly due to the increased time put into lecture preparation). Given the small sample size of this report it may simply be due to a lack of statistical power as well. Unchanged learning during lecture preparation may be due to similar factors.
ACCEPTED MANUSCRIPT Important to note is that residents reported increased time preparing SAM lectures. In discussion with the residents this was likely due to multiple factors.
T
Residents were anxious to not be the first resident not to receive SAM approval
RI P
from the ABR, thus ensured presentations were well prepared. All material was referenced and figures in the presentation were either referenced (as per ABR
SC
instructions) or generated by the resident. Once SAM deemed status is achieved
NU
the amount of time a resident invests in a lecture may return to baseline. There were not any other reported “negative” implications for instituting SAM didactics;
MA
the overall impression of the residents was favorable, especially citing the increased medical and physics faculty attendance as positive outcomes of SAM
PT
ED
lectures.
Limitations of this study include that it is a single institution experience and that
CE
the survey responses are a surrogate for actual improvement in resident
AC
education. It is difficult to assess if the quality of resident education improved with this intervention. Prior data indicate that resident participation in self-assessment activities increases initial board certification success15. Future studies may evaluate institutional board certification pass rate before and after implementing resident lectures with SAM credit. However, we emphasize that the import of this study is not whether or not the education was improved but that it describes a methodology to combine resident education with faculty maintenance of certification requirements. We propose that the strength of this work is that no other institution has published their success in integrating SAM with resident
ACCEPTED MANUSCRIPT generated lectures. The benefits of this combination are briefly assessed through our single institution survey but require additional studies. Further follow-up is
T
required to see if the SAM component of lectures continues to maintain high
RI P
faculty attendance at lectures. Finally, this initiative primarily benefits radiation oncologists at academic / teaching hospitals where residents rotate and private
NU
SC
practice physicians may not benefit from this model.
Although the ABR does not charge fees to review SAM didactic applications,
MA
there was an institutional CME administrative fee of approximately $1,000 for a 3 year period. This was a flat fee independent of the number of SAM credits
ED
awarded or faculty recipients. Other CME offices and even professional societies
PT
may charge fees, which may deter other institutions from pursuing this process. While the exact difference in costs between the resident SAM didactics and
CE
individuals purchasing SAM materials is difficult to project, it is important to note
AC
that it does save the individual physician from out of pocket or personal education/research fund expenses. Importantly, if too many departments pursue SAM approval this may initially saturate the resources available to approve the SAM applications. However, if these institutions can eventually obtain SAM deemed status then they could independently administer SAM CME credit to their faculty without drawing on the society’s resources.
In an era of competing priorities, integrating SAMs with resident generated didactics may increase faculty participation, encourage MOC compliance,
ACCEPTED MANUSCRIPT eliminate SAM associated costs and improve the quality of resident education. Although additional studies are needed to clarify the impact of resident generated
T
SAM didactics on MOC compliance and resident education, this work establishes
RI P
the feasibility of a department producing resident generated SAM lectures that
AC
CE
PT
ED
MA
NU
SC
can be presented on site during the work day at no cost to its faculty.
ACCEPTED MANUSCRIPT References
AC
CE
PT
ED
MA
NU
SC
RI P
T
1. Davis NL, Willis CE. A new metric for continuing medical education credit. J Contin Educ Health Prof. 2004;24(3):139-144. doi:10.1002/chp.1340240304. 2. Board of Directors of the American Board of Medical Specialties. Standards for the ABMS Program for Maintenance of Certification (MOC). January 2014. 3. Sharp LK, Bashook PG, Lipsky MS, Horowitz SD, Miller SH. Specialty board certification and clinical outcomes: the missing link. Acad Med J Assoc Am Med Coll. 2002;77(6):534-542. 4. Davis DA, Mazmanian PE, Fordis M, Van Harrison R, Thorpe KE, Perrier L. Accuracy of physician self-assessment compared with observed measures of competence: a systematic review. JAMA. 2006;296(9):1094-1102. doi:10.1001/jama.296.9.1094. 5. Eva KW, Regehr G. Self-assessment in the health professions: a reformulation and research agenda. Acad Med J Assoc Am Med Coll. 2005;80(10 Suppl):S46-54. 6. Sibley JC, Sackett DL, Neufeld V, Gerrard B, Rudnick KV, Fraser W. A randomized trial of continuing medical education. N Engl J Med. 1982;306(9):511-515. doi:10.1056/NEJM198203043060904. 7. Gordon MJ. A review of the validity and accuracy of self-assessments in health professions training. Acad Med J Assoc Am Med Coll. 1991;66(12):762-769. 8. Rosenblum ND, Nagler J, Lovejoy FH, Hafler JP. The pedagogic characteristics of a clinical conference for senior residents and faculty. Arch Pediatr Adolesc Med. 1995;149(9):1023-1028. 9. Joyner BD, Nicholson C, Seidel K. Medical Knowledge: The Importance of Faculty Involvement and Curriculum in Graduate Medical Education. J Urol. 2006;175(5):1843-1846. doi:10.1016/S0022-5347(05)00981-X. 10. Lefebvre CW, Hiestand B, Bond MC, et al. Increasing Faculty Attendance at Emergency Medicine Resident Conferences: Does CME Credit Make a Difference? J Grad Med Educ. 2013;5(1):41-45. doi:10.4300/JGME-D-12-00030.1. 11. American Society for Radiation Oncology. ASTRO Educational Resources. https://www.astro.org/Educational-Resources/Index.aspx. 12. Kim H, Malatesta TM, Simone NL, et al. A single activity with a practice quality improvement project for faculty and a quality improvement project for residents. Pract Radiat Oncol. 2016;6(2):114-118. doi:10.1016/j.prro.2015.10.014. 13. Rosenbluth G, Tabas JA, Baron RB. What’s in It for Me? Maintenance of Certification as an Incentive for Faculty Supervision of Resident Quality Improvement Projects. Acad Med J Assoc Am Med Coll. 2016;91(1):56-59. doi:10.1097/ACM.0000000000000797. 14. The American Board of Radiology. Maintenance of Certification - SAM Information. http://www.theabr.org/moc-gen-sam-information. Accessed March 17, 2016. 15. Peterson LE, Blackburn B, King MR. Completing self-assessment modules during residency is associated with better certification exam results. Fam Med. 2014;46(8):597602.
ACCEPTED MANUSCRIPT
Table 1. Survey results (percent)
25
I
N
D
88
13
0
.29 25
0
.29 38 .73
100
0
88
D
60
0
0
0
80
13
0
.73
25
0
67
0
80
.18
.25 20
33
20
22
0
.001
<.0001
0
89
20
11
0
.0006
0
.004
0
.18
.039 0
78
22 .18
0
67
33 .51
0
.33
.039
100
0
0
.008
75
.008 25 .29
0
.29
MCP, main campus physician; ACP, affiliate campus physician; I, increased; N, no change; D, decreased. Sum of percentages may not equal 100 due to rounding and participant decline to response.
AC
0
Aggregate p-value
.69
.38
.38 13
44
.38
80
88
Physicist (n=9) I N D
.38
.039
MA 63
N
.004
75
38
0
NU
63
I
.07
SC
75
PT
CE
0
ACP (n=5)
T
75
ED
the quality/educational value of medical resident didactic sessions? p-value the attending physician attendance/participation in medical resident didactic sessions? p-value the physicist attendance/participation in medical resident didactic sessions? p-value your motivation to attend medical resident didactic sessions? p-value your engagement in main campus activities/resident education? p-value attending physician (advisor) input into medical resident-prepared lectures? p-value affected medical resident hours dedicated to preparing didactic sessions? p-value learning during the preparation of each lecture? p-value
MCP (n=9)
RI P
Resident (n=8) I N D
How has the implementation of CME/SAM certified lectures affected: