- Email: [email protected]
Responsible Milestone-Based Educational Handover With Individualized Learning Plan From Undergraduate to Graduate Pediatric Medical Education
Accepted Manuscript Responsible Milestone-Based Educational Handover with Individualized Learning Plan from Undergraduate to Graduate Pediatric Medical Education Jocelyn Huang Schiller, MD, Heather L. Burrows, MD,, Amy E. Fleming, Meg G. Keeley, M.D, Lauren Wozniak, Sally A. Santen, MD, PII:
S1876-2859(17)30490-4
DOI:
10.1016/j.acap.2017.09.010
Reference:
ACAP 1096
To appear in:
Academic Pediatrics
Received Date: 25 April 2017 Revised Date:
21 August 2017
Accepted Date: 2 September 2017
Please cite this article as: Schiller JH, Burrows HL, Fleming, AE, Keeley MG, Wozniak L, Santen SA, Responsible Milestone-Based Educational Handover with Individualized Learning Plan from Undergraduate to Graduate Pediatric Medical Education, Academic Pediatrics (2017), doi: 10.1016/ j.acap.2017.09.010. This is a PDF file of an unedited manuscript that has been accepted for publication. As a service to our customers we are providing this early version of the manuscript. The manuscript will undergo copyediting, typesetting, and review of the resulting proof before it is published in its final form. Please note that during the production process errors may be discovered which could affect the content, and all legal disclaimers that apply to the journal pertain.
ACCEPTED MANUSCRIPT Title: Responsible Milestone-Based Educational Handover with Individualized Learning Plan from Undergraduate to Graduate Pediatric Medical Education Jocelyn Schiller, MD, Heather Burrows, MD, PhD, Amy Fleming, MD, MHPE, Meg Keeley, MD, Lauren Wozniak, Sally A Santen, MD, PhD
SC
TE D
Amy E. Fleming, MD Department of Pediatrics Vanderbilt University School of Medicine 201 Light Hall Nashville, TN 37232 [email protected]
Lauren Wozniak, MPH University of Michigan Medical School 2635 Gladstone Ave. Ann Arbor, MI 48104 [email protected]
M AN U
Authors: Heather L Burrows, MD, PhD Department of Pediatrics University of Michigan Medical School 3237 Medical Professional Building Ann Arbor, MI 48109 [email protected]
RI PT
Corresponding Author: Jocelyn Huang Schiller, MD Department of Pediatrics University of Michigan Medical School 1540 E. Hospital Drive Mott 12-525A/SPC 4280 Ann Arbor, MI 48109 Phone: 734-763-5359 FAX: 734-647-5624 [email protected]
Sally A Santen, MD, PHD Department of Emergency Medicine University of Michigan Medical School 6121 Taubman Health Sciences Library 1135 Catherine Street Ann Arbor, MI 48109-5726 [email protected]
AC C
EP
Meg G. Keeley M.D. Department of Pediatrics University of Virginia Health System PO Box 800739 Charlottesville, VA 22908 [email protected]
Key Words: Medical student, pediatrics, graduate medical education, handover, milestones Running title: Responsible Milestone Educational Handover for Pediatric Students Text word count:
Funding: The authors would like to thank the University of Michigan Graduate Medical Education Innovations Program Grant for funding to support the dissemination of our work at the Council on Medical Student Education in Pediatrics Annual Meeting 2017 and Pediatric Academic Societies Annual Meeting 2017. University of Michigan Medical School has an Accelerating Change in Medical Education Grant from the American Medicine Association. The authors have no potential conflicts of interest or corporate sponsors. 1
ACCEPTED MANUSCRIPT What’s New?
Communication of medical students’ pediatric milestone assessments and individual learning plans from medical schools to pediatric residency directors allows for effective educational
RI PT
handovers promoting the continuum of education. Existing undergraduate medical education assessments can provide meaningful data to determine most pediatric milestone levels.
SC
Background
Milestones are competency-based, developmental outcomes marking progress from
M AN U
medical school through residency into practice. The Accreditation Council for Graduate Medical Education (ACGME) defined milestones for all specialties including pediatrics.1 As medical education moves towards competency-based assessment, there are calls for a continuum, including communication between undergraduate (UME) and graduate medical education
TE D
(GME).2
Currently, the UME to GME educational handover is the residency application, including the Medical Student Performance Evaluation (MSPE), transcripts and reference letters. The
EP
MSPE is created almost a year before internship, without an update. Also, there are concerns regarding the trustworthiness of the MSPE.2 Although transparency may improve with the 2016
AC C
MSPE recommendations, most schools do not include competencies, and if included, do not provide the detail needed to determine specialty specific milestones.3 These communication failures hinder learners’ education, wastes educational resources and may cause harm to patients.2 To address these issues, one institution’s departments of emergency medicine, surgery, and obstetrics/gynecology developed specialty specific, milestone-based competency
2
ACCEPTED MANUSCRIPT handovers between UME and GME after the National Resident Matching Program Match.4-6 Such efforts in pediatrics have yet to be described. We sought to develop a Pediatric Milestone Handover with individual learning plan (MH-ILP) to communicate with residency program directors (PDs) after the Match. Using a
RI PT
competency-based framework,7 we shared students’ attainment of pediatric ACGME
milestones. Pediatric milestones to allow schools and residencies to share a framework for assessment and communication. Using self-determination theory, ILPs were added to
SC
encourage student self-improvement through goal setting.1,8,9 In this study, we determined the
Educational Approach and Innovation
M AN U
feasibility and utility of the MH-ILP from the perspective of pediatric PDs.
In 2016, graduating students entering pediatric residencies were identified at three schools. Students voluntarily participated; written consent was obtained. This study was
TE D
approved by each institution’s Institutional Review Boards. Pediatric competency assessment committees comprised of PDs, clerkship directors,
EP
and internship readiness course faculty were established at each school. In spring 2016, committees reviewed the performance of their school’s participating students. After
AC C
investigators discussed data sources, each school’s committee used their own understanding of milestones to plot students. Data included performance assessments from pediatric courses (clerkships, acting internships, fourth year electives, and internship readiness courses) and observed structured clinical examinations. By consensus, committees mapped students’ assessments to the milestones and determined milestone level (Online Appendix).1 Milestones without data were marked as “not assessed.” Although 49 pediatric milestones exist, we used
3
ACCEPTED MANUSCRIPT the 21 milestones required for ACGME reporting due to limited committee experience with the other milestones. Students were trained on the development of Specific, Measurable, Attainable, Realistic, Timely goals. Students performed self-assessments using the milestones and set learning goals.
RI PT
Most met with a committee member to compare their self-assessment with committee
assessment and further refine their ILP. Two investigators reviewed the ILPs and discussed the data, codes, and themes until they reached consensus.
SC
MH-ILP included committee milestone assessments and student–identified learning
M AN U
goals. The post-Match MH-ILP was sent to their residency PD with an explanation and a survey. PDs were asked about the ability of the MH-ILP and MSPE to describe students’ strengths and weaknesses on a Likert scale (1= not at all, 5= extremely well) (Table 1). A similar survey was
Results
TE D
sent 6 months later.
MH-ILPs were created for 27 students who matched into 19 different programs.
EP
Committees were able to consistently assess 16 of the 21 pediatric milestones. Students were assessed to be performing at level one or greater. None of the schools consistently assessed the
AC C
milestone “Practice-Based Learning and Improvement 3” regarding quality improvement. Students developed ILP goals aimed at the first internship months. Although students’
141 goals covered all ACGME competency domains, the three most common sub-competencies chosen were perform appropriate learning activities (14%), identify strengths and weaknesses (13%) and organize/prioritize care (13%). Sixteen PDs (84%) responded to our initial survey; 10 (53%) responded to the follow-up (Table). In the follow-up survey, PDs felt the MH-ILP described students’ strengths and 4
ACCEPTED MANUSCRIPT weaknesses moderately well (strengths mean 3.9, weaknesses mean 3.4). Milestones correlated moderately with the trainee’s mid-internship milestone assessments (mean 3.9). One respondent commented the MH-ILP “correlated with their milestones here” and another PD noted, “As a confirmation, seems to be continuing same positive trend,” lending support for
RI PT
our milestone level assessments. Most (81%) felt the new MH-ILP provided more useful information over the traditional MSPE.
SC
Discussion and Next Steps
M AN U
This innovation demonstrates graduating students entering pediatrics can be assessed on the majority of the pediatric required milestones and handover from UME to GME is moderately useful to pediatric PDs. Importantly, the MH-ILP provided better description of the trainee compared to the MSPE.
The milestone framework1,7 was helpful in interpreting our data and confirming our
TE D
observations that these graduates were competent to begin residency and student performance varied. Although we did not record the time spent collecting and reviewing
EP
student data, there was significant effort involved, which is likely to be streamlined with experience. We do believe, however, that communication between UME and GME will improve
AC C
the trainees’ educational continuum. Although PDs had mixed responses to the MH-ILP, as medical education continues to dialogue about competencies, our shared understanding will increase. If more schools communicated on this level, program directors would come to expect the information. They could use the information to tailor the intern year, perhaps mitigating some of the “July effect” and monitoring progress of learners to help them achieve learning goals.10
5
ACCEPTED MANUSCRIPT We are working to improve validity evidence; for some of the competency domains there was rich assessment data, in others there was less robust data. The small n and lower response rate may reflect response bias and limits generalizability of our findings, so larger
consistent practice standards will need to be implemented.
RI PT
studies are indicated. For the MH-ILP to be useful, broader use at more institutions and
In conclusion, existing UME assessments can provide meaningful data to determine most pediatric milestone levels. The MH-ILP provides an opportunity to improve the
SC
assessment of learners and contribute to communication between UME and GME, further
AC C
EP
TE D
M AN U
developing the continuum of learning.
6
ACCEPTED MANUSCRIPT References: 1
The Accreditation Council for Graduate Medical Education and the American Board of Pediatrics. The Pediatric
Milestone Project. 2015. Retrieved from https://acgme.org/Portals/0/PDFs/Milestones/PediatricsMilestones.pdf
Warm EJ, Englander R, Pereira A, Barach P. Improving learner handovers in medical education. Acad Med.
RI PT
2
2017;92(7):927-31. 3
Association of American Medical Colleges. Recommendations for revising the medical student performance
evaluation.
Sozner CN, Lypson ML, House JH et al. Reporting achievement of medical student milestones to residency
M AN U
4
SC
https://www.aamc.org/download/470400/data/mspe-recommendations.pdf; 2016 accessed 25.07.17.
program directors: An educational handover. Acad Med. 2016;91:676-684. 5
Wancata LM, Morgan H, Sandhu G et al. Using the ACGME milestones as a handover tool from medical school to
surgery residency. J Surg Educ. 2017;74(3):519-29.
Morgan H, Skinner B, Marzano D, et al. Bridging the continuum: Lessons learned from creating a competency-
TE D
6
based educational handover in obstetrics and gynecology. Med Sci Educ. 2016;26:443-447. 7
Frank JR, Snell LS, ten Cate O, et al. Competency-based medical education: theory to practice. Med Teach.
8
EP
2010;32:638-45.
ABIM Foundation, ACP-ASIM Foundation and European Federation of Internal Medicine. Medical professionalism
9
AC C
in the new millennium: A physician charter. Ann Intern Med. 2002;136:243-246.
Accreditation Council for Graduate Medical Education. ACGME program requirements for graduate medical
education in pediatrics. 2016. Retrieved from https://www.acgme.org/Portals/0/PFAssets/ProgramRequirements/320_pediatrics_2016.pdf 10
Pangaro L. “Forward feeding” about students’ progress: More information will enable better policy. Acad Med.
2008;83:802-803.
7
ACCEPTED MANUSCRIPT
AC C
EP
TE D
M AN U
SC
RI PT
Acknowledgements: The authors would like to thank the University of Michigan Graduate Medical Educations Innovations Program Grant for funding to support the dissemination of our work at the Council on Medical Student Education in Pediatrics Annual Meeting 2017 and Pediatric Academic Societies Annual Meeting 2017. The authors would also like to thank the competency committee members Kim Lomis, Mark Mendelsohn, Jeremy Middleton, David Stewart, Rebecca Swan and Linda Waggoner-Fountain.
8
ACCEPTED MANUSCRIPT
Table 1: Pediatric residency program director perceptions of the Medical Student Performance Evaluation (MSPE) and Milestone Handover-Individual Learning Plan (MH-ILP)
4.2 (0.63)
3.1 (1.20)
SC
2.8 (0.91)
Six month follow-up survey average(n=10, 53% response rate)
RI PT
4.0 (0.65)
M AN U
2.4 (0.89)
3.9 (1.10)
3.0 (0.93)
3.4 (0.97)
3.6 (1.26)
3.8 (0.92)
19% early intervention for an area of weakness; 13% advanced curriculum for an area of strength; 44% no change; 31% uncertain; 19% other
3.9 (0.88) 50% did not use information; 10% early intervention for an area of weakness; 10% used to confirm existing opinion; 10% correlated with residency milestones. 20% for "information only"
EP
N/A
How did you use the MH-ILP in your approach to the new trainee(s)? Does the MH-ILP provide you with more useful information over the traditional MSPE? 81% yes, 19% no N/A The MH-ILP provided useful information in addition to the traditional MSPE B N/A Data are presented as mean (SD) or % except where otherwise noted.
AC C
N/A
3.5 (0.97)
TE D
Survey Questions How well did the MSPE describe new trainees' strengths?A How well did the MSPE describe new trainees' weaknesses? A How well does the MSPE describe level one milestones of a new trainee? A How well did the MH-ILP describe a new trainee's strengths? A How well did the MH-ILP describe a new trainee's weaknesses? A How accurately did the MHILP describe milestones of your new trainee(s)? A How well the MH-ILP correlate with the trainee's mid-internship milestone assessment? A
Initial survey average (n=16, 84% response rate)
3.7 (1.42)
A- Likert scale, 1= not at all, 2= slightly, 3=somewhat, 4=moderately, 5=extremely well B- Likert scale, 1= strongly disagree, 2= disagree, 3= neutral, 4=agree, 5= strongly agree N/A= not applicable
S1876-2859(17)30490-4
DOI:
10.1016/j.acap.2017.09.010
Reference:
ACAP 1096
To appear in:
Academic Pediatrics
Received Date: 25 April 2017 Revised Date:
21 August 2017
Accepted Date: 2 September 2017
Please cite this article as: Schiller JH, Burrows HL, Fleming, AE, Keeley MG, Wozniak L, Santen SA, Responsible Milestone-Based Educational Handover with Individualized Learning Plan from Undergraduate to Graduate Pediatric Medical Education, Academic Pediatrics (2017), doi: 10.1016/ j.acap.2017.09.010. This is a PDF file of an unedited manuscript that has been accepted for publication. As a service to our customers we are providing this early version of the manuscript. The manuscript will undergo copyediting, typesetting, and review of the resulting proof before it is published in its final form. Please note that during the production process errors may be discovered which could affect the content, and all legal disclaimers that apply to the journal pertain.
ACCEPTED MANUSCRIPT Title: Responsible Milestone-Based Educational Handover with Individualized Learning Plan from Undergraduate to Graduate Pediatric Medical Education Jocelyn Schiller, MD, Heather Burrows, MD, PhD, Amy Fleming, MD, MHPE, Meg Keeley, MD, Lauren Wozniak, Sally A Santen, MD, PhD
SC
TE D
Amy E. Fleming, MD Department of Pediatrics Vanderbilt University School of Medicine 201 Light Hall Nashville, TN 37232 [email protected]
Lauren Wozniak, MPH University of Michigan Medical School 2635 Gladstone Ave. Ann Arbor, MI 48104 [email protected]
M AN U
Authors: Heather L Burrows, MD, PhD Department of Pediatrics University of Michigan Medical School 3237 Medical Professional Building Ann Arbor, MI 48109 [email protected]
RI PT
Corresponding Author: Jocelyn Huang Schiller, MD Department of Pediatrics University of Michigan Medical School 1540 E. Hospital Drive Mott 12-525A/SPC 4280 Ann Arbor, MI 48109 Phone: 734-763-5359 FAX: 734-647-5624 [email protected]
Sally A Santen, MD, PHD Department of Emergency Medicine University of Michigan Medical School 6121 Taubman Health Sciences Library 1135 Catherine Street Ann Arbor, MI 48109-5726 [email protected]
AC C
EP
Meg G. Keeley M.D. Department of Pediatrics University of Virginia Health System PO Box 800739 Charlottesville, VA 22908 [email protected]
Key Words: Medical student, pediatrics, graduate medical education, handover, milestones Running title: Responsible Milestone Educational Handover for Pediatric Students Text word count:
Funding: The authors would like to thank the University of Michigan Graduate Medical Education Innovations Program Grant for funding to support the dissemination of our work at the Council on Medical Student Education in Pediatrics Annual Meeting 2017 and Pediatric Academic Societies Annual Meeting 2017. University of Michigan Medical School has an Accelerating Change in Medical Education Grant from the American Medicine Association. The authors have no potential conflicts of interest or corporate sponsors. 1
ACCEPTED MANUSCRIPT What’s New?
Communication of medical students’ pediatric milestone assessments and individual learning plans from medical schools to pediatric residency directors allows for effective educational
RI PT
handovers promoting the continuum of education. Existing undergraduate medical education assessments can provide meaningful data to determine most pediatric milestone levels.
SC
Background
Milestones are competency-based, developmental outcomes marking progress from
M AN U
medical school through residency into practice. The Accreditation Council for Graduate Medical Education (ACGME) defined milestones for all specialties including pediatrics.1 As medical education moves towards competency-based assessment, there are calls for a continuum, including communication between undergraduate (UME) and graduate medical education
TE D
(GME).2
Currently, the UME to GME educational handover is the residency application, including the Medical Student Performance Evaluation (MSPE), transcripts and reference letters. The
EP
MSPE is created almost a year before internship, without an update. Also, there are concerns regarding the trustworthiness of the MSPE.2 Although transparency may improve with the 2016
AC C
MSPE recommendations, most schools do not include competencies, and if included, do not provide the detail needed to determine specialty specific milestones.3 These communication failures hinder learners’ education, wastes educational resources and may cause harm to patients.2 To address these issues, one institution’s departments of emergency medicine, surgery, and obstetrics/gynecology developed specialty specific, milestone-based competency
2
ACCEPTED MANUSCRIPT handovers between UME and GME after the National Resident Matching Program Match.4-6 Such efforts in pediatrics have yet to be described. We sought to develop a Pediatric Milestone Handover with individual learning plan (MH-ILP) to communicate with residency program directors (PDs) after the Match. Using a
RI PT
competency-based framework,7 we shared students’ attainment of pediatric ACGME
milestones. Pediatric milestones to allow schools and residencies to share a framework for assessment and communication. Using self-determination theory, ILPs were added to
SC
encourage student self-improvement through goal setting.1,8,9 In this study, we determined the
Educational Approach and Innovation
M AN U
feasibility and utility of the MH-ILP from the perspective of pediatric PDs.
In 2016, graduating students entering pediatric residencies were identified at three schools. Students voluntarily participated; written consent was obtained. This study was
TE D
approved by each institution’s Institutional Review Boards. Pediatric competency assessment committees comprised of PDs, clerkship directors,
EP
and internship readiness course faculty were established at each school. In spring 2016, committees reviewed the performance of their school’s participating students. After
AC C
investigators discussed data sources, each school’s committee used their own understanding of milestones to plot students. Data included performance assessments from pediatric courses (clerkships, acting internships, fourth year electives, and internship readiness courses) and observed structured clinical examinations. By consensus, committees mapped students’ assessments to the milestones and determined milestone level (Online Appendix).1 Milestones without data were marked as “not assessed.” Although 49 pediatric milestones exist, we used
3
ACCEPTED MANUSCRIPT the 21 milestones required for ACGME reporting due to limited committee experience with the other milestones. Students were trained on the development of Specific, Measurable, Attainable, Realistic, Timely goals. Students performed self-assessments using the milestones and set learning goals.
RI PT
Most met with a committee member to compare their self-assessment with committee
assessment and further refine their ILP. Two investigators reviewed the ILPs and discussed the data, codes, and themes until they reached consensus.
SC
MH-ILP included committee milestone assessments and student–identified learning
M AN U
goals. The post-Match MH-ILP was sent to their residency PD with an explanation and a survey. PDs were asked about the ability of the MH-ILP and MSPE to describe students’ strengths and weaknesses on a Likert scale (1= not at all, 5= extremely well) (Table 1). A similar survey was
Results
TE D
sent 6 months later.
MH-ILPs were created for 27 students who matched into 19 different programs.
EP
Committees were able to consistently assess 16 of the 21 pediatric milestones. Students were assessed to be performing at level one or greater. None of the schools consistently assessed the
AC C
milestone “Practice-Based Learning and Improvement 3” regarding quality improvement. Students developed ILP goals aimed at the first internship months. Although students’
141 goals covered all ACGME competency domains, the three most common sub-competencies chosen were perform appropriate learning activities (14%), identify strengths and weaknesses (13%) and organize/prioritize care (13%). Sixteen PDs (84%) responded to our initial survey; 10 (53%) responded to the follow-up (Table). In the follow-up survey, PDs felt the MH-ILP described students’ strengths and 4
ACCEPTED MANUSCRIPT weaknesses moderately well (strengths mean 3.9, weaknesses mean 3.4). Milestones correlated moderately with the trainee’s mid-internship milestone assessments (mean 3.9). One respondent commented the MH-ILP “correlated with their milestones here” and another PD noted, “As a confirmation, seems to be continuing same positive trend,” lending support for
RI PT
our milestone level assessments. Most (81%) felt the new MH-ILP provided more useful information over the traditional MSPE.
SC
Discussion and Next Steps
M AN U
This innovation demonstrates graduating students entering pediatrics can be assessed on the majority of the pediatric required milestones and handover from UME to GME is moderately useful to pediatric PDs. Importantly, the MH-ILP provided better description of the trainee compared to the MSPE.
The milestone framework1,7 was helpful in interpreting our data and confirming our
TE D
observations that these graduates were competent to begin residency and student performance varied. Although we did not record the time spent collecting and reviewing
EP
student data, there was significant effort involved, which is likely to be streamlined with experience. We do believe, however, that communication between UME and GME will improve
AC C
the trainees’ educational continuum. Although PDs had mixed responses to the MH-ILP, as medical education continues to dialogue about competencies, our shared understanding will increase. If more schools communicated on this level, program directors would come to expect the information. They could use the information to tailor the intern year, perhaps mitigating some of the “July effect” and monitoring progress of learners to help them achieve learning goals.10
5
ACCEPTED MANUSCRIPT We are working to improve validity evidence; for some of the competency domains there was rich assessment data, in others there was less robust data. The small n and lower response rate may reflect response bias and limits generalizability of our findings, so larger
consistent practice standards will need to be implemented.
RI PT
studies are indicated. For the MH-ILP to be useful, broader use at more institutions and
In conclusion, existing UME assessments can provide meaningful data to determine most pediatric milestone levels. The MH-ILP provides an opportunity to improve the
SC
assessment of learners and contribute to communication between UME and GME, further
AC C
EP
TE D
M AN U
developing the continuum of learning.
6
ACCEPTED MANUSCRIPT References: 1
The Accreditation Council for Graduate Medical Education and the American Board of Pediatrics. The Pediatric
Milestone Project. 2015. Retrieved from https://acgme.org/Portals/0/PDFs/Milestones/PediatricsMilestones.pdf
Warm EJ, Englander R, Pereira A, Barach P. Improving learner handovers in medical education. Acad Med.
RI PT
2
2017;92(7):927-31. 3
Association of American Medical Colleges. Recommendations for revising the medical student performance
evaluation.
Sozner CN, Lypson ML, House JH et al. Reporting achievement of medical student milestones to residency
M AN U
4
SC
https://www.aamc.org/download/470400/data/mspe-recommendations.pdf; 2016 accessed 25.07.17.
program directors: An educational handover. Acad Med. 2016;91:676-684. 5
Wancata LM, Morgan H, Sandhu G et al. Using the ACGME milestones as a handover tool from medical school to
surgery residency. J Surg Educ. 2017;74(3):519-29.
Morgan H, Skinner B, Marzano D, et al. Bridging the continuum: Lessons learned from creating a competency-
TE D
6
based educational handover in obstetrics and gynecology. Med Sci Educ. 2016;26:443-447. 7
Frank JR, Snell LS, ten Cate O, et al. Competency-based medical education: theory to practice. Med Teach.
8
EP
2010;32:638-45.
ABIM Foundation, ACP-ASIM Foundation and European Federation of Internal Medicine. Medical professionalism
9
AC C
in the new millennium: A physician charter. Ann Intern Med. 2002;136:243-246.
Accreditation Council for Graduate Medical Education. ACGME program requirements for graduate medical
education in pediatrics. 2016. Retrieved from https://www.acgme.org/Portals/0/PFAssets/ProgramRequirements/320_pediatrics_2016.pdf 10
Pangaro L. “Forward feeding” about students’ progress: More information will enable better policy. Acad Med.
2008;83:802-803.
7
ACCEPTED MANUSCRIPT
AC C
EP
TE D
M AN U
SC
RI PT
Acknowledgements: The authors would like to thank the University of Michigan Graduate Medical Educations Innovations Program Grant for funding to support the dissemination of our work at the Council on Medical Student Education in Pediatrics Annual Meeting 2017 and Pediatric Academic Societies Annual Meeting 2017. The authors would also like to thank the competency committee members Kim Lomis, Mark Mendelsohn, Jeremy Middleton, David Stewart, Rebecca Swan and Linda Waggoner-Fountain.
8
ACCEPTED MANUSCRIPT
Table 1: Pediatric residency program director perceptions of the Medical Student Performance Evaluation (MSPE) and Milestone Handover-Individual Learning Plan (MH-ILP)
4.2 (0.63)
3.1 (1.20)
SC
2.8 (0.91)
Six month follow-up survey average(n=10, 53% response rate)
RI PT
4.0 (0.65)
M AN U
2.4 (0.89)
3.9 (1.10)
3.0 (0.93)
3.4 (0.97)
3.6 (1.26)
3.8 (0.92)
19% early intervention for an area of weakness; 13% advanced curriculum for an area of strength; 44% no change; 31% uncertain; 19% other
3.9 (0.88) 50% did not use information; 10% early intervention for an area of weakness; 10% used to confirm existing opinion; 10% correlated with residency milestones. 20% for "information only"
EP
N/A
How did you use the MH-ILP in your approach to the new trainee(s)? Does the MH-ILP provide you with more useful information over the traditional MSPE? 81% yes, 19% no N/A The MH-ILP provided useful information in addition to the traditional MSPE B N/A Data are presented as mean (SD) or % except where otherwise noted.
AC C
N/A
3.5 (0.97)
TE D
Survey Questions How well did the MSPE describe new trainees' strengths?A How well did the MSPE describe new trainees' weaknesses? A How well does the MSPE describe level one milestones of a new trainee? A How well did the MH-ILP describe a new trainee's strengths? A How well did the MH-ILP describe a new trainee's weaknesses? A How accurately did the MHILP describe milestones of your new trainee(s)? A How well the MH-ILP correlate with the trainee's mid-internship milestone assessment? A
Initial survey average (n=16, 84% response rate)
3.7 (1.42)
A- Likert scale, 1= not at all, 2= slightly, 3=somewhat, 4=moderately, 5=extremely well B- Likert scale, 1= strongly disagree, 2= disagree, 3= neutral, 4=agree, 5= strongly agree N/A= not applicable