- Email: [email protected]
Enhancing Pediatric Fellows' Research Training: Development of an Office of Pediatric Clinical Fellowships
Enhancing Pediatric Fellows' Research Training: Development of an Office of Pediatric Clinical Fellowships Melissa Klein, MD, MEd, Gurjit K. Khurana Hershey, MD, PhD, Prasad Devarajan, MD, Louis J. Muglia, MD, PhD, Kathryn A. Wikenheiser-Brokamp, MD, PhD, Jill Loch, MEd, Margaret K. Hostetter, MD, Arnold W. Strauss, MD, and Thomas G. DeWitt, MD
T
raining fellows, the next generation of academic pediatricians, is challenging in rapidly changing and expanding clinical and research environments. Research methods and techniques are revolutionizing both translational and basic research. Investigators who can seamlessly transfer research from bench-to-bedside have become a critical asset. This bench-to-bedside approach has benefited greatly from more rigorous research in quality improvement and education,1-3 shifting from process to focus on patient outcomes.4-6 Academic institutions, their faculty, and fellowship programs must be equipped to effectively train fellows in these research methodologies, employing collaborative and interdisciplinary approaches to foster team science and allow all programs to utilize the array of institutional resources.
Institutional Need for Standardized Fellowship Training Accreditation Council for Graduate Medical Education (ACGME) accredited fellowships are designed to train the next generation of clinicians, researchers, and leaders.7 Although fellowships are often structured to emphasize scholarship, limited divisional resources and mentorship can impact fellows’ abilities to achieve the desired outcomes. An informal needs assessment by institutional leadership raised concern for such limitations and noted variability in recruitment, research training and collaboration. The Office of Pediatric Clinical Fellowships (OPCF) was developed to address this gap by augmenting research and recruitment support.
Utilizing a Logic Model to Conceptualize the OPCF Institutional and graduate medical education leadership utilized a logic model framework8,9 to build the OPCF. We envisioned exceptional, multidisciplinary, interdivisional research training with diminished variability between fellowship programs (Figure; available at www. jpeds.com). The leadership team and key stakeholders
defined OPCF elements, highlighting how the OPCF program (inputs/resources) would lead to desired products (outputs/activities). Both short- and long-term outcomes were conceived as critical components for evaluation purposes. Short-term outcomes reflect lower level learning, and long-term outcomes relate to the application of learning.10-13
Development of the OPCF With support from the department chair and the designated institutional official, who defined scope and funding, key stakeholders’ roles were determined. Faculty with varied research expertise (basic science, translational, clinical, quality improvement, and educational) were identified as steering committee members to enhance cross-divisional research training, collaboration, and to foster team science. This also helped to create a sense of community among the pediatric fellowship programs. After reviewing the literature for similar programs,14,15 OPCF leadership proposed to the institution’s 33 ACGME-accredited fellowship directors a centralized resource. As currently developed, the OPCF contains elements to assist fellowship directors in: (1) recruitment; (2) research education; (3) fostering interdisciplinary research; (4) implementing standardized assessment tools (eg, individualized development plans, scholarship oversight committee documents); and (5) tracking outcomes and applying for training grants through a centralized database. Our graduate medical education office assists fellowship directors in the ACGME administrative and regulatory tasks. The OPCF role is complimentary and focuses on research infrastructure, outcome tracking, and recruitment enhancement. With these defined roles, the OPCF manager was partnered with an application specialist for website development and data repository creation. Because recruiting the best fellows was critical, a human resources recruitment specialist was hired to optimize the recruitment process.
From the Cincinnati Children’s Hospital Medical Center, Cincinnati, OH
ACGME OPCF
Accreditation Council for Graduate Medical Education Office of Pediatric Clinical Fellowships
The authors declare no conflicts of interest. 0022-3476//$ - see front matter. Copyright ª 2015 Elsevier Inc. All rights reserved. http://dx.doi.org/10.1016/j.jpeds.2015.05.053
506
Vol. 167, No. 3 September 2015
Fellow Research Education and Funding Opportunities Research training enhancements was prioritized as the OPCF’s most critical role. The institution’s preexisting curricula were augmented by the novel “Fellows’ Crosstalk.” In this bimonthly seminar, fellows present their research to other fellows and faculty with diverse research backgrounds. Each presenter receives structured feedback on positive aspects of his/her project, suggestions for improvement, and potential interdisciplinary collaborations. In addition, an institutional grant funding mechanism was implemented, which provides senior fellows the opportunity to write a proposal in a format consistent with national grant funding requirements. The proposals are reviewed by a panel of faculty with prior grant funding success, and fellows receive feedback on their application’s quality and suggestions for improvement. The top 3 proposals are awarded funding designed to assist the fellow in academic pursuits, provide support as they transition to junior faculty positions, and prepare them for their next grant application. Additionally, the OPCF created a repository of standardized tools to aid fellow development and prior successful training grant proposals that fellowship directors can reference. Enhanced outcome tracking capabilities will augment division- and program-based recruitment and funding procurement.
Short-Term Outcomes The establishment and tracking of objective short-term outcomes was critical for evaluating OCPF’s success. First, investment in enhanced research training and mentorship through Fellows’ Crosstalk, availability of standardized development tools, and internal grant awards has led to increased cross-divisional collaborations, national presentations, and peer-reviewed publications. Second, serial comparison of the match results from the ACGME-accredited fellowships serve as an indicator of program competiveness, reflecting the recruitment process. Third, an increased number of proposals for the new internal grant program may enhance the quality of scientific writing leading to higher rates of external funding for fellows and junior faculty. Fourth, fellow scholarly accomplishments leading to enhanced national exposure may increase the visibility of
our fellowship programs. Finally, developing, implementing, and tracking outcome measures in an accessible, centralized, and shared database should facilitate increased applications, and reapplications, for training grants.
Long-Term Outcomes The long-term outcomes focus on fellows’ academic success. Our primary outcome metric is that an increased proportion of graduates receive academic appointments with significant research expectations. We will also continuously assess faculty success in receiving independent funding, and whether divisions sustain, or develop, funded fellowship programs. We believe the OPCF fostered community of interdivisional research will make our institution a model for training to prepare fellows for careers as academic pediatric leaders.
Challenges Establishing the OCPF entailed defining, expecting, and managing challenges. These centered on defining key stakeholders, creating buy-in, and securing funding. By expecting these challenges and dealing with them in a collaborative manner, we were able to overcome them.
Conclusion Consistent recruitment of top candidates into well-designed, interdisciplinary, mentored fellowship programs should enhance the training experience and increase potential academic success. Assisting fellowship directors in tracking outcomes and applying for program funding may improve the quality and desirability of all institutional fellowship programs. The OPCF was designed to support both fellows and fellowship directors as we strive to develop exceptionally-trained academic pediatricians, a goal that is critical in this era of rapidly evolving science. n Reprint requests: Melissa Klein, MD, MEd, Cincinnati Children’s Hospital Medical Center, 3333 Burnet Ave, MLC 2011, Cincinnati, OH 45229. E-mail: [email protected]
References available at www.jpeds.com
507
THE JOURNAL OF PEDIATRICS
www.jpeds.com
References 1. Taylor MJ, McNicholas C, Nicolay C, Darzi A, Bell D, Reed JE. Systematic review of the application of the plan-do-study-act method to improve quality in healthcare. BMJ Qual Saf 2014;23:290-8. 2. Irby DM, Hodgson CS, Muller JH. Promoting research in medical education at the University of California, San Francisco, School of Medicine. Acad Med 2004;79:981-4. 3. Glassick CE. Boyer’s expanded definitions of scholarship, the standards for assessing scholarship, and the elusiveness of the scholarship of teaching. Acad Med 2000;75:877-80. 4. Wong BM, Levinson W, Shojania KG. Quality improvement in medical education: current state and future directions. Med Educ 2012;46:107-19. 5. Starmer AJ, Spector ND, Srivastava R, West DC, Rosenbluth G, Allen AD, et al. Changes in medical errors after implementation of a handoff program. N Engl J Med 2014;371:1803-12. 6. Chen FM, Bauchner H, Burstin H. A call for outcomes research in medical education. Acad Med 2004;79:955-60. 7. ACGME. ACGME Common Program Requirements for 1-Year Fellowships. https://www.acgme.org/acgmeweb/Portals/0/PFAssets/Program Requirements/CPRs2013.pdf. Accessed November 7, 2014.
Vol. 167, No. 3 8. Armstrong EG, Barsion SJ. Using an outcomes-logic-model approach to evaluate a faculty development program for medical educators. Acad Med 2006;81:483-8. 9. Balmer DF, Schwartz A. Innovation in pediatric residency education: the role of evaluation. Pediatrics 2010;126:1-3. 10. Wilkes M, Bligh J. Evaluating educational interventions. BMJ 1999;318: 1269-72. 11. Morrison J. ABC of learning and teaching in medicine: evaluation. BMJ 2003;326:385-7. 12. Hutchinson L. Evaluating and researching the effectiveness of educational interventions. BMJ 1999;318:1267-9. 13. Kirkpatrick DL. Evaluating training programs: A collection of articles from the Journal of the American Society for Training and Development. Madison, Wisconsin: ASTD; 1975. 14. Hospital BCs. Office of Fellowship Training Webpage. http://www.childrens hospital.org/research-and-innovation/research/research-administration/ office-of-fellowship-training. Accessed May 1, 2014. 15. Philadelphia CsHo. Children’s Hospital of Philadelphia Office of Fellowships. http://www.chop.edu/pediatric-fellowships/office-fellowshipprograms#.VF0XuGYo7IV. Accessed May 1, 2014.
Figure. Logic model of development and evaluation of the OPCF. HR, human resources. 507.e1
Klein et al
T
raining fellows, the next generation of academic pediatricians, is challenging in rapidly changing and expanding clinical and research environments. Research methods and techniques are revolutionizing both translational and basic research. Investigators who can seamlessly transfer research from bench-to-bedside have become a critical asset. This bench-to-bedside approach has benefited greatly from more rigorous research in quality improvement and education,1-3 shifting from process to focus on patient outcomes.4-6 Academic institutions, their faculty, and fellowship programs must be equipped to effectively train fellows in these research methodologies, employing collaborative and interdisciplinary approaches to foster team science and allow all programs to utilize the array of institutional resources.
Institutional Need for Standardized Fellowship Training Accreditation Council for Graduate Medical Education (ACGME) accredited fellowships are designed to train the next generation of clinicians, researchers, and leaders.7 Although fellowships are often structured to emphasize scholarship, limited divisional resources and mentorship can impact fellows’ abilities to achieve the desired outcomes. An informal needs assessment by institutional leadership raised concern for such limitations and noted variability in recruitment, research training and collaboration. The Office of Pediatric Clinical Fellowships (OPCF) was developed to address this gap by augmenting research and recruitment support.
Utilizing a Logic Model to Conceptualize the OPCF Institutional and graduate medical education leadership utilized a logic model framework8,9 to build the OPCF. We envisioned exceptional, multidisciplinary, interdivisional research training with diminished variability between fellowship programs (Figure; available at www. jpeds.com). The leadership team and key stakeholders
defined OPCF elements, highlighting how the OPCF program (inputs/resources) would lead to desired products (outputs/activities). Both short- and long-term outcomes were conceived as critical components for evaluation purposes. Short-term outcomes reflect lower level learning, and long-term outcomes relate to the application of learning.10-13
Development of the OPCF With support from the department chair and the designated institutional official, who defined scope and funding, key stakeholders’ roles were determined. Faculty with varied research expertise (basic science, translational, clinical, quality improvement, and educational) were identified as steering committee members to enhance cross-divisional research training, collaboration, and to foster team science. This also helped to create a sense of community among the pediatric fellowship programs. After reviewing the literature for similar programs,14,15 OPCF leadership proposed to the institution’s 33 ACGME-accredited fellowship directors a centralized resource. As currently developed, the OPCF contains elements to assist fellowship directors in: (1) recruitment; (2) research education; (3) fostering interdisciplinary research; (4) implementing standardized assessment tools (eg, individualized development plans, scholarship oversight committee documents); and (5) tracking outcomes and applying for training grants through a centralized database. Our graduate medical education office assists fellowship directors in the ACGME administrative and regulatory tasks. The OPCF role is complimentary and focuses on research infrastructure, outcome tracking, and recruitment enhancement. With these defined roles, the OPCF manager was partnered with an application specialist for website development and data repository creation. Because recruiting the best fellows was critical, a human resources recruitment specialist was hired to optimize the recruitment process.
From the Cincinnati Children’s Hospital Medical Center, Cincinnati, OH
ACGME OPCF
Accreditation Council for Graduate Medical Education Office of Pediatric Clinical Fellowships
The authors declare no conflicts of interest. 0022-3476//$ - see front matter. Copyright ª 2015 Elsevier Inc. All rights reserved. http://dx.doi.org/10.1016/j.jpeds.2015.05.053
506
Vol. 167, No. 3 September 2015
Fellow Research Education and Funding Opportunities Research training enhancements was prioritized as the OPCF’s most critical role. The institution’s preexisting curricula were augmented by the novel “Fellows’ Crosstalk.” In this bimonthly seminar, fellows present their research to other fellows and faculty with diverse research backgrounds. Each presenter receives structured feedback on positive aspects of his/her project, suggestions for improvement, and potential interdisciplinary collaborations. In addition, an institutional grant funding mechanism was implemented, which provides senior fellows the opportunity to write a proposal in a format consistent with national grant funding requirements. The proposals are reviewed by a panel of faculty with prior grant funding success, and fellows receive feedback on their application’s quality and suggestions for improvement. The top 3 proposals are awarded funding designed to assist the fellow in academic pursuits, provide support as they transition to junior faculty positions, and prepare them for their next grant application. Additionally, the OPCF created a repository of standardized tools to aid fellow development and prior successful training grant proposals that fellowship directors can reference. Enhanced outcome tracking capabilities will augment division- and program-based recruitment and funding procurement.
Short-Term Outcomes The establishment and tracking of objective short-term outcomes was critical for evaluating OCPF’s success. First, investment in enhanced research training and mentorship through Fellows’ Crosstalk, availability of standardized development tools, and internal grant awards has led to increased cross-divisional collaborations, national presentations, and peer-reviewed publications. Second, serial comparison of the match results from the ACGME-accredited fellowships serve as an indicator of program competiveness, reflecting the recruitment process. Third, an increased number of proposals for the new internal grant program may enhance the quality of scientific writing leading to higher rates of external funding for fellows and junior faculty. Fourth, fellow scholarly accomplishments leading to enhanced national exposure may increase the visibility of
our fellowship programs. Finally, developing, implementing, and tracking outcome measures in an accessible, centralized, and shared database should facilitate increased applications, and reapplications, for training grants.
Long-Term Outcomes The long-term outcomes focus on fellows’ academic success. Our primary outcome metric is that an increased proportion of graduates receive academic appointments with significant research expectations. We will also continuously assess faculty success in receiving independent funding, and whether divisions sustain, or develop, funded fellowship programs. We believe the OPCF fostered community of interdivisional research will make our institution a model for training to prepare fellows for careers as academic pediatric leaders.
Challenges Establishing the OCPF entailed defining, expecting, and managing challenges. These centered on defining key stakeholders, creating buy-in, and securing funding. By expecting these challenges and dealing with them in a collaborative manner, we were able to overcome them.
Conclusion Consistent recruitment of top candidates into well-designed, interdisciplinary, mentored fellowship programs should enhance the training experience and increase potential academic success. Assisting fellowship directors in tracking outcomes and applying for program funding may improve the quality and desirability of all institutional fellowship programs. The OPCF was designed to support both fellows and fellowship directors as we strive to develop exceptionally-trained academic pediatricians, a goal that is critical in this era of rapidly evolving science. n Reprint requests: Melissa Klein, MD, MEd, Cincinnati Children’s Hospital Medical Center, 3333 Burnet Ave, MLC 2011, Cincinnati, OH 45229. E-mail: [email protected]
References available at www.jpeds.com
507
THE JOURNAL OF PEDIATRICS
www.jpeds.com
References 1. Taylor MJ, McNicholas C, Nicolay C, Darzi A, Bell D, Reed JE. Systematic review of the application of the plan-do-study-act method to improve quality in healthcare. BMJ Qual Saf 2014;23:290-8. 2. Irby DM, Hodgson CS, Muller JH. Promoting research in medical education at the University of California, San Francisco, School of Medicine. Acad Med 2004;79:981-4. 3. Glassick CE. Boyer’s expanded definitions of scholarship, the standards for assessing scholarship, and the elusiveness of the scholarship of teaching. Acad Med 2000;75:877-80. 4. Wong BM, Levinson W, Shojania KG. Quality improvement in medical education: current state and future directions. Med Educ 2012;46:107-19. 5. Starmer AJ, Spector ND, Srivastava R, West DC, Rosenbluth G, Allen AD, et al. Changes in medical errors after implementation of a handoff program. N Engl J Med 2014;371:1803-12. 6. Chen FM, Bauchner H, Burstin H. A call for outcomes research in medical education. Acad Med 2004;79:955-60. 7. ACGME. ACGME Common Program Requirements for 1-Year Fellowships. https://www.acgme.org/acgmeweb/Portals/0/PFAssets/Program Requirements/CPRs2013.pdf. Accessed November 7, 2014.
Vol. 167, No. 3 8. Armstrong EG, Barsion SJ. Using an outcomes-logic-model approach to evaluate a faculty development program for medical educators. Acad Med 2006;81:483-8. 9. Balmer DF, Schwartz A. Innovation in pediatric residency education: the role of evaluation. Pediatrics 2010;126:1-3. 10. Wilkes M, Bligh J. Evaluating educational interventions. BMJ 1999;318: 1269-72. 11. Morrison J. ABC of learning and teaching in medicine: evaluation. BMJ 2003;326:385-7. 12. Hutchinson L. Evaluating and researching the effectiveness of educational interventions. BMJ 1999;318:1267-9. 13. Kirkpatrick DL. Evaluating training programs: A collection of articles from the Journal of the American Society for Training and Development. Madison, Wisconsin: ASTD; 1975. 14. Hospital BCs. Office of Fellowship Training Webpage. http://www.childrens hospital.org/research-and-innovation/research/research-administration/ office-of-fellowship-training. Accessed May 1, 2014. 15. Philadelphia CsHo. Children’s Hospital of Philadelphia Office of Fellowships. http://www.chop.edu/pediatric-fellowships/office-fellowshipprograms#.VF0XuGYo7IV. Accessed May 1, 2014.
Figure. Logic model of development and evaluation of the OPCF. HR, human resources. 507.e1
Klein et al