- Email: [email protected]
HRSA National Faculty Development Scholars Program: Community-Based Teaching Track
APA/HRSA National Faculty Development Scholars Program: Community-Based Teaching Track Kenneth B. Roberts, MD; Jeffrey M. Devries, MD, MPH Objectives.—Goals and objectives of the APA/HRSA National Faculty Development Scholars Program are described in a companion article in this supplement. Program objectives of the Community-Based Teaching Track were to 1) identify individuals with faculty development skills to serve as regional leaders, conducting local and regional workshops; 2) create a national network of leaders, thereby promoting sustainability; and 3) increase educational contributions of office-based preceptors. Participant objectives were to 1) meet expectations of the program; 2) possess knowledge, skills, and attitudes needed to recruit and retain community-based preceptors; 3) possess knowledge and skills needed to conduct faculty development workshops for practitioners; and 4) form local and national collegial networks. Methods.—Academicians and practitioners were selected to form 2 cohorts, starting in 1999 (N 5 20) and 2000 (N 5 22), respectively. Workshops were developed on the basis of projected needs and iterative needs assessments of the participants. Each participant was required to conduct workshops between meetings, first locally and then regionally or nationally. Outcomes.—Forty-two participants conducted 307 workshops attended by 3815 individuals. Participants reported specific knowledge and skills gained from the program and increases in multiple areas of competence. Networks were formed, leading to collaborative regional and national presentations and formation of an APA Faculty Development Special Interest Group. Participants reported increased recruitment of training sites and preceptors, with enhanced relationships. Conclusion.—Program and participant objectives of the Community-Based Teaching Track were met, including implementation of multiple new workshops, development of skills, and creation of a support network to enhance the sustainability of that success. KEY WORDS:
community-based; education; faculty development
Ambulatory Pediatrics 2004;4:92 97
D
uring the past 2 decades, there has been increasing recognition of the value and importance of community-based teaching.1,2 Two pediatric programs in the late 1980s extended the resident continuity experience to office practices, identified the ‘‘nuts and bolts’’ necessary for such initiatives,3 demonstrated their financial feasibility,4 and documented both immediate and long-term educational benefits.5–7 Guidelines for the incorporation of office practices into the medical student clerkship experience were also developed.8 However, a survey in 1991 demonstrated that most residents still had very limited exposure to office practices, particularly before the third year of residency, and the duration of the experience was generally 1 month or less.9 Programs began to develop office-based experiences for residents and students. As initial efforts were necessarily directed at the logistics of preceptor recruitment and trainee placement, faculty development to promote effective and efficient learning in these settings was often omitted,
possibly limiting the effectiveness of such programs.10 In a national conference on Pediatric Education in Community Settings,11 faculty development was identified as an important component of successful, community-based education programs.12 At the time, there were no initiatives at the national level to coordinate pediatric faculty development programs for practitioner-preceptors or to increase the number and skills of leaders in faculty development for community-based teaching activities. The APA/HRSA National Faculty Development Scholars Program was designed to fill this need. The objectives of the overall program are described elsewhere in this supplement.13 The specific program objectives of the CommunityBased Teaching Track were to: 1) Identify individuals with core skills in faculty development to serve as regional leaders, conducting local and regional faculty development workshops; 2) Create a national network of these leaders, resulting in an increased national presence of faculty development and sustained efforts after the period of HRSA contract funding was completed; and 3) Improve and increase the contributions of office-based practitioner-preceptors to pediatric medical education.
From the Moses Cone Health System (Dr Roberts), Greensboro, NC, and University of North Carolina School of Medicine, Chapel Hill, NC and Oakwood Healthcare System (Dr Devries), Dearborn, MI, and University of Michigan Medical School, Ann Arbor, MI. Address correspondence to Kenneth B. Roberts, MD, Moses Cone Hospital, Pediatric Teaching Program, 1200 N Elm St, Greensboro, NC 27401 (e-mail: [email protected]). Received for publication April 26, 2003; accepted August 4, 2003. AMBULATORY PEDIATRICS Copyright q 2004 by Ambulatory Pediatric Association
Participant objectives were for them to: 1) Meet the expectations of the program (eg, attend all sessions, conduct regional workshops between program meetings);
92
Volume 4, Number 1 Supplement January–February 2004
AMBULATORY PEDIATRICS
National Faculty Development Scholars Program: Community-Based Teaching
2) Possess the requisite knowledge, skills, and attitudes to recruit and retain community-based preceptors; 3) Possess the requisite knowledge and skills to plan and conduct faculty development workshops for practitioners on topics related to office-based precepting (eg, principles of adult learning, effective and efficient teaching, evaluation, feedback); and 4) Form local and national collegial networks. PROGRAM OVERVIEW Two cohorts of 20 individuals were planned, consisting of 2 individuals from each of the Ambulatory Pediatric Association’s 10 geographic regions, comprising 40 participants in all. A series of ‘‘Train the Trainer’’ workshops was designed with a minimum of 3 per cohort. Participants were expected to attend all sessions and to plan and conduct faculty development workshops during the course of the program. Between the first and second sessions, participants were expected to conduct a faculty development workshop in their own institution or community. Between the second and third sessions, they were expected to conduct a regional or national faculty development workshop. The process by which participants were recruited is reported elsewhere in this supplement.13 In the selection process for the Community-Based Teaching Track, priority was given to individuals who had experience in faculty development and in community-based education and who were positioned in their institutions to accomplish the assigned tasks of conducting workshops. A letter from the department chair was required, committing support for the individual’s time and activities and endorsing the individual’s ability to meet the objectives of the program by virtue of position, interest, and skills. Interestingly, the majority of applicants lacked the significant experience in conducting faculty development workshops that had initially been presumed to be necessary to achieve the program’s goals. This was interpreted as verification of the paucity of experienced leaders in this field, thereby confirming the need for this initiative. Of 33 applicants for the first cohort, 20 (61%) were selected; of 35 applicants for the second cohort, 22 (63%) were selected. The first cohort consisted primarily of hospital-based faculty from institutions with communitybased teaching programs. The second cohort included more practitioners who represented their communities in such efforts. Among the 2 cohorts, 26% of participants were community preceptors. A series of three 2-day workshops was spread over 12 months (at 0, 6, and 12 months). Scheduling the workshops 6 months apart facilitated reinforcement, a progressive curriculum, increasing expectations, and the opportunity for participants to conduct their own workshops between sessions. Each scholars program meeting contained both plenary presentations and track-specific sessions. In light of the overlapping roles played within their departments by many of the participants, it was neither unusual nor unexpected for members of any of the tracks to express interest in the activities of the other tracks.
93
Cross-track sessions were incorporated to emphasize that the 3 elements (clinical teaching, educational scholarship, and leadership) were all vital, interrelated components of successful programs. The curriculum for the Train the Trainer sessions was initially developed in accordance with the anticipated needs of the participants, based upon review of the literature, consensus of program leaders, and results of an email survey sent to participants before the first meeting. Participants were requested to provide personal introductions to the group, as well as responses to the following questions: 1) What strengths and faculty development experience do you bring to the program? 2) What, specifically, do you expect/hope to achieve from the program? 3) In which areas might you be willing to contribute (eg, ‘‘I teach a session on ‘Creating a Learning Environment’ that practitioners like and have told me is useful. I would be happy to demonstrate it.’’; ‘‘I have a method for teaching . . . which I think works well. . . . ’’)? The initial session focused on teaching knowledge, skills, and attitudes needed to develop and maintain successful community-based education programs and to plan and conduct faculty development workshops. During the course of the first and all subsequent meetings, verbal surveys of participants were utilized to identify newly perceived needs or shifting priorities on the basis of previous experience in the program, as well as the needs and priorities identified through participants conducting their own workshops between program meetings. Additionally, needs assessment surveys were conducted by e-mail between meetings. For example, the following survey was sent requesting the participants to respond to the entire group: 1) What do you need to recruit, develop, and retain community faculty? Which of these should we spend time on? 2) What do those faculty need from you? What should we spend time on? 3) What content areas should we address? 4) What teaching skills do you want to develop in your community preceptors? Which of these should we spend time on? 5) What teaching skills do you want to develop in yourself to make your faculty development efforts more effective? The responses to this and subsequent needs assessments were used to design the workshops, occasionally with midcourse corrections responding to the feedback. Topics are listed in Table 1. Presentations were provided by recognized national leaders, including those from academic medical center backgrounds as well as experienced community preceptors whose ‘‘legitimacy’’ was recognized from the perspective of community-based practitioners.14 In later sessions, participants were asked to volunteer to lead topic discussions, with the intent of mastering the
94
Roberts & Devries
Table 1. Topics Addressed in the Community-Based Teaching Track ● Developing and maintaining teaching programs in practitioners’ offices ● Adult learning theory and its application ● Creating a constructive learning climate ● Educational planning process (goals, needs assessment, objectives, methods, evaluation) ● Strategies, methods, skills, and instruments to conduct workshops (eg, warm-ups and ‘‘ice-breakers,’’ ‘‘microskills,’’ modeling and ‘‘activated demonstration,’’ role-playing, facilitation of small discussions) ● Learner feedback and evaluation ● Preceptor feedback and evaluation ● Evaluation of workshops ● Technology and various media in medical education ● Distance learning strategies ● Planning and conducting faculty development sessions of various lengths (eg, 1-hour session, full-day workshop) ● Management of ‘‘problem learners’’ ● Advanced faculty development strategies
material, experimenting with various instructional methodologies, and receiving feedback on their teaching skills in a supportive environment. Basic elements of key educational principles (eg, adult learning theory, the educational planning process, and design of constructive learning climates) were addressed. The structure and function of workshops were addressed in the opening plenary session for participants in all 3 tracks to prepare individuals for the sessions in which they would participate during the program and to serve as a template for the sessions they would develop and lead in their regions. In the Community-Based Teaching Track, specific content for faculty development workshops that was familiar to many of the participants was included both to achieve consensus on key issues to be taught and to facilitate discussion on effective teaching methods. (Examples of such content addressed in the 3 workshops include ‘‘efficient teaching,’’ ‘‘feedback,’’ ‘‘evaluation,’’ and ‘‘management of problem learners.’’) Participants were encouraged to consider both the content of sessions and the methods used in teaching. Each Community-Based Teaching Track session closed with reflections about both the content of the session and the methods that had been used to teach the content. The objectives of this reflection activity were to reinforce key messages and to improve teaching techniques by identifying the advantages and disadvantages of various instructional strategies. Initially, one of the track leaders served as the reflector, reporting observations on what he perceived to be key elements of the presentation (content and process). Discussion ensued, reinforcing valuable lessons and generating ideas for improvement. In subsequent sessions, volunteers from the group rotated the ‘‘reflector’’ role responsibility. Through this activity, participants recognized the difficulty of trying simultaneously to present material to a group, gauge audience reaction, and troubleshoot environmental problems (eg, room lighting, changes in room temperature) when they occurred. Recognizing the value of assigning these tasks to an additional person, many of the scholars recruited a colleague to attend their
AMBULATORY PEDIATRICS
faculty development workshops to provide this valuable reflection and feedback. The second and third meetings included scheduled opportunities for sharing and reflecting about the local, regional, and national workshops conducted by participants between meetings. This additional reflection activity encouraged self-evaluation in a supportive, constructive environment. OUTCOMES The first program objective of the Community-Based Teaching Track was to identify individuals with skills in faculty development who would serve as regional leaders to conduct local and regional faculty development workshops. Related participant objectives were to attain the knowledge and skills to plan and conduct faculty development workshops for practitioners on topics related to office-based precepting (eg, principles of adult learning, effective and efficient teaching, evaluation, feedback). Of the 42 participants comprising the 2 cohorts, 40 (95%) completed the program. Participants reported high satisfaction with the individual sessions and actively contributed to both the planning and the delivery of program components. They conducted a total of 307 workshops (an average of 7.4 workshops per participant) attended by 3815 individuals. Participants reported that they incorporated into their workshops knowledge and skills gained in the APA/ HRSA National Faculty Development Scholars Program in the following areas: 1) Planning workshops (eg, development, time management, organization); 2) Teaching skills (eg, needs assessment, interactivity, facilitation skills, use of media, distilling key points); 3) Specific content (eg, problem learner, feedback, adult learning theory, ‘‘microskills’’); and 4) Evaluation tools and techniques. Many of the scholars gave presentations at track-specific sessions and cross-track sessions, permitting direct observations of their skills. Evaluation ratings by track leaders and program participants were high. Evaluation ratings by attendees of scholars’ local, regional, and national workshops were also high. Finally, participants reported their self-perception of increased competence by the end of the program in 12 areas related to workshop development, community-based teaching, and scholarship (Table 2). The second program objective of this track was to create a national network of these leaders, resulting in an increased national presence of faculty development and sustained efforts beyond the period of HRSA funding. The corresponding participant objective was for participants to form local and national collegial networks. Of the 40 who completed the program, 33 (83%) conducted 13 national workshops at meetings of the Association of American Medical Colleges, the Council on Medical Student Education in Pediatrics, and the Pediatric Academic Societies (PAS). More than 40% (17 of 40) were promoted, either
AMBULATORY PEDIATRICS
National Faculty Development Scholars Program: Community-Based Teaching
95
Table 2. Retrospective Pre- and Post-assessment of Perceived Competencies of Community-Based Teaching Track Participants (N 5 39)† Competencies‡
Pre-Program
Post-Program
Change
Set up your workshop(s): logistics, PR, budget Identify and access additional curriculum materials/resources for your workshops (eg, COMSEP) Conduct your workshop Exhibit confidence as a faculty developer during your workshop Utilize curriculum training materials in your workshop Refer your workshop attendees to other materials/resources Retain key faculty and personnel Use technology in your teaching Teach your workshop with an emphasis on the following concepts essential to primary care general pediatrics: ambulatory care, continuity of care, EBM, prevention, psychosocial aspects, health care delivery systems, population-based medicine, cost-of case analyses, utilization of interdisciplinary health care teams, utilization of innovative technology Develop criteria to judge your workshop success Use the criteria to judge your workshop success Enhance your own scholarly career in pediatrics
3.1 3.0
4.0 4.0
0.9* 1.0*
3.1 3.1 3.1 2.7 3.1 3.1 2.9
4.0 4.0 4.1 3.9 3.6 3.5 3.4
0.9* 0.9* 1.1* 1.2* 0.5* 0.4* 0.5*
2.8 2.9 2.9
3.6 3.7 3.7
0.8* 0.8* 0.8*
†Scale: 1 5 unsatisfactory; 3 5 good; 5 5 excellent. ‡PR indicates public relations; COMSEP, Council on Medical Student Education in Pediatrics; and EBM, evidence-based medicine. *P , .001.
in academic rank or in their role in education (eg, Vice Chair for Pediatric Education). The participants were active in forming a Faculty Development Special Interest Group within the Ambulatory Pediatric Association, which has met at each of the past 3 annual PAS meetings and has developed a curriculum strategy recently approved by the APA Board of Directors to sustain faculty development efforts at the national level. Networking occurred among program participants at both national and local levels. Members of the Community-Based Teaching Track reported the benefit of the reflection exercises and sought to recreate the academically invigorating atmosphere in their own institutions by networking with individuals in other departments and disciplines. From the beginning to the end of the program, they reported an increase in 5 of 6 key items in the Colleagues category of the 4/20 Habits of Academic Success Inventory15 (Table 3). Of note is that the item demonstrating the greatest percent increase was ‘‘willingness to borrow resources (eg, lecture notes and slides) from colleagues’’—a relatively straightforward gesture of cooperation that might have been presumed beforehand. This may serve to demonstrate that collegiality cannot be as-
sumed and that success in establishing networking and cooperation is a significant achievement. The third program objective was to improve and increase the contributions of office-based practitioner-preceptors to pediatric medical education. The corresponding participant objectives were to possess the knowledge, skills, and attitudes needed to recruit and retain community-based preceptors and to conduct faculty development workshops for practitioner-preceptors. Of 32 participants who responded to a question regarding organizational outcomes of the APA/HRSA National Faculty Development Scholars Program, 21 (66%) reported success in recruiting more community sites for learners, enhanced relations with community preceptors, and increased formal acknowledgment of the contributions of community preceptors (through awards, access to university resources, etc) at their institutions. One participant in the CommunityBased Teaching Track now leads the Resident Education and Training Special Interest Group of the American Academy of Pediatrics’ Section on Community Pediatrics, which is developing a ‘‘Starter Kit’’ to assist practitioners interested in precepting trainees in their offices.
Table 3. Habits of Academic Success15 Identified by Participants of the Community-Based Teaching Track (N 5 22)† Colleagues
Pre-Program
Post-Program
Change
Associate/collaborate with distinguished colleagues in your field Have a colleague network which includes senior colleagues, peers, administrators, and staff Collaborate with colleagues on writing, teaching, research, and administrative tasks Regularly obtain guidance and feedback from a senior colleague(s) Maintain regular contact with professional colleagues outside your institution Borrow resources from colleagues pertinent to a new assignment (eg, lecture notes and slides)
2.2
3.3
1.1*
2.2
3.1
0.9*
2.4 2.5 1.8
3.1 2.7 2.9
0.7* 0.2 1.1*
1.7
3.0
1.3*
†Scale: 1 5 not true; 4 5 very true. *P , .001.
96
Roberts & Devries
DISCUSSION The office practice of pediatrics is fast paced with a need for efficiency. Teaching in such settings is challenging,16 because in competition for time and attention, the primary focus is clearly on patient care rather than teaching. Practitioner-preceptors generally are volunteers who have limited experience with instructing trainees. The primary reward for such volunteers is satisfaction and stimulation, which learners do not always provide. Faculty development workshops can enhance this satisfaction and stimulation by improving teaching skills, clarifying curriculum expectations, and creating a supportive cohort of interested practitioner-preceptor colleagues.12,17 HRSA has provided funding for faculty development programs under Title VII for more than 2 decades. These grants permitted individual institutions to develop enhanced capabilities and, in some instances, national reputations. However, there were no organized systems to disseminate achievements and lessons learned either regionally or nationally. The APA/HRSA National Faculty Development Scholars Program sought to create such a network and succeeded. Scholars in the Community-Based Teaching Track have conducted workshops locally, regionally, and nationally. They have collaborated within the track to conduct national workshops and across tracks for both regional and national workshops. The scholars have recognized the value of assembling a national cohort for collegial support, intellectual stimulation, and collaboration. During the initial formation of this program, it was thought that participants would need to be somewhat senior faculty members with several years of experience conducting faculty development workshops in order to possess the minimum skills needed to advance to a position of regional leadership in this field. However, the majority of applicants lacked such extensive experience. The reason was not studied, but a likely explanation was the paucity of experienced leaders in this field—a fact recognized by the program’s leaders and affirming the need for such a program. Despite many participants’ lack of previous experience, the outcomes were impressive. The evaluation component consisted of multiple measures, including full program participation; the number of institutional, regional, and national workshops conducted between program meetings; observations of participants’ presentations at program sessions; and participants’ perceptions of having improved knowledge, skills, and comfort as a result of their involvement. Their own perceptions of achievement were based partially on their interpretation of the written and verbal feedback they received in response to the workshops that they conducted between program meetings. For some, even the fact that they initiated and conducted a workshop on this topic for the first time confirmed their enhanced confidence. Nevertheless, the evaluation process did not include many objective measures of actual acquisition of knowledge or skills. This competence should not be assumed, especially inasmuch as these leaders will likely become the teachers of subsequent workshop organizers and must possess solid
AMBULATORY PEDIATRICS
skills in order to pass these on to more junior colleagues. It will be important for further studies to evaluate the competence of these graduates—perhaps utilizing an evaluation instrument on paper or by computer but, more importantly, within the actual workshop setting. The benefits for the participants and for their individual programs have already been quantified. The long-term benefits to be derived through these scholars recruiting and training their colleagues—thereby creating an increasing network of faculty development leaders—are just beginning to be felt. The test will be whether efforts will be sustained beyond the period of HRSA funding and whether necessary institutional and external funding can be obtained. Participants reported increased commitment from their department chairs for resources to support faculty development and increased collaboration across departments and institutions. The formation and vitality of the new APA Faculty Development Special Interest Group and the creation of a Starter Kit to be disseminated by the American Academy of Pediatrics are also encouraging. An important longer term measure of success is whether the increasing number and quality of community-based teaching programs are sustained and spread to include additional institutions and whether, in the process, students, residents, practitioners, and academic institutions continue to benefit. Longitudinal follow-up studies to determine the extent of sustainability of this activity, and its dissemination throughout regions, will be necessary to assess the true long-term impact of this initiative. Finally, the most important outcome—albeit the most difficult to measure— is whether children become the ultimate beneficiaries of these educational activities. CONCLUSION The 3 program objectives of the Community-Based Teaching Track were achieved: 1) Individuals with core skills in faculty development were identified and have served as regional leaders, providing local and regional faculty development. 2) These individuals formed a national network, resulting in an increased national presence of faculty development as reflected in the APA Special Interest Group and workshops at national meetings. 3) Participants report improved and increased participation of office-based practitioner-preceptors in pediatric medical education and have undertaken efforts to facilitate increased participation. The 4 participant objectives were also achieved by the conclusion of the program. The participants 1) met the expectations of the program (eg, attended all sessions, conducted workshops); 2) possessed knowledge, skills, and attitudes to recruit and retain community-based preceptors; 3) possessed the knowledge and skills needed to plan and conduct faculty development workshops for practitioners on basic topics related to office-based precepting (eg, principles of adult learning, effective and efficient teaching, evaluation, feedback); and 4) formed local and national collegial networks expected to enhance the sustainability of this initiative.
AMBULATORY PEDIATRICS
National Faculty Development Scholars Program: Community-Based Teaching
ACKNOWLEDGMENTS The authors gratefully acknowledge the energy, active participation, productivity, and contributions of the National Faculty Development Scholars in the Community-Based Teaching Track. This work was supported by US Department of Health Resources and Services Administration (HRSA 240-97-0043) ‘‘Faculty Development of General Pediatrics Generalist Faculty Teaching in Community-Based Ambulatory Settings.’’
REFERENCES 1. Association of American Medical Colleges. Physicians for the twenty-first century: report of the project panel on the general professional education of the physician and college preparation for medicine. J Med Educ. 1984;59(suppl):1–200. 2. Council on Graduate Medical Education Eighth Report. Patient Care Physician Supply and Requirements: Testing the COGME Assumptions. Washington, DC: US Dept of Health and Human Services, Public Health Service, Health Resources and Services Administration, Bureau of Health Professions, Division of Medicine and Dentistry; 1996. Publication HRSA-P-DM 98–3. 3. Sargent JR, Osborn LM, Roberts KB, DeWitt TG. Establishment of primary care continuity experiences in community pediatricians’ offices: nuts and bolts. Pediatrics. 1993;91:1185– 1189. 4. Sargent JR, Osborn LM. Resident training in community pediatricians’ offices: not a financial drain. Am J Dis Child. 1990; 144:1356–1359. 5. Osborn LM, Sargent JR, Williams SD. Effects of time-in-clinic, clinic setting, and faculty supervision on the continuity clinic experience. Pediatrics. 1993;91:1089–1093. 6. Roberts KB, Starr S, DeWitt TG. The University of Massachusetts Medical Center office-based continuity experience: are we
7.
8.
9.
10.
11. 12. 13.
14. 15. 16.
17.
97
preparing pediatrics residents for primary care practice? Pediatrics. 1997;100:e2. Roberts KB, Starr S, DeWitt TG. Resident preparedness for practice: a longitudinal cohort study. Ambulatory Pediatr. 2002; 2:132–135. Scheiner AP. Guidelines for medical student education in community-based pediatric offices. American Academy of Pediatrics Council on Pediatric Education Subcommittee on Medical Student Curriculum. Pediatrics. 1994;93:956–959. Greenberg LW, Getson P, Brasseux C, et al. How are pediatric training programs preparing residents for practice? Am J Dis Child. 1991;145:1389–1392. Rice TD, Holmes SE, Drutz JE. Comparison of continuity clinic experience by practice setting and postgraduate level. Arch Pediatr Adolesc Med. 1996;150:1299–1304. Roberts KB, DeWitt T, eds. Pediatric resident education in community settings. Pediatrics. 1996;98:1249–1301. DeWitt TG. Faculty development for community practitioners. Pediatrics. 1996;98:1273–1276. Osborn LM, Roberts KB, Greenberg L, et al. The APA/HRSA Faculty Development Scholars Program: introduction to the supplement. Ambul Pediatr. 2004;4(suppl 1):83–87. Young LM. The perspective of the community pediatrician. Pediatrics. 1996;98:1255–1258. Simpson DE. 4/20 Habits of Academic Success Inventory. Milwaukee, Wis: Medical College of Wisconsin; 2002. Roberts KB, DeWitt TG. Faculty development of pediatric practitioners: complexities in teaching clinical precepting. Pediatrics. 1996;97:389–393. DeWitt TG, Goldberg RL, Roberts KB. Developing community faculty: principles, practice, and evaluation. AJDC. 1993;147: 49–53.
community-based; education; faculty development
Ambulatory Pediatrics 2004;4:92 97
D
uring the past 2 decades, there has been increasing recognition of the value and importance of community-based teaching.1,2 Two pediatric programs in the late 1980s extended the resident continuity experience to office practices, identified the ‘‘nuts and bolts’’ necessary for such initiatives,3 demonstrated their financial feasibility,4 and documented both immediate and long-term educational benefits.5–7 Guidelines for the incorporation of office practices into the medical student clerkship experience were also developed.8 However, a survey in 1991 demonstrated that most residents still had very limited exposure to office practices, particularly before the third year of residency, and the duration of the experience was generally 1 month or less.9 Programs began to develop office-based experiences for residents and students. As initial efforts were necessarily directed at the logistics of preceptor recruitment and trainee placement, faculty development to promote effective and efficient learning in these settings was often omitted,
possibly limiting the effectiveness of such programs.10 In a national conference on Pediatric Education in Community Settings,11 faculty development was identified as an important component of successful, community-based education programs.12 At the time, there were no initiatives at the national level to coordinate pediatric faculty development programs for practitioner-preceptors or to increase the number and skills of leaders in faculty development for community-based teaching activities. The APA/HRSA National Faculty Development Scholars Program was designed to fill this need. The objectives of the overall program are described elsewhere in this supplement.13 The specific program objectives of the CommunityBased Teaching Track were to: 1) Identify individuals with core skills in faculty development to serve as regional leaders, conducting local and regional faculty development workshops; 2) Create a national network of these leaders, resulting in an increased national presence of faculty development and sustained efforts after the period of HRSA contract funding was completed; and 3) Improve and increase the contributions of office-based practitioner-preceptors to pediatric medical education.
From the Moses Cone Health System (Dr Roberts), Greensboro, NC, and University of North Carolina School of Medicine, Chapel Hill, NC and Oakwood Healthcare System (Dr Devries), Dearborn, MI, and University of Michigan Medical School, Ann Arbor, MI. Address correspondence to Kenneth B. Roberts, MD, Moses Cone Hospital, Pediatric Teaching Program, 1200 N Elm St, Greensboro, NC 27401 (e-mail: [email protected]). Received for publication April 26, 2003; accepted August 4, 2003. AMBULATORY PEDIATRICS Copyright q 2004 by Ambulatory Pediatric Association
Participant objectives were for them to: 1) Meet the expectations of the program (eg, attend all sessions, conduct regional workshops between program meetings);
92
Volume 4, Number 1 Supplement January–February 2004
AMBULATORY PEDIATRICS
National Faculty Development Scholars Program: Community-Based Teaching
2) Possess the requisite knowledge, skills, and attitudes to recruit and retain community-based preceptors; 3) Possess the requisite knowledge and skills to plan and conduct faculty development workshops for practitioners on topics related to office-based precepting (eg, principles of adult learning, effective and efficient teaching, evaluation, feedback); and 4) Form local and national collegial networks. PROGRAM OVERVIEW Two cohorts of 20 individuals were planned, consisting of 2 individuals from each of the Ambulatory Pediatric Association’s 10 geographic regions, comprising 40 participants in all. A series of ‘‘Train the Trainer’’ workshops was designed with a minimum of 3 per cohort. Participants were expected to attend all sessions and to plan and conduct faculty development workshops during the course of the program. Between the first and second sessions, participants were expected to conduct a faculty development workshop in their own institution or community. Between the second and third sessions, they were expected to conduct a regional or national faculty development workshop. The process by which participants were recruited is reported elsewhere in this supplement.13 In the selection process for the Community-Based Teaching Track, priority was given to individuals who had experience in faculty development and in community-based education and who were positioned in their institutions to accomplish the assigned tasks of conducting workshops. A letter from the department chair was required, committing support for the individual’s time and activities and endorsing the individual’s ability to meet the objectives of the program by virtue of position, interest, and skills. Interestingly, the majority of applicants lacked the significant experience in conducting faculty development workshops that had initially been presumed to be necessary to achieve the program’s goals. This was interpreted as verification of the paucity of experienced leaders in this field, thereby confirming the need for this initiative. Of 33 applicants for the first cohort, 20 (61%) were selected; of 35 applicants for the second cohort, 22 (63%) were selected. The first cohort consisted primarily of hospital-based faculty from institutions with communitybased teaching programs. The second cohort included more practitioners who represented their communities in such efforts. Among the 2 cohorts, 26% of participants were community preceptors. A series of three 2-day workshops was spread over 12 months (at 0, 6, and 12 months). Scheduling the workshops 6 months apart facilitated reinforcement, a progressive curriculum, increasing expectations, and the opportunity for participants to conduct their own workshops between sessions. Each scholars program meeting contained both plenary presentations and track-specific sessions. In light of the overlapping roles played within their departments by many of the participants, it was neither unusual nor unexpected for members of any of the tracks to express interest in the activities of the other tracks.
93
Cross-track sessions were incorporated to emphasize that the 3 elements (clinical teaching, educational scholarship, and leadership) were all vital, interrelated components of successful programs. The curriculum for the Train the Trainer sessions was initially developed in accordance with the anticipated needs of the participants, based upon review of the literature, consensus of program leaders, and results of an email survey sent to participants before the first meeting. Participants were requested to provide personal introductions to the group, as well as responses to the following questions: 1) What strengths and faculty development experience do you bring to the program? 2) What, specifically, do you expect/hope to achieve from the program? 3) In which areas might you be willing to contribute (eg, ‘‘I teach a session on ‘Creating a Learning Environment’ that practitioners like and have told me is useful. I would be happy to demonstrate it.’’; ‘‘I have a method for teaching . . . which I think works well. . . . ’’)? The initial session focused on teaching knowledge, skills, and attitudes needed to develop and maintain successful community-based education programs and to plan and conduct faculty development workshops. During the course of the first and all subsequent meetings, verbal surveys of participants were utilized to identify newly perceived needs or shifting priorities on the basis of previous experience in the program, as well as the needs and priorities identified through participants conducting their own workshops between program meetings. Additionally, needs assessment surveys were conducted by e-mail between meetings. For example, the following survey was sent requesting the participants to respond to the entire group: 1) What do you need to recruit, develop, and retain community faculty? Which of these should we spend time on? 2) What do those faculty need from you? What should we spend time on? 3) What content areas should we address? 4) What teaching skills do you want to develop in your community preceptors? Which of these should we spend time on? 5) What teaching skills do you want to develop in yourself to make your faculty development efforts more effective? The responses to this and subsequent needs assessments were used to design the workshops, occasionally with midcourse corrections responding to the feedback. Topics are listed in Table 1. Presentations were provided by recognized national leaders, including those from academic medical center backgrounds as well as experienced community preceptors whose ‘‘legitimacy’’ was recognized from the perspective of community-based practitioners.14 In later sessions, participants were asked to volunteer to lead topic discussions, with the intent of mastering the
94
Roberts & Devries
Table 1. Topics Addressed in the Community-Based Teaching Track ● Developing and maintaining teaching programs in practitioners’ offices ● Adult learning theory and its application ● Creating a constructive learning climate ● Educational planning process (goals, needs assessment, objectives, methods, evaluation) ● Strategies, methods, skills, and instruments to conduct workshops (eg, warm-ups and ‘‘ice-breakers,’’ ‘‘microskills,’’ modeling and ‘‘activated demonstration,’’ role-playing, facilitation of small discussions) ● Learner feedback and evaluation ● Preceptor feedback and evaluation ● Evaluation of workshops ● Technology and various media in medical education ● Distance learning strategies ● Planning and conducting faculty development sessions of various lengths (eg, 1-hour session, full-day workshop) ● Management of ‘‘problem learners’’ ● Advanced faculty development strategies
material, experimenting with various instructional methodologies, and receiving feedback on their teaching skills in a supportive environment. Basic elements of key educational principles (eg, adult learning theory, the educational planning process, and design of constructive learning climates) were addressed. The structure and function of workshops were addressed in the opening plenary session for participants in all 3 tracks to prepare individuals for the sessions in which they would participate during the program and to serve as a template for the sessions they would develop and lead in their regions. In the Community-Based Teaching Track, specific content for faculty development workshops that was familiar to many of the participants was included both to achieve consensus on key issues to be taught and to facilitate discussion on effective teaching methods. (Examples of such content addressed in the 3 workshops include ‘‘efficient teaching,’’ ‘‘feedback,’’ ‘‘evaluation,’’ and ‘‘management of problem learners.’’) Participants were encouraged to consider both the content of sessions and the methods used in teaching. Each Community-Based Teaching Track session closed with reflections about both the content of the session and the methods that had been used to teach the content. The objectives of this reflection activity were to reinforce key messages and to improve teaching techniques by identifying the advantages and disadvantages of various instructional strategies. Initially, one of the track leaders served as the reflector, reporting observations on what he perceived to be key elements of the presentation (content and process). Discussion ensued, reinforcing valuable lessons and generating ideas for improvement. In subsequent sessions, volunteers from the group rotated the ‘‘reflector’’ role responsibility. Through this activity, participants recognized the difficulty of trying simultaneously to present material to a group, gauge audience reaction, and troubleshoot environmental problems (eg, room lighting, changes in room temperature) when they occurred. Recognizing the value of assigning these tasks to an additional person, many of the scholars recruited a colleague to attend their
AMBULATORY PEDIATRICS
faculty development workshops to provide this valuable reflection and feedback. The second and third meetings included scheduled opportunities for sharing and reflecting about the local, regional, and national workshops conducted by participants between meetings. This additional reflection activity encouraged self-evaluation in a supportive, constructive environment. OUTCOMES The first program objective of the Community-Based Teaching Track was to identify individuals with skills in faculty development who would serve as regional leaders to conduct local and regional faculty development workshops. Related participant objectives were to attain the knowledge and skills to plan and conduct faculty development workshops for practitioners on topics related to office-based precepting (eg, principles of adult learning, effective and efficient teaching, evaluation, feedback). Of the 42 participants comprising the 2 cohorts, 40 (95%) completed the program. Participants reported high satisfaction with the individual sessions and actively contributed to both the planning and the delivery of program components. They conducted a total of 307 workshops (an average of 7.4 workshops per participant) attended by 3815 individuals. Participants reported that they incorporated into their workshops knowledge and skills gained in the APA/ HRSA National Faculty Development Scholars Program in the following areas: 1) Planning workshops (eg, development, time management, organization); 2) Teaching skills (eg, needs assessment, interactivity, facilitation skills, use of media, distilling key points); 3) Specific content (eg, problem learner, feedback, adult learning theory, ‘‘microskills’’); and 4) Evaluation tools and techniques. Many of the scholars gave presentations at track-specific sessions and cross-track sessions, permitting direct observations of their skills. Evaluation ratings by track leaders and program participants were high. Evaluation ratings by attendees of scholars’ local, regional, and national workshops were also high. Finally, participants reported their self-perception of increased competence by the end of the program in 12 areas related to workshop development, community-based teaching, and scholarship (Table 2). The second program objective of this track was to create a national network of these leaders, resulting in an increased national presence of faculty development and sustained efforts beyond the period of HRSA funding. The corresponding participant objective was for participants to form local and national collegial networks. Of the 40 who completed the program, 33 (83%) conducted 13 national workshops at meetings of the Association of American Medical Colleges, the Council on Medical Student Education in Pediatrics, and the Pediatric Academic Societies (PAS). More than 40% (17 of 40) were promoted, either
AMBULATORY PEDIATRICS
National Faculty Development Scholars Program: Community-Based Teaching
95
Table 2. Retrospective Pre- and Post-assessment of Perceived Competencies of Community-Based Teaching Track Participants (N 5 39)† Competencies‡
Pre-Program
Post-Program
Change
Set up your workshop(s): logistics, PR, budget Identify and access additional curriculum materials/resources for your workshops (eg, COMSEP) Conduct your workshop Exhibit confidence as a faculty developer during your workshop Utilize curriculum training materials in your workshop Refer your workshop attendees to other materials/resources Retain key faculty and personnel Use technology in your teaching Teach your workshop with an emphasis on the following concepts essential to primary care general pediatrics: ambulatory care, continuity of care, EBM, prevention, psychosocial aspects, health care delivery systems, population-based medicine, cost-of case analyses, utilization of interdisciplinary health care teams, utilization of innovative technology Develop criteria to judge your workshop success Use the criteria to judge your workshop success Enhance your own scholarly career in pediatrics
3.1 3.0
4.0 4.0
0.9* 1.0*
3.1 3.1 3.1 2.7 3.1 3.1 2.9
4.0 4.0 4.1 3.9 3.6 3.5 3.4
0.9* 0.9* 1.1* 1.2* 0.5* 0.4* 0.5*
2.8 2.9 2.9
3.6 3.7 3.7
0.8* 0.8* 0.8*
†Scale: 1 5 unsatisfactory; 3 5 good; 5 5 excellent. ‡PR indicates public relations; COMSEP, Council on Medical Student Education in Pediatrics; and EBM, evidence-based medicine. *P , .001.
in academic rank or in their role in education (eg, Vice Chair for Pediatric Education). The participants were active in forming a Faculty Development Special Interest Group within the Ambulatory Pediatric Association, which has met at each of the past 3 annual PAS meetings and has developed a curriculum strategy recently approved by the APA Board of Directors to sustain faculty development efforts at the national level. Networking occurred among program participants at both national and local levels. Members of the Community-Based Teaching Track reported the benefit of the reflection exercises and sought to recreate the academically invigorating atmosphere in their own institutions by networking with individuals in other departments and disciplines. From the beginning to the end of the program, they reported an increase in 5 of 6 key items in the Colleagues category of the 4/20 Habits of Academic Success Inventory15 (Table 3). Of note is that the item demonstrating the greatest percent increase was ‘‘willingness to borrow resources (eg, lecture notes and slides) from colleagues’’—a relatively straightforward gesture of cooperation that might have been presumed beforehand. This may serve to demonstrate that collegiality cannot be as-
sumed and that success in establishing networking and cooperation is a significant achievement. The third program objective was to improve and increase the contributions of office-based practitioner-preceptors to pediatric medical education. The corresponding participant objectives were to possess the knowledge, skills, and attitudes needed to recruit and retain community-based preceptors and to conduct faculty development workshops for practitioner-preceptors. Of 32 participants who responded to a question regarding organizational outcomes of the APA/HRSA National Faculty Development Scholars Program, 21 (66%) reported success in recruiting more community sites for learners, enhanced relations with community preceptors, and increased formal acknowledgment of the contributions of community preceptors (through awards, access to university resources, etc) at their institutions. One participant in the CommunityBased Teaching Track now leads the Resident Education and Training Special Interest Group of the American Academy of Pediatrics’ Section on Community Pediatrics, which is developing a ‘‘Starter Kit’’ to assist practitioners interested in precepting trainees in their offices.
Table 3. Habits of Academic Success15 Identified by Participants of the Community-Based Teaching Track (N 5 22)† Colleagues
Pre-Program
Post-Program
Change
Associate/collaborate with distinguished colleagues in your field Have a colleague network which includes senior colleagues, peers, administrators, and staff Collaborate with colleagues on writing, teaching, research, and administrative tasks Regularly obtain guidance and feedback from a senior colleague(s) Maintain regular contact with professional colleagues outside your institution Borrow resources from colleagues pertinent to a new assignment (eg, lecture notes and slides)
2.2
3.3
1.1*
2.2
3.1
0.9*
2.4 2.5 1.8
3.1 2.7 2.9
0.7* 0.2 1.1*
1.7
3.0
1.3*
†Scale: 1 5 not true; 4 5 very true. *P , .001.
96
Roberts & Devries
DISCUSSION The office practice of pediatrics is fast paced with a need for efficiency. Teaching in such settings is challenging,16 because in competition for time and attention, the primary focus is clearly on patient care rather than teaching. Practitioner-preceptors generally are volunteers who have limited experience with instructing trainees. The primary reward for such volunteers is satisfaction and stimulation, which learners do not always provide. Faculty development workshops can enhance this satisfaction and stimulation by improving teaching skills, clarifying curriculum expectations, and creating a supportive cohort of interested practitioner-preceptor colleagues.12,17 HRSA has provided funding for faculty development programs under Title VII for more than 2 decades. These grants permitted individual institutions to develop enhanced capabilities and, in some instances, national reputations. However, there were no organized systems to disseminate achievements and lessons learned either regionally or nationally. The APA/HRSA National Faculty Development Scholars Program sought to create such a network and succeeded. Scholars in the Community-Based Teaching Track have conducted workshops locally, regionally, and nationally. They have collaborated within the track to conduct national workshops and across tracks for both regional and national workshops. The scholars have recognized the value of assembling a national cohort for collegial support, intellectual stimulation, and collaboration. During the initial formation of this program, it was thought that participants would need to be somewhat senior faculty members with several years of experience conducting faculty development workshops in order to possess the minimum skills needed to advance to a position of regional leadership in this field. However, the majority of applicants lacked such extensive experience. The reason was not studied, but a likely explanation was the paucity of experienced leaders in this field—a fact recognized by the program’s leaders and affirming the need for such a program. Despite many participants’ lack of previous experience, the outcomes were impressive. The evaluation component consisted of multiple measures, including full program participation; the number of institutional, regional, and national workshops conducted between program meetings; observations of participants’ presentations at program sessions; and participants’ perceptions of having improved knowledge, skills, and comfort as a result of their involvement. Their own perceptions of achievement were based partially on their interpretation of the written and verbal feedback they received in response to the workshops that they conducted between program meetings. For some, even the fact that they initiated and conducted a workshop on this topic for the first time confirmed their enhanced confidence. Nevertheless, the evaluation process did not include many objective measures of actual acquisition of knowledge or skills. This competence should not be assumed, especially inasmuch as these leaders will likely become the teachers of subsequent workshop organizers and must possess solid
AMBULATORY PEDIATRICS
skills in order to pass these on to more junior colleagues. It will be important for further studies to evaluate the competence of these graduates—perhaps utilizing an evaluation instrument on paper or by computer but, more importantly, within the actual workshop setting. The benefits for the participants and for their individual programs have already been quantified. The long-term benefits to be derived through these scholars recruiting and training their colleagues—thereby creating an increasing network of faculty development leaders—are just beginning to be felt. The test will be whether efforts will be sustained beyond the period of HRSA funding and whether necessary institutional and external funding can be obtained. Participants reported increased commitment from their department chairs for resources to support faculty development and increased collaboration across departments and institutions. The formation and vitality of the new APA Faculty Development Special Interest Group and the creation of a Starter Kit to be disseminated by the American Academy of Pediatrics are also encouraging. An important longer term measure of success is whether the increasing number and quality of community-based teaching programs are sustained and spread to include additional institutions and whether, in the process, students, residents, practitioners, and academic institutions continue to benefit. Longitudinal follow-up studies to determine the extent of sustainability of this activity, and its dissemination throughout regions, will be necessary to assess the true long-term impact of this initiative. Finally, the most important outcome—albeit the most difficult to measure— is whether children become the ultimate beneficiaries of these educational activities. CONCLUSION The 3 program objectives of the Community-Based Teaching Track were achieved: 1) Individuals with core skills in faculty development were identified and have served as regional leaders, providing local and regional faculty development. 2) These individuals formed a national network, resulting in an increased national presence of faculty development as reflected in the APA Special Interest Group and workshops at national meetings. 3) Participants report improved and increased participation of office-based practitioner-preceptors in pediatric medical education and have undertaken efforts to facilitate increased participation. The 4 participant objectives were also achieved by the conclusion of the program. The participants 1) met the expectations of the program (eg, attended all sessions, conducted workshops); 2) possessed knowledge, skills, and attitudes to recruit and retain community-based preceptors; 3) possessed the knowledge and skills needed to plan and conduct faculty development workshops for practitioners on basic topics related to office-based precepting (eg, principles of adult learning, effective and efficient teaching, evaluation, feedback); and 4) formed local and national collegial networks expected to enhance the sustainability of this initiative.
AMBULATORY PEDIATRICS
National Faculty Development Scholars Program: Community-Based Teaching
ACKNOWLEDGMENTS The authors gratefully acknowledge the energy, active participation, productivity, and contributions of the National Faculty Development Scholars in the Community-Based Teaching Track. This work was supported by US Department of Health Resources and Services Administration (HRSA 240-97-0043) ‘‘Faculty Development of General Pediatrics Generalist Faculty Teaching in Community-Based Ambulatory Settings.’’
REFERENCES 1. Association of American Medical Colleges. Physicians for the twenty-first century: report of the project panel on the general professional education of the physician and college preparation for medicine. J Med Educ. 1984;59(suppl):1–200. 2. Council on Graduate Medical Education Eighth Report. Patient Care Physician Supply and Requirements: Testing the COGME Assumptions. Washington, DC: US Dept of Health and Human Services, Public Health Service, Health Resources and Services Administration, Bureau of Health Professions, Division of Medicine and Dentistry; 1996. Publication HRSA-P-DM 98–3. 3. Sargent JR, Osborn LM, Roberts KB, DeWitt TG. Establishment of primary care continuity experiences in community pediatricians’ offices: nuts and bolts. Pediatrics. 1993;91:1185– 1189. 4. Sargent JR, Osborn LM. Resident training in community pediatricians’ offices: not a financial drain. Am J Dis Child. 1990; 144:1356–1359. 5. Osborn LM, Sargent JR, Williams SD. Effects of time-in-clinic, clinic setting, and faculty supervision on the continuity clinic experience. Pediatrics. 1993;91:1089–1093. 6. Roberts KB, Starr S, DeWitt TG. The University of Massachusetts Medical Center office-based continuity experience: are we
7.
8.
9.
10.
11. 12. 13.
14. 15. 16.
17.
97
preparing pediatrics residents for primary care practice? Pediatrics. 1997;100:e2. Roberts KB, Starr S, DeWitt TG. Resident preparedness for practice: a longitudinal cohort study. Ambulatory Pediatr. 2002; 2:132–135. Scheiner AP. Guidelines for medical student education in community-based pediatric offices. American Academy of Pediatrics Council on Pediatric Education Subcommittee on Medical Student Curriculum. Pediatrics. 1994;93:956–959. Greenberg LW, Getson P, Brasseux C, et al. How are pediatric training programs preparing residents for practice? Am J Dis Child. 1991;145:1389–1392. Rice TD, Holmes SE, Drutz JE. Comparison of continuity clinic experience by practice setting and postgraduate level. Arch Pediatr Adolesc Med. 1996;150:1299–1304. Roberts KB, DeWitt T, eds. Pediatric resident education in community settings. Pediatrics. 1996;98:1249–1301. DeWitt TG. Faculty development for community practitioners. Pediatrics. 1996;98:1273–1276. Osborn LM, Roberts KB, Greenberg L, et al. The APA/HRSA Faculty Development Scholars Program: introduction to the supplement. Ambul Pediatr. 2004;4(suppl 1):83–87. Young LM. The perspective of the community pediatrician. Pediatrics. 1996;98:1255–1258. Simpson DE. 4/20 Habits of Academic Success Inventory. Milwaukee, Wis: Medical College of Wisconsin; 2002. Roberts KB, DeWitt TG. Faculty development of pediatric practitioners: complexities in teaching clinical precepting. Pediatrics. 1996;97:389–393. DeWitt TG, Goldberg RL, Roberts KB. Developing community faculty: principles, practice, and evaluation. AJDC. 1993;147: 49–53.