- Email: [email protected]
Regional teaching improvement programs for community-based teachers
SPECIAL ARTICLE
Regional Teaching Improvement Programs for Community-based Teachers Kelley M. Skeff, MD, PhD, Georgette A. Stratos, PhD, Merlynn R. Bergen, PhD, Kindra Sampson, MA, Susan L. Deutsch, MD PURPOSE: Community-based clinical teachers provide an important cadre of faculty for medical education. This study was designed to examine the feasibility and value of an American College of Physicians–sponsored regional teaching improvement program for community-based teachers. SUBJECTS AND METHODS: We conducted five regional (Connecticut, New Hampshire/Vermont, New York, Ohio, and Virginia) 1- to 2-day teaching-improvement workshops for 282 faculty (49% community based, 51% university based). The workshops were conducted by regional facilitators trained by the Stanford Faculty Development Program using large group and small group instructional methods to teach participants a framework for analyzing teaching, to increase their repertoire of teaching behaviors, to define personal teaching goals, and to identify the educational needs of their teaching site. Participants used Likert ratings [1 (low) to 5 (high) scale] to assess workshop quality, facilitator effectiveness, and rewards for and barriers to teaching in their clinics. Using retrospective
pre- and postintervention ratings, participants also assessed workshop impacts on teacher knowledge, attitudes, and skills. Finally, participants completed open-ended questions to identify recommended changes to improve their clinic as an educational site for students and residents. RESULTS: At all sites, participants evaluated the program as highly useful (4.6 6 0.6, mean 6 SD). Participants’ ratings indicated that the program had a positive effect on their knowledge of teaching principles (4.0 6 0.9), an increase in their teaching ability (P ,0.001), and an increase in their sense of integration with their affiliated institution (P ,0.001). CONCLUSIONS: Regional training of university and community faculty can be an effective way of promoting the improvement of teaching and the collaboration between community-based teachers and academic centers. National physician organizations and regionally based facilitators can provide important resources for the delivery of such training. Am J Med. 1999;106:76 – 80. q1999 by Excerpta Medica, Inc.
M
importance of community-based teaching has prompted foundations and professional organizations to become involved. In 1994 the American College of Physicians (ACP), the largest professional organization representing internists, embarked on a community-based teaching project supported by the Pew Charitable Trusts. This project was intended to enhance the role of the internisteducator. The project’s goals were 1) to improve the education of medical students and internal medicine residents by providing quality training experiences in community-based practices, 2) to strengthen internal medicine by providing educational programs that would produce appropriately trained internists, 3) to improve the attractiveness of internal medicine as a career by providing realistic role models, 4) to involve practitioners in education and to help bridge the gap between academic and private practice, 5) to promote networking between practitioners in academic and private practice, 6) to develop effective resources for office-based teaching to support programs and to aid community-based teachers in program implementation. Community-based practitioners have reported insecurity in their teaching skills (6). To address this concern, a
any medical schools and residency programs are expanding community-based teaching programs. Community-based teachers bring a background and experience relevant to current medical education. Moreover, students and residents have found that, compared to other settings, community-based settings can provide a broader range of patient problems, more supervision from the attending physician, and greater responsibility for patient care (1–3). However, it is difficult to recruit and retain skilled community-based teachers (4). Research investigating the needs of community-based teachers has revealed that they desire such external incentives as clinical titles and continuing medical education credit for their efforts, and they desire to improve their teaching (4,5). In the past, ensuring teaching quality has been the responsibility of medical schools. However, the increasing From the Stanford Faculty Development Program, Palo Alto, California (KMS, GAS, MRB, KS); and Primary Care Education, Cornell University Medical College, New York, New York. Requests for reprints should be addressed to Kelley M. Skeff, MD, PhD, Stanford Faculty Development Program, 1000 Welch Road, Suite 1, Palo Alto, California. Manuscript submitted December 10, 1997, and accepted August 17, 1998. 76
q1999 by Excerpta Medica, Inc. All rights reserved.
0002-9343/99/$–see front matter PII S0002-9343(98)00360-X
Regional Teaching Improvement Programs/Skeff et al
national cadre of physician-facilitators trained by the Stanford Faculty Development Program in Clinical Teaching provided a resource for regional ACP chapters to use in training community-based teachers, using a “mass-training” model at 1- or 2-day workshops (7). These workshops were designed to address communitybased teachers’ needs for faculty development while also providing an opportunity for collegial exchange between the community-based teachers and faculty at their affiliated medical schools.
METHODS Sites and Participants Two planning sessions were held (Connecticut and California) to confirm the utility of the content and approach used by the Stanford program for community-based teaching; ACP governors, clerkship directors, and community-based teachers participated. In response to requests from regional governors, the ACP chose to conduct programs in Connecticut, Ohio, New York, New Hampshire/Vermont, and Virginia. Each statewide program invited community-based and university-based physicians from the surrounding area to participate in the program, although emphasis was placed on communitybased faculty.
Design of the Regional Workshops The regional workshops were conducted by faculty trained at Stanford during a 1-month program for clinical teaching facilitators (8,9). Facilitators are trained to deliver a series of seven 2-hour seminars in a small group instructional format (six to eight participants). This seminar series covers the topics of promotion of a positive learning climate, control of the teaching session, communication of educational goals, promotion of understanding and retention, evaluation of the learners, provision of feedback, and promotion of self-directed learning. The 1to 2-day workshops for community-based physicians took place between November 1995 and June 1996 and focused on only two or three topics: promoting a positive learning climate (five workshop sites), communicating educational goals (two sites), and providing feedback (four sites). These topics were chosen as particularly useful for community-based physicians. Usually one content area was discussed in the morning (eg, learning climate) and one in the afternoon (eg, providing feedback). Each topic was initially presented during a brief lecture. During subsequent small groups, participants reviewed videotaped re-enactments of actual outpatient teaching episodes, participated in a videotaped role play exercise about issues faced in community-based teaching, and reviewed the videotape to discuss the educational effectiveness of the teaching behaviors used and to consider alternative teaching techniques. Participants repeated the role
play for the practice and evaluation of new behaviors. Finally, all participants defined their own personal teaching goals based on their experience in the session and developed recommendations for improving the effectiveness of teaching in their clinic.
Data Collection and Analysis Immediately following each regional session, participants completed a questionnaire about demographic information; ratings of the program and the facilitators; ratings of the workshop’s effects on their knowledge, skills, and attitudes about office-based teaching; and their opinions regarding the rewards for, and barriers to, office-based teaching. The evaluation instrument used five-point Likert scales for rating the workshop and the facilitators (1 5 low; 5 5 high), as well as retrospective before- and afterintervention ratings to assess changes in teacher knowledge, skills, and attitudes. [In previous research, retrospective self-assessment has been shown to be a more valid reflection of actual change than traditional beforeafter ratings (10,11).] The questionnaire also included open-ended comments about the effect of the workshop, including anticipated application of the seminar content to their own teaching, recommendations for improving the clinic environment for teaching, and ways the workshop could be improved. We compared ratings among the five regional programs using analysis of variance. We used paired t tests to compare before and after ratings, unpaired t tests to compare ratings of university- and community-based participants, and repeated measures analysis of variance to compare before and after ratings among the different types (community based, university based) of physicians. Continuous values are presented as mean 6 SD. Statistical significance was set at P , 0.05.
RESULTS The workshops were attended by 282 faculty in the five regions (Connecticut, 30; New York, 107; Ohio, 60 on day 1 and 30 on day 2; Virginia, 61; New Hampshire/ Vermont, 24), with 210 (74%) participants completing the questionnaire. In Ohio, the only site with a 2-day program, several participants did not attend the second day when the questionnaire was administered. Participants’ average age was 42 years (range, 28 to 71 years); 62% were male; 72% were general internists with 10% family physicians, 6% pediatricians, and 12% subspecialists; 51% were university based and 49% community based; and 40% reported previous faculty development program experience. Approximately 85% had prior experience teaching students or residents. Participants reported that 64% of their patients were fee-for-service, and 36% were in capitated insurance programs. January 1999
THE AMERICAN JOURNAL OF MEDICINEt
Volume 106 77
Regional Teaching Improvement Programs/Skeff et al
Table 1. Mean (6 SD) Values of Physicians’ Self-Rated Teaching Skills before and after Training on a 1 (low) to 5 (high) Scale Skill General clinical teaching ability Learning climate Communication of goals Feedback
N
Before After Training Training P Value
182
3.6 6 0.7 4.1 6 0.5 ,0.001
191 4.1 6 0.6 4.5 6 0.4 ,0.001 41* 2.8 6 0.7 4.2 6 0.7 ,0.001 176
3.1 6 0.7 4.3 6 0.6 ,0.001
* This topic was taught at only three of the five sites.
Program Ratings
Participants rated the program as highly useful (4.6 6 0.6 on a five-point scale). This rating significantly exceeded the participants’ ratings of how useful they thought the sessions would be before attending the workshop (3.9 6 0.9; P ,0.001). Participants strongly recommended the experience to colleagues (4.5 6 0.7). They gave high ratings to the large-group presentations (4.4 6 0.6), the small-group discussions (4.6 6 0.5), and the workshops in each topic area (learning climate 4.5 6 0.7, communication of goals 4.5 6 0.7, and feedback 4.5 6 0.6). Ratings indicated that the workshops benefited their knowledge, skills, and attitudes regarding office-based teaching, including knowledge of the principles of effective teaching (4.0 6 0.9) and knowledge of themselves as teachers. There were significant increases in ratings of awareness of teaching strengths (3.5 6 0.9 before and 4.2 6 0.7 after; P ,.001) and awareness of teaching problems (3.1 6 0.8 before and 4.2 6 0.7 after; P ,.001). Those without prior faculty development experience reported significantly more material as new (63%) than those with prior experience (49%; P ,0.001). Participants’ ratings of their abilities before and after the seminar revealed statistically significant changes in behaviors pertaining to fostering a positive learning climate, communicating goals, and providing feedback, as well as general teaching ability (Table 1). The presession ratings of communication of goals and feedback indicated that participants believed that these areas were in greatest need of improvement. Finally, participants indicated that the sessions had a positive effect (4.2 6 0.8) on cooperative interactions with colleagues.
Table 3. Mean (6 SD) Values of Participant Ratings of Rewards for and Barriers to Teaching on a 1 (strongly disagree) to 5 (strongly agree) Scale Measurement Rewards Personal gratification Affiliation with institutions Status Financial Barriers Time Space Financial Lack of confidence in teaching skills Patient resistance Lack of opportunity to teach
N
Rating
175 168 161 169
4.6 6 0.6 3.8 6 1.1 3.2 6 1.1 2.1 6 1.2
178 175 172 175 173 172
4.1 6 1.0 3.2 6 1.3 2.8 6 1.3 2.2 6 1.0 2.1 6 1.0 1.9 6 1.0
Participants rated the effect of the program on their enthusiasm and their willingness to teach as between 4.0 and 4.3 (1 to 5 scale, 1 5 negative, 5 5 positive). Participants’ ratings also indicated a significant increase in their potential as teachers, desire to meet with colleagues regarding education, and sense of integration into their affiliated institution (Table 2).
Participation in Teaching The most commonly reported reasons for physicians to teach are personal gratification and institutional affiliation (Table 3). Faculty who had previously taught in their offices gave high ratings to their enjoyment of teaching (4.3 6 0.9). Several participants mentioned reimbursement, including money or reductions in tuition for courses in continuing medical education as possible rewards. Participants rated time and space as the largest barriers to teaching. Ratings of patient acceptance of office-based teaching were similar and positive among faculty who had taught residents in their offices and those who had not (4.0 6 0.7). However, faculty who had not taught students rated patients’ responses to having students in their offices as lower (3.5 6 1.0) than faculty who had already had experiences (3.9 6 0.7). Faculty made several recommendations to improve the clinic environment for effective teaching, including the
Table 2. Mean (6 SD) Participant Ratings of Attitudes before and after Training on a 1 (low) to 5 (high) Scale Attitude
N
Before Training
After Training
P Value
Potential to contribute as a teacher To students To residents Desire to meet with colleagues to discuss eduational issues Sense of integration into affiliated institutions’ teaching programs
184 180 186 182
3.7 6 0.9 3.7 6 0.8 3.3 6 1.0 3.3 6 1.0
4.3 6 0.7 4.4 6 0.7 4.1 6 0.8 3.9 6 0.9
,0.001 ,0.001 ,0.001 ,0.001
78
January 1999
THE AMERICAN JOURNAL OF MEDICINEt Volume 106
Regional Teaching Improvement Programs/Skeff et al
Table 4. Comparison of Community-Based and University-Based Physicians’ Ratings before and after the Workshop on a 1 (low) to 5 (high) Scale CommunityBased Teachers (N 5 100)
Clinical teaching ability Awareness of clinical teaching strengths Sense of integration into affiliated institution Potential to contribute to residents Anxiety regarding ability to teach residents
UniversityBased Teachers (N 5 103)
Before versus After† (P Value)
Interaction†† (P Value)
Before
After
Before
After
Community versus University* (P Value)
3.4 3.4
4.0 4.1
3.9 3.7
4.2 4.3
,0.001 0.02
,0.001 ,0.001
0.001 0.2
3.0
3.7
3.6
4.0
0.001
,0.001
0.02
3.6
4.3
3.9
4.4
0.09
,0.001
0.3
2.9
2.5
2.6
2.3
0.07
,0.001
0.4
* The P values indicate whether the community-based teachers’ ratings were statistically different from those of the university-based teachers. † The P values indicate whether the teachers’ ratings before the workshop were statistically different from ratings after the workshop. †† The P values indicate whether the before–after changes of the community-based teachers were statistically different from those of the universitybased teachers.
involvement of office staff and patients in teaching, adding chairs in the examination rooms, videotaping training sessions, and using computers in teaching. The comments focused on the need to take more time for effective teaching and highlighted participants’ suggestions to take time early and regularly in the rotation for evaluation, reduce pressure of patient volume when teaching, set aside time at the middle and end of day to review cases, decrease the ratios of teachers to learners, and increase the amount of direct observation. Although the course was organized for communitybased teachers, approximately half of the participants were university based. Faculty from both groups indicated that more than half of the material presented in the seminars was new (community based, 60%; university based, 56%). Ratings from both groups indicated similar achievement of the goals of the sessions and the rewards for teaching, and a similar rank ordering of the barriers to teaching. Also, both groups indicated that a similar proportion of patients had capitated reimbursement in their practices (community based, 36%; university based, 39%). University-based teachers rated space as a greater barrier to teaching than community-based teachers (3.4 6 1.2 vs 3.0 6 1.4; P 5 0.03), whereas community-based teachers rated the impact of the workshop on their philosophy of teaching more highly than university-based teachers (3.9 6 0.9 vs 3.5 6 1.1; P 5 0.01). Communitybased teachers rated their general clinical teaching ability, awareness of their teaching strengths, and sense of integration into their affiliated institution’s teaching programs lower than did university-based teachers (Table 4). There were significant increases from before to after the workshop in all five areas. In two areas, general teaching
ability and sense of integration, community-based teachers reported greater improvement than did universitybased teachers.
DISCUSSION The results of this study are relevant to several areas of medical education, including the usefulness of methods to improve teaching skills, the potential for teaching improvement activities to increase collaboration between community-based teachers and affiliated institutions, the potential for national organizations to contribute to teaching improvement, and the rewards and barriers affecting community-based teachers (12,13). This study indicates that faculty development programs can improve knowledge and skills about office-based teaching and enhance the legitimacy and potential of community-based teachers as clinical instructors. The consistency of our results across regions suggests that both university and community physicians nationwide can benefit. Most participants were already experienced teachers, showing that experience alone does not accomplish the benefits of faculty development programs. Regional training with both large- and small-group educational methods appears to accomplish many goals. Use of trained facilitators to deliver a tested curriculum provided consistent training in several regions. Participants acquired new teaching skills and enhanced enthusiasm for teaching. Indeed, training practitioners to be effective teachers may increase their desire to teach. The program appeared to enhance interactions among institutions and their faculty, indicating a benefit to training community and university faculty together. Participants January 1999
THE AMERICAN JOURNAL OF MEDICINEt
Volume 106 79
Regional Teaching Improvement Programs/Skeff et al
reported a greater sense of partnership with their affiliated institution after the session and an increased desire to discuss education with colleagues. Simultaneous training may have led to increased cross-fertilization of ideas and enhanced the improvement of teaching skills. Thus, the program fostered collegiality among university- and community-based physicians around a common goal, the improvement of teaching. The results also support the need for participation in comprehensive teaching-improvement programs. Although 40% of the faculty had previously participated in faculty development methods, these faculty reported that more than half of the content was new. Moreover, following participation, faculty indicated a desire for ongoing training to cover other educational topics and skills specific to community-based teaching. Training during regional meetings, national meetings, or local programs could be dedicated to this purpose. The success of this program highlights the potential role of national organizations to improve medical education. Alerted by the challenge of attracting students to the field of internal medicine, the ACP developed a program to increase the role of community-based physicians in teaching internal medicine. Assisting them to be more effective teachers was one part of that program. The participants’ ratings and comments about rewards for, and barriers to, teaching were provocative. Although participants’ ratings indicate that institutional recognition for their efforts is important, faculty in general saw teaching as its own reward, with the major incentive being personal gratification. Understandably, the participants identified time as the major barrier to teaching. Learning more about teaching enhanced teachers’ recognition of the time required for excellent teaching and increased their desire to spend that time. This is a substantial challenge, as adequate time for many components of outpatient teaching, such as discussing cases, observing learners, and providing feedback, stands in opposition to clinical productivity requirements that currently are being emphasized. Although the increased application of computer-assisted instruction and self-directed learning methods will provide learning opportunities that do not require the immediate availability of the clinical teacher, time with experienced teachers should always be an essential element of medical education. It is incumbent upon medical institutions to monitor the constraints on teachers’ time, since the ultimate result of reduced time may be less well trained physicians. The evaluation of this teaching improvement ap-
80
January 1999
THE AMERICAN JOURNAL OF MEDICINEt Volume 106
proach for community physicians was based on self-report. Additional measures of faculty’s pre- and postworkshop teaching performance in the clinical setting and the impact on students were not obtained. Despite these limitations, the positive responses of the participants suggest that this program had a meaningful impact. We conclude that teaching improvement programs for community and university physicians should be a continuing goal of local institutions and national organizations. National organizations’ support for faculty development may benefit not only the physician-teachers themselves, but also their institutions and, ultimately, the profession. At a time when physicians are facing new time constraints, enhanced teaching skills may foster physician satisfaction by emphasizing and improving this potentially gratifying portion of their professional lives.
REFERENCES 1. Osborn LM, Sargent JR, Williams SD. Effects of time-in-clinic, clinic settings, and faculty supervision on the continuity clinic experience. Pediatrics. 1993;19:1089 –1093. 2. Packman CH, Krackov SK, Groff GD, Cohen J. The Rochester practice-based experience: an experiment in medical education. Arch Intern Med. 1994;154:1253–1260. 3. Napodano RJ, Schuster BL, Krackov SK, Young LE, Griner MD, Pratt R. Use of private offices in education of residents in internal medicine. Arch Intern Med. 1984;144:303–305. 4. Foley R, Yonke A, Smith J, Roe B, Vance J. Recruiting and retaining volunteer community preceptors. Acad Med. 1996;71:460 – 463. 5. Langlois JP. Support of community preceptors: what do they need? Fam Med. 1995;27:641– 645. 6. Sargent JR, Osborn LM, Roberts KB, DeWitt TG. Establishment of primary care continuity experiences in community pediatricians’ offices: nuts and bolts. Pediatrics. 1993;91:1185–1189. 7. Litzelman DK, Stratos GA, Skeff KM. The effect of a clinical teaching retreat on residents’ teaching skills. Acad Med. 1994;69:433– 434. 8. Skeff KM, Stratos GA, Bergen MR, et al. The Stanford faculty development program: a dissemination approach to faculty development for medical teachers. Teach Learn Med. 1992;4:180 –187. 9. Skeff KM, Stratos GA, Berman J, Bergen M. Improving clinical teaching. Evaluation of a national dissemination program. Arch Intern Med. 1992;152:1156 –1161. 10. Skeff KM, Stratos GA, Bergen MR. Evaluation of a medical faculty development program: a comparison of traditional pre/post and retrospective pre/post self-assessment ratings. Eval Health Prof. 1992;15:351–366. 11. Bray JH, Howard GS. Methodological considerations in the evaluation of a teacher-training program. J Ed Psych. 1980;72:62–70. 12. Glenn JK, Reid JC, Mahaffy J, Schurtleff H. Teaching behaviors in the attending-resident interaction. J Fam Pract. 1984;18:297–304. 13. DeWitt TG, Goldberg RL, Roberts KB. Developing community faculty. Principles, practice, and evaluation. Am J Dis Child. 1993;147: 49 –53.
Regional Teaching Improvement Programs for Community-based Teachers Kelley M. Skeff, MD, PhD, Georgette A. Stratos, PhD, Merlynn R. Bergen, PhD, Kindra Sampson, MA, Susan L. Deutsch, MD PURPOSE: Community-based clinical teachers provide an important cadre of faculty for medical education. This study was designed to examine the feasibility and value of an American College of Physicians–sponsored regional teaching improvement program for community-based teachers. SUBJECTS AND METHODS: We conducted five regional (Connecticut, New Hampshire/Vermont, New York, Ohio, and Virginia) 1- to 2-day teaching-improvement workshops for 282 faculty (49% community based, 51% university based). The workshops were conducted by regional facilitators trained by the Stanford Faculty Development Program using large group and small group instructional methods to teach participants a framework for analyzing teaching, to increase their repertoire of teaching behaviors, to define personal teaching goals, and to identify the educational needs of their teaching site. Participants used Likert ratings [1 (low) to 5 (high) scale] to assess workshop quality, facilitator effectiveness, and rewards for and barriers to teaching in their clinics. Using retrospective
pre- and postintervention ratings, participants also assessed workshop impacts on teacher knowledge, attitudes, and skills. Finally, participants completed open-ended questions to identify recommended changes to improve their clinic as an educational site for students and residents. RESULTS: At all sites, participants evaluated the program as highly useful (4.6 6 0.6, mean 6 SD). Participants’ ratings indicated that the program had a positive effect on their knowledge of teaching principles (4.0 6 0.9), an increase in their teaching ability (P ,0.001), and an increase in their sense of integration with their affiliated institution (P ,0.001). CONCLUSIONS: Regional training of university and community faculty can be an effective way of promoting the improvement of teaching and the collaboration between community-based teachers and academic centers. National physician organizations and regionally based facilitators can provide important resources for the delivery of such training. Am J Med. 1999;106:76 – 80. q1999 by Excerpta Medica, Inc.
M
importance of community-based teaching has prompted foundations and professional organizations to become involved. In 1994 the American College of Physicians (ACP), the largest professional organization representing internists, embarked on a community-based teaching project supported by the Pew Charitable Trusts. This project was intended to enhance the role of the internisteducator. The project’s goals were 1) to improve the education of medical students and internal medicine residents by providing quality training experiences in community-based practices, 2) to strengthen internal medicine by providing educational programs that would produce appropriately trained internists, 3) to improve the attractiveness of internal medicine as a career by providing realistic role models, 4) to involve practitioners in education and to help bridge the gap between academic and private practice, 5) to promote networking between practitioners in academic and private practice, 6) to develop effective resources for office-based teaching to support programs and to aid community-based teachers in program implementation. Community-based practitioners have reported insecurity in their teaching skills (6). To address this concern, a
any medical schools and residency programs are expanding community-based teaching programs. Community-based teachers bring a background and experience relevant to current medical education. Moreover, students and residents have found that, compared to other settings, community-based settings can provide a broader range of patient problems, more supervision from the attending physician, and greater responsibility for patient care (1–3). However, it is difficult to recruit and retain skilled community-based teachers (4). Research investigating the needs of community-based teachers has revealed that they desire such external incentives as clinical titles and continuing medical education credit for their efforts, and they desire to improve their teaching (4,5). In the past, ensuring teaching quality has been the responsibility of medical schools. However, the increasing From the Stanford Faculty Development Program, Palo Alto, California (KMS, GAS, MRB, KS); and Primary Care Education, Cornell University Medical College, New York, New York. Requests for reprints should be addressed to Kelley M. Skeff, MD, PhD, Stanford Faculty Development Program, 1000 Welch Road, Suite 1, Palo Alto, California. Manuscript submitted December 10, 1997, and accepted August 17, 1998. 76
q1999 by Excerpta Medica, Inc. All rights reserved.
0002-9343/99/$–see front matter PII S0002-9343(98)00360-X
Regional Teaching Improvement Programs/Skeff et al
national cadre of physician-facilitators trained by the Stanford Faculty Development Program in Clinical Teaching provided a resource for regional ACP chapters to use in training community-based teachers, using a “mass-training” model at 1- or 2-day workshops (7). These workshops were designed to address communitybased teachers’ needs for faculty development while also providing an opportunity for collegial exchange between the community-based teachers and faculty at their affiliated medical schools.
METHODS Sites and Participants Two planning sessions were held (Connecticut and California) to confirm the utility of the content and approach used by the Stanford program for community-based teaching; ACP governors, clerkship directors, and community-based teachers participated. In response to requests from regional governors, the ACP chose to conduct programs in Connecticut, Ohio, New York, New Hampshire/Vermont, and Virginia. Each statewide program invited community-based and university-based physicians from the surrounding area to participate in the program, although emphasis was placed on communitybased faculty.
Design of the Regional Workshops The regional workshops were conducted by faculty trained at Stanford during a 1-month program for clinical teaching facilitators (8,9). Facilitators are trained to deliver a series of seven 2-hour seminars in a small group instructional format (six to eight participants). This seminar series covers the topics of promotion of a positive learning climate, control of the teaching session, communication of educational goals, promotion of understanding and retention, evaluation of the learners, provision of feedback, and promotion of self-directed learning. The 1to 2-day workshops for community-based physicians took place between November 1995 and June 1996 and focused on only two or three topics: promoting a positive learning climate (five workshop sites), communicating educational goals (two sites), and providing feedback (four sites). These topics were chosen as particularly useful for community-based physicians. Usually one content area was discussed in the morning (eg, learning climate) and one in the afternoon (eg, providing feedback). Each topic was initially presented during a brief lecture. During subsequent small groups, participants reviewed videotaped re-enactments of actual outpatient teaching episodes, participated in a videotaped role play exercise about issues faced in community-based teaching, and reviewed the videotape to discuss the educational effectiveness of the teaching behaviors used and to consider alternative teaching techniques. Participants repeated the role
play for the practice and evaluation of new behaviors. Finally, all participants defined their own personal teaching goals based on their experience in the session and developed recommendations for improving the effectiveness of teaching in their clinic.
Data Collection and Analysis Immediately following each regional session, participants completed a questionnaire about demographic information; ratings of the program and the facilitators; ratings of the workshop’s effects on their knowledge, skills, and attitudes about office-based teaching; and their opinions regarding the rewards for, and barriers to, office-based teaching. The evaluation instrument used five-point Likert scales for rating the workshop and the facilitators (1 5 low; 5 5 high), as well as retrospective before- and afterintervention ratings to assess changes in teacher knowledge, skills, and attitudes. [In previous research, retrospective self-assessment has been shown to be a more valid reflection of actual change than traditional beforeafter ratings (10,11).] The questionnaire also included open-ended comments about the effect of the workshop, including anticipated application of the seminar content to their own teaching, recommendations for improving the clinic environment for teaching, and ways the workshop could be improved. We compared ratings among the five regional programs using analysis of variance. We used paired t tests to compare before and after ratings, unpaired t tests to compare ratings of university- and community-based participants, and repeated measures analysis of variance to compare before and after ratings among the different types (community based, university based) of physicians. Continuous values are presented as mean 6 SD. Statistical significance was set at P , 0.05.
RESULTS The workshops were attended by 282 faculty in the five regions (Connecticut, 30; New York, 107; Ohio, 60 on day 1 and 30 on day 2; Virginia, 61; New Hampshire/ Vermont, 24), with 210 (74%) participants completing the questionnaire. In Ohio, the only site with a 2-day program, several participants did not attend the second day when the questionnaire was administered. Participants’ average age was 42 years (range, 28 to 71 years); 62% were male; 72% were general internists with 10% family physicians, 6% pediatricians, and 12% subspecialists; 51% were university based and 49% community based; and 40% reported previous faculty development program experience. Approximately 85% had prior experience teaching students or residents. Participants reported that 64% of their patients were fee-for-service, and 36% were in capitated insurance programs. January 1999
THE AMERICAN JOURNAL OF MEDICINEt
Volume 106 77
Regional Teaching Improvement Programs/Skeff et al
Table 1. Mean (6 SD) Values of Physicians’ Self-Rated Teaching Skills before and after Training on a 1 (low) to 5 (high) Scale Skill General clinical teaching ability Learning climate Communication of goals Feedback
N
Before After Training Training P Value
182
3.6 6 0.7 4.1 6 0.5 ,0.001
191 4.1 6 0.6 4.5 6 0.4 ,0.001 41* 2.8 6 0.7 4.2 6 0.7 ,0.001 176
3.1 6 0.7 4.3 6 0.6 ,0.001
* This topic was taught at only three of the five sites.
Program Ratings
Participants rated the program as highly useful (4.6 6 0.6 on a five-point scale). This rating significantly exceeded the participants’ ratings of how useful they thought the sessions would be before attending the workshop (3.9 6 0.9; P ,0.001). Participants strongly recommended the experience to colleagues (4.5 6 0.7). They gave high ratings to the large-group presentations (4.4 6 0.6), the small-group discussions (4.6 6 0.5), and the workshops in each topic area (learning climate 4.5 6 0.7, communication of goals 4.5 6 0.7, and feedback 4.5 6 0.6). Ratings indicated that the workshops benefited their knowledge, skills, and attitudes regarding office-based teaching, including knowledge of the principles of effective teaching (4.0 6 0.9) and knowledge of themselves as teachers. There were significant increases in ratings of awareness of teaching strengths (3.5 6 0.9 before and 4.2 6 0.7 after; P ,.001) and awareness of teaching problems (3.1 6 0.8 before and 4.2 6 0.7 after; P ,.001). Those without prior faculty development experience reported significantly more material as new (63%) than those with prior experience (49%; P ,0.001). Participants’ ratings of their abilities before and after the seminar revealed statistically significant changes in behaviors pertaining to fostering a positive learning climate, communicating goals, and providing feedback, as well as general teaching ability (Table 1). The presession ratings of communication of goals and feedback indicated that participants believed that these areas were in greatest need of improvement. Finally, participants indicated that the sessions had a positive effect (4.2 6 0.8) on cooperative interactions with colleagues.
Table 3. Mean (6 SD) Values of Participant Ratings of Rewards for and Barriers to Teaching on a 1 (strongly disagree) to 5 (strongly agree) Scale Measurement Rewards Personal gratification Affiliation with institutions Status Financial Barriers Time Space Financial Lack of confidence in teaching skills Patient resistance Lack of opportunity to teach
N
Rating
175 168 161 169
4.6 6 0.6 3.8 6 1.1 3.2 6 1.1 2.1 6 1.2
178 175 172 175 173 172
4.1 6 1.0 3.2 6 1.3 2.8 6 1.3 2.2 6 1.0 2.1 6 1.0 1.9 6 1.0
Participants rated the effect of the program on their enthusiasm and their willingness to teach as between 4.0 and 4.3 (1 to 5 scale, 1 5 negative, 5 5 positive). Participants’ ratings also indicated a significant increase in their potential as teachers, desire to meet with colleagues regarding education, and sense of integration into their affiliated institution (Table 2).
Participation in Teaching The most commonly reported reasons for physicians to teach are personal gratification and institutional affiliation (Table 3). Faculty who had previously taught in their offices gave high ratings to their enjoyment of teaching (4.3 6 0.9). Several participants mentioned reimbursement, including money or reductions in tuition for courses in continuing medical education as possible rewards. Participants rated time and space as the largest barriers to teaching. Ratings of patient acceptance of office-based teaching were similar and positive among faculty who had taught residents in their offices and those who had not (4.0 6 0.7). However, faculty who had not taught students rated patients’ responses to having students in their offices as lower (3.5 6 1.0) than faculty who had already had experiences (3.9 6 0.7). Faculty made several recommendations to improve the clinic environment for effective teaching, including the
Table 2. Mean (6 SD) Participant Ratings of Attitudes before and after Training on a 1 (low) to 5 (high) Scale Attitude
N
Before Training
After Training
P Value
Potential to contribute as a teacher To students To residents Desire to meet with colleagues to discuss eduational issues Sense of integration into affiliated institutions’ teaching programs
184 180 186 182
3.7 6 0.9 3.7 6 0.8 3.3 6 1.0 3.3 6 1.0
4.3 6 0.7 4.4 6 0.7 4.1 6 0.8 3.9 6 0.9
,0.001 ,0.001 ,0.001 ,0.001
78
January 1999
THE AMERICAN JOURNAL OF MEDICINEt Volume 106
Regional Teaching Improvement Programs/Skeff et al
Table 4. Comparison of Community-Based and University-Based Physicians’ Ratings before and after the Workshop on a 1 (low) to 5 (high) Scale CommunityBased Teachers (N 5 100)
Clinical teaching ability Awareness of clinical teaching strengths Sense of integration into affiliated institution Potential to contribute to residents Anxiety regarding ability to teach residents
UniversityBased Teachers (N 5 103)
Before versus After† (P Value)
Interaction†† (P Value)
Before
After
Before
After
Community versus University* (P Value)
3.4 3.4
4.0 4.1
3.9 3.7
4.2 4.3
,0.001 0.02
,0.001 ,0.001
0.001 0.2
3.0
3.7
3.6
4.0
0.001
,0.001
0.02
3.6
4.3
3.9
4.4
0.09
,0.001
0.3
2.9
2.5
2.6
2.3
0.07
,0.001
0.4
* The P values indicate whether the community-based teachers’ ratings were statistically different from those of the university-based teachers. † The P values indicate whether the teachers’ ratings before the workshop were statistically different from ratings after the workshop. †† The P values indicate whether the before–after changes of the community-based teachers were statistically different from those of the universitybased teachers.
involvement of office staff and patients in teaching, adding chairs in the examination rooms, videotaping training sessions, and using computers in teaching. The comments focused on the need to take more time for effective teaching and highlighted participants’ suggestions to take time early and regularly in the rotation for evaluation, reduce pressure of patient volume when teaching, set aside time at the middle and end of day to review cases, decrease the ratios of teachers to learners, and increase the amount of direct observation. Although the course was organized for communitybased teachers, approximately half of the participants were university based. Faculty from both groups indicated that more than half of the material presented in the seminars was new (community based, 60%; university based, 56%). Ratings from both groups indicated similar achievement of the goals of the sessions and the rewards for teaching, and a similar rank ordering of the barriers to teaching. Also, both groups indicated that a similar proportion of patients had capitated reimbursement in their practices (community based, 36%; university based, 39%). University-based teachers rated space as a greater barrier to teaching than community-based teachers (3.4 6 1.2 vs 3.0 6 1.4; P 5 0.03), whereas community-based teachers rated the impact of the workshop on their philosophy of teaching more highly than university-based teachers (3.9 6 0.9 vs 3.5 6 1.1; P 5 0.01). Communitybased teachers rated their general clinical teaching ability, awareness of their teaching strengths, and sense of integration into their affiliated institution’s teaching programs lower than did university-based teachers (Table 4). There were significant increases from before to after the workshop in all five areas. In two areas, general teaching
ability and sense of integration, community-based teachers reported greater improvement than did universitybased teachers.
DISCUSSION The results of this study are relevant to several areas of medical education, including the usefulness of methods to improve teaching skills, the potential for teaching improvement activities to increase collaboration between community-based teachers and affiliated institutions, the potential for national organizations to contribute to teaching improvement, and the rewards and barriers affecting community-based teachers (12,13). This study indicates that faculty development programs can improve knowledge and skills about office-based teaching and enhance the legitimacy and potential of community-based teachers as clinical instructors. The consistency of our results across regions suggests that both university and community physicians nationwide can benefit. Most participants were already experienced teachers, showing that experience alone does not accomplish the benefits of faculty development programs. Regional training with both large- and small-group educational methods appears to accomplish many goals. Use of trained facilitators to deliver a tested curriculum provided consistent training in several regions. Participants acquired new teaching skills and enhanced enthusiasm for teaching. Indeed, training practitioners to be effective teachers may increase their desire to teach. The program appeared to enhance interactions among institutions and their faculty, indicating a benefit to training community and university faculty together. Participants January 1999
THE AMERICAN JOURNAL OF MEDICINEt
Volume 106 79
Regional Teaching Improvement Programs/Skeff et al
reported a greater sense of partnership with their affiliated institution after the session and an increased desire to discuss education with colleagues. Simultaneous training may have led to increased cross-fertilization of ideas and enhanced the improvement of teaching skills. Thus, the program fostered collegiality among university- and community-based physicians around a common goal, the improvement of teaching. The results also support the need for participation in comprehensive teaching-improvement programs. Although 40% of the faculty had previously participated in faculty development methods, these faculty reported that more than half of the content was new. Moreover, following participation, faculty indicated a desire for ongoing training to cover other educational topics and skills specific to community-based teaching. Training during regional meetings, national meetings, or local programs could be dedicated to this purpose. The success of this program highlights the potential role of national organizations to improve medical education. Alerted by the challenge of attracting students to the field of internal medicine, the ACP developed a program to increase the role of community-based physicians in teaching internal medicine. Assisting them to be more effective teachers was one part of that program. The participants’ ratings and comments about rewards for, and barriers to, teaching were provocative. Although participants’ ratings indicate that institutional recognition for their efforts is important, faculty in general saw teaching as its own reward, with the major incentive being personal gratification. Understandably, the participants identified time as the major barrier to teaching. Learning more about teaching enhanced teachers’ recognition of the time required for excellent teaching and increased their desire to spend that time. This is a substantial challenge, as adequate time for many components of outpatient teaching, such as discussing cases, observing learners, and providing feedback, stands in opposition to clinical productivity requirements that currently are being emphasized. Although the increased application of computer-assisted instruction and self-directed learning methods will provide learning opportunities that do not require the immediate availability of the clinical teacher, time with experienced teachers should always be an essential element of medical education. It is incumbent upon medical institutions to monitor the constraints on teachers’ time, since the ultimate result of reduced time may be less well trained physicians. The evaluation of this teaching improvement ap-
80
January 1999
THE AMERICAN JOURNAL OF MEDICINEt Volume 106
proach for community physicians was based on self-report. Additional measures of faculty’s pre- and postworkshop teaching performance in the clinical setting and the impact on students were not obtained. Despite these limitations, the positive responses of the participants suggest that this program had a meaningful impact. We conclude that teaching improvement programs for community and university physicians should be a continuing goal of local institutions and national organizations. National organizations’ support for faculty development may benefit not only the physician-teachers themselves, but also their institutions and, ultimately, the profession. At a time when physicians are facing new time constraints, enhanced teaching skills may foster physician satisfaction by emphasizing and improving this potentially gratifying portion of their professional lives.
REFERENCES 1. Osborn LM, Sargent JR, Williams SD. Effects of time-in-clinic, clinic settings, and faculty supervision on the continuity clinic experience. Pediatrics. 1993;19:1089 –1093. 2. Packman CH, Krackov SK, Groff GD, Cohen J. The Rochester practice-based experience: an experiment in medical education. Arch Intern Med. 1994;154:1253–1260. 3. Napodano RJ, Schuster BL, Krackov SK, Young LE, Griner MD, Pratt R. Use of private offices in education of residents in internal medicine. Arch Intern Med. 1984;144:303–305. 4. Foley R, Yonke A, Smith J, Roe B, Vance J. Recruiting and retaining volunteer community preceptors. Acad Med. 1996;71:460 – 463. 5. Langlois JP. Support of community preceptors: what do they need? Fam Med. 1995;27:641– 645. 6. Sargent JR, Osborn LM, Roberts KB, DeWitt TG. Establishment of primary care continuity experiences in community pediatricians’ offices: nuts and bolts. Pediatrics. 1993;91:1185–1189. 7. Litzelman DK, Stratos GA, Skeff KM. The effect of a clinical teaching retreat on residents’ teaching skills. Acad Med. 1994;69:433– 434. 8. Skeff KM, Stratos GA, Bergen MR, et al. The Stanford faculty development program: a dissemination approach to faculty development for medical teachers. Teach Learn Med. 1992;4:180 –187. 9. Skeff KM, Stratos GA, Berman J, Bergen M. Improving clinical teaching. Evaluation of a national dissemination program. Arch Intern Med. 1992;152:1156 –1161. 10. Skeff KM, Stratos GA, Bergen MR. Evaluation of a medical faculty development program: a comparison of traditional pre/post and retrospective pre/post self-assessment ratings. Eval Health Prof. 1992;15:351–366. 11. Bray JH, Howard GS. Methodological considerations in the evaluation of a teacher-training program. J Ed Psych. 1980;72:62–70. 12. Glenn JK, Reid JC, Mahaffy J, Schurtleff H. Teaching behaviors in the attending-resident interaction. J Fam Pract. 1984;18:297–304. 13. DeWitt TG, Goldberg RL, Roberts KB. Developing community faculty. Principles, practice, and evaluation. Am J Dis Child. 1993;147: 49 –53.