- Email: [email protected]
Evaluation of an online faculty appraisal instrument: Comparison of resident and faculty perceptions1
Radiology Resident Education
Evaluation of an Online Faculty Appraisal Instrument: Comparison of Resident and Faculty Perceptions1 Phillip M. Boiselle, MD, Richard Jennette, Kevin Donohoe, MD
Rationale and Objectives. Our purpose was to compare faculty and resident perceptions regarding an online faculty appraisal instrument and the potential impact of the appraisal process. Materials and Methods. Faculty members and residents at an academic medical center diagnostic radiology department were asked to complete anonymous surveys that sought feedback regarding an online faculty appraisal form and process. Questions were asked regarding the relative importance of various faculty performance measures, preferences for narrative versus quantitative components of the instrument, and the likely impact of the evaluation process on future faculty behavior. Results. The survey was completed by 19 (45%) of 42 faculty members and by 16 (80%) of 20 residents who participated in the initial faculty appraisal process. Residents and faculty both assigned generally high rankings to the six measures of faculty performance. The vast majority (86 %) of faculty and residents found the narrative component of the survey (either alone or in combination with quantitative data) most helpful. Based on the appraisal process, 74% of the faculty planned to modify one or two aspects of their behavior, whereas a significant minority (44%) of residents thought that the faculty would not make any changes. Conclusion. Faculty and residents generally agree on a core set of faculty performance measures, and both groups show a preference for an appraisal instrument that incorporates a narrative component. Concerning the perceived impact of the appraisal process, faculty members were slightly more optimistic than residents regarding its ability to serve as an impetus for faculty behavioral changes. Key Words. Radiology faculty evaluation. ©
AUR, 2004
The Accreditation Council for Graduate and Medical Education (ACGME) requires evaluation of faculty by residents on a regular basis (1); however, it does not stipulate which specific instruments or methods are to be employed for this process. Several radiology faculty evaluation instruments have been described in the medical education literature, ranging from a simple, one-page evaluation form to more complicated surveys containing more than 60 items (2– 6). Although these evaluation forms are priAcad Radiol 2004; 11:1071–1077 1 From the Department of Radiology, Beth Israel Deaconess Medical Center (P.M.B., R.J., K.D.) and Harvard Medical School, 330 Brookline Avenue, Boston, MA 02215 (P.M.B., K.D.). Address correspondence to P.M.B.
© AUR, 2004 doi:10.1016/j.acra.2004.03.055
marily quantitative, it has been shown that the incorporation of a qualitative component can result in improvements in faculty teaching performance (4). The general complexity and time-consuming nature of appraisal instruments can serve as a barrier to widespread participation in the appraisal process (3, 6). Notably, however, the use of shorter evaluation forms and the employment of an Internet-based evaluation system have been reported to enhance participation (6). Interestingly, there are few data in the literature regarding faculty and resident perceptions of the faculty appraisal process. After developing and implementing an Internet-based faculty appraisal instrument, we constructed a survey to solicit feedback from our departmental faculty and residents regarding their perceptions of 1071
BOISELLE ET AL
both the appraisal form and process. We were specifically interested in whether residents and faculty would differ with regard to: (1) which faculty performance measures are most important; (2) whether the quantitative or qualitative aspect of the instrument is most helpful; and (3) the likely impact of the evaluation process on faculty behavior and commitment to teaching. In this article, we describe the results of this survey.
MATERIALS AND METHODS Evaluation Form and Process In May 2003, we developed an Internet-based faculty appraisal instrument that was designed to solicit both quantitative and qualitative information from residents regarding faculty teaching performance. We specifically limited the scope of the quantitative aspect of the survey because we were concerned that an exhaustive list of items would be perceived as a barrier to participation in the survey and that it could potentially have a negative impact on the residents’ willingness to provide detailed answers to the qualitative component of the instrument. As shown in Appendix 1, the appraisal instrument is a one-page form comprising quantitative and qualitative components. The quantitative component lists five different measures of faculty performance: (1) quality of teaching in a conference setting; (2) quality of teaching during rotations; (3) availability and punctuality; (4) professionalism and attitude; and (5) effectiveness as a mentor. For each of these measures, residents were asked to grade the faculty member on a 4-point scale (1 ⫽ below expectations, 2 ⫽ meets expectations, 3 ⫽ often exceeds expectations, 4 ⫽ consistently exceeds expectations). The quantitative aspect of the survey concluded by asking the residents to grade the faculty member’s overall evaluation on a 5-point scale (1 ⫽ poor, 2 ⫽ fair, 3 ⫽ satisfactory, 4 ⫽ very good, 5 ⫽ exemplary). The qualitative component of the survey comprised two questions: (1) “What are this faculty member’s major strengths” and (2) “Are there any areas in which this faculty member could work to improve your educational experience?” As recommended by Sledge (7), these questions were specifically phrased in a manner that avoided judgmental words such as “good” and “bad.” Ample space was provided to answer the qualitative questions. Residents were assured that their individual responses would remain anonymous, and they were encouraged to be candid and professional in their narrative comments. Although residents were asked to evaluate all members of the faculty, they were given the option to not evaluate 1072
Academic Radiology, Vol 11, No 9, September 2004
a specific faculty member if they had not worked with that individual during the 12 month period before the evaluation. Residents were asked to complete the faculty surveys during a 2-week period in June 2003, and they were provided access to a computer room with Internet access for this purpose. Additionally, residents had the option of completing the evaluations on a personal computer at home. In August 2003, the faculty members received the results of the survey, accompanied by a letter from the departmental chairman and the residency program directors that explained the survey form and process. Each faculty member received an analysis of his or her quantitative data and an unedited listing of all narrative comments. For the overall faculty evaluation score, each faculty member also received a comparative listing of the mean score for the faculty as a whole. Resident confidentiality was maintained by blinding the faculty to the authors of narrative comments and to the names of residents who participated in their evaluation. Faculty and Resident Survey In October 2003, the faculty members and residents were asked to complete separate online surveys that sought feedback regarding the evaluation form and process. The specific surveys are listed in Appendices 2 and 3. Residents and faculty members were informed that their individual responses were anonymous, and they were encouraged to be candid in their narrative comments. To enhance the response rate, the initial e-mail request to complete the survey was followed by three additional requests during a 2-week period. The resident and faculty surveys asked similar questions about the evaluation form and process. Both surveys began by asking the respondent to rank the six faculty performance factors that were listed on the appraisal instrument on a scale of 1 to 6 (1 ⫽ least important to 6 ⫽ most important). Equivalent rankings were permitted. This question was followed by an opportunity to provide narrative input asking for additional areas of faculty performance to be included on future evaluations, as well as specific changes that should be made to the appraisal form or process. Subsequently, the faculty and residents were both asked which component (quantitative, qualitative, both) of the appraisal form was most helpful. The survey concluded with a series of questions regarding the respondent’s perceptions of the degree to which the appraisal process provided faculty members with insight into their strengths and weaknesses as educators, and the
Academic Radiology, Vol 11, No 9, September 2004 EVALUATION OF AN ONLINE FACULTY APPRAISAL INSTRUMENT
Table 1 Comparison of Faculty and Resident Rankings of Faculty Performance Measures
Performance Measure Teaching in conference Teaching during rotation Availability and punctuality Professionalism and attitude Effectiveness as a mentor Overall evaluation of performance
Faculty Ranking (Median and Range)
Resident Ranking (Median and Range)
P Value
4 (1–6) 5 (1–6)
4 (1–6) 5 (2–6)
0.45 0.56
4 (1–6)
5 (1–6)
0.36
5 (1–6)
4 (1–6)
0.48
4 (1–6)
2 (1–5)
0.06
6 (1–6)
3.5 (1–6)
0.13
perceived impact of the process on future faculty behavioral changes and commitment to resident education. A two-tailed, two-sample equal variance t-test was used to compare the faculty and resident rankings of the six faculty performance measures, with P values ⬍.05 considered statistically significant (8).
RESULTS Twenty (71%) of 28 residents participated in the initial faculty appraisal process of 42 faculty members. With regard to the online survey requesting feedback about the appraisal form and process, it was completed by 19 (45%) of 42 faculty members and by 16 (80%) of 20 residents who completed the initial survey. As shown in Table 1, the residents and faculty members both assigned generally high rankings to the six measures of faculty performance. However, the residents gave a low ranking (median ⫽ 2) to the category of “Effectiveness as a Mentor,” whereas the faculty members gave this category a higher (median ⫽ 4) ranking. This difference approached, but did not meet, statistical significance (P ⫽ .06). Although faculty and resident median rankings were either equivalent or within one rank level of one another for four of the six parameters, they differed by 2 to 2.5 rank levels for the categories of “Effectiveness as a Mentor” and “Overall evaluation of performance.” The latter received a median ranking of 6 by the faculty compared with a ranking of 3.5 by the residents. When asked to list additional areas of faculty performance that should be assessed in the future, a slight majority (51%) of the residents and faculty stated that no
additional measures were necessary. The most commonly listed additional areas of faculty performance related to the willingness of faculty members to share in the workload (n ⫽ 5, four residents and one faculty member) and the degree to which faculty members give feedback to residents about their performance (n ⫽ 2, one resident and one faculty). The most commonly listed changes to the evaluation form were to add a space for narrative comments below each quantitative assessment (n ⫽ 2, both faculty members) and to add a response choice of “not applicable” on the quantitative grading scale (n ⫽2, both residents). The most commonly listed changes to the evaluation process included: greater participation by the residents (n ⫽ 4, all faculty members); completion of the instrument at the end of each rotation rather than yearly (n ⫽ 4, three residents and one faculty member); faster turnaround time for results (n ⫽ 2, both faculty members); and to change the process to one that is not anonymous (n ⫽ 2, both faculty members). As shown in Table 2, 89% of the faculty and 81% of the residents reported that they found either the narrative component alone or a combination of the narrative and quantitative component of the survey most helpful. In contrast, only 5% of faculty members and 19% of residents reported that the quantitative component of the survey alone was most helpful. The faculty members strongly agreed (median rank ⫽ 4 on scale of 1 to 4) that the narrative comments of the residents were generally constructive and professional. The faculty mostly agreed (median rank ⫽ 3) and the residents strongly agreed (median rank ⫽ 4) that the level of detail of the quantitative aspect of the survey was satisfactory. The faculty members and residents both strongly agreed (median ranks ⫽ 4 and 3.5, respectively) that the results of the evaluation would help faculty members to gain greater insight into their relative strengths and weaknesses as educators. Based on the results of the evaluation, 74% of the faculty members planned to make one (58%) or two (16%)
Table 2 Comparison of Faculty and Resident Preferences for Quantitative Versus Qualitative Components of Evaluation
Narrative comments Quantitative data Both Neither
Faculty (n ⫽ 19)
Residents (n ⫽ 16)
Total (n ⫽ 35)
47% (9) 5% (1) 42% (8) 5% (1)
31% (5) 19% (3) 50% (8) 0
40% (14) 11% (4) 46% (16) 3% (1)
1073
BOISELLE ET AL
changes to their teaching or practice. A slight majority (56%) of residents predicted that individual faculty members would likely modify one aspect of their teaching or practice, whereas 44% thought that the faculty would not make any changes. Concerning the impact of the survey on the faculty members’ commitment to resident education, a slight majority (58%) of faculty members stated that it had no impact, whereas 37% stated it had a positive impact, and 5% reported a negative impact. In contrast, a slight majority of residents (56%) predicted that it would have a positive impact. Most (84%) faculty members reported that the evaluation process did not change their opinion of the residents. Those who reported a change in their opinion listed favorable comments such as “. . .the residents have made a strong statement about their commitment to the program and their desire to learn . . . (they) showed a remarkable maturity in their responses.” Finally, regarding the results of the faculty appraisal process, most faculty (63%) stated that their individual results were “as expected,” whereas 32% received a higher than expected outcome and 5% reported a lower than expected outcome. DISCUSSION We performed a survey to solicit feedback from our departmental faculty and residents regarding their perceptions of an Internet-based faculty appraisal form comprised of quantitative and qualitative elements. Our results show that faculty and residents generally agree on the core set of faculty performance measures that are assessed by the instrument, and that both groups show a preference for an instrument that incorporates a narrative component. Concerning the perceived impact of the appraisal process, the faculty members were slightly more optimistic than the residents regarding its ability to serve as an impetus for faculty behavioral changes. For an appraisal instrument to be effective, it seems plausible that it should measure behaviors and qualities of faculty teaching that are considered important by both residents and faculty members. Previous research by Collins et al. (2) provides insight into the spectrum of faculty behaviors that are considered important to residents. These authors used the technique of critical incidence interviewing to develop a quantitative faculty appraisal instrument comprised of 53 faculty behaviors. As noted by the authors, a limitation of the instrument is that the items were derived only from resident input (2). This factor may have contributed to the mixed faculty response to their instrument. Although these authors did not directly 1074
Academic Radiology, Vol 11, No 9, September 2004
compare resident and faculty rankings of specific measures, they provided a summary of faculty concerns regarding specific appraisal items. Many of the items that were described by their faculty members as unnecessary or inappropriate, including items related to faculty expertise and clinical skills, were not included in our appraisal instrument. This factor may have contributed to the generally positive faculty response to our instrument. Despite the relatively brief quantitative component of our instrument, our residents and faculty both agreed that the degree of detail in this component of the survey was sufficient. Notably, there was general agreement regarding the importance of the faculty qualities that we measured, with only one exception. Residents gave a relatively low ranking to the item of “Effectiveness as a mentor.” The reason for this ranking is uncertain, but it is likely that the residents perceive that a faculty member can be an effective teacher to a group of residents without necessarily being effective in a one-to-one relationship as a mentor. Only a minority of respondents listed additional measures of faculty performance that should be added to our instrument. In response to our survey results, we plan to add two additional performance measures (“assists resident with workload” and “provides feedback”) to our instrument and to remove “Effectiveness as a mentor.” Although the relatively few quantitative measures listed in our instrument are a potential limitation, this factor can also be viewed as a relative advantage. As noted by Williamson et al., the complex and time-consuming nature of appraisal instruments is an important barrier to participation in the evaluation process (6). Although we did not design our survey to directly assess the impact of this factor, it seems likely that the brevity of the quantitative aspect of the form positively influenced the overall resident participation rate and the time that individual residents devoted to the qualitative component of the instrument. Williamson et al. showed an increase in resident response rate from 11% to 85% after shortening their appraisal form to one-page and constructing a web-based design (6). Cohan et al. have shown that an evaluation instrument that solicits specific comments from residents regarding faculty strengths and weaknesses can effect improvements in faculty teaching performance, particularly among those considered to be the weakest teachers (4). In our survey, both residents and faculty found such a narrative component to be a helpful component of the appraisal process. Indeed, the vast majority of respondents expressed a preference for either the narrative component alone or a combination of the narrative and quantitative components of the instrument. One of the requested changes to our in-
Academic Radiology, Vol 11, No 9, September 2004 EVALUATION OF AN ONLINE FACULTY APPRAISAL INSTRUMENT
strument was to add a space for narrative comments below each quantitative assessment. This construct has been shown to be effective by Cohan et al. (4), and we plan to incorporate this change to our instrument. The ultimate goal of a faculty appraisal instrument is to improve faculty teaching performance. Previous studies have shown that student and resident evaluations of faculty members can lead to improvements in the perceived teaching performance of faculty and in the overall quality of a teaching program (4, 8 –12). On the other hand, if the residents doubt that their input will produce constructive change in faculty performance, they may be less likely to participate in the evaluation process (6). We were pleased to find that 74% of our faculty planned to modify one or two aspects of their behavior based on the results of the appraisal process. However, it was somewhat concerning that a significant minority (44%) of residents thought that the faculty would not make any changes. We plan to follow-up with individual faculty members to determine the actual number and type of changes that are made. If the faculty members are successful at implementing behavioral changes, sharing this information with residents has the potential to enhance their experience of (and perhaps future participation in) the evaluation process. Several limitations of our survey should be mentioned. First, it should be noted that our survey was performed at a single institution. We plan to enlist other sites to use our appraisal instrument and to subsequently collect survey data to determine whether our findings are reflective of residents and faculty in diagnostic radiology in general. Second, our survey was designed to assess initial perceptions of an appraisal instrument form and process rather than to measure specific outcomes such as teaching performance. We plan to collect longitudinal data regarding the impact of the appraisal process upon future teaching performance and behaviors. A third limitation, which is shared by most physician survey studies, is that our survey was completed by slightly less than half of the faculty members to whom it was disseminated. However, our faculty response rate of 45% is greater than the reported response rates for other surveys of physicians published in the radiology literature (13, 14). Finally, we did not directly compare our instrument with others that have been previously reported in the medical literature. Future randomized, prospective studies would be helpful to quantify the differences in response rate, outcomes, and perceptions of various instruments and differing methods of dissemination (paper-based versus web-based). In summary, our results show that faculty and residents generally agree on a core set of faculty performance mea-
sures, and that both groups show a preference for an appraisal instrument that incorporates a narrative component. Concerning the perceived impact of the appraisal process, our faculty members were slightly more optimistic than the residents regarding its ability to serve as an impetus for faculty behavioral changes. ACKNOWLEDGMENT
We thank Ellen C. Boiselle for her thoughtful review of this manuscript, Larry Barbaras for computer assistance, and J. Anthony Parker, MD, for assistance with statistical analysis. References 1. Collins J. Evaluation of residents, faculty, and program. Acad Radiol 2003; 10(Suppl 1):S35–S43. 2. Collins J, Albanese M, Thakor SK, et al. Development of a radiology faculty appraisal instrument by using critical incident interviewing. Acad Radiol 1997; 4:795– 801. 3. Collins J, Albanese MA, Scanlan KA, et al. Bi-institutional implementation of an experimental appraisal instrument. Acad Radiol 1998; 5:804 – 809. 4. Cohan RH, Dunnick NR, Blane CE, et al. Improvement of faculty teaching performance: Efficacy of resident evaluations. Acad Radiol 1996; 3:63– 67. 5. Love MB, Revesz G, Degnan JW. Teaching effectiveness in the radiology residency: A method of evaluation by residents. Invest Radiol 1990; 25:287–291. 6. Williamson KB, Jackson VP, Shuman LA, et al. Online evaluation in radiology residency programs. Acad Radiol 2003; 10:83– 86. 7. Sledge WH. Resource identification: a use of psychiatric residents’ evaluation of faculty. J Med Educ 1978; 53:149 –151. 8. Dawson-Saunders B, Trapp RG. Basic and clinical biostatistics. East Norwalk, CT: Appleton and Lange; 1994. 9. Kelly J, Woiwode D. Faculty evaluation by residents in a family medicine residency program. J Fam Practice 1977; 4:693– 695. 10. Ramsbottom-Lucier MT, Gillmore GM, Irby DM, et al. Evaluation of clinical teaching by general internal medicine faculty in outpatient and inpatient settings. Acad Med 1994; 69:152–154. 11. Sall S, Gromisch DS, Rubin SH, et al. Improvement of faculty teaching performance in a department of obstetrics and gynecology by student evaluation. Am J Obstet Gynecol 1976; 124:217–221. 12. Whitman N, Schwenk T. Faculty evaluation as a means of faculty development. J Fam Practice 1982; 14:1097–1101. 13. Munden RF, Hess KR. “Ditzels” on chest CT: Survey of members of the Society of Thoracic Radiology. AJR Am J Roentgenol 2001; 176: 1363–1369. 14. Boiselle PM, Reddy SS, Villas PA, et al. Pulmonary embolus in pregnant patients: Survey of ventilation-perfusion imaging policies and practices. Radiology 1998; 207:201–206.
APPENDIX 1 FACULTY EVALUATION FORM Faculty Name: ________________________________ The residency program directors and other faculty members value your input about their performance. Please be candid and professional in your comments. Your response will be anonymous. 1075
BOISELLE ET AL
If you do not feel that you have worked closely enough with this faculty member in the last 12 months to be able to complete this evaluation form, please check here ____ and do not complete the evaluation. Otherwise, please complete the following evaluation: 1. Please grade each of the following measures of faculty performance using the following scale (1 Falls below expectations, 2 Meets expectations, 3 Often exceeds expectations, and 4 Consistently exceeds expectations): a. Quality of Teaching in Conference Setting b. Quality of Teaching During Rotations c. Availability and Puctuality d. Professionalism and Attitude e. Effectiveness as Mentor (Clinical and/or Research) 2. Please grade this faculty member’s overall performance using the following scale (1 Poor, 2 Fair, 3 Satisfactory, 4 Very Good, 5 Exemplary) Narrative: What are this faculty member’s major strengths?1 Narrative: Are there any areas in which this faculty member could work to improve your educational experience?1
APPENDIX 2 FACULTY SURVEY OF APPRAISAL INSTRUMENT AND PROCESS 1. The six factors on which faculty members were rated by the residents are listed below. Please rank these factors in the order of increasing importance to you as a faculty member (1 ⫽ least important; 6 ⫽ most important): – Quality of teaching in a conference setting – Quality of teaching during rotations – Availability and punctuality – Professionalism and attitude – Effectiveness as a mentor – Overall evaluation of faculty performance 2. Are there any additional areas of faculty performance for which you would like to be evaluated in the future? Please list: _______________________ 1The reader should note here that a greater amount of space was provided for narrative comments on the internet-based form.
1076
Academic Radiology, Vol 11, No 9, September 2004
3. If you could make one change to the evaluation form (eg, the types of questions that are asked), what would it be? 4. If you could make one change to the evaluation process (eg, the frequency of the evaluation, the method of distribution of results, the degree of participation by the residents), what would it be? 5. Which ONE of the following aspects of the evaluation did you find MOST helpful? (Choose one): – – – –
Quantitative data Narrative comments of residents Both were equally helpful Neither was helpful
Please select a response from 1 to 4 regarding whether you agree or disagree with the following 3 statements: (1 ⫽ strongly disagree; 2 ⫽ slightly disagree; 3 ⫽ mostly agree; 4 ⫽ strongly agree) 6. The narrative comments of the residents were generally constructive and professional. 7. The level of detail of the quantitative aspect of the survey was satisfactory. 8. The results of the evaluation helped me to gain greater insight into my relative strengths and weaknesses as an educator. 9. Based on the results of the evaluation, do you plan to make one or more changes in your teaching or practice (eg, “I will work to improve my teaching in a rotation setting” or “I will try to be more available and accessible”)? (Choose one): 1 –I do not plan to make any changes to my teaching or practice. 2 –I plan to modify one aspect of my teaching or practice. 3 –I plan to modify two aspects of my teaching or practice. 4 –I plan to modify more than two aspects of my teaching or practice. 10. Have the results of the survey had an impact on your commitment to resident education? – Negative impact. I am less committed to resident education. – No impact. My level of commitment to resident education has not changed. – Positive impact. My level of commitment to resident education has increased.
Academic Radiology, Vol 11, No 9, September 2004 EVALUATION OF AN ONLINE FACULTY APPRAISAL INSTRUMENT
11. Have the results of the survey changed your opinion of the residents? – Yes (if yes, please list how it has changed) ___________________________ – No 12. Regarding the overall evaluation score of faculty performance that you received, please select one: – It was lower than I expected – It was about what I expected – It was higher than I expected
APPENDIX 3 RESIDENT SURVEY OF APPRAISAL INSTRUMENT AND PROCESS
1. The 6 factors upon which faculty members were rated by the residents are listed below. Please rank these faculty performance factors in the order of increasing importance to you as a resident (1 ⫽ least important; 6 ⫽ most important): – Quality of teaching in a conference setting – Quality of teaching during rotations – Availability and punctuality – Professionalism and attitude – Effectiveness as a mentor – Overall evaluation of faculty performance 2. Are there any additional areas of faculty performance for which you would like to be able to evaluate faculty members in the future? Please list: ________________________ 3. If you could make one change to the evaluation form (eg, the types of questions that are asked), what would it be? 4. If you could make one change to the evaluation process (eg, the frequency of the evaluation, the degree of participation by the residents), what would it be?
5. Which ONE of the following aspects of the evaluation did you believe was MOST helpful? (Choose one): – – – –
Quantitative data Narrative comments Both were equally helpful Neither was helpful
Please select a response from 1 to 4 regarding whether you agree or disagree with the following 3 statements: (1 ⫽ strongly disagree; 2 ⫽ slightly disagree; 3 ⫽ mostly agree; 4 ⫽ strongly agree) 6. The level of detail of the quantitative aspect of the survey was satisfactory. 7. The results of the evaluation will help our faculty members to gain greater insight into their relative strengths and weaknesses as educators. 8. Based on the results of the evaluation, do you feel that faculty members will change one or more aspects of their teaching or practice (eg, “They will work to improve their teaching in a rotation setting” or “They will try to be more available and accessible”)? (Choose one): 1 –Most faculty members will not make any changes to their teaching or practice. 2 –Most faculty members will likely modify one aspect of their teaching or practice. 3 –Most faculty members will likely modify two aspects of their teaching or practice. 4 –Most faculty members will likely modify more than two aspects of their teaching or practice. 9. Do you believe that the faculty evaluation results will have an impact on the commitment of our faculty to resident education? – Negative impact. Faculty members will be less committed to resident education. – No impact. The level of commitment to resident education will not change. – Positive impact. The level of commitment to resident education will increase.
1077
Evaluation of an Online Faculty Appraisal Instrument: Comparison of Resident and Faculty Perceptions1 Phillip M. Boiselle, MD, Richard Jennette, Kevin Donohoe, MD
Rationale and Objectives. Our purpose was to compare faculty and resident perceptions regarding an online faculty appraisal instrument and the potential impact of the appraisal process. Materials and Methods. Faculty members and residents at an academic medical center diagnostic radiology department were asked to complete anonymous surveys that sought feedback regarding an online faculty appraisal form and process. Questions were asked regarding the relative importance of various faculty performance measures, preferences for narrative versus quantitative components of the instrument, and the likely impact of the evaluation process on future faculty behavior. Results. The survey was completed by 19 (45%) of 42 faculty members and by 16 (80%) of 20 residents who participated in the initial faculty appraisal process. Residents and faculty both assigned generally high rankings to the six measures of faculty performance. The vast majority (86 %) of faculty and residents found the narrative component of the survey (either alone or in combination with quantitative data) most helpful. Based on the appraisal process, 74% of the faculty planned to modify one or two aspects of their behavior, whereas a significant minority (44%) of residents thought that the faculty would not make any changes. Conclusion. Faculty and residents generally agree on a core set of faculty performance measures, and both groups show a preference for an appraisal instrument that incorporates a narrative component. Concerning the perceived impact of the appraisal process, faculty members were slightly more optimistic than residents regarding its ability to serve as an impetus for faculty behavioral changes. Key Words. Radiology faculty evaluation. ©
AUR, 2004
The Accreditation Council for Graduate and Medical Education (ACGME) requires evaluation of faculty by residents on a regular basis (1); however, it does not stipulate which specific instruments or methods are to be employed for this process. Several radiology faculty evaluation instruments have been described in the medical education literature, ranging from a simple, one-page evaluation form to more complicated surveys containing more than 60 items (2– 6). Although these evaluation forms are priAcad Radiol 2004; 11:1071–1077 1 From the Department of Radiology, Beth Israel Deaconess Medical Center (P.M.B., R.J., K.D.) and Harvard Medical School, 330 Brookline Avenue, Boston, MA 02215 (P.M.B., K.D.). Address correspondence to P.M.B.
© AUR, 2004 doi:10.1016/j.acra.2004.03.055
marily quantitative, it has been shown that the incorporation of a qualitative component can result in improvements in faculty teaching performance (4). The general complexity and time-consuming nature of appraisal instruments can serve as a barrier to widespread participation in the appraisal process (3, 6). Notably, however, the use of shorter evaluation forms and the employment of an Internet-based evaluation system have been reported to enhance participation (6). Interestingly, there are few data in the literature regarding faculty and resident perceptions of the faculty appraisal process. After developing and implementing an Internet-based faculty appraisal instrument, we constructed a survey to solicit feedback from our departmental faculty and residents regarding their perceptions of 1071
BOISELLE ET AL
both the appraisal form and process. We were specifically interested in whether residents and faculty would differ with regard to: (1) which faculty performance measures are most important; (2) whether the quantitative or qualitative aspect of the instrument is most helpful; and (3) the likely impact of the evaluation process on faculty behavior and commitment to teaching. In this article, we describe the results of this survey.
MATERIALS AND METHODS Evaluation Form and Process In May 2003, we developed an Internet-based faculty appraisal instrument that was designed to solicit both quantitative and qualitative information from residents regarding faculty teaching performance. We specifically limited the scope of the quantitative aspect of the survey because we were concerned that an exhaustive list of items would be perceived as a barrier to participation in the survey and that it could potentially have a negative impact on the residents’ willingness to provide detailed answers to the qualitative component of the instrument. As shown in Appendix 1, the appraisal instrument is a one-page form comprising quantitative and qualitative components. The quantitative component lists five different measures of faculty performance: (1) quality of teaching in a conference setting; (2) quality of teaching during rotations; (3) availability and punctuality; (4) professionalism and attitude; and (5) effectiveness as a mentor. For each of these measures, residents were asked to grade the faculty member on a 4-point scale (1 ⫽ below expectations, 2 ⫽ meets expectations, 3 ⫽ often exceeds expectations, 4 ⫽ consistently exceeds expectations). The quantitative aspect of the survey concluded by asking the residents to grade the faculty member’s overall evaluation on a 5-point scale (1 ⫽ poor, 2 ⫽ fair, 3 ⫽ satisfactory, 4 ⫽ very good, 5 ⫽ exemplary). The qualitative component of the survey comprised two questions: (1) “What are this faculty member’s major strengths” and (2) “Are there any areas in which this faculty member could work to improve your educational experience?” As recommended by Sledge (7), these questions were specifically phrased in a manner that avoided judgmental words such as “good” and “bad.” Ample space was provided to answer the qualitative questions. Residents were assured that their individual responses would remain anonymous, and they were encouraged to be candid and professional in their narrative comments. Although residents were asked to evaluate all members of the faculty, they were given the option to not evaluate 1072
Academic Radiology, Vol 11, No 9, September 2004
a specific faculty member if they had not worked with that individual during the 12 month period before the evaluation. Residents were asked to complete the faculty surveys during a 2-week period in June 2003, and they were provided access to a computer room with Internet access for this purpose. Additionally, residents had the option of completing the evaluations on a personal computer at home. In August 2003, the faculty members received the results of the survey, accompanied by a letter from the departmental chairman and the residency program directors that explained the survey form and process. Each faculty member received an analysis of his or her quantitative data and an unedited listing of all narrative comments. For the overall faculty evaluation score, each faculty member also received a comparative listing of the mean score for the faculty as a whole. Resident confidentiality was maintained by blinding the faculty to the authors of narrative comments and to the names of residents who participated in their evaluation. Faculty and Resident Survey In October 2003, the faculty members and residents were asked to complete separate online surveys that sought feedback regarding the evaluation form and process. The specific surveys are listed in Appendices 2 and 3. Residents and faculty members were informed that their individual responses were anonymous, and they were encouraged to be candid in their narrative comments. To enhance the response rate, the initial e-mail request to complete the survey was followed by three additional requests during a 2-week period. The resident and faculty surveys asked similar questions about the evaluation form and process. Both surveys began by asking the respondent to rank the six faculty performance factors that were listed on the appraisal instrument on a scale of 1 to 6 (1 ⫽ least important to 6 ⫽ most important). Equivalent rankings were permitted. This question was followed by an opportunity to provide narrative input asking for additional areas of faculty performance to be included on future evaluations, as well as specific changes that should be made to the appraisal form or process. Subsequently, the faculty and residents were both asked which component (quantitative, qualitative, both) of the appraisal form was most helpful. The survey concluded with a series of questions regarding the respondent’s perceptions of the degree to which the appraisal process provided faculty members with insight into their strengths and weaknesses as educators, and the
Academic Radiology, Vol 11, No 9, September 2004 EVALUATION OF AN ONLINE FACULTY APPRAISAL INSTRUMENT
Table 1 Comparison of Faculty and Resident Rankings of Faculty Performance Measures
Performance Measure Teaching in conference Teaching during rotation Availability and punctuality Professionalism and attitude Effectiveness as a mentor Overall evaluation of performance
Faculty Ranking (Median and Range)
Resident Ranking (Median and Range)
P Value
4 (1–6) 5 (1–6)
4 (1–6) 5 (2–6)
0.45 0.56
4 (1–6)
5 (1–6)
0.36
5 (1–6)
4 (1–6)
0.48
4 (1–6)
2 (1–5)
0.06
6 (1–6)
3.5 (1–6)
0.13
perceived impact of the process on future faculty behavioral changes and commitment to resident education. A two-tailed, two-sample equal variance t-test was used to compare the faculty and resident rankings of the six faculty performance measures, with P values ⬍.05 considered statistically significant (8).
RESULTS Twenty (71%) of 28 residents participated in the initial faculty appraisal process of 42 faculty members. With regard to the online survey requesting feedback about the appraisal form and process, it was completed by 19 (45%) of 42 faculty members and by 16 (80%) of 20 residents who completed the initial survey. As shown in Table 1, the residents and faculty members both assigned generally high rankings to the six measures of faculty performance. However, the residents gave a low ranking (median ⫽ 2) to the category of “Effectiveness as a Mentor,” whereas the faculty members gave this category a higher (median ⫽ 4) ranking. This difference approached, but did not meet, statistical significance (P ⫽ .06). Although faculty and resident median rankings were either equivalent or within one rank level of one another for four of the six parameters, they differed by 2 to 2.5 rank levels for the categories of “Effectiveness as a Mentor” and “Overall evaluation of performance.” The latter received a median ranking of 6 by the faculty compared with a ranking of 3.5 by the residents. When asked to list additional areas of faculty performance that should be assessed in the future, a slight majority (51%) of the residents and faculty stated that no
additional measures were necessary. The most commonly listed additional areas of faculty performance related to the willingness of faculty members to share in the workload (n ⫽ 5, four residents and one faculty member) and the degree to which faculty members give feedback to residents about their performance (n ⫽ 2, one resident and one faculty). The most commonly listed changes to the evaluation form were to add a space for narrative comments below each quantitative assessment (n ⫽ 2, both faculty members) and to add a response choice of “not applicable” on the quantitative grading scale (n ⫽2, both residents). The most commonly listed changes to the evaluation process included: greater participation by the residents (n ⫽ 4, all faculty members); completion of the instrument at the end of each rotation rather than yearly (n ⫽ 4, three residents and one faculty member); faster turnaround time for results (n ⫽ 2, both faculty members); and to change the process to one that is not anonymous (n ⫽ 2, both faculty members). As shown in Table 2, 89% of the faculty and 81% of the residents reported that they found either the narrative component alone or a combination of the narrative and quantitative component of the survey most helpful. In contrast, only 5% of faculty members and 19% of residents reported that the quantitative component of the survey alone was most helpful. The faculty members strongly agreed (median rank ⫽ 4 on scale of 1 to 4) that the narrative comments of the residents were generally constructive and professional. The faculty mostly agreed (median rank ⫽ 3) and the residents strongly agreed (median rank ⫽ 4) that the level of detail of the quantitative aspect of the survey was satisfactory. The faculty members and residents both strongly agreed (median ranks ⫽ 4 and 3.5, respectively) that the results of the evaluation would help faculty members to gain greater insight into their relative strengths and weaknesses as educators. Based on the results of the evaluation, 74% of the faculty members planned to make one (58%) or two (16%)
Table 2 Comparison of Faculty and Resident Preferences for Quantitative Versus Qualitative Components of Evaluation
Narrative comments Quantitative data Both Neither
Faculty (n ⫽ 19)
Residents (n ⫽ 16)
Total (n ⫽ 35)
47% (9) 5% (1) 42% (8) 5% (1)
31% (5) 19% (3) 50% (8) 0
40% (14) 11% (4) 46% (16) 3% (1)
1073
BOISELLE ET AL
changes to their teaching or practice. A slight majority (56%) of residents predicted that individual faculty members would likely modify one aspect of their teaching or practice, whereas 44% thought that the faculty would not make any changes. Concerning the impact of the survey on the faculty members’ commitment to resident education, a slight majority (58%) of faculty members stated that it had no impact, whereas 37% stated it had a positive impact, and 5% reported a negative impact. In contrast, a slight majority of residents (56%) predicted that it would have a positive impact. Most (84%) faculty members reported that the evaluation process did not change their opinion of the residents. Those who reported a change in their opinion listed favorable comments such as “. . .the residents have made a strong statement about their commitment to the program and their desire to learn . . . (they) showed a remarkable maturity in their responses.” Finally, regarding the results of the faculty appraisal process, most faculty (63%) stated that their individual results were “as expected,” whereas 32% received a higher than expected outcome and 5% reported a lower than expected outcome. DISCUSSION We performed a survey to solicit feedback from our departmental faculty and residents regarding their perceptions of an Internet-based faculty appraisal form comprised of quantitative and qualitative elements. Our results show that faculty and residents generally agree on the core set of faculty performance measures that are assessed by the instrument, and that both groups show a preference for an instrument that incorporates a narrative component. Concerning the perceived impact of the appraisal process, the faculty members were slightly more optimistic than the residents regarding its ability to serve as an impetus for faculty behavioral changes. For an appraisal instrument to be effective, it seems plausible that it should measure behaviors and qualities of faculty teaching that are considered important by both residents and faculty members. Previous research by Collins et al. (2) provides insight into the spectrum of faculty behaviors that are considered important to residents. These authors used the technique of critical incidence interviewing to develop a quantitative faculty appraisal instrument comprised of 53 faculty behaviors. As noted by the authors, a limitation of the instrument is that the items were derived only from resident input (2). This factor may have contributed to the mixed faculty response to their instrument. Although these authors did not directly 1074
Academic Radiology, Vol 11, No 9, September 2004
compare resident and faculty rankings of specific measures, they provided a summary of faculty concerns regarding specific appraisal items. Many of the items that were described by their faculty members as unnecessary or inappropriate, including items related to faculty expertise and clinical skills, were not included in our appraisal instrument. This factor may have contributed to the generally positive faculty response to our instrument. Despite the relatively brief quantitative component of our instrument, our residents and faculty both agreed that the degree of detail in this component of the survey was sufficient. Notably, there was general agreement regarding the importance of the faculty qualities that we measured, with only one exception. Residents gave a relatively low ranking to the item of “Effectiveness as a mentor.” The reason for this ranking is uncertain, but it is likely that the residents perceive that a faculty member can be an effective teacher to a group of residents without necessarily being effective in a one-to-one relationship as a mentor. Only a minority of respondents listed additional measures of faculty performance that should be added to our instrument. In response to our survey results, we plan to add two additional performance measures (“assists resident with workload” and “provides feedback”) to our instrument and to remove “Effectiveness as a mentor.” Although the relatively few quantitative measures listed in our instrument are a potential limitation, this factor can also be viewed as a relative advantage. As noted by Williamson et al., the complex and time-consuming nature of appraisal instruments is an important barrier to participation in the evaluation process (6). Although we did not design our survey to directly assess the impact of this factor, it seems likely that the brevity of the quantitative aspect of the form positively influenced the overall resident participation rate and the time that individual residents devoted to the qualitative component of the instrument. Williamson et al. showed an increase in resident response rate from 11% to 85% after shortening their appraisal form to one-page and constructing a web-based design (6). Cohan et al. have shown that an evaluation instrument that solicits specific comments from residents regarding faculty strengths and weaknesses can effect improvements in faculty teaching performance, particularly among those considered to be the weakest teachers (4). In our survey, both residents and faculty found such a narrative component to be a helpful component of the appraisal process. Indeed, the vast majority of respondents expressed a preference for either the narrative component alone or a combination of the narrative and quantitative components of the instrument. One of the requested changes to our in-
Academic Radiology, Vol 11, No 9, September 2004 EVALUATION OF AN ONLINE FACULTY APPRAISAL INSTRUMENT
strument was to add a space for narrative comments below each quantitative assessment. This construct has been shown to be effective by Cohan et al. (4), and we plan to incorporate this change to our instrument. The ultimate goal of a faculty appraisal instrument is to improve faculty teaching performance. Previous studies have shown that student and resident evaluations of faculty members can lead to improvements in the perceived teaching performance of faculty and in the overall quality of a teaching program (4, 8 –12). On the other hand, if the residents doubt that their input will produce constructive change in faculty performance, they may be less likely to participate in the evaluation process (6). We were pleased to find that 74% of our faculty planned to modify one or two aspects of their behavior based on the results of the appraisal process. However, it was somewhat concerning that a significant minority (44%) of residents thought that the faculty would not make any changes. We plan to follow-up with individual faculty members to determine the actual number and type of changes that are made. If the faculty members are successful at implementing behavioral changes, sharing this information with residents has the potential to enhance their experience of (and perhaps future participation in) the evaluation process. Several limitations of our survey should be mentioned. First, it should be noted that our survey was performed at a single institution. We plan to enlist other sites to use our appraisal instrument and to subsequently collect survey data to determine whether our findings are reflective of residents and faculty in diagnostic radiology in general. Second, our survey was designed to assess initial perceptions of an appraisal instrument form and process rather than to measure specific outcomes such as teaching performance. We plan to collect longitudinal data regarding the impact of the appraisal process upon future teaching performance and behaviors. A third limitation, which is shared by most physician survey studies, is that our survey was completed by slightly less than half of the faculty members to whom it was disseminated. However, our faculty response rate of 45% is greater than the reported response rates for other surveys of physicians published in the radiology literature (13, 14). Finally, we did not directly compare our instrument with others that have been previously reported in the medical literature. Future randomized, prospective studies would be helpful to quantify the differences in response rate, outcomes, and perceptions of various instruments and differing methods of dissemination (paper-based versus web-based). In summary, our results show that faculty and residents generally agree on a core set of faculty performance mea-
sures, and that both groups show a preference for an appraisal instrument that incorporates a narrative component. Concerning the perceived impact of the appraisal process, our faculty members were slightly more optimistic than the residents regarding its ability to serve as an impetus for faculty behavioral changes. ACKNOWLEDGMENT
We thank Ellen C. Boiselle for her thoughtful review of this manuscript, Larry Barbaras for computer assistance, and J. Anthony Parker, MD, for assistance with statistical analysis. References 1. Collins J. Evaluation of residents, faculty, and program. Acad Radiol 2003; 10(Suppl 1):S35–S43. 2. Collins J, Albanese M, Thakor SK, et al. Development of a radiology faculty appraisal instrument by using critical incident interviewing. Acad Radiol 1997; 4:795– 801. 3. Collins J, Albanese MA, Scanlan KA, et al. Bi-institutional implementation of an experimental appraisal instrument. Acad Radiol 1998; 5:804 – 809. 4. Cohan RH, Dunnick NR, Blane CE, et al. Improvement of faculty teaching performance: Efficacy of resident evaluations. Acad Radiol 1996; 3:63– 67. 5. Love MB, Revesz G, Degnan JW. Teaching effectiveness in the radiology residency: A method of evaluation by residents. Invest Radiol 1990; 25:287–291. 6. Williamson KB, Jackson VP, Shuman LA, et al. Online evaluation in radiology residency programs. Acad Radiol 2003; 10:83– 86. 7. Sledge WH. Resource identification: a use of psychiatric residents’ evaluation of faculty. J Med Educ 1978; 53:149 –151. 8. Dawson-Saunders B, Trapp RG. Basic and clinical biostatistics. East Norwalk, CT: Appleton and Lange; 1994. 9. Kelly J, Woiwode D. Faculty evaluation by residents in a family medicine residency program. J Fam Practice 1977; 4:693– 695. 10. Ramsbottom-Lucier MT, Gillmore GM, Irby DM, et al. Evaluation of clinical teaching by general internal medicine faculty in outpatient and inpatient settings. Acad Med 1994; 69:152–154. 11. Sall S, Gromisch DS, Rubin SH, et al. Improvement of faculty teaching performance in a department of obstetrics and gynecology by student evaluation. Am J Obstet Gynecol 1976; 124:217–221. 12. Whitman N, Schwenk T. Faculty evaluation as a means of faculty development. J Fam Practice 1982; 14:1097–1101. 13. Munden RF, Hess KR. “Ditzels” on chest CT: Survey of members of the Society of Thoracic Radiology. AJR Am J Roentgenol 2001; 176: 1363–1369. 14. Boiselle PM, Reddy SS, Villas PA, et al. Pulmonary embolus in pregnant patients: Survey of ventilation-perfusion imaging policies and practices. Radiology 1998; 207:201–206.
APPENDIX 1 FACULTY EVALUATION FORM Faculty Name: ________________________________ The residency program directors and other faculty members value your input about their performance. Please be candid and professional in your comments. Your response will be anonymous. 1075
BOISELLE ET AL
If you do not feel that you have worked closely enough with this faculty member in the last 12 months to be able to complete this evaluation form, please check here ____ and do not complete the evaluation. Otherwise, please complete the following evaluation: 1. Please grade each of the following measures of faculty performance using the following scale (1 Falls below expectations, 2 Meets expectations, 3 Often exceeds expectations, and 4 Consistently exceeds expectations): a. Quality of Teaching in Conference Setting b. Quality of Teaching During Rotations c. Availability and Puctuality d. Professionalism and Attitude e. Effectiveness as Mentor (Clinical and/or Research) 2. Please grade this faculty member’s overall performance using the following scale (1 Poor, 2 Fair, 3 Satisfactory, 4 Very Good, 5 Exemplary) Narrative: What are this faculty member’s major strengths?1 Narrative: Are there any areas in which this faculty member could work to improve your educational experience?1
APPENDIX 2 FACULTY SURVEY OF APPRAISAL INSTRUMENT AND PROCESS 1. The six factors on which faculty members were rated by the residents are listed below. Please rank these factors in the order of increasing importance to you as a faculty member (1 ⫽ least important; 6 ⫽ most important): – Quality of teaching in a conference setting – Quality of teaching during rotations – Availability and punctuality – Professionalism and attitude – Effectiveness as a mentor – Overall evaluation of faculty performance 2. Are there any additional areas of faculty performance for which you would like to be evaluated in the future? Please list: _______________________ 1The reader should note here that a greater amount of space was provided for narrative comments on the internet-based form.
1076
Academic Radiology, Vol 11, No 9, September 2004
3. If you could make one change to the evaluation form (eg, the types of questions that are asked), what would it be? 4. If you could make one change to the evaluation process (eg, the frequency of the evaluation, the method of distribution of results, the degree of participation by the residents), what would it be? 5. Which ONE of the following aspects of the evaluation did you find MOST helpful? (Choose one): – – – –
Quantitative data Narrative comments of residents Both were equally helpful Neither was helpful
Please select a response from 1 to 4 regarding whether you agree or disagree with the following 3 statements: (1 ⫽ strongly disagree; 2 ⫽ slightly disagree; 3 ⫽ mostly agree; 4 ⫽ strongly agree) 6. The narrative comments of the residents were generally constructive and professional. 7. The level of detail of the quantitative aspect of the survey was satisfactory. 8. The results of the evaluation helped me to gain greater insight into my relative strengths and weaknesses as an educator. 9. Based on the results of the evaluation, do you plan to make one or more changes in your teaching or practice (eg, “I will work to improve my teaching in a rotation setting” or “I will try to be more available and accessible”)? (Choose one): 1 –I do not plan to make any changes to my teaching or practice. 2 –I plan to modify one aspect of my teaching or practice. 3 –I plan to modify two aspects of my teaching or practice. 4 –I plan to modify more than two aspects of my teaching or practice. 10. Have the results of the survey had an impact on your commitment to resident education? – Negative impact. I am less committed to resident education. – No impact. My level of commitment to resident education has not changed. – Positive impact. My level of commitment to resident education has increased.
Academic Radiology, Vol 11, No 9, September 2004 EVALUATION OF AN ONLINE FACULTY APPRAISAL INSTRUMENT
11. Have the results of the survey changed your opinion of the residents? – Yes (if yes, please list how it has changed) ___________________________ – No 12. Regarding the overall evaluation score of faculty performance that you received, please select one: – It was lower than I expected – It was about what I expected – It was higher than I expected
APPENDIX 3 RESIDENT SURVEY OF APPRAISAL INSTRUMENT AND PROCESS
1. The 6 factors upon which faculty members were rated by the residents are listed below. Please rank these faculty performance factors in the order of increasing importance to you as a resident (1 ⫽ least important; 6 ⫽ most important): – Quality of teaching in a conference setting – Quality of teaching during rotations – Availability and punctuality – Professionalism and attitude – Effectiveness as a mentor – Overall evaluation of faculty performance 2. Are there any additional areas of faculty performance for which you would like to be able to evaluate faculty members in the future? Please list: ________________________ 3. If you could make one change to the evaluation form (eg, the types of questions that are asked), what would it be? 4. If you could make one change to the evaluation process (eg, the frequency of the evaluation, the degree of participation by the residents), what would it be?
5. Which ONE of the following aspects of the evaluation did you believe was MOST helpful? (Choose one): – – – –
Quantitative data Narrative comments Both were equally helpful Neither was helpful
Please select a response from 1 to 4 regarding whether you agree or disagree with the following 3 statements: (1 ⫽ strongly disagree; 2 ⫽ slightly disagree; 3 ⫽ mostly agree; 4 ⫽ strongly agree) 6. The level of detail of the quantitative aspect of the survey was satisfactory. 7. The results of the evaluation will help our faculty members to gain greater insight into their relative strengths and weaknesses as educators. 8. Based on the results of the evaluation, do you feel that faculty members will change one or more aspects of their teaching or practice (eg, “They will work to improve their teaching in a rotation setting” or “They will try to be more available and accessible”)? (Choose one): 1 –Most faculty members will not make any changes to their teaching or practice. 2 –Most faculty members will likely modify one aspect of their teaching or practice. 3 –Most faculty members will likely modify two aspects of their teaching or practice. 4 –Most faculty members will likely modify more than two aspects of their teaching or practice. 9. Do you believe that the faculty evaluation results will have an impact on the commitment of our faculty to resident education? – Negative impact. Faculty members will be less committed to resident education. – No impact. The level of commitment to resident education will not change. – Positive impact. The level of commitment to resident education will increase.
1077