- Email: [email protected]
Correlates of honor ratings in a clinical clerkship employing a faculty forum evaluation system
Evaluation and Program Planning, Vol. 7, pp. 127-130, 1984
0149-7189/84 $3.00 + .OO Copyright D 1984 Pergamon Press Ltd
Printed in the USA. All rights reserved.
CORRELATES OF HONOR RATINGS IN A CLINICAL EMPLOYING A FACULTY FORUM EVALUATION
CLERKSHIP SYSTEM
DEBRA A. DAROSA and ROLAND FOLSE Department of Surgery, Southern Illinois University School of Medicine
JOHN F. MARKUS Office of Educational Development, University of Texas Medical Branch
ABSTRACT The purpose of this study was to determine whether or not suspected variables affected a surgery clerk’s chances of being awarded an honor rating. Findings indicated a significant relationship between a student receiving an honor rating and his or her preceptor's predetermined level of student advocacy, the number of completed patient interview and physical examination write-ups, and final examination scores. There was no significant relationship found between honor ratings and the preceptor’s status, the sequence in which the student was discussed or the length of time spent discussing the student at the final evaluation meeting, or the number of clinical faculty present at the meeting. These results lend support to the faculty forum evaluation approach, but suggest a need for further scrutiny of some influencing variables to ensure all students are fairly considered and honor ratings judiciously awarded.
There are numerous evaluation methods and models by which faculty can assess student performance. No one system has proven to be more valid than the others. Yet few of the responsibilities of clinical faculty in medical schools pose as many significant intellectual, emotional, ethical, and possibly legal demands as does the process of accurately evaluating the clinical performance of undergraduate medical students and residents (Kapp, 1981). Numerous research efforts have been directed toward the problems associated with objectively evaluating the clinical performance of medical students. Wigton (1980) found that the assessment of clinical competence can be significantly influenced by the personal characteristics of the students. He also noted that faculty do not appear to share common evaluation standards. These findings are supported by other investigators whose research indicated low levels of agreement among faculty evaluating students (O’Donohue & Wergin, 1978), as well as low correlations among different measures of clinical skills (Dowaliby & Andrew, 1976). Printen, Chappell, and Whitney (1973) con-
eluded that the failure of faculty to use the full range of the rating scales thwarted differentiation among levels of student abilities. Another disturbing research finding was that residents and full professor faculty assigned a proportionally greater number of inaccurate grades to students than did junior faculty (Oaks, Scheinok, & Husted, 1969). The principal finding widespread throughout the student evaluation literature was that the evaluation system employed by faculty was one of the major obstacles to providing accurate student assessments. It was, therefore, the purpose of this study to determine if certain variables operant in a group forum evaluation system used in a surgery clerkship biased student assessment outcomes. Southern Illinois University School of Medicine is a community-based school with a mastery-based program. Details of this curriculum have been described elsewhere (Silber, Williams, Paiva, Taylor, & Robinson, 1978). Medical students are assessed on a pass/fail grading system during their basic science years and a
Reprint requests should be sent to Debra DaRosa, PhD, P.O. Box 3926, Southern Illinois University School of Medicine, Springfield, IL 62708. 127
128
DEBRA A. DAROSA, ROLAND FOLSE, and JOHN F. MARKUS
pass/fail/honors system in those clinical clerkships electing to formally recognize outstanding students. One or more honor ratings can be awarded to clerks based on demonstrated exceptional performance in the areas of clinical sophistication, cognitive knowledge, communications skills, personal and professional maturity, and/or technical skills. These honor rating categories have been operationally defined with subheadings of behaviors or activities that correspond to each major category. Although not all clerkships choose to use the honor rating system, the overall student evaluation system is much the same. Following every clerkship, a formal narrative description of each student’s performance is submitted to the dean’s office by the clinical clerkship director. Formal letters of recommendation are drafted based on the culmination of these written evaluations and subsequently forwarded to the various departments where the student is applying for residency. The absence of grades or assigned class ranks can make the earning of honor ratings very important to a student, for it serves to highlight exceptional performance. Hence, it is a critical responsibility of the clinical faculty using the honor system to ensure all students are fairly considered and honor ratings judiciously awarded. The Department of Surgery at Southern Illinois University takes a somewhat traditional approach to evaluating students, using largely subjective and some objective measures. Each student is assigned to a surgical faculty member who serves as his or her preceptor. Preceptors are responsible for reviewing copies of their assigned students’ history and physical examination write-ups, monitoring students’ progress through the lo-week clerkships, counseling, and presenting assessments of their students at the final evaluation meeting. Every faculty member completes a Student Evaluation Form on each student with whom he or she had contact. These forms provide for a scaled assessment of the student’s performance in each of the honor rating categories. The ratings are based on a 4-point scale with
“1” being exceptional and “4” unsatisfactory. Each preceptor receives all the faculty completed Student Evaluation Forms, test scores, and other pertinent information for his or her student for review. All faculty are scheduled to attend the final evaluation meeting. At this faculty forum, each preceptor presents a summary of his or her students’ performances. Nominations by any faculty member can subsequently be made following each preceptor presentation for awarding a student one or more honor ratings in any of the aforementioned five categories. Students can be given an honor rating in one or more of these categories. If faculty consensus is reached, the honor rating or ratings are given. It was hypothesized that certain factors might be influencing the granting of honor ratings in the surgery clerkship. This study was designed to determine if preidentified variables, some latent and some independent, were biasing student chances of receiving honor recognition, upon completing the clerkship. Specifically, the study investigated whether or not there was a relationship between clerks being granted an honor rating and 1. if a. b. c.
2. 3. 4. 5. 6.
their faculty preceptor for the clerkship: was an assistant, associate, or full professor? had a full- or part-time faculty appointment? was considered a strong or weak student advocate as judged by the Department Chairman using a l-5 Likert scale? d. had been with the school for a number of years? the sequence in which the student was discussed at the final evaluation meeting? the number of minutes spent by the faculty discussing the student? the number of physician faculty present at the final student evaluation meeting? students’ scores on their final multiple-choice examinations? the number of patient interview and physical examination write-ups completed during the clerkship?
METHODS Subjects included 51 third-year medical students who had been evaluated during one of three lo-week surgery clerkships and 22 clinical faculty members who served as preceptors during these clerkship rotations. The primary investigator of this study, a full-time nonclinical faculty member, attended the final student evaluation meetings. Notes were taken by her, unbeknownst by the clinical faculty, pertaining to the sequence in which students were discussed, the number of minutes spent discussing each student, and the number of physician faculty present at the meeting. Data related to the preceptors’ academic rank, employment status, and longevity were gathered by
reviewing personnel records. The “student advocacy” rating (Question lc) was based on the department chairman’s perception of each faculty member’s professional area of interest and emphasis. A rating of “1” was given to faculty whose primary accentuation was in research or other scholarly activities and less in undergraduate teaching. A rating of “5” meant the faculty member spent a good deal of time with their preceptee and was known to take a very strong interest in teaching. The chairman was not informed of the use of or intent for this information. Pearson’s product-moment correlation coefficients were calculated to determine if there was a significant
Correlates of Honor Ratings relationship between each variable and whether or not students received at least one honor rating. The Statistical Package for the Social Sciences (SPSS) softRESULTS
TABLE I
Selected
Factors
1. Preceptor characteristics a. assistant, associate, or full professor b. full- or part-time status G. low or high student advocate rating d. longevity with the medical school 2. No. of completed write-ups 3. Composite examination score 4. No. of clinical faculty present at meeting 5. Sequence student was discussed at meeting 6. Length of time spent discussing student
*p < .02 “p < .Ol
ware program was used to complete the analysis (Nie, Hull, Jenkins, Steinbrenner, & Bent, 1975).
AND DISCUSSION
Thirty-one (61%) students evaluated received at least one honor rating. Significant correlations were found between students receiving at least one honor rating and three variables (see Table 1). Faculty rated as high student advocates, those tending to take a strong interest in teaching and students, were more likely to have honor-rated students. This finding raises the question as to how fair it is to randomly assign preceptors to the students for the positive correlation indicates that preceptees of high student advocates are at an advantage over others in terms of gaining honor recognition. This may have resulted because faculty highly interested in teaching may know their students better than research oriented faculty. Each preceptor was aware of all student examination grades prior to the final evaluation meeting. Test score information was provided to all faculty at the meeting who did not serve as preceptors. Data indicated that three honor ratings in the “cognitive knowledge” category were given per cierkship. These three were usually given to the students with the three highest scores on the surgery final written exam. It is apparent that written exam scores were used as the major indicator of students’ knowledge even though a subjective appraisal of knowledge was made by faculty continuously throughout the clerkship. Faculty are with the students daily in the operating room, outpatient clinics, or on rounds where students are probed with questions. The fact that honor ratings were related to exam scores can lead to one of two conclusions. The first conclusion, which is unlikely, is that faculty subjective ratings of students were highly consistent with examination scores. The other conclusion
CORRELATIONS BETWEEN STUDENT HONOR AND SELECTED FACTORS FOR 51 THtRD-YEAR CLERKS AND 22 PRECEPTORS
129
RATINGS SURGERY
f .054 -.143 ,288’ .144 .296’* ,661* * -.041 ,183 -.172
may be that faculty relied strongly on exam scores, and less on their subjective ratings of cognitive knowledge when awarding honor ratings. Information about the total number of histories and physical examinations performed by students is known by each preceptor prior to the meeting. The significant correlation may be due to the fact that the more aggressive students, those actively seeking out an above average number of patients, were more visible on the wards and subsequently recognized by faculty. The quantity of workups certainly does not imply quality, but likely reflects industry, interest, and other characteristics typically held high by clerkship faculty. Data need to be collected and provided to faculty for the final evaluation meeting to ensure that the caliber of the workups are acceptable regardless of the total number completed. Because the evaluation meetings are often lengthy, it was hypothesized that the patience and endurance of the faculty might decrease with time. It was thought that faculty might hesitate to speak out on behalf of a student near the end of the meeting, thereby resulting in shorter discussions per clerk. The findings did not support these assumptions for the length of time as well as the sequence in which students were discussed did not significantly correlate with honor awards. There were insignificant correlations for the number of clinical faculty present during the meeting, university rank of the preceptor, number of years the preceptor was with the school, and whether the preceptor was a full- or part-time faculty member. This indicated that preceptor “status” did not influence the dissemination of honor ratings. The fact that no signi~cant correlations were found related to the chronological order in which students were discussed, the amount of time taken to discuss the student’s performance, or the number of attendings at the final evaluation meeting reflects positively on this forum evaluation approach. The key advantage of this approach is that it provides for oral input of the individuals who monitored and assessed student progress throughout the clerkship. Different faculty members evaluated students in varying settings, performing different skills, and at one or more taxonomy levels. To evaluate clerks’ performance without a forum of people who had student contact could result in making important decisions void of necessary information. Evaluating students and awarding honor ratings based solely on facultycompleted evaluation forms without open verbal exchange could lead to improper conclusions being drawn.
130
DEBRA A. DAROSA, ROLAND FOLSE, and JOHN F. MARKUS
The results of this study showed the forum approach itself does not create bias in the manner in which it is conducted. This is a very common method for assessing clerkship students that has to date not been studied. This research effort provides evidence that this method does not necessarily bias student assessment outcomes. If the variables under study were found to relate to honor recognition, immediate changes would be required to correct the inequities. If biases would have been found, the number of students that would have been effected would have been very high considering the widespread use of the faculty forum system for medical student assessment.
The preceptor advocacy level was found to produce biased evaluative results, which calls for further investigation and subsequent intervention to correct it. The remaining nonperformance variables, however, did not influence honors distribution. This should prove reassuring to faculty using the forum system, as well as the students assessed by it. It is the responsibility of the faculty to ensure the evaluation mechanisms employed provide for accurate assessments so that all students are considered fairly, and that the most deserving consistently receive honor recognition.
REFERENCES DOWALIBY,F. J., & ANDREW,B. J. (1976).Relationships tween
and examination Journal of Medical Education, 51, 181-188. KAPP, clinical
clinical
competence
M. (1981). performance.
Legal
ratings
issues
in faculty
be-
evaluation
of student
Journal of Medical Education, 56, 559-564.
NIE, N. H., HULL, C. H., JENKINS, J. G., STEINBRENNER, K., & BENT, D. H. (1975). Statisticalpackage for the social sciences (2nd ed.) New York: McGraw-Hill. OAKS, W. W., SCHEINOK, P. A., & HUSTED, F. L. (1969). Objective evaluation of a method of assessing student performance in a clinical clerkship. Journal of Medical Education, 44, 207-213. O’DONOHUE,
W. J., & WERGIN,
J. F. (1978).
Evaluation
medical
students
during
a clinical
clerkship
in internal
medicine.
Journal of Medical Education, 53, 55-58.
performance.
of
PRINTEN, K., CHAPPELL, W., & WHITNEY, D. (1973). Clinical performance evaluation of junior medical students. Journal of Medical Education, 48, 343-348. SILBER, D. L., WILLIAMS, R. G., PAIVA, R. E., TAYLOR, D. D., & ROBINSON, R. (1978). The SIU medical curriculum: Systemwide objectives-based instruction. Journal of Medical Educa-
tion, 53, 473-479. WIGTON, R. S. (1980). The effects of student personal characteristics on the evaluation of clinical performance. Journal of
Medical Education, 55, 423-427.
0149-7189/84 $3.00 + .OO Copyright D 1984 Pergamon Press Ltd
Printed in the USA. All rights reserved.
CORRELATES OF HONOR RATINGS IN A CLINICAL EMPLOYING A FACULTY FORUM EVALUATION
CLERKSHIP SYSTEM
DEBRA A. DAROSA and ROLAND FOLSE Department of Surgery, Southern Illinois University School of Medicine
JOHN F. MARKUS Office of Educational Development, University of Texas Medical Branch
ABSTRACT The purpose of this study was to determine whether or not suspected variables affected a surgery clerk’s chances of being awarded an honor rating. Findings indicated a significant relationship between a student receiving an honor rating and his or her preceptor's predetermined level of student advocacy, the number of completed patient interview and physical examination write-ups, and final examination scores. There was no significant relationship found between honor ratings and the preceptor’s status, the sequence in which the student was discussed or the length of time spent discussing the student at the final evaluation meeting, or the number of clinical faculty present at the meeting. These results lend support to the faculty forum evaluation approach, but suggest a need for further scrutiny of some influencing variables to ensure all students are fairly considered and honor ratings judiciously awarded.
There are numerous evaluation methods and models by which faculty can assess student performance. No one system has proven to be more valid than the others. Yet few of the responsibilities of clinical faculty in medical schools pose as many significant intellectual, emotional, ethical, and possibly legal demands as does the process of accurately evaluating the clinical performance of undergraduate medical students and residents (Kapp, 1981). Numerous research efforts have been directed toward the problems associated with objectively evaluating the clinical performance of medical students. Wigton (1980) found that the assessment of clinical competence can be significantly influenced by the personal characteristics of the students. He also noted that faculty do not appear to share common evaluation standards. These findings are supported by other investigators whose research indicated low levels of agreement among faculty evaluating students (O’Donohue & Wergin, 1978), as well as low correlations among different measures of clinical skills (Dowaliby & Andrew, 1976). Printen, Chappell, and Whitney (1973) con-
eluded that the failure of faculty to use the full range of the rating scales thwarted differentiation among levels of student abilities. Another disturbing research finding was that residents and full professor faculty assigned a proportionally greater number of inaccurate grades to students than did junior faculty (Oaks, Scheinok, & Husted, 1969). The principal finding widespread throughout the student evaluation literature was that the evaluation system employed by faculty was one of the major obstacles to providing accurate student assessments. It was, therefore, the purpose of this study to determine if certain variables operant in a group forum evaluation system used in a surgery clerkship biased student assessment outcomes. Southern Illinois University School of Medicine is a community-based school with a mastery-based program. Details of this curriculum have been described elsewhere (Silber, Williams, Paiva, Taylor, & Robinson, 1978). Medical students are assessed on a pass/fail grading system during their basic science years and a
Reprint requests should be sent to Debra DaRosa, PhD, P.O. Box 3926, Southern Illinois University School of Medicine, Springfield, IL 62708. 127
128
DEBRA A. DAROSA, ROLAND FOLSE, and JOHN F. MARKUS
pass/fail/honors system in those clinical clerkships electing to formally recognize outstanding students. One or more honor ratings can be awarded to clerks based on demonstrated exceptional performance in the areas of clinical sophistication, cognitive knowledge, communications skills, personal and professional maturity, and/or technical skills. These honor rating categories have been operationally defined with subheadings of behaviors or activities that correspond to each major category. Although not all clerkships choose to use the honor rating system, the overall student evaluation system is much the same. Following every clerkship, a formal narrative description of each student’s performance is submitted to the dean’s office by the clinical clerkship director. Formal letters of recommendation are drafted based on the culmination of these written evaluations and subsequently forwarded to the various departments where the student is applying for residency. The absence of grades or assigned class ranks can make the earning of honor ratings very important to a student, for it serves to highlight exceptional performance. Hence, it is a critical responsibility of the clinical faculty using the honor system to ensure all students are fairly considered and honor ratings judiciously awarded. The Department of Surgery at Southern Illinois University takes a somewhat traditional approach to evaluating students, using largely subjective and some objective measures. Each student is assigned to a surgical faculty member who serves as his or her preceptor. Preceptors are responsible for reviewing copies of their assigned students’ history and physical examination write-ups, monitoring students’ progress through the lo-week clerkships, counseling, and presenting assessments of their students at the final evaluation meeting. Every faculty member completes a Student Evaluation Form on each student with whom he or she had contact. These forms provide for a scaled assessment of the student’s performance in each of the honor rating categories. The ratings are based on a 4-point scale with
“1” being exceptional and “4” unsatisfactory. Each preceptor receives all the faculty completed Student Evaluation Forms, test scores, and other pertinent information for his or her student for review. All faculty are scheduled to attend the final evaluation meeting. At this faculty forum, each preceptor presents a summary of his or her students’ performances. Nominations by any faculty member can subsequently be made following each preceptor presentation for awarding a student one or more honor ratings in any of the aforementioned five categories. Students can be given an honor rating in one or more of these categories. If faculty consensus is reached, the honor rating or ratings are given. It was hypothesized that certain factors might be influencing the granting of honor ratings in the surgery clerkship. This study was designed to determine if preidentified variables, some latent and some independent, were biasing student chances of receiving honor recognition, upon completing the clerkship. Specifically, the study investigated whether or not there was a relationship between clerks being granted an honor rating and 1. if a. b. c.
2. 3. 4. 5. 6.
their faculty preceptor for the clerkship: was an assistant, associate, or full professor? had a full- or part-time faculty appointment? was considered a strong or weak student advocate as judged by the Department Chairman using a l-5 Likert scale? d. had been with the school for a number of years? the sequence in which the student was discussed at the final evaluation meeting? the number of minutes spent by the faculty discussing the student? the number of physician faculty present at the final student evaluation meeting? students’ scores on their final multiple-choice examinations? the number of patient interview and physical examination write-ups completed during the clerkship?
METHODS Subjects included 51 third-year medical students who had been evaluated during one of three lo-week surgery clerkships and 22 clinical faculty members who served as preceptors during these clerkship rotations. The primary investigator of this study, a full-time nonclinical faculty member, attended the final student evaluation meetings. Notes were taken by her, unbeknownst by the clinical faculty, pertaining to the sequence in which students were discussed, the number of minutes spent discussing each student, and the number of physician faculty present at the meeting. Data related to the preceptors’ academic rank, employment status, and longevity were gathered by
reviewing personnel records. The “student advocacy” rating (Question lc) was based on the department chairman’s perception of each faculty member’s professional area of interest and emphasis. A rating of “1” was given to faculty whose primary accentuation was in research or other scholarly activities and less in undergraduate teaching. A rating of “5” meant the faculty member spent a good deal of time with their preceptee and was known to take a very strong interest in teaching. The chairman was not informed of the use of or intent for this information. Pearson’s product-moment correlation coefficients were calculated to determine if there was a significant
Correlates of Honor Ratings relationship between each variable and whether or not students received at least one honor rating. The Statistical Package for the Social Sciences (SPSS) softRESULTS
TABLE I
Selected
Factors
1. Preceptor characteristics a. assistant, associate, or full professor b. full- or part-time status G. low or high student advocate rating d. longevity with the medical school 2. No. of completed write-ups 3. Composite examination score 4. No. of clinical faculty present at meeting 5. Sequence student was discussed at meeting 6. Length of time spent discussing student
*p < .02 “p < .Ol
ware program was used to complete the analysis (Nie, Hull, Jenkins, Steinbrenner, & Bent, 1975).
AND DISCUSSION
Thirty-one (61%) students evaluated received at least one honor rating. Significant correlations were found between students receiving at least one honor rating and three variables (see Table 1). Faculty rated as high student advocates, those tending to take a strong interest in teaching and students, were more likely to have honor-rated students. This finding raises the question as to how fair it is to randomly assign preceptors to the students for the positive correlation indicates that preceptees of high student advocates are at an advantage over others in terms of gaining honor recognition. This may have resulted because faculty highly interested in teaching may know their students better than research oriented faculty. Each preceptor was aware of all student examination grades prior to the final evaluation meeting. Test score information was provided to all faculty at the meeting who did not serve as preceptors. Data indicated that three honor ratings in the “cognitive knowledge” category were given per cierkship. These three were usually given to the students with the three highest scores on the surgery final written exam. It is apparent that written exam scores were used as the major indicator of students’ knowledge even though a subjective appraisal of knowledge was made by faculty continuously throughout the clerkship. Faculty are with the students daily in the operating room, outpatient clinics, or on rounds where students are probed with questions. The fact that honor ratings were related to exam scores can lead to one of two conclusions. The first conclusion, which is unlikely, is that faculty subjective ratings of students were highly consistent with examination scores. The other conclusion
CORRELATIONS BETWEEN STUDENT HONOR AND SELECTED FACTORS FOR 51 THtRD-YEAR CLERKS AND 22 PRECEPTORS
129
RATINGS SURGERY
f .054 -.143 ,288’ .144 .296’* ,661* * -.041 ,183 -.172
may be that faculty relied strongly on exam scores, and less on their subjective ratings of cognitive knowledge when awarding honor ratings. Information about the total number of histories and physical examinations performed by students is known by each preceptor prior to the meeting. The significant correlation may be due to the fact that the more aggressive students, those actively seeking out an above average number of patients, were more visible on the wards and subsequently recognized by faculty. The quantity of workups certainly does not imply quality, but likely reflects industry, interest, and other characteristics typically held high by clerkship faculty. Data need to be collected and provided to faculty for the final evaluation meeting to ensure that the caliber of the workups are acceptable regardless of the total number completed. Because the evaluation meetings are often lengthy, it was hypothesized that the patience and endurance of the faculty might decrease with time. It was thought that faculty might hesitate to speak out on behalf of a student near the end of the meeting, thereby resulting in shorter discussions per clerk. The findings did not support these assumptions for the length of time as well as the sequence in which students were discussed did not significantly correlate with honor awards. There were insignificant correlations for the number of clinical faculty present during the meeting, university rank of the preceptor, number of years the preceptor was with the school, and whether the preceptor was a full- or part-time faculty member. This indicated that preceptor “status” did not influence the dissemination of honor ratings. The fact that no signi~cant correlations were found related to the chronological order in which students were discussed, the amount of time taken to discuss the student’s performance, or the number of attendings at the final evaluation meeting reflects positively on this forum evaluation approach. The key advantage of this approach is that it provides for oral input of the individuals who monitored and assessed student progress throughout the clerkship. Different faculty members evaluated students in varying settings, performing different skills, and at one or more taxonomy levels. To evaluate clerks’ performance without a forum of people who had student contact could result in making important decisions void of necessary information. Evaluating students and awarding honor ratings based solely on facultycompleted evaluation forms without open verbal exchange could lead to improper conclusions being drawn.
130
DEBRA A. DAROSA, ROLAND FOLSE, and JOHN F. MARKUS
The results of this study showed the forum approach itself does not create bias in the manner in which it is conducted. This is a very common method for assessing clerkship students that has to date not been studied. This research effort provides evidence that this method does not necessarily bias student assessment outcomes. If the variables under study were found to relate to honor recognition, immediate changes would be required to correct the inequities. If biases would have been found, the number of students that would have been effected would have been very high considering the widespread use of the faculty forum system for medical student assessment.
The preceptor advocacy level was found to produce biased evaluative results, which calls for further investigation and subsequent intervention to correct it. The remaining nonperformance variables, however, did not influence honors distribution. This should prove reassuring to faculty using the forum system, as well as the students assessed by it. It is the responsibility of the faculty to ensure the evaluation mechanisms employed provide for accurate assessments so that all students are considered fairly, and that the most deserving consistently receive honor recognition.
REFERENCES DOWALIBY,F. J., & ANDREW,B. J. (1976).Relationships tween
and examination Journal of Medical Education, 51, 181-188. KAPP, clinical
clinical
competence
M. (1981). performance.
Legal
ratings
issues
in faculty
be-
evaluation
of student
Journal of Medical Education, 56, 559-564.
NIE, N. H., HULL, C. H., JENKINS, J. G., STEINBRENNER, K., & BENT, D. H. (1975). Statisticalpackage for the social sciences (2nd ed.) New York: McGraw-Hill. OAKS, W. W., SCHEINOK, P. A., & HUSTED, F. L. (1969). Objective evaluation of a method of assessing student performance in a clinical clerkship. Journal of Medical Education, 44, 207-213. O’DONOHUE,
W. J., & WERGIN,
J. F. (1978).
Evaluation
medical
students
during
a clinical
clerkship
in internal
medicine.
Journal of Medical Education, 53, 55-58.
performance.
of
PRINTEN, K., CHAPPELL, W., & WHITNEY, D. (1973). Clinical performance evaluation of junior medical students. Journal of Medical Education, 48, 343-348. SILBER, D. L., WILLIAMS, R. G., PAIVA, R. E., TAYLOR, D. D., & ROBINSON, R. (1978). The SIU medical curriculum: Systemwide objectives-based instruction. Journal of Medical Educa-
tion, 53, 473-479. WIGTON, R. S. (1980). The effects of student personal characteristics on the evaluation of clinical performance. Journal of
Medical Education, 55, 423-427.