- Email: [email protected]
113 Emergency Medicine Residents' Self-assessments Play a Critical Role When Receiving Feedback From Evaluators
Research Forum Abstracts
111
A Comparison of Individualized Feedback Versus Standard Didactic Lecture to Teach Interpersonal Communication Skills to Emergency Medicine Residents: A Multicenter Randomized Controlled Trial
Leber M, He C, Akhtar S, Asher S, Bania T, Di C, Steinberg E, Webster A, Clark M/St. Luke’s/Roosevelt Hospital Center, New York, NY; Beth Israel Medical Center, New York, NY; Albany Medical Center, Albany, NY
Background: Effective communication and interpersonal skills by residents are integral to providing quality patient care and to achieving patient satisfaction. The Accreditation Council for Graduate Medical Education (ACGME) views these skills as critical to residency education and includes them among the core competencies. There is a lack of evidence-based literature regarding the best methods to teach and evaluate these skills. Study Objectives: We hypothesize that providing residents with individualized direct feedback of actual patient encounters is an effective way of teaching interpersonal communication skills and is a superior teaching method to the traditional didactic format. Methods: We randomized 35 hypothesis-blinded residents at 3 ACGME accredited emergency medicine residencies into 2 intervention groups: didactic lecture or individualized feedback. A questionnaire was utilized which contained 9 questions about physician communication and interpersonal skills and one summary question that indicated the patient’s overall experience with the resident by asking if the patient would want to be cared for by that resident again. Twenty-five questionnaires were collected from patients for each resident prior to intervention based on a convenience sample. Each positive response on the survey was given one point, with a maximum score of 9 points per survey. After each resident completed the assigned intervention, an additional 25 questionnaires were collected from his or her patients. We compared the results of the pre- and post-intervention questionnaire scores to measure improvement of the resident’s interpersonal skills in response to the intervention. We also compared the improvement in the patients’ overall experience between the 2 groups. Results: Thirty-five residents at the 3 ACGME accredited emergency medicine residency training programs were randomized into 2 intervention groups (16 didactic lecture, 19 direct feedback). Residents in the direct feedback group demonstrated greater improvement of interpersonal skills, 7.64% (95 CI 4.8%-10.5%), compared to the didactic lecture group, 2.40% (95 CI -1.1%-5.9%). Additionally, residents in the direct feedback group demonstrated greater improvement in their patients’ overall desire to be cared for by that resident again, 6.86% (95 CI 3.8%-10.0%), compared to the didactic lecture group, 0.54% (95 CI -3.5%-4.6%). Conclusion: Direct feedback based on patient encounter questionnaires is an effective method of teaching communication and interpersonal skills to emergency medicine residents and is superior to traditional didactic lectures. This is a promising new strategy which can be incorporated into the teaching and evaluation of emergency medicine ACGME core competencies.
112
Resident Education in Ultrasonography: Assessment of Identification of Basic Cardiac Anatomy and Pathology After a Multimedia Tutorial
Bharati A, Datta A, Gupta S, Chun-Lema P, Ryan J/New York Hospital queens, Flushing, NY; New York Hospital Queens, Flushing, NY
Study Objectives: It has been suggested that independent study and individual educational activities can be more effective than didactic lectures in resident education. Past studies have shown that multimedia devices (such as simulation and image review) can be effective in skill development among medical students. Our primary objective was to show a multimedia application can be effective in teaching emergency medicine residents basic echocardiography for identification of normal anatomy and common pathology. We re-examined the subjects at 6 months to evaluate their retention of information. Methods: This was an observational study. The subjects were emergency medicine residents enrolled in a 3-year residency in emergency medicine approved by the Accreditation Council for Graduate Medical Education (ACGME). Subjects were then divided into 2 groups. Half the residents from each PGY year were randomly assigned to each group. Both groups received a pre-test (20 multiple choice questions) prior to the start of the learning activity. Group C received a didactic lecture based on emergency echocardiography. The lecture format was case-based presentation and was
Volume , . : October
delivered by a senior member of the ultrasonography division at New York Hospital Queens. Group AB received a 1-hour case-based multimedia tutorial in cardiac ultrasonography (developed by the author). Both presentations were identical in format, cases and images. At the end of the learning activity, students were given a post test (20 multiple choice questions) and re-evaluated 6 months after the activity for retention of educational objectives. All subjects were evaluated with a hands-on test before and after the learning activity (with follow up after 6 months). Residents were given individual ID tags and were only included in the results if they had completed both phases of the study. Results: Twenty-five residents were enrolled in the study with 13 enrolled in Group AB and 12 enrolled in Group C with median pretest scores 68 (59-70) and 62.5 (52.5-75) respectively, post-test scores 90 (87-95) and 77.5 (67.5-85), 6 month follow up scores of 82.5 (80-90) and 80 (70-90). Prior to participation, residents were assessed for competency performing 4 views of cardiac ultrasonography (long axis, short axis, subxyphoid, apical). Groups AB and C scored 75 (75-100) and 75 (50100) respectively. Immediately after the learning activity each group was assessed in obtaining 4 views as stated previously. Groups AB and C scored 75 (50-100) and 62.5 (0-100) respectively. There was a significant difference between AB and C groups from intial pre-test to post-test with p value of p⬍0.001, t⫽ 6.7 and p⫽0.005, t⫽3.5 but there was no significant difference at 6 months when compared to post test with p⫽0.7, t⫽0.3 and p⫽0.4 , t⫽0.7 respectively. Similarly there was no difference between hands on scores at 0 and 6 months between groups AB and C with p⫽0.5, t⫽0.6 and p⫽0.4, t⫽0.9. Conclusion: Individual modules can be helpful in teaching residents cardiac ultrasonography. Their use as a part of the residency curriculum can improve identification of cardiac anatomy and pathology. Translation of knowledge to practical use in image acquisition and retention of knowledge at 6 months is not shown in this study. Further studies need to be done to assess the utility of this teaching module.
113
Emergency Medicine Residents’ Self-assessments Play a Critical Role When Receiving Feedback From Evaluators
Bush C, Bounds R, Aghera A, Rodriguez N, Davis B, Santen S/Michigan State University College of Human Medicine, Grand Rapids, MI; Christiana Care Health Systems, Newark, DE; Maimonides Medical Center, Brooklyn, NY; University of Wisconsin, Madison, WI; University of Michigan Health System, Ann Arbor, MI
Study Objectives: One of the greatest challenges in medical education lies in the provision of valuable feedback that motivates residents to improve. Educators often focus on enhancing the quality of feedback, but rarely consider the influence of residents’ own self-assessment. This study evaluates the generation of learning goals by emergency medicine residents after participating in an oral board scenario, performing a self-assessment, and receiving feedback. We hypothesize that their learning goals are based more on the resident’s own self assessment than on the specific feedback given, and that the greatest impact will be provided when the self-assessment and the feedback are in agreement. Methods: This multicenter prospective educational study at 4 academic programs involved senior emergency medicine residents during a 6-month period. Volunteers were asked to participate in a standard oral board scenario. After the scenario, the resident completed a standardized self-assessment form. Next, the 4 examiners provided specific feedback with both positive and negative points. The feedback was standardized with a checklist, which was validated on a representative sample of the target population. The resident was then asked to generate “SMART” learning goals (Specific, Measurable, Attainable, Realistic, and Timely) (Chang 2011). The primary outcome was the source of the learning goals and whether the learning goals came from the resident’s self-assessment, feedback or both. Descriptive statistics (frequency tables, confidence intervals) were used to summarize the data. Quantitative data were expressed as the mean ⫹/⫺ SEM, while nominal data were expressed as a percentage (frequency tables). Results: Of a total of 76 senior emergency medicine residents eligible for study participation, 75 volunteered to participate. Three residents could not complete the study because of time constraints. Overall, 72 residents (95%) were enrolled in the study. Sixty-one (82%) of the residents achieved greater than minimum competence in overall scores. After completing the scenario, these volunteers generated 226 learning goals (mean 3.1 ⫹/⫺ 1.3). Residents’ perceptions of their own performance were more likely to result in the generation of learning goals than was examiner
Annals of Emergency Medicine S41
Research Forum Abstracts feedback (47% versus 27%) (Table). When there was disagreement between resident and examiner, learning goals favored self-assessment over feedback (4% versus 1%).
115
Review of Emergency Medicine Publications: Methodology, Ethics and Setting
Ducasse J, Dehours E, Houze Cerfon V, Bounes V/Samu 31 - Chu Purpan, Toulouse, France
Conclusions: The findings highlight the perceived strengths and weaknesses in clinical competence and self-assessment skills and provide direction for program training needs. While self-assessments may not realistically indicate ability, it is still critical to determine how students perceive their ability because their opinions drive their learning goals. The interactive examination appeared to be a convenient tool for providing deeper insight into students’ ability to prioritize, self-assess and steer their own learning.
114
Emergency Medicine Residency Applicant Interview Scoring: Can a Global Visual Analog Scale Produce Reliable Results?
Roper J, Melanson S, Jeanmonod R/St. Luke’s University Hospital, Bethlehem, PA
Study Objective: The interview process is integral to choosing emergency medicine residents. Historically, the interview score has accounted for 75% of the variability in rank position at our institution, and this has been reported in another study. There is little data exploring the reliability of interview scoring in emergency medicine . We sought to determine if a single global visual analog scale (VAS) to score applicants would create a broad enough spread of values to be useful while being reliable between faculty interviewers. Methods: This is a retrospective observational study performed at a communitybased university-affiliated institution with 8 allopathic emergency medicine positions each year. Applicants in the 2011 pool were interviewed by 3 or 4 individuals drawn from a pool of 9 academic faculty and 1 chief resident. Interviewers scored applicants using a VAS which was 100mm long with the words “bottom of the rank list” on one side and “top of the rank list” on the other. Interviewers were instructed to place a single hash mark on the VAS corresponding to where they believed the applicant should be ranked. The distance in millimeters from the “bottom of the rank list” side was measured and recorded as the applicant’s score. Interviewers had access to the applicants’ academic records and board scores for 15 to 30 minutes prior to the interview. Scores by each of the 10 interviewers were compared to determine if any interviewer scored applicants consistently lower or higher than the global pool of interviewers. Each applicant’s scores were analyzed for reliability among the interviewers. Regression analysis was performed on interview scores as compared to board scores as well as rank position. Scores for the entire interview season were examined through descriptive statistics. This study was institutional review board exempt. Results: Over 600 applications were received in the 2011 season, and about 100 applicants were invited to interview. 91 applicants accepted the invitation and were interviewed during the 2011 interview season. The mean interview score for these applicants was 61.9 (median 61.7) with a standard of deviation of 16.5. The 10 interviewers did not differ significantly in their scoring of applicants (ANOVA p⫽NS). For applicants interviewed by 4 interviewers, the intraclass correlation was 0.85. For those interviewed by 3 interviewers, the intraclass correlation was 0.84. In regression analysis of interview scores and USMLE Step 1 scores, r ⫽ 0.28 (r squared ⫽ 0.08). In regression analysis of interview scores and final rank position, r ⫽ 0.87 (r squared ⫽ 0.75). Conclusion: A global VAS provides a wide distribution of scores and is a reliable method for assessing interview applicants with 3 or 4 interviewers. Interview score was responsible for 75% of the variability in position on the rank list. There was only a weak relationship between interview score and USMLE Step 1 score. Future studies will determine if applicant interview scores correlate with clinical success in residency.
S42 Annals of Emergency Medicine
Study Objective: To describe emergency medicine publications in terms of methodology, approval by IRB, method of consent, external validity and setting (eg, out-of-hospital or emergency department). Methods: The 12 top ranked emergency journals were selected. We manually reviewed the last 30 original articles in each emergency medicine journal, to represent more than 2 months of publications for all emergency medicine journals (range 2 to 6 months). Only clinical original articles on human subjects were included. To ensure accurate data transcription, each article was read at least twice by 2 different reviewers and graded by written criteria using an extraction standard chart. Results: Over the articles reviewed, 330 were analyzed. 189 (57.3%) were prospective studies, 29 (8.8%) were randomized studies. 226 studies (68.5%) mentioned an institutional review board approval or a waiver of authorization, and an informed consent was not mentioned in 227 (68.8%) of studies. 59 (17.9%) were conducted in a out-of-hospital setting. Two hundred and thirty 8 (72.1%) of these studies were at single center institutions, the USA contributed 158 (47.9%) of the total publications. Trauma, Management-Overcrowding-Policies and Cardiology (including cardiac arrest) represented more than 50% of all the publications. Conclusion: This study describes publications in the field of emergency medicine. Randomized studies represent 9% of publications, most studies are cross sectional, and more than half have a retrospective design. We found that in one third of the studies, an institutional review board review was not mentioned and informed consent was not specified in 2/3 studies. Emergency medicine research volume, quality, and grants activity must increase in order for emergency medicine to progress within academic medicine.
116
Impact of Learners on Emergency Medicine Attending Physician Productivity
Bhat R, Maloy K, Dubin J/Georgetown University Hospital, Washington, DC; Washington Hospital Center, Washington, DC
Background: Several prior studies have examined the impact of learners (medical students or residents) on overall emergency department flow as well as the impact of resident training level on the number of patients seen by residents per hour. No study to date has specifically examined the impact of learners on emergency medicine attending physician productivity, with regards to patients per hour (PPH). We sought to evaluate whether learners increase, decrease, or have no effect on the productivity of emergency medicine attending physicians in a teaching program with one student or resident per attending. Methods: This was a retrospective database review of an urban, academic tertiary care center with 3 separate teams on the acute care side of the emergency department. Each team was staffed with one attending physician paired with either one resident, one medical student or with no learners. All shifts from July 1, 2008 to June 30th 2010 were reviewed using an electronic database. A shift was predefined as “Resident” if ⬎ 5 patients were seen by a resident, “Medical Student” if any patients were seen by a medical student, and “No Learners” if no patients were seen by a medical student or resident. Shifts were removed from analysis if more than one learner saw patients during the shift. Resident shifts were further stratified by emergency medicine training level or off service rotator. For each type of shift, the total number of patients seen by the attending physician was then divided by 8 hours (shift duration) to arrive at number of patients per hour. Results: A total of 7360 shifts were analyzed with 2778 removed due to multiple learners on a team. For the 2199 shifts with attending physicians with no learners, the average number of PPH was 1.87(95% CI 1.86,1.89). For the 514 medical student shifts, the average PPH was 1.87(95% CI 1.84,1.90), p ⫽ 0.99. For the 1935 resident shifts, the average PPH was 1.99(95% CI 1.97,2.00), p ⬍0.005, with no statistically significant difference found between emergency medicine 1: 1.98, emergency medicine 2: 1.99, emergency medicine 3: 1.99, off-service: 1.99. Conclusion: The addition of a resident to an attending team in a one-on-one teaching model increased attending productivity while the addition of a medical student had no impact. The results of this study may help guide emergency departments seeking to expand or establish a residency training program to assess the productivity impact of this decision.
Volume , . : October
111
A Comparison of Individualized Feedback Versus Standard Didactic Lecture to Teach Interpersonal Communication Skills to Emergency Medicine Residents: A Multicenter Randomized Controlled Trial
Leber M, He C, Akhtar S, Asher S, Bania T, Di C, Steinberg E, Webster A, Clark M/St. Luke’s/Roosevelt Hospital Center, New York, NY; Beth Israel Medical Center, New York, NY; Albany Medical Center, Albany, NY
Background: Effective communication and interpersonal skills by residents are integral to providing quality patient care and to achieving patient satisfaction. The Accreditation Council for Graduate Medical Education (ACGME) views these skills as critical to residency education and includes them among the core competencies. There is a lack of evidence-based literature regarding the best methods to teach and evaluate these skills. Study Objectives: We hypothesize that providing residents with individualized direct feedback of actual patient encounters is an effective way of teaching interpersonal communication skills and is a superior teaching method to the traditional didactic format. Methods: We randomized 35 hypothesis-blinded residents at 3 ACGME accredited emergency medicine residencies into 2 intervention groups: didactic lecture or individualized feedback. A questionnaire was utilized which contained 9 questions about physician communication and interpersonal skills and one summary question that indicated the patient’s overall experience with the resident by asking if the patient would want to be cared for by that resident again. Twenty-five questionnaires were collected from patients for each resident prior to intervention based on a convenience sample. Each positive response on the survey was given one point, with a maximum score of 9 points per survey. After each resident completed the assigned intervention, an additional 25 questionnaires were collected from his or her patients. We compared the results of the pre- and post-intervention questionnaire scores to measure improvement of the resident’s interpersonal skills in response to the intervention. We also compared the improvement in the patients’ overall experience between the 2 groups. Results: Thirty-five residents at the 3 ACGME accredited emergency medicine residency training programs were randomized into 2 intervention groups (16 didactic lecture, 19 direct feedback). Residents in the direct feedback group demonstrated greater improvement of interpersonal skills, 7.64% (95 CI 4.8%-10.5%), compared to the didactic lecture group, 2.40% (95 CI -1.1%-5.9%). Additionally, residents in the direct feedback group demonstrated greater improvement in their patients’ overall desire to be cared for by that resident again, 6.86% (95 CI 3.8%-10.0%), compared to the didactic lecture group, 0.54% (95 CI -3.5%-4.6%). Conclusion: Direct feedback based on patient encounter questionnaires is an effective method of teaching communication and interpersonal skills to emergency medicine residents and is superior to traditional didactic lectures. This is a promising new strategy which can be incorporated into the teaching and evaluation of emergency medicine ACGME core competencies.
112
Resident Education in Ultrasonography: Assessment of Identification of Basic Cardiac Anatomy and Pathology After a Multimedia Tutorial
Bharati A, Datta A, Gupta S, Chun-Lema P, Ryan J/New York Hospital queens, Flushing, NY; New York Hospital Queens, Flushing, NY
Study Objectives: It has been suggested that independent study and individual educational activities can be more effective than didactic lectures in resident education. Past studies have shown that multimedia devices (such as simulation and image review) can be effective in skill development among medical students. Our primary objective was to show a multimedia application can be effective in teaching emergency medicine residents basic echocardiography for identification of normal anatomy and common pathology. We re-examined the subjects at 6 months to evaluate their retention of information. Methods: This was an observational study. The subjects were emergency medicine residents enrolled in a 3-year residency in emergency medicine approved by the Accreditation Council for Graduate Medical Education (ACGME). Subjects were then divided into 2 groups. Half the residents from each PGY year were randomly assigned to each group. Both groups received a pre-test (20 multiple choice questions) prior to the start of the learning activity. Group C received a didactic lecture based on emergency echocardiography. The lecture format was case-based presentation and was
Volume , . : October
delivered by a senior member of the ultrasonography division at New York Hospital Queens. Group AB received a 1-hour case-based multimedia tutorial in cardiac ultrasonography (developed by the author). Both presentations were identical in format, cases and images. At the end of the learning activity, students were given a post test (20 multiple choice questions) and re-evaluated 6 months after the activity for retention of educational objectives. All subjects were evaluated with a hands-on test before and after the learning activity (with follow up after 6 months). Residents were given individual ID tags and were only included in the results if they had completed both phases of the study. Results: Twenty-five residents were enrolled in the study with 13 enrolled in Group AB and 12 enrolled in Group C with median pretest scores 68 (59-70) and 62.5 (52.5-75) respectively, post-test scores 90 (87-95) and 77.5 (67.5-85), 6 month follow up scores of 82.5 (80-90) and 80 (70-90). Prior to participation, residents were assessed for competency performing 4 views of cardiac ultrasonography (long axis, short axis, subxyphoid, apical). Groups AB and C scored 75 (75-100) and 75 (50100) respectively. Immediately after the learning activity each group was assessed in obtaining 4 views as stated previously. Groups AB and C scored 75 (50-100) and 62.5 (0-100) respectively. There was a significant difference between AB and C groups from intial pre-test to post-test with p value of p⬍0.001, t⫽ 6.7 and p⫽0.005, t⫽3.5 but there was no significant difference at 6 months when compared to post test with p⫽0.7, t⫽0.3 and p⫽0.4 , t⫽0.7 respectively. Similarly there was no difference between hands on scores at 0 and 6 months between groups AB and C with p⫽0.5, t⫽0.6 and p⫽0.4, t⫽0.9. Conclusion: Individual modules can be helpful in teaching residents cardiac ultrasonography. Their use as a part of the residency curriculum can improve identification of cardiac anatomy and pathology. Translation of knowledge to practical use in image acquisition and retention of knowledge at 6 months is not shown in this study. Further studies need to be done to assess the utility of this teaching module.
113
Emergency Medicine Residents’ Self-assessments Play a Critical Role When Receiving Feedback From Evaluators
Bush C, Bounds R, Aghera A, Rodriguez N, Davis B, Santen S/Michigan State University College of Human Medicine, Grand Rapids, MI; Christiana Care Health Systems, Newark, DE; Maimonides Medical Center, Brooklyn, NY; University of Wisconsin, Madison, WI; University of Michigan Health System, Ann Arbor, MI
Study Objectives: One of the greatest challenges in medical education lies in the provision of valuable feedback that motivates residents to improve. Educators often focus on enhancing the quality of feedback, but rarely consider the influence of residents’ own self-assessment. This study evaluates the generation of learning goals by emergency medicine residents after participating in an oral board scenario, performing a self-assessment, and receiving feedback. We hypothesize that their learning goals are based more on the resident’s own self assessment than on the specific feedback given, and that the greatest impact will be provided when the self-assessment and the feedback are in agreement. Methods: This multicenter prospective educational study at 4 academic programs involved senior emergency medicine residents during a 6-month period. Volunteers were asked to participate in a standard oral board scenario. After the scenario, the resident completed a standardized self-assessment form. Next, the 4 examiners provided specific feedback with both positive and negative points. The feedback was standardized with a checklist, which was validated on a representative sample of the target population. The resident was then asked to generate “SMART” learning goals (Specific, Measurable, Attainable, Realistic, and Timely) (Chang 2011). The primary outcome was the source of the learning goals and whether the learning goals came from the resident’s self-assessment, feedback or both. Descriptive statistics (frequency tables, confidence intervals) were used to summarize the data. Quantitative data were expressed as the mean ⫹/⫺ SEM, while nominal data were expressed as a percentage (frequency tables). Results: Of a total of 76 senior emergency medicine residents eligible for study participation, 75 volunteered to participate. Three residents could not complete the study because of time constraints. Overall, 72 residents (95%) were enrolled in the study. Sixty-one (82%) of the residents achieved greater than minimum competence in overall scores. After completing the scenario, these volunteers generated 226 learning goals (mean 3.1 ⫹/⫺ 1.3). Residents’ perceptions of their own performance were more likely to result in the generation of learning goals than was examiner
Annals of Emergency Medicine S41
Research Forum Abstracts feedback (47% versus 27%) (Table). When there was disagreement between resident and examiner, learning goals favored self-assessment over feedback (4% versus 1%).
115
Review of Emergency Medicine Publications: Methodology, Ethics and Setting
Ducasse J, Dehours E, Houze Cerfon V, Bounes V/Samu 31 - Chu Purpan, Toulouse, France
Conclusions: The findings highlight the perceived strengths and weaknesses in clinical competence and self-assessment skills and provide direction for program training needs. While self-assessments may not realistically indicate ability, it is still critical to determine how students perceive their ability because their opinions drive their learning goals. The interactive examination appeared to be a convenient tool for providing deeper insight into students’ ability to prioritize, self-assess and steer their own learning.
114
Emergency Medicine Residency Applicant Interview Scoring: Can a Global Visual Analog Scale Produce Reliable Results?
Roper J, Melanson S, Jeanmonod R/St. Luke’s University Hospital, Bethlehem, PA
Study Objective: The interview process is integral to choosing emergency medicine residents. Historically, the interview score has accounted for 75% of the variability in rank position at our institution, and this has been reported in another study. There is little data exploring the reliability of interview scoring in emergency medicine . We sought to determine if a single global visual analog scale (VAS) to score applicants would create a broad enough spread of values to be useful while being reliable between faculty interviewers. Methods: This is a retrospective observational study performed at a communitybased university-affiliated institution with 8 allopathic emergency medicine positions each year. Applicants in the 2011 pool were interviewed by 3 or 4 individuals drawn from a pool of 9 academic faculty and 1 chief resident. Interviewers scored applicants using a VAS which was 100mm long with the words “bottom of the rank list” on one side and “top of the rank list” on the other. Interviewers were instructed to place a single hash mark on the VAS corresponding to where they believed the applicant should be ranked. The distance in millimeters from the “bottom of the rank list” side was measured and recorded as the applicant’s score. Interviewers had access to the applicants’ academic records and board scores for 15 to 30 minutes prior to the interview. Scores by each of the 10 interviewers were compared to determine if any interviewer scored applicants consistently lower or higher than the global pool of interviewers. Each applicant’s scores were analyzed for reliability among the interviewers. Regression analysis was performed on interview scores as compared to board scores as well as rank position. Scores for the entire interview season were examined through descriptive statistics. This study was institutional review board exempt. Results: Over 600 applications were received in the 2011 season, and about 100 applicants were invited to interview. 91 applicants accepted the invitation and were interviewed during the 2011 interview season. The mean interview score for these applicants was 61.9 (median 61.7) with a standard of deviation of 16.5. The 10 interviewers did not differ significantly in their scoring of applicants (ANOVA p⫽NS). For applicants interviewed by 4 interviewers, the intraclass correlation was 0.85. For those interviewed by 3 interviewers, the intraclass correlation was 0.84. In regression analysis of interview scores and USMLE Step 1 scores, r ⫽ 0.28 (r squared ⫽ 0.08). In regression analysis of interview scores and final rank position, r ⫽ 0.87 (r squared ⫽ 0.75). Conclusion: A global VAS provides a wide distribution of scores and is a reliable method for assessing interview applicants with 3 or 4 interviewers. Interview score was responsible for 75% of the variability in position on the rank list. There was only a weak relationship between interview score and USMLE Step 1 score. Future studies will determine if applicant interview scores correlate with clinical success in residency.
S42 Annals of Emergency Medicine
Study Objective: To describe emergency medicine publications in terms of methodology, approval by IRB, method of consent, external validity and setting (eg, out-of-hospital or emergency department). Methods: The 12 top ranked emergency journals were selected. We manually reviewed the last 30 original articles in each emergency medicine journal, to represent more than 2 months of publications for all emergency medicine journals (range 2 to 6 months). Only clinical original articles on human subjects were included. To ensure accurate data transcription, each article was read at least twice by 2 different reviewers and graded by written criteria using an extraction standard chart. Results: Over the articles reviewed, 330 were analyzed. 189 (57.3%) were prospective studies, 29 (8.8%) were randomized studies. 226 studies (68.5%) mentioned an institutional review board approval or a waiver of authorization, and an informed consent was not mentioned in 227 (68.8%) of studies. 59 (17.9%) were conducted in a out-of-hospital setting. Two hundred and thirty 8 (72.1%) of these studies were at single center institutions, the USA contributed 158 (47.9%) of the total publications. Trauma, Management-Overcrowding-Policies and Cardiology (including cardiac arrest) represented more than 50% of all the publications. Conclusion: This study describes publications in the field of emergency medicine. Randomized studies represent 9% of publications, most studies are cross sectional, and more than half have a retrospective design. We found that in one third of the studies, an institutional review board review was not mentioned and informed consent was not specified in 2/3 studies. Emergency medicine research volume, quality, and grants activity must increase in order for emergency medicine to progress within academic medicine.
116
Impact of Learners on Emergency Medicine Attending Physician Productivity
Bhat R, Maloy K, Dubin J/Georgetown University Hospital, Washington, DC; Washington Hospital Center, Washington, DC
Background: Several prior studies have examined the impact of learners (medical students or residents) on overall emergency department flow as well as the impact of resident training level on the number of patients seen by residents per hour. No study to date has specifically examined the impact of learners on emergency medicine attending physician productivity, with regards to patients per hour (PPH). We sought to evaluate whether learners increase, decrease, or have no effect on the productivity of emergency medicine attending physicians in a teaching program with one student or resident per attending. Methods: This was a retrospective database review of an urban, academic tertiary care center with 3 separate teams on the acute care side of the emergency department. Each team was staffed with one attending physician paired with either one resident, one medical student or with no learners. All shifts from July 1, 2008 to June 30th 2010 were reviewed using an electronic database. A shift was predefined as “Resident” if ⬎ 5 patients were seen by a resident, “Medical Student” if any patients were seen by a medical student, and “No Learners” if no patients were seen by a medical student or resident. Shifts were removed from analysis if more than one learner saw patients during the shift. Resident shifts were further stratified by emergency medicine training level or off service rotator. For each type of shift, the total number of patients seen by the attending physician was then divided by 8 hours (shift duration) to arrive at number of patients per hour. Results: A total of 7360 shifts were analyzed with 2778 removed due to multiple learners on a team. For the 2199 shifts with attending physicians with no learners, the average number of PPH was 1.87(95% CI 1.86,1.89). For the 514 medical student shifts, the average PPH was 1.87(95% CI 1.84,1.90), p ⫽ 0.99. For the 1935 resident shifts, the average PPH was 1.99(95% CI 1.97,2.00), p ⬍0.005, with no statistically significant difference found between emergency medicine 1: 1.98, emergency medicine 2: 1.99, emergency medicine 3: 1.99, off-service: 1.99. Conclusion: The addition of a resident to an attending team in a one-on-one teaching model increased attending productivity while the addition of a medical student had no impact. The results of this study may help guide emergency departments seeking to expand or establish a residency training program to assess the productivity impact of this decision.
Volume , . : October