- Email: [email protected]
Computerized tracking of emergency medicine resident clinical experience
ORIGINAL CONTRIBUTION computer; residency, clinical experience
Computerized Tracking of Emergency Medicine Resident Clinical Experience Although we c o m m o n l y assume that because residents spend a given number of months in the emergency department they achieve adequate exposure to all necessary clinical entities, this has never been shown. We suspect, rather, that great variability exists among residents in the number and variety of patients they see; and that with respect to the ED, there are important diagnoses that are rare or absent in the clinical pathology of a training program. To confirm these hypotheses, we implemented a computerized system of recording patients and diagnoses managed in the ED by the 33 residents of the University of Illinois Affiliated Hospitals Emergency Medicine Residency. We collected data for nine months and accumulated 2,152 shifts of clinical experience. These data confirm our hypotheses. We found that senior residents managed an average of 11.9 + 2.3 patients per ten-hour shift, but the quickest resident saw almost twice as m a n y patients as the slowest. Junior residents saw fewer patients, 8.5 ++1.4 patients per shift, but maintained a twofold difference between the fastest and slowest. Furthermore, there are important diagnoses that present too rarely for each resident to become facile in their management. We found that 22.7% of the 554 diagnoses listed in the Emergency Medicine Core Content never once presented to the ED. An additional 34.7% of these diagnoses did present, but so rarely that each resident could not possibly manage one case during a residency. The Length of Training R& port of the American College of Emergency Physicians provides objective guidelines for the number of encounters a resident should have with 283 clinical entities. In this study, residents fell short of these guidelines with 50.5% of diagnoses. While absolute quantity of exposure does not assure competence in management, we recommend that each residency monitor the experience of its residents. This allows a residency to change its curriculum to m a k e optimum use of available pathology, as well as to supplem e n t deficiencies in clinical experience with case simulations. [Langdorf MI, Strange G, Macneil P: Computerized tracking of emergency medicine resident clinical experience. Ann Emerg Med July 1990;19:764-773.]
Mark I Langdorf, MD, MHPE* Irvine, California Gary Strange, MD, FACEPt Chicago, Illinois Philip Macneil, MD:~ New York, New York From the Emergency Medicine Residency, University of California, Irvine;* University of Illinois Affiliated Emergency Medicine Residency, Chicago;¢ and Metropolitan Hospital Emergency Medicine Residency, New York.¢ Received for publication August 7, 1989. Revision received December 19, 1989. Accepted for publication March 26, 1990. Presented at the Second International Conference on Emergency Medicine in Brisbane, Australia, October 1988. Supported in part by the 1988 Educational Methodology Grant from the Society of Teachers of Emergency Medicine and the Emergency Medicine Foundation. Address for reprints: Mark I Langdorf, MD, MHPE, University of California, Irvine, Medical Center, Division of Emergency Medicine, 101 The City Drive, Route 128, Orange, California 92668.
INTRODUCTION Graduate medical education in the United States is based on an apprenticeship model, in which a trainee spends an extended period under the tutelage of an experienced practitioner and learns his approach to clinical medicine. 1-3 The clinical situations to which the trainee is exposed depend solely on the patients who present for treatment. In times past, with this close mentor-student relationship, the practitioner had complete knowledge concerning the breadth and depth of training. Modern graduate medical education, however, has become too complex for any one person to adequately supervise the residents' training. 1 Residents work around the clock, at various sites within the hospital, and, often, at various hospitals. The clinical training in emergency medicine occurs on a random basis dependent on which patients present to the emergency department, and so uniformity of exposure to clinical medicine is difficult to achieve. No attempt has yet been made to assess the variability in clinical exposure among residents or to devise a method by which we can offer each resident a more complete clinical education. Much time and effort has gone into the development of a Core Content
19:7 Jury 1990
Annals of Emergency Medicine
764/63
RESIDENT EXPERIENCE Langdorf, Strange & Macneil
FIGURE 1. Clinical categories and
abbreviations. Statement (CSS) for emergency medicine and into the sequencing of didactic sessions to cover the material. However, residents in emergency medicine spend only 10% to 15% of their training in the didactic arena. The remainder of time is spent seeing patients under faculty supervision. It seems prudent, therefore, to put forth at least as much effort to assure an equally complete clinical exposure. If the ultimate goal in emergency medicine graduate education is to provide as thorough an education as possible for each resident, then four e s s e n t i a l q u e s t i o n s m u s t be addressed: What is the current clinical exposure of residents? What clinical exposure constitutes optimum training that fulfills established learning objectives? What are the weaknesses of the current clinical exposure, ie, how does reality fall short of the opt i m u m ? and, H o w can alternate training methods be used to rectify these deficiencies? This study was designed to address the first and third questions in this sequence for one emergency medicine residency. We hypothesized that a catalog of resident clinical experience in the ED would demonstrate that there is great variability among residents in clinical exposure and some important clinical entities present so rarely that it is unlikely that each resident would have clinical exposure. While clinical exposure alone is not sufficient to assure competence, it is a vital component of any training p r o g r a m . 4-6 As D a i l e y paraphrased Osler: "To see patients without reading about them is to go to sea without charts; but to read about disease without seeing patients is not to go to sea at all. ''4 Case management must also be supervised by experienced emergency medicine faculty so that the resident receives optimum benefit from clinical exposure in terms of bedside teaching and problem solving. METHODS A program to track the ED clinical exposure of all 33 residents in the University of Illinois Affiliated Hospitals E m e r g e n c y M e d i c i n e Resid e n c y was developed and imple-
64/765
Abdominal and gastrointestinal disorders (GI) Cardiovascular disorders (CV) Cutaneous disorders (Skin) Disorders related to the immune system (Immune) Disorders caused by biologic agents (ID) Disorders due to chemical and environmental agents (Env/drug) Hematologic disorders (Heme) Hormonal, metabolic, and nutritional disorders (Hormone/metab) Disorders of the head and neck (ENT) Disorders primarily presenting in infancy and childhood (Peds) Musculoskeletal disorders (Ortho) Nervous system disorders (Neuro) Psychobehavioral disorders (Psych) Thoracic-respiratory disorders (Chest) Urogenital disorders (GU) m e n t e d during the academic year 1987-88. There were eight first-, 12 second-, and 13 third-year residents. All of the residents came directly from medical school. All patients seen by residents during their emergency medicine rotations from July 1987 through March 1988 at the three participating hospitals, Mercy Hospital and Medical Center, Illinois Masonic Medical Center, both in Chicago, and Lutheran General Hospital in suburban Park Ridge, had data entered in the computer catalog by data processing clerks. These data consisted of a resident code number; up to three diagnostic codes taken from the International Classification of Diseases, Ninth Revision, Clinical Modification (ICD9-CM); 7 the date of visit; medical record number; and whether the patient was admitted to the hospital. We did not track any index of severity of illness. We compiled and sorted these data by both diagnosis and resident using a microcomputer with dBase III+ software. We assessed variability among residents in exposure to diagnoses and the incidence of diagnoses that are rare among the pathology of the hospitals. The three EDs had a combined yearly census of more than 100,000 patients per year, and two of the three are Level I trauma centers. Residents spent 19 of their 22 emergency medicine (86%) and of their 34 total clinical months (56%) in EDs that were served by the catalog system. Because of constraints of time and resources, we made no at-
Annals of Emergency Medicine
tempt to document clinical exposure in outside rotations such as in the community hospital, pediatric ED (at another institution), trauma unit, or off-service rotations of medicine, surgery, obstetrics and gynecology, ICU, or CCU. Within a class, residents spent a closely similar a m o u n t of time at each of the three hospitals. First- and secon&year residents spent one and two months, respectively, in each of the three EDs, while senior residents spent either three or four months at each. All residents of the same year had an equal responsibility to work shifts at any time of day. The catalog system uses ICD9-CM coding, as has been done in previous studies. 8-1o Although this system is not perfect for our purposes, it has the distinct advantage of already being in place at the three hospitals. The ICD9-CM is at times too specific in coding diagnoses. Therefore, we searched multiple codes to include all possible labels for the same condition. The ICD9-CM coding was done by two medical records clerks at Illinois Masonic Medical Center, by the attending emergency physicians (80% of charts) and one hospital billing clerk (20% of charts) at Lutheran General Hospital, and by two ED billing clerks at Mercy Hospital and Medical Center. Each clerk had taken a basic ICD9-CM course, including medical terminology, and had from one to three years' experience coding diagnoses. We compared the data base with
19:7 July 1990
TABLE 1. Proportion of each CCS area in which diagnoses did not present to the ED
No. not seen No. of diagnoses
GI Skin GU
CV
Chest
9
6
7
6
7
8
16
4
8
33
0
4
1
9
8
126
50
33
54
42
31
57
62
19
27
60
6
30
28
29
26
554
22.6
14,0
25.8
21.1
29.6
55.0
0.0
13.3
31.0
30.8
22.7
Percentage not seen 18.0 18.2 13.0 14.3
ENT Env/Drug Heme Hormone
ID
Immune
Neuro
O~ho Peds
3.6
Psych
Total
T ~ G E 2. Proportion of each CCS area in which diagnoses were encountered at least once but less than seven times GI Skin GU
CV
Chest
ENT Env/Drug Heme Hormone
ID
Immune
Neuro
O~ho Peds
Psych
Total
12
6
192
28
29
26
554
21.4
41.4
23.1
34.7
No, seen less than seven times
12
14
18
15
10
21
29
9
9
16
4
11
6
No. of diagnoses
50
33
54
42
31
57
62
19
27
60
6
30
32.3
36.8
46.8
60.0
33.3
26.7
36.7
Percentage seen less than seven times 24.0 42.4 33.3 35.7
66.7
TABLE 3. Proportion of each LOT Report diagnosis in which cases seen fall short of recommendations GI No, that fall short of LOT Report recommendations
Skin GU
CV
Chest
ENT Env/Drug Heme Hormone
ID
Immune
Neuro
Ortho Peds
Psych
Total
7
10
12
6
6
17
21
4
9
2
11
11
4
15
8
143
No. of diagnoses
16
17
22
13
16
36
29
8
16
12
19
20
12
24
23
283
Percentage of diagnoses
43.8 58.8 54.5 46.2
37.5
47.2
72.4
56.3
16.7
57.9
55.0
33.3
62.5
34.8
50.5
two different measures of a complete clinical experience. The American College of Emergency Physicians (ACEP) and The American Board of Emergency Medicine (ABEM) have set forth a CCS that lists 554 clinical entities that should be covered in an emergency medicine residency. 1t No numbers of patient encounters for a given diagnosis are contained in this document; it suggests only that the clinical topic should be included in the curriculum. It is divided into 20 categories, 15 of them clinical, and for the most part each is centered about an organ system. The clinical categories are s h o w n in Figure 1. Some of the 554 diagnostic categories are too broad to provide a full accounting of resident work. We chose to report only the 554 diagnostic categories from the CCS. The second measure of a complete clinical training to which we compared the data base is the Length of Training Report (LOT Report) developed by ACEP in 1983 to address the 19:7 July 1990
50.0
issue of expanding emergency medic i n e r e s i d e n c y f r o m 24 to 36 months, lz It contains 283 tasks with which a resident should have experience (eg, " p e r f o r m , " " e v a l u a t e , " "compare and contrast, .... differentiate, .... recognize," etc), along with the number of patient contacts that are t h o u g h t necessary to achieve each skill. The report contains approximately 100 more categories, but these had no patient-contact recommendations. We realized that some of the clinical entities contained in the CCS and the LOT Report would not be routinely written on the ED record as diagnoses. Yet, to obtain a complete accounting of experience with the CCS and LOT Report diagnoses, all diagnoses written on the chart needed to be matched with the m o s t appropriate clinical category in the documents. One of the authors judged where and whether the 2,220 ED diagnoses contained in the data base related to the 554 CCS clinical entities Annals of Emergency Medicine
and the 283 LOT Report categories. We used a liberal pairing philosophy of ED record diagnoses with categories in the documents so that resident experience would be best represented in terms of the CCS and LOT Reports. Furthermore, many CCS diagnoses, such as "anorexia," "crisis intervention," or "chills" would never be listed as diagnoses on the ED record. In these cases, we did not report the diagnoses as ones rarely or never seen in Tables 1-3 and Figure 2. There was no confusion concerning where in the LOT Report or CCS to place diagnoses of pediatric pa tients. Diseases confined predomi~ nantly to children, such as neuroblastoma, are listed under pediatrics, while diseases that occur both in adults and children are listed by organ system. For example, a fracture in a 6-year-old would be coded under orthopedics, because there are no orthopedic diagnoses under pediatrics in the LOT Report or the CCS. 766/65
RESIDENT EXPERIENCE Langdorf, Strange & Macneil
No. not seen or seen less than seven times
21
No. of diagnoses 50 Percentage of diagnoses 42.0 100
20
25
21
17
29
45
13
17
49
4
15
7
21
14
318
33 60.6
54 46.3
42 50.0
31 54.8
57 50.1
62 72.6
19 68.4
27 63.0
60 81.7
6 66.7
30 50.0
28 25.0
29 72.4
26 53.8
554 57.4
S k
Gu
Cv
C h
E n
E n
H e
H 0
1 D
I m
N e
O r
P e
P s
Total
i
e
t
v
m
r
m
u
t
d
y
n
s
/
e
rn
u
r
h
s
c
t
D r
o n
n e
o
o
u g
e
75
50
25
0 Gi
FIGURE 2. Proportion of each CCS area in which diagnoses were either not encountered (solid areas) or were seen less than seven t i m e s (shaded areas). This is a summation of Tables 1 and 2.
Approximately 90% of the charts were catalogued during the ninemonth period, the remainder falling victim to noncompliance.
RESULTS The residents saw 20,856 patients during the n i n e - m o n t h period. Because some of the patients had more than one diagnosis entered on the ED chart, the total number of diagnoses was 25,959. If more than one resident saw a patient, eg, a senior resident supervising the work of an intern, both received credit for that diagnosis. Overall, there were 2,220 different ED diagnoses. Table 4 describes the number of shifts and months of work in the ED for which we collected data. Because of residents' outside rotations, we did not collect nine months of data on any one resident. As residents progress in their training, they spend a greater proportion of time working in the ED. Therefore, we collected a 66/767
greater number of months of data for third-year residents (senior residents) than for second-year (junior residents) or first-year residents (interns). Senior residents work fewer shifts per month than their junior counterparts due to the seniors' nonclinical responsibilities. This accounts for the parity between juniors and seniors w i t h respect to n u m b e r of shifts, despite the disparity in the number of months. Overall the data base c o n t a i n s 2,151.6 ten-hour shifts of clinical experience data. The total number of shifts that an average resident would work in the ED during the three-year residency is 306. Therefore, the data base contains almost exactly the aggregate clinical experience of a hypothetical group of seven residents during a complete residency. This number seven becomes important when we e x a m i n e the f r e q u e n c y w i t h which diagnoses from the CCS present to the ED. If a given diagnosis is listed in the data base fewer than seven times, it would be impossible for each of the hypothetical seven residents to see even one case during an entire residency. As shown in Table 4, there is considerable variability between residents in the number of patients seen A n n a l s of E m e r g e n c y M e d i c i n e
h
per shift, both across all residents and within a given year of training. Among senior residents, the fastest saw 15.91 patients per shift while the slowest saw 8.18 patients per shift. A m o n g junior residents, p a t i e n t s seen r a n g e d f r o m 4.91 to 10.59. Among interns, patients seen ranged from 3.78 to 9.11. Overall, there was more than a fourfold difference between the speed at which the slowest resident saw patients and that at which the fastest did. This great difference might be expected from differences in clinical skill across years of training. But even within a given year, the fastest resident saw 1.9 to 2.4 times as m a n y patients as the slowest. Not only did we find wide variation in the total number of patients seen, but we found even greater variability when we examined individual diagnoses. Table 5 shows wide variation in the 18 diagnoses chosen for their importance toward training an emergency physician. This degree of variation permeated the entire data base. For individual diagnoses, some residents saw ten times as m a n y cases as others. There were 126 CCS diagnoses (available as an appendix from the authors) for which no patients were 19:7 July 1990
TABLE 4. Scope of the clinical experience data base and variability between residents in amount of experience
All Residents
First-Year Residents
Second-Year Residents
Third-Year Residents
No. of residents
33
8
12
13
Months of ED work for which patient data were collected (mean)
4.8
2.3
4.7
6.5
36.2 _+ 13.4
75.8 ___ 15.4
73.4
7.27 _+ 2.11
8.54 _+ 1.42
11.86 _+ 2.33
Shifts of patient data collected (ten-hour) Total shifts of data collected Number of patients seen per shift (mean +_ SD) Range of patients seen per shift Least Most Fold difference between least and most
65.2
_+ 22.5
_+ 15.4
2,151.6* 9.54 +_ 2.75 3.78 15.91
3.78 9.11
4.91 10.59
8.18 15.91
4.21
2.41
2.16
1.94
*Seven residency equivalents of data.
seen. Table 1 breaks down these 126 diagnoses by o r g a n s y s t e m and expresses t h e m as a percentage of the total n u m b e r of CCS diagnoses, 554. All totaled, none of the 33 residents saw a single case of 22.7% of the diagnoses that ACEP and ABEM state should be included in the emergency m e d i c i n e curriculum. Because the data base contains the e q u i v a l e n t of a c o m p l e t e t h r e e - y e a r r e s i d e n c y for s e v e n r e s i d e n t s , it is i n s t r u c t i v e to e x a m i n e w h i c h diagnoses presented less than seven times. Even a s s u m i n g t h a t c l i n i c a l experience could be u n i f o r m among residents, if a diagnosis does n o t pres e n t s e v e n t i m e s in t h e d a t a base, then it w o u l d be v i r t u a l l y impossible for each resident to see even one case during an entire residency. Table 2 s u m m a r i z e s the diagnoses seen less t h a n seven t i m e s and expresses t h e m as a percentage of the total n u m b e r of diagnoses in each organ system. Overall, 192 of the 554 CCS diagnoses (34.7%) were seen too i n f r e q u e n t l y for each r e s i d e n t to be assured t h e c h a n c e to see even one case. (A full list of these diagnoses is available from the authors.) If w e c o m b i n e t h o s e C C S diagnoses t h a t were either not seen at all or s e e n so i n f r e q u e n t l y t h a t e v e r y resident could not possibly manage even one case during the residency, w e see t h a t 318 of 554 d i a g n o s e s (57.4%) fall into this category. Figure I9:7 July 1990
2 presents these results graphically. W h e n we compare the data base to t h e L O T Report, w e find s i m i l a r l y that, for m a n y diagnoses, resident experience falls far short of t h a t w h i c h ACEP r e c o m m e n d s . Table 6 s u m m a rizes these diagnoses with which m o s t r e s i d e n t s h a d no e x p e r i e n c e , l i s t i n g t h e n u m b e r , of 33, w h o did not see a p a t i e n t in this category. Alt h o u g h t h e r e are s o m e s e e m i n g l y c o m m o n entities here, the data base nevertheless shows a p a u c i t y of resident experience. We w a n t e d to ascertain for w h i c h of t h e 283 L O T R e p o r t c a t e g o r i e s was it i m p o s s i b l e for enough cases to p r e s e n t to fulfill the r e c o m m e n d e d e x p e r i e n c e for each resident. To do this, w e m a d e the idealized assumption t h a t pathology w o u l d be evenly spread among residents w h e n it did present. R e m e m b e r i n g that the data base represents enough shifts to provide a full r e s i d e n c y to seven resid e n t s , w e d i v i d e d t h e n u m b e r of cases contained in the data base for each diagnosis by the n u m b e r recomm e n d e d in the LOT Report for t h a t diagnosis, d i v i d e d by seven. If t h a t n u m b e r fell short of one, t h e n there c o u l d n o t p o s s i b l y be sufficient pathology available to fulfill LOT Report r e c o m m e n d a t i o n s . Table 3 shows h o w often this occurs for the 15 broad organ systems (the same 15 as used in the CCS, b u t t h e i n d i v i d u a l diagnoses s o m e w h a t Annals of Emergency Medicine
different). Overall, for m o r e than half (50.5%) of the LOT Report diagnoses or actions, residents do n o t manage enough cases in the ED to m e e t current r e c o m m e n d a t i o n s .
DISCUSSION Only one previously published w o r k addresses the issue of inter-resi d e n t v a r i a b i l i t y in c l i n i c a l e x p e r i ence. Lamb et al followed a group of p e d i a t r i c i n t e r n s t h r o u g h t h e i r ED experience. 13 T h e y found t h a t the average n u m b e r of patients seen varied by m o r e t h a n 42%, f r o m a l o w of 17.6 to a h i g h of 25.1 p a t i e n t s p e r shift. Moreover, w h e n a few individual diagnoses were compared, the n u m b e r of cases of pharyngitis varied 73%, " a c c i d e n t s " 3 5 % , a n d h e m o p h i l i a 39%. T h e n u m b e r of l u m b a r p u n c t u r e s done ranged from four to 26. Of course, some of the v a r i a b i l i t y m a y be due to differing labels placed on children w i t h the same conditions or to different thresholds for performing d i a g n o s t i c tests. N e v e r t h e l e s s , this information concurs with our findings of inter-resident variability. Two previous a t t e m p t s have been m a d e to catalog emergency m e d i c i n e resident clinical experience. Goldm a n set up a m a n u a l log at A k r o n C i t y H o s p i t a l in 1984. ~4 C u r r e n t l y , residents are required to s u b m i t daily p a t i e n t lists, and this i n f o r m a t i o n is used to s u p p o r t d i s c i p l i n a r y a c t i o n for residents w h o do not perform ade768/67
RESIDENT EXPERIENCE Langdorf, Strange & Macneil
TABLE 5. Variability in number of cases seen per resident for 18 exemplary diagnoses Patients Seen Diagnosis
Range
Year
Mean + SD
High
Chronic obstructive pulmonary disease
2 3
5.3 _+ 2.3 5.8 _+ 3.7
9 11
1 1
Atrial fibrillation/flutter
2 3
2.4 _+ 1.0 2.3 _+ 1.6
4 6
0 0
Congestive heart failure
2 3
14.3 _+ 3.0 11.1 _+ 4.5
20 19
10 5
Chest pain Rule-out myocardial infarction
2 3
21.2 _+ 6.4 21.2 _+ 8.4
34 36
12 9
Syncope
2 3
8.8 _+ 3.5 9.5 _+ 4.5
15 21
2 4
©titis media
2 3
19.6 _+ 9.5 19.8 _+ 10.5
36 37
6 5
Vertigo/dizziness
2 3
3.6 _+ 1.7 4.8 -+ 3.5
6 12
0 2
Blunt trauma (contusions)
2 3
39.8 -+ t l . 9 69.1 -+ 16.2
72 90
29 44
Burns
2 3
3.3 -+ 2.3 6.2 _+ 3.2
7 13
0 3
Multiple trauma
2 3
6.8 _+ 3.3 4.8 +_ 3.5
14 12
2 0
Abdominal pain
2 3
21.4 _+ 8.6 36.9 _+ 11.9
38 59
7 15
Ectopic pregnancy
2 3
1.2 + 1.0 1.8 _+ 2.0
3 6
0 0
Diabetes mellitus
2 3
5.3 _+ 3.1 5.8 _+ 3.6
11 12
2 1
Brain injury
2 3
7.6 _+ 3.0 13.4 _+ 10.1
12 44
4 4
Headache
2 3
7.3 _+ 3.3 12.1 _+ 4.1
12 19
1 6
Dislocation/subluxation
2 3
1.8 _+ 1.5 2.7 _+ 1.5
4 5
0 0
Fractures
2 3
22.3 _+ 10.2 32.8 _+ 9.3
38 48
8 9
Lacerations
2 3
53,0 + 11.8 78.8 _+ 31.0
73 137
37 30
quately. Unfortunately, no systematic analysis of the data has been performed to obtain a profile of individual resident experience. Furthermore, the clinical curriculum has not been altered in response to these data. is An Emergency Department Critical Care Registry has been implemented at Hennepin County Medical Center in Minneapolis. 16 Its purpose is quite similar to this project in that it m o n i t o r s resident clinical exposure, as well as provides a data base from which to conduct retrospective 68/769
clinical research. However, it only records critical patients, not the total clinical spectrum, and no data have been published addressing the interresident variability or rare-pathology issues. Other specialties, especially family practice, have been m u c h more aggressive in documenting resident experience. T h e i r m o t i v a t i o n s t e m s from the necessity of providing this i n f o r m a t i o n to hospital credentials boards in order for their graduates to obtain staff privilegesJ 7 The AmeriAnnals of Emergency Medicine
Low
can Academy of Family Practice developed a manual index-card system for recording patient and diagnosis information in 1978, TM and, by 1981, 82% of programs surveyed reported s o m e f o r m of resident experience documentation. 17 A 1982 survey revealed that 84% of the 354 family practice residencies either had a computer catalog system in place or were developing one. Eighty-five percent of those with the computer system o p e r a t i o n a l w e r e c o l l e c t i n g diagnoses, and most reviewed these data 19:7 July 1990
TABLE 6. Diagnoses from the L O T Report where m o s t residents did not see even a single case Not Seen By Residents N = 33
Meningococcemia/meningitis Congenital heart disease Noncardiac pulmonary edema Cardiogenic shock Testicular torsion Ethylene glycol poisoning Addisonian crisis Reye's syndrome Compartment syndrome Pericardial tamponade Anaphylactic shock Pre-eclampsia/eclampsia Rhabdomyolosis Sudden infant death syndrome Septic shock Tricyclic antidepressant overdose Glaucoma Heat illness Intussusception Hyperthyroidism/thyroid storm Drowning Complications of transfusion Ascending cholangitis Cerebral aneurysm/arteriovenous malformation Cocaine intoxication Kidney/bladder/urethra injury Cavernous sinus thrombosis Sucking chest wound Supraventricular tachycardia Epiglottitis Salicylate poisoning Spinal cord injuries Atrioventricular block Encephalitis Acetaminophen overdose Diabetic ketoacidosis Electrical injuries
with the residents. 19 Unfortunately, published reports 2o-2~ again have not dealt with the issue of inter-resident variability. Furthermore, no mention was made in these reports of deficiencies in pathology that could preclude a complete training.
19:7 July 1990
32 31 31 31 31 31 31 31 31 31 31 31 31 31 31 31 31 31 3O 30 29 29 29 29 29 29 28 28 28 28 28 27 27 26 25 25 24
As with any complicated system, this computer catalog has many potential sources of error as a measuring device. We have attempted to correct these as m u c h as possible, and we believe that this report is an accurate representation of resident Annals of Emergency Medicine
e x p e r i e n c e in the ED over n i n e m o n t h s in a large urban/suburban emergency medicine program. We assured compliance by residents in placing their code number on the chart of every ED patient they saw. O n e of the i n v e s t i g a t o r s checked every chart from two of the three hospitals (Lutheran General Hospital and Illinois Masonic Medical Center) for resident signatures and added code numbers when necessary. Toward the end of the data collection period, these additions became less frequent as the residents became used to recording their numbers. The data collection began four months into the academic year at the third hospital (Mercy Hospital and Medical Center), w h e n compliance with coding was already good. A comparison between patients per shift seen at Illinois Masonic Medical Center and Mercy Hospital and Medical Center for all 33 residents revealed closely similar results, indicating that compliance with coding was as good at Mercy Hospital and Medical Center as it was at the hospitals where every chart was checked. Because the clerks who did the coding at all three hospitals did so as part of the billing function of the hospital, it is unlikely that a significant number of incorrect codes was recorded. Social scientists have long known that the very fact that people are known to be under study can alter their behavior. Could residents have begun to write down more diagnoses on the ED record than really existed in order to look faster? After reviewing approximately 2,000 charts, we did n o t find obvious examples of this. F u r t h e r m o r e , w r i t i n g d o w n more diagnoses would not have altered a resident's overall patients~pershift outcome, which would be the most desirable of the measured variables. On the positive side, perhaps the knowledge that diagnoses were being catalogued encouraged residents to write down diagnoses that were present (eg, laceration to face in the multiple trauma patient) but would not ordinarily have been recorded, giving us a more complete record of resident experience. It is impossible to discern which of these effects, if either, would predominate. Before we address the significance
770/69
RESIDENT EXPERIENCE Langdorf, Strange & Macneil
of resident experience falling so far short of what ACEP and ABEM consider optimal, it is important to point out what the computer catalog did n o t collect. First, it represents the training setting of only 19 of the 34 months spent in clinical work during residency. Experience gained on the medical and surgical wards was not represented. We acknowledge that a great deal of learning can take place on the wards, but the situations tend not to be emergent, and the supervision is generally not by faculty. The experience, again not in the computer catalog, gained in the intensive and cardiac care units is certainly much more acute in nature, but the lack of immediate resource to faculty, at least in our institutions, remains. This is not to imply that off-service rotations are less valuable than ED experience. In fact, it is possible that some or many of the deficiencies identified for ED experience could be provided during nonED rotations. This study demonstrates that we must either rely extensively on clinical experience gained outside the ED or we must teach the case m a n a g e m e n t of m a n y c o n d i t i o n s some other way. Pediatrics is clearly under-represented in the catalog. Emergency m e d i c i n e residents see pediatric cases without the aid of pediatricians in two of the three hospitals (and after midnight at the third), but a great deal of exposure occurs during a twomonth pediatric emergency medicine rotation at a hospital that did not participate in the computer catalog. We did not collect equal amounts of data for all four seasons. Twentyfour calendar months of data from three hospitals were comprised of nine winter months, six months each of spring and fall, and only three summer months. Therefore, a clinical entity with seasonal predilection to summer (eg, hyperthermia, drowning) would be theoretically underrepresented in the data base. There is no reason why the results of this study should be generalizable to other emergency medicine residencies. The pathology that presents to another program's hospitals would be inherently different, depending on the social environment adjacent to the ED, the a v a i l a b i l i t y of o t h e r sources of medical care, the emergency medical services system, and 70/771
the presence or absence of trauma center designation, among others. Furthermore, geography and weather can play an important part in determ i n i n g pathology. This data base would be expected to contain, for example, more lightning injuries and fewer snakebites than a similar tracking system based in Southern California. We believe, therefore, that it is important for each residency to monitor its own pathology in order to identify deficiencies in clinical experience. One could make an argument that the reason there is such wide variability between residents, or that some diagnoses apparently never present, is that certain residents, or even the entire residency, see the diagnosis but fail to recognize it. Perhaps patients with thyroid storm are incorrectly diagnosed as sepsis over and over again. There are two responses to this possibility. First, it is unlikely that, with 24-hour faculty supervision, residents could routinely repeat the same errors unnoticed. Second, if they do, and an important clinical entity is found by this system to present very rarely or not at all, these diagnoses are precisely the ones that we would target for teaching to residents by simulation. Following a session with a simulated patient, the resident would then not misdiagnose the patient on next presentation. Because the purpose of this catalog is to improve the educational experience, it does not particularly matter if the diagnosis never presents, or presents and is not recognized, so long as the education concerning that diagnosis takes place. It would be optimal to compare admitting diagnoses with final diagnoses as a measure of quality control for the department. However, the logistics of such a comparison were too difficult for this present study. A possible explanation for the large gaps in pathology identified here is the situation in which, shortly after presentation to the ED, a patient obviously is ill enough to warrant admission, but a diagnosis is not yet evident. It is common to make an educated guess as to the diagnosis in order to set in motion the admitting process. The risk arises that the "preliminary ED diagnosis" might not get replaced with the "definitive ED diagnosis." For example, "syncopal epiAnnals of Emergency Medicine
sode" might remain as the admitting diagnosis, even though the patient develops v e n t r i c u l a r t a c h y c a r d i a while in the ED. Then the resident managing the case fails to get credit for the more specific diagnosis. We admit that this situation occurs infrequently, and might make some of our rare specific diagnoses a bit more c o m m o n . However, this does not change the overall a r g u m e n t that monitoring resident cases is useful and necessary. By using a generous pairing system for ED record diagnoses as they relate to LOT Report and CCS diagnoses, we tried to avoid the criticism that a restrictive pairing was responsible for falsely showing that there are large gaps in available pathology. Whenever an ED record diagnosis could reasonably fit into a category in the documents, it was assigned there to more completely reflect resident activities. Overall, we believe that the conversion of ED record diagnoses to diagnoses contained in the CCS and LOT Reports is valid. Our findings suggest that ostensibly common conditions are not so common, even if this is contrary to intuition. The data clearly show that there is wide variability between residents in the number and types of diagnoses seen, confirming our first hypothesis. There are four possible reasons for this. First, some residents are inherently more skillful and make decisions faster and with more confidence, thereby seeing more patients. Second, some residents may choose to see more critically ill patients who take longer to manage, and therefore the resident might appear to be slow. No measure of the severity of illness of the residents' patients was included in this computer catalog. Third, there are some important variations in the types of pathology that present to the three hospitals. For example, Lutheran General Hospital, which is located in an uppermiddle-class suburb, sees significant heart disease, but very rarely any pelvic inflammatory disease or sickle cell crises. Mercy Hospital and Medical Center, located in a poor area, does see these latter two entities, as well as some penetrating trauma. Illinois Masonic Medical Center sees many more AIDS-related cases than do the other two hospitals. In the residents' second year, time is divided equally among all three hospitals, 19:7 July 1990
but during the senior year, chief residentS spend m o r e of their t i m e at one of the three. This alters the available pathology for t h a t y e a r and can account for part of the variability during the third year. However, referring again to Table 5, there is no great difference in the range of patients seen with a given diagnosis b e t w e e n second- and third-year residents. Fourth, and w e believe, m o s t important, t h e r e is the u n c o n t r o l l a b l e element of chance which, especially with u n c o m m o n diagnoses, can provide a few residents w i t h first-hand clinical e x p e r i e n c e w h i l e depriving most others. The reasons behind the variability are n o t v e r y i m p o r t a n t , a l t h o u g h identifying those residents who clearly c a n n o t h a n d l e a t y p i c a l patient load is useful. T h e true value of this catalog is to identify deficiencies i n i n d i v i d u a l and c o l l e c t i v e experience in order to change the curriculum accordingly, and to guide supplem e n t a t i o n of t h e s e d e f i c i e n c i e s through case simulation. Residency directors have long known t h a t there could not possibly be e n o u g h c a s e s of e p i g l o t t i t i s or meningococcemia so that every resident gains first-hand experience. But these data are alarming in that they show t h a t 22.7% (126 of 554) of the diagnoses c o n t a i n e d in t h e CCS for emergency m e d i c i n e did not present even once d u r i n g the e q u i v a l e n t of seven c o m p l e t e residencies. Furthermore, an a d d i t i o n a l i92 d i a g n o s e s (34.7%) p r e s e n t e d less t h a n s e v e n times, m a k i n g it v i r t u a l l y impossible for e v e r y r e s i d e n t to see e v e n one case. Overall, it appears that 58% of the CCS diagnoses w i l l n o t p r e s e n t to the ED often enough to provide faculty-supervised case m a n a g e m e n t to the residents. A n o t h e r w a y to assess the p r o b l e m of absent pathology is to compare the reality of a c t u a l n u m b e r of cases seen with t h e ideal. T h i s a d d r e s s e s t h e third of the four essential questions posed i n t h e i n t r o d u c t i o n , w h i c h needs an answer in order to provide each resident w i t h a c o m p l e t e clinical experience. The L O T Report 12 is the o n l y d o c u m e n t t h a t a d d r e s s e s what w o u l d be t h e o p t i m u m quantity of experience for a resident. Unfortunately, its r e c o m m e n d a t i o n s are not based on hard data, but rather on the opinions of recognized educators in e m e r g e n c y m e d i c i n e . A l t h o u g h 19:7 July 1990
this d o c u m e n t uses the s a m e structure as the CCS, it c o m b i n e s the 554 diagnoses i n t o 283 broader diagnostic categories and then assigns a recomm e n d e d n u m b e r of c a s e s to e a c h . Again, the data s h o w that actual exp e r i e n c e f a l l s far s h o r t ; o n l y i n 49.5% (140 of 283) of diagnostic categories did r e s i d e n t s m e e t or exceed the LOT Report recommendations. Clearly, our second hypothesis is also true - s o m e clinical entities do present so rarely as to m a k e it u n l i k e l y t h a t each r e s i d e n t w o u l d e n c o u n t e r t h e m in the ED frequently enough to p r o m o t e expert m a n a g e m e n t . D e s p i t e t h e a p p a r e n t a b s e n c e of m a n y conditions from the cases that p r e s e n t to t h e s t u d y EDs, t h e resid e n t s p r o b a b l y do l e a r n a b o u t t h e r e c o g n i t i o n of t h e s e c o n d i t i o n s through discussions of differential diagnosis w i t h m o r e senior emergency physicians. We agree that a resident can l e a r n m u c h a b o u t a d i a g n o s i s w i t h o u t first-hand case m a n a g e m e n t . H o w e v e r , c o n s i d e r a t i o n of a diagn o s i s is o n l y part of t h e process of p a t i e n t m a n a g e m e n t . Unless t h e pat i e n t actually has the disease, appropriate t h e r a p y is less l i k e l y to be discussed. T h e r e f o r e , w e b e l i e v e (but have not s h o w n here) t h a t actual experience w i t h a clinical disease is the o p t i m a l w a y to teach m a n a g e m e n t . It is therefore useful to m o n i t o r the residents' actual p a t i e n t contact in the c l i n i c a l a r e n a , i n a d d i t i o n to fostering learning through discussions of differential diagnosis. One could argue that a m o n i t o r i n g s y s t e m s u c h as this is u n n e c e s s a r y , evidenced b y the thousands of competent, board-certified emergency physicians w h o are n o w in practice b u t never had the benefit of such a system. We agree t h a t it is possible to train an e m e r g e n c y physician w i t h the c u r r e n t c u r r i c u l u m d e s p i t e the l a c k of personal experience w i t h case m a n a g e m e n t , as t h e p r o g r a m u n d e r s t u d y has done for a l m o s t ten years. However, each e m e r g e n c y physician can t h i n k back to the t i m e w h e n he completed training and remember areas of w e a k n e s s in k n o w l e d g e or m a n a g e m e n t w i t h at l e a s t s e v e r a l clinical entities. O v e r y e a r s of p r a c t i c e , w e h a v e shored up o u r a b i l i t y to h a n d l e the u n c o m m o n presentation, but we susp e c t t h e s e leaps of k n o w l e d g e were sparked by seeing a p a t i e n t and experiencing a bit of panic as we realized Annals of Emergency Medicine
t h a t w e w e r e u n p r e p a r e d . T h e authors seek to begin this m a t u r a t i o n process during residency so t h a t graduates will be better prepared for independent practice. The current m e t h o d of t r a i n i n g is good, b u t a m o n i t o r i n g system, w i t h appropriate s u p p l e m e n t a t i o n of cases, has great potential to m a k e it better. CONCLUSION T h e results of this study s h o w that there is great variability in the n u m bers and k i n d s of patients that emergency m e d i c i n e residents see in the ED and t h a t there are large gaps in the available ED pathology of even a good residency. We r e c o m m e n d that e a c h p r o g r a m set up a m o n i t o r i n g s y s t e m to identify deficiencies in ind i v i d u a l a n d g r o u p e x p e r i e n c e and s u p p l e m e n t t r a i n i n g w i t h case simulations. The authors thank the residents of the University of Illinois Affiliated Hospitals Emergency Medicine Residency, Society of Teachers of Emergency Medicine/ Emergency Medicine Foundation, and Richard Foley, Phi), for invaluable help. This article is dedicated to Harold Jayne, MD, FACEP, in memoriam.
REFERENCES
1. Guze SB, Myers JD: The quality of graduate medical education, in graduate medical education: Proposals for the eighties. J Med Educ 1981;56:21-36. 2. Podgorny G: Graduate education in eme> gency medicine: Duration of training. Ann Emerg Med 1982;11:592-593. 3. Irby DM: Clinical teaching and the clinical teacher. J Med Educ 1986;61(9 part 2):35-45. 4. Dailey RH: Residency essentials: Quantify to assure quality. A n n Emerg Med 1985;14: 928-929. 5. Frumkin K: The future of emergency medicine residency training. A n n Emerg Med 1985;14:378-379. 6. white JD: Defining emergency medicine residency training (letter). Ann Emerg Med 1986; 15:872. 7. US Department of Health and Human Serk vices, Public Health Service - Health Care Fi nancing Administration: International Classification of Diseases, 9th Revision, Clinical Modification. Washington, DC, US Government
Printing Office, 1980. 8. Smith MV, Freedman AM, Schwartz RW, et . al: A microcomputerized case log for surgical residents. Curt Surg 1984;41:371-373. 9. Anastasio GD, White TR, Fries JC: Comput erized prescription inventory program for the education of residents (PIPER). J Faro Pratt 1986;23:598-600. 10. Quattlebaum TG: Microcomputer analysis and management of residency training experi772/71
RESIDENT EXPERIENCE Langdorf, Strange & Macneil
ences. C o m p u t M e t h o d s Programs B i o m e d 1985;20:169-172. 11. Hamilton GC, Lumpkin JR, Tomlanovich MC, et al: Special Committee on the Core Content Revision: Emergency medicine core content. Ann Emerg Med 1986;15:853-862. 12. Lumpkin JR: Length of training in emergency medicine. Unpublished report. American College of Emergency Physicians, December 9, 1983. 13. Lamb GA, Weinberger HL, Schneiderman H, et al: Systematic utilization of data for analysis of a pediatric emergency-room experience. Pediatrics 1975;55:266-274.
14. Goldman GE: The need for some conformity in emergency medicine? Ann Emerg Med 1984;13:212.
19. Lutz L, Green L: Use of computer systems in family practice residencies. J Faro Praet 1982;14:369-375.
15. Goldman GE, personal communication, 1987.
20. Schneeweiss R: Diagnostic clusters: A new approach for reporting the diagnostic content of family practice residents' ambulatory experiences. J Faro Praet 1985;20:487-492.
16. Plummer D, Clinton J, Ruiz E: Emergency department critical care registry (abstract). Ann Emerg IVied 1987;16:502. 17. Curtis P, Resnick J, Greganti MA: Documentation of inpatient experiences of resident physicians. J Med Educ 1985;60:408-410. 18. Resnick JC, Curtis P: Inpatient documentation for family practice residents. J Fam Pract 1982; 15: 798-801.
21. Shear CL, Wall EM: Diagnosis cluster frequency in a community-based family practice residency program. West f Med 1985;142: 854-857. 22. Meza E, Lapsys FX: Adult medicine inpatient experience: A comparison of family practice and internal medicine residency services. J Faro Pract 1981;13:701-705.
See related editorial, p 827
72/773
Annals of Emergency Medicine
19:7 July 1990
Computerized Tracking of Emergency Medicine Resident Clinical Experience Although we c o m m o n l y assume that because residents spend a given number of months in the emergency department they achieve adequate exposure to all necessary clinical entities, this has never been shown. We suspect, rather, that great variability exists among residents in the number and variety of patients they see; and that with respect to the ED, there are important diagnoses that are rare or absent in the clinical pathology of a training program. To confirm these hypotheses, we implemented a computerized system of recording patients and diagnoses managed in the ED by the 33 residents of the University of Illinois Affiliated Hospitals Emergency Medicine Residency. We collected data for nine months and accumulated 2,152 shifts of clinical experience. These data confirm our hypotheses. We found that senior residents managed an average of 11.9 + 2.3 patients per ten-hour shift, but the quickest resident saw almost twice as m a n y patients as the slowest. Junior residents saw fewer patients, 8.5 ++1.4 patients per shift, but maintained a twofold difference between the fastest and slowest. Furthermore, there are important diagnoses that present too rarely for each resident to become facile in their management. We found that 22.7% of the 554 diagnoses listed in the Emergency Medicine Core Content never once presented to the ED. An additional 34.7% of these diagnoses did present, but so rarely that each resident could not possibly manage one case during a residency. The Length of Training R& port of the American College of Emergency Physicians provides objective guidelines for the number of encounters a resident should have with 283 clinical entities. In this study, residents fell short of these guidelines with 50.5% of diagnoses. While absolute quantity of exposure does not assure competence in management, we recommend that each residency monitor the experience of its residents. This allows a residency to change its curriculum to m a k e optimum use of available pathology, as well as to supplem e n t deficiencies in clinical experience with case simulations. [Langdorf MI, Strange G, Macneil P: Computerized tracking of emergency medicine resident clinical experience. Ann Emerg Med July 1990;19:764-773.]
Mark I Langdorf, MD, MHPE* Irvine, California Gary Strange, MD, FACEPt Chicago, Illinois Philip Macneil, MD:~ New York, New York From the Emergency Medicine Residency, University of California, Irvine;* University of Illinois Affiliated Emergency Medicine Residency, Chicago;¢ and Metropolitan Hospital Emergency Medicine Residency, New York.¢ Received for publication August 7, 1989. Revision received December 19, 1989. Accepted for publication March 26, 1990. Presented at the Second International Conference on Emergency Medicine in Brisbane, Australia, October 1988. Supported in part by the 1988 Educational Methodology Grant from the Society of Teachers of Emergency Medicine and the Emergency Medicine Foundation. Address for reprints: Mark I Langdorf, MD, MHPE, University of California, Irvine, Medical Center, Division of Emergency Medicine, 101 The City Drive, Route 128, Orange, California 92668.
INTRODUCTION Graduate medical education in the United States is based on an apprenticeship model, in which a trainee spends an extended period under the tutelage of an experienced practitioner and learns his approach to clinical medicine. 1-3 The clinical situations to which the trainee is exposed depend solely on the patients who present for treatment. In times past, with this close mentor-student relationship, the practitioner had complete knowledge concerning the breadth and depth of training. Modern graduate medical education, however, has become too complex for any one person to adequately supervise the residents' training. 1 Residents work around the clock, at various sites within the hospital, and, often, at various hospitals. The clinical training in emergency medicine occurs on a random basis dependent on which patients present to the emergency department, and so uniformity of exposure to clinical medicine is difficult to achieve. No attempt has yet been made to assess the variability in clinical exposure among residents or to devise a method by which we can offer each resident a more complete clinical education. Much time and effort has gone into the development of a Core Content
19:7 Jury 1990
Annals of Emergency Medicine
764/63
RESIDENT EXPERIENCE Langdorf, Strange & Macneil
FIGURE 1. Clinical categories and
abbreviations. Statement (CSS) for emergency medicine and into the sequencing of didactic sessions to cover the material. However, residents in emergency medicine spend only 10% to 15% of their training in the didactic arena. The remainder of time is spent seeing patients under faculty supervision. It seems prudent, therefore, to put forth at least as much effort to assure an equally complete clinical exposure. If the ultimate goal in emergency medicine graduate education is to provide as thorough an education as possible for each resident, then four e s s e n t i a l q u e s t i o n s m u s t be addressed: What is the current clinical exposure of residents? What clinical exposure constitutes optimum training that fulfills established learning objectives? What are the weaknesses of the current clinical exposure, ie, how does reality fall short of the opt i m u m ? and, H o w can alternate training methods be used to rectify these deficiencies? This study was designed to address the first and third questions in this sequence for one emergency medicine residency. We hypothesized that a catalog of resident clinical experience in the ED would demonstrate that there is great variability among residents in clinical exposure and some important clinical entities present so rarely that it is unlikely that each resident would have clinical exposure. While clinical exposure alone is not sufficient to assure competence, it is a vital component of any training p r o g r a m . 4-6 As D a i l e y paraphrased Osler: "To see patients without reading about them is to go to sea without charts; but to read about disease without seeing patients is not to go to sea at all. ''4 Case management must also be supervised by experienced emergency medicine faculty so that the resident receives optimum benefit from clinical exposure in terms of bedside teaching and problem solving. METHODS A program to track the ED clinical exposure of all 33 residents in the University of Illinois Affiliated Hospitals E m e r g e n c y M e d i c i n e Resid e n c y was developed and imple-
64/765
Abdominal and gastrointestinal disorders (GI) Cardiovascular disorders (CV) Cutaneous disorders (Skin) Disorders related to the immune system (Immune) Disorders caused by biologic agents (ID) Disorders due to chemical and environmental agents (Env/drug) Hematologic disorders (Heme) Hormonal, metabolic, and nutritional disorders (Hormone/metab) Disorders of the head and neck (ENT) Disorders primarily presenting in infancy and childhood (Peds) Musculoskeletal disorders (Ortho) Nervous system disorders (Neuro) Psychobehavioral disorders (Psych) Thoracic-respiratory disorders (Chest) Urogenital disorders (GU) m e n t e d during the academic year 1987-88. There were eight first-, 12 second-, and 13 third-year residents. All of the residents came directly from medical school. All patients seen by residents during their emergency medicine rotations from July 1987 through March 1988 at the three participating hospitals, Mercy Hospital and Medical Center, Illinois Masonic Medical Center, both in Chicago, and Lutheran General Hospital in suburban Park Ridge, had data entered in the computer catalog by data processing clerks. These data consisted of a resident code number; up to three diagnostic codes taken from the International Classification of Diseases, Ninth Revision, Clinical Modification (ICD9-CM); 7 the date of visit; medical record number; and whether the patient was admitted to the hospital. We did not track any index of severity of illness. We compiled and sorted these data by both diagnosis and resident using a microcomputer with dBase III+ software. We assessed variability among residents in exposure to diagnoses and the incidence of diagnoses that are rare among the pathology of the hospitals. The three EDs had a combined yearly census of more than 100,000 patients per year, and two of the three are Level I trauma centers. Residents spent 19 of their 22 emergency medicine (86%) and of their 34 total clinical months (56%) in EDs that were served by the catalog system. Because of constraints of time and resources, we made no at-
Annals of Emergency Medicine
tempt to document clinical exposure in outside rotations such as in the community hospital, pediatric ED (at another institution), trauma unit, or off-service rotations of medicine, surgery, obstetrics and gynecology, ICU, or CCU. Within a class, residents spent a closely similar a m o u n t of time at each of the three hospitals. First- and secon&year residents spent one and two months, respectively, in each of the three EDs, while senior residents spent either three or four months at each. All residents of the same year had an equal responsibility to work shifts at any time of day. The catalog system uses ICD9-CM coding, as has been done in previous studies. 8-1o Although this system is not perfect for our purposes, it has the distinct advantage of already being in place at the three hospitals. The ICD9-CM is at times too specific in coding diagnoses. Therefore, we searched multiple codes to include all possible labels for the same condition. The ICD9-CM coding was done by two medical records clerks at Illinois Masonic Medical Center, by the attending emergency physicians (80% of charts) and one hospital billing clerk (20% of charts) at Lutheran General Hospital, and by two ED billing clerks at Mercy Hospital and Medical Center. Each clerk had taken a basic ICD9-CM course, including medical terminology, and had from one to three years' experience coding diagnoses. We compared the data base with
19:7 July 1990
TABLE 1. Proportion of each CCS area in which diagnoses did not present to the ED
No. not seen No. of diagnoses
GI Skin GU
CV
Chest
9
6
7
6
7
8
16
4
8
33
0
4
1
9
8
126
50
33
54
42
31
57
62
19
27
60
6
30
28
29
26
554
22.6
14,0
25.8
21.1
29.6
55.0
0.0
13.3
31.0
30.8
22.7
Percentage not seen 18.0 18.2 13.0 14.3
ENT Env/Drug Heme Hormone
ID
Immune
Neuro
O~ho Peds
3.6
Psych
Total
T ~ G E 2. Proportion of each CCS area in which diagnoses were encountered at least once but less than seven times GI Skin GU
CV
Chest
ENT Env/Drug Heme Hormone
ID
Immune
Neuro
O~ho Peds
Psych
Total
12
6
192
28
29
26
554
21.4
41.4
23.1
34.7
No, seen less than seven times
12
14
18
15
10
21
29
9
9
16
4
11
6
No. of diagnoses
50
33
54
42
31
57
62
19
27
60
6
30
32.3
36.8
46.8
60.0
33.3
26.7
36.7
Percentage seen less than seven times 24.0 42.4 33.3 35.7
66.7
TABLE 3. Proportion of each LOT Report diagnosis in which cases seen fall short of recommendations GI No, that fall short of LOT Report recommendations
Skin GU
CV
Chest
ENT Env/Drug Heme Hormone
ID
Immune
Neuro
Ortho Peds
Psych
Total
7
10
12
6
6
17
21
4
9
2
11
11
4
15
8
143
No. of diagnoses
16
17
22
13
16
36
29
8
16
12
19
20
12
24
23
283
Percentage of diagnoses
43.8 58.8 54.5 46.2
37.5
47.2
72.4
56.3
16.7
57.9
55.0
33.3
62.5
34.8
50.5
two different measures of a complete clinical experience. The American College of Emergency Physicians (ACEP) and The American Board of Emergency Medicine (ABEM) have set forth a CCS that lists 554 clinical entities that should be covered in an emergency medicine residency. 1t No numbers of patient encounters for a given diagnosis are contained in this document; it suggests only that the clinical topic should be included in the curriculum. It is divided into 20 categories, 15 of them clinical, and for the most part each is centered about an organ system. The clinical categories are s h o w n in Figure 1. Some of the 554 diagnostic categories are too broad to provide a full accounting of resident work. We chose to report only the 554 diagnostic categories from the CCS. The second measure of a complete clinical training to which we compared the data base is the Length of Training Report (LOT Report) developed by ACEP in 1983 to address the 19:7 July 1990
50.0
issue of expanding emergency medic i n e r e s i d e n c y f r o m 24 to 36 months, lz It contains 283 tasks with which a resident should have experience (eg, " p e r f o r m , " " e v a l u a t e , " "compare and contrast, .... differentiate, .... recognize," etc), along with the number of patient contacts that are t h o u g h t necessary to achieve each skill. The report contains approximately 100 more categories, but these had no patient-contact recommendations. We realized that some of the clinical entities contained in the CCS and the LOT Report would not be routinely written on the ED record as diagnoses. Yet, to obtain a complete accounting of experience with the CCS and LOT Report diagnoses, all diagnoses written on the chart needed to be matched with the m o s t appropriate clinical category in the documents. One of the authors judged where and whether the 2,220 ED diagnoses contained in the data base related to the 554 CCS clinical entities Annals of Emergency Medicine
and the 283 LOT Report categories. We used a liberal pairing philosophy of ED record diagnoses with categories in the documents so that resident experience would be best represented in terms of the CCS and LOT Reports. Furthermore, many CCS diagnoses, such as "anorexia," "crisis intervention," or "chills" would never be listed as diagnoses on the ED record. In these cases, we did not report the diagnoses as ones rarely or never seen in Tables 1-3 and Figure 2. There was no confusion concerning where in the LOT Report or CCS to place diagnoses of pediatric pa tients. Diseases confined predomi~ nantly to children, such as neuroblastoma, are listed under pediatrics, while diseases that occur both in adults and children are listed by organ system. For example, a fracture in a 6-year-old would be coded under orthopedics, because there are no orthopedic diagnoses under pediatrics in the LOT Report or the CCS. 766/65
RESIDENT EXPERIENCE Langdorf, Strange & Macneil
No. not seen or seen less than seven times
21
No. of diagnoses 50 Percentage of diagnoses 42.0 100
20
25
21
17
29
45
13
17
49
4
15
7
21
14
318
33 60.6
54 46.3
42 50.0
31 54.8
57 50.1
62 72.6
19 68.4
27 63.0
60 81.7
6 66.7
30 50.0
28 25.0
29 72.4
26 53.8
554 57.4
S k
Gu
Cv
C h
E n
E n
H e
H 0
1 D
I m
N e
O r
P e
P s
Total
i
e
t
v
m
r
m
u
t
d
y
n
s
/
e
rn
u
r
h
s
c
t
D r
o n
n e
o
o
u g
e
75
50
25
0 Gi
FIGURE 2. Proportion of each CCS area in which diagnoses were either not encountered (solid areas) or were seen less than seven t i m e s (shaded areas). This is a summation of Tables 1 and 2.
Approximately 90% of the charts were catalogued during the ninemonth period, the remainder falling victim to noncompliance.
RESULTS The residents saw 20,856 patients during the n i n e - m o n t h period. Because some of the patients had more than one diagnosis entered on the ED chart, the total number of diagnoses was 25,959. If more than one resident saw a patient, eg, a senior resident supervising the work of an intern, both received credit for that diagnosis. Overall, there were 2,220 different ED diagnoses. Table 4 describes the number of shifts and months of work in the ED for which we collected data. Because of residents' outside rotations, we did not collect nine months of data on any one resident. As residents progress in their training, they spend a greater proportion of time working in the ED. Therefore, we collected a 66/767
greater number of months of data for third-year residents (senior residents) than for second-year (junior residents) or first-year residents (interns). Senior residents work fewer shifts per month than their junior counterparts due to the seniors' nonclinical responsibilities. This accounts for the parity between juniors and seniors w i t h respect to n u m b e r of shifts, despite the disparity in the number of months. Overall the data base c o n t a i n s 2,151.6 ten-hour shifts of clinical experience data. The total number of shifts that an average resident would work in the ED during the three-year residency is 306. Therefore, the data base contains almost exactly the aggregate clinical experience of a hypothetical group of seven residents during a complete residency. This number seven becomes important when we e x a m i n e the f r e q u e n c y w i t h which diagnoses from the CCS present to the ED. If a given diagnosis is listed in the data base fewer than seven times, it would be impossible for each of the hypothetical seven residents to see even one case during an entire residency. As shown in Table 4, there is considerable variability between residents in the number of patients seen A n n a l s of E m e r g e n c y M e d i c i n e
h
per shift, both across all residents and within a given year of training. Among senior residents, the fastest saw 15.91 patients per shift while the slowest saw 8.18 patients per shift. A m o n g junior residents, p a t i e n t s seen r a n g e d f r o m 4.91 to 10.59. Among interns, patients seen ranged from 3.78 to 9.11. Overall, there was more than a fourfold difference between the speed at which the slowest resident saw patients and that at which the fastest did. This great difference might be expected from differences in clinical skill across years of training. But even within a given year, the fastest resident saw 1.9 to 2.4 times as m a n y patients as the slowest. Not only did we find wide variation in the total number of patients seen, but we found even greater variability when we examined individual diagnoses. Table 5 shows wide variation in the 18 diagnoses chosen for their importance toward training an emergency physician. This degree of variation permeated the entire data base. For individual diagnoses, some residents saw ten times as m a n y cases as others. There were 126 CCS diagnoses (available as an appendix from the authors) for which no patients were 19:7 July 1990
TABLE 4. Scope of the clinical experience data base and variability between residents in amount of experience
All Residents
First-Year Residents
Second-Year Residents
Third-Year Residents
No. of residents
33
8
12
13
Months of ED work for which patient data were collected (mean)
4.8
2.3
4.7
6.5
36.2 _+ 13.4
75.8 ___ 15.4
73.4
7.27 _+ 2.11
8.54 _+ 1.42
11.86 _+ 2.33
Shifts of patient data collected (ten-hour) Total shifts of data collected Number of patients seen per shift (mean +_ SD) Range of patients seen per shift Least Most Fold difference between least and most
65.2
_+ 22.5
_+ 15.4
2,151.6* 9.54 +_ 2.75 3.78 15.91
3.78 9.11
4.91 10.59
8.18 15.91
4.21
2.41
2.16
1.94
*Seven residency equivalents of data.
seen. Table 1 breaks down these 126 diagnoses by o r g a n s y s t e m and expresses t h e m as a percentage of the total n u m b e r of CCS diagnoses, 554. All totaled, none of the 33 residents saw a single case of 22.7% of the diagnoses that ACEP and ABEM state should be included in the emergency m e d i c i n e curriculum. Because the data base contains the e q u i v a l e n t of a c o m p l e t e t h r e e - y e a r r e s i d e n c y for s e v e n r e s i d e n t s , it is i n s t r u c t i v e to e x a m i n e w h i c h diagnoses presented less than seven times. Even a s s u m i n g t h a t c l i n i c a l experience could be u n i f o r m among residents, if a diagnosis does n o t pres e n t s e v e n t i m e s in t h e d a t a base, then it w o u l d be v i r t u a l l y impossible for each resident to see even one case during an entire residency. Table 2 s u m m a r i z e s the diagnoses seen less t h a n seven t i m e s and expresses t h e m as a percentage of the total n u m b e r of diagnoses in each organ system. Overall, 192 of the 554 CCS diagnoses (34.7%) were seen too i n f r e q u e n t l y for each r e s i d e n t to be assured t h e c h a n c e to see even one case. (A full list of these diagnoses is available from the authors.) If w e c o m b i n e t h o s e C C S diagnoses t h a t were either not seen at all or s e e n so i n f r e q u e n t l y t h a t e v e r y resident could not possibly manage even one case during the residency, w e see t h a t 318 of 554 d i a g n o s e s (57.4%) fall into this category. Figure I9:7 July 1990
2 presents these results graphically. W h e n we compare the data base to t h e L O T Report, w e find s i m i l a r l y that, for m a n y diagnoses, resident experience falls far short of t h a t w h i c h ACEP r e c o m m e n d s . Table 6 s u m m a rizes these diagnoses with which m o s t r e s i d e n t s h a d no e x p e r i e n c e , l i s t i n g t h e n u m b e r , of 33, w h o did not see a p a t i e n t in this category. Alt h o u g h t h e r e are s o m e s e e m i n g l y c o m m o n entities here, the data base nevertheless shows a p a u c i t y of resident experience. We w a n t e d to ascertain for w h i c h of t h e 283 L O T R e p o r t c a t e g o r i e s was it i m p o s s i b l e for enough cases to p r e s e n t to fulfill the r e c o m m e n d e d e x p e r i e n c e for each resident. To do this, w e m a d e the idealized assumption t h a t pathology w o u l d be evenly spread among residents w h e n it did present. R e m e m b e r i n g that the data base represents enough shifts to provide a full r e s i d e n c y to seven resid e n t s , w e d i v i d e d t h e n u m b e r of cases contained in the data base for each diagnosis by the n u m b e r recomm e n d e d in the LOT Report for t h a t diagnosis, d i v i d e d by seven. If t h a t n u m b e r fell short of one, t h e n there c o u l d n o t p o s s i b l y be sufficient pathology available to fulfill LOT Report r e c o m m e n d a t i o n s . Table 3 shows h o w often this occurs for the 15 broad organ systems (the same 15 as used in the CCS, b u t t h e i n d i v i d u a l diagnoses s o m e w h a t Annals of Emergency Medicine
different). Overall, for m o r e than half (50.5%) of the LOT Report diagnoses or actions, residents do n o t manage enough cases in the ED to m e e t current r e c o m m e n d a t i o n s .
DISCUSSION Only one previously published w o r k addresses the issue of inter-resi d e n t v a r i a b i l i t y in c l i n i c a l e x p e r i ence. Lamb et al followed a group of p e d i a t r i c i n t e r n s t h r o u g h t h e i r ED experience. 13 T h e y found t h a t the average n u m b e r of patients seen varied by m o r e t h a n 42%, f r o m a l o w of 17.6 to a h i g h of 25.1 p a t i e n t s p e r shift. Moreover, w h e n a few individual diagnoses were compared, the n u m b e r of cases of pharyngitis varied 73%, " a c c i d e n t s " 3 5 % , a n d h e m o p h i l i a 39%. T h e n u m b e r of l u m b a r p u n c t u r e s done ranged from four to 26. Of course, some of the v a r i a b i l i t y m a y be due to differing labels placed on children w i t h the same conditions or to different thresholds for performing d i a g n o s t i c tests. N e v e r t h e l e s s , this information concurs with our findings of inter-resident variability. Two previous a t t e m p t s have been m a d e to catalog emergency m e d i c i n e resident clinical experience. Goldm a n set up a m a n u a l log at A k r o n C i t y H o s p i t a l in 1984. ~4 C u r r e n t l y , residents are required to s u b m i t daily p a t i e n t lists, and this i n f o r m a t i o n is used to s u p p o r t d i s c i p l i n a r y a c t i o n for residents w h o do not perform ade768/67
RESIDENT EXPERIENCE Langdorf, Strange & Macneil
TABLE 5. Variability in number of cases seen per resident for 18 exemplary diagnoses Patients Seen Diagnosis
Range
Year
Mean + SD
High
Chronic obstructive pulmonary disease
2 3
5.3 _+ 2.3 5.8 _+ 3.7
9 11
1 1
Atrial fibrillation/flutter
2 3
2.4 _+ 1.0 2.3 _+ 1.6
4 6
0 0
Congestive heart failure
2 3
14.3 _+ 3.0 11.1 _+ 4.5
20 19
10 5
Chest pain Rule-out myocardial infarction
2 3
21.2 _+ 6.4 21.2 _+ 8.4
34 36
12 9
Syncope
2 3
8.8 _+ 3.5 9.5 _+ 4.5
15 21
2 4
©titis media
2 3
19.6 _+ 9.5 19.8 _+ 10.5
36 37
6 5
Vertigo/dizziness
2 3
3.6 _+ 1.7 4.8 -+ 3.5
6 12
0 2
Blunt trauma (contusions)
2 3
39.8 -+ t l . 9 69.1 -+ 16.2
72 90
29 44
Burns
2 3
3.3 -+ 2.3 6.2 _+ 3.2
7 13
0 3
Multiple trauma
2 3
6.8 _+ 3.3 4.8 +_ 3.5
14 12
2 0
Abdominal pain
2 3
21.4 _+ 8.6 36.9 _+ 11.9
38 59
7 15
Ectopic pregnancy
2 3
1.2 + 1.0 1.8 _+ 2.0
3 6
0 0
Diabetes mellitus
2 3
5.3 _+ 3.1 5.8 _+ 3.6
11 12
2 1
Brain injury
2 3
7.6 _+ 3.0 13.4 _+ 10.1
12 44
4 4
Headache
2 3
7.3 _+ 3.3 12.1 _+ 4.1
12 19
1 6
Dislocation/subluxation
2 3
1.8 _+ 1.5 2.7 _+ 1.5
4 5
0 0
Fractures
2 3
22.3 _+ 10.2 32.8 _+ 9.3
38 48
8 9
Lacerations
2 3
53,0 + 11.8 78.8 _+ 31.0
73 137
37 30
quately. Unfortunately, no systematic analysis of the data has been performed to obtain a profile of individual resident experience. Furthermore, the clinical curriculum has not been altered in response to these data. is An Emergency Department Critical Care Registry has been implemented at Hennepin County Medical Center in Minneapolis. 16 Its purpose is quite similar to this project in that it m o n i t o r s resident clinical exposure, as well as provides a data base from which to conduct retrospective 68/769
clinical research. However, it only records critical patients, not the total clinical spectrum, and no data have been published addressing the interresident variability or rare-pathology issues. Other specialties, especially family practice, have been m u c h more aggressive in documenting resident experience. T h e i r m o t i v a t i o n s t e m s from the necessity of providing this i n f o r m a t i o n to hospital credentials boards in order for their graduates to obtain staff privilegesJ 7 The AmeriAnnals of Emergency Medicine
Low
can Academy of Family Practice developed a manual index-card system for recording patient and diagnosis information in 1978, TM and, by 1981, 82% of programs surveyed reported s o m e f o r m of resident experience documentation. 17 A 1982 survey revealed that 84% of the 354 family practice residencies either had a computer catalog system in place or were developing one. Eighty-five percent of those with the computer system o p e r a t i o n a l w e r e c o l l e c t i n g diagnoses, and most reviewed these data 19:7 July 1990
TABLE 6. Diagnoses from the L O T Report where m o s t residents did not see even a single case Not Seen By Residents N = 33
Meningococcemia/meningitis Congenital heart disease Noncardiac pulmonary edema Cardiogenic shock Testicular torsion Ethylene glycol poisoning Addisonian crisis Reye's syndrome Compartment syndrome Pericardial tamponade Anaphylactic shock Pre-eclampsia/eclampsia Rhabdomyolosis Sudden infant death syndrome Septic shock Tricyclic antidepressant overdose Glaucoma Heat illness Intussusception Hyperthyroidism/thyroid storm Drowning Complications of transfusion Ascending cholangitis Cerebral aneurysm/arteriovenous malformation Cocaine intoxication Kidney/bladder/urethra injury Cavernous sinus thrombosis Sucking chest wound Supraventricular tachycardia Epiglottitis Salicylate poisoning Spinal cord injuries Atrioventricular block Encephalitis Acetaminophen overdose Diabetic ketoacidosis Electrical injuries
with the residents. 19 Unfortunately, published reports 2o-2~ again have not dealt with the issue of inter-resident variability. Furthermore, no mention was made in these reports of deficiencies in pathology that could preclude a complete training.
19:7 July 1990
32 31 31 31 31 31 31 31 31 31 31 31 31 31 31 31 31 31 3O 30 29 29 29 29 29 29 28 28 28 28 28 27 27 26 25 25 24
As with any complicated system, this computer catalog has many potential sources of error as a measuring device. We have attempted to correct these as m u c h as possible, and we believe that this report is an accurate representation of resident Annals of Emergency Medicine
e x p e r i e n c e in the ED over n i n e m o n t h s in a large urban/suburban emergency medicine program. We assured compliance by residents in placing their code number on the chart of every ED patient they saw. O n e of the i n v e s t i g a t o r s checked every chart from two of the three hospitals (Lutheran General Hospital and Illinois Masonic Medical Center) for resident signatures and added code numbers when necessary. Toward the end of the data collection period, these additions became less frequent as the residents became used to recording their numbers. The data collection began four months into the academic year at the third hospital (Mercy Hospital and Medical Center), w h e n compliance with coding was already good. A comparison between patients per shift seen at Illinois Masonic Medical Center and Mercy Hospital and Medical Center for all 33 residents revealed closely similar results, indicating that compliance with coding was as good at Mercy Hospital and Medical Center as it was at the hospitals where every chart was checked. Because the clerks who did the coding at all three hospitals did so as part of the billing function of the hospital, it is unlikely that a significant number of incorrect codes was recorded. Social scientists have long known that the very fact that people are known to be under study can alter their behavior. Could residents have begun to write down more diagnoses on the ED record than really existed in order to look faster? After reviewing approximately 2,000 charts, we did n o t find obvious examples of this. F u r t h e r m o r e , w r i t i n g d o w n more diagnoses would not have altered a resident's overall patients~pershift outcome, which would be the most desirable of the measured variables. On the positive side, perhaps the knowledge that diagnoses were being catalogued encouraged residents to write down diagnoses that were present (eg, laceration to face in the multiple trauma patient) but would not ordinarily have been recorded, giving us a more complete record of resident experience. It is impossible to discern which of these effects, if either, would predominate. Before we address the significance
770/69
RESIDENT EXPERIENCE Langdorf, Strange & Macneil
of resident experience falling so far short of what ACEP and ABEM consider optimal, it is important to point out what the computer catalog did n o t collect. First, it represents the training setting of only 19 of the 34 months spent in clinical work during residency. Experience gained on the medical and surgical wards was not represented. We acknowledge that a great deal of learning can take place on the wards, but the situations tend not to be emergent, and the supervision is generally not by faculty. The experience, again not in the computer catalog, gained in the intensive and cardiac care units is certainly much more acute in nature, but the lack of immediate resource to faculty, at least in our institutions, remains. This is not to imply that off-service rotations are less valuable than ED experience. In fact, it is possible that some or many of the deficiencies identified for ED experience could be provided during nonED rotations. This study demonstrates that we must either rely extensively on clinical experience gained outside the ED or we must teach the case m a n a g e m e n t of m a n y c o n d i t i o n s some other way. Pediatrics is clearly under-represented in the catalog. Emergency m e d i c i n e residents see pediatric cases without the aid of pediatricians in two of the three hospitals (and after midnight at the third), but a great deal of exposure occurs during a twomonth pediatric emergency medicine rotation at a hospital that did not participate in the computer catalog. We did not collect equal amounts of data for all four seasons. Twentyfour calendar months of data from three hospitals were comprised of nine winter months, six months each of spring and fall, and only three summer months. Therefore, a clinical entity with seasonal predilection to summer (eg, hyperthermia, drowning) would be theoretically underrepresented in the data base. There is no reason why the results of this study should be generalizable to other emergency medicine residencies. The pathology that presents to another program's hospitals would be inherently different, depending on the social environment adjacent to the ED, the a v a i l a b i l i t y of o t h e r sources of medical care, the emergency medical services system, and 70/771
the presence or absence of trauma center designation, among others. Furthermore, geography and weather can play an important part in determ i n i n g pathology. This data base would be expected to contain, for example, more lightning injuries and fewer snakebites than a similar tracking system based in Southern California. We believe, therefore, that it is important for each residency to monitor its own pathology in order to identify deficiencies in clinical experience. One could make an argument that the reason there is such wide variability between residents, or that some diagnoses apparently never present, is that certain residents, or even the entire residency, see the diagnosis but fail to recognize it. Perhaps patients with thyroid storm are incorrectly diagnosed as sepsis over and over again. There are two responses to this possibility. First, it is unlikely that, with 24-hour faculty supervision, residents could routinely repeat the same errors unnoticed. Second, if they do, and an important clinical entity is found by this system to present very rarely or not at all, these diagnoses are precisely the ones that we would target for teaching to residents by simulation. Following a session with a simulated patient, the resident would then not misdiagnose the patient on next presentation. Because the purpose of this catalog is to improve the educational experience, it does not particularly matter if the diagnosis never presents, or presents and is not recognized, so long as the education concerning that diagnosis takes place. It would be optimal to compare admitting diagnoses with final diagnoses as a measure of quality control for the department. However, the logistics of such a comparison were too difficult for this present study. A possible explanation for the large gaps in pathology identified here is the situation in which, shortly after presentation to the ED, a patient obviously is ill enough to warrant admission, but a diagnosis is not yet evident. It is common to make an educated guess as to the diagnosis in order to set in motion the admitting process. The risk arises that the "preliminary ED diagnosis" might not get replaced with the "definitive ED diagnosis." For example, "syncopal epiAnnals of Emergency Medicine
sode" might remain as the admitting diagnosis, even though the patient develops v e n t r i c u l a r t a c h y c a r d i a while in the ED. Then the resident managing the case fails to get credit for the more specific diagnosis. We admit that this situation occurs infrequently, and might make some of our rare specific diagnoses a bit more c o m m o n . However, this does not change the overall a r g u m e n t that monitoring resident cases is useful and necessary. By using a generous pairing system for ED record diagnoses as they relate to LOT Report and CCS diagnoses, we tried to avoid the criticism that a restrictive pairing was responsible for falsely showing that there are large gaps in available pathology. Whenever an ED record diagnosis could reasonably fit into a category in the documents, it was assigned there to more completely reflect resident activities. Overall, we believe that the conversion of ED record diagnoses to diagnoses contained in the CCS and LOT Reports is valid. Our findings suggest that ostensibly common conditions are not so common, even if this is contrary to intuition. The data clearly show that there is wide variability between residents in the number and types of diagnoses seen, confirming our first hypothesis. There are four possible reasons for this. First, some residents are inherently more skillful and make decisions faster and with more confidence, thereby seeing more patients. Second, some residents may choose to see more critically ill patients who take longer to manage, and therefore the resident might appear to be slow. No measure of the severity of illness of the residents' patients was included in this computer catalog. Third, there are some important variations in the types of pathology that present to the three hospitals. For example, Lutheran General Hospital, which is located in an uppermiddle-class suburb, sees significant heart disease, but very rarely any pelvic inflammatory disease or sickle cell crises. Mercy Hospital and Medical Center, located in a poor area, does see these latter two entities, as well as some penetrating trauma. Illinois Masonic Medical Center sees many more AIDS-related cases than do the other two hospitals. In the residents' second year, time is divided equally among all three hospitals, 19:7 July 1990
but during the senior year, chief residentS spend m o r e of their t i m e at one of the three. This alters the available pathology for t h a t y e a r and can account for part of the variability during the third year. However, referring again to Table 5, there is no great difference in the range of patients seen with a given diagnosis b e t w e e n second- and third-year residents. Fourth, and w e believe, m o s t important, t h e r e is the u n c o n t r o l l a b l e element of chance which, especially with u n c o m m o n diagnoses, can provide a few residents w i t h first-hand clinical e x p e r i e n c e w h i l e depriving most others. The reasons behind the variability are n o t v e r y i m p o r t a n t , a l t h o u g h identifying those residents who clearly c a n n o t h a n d l e a t y p i c a l patient load is useful. T h e true value of this catalog is to identify deficiencies i n i n d i v i d u a l and c o l l e c t i v e experience in order to change the curriculum accordingly, and to guide supplem e n t a t i o n of t h e s e d e f i c i e n c i e s through case simulation. Residency directors have long known t h a t there could not possibly be e n o u g h c a s e s of e p i g l o t t i t i s or meningococcemia so that every resident gains first-hand experience. But these data are alarming in that they show t h a t 22.7% (126 of 554) of the diagnoses c o n t a i n e d in t h e CCS for emergency m e d i c i n e did not present even once d u r i n g the e q u i v a l e n t of seven c o m p l e t e residencies. Furthermore, an a d d i t i o n a l i92 d i a g n o s e s (34.7%) p r e s e n t e d less t h a n s e v e n times, m a k i n g it v i r t u a l l y impossible for e v e r y r e s i d e n t to see e v e n one case. Overall, it appears that 58% of the CCS diagnoses w i l l n o t p r e s e n t to the ED often enough to provide faculty-supervised case m a n a g e m e n t to the residents. A n o t h e r w a y to assess the p r o b l e m of absent pathology is to compare the reality of a c t u a l n u m b e r of cases seen with t h e ideal. T h i s a d d r e s s e s t h e third of the four essential questions posed i n t h e i n t r o d u c t i o n , w h i c h needs an answer in order to provide each resident w i t h a c o m p l e t e clinical experience. The L O T Report 12 is the o n l y d o c u m e n t t h a t a d d r e s s e s what w o u l d be t h e o p t i m u m quantity of experience for a resident. Unfortunately, its r e c o m m e n d a t i o n s are not based on hard data, but rather on the opinions of recognized educators in e m e r g e n c y m e d i c i n e . A l t h o u g h 19:7 July 1990
this d o c u m e n t uses the s a m e structure as the CCS, it c o m b i n e s the 554 diagnoses i n t o 283 broader diagnostic categories and then assigns a recomm e n d e d n u m b e r of c a s e s to e a c h . Again, the data s h o w that actual exp e r i e n c e f a l l s far s h o r t ; o n l y i n 49.5% (140 of 283) of diagnostic categories did r e s i d e n t s m e e t or exceed the LOT Report recommendations. Clearly, our second hypothesis is also true - s o m e clinical entities do present so rarely as to m a k e it u n l i k e l y t h a t each r e s i d e n t w o u l d e n c o u n t e r t h e m in the ED frequently enough to p r o m o t e expert m a n a g e m e n t . D e s p i t e t h e a p p a r e n t a b s e n c e of m a n y conditions from the cases that p r e s e n t to t h e s t u d y EDs, t h e resid e n t s p r o b a b l y do l e a r n a b o u t t h e r e c o g n i t i o n of t h e s e c o n d i t i o n s through discussions of differential diagnosis w i t h m o r e senior emergency physicians. We agree that a resident can l e a r n m u c h a b o u t a d i a g n o s i s w i t h o u t first-hand case m a n a g e m e n t . H o w e v e r , c o n s i d e r a t i o n of a diagn o s i s is o n l y part of t h e process of p a t i e n t m a n a g e m e n t . Unless t h e pat i e n t actually has the disease, appropriate t h e r a p y is less l i k e l y to be discussed. T h e r e f o r e , w e b e l i e v e (but have not s h o w n here) t h a t actual experience w i t h a clinical disease is the o p t i m a l w a y to teach m a n a g e m e n t . It is therefore useful to m o n i t o r the residents' actual p a t i e n t contact in the c l i n i c a l a r e n a , i n a d d i t i o n to fostering learning through discussions of differential diagnosis. One could argue that a m o n i t o r i n g s y s t e m s u c h as this is u n n e c e s s a r y , evidenced b y the thousands of competent, board-certified emergency physicians w h o are n o w in practice b u t never had the benefit of such a system. We agree t h a t it is possible to train an e m e r g e n c y physician w i t h the c u r r e n t c u r r i c u l u m d e s p i t e the l a c k of personal experience w i t h case m a n a g e m e n t , as t h e p r o g r a m u n d e r s t u d y has done for a l m o s t ten years. However, each e m e r g e n c y physician can t h i n k back to the t i m e w h e n he completed training and remember areas of w e a k n e s s in k n o w l e d g e or m a n a g e m e n t w i t h at l e a s t s e v e r a l clinical entities. O v e r y e a r s of p r a c t i c e , w e h a v e shored up o u r a b i l i t y to h a n d l e the u n c o m m o n presentation, but we susp e c t t h e s e leaps of k n o w l e d g e were sparked by seeing a p a t i e n t and experiencing a bit of panic as we realized Annals of Emergency Medicine
t h a t w e w e r e u n p r e p a r e d . T h e authors seek to begin this m a t u r a t i o n process during residency so t h a t graduates will be better prepared for independent practice. The current m e t h o d of t r a i n i n g is good, b u t a m o n i t o r i n g system, w i t h appropriate s u p p l e m e n t a t i o n of cases, has great potential to m a k e it better. CONCLUSION T h e results of this study s h o w that there is great variability in the n u m bers and k i n d s of patients that emergency m e d i c i n e residents see in the ED and t h a t there are large gaps in the available ED pathology of even a good residency. We r e c o m m e n d that e a c h p r o g r a m set up a m o n i t o r i n g s y s t e m to identify deficiencies in ind i v i d u a l a n d g r o u p e x p e r i e n c e and s u p p l e m e n t t r a i n i n g w i t h case simulations. The authors thank the residents of the University of Illinois Affiliated Hospitals Emergency Medicine Residency, Society of Teachers of Emergency Medicine/ Emergency Medicine Foundation, and Richard Foley, Phi), for invaluable help. This article is dedicated to Harold Jayne, MD, FACEP, in memoriam.
REFERENCES
1. Guze SB, Myers JD: The quality of graduate medical education, in graduate medical education: Proposals for the eighties. J Med Educ 1981;56:21-36. 2. Podgorny G: Graduate education in eme> gency medicine: Duration of training. Ann Emerg Med 1982;11:592-593. 3. Irby DM: Clinical teaching and the clinical teacher. J Med Educ 1986;61(9 part 2):35-45. 4. Dailey RH: Residency essentials: Quantify to assure quality. A n n Emerg Med 1985;14: 928-929. 5. Frumkin K: The future of emergency medicine residency training. A n n Emerg Med 1985;14:378-379. 6. white JD: Defining emergency medicine residency training (letter). Ann Emerg Med 1986; 15:872. 7. US Department of Health and Human Serk vices, Public Health Service - Health Care Fi nancing Administration: International Classification of Diseases, 9th Revision, Clinical Modification. Washington, DC, US Government
Printing Office, 1980. 8. Smith MV, Freedman AM, Schwartz RW, et . al: A microcomputerized case log for surgical residents. Curt Surg 1984;41:371-373. 9. Anastasio GD, White TR, Fries JC: Comput erized prescription inventory program for the education of residents (PIPER). J Faro Pratt 1986;23:598-600. 10. Quattlebaum TG: Microcomputer analysis and management of residency training experi772/71
RESIDENT EXPERIENCE Langdorf, Strange & Macneil
ences. C o m p u t M e t h o d s Programs B i o m e d 1985;20:169-172. 11. Hamilton GC, Lumpkin JR, Tomlanovich MC, et al: Special Committee on the Core Content Revision: Emergency medicine core content. Ann Emerg Med 1986;15:853-862. 12. Lumpkin JR: Length of training in emergency medicine. Unpublished report. American College of Emergency Physicians, December 9, 1983. 13. Lamb GA, Weinberger HL, Schneiderman H, et al: Systematic utilization of data for analysis of a pediatric emergency-room experience. Pediatrics 1975;55:266-274.
14. Goldman GE: The need for some conformity in emergency medicine? Ann Emerg Med 1984;13:212.
19. Lutz L, Green L: Use of computer systems in family practice residencies. J Faro Praet 1982;14:369-375.
15. Goldman GE, personal communication, 1987.
20. Schneeweiss R: Diagnostic clusters: A new approach for reporting the diagnostic content of family practice residents' ambulatory experiences. J Faro Praet 1985;20:487-492.
16. Plummer D, Clinton J, Ruiz E: Emergency department critical care registry (abstract). Ann Emerg IVied 1987;16:502. 17. Curtis P, Resnick J, Greganti MA: Documentation of inpatient experiences of resident physicians. J Med Educ 1985;60:408-410. 18. Resnick JC, Curtis P: Inpatient documentation for family practice residents. J Fam Pract 1982; 15: 798-801.
21. Shear CL, Wall EM: Diagnosis cluster frequency in a community-based family practice residency program. West f Med 1985;142: 854-857. 22. Meza E, Lapsys FX: Adult medicine inpatient experience: A comparison of family practice and internal medicine residency services. J Faro Pract 1981;13:701-705.
See related editorial, p 827
72/773
Annals of Emergency Medicine
19:7 July 1990