- Email: [email protected]
Comparative analysis of successful and unsuccessful candidates for the pediatric surgical matching program
Comparative
Analysis of Successful and Unsuccessful Candidates the Pediatric Surgical Matching Program
for
By Maureen A. Hirthler, Philip L. Glick, James M. Hassett, Jr, Jon Rossman, Pauline Mendola, James E. Allen, Theodore C. Jewett, Jr, and Donald R. Cooney Buffalo, l Only one third of the applicants for training in pediatric surgery obtain a position through the pediatric surgery matching program. In order to identify factors that contribute to a successful outcome, we conducted a retrospective survey of all participants in the matching process for positions during the years 1983 to 1991. This survey was designed to identify characteristics associated with success in the match through comparison of successful and unsuccessful applicants. Significant factors associated with a successful match included: a greater incidence of residencysupported research (P = .012) with a greater number of publications (P = .003) and national presentations (P = .014), specifically at the annual meetings of the American Pediatric Surgical Association (P = .05) and the American Academy of Pediatrics (P = .05). In addition, successful candidates had more contact with (P = .004) and letters of recommendation from (P = .015) well-known pediatric surgeons involved in the general surgical residency program. This information should be invaluable to those counseling medical students and residents interested in a career in pediatric surgery. Copyright o 7992 by WA?. Saunders Company INDEX WORDS: Graduate gery Match Program.
medical education;
Pediatric Sur-
S
ELECTION of residents for training in pediatric surgery has traditionally been organized through the Pediatric Surgical Matching Program. This program was conducted by the Education Committee of the American Pediatric Surgical Association (APSA)’ from the mid-1970s until this past year, when the National Intern and Resident Matching Program assumed its administration. The Association of Training Program Directors anticipates no alteration in the actual selection process for pediatric surgical residents following this change. There continues to be significant competition for a limited number of positions, and controversy surrounds the announcement
From the Departments of Surgery and Pediattics, The School of Medicine and Biomedical Sciences, The University at Buffalo, State University of New York, The Children’s Hospital of Buffalo, Buffalo, NY Supported in part by The Women’s and Children S Health Research Foundation of the Children’s Hospital of Buffalo, Buffalo, NY Presented at the 22nd Annual Meeting of the American Pediatric SurgicalAssociation, Lake Buena Vista, Florida, May 15-18, 1991. Address reprint requests to Philip L. Click, MD, Department of Pediatric Surgery, The Children’s Hospital of Buffalo, 219 Bryant St, Buffalo, NY14222. Copyright o 1992 by WB. Saunders Company 0022-3468/92/2702-0002$03.00/O 142
New
York
of each year’s results.’ Speculation exists among pediatric surgeons, match participants, and general surgery residents regarding the attributes that contribute to a candidate’s successful outcome in the matching process. Such information might be helpful in counseling prospective apphcants in order to optimize their chances of success in the match, as well as to prepare them for a career in pediatric surgery. In an attempt to identify those objective factors that contributed to success, we designed a questionnaire that was distributed to all participants in the matching process for positions during the years 1983 to 1991. MATERIALS AND METHODS Participants in the match for the years of the study were identified through information provided by the Education Committee of APSA. Unsuccessful candidates were located through those same listings (recent applicants) or various medical directories, or by contacting the residencies where the candidates had trained in general surgery. Repeat mailings and telephone contacts were attempted for nonresponders. The questionnaire was designed for rapid completion, with primarily numerical and yes/no answers required. Questions included demographic and educational data, research experience, presentations, publications, exposure to pediatric surgery and pediatric surgeons, and evaluation of the matching process. Confidential coding was used for data input. Additional comments were encouraged. Response rates were calculated as the percentage of contacts that resulted in completed questionnaires3 Unsuccessful candidates responded much less frequently than successful ones. Analysis of curriculum vitae and the Directory of the American College of Surgeons confirmed that unsuccessful respondents (n = 10) did not differ from unsuccessful nonrespondents (n = 7) with regard to postresidency training, type of practice, or membership in national organizations. To assess the reliability of self-supplied data on the questionnaires, we compared responses to several items with data from the curriculum vitae (n = 15). No significant differences were found for age, change in residency, number of publications, and number of national presentations. Statistical analysis of all data was performed by either Student’s t test or x2 analysis with the Yates correction for small sample size, with significance assigned to P < .05. RESULTS
The number of participants for each year of the match included in the study are shown in Table 1. Included in each year are both first-time and repeat applicants, so that the total numbers of applicants in the table are greater than the actual number of JournatofPediatfic Surgery, Vol 27, No 2 (February), 1992: pp 142-149
PEDIATRIC SURGICAL MATCHING
143
PROGRAM
Table 1. Match Participants by Year
TotalApplicants
Returns
NO.
18
13
25
16
12
38
2
85
61
18
15
43
7
86
62
16
11
46
2
87
66
20
12
46
8
88
60
17
14
43
2
89
70
22
19
48
8
90
49
22
14
27
10
91
64
25
23
40
12
174
133
356
51
Year
83
43
84
54
Total
NO.
116 Not returned
Unsuccessful
Successful
Returns
20 Incorrect address
46 Incorrect phone 51 Returned 18 Not interested Fig 2.
candidates. Twenty-seven (16%) of the successful candidates were repeat applicants, as were 47 (19%) of the unsuccessful group. The distribution of the response to the questionnaires is shown in Fig 1. One hundred seventy-four successful candidates were contacted, with a 76% response rate (n = 133). Two hundred fifty-one unsuccessful candidates were initially identified. Sixty-six could not be located by mail or telephone. Onehundred eighty-five questionnaires were mailed to unsuccessful candidates and not returned by the post office; we assumed this indicated a successful contact.J Fifty-one unsuccessful candidates completed the questionnaire for a response rate of 28%. Finally, 18 communicated to us that they were not interested in participating, and in 116 no response could be obtained (Fig 2). Demographic data for all respondents are found in Table 2. There were no differences noted between successful and unsuccessful candidates with regard to 133
(76%)
134
41
(24%)
(72%)
51 (28%)
L Study
R
Study
NR
Control
R
Control
No second contact
NR
Fig 1. Distribution of response data. Successful (study) n = 174; unsuccessful (control) n = 185. R, responders: NR, nonresponders.
Contact results, unsuccessful candidates (n = 251).
age, marital status, spouse’s occupation, or number of children. Successful candidates had a greater incidence of residency-supported research, contact with wellknown pediatric surgeons, and letters of recommendation from pediatric surgeons involved in their general surgical residency program (Table 3). In addition, they had a greater number of publications and national presentations, as depicted in Table 4. The number of papers presented at the annual meetings of APSA and the Surgical Section of the American Academy of Pediatrics (AAP) were also significantly greater in the successful group. The data also indicated no differences between the successful and unsuccessful candidates with regard to the association of the general surgery training program with a pediatric surgical residency, letters of recommendation from pediatric surgeons not involved with the general surgical residency, the type of research performed (clinical v basic science), and election to premedical and medical honor societies. Analysis of other items (Table 4) showed that both successful and unsuccessful candidates applied to and interviewed at approximately two thirds of the programs. Rank analysis of factors important to the candidate (Table 5) indicates that unsuccessful candidates were more likely to place the programs where they felt they had the best chance of matching at the top of their match lists. Salary and hospital environment were more important to successful candidates. It was of particular interest that 29% of successful applicants were trained at one of nine general surgical programs (Table 6). Strong personal feelings about the matching process were communicated to the authors by both successful and unsuccessful candidates, but analysis of these data is beyond the scope of this report, and will be presented at a later time.
144
HIRTHLER ET AL
Table 2. Demographic Information Unsuccessful
Successful
f
X2
P Value
Age at time of PSMP (mean ? SD)
31.2 r 2.5
30.9 2 1.97
0.07
0.48
No. of children (mean + SD)
0.81 + 1.4
0.88 2 1.1
0.31
0.76
Marital Status (no.) Yes
33
70
No
20
31
Yes
19
34
No
9
36
0.49
0.48
0.27
0.14
Medical spouse (no.)
Abbreviation: PSMP, Pediatric Surgery Matching Program.
DISCUSSION
The results of this survey indicate that general surgical residencies must provide prospective candidates with the requisite opportunities that enable them to successfully compete in the pediatric surgery matching process. University-based general surgery programs with strong divisions of pediatric surgery, in which the resident is encouraged to do research, write papers, and make national presentations, meet that goal. It is important that the prospective candidate choose this type of program when interviewing as a medical student for general surgery. Many applicants fortuitously find themselves in such a program at the time of their decision to enter the match. Otherwise, the applicant must search outside the residency for the experience necessary to compete successfully, a more difficult and less satisfactory approach. Strongly supporting this conclusion is the fact that 29% of successful candidates matched directly from nine surgical programs of acknowledged excellence. The data presented in this paper support several additional conclusions. First, the selection process encourages and rewards high academic achievement.
Second, there are more strong candidates applying than can be accepted; thus, the future of pediatric surgery as an academically strong surgical subspecialty is likely to be maintained. Third, this degree of competition assures that pediatric surgery will continue to have strong role models who will influence the career choices of medical students and surgical residents. The methodology of this type of study creates some concern about the objectivity of the responses. When gathering information by questionnaire, it is impossible to ascertain the validity of self-reporting. However, when comparing completed questionnaires with candidates’ curriculum vitae, we found no significant differences. The low proportion of respondents in the unsuccessful group must also be addressed. Although mail questionnaires have a higher probability of reaching their target than telephone surveys,3 there are still a number of factors precluding a high response rate. These include the difficulty of locating people in a profession that involves frequent geographical changes, screening of mail by secretarial staff, a lack
Table 3. Study Items Unsuccessful Question
Successful
Yes
NO
Yes
NO
Research during residency
31
22
79
21
PValue
0.013’
Exposure to well-known pediatric surgeons
35
16
90
11
0.004*
Letter from Department of Pediatric Surgery
44
9
96
A
0.015*
(home institution) 0.001*
Time off for research
17
21
74
13
Change in residency
7
46
4
97
0.074
Phi Beta Kappa
8
30
22
63
0.730
Alpha Omega Alpha
9
29
34
51
0.121
Associated training program
9
44
28
73
0.199
36
16
71
30
1.000
Amount of money spent (over $4,000) Recommendations: Chairman
38
0
84
2
0.860
Pediatric surgeon (other institution)
19
31
34
57
1.000
General surgeon (home institution)
45
7
a5
12
1.000
Pediatric surgeon (research institution)
18
28
49
38
0.08
NOTE. Data analyzed by x2. *Significant difference.
145
PEDIATRIC SURGICAL MATCHING PROGRAM
Table 4. Study Items
Table 6. General Surgery Residencies With Greater Than Four Successful Match Participants
No. of publications (mean)
4.5
a.2
0.003” 0.014*
No. of
Program
2.0
3.7
No. of applications (mean)
12.5
12.9
0.72
University of Michigan
No. of interviews (mean)
11.3
11.9
0.553
Massachusetts General Hospital
No. of presentations (mean)
No. of papers presented APSA 1
13
8
>l
6
14
0.05x
AAP >l
11
5
7
17
0.05*
SuccessfulApplicants 12 7
Peter Bent Brigham
5
Parkland (Dallas, TX)
5
University of California (San Francisco)
5
University of Texas (Houston)
5
University of Illinois
4
Indiana University
4
Medical College of Virginia Total
4 51 (29%)
NOTE. Data analyzed by t test. Abbreviations: APSA, American Pediatric Surgical Association; AAP, American Academy of Pediatrics.
cess. All of these
*Significant difference.
of interest in participating in a study that provides no benefit to the respondent, or simple forgetfulness. When contacted during the second pass, several parties declined to participate in the study, citing a desire to have no further involvement with the proTable 5. Rank Analysis of Attributes of Training Programs
Hospital reputation
4.00 * 1.1
3.9 + 1.2
City, cost of living
2.77 ? 1.4
2.83 + 1.4
Director and staff reputation
4.24 2 0.97
4.20 + 1.12
Personalities
4.12 ? 0.96
4.22 + 1.17
Interviews with director
4.02 2 1.00
4.04 + 1.97
No. of cases
3.10 2 1.07
3.46 + 1.18
Organization of clinical service
3.28 2 1.03
3.31 * 1.19
Housestaff quality
2.79 -t 0.98
3.06 + 1.19
Salary, call schedule*
1.18 + 0.44
1.55 2 0.85
Physical plantt
1.98 + 0.97
2.44 * 1.04
Special programs
2.69 r 1.06
3.06 t 1.19
Current positions of former graduates
2.80 + 0.98
3.06 2 1.19
Estimated chance of matching*
3.18 r 1.40
2.35 z? 1.35
NOTE. 1, Low; 5, High. Data given as mean -c SD. *P =
,005.
tP = .Ol.
factors contributed to the low number of respondents in the unsuccessful group. Therefore, it is especially important to determine whether the unsuccessful nonrespondents differed greatly from the unsuccessful respondents, because the distribution of the data could be affected if nonrespondents were extreme in either direction.’ We were able to indirectly check several factors, such as completion of other postresidency training, type of practice, and membership in national organizations, which were similar in both groups. We were confident that we had a representative sample, albeit small, of the unsuccessful group. However, the data show bias in favor of the experiences and responses of the successful participants. It is possible that other differences between the successful and unsuccessful groups could exist and not be detected in our analysis. This study should not be construed as an absolute formula for success in the match because many subjective factors remain outside this analysis. The data presented herein are intended to encourage discussion about the manner in which future pediatric surgeons are chosen, and to provide a guide for advising medical students and general surgical residents considering a career in pediatric surgery.
REFERENCES 1. Johnson DG: Presidential address: Excellence in search of recognition. J Pediatr Surg 21:1019-1031, 1986 2. O’Neill JA: Presidential address: Pediatric surgery: Whither goest thou? J Pediatr Surg 25:2-10. 1990
3. Dillman DA: Mail and Telephone Surveys. The Total Design Method. New York, NY, Wiley, 1978 4. Rossi PH, Wright JD, Anderson AB: Handbook of Survey Research. San Diego, CA, Academic, 1983
Discussion B.M. Rodgers (Charlottesville, VA): I would like to first congratulate the authors and I hope the audience recognizes the incredible amount of work that went into tracking down the hundreds of individuals who
have participated in the Pediatric Surgery Matching Program during the last several years. The data are particularly helpful to those of us who are not associated with a training program, because we are
146
frequently advising students and residents before they enter the matching process. A few comments about the Pediatric Surgery Matching Program: it was started under the auspices of APSA through the Education Committee and the efforts of Lucian Leape in the mid-1970s. I think most individuals in this audience would agree that the match has been very successful from our viewpoint and that of the young people going through the process. However, one of the problems is that over 65% of the respondents indicated that they spent over $4,000 in the matching process. It was surprising to me that most respondents interviewed at only two thirds of the programs, because our residents in Virginia usually visit all of the programs. They may be away from their general surgery training program for 6 weeks. We need to look at how we can ease the travel and financial burden on the applicants. The major methodological problem with this questionnaire survey is that the percentage of respondents was quite low for unsuccessful candidates compared with successful applicants. Findings of no surprise were that the successful resident needs to have contact and support of a well-known pediatric surgeon, residency-sponsored research, a training program with academic activity, publications, and national presentations. The association of a general surgery training program with a pediatric surgery training program was not important for success in matching. I was surprised that election to AOA and Phi Beta Kappa did not make any difference. It is obvious that we need to begin counseling these young people when they are medical students rather than residents. Did the authors detect any difference in outcome based on when the applicants chose pediatric surgery? What comments did the respondents make about the training programs? Was there a gender bias in terms of selection of applicants? I enjoyed this paper and believe it will be helpful in counseling young people interested in our specialty. D.R. King (Columbus, OH): Would you clarify for me the association between successful candidates as far as their relationship to training programs that have an associated pediatric surgical training program. What are the data on internal selection? Did you evaluate how many individuals stayed at their own institution? L. Leupe (Boston, MA): I really don’t deserve the credit. It was Jim Rosenkrantz who started the matching program, but the interesting thing I think is that the matching program became what it was because of the leadership of APSA. When we first started, there were only five programs that participated because the thinking at the time was that you
HIRTHLER ET AL
didn’t want to pick somebody you didn’t know. A number of the programs would only appoint people into their pediatric surgical program who had been at their hospital and, preferably, who had been at a surgical residency where they had a chance to get to know them. When the match was initially proposed, there was a great deal of skepticism but the leadership of APSA felt strongly that our society would be much stronger if we had an open selection process and put a lot of pressure on the program directors. Within a matter of 2 years we had complete cooperation and I think our strength today derives directly from that. J.A. Huller (Baltimore, MD): I think that this paper will be very helpful to each of us in our own environment to give to the Chairmen of the Departments of General Surgery because they also ask what the various requirements are for candidates. I think this information is vital to them in identifying a research opportunity, early contact with pediatric surgeons, and things that are often missing in some of the top candidates who don’t begin to think about pediatric surgery until they are at the PGY 4,5, and 6 level. I also think that the important introduction of the pediatric surgery residents conference more than 10 years ago by Dr Altman and Dr Randolph has played a very important role for us in pediatric surgery because it has given general surgery residents an opportunity to have a format for the presentation of pertinent research and I not only want to congratulate them publicly but I think it is an important continuing effort on the part of APSA that we make that form available to our residents, both general surgery residents and those who are in the pediatric surgery training programs. AA. delorimier (San Francisco, CA): In looking at the list of where the successful candidates came from, it seemed as if the majority of them came from areas where they didn’t have a children’s hospital. Is this a misinterpretation of what was there or are many of our really good people getting turned off from their exposure to the children’s hospital environment because there is such a crush of heavyweights at the top that they get very little involvement when they rotate through the services? R.J. Touloukian (New Haven, CT): I think that we want to cozy up to these data because we feel very comfortable with the results of this survey, although the authors have pointed out that there is a bias here built on the fact that only 25% of the unsuccessful candidates responded. I think that most statisticians would state that this was an invalid study. I guess the question that we have really of those of us that are going to look critically at it is whether or not in
PEDIATRIC SURGICAL MATCHING
PROGRAM
uncovering those 75%, we might find a certain significant percentage of disgruntled applicants who felt that they were very qualified based on all of the criteria that you have identified as being appropriate for training in pediatric surgery. I am not sure that this study really answers the question that nags at me. Perhaps one could go back and look and see where the remainder of the unsuccessful candidates who didn’t respond to this questionnaire actually trained and if their demographic data are different from the successful candidates. P.K. Donahoe (Boston, IVIA): I would echo Dr Rodgers comments about the cost to our candidates and the travel time. I would think that he probably underestimated the cost at $4,000. I would imagine with today’s airfares, it is probably in the range of $10,000 for each candidate and we ought to consider organizing interviews in this forum to reduce that cost and the time commitment. M.H. Nahmad (Miami, FL): I wonder if the program directors were asked how they chose their candidates and why? M.A. Hirthler (response): In regard to the last question, we distributed a questionnaire to the training program directors at this meeting so I would hope that next year we will be able to tell you exactly how they choose their candidates. I think Dr Rodgers and Dr Donohoe pointed out what most people had to say about the match: it costs too much money and it takes too much time. Almost all the candidates, both successful and unsuccessful, questioned the value of visiting the actual programs, because obviously people are going to go any place where they match. It is an illusion that it’s a buyer’s market, it’s a seller’s market. When people did spend the money and visit these children’s hospitals, they often didn’t meet all the faculty. Some people didn’t even meet the training program director. The general surgery residencies offered very little support. Dr Rodger’s program in Charlottesville may be the only place that allowed their candidates to interview at all of the programs. Most people only got to two thirds of the programs because they either had to go back to work because of pressure from their fellow residents or they ran out of money. People also commented that the whole procedure was physically and mentally exhausting. The other comments that candidates made were more subjective. People professed to have a general knowledge of inside candidates and felt that programs that had a favorite son or daughter should so state at the beginning of the process and save people some time and money. Many people said they were falsely assured by training program directors either at the time of their interview or later by mail
147
that they were either going to match at that program or another program and so occasionally didn’t continue to interview. If they didn’t match, they felt a little bit more disappointed and more bitter about the whole procedure. Finally, the only comment I have on gender bias (because there really were so few women involved in the process) is that female candidates felt that there were inequitable emphases placed during their interviews on their family situations and their spouses’ career and career plans rather than focus on their own goals. To address some more of Dr Rodger’s comments about things that were significant and things that weren’t, I think that election to AOA and Phi Beta Kapa were not significant because about the same percentage of people (10%) in both the successful and unsuccessful groups belonged to those organizations and so it didn’t really help to distinguish the two. I think it is a true statement that the association of a pediatric surgery training program with your general surgery residency does not help. It was a real fact that most of the candidates came from places that didn’t have a training program and I would like to think that’s because those residents have a better feel for pediatric surgery because they get to do more surgery, they work more closely with the attending staff, and it’s not quite as dilute an experience at a place that has a number of fellows. I don’t have any data on internal matching because they would be anecdotal because there are really just a few instances in which that happened and I don’t think you can draw any conclusions from the number of people that were involved. As far as how candidates ranked programs, 1 think the best explanation I have for that is that when we asked people how they thought Dr Rodger’s match program worked and whether the program favored the candidate, the program, or neither, I would say that one third said it favored the candidate and one third said it favored the programs and one third said they didn’t have a clue as to how the program worked. I think that’s why people tended to think that putting their program where they thought they had the best chance of matching at the top of their list would help them, when in fact it didn’t have anything to do with what happened. Dr Haller, as far as the pediatric surgical residents conference, again that was a piece of data that was the same really in both groups, the same number of people participated so it wasn’t a distinguishing factor where APSA and AOA did differ in the two groups. Finally, I think your comment about how important this analysis is holds not only for pediatric surgery but for all sorts of surgical subspecialties. Most people chose a career in pediatric surgery during their residencies. We have looked at that data and, although people may have
148
expressed some interest when they were medical students, almost everybody decided on it during their second year of residency when they had their main pediatric surgical experience. It is a kind of Catch 22. You have to hope that you are in a place that will provide you with all the opportunities that we talked
HIRTHLER ET AL
about and so it makes it a little more difficult for the candidate who decides during residency and finds they are not in a great residency. I think that as time goes on we will have to encourage more medical students to think early about their career plans.
Analysis of Successful and Unsuccessful Candidates the Pediatric Surgical Matching Program
for
By Maureen A. Hirthler, Philip L. Glick, James M. Hassett, Jr, Jon Rossman, Pauline Mendola, James E. Allen, Theodore C. Jewett, Jr, and Donald R. Cooney Buffalo, l Only one third of the applicants for training in pediatric surgery obtain a position through the pediatric surgery matching program. In order to identify factors that contribute to a successful outcome, we conducted a retrospective survey of all participants in the matching process for positions during the years 1983 to 1991. This survey was designed to identify characteristics associated with success in the match through comparison of successful and unsuccessful applicants. Significant factors associated with a successful match included: a greater incidence of residencysupported research (P = .012) with a greater number of publications (P = .003) and national presentations (P = .014), specifically at the annual meetings of the American Pediatric Surgical Association (P = .05) and the American Academy of Pediatrics (P = .05). In addition, successful candidates had more contact with (P = .004) and letters of recommendation from (P = .015) well-known pediatric surgeons involved in the general surgical residency program. This information should be invaluable to those counseling medical students and residents interested in a career in pediatric surgery. Copyright o 7992 by WA?. Saunders Company INDEX WORDS: Graduate gery Match Program.
medical education;
Pediatric Sur-
S
ELECTION of residents for training in pediatric surgery has traditionally been organized through the Pediatric Surgical Matching Program. This program was conducted by the Education Committee of the American Pediatric Surgical Association (APSA)’ from the mid-1970s until this past year, when the National Intern and Resident Matching Program assumed its administration. The Association of Training Program Directors anticipates no alteration in the actual selection process for pediatric surgical residents following this change. There continues to be significant competition for a limited number of positions, and controversy surrounds the announcement
From the Departments of Surgery and Pediattics, The School of Medicine and Biomedical Sciences, The University at Buffalo, State University of New York, The Children’s Hospital of Buffalo, Buffalo, NY Supported in part by The Women’s and Children S Health Research Foundation of the Children’s Hospital of Buffalo, Buffalo, NY Presented at the 22nd Annual Meeting of the American Pediatric SurgicalAssociation, Lake Buena Vista, Florida, May 15-18, 1991. Address reprint requests to Philip L. Click, MD, Department of Pediatric Surgery, The Children’s Hospital of Buffalo, 219 Bryant St, Buffalo, NY14222. Copyright o 1992 by WB. Saunders Company 0022-3468/92/2702-0002$03.00/O 142
New
York
of each year’s results.’ Speculation exists among pediatric surgeons, match participants, and general surgery residents regarding the attributes that contribute to a candidate’s successful outcome in the matching process. Such information might be helpful in counseling prospective apphcants in order to optimize their chances of success in the match, as well as to prepare them for a career in pediatric surgery. In an attempt to identify those objective factors that contributed to success, we designed a questionnaire that was distributed to all participants in the matching process for positions during the years 1983 to 1991. MATERIALS AND METHODS Participants in the match for the years of the study were identified through information provided by the Education Committee of APSA. Unsuccessful candidates were located through those same listings (recent applicants) or various medical directories, or by contacting the residencies where the candidates had trained in general surgery. Repeat mailings and telephone contacts were attempted for nonresponders. The questionnaire was designed for rapid completion, with primarily numerical and yes/no answers required. Questions included demographic and educational data, research experience, presentations, publications, exposure to pediatric surgery and pediatric surgeons, and evaluation of the matching process. Confidential coding was used for data input. Additional comments were encouraged. Response rates were calculated as the percentage of contacts that resulted in completed questionnaires3 Unsuccessful candidates responded much less frequently than successful ones. Analysis of curriculum vitae and the Directory of the American College of Surgeons confirmed that unsuccessful respondents (n = 10) did not differ from unsuccessful nonrespondents (n = 7) with regard to postresidency training, type of practice, or membership in national organizations. To assess the reliability of self-supplied data on the questionnaires, we compared responses to several items with data from the curriculum vitae (n = 15). No significant differences were found for age, change in residency, number of publications, and number of national presentations. Statistical analysis of all data was performed by either Student’s t test or x2 analysis with the Yates correction for small sample size, with significance assigned to P < .05. RESULTS
The number of participants for each year of the match included in the study are shown in Table 1. Included in each year are both first-time and repeat applicants, so that the total numbers of applicants in the table are greater than the actual number of JournatofPediatfic Surgery, Vol 27, No 2 (February), 1992: pp 142-149
PEDIATRIC SURGICAL MATCHING
143
PROGRAM
Table 1. Match Participants by Year
TotalApplicants
Returns
NO.
18
13
25
16
12
38
2
85
61
18
15
43
7
86
62
16
11
46
2
87
66
20
12
46
8
88
60
17
14
43
2
89
70
22
19
48
8
90
49
22
14
27
10
91
64
25
23
40
12
174
133
356
51
Year
83
43
84
54
Total
NO.
116 Not returned
Unsuccessful
Successful
Returns
20 Incorrect address
46 Incorrect phone 51 Returned 18 Not interested Fig 2.
candidates. Twenty-seven (16%) of the successful candidates were repeat applicants, as were 47 (19%) of the unsuccessful group. The distribution of the response to the questionnaires is shown in Fig 1. One hundred seventy-four successful candidates were contacted, with a 76% response rate (n = 133). Two hundred fifty-one unsuccessful candidates were initially identified. Sixty-six could not be located by mail or telephone. Onehundred eighty-five questionnaires were mailed to unsuccessful candidates and not returned by the post office; we assumed this indicated a successful contact.J Fifty-one unsuccessful candidates completed the questionnaire for a response rate of 28%. Finally, 18 communicated to us that they were not interested in participating, and in 116 no response could be obtained (Fig 2). Demographic data for all respondents are found in Table 2. There were no differences noted between successful and unsuccessful candidates with regard to 133
(76%)
134
41
(24%)
(72%)
51 (28%)
L Study
R
Study
NR
Control
R
Control
No second contact
NR
Fig 1. Distribution of response data. Successful (study) n = 174; unsuccessful (control) n = 185. R, responders: NR, nonresponders.
Contact results, unsuccessful candidates (n = 251).
age, marital status, spouse’s occupation, or number of children. Successful candidates had a greater incidence of residency-supported research, contact with wellknown pediatric surgeons, and letters of recommendation from pediatric surgeons involved in their general surgical residency program (Table 3). In addition, they had a greater number of publications and national presentations, as depicted in Table 4. The number of papers presented at the annual meetings of APSA and the Surgical Section of the American Academy of Pediatrics (AAP) were also significantly greater in the successful group. The data also indicated no differences between the successful and unsuccessful candidates with regard to the association of the general surgery training program with a pediatric surgical residency, letters of recommendation from pediatric surgeons not involved with the general surgical residency, the type of research performed (clinical v basic science), and election to premedical and medical honor societies. Analysis of other items (Table 4) showed that both successful and unsuccessful candidates applied to and interviewed at approximately two thirds of the programs. Rank analysis of factors important to the candidate (Table 5) indicates that unsuccessful candidates were more likely to place the programs where they felt they had the best chance of matching at the top of their match lists. Salary and hospital environment were more important to successful candidates. It was of particular interest that 29% of successful applicants were trained at one of nine general surgical programs (Table 6). Strong personal feelings about the matching process were communicated to the authors by both successful and unsuccessful candidates, but analysis of these data is beyond the scope of this report, and will be presented at a later time.
144
HIRTHLER ET AL
Table 2. Demographic Information Unsuccessful
Successful
f
X2
P Value
Age at time of PSMP (mean ? SD)
31.2 r 2.5
30.9 2 1.97
0.07
0.48
No. of children (mean + SD)
0.81 + 1.4
0.88 2 1.1
0.31
0.76
Marital Status (no.) Yes
33
70
No
20
31
Yes
19
34
No
9
36
0.49
0.48
0.27
0.14
Medical spouse (no.)
Abbreviation: PSMP, Pediatric Surgery Matching Program.
DISCUSSION
The results of this survey indicate that general surgical residencies must provide prospective candidates with the requisite opportunities that enable them to successfully compete in the pediatric surgery matching process. University-based general surgery programs with strong divisions of pediatric surgery, in which the resident is encouraged to do research, write papers, and make national presentations, meet that goal. It is important that the prospective candidate choose this type of program when interviewing as a medical student for general surgery. Many applicants fortuitously find themselves in such a program at the time of their decision to enter the match. Otherwise, the applicant must search outside the residency for the experience necessary to compete successfully, a more difficult and less satisfactory approach. Strongly supporting this conclusion is the fact that 29% of successful candidates matched directly from nine surgical programs of acknowledged excellence. The data presented in this paper support several additional conclusions. First, the selection process encourages and rewards high academic achievement.
Second, there are more strong candidates applying than can be accepted; thus, the future of pediatric surgery as an academically strong surgical subspecialty is likely to be maintained. Third, this degree of competition assures that pediatric surgery will continue to have strong role models who will influence the career choices of medical students and surgical residents. The methodology of this type of study creates some concern about the objectivity of the responses. When gathering information by questionnaire, it is impossible to ascertain the validity of self-reporting. However, when comparing completed questionnaires with candidates’ curriculum vitae, we found no significant differences. The low proportion of respondents in the unsuccessful group must also be addressed. Although mail questionnaires have a higher probability of reaching their target than telephone surveys,3 there are still a number of factors precluding a high response rate. These include the difficulty of locating people in a profession that involves frequent geographical changes, screening of mail by secretarial staff, a lack
Table 3. Study Items Unsuccessful Question
Successful
Yes
NO
Yes
NO
Research during residency
31
22
79
21
PValue
0.013’
Exposure to well-known pediatric surgeons
35
16
90
11
0.004*
Letter from Department of Pediatric Surgery
44
9
96
A
0.015*
(home institution) 0.001*
Time off for research
17
21
74
13
Change in residency
7
46
4
97
0.074
Phi Beta Kappa
8
30
22
63
0.730
Alpha Omega Alpha
9
29
34
51
0.121
Associated training program
9
44
28
73
0.199
36
16
71
30
1.000
Amount of money spent (over $4,000) Recommendations: Chairman
38
0
84
2
0.860
Pediatric surgeon (other institution)
19
31
34
57
1.000
General surgeon (home institution)
45
7
a5
12
1.000
Pediatric surgeon (research institution)
18
28
49
38
0.08
NOTE. Data analyzed by x2. *Significant difference.
145
PEDIATRIC SURGICAL MATCHING PROGRAM
Table 4. Study Items
Table 6. General Surgery Residencies With Greater Than Four Successful Match Participants
No. of publications (mean)
4.5
a.2
0.003” 0.014*
No. of
Program
2.0
3.7
No. of applications (mean)
12.5
12.9
0.72
University of Michigan
No. of interviews (mean)
11.3
11.9
0.553
Massachusetts General Hospital
No. of presentations (mean)
No. of papers presented APSA 1
13
8
>l
6
14
0.05x
AAP >l
11
5
7
17
0.05*
SuccessfulApplicants 12 7
Peter Bent Brigham
5
Parkland (Dallas, TX)
5
University of California (San Francisco)
5
University of Texas (Houston)
5
University of Illinois
4
Indiana University
4
Medical College of Virginia Total
4 51 (29%)
NOTE. Data analyzed by t test. Abbreviations: APSA, American Pediatric Surgical Association; AAP, American Academy of Pediatrics.
cess. All of these
*Significant difference.
of interest in participating in a study that provides no benefit to the respondent, or simple forgetfulness. When contacted during the second pass, several parties declined to participate in the study, citing a desire to have no further involvement with the proTable 5. Rank Analysis of Attributes of Training Programs
Hospital reputation
4.00 * 1.1
3.9 + 1.2
City, cost of living
2.77 ? 1.4
2.83 + 1.4
Director and staff reputation
4.24 2 0.97
4.20 + 1.12
Personalities
4.12 ? 0.96
4.22 + 1.17
Interviews with director
4.02 2 1.00
4.04 + 1.97
No. of cases
3.10 2 1.07
3.46 + 1.18
Organization of clinical service
3.28 2 1.03
3.31 * 1.19
Housestaff quality
2.79 -t 0.98
3.06 + 1.19
Salary, call schedule*
1.18 + 0.44
1.55 2 0.85
Physical plantt
1.98 + 0.97
2.44 * 1.04
Special programs
2.69 r 1.06
3.06 t 1.19
Current positions of former graduates
2.80 + 0.98
3.06 2 1.19
Estimated chance of matching*
3.18 r 1.40
2.35 z? 1.35
NOTE. 1, Low; 5, High. Data given as mean -c SD. *P =
,005.
tP = .Ol.
factors contributed to the low number of respondents in the unsuccessful group. Therefore, it is especially important to determine whether the unsuccessful nonrespondents differed greatly from the unsuccessful respondents, because the distribution of the data could be affected if nonrespondents were extreme in either direction.’ We were able to indirectly check several factors, such as completion of other postresidency training, type of practice, and membership in national organizations, which were similar in both groups. We were confident that we had a representative sample, albeit small, of the unsuccessful group. However, the data show bias in favor of the experiences and responses of the successful participants. It is possible that other differences between the successful and unsuccessful groups could exist and not be detected in our analysis. This study should not be construed as an absolute formula for success in the match because many subjective factors remain outside this analysis. The data presented herein are intended to encourage discussion about the manner in which future pediatric surgeons are chosen, and to provide a guide for advising medical students and general surgical residents considering a career in pediatric surgery.
REFERENCES 1. Johnson DG: Presidential address: Excellence in search of recognition. J Pediatr Surg 21:1019-1031, 1986 2. O’Neill JA: Presidential address: Pediatric surgery: Whither goest thou? J Pediatr Surg 25:2-10. 1990
3. Dillman DA: Mail and Telephone Surveys. The Total Design Method. New York, NY, Wiley, 1978 4. Rossi PH, Wright JD, Anderson AB: Handbook of Survey Research. San Diego, CA, Academic, 1983
Discussion B.M. Rodgers (Charlottesville, VA): I would like to first congratulate the authors and I hope the audience recognizes the incredible amount of work that went into tracking down the hundreds of individuals who
have participated in the Pediatric Surgery Matching Program during the last several years. The data are particularly helpful to those of us who are not associated with a training program, because we are
146
frequently advising students and residents before they enter the matching process. A few comments about the Pediatric Surgery Matching Program: it was started under the auspices of APSA through the Education Committee and the efforts of Lucian Leape in the mid-1970s. I think most individuals in this audience would agree that the match has been very successful from our viewpoint and that of the young people going through the process. However, one of the problems is that over 65% of the respondents indicated that they spent over $4,000 in the matching process. It was surprising to me that most respondents interviewed at only two thirds of the programs, because our residents in Virginia usually visit all of the programs. They may be away from their general surgery training program for 6 weeks. We need to look at how we can ease the travel and financial burden on the applicants. The major methodological problem with this questionnaire survey is that the percentage of respondents was quite low for unsuccessful candidates compared with successful applicants. Findings of no surprise were that the successful resident needs to have contact and support of a well-known pediatric surgeon, residency-sponsored research, a training program with academic activity, publications, and national presentations. The association of a general surgery training program with a pediatric surgery training program was not important for success in matching. I was surprised that election to AOA and Phi Beta Kappa did not make any difference. It is obvious that we need to begin counseling these young people when they are medical students rather than residents. Did the authors detect any difference in outcome based on when the applicants chose pediatric surgery? What comments did the respondents make about the training programs? Was there a gender bias in terms of selection of applicants? I enjoyed this paper and believe it will be helpful in counseling young people interested in our specialty. D.R. King (Columbus, OH): Would you clarify for me the association between successful candidates as far as their relationship to training programs that have an associated pediatric surgical training program. What are the data on internal selection? Did you evaluate how many individuals stayed at their own institution? L. Leupe (Boston, MA): I really don’t deserve the credit. It was Jim Rosenkrantz who started the matching program, but the interesting thing I think is that the matching program became what it was because of the leadership of APSA. When we first started, there were only five programs that participated because the thinking at the time was that you
HIRTHLER ET AL
didn’t want to pick somebody you didn’t know. A number of the programs would only appoint people into their pediatric surgical program who had been at their hospital and, preferably, who had been at a surgical residency where they had a chance to get to know them. When the match was initially proposed, there was a great deal of skepticism but the leadership of APSA felt strongly that our society would be much stronger if we had an open selection process and put a lot of pressure on the program directors. Within a matter of 2 years we had complete cooperation and I think our strength today derives directly from that. J.A. Huller (Baltimore, MD): I think that this paper will be very helpful to each of us in our own environment to give to the Chairmen of the Departments of General Surgery because they also ask what the various requirements are for candidates. I think this information is vital to them in identifying a research opportunity, early contact with pediatric surgeons, and things that are often missing in some of the top candidates who don’t begin to think about pediatric surgery until they are at the PGY 4,5, and 6 level. I also think that the important introduction of the pediatric surgery residents conference more than 10 years ago by Dr Altman and Dr Randolph has played a very important role for us in pediatric surgery because it has given general surgery residents an opportunity to have a format for the presentation of pertinent research and I not only want to congratulate them publicly but I think it is an important continuing effort on the part of APSA that we make that form available to our residents, both general surgery residents and those who are in the pediatric surgery training programs. AA. delorimier (San Francisco, CA): In looking at the list of where the successful candidates came from, it seemed as if the majority of them came from areas where they didn’t have a children’s hospital. Is this a misinterpretation of what was there or are many of our really good people getting turned off from their exposure to the children’s hospital environment because there is such a crush of heavyweights at the top that they get very little involvement when they rotate through the services? R.J. Touloukian (New Haven, CT): I think that we want to cozy up to these data because we feel very comfortable with the results of this survey, although the authors have pointed out that there is a bias here built on the fact that only 25% of the unsuccessful candidates responded. I think that most statisticians would state that this was an invalid study. I guess the question that we have really of those of us that are going to look critically at it is whether or not in
PEDIATRIC SURGICAL MATCHING
PROGRAM
uncovering those 75%, we might find a certain significant percentage of disgruntled applicants who felt that they were very qualified based on all of the criteria that you have identified as being appropriate for training in pediatric surgery. I am not sure that this study really answers the question that nags at me. Perhaps one could go back and look and see where the remainder of the unsuccessful candidates who didn’t respond to this questionnaire actually trained and if their demographic data are different from the successful candidates. P.K. Donahoe (Boston, IVIA): I would echo Dr Rodgers comments about the cost to our candidates and the travel time. I would think that he probably underestimated the cost at $4,000. I would imagine with today’s airfares, it is probably in the range of $10,000 for each candidate and we ought to consider organizing interviews in this forum to reduce that cost and the time commitment. M.H. Nahmad (Miami, FL): I wonder if the program directors were asked how they chose their candidates and why? M.A. Hirthler (response): In regard to the last question, we distributed a questionnaire to the training program directors at this meeting so I would hope that next year we will be able to tell you exactly how they choose their candidates. I think Dr Rodgers and Dr Donohoe pointed out what most people had to say about the match: it costs too much money and it takes too much time. Almost all the candidates, both successful and unsuccessful, questioned the value of visiting the actual programs, because obviously people are going to go any place where they match. It is an illusion that it’s a buyer’s market, it’s a seller’s market. When people did spend the money and visit these children’s hospitals, they often didn’t meet all the faculty. Some people didn’t even meet the training program director. The general surgery residencies offered very little support. Dr Rodger’s program in Charlottesville may be the only place that allowed their candidates to interview at all of the programs. Most people only got to two thirds of the programs because they either had to go back to work because of pressure from their fellow residents or they ran out of money. People also commented that the whole procedure was physically and mentally exhausting. The other comments that candidates made were more subjective. People professed to have a general knowledge of inside candidates and felt that programs that had a favorite son or daughter should so state at the beginning of the process and save people some time and money. Many people said they were falsely assured by training program directors either at the time of their interview or later by mail
147
that they were either going to match at that program or another program and so occasionally didn’t continue to interview. If they didn’t match, they felt a little bit more disappointed and more bitter about the whole procedure. Finally, the only comment I have on gender bias (because there really were so few women involved in the process) is that female candidates felt that there were inequitable emphases placed during their interviews on their family situations and their spouses’ career and career plans rather than focus on their own goals. To address some more of Dr Rodger’s comments about things that were significant and things that weren’t, I think that election to AOA and Phi Beta Kapa were not significant because about the same percentage of people (10%) in both the successful and unsuccessful groups belonged to those organizations and so it didn’t really help to distinguish the two. I think it is a true statement that the association of a pediatric surgery training program with your general surgery residency does not help. It was a real fact that most of the candidates came from places that didn’t have a training program and I would like to think that’s because those residents have a better feel for pediatric surgery because they get to do more surgery, they work more closely with the attending staff, and it’s not quite as dilute an experience at a place that has a number of fellows. I don’t have any data on internal matching because they would be anecdotal because there are really just a few instances in which that happened and I don’t think you can draw any conclusions from the number of people that were involved. As far as how candidates ranked programs, 1 think the best explanation I have for that is that when we asked people how they thought Dr Rodger’s match program worked and whether the program favored the candidate, the program, or neither, I would say that one third said it favored the candidate and one third said it favored the programs and one third said they didn’t have a clue as to how the program worked. I think that’s why people tended to think that putting their program where they thought they had the best chance of matching at the top of their list would help them, when in fact it didn’t have anything to do with what happened. Dr Haller, as far as the pediatric surgical residents conference, again that was a piece of data that was the same really in both groups, the same number of people participated so it wasn’t a distinguishing factor where APSA and AOA did differ in the two groups. Finally, I think your comment about how important this analysis is holds not only for pediatric surgery but for all sorts of surgical subspecialties. Most people chose a career in pediatric surgery during their residencies. We have looked at that data and, although people may have
148
expressed some interest when they were medical students, almost everybody decided on it during their second year of residency when they had their main pediatric surgical experience. It is a kind of Catch 22. You have to hope that you are in a place that will provide you with all the opportunities that we talked
HIRTHLER ET AL
about and so it makes it a little more difficult for the candidate who decides during residency and finds they are not in a great residency. I think that as time goes on we will have to encourage more medical students to think early about their career plans.