- Email: [email protected]
Normalized Competitive Index: Analyzing Trends in Surgical Fellowship Training Over the Past Decade (2009-2018)
ARTICLE IN PRESS ORIGINAL REPORTS
Normalized Competitive Index: Analyzing Trends in Surgical Fellowship Training Over the Past Decade (2009-2018) Christopher G. Yheulon, MD, FACS,*, William C. Cole, MD,† Justin J. Ernat, MD,‡ and S. Scott Davis Jr., MD, FACS* Division of General and GI Surgery, Emory University Hospital, Atlanta Georgia; †Department of Surgery, Tripler Army Medical Center, Honolulu, Hawaii; and ‡Department of Orthopaedic Surgery, Blanchfield Army Community Hospital, Fort Campbell, Kentucky
*
OBJECTIVE: There is a lack of literature describing how competitive surgical fellowships are, especially across specialties. Such information would be valuable to prospective candidates, especially as immediate postresidency subspecialty training becomes the norm for general surgery. Match-rates alone may be misleading indicators as programs may not fill positions with unqualified applicants. We propose a simple metric to analyze the competitiveness of various surgical subspecialties to each other and themselves over time. DESIGN: Retrospective cohort study. The Competitive Index is defined as the percentage of filled programs within each specialty divided by the match-rate for that specialty. For ease of comparison, a Normalized Competitive Index (NCI) was developed, normalizing the metric for all specialties in that year to a value of 1. SETTING: The National Resident Matching Program, The Fellowship Council, and the San Francisco Match publicly available match data from 2009 to 2018. PARTICIPANTS: General Surgery Associated Fellowship
Applicants (Abdominal Transplant, Colorectal, Surgical Oncology, Minimally Invasive Surgery, Pediatric, Plastic, Critical Care, Thoracic, and Vascular).
competitive than other specialties (NCI 1.67, p < 0.0001), while surgical critical care (NCI 0.58, p < 0.0001) and vascular (NCI 0.90, p < 0.0492) were significantly less competitive. When comparing the NCI within each specialty from the first 5 years (2009-2013) to the last 5 years, (2014-2018), surgical critical care (NCI 0.54 vs. 0.62, p = 0.0462) and thoracic (NCI 0.74 vs. 1.08, p=0.0025) became significantly more competitive, while transplant (NCI 1.10 vs. 0.92, p = 0.0343) and colorectal (NCI 1.32 vs. 1.09, p = 0.0021) became significantly less competitive. CONCLUSION: The NCI is a metric which might be use-
ful to prospective applicants and which could be provided annually by organizations sponsoring fellowship matching processes. Further research must be performed to establish what defines a qualified applicant in each specialty. ( J Surg Ed 000:1 8. Published by Elsevier Inc. on behalf of Association of Program Directors in Surgery.) COMPETENCIES: Professionalism, Interpersonal and
Communication Skills, Systems Based Practice KEY WORDS: general surgery, fellowship, match, com-
petitive, comparison
RESULTS: The overall match rate for all specialties was
74.6% and 84.0% of all programs were filled. Over the past decade, pediatric surgery was significantly more
INTRODUCTION
Funding: The authors received no funding or other support for the creation of this manuscript to include the following organizations: National Institutes of Health (NIH); Wellcome Trust; Howard Hughes Medical Institute (HHMI); and other(s). Correspondence: Inquiries to Christopher G. Yheulon, MD, FACS, Division of General and GI Surgery, Emory University Hospital, 1364 Clifton Road, NE, Atlanta 30322, GA; e-mail: [email protected]
There is an increasing trend in early subspecialization after completion of a general surgery residency. From 1993 to 2005, the number of graduating residents pursuing immediate fellowship training increased from 67% to 77%.1 Even that figure may be underestimated, as some programs report rates as high as 95%.2 This can put
Journal of Surgical Education Published by Elsevier Inc. on behalf of Association of Program Directors in 1931-7204/$30.00 Surgery. https://doi.org/10.1016/j.jsurg.2019.07.023
1
2
55.0% 48.6% 47.2% N/A 27.6% N/A 80.4% 58.1% 62.4% 89.3% 66.4% 73.4% N/A 52.1% N/A 88.0% 70.5% 70.0%
Top 3 Choice Match Rate (Matches)
37.3% 30.1% 29.3% N/A 16.9% N/A 59.9% 35.8% 39.2%
Top Choice Match Rate (Applicants)
60.7% 41.1% 45.5% N/A 32.0% N/A 65.6% 43.4% 44.0%
Top Choice Match Rate (Matches)
94.1% 80.0% 81.5% 77.7% 57.0% N/A 94.2% 86.4% 92.5%
US Allopathic Match Rate
69.5% 98.3% 98.5% 89.2% 99.0% 94.3% 61.1% 82.9% 89.7% 61.5% 73.2% 64.3% 67.1% 52.9% 75.0% 91.3% 82.4% 89.1%
N/A, Not Available; US, United States. *Complex Surgical Oncology data only available for 2014 to 2018.
84.0% 74.6%
6.2 10.6 9.2 N/A 16.6 N/A 6.1 10.3 10.5
Program Ranks per Applicant
Total Specialty type Abdominal transplant surgery Colorectal surgery Complex surgical oncology* Minimally invasive surgery Pediatric surgery Plastic surgery Surgical critical care Thoracic surgery Vascular surgery
The National Resident Match Program, The Fellowship Council, and the San Francisco Match websites all provide publicly available data on fellowship programs, positions, applicants, and matches. The National Resident Match Program website (https://nrmp.org) provides data on abdominal transplant surgery, colorectal surgery, complex surgical oncology (CSO), pediatric surgery, surgical critical care (SCC), thoracic surgery, and vascular surgery. The Fellowship Council website (https://www.fellowshipcouncil.org) provided data on minimally invasive surgery (MIS). The San Francisco Match website (https://www.sfmatch.org) provided data for independent plastic surgery fellowships. These databases do not include integrative residency programs for any specialty, the most notable exclusions being thoracic surgery, vascular surgery, and plastic surgery. Data were available from 2009 to 2018, with the exception of CSO which was only available from 2014 to 2018. Institutional Review Board approval was not required as all
Percent Filled Programs
Inclusion of specialties
Match Rate
METHODS
TABLE 1. Surgical SubSpecialty Applicant and Match Characteristics 2009 to 2018
immense pressure on junior residents, and even medical students considering a career in surgery. A study by Reed et al. in 2010 found that second year residents (48%) are the most undecided with regard to future specialization. This percentage decreases dramatically in the third (32%), fourth (11%), and fifth (2%) postgraduate years.3 These authors also explored the relative importance of various influences in specialty choice. Nearly a quarter (23.5%) of residents responding to their survey reported “concern about matching” into a certain specialty as a factor. This was more than those concerned about future income potential (20.9%). Though existing literature defines what determines competitive applicants in some subspecialties, there is little or none available for others.4-8 “Match to applicant” (i.e., match rate) and “applicant to positions” ratios may be misleading indicators, as programs may not fill positions with unqualified applicants. In addition, trainee preferences change over time, making the measurement of “competitiveness” dynamic, not static. To address this issue, we propose a simple metric to compare the competitiveness of various surgical subspecialties. The Competitive Index is defined as the percentage of filled programs within each specialty divided by the match rate for that specialty. For ease of comparison, a Normalized Competitive Index (NCI) was developed, which standardizes the metric for all specialties in a given time period to a reference value of 1.
Top 3 Choice Match Rate (Applicants)
ARTICLE IN PRESS
Journal of Surgical Education Volume 00 /Number 00 Month 2019
ARTICLE IN PRESS data were publicly available. Data for all websites were accessed on February 2nd, 2019. Definition of Metrics The Competitive Index is defined as the percentage of filled programs (filled programs/unfilled programs) divided by the match rate (number of matches/number of applicants). The NCI normalizes the competitive indices for all specialties within a given year to a reference value of 1. A cumulative competitive index is calculated for each year (% of filled programs for all specialties divided by match rate for all specialties in that year) and the competitive index for each specialty for that year is divided by that cumulative value. The NCI simplifies comparisons within and across specialties over time. An NCI of 2 would imply that a specialty is twice as competitive as the average specialty for that year. In addition, if 2 specialties had NCIs of 1.5 and 0.75 in the same year, it can be inferred that 1 specialty is twice as competitive as the other specialty for that year. Statistical Analysis Surgical specialties were first analyzed over the entire 10-year period (2009-2018). Match rates and percentage of filled programs for each specialty were compared to the remaining other specialties excluding the specialty of interest. Subgroup analysis was performed between the first 5 years (2009-2013) and the most recent 5 years (2014-2018). CSO was not included in subgroup analyses, as data were not available from 2009 to 2013. Percentage of filled programs and match rates were compared with a chi-square test and the NCI with a Student t test. Pearson correlation coefficients were calculated to interpret the association between the NCI and other metrics, including total program ranks per applicant, US allopathic match
rate, top choice match rate, and top-3 choice match rate. These comparisons were not performed for MIS or plastic surgery, as complete data were unavailable for plastic surgery and MIS. Data were extracted and analyzed using a Microsoft Excel Spreadsheet (Microsoft Corporation, Redmond, Washington).
RESULTS Cumulative Analysis Applicant and match data were available for 9 surgical subspecialties (Table 1). The overall match rate for all specialties was 74.6%, and 84.0% of all programs were filled. Overall, match rates over the decade were relatively static. The exceptions were colorectal surgery, with an 8% increase, and thoracic surgery, with a 16% decrease. Match rates were greater than 60% for specialties other than pediatric surgery (53%). Abdominal transplant surgery, CSO, MIS, and pediatric surgery all had significantly lower match rates, while SCC, vascular surgery, and thoracic surgery all had significantly higher match rates (Table 2). The number of filled programs also generally increased, excepting abdominal transplant, colorectal, and pediatric surgery, which held steady. Overall, programs filled >80% of positions other than transplant, 69% filled, and critical care, 61% filled. Colorectal surgery, CSO, MIS, pediatric surgery, plastic surgery, and vascular surgery all had a significantly higher percentage of programs filled. Abdominal transplant surgery and SCC both had a significantly lower percentage of programs filled. Over the past decade, pediatric surgery was significantly more competitive than other specialties (NCI 1.67, p < 0.0001), while SCC (NCI 0.58, p < 0.0001) and vascular surgery (NCI 0.90, p < 0.0492) were significantly less competitive (Table 3).
TABLE 2. Comparison of Match Rates, Percent Filled Programs, and NCI Between Specialties, 2009 to 2018 Match Rate Total Specialty type Abdominal transplant surgery Colorectal surgery Complex surgical oncology* Minimally invasive surgery Pediatric surgery Plastic surgery Surgical critical care Thoracic surgery Vascular surgery
p Value
74.6% 61.5% 73.2% 64.3% 67.1% 52.9% 75.0% 91.3% 82.4% 89.1%
Filled Programs
p Value
NCI
p Value
<0.0001 <0.0001 <0.0001 <0.0001 <0.0001 <0.0001 <0.0001 0.3724 <0.0001
1.01 1.20 1.30 1.19 1.67 1.13 0.58 0.91 0.90
0.4292 0.2277 0.1321 0.2887 <0.0001 0.6712 <0.0001 0.0663 0.0492
84.0% <0.0001 0.2544 <0.0001 <0.0001 <0.0001 0.7181 <0.0001 <0.0001 <0.0001
69.5% 98.3% 98.5% 89.2% 99.0% 94.3% 61.1% 82.9% 89.7%
NCI, Normalized Competitive Index. Bold value are denotes statistical significance. *Complex Surgical Oncology data only available for 2014 to 2018.
Journal of Surgical Education Volume 00 /Number 00 Month 2019
3
4
0.0343 0.0021 0.3097 0.9748 0.3889 0.0462 0.0025 0.1129
Subgroup Analysis
Changes in Number of Applicants and Positions Over Time
69.0% 99.3% 92.0% 99.0% 96.8% 67.7% 95.0% 92.1% 70.0% 97.1% 86.2% 99.0% 92.1% 52.0% 72.4% 87.3%
0.7885 0.0636 0.0006 0.9686 0.0264 <0.0001 <0.0001 0.0161
87.5% 80.2%
<0.0001
1.10 1.32 1.15 1.67 1.19 0.54 0.74 0.92
0.92 1.09 1.23 1.67 1.07 0.62 1.08 0.87
Subgroup analysis was performed comparing the NCI within each specialty from the first 5 years (2009-2013) to the most recent 5 years, (2014-2018). SCC (NCI 0.54 vs. 0.62, p = 0.0462) and thoracic surgery (NCI 0.74 vs. 1.08, p = 0.0025) became significantly more competitive. Transplant surgery (NCI 1.10 vs. 0.92, p = 0.0343) and colorectal surgery (NCI 1.32 vs. 1.09, p = 0.0021) became significantly less competitive (Table 3 and Fig. 1).
The total number of applicants and positions offered for each specialty were also analyzed comparing pools from 2009-2013 to 2014-2018 (Table 4). There was a significant increase in the number of applicants in SCC (117 vs. 193, p = 0.0005) and thoracic surgery (93 vs. 115, p = 0.0163) while there was a decrease in the number of plastic surgery applicants (125 vs. 96, p = 0.0323). There was a significant increase in the number of positions offered for colorectal surgery (84 vs. 93, p = 0.0036) and SCC (164 vs. 229, p = 0.0078) while there was a decrease in the number of positions offered in plastic surgery (95 vs. 76, p = 0.0022) and thoracic surgery (110 vs. 90, p = 0.0011).
As expected, there is a high positive correlation between NCI and percentage of filled programs and a high negative correlation between NCI and match rate (Table 5).9 There were also high negative correlations between NCI and allopathic match rate, top choice match rate (for matriculates and applicants), and top-3 choice match rate (for matriculates and applicants). There was a moderate positive correlation between NCI and total program ranks per applicant.
NCI, Normalized Competitive Index. Bold value are denotes statistical significance.
64.0% 77.3% 64.0% 50.2% 77.7% 92.7% 75.1% 89.6% 59.3% 69.1% 70.8% 56.1% 73.0% 89.0% 91.6% 88.5%
0.1505 0.0013 0.0005 0.1014 0.0738 0.0142 <0.0001 0.5681
74.1% 75.1%
0.2600
Correlation with other metrics
Total Specialty Type Abdominal Transplant Surgery Colorectal Surgery Minimally Invasive Surgery Pediatric Surgery Plastic Surgery Surgical Critical Care Thoracic Surgery Vascular Surgery
Filled Programs Filled Programs 2009-2013 2014-2018 p value Match Rate Match Rate 2009-2013 2014-2018
TABLE 3. Interdisciplinary Comparison of Match Rates and Competitive Indices Over Time
p value
NCI NCI 2009-2013 2014-2018 p value
ARTICLE IN PRESS
DISCUSSION It is unsurprising that pediatric surgery is the most competitive surgical fellowship. Previous studies have demonstrated low match rates and pediatric surgery as a specialty is the most well studied regarding matriculation into fellowship training.4,10-12 It is also unsurprising that SCC is the least competitive subspecialty as previous papers described protocols to make the field more attractive to top applicants.13,14 Vascular Surgery is also statistically less competitive. This may be attributable to the introduction of integrated vascular surgery residencies beginning in 2007.8 However, we do not find similar
Journal of Surgical Education Volume 00 /Number 00 Month 2019
ARTICLE IN PRESS
FIGURE 1. Normalized Competitive Index trends from 2009 to 2018 for transplant, colorectal, critical care, and thoracic surgery.
TABLE 4. Interdisciplinary Comparison of Number of Applicants and Positions Available Over Time No. of No. of No. of No. of Applicants Applicants % p Positions Positions Percent p 2009-2013 2014-2018 Change value 2009-2013 2014-2018 Change value Specialty Type Abdominal transplant surgery Colorectal surgery Minimally invasive surgery Pediatric surgery Plastic surgery Surgical critical care Thoracic surgery Vascular surgery
94 119 214 72 125 117 93 119
85 120 252 84 96 193 115 123
10% 1% 18% 17% 23% 66% 24% 3%
0.2321 0.8963 0.0658 0.0923 0.0323 0.0005 0.0163 0.4991
77 84 173 41 95 164 110 117
74 93 175 43 76 229 90 118
4% 11% 1% 4% 20% 40% 18% 0%
0.3285 0.0036 0.7232 0.5034 0.0078 0.0022 0.0011 0.8445
NCI, Normalized Competitive Index. Bold value are denotes statistical significance.
findings in plastic surgery or cardiothoracic surgery, both of which offer integrated residency programs. What is surprising is the change in competitiveness in certain subspecialties over time. Colorectal surgery has recently been recognized as one of the most competitive specialties, though our data suggests this may be waning over the last 5 years.15 However, this may be explained by the significant increase in number of positions offered in colorectal surgery with virtually no change in the number of applicants. Transplant surgery has also become less competitive, though there is a near Journal of Surgical Education Volume 00 /Number 00 Month 2019
complete absence of data regarding any component of the abdominal transplant application and match process.16 Despite reports stating a lack of interest in thoracic surgery fellowship, our data indicates thoracic surgery has become more competitive over the last 5 years. This may be due to a supply and demand effect from a decrease in cardiothoracic surgery fellowship positions (115 from 2009 to 2013, 90 from 2014 to 2018) stemming from an increase in integrated residency positions.6 Furthermore, interest in thoracic surgery seems to be 5
ARTICLE IN PRESS
TABLE 5. Correlation Between NCI and Other Metrics Correlation Coefficient Metric Match rate Percent Filled programs Total program ranks per applicant* US Allopathic match rate** Match rate—top choice (matches)* Match rate—top choice (applicants)* Match rate—top 3 choice (matches)* Match rate—top 3 choice (applicants)*
0.86 0.73 0.62 0.87 0.70 0.90 0.74 0.95
NCI, Normalized Competitive Index. *Data unavailable for plastic surgery and minimally invasive surgery. **Data unavailable for plastic surgery.
outpacing the reduction in number of positions as we see a paradoxical increase in number of applicants (93 vs. 115) despite this reduction in number of programs. This finding is not replicated with either plastic surgery or vascular surgery, both of which similarly have a significant proportion of their trainees within integrated residency programs. SCC has also become more competitive over the last 5 years, part of which is likely attributable to the significant increase in number of applicants (117 vs. 193). Although there is also a significant increase in the number of programs, there is outpaced by the proportional increase in number of applicants (66% vs. 40%). Choosing a subspecialty is a complex process. Many factors influence a trainee’s decision to pursue a fellowship, including, as one would expect, the types of procedures and techniques involved, anticipated work-life balance, personal experiences, and mentorship.1,3,17-23 These factors are either widely known, in the case of types of procedures performed, or quite subjective and difficult to measure, as mentorship.24 For a significant proportion of trainees, perceived ability to successfully match into a given subspecialty is also a factor in fellowship choice.3 On their surface, existing metrics seem to provide this information. However, these may be misleading. Overall, the results here seem encouraging to trainees, with favorable match rates, nearly 75%, outside of a few select specialties. However, simple match-to-applicant or applicant-to-position ratios may fail to fully capture the competitiveness of applicants or selectivity of programs. They may also confound the popularity of a subspecialty with its competitiveness. Thus, using existing measures may dissuade applicants from choosing a fellowship for which they would actually be suited or encourage them to seek further training they cannot obtain. These potentially misguided choices, based on imperfect data, have downstream effects for the healthcare system. Such effects include the availability of subspecialists, as well 6
as real financial implications for the trainee, in the form of debt incurred during the fellowship interview process.25,26 This study has many limitations. This study was not completely inclusive of all surgical fellowships; endocrine surgery and burn surgery do not have publicly available match data and were not included in analysis. However, these 2 specialties are both small and only represent a small fraction of all fellowship applicants. The main limitation is the absence of individual metrics that likely independently affect the match including test scores, research, letters of recommendation, and interviews. In addition, what defines a “competitive” applicant is probably not consistent across fields. For example, an applicant deemed “uncompetitive” by pediatric surgery standards may still be considered “competitive” by many if not all other subspecialties. The degree to which perceived ability to match, based on metrics other than the NCI, influenced match rates and program vacancies within the data set cannot be determined. Indeed, the influence of any of the above-listed factors—type of techniques, case mix, work-life balance, etc.—already at play within the data set cannot be known, and one must suspect that a certain degree of self-selection has already been occurring. This may lead the NCI to become a measure, not of the competitiveness of each program, but of the ability of applicants to select suitable programs for themselves. Current researchers hypothesize that mathematical algorithms should be used to group fellowship candidates into competitive tiers which may improve match efficiency and decrease the burden of the match process for both applicants and programs.27,28 Additionally, the data may be affected by the evolution of surgical training programs, including the creation of combined or integrated programs, new programs, and entirely new subspecialties, as has occurred over time. However, the NCI should be able to account for this by including both the match rate and program vacancy rate. In either case, the NCI would accurately capture the changes to the specialty’s competitiveness. In the future, the NCI could be applied to individual programs within each subspecialty, as well. This could aid applicants in directing the efforts and save them time, money, and hassle by allowing them to limit the number of programs they apply to.
CONCLUSION Pediatric surgery is the most competitive surgical subspecialty. SCC is the least competitive surgical subspecialty, though it has become increasingly more competitive over recent years. Journal of Surgical Education Volume 00 /Number 00 Month 2019
ARTICLE IN PRESS The NCI is a novel yet simple metric that offers surgical trainees considering fellowship a more inclusive and accurate idea of the competitiveness of each subspecialty. The metric also provides additional objective information for program directors and mentors to set and manage trainee expectations. Overall, the NCI allows trainees to make a more informed decision on the pursuit of fellowship training.
REFERENCES
the pediatric surgical matching program. J Pediatr Surg. 1992;27:142–145. discussion 145-148. 12. Little DC, Yoder SM, Grikscheit TC, et al. Cost con-
siderations and applicant characteristics for the pediatric surgery match. J Pediatr Surg. 2005;40:69–73. discussion 73-64. 13. Hadzikadic L, Burke PA, Esposito TJ, Agarwal S. Sur-
gical resident perceptions of trauma surgery as a specialty. Arch Surg. 2010;145:445–450. 14. Richardson JD, Franklin GA, Rodriguez JL. Can we
1. Borman KR, Vick LR, Biester TW, Mitchell ME.
Changing demographics of residents choosing fellowships: longterm data from the American Board of Surgery. J Am Coll Surg. 2008;206:782–788. discussion 788-789. 2. Longo WE, Sumpio B, Duffy A, Seashore J, Udelsman
R. Early specialization in surgery: the new frontier. Yale J Biol Med. 2008;81:187–191.
make training in surgical critical care more attractive. J Trauma Acute Care Surg. 2005;59:1247–1249. 15. Bailey MB, Miller PE, Pawlak SE, et al. Current state
of colorectal surgery training: a survey of program directors, current and recently matched colorectal residents, and recent colorectal graduates. Dis Colon Rectum. 2016;59:140–147. 16. Hoffman A, Grant W, McCormick M, Jezewski E,
dominating choice of surgical specialty. J Am Coll Surg. 2010;210:319–324.
Matemavi P, Langnas A. Gendered differences in letters of recommendation for transplant surgery fellowship applicants. J Surg Educ. 2019;76:427–432.
4. Fraser JD, Aguayo P, St Peter S, et al. Analysis of the
17. Ellis MC, Dhungel B, Weerasinghe R, Vetto JT, Deve-
pediatric surgery match: factors predicting outcome. Pediatr Surg Int. 2011;27:1239–1244.
ney K. Trends in research time, fellowship training, and practice patterns among general surgery graduates. J Surg Educ. 2011;68:309–312.
3. Reed CE, Vaporciyan AA, Erikson C, et al. Factors
5. Hammoudeh ZS, Mostafa G, Saint-Cyr M, Bakri K,
Mardini S. The “independent” plastic surgery match: analysis of changes in recent years and applicant preferences. J Surg Educ. 2016;73:1039–1045. 6. Pousatis SM, Marshall MB. Trends in applications for
18. Fitzgerald JEF, Milburn JA, Khera G, Davies RSM,
Hornby ST, Giddings CEB. Clinical fellowships in surgical training: analysis of a national pan-specialty workforce survey. World J Surg. 2013;37:945–952.
thoracic fellowship in comparison with other subspecialties. Ann Thoracic Surg. 2014;97:624–632. discussion 623-632.
19. Grosfeld JL. General surgery and fellowship training:
7. Wach MM, Ruff SM, Ayabe RI, et al. An examination
20. Grover BT, Kothari SN. Fellowship training: need
of applicants and factors associated with matriculation to Complex General Surgical Oncology Fellowship Training Programs. Ann Surg Oncol. 2018;25:3436–3442.
and contributions. Surg Clin N Am. 2016;96:47–57.
8. Zayed MA, Dalman RL, Lee JT. A comparison of 0 + 5
versus 5 + 2 applicants to vascular surgery training programs. J Vasc Surg. 2012;56:1448–1452. 9. Mukaka MM. Statistics corner: A guide to appropri-
Mutually beneficial or competing entities. Surgery. 2002;132:526–528.
21. Muffly TM, Barber MD, Karafa MT, Kattan MW,
Shniter A, Jelovsek JE. Nomogram to predict successful placement in surgical subspecialty fellowships using applicant characteristics. J Surg Educ. 2012;69:364–370. 22. Stitzenberg KB, Sheldon GF. Progressive specializa-
ate use of correlation coefficient in medical research. Malawi Medical Journal. 2012;24:69–71.
tion within general surgery: adding to the complexity of workforce planning. J Am Coll Surg. 2005;201:925–932.
10. Beres A, Baird R, Puligandla PS. Success in the pedi-
23. Udgiri N, Raghavan K, Kella V. Factors dominating
atric surgery match: a survey of the 2010 applicant pool. J Pediatr Surg. 2011;46:957–961.
choice of surgical specialty among foreign medical graduates. J Am Coll Surg. 2010;211:297.
11. Hirthler MA, Glick PL, Hassett JM Jr., et al. Comparative
24. Marshall DC, Salciccioli JD, Walton SJ, Pitkin J, Shal-
analysis of successful and unsuccessful candidates for
houb J, Malietzis G. Medical student experience in
Journal of Surgical Education Volume 00 /Number 00 Month 2019
7
ARTICLE IN PRESS surgery influences their career choices: a systematic review of the literature. J Surg Educ. 2015;72: 438–445. 25. Eidt JF. Analysis of the current applicant pool to vas-
current and projected supply of pediatric surgeons in the context of a rapidly changing process for specialty and subspecialty training. Ann Surg. 2017;265:609–615.
cular surgery: who are they and where do they come from. Semin Vasc Surg. 2006;19:172–179.
27. Melcher ML, Ashlagi I, Wapnir I. Matching for fel-
26. Ricketts TC, Adamson WT, Fraher EP, et al. Future
28. Miele NF. Reducing the burden of fellowship inter-
supply of pediatric surgeons: analytical study of the
8
lowship interviews. JAMA. 2018;320:1639–1640. views. JAMA. 2019;321. 1106-1106.
Journal of Surgical Education Volume 00 /Number 00 Month 2019
Normalized Competitive Index: Analyzing Trends in Surgical Fellowship Training Over the Past Decade (2009-2018) Christopher G. Yheulon, MD, FACS,*, William C. Cole, MD,† Justin J. Ernat, MD,‡ and S. Scott Davis Jr., MD, FACS* Division of General and GI Surgery, Emory University Hospital, Atlanta Georgia; †Department of Surgery, Tripler Army Medical Center, Honolulu, Hawaii; and ‡Department of Orthopaedic Surgery, Blanchfield Army Community Hospital, Fort Campbell, Kentucky
*
OBJECTIVE: There is a lack of literature describing how competitive surgical fellowships are, especially across specialties. Such information would be valuable to prospective candidates, especially as immediate postresidency subspecialty training becomes the norm for general surgery. Match-rates alone may be misleading indicators as programs may not fill positions with unqualified applicants. We propose a simple metric to analyze the competitiveness of various surgical subspecialties to each other and themselves over time. DESIGN: Retrospective cohort study. The Competitive Index is defined as the percentage of filled programs within each specialty divided by the match-rate for that specialty. For ease of comparison, a Normalized Competitive Index (NCI) was developed, normalizing the metric for all specialties in that year to a value of 1. SETTING: The National Resident Matching Program, The Fellowship Council, and the San Francisco Match publicly available match data from 2009 to 2018. PARTICIPANTS: General Surgery Associated Fellowship
Applicants (Abdominal Transplant, Colorectal, Surgical Oncology, Minimally Invasive Surgery, Pediatric, Plastic, Critical Care, Thoracic, and Vascular).
competitive than other specialties (NCI 1.67, p < 0.0001), while surgical critical care (NCI 0.58, p < 0.0001) and vascular (NCI 0.90, p < 0.0492) were significantly less competitive. When comparing the NCI within each specialty from the first 5 years (2009-2013) to the last 5 years, (2014-2018), surgical critical care (NCI 0.54 vs. 0.62, p = 0.0462) and thoracic (NCI 0.74 vs. 1.08, p=0.0025) became significantly more competitive, while transplant (NCI 1.10 vs. 0.92, p = 0.0343) and colorectal (NCI 1.32 vs. 1.09, p = 0.0021) became significantly less competitive. CONCLUSION: The NCI is a metric which might be use-
ful to prospective applicants and which could be provided annually by organizations sponsoring fellowship matching processes. Further research must be performed to establish what defines a qualified applicant in each specialty. ( J Surg Ed 000:1 8. Published by Elsevier Inc. on behalf of Association of Program Directors in Surgery.) COMPETENCIES: Professionalism, Interpersonal and
Communication Skills, Systems Based Practice KEY WORDS: general surgery, fellowship, match, com-
petitive, comparison
RESULTS: The overall match rate for all specialties was
74.6% and 84.0% of all programs were filled. Over the past decade, pediatric surgery was significantly more
INTRODUCTION
Funding: The authors received no funding or other support for the creation of this manuscript to include the following organizations: National Institutes of Health (NIH); Wellcome Trust; Howard Hughes Medical Institute (HHMI); and other(s). Correspondence: Inquiries to Christopher G. Yheulon, MD, FACS, Division of General and GI Surgery, Emory University Hospital, 1364 Clifton Road, NE, Atlanta 30322, GA; e-mail: [email protected]
There is an increasing trend in early subspecialization after completion of a general surgery residency. From 1993 to 2005, the number of graduating residents pursuing immediate fellowship training increased from 67% to 77%.1 Even that figure may be underestimated, as some programs report rates as high as 95%.2 This can put
Journal of Surgical Education Published by Elsevier Inc. on behalf of Association of Program Directors in 1931-7204/$30.00 Surgery. https://doi.org/10.1016/j.jsurg.2019.07.023
1
2
55.0% 48.6% 47.2% N/A 27.6% N/A 80.4% 58.1% 62.4% 89.3% 66.4% 73.4% N/A 52.1% N/A 88.0% 70.5% 70.0%
Top 3 Choice Match Rate (Matches)
37.3% 30.1% 29.3% N/A 16.9% N/A 59.9% 35.8% 39.2%
Top Choice Match Rate (Applicants)
60.7% 41.1% 45.5% N/A 32.0% N/A 65.6% 43.4% 44.0%
Top Choice Match Rate (Matches)
94.1% 80.0% 81.5% 77.7% 57.0% N/A 94.2% 86.4% 92.5%
US Allopathic Match Rate
69.5% 98.3% 98.5% 89.2% 99.0% 94.3% 61.1% 82.9% 89.7% 61.5% 73.2% 64.3% 67.1% 52.9% 75.0% 91.3% 82.4% 89.1%
N/A, Not Available; US, United States. *Complex Surgical Oncology data only available for 2014 to 2018.
84.0% 74.6%
6.2 10.6 9.2 N/A 16.6 N/A 6.1 10.3 10.5
Program Ranks per Applicant
Total Specialty type Abdominal transplant surgery Colorectal surgery Complex surgical oncology* Minimally invasive surgery Pediatric surgery Plastic surgery Surgical critical care Thoracic surgery Vascular surgery
The National Resident Match Program, The Fellowship Council, and the San Francisco Match websites all provide publicly available data on fellowship programs, positions, applicants, and matches. The National Resident Match Program website (https://nrmp.org) provides data on abdominal transplant surgery, colorectal surgery, complex surgical oncology (CSO), pediatric surgery, surgical critical care (SCC), thoracic surgery, and vascular surgery. The Fellowship Council website (https://www.fellowshipcouncil.org) provided data on minimally invasive surgery (MIS). The San Francisco Match website (https://www.sfmatch.org) provided data for independent plastic surgery fellowships. These databases do not include integrative residency programs for any specialty, the most notable exclusions being thoracic surgery, vascular surgery, and plastic surgery. Data were available from 2009 to 2018, with the exception of CSO which was only available from 2014 to 2018. Institutional Review Board approval was not required as all
Percent Filled Programs
Inclusion of specialties
Match Rate
METHODS
TABLE 1. Surgical SubSpecialty Applicant and Match Characteristics 2009 to 2018
immense pressure on junior residents, and even medical students considering a career in surgery. A study by Reed et al. in 2010 found that second year residents (48%) are the most undecided with regard to future specialization. This percentage decreases dramatically in the third (32%), fourth (11%), and fifth (2%) postgraduate years.3 These authors also explored the relative importance of various influences in specialty choice. Nearly a quarter (23.5%) of residents responding to their survey reported “concern about matching” into a certain specialty as a factor. This was more than those concerned about future income potential (20.9%). Though existing literature defines what determines competitive applicants in some subspecialties, there is little or none available for others.4-8 “Match to applicant” (i.e., match rate) and “applicant to positions” ratios may be misleading indicators, as programs may not fill positions with unqualified applicants. In addition, trainee preferences change over time, making the measurement of “competitiveness” dynamic, not static. To address this issue, we propose a simple metric to compare the competitiveness of various surgical subspecialties. The Competitive Index is defined as the percentage of filled programs within each specialty divided by the match rate for that specialty. For ease of comparison, a Normalized Competitive Index (NCI) was developed, which standardizes the metric for all specialties in a given time period to a reference value of 1.
Top 3 Choice Match Rate (Applicants)
ARTICLE IN PRESS
Journal of Surgical Education Volume 00 /Number 00 Month 2019
ARTICLE IN PRESS data were publicly available. Data for all websites were accessed on February 2nd, 2019. Definition of Metrics The Competitive Index is defined as the percentage of filled programs (filled programs/unfilled programs) divided by the match rate (number of matches/number of applicants). The NCI normalizes the competitive indices for all specialties within a given year to a reference value of 1. A cumulative competitive index is calculated for each year (% of filled programs for all specialties divided by match rate for all specialties in that year) and the competitive index for each specialty for that year is divided by that cumulative value. The NCI simplifies comparisons within and across specialties over time. An NCI of 2 would imply that a specialty is twice as competitive as the average specialty for that year. In addition, if 2 specialties had NCIs of 1.5 and 0.75 in the same year, it can be inferred that 1 specialty is twice as competitive as the other specialty for that year. Statistical Analysis Surgical specialties were first analyzed over the entire 10-year period (2009-2018). Match rates and percentage of filled programs for each specialty were compared to the remaining other specialties excluding the specialty of interest. Subgroup analysis was performed between the first 5 years (2009-2013) and the most recent 5 years (2014-2018). CSO was not included in subgroup analyses, as data were not available from 2009 to 2013. Percentage of filled programs and match rates were compared with a chi-square test and the NCI with a Student t test. Pearson correlation coefficients were calculated to interpret the association between the NCI and other metrics, including total program ranks per applicant, US allopathic match
rate, top choice match rate, and top-3 choice match rate. These comparisons were not performed for MIS or plastic surgery, as complete data were unavailable for plastic surgery and MIS. Data were extracted and analyzed using a Microsoft Excel Spreadsheet (Microsoft Corporation, Redmond, Washington).
RESULTS Cumulative Analysis Applicant and match data were available for 9 surgical subspecialties (Table 1). The overall match rate for all specialties was 74.6%, and 84.0% of all programs were filled. Overall, match rates over the decade were relatively static. The exceptions were colorectal surgery, with an 8% increase, and thoracic surgery, with a 16% decrease. Match rates were greater than 60% for specialties other than pediatric surgery (53%). Abdominal transplant surgery, CSO, MIS, and pediatric surgery all had significantly lower match rates, while SCC, vascular surgery, and thoracic surgery all had significantly higher match rates (Table 2). The number of filled programs also generally increased, excepting abdominal transplant, colorectal, and pediatric surgery, which held steady. Overall, programs filled >80% of positions other than transplant, 69% filled, and critical care, 61% filled. Colorectal surgery, CSO, MIS, pediatric surgery, plastic surgery, and vascular surgery all had a significantly higher percentage of programs filled. Abdominal transplant surgery and SCC both had a significantly lower percentage of programs filled. Over the past decade, pediatric surgery was significantly more competitive than other specialties (NCI 1.67, p < 0.0001), while SCC (NCI 0.58, p < 0.0001) and vascular surgery (NCI 0.90, p < 0.0492) were significantly less competitive (Table 3).
TABLE 2. Comparison of Match Rates, Percent Filled Programs, and NCI Between Specialties, 2009 to 2018 Match Rate Total Specialty type Abdominal transplant surgery Colorectal surgery Complex surgical oncology* Minimally invasive surgery Pediatric surgery Plastic surgery Surgical critical care Thoracic surgery Vascular surgery
p Value
74.6% 61.5% 73.2% 64.3% 67.1% 52.9% 75.0% 91.3% 82.4% 89.1%
Filled Programs
p Value
NCI
p Value
<0.0001 <0.0001 <0.0001 <0.0001 <0.0001 <0.0001 <0.0001 0.3724 <0.0001
1.01 1.20 1.30 1.19 1.67 1.13 0.58 0.91 0.90
0.4292 0.2277 0.1321 0.2887 <0.0001 0.6712 <0.0001 0.0663 0.0492
84.0% <0.0001 0.2544 <0.0001 <0.0001 <0.0001 0.7181 <0.0001 <0.0001 <0.0001
69.5% 98.3% 98.5% 89.2% 99.0% 94.3% 61.1% 82.9% 89.7%
NCI, Normalized Competitive Index. Bold value are denotes statistical significance. *Complex Surgical Oncology data only available for 2014 to 2018.
Journal of Surgical Education Volume 00 /Number 00 Month 2019
3
4
0.0343 0.0021 0.3097 0.9748 0.3889 0.0462 0.0025 0.1129
Subgroup Analysis
Changes in Number of Applicants and Positions Over Time
69.0% 99.3% 92.0% 99.0% 96.8% 67.7% 95.0% 92.1% 70.0% 97.1% 86.2% 99.0% 92.1% 52.0% 72.4% 87.3%
0.7885 0.0636 0.0006 0.9686 0.0264 <0.0001 <0.0001 0.0161
87.5% 80.2%
<0.0001
1.10 1.32 1.15 1.67 1.19 0.54 0.74 0.92
0.92 1.09 1.23 1.67 1.07 0.62 1.08 0.87
Subgroup analysis was performed comparing the NCI within each specialty from the first 5 years (2009-2013) to the most recent 5 years, (2014-2018). SCC (NCI 0.54 vs. 0.62, p = 0.0462) and thoracic surgery (NCI 0.74 vs. 1.08, p = 0.0025) became significantly more competitive. Transplant surgery (NCI 1.10 vs. 0.92, p = 0.0343) and colorectal surgery (NCI 1.32 vs. 1.09, p = 0.0021) became significantly less competitive (Table 3 and Fig. 1).
The total number of applicants and positions offered for each specialty were also analyzed comparing pools from 2009-2013 to 2014-2018 (Table 4). There was a significant increase in the number of applicants in SCC (117 vs. 193, p = 0.0005) and thoracic surgery (93 vs. 115, p = 0.0163) while there was a decrease in the number of plastic surgery applicants (125 vs. 96, p = 0.0323). There was a significant increase in the number of positions offered for colorectal surgery (84 vs. 93, p = 0.0036) and SCC (164 vs. 229, p = 0.0078) while there was a decrease in the number of positions offered in plastic surgery (95 vs. 76, p = 0.0022) and thoracic surgery (110 vs. 90, p = 0.0011).
As expected, there is a high positive correlation between NCI and percentage of filled programs and a high negative correlation between NCI and match rate (Table 5).9 There were also high negative correlations between NCI and allopathic match rate, top choice match rate (for matriculates and applicants), and top-3 choice match rate (for matriculates and applicants). There was a moderate positive correlation between NCI and total program ranks per applicant.
NCI, Normalized Competitive Index. Bold value are denotes statistical significance.
64.0% 77.3% 64.0% 50.2% 77.7% 92.7% 75.1% 89.6% 59.3% 69.1% 70.8% 56.1% 73.0% 89.0% 91.6% 88.5%
0.1505 0.0013 0.0005 0.1014 0.0738 0.0142 <0.0001 0.5681
74.1% 75.1%
0.2600
Correlation with other metrics
Total Specialty Type Abdominal Transplant Surgery Colorectal Surgery Minimally Invasive Surgery Pediatric Surgery Plastic Surgery Surgical Critical Care Thoracic Surgery Vascular Surgery
Filled Programs Filled Programs 2009-2013 2014-2018 p value Match Rate Match Rate 2009-2013 2014-2018
TABLE 3. Interdisciplinary Comparison of Match Rates and Competitive Indices Over Time
p value
NCI NCI 2009-2013 2014-2018 p value
ARTICLE IN PRESS
DISCUSSION It is unsurprising that pediatric surgery is the most competitive surgical fellowship. Previous studies have demonstrated low match rates and pediatric surgery as a specialty is the most well studied regarding matriculation into fellowship training.4,10-12 It is also unsurprising that SCC is the least competitive subspecialty as previous papers described protocols to make the field more attractive to top applicants.13,14 Vascular Surgery is also statistically less competitive. This may be attributable to the introduction of integrated vascular surgery residencies beginning in 2007.8 However, we do not find similar
Journal of Surgical Education Volume 00 /Number 00 Month 2019
ARTICLE IN PRESS
FIGURE 1. Normalized Competitive Index trends from 2009 to 2018 for transplant, colorectal, critical care, and thoracic surgery.
TABLE 4. Interdisciplinary Comparison of Number of Applicants and Positions Available Over Time No. of No. of No. of No. of Applicants Applicants % p Positions Positions Percent p 2009-2013 2014-2018 Change value 2009-2013 2014-2018 Change value Specialty Type Abdominal transplant surgery Colorectal surgery Minimally invasive surgery Pediatric surgery Plastic surgery Surgical critical care Thoracic surgery Vascular surgery
94 119 214 72 125 117 93 119
85 120 252 84 96 193 115 123
10% 1% 18% 17% 23% 66% 24% 3%
0.2321 0.8963 0.0658 0.0923 0.0323 0.0005 0.0163 0.4991
77 84 173 41 95 164 110 117
74 93 175 43 76 229 90 118
4% 11% 1% 4% 20% 40% 18% 0%
0.3285 0.0036 0.7232 0.5034 0.0078 0.0022 0.0011 0.8445
NCI, Normalized Competitive Index. Bold value are denotes statistical significance.
findings in plastic surgery or cardiothoracic surgery, both of which offer integrated residency programs. What is surprising is the change in competitiveness in certain subspecialties over time. Colorectal surgery has recently been recognized as one of the most competitive specialties, though our data suggests this may be waning over the last 5 years.15 However, this may be explained by the significant increase in number of positions offered in colorectal surgery with virtually no change in the number of applicants. Transplant surgery has also become less competitive, though there is a near Journal of Surgical Education Volume 00 /Number 00 Month 2019
complete absence of data regarding any component of the abdominal transplant application and match process.16 Despite reports stating a lack of interest in thoracic surgery fellowship, our data indicates thoracic surgery has become more competitive over the last 5 years. This may be due to a supply and demand effect from a decrease in cardiothoracic surgery fellowship positions (115 from 2009 to 2013, 90 from 2014 to 2018) stemming from an increase in integrated residency positions.6 Furthermore, interest in thoracic surgery seems to be 5
ARTICLE IN PRESS
TABLE 5. Correlation Between NCI and Other Metrics Correlation Coefficient Metric Match rate Percent Filled programs Total program ranks per applicant* US Allopathic match rate** Match rate—top choice (matches)* Match rate—top choice (applicants)* Match rate—top 3 choice (matches)* Match rate—top 3 choice (applicants)*
0.86 0.73 0.62 0.87 0.70 0.90 0.74 0.95
NCI, Normalized Competitive Index. *Data unavailable for plastic surgery and minimally invasive surgery. **Data unavailable for plastic surgery.
outpacing the reduction in number of positions as we see a paradoxical increase in number of applicants (93 vs. 115) despite this reduction in number of programs. This finding is not replicated with either plastic surgery or vascular surgery, both of which similarly have a significant proportion of their trainees within integrated residency programs. SCC has also become more competitive over the last 5 years, part of which is likely attributable to the significant increase in number of applicants (117 vs. 193). Although there is also a significant increase in the number of programs, there is outpaced by the proportional increase in number of applicants (66% vs. 40%). Choosing a subspecialty is a complex process. Many factors influence a trainee’s decision to pursue a fellowship, including, as one would expect, the types of procedures and techniques involved, anticipated work-life balance, personal experiences, and mentorship.1,3,17-23 These factors are either widely known, in the case of types of procedures performed, or quite subjective and difficult to measure, as mentorship.24 For a significant proportion of trainees, perceived ability to successfully match into a given subspecialty is also a factor in fellowship choice.3 On their surface, existing metrics seem to provide this information. However, these may be misleading. Overall, the results here seem encouraging to trainees, with favorable match rates, nearly 75%, outside of a few select specialties. However, simple match-to-applicant or applicant-to-position ratios may fail to fully capture the competitiveness of applicants or selectivity of programs. They may also confound the popularity of a subspecialty with its competitiveness. Thus, using existing measures may dissuade applicants from choosing a fellowship for which they would actually be suited or encourage them to seek further training they cannot obtain. These potentially misguided choices, based on imperfect data, have downstream effects for the healthcare system. Such effects include the availability of subspecialists, as well 6
as real financial implications for the trainee, in the form of debt incurred during the fellowship interview process.25,26 This study has many limitations. This study was not completely inclusive of all surgical fellowships; endocrine surgery and burn surgery do not have publicly available match data and were not included in analysis. However, these 2 specialties are both small and only represent a small fraction of all fellowship applicants. The main limitation is the absence of individual metrics that likely independently affect the match including test scores, research, letters of recommendation, and interviews. In addition, what defines a “competitive” applicant is probably not consistent across fields. For example, an applicant deemed “uncompetitive” by pediatric surgery standards may still be considered “competitive” by many if not all other subspecialties. The degree to which perceived ability to match, based on metrics other than the NCI, influenced match rates and program vacancies within the data set cannot be determined. Indeed, the influence of any of the above-listed factors—type of techniques, case mix, work-life balance, etc.—already at play within the data set cannot be known, and one must suspect that a certain degree of self-selection has already been occurring. This may lead the NCI to become a measure, not of the competitiveness of each program, but of the ability of applicants to select suitable programs for themselves. Current researchers hypothesize that mathematical algorithms should be used to group fellowship candidates into competitive tiers which may improve match efficiency and decrease the burden of the match process for both applicants and programs.27,28 Additionally, the data may be affected by the evolution of surgical training programs, including the creation of combined or integrated programs, new programs, and entirely new subspecialties, as has occurred over time. However, the NCI should be able to account for this by including both the match rate and program vacancy rate. In either case, the NCI would accurately capture the changes to the specialty’s competitiveness. In the future, the NCI could be applied to individual programs within each subspecialty, as well. This could aid applicants in directing the efforts and save them time, money, and hassle by allowing them to limit the number of programs they apply to.
CONCLUSION Pediatric surgery is the most competitive surgical subspecialty. SCC is the least competitive surgical subspecialty, though it has become increasingly more competitive over recent years. Journal of Surgical Education Volume 00 /Number 00 Month 2019
ARTICLE IN PRESS The NCI is a novel yet simple metric that offers surgical trainees considering fellowship a more inclusive and accurate idea of the competitiveness of each subspecialty. The metric also provides additional objective information for program directors and mentors to set and manage trainee expectations. Overall, the NCI allows trainees to make a more informed decision on the pursuit of fellowship training.
REFERENCES
the pediatric surgical matching program. J Pediatr Surg. 1992;27:142–145. discussion 145-148. 12. Little DC, Yoder SM, Grikscheit TC, et al. Cost con-
siderations and applicant characteristics for the pediatric surgery match. J Pediatr Surg. 2005;40:69–73. discussion 73-64. 13. Hadzikadic L, Burke PA, Esposito TJ, Agarwal S. Sur-
gical resident perceptions of trauma surgery as a specialty. Arch Surg. 2010;145:445–450. 14. Richardson JD, Franklin GA, Rodriguez JL. Can we
1. Borman KR, Vick LR, Biester TW, Mitchell ME.
Changing demographics of residents choosing fellowships: longterm data from the American Board of Surgery. J Am Coll Surg. 2008;206:782–788. discussion 788-789. 2. Longo WE, Sumpio B, Duffy A, Seashore J, Udelsman
R. Early specialization in surgery: the new frontier. Yale J Biol Med. 2008;81:187–191.
make training in surgical critical care more attractive. J Trauma Acute Care Surg. 2005;59:1247–1249. 15. Bailey MB, Miller PE, Pawlak SE, et al. Current state
of colorectal surgery training: a survey of program directors, current and recently matched colorectal residents, and recent colorectal graduates. Dis Colon Rectum. 2016;59:140–147. 16. Hoffman A, Grant W, McCormick M, Jezewski E,
dominating choice of surgical specialty. J Am Coll Surg. 2010;210:319–324.
Matemavi P, Langnas A. Gendered differences in letters of recommendation for transplant surgery fellowship applicants. J Surg Educ. 2019;76:427–432.
4. Fraser JD, Aguayo P, St Peter S, et al. Analysis of the
17. Ellis MC, Dhungel B, Weerasinghe R, Vetto JT, Deve-
pediatric surgery match: factors predicting outcome. Pediatr Surg Int. 2011;27:1239–1244.
ney K. Trends in research time, fellowship training, and practice patterns among general surgery graduates. J Surg Educ. 2011;68:309–312.
3. Reed CE, Vaporciyan AA, Erikson C, et al. Factors
5. Hammoudeh ZS, Mostafa G, Saint-Cyr M, Bakri K,
Mardini S. The “independent” plastic surgery match: analysis of changes in recent years and applicant preferences. J Surg Educ. 2016;73:1039–1045. 6. Pousatis SM, Marshall MB. Trends in applications for
18. Fitzgerald JEF, Milburn JA, Khera G, Davies RSM,
Hornby ST, Giddings CEB. Clinical fellowships in surgical training: analysis of a national pan-specialty workforce survey. World J Surg. 2013;37:945–952.
thoracic fellowship in comparison with other subspecialties. Ann Thoracic Surg. 2014;97:624–632. discussion 623-632.
19. Grosfeld JL. General surgery and fellowship training:
7. Wach MM, Ruff SM, Ayabe RI, et al. An examination
20. Grover BT, Kothari SN. Fellowship training: need
of applicants and factors associated with matriculation to Complex General Surgical Oncology Fellowship Training Programs. Ann Surg Oncol. 2018;25:3436–3442.
and contributions. Surg Clin N Am. 2016;96:47–57.
8. Zayed MA, Dalman RL, Lee JT. A comparison of 0 + 5
versus 5 + 2 applicants to vascular surgery training programs. J Vasc Surg. 2012;56:1448–1452. 9. Mukaka MM. Statistics corner: A guide to appropri-
Mutually beneficial or competing entities. Surgery. 2002;132:526–528.
21. Muffly TM, Barber MD, Karafa MT, Kattan MW,
Shniter A, Jelovsek JE. Nomogram to predict successful placement in surgical subspecialty fellowships using applicant characteristics. J Surg Educ. 2012;69:364–370. 22. Stitzenberg KB, Sheldon GF. Progressive specializa-
ate use of correlation coefficient in medical research. Malawi Medical Journal. 2012;24:69–71.
tion within general surgery: adding to the complexity of workforce planning. J Am Coll Surg. 2005;201:925–932.
10. Beres A, Baird R, Puligandla PS. Success in the pedi-
23. Udgiri N, Raghavan K, Kella V. Factors dominating
atric surgery match: a survey of the 2010 applicant pool. J Pediatr Surg. 2011;46:957–961.
choice of surgical specialty among foreign medical graduates. J Am Coll Surg. 2010;211:297.
11. Hirthler MA, Glick PL, Hassett JM Jr., et al. Comparative
24. Marshall DC, Salciccioli JD, Walton SJ, Pitkin J, Shal-
analysis of successful and unsuccessful candidates for
houb J, Malietzis G. Medical student experience in
Journal of Surgical Education Volume 00 /Number 00 Month 2019
7
ARTICLE IN PRESS surgery influences their career choices: a systematic review of the literature. J Surg Educ. 2015;72: 438–445. 25. Eidt JF. Analysis of the current applicant pool to vas-
current and projected supply of pediatric surgeons in the context of a rapidly changing process for specialty and subspecialty training. Ann Surg. 2017;265:609–615.
cular surgery: who are they and where do they come from. Semin Vasc Surg. 2006;19:172–179.
27. Melcher ML, Ashlagi I, Wapnir I. Matching for fel-
26. Ricketts TC, Adamson WT, Fraher EP, et al. Future
28. Miele NF. Reducing the burden of fellowship inter-
supply of pediatric surgeons: analytical study of the
8
lowship interviews. JAMA. 2018;320:1639–1640. views. JAMA. 2019;321. 1106-1106.
Journal of Surgical Education Volume 00 /Number 00 Month 2019