- Email: [email protected]
Residency Research Requirement as a Predictor of Future Publication Productivity
Residency Research Requirement as a Predictor of Future Publication Productivity Phillip O. Ozuah, MD, PhD
T
he Accreditation Council for Graduate Medical Education (ACGME) requires that pediatric residency programs encourage and provide residents with an opportunity to participate in research and other scholarly activities.1 However, research projects are often difficult to incorporate into residency training because of several barriers, including funding, faculty mentors, and the lack of protected time.2-7 Despite these barriers, a number of pediatric residency programs have introduced a residency research requirement.5 In 1999, we initiated a research requirement in the primary care pediatric residency track at our institution. Earlier studies revealed that trainees who performed research were considered more competitive as candidates for future training.8-13 However, it is unknown whether having a residency research requirement is related to a greater number of future publications. The goal of this report is to share our experience with a residency research requirement and to encourage other departments and programs to consider the effectiveness of such an approach.
Methods We conducted a matched case-controlled study of a cohort of pediatric residents who graduated between the 2001 and 2005 academic years from the Children’s Hospital at Montefiore/ Albert Einstein College of Medicine, Bronx, New York. The institutional review board at the Montefiore Medical Center approved the study. Cases were defined as all residents in the primary care pediatric residency track, which had a research requirement. For each case, we identified 2 control subjects from the same institution, but in the categorical pediatric residency track, which did not have a research requirement. Cases and control subjects were matched on these characteristics: 1) reputation of the medical school from which the resident graduated (on the basis of the US News and World Report annual ranking of America’s best graduate schools in 200714); 2) memberships in Phi Beta Kappa Society (the US undergraduate honors organization) and Alpha Omega Alpha (the US national honor medical society); 3) academic performance in the medical school (on the basis of grades achieved); 4) residency interview rank score; 5) sex; and 6) year of graduation from the residency program. Reputation of the medical school from which the resident graduated was used as a proxy for academic environment before entering the residency program. Medical school performance and memberships in honor organizations served as markers of residents’ scholastic aptitude.
Data Analysis Data on number of publications before, during, and after residency were obtained from the PubMed database (from inception until December 1, 2006) by using subjects’ names and the institution where the research was conducted. For the matched case-controlled analysis, we calculated publication rates of cases and matched control subjects before residency training and during and after residency training. We performed 2 sets of secondary analyses. To partially control for selection bias, we compared the mean number of publications between all cases and the subset of categorical residents who applied to the primary care residency track but did not match because of the program’s space limitation. Additionally, because fellowship training has been related to increased publication rates,15,16 we collected data on the unmatched subsets of residents who pursued fellowship training. To determine the effect of fellowship training on publication productivity, we compared the mean number of publications for the subgroup of cases who pursued fellowships versus the subgroup of control subjects who pursued fellowships and the mean number of publications for the subgroup of cases who pursued fellowships versus the subgroup of categorical residents who applied but did not match to the primary care residency track and pursued fellowships. We used generalized estimating equations (GEEs) and conditional logistic regression for matched data. For secondary analyses, we used the independent samples t test to compare the mean number of publications in unmatched subsets of residents.
Results Of the 60 participants, cases (n = 20) and control subjects (n = 40) had similar publication rates before residency (0.5 vs. 0.45; P = .12). However, cases had a higher publication rate during and after residency (2.7 vs. 0.4; P = .006) (Table I; available at www.jpeds.com). When restricted to publications as the lead author, cases continued to have a higher publication rate during and after residency (Table I). With GEE statistics to compare means, we found that cases had a significantly greater mean number of publications during
From the Children’s Hospital at Montefiore and the Department of Pediatrics, Albert Einstein College of Medicine, Bronx, NY This study was presented in part at the 2007 Pediatric Academic Societies’ Annual Meeting in Toronto, Canada, May 5-8, 2007. The authors declare no conflicts of interest. 0022-3476/$ - see front matter. Copyright ª 2009 Mosby Inc. All rights reserved. 10.1016/j.jpeds.2009.03.056
1
THE JOURNAL OF PEDIATRICS
www.jpeds.com
and after residency (2.7 versus 0.4, P = .006). Conditional logistic regression analysis revealed that cases were at least 4 times more likely to publish during and after residency (odds ratio [OR] 4.25; 95% CI, 1.50-12.06). There were 23 categorical residents who applied but did not match to the primary care residency track. With a comparison of cases (n = 20) and categorical residents (n = 23) who applied but did not match to the primary care track, cases were revealed to have a significantly greater mean number of publications during and after residency (2.65 versus 0.26; P = .022; Table II; available at www.jpeds.com). We compared the subgroups of cases and control subjects who pursued fellowships. We found that 45% of cases (n = 9) and 65% of control subjects (n = 26) pursued fellowship training. With independent samples t test analysis, these 2 subgroups were revealed to have a similar mean number of publications before residency (0.1 versus 0.5, P = .12). As compared with control subjects who pursued fellowships, cases who went on to do fellowships had a significantly greater mean number of publications during and after residency (4.0 versus 0.5, P = .035). We also compared the subgroup of cases who pursued fellowships versus the subgroup of categorical residents who applied but did not match to the primary care residency track and pursued fellowships. For this comparison, we found that 45% of cases and 48% of categorical residents (n = 11) pursued fellowships. These 2 subgroups had similar mean number of publications before residency (0.11 versus 0.0, P = .35). However, cases with subsequent fellowship training had a significantly greater mean number of publications during and after residency (4.0 versus 0.5, P = .015).
Discussion A research requirement in our residency program was associated with significantly greater publication productivity during and after residency training. This study evaluated the relationship between a residency research requirement and future publication productivity. Residents in our study had 6 to 8 weeks of protected time for research during residency and had faculty mentors with research experience. Residents were required to present abstracts at regional and national research meetings, and were encouraged to submit a manuscript for publication in a peer-reviewed journal. There has been some discussion among pediatric medical educators about the necessity of a residency research requirement. Proponents of a research requirement contend that research experience imparts skills that are of great importance to the practice of medicine, such as literature searching and its critical appraisal.4,17 Additionally, early research training has been associated with continued scholarly work and may guide residents’ career choices.8-13 However, research activity may take time away from clinical learning, the primary goal of residency training. Earlier reports suggest that intrinsic interest in research, motivation, and self-selection have been related to the choice 2
Vol. 155, No. 1 of a research career and a greater number of publications.18,19 Although we used the same criteria to select applicants to either track, we wondered whether interest in the primary care residency track with a research requirement would be associated with a greater number of future publications. After partially controlling for selection bias, we found that although trainees in the research requirement track had a significantly greater mean number of publications, having an interest in the primary care residency track did not, by itself, predict future publication productivity. The criteria for selection of residents to either track were the same. There was no requirement for earlier research experience or publications, as evidenced by similar publication rates in both groups before residency. We found that the differences in publication productivity during and after residency persisted even in the subgroups of residents who pursued fellowships. Fellowship training has been associated with increased publication rates.15,16 During a pediatric fellowship, there is generally a 24-month period allocated for research activities, as compared with the 6 to 8 weeks of protected time that we provided to our residents. Our findings suggest that having protected time for research may not be the sole determining factor in predicting a greater number of future publications. Publishing early in one’s career, as suggested by our findings and earlier reports,20,21 may be an important predictor of future publication productivity. Our study had some limitations. We had a small sample size, and the study was conducted at a single institution. We used a single database, the PubMed, to collect data on the number of publications. Thus, only peer-reviewed publications indexed in the PubMed database were selected. However, when we included publications in the popular pediatric journals not indexed in the PubMed, such as Contemporary Pediatrics, residents in the primary care track had even greater number of publications. We excluded these publications from the analysis because of the possibility of inadvertent omission of papers authored by our trainees and published in the variety of other medical journals not indexed in the PubMed. Residency research requirement in our training program was associated with a significantly greater mean number of future publications. The differences in publication productivity persisted even after controlling for selection bias and the pursuit of future fellowship training. Timing of the research experience and the first publication, rather than having protected time for research, may be an important predictor of future publication productivity. Depending on the goals of a pediatric residency program, a research requirement could be an effective means of promoting scholarship among trainees. n Reprint requests: Phillip O. Ozuah, MD, PhD, Professor and University Chairman, Physician-in-Chief, Department of Pediatrics, Albert Einstein College of Medicine, Children’s Hospital at Montefiore, 3415 Bainbridge Ave, Bronx, NY 10467. E-mail: [email protected].
References available at www.jpeds.com Ozuah
July 2009
Table I. Publication rates for cases and matched control subjects Publication rates Before residency During and after residency During and after residency (restricted to publications as the lead author)
Cases (n = 20)
Matched controls (n = 40)
P value*
0.5 2.7 2.1
0.45 0.4 0.3
.12 .006 .003
*P values obtained with generalized estimating equations.
Table II. Mean number of publications for cases versus categorical residents who did not match to the primary care track Mean number of publications
Cases (n = 20)
Categorical residents (n = 23)
P value*
Before residency During and after residency During and after residency (restricted to publications as the lead author)
0.50 2.65 2.10
0.13 0.26 0.22
.209 .009 .022
*P values obtained with independent samples t test.
References 1. Accreditation Council for Graduate Medical Education. Program requirements for residency education in pediatrics. Available at: http:// www.acgme.org/acWebsite/downloads/RRC_progReq/320pr01012006. pdf. Accessed Sep 20, 2007. 2. Carek PJ. Graduate surveys: an opportunity for residency research. Ann Fam Med 2005;3:376-7. 3. Rivera JA, Levine RB, Wright SM. Completing a scholarly project during residency training. Perspectives of residents who have been successful. J Gen Intern Med 2005;20:366-9. 4. Levine RB, Hebert RS, Wright SM. Resident research and scholarly activity in internal medicine residency training programs. J Gen Intern Med 2005;20:155-9. 5. Brouhard BH, Doylel W, Aceves J, McHugh MJ. Research in pediatric residency programs. Pediatrics 1996;97:71-3. 6. Neale AV, West P, French L. Surviving your resident research requirement. JAMA 1998;280:1802. 7. Mylonakis E, Koutkia P. The realities of resident research requirements. JAMA 1999;281:2089. 8. Hayward RA, Taweel F. Data and the internal medicine houseofficer: alumni’s views of the educational value of a residency program’s research requirement. J Gen Intern Med 1993;8:140-2. 9. Levine DM. Career intentions in internal medicine. Ann Intern Med 1981;94:704-6. 10. Miller ED. Clinical investigators—the endangered species revisited. JAMA 2001;286:845-6.
11. Solomon SS, Tom SC, Pichert J, Wasserman D, Powers AC. Impact of medical students research in the development of physician-scientists. J Investig Med 2003;51:149-56. 12. Lessin MS, Klein MD. Does research during general surgery residency correlate with academic pursuits after pediatric surgery residency? J Pediatr Surg 1995;30:1310-3. 13. Dunn JC, Lai EC, Brooks CM, Stabile BE, Fonkalsrud EW. The outcome of research training during surgical residency. J Pediatr Surg 1998;33:362-4. 14. US News and World Report. America’s Best Graduate Schools 2007. Available at: www.usnews.com/usnews/edu/grad/rankings/med/brief/ mdrrank_brief.php. Accessed Sep 24, 2007. 15. Powner DJ, Thomas EA. Research curricula in critical care fellowships— a survey. Crit Care Med 1996;24:1079-82. 16. Steiner JF, Lanphear BP, Curtis P, Vu KO. Indicators of early research productivity among primary care fellows. J Gen Intern Med 2002;17: 845-51. 17. Potti A, Mariani P, Saeed M, Smego RA Jr.. Residents as researchers: expectations, requirements, and productivity. Am J Med 2003;115:510-4. 18. Salive ME, Stein DH. Predictors and correlates of prevention research careers: a national study of residency graduates. Prev Med 1999;28:430-6. 19. Straus SE, Straus C, Tzanetos K; International Campaign to Revitalise Academic Medicine. Career choice in academic medicine: systematic review. J Gen Intern Med 2006;21:1222-9. 20. Ledley FD, Lovejoy FH Jr. Prospects for academically trained pediatricians in academic medicine. Clin Invest Med 1992;15:518-26. 21. Dorsey ER, Raphael BA, Balcer LJ, Galetta SL. Predictors of future publication record and academic rank in a cohort of neurology residents. Neurology 2006;67:1335-7.
Residency Research Requirement as a Predictor of Future Publication Productivity
2.e1
T
he Accreditation Council for Graduate Medical Education (ACGME) requires that pediatric residency programs encourage and provide residents with an opportunity to participate in research and other scholarly activities.1 However, research projects are often difficult to incorporate into residency training because of several barriers, including funding, faculty mentors, and the lack of protected time.2-7 Despite these barriers, a number of pediatric residency programs have introduced a residency research requirement.5 In 1999, we initiated a research requirement in the primary care pediatric residency track at our institution. Earlier studies revealed that trainees who performed research were considered more competitive as candidates for future training.8-13 However, it is unknown whether having a residency research requirement is related to a greater number of future publications. The goal of this report is to share our experience with a residency research requirement and to encourage other departments and programs to consider the effectiveness of such an approach.
Methods We conducted a matched case-controlled study of a cohort of pediatric residents who graduated between the 2001 and 2005 academic years from the Children’s Hospital at Montefiore/ Albert Einstein College of Medicine, Bronx, New York. The institutional review board at the Montefiore Medical Center approved the study. Cases were defined as all residents in the primary care pediatric residency track, which had a research requirement. For each case, we identified 2 control subjects from the same institution, but in the categorical pediatric residency track, which did not have a research requirement. Cases and control subjects were matched on these characteristics: 1) reputation of the medical school from which the resident graduated (on the basis of the US News and World Report annual ranking of America’s best graduate schools in 200714); 2) memberships in Phi Beta Kappa Society (the US undergraduate honors organization) and Alpha Omega Alpha (the US national honor medical society); 3) academic performance in the medical school (on the basis of grades achieved); 4) residency interview rank score; 5) sex; and 6) year of graduation from the residency program. Reputation of the medical school from which the resident graduated was used as a proxy for academic environment before entering the residency program. Medical school performance and memberships in honor organizations served as markers of residents’ scholastic aptitude.
Data Analysis Data on number of publications before, during, and after residency were obtained from the PubMed database (from inception until December 1, 2006) by using subjects’ names and the institution where the research was conducted. For the matched case-controlled analysis, we calculated publication rates of cases and matched control subjects before residency training and during and after residency training. We performed 2 sets of secondary analyses. To partially control for selection bias, we compared the mean number of publications between all cases and the subset of categorical residents who applied to the primary care residency track but did not match because of the program’s space limitation. Additionally, because fellowship training has been related to increased publication rates,15,16 we collected data on the unmatched subsets of residents who pursued fellowship training. To determine the effect of fellowship training on publication productivity, we compared the mean number of publications for the subgroup of cases who pursued fellowships versus the subgroup of control subjects who pursued fellowships and the mean number of publications for the subgroup of cases who pursued fellowships versus the subgroup of categorical residents who applied but did not match to the primary care residency track and pursued fellowships. We used generalized estimating equations (GEEs) and conditional logistic regression for matched data. For secondary analyses, we used the independent samples t test to compare the mean number of publications in unmatched subsets of residents.
Results Of the 60 participants, cases (n = 20) and control subjects (n = 40) had similar publication rates before residency (0.5 vs. 0.45; P = .12). However, cases had a higher publication rate during and after residency (2.7 vs. 0.4; P = .006) (Table I; available at www.jpeds.com). When restricted to publications as the lead author, cases continued to have a higher publication rate during and after residency (Table I). With GEE statistics to compare means, we found that cases had a significantly greater mean number of publications during
From the Children’s Hospital at Montefiore and the Department of Pediatrics, Albert Einstein College of Medicine, Bronx, NY This study was presented in part at the 2007 Pediatric Academic Societies’ Annual Meeting in Toronto, Canada, May 5-8, 2007. The authors declare no conflicts of interest. 0022-3476/$ - see front matter. Copyright ª 2009 Mosby Inc. All rights reserved. 10.1016/j.jpeds.2009.03.056
1
THE JOURNAL OF PEDIATRICS
www.jpeds.com
and after residency (2.7 versus 0.4, P = .006). Conditional logistic regression analysis revealed that cases were at least 4 times more likely to publish during and after residency (odds ratio [OR] 4.25; 95% CI, 1.50-12.06). There were 23 categorical residents who applied but did not match to the primary care residency track. With a comparison of cases (n = 20) and categorical residents (n = 23) who applied but did not match to the primary care track, cases were revealed to have a significantly greater mean number of publications during and after residency (2.65 versus 0.26; P = .022; Table II; available at www.jpeds.com). We compared the subgroups of cases and control subjects who pursued fellowships. We found that 45% of cases (n = 9) and 65% of control subjects (n = 26) pursued fellowship training. With independent samples t test analysis, these 2 subgroups were revealed to have a similar mean number of publications before residency (0.1 versus 0.5, P = .12). As compared with control subjects who pursued fellowships, cases who went on to do fellowships had a significantly greater mean number of publications during and after residency (4.0 versus 0.5, P = .035). We also compared the subgroup of cases who pursued fellowships versus the subgroup of categorical residents who applied but did not match to the primary care residency track and pursued fellowships. For this comparison, we found that 45% of cases and 48% of categorical residents (n = 11) pursued fellowships. These 2 subgroups had similar mean number of publications before residency (0.11 versus 0.0, P = .35). However, cases with subsequent fellowship training had a significantly greater mean number of publications during and after residency (4.0 versus 0.5, P = .015).
Discussion A research requirement in our residency program was associated with significantly greater publication productivity during and after residency training. This study evaluated the relationship between a residency research requirement and future publication productivity. Residents in our study had 6 to 8 weeks of protected time for research during residency and had faculty mentors with research experience. Residents were required to present abstracts at regional and national research meetings, and were encouraged to submit a manuscript for publication in a peer-reviewed journal. There has been some discussion among pediatric medical educators about the necessity of a residency research requirement. Proponents of a research requirement contend that research experience imparts skills that are of great importance to the practice of medicine, such as literature searching and its critical appraisal.4,17 Additionally, early research training has been associated with continued scholarly work and may guide residents’ career choices.8-13 However, research activity may take time away from clinical learning, the primary goal of residency training. Earlier reports suggest that intrinsic interest in research, motivation, and self-selection have been related to the choice 2
Vol. 155, No. 1 of a research career and a greater number of publications.18,19 Although we used the same criteria to select applicants to either track, we wondered whether interest in the primary care residency track with a research requirement would be associated with a greater number of future publications. After partially controlling for selection bias, we found that although trainees in the research requirement track had a significantly greater mean number of publications, having an interest in the primary care residency track did not, by itself, predict future publication productivity. The criteria for selection of residents to either track were the same. There was no requirement for earlier research experience or publications, as evidenced by similar publication rates in both groups before residency. We found that the differences in publication productivity during and after residency persisted even in the subgroups of residents who pursued fellowships. Fellowship training has been associated with increased publication rates.15,16 During a pediatric fellowship, there is generally a 24-month period allocated for research activities, as compared with the 6 to 8 weeks of protected time that we provided to our residents. Our findings suggest that having protected time for research may not be the sole determining factor in predicting a greater number of future publications. Publishing early in one’s career, as suggested by our findings and earlier reports,20,21 may be an important predictor of future publication productivity. Our study had some limitations. We had a small sample size, and the study was conducted at a single institution. We used a single database, the PubMed, to collect data on the number of publications. Thus, only peer-reviewed publications indexed in the PubMed database were selected. However, when we included publications in the popular pediatric journals not indexed in the PubMed, such as Contemporary Pediatrics, residents in the primary care track had even greater number of publications. We excluded these publications from the analysis because of the possibility of inadvertent omission of papers authored by our trainees and published in the variety of other medical journals not indexed in the PubMed. Residency research requirement in our training program was associated with a significantly greater mean number of future publications. The differences in publication productivity persisted even after controlling for selection bias and the pursuit of future fellowship training. Timing of the research experience and the first publication, rather than having protected time for research, may be an important predictor of future publication productivity. Depending on the goals of a pediatric residency program, a research requirement could be an effective means of promoting scholarship among trainees. n Reprint requests: Phillip O. Ozuah, MD, PhD, Professor and University Chairman, Physician-in-Chief, Department of Pediatrics, Albert Einstein College of Medicine, Children’s Hospital at Montefiore, 3415 Bainbridge Ave, Bronx, NY 10467. E-mail: [email protected].
References available at www.jpeds.com Ozuah
July 2009
Table I. Publication rates for cases and matched control subjects Publication rates Before residency During and after residency During and after residency (restricted to publications as the lead author)
Cases (n = 20)
Matched controls (n = 40)
P value*
0.5 2.7 2.1
0.45 0.4 0.3
.12 .006 .003
*P values obtained with generalized estimating equations.
Table II. Mean number of publications for cases versus categorical residents who did not match to the primary care track Mean number of publications
Cases (n = 20)
Categorical residents (n = 23)
P value*
Before residency During and after residency During and after residency (restricted to publications as the lead author)
0.50 2.65 2.10
0.13 0.26 0.22
.209 .009 .022
*P values obtained with independent samples t test.
References 1. Accreditation Council for Graduate Medical Education. Program requirements for residency education in pediatrics. Available at: http:// www.acgme.org/acWebsite/downloads/RRC_progReq/320pr01012006. pdf. Accessed Sep 20, 2007. 2. Carek PJ. Graduate surveys: an opportunity for residency research. Ann Fam Med 2005;3:376-7. 3. Rivera JA, Levine RB, Wright SM. Completing a scholarly project during residency training. Perspectives of residents who have been successful. J Gen Intern Med 2005;20:366-9. 4. Levine RB, Hebert RS, Wright SM. Resident research and scholarly activity in internal medicine residency training programs. J Gen Intern Med 2005;20:155-9. 5. Brouhard BH, Doylel W, Aceves J, McHugh MJ. Research in pediatric residency programs. Pediatrics 1996;97:71-3. 6. Neale AV, West P, French L. Surviving your resident research requirement. JAMA 1998;280:1802. 7. Mylonakis E, Koutkia P. The realities of resident research requirements. JAMA 1999;281:2089. 8. Hayward RA, Taweel F. Data and the internal medicine houseofficer: alumni’s views of the educational value of a residency program’s research requirement. J Gen Intern Med 1993;8:140-2. 9. Levine DM. Career intentions in internal medicine. Ann Intern Med 1981;94:704-6. 10. Miller ED. Clinical investigators—the endangered species revisited. JAMA 2001;286:845-6.
11. Solomon SS, Tom SC, Pichert J, Wasserman D, Powers AC. Impact of medical students research in the development of physician-scientists. J Investig Med 2003;51:149-56. 12. Lessin MS, Klein MD. Does research during general surgery residency correlate with academic pursuits after pediatric surgery residency? J Pediatr Surg 1995;30:1310-3. 13. Dunn JC, Lai EC, Brooks CM, Stabile BE, Fonkalsrud EW. The outcome of research training during surgical residency. J Pediatr Surg 1998;33:362-4. 14. US News and World Report. America’s Best Graduate Schools 2007. Available at: www.usnews.com/usnews/edu/grad/rankings/med/brief/ mdrrank_brief.php. Accessed Sep 24, 2007. 15. Powner DJ, Thomas EA. Research curricula in critical care fellowships— a survey. Crit Care Med 1996;24:1079-82. 16. Steiner JF, Lanphear BP, Curtis P, Vu KO. Indicators of early research productivity among primary care fellows. J Gen Intern Med 2002;17: 845-51. 17. Potti A, Mariani P, Saeed M, Smego RA Jr.. Residents as researchers: expectations, requirements, and productivity. Am J Med 2003;115:510-4. 18. Salive ME, Stein DH. Predictors and correlates of prevention research careers: a national study of residency graduates. Prev Med 1999;28:430-6. 19. Straus SE, Straus C, Tzanetos K; International Campaign to Revitalise Academic Medicine. Career choice in academic medicine: systematic review. J Gen Intern Med 2006;21:1222-9. 20. Ledley FD, Lovejoy FH Jr. Prospects for academically trained pediatricians in academic medicine. Clin Invest Med 1992;15:518-26. 21. Dorsey ER, Raphael BA, Balcer LJ, Galetta SL. Predictors of future publication record and academic rank in a cohort of neurology residents. Neurology 2006;67:1335-7.
Residency Research Requirement as a Predictor of Future Publication Productivity
2.e1