- Email: [email protected]
How well does early-career investigators' cardiovascular outcomes research training align with funded outcomes research?
How well does early-career investigators’ cardiovascular outcomes research training align with funded outcomes research? Matthew J. Crowley MD, MHS, Sana M. Al-Khatib MD, MHS, Tracy Y. Wang MD, MHS, MSc, Prateeti Khazanie MD, MPH, Nancy R. Kressin PhD, Harlan M. Krumholz MD, SM, Catarina I Kiefe PhD, MD, Barbara L. Wells PhD, Sean M. O’Brien PhD, Eric D. Peterson MD, MPH, Gillian D. Sanders PhD PII: DOI: Reference:
S0002-8703(17)30280-6 doi: 10.1016/j.ahj.2017.09.008 YMHJ 5531
To appear in:
American Heart Journal
Received date: Accepted date:
27 January 2017 12 September 2017
Please cite this article as: Crowley Matthew J., Al-Khatib Sana M., Wang Tracy Y., Khazanie Prateeti, Kressin Nancy R., Krumholz Harlan M., Kiefe Catarina I, Wells Barbara L., O’Brien Sean M., Peterson Eric D., Sanders Gillian D., How well does early-career investigators’ cardiovascular outcomes research training align with funded outcomes research?, American Heart Journal (2017), doi: 10.1016/j.ahj.2017.09.008
This is a PDF file of an unedited manuscript that has been accepted for publication. As a service to our customers we are providing this early version of the manuscript. The manuscript will undergo copyediting, typesetting, and review of the resulting proof before it is published in its final form. Please note that during the production process errors may be discovered which could affect the content, and all legal disclaimers that apply to the journal pertain.
ACCEPTED MANUSCRIPT
How well does early-career investigators’ cardiovascular outcomes research training align
T
with funded outcomes research?
RI P
Running Title: Do research training and funding align?
SC
Matthew J. Crowley, MD, MHS1,2,3, Sana M. Al-Khatib, MD1,4, MHS, Tracy Y. Wang, MD, MHS,
NU
MSc1,4, Prateeti Khazanie, MD, MPH5; Nancy R. Kressin PhD 6, 11, Harlan M. Krumholz MD, SM7, Catarina I Kiefe PhD, MD8, Barbara L. Wells PhD9, Sean M. O’Brien PhD3, 10, Eric D. Peterson MD,
MA
MPH.1,4, Gillian D. Sanders PhD1,2,4
1
ED
Department of Medicine, Duke University Medical Center, Durham, NC
2
Evidence-Based Practice Center, Duke Clinical Research Institute, Durham, NC
3
PT
Center for Health Services Research in Primary Care, Durham VA Medical Center, Durham, NC
4
CE
Duke Clinical Research Institute, Duke University, Durham, NC
5
Department of Medicine, University of Colorado School of Medicine, Denver, CO
6
AC
Department of Medicine, Boston University School of Medicine, Boston MA
7
Department of Medicine, Yale University, New Haven CT
8
Department of Quantitative Health Sciences, University of Massachusetts Medical School, Worcester
MA 9
Clinical Applications and Prevention Branch, Division of Cardiovascular Sciences, National Heart Lung
Blood Institute, National Institutes of Health 10
Department of Biostatistics and Bioinformatics, Duke University School of Medicine, Durham, NC
11
VA Boston Healthcare System
ACCEPTED MANUSCRIPT Corresponding Author: Gillian D. Sanders, Ph.D., Duke Clinical Research Institute, Duke Box 3485, 7020 N Pavilion Bldg., Durham, NC 27710. Phone: (919) 668-7824, Fax: (919) 668-7018, E-mail
RI P
T
[email protected]
Disclaimer: The views expressed in this manuscript are those of the authors and do not necessarily
NU
the U.S. Department of Health and Human Services
SC
represent the views of the National Heart, Lung, and Blood Institute; the National Institutes of Health; or
MA
Funding: This project was supported by the Centers for Cardiovascular Outcomes Research coordinating unit grant number U01HL107023 from the National Heart, Lung, and Blood Institute.
ED
Key Words: training, education, outcomes research, comparative effectiveness
AC
CE
PT
Word Count: 3,160 (Text)
ACCEPTED MANUSCRIPT ABSTRACT Background: Outcomes research training programs should prepare trainees to successfully
T
compete for research funding. We examined how early-career investigators’ prior and desired
RI P
training aligns with recently funded cardiovascular (CV) outcomes research.
SC
Methods/Results: We: 1) reviewed literature to identify 13 core competency areas in CV
NU
outcomes research; 2) surveyed early-career investigators to understand their prior and desired training in each competency area; 3) examined recently funded grants commonly pursued by
MA
early career outcomes researchers to ascertain available funding in competency areas; and 4) analyzed alignment between investigator training and funded research in each competency area.
ED
We evaluated 185 survey responses from early-career investigators (response rate 28%) and 521
PT
funded grants from 2010-2014. Respondents’ prior training aligned with funded grants in the areas of clinical epidemiology, observational research, randomized controlled trials, and
CE
implementation/dissemination research. Funding in community-engaged research and health
AC
informatics was more common than prior training in these areas. Respondents’ prior training in biostatistics and systematic review was more common than funded grants focusing on these specific areas. Respondents’ desired training aligned similarly with funded grants, with some exceptions; for example, desired training in health economics/cost-effectiveness research was more common than funded grants in these areas. Restricting to cardiovascular grants (n=132) and NHLBI-funded grants (n=170) produced similar results.
Conclusions: Identifying mismatch between funded grants in outcomes research and earlycareer investigators’ prior/desired training may help efforts to harmonize investigator interests,
ACCEPTED MANUSCRIPT training, and funding. Our findings suggest a need for further consideration of how to best
T
prepare early-career investigators for funding success.
AC
CE
PT
ED
MA
NU
SC
RI P
Abstract word count: 249
ACCEPTED MANUSCRIPT
Background
T
Cardiovascular disease (CVD) is the leading cause of death in the United States.[1]
RI P
Continued innovation in CVD care is needed to improve outcomes for patients with CVD.[2,3] Because cardiovascular outcomes research is central to the advancement of CVD care, this field
SC
captures the interest of many fellows and early-career investigators.
NU
Unfortunately, across clinical disciplines, reductions in research funding combined with competing clinical demands may threaten the emergence of the next generation of outcomes
MA
researchers.[4] In the current competitive research funding environment, training programs must prepare researchers to contend effectively for available funding opportunities, and to leverage
ED
these opportunities to foster long-term success. Programs like the National Heart, Lung, and
PT
Blood Institute (NHLBI) Centers for Cardiovascular Outcomes Research (CCOR) and the American Heart Association (AHA) Pharmaceutical Roundtable Outcomes Research Centers
CE
represent recent efforts to support high-quality outcomes cardiovascular (CV) research training
AC
for early-career investigators,[5-7] as do investigator-initiated, academic center-based T32 programs and VA health services research and development (HSR&D) programs.[8] In order to optimize trainees’ prospects, training programs must not only understand current clinical and policy challenges, but also how early-career CV investigators’ prior and desired training aligns with areas of research that are likely to receive funding. While little is known about how training aligns with funded grants, this information could facilitate matching investigator interests and training with areas in which funding is currently more prevalent, thereby facilitating early-career investigators’ transition to independent research. With support from the NHLBI CCOR initiative, we sought to determine how well earlycareer investigators’ prior and desired training aligns with recent trends in contemporary
ACCEPTED MANUSCRIPT outcomes research funding. We defined early stage investigators as being within 10 years from their completion of training. Given current funding priorities in the United States, our goal was
T
to provide outcomes research training programs with information to help better prepare their
RI P
trainees to become successful, independent investigators in CV outcomes research. In addition to characterizing current alignment between trainee interests and funding opportunities, we also
SC
aimed to demonstrate a process by which programs might continually examine this alignment as
NU
the funding environment evolves.
MA
Methods
In carrying out this project, we completed 4 steps: 1) we reviewed literature to identify
ED
core competency areas in outcomes research; 2) we surveyed early-career investigators (defined
PT
as within 10 years of completing their last training program) to understand their prior and desired training in each competency area; 3) we examined recently funded grants from relevant agencies
CE
in the competency areas; and 4) we descriptively analyzed alignment between early career
AC
investigators’ reported training and funded grants in each competency area.
Determination of Core Outcomes Research Competency Areas Our initial core competency areas in outcomes research was based on a list generated by the Clinical and Translational Science Awards (CTSA) Consortium Strategic Goal Committee on Comparative Effectiveness Research Workgroup on Workforce Development.[9] This initial list combined information from four expert groups [10-13] as well as a survey of 33 CTSA programs [12, 14] regarding their assessment of comparative effectiveness research capacity and needs.
ACCEPTED MANUSCRIPT Through discussions with CCOR principal investigator participants, we further expanded our list
T
to reflect additional core competency areas specific to cardiovascular outcomes research.
RI P
Early-Career Investigator Survey
We developed a survey for early-career outcomes research investigators (defined as
SC
investigators within 10 years of completing their last training program) with guidance from
network to assure clarity and content validity.
NU
CCOR investigators. We pilot tested the survey with 9 early-stage CV investigators in the CCOR
MA
Our survey consisted of 15 questions addressing respondent demographics, clinical training and specialty, mentorship, career goals, and impressions about their future prospects in
ED
CV outcomes research (Supplemental Appendix). Pertinent to the current project, the survey
PT
presented respondents with the following scenario addressing their prior and desired training: “Below we have included a list of competencies in outcomes research. Thinking about training
CE
opportunities available to you, please check all competency areas in which you have chosen to
AC
pursue training. Please also indicate those competency areas in which you would desire training in if they were available.” For their prior training, respondents were asked to mark ‘0’ for “no prior (or ongoing) training,” ‘1’ for “some prior (or ongoing) training,” and ‘2’ for “substantial prior (or ongoing) training.” For their desired training, respondents were asked to mark ‘0’ for “no desire,” ‘1’ for “some desire,” and ‘2’ for “strong desire.” To facilitate analysis, we ultimately dichotomized survey responses: for prior training, we classified responses as “substantial prior training” versus “no/some prior training” and for desired training, we classified responses as “strong desire” versus “no/some desire.”
ACCEPTED MANUSCRIPT From April to November 2014, we surveyed early-career investigator grantees funded through National Heart Lung and Blood Institute (NHLBI) and Agency for Healthcare Research
T
and Quality (AHRQ) funding opportunities and investigators funded by the Department of
RI P
Veterans Affairs (VA), Patient Centered Outcomes Research Institute (PCORI) seeking responses only from those currently defined as early-career investigators (defined as those who
SC
were within 10 years from completing their training). We supplemented these awardees with
NU
solicitations to individuals attending early investigator programs at Quality of Care and Outcomes Research (QCOR), or who were early investigators from CCOR, NHLBI T32
MA
cardiovascular programs, VA Health Services Research programs, and the American Heart Association (AHA) Roundtable programs. We approached potential respondents by e-mail,
ED
asking them to participate in an anonymous electronic survey. Three separate requests for
PT
participation were sent to each potential respondent over the course of 12 weeks. All surveys were anonymous, and no identifiable personal information was collected from any responder.
CE
Therefore, no informed consent was obtained from participants. This research, including the
AC
surveys, was approved by the institutional review board at Duke University Medical Center.
Review of Funded Grants We reviewed successfully funded grants from 2010-2014. We were interested in exploring funding available to early-career investigators in CV outcomes research, and therefore funding agencies of interest for this project included NHLBI, AHRQ, PCORI, VA HSR&D, AHA, American College of Cardiology (ACC), and the Robert Wood Johnson Foundation (RWJF). We were limited to those agencies which made publically available summaries of their funded grants which restricted our included set to: NHLBI (N=170), AHRQ (N=30), PCORI
ACCEPTED MANUSCRIPT (N=279), and the VA HSR&D (N=42). Although AHRQ, PCORI, and VA HSR&D do not specifically target cardiovascular outcomes research, these funders are key sources of potential
T
funding for young investigators regardless of clinical domain and therefore included in our
RI P
analysis.
We reviewed each grant summary to determine which of our core competency areas (if
SC
any) were relevant to the project; more than one competency area could be selected for each
NU
grant. We also identified which grants focused on cardiovascular research. All grants were reviewed by a member of our project team, and overread by another team member. When
MA
disagreements arose between these two reviewers, they were resolved through discussions
ED
between the two primary reviewers or were adjudicated by a third team member.
PT
Alignment Between Training and Funded Grants We analyzed data gathered from our early-career investigator survey and funded grant
CE
review to evaluate alignment between investigators’ prior/desired training and funded research
AC
during the review period. Based on the proportion of the survey respondents endorsing “substantial prior training” for each competency, we allocated our core competency areas into quartiles of prior training (with the top quartile including areas for which the greatest proportion of respondents reported substantial prior training). We repeated this process for desired training based on the proportion of the survey respondents expressing “strong desire” for additional training. We then allocated our competency areas into quartiles based on the proportion of funded grants relevant to the competency (such that the top quartile included those competency areas represented by the greatest proportion of funded grants).
ACCEPTED MANUSCRIPT In order to examine how early-career investigators’ prior and desired training aligned with successfully funded grants within our core competency areas, we cross-tabulated the prior
T
and desired training quartiles with the quartiles from our grant review. These tables allowed us to
RI P
identify competency areas for which there was general alignment between training and funded grants, as well as those for which there was mismatch. Competency areas for which there was
SC
mismatch included those where investigators’ interest appeared to exceed funded research, as
NU
well as areas where funded research appeared to exceed investigator interest (Figure 1). We generated separate tables reflecting alignment between investigators’: 1) prior
MA
training and available funding; and 2) desired training and available funding. In order to specifically inform cardiovascular outcomes research training programs and specifically the
ED
NHLBI CCOR program which funded this work, we also generated tables limited to CV-focused
CE
Analyses
PT
grants only (N=132) and NHLBI grants only.
AC
We described baseline characteristics of survey respondents using SAS version 9.2 (SAS Institute Inc, Cary, North Carolina). All other analyses were descriptive.
Funding This project was supported by the Centers for Cardiovascular Outcomes Research coordinating unit grant number U01HL107023 from the National Heart, Lung, and Blood Institute. The authors are solely responsible for the design and conduct of this study, all study analyses, the drafting and editing of the paper and its final contents.
ACCEPTED MANUSCRIPT
Results
T
We identified 13 core competency areas in outcomes research, which are displayed in
RI P
Table 1. We sent our early-career investigator survey to 662 eligible participants, and received a total of 185 responses (28%). Because the survey response was anonymous, we could not obtain
SC
information on non-responders. Table 2 summarizes the survey respondents’ characteristics.
NU
Respondents were equally split by sex; most were between 31 and 40 years of age, and the most common specialty was Cardiology. The majority of respondents had completed training over 3
MA
years ago and spent more than 50% of their time on research; over 70% of respondents wished to spend more than 50% time on research in the future. Most respondents had entered into their
ED
training with the intention to remain in academics and finished training with the same goal.
PT
Table 1 summarizes how many of the 185 survey responses endorsed prior or desired training in each competency area, as well as how many of the 521 funded grants were relevant to
AC
training and funding.
CE
each core competency area. Supplemental Table 1 indicates the absolute mismatch between
Alignment Between Training and Funded Grants Based on our cross-tabulation (Figure 2), respondents’ prior training aligned with funded grants in the areas of clinical epidemiology, observational research, randomized controlled trials, and implementation and dissemination research. There was also relative alignment between low rates of prior training and fewer funded grants in the areas of pharmacoepidemiology, practicebased network research, decision analysis, and health economics. Areas of mismatch where respondents reported less prior training but more grants had been funded included community-
ACCEPTED MANUSCRIPT engaged research and biomedical/health informatics/information needs assessment. Areas with fewer funded grants where respondents felt well-trained included biostatistics, systematic review,
T
and practical clinical trials.
RI P
We found that respondents’ desired training aligned with funded grants in the competency areas of randomized controlled trials, implementation and dissemination research,
SC
clinical epidemiology, and observational research (Figure 3). There was relative alignment
NU
between low desire for training and fewer funded grants in the areas of pharmacoepidemiology, systematic review, practice-based network research, and decision analysis. Areas of mismatch
MA
where respondents did not express a strong desire for further training despite a higher number of funded grants included community-engaged research and health informatics/information needs
ED
assessment. Less-funded areas where respondents desired additional training included
PT
biostatistics, practical clinical trials, and health economics/cost-effectiveness research. We also assessed the success of respondents in receiving funding by those who had
CE
significant training in core competency areas. Of the 185 trainees, 163 (88%) had submitted one
AC
or more proposals for research funding in the past 3 years and 140 (86% of 163) of these trainees had one or more of their submitted proposals funded. The percent of applicants who had at least one proposal funded was 93% (101 of 109) in the subgroup of trainees who had significant training in any of the top quartile core competency areas associated with grant funding (Table 1) compared to 76% (38 of 50) in the subgroup of trainees without substantial training in those core competency areas (Chi-square p = 0.003). Compared to our analysis of all funded grants, restricting to CV-focused grants (n=132, Supplemental Figures 1 and 2) led to no changes in quartile alignment between funded grants and investigators’ prior and desired training. Restricting to NHLBI grants (n=170, Supplemental
ACCEPTED MANUSCRIPT Figures 3 and 4) yielded results that were generally similar to the ‘all grants’ analysis; the only change in quartile alignment was in the area of practical clinical trials, for which there was
T
relatively more funding in the NHLBI pool of grants. This difference led to greater alignment
RI P
with investigators’ prior and desired training.
SC
Discussion
NU
Given the current challenging outcomes research funding landscape, focusing training efforts on areas where cardiovascular investigators have good prospects for funding success may
MA
be important. Conversely, identifying and understanding mismatch between training and funding – which could reflect a variety of factors, including changing perceptions among trainees,
ED
evolution in research methods, and shifts in the funding landscape – might help provide greater
PT
guidance for outcomes research training programs and funders. In this NHLBI-funded examination of alignment between early-career investigators’ prior/desired training and funded
CE
grants, we found several core competency areas where training and funding aligned, and others
AC
in which mismatch existed. Harmonizing research interests and training with funding could help promote success for early-career outcomes research investigators.
Areas of Alignment Between Training and Funding Areas where we found alignment between early-career investigators’ prior/desired training and funded grants included randomized controlled trials, clinical epidemiology, observational research, and implementation and dissemination research. Given that there appears to be both a strong interest and grant support in these domains, they may continue to represent high-yield areas of focus for outcomes research training programs. Conversely, we consistently
ACCEPTED MANUSCRIPT found relatively little investigator prior/desired training or currently available funding in pharmacoepidemiology, practice-based network research, and decision analysis. Consequently,
T
these may be less fruitful targets for outcomes research training.
RI P
Of note, while we found relatively few funded grants in health economics/costeffectiveness research and systematic review, we identified inconsistencies in investigators’ prior
SC
training and desire for further training in these competency areas. While respondents endorsed
NU
little prior training in health economics/cost-effectiveness research – aligning with the relative paucity of funded grants in this area – their desire for further training was higher, resulting in
MA
mismatch with funding. In contrast, respondents reported ample prior training in systematic review, which was discrepant with the low funding in this area; however, investigators’
ED
relatively low desire for further training in systematic review aligned with funding. Early-career
PT
investigators and outcomes research programs should carefully consider how to prioritize training in these competency areas, recognizing that some topics or processes (e.g., systematic
CE
literature review) may represent important skills for CV outcomes researchers, regardless of
AC
funding opportunities.
Areas of Mismatch Between Training and Funding We found a consistent mismatch between early-career investigators’ prior/desired training and funded grants in some areas. Investigators’ training appeared to exceed awarded grants in the area of biostatistics and practical clinical trials. However, it is important to note that competency in biostatistics is critical to research success across outcomes research disciplines. For this reason, even though biostatistics is seldom the sole focus of any cardiovascular outcomes research funding proposal, programs should continue to focus on biostatistics as a core
ACCEPTED MANUSCRIPT competency for future outcomes researchers. Conversely, areas for which funding was awarded but in which investigators expressed relatively little interest included community-engaged
T
research and health informatics/information needs assessment. These areas may represent
RI P
‘missed opportunities’ for outcomes research training programs, and may warrant additional consideration as fruitful targets for training. These findings might also support the notion that
SC
these areas could be fruitful for development of collaborative relationships between outcomes
NU
researchers and researchers with expertise in community-engaged research and health
MA
informatics.
Limitations
ED
Although this analysis is the largest to date to examine alignment between early career
PT
investigators’ training perspectives and funded grants, it has limitations. While we attempted to assure the broadest possible survey dissemination, we could not evaluate our sample’s
CE
comprehensiveness and representativeness because: 1) the true denominator for outcomes
AC
research trainees is unknown; and 2) the survey was anonymous. Our survey response rate was relatively low (28%), so may not represent all trainee perspectives. However, certain factors may have artificially lowered our apparent response rate, making 28% a conservative estimate. First, because we reached out to a large set of awardees who may not be early-career investigators (awardees from PCORI) or who may have transitioned to mid-career (awardees from training grants within NHLBI and AHRQ who were funded in the earlier years of our included sample and who by the time of the survey may have been past 10 years since training) we were unable to ascertain which awardees failed to respond because they were no longer within 10 years of training, so not relevant to our study. Second, our survey was distributed to 314 QCOR attendees
ACCEPTED MANUSCRIPT directly by AHA. Because this private email distribution list could not be checked against our other distribution lists (awardees, T32 trainees) for duplicates, individuals on this list may have
T
received duplicate survey requests. Our true denominator was therefore unclear and our response
RI P
rate is a conservative estimate.
Another limitation is that we limited our review to major funding agencies that: 1) often
SC
support early-career investigators; and 2) made public their funded grant summaries. The latter
NU
requirement meant that we may have omitted some funders of early-career investigators in CV outcomes research (e.g., AHA, RWJF), and may have missed available funding in other areas
MA
(e.g., professional societies for systematic reviews, internal funding mechanisms for quality improvement efforts, etc.). Further, we did not include unsuccessful grants in this analysis,
ED
because information on unfunded proposals is not publicly available. Including unsuccessful
PT
grants in future analyses may provide additional context that is valuable to training programs. Because funding trends evolve over time (e.g., recently available funding for community-
CE
engaged research through PCORI), it will be important to update this exercise periodically.
AC
Similarly, competencies of interest evolve over time – for example, while patient-reported outcomes were not part of this analysis, recent years have increased the focus on use of these constructs to understand patient perspectives.[15] Therefore, the present analysis is not simply static guidance for programs, but also a demonstration of a dynamic process programs might use to assure responsiveness to changes in the funding landscape and relevant competencies.
Conclusions In this review of 521 funded grants from 2010-2014 and survey of 185 early-career investigators from CV outcomes research training programs, we identified numerous areas of
ACCEPTED MANUSCRIPT alignment between investigator interest and recent funding, and several areas of mismatch. Trends in alignment with recent funding were similar for respondents’ prior training and their
T
desired training. Our findings may inform efforts to improve alignment between training and
RI P
funding, which could promote success for early-career investigators. A critical step toward this success will be continued recognition of early-career investigators’ perspectives, as they possess
SC
the best understanding of “on the ground” conditions surrounding the transition to independent
MA
NU
investigators.
Acknowledgements
ED
MJC is supported by a Career Development Award from VHA Health Services Research
AC
CE
PT
and Development (CDA 13-261).
ACCEPTED MANUSCRIPT References 1. Mozaffarian D, Benjamin EJ, Go AS, Arnett DK, Blaha MJ, Cushman M, de Ferranti S,
RI P
T
Després J-P, Fullerton HJ, Howard VJ, Huffman MD, Judd SE, Kissela BM, Lackland DT, Lichtman JH, Lisabeth LD, Liu S, Mackey RH, Matchar DB, McGuire DK, Mohler
SC
ER 3rd, Moy CS, Muntner P, Mussolino ME, Nasir K, Neumar RW, Nichol G, Palaniappan L, Pandey DK, Reeves MJ, Rodriguez CJ, Sorlie PD, Stein J, Towfighi A,
NU
Turan TN, Virani SS, Willey JZ, Woo D, Yeh RW, Turner MB; on behalf of the
MA
American Heart Association Statistics Committee and Stroke Statistics Subcommittee. Heart disease and stroke statistics—2015 update: a report from the American Heart
ED
Association. Circulation. 2015; 131:e29–e322. 2. Krumholz HM, Peterson ED, Ayanian JZ, Chin MH, DeBusk RF, Goldman L, Kiefe
PT
CI, Powe NR, Rumsfeld JS, Spertus JA, Weintraub WS; National Heart, Lung, and Blood
CE
Institute working group. Report of the National Heart, Lung, and Blood Institute working
66.
AC
group on outcomes research in cardiovascular disease. Circulation. 2005; 111(23):3158-
3. Krumholz HM. Outcomes Research: Generating Evidence for Best Practice and Policies. Circulation. 2008; 118:309-318. 4. Tong CW, Ahmad T, Brittain EL, Bunch TJ, Damp JB, Dardas T, Hijar A, Hill JA, Hilliard AA, Houser SR, Jahangir E, Kates AM, Kim D, Lindman BR, Ryan JJ, Rzeszut AK, Sivaram CA, Valente AM, Freeman AM. Challenges facing early career academic cardiologists. J Am Coll Cardiol. 2014; 63(21):2199-208. 5. Centers for Cardiovascular Outcomes Research (CCOR). About CCOR. Available at: https://www.cardiovascular-outcomes.org/. Accessed May 26, 2016.
ACCEPTED MANUSCRIPT 6. Cook NL, Bonds, DE, Kiefe CI, Curtis JP, Krumholz HM, Kressin, NR, Peterson ED. Centers for Cardiovascular Outcomes Research: Defining a Collaborative Vision.
T
Circulation: Cardiovascular Quality and Outcomes. 2013;6(2):223-8.
RI P
7. Peterson ED, Spertus JA, Cohen DJ, et al. Vision and creation of the American Heart Association pharmaceutical roundtable outcomes research centers. Circulation.
SC
Cardiovascular quality and outcomes. 2009; 2(6):663-70.
NU
8. National Institutes of Health. National Heart, Lung, and Blood Institute Training and Career Development. Available at: https://www.nhlbi.nih.gov/research/training, Accessed
MA
September 28, 2016.
9. Kroenke K, Kapoor W, Helfand M, Meltzer DO, McDonald MA, Selker H. Training and
ED
career development for comparative effectiveness research workforce development:
PT
CTSA Consortium Strategic Goal Committee on comparative effectiveness research workgroup on workforce development Clin Transl Sci. 2010; 3(5):258-62.
CE
10. Committee on Comparative Effectiveness Research Prioritization, Institute of Medicine
AC
(IOM) Initial National Priorities for Comparative Effectiveness Research. Washington, DC: The National Academies Press; 2009. 11. Hersh W, Carey T, Ricketts T, Helfand M, Floyd N, Shiffman R, Hickman D. Institute of Medicine Roundtable on Evidence-Based Medicine. 2009. A Framework for the Workforce Required for Comparative Effectiveness Research. 12. CTSA Strategic Goal 4B-1 Subcommittee. Assessment of Comparative Effectiveness Research Capacity and Needs for CTSA Institutions. 2009. 13. CTSA Education and Career Development Workgroup. Core Competencies in Clinical and Translational Research for Master’s Candidates. 2009.
ACCEPTED MANUSCRIPT 14. Selker HP, Strom BL, Ford DE, Meltzer DO, Pauker SG, Pincus HA, Rich EC, Tompkins
C, Whitlock EP. White paper on CTSA Consortium role in facilitating comparative
T
effectiveness research. Clin Transl Sci. 2010;3:29–37.
RI P
15. Tucker CA, Cieza A, Riley AW, Stucki G, Lai JS, Bedirhan Ustun T, Kostanjsek N,
Riley W, Cella D, Forrest CB. Concept analysis of the patient reported
SC
outcomes measurement information system (PROMIS(®)) and the international
NU
classification of functioning, disability and health (ICF). Qual Life Res. 2014
AC
CE
PT
ED
MA
Aug;23(6):1677-86.
ACCEPTED MANUSCRIPT Tables
Substantial prior training
Strong desire for training
Quart.
N(%)
Quart.
N(%)
Grants Core competency area
Quart.
521(100)
-
185(100)
-
185(100)
-
Clinical epidemiology
383 (72)
1
72 (39)
1
45 (24)
2
Community-engaged research/patient & stakeholder engagement
309 (58)
1
18 (10)
3
36 (19)
3
Observational research
218 (42)
1
105 (57)
1
50 (27)
2
122 (23)
2
48 (26)
2
61 (33)
1
84 (16)
2
12 (6)
4
29 (16)
4
78 (15)
2
30 (16)
2
66 (36)
1
51 (10)
3
15 (8)
3
41 (22)
3
50 (10)
3
33 (18)
2
94 (51)
1
26 (5)
3
20 (11)
3
59 (32)
2
26 (5)
3
4 (2)
4
28 (15)
4
22 (4)
4
87 (47)
1
61 (33)
1
7 (1)
4
30 (16)
2
36 (19)
3
4 (1)
4
13 (7)
4
20 (11)
4
Randomized controlled trials Biomedical/health informatics & information needs assessment
MA
Implementation and dissemination research Decision analysis/cognitive sciences Practical clinical trials
ED
Health economics/cost-effectiveness research Practice-based network research Biostatistics
PT
Systematic reviews
AC
CE
Pharmacoepidemiology
SC
Total Ns: number of grants reviewed and of survey respondents
NU
N (%)
RI P
T
Table 1. Identified core competencies, with number of grants relevant to each competency area (with quartiles) and number of survey respondents indicating substantial prior training and strong desire for training in each area (with quartiles)
ACCEPTED MANUSCRIPT Table 2. Characteristics of survey respondents.
Age ≤ 30 years 31-35 years 36-40 years 41-45 years >45 years
15 58 64 32 16
(8) (31) (35) (17) (9)
Training/Career Stage In research training Early stages <3 years after completing training 3-5 years after completing training 6-10 years after completing training
32 14 36 53 50
(17) (8) (19) (29) (27)
81 17 14 6 4 2 2 1 33 23 2
(44) (9) (8) (3) (2) (1) (1) (1) (18) (12) (1)
AC
MA ED
PT
CE
Clinical Specialty Cardiology Internal Medicine Psychiatry/Psychology Pediatrics Surgery Family Medicine Hospitalist Neurology Other Non-clinical Missing
RI P
91 (49) 92 (50) 2 (1)
NU
Gender Male Female Missing
T
N (%)
SC
Respondent characteristic
ACCEPTED MANUSCRIPT Figures Figure 1. Cross-tabulation of quartiles of early-career investigator prior and desired training and
T
funded grants for core competency areas.
RI P
SC
NU MA
4th quartile
ED
3rd quartile
PT
2nd quartile
nd
CE
1st quartile
AC
Survey results
Grant review results 1 quartile 2 quartile 3rd quartile 4th quartile Alignment: Mismatch: Areas with more prior/desired training Areas with more prior/desired training and and more funded grants fewer funded grants Mismatch: Alignment: Areas with less prior/desired training and Areas with less prior/desired training and more funded grants fewer funded grants st
ACCEPTED MANUSCRIPT Figure 2. Alignment of quartiles between funded grants and investigators’ prior training (all grants).
T Randomized controlled trials Implementation and dissemination
nd
2 quartile
MA
3rd quartile
Biomedical/Health informatics and information needs assessment
AC
CE
PT
ED
4th quartile
Practical clinical trials
4th quartile Biostatistics
Systematic reviews
Health economics/ cost-effectiveness research Decision analysis/ cognitive sciences Practice-based network research
NU
Communityengaged research
RI P
1st quartile
1 quartile Observational research Clinical epidemiology
SC
Prior training
Successfully funded grants (all grants) 2nd quartile 3rd quartile
st
Pharmacoepidemiology
ACCEPTED MANUSCRIPT Figure 3. Alignment of quartiles between funded grants and investigators’ desired training (all grants).
nd
2 quartile
3rd quartile
Observational research Clinical epidemiology Communityengaged research
SC
1st quartile
Biomedical/health informatics and information needs assessment
th
AC
CE
PT
ED
MA
4 quartile
4th quartile Biostatistics
RI P
T
1 quartile
Successfully-funded grants (all grants) 2nd quartile 3rd quartile Implementation Practical clinical and dissemination trials Randomized controlled trials Health economics/ cost-effectiveness research Decision analysis/ cognitive sciences Practice-based network research
NU
Desired training
st
Systematic reviews Pharmacoepidemiology
S0002-8703(17)30280-6 doi: 10.1016/j.ahj.2017.09.008 YMHJ 5531
To appear in:
American Heart Journal
Received date: Accepted date:
27 January 2017 12 September 2017
Please cite this article as: Crowley Matthew J., Al-Khatib Sana M., Wang Tracy Y., Khazanie Prateeti, Kressin Nancy R., Krumholz Harlan M., Kiefe Catarina I, Wells Barbara L., O’Brien Sean M., Peterson Eric D., Sanders Gillian D., How well does early-career investigators’ cardiovascular outcomes research training align with funded outcomes research?, American Heart Journal (2017), doi: 10.1016/j.ahj.2017.09.008
This is a PDF file of an unedited manuscript that has been accepted for publication. As a service to our customers we are providing this early version of the manuscript. The manuscript will undergo copyediting, typesetting, and review of the resulting proof before it is published in its final form. Please note that during the production process errors may be discovered which could affect the content, and all legal disclaimers that apply to the journal pertain.
ACCEPTED MANUSCRIPT
How well does early-career investigators’ cardiovascular outcomes research training align
T
with funded outcomes research?
RI P
Running Title: Do research training and funding align?
SC
Matthew J. Crowley, MD, MHS1,2,3, Sana M. Al-Khatib, MD1,4, MHS, Tracy Y. Wang, MD, MHS,
NU
MSc1,4, Prateeti Khazanie, MD, MPH5; Nancy R. Kressin PhD 6, 11, Harlan M. Krumholz MD, SM7, Catarina I Kiefe PhD, MD8, Barbara L. Wells PhD9, Sean M. O’Brien PhD3, 10, Eric D. Peterson MD,
MA
MPH.1,4, Gillian D. Sanders PhD1,2,4
1
ED
Department of Medicine, Duke University Medical Center, Durham, NC
2
Evidence-Based Practice Center, Duke Clinical Research Institute, Durham, NC
3
PT
Center for Health Services Research in Primary Care, Durham VA Medical Center, Durham, NC
4
CE
Duke Clinical Research Institute, Duke University, Durham, NC
5
Department of Medicine, University of Colorado School of Medicine, Denver, CO
6
AC
Department of Medicine, Boston University School of Medicine, Boston MA
7
Department of Medicine, Yale University, New Haven CT
8
Department of Quantitative Health Sciences, University of Massachusetts Medical School, Worcester
MA 9
Clinical Applications and Prevention Branch, Division of Cardiovascular Sciences, National Heart Lung
Blood Institute, National Institutes of Health 10
Department of Biostatistics and Bioinformatics, Duke University School of Medicine, Durham, NC
11
VA Boston Healthcare System
ACCEPTED MANUSCRIPT Corresponding Author: Gillian D. Sanders, Ph.D., Duke Clinical Research Institute, Duke Box 3485, 7020 N Pavilion Bldg., Durham, NC 27710. Phone: (919) 668-7824, Fax: (919) 668-7018, E-mail
RI P
T
[email protected]
Disclaimer: The views expressed in this manuscript are those of the authors and do not necessarily
NU
the U.S. Department of Health and Human Services
SC
represent the views of the National Heart, Lung, and Blood Institute; the National Institutes of Health; or
MA
Funding: This project was supported by the Centers for Cardiovascular Outcomes Research coordinating unit grant number U01HL107023 from the National Heart, Lung, and Blood Institute.
ED
Key Words: training, education, outcomes research, comparative effectiveness
AC
CE
PT
Word Count: 3,160 (Text)
ACCEPTED MANUSCRIPT ABSTRACT Background: Outcomes research training programs should prepare trainees to successfully
T
compete for research funding. We examined how early-career investigators’ prior and desired
RI P
training aligns with recently funded cardiovascular (CV) outcomes research.
SC
Methods/Results: We: 1) reviewed literature to identify 13 core competency areas in CV
NU
outcomes research; 2) surveyed early-career investigators to understand their prior and desired training in each competency area; 3) examined recently funded grants commonly pursued by
MA
early career outcomes researchers to ascertain available funding in competency areas; and 4) analyzed alignment between investigator training and funded research in each competency area.
ED
We evaluated 185 survey responses from early-career investigators (response rate 28%) and 521
PT
funded grants from 2010-2014. Respondents’ prior training aligned with funded grants in the areas of clinical epidemiology, observational research, randomized controlled trials, and
CE
implementation/dissemination research. Funding in community-engaged research and health
AC
informatics was more common than prior training in these areas. Respondents’ prior training in biostatistics and systematic review was more common than funded grants focusing on these specific areas. Respondents’ desired training aligned similarly with funded grants, with some exceptions; for example, desired training in health economics/cost-effectiveness research was more common than funded grants in these areas. Restricting to cardiovascular grants (n=132) and NHLBI-funded grants (n=170) produced similar results.
Conclusions: Identifying mismatch between funded grants in outcomes research and earlycareer investigators’ prior/desired training may help efforts to harmonize investigator interests,
ACCEPTED MANUSCRIPT training, and funding. Our findings suggest a need for further consideration of how to best
T
prepare early-career investigators for funding success.
AC
CE
PT
ED
MA
NU
SC
RI P
Abstract word count: 249
ACCEPTED MANUSCRIPT
Background
T
Cardiovascular disease (CVD) is the leading cause of death in the United States.[1]
RI P
Continued innovation in CVD care is needed to improve outcomes for patients with CVD.[2,3] Because cardiovascular outcomes research is central to the advancement of CVD care, this field
SC
captures the interest of many fellows and early-career investigators.
NU
Unfortunately, across clinical disciplines, reductions in research funding combined with competing clinical demands may threaten the emergence of the next generation of outcomes
MA
researchers.[4] In the current competitive research funding environment, training programs must prepare researchers to contend effectively for available funding opportunities, and to leverage
ED
these opportunities to foster long-term success. Programs like the National Heart, Lung, and
PT
Blood Institute (NHLBI) Centers for Cardiovascular Outcomes Research (CCOR) and the American Heart Association (AHA) Pharmaceutical Roundtable Outcomes Research Centers
CE
represent recent efforts to support high-quality outcomes cardiovascular (CV) research training
AC
for early-career investigators,[5-7] as do investigator-initiated, academic center-based T32 programs and VA health services research and development (HSR&D) programs.[8] In order to optimize trainees’ prospects, training programs must not only understand current clinical and policy challenges, but also how early-career CV investigators’ prior and desired training aligns with areas of research that are likely to receive funding. While little is known about how training aligns with funded grants, this information could facilitate matching investigator interests and training with areas in which funding is currently more prevalent, thereby facilitating early-career investigators’ transition to independent research. With support from the NHLBI CCOR initiative, we sought to determine how well earlycareer investigators’ prior and desired training aligns with recent trends in contemporary
ACCEPTED MANUSCRIPT outcomes research funding. We defined early stage investigators as being within 10 years from their completion of training. Given current funding priorities in the United States, our goal was
T
to provide outcomes research training programs with information to help better prepare their
RI P
trainees to become successful, independent investigators in CV outcomes research. In addition to characterizing current alignment between trainee interests and funding opportunities, we also
SC
aimed to demonstrate a process by which programs might continually examine this alignment as
NU
the funding environment evolves.
MA
Methods
In carrying out this project, we completed 4 steps: 1) we reviewed literature to identify
ED
core competency areas in outcomes research; 2) we surveyed early-career investigators (defined
PT
as within 10 years of completing their last training program) to understand their prior and desired training in each competency area; 3) we examined recently funded grants from relevant agencies
CE
in the competency areas; and 4) we descriptively analyzed alignment between early career
AC
investigators’ reported training and funded grants in each competency area.
Determination of Core Outcomes Research Competency Areas Our initial core competency areas in outcomes research was based on a list generated by the Clinical and Translational Science Awards (CTSA) Consortium Strategic Goal Committee on Comparative Effectiveness Research Workgroup on Workforce Development.[9] This initial list combined information from four expert groups [10-13] as well as a survey of 33 CTSA programs [12, 14] regarding their assessment of comparative effectiveness research capacity and needs.
ACCEPTED MANUSCRIPT Through discussions with CCOR principal investigator participants, we further expanded our list
T
to reflect additional core competency areas specific to cardiovascular outcomes research.
RI P
Early-Career Investigator Survey
We developed a survey for early-career outcomes research investigators (defined as
SC
investigators within 10 years of completing their last training program) with guidance from
network to assure clarity and content validity.
NU
CCOR investigators. We pilot tested the survey with 9 early-stage CV investigators in the CCOR
MA
Our survey consisted of 15 questions addressing respondent demographics, clinical training and specialty, mentorship, career goals, and impressions about their future prospects in
ED
CV outcomes research (Supplemental Appendix). Pertinent to the current project, the survey
PT
presented respondents with the following scenario addressing their prior and desired training: “Below we have included a list of competencies in outcomes research. Thinking about training
CE
opportunities available to you, please check all competency areas in which you have chosen to
AC
pursue training. Please also indicate those competency areas in which you would desire training in if they were available.” For their prior training, respondents were asked to mark ‘0’ for “no prior (or ongoing) training,” ‘1’ for “some prior (or ongoing) training,” and ‘2’ for “substantial prior (or ongoing) training.” For their desired training, respondents were asked to mark ‘0’ for “no desire,” ‘1’ for “some desire,” and ‘2’ for “strong desire.” To facilitate analysis, we ultimately dichotomized survey responses: for prior training, we classified responses as “substantial prior training” versus “no/some prior training” and for desired training, we classified responses as “strong desire” versus “no/some desire.”
ACCEPTED MANUSCRIPT From April to November 2014, we surveyed early-career investigator grantees funded through National Heart Lung and Blood Institute (NHLBI) and Agency for Healthcare Research
T
and Quality (AHRQ) funding opportunities and investigators funded by the Department of
RI P
Veterans Affairs (VA), Patient Centered Outcomes Research Institute (PCORI) seeking responses only from those currently defined as early-career investigators (defined as those who
SC
were within 10 years from completing their training). We supplemented these awardees with
NU
solicitations to individuals attending early investigator programs at Quality of Care and Outcomes Research (QCOR), or who were early investigators from CCOR, NHLBI T32
MA
cardiovascular programs, VA Health Services Research programs, and the American Heart Association (AHA) Roundtable programs. We approached potential respondents by e-mail,
ED
asking them to participate in an anonymous electronic survey. Three separate requests for
PT
participation were sent to each potential respondent over the course of 12 weeks. All surveys were anonymous, and no identifiable personal information was collected from any responder.
CE
Therefore, no informed consent was obtained from participants. This research, including the
AC
surveys, was approved by the institutional review board at Duke University Medical Center.
Review of Funded Grants We reviewed successfully funded grants from 2010-2014. We were interested in exploring funding available to early-career investigators in CV outcomes research, and therefore funding agencies of interest for this project included NHLBI, AHRQ, PCORI, VA HSR&D, AHA, American College of Cardiology (ACC), and the Robert Wood Johnson Foundation (RWJF). We were limited to those agencies which made publically available summaries of their funded grants which restricted our included set to: NHLBI (N=170), AHRQ (N=30), PCORI
ACCEPTED MANUSCRIPT (N=279), and the VA HSR&D (N=42). Although AHRQ, PCORI, and VA HSR&D do not specifically target cardiovascular outcomes research, these funders are key sources of potential
T
funding for young investigators regardless of clinical domain and therefore included in our
RI P
analysis.
We reviewed each grant summary to determine which of our core competency areas (if
SC
any) were relevant to the project; more than one competency area could be selected for each
NU
grant. We also identified which grants focused on cardiovascular research. All grants were reviewed by a member of our project team, and overread by another team member. When
MA
disagreements arose between these two reviewers, they were resolved through discussions
ED
between the two primary reviewers or were adjudicated by a third team member.
PT
Alignment Between Training and Funded Grants We analyzed data gathered from our early-career investigator survey and funded grant
CE
review to evaluate alignment between investigators’ prior/desired training and funded research
AC
during the review period. Based on the proportion of the survey respondents endorsing “substantial prior training” for each competency, we allocated our core competency areas into quartiles of prior training (with the top quartile including areas for which the greatest proportion of respondents reported substantial prior training). We repeated this process for desired training based on the proportion of the survey respondents expressing “strong desire” for additional training. We then allocated our competency areas into quartiles based on the proportion of funded grants relevant to the competency (such that the top quartile included those competency areas represented by the greatest proportion of funded grants).
ACCEPTED MANUSCRIPT In order to examine how early-career investigators’ prior and desired training aligned with successfully funded grants within our core competency areas, we cross-tabulated the prior
T
and desired training quartiles with the quartiles from our grant review. These tables allowed us to
RI P
identify competency areas for which there was general alignment between training and funded grants, as well as those for which there was mismatch. Competency areas for which there was
SC
mismatch included those where investigators’ interest appeared to exceed funded research, as
NU
well as areas where funded research appeared to exceed investigator interest (Figure 1). We generated separate tables reflecting alignment between investigators’: 1) prior
MA
training and available funding; and 2) desired training and available funding. In order to specifically inform cardiovascular outcomes research training programs and specifically the
ED
NHLBI CCOR program which funded this work, we also generated tables limited to CV-focused
CE
Analyses
PT
grants only (N=132) and NHLBI grants only.
AC
We described baseline characteristics of survey respondents using SAS version 9.2 (SAS Institute Inc, Cary, North Carolina). All other analyses were descriptive.
Funding This project was supported by the Centers for Cardiovascular Outcomes Research coordinating unit grant number U01HL107023 from the National Heart, Lung, and Blood Institute. The authors are solely responsible for the design and conduct of this study, all study analyses, the drafting and editing of the paper and its final contents.
ACCEPTED MANUSCRIPT
Results
T
We identified 13 core competency areas in outcomes research, which are displayed in
RI P
Table 1. We sent our early-career investigator survey to 662 eligible participants, and received a total of 185 responses (28%). Because the survey response was anonymous, we could not obtain
SC
information on non-responders. Table 2 summarizes the survey respondents’ characteristics.
NU
Respondents were equally split by sex; most were between 31 and 40 years of age, and the most common specialty was Cardiology. The majority of respondents had completed training over 3
MA
years ago and spent more than 50% of their time on research; over 70% of respondents wished to spend more than 50% time on research in the future. Most respondents had entered into their
ED
training with the intention to remain in academics and finished training with the same goal.
PT
Table 1 summarizes how many of the 185 survey responses endorsed prior or desired training in each competency area, as well as how many of the 521 funded grants were relevant to
AC
training and funding.
CE
each core competency area. Supplemental Table 1 indicates the absolute mismatch between
Alignment Between Training and Funded Grants Based on our cross-tabulation (Figure 2), respondents’ prior training aligned with funded grants in the areas of clinical epidemiology, observational research, randomized controlled trials, and implementation and dissemination research. There was also relative alignment between low rates of prior training and fewer funded grants in the areas of pharmacoepidemiology, practicebased network research, decision analysis, and health economics. Areas of mismatch where respondents reported less prior training but more grants had been funded included community-
ACCEPTED MANUSCRIPT engaged research and biomedical/health informatics/information needs assessment. Areas with fewer funded grants where respondents felt well-trained included biostatistics, systematic review,
T
and practical clinical trials.
RI P
We found that respondents’ desired training aligned with funded grants in the competency areas of randomized controlled trials, implementation and dissemination research,
SC
clinical epidemiology, and observational research (Figure 3). There was relative alignment
NU
between low desire for training and fewer funded grants in the areas of pharmacoepidemiology, systematic review, practice-based network research, and decision analysis. Areas of mismatch
MA
where respondents did not express a strong desire for further training despite a higher number of funded grants included community-engaged research and health informatics/information needs
ED
assessment. Less-funded areas where respondents desired additional training included
PT
biostatistics, practical clinical trials, and health economics/cost-effectiveness research. We also assessed the success of respondents in receiving funding by those who had
CE
significant training in core competency areas. Of the 185 trainees, 163 (88%) had submitted one
AC
or more proposals for research funding in the past 3 years and 140 (86% of 163) of these trainees had one or more of their submitted proposals funded. The percent of applicants who had at least one proposal funded was 93% (101 of 109) in the subgroup of trainees who had significant training in any of the top quartile core competency areas associated with grant funding (Table 1) compared to 76% (38 of 50) in the subgroup of trainees without substantial training in those core competency areas (Chi-square p = 0.003). Compared to our analysis of all funded grants, restricting to CV-focused grants (n=132, Supplemental Figures 1 and 2) led to no changes in quartile alignment between funded grants and investigators’ prior and desired training. Restricting to NHLBI grants (n=170, Supplemental
ACCEPTED MANUSCRIPT Figures 3 and 4) yielded results that were generally similar to the ‘all grants’ analysis; the only change in quartile alignment was in the area of practical clinical trials, for which there was
T
relatively more funding in the NHLBI pool of grants. This difference led to greater alignment
RI P
with investigators’ prior and desired training.
SC
Discussion
NU
Given the current challenging outcomes research funding landscape, focusing training efforts on areas where cardiovascular investigators have good prospects for funding success may
MA
be important. Conversely, identifying and understanding mismatch between training and funding – which could reflect a variety of factors, including changing perceptions among trainees,
ED
evolution in research methods, and shifts in the funding landscape – might help provide greater
PT
guidance for outcomes research training programs and funders. In this NHLBI-funded examination of alignment between early-career investigators’ prior/desired training and funded
CE
grants, we found several core competency areas where training and funding aligned, and others
AC
in which mismatch existed. Harmonizing research interests and training with funding could help promote success for early-career outcomes research investigators.
Areas of Alignment Between Training and Funding Areas where we found alignment between early-career investigators’ prior/desired training and funded grants included randomized controlled trials, clinical epidemiology, observational research, and implementation and dissemination research. Given that there appears to be both a strong interest and grant support in these domains, they may continue to represent high-yield areas of focus for outcomes research training programs. Conversely, we consistently
ACCEPTED MANUSCRIPT found relatively little investigator prior/desired training or currently available funding in pharmacoepidemiology, practice-based network research, and decision analysis. Consequently,
T
these may be less fruitful targets for outcomes research training.
RI P
Of note, while we found relatively few funded grants in health economics/costeffectiveness research and systematic review, we identified inconsistencies in investigators’ prior
SC
training and desire for further training in these competency areas. While respondents endorsed
NU
little prior training in health economics/cost-effectiveness research – aligning with the relative paucity of funded grants in this area – their desire for further training was higher, resulting in
MA
mismatch with funding. In contrast, respondents reported ample prior training in systematic review, which was discrepant with the low funding in this area; however, investigators’
ED
relatively low desire for further training in systematic review aligned with funding. Early-career
PT
investigators and outcomes research programs should carefully consider how to prioritize training in these competency areas, recognizing that some topics or processes (e.g., systematic
CE
literature review) may represent important skills for CV outcomes researchers, regardless of
AC
funding opportunities.
Areas of Mismatch Between Training and Funding We found a consistent mismatch between early-career investigators’ prior/desired training and funded grants in some areas. Investigators’ training appeared to exceed awarded grants in the area of biostatistics and practical clinical trials. However, it is important to note that competency in biostatistics is critical to research success across outcomes research disciplines. For this reason, even though biostatistics is seldom the sole focus of any cardiovascular outcomes research funding proposal, programs should continue to focus on biostatistics as a core
ACCEPTED MANUSCRIPT competency for future outcomes researchers. Conversely, areas for which funding was awarded but in which investigators expressed relatively little interest included community-engaged
T
research and health informatics/information needs assessment. These areas may represent
RI P
‘missed opportunities’ for outcomes research training programs, and may warrant additional consideration as fruitful targets for training. These findings might also support the notion that
SC
these areas could be fruitful for development of collaborative relationships between outcomes
NU
researchers and researchers with expertise in community-engaged research and health
MA
informatics.
Limitations
ED
Although this analysis is the largest to date to examine alignment between early career
PT
investigators’ training perspectives and funded grants, it has limitations. While we attempted to assure the broadest possible survey dissemination, we could not evaluate our sample’s
CE
comprehensiveness and representativeness because: 1) the true denominator for outcomes
AC
research trainees is unknown; and 2) the survey was anonymous. Our survey response rate was relatively low (28%), so may not represent all trainee perspectives. However, certain factors may have artificially lowered our apparent response rate, making 28% a conservative estimate. First, because we reached out to a large set of awardees who may not be early-career investigators (awardees from PCORI) or who may have transitioned to mid-career (awardees from training grants within NHLBI and AHRQ who were funded in the earlier years of our included sample and who by the time of the survey may have been past 10 years since training) we were unable to ascertain which awardees failed to respond because they were no longer within 10 years of training, so not relevant to our study. Second, our survey was distributed to 314 QCOR attendees
ACCEPTED MANUSCRIPT directly by AHA. Because this private email distribution list could not be checked against our other distribution lists (awardees, T32 trainees) for duplicates, individuals on this list may have
T
received duplicate survey requests. Our true denominator was therefore unclear and our response
RI P
rate is a conservative estimate.
Another limitation is that we limited our review to major funding agencies that: 1) often
SC
support early-career investigators; and 2) made public their funded grant summaries. The latter
NU
requirement meant that we may have omitted some funders of early-career investigators in CV outcomes research (e.g., AHA, RWJF), and may have missed available funding in other areas
MA
(e.g., professional societies for systematic reviews, internal funding mechanisms for quality improvement efforts, etc.). Further, we did not include unsuccessful grants in this analysis,
ED
because information on unfunded proposals is not publicly available. Including unsuccessful
PT
grants in future analyses may provide additional context that is valuable to training programs. Because funding trends evolve over time (e.g., recently available funding for community-
CE
engaged research through PCORI), it will be important to update this exercise periodically.
AC
Similarly, competencies of interest evolve over time – for example, while patient-reported outcomes were not part of this analysis, recent years have increased the focus on use of these constructs to understand patient perspectives.[15] Therefore, the present analysis is not simply static guidance for programs, but also a demonstration of a dynamic process programs might use to assure responsiveness to changes in the funding landscape and relevant competencies.
Conclusions In this review of 521 funded grants from 2010-2014 and survey of 185 early-career investigators from CV outcomes research training programs, we identified numerous areas of
ACCEPTED MANUSCRIPT alignment between investigator interest and recent funding, and several areas of mismatch. Trends in alignment with recent funding were similar for respondents’ prior training and their
T
desired training. Our findings may inform efforts to improve alignment between training and
RI P
funding, which could promote success for early-career investigators. A critical step toward this success will be continued recognition of early-career investigators’ perspectives, as they possess
SC
the best understanding of “on the ground” conditions surrounding the transition to independent
MA
NU
investigators.
Acknowledgements
ED
MJC is supported by a Career Development Award from VHA Health Services Research
AC
CE
PT
and Development (CDA 13-261).
ACCEPTED MANUSCRIPT References 1. Mozaffarian D, Benjamin EJ, Go AS, Arnett DK, Blaha MJ, Cushman M, de Ferranti S,
RI P
T
Després J-P, Fullerton HJ, Howard VJ, Huffman MD, Judd SE, Kissela BM, Lackland DT, Lichtman JH, Lisabeth LD, Liu S, Mackey RH, Matchar DB, McGuire DK, Mohler
SC
ER 3rd, Moy CS, Muntner P, Mussolino ME, Nasir K, Neumar RW, Nichol G, Palaniappan L, Pandey DK, Reeves MJ, Rodriguez CJ, Sorlie PD, Stein J, Towfighi A,
NU
Turan TN, Virani SS, Willey JZ, Woo D, Yeh RW, Turner MB; on behalf of the
MA
American Heart Association Statistics Committee and Stroke Statistics Subcommittee. Heart disease and stroke statistics—2015 update: a report from the American Heart
ED
Association. Circulation. 2015; 131:e29–e322. 2. Krumholz HM, Peterson ED, Ayanian JZ, Chin MH, DeBusk RF, Goldman L, Kiefe
PT
CI, Powe NR, Rumsfeld JS, Spertus JA, Weintraub WS; National Heart, Lung, and Blood
CE
Institute working group. Report of the National Heart, Lung, and Blood Institute working
66.
AC
group on outcomes research in cardiovascular disease. Circulation. 2005; 111(23):3158-
3. Krumholz HM. Outcomes Research: Generating Evidence for Best Practice and Policies. Circulation. 2008; 118:309-318. 4. Tong CW, Ahmad T, Brittain EL, Bunch TJ, Damp JB, Dardas T, Hijar A, Hill JA, Hilliard AA, Houser SR, Jahangir E, Kates AM, Kim D, Lindman BR, Ryan JJ, Rzeszut AK, Sivaram CA, Valente AM, Freeman AM. Challenges facing early career academic cardiologists. J Am Coll Cardiol. 2014; 63(21):2199-208. 5. Centers for Cardiovascular Outcomes Research (CCOR). About CCOR. Available at: https://www.cardiovascular-outcomes.org/. Accessed May 26, 2016.
ACCEPTED MANUSCRIPT 6. Cook NL, Bonds, DE, Kiefe CI, Curtis JP, Krumholz HM, Kressin, NR, Peterson ED. Centers for Cardiovascular Outcomes Research: Defining a Collaborative Vision.
T
Circulation: Cardiovascular Quality and Outcomes. 2013;6(2):223-8.
RI P
7. Peterson ED, Spertus JA, Cohen DJ, et al. Vision and creation of the American Heart Association pharmaceutical roundtable outcomes research centers. Circulation.
SC
Cardiovascular quality and outcomes. 2009; 2(6):663-70.
NU
8. National Institutes of Health. National Heart, Lung, and Blood Institute Training and Career Development. Available at: https://www.nhlbi.nih.gov/research/training, Accessed
MA
September 28, 2016.
9. Kroenke K, Kapoor W, Helfand M, Meltzer DO, McDonald MA, Selker H. Training and
ED
career development for comparative effectiveness research workforce development:
PT
CTSA Consortium Strategic Goal Committee on comparative effectiveness research workgroup on workforce development Clin Transl Sci. 2010; 3(5):258-62.
CE
10. Committee on Comparative Effectiveness Research Prioritization, Institute of Medicine
AC
(IOM) Initial National Priorities for Comparative Effectiveness Research. Washington, DC: The National Academies Press; 2009. 11. Hersh W, Carey T, Ricketts T, Helfand M, Floyd N, Shiffman R, Hickman D. Institute of Medicine Roundtable on Evidence-Based Medicine. 2009. A Framework for the Workforce Required for Comparative Effectiveness Research. 12. CTSA Strategic Goal 4B-1 Subcommittee. Assessment of Comparative Effectiveness Research Capacity and Needs for CTSA Institutions. 2009. 13. CTSA Education and Career Development Workgroup. Core Competencies in Clinical and Translational Research for Master’s Candidates. 2009.
ACCEPTED MANUSCRIPT 14. Selker HP, Strom BL, Ford DE, Meltzer DO, Pauker SG, Pincus HA, Rich EC, Tompkins
C, Whitlock EP. White paper on CTSA Consortium role in facilitating comparative
T
effectiveness research. Clin Transl Sci. 2010;3:29–37.
RI P
15. Tucker CA, Cieza A, Riley AW, Stucki G, Lai JS, Bedirhan Ustun T, Kostanjsek N,
Riley W, Cella D, Forrest CB. Concept analysis of the patient reported
SC
outcomes measurement information system (PROMIS(®)) and the international
NU
classification of functioning, disability and health (ICF). Qual Life Res. 2014
AC
CE
PT
ED
MA
Aug;23(6):1677-86.
ACCEPTED MANUSCRIPT Tables
Substantial prior training
Strong desire for training
Quart.
N(%)
Quart.
N(%)
Grants Core competency area
Quart.
521(100)
-
185(100)
-
185(100)
-
Clinical epidemiology
383 (72)
1
72 (39)
1
45 (24)
2
Community-engaged research/patient & stakeholder engagement
309 (58)
1
18 (10)
3
36 (19)
3
Observational research
218 (42)
1
105 (57)
1
50 (27)
2
122 (23)
2
48 (26)
2
61 (33)
1
84 (16)
2
12 (6)
4
29 (16)
4
78 (15)
2
30 (16)
2
66 (36)
1
51 (10)
3
15 (8)
3
41 (22)
3
50 (10)
3
33 (18)
2
94 (51)
1
26 (5)
3
20 (11)
3
59 (32)
2
26 (5)
3
4 (2)
4
28 (15)
4
22 (4)
4
87 (47)
1
61 (33)
1
7 (1)
4
30 (16)
2
36 (19)
3
4 (1)
4
13 (7)
4
20 (11)
4
Randomized controlled trials Biomedical/health informatics & information needs assessment
MA
Implementation and dissemination research Decision analysis/cognitive sciences Practical clinical trials
ED
Health economics/cost-effectiveness research Practice-based network research Biostatistics
PT
Systematic reviews
AC
CE
Pharmacoepidemiology
SC
Total Ns: number of grants reviewed and of survey respondents
NU
N (%)
RI P
T
Table 1. Identified core competencies, with number of grants relevant to each competency area (with quartiles) and number of survey respondents indicating substantial prior training and strong desire for training in each area (with quartiles)
ACCEPTED MANUSCRIPT Table 2. Characteristics of survey respondents.
Age ≤ 30 years 31-35 years 36-40 years 41-45 years >45 years
15 58 64 32 16
(8) (31) (35) (17) (9)
Training/Career Stage In research training Early stages <3 years after completing training 3-5 years after completing training 6-10 years after completing training
32 14 36 53 50
(17) (8) (19) (29) (27)
81 17 14 6 4 2 2 1 33 23 2
(44) (9) (8) (3) (2) (1) (1) (1) (18) (12) (1)
AC
MA ED
PT
CE
Clinical Specialty Cardiology Internal Medicine Psychiatry/Psychology Pediatrics Surgery Family Medicine Hospitalist Neurology Other Non-clinical Missing
RI P
91 (49) 92 (50) 2 (1)
NU
Gender Male Female Missing
T
N (%)
SC
Respondent characteristic
ACCEPTED MANUSCRIPT Figures Figure 1. Cross-tabulation of quartiles of early-career investigator prior and desired training and
T
funded grants for core competency areas.
RI P
SC
NU MA
4th quartile
ED
3rd quartile
PT
2nd quartile
nd
CE
1st quartile
AC
Survey results
Grant review results 1 quartile 2 quartile 3rd quartile 4th quartile Alignment: Mismatch: Areas with more prior/desired training Areas with more prior/desired training and and more funded grants fewer funded grants Mismatch: Alignment: Areas with less prior/desired training and Areas with less prior/desired training and more funded grants fewer funded grants st
ACCEPTED MANUSCRIPT Figure 2. Alignment of quartiles between funded grants and investigators’ prior training (all grants).
T Randomized controlled trials Implementation and dissemination
nd
2 quartile
MA
3rd quartile
Biomedical/Health informatics and information needs assessment
AC
CE
PT
ED
4th quartile
Practical clinical trials
4th quartile Biostatistics
Systematic reviews
Health economics/ cost-effectiveness research Decision analysis/ cognitive sciences Practice-based network research
NU
Communityengaged research
RI P
1st quartile
1 quartile Observational research Clinical epidemiology
SC
Prior training
Successfully funded grants (all grants) 2nd quartile 3rd quartile
st
Pharmacoepidemiology
ACCEPTED MANUSCRIPT Figure 3. Alignment of quartiles between funded grants and investigators’ desired training (all grants).
nd
2 quartile
3rd quartile
Observational research Clinical epidemiology Communityengaged research
SC
1st quartile
Biomedical/health informatics and information needs assessment
th
AC
CE
PT
ED
MA
4 quartile
4th quartile Biostatistics
RI P
T
1 quartile
Successfully-funded grants (all grants) 2nd quartile 3rd quartile Implementation Practical clinical and dissemination trials Randomized controlled trials Health economics/ cost-effectiveness research Decision analysis/ cognitive sciences Practice-based network research
NU
Desired training
st
Systematic reviews Pharmacoepidemiology