Examining state health agency epidemiologists and their training needs

Annals of Epidemiology xxx (2016) 1e6

Contents lists available at ScienceDirect

Annals of Epidemiology journal homepage: www.annalsofepidemiology.org

Original article

Examining state health agency epidemiologists and their training needs Theresa Chapple-McGruder PhD, MPH a, *, Jonathon P. Leider PhD a, Angela J. Beck PhD, MPH b, Brian C. Castrucci MA a, Elizabeth Harper DrPH c, Katie Sellers DrPH, MPH c, Jessica Arrazola MPH d, Jeff Engel MD d a

de Beaumont Foundation, Bethesda, MD Department of Health Management and Policy, Center of Excellence in Public Health Workforce Studies, University of Michigan School of Public Health, Ann Arbor Association of State and Territorial Health Officials, Arlington, VA d Council of State and Territorial Epidemiologists, Atlanta, GA b c

a r t i c l e i n f o

a b s t r a c t

Article history: Received 5 August 2016 Accepted 18 November 2016 Available online xxx

Purpose: Despite increases in formal education, changing trends affecting epidemiologic practice prompted concerns over whether epidemiologists had sufficient training. Methods: This study sought to explain factors that predicted low self-reported proficiency levels among daily important work tasks of state health agencies’ epidemiologists. The number of knowledge gaps, instances where epidemiologists identified a work-related task both as ‘very’ important in their daily work and felt they were “unable to perform” or performed at a “beginner” level, was studied, and predictor variables were assessed. A total of 681 epidemiologists responded to the 2014 Public Health Workforce Interests and Needs Survey, a national survey of state health agency workers; epidemiologists represented 7% of all respondents. Results: Epidemiologists at state health agencies worked mostly in communicable disease (31%) or general surveillance (26%). Epidemiologists reported eight key daily work-related activities with an average of three training gaps. Factors that decreased the likelihood of epidemiologists’ low proficiency in performing key activities were the presence of internal trainings (adjusted odds ratio ¼ 0.69, 95% confidence interval, 0.49e0.99) and length of time working in public health (adjusted odds ratio ¼ 0.95, 95% confidence interval, 0.93e0.98). Conclusion: Although formal education of epidemiologists is on the rise, state health agencies’ epidemiologists feel unprepared to tackle one-third of their important daily tasks. Ó 2016 de Beaumont Foundation. Published by Elsevier Inc. This is an open access article under the CC BY-NC-ND license (http://creativecommons.org/licenses/by-nc-nd/4.0/).

Keywords: Epidemiology Workforce Training Education

Introduction Epidemiologists play a vital role in investigating causes of disease in populations, identifying people who are at risk of illness, and determining how to control the spread of disease [1]. All state health agencies employ epidemiologists [2], only about 36% of local health departments have epidemiologists [3]. Overall, epidemiologists comprise roughly 2% of the state and local public health workforce [4]. Developing a well-trained, competent epidemiology workforce is an ongoing public health practice concern. A 2001 national survey of state health agencies (SHAs) found that nearly

* Corresponding author. de Beaumont Foundation, 7501 Wisconsin Avenue, Suite 1310E, Bethesda, MD 20814. Tel.: 301-685-5021; fax: 301-961-5802. E-mail address: [email protected] (T. Chapple-McGruder).

57% of employed epidemiologists had formal academic training in epidemiology [5]. By 2013, this percentage increased to 88% [6], a possible outcome of the competency-based epidemiology workforce development initiatives that had been implemented among the current and future workforce over the past decade [7e9]. Despite the reported increase in formal epidemiology training [10], changing trends affecting epidemiologic research and practice prompt continued concerns about whether the workforce has sufficient training to deliver public health services. SHA capacity to deliver epidemiology-related Essential Services of Public Health and program area capacity has been assessed through periodic surveys issued by the Council of State and Territorial Epidemiologists (CSTEs) since the early 2000s [4e6]. The most recent survey found that surveillance and epidemiology capacity increased to its highest levels in 2013 for all eight monitored program areas.

http://dx.doi.org/10.1016/j.annepidem.2016.11.007 1047-2797/Ó 2016 de Beaumont Foundation. Published by Elsevier Inc. This is an open access article under the CC BY-NC-ND license (http://creativecommons.org/licenses/bync-nd/4.0/).

Please cite this article in press as: Chapple-McGruder T, et al., Examining state health agency epidemiologists and their training needs, Annals of Epidemiology (2016), http://dx.doi.org/10.1016/j.annepidem.2016.11.007

2

T. Chapple-McGruder et al. / Annals of Epidemiology xxx (2016) 1e6

However, fewer than half the states reported substantial-to-full surveillance and epidemiology capacity in environmental health, injury, oral health, occupational health, substance abuse, and mental health program areas [6,11e13]. Nearly one-third of states also reported a need for enhanced capacity to carry out public health services related to evaluating effectiveness, accessibility, and quality of personal and population-based health services and researching for new insights and innovative solutions to health problems [6,11e13]. These capacity studies point to potential training needs for SHA epidemiologists. In addition, a recent project implemented by a working group of senior epidemiologists identified 12 macro-level trends that are impacting epidemiology training needs, including availability of big data/informatics, changing health communication environment, health care system reform, shifting demographics, globalization, emerging high-throughput technologies, greater focus on accountability, privacy changes, greater focus on “upstream” causes of disease, emergence of translational sciences, transdisciplinary science, and the evolving funding environment [14]. To address these emerging trends, epidemiologists need to be innovative with the ability to design systems, collaborate across sectors, critically evaluate data and determine its utility, and communicate the importance of surveillance to policy makers, health systems, and the general public especially to articulate the need for adequate funding and support [15]. The findings suggest that traditional epidemiology curricula in schools and programs of public health should be expanded to incorporate competencies related to the identified trends and skills through mentoring and practice application, and that lifelong learning initiatives should be used to ensure workers continue to acquire skills as public health practice needs shift. It is important to understand the daily tasks of epidemiologists, which may vary by health department, and current proficiency levels in carrying out prescribed tasks to adapt continuing education opportunities and public health curricula to better address the training needs of applied epidemiologists. The 2014 Public Health Workforce Interests and Needs Survey (PH WINS), a joint project of the de Beaumont Foundation and Association of State and Territorial Health Officials, collected data from state and local health department workers to identify potential training needs [16]. The purpose of this study was to identify predictors of low self-reported proficiency in a number of important daily work tasks for SHA epidemiologist respondents to PH WINS. The number of training gaps was also studied to determine what factors could reduce the number of important work-related tasks in which epidemiologists rate themselves as beginners or unable to perform. The empirical findings will supplement the findings of previous studies by identifying areas of low proficiency and high training needs among this important segment of the public health workforce. Methods Sampling and broader survey methodology have been written about extensively elsewhere [17]. In brief, PH WINS was conceived in 2013 and fielded in fall 2014 to gauge public health practitioners’ perspectives on workplace environment, job satisfaction, national trends, and training needs and to gather demographics on the workforce. PH WINS was fielded in three framesea nationally representative frame of SHA staff, staff from a group of large, urban local health departments, and staff from other, smaller local health departments. This study uses data from the first frame only, the nationally representative frame of permanently employed SHA central office staff. Respondents were sampled from pairing the 10 Health and Human Services regions of the United States into five contiguous regions. Of the 50 states, 13 did not participate, this was

known during the design phase, which allowed for the sampling design and weights to be created at both the state and regional levels. Approximately, 25,000 invitations were sent to central office employees. After accounting for invalid e-mail addresses and staff who had left their position, the response rate was 46% (n ¼ 10,246). Balanced repeated replication weights were used to construct robust variance estimators and account for the complex sampling design. The PH WINS instrument was created mainly through utilization of previously used or validated questions; it draws heavily from the Centers for Disease Control and Prevention’s Project Officer Survey, the 2009 Epidemiology Capacity Assessment, the Federal Employee Viewpoint Survey, the Public Health Foundation Worker Survey, and the Job in General Scale [12,18e22]. Questions were added relating to training needs and awareness of national trends. Cognitive interviews were conducted, aimed at comprehensibility and accessibility, especially of newly added questions. A revised instrument was pretested with three groups of public health practitioners and state and local health departments. Most analyses in this article were limited to individuals who self-identified as epidemiologist. Descriptive analyses and bivariate comparisons were conducted using Pearson’s c2 and Tukey’s tests for multiple comparisons, as appropriate. In addition to demographic information, analyses were conducted around perceived training needs, gaps in skill level, and important daily work-related tasks. Logistic regression was used to determine factors that explain noted discrepancies in importance of daily work tasks and low selfrated proficiency of those tasks. A composite variable was created to account for all respondents who reported a low proficiency level, rating themselves as beginner, or unable to perform in any of the eight tasks described by at least 50% of epidemiologists as important to their daily work as an SHA epidemiologist. Predictor variables were assessed for confounding, collinearity, and model fit to determine which factors best explained SHA epidemiologists reporting inability to perform important tasks or performing tasks at the beginner level. Backward elimination approach was used to determine which variables should remain in the model. Variables were kept in the model that were significantly at a P < .05. PH WINS was deemed exempt by the Chesapeake Institutional Review Board. Results In total, 681 permanently employed epidemiologists from SHA central offices responded to PH WINS, accounting for about 7% of all responses. After weighting and poststratification, data from PH WINS suggest that about 2850 epidemiologists were permanently employed at SHAs central offices nationwide (95% confidence interval: 2540e3168; Table 1). Approximately, 74% of SHA epidemiologists were female, 71% were non-Hispanic White, and half were aged older than 40 years. Overall, epidemiologists at SHAs work mostly in communicable disease (31%) or general surveillance (26%). Epidemiologists were among the most highly educated staff types in the SHA workforce. Approximately, 75% of the workforce overall had a bachelor’s degree, compared with 99% of epidemiologists (P < .001). Similarly, about 38% of the workforce had a master’s degree, compared with 88% of epidemiologists (P < .001). About 9% of SHA staff had a doctoral degree, compared with 26% of epidemiologists (P < .001). Epidemiologists also more frequently had public healtheoriented degrees (77%) compared with the workforce overall (17%), generally through attainment of a Master of Public Health (64%) or a related master’s degree. Educational attainment for bachelor’s degrees overall was consistent by age (Fig. 1). While the proportion of epidemiologists with a doctorate increased with ascending age, the proportion of staff with a public

T. Chapple-McGruder et al. / Annals of Epidemiology xxx (2016) 1e6

3

Table 1 Estimated demographics of epidemiologists working in US state health agency’s central offices, 2014 (n ¼ 2849, 95% CI: 2530e3168) Variable Gender Male Female Race/ethnicity American Indian or Alaskan Native Asian Black or African American Hispanic or Latino Native Hawaiian or other Pacific Islander White Two or more races Time in public health (y) 0e5 6e10 11e15 16e20 21 Age (y) 21e25 26e30 31e35 36e40 41e45 46e50 51e55 56e60 61e65 66e70 Supervisory status Nonsupervisor Team leader Supervisor Manager Executive Educational attainment (general) Associates Bachelor’s Master’s Doctorate Educational attainment (in public health) Bachelor’s Master’s Doctorate Public Health degree (any level) Program area Chronic disease Communicable disease Environmental health Maternal/child health Assessment (general surveillance) All other

Point estimates (%)

CIs (%)

26 74

19e34 66e81

0 9 9 6 0 71 5

0e0 6e12 5e13 4e8 0e0 66e77 3e6

23 25 22 11 20

20e26 21e28 18e25 8e13 16e24

2 17 16 16 14 11 7 7 8 2

1e4 13e20 13e20 11e20 10e18 8e13 5e8 4e10 5e11 1e4

49 20 21 10 0

43e55 14e25 18e25 7e12 0e1

5 99 88 26

3e6 98e100 85e91 23e30

3 73 12 77

2e5 70e76 10e15 74e80

7 31 8 11 26 17

4e9 25e37 6e10 8e14 22e30 14e20

CI ¼ confidence interval. Totals may not sum to 100% because of rounding. Educational attainment questions permitted respondents to select all degrees attained.

health master’s degree (most typically a Master of Public Health) decreased among older staff. Among epidemiologists in the younger half of the workforce (aged 48 years and younger), about 82% (95% confidence interval: 79%e85%) had a public healtheoriented master’s degree. Among those aged 49 years and older, 51% had a public healtheoriented master’s degree (P < .001), although 80% had a master’s degree of some kind. Practitioner perspectives on importance and self-reported skill related to a number of training areas and competencies were also assessed (Table 2). Eighty-eight percent of epidemiologists rated interpreting public health data to answer questions as very important in their daily work; 84% rated gathering reliable information to answer questions as very important. Most

Fig. 1. Educational attainment of U.S. state health agency epidemiology staff by age at time of survey, 2014. (A) Overall, (B) public health degree.

epidemiologists also rated skills related to communicating clearly and persuasively, using evidence, collaborating with peers, change management, and systems thinking as very important to their daily work. However, with the exception of “gathering reliable information to answer questions,” between 16% and 32% of epidemiologists reported being an expert in these competency areas. Fiftythree percent of epidemiologists reported being an expert at gathering reliable information, compared with 40% of staff within the public health sciences and 33% of all SHA staff (P ¼ .002 and P < .001, respectively). While a relatively small proportion of epidemiologists felt they were unable to perform or a beginner at tasks related to gathering, interpreting, and communicating information accessibly, between 16% and 23% of epidemiologists did not feel they were proficient or expert in the other top tasks they rated as very important to their daily work. A respondent variable was generated that counted the number of self-identified training gaps (i.e., each instance where an individual identified a work-related task as “somewhat” or “very” important in their day-to-day work and felt they were “unable to perform” or were “beginner” level proficiency). Among epidemiologists, the average number of training gaps was 3.0. Among all SHA staff, the average number of gaps was 2.7 (P ¼ .002). Bivariate comparisons show differences by education, but among all SHA staff, increasing education was statistically significantly associated with a smaller number of gaps; among epidemiologists, master’seducated staff had more gaps than either bachelor’s (P ¼ .65) or doctoral recipients (P < .001; Fig. 2). The likelihood of an epidemiologist at a SHA rating themselves as beginners or unable to perform at least one of the eight key work-related activities was highest for epidemiologists with a bachelor’s degree (adjusted odds ratio [AOR] ¼ 2.23, 95% confidence interval: 0.96e5.16) and elevated for those with a master’s degree (AOR ¼ 1.61, 95% confidence interval: 1.03e2.53) when

4

T. Chapple-McGruder et al. / Annals of Epidemiology xxx (2016) 1e6

Table 2 Tasks at least 50% of epidemiologists at US state health agencies find very important to their daily work and self-reported task proficiency, 2014 (n ¼ 661) Daily work tasks

Daily work importance

Ability to perform work

Very Expert (%) Beginner or important (%) unable to perform (%) 1. Interpreting public health data to answer questions 2. Gathering reliable information to answer questions 3. Communicating ideas and information in a way that different audiences understand 4. Applying evidence-based approaches to solve public health issues 5. Engaging staff within your health department to collaborate on projects 6. Managing change in response to dynamic, evolving circumstances 7. Assessing the broad array of factors that influence specific public health practices 8. Communicating in a way that persuades others to act 9. Finding evidence on public health efforts that work 10. Addressing the needs of diverse populations in a culturally sensitive way 11. Engaging partners outside your health department to collaborate on projects 12. Applying quality improvement concepts in my work 13. Collaborating with diverse communities to identify and solve health problems 14. Ensuring that programs are managed within the current and forecasted budget constraints 15. Understanding the relationship between a new policy and many types of public health problems 16. Preparing a program budget with justification 17. Anticipating the changes in your environment (physical, political, and environmental) that may influence your work 18. Influencing policy development

88

56

4

84

53

3

73

32

8

62

23

16

54

20

15

54

16

21

53

19

20

50

16

23

49

20

16

48

16

26

46

15

22

44

16

25

39

13

24

36

9

29

34

7

36

32

11

30

31

8

27

24

7

42

compared with those with a doctoral degree, although it was only significant for those with a master’s degree. The greatest factor found to decrease the likelihood of epidemiologists being performing at a beginner level or being unable to perform a task was the availability of on-site trainings (AOR ¼ 0.69, 95% confidence interval: 0.49e0.99). Working in public health for more than 15 years was also found to decrease the likelihood of epidemiologists having a low proficiency in daily work tasks (AOR ¼ 0.95, 95% confidence interval: 0.93e0.98; Table 3).

Discussion When asked to rate the importance of several health department tasks, interpreting public health data to answer questions and gathering reliable information to answer questions were the two tasks that were identified as very important by the largest

Table 3 The odds of an US state health agency epidemiologist being a beginner or unable to perform important daily work activities, 2014 (n ¼ 681) Covariates Highest education attained Bachelor’s Master’s Doctoral Years in public health 15þ 1e15 Supervisory role Nonsupervisory role Has staff positions for internal training No training staff Person of color White staff member

AOR

95% CI

P value

2.23 1.61 (ref)

0.96e5.16 1.03e2.53

.06 .04

0.95 (ref) 1.08 (ref) 0.69 (ref) 0.68 (ref)

0.93e0.98

<.001

0.71e1.64

.73

0.49e0.99

.05

0.45e1.03

.07

CI ¼ confidence interval. The dependent variable is whether a respondent indicated they were unable to perform/beginner in at least one key epidemiology-related training area, specifically gathering reliable information to answer questions, interpreting public health data to answer questions, applying evidence-based approaches to solve public health issues, and/or assessing the broad array of factors that influence specific public health problems.

proportion of epidemiologists. The proportion of epidemiologists reporting being a beginner or unable to perform these tasks was less than 5%. This is similar to the results from the CSTEs’ 2013 report that most epidemiologists working in an SHA possessed capacity to fulfill at least 8 of the 10 essential public health services [6]. However, in addition to these traditional epidemiologic tasks, more than half of all epidemiologists cited several tasks that were not specific to epidemiology, but more cross cutting: engaging staff within their health department to collaborate on projects, managing change in response to dynamic evolving circumstances, and communicating in a way that persuades others to act. For these cross-cutting skills, approximately one-fifth of epidemiologists reported being a beginner or unable to perform these tasks. This demonstrates when designing continuing education options for epidemiologists, there may be less of a need for specialty-specific training in lieu of more cross-cutting training that provides knowledge and skills that are not typically included in specialty area training. An overwhelming majority of SHA epidemiologists had degrees in public health, which may be the result of changes in educational attainment hiring requirements. This was underscored by those reporting bachelor’s degrees as highest education attained also reporting the greatest number of years worked in public health. This may explain why those with master’s degrees identified more training gaps in comparison with those with bachelor’s degrees. With the most new SHA epidemiologists having public health training, this suggests that schools and programs of public health may have a greater influence on the epidemiology workforce, more so than the entire public health workforce. Schools and programs training the greatest number of epidemiologists should be encouraged to consider changes in their curricula that could potentially address some of the nontraditional, cross-cutting tasks that practicing epidemiologists identified as very important but for which there exists a training gap. Increased experience was associated with reduced odds of being a beginner or being unable to perform a task. However, the effect size was small, especially among early career professionals (<1 year, 1e2 years, and 3e5 years). This suggests that experience alone is not an adequate professional development strategy. Health departments may want to consider requiring professional development plans or training for epidemiologic staff, especially those who are early in their careers.

T. Chapple-McGruder et al. / Annals of Epidemiology xxx (2016) 1e6

5

Fig. 2. Proportion of U.S. state health agency staff by number of self-assessed training gaps, by educational attainment, 2014.

We identified a strong association between having on-site training and a decreased odds of being a beginner or unable to perform tasks. For health departments that value a well-trained epidemiologic workforce, this finding suggests that investment in on-site training for staff may contribute to reducing training gaps among the epidemiologic workforce. This is an example of a proactive approach in which health departments can engage that may help to address identified skill deficiencies in the epidemiologic workforce. For agencies unable to provide on-site training, identifying other opportunities to foster professional growth is important to maintain and attract a skilled and diverse workforce [15]. Partnership with academic institutions and membership in professional association, such as CSTE, are other methods of connecting epidemiologists with continuing education. Further research is needed to determine the influence of on-site and off-site training resources on the training gaps among the broader public health workforce.

Nevada, New Hampshire, New Mexico, and Oregon), thus impacting both representation and the total number of respondents. Also, some respondents commented that because of the detail requested in some questions, anonymity may not exist at the state level. This may have led to an overrepresentation of people reporting high levels of proficiency for work-related tasks. Finally, epidemiologists were one of the many professions within health departments that received this survey, only accounting for about 7% of respondents. Having small numbers may have impacted the statistical power of the models and limited the number of factors that could be assessed within each model. Similarly, the data were weighted to be representative of the entire governmental public health workforce working in state health departments, not of epidemiologists specifically. Overall, we believe these limitations may have biased our results toward the null, and therefore, any associations seen are representative of true associations.

Limitations

Conclusions

This study is subject to at least three known limitations. While the sampling scheme sought to be representative of all the states, 13 states opted not to participate in the survey (Alabama, Colorado, Florida, Hawaii, Idaho, Iowa, Kentucky, Maine, Montana,

Epidemiologists are one of the most highly educated groups of professionals working in SHAs. However, sizable knowledge gaps exist in five of the nine main activities that SHA epidemiologists rated as important in their daily work. This leads to the question,

6

T. Chapple-McGruder et al. / Annals of Epidemiology xxx (2016) 1e6

are public health programs and schools appropriately preparing students to work as applied epidemiologists? In examining reported knowledge and performance gaps, we note, on average, epidemiologists reported three such gaps. An increase in formal education did not statistically attenuate these gaps. The only factors that did were years of experience in public health and the availability of on-site training. The lessons learned from on-the-job training could be incorporated into public health continuing education programs, as nearly a quarter of epidemiologists reported no opportunities to receive on-site training from their jobs. Ongoing education and training is needed to maintain a competent and skilled applied epidemiology workforce. Acknowledgments The authors would like to acknowledge the 37 state health departments that participated in the Public Health Workforce Interests and Needs Survey. Funding: Public Health Workforce Interests and Needs Survey was funded by the de Beaumont Foundation and conducted by the Association of State and Territorial Health Officials and the de Beaumont Foundation. References [1] Centers for Disease Control and Prevention. Who are epidemiologists? Available at: http://www.cdc.gov/EXCITE/epidemiologists.html. Published 2014. Accessed August 19, 2015. [2] Association of State and Territorial Health Officials. Profile of State Public Health, volume 3. Arlington, VA: ASTHO; 2013. Available at: http://www. astho.org/Profile/Volume-Three/. Accessed August 19, 2015. [3] National Association of County and City Health Officials. 2013 National Profile of Local Health Departments. Washington DC: NACCHO; 2014. Available at: http://www.naccho.org/topics/infrastructure/profile/upload/2013-NationalProfile-of-Local-Health-Departments-report.pdf. Accessed August 19, 2015. [4] Beck AJ, Boulton ML, Coronado F. Enumeration of the governmental public health workforce, 2014. Am J Prev Med 2014;47(5S3):S306e13. [5] Centers for Disease Control and Prevention. Assessment of the epidemiologic capacity in state and territorial health departments-United States, 2001. MMWR Morb Mortal Wkly Rep 2003;52(43):1049e51.

[6] Hadler JL, Lampkins R, Lemmings J, Lichtenstein M, Huang M, Engel J, et al. Assessment of epidemiology capacity in state health departments-United States, 2013. MMWR 2015;64(14):394e8. [7] Dick VR, Master’s AE, McConnon PJ, Engel JP, Underwood VN, Harrison RJ, et al. The CDC/Council of State and Territorial Epidemiologists Applied Epidemiology Fellowship program: evaluation of the first 9 years. Am J Prev Med 2014;47(5S3):S376e82. [8] Council of State and Territorial Epidemiologists. Competencies for Applied Epidemiologists in Governmental Public Health Agencies. Atlanta, GA: CSTE; 2014. Available at: http://www.cste.org/group/CSTECDCAEC. Accessed August 19, 2015. [9] Council of State and Territorial Epidemiologists. Epidemiology Workforce Training Catalog. 2015. Available at: http://www.cste2.org/workforcetraining/ default2.aspx. Accessed August 19, 2015. [10] Beck AJ, Boulton ML. Challenges to recruitment and retention of the state health department epidemiology workforce. Am J Prev Med 2012;42(1):76e80. [11] Boulton ML, Lemmings J, Beck AJ. Assessment of epidemiology capacity in state health departments, 2001-2006. J Public Health Manag Pract 2009;15(4):328e36. [12] Boulton ML, Hadler J, Neck AJ, Ferland L, Lichtveld M. Assessment of epidemiology in state health departments, 2004-2009. Public Health Rep 2011;126(1):84e93. [13] Council of State and Territorial Epidemiologists. 2013 National Assessment of Epidemiology Capacity. Atlanta, GA: CSTE; 2014. Available at: http://www.cste2. org/2013eca/CSTEEpidemiologyCapacityAssessment2014-final2.pdf. Accessed August 19, 2015. [14] Brownson RC, Samet JM, Chavez GF, Davies MM, Galea S, Hiatt RA, et al. Charting a future for epidemiologic training. Ann Epidemiol 2015;25(6):458e65. [15] Smith P, Handler J, Stanbury M, Rolfs R, Hopkins R. Blueprint version 2.0: updating public health surveillance for the 21st century. J Public Health Manag Pract 2013;19(3):231e9. [16] Sellers K, Leider JP, Harper E, Castrucci BC, Bharthapudi K, Liss-Levinson R, et al. The public health workforce interest and needs survey: the first national survey of state health agency employees. J Public Health Manag Pract 2015;21(6):S13e25. [17] Leider JP, Bharthapudi K, Pineau V, Liu L, Harper E. The methods behind PH WINS. J Public Health Manag Pract 2015;21(6):S28e35. [18] Centers for Disease Control and Prevention. 2009-2010 National Adult Tobacco Survey. Atlanta, GA: Centers for Disease Control and Prevention; 2011. [19] Centers for Disease Control and Prevention. CDC’s Technical Assistance & Service Improvement Initiative: Summary of Assessment. Atlanta, GA: Centers for Disease Control and Prevention; 2011. [20] Office of Personnel Management. Federal Employee Viewpoint Survey Results. Washington, DC: Office of Personnel Management; 2014. [21] Public Health Foundation. Public Health Workforce Survey Instrument. Washington, DC: Public Health Foundation; 2010. [22] Ironson GH, Smith PC, Brannick MT, Gibson W, Paul K. Construction of a job in General Scale: a comparison of global, composite, and specific measures. J Appl Psychol 1989;74(2):193.