- Email: [email protected]
A post-conference evaluation of the 2015 National Occupational Injury Research Symposium
JSR-01367; No of Pages 4 Journal of Safety Research xxx (2016) xxx–xxx
Contents lists available at ScienceDirect
Journal of Safety Research journal homepage: www.elsevier.com/locate/jsr
3Q2 4
F
2
A post-conference evaluation of the 2015 National Occupational Injury Research Symposium Christine R. Schuler, a,⁎ Dawn N. Castillo, a Cammie Chaumont Menéndez, a Sergey Sinelnikov, b Sydney Webb, a Emily Chavez b
O
1Q1
5 6 7
a
8
a r t i c l e
9 10 11 12 13 39 40 41 42 43 44
Article history: Received 8 December 2016 Accepted 14 December 2016 Available online xxxx
49
1. Introduction
50
The National Occupational Injury Research Symposium (NOIRS) is the only national forum that is focused on occupational injury research findings, data and methods, and prevention strategies, and has been convened every three to five years since 1997. The theme of the sixth symposium, held in May 2015, was “Advancing Occupational Injury Research through Integration and Partnership.” NOIRS 2015 was sponsored by the National Institute for Occupational Safety and Health's (NIOSH) Division of Safety Research (DSR), and co-sponsored by the National Safety Council (NSC), the American Society of Safety Engineers (ASSE), Liberty Mutual Research Institute for Safety (LMRIS), and the Society for Advancement of Violence and Injury Research (SAVIR).
a b s t r a c t
Introduction: The National Occupational Injury Research, Symposium (NOIRS) is the only national forum focused on occupational injury research findings, data and methods, and prevention strategies; it has been convened every 3–5 years since 1997. Held in May 2015, the 6th symposium's theme was “Advancing Occupational Injury Research through Integration and Partnership.” Organizers requested that attendees complete a post-meeting evaluation to assess meeting impact, and gather information useful in planning subsequent meetings and activities. Method: The questionnaire was publicized via a quick response code and link to the survey on symposium book cover, and mentioned at each scientific session. The online survey was designed to be completed in ≤15 min; no identifying information was collected. Survey link remained open for seven days post-symposium. Results: About 50% of registered attendees responded. Almost half were attending their first NOIRS. Most were researchers (69%); 45% were affiliated with government and 38% with university or research institute. Five of six reported that the symposium mostly or completely met expectations. Reasons for attending included gaining exposure to new areas of research (87%), sharing their research (80%), and to develop new ideas for conducting research (79%). The majority (90%) reported that the symposium provided adequate networking opportunities. The conference venue was reported as good or better by 69%, moreso among repeat attendees (77%) compared to first-timers (61%). Discussion: The evaluation demonstrated that NOIRS was valuable to attendees, and provided a forum for sharing research results, developing new research ideas, and networking. Respondents provided input on different aspects of NOIRS and suggestions useful in planning next NOIRS, tentatively scheduled for 2018. NOIRS 2015 objectives for integration across disciplines and partnership with industry and safety professionals were partially met. In planning NOIRS 2018, more attention should be paid to attracting and engaging a broader spectrum of attendees. © 2016 Published by Elsevier Ltd.
Q3 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38
NOIRS 2015 was held in Kingwood, West Virginia (WV), about an hour's drive from the NIOSH campus in Morgantown, WV. Previous NOIRS had been held in Morgantown, WV and Pittsburgh, PA. The venue was a National Guard training complex and was economical. There were no registration fees. The venue was comfortable and in a beautiful but rural location. The nearest large airport was about a two and a half hour drive away. The symposium was attended by more than 250 occupational injury researchers, safety professionals, and students from six countries (including the United States), representing government (state and federal) agencies, universities, research institutes, private sector, labor, and non-profit/non-governmental organizations. The event was publicized through email, social media, and the efforts of the co-sponsors. The two and a half day conference agenda included opening and closing plenaries, eight groups of four to five concurrent topic-specific scientific sessions (with a total of 135 platform presentations), a tutorial session, and two evening networking events, one including a poster session with
61
P
i n f o
R O
National Institute for Occupational Safety and Health, Division of Safety Research, 1095 Willowdale Road, MS H1900, Morgantown, WV 26505, United States National Safety Council, 1121 Spring Lake Drive, Itasca, IL 60143, United States
D
b
53 54 55 56 57 58 59 60
U
51 52
N C O
R
R
E
C
T
E
Keywords: Surveys and questionnaires Occupational injuries Occupational health Safety Conference
⁎ Corresponding author. E-mail address: [email protected] (C.R. Schuler).
http://dx.doi.org/10.1016/j.jsr.2016.12.009 0022-4375/© 2016 Published by Elsevier Ltd.
Please cite this article as: Schuler, C.R., et al., A post-conference evaluation of the 2015 National Occupational Injury Research Symposium, Journal of Safety Research (2016), http://dx.doi.org/10.1016/j.jsr.2016.12.009
62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77
126
3. Survey results
127 128
Among the 266 individuals who registered to attend NOIRS 2015, 134 (50%) took part in the conference evaluation; 90% (120/134) completed the full evaluation. Almost half of the respondents were attending their first NOIRS (48%, 64/134) (Table 1). Respondents were equally split between genders, and were mostly U.S. residents. Professions were predominantly research-related (69%); 45% were affiliated with federal or state governments, and more than a third were from academia. When asked about the meeting location, 74% reported that they would attend again if offered at the same location; only 23% reported that the venue was easy to locate. Five of six respondents (83%) reported that NOIRS 2015 completely or mostly met their expectations (Table 1). Most respondents either agreed or strongly agreed that the conference was well-run (i.e., sessions and
103 104
108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123
129 130 131 132 133 134 135 136 137 138 139
C
101 102
E
99 100
R
97 98
R
95 96
O
93 94
C
91 92
N
89 90
U
87 88
F
124 125
The conference organizers developed a post-conference evaluation questionnaire to assess the value and impact of NOIRS 2015, and to better inform planning for the next symposium, tentatively scheduled for 2018. Areas of particular interest from the survey were conference experience and impact, particularly how well various aspects of the conference met the expectations of attendees, how attendees intended to use the knowledge and contacts obtained at the conference, and what might be improved for the next NOIRS; a comparison of the experiences of first-time attendees and those who had attended one or more prior NOIRS; as well as demographic and professional information. The online survey was designed to be completed in less than 15 min, and no personally identifying information was collected. The instrument included closed-ended questions with response options, as well as some open-ended questions that required narrative responses. The survey was publicized throughout the meeting via a quick response (QR) code and link to the survey on the cover of the symposium book, and mentioned at the beginning of each scientific session. An email was distributed after the meeting to remind all registered attendees to complete the evaluation within seven days.
85 86
O
107
84
R O
2. Survey instrument and methods
82 83
P
106
80 81
presentations kept to schedule, 90%), that the scientific content was organized appropriately into sessions (88%), that there was adequate opportunity for networking with other attendees (90%), and that there was adequate time for audience members to interact with speakers (82%). A slightly smaller majority believed that the topics presented were relevant to them (75%). Responses were not dissimilar between first-time and repeat attendees. When asked about various aspects of the meeting, most respondents reported that the concurrent scientific sessions (84%) and their conversations and interactions with conference participants (83%) completely or mostly contributed to their learning experience (Table 1). The combined evening networking and poster session was reported to contribute less to the learning experience (47% completely or mostly). Responses were similar between first-time and repeat attendees, although 90% of repeat attendees reported benefitting from their interactions with conference participants compared to 76% of first-time attendees. When asked why they decided to attend NOIRS 2015, 87% reported that gaining exposure to new areas of research was important or very important, 80% reported to share their research, and 79% reported to develop new ideas for conducting research (Table 1). Having the opportunity to recharge or refocus was reported as important or very important by 47%. Although not reported as important or very important by many, there were differences in relative importance between first-time attendees and repeat attendees for meeting or identifying potential employers (21% and 6%, respectively), developing new practice approaches from what was learned at NOIRS (58% and 48%), deciding where to continue their public health education (11% and 5%), and the opportunity to recharge or refocus (40% and 53%). When asked for specific information about what they gained at NOIRS that could not be gained elsewhere, respondents remarked on the attendees (responses included interesting multidisciplinary group, opportunity for a safety practitioner to interact with researchers, inspired by the energy and commitment of the presenters and attendees); the size of the conference (size makes it easier to network, small conference environment encourages more interaction between speakers and audiences and also between participants, opportunity for one-on-one conversations with key people); the focused nature of the meeting (concentrated on occupational injury research); and the atmosphere (relaxed, conducive to learning and interacting). The respondents also provided many suggestions for what they'd like to see offered at future NOIRS, including topic areas (more intervention research, more sleep and fatigue research, psychological/cognitive demands of changing/hazardous work environments, cost effectiveness of research, workplace road safety, other transportation-related aspects, relationship of injury with psychosocial environment, impact of restructuring of work on injury causation, causes of rail disasters, national or state level campaigns for injury prevention, economic costs of injury to families and communities, young workers as a vulnerable population); tutorials (conducting in-house workplace research, statistical software sessions, more contemporary methods, advanced epidemiologic and statistical methods, how to disseminate research, grants and funding opportunities from NIOSH and others, how to measure impact and evaluate success, use of social media and related metrics); pre-conference workshops (systematic reviews, update on research on underreporting of illness/injury and how to mitigate/address it); more efforts to engage industry partners in a meaningful way; and presentations from the political realm (senator, representative, political strategist, updating legislation). When asked for examples of why they might recommend NOIRS to a colleague, comments included: learn about current occupational injury research from both NIOSH and non-NIOSH researchers; both research and practice, and the integration of the two, are highlighted; strong co-sponsors; great opportunity for networking; scientific quality; accessibility of presenters and established researchers; it's an evidencebased conference; cost was low, making it easy for students to attend;
T
105
more than 30 poster presentations. The day before the conference, NIOSH offered a workshop on Injury Surveillance Tools and Techniques and tours of five engineering research laboratories at the NIOSH campus. The conference opening plenary focused on the conference theme and included presentations by John Howard, MD, Director of NIOSH; David Michaels, PhD, MPH, Assistant Secretary of Labor for Occupational Safety and Health; Deborah A. P. Hersman, MS, President and Chief Executive Officer of NSC; and Thomas F. Cecich, CSP, CIH, Senior Vice President of ASSE. The closing plenary included presentations on research-to-practice from a panel including Scott Schneider, CIH, Director of Occupational Safety and Health, Laborers' Health and Safety Fund of North America; Cindy DePrater, Vice President of Environmental Health and Safety, Turner Construction Company; Brad Boehler, President, Linamar Skyjack Group; Letitia Davis, ScD, EdM, Director of Occupational Health Surveillance Program, Massachusetts Department of Health; and Joan Mazur, PhD, Professor, College of Education, University of Kentucky and Southeast Center for Agricultural Health and Injury Prevention. The closing plenary concluded with an interactive discussion between audience and panel members on partnering throughout the research process. Concurrent scientific sessions included topics related to various industry sectors (e.g., construction, agriculture), high-risk and vulnerable worker groups (e.g., fire fighters, young workers), types of research (e.g., surveillance, laboratory, economics), injury causes (e.g., motor vehicle, violence, ergonomics, falls), prevention strategies and evaluation, and safety climate and culture, representing the spectrum of occupational injury topics.
D
78 79
C.R. Schuler et al. / Journal of Safety Research xxx (2016) xxx–xxx
E
2
Please cite this article as: Schuler, C.R., et al., A post-conference evaluation of the 2015 National Occupational Injury Research Symposium, Journal of Safety Research (2016), http://dx.doi.org/10.1016/j.jsr.2016.12.009
140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170 171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205
C.R. Schuler et al. / Journal of Safety Research xxx (2016) xxx–xxx Table 1 NOIRS conference evaluation results by attendance at previous NOIRS.
49% 92%
36% 9% 12% 21% 12% 0% 10%
47% 19% 15% 6% 2% 3% 8%
42% 14% 13% 13% 7% 2% 9%
40% 40% 12% 3% 5% 61% 71% 16% 82% 92% 89% 85% 82% 72%
50% 37% 11% 2% 0% 77% 77% 31% 83% 89% 88% 95% 82% 78%
45% 38% 12% 3% 3% 69% 74% 23% 83% 90% 88% 90% 82% 75%
4. Discussion
212 213
In attempting to assess the value of NOIRS 2015, and what might be introduced or improved in the next iteration, few examples of other conference evaluations were found in the literature. Wren, Allen, Proffitt, Riley, and Aiken (2013) reflected on the value of the World Conference on Injury Prevention and Safety Promotion. At this much larger conference (almost 900 delegates from 64 countries), the authors observed that it is difficult to assess the value and impact of such a
218
U
211
216 217
84% 83% 77% 67% 61% 47%
85% 80% 77% 70% 55% 48% 53% 6% 5% 5%
87% 80% 79% 69% 55% 53% 47% 13% 8% 6%
R
3
209 210
214 215
81% 90% 82% 65% 63% 52%
Responses: n = 54 1st time (16% missing); n = 57 previous (19% missing); n = 111 total (17% missing). Responses: n = 58 1st time (9% missing); n = 62 previous (11% missing); n = 120 total (10% missing). Responses: n = 59 1st time (8% missing); n = 61 previous (13% missing); n = 120 total (10% missing). 4 Percent who responded “good,” “very good,” or “excellent”. 5 Percent who responded “easy” or “very easy”. 6 Responses: n = 62 1st time (3% missing); n = 66 previous (6% missing); n = 128 total (5% missing). 7 Percent who responded “completely met” or “mostly met”. 8 Responses: n = 61 1st time (5% missing); n = 65 previous (7% missing); n = 126 total (6% missing). 9 Percent who responded “agree” or “strongly agree”. 10 Percent who responded “disagree” or “strongly disagree” to statement “topics were NOT relevant to me”. 11 Responses: n = 59 1st time (8% missing); n = 62 previous (11% missing); n = 121 total (10% missing). 12 Percent who responded “important” or “very important”. 2
manageable size; more people in safety should understand the kind of safety research that is happening; a comfortable environment; very specialized for occupational safety and health; and no other conference like it in the United States for those who do research related to occupational injuries.
207 208
O
R O D
88% 76% 71% 69% 59% 42% 89% 81% 81% 68% 56% 58% 40% 21% 11% 8%
F
37% 97%
N C O
206
61% 86%
E
1
Total (100%, n = 134)
C
t1:47 t1:48 t1:49 t1:50 t1:51 t1:52 t1:53 t1:54 t1:55 t1:56 t1:57 t1:58
Attended previous NOIRS (52%, n = 70)
E
Gender: % female1 Country of residency: % US2 Profession2 Public health research Engineering research Other research Public health practitioner/ Safety professional Student Educator Other Affiliation2 Federal/state government University/research institute Private sector/non-profit/non-government org Union Other Positively rated the meeting venue3,4 Attend if next NOIRS held in this venue: % yes2 Easy to locate meeting venue2,5 NOIRS met expectations6,7 NOIRS was well-run8,9 NOIRS was well-organized8,9 Adequate opportunity for networking8,9 Adequate time for speaker/audience interaction8,9 Topics were relevant to me8,10 Aspects contributing to learning experience11,7 Concurrent sessions Interactions with other participants Interactions with speakers Opening plenary Closing plenary Poster session Reasons for attending6,12 Exposure to new areas of research Share my research Develop new ideas for conducting research Develop new ideas for disseminating research Meet/identify potential research partners Develop new practice approaches Recharge/refocus Meet/identify potential employers Decide where to continue publ health education Decide whether to continue publ health research
R
t1:4 t1:5 t1:6 t1:7 t1:8 t1:9 t1:10 t1:11 t1:12 t1:13 t1:14 t1:15 t1:16 t1:17 t1:18 t1:19 t1:20 t1:21 t1:22 t1:23 t1:24 t1:25 t1:26 t1:27 t1:28 t1:29 t1:30 t1:31 t1:32 t1:33 t1:34 t1:35 t1:36 t1:37 t1:38 t1:39 t1:40 t1:41 t1:42 t1:43 t1:44 t1:45 t1:46
1st time attendees (48%, n = 64)
P
t1:3
T
t1:1 t1:2
3
meeting, given that only those attending could be queried. They conducted two surveys, one at the end of the meeting and a second two months later. They recommended that, prior to the next conference, organizers define conference objectives, identify key audiences and stakeholders, define how to measure value and impact, and identify the information that should be gathered during and after to measure value and impact. Arellano et al. (2014) evaluated a maternal and child health epidemiology and urban leadership conference at registration and six months post-conference for development of networking interactions, transfer of knowledge gained, application of knowledge gained, and use of knowledge gained. They concluded that conferences can lead to expanded networking and sharing of information, application of new knowledge, and translation of knowledge to propel action. This was the first time NOIRS conference organizers attempted to systematically collect information from those attending as to their
Please cite this article as: Schuler, C.R., et al., A post-conference evaluation of the 2015 National Occupational Injury Research Symposium, Journal of Safety Research (2016), http://dx.doi.org/10.1016/j.jsr.2016.12.009
219 220 221 222 223 224 225 226 227 228 229 230 231 232 233
257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290
300
293 294 295 296 297 298 299
O
F
The findings and conclusions in this report are those of the authors 301 and do not necessarily represent the views of the National Institute for 302 Occupational Safety and Health. 303 References
304
Arellano, D. E., Goodman, D. A., Howlette, T., Kroelinger, C. D., Law, M., Phillips, D., ... Fitzgerald, M. (2014). Evaluation of the 2012 18th Maternal and Child Health (MCH) Epidemiology and 22nd CityMatCH MCH Urban Leadership Conference: Six month impact on science, program, and policy. Maternal and Child Health Journal, 18(7), 1565–1571. http://dx.doi.org/10.1007/s10995-014-1585-x. Wren, J., Allen, K., Proffitt, C., Riley, H., & Aiken, M. (2013). What is the value and impact of the safety world conference? Evaluators' reflections of safety 2012 Injury Prevention, 19(6), 434–435. http://dx.doi.org/10.1136/injuryprev-2013-040889.
305 306 307 308 309 310 311 312
Christine R. Schuler, PhD is the associate director for science for the Division of Safety Research at NIOSH. Dr. Schuler was a research epidemiologist at NIOSH for 15 years before joining the Division of Safety Research. She conducted field studies in respiratory epidemiology in manufacturing facilities, where the workers taught her a great deal about translating the knowledge gained from conducting such studies and the impact science can have in making the workplace safer.
313 314 315 316 317 318 319
Dawn N. Castillo, MPH is the director of the Division of Safety Research at NIOSH. The Division serves as the focal point for traumatic occupational injury research and prevention programs at NIOSH. Prior to being appointed as division director in 2011, Ms. Castillo served as chief of the division branch responsible for occupational injury data collection, analysis and interpretation. Ms. Castillo is an epidemiologist and has interests in pragmatic research that can be translated into practice to improve worker safety.
320 321 322 323 324 325 326 327 328 329 330 331 332 333 334
R O
255 256
Disclaimer
P
253 254
291 292
D
251 252
reported that the symposium largely met their reasons for attending, which included being exposed to new areas of research, sharing their research, and developing new ideas for research. Respondents also reported favorably on opportunities to network. Respondents were generous with suggestions for improvements for the next NOIRS, which will be very useful to the NOIRS 2018 planning committee. The planning committee will also need to consider feedback on the location as they work to organize the next symposium that is economical to NIOSH and co-sponsors, and affordable and appealing to attendees.
T
249 250
C
247 248
E
245 246
R
243 244
R
241 242
O
240
C
238 239
N
236 237
expectations and experiences, and the perception of benefits gained immediately post-conference. The symposium theme was “Advancing Occupational Injury Research through Integration and Partnership.” The goals for integration and partnership were multi-faceted, including providing a forum for NIOSH and non-NIOSH researchers to learn about and build from the work of each other, encouraging sharing of information and discussions across disciplines, encouraging participation by students to help integrate them into the field, and encouraging discussions between researchers and practitioners. Registration and survey responses suggest success in providing a forum for NIOSH and nonNIOSH researchers to interact, with a good mix of researchers from NIOSH, other government agencies, university and research institutes, and positive responses to questions about sharing of research findings and networking. Survey responses suggest that if future NOIRS have similar goals for integration and partnership, more attention should be paid to attracting and engaging disciplines beyond public health, students, industry, and safety professionals. There was no registration fee for NOIRS 2015. It was hoped that this would increase conference attendance, especially among students, who often have little or no funding for meetings. NOIRS 2015 had 266 registrants, which was about a 20% increase over the previous two NOIRS, held in 2008 and 2011 with registration fees of $150 and $230, respectively. Since attendees at previous NOIRS had not been surveyed, it is not possible to evaluate the impact of cost on specific subsets of attendees, but the general increase in registration was promising. However, survey responses also suggest that the rural location presented challenges for some attendees and attendance at future NOIRS might be negatively impacted if again held in a difficult to reach location. The time of year (May) may have also affected attendance, as it may have conflicted with end-of-year testing or graduation. Interpretation of these survey results may be limited by only collecting information immediately after the conference, and not at some additional interval afterward (e.g., six months). Therefore, it is not possible to assess whether the perceived value in the short-term would still hold true later on. Although only 50% of those attending participated in the conference evaluation, the participants were similar to the overall registrants with respect to gender distribution, percent identifying the United States as their country of residence, and profession (e.g., percent from academia, either as faculty or students). Thus, the authors are satisfied that these results are less likely to be biased because of differential participation. NOIRS 2015 included objectives for integration and partnership. These objectives, which were partially met, hold promise. Increasing the mix of scientific disciplines and multidisciplinary work at NOIRS may help advance the occupational injury research field. Engaging more students may help establish a future stream of researchers and safety professionals to build and innovate upon current research approaches. Deliberate efforts to engage and provide a role for industry representatives and safety professionals at NOIRS may help ensure the relevance of occupational injury research to those in position to improve worker safety in the field, provide a bridge for translating research into practice, and provide a venue for networking among researchers, industry, and students. The committee planning the next conference should consider how NOIRS 2018 might be organized, publicized, and executed to best achieve these objectives. Overall, the authors believe the survey demonstrated that NOIRS has value to the occupational injury research community. Respondents
U
234 235
C.R. Schuler et al. / Journal of Safety Research xxx (2016) xxx–xxx
E
4
Cammie Chaumont Menéndez, MPH, PhD is a research epidemiologist within the Analysis and Field Evaluations Branch of the Division of Safety Research at NIOSH. Dr. Chaumont Menéndez first came to NIOSH in 2007 as an epidemic intelligence service officer and has continued her field research focused on the epidemiology of occupational injuries among high risk and vulnerable worker populations and identifying effective and cost-effective interventions to prevent them so workers can lead healthful and productive lives. Sergey Sinelnikov, MPH is a researcher at the National Safety Council (NSC) whose mission is to eliminate preventable deaths at work, in homes and communities, and on the road through leadership, research, education, and advocacy. Since joining the NSC in 2001, Mr. Sinelnikov has conducted research involving the evaluation of injury prevention interventions delivered in community, workplace, and transportation settings.
335 336 337 338 339 340
341 342 343 344 345 346 347 Emily Chavez, MPH joined the National Safety Council in 2010 as a research associate 348 conducting program evaluation for community, transportation and workplace injury pre- 349 vention interventions. Ms. Chavez is currently a senior program manager in the Advocacy 350 Department leading the launch of the fatigue initiative. 351 352 Sydney Webb, PhD is a health communications fellow in the Division of Safety Research at NIOSH. In this role she is responsible for developing strategic, multi-channel communication plans to promote Division research and products. Prior to joining NIOSH in 2014, Dr. Webb served as the director of member communications and engagement at a health and wellness company. During her tenure as director she led her team to win nine nationally recognized awards for its communication initiatives.
Please cite this article as: Schuler, C.R., et al., A post-conference evaluation of the 2015 National Occupational Injury Research Symposium, Journal of Safety Research (2016), http://dx.doi.org/10.1016/j.jsr.2016.12.009
Contents lists available at ScienceDirect
Journal of Safety Research journal homepage: www.elsevier.com/locate/jsr
3Q2 4
F
2
A post-conference evaluation of the 2015 National Occupational Injury Research Symposium Christine R. Schuler, a,⁎ Dawn N. Castillo, a Cammie Chaumont Menéndez, a Sergey Sinelnikov, b Sydney Webb, a Emily Chavez b
O
1Q1
5 6 7
a
8
a r t i c l e
9 10 11 12 13 39 40 41 42 43 44
Article history: Received 8 December 2016 Accepted 14 December 2016 Available online xxxx
49
1. Introduction
50
The National Occupational Injury Research Symposium (NOIRS) is the only national forum that is focused on occupational injury research findings, data and methods, and prevention strategies, and has been convened every three to five years since 1997. The theme of the sixth symposium, held in May 2015, was “Advancing Occupational Injury Research through Integration and Partnership.” NOIRS 2015 was sponsored by the National Institute for Occupational Safety and Health's (NIOSH) Division of Safety Research (DSR), and co-sponsored by the National Safety Council (NSC), the American Society of Safety Engineers (ASSE), Liberty Mutual Research Institute for Safety (LMRIS), and the Society for Advancement of Violence and Injury Research (SAVIR).
a b s t r a c t
Introduction: The National Occupational Injury Research, Symposium (NOIRS) is the only national forum focused on occupational injury research findings, data and methods, and prevention strategies; it has been convened every 3–5 years since 1997. Held in May 2015, the 6th symposium's theme was “Advancing Occupational Injury Research through Integration and Partnership.” Organizers requested that attendees complete a post-meeting evaluation to assess meeting impact, and gather information useful in planning subsequent meetings and activities. Method: The questionnaire was publicized via a quick response code and link to the survey on symposium book cover, and mentioned at each scientific session. The online survey was designed to be completed in ≤15 min; no identifying information was collected. Survey link remained open for seven days post-symposium. Results: About 50% of registered attendees responded. Almost half were attending their first NOIRS. Most were researchers (69%); 45% were affiliated with government and 38% with university or research institute. Five of six reported that the symposium mostly or completely met expectations. Reasons for attending included gaining exposure to new areas of research (87%), sharing their research (80%), and to develop new ideas for conducting research (79%). The majority (90%) reported that the symposium provided adequate networking opportunities. The conference venue was reported as good or better by 69%, moreso among repeat attendees (77%) compared to first-timers (61%). Discussion: The evaluation demonstrated that NOIRS was valuable to attendees, and provided a forum for sharing research results, developing new research ideas, and networking. Respondents provided input on different aspects of NOIRS and suggestions useful in planning next NOIRS, tentatively scheduled for 2018. NOIRS 2015 objectives for integration across disciplines and partnership with industry and safety professionals were partially met. In planning NOIRS 2018, more attention should be paid to attracting and engaging a broader spectrum of attendees. © 2016 Published by Elsevier Ltd.
Q3 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38
NOIRS 2015 was held in Kingwood, West Virginia (WV), about an hour's drive from the NIOSH campus in Morgantown, WV. Previous NOIRS had been held in Morgantown, WV and Pittsburgh, PA. The venue was a National Guard training complex and was economical. There were no registration fees. The venue was comfortable and in a beautiful but rural location. The nearest large airport was about a two and a half hour drive away. The symposium was attended by more than 250 occupational injury researchers, safety professionals, and students from six countries (including the United States), representing government (state and federal) agencies, universities, research institutes, private sector, labor, and non-profit/non-governmental organizations. The event was publicized through email, social media, and the efforts of the co-sponsors. The two and a half day conference agenda included opening and closing plenaries, eight groups of four to five concurrent topic-specific scientific sessions (with a total of 135 platform presentations), a tutorial session, and two evening networking events, one including a poster session with
61
P
i n f o
R O
National Institute for Occupational Safety and Health, Division of Safety Research, 1095 Willowdale Road, MS H1900, Morgantown, WV 26505, United States National Safety Council, 1121 Spring Lake Drive, Itasca, IL 60143, United States
D
b
53 54 55 56 57 58 59 60
U
51 52
N C O
R
R
E
C
T
E
Keywords: Surveys and questionnaires Occupational injuries Occupational health Safety Conference
⁎ Corresponding author. E-mail address: [email protected] (C.R. Schuler).
http://dx.doi.org/10.1016/j.jsr.2016.12.009 0022-4375/© 2016 Published by Elsevier Ltd.
Please cite this article as: Schuler, C.R., et al., A post-conference evaluation of the 2015 National Occupational Injury Research Symposium, Journal of Safety Research (2016), http://dx.doi.org/10.1016/j.jsr.2016.12.009
62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77
126
3. Survey results
127 128
Among the 266 individuals who registered to attend NOIRS 2015, 134 (50%) took part in the conference evaluation; 90% (120/134) completed the full evaluation. Almost half of the respondents were attending their first NOIRS (48%, 64/134) (Table 1). Respondents were equally split between genders, and were mostly U.S. residents. Professions were predominantly research-related (69%); 45% were affiliated with federal or state governments, and more than a third were from academia. When asked about the meeting location, 74% reported that they would attend again if offered at the same location; only 23% reported that the venue was easy to locate. Five of six respondents (83%) reported that NOIRS 2015 completely or mostly met their expectations (Table 1). Most respondents either agreed or strongly agreed that the conference was well-run (i.e., sessions and
103 104
108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123
129 130 131 132 133 134 135 136 137 138 139
C
101 102
E
99 100
R
97 98
R
95 96
O
93 94
C
91 92
N
89 90
U
87 88
F
124 125
The conference organizers developed a post-conference evaluation questionnaire to assess the value and impact of NOIRS 2015, and to better inform planning for the next symposium, tentatively scheduled for 2018. Areas of particular interest from the survey were conference experience and impact, particularly how well various aspects of the conference met the expectations of attendees, how attendees intended to use the knowledge and contacts obtained at the conference, and what might be improved for the next NOIRS; a comparison of the experiences of first-time attendees and those who had attended one or more prior NOIRS; as well as demographic and professional information. The online survey was designed to be completed in less than 15 min, and no personally identifying information was collected. The instrument included closed-ended questions with response options, as well as some open-ended questions that required narrative responses. The survey was publicized throughout the meeting via a quick response (QR) code and link to the survey on the cover of the symposium book, and mentioned at the beginning of each scientific session. An email was distributed after the meeting to remind all registered attendees to complete the evaluation within seven days.
85 86
O
107
84
R O
2. Survey instrument and methods
82 83
P
106
80 81
presentations kept to schedule, 90%), that the scientific content was organized appropriately into sessions (88%), that there was adequate opportunity for networking with other attendees (90%), and that there was adequate time for audience members to interact with speakers (82%). A slightly smaller majority believed that the topics presented were relevant to them (75%). Responses were not dissimilar between first-time and repeat attendees. When asked about various aspects of the meeting, most respondents reported that the concurrent scientific sessions (84%) and their conversations and interactions with conference participants (83%) completely or mostly contributed to their learning experience (Table 1). The combined evening networking and poster session was reported to contribute less to the learning experience (47% completely or mostly). Responses were similar between first-time and repeat attendees, although 90% of repeat attendees reported benefitting from their interactions with conference participants compared to 76% of first-time attendees. When asked why they decided to attend NOIRS 2015, 87% reported that gaining exposure to new areas of research was important or very important, 80% reported to share their research, and 79% reported to develop new ideas for conducting research (Table 1). Having the opportunity to recharge or refocus was reported as important or very important by 47%. Although not reported as important or very important by many, there were differences in relative importance between first-time attendees and repeat attendees for meeting or identifying potential employers (21% and 6%, respectively), developing new practice approaches from what was learned at NOIRS (58% and 48%), deciding where to continue their public health education (11% and 5%), and the opportunity to recharge or refocus (40% and 53%). When asked for specific information about what they gained at NOIRS that could not be gained elsewhere, respondents remarked on the attendees (responses included interesting multidisciplinary group, opportunity for a safety practitioner to interact with researchers, inspired by the energy and commitment of the presenters and attendees); the size of the conference (size makes it easier to network, small conference environment encourages more interaction between speakers and audiences and also between participants, opportunity for one-on-one conversations with key people); the focused nature of the meeting (concentrated on occupational injury research); and the atmosphere (relaxed, conducive to learning and interacting). The respondents also provided many suggestions for what they'd like to see offered at future NOIRS, including topic areas (more intervention research, more sleep and fatigue research, psychological/cognitive demands of changing/hazardous work environments, cost effectiveness of research, workplace road safety, other transportation-related aspects, relationship of injury with psychosocial environment, impact of restructuring of work on injury causation, causes of rail disasters, national or state level campaigns for injury prevention, economic costs of injury to families and communities, young workers as a vulnerable population); tutorials (conducting in-house workplace research, statistical software sessions, more contemporary methods, advanced epidemiologic and statistical methods, how to disseminate research, grants and funding opportunities from NIOSH and others, how to measure impact and evaluate success, use of social media and related metrics); pre-conference workshops (systematic reviews, update on research on underreporting of illness/injury and how to mitigate/address it); more efforts to engage industry partners in a meaningful way; and presentations from the political realm (senator, representative, political strategist, updating legislation). When asked for examples of why they might recommend NOIRS to a colleague, comments included: learn about current occupational injury research from both NIOSH and non-NIOSH researchers; both research and practice, and the integration of the two, are highlighted; strong co-sponsors; great opportunity for networking; scientific quality; accessibility of presenters and established researchers; it's an evidencebased conference; cost was low, making it easy for students to attend;
T
105
more than 30 poster presentations. The day before the conference, NIOSH offered a workshop on Injury Surveillance Tools and Techniques and tours of five engineering research laboratories at the NIOSH campus. The conference opening plenary focused on the conference theme and included presentations by John Howard, MD, Director of NIOSH; David Michaels, PhD, MPH, Assistant Secretary of Labor for Occupational Safety and Health; Deborah A. P. Hersman, MS, President and Chief Executive Officer of NSC; and Thomas F. Cecich, CSP, CIH, Senior Vice President of ASSE. The closing plenary included presentations on research-to-practice from a panel including Scott Schneider, CIH, Director of Occupational Safety and Health, Laborers' Health and Safety Fund of North America; Cindy DePrater, Vice President of Environmental Health and Safety, Turner Construction Company; Brad Boehler, President, Linamar Skyjack Group; Letitia Davis, ScD, EdM, Director of Occupational Health Surveillance Program, Massachusetts Department of Health; and Joan Mazur, PhD, Professor, College of Education, University of Kentucky and Southeast Center for Agricultural Health and Injury Prevention. The closing plenary concluded with an interactive discussion between audience and panel members on partnering throughout the research process. Concurrent scientific sessions included topics related to various industry sectors (e.g., construction, agriculture), high-risk and vulnerable worker groups (e.g., fire fighters, young workers), types of research (e.g., surveillance, laboratory, economics), injury causes (e.g., motor vehicle, violence, ergonomics, falls), prevention strategies and evaluation, and safety climate and culture, representing the spectrum of occupational injury topics.
D
78 79
C.R. Schuler et al. / Journal of Safety Research xxx (2016) xxx–xxx
E
2
Please cite this article as: Schuler, C.R., et al., A post-conference evaluation of the 2015 National Occupational Injury Research Symposium, Journal of Safety Research (2016), http://dx.doi.org/10.1016/j.jsr.2016.12.009
140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170 171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205
C.R. Schuler et al. / Journal of Safety Research xxx (2016) xxx–xxx Table 1 NOIRS conference evaluation results by attendance at previous NOIRS.
49% 92%
36% 9% 12% 21% 12% 0% 10%
47% 19% 15% 6% 2% 3% 8%
42% 14% 13% 13% 7% 2% 9%
40% 40% 12% 3% 5% 61% 71% 16% 82% 92% 89% 85% 82% 72%
50% 37% 11% 2% 0% 77% 77% 31% 83% 89% 88% 95% 82% 78%
45% 38% 12% 3% 3% 69% 74% 23% 83% 90% 88% 90% 82% 75%
4. Discussion
212 213
In attempting to assess the value of NOIRS 2015, and what might be introduced or improved in the next iteration, few examples of other conference evaluations were found in the literature. Wren, Allen, Proffitt, Riley, and Aiken (2013) reflected on the value of the World Conference on Injury Prevention and Safety Promotion. At this much larger conference (almost 900 delegates from 64 countries), the authors observed that it is difficult to assess the value and impact of such a
218
U
211
216 217
84% 83% 77% 67% 61% 47%
85% 80% 77% 70% 55% 48% 53% 6% 5% 5%
87% 80% 79% 69% 55% 53% 47% 13% 8% 6%
R
3
209 210
214 215
81% 90% 82% 65% 63% 52%
Responses: n = 54 1st time (16% missing); n = 57 previous (19% missing); n = 111 total (17% missing). Responses: n = 58 1st time (9% missing); n = 62 previous (11% missing); n = 120 total (10% missing). Responses: n = 59 1st time (8% missing); n = 61 previous (13% missing); n = 120 total (10% missing). 4 Percent who responded “good,” “very good,” or “excellent”. 5 Percent who responded “easy” or “very easy”. 6 Responses: n = 62 1st time (3% missing); n = 66 previous (6% missing); n = 128 total (5% missing). 7 Percent who responded “completely met” or “mostly met”. 8 Responses: n = 61 1st time (5% missing); n = 65 previous (7% missing); n = 126 total (6% missing). 9 Percent who responded “agree” or “strongly agree”. 10 Percent who responded “disagree” or “strongly disagree” to statement “topics were NOT relevant to me”. 11 Responses: n = 59 1st time (8% missing); n = 62 previous (11% missing); n = 121 total (10% missing). 12 Percent who responded “important” or “very important”. 2
manageable size; more people in safety should understand the kind of safety research that is happening; a comfortable environment; very specialized for occupational safety and health; and no other conference like it in the United States for those who do research related to occupational injuries.
207 208
O
R O D
88% 76% 71% 69% 59% 42% 89% 81% 81% 68% 56% 58% 40% 21% 11% 8%
F
37% 97%
N C O
206
61% 86%
E
1
Total (100%, n = 134)
C
t1:47 t1:48 t1:49 t1:50 t1:51 t1:52 t1:53 t1:54 t1:55 t1:56 t1:57 t1:58
Attended previous NOIRS (52%, n = 70)
E
Gender: % female1 Country of residency: % US2 Profession2 Public health research Engineering research Other research Public health practitioner/ Safety professional Student Educator Other Affiliation2 Federal/state government University/research institute Private sector/non-profit/non-government org Union Other Positively rated the meeting venue3,4 Attend if next NOIRS held in this venue: % yes2 Easy to locate meeting venue2,5 NOIRS met expectations6,7 NOIRS was well-run8,9 NOIRS was well-organized8,9 Adequate opportunity for networking8,9 Adequate time for speaker/audience interaction8,9 Topics were relevant to me8,10 Aspects contributing to learning experience11,7 Concurrent sessions Interactions with other participants Interactions with speakers Opening plenary Closing plenary Poster session Reasons for attending6,12 Exposure to new areas of research Share my research Develop new ideas for conducting research Develop new ideas for disseminating research Meet/identify potential research partners Develop new practice approaches Recharge/refocus Meet/identify potential employers Decide where to continue publ health education Decide whether to continue publ health research
R
t1:4 t1:5 t1:6 t1:7 t1:8 t1:9 t1:10 t1:11 t1:12 t1:13 t1:14 t1:15 t1:16 t1:17 t1:18 t1:19 t1:20 t1:21 t1:22 t1:23 t1:24 t1:25 t1:26 t1:27 t1:28 t1:29 t1:30 t1:31 t1:32 t1:33 t1:34 t1:35 t1:36 t1:37 t1:38 t1:39 t1:40 t1:41 t1:42 t1:43 t1:44 t1:45 t1:46
1st time attendees (48%, n = 64)
P
t1:3
T
t1:1 t1:2
3
meeting, given that only those attending could be queried. They conducted two surveys, one at the end of the meeting and a second two months later. They recommended that, prior to the next conference, organizers define conference objectives, identify key audiences and stakeholders, define how to measure value and impact, and identify the information that should be gathered during and after to measure value and impact. Arellano et al. (2014) evaluated a maternal and child health epidemiology and urban leadership conference at registration and six months post-conference for development of networking interactions, transfer of knowledge gained, application of knowledge gained, and use of knowledge gained. They concluded that conferences can lead to expanded networking and sharing of information, application of new knowledge, and translation of knowledge to propel action. This was the first time NOIRS conference organizers attempted to systematically collect information from those attending as to their
Please cite this article as: Schuler, C.R., et al., A post-conference evaluation of the 2015 National Occupational Injury Research Symposium, Journal of Safety Research (2016), http://dx.doi.org/10.1016/j.jsr.2016.12.009
219 220 221 222 223 224 225 226 227 228 229 230 231 232 233
257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290
300
293 294 295 296 297 298 299
O
F
The findings and conclusions in this report are those of the authors 301 and do not necessarily represent the views of the National Institute for 302 Occupational Safety and Health. 303 References
304
Arellano, D. E., Goodman, D. A., Howlette, T., Kroelinger, C. D., Law, M., Phillips, D., ... Fitzgerald, M. (2014). Evaluation of the 2012 18th Maternal and Child Health (MCH) Epidemiology and 22nd CityMatCH MCH Urban Leadership Conference: Six month impact on science, program, and policy. Maternal and Child Health Journal, 18(7), 1565–1571. http://dx.doi.org/10.1007/s10995-014-1585-x. Wren, J., Allen, K., Proffitt, C., Riley, H., & Aiken, M. (2013). What is the value and impact of the safety world conference? Evaluators' reflections of safety 2012 Injury Prevention, 19(6), 434–435. http://dx.doi.org/10.1136/injuryprev-2013-040889.
305 306 307 308 309 310 311 312
Christine R. Schuler, PhD is the associate director for science for the Division of Safety Research at NIOSH. Dr. Schuler was a research epidemiologist at NIOSH for 15 years before joining the Division of Safety Research. She conducted field studies in respiratory epidemiology in manufacturing facilities, where the workers taught her a great deal about translating the knowledge gained from conducting such studies and the impact science can have in making the workplace safer.
313 314 315 316 317 318 319
Dawn N. Castillo, MPH is the director of the Division of Safety Research at NIOSH. The Division serves as the focal point for traumatic occupational injury research and prevention programs at NIOSH. Prior to being appointed as division director in 2011, Ms. Castillo served as chief of the division branch responsible for occupational injury data collection, analysis and interpretation. Ms. Castillo is an epidemiologist and has interests in pragmatic research that can be translated into practice to improve worker safety.
320 321 322 323 324 325 326 327 328 329 330 331 332 333 334
R O
255 256
Disclaimer
P
253 254
291 292
D
251 252
reported that the symposium largely met their reasons for attending, which included being exposed to new areas of research, sharing their research, and developing new ideas for research. Respondents also reported favorably on opportunities to network. Respondents were generous with suggestions for improvements for the next NOIRS, which will be very useful to the NOIRS 2018 planning committee. The planning committee will also need to consider feedback on the location as they work to organize the next symposium that is economical to NIOSH and co-sponsors, and affordable and appealing to attendees.
T
249 250
C
247 248
E
245 246
R
243 244
R
241 242
O
240
C
238 239
N
236 237
expectations and experiences, and the perception of benefits gained immediately post-conference. The symposium theme was “Advancing Occupational Injury Research through Integration and Partnership.” The goals for integration and partnership were multi-faceted, including providing a forum for NIOSH and non-NIOSH researchers to learn about and build from the work of each other, encouraging sharing of information and discussions across disciplines, encouraging participation by students to help integrate them into the field, and encouraging discussions between researchers and practitioners. Registration and survey responses suggest success in providing a forum for NIOSH and nonNIOSH researchers to interact, with a good mix of researchers from NIOSH, other government agencies, university and research institutes, and positive responses to questions about sharing of research findings and networking. Survey responses suggest that if future NOIRS have similar goals for integration and partnership, more attention should be paid to attracting and engaging disciplines beyond public health, students, industry, and safety professionals. There was no registration fee for NOIRS 2015. It was hoped that this would increase conference attendance, especially among students, who often have little or no funding for meetings. NOIRS 2015 had 266 registrants, which was about a 20% increase over the previous two NOIRS, held in 2008 and 2011 with registration fees of $150 and $230, respectively. Since attendees at previous NOIRS had not been surveyed, it is not possible to evaluate the impact of cost on specific subsets of attendees, but the general increase in registration was promising. However, survey responses also suggest that the rural location presented challenges for some attendees and attendance at future NOIRS might be negatively impacted if again held in a difficult to reach location. The time of year (May) may have also affected attendance, as it may have conflicted with end-of-year testing or graduation. Interpretation of these survey results may be limited by only collecting information immediately after the conference, and not at some additional interval afterward (e.g., six months). Therefore, it is not possible to assess whether the perceived value in the short-term would still hold true later on. Although only 50% of those attending participated in the conference evaluation, the participants were similar to the overall registrants with respect to gender distribution, percent identifying the United States as their country of residence, and profession (e.g., percent from academia, either as faculty or students). Thus, the authors are satisfied that these results are less likely to be biased because of differential participation. NOIRS 2015 included objectives for integration and partnership. These objectives, which were partially met, hold promise. Increasing the mix of scientific disciplines and multidisciplinary work at NOIRS may help advance the occupational injury research field. Engaging more students may help establish a future stream of researchers and safety professionals to build and innovate upon current research approaches. Deliberate efforts to engage and provide a role for industry representatives and safety professionals at NOIRS may help ensure the relevance of occupational injury research to those in position to improve worker safety in the field, provide a bridge for translating research into practice, and provide a venue for networking among researchers, industry, and students. The committee planning the next conference should consider how NOIRS 2018 might be organized, publicized, and executed to best achieve these objectives. Overall, the authors believe the survey demonstrated that NOIRS has value to the occupational injury research community. Respondents
U
234 235
C.R. Schuler et al. / Journal of Safety Research xxx (2016) xxx–xxx
E
4
Cammie Chaumont Menéndez, MPH, PhD is a research epidemiologist within the Analysis and Field Evaluations Branch of the Division of Safety Research at NIOSH. Dr. Chaumont Menéndez first came to NIOSH in 2007 as an epidemic intelligence service officer and has continued her field research focused on the epidemiology of occupational injuries among high risk and vulnerable worker populations and identifying effective and cost-effective interventions to prevent them so workers can lead healthful and productive lives. Sergey Sinelnikov, MPH is a researcher at the National Safety Council (NSC) whose mission is to eliminate preventable deaths at work, in homes and communities, and on the road through leadership, research, education, and advocacy. Since joining the NSC in 2001, Mr. Sinelnikov has conducted research involving the evaluation of injury prevention interventions delivered in community, workplace, and transportation settings.
335 336 337 338 339 340
341 342 343 344 345 346 347 Emily Chavez, MPH joined the National Safety Council in 2010 as a research associate 348 conducting program evaluation for community, transportation and workplace injury pre- 349 vention interventions. Ms. Chavez is currently a senior program manager in the Advocacy 350 Department leading the launch of the fatigue initiative. 351 352 Sydney Webb, PhD is a health communications fellow in the Division of Safety Research at NIOSH. In this role she is responsible for developing strategic, multi-channel communication plans to promote Division research and products. Prior to joining NIOSH in 2014, Dr. Webb served as the director of member communications and engagement at a health and wellness company. During her tenure as director she led her team to win nine nationally recognized awards for its communication initiatives.
Please cite this article as: Schuler, C.R., et al., A post-conference evaluation of the 2015 National Occupational Injury Research Symposium, Journal of Safety Research (2016), http://dx.doi.org/10.1016/j.jsr.2016.12.009