Leading indicators of occupational health and safety: An employee and workplace level validation study

Safety Science 85 (2016) 293–304

Contents lists available at ScienceDirect

Safety Science journal homepage: www.elsevier.com/locate/ssci

Leading indicators of occupational health and safety: An employee and workplace level validation study Tracey Shea ⇑, Helen De Cieri 1, Ross Donohue 2, Brian Cooper 3, Cathy Sheehan 4 Monash University, Department of Management, Faculty of Business and Economics, P.O. Box 197, Caulfield East, VIC 3145, Australia

a r t i c l e

i n f o

Article history: Received 10 April 2015 Received in revised form 22 October 2015 Accepted 2 January 2016

Keywords: Leading indicators Occupational health and safety Rasch analysis Scale validation

a b s t r a c t There is growing interest in advancing knowledge and practice on the use of leading indicators to measure occupational health and safety (OHS) performance in organizations. In response we present psychometric analysis of the Organizational Performance Metric – Monash University (OPM-MU), which is a recently developed measure of leading indicators of OHS with several adaptations made as part of our investigation. Based on a national survey conducted with 3605 employees in 66 workplaces from several major organizations in Australia, we applied classical test (exploratory factor analysis) and item response (Rasch model analysis) theories to conduct a psychometric evaluation of the OPM-MU. Results revealed that the OPM-MU displayed good psychometric properties and evidence for both construct and criterion validity at employee and workplace levels. The OPM-MU could be used as an initial ‘flag’ of the leading indicators of OHS and has the potential to be a benchmarking tool for workplaces both within and across organizations. This paper represents an important advancement in the field of leading indicators of OHS performance and demonstrates that the OPM-MU is a promising new tool with demonstrated reliability and validity. Ó 2016 Elsevier Ltd. All rights reserved.

1. Introduction Occupational health and safety (OHS) has been, and continues to be, a priority area for policy-makers, managers and workers. Occupational injuries and diseases result in significant costs to employers and impact on the private and social lives of individuals (Battaglia et al., 2015; Lebeau et al., 2014). In view of the consequences of effective OHS, the current study adds to research on a proactive and positive OHS management approach through the investigation of leading indicators of OHS. Research on leading indicators has grown in recent years with contributions from a range of sectors including academia, industry, and government (Reiman and Pietikäinen, 2012; Sinelnikov et al., 2015). Leading indicators can be thought of as precursors to harm that provide early warning signals of potential failure; as such they offer organizations the opportunity to detect and mitigate risks ⇑ Corresponding author. Tel.: +61 3 990 34314. E-mail addresses: [email protected] (T. Shea), [email protected] (H. De Cieri), [email protected] (R. Donohue), [email protected] (B. Cooper), [email protected] (C. Sheehan). 1 Tel.: +61 3 990 32013. 2 Tel.: +61 3 990 31548. 3 Tel.: +61 3 990 31233. 4 Tel.: +61 3 990 32228. http://dx.doi.org/10.1016/j.ssci.2016.01.015 0925-7535/Ó 2016 Elsevier Ltd. All rights reserved.

or risk increases before an OHS incident occurs or a hazardous state is reached (Sinelnikov et al., 2015). They can be viewed as measures of the positive steps that organizations take that may prevent an OHS incident from occurring (Grabowski et al., 2007; Lingard et al., 2011). Baker et al. (2007: H2) define leading indicators as ‘‘A metric that attempts to measure some variable that is believed to be an indicator or precursor of future safety performance.” While some analysis has focused on leading indicators of safety performance (e.g., Hinze et al., 2013; Reiman and Pietikäinen, 2012), we build on the recent work by Sinelnikov et al. (2015) and address the broader concept of OHS performance. A common approach to the measurement of OHS performance is the separation of leading indicators from lagging indicators (Dyreborg, 2009; Hopkins, 2009; Kjellén, 2009). This has emerged in response to the heavy reliance in many OHS organizational initiatives on lagging indicators such as injury rates. Lagging indicators are measures of OHS outcomes or outputs and provide a measure of past performance (Erikson, 2009). OHS outcomes are tangible events or results, such as accidents, injuries, or fatalities (Christian et al., 2009). Examples of lagging indicators include: near-misses; lost time injury frequency rate; medical treatment injury frequency rate; and claims for compensation. In contrast to leading indicators, lagging indicators measure events or outcomes that have already happened (Hopkins, 2009); they are

294

T. Shea et al. / Safety Science 85 (2016) 293–304

‘‘failure-focused [and] are less useful in helping organizations drive continuous improvements” (Sinelnikov et al., 2015: p. 240). Safety culture within an organization is a dynamic phenomenon and focusing on lagging indicators, which change after the actual risk level of the organization has altered, may be of limited value (Reiman and Pietikäinen, 2012). As Blair and O’Toole (2010: 29) have explained, leading indicators, on the other hand, ‘‘. . . measure actions, behaviors and processes, the things people actually do for safety, and not simply the safety-related failures typically tracked by trailing [lagging] measures.” Given the high costs in human, social, economic and financial terms related to OHS outcomes, it is important to understand how OHS leading indicators and various workplace contextual factors and working conditions may influence OHS outcomes (Clarke, 2006b, 2013; Nahrgang et al., 2011). It would generally be expected that more positive OHS leading indicators and a greater presence of health and safety features in the workplace would be negatively associated with OHS outcomes such as work-related injuries or claims for compensation. We acknowledge Hopkins’ (2009) argument that there may be little achieved by trying to develop precise meanings of the two terms (leading and lagging) because in different contexts they are used to refer to different indicators. However, as discussed by Dyreborg (2009), this distinction and the causal relationships between leading and lagging indicators is important. This idea is particularly important in psychometric analysis where it is necessary to establish a pattern of relationships between a construct under examination and measures of other related and unrelated constructs. Despite some definitional overlap, Sinelnikov et al. (2015: 240) recently noted that ‘‘a general consensus exists for the use of leading indicators as a measure of OHS performance”. Sinelnikov et al. (2015: 248) called for the development and validation of a ‘‘standard index of leading indicators that could be used for benchmarking across organizations.” Our study seeks to address this call. In this paper, we initially examine the literature to present the current status of leading indicators of OHS along with extant measures of this construct with the aim of validating the most theoretically sound and practical exemplar of those measures. Specifically, we conducted a review of the literature on the leading indicators of OHS that would enable us to: 1. categorize the features of leading indicators of OHS and 2. identify existing measures of leading indicators of OHS that are consistent with those categorizations. Following this examination of the literature the aim of this paper was to: 3. identify and validate a short and practical measure of leading indicators that has the potential to be a benchmarking tool for workplaces both within and across organizations. The discussion in following sections will highlight the elements of leading indicators of OHS and potential measures of the construct. This will be followed by a discussion of the Organizational Performance Metric (OPM: IWH, 2011, 2013) which was selected as the exemplar measure for validation in this study. 1.1. Identifying the features of leading indicators of OHS performance Several different categorizations of factors exist that could be considered as leading indicators of OHS performance. Developments in research have broadened the focal point from safety climate (Flin et al., 2000) to safety management systems (Fernández-Muñiz et al., 2009), safety performance (Reiman and Pietikäinen, 2012), and most recently to OHS performance

(Sinelnikov et al., 2015). Several of these categorizations are focused on micro-level metrics and/or have been validated within one industry. We adopt an inclusive approach and, following Sinelnikov et al.’s (2015) call, we seek to identify leading indicators at a high level, to facilitate benchmarking across industries and organizational contexts. Based on our review of the literature, we propose that the construct of leading indicators of OHS performance encompasses 10 areas: OHS systems (policies, procedures, practices); management commitment and leadership; OHS training, interventions, information, tools and resources; workplace OHS inspections and audits; consultation and communication about OHS; prioritization of OHS; OHS empowerment and employee involvement in decision making; OHS accountability; positive feedback and recognition for OHS; and risk management. Table 1 displays each area with descriptors and examples of key authors who have discussed each feature. The 10 identified areas are consistent with, and extend upon, previous efforts to identify the important characteristics of leading indicators of OHS performance. 1.2. Measuring leading indicators of OHS performance We now turn to an examination of the literature on the measurement of leading indicators where we report on a literature search conducted to identify measures of leading indicators of OHS. The purpose of the review was to search both the academic and grey literatures to examine the scope of the leading indicators construct and identify a measure of lead indicators that could be used to make a preliminary assessment of the predictors of OHS performance in workplaces. We applied the following inclusion criteria, each measure should: (1) address the features of leading indicators of OHS performance; (2) be a generic measure that could be applied across different industries and at different levels of analysis within the organization; and (3) be comprised of Likerttype items that could be summed or averaged to yield a scale score. All measures that represented lagging indicators of OHS or safety, measures that were designed for a specific industry or type of job, and measures that assessed safety in relation to employee experiences at work were excluded. Additionally, studies that investigated leading indicators of OHS through extensive surveys but did not present their items as a well-defined scale were also excluded (e.g., Geldart et al., 2010; Mariscal et al., 2012). Several academic databases were searched (Business Source Complete, EMBASE, PsychInfo, Emerald, Science Direct) using a timeline from January 2000 to June 2013. A systematic search of the table of contents of academic journals that publish articles on OHS or safety was also conducted in the following journals: Academy of Management Journal, Accident Analysis and Prevention, Journal of Applied Psychology, Journal of Management, Journal of Occupational and Organizational Psychology, Journal of Occupational Health and Safety in Australia and New Zealand, Journal of Safety Research, Professional Safety and Safety Science. The search of grey literature focused on webpages from organizations and government bodies that conduct research and/or collect statistical information on safety in Australia (e.g., WorkSafe Victoria, SafeWork Australia, Safety Institute of Australia, WorkSafe WA, Work Cover NSW); the United Kingdom (Health & Safety Executive); North America (Institute for Work & Health in Canada, National Institute of Safety & Occupational Health in USA) and globally (SAI Global, International Labour Organization, World Health Organization). As leading indicators is a variable primarily used in the field of economics, a key search in the Business Source Complete database using this term resulted in approximately 10,000 citations. Consequently, we conducted a Boolean search where the lead indicator term was coupled with other safety key words and phrases such

295

T. Shea et al. / Safety Science 85 (2016) 293–304 Table 1 Leading indicators of occupational health and safety. Indicator

Description

Selected authors

Accountability for OHS

A workplace culture that emphasizes a sense of shared responsibility and accountability for OHS, by actively applying scrutiny and transparency in reporting, is likely to influence behavior in the workplace This refers to regular, formal and informal communication and consultation about OHS. Employee surveys may be one way of gathering information from employees regarding their perceptions of OHS It is widely understood that employee involvement in decision making will lead to ‘ownership’ of their behavior and positive outcomes, such as safety behavior. Several researchers have investigated the role of empowerment and engagement in OHS and found that empowerment of workers and supervisors to make decisions with regard to OHS (e.g., to stop work that is unsafe) is a leading indicator of OHS performance As with any organizational initiative, management commitment is key to OHS. This includes managers at all levels, from board and senior executive levels to front-line supervisors. Effective commitment is demonstrated in active engagement in areas such as information gathering about OHS, building trust so all employees view managers as committed to OHS, managers’ behavior demonstrating that they are OHS role models; and managers demonstrating that OHS is a high priority across the organization It is suggested that high performance on OHS will be reinforced by positive feedback and recognition for past performance. Such recognition should not, however, include rewards that might lead to under-reporting of incidents or injuries The tendency for safety to be traded off against productivity has been discussed at length by OHS academics. Rather than view safety and productivity as competing goals, OHS embedded in the organization as a high priority alongside efficiency and productivity can be viewed as a leading indicator of OHS performance This refers to the integration of risk management with the management of OHS; aspects of risk management include risk assessment, control, inspection and maintenance. Risks may be associated with psychosocial, physical and/or physiological dimensions of OHS These systems refer to workplace policies, processes and practices designed to control and monitor OHS, and are typically implemented and maintained by managers and in work groups Along with the resourcing of OHS with suitably qualified OHS specialist expertise, the provision of OHS training, information, tools and resources are key leading indicators of OHS performance. This includes preparedness to act and having a response plan in place A phrase often attributed to management scholar Peter Drucker: is ‘‘What gets measured, gets managed.” An important implication of this is that the conduct of an audit or inspection may not in itself be adequate as a leading indicator of OHS performance. Inspections and audits should be designed to provide appropriate and comprehensive information. Appropriate and timely corrective action should be taken to address issues identified in audits or inspections

Dyreborg (2009) and Fernández-Muñiz et al. (2009)

Consultation and communication about OHS Empowerment and employee involvement in decision making about OHS

Management commitment and leadership

Positive feedback and recognition for OHS

Prioritization of OHS

Risk management

Systems for OHS (policies, procedures, practices) Training, interventions, information, tools and resources for OHS Workplace OHS inspections and audits

as: antecedents of safety, health and safety management, occupational health and safety, organizational safety measurement, performance indicators, positive performance measures, safety climate, safety culture, safety management system, safety metrics, safety performance and work safety indicator to focus the search. We found few measures in the literature, however, that were designed with the specific intention of measuring leading indicators of OHS performance. Therefore, we extended the search to include measures that referred to organizational safety practices that could be perceived as leading indicators of OHS performance, such as safety climate. Table 2 summarises the outcomes of the review process (also see Appendix A). Forty-two scales met our inclusion criteria and, notably, only two scales were found that were explicitly labelled by their developers as measures of leading indicators of OHS; of those remaining, most were described by their authors as measures of safety climate. Each of the 42 measures contained items that addressed some of the 10 features of leading indicators of OHS. Items representing the concept of management commitment and leadership were found in almost all measures. Similarly, items that addressed OHS training, interventions, information, tools and resources along with OHS empowerment and employee involvement in decision making were found in more than half of the measures identified in the review.

DeJoy et al. (2004), Grabowski et al. (2007) and Health and Safety Executive (2005) Nahrgang et al. (2011), Wiegand (2007) and Wurzelbacher and Jin (2011)

Choudhry et al. (2007), Frazier et al. (2013), Health and Safety Executive (2005), Lingard et al. (2011) and Zohar (2010)

Daniels and Marlow (2005)

Glendon and Litherland (2001), Health and Safety Executive (2005), Van Dyck et al. (2013) and Zanko and Dawson (2012)

Fernández-Muñiz et al. (2009), Hopkins (2009), Kjellén (2009) and Pidgeon (1991)

Frazier et al. (2013), Payne et al. (2009), Pidgeon (1991), Wachter and Yorio (2014) and Wurzelbacher and Jin (2011) Health and Safety Executive (2005) and Lingard et al. (2011)

Carson and Snowden (2010) and Hallowell et al. (2013)

The elements of the leading indicators construct that were least likely to be incorporated into extant measures of safety were risk management, workplace OHS inspections and audits, OHS accountability and positive feedback and recognition for OHS. Items addressing these concepts were included in less than one third of the safety scales. The measures identified in the literature review were appraised on the basis of their theoretical significance and practical value. Several measures were found that addressed the leading indicators construct to some degree. It should be noted that the measurement of leading indicators of OHS was not the intended purpose for the development for most of these measures. However, they were evaluated to determine whether it was possible to either use or adapt one of them as a measure of leading indicators of OHS. These measures were: a 70-item safety culture scale (Frazier et al., 2013); the 50-item Nordic Occupational Safety Climate Questionnaire (Kines et al., 2011); a 22-item safety climate index (Hon et al., 2013); a 19-item measure of organizational policies and practices (Amick et al., 2000); and an 8-item measure of organizational safety performance (OPM: IWH, 2011, 2013). The five measures listed above incorporated most but not all elements (i.e., eight or more components) of the leading indicators construct thus performing reasonably well in terms of the first

296

T. Shea et al. / Safety Science 85 (2016) 293–304

Table 2 Summary of leading indicator scales. Review summary

N

%

Scale type Leading indicators of OHS OHS leadership Positive performance indicators Safety climate Safety culture Safety efficacy Safety management Safety obligations

2 4 1 26 3 1 4 1

4.8 9.5 2.4 61.9 7.1 2.4 9.5 2.4

19 34 25 12 20 19 26

45.2 81.0 59.5 28.6 47.6 45.2 61.9

13 13 9

31.0 31.0 21.4

Elements of leading indicators of OHSa OHS systems (policies, procedures, practices) Management commitment and leadership OHS training, interventions, information, tools and resources Workplace OHS inspections and audits Consultation and communication about OHS Prioritization of OHS OHS empowerment and employee involvement in decision making OHS accountability Positive feedback and recognition for OHS Risk management

a Note that, due to the varying number of elements of leading indicators of OHS addressed within each measure, the percentages do not add to 100.

evaluation criterion as they covered many of the 10 features of leading indicators of OHS, even if indirectly. All measures were generic in that they could be administered across industries, although three of the measures contained questions that were a combination of generic and employee centered questions (Frazier et al., 2013; Hon et al., 2013; Kines et al., 2011). The items of the measures developed by Amick et al. (2000) and IWH (2011, 2013) were written in a generic form. All measures contained items measured on a Likert type scale that could be summed or averaged to a single score. While the review revealed the scope of the leading indicators of OHS construct and how it could be measured, it is also important to consider the potential practical application of a scale, such as whether a scale is of a practical length for incorporating into surveys alongside other constructs. The measures developed by Frazier et al. (2013) and Kines et al. (2011) were extensive in length and therefore impractical for this purpose. An examination of the other three measures shows that the OPM was the shortest. The 8-item OPM, satisfied the initial review criteria and was a short, easy to administer measure of the leading indicators of OHS. 1.3. Organizational performance metric The OPM, developed at the Institute for Work & Health in Canada (IWH, 2011, 2013), was designed specifically to measure leading indicators of OHS. Research to date into the psychometric properties of the OPM in North America indicates that this scale is a reliable measure of leading indicators of OHS (IWH, 2011, 2013). The OPM is a generic measure that can be applied across workplace and industry contexts in order to benchmark and obtain a broad, comparable overview of OHS. Unlike most other measures in our review, the OPM is also a concise scale that is brief enough to potentially be used as a preliminary barometer of organizational safety and an initial ‘‘flag” for specific OHS issues that could subsequently be examined in more depth. As reported by IWH (2011), the items comprising the OPM were written such that any member of an organization could respond to them meaningfully, indicating that it can be used at multiple levels within an organization. This tool was designed to be used in organizations as an initial step, to be followed by a more in-depth analysis of each of the indicators of OHS performance. While the OPM was selected as the most

suitable exemplar for validation in this study, we acknowledge that there are likely to be some shortcomings in the measure and we sought to determine whether there was room for improvement. The OPM has some initial evidence of reliability and validity in North America (IWH, 2011, 2013) but a detailed psychometric investigation has yet to be published. The current study builds on the initial investigations in North America (IWH, 2011, 2013) by examining the OPM in the Australian context and applying the OPM at both the employee and workplace level. Leading indicators such as management commitment or OHS systems are expected to influence employee knowledge, skills, attitudes, and perceptions; in a positive sense, leading indicators are designed to keep the workplace safe and healthy. Some studies (e.g., Fernández-Muñiz et al., 2009) have focused on the objective features of safety management systems (i.e., the presence of policies or the frequency of organizational practices to improve safety) by surveying firms’ human resource (HR) managers or safety officers. In OHS management research, as in cognate areas such as human resource management, however, employee perceptions and behaviors are increasingly used to assess and gauge the impact of organizational practices (Wachter and Yorio, 2014). Indeed, recent research indicates that employee perceptions and experiences of HR systems may differ considerably from what is reported by their managers (Jiang et al., 2013; Liao et al., 2009). Purcell and Hutchinson (2007) suggested that it is the actual practices and the way they are implemented that employees perceive and react to, as they judge and respond to each in terms of utility or satisfaction. Employees are thus widely argued to be the most appropriate data source if the aim is to examine actual practices implemented in the organization (see for example Wright et al., 2003). There are also related concerns about the reliability of single ratings from organizations, taken for example from a single HR or OHS manager (Gerhart et al., 2000). Surveying multiple employees is one way to generate potentially more reliable data in assessing OHS indicators. To date, it appears that the OPM has only been validated using single informants (IWH, 2013). As noted above, a major goal of our study is to extend this validation to multiple informants at the employee level. Initial analysis to establish a relationship between the OPM and employee behaviors and employee level OHS outcomes is important; however, the OPM also needs to have demonstrated validity in terms of outcomes such as lost-time injury rates to be of practical value for organizations. An analysis at both the employee and workplace levels provides an adaptation and extensive validation of the original OPM and will consolidate its utility as a benchmarking tool within and between organizations. As discussed above, the aim of the current study was to determine whether the OPM is a reliable and valid measure of leading indicators of OHS. Specifically, we examine the OPM and adapt it to address identified limitations in the original version of the OPM. Then, we investigate the latent structure and psychometric properties of the adapted OPM using exploratory factor analysis and Rasch model analysis; and also examine the relationship between the adapted OPM and other measures of OHS and OHS outcomes at both employee and workplace levels. An evaluation at the workplace level extends our understanding from how the adapted OPM is associated with employee OHS outcomes to how it is associated with objective measures of OHS at the workplace level that might be regularly tracked within organizations.

2. Method 2.1. Sample The sample for this study was recruited via a national multiindustry survey conducted in Australia from September 2013 to

T. Shea et al. / Safety Science 85 (2016) 293–304

November 2014, using both pencil and paper and online methods of administration. Using our professional networks, we approached 20 OHS managers and invited their organizations to participate in the research. This resulted in six organizations being recruited into the study, with 66 workplaces (a single worksite of the organization) being sampled. Overall, the survey was made available to 10,362 employees and 3605 employees responded resulting in a response rate of 35%. The project was approved by the university’s Human Research Ethics Committee and all respondents were assured of confidentiality and anonymity. More than half of the respondents were male (60%). Respondents were classified as being employed in management positions (5%), supervisory positions (19%) or non-supervisory roles (76%) and most respondents were permanent employees (73%), rather than casual employees or contractors. Six industries were represented in the sample by workplaces from: arts and recreation (19%); construction (11%); electricity, gas, water and waste services (2%); healthcare and social assistance (26%), mining (17%) and transport, postal and warehousing (25%). 2.2. Measures The employee questionnaire that was used in this study included both perceptual measures of safety and self-reported OHS outcomes that were used to establish construct and criterion validity of the OPM. In completing the measures, respondents were asked to report in relation to the workplace they work in most often rather than the organization overall. Our literature review identified the OPM as a theoretically sound and practical measure of OHS leading indicators. However, inspection of the OPM led us to make adaptations to the scale. First, in order to adequately and consistently address OHS, we adapted some of the OPM items by replacing ‘safety’ with ‘health and safety.’ Second, initial piloting with the percentage response options revealed a substantial ceiling effect so we replaced the percentage response scale [0–20%, 21–40%, 41–60%, 61–80%, 81–100%] with a Likert scale. The adapted version of the OPM, hereafter referred to as the Organizational Performance Metric – Monash University (OPM-MU), and other perceptual measures were rated on fivepoint scales from strongly disagree (1) to strongly agree (5). The following section describes the additional perceptual measures and OHS incidents that were used to validate the OPM-MU. 2.2.1. Perceptual measures The perceptual measures included in the questionnaire were the Safety Climate, Safety Motivation, Safety Compliance and Safety Participation scales (Neal and Griffin, 2006); and Employee Safety Control scale (Huang et al., 2004). These measures were selected to enable an evaluation of how the OPM is associated with different elements of OHS, namely employee perceptions and behaviors, within the workplace. These five measures each contained three items. Each of these measures displayed very good reliability: Safety Climate (a = .94), Safety Motivation (a = .86), Safety Compliance (a = .90), Safety Participation (a = .86) and Safety Control (a = .79). 2.2.2. Self-reported OHS outcomes Self-reported OHS outcomes over the preceding 12 months were also used as a test of criterion validity. Specifically, respondents were asked to report the number of OHS incidents they had been personally involved in at work. These self-reported measures (sourced from Probst and Estrada, 2010) were: reported OHS incidents (OHS incidents that were reported to management); unreported OHS incidents (OHS incidents that were not reported to management); and near misses (situations that could have caused an injury/illness but did not).

297

2.2.3. Workplace OHS outcomes The OHS manager in each organization was contacted to collect the measures used at the workplace level. Measures included the number of: reported OHS incidents (occurrences of injury/disease reported to management by workers); reported near misses (unplanned incidents reported to management by workers that occurred at the workplace and although not resulting in any injury or disease, had the potential to do so); reported hazards (any activity, procedure, plant, process, substance, situation or any other circumstance that could cause, or contribute to causing, a major incident which has been reported by a worker to management). Our inclusion of multiple measures of OHS outcomes is consistent with recent research advising investigation of a range of lagging indicators (Christian et al., 2009; O’Neill et al., 2013). Measures collected at workplace level also included the number of: lost time injuries (occurrences that resulted in a fatality, permanent disability or time lost from work of one day/shift or more); medical-treatment injuries (occurrences which were not losttime injuries and for which medical treatment was administered excluding first aid treatment); and the number of hours worked in the preceding three month time period. Lost-time injury rates were calculated as the number of lost-time injuries divided by the total number of hours worked multiplied by 1,000,000. Similarly, medical-treatment injury rates were calculated as the number of medical-treatment injuries divided by the total number of hours worked multiplied by 1,000,000 (Worksafe Australia, 1990). 2.3. Procedure All employees at each participating workplace had the opportunity to complete the survey. The invitation to participate in and distribution of the survey depended on both workplace context and whether the questionnaire was administered using pencil and paper or online. Pencil and paper surveys were distributed at toolbox meetings or through the internal mail. In workplaces where employees gathered for toolbox meetings, provision was made for the researchers to attend meetings to distribute information sheets and surveys. Respondents completed the questionnaires at the toolbox meeting and returned them in a sealed envelope directly to the researchers. In other organizations the survey was distributed through the internal mail where respondents completed the survey in their own time and returned the questionnaires in a sealed envelope to a secure central collection point within their workplace. Where the questionnaire was administered online, the researchers provided the OHS manager with an email containing a link to the online survey; this email was sent by the OHS manager to employees (privacy legislation in Australia prevents employers from sharing employee emails with researchers). Respondents completing the surveys online or those who completed the pencil and paper surveys at their own pace were sent two reminders, via a global email or a newsletter sent by the OHS manager, two and four weeks after the initial invitation. To collect the workplace-level data, three months after the administration of the employee questionnaire a separate questionnaire was sent to the OHS manager in each of the six participating organizations. The OHS manager was asked to provide data sourced from the organizational records on several OHS outcomes for each of the participating workplaces for the preceding three months. 2.4. Statistical analysis The aim of this paper was to conduct a psychometric evaluation that includes exploratory factor analysis (EFA), Rasch model analyses and reliability analysis (Cronbach’s alpha) and validation of the OPM-MU with an approach that is largely based on methods

298

T. Shea et al. / Safety Science 85 (2016) 293–304

recommended by DeVellis (2012). This process includes an examination of the latent construct (i.e., leading indicators of OHS as measured by the OPM-MU), determination of scale reliability and an evaluation of scale validity. Each of these components of the evaluation process is discussed below. The dominant measurement paradigm over the past century has been classical test theory (e.g., EFA) but more recently item response theory (e.g. Rasch model analysis) has grown in importance (Embretson and Reise, 2000). Classical test theory is often used to develop new or evaluate existing scales and Rasch modelling has been used extensively in education, psychology, medicine and to a much lesser degree marketing. In other fields of research such as psychology and education the debate over the merits of both classical test and item response theories of measurement is ongoing (for example see Streiner, 2010). However, we do not seek to argue for the supremacy of either theory; our aim is to use both methods to enable us to capitalize on the strengths of both theories and use the differences in analysis to provide a more extensive evaluation of the OPM-MU. We used exploratory factor analysis because, no peer reviewed studies have been published to demonstrate the factor structure of the OPM-MU. Correlations with other perceptual measures of OHS and OHS incidents measured at the employee and workplace levels were also conducted to assess both construct and criterion validity of the OPM-MU. Preliminary analysis was conducted to assess the quality of the data, generate means, standard deviations and correlations amongst variables. Inspection of responses to the individual items of the OPM revealed 34 respondents had 25% or more missing data; these respondents were removed leaving 3571 cases for analysis. The sample was then divided into three subsets to conduct: exploratory factor analysis and reliability (n = 2571), Rasch model analysis (n = 500) and correlational analysis (n = 500). Each group was compared on basic demographic details and found to be roughly equivalent for: gender, level within the organization, employee status, industry, organization, mode of administration and the number of respondents experiencing OHS incidents. There were no significant differences on these variables between paperand-pencil and online modes of administration. 2.4.1. Exploratory factor analysis Exploratory factor analysis was conducted on the first subset of the sample using SPSS 21.0 (IBM Corp, 2012) with principal axis factoring as the extraction method and, as potential factors were expected to be correlated, an oblique rotation method (promax) was used. The number of factors retained was chosen with parallel analysis (Horn, 1965), Kaiser’s criterion and inspection of the scree plot. A good solution requires: a minimum of 50% explanatory variance; good communalities (ideally >.6); each factor should be represented by at least three items with each item displaying significant loadings (>.4) on one factor only; and ideally no more than five percent of the residual correlations should be greater than .05 (Tabachnick and Fidell, 2000). 2.4.2. Rasch model analysis The Rasch model analysis was conducted on the second subset of the sample using RUMM2030 software (Andrich et al., 2012) and model fit was evaluated using three measures of overall model fit provided in the RUMM2030 program. These measures are: the item–trait interaction statistic; item fit residual values; and person fit residual values. In good model fit the item–trait interaction statistic has a non-significant chi-square value and the fit residual values for respondents and items have a mean of zero and a standard deviation of one. The RUMM2030 program also provides a measure of internal consistency, the person separation index (PSI), that is measured on the same scale as Cronbach’s alpha.

If the item–trait interaction statistic is significant then model misfit can be investigated first by checking for out of range item or respondent fit statistics (i.e., fit residuals with a mean greater than 1.0 or standard deviation greater than 1.5) followed by an inspection of individual item and/or respondent fit residuals and their corresponding chi-square probability values. These individual fit residuals should not have an absolute value greater than 2.5. A fit residual value less than 2.5 indicates overfit and a fit residual value greater than +2.5 represents misfit that needs to be addressed. The individual chi-square probability values should also be non-significant (Bonferroni corrected). Item misfit indicates that the item deviates from its expected relationship with the other items and does not adequately represent the latent trait in question (Bhakta et al., 2005). Where misfitting respondents are identified this means that the respondent’s answers deviate from model expectations. An investigation of model fit using items or respondent fit statistics is just an initial step in the analysis; there are other aspects of the scale and the way in which study participants respond to the scale items that might impact on model fit and should also be investigated. These elements are: item thresholds (proper use of response options); dimensionality; local dependence; targeting; and differential item functioning (DIF). An examination of each issue will provide a greater understanding of the scale and its application in different research settings. As readers may not be familiar with Rasch analysis, the following is a brief summary of each of the additional elements of the Rasch process that can be investigated using the RUMM2030 program. Checking for disordered thresholds is an important step beyond item fit statistics when assessing item fit as disordered thresholds can result from inconsistent responding or using response terms that are semantically similar (e.g., often, frequently). Differential item functioning will also impact on model fit and occurs when respondents from different subgroups within the sample exhibit the same level of the measured trait but respond differently across some items. Local dependence, which may arise from response dependence or multidimensionality, occurs where the response to one item is dependent on the response to another. In RUMM2030 an analysis of residual correlations will test for response dependency (absolute values >.30) and a principal components analysis of the residuals can be used to investigate multidimensionality. The items will load either positively or negatively on the first residual factor forming two subsets of items. Multidimensionality is indicated when the number of cases with significantly different scores on each subset of items exceed 5%. If the number is greater than 5% then a confidence interval is applied and if the confidence interval contains the value .05 then the scale can be considered unidimensional. Finally, to investigate whether the scale is well targeted RUMM2030 provides a person-item threshold map, a visual display of the respondents and items calibrated to the Rasch continuum. The ideal would be a good spread of items across the range of scores, which indicates that a scale is well targeted. Gaps along the continuum would indicate levels of the latent trait that are not targeted by the scale. More detailed explanations of Rasch analysis in general, specific fit statistics and the processes outlined above can be found elsewhere (Bond and Fox, 2007; Pallant and Tennant, 2007; Salzberger and Koller, 2013). 2.4.3. Construct and criterion validity The validation of a scale, ideally, requires evidence of several types of validity including construct and criterion validity (AERA, 1999). Validity is specified on the basis of pre-determined theoretical relationships and evaluated from the pattern of correlations that emerges between the scale being examined and measures of theoretically related and unrelated constructs (AERA, 1999;

299

T. Shea et al. / Safety Science 85 (2016) 293–304

DeVellis, 2012). Typically, construct validity refers to a scale’s relationship to other measures; that is, the degree to which it can be associated with other related (convergent) or differentiated from unrelated (discriminant) measures. Criterion validity generally refers to the associations between the scale being evaluated (i.e., OPM-MU) and other criterion that are expected to be related (e.g., OHS incidents), where the measurement of the scale score may be concurrent with (i.e., concurrent validity) or precede (i.e., predictive validity) the criterion of interest (DeVellis, 2012). In this study, correlational analysis was conducted on the third subset of the sample using Spearman’s rho (rs), due to the skewed distribution of OHS outcome variables at both employee and workplace levels. Construct validity was evaluated by observing the pattern of correlations between the OPM-MU and other measures of OHS. As a measure of leading indicators of OHS, the OPM-MU should correlate more strongly with perceptions or behaviors that precede harm such as complying with safety rules (Safety Compliance), taking a proactive stance on creating a safer workplace (Safety Participation) and the experience of control over one’s safety at work (Safety Control). The OPM-MU was expected to correlate less strongly with a scale that measures intrinsic employee traits (Safety Motivation). Concurrent validity was examined by correlating respondent scores on the OPM-MU with self-reported OHS outcomes (reported incidents, unreported incidents and near misses). Predictive validity was examined by aggregating OPM-MU scores to the workplace level and correlating workplace level OPM-MU scores to workplace OHS incidents (reported incidents, reported hazards, reported near misses, lost time injury frequency rates and medical-treatment injury frequency rates). 3. Results 3.1. Exploratory factor analysis Exploratory factor analysis was conducted with subset one using principal axis factoring. The KMO statistic was .93 and Bartlett’s Test of Sphericity was significant indicating that the data were suitable for factor analysis and each item was significantly correlated above .3 with all other items (p < .001). Initial inspection of the results showed that the eigenvalues and scree plot suggested a one-factor structure that was supported by the parallel analysis. The pattern matrix in Table 3 shows a clean one-factor structure with all items loading significantly on one factor and explaining 50% of the variance in the underlying construct. Reliability analysis using Cronbach’s alpha showed that the OPMMU had excellent reliability (a = .88). 3.2. Rasch analysis The eight items of the OPM-MU were also subjected to Rasch analysis using a second subset of the sample. This analysis revealed good overall model fit, the item–trait interaction statistic was not significant (X2 = 62.0, df = 56, p = .27). The item fit statistics (M = 0.2, SD = 1.6) were slightly out of range indicating some item misfit and the respondent fit statistics (M = 0.5, SD = 1.4) were within range. An inspection of the threshold map showed that the item everyone at this workplace values ongoing OHS improvement in this workplace had a disordered threshold. Examination of the category probability curve for this item revealed that respondents were not using the middle response category (value three) in the predicted way; however, given that overall model fit was good, the original scoring was retained. The respondent separation index was .87 which was roughly equivalent to Cronbach’s alpha obtained from the first stage of the analysis

Table 3 Pattern matrix for the OPM-MU. Items

Loadings

Workers and supervisors have the information they need to work safely Employees are always involved in decisions affecting their health and safety This workplace considers OHS at least as important as production and quality in the way work is done Those in charge of OHS have the authority to make the changes they have identified as necessary Everyone has the tools and/or equipment they need to complete their work safely Everyone at this workplace values ongoing OHS improvement in this workplace Those who act safely receive positive recognition Formal OHS audits at regular intervals are a normal part of our workplace

0.77 0.77 0.72 0.71 0.69 0.69 0.66 0.63

(a = .88). The two most diverse subsets of items were identified in a principal components analysis (set 1 – items 1, 2, 3; set two – items 4, 5, 6,7, 8); independent t-tests revealed that 6.8% of cases within the sample had significantly different scores for these subsets of items, however, suggesting that the scale could be multidimensional. A 95% confidence interval was calculated for these ttests and which ranged from .05 to .09. As this range contains the value .05 we can conclude that this subscale is unidimensional. There were no residual correlations at or above .3 indicating that there was no response dependence among items or redundancy in the scale. Table 4 displays the individual item fit statistics for the OPMMU. The location scores reveal that the item workers and supervisors have the information they need to work safely was the easiest item for the sample to endorse. The item those who act safely receive positive recognition was the hardest item for the respondents in the sample to endorse. The person-item threshold distribution shown in Fig. 1 shows the respondents and items calibrated to the same scale where respondent scores are shown in the upper panel (person distribution) and the items are represented in the lower portion (item distribution). Respondents who report lower levels of leading indicators of OHS in their workplace cluster to the left of the graph and respondents who report higher levels of leading indicators of OHS in their workplace cluster to the right. The map revealed coverage of items across lower and middle scores in the sample, but there were no items to cover highest levels of workplace performance on leading indicators of safety in the sample. The mean estimate of the latent construct (leading indicators) was 0.98 (SD = 1.59). In Rasch analysis mean location scores for both respondents and items are set to zero and a mean estimate that is close to zero means that a scale is reasonably well targeted to the sample, or more specifically, the items are not too easy or too hard to endorse. In this instance a score that is a little above zero shows that the scale is reasonably well targeted to the sample. Differential item functioning (DIF) was investigated for five variables representing respondent and workplace characteristics. This was done to determine the impact of these variables on the way the items were answered. DIF was investigated for the following characteristics: level in organization (manager, supervisor, employee), employee status (permanent, contingent), gender (male, female), whether the respondent had experienced an OHS incident in the past 12 months (no incidents, incidents) and survey administration method (paper and pencil with group administration, paper and pencil with individual administration and online). No differential item functioning was observed on any items for level in organization, employee status or experience of OHS incidents. Differential item functioning was observed on one item for

300

T. Shea et al. / Safety Science 85 (2016) 293–304

Table 4 Individual item fit statistics for the OPM-MU. Items

Locat

Formal OHS audits at regular intervals are a normal part of our workplace Everyone at this workplace values ongoing OHS improvement in this workplace This workplace considers OHS at least as important as production and quality in the way work is done Workers and supervisors have the information they need to work safely Employees are always involved in decisions affecting their health and safety Those in charge of OHS have the authority to make the changes they have identified as necessary Those who act safely receive positive recognition Everyone has the tools and/or equipment they need to complete their work safely

0.05 0.56 0.10 0.57 0.41 0.12 0.63 0.25

SE 0.06 0.07 0.06 0.07 0.06 0.07 0.06 0.06

FitRes (df) 1.70 1.58 1.36 2.17 1.81 0.57 0.96 0.77

(420.32) (422.05) (419.45) (421.18) (421.18) (421.18) (420.32) (420.32)

ChiSq (df)

Prob

7.99 4.33 9.93 9.29 10.59 10.61 3.65 5.65

0.33 0.74 0.19 0.23 0.16 0.16 0.82 0.58

(7) (7) (7) (7) (7) (7) (7)

Locat = Rasch location, SE = Standard error, FitRes = Fit Residual, ChiSq = Chi square.

gender: employees are always involved in decisions affecting their health and safety. Male respondents were more likely to endorse this item compared to female respondents. Differential item functioning was observed for three items on mode of administration. For the item this workplace considers OHS at least as important as production and quality in the way work is done, respondents who filled in the paper and pencil survey in a group tended to rate this item slightly lower than those who completed the questionnaire online or individually. For the item, employees are always involved in decisions affecting their health and safety, respondents who filled in the paper and pencil survey individually tended to rate this item slightly lower than those who completed the questionnaire online or in a group. Finally for the item, those who act safely receive positive recognition, respondents who filled in the paper and pencil survey in a group tended to rate this item slightly higher than those who completed the questionnaire online or individually. Despite these differences on the basis of mode of administration, it could be expected that the differential item functioning would cancel out at the scale level. 3.3. Construct validity Finally, correlations between scores on the OPM-MU, scores for other perceptual measures and self-reported OHS incidents were conducted with subset three to assess the construct validity and of the OPM-MU. These correlations are displayed in Table 5. There were statistically significant positive correlations between the OPM-MU and all other perceptual measures. The moderate correlations between the OPM-MU and measures of

employee safety behaviors (compliance, participation, and control) indicate that the OPM-MU is associated with employee behaviors, but can be distinguished from another element of safety intrinsic to the employee (safety motivation), which correlated more weakly with the OPM-MU compared to the other measures.

3.4. Criterion validity Correlations between the OPM-MU and self-reported OHS incidents varied across specific types of incidents (reported, not reported and near misses). There was a negative correlation between scores on the OPM-MU and the number of OHS incidents that respondents experienced but did not report to management (rs = .18, p < .001). There was also a negative correlation between scores on the OPM-MU and near misses (rs = .23, p < .001). Respondents who obtained higher scores on the OPM-MU were less likely to be involved in either OHS incidents that they did not report to management or near misses. There was no statistically significant correlation between scores on the OPM-MU and incidents that were reported to management (rs = .07, p > .05). Finally, we examined the predictive validity of the OPM-MU by examining its relationships with OHS outcomes collected at the workplace level, using a three month time lag. We first calculated rwg, ICC(1), and ICC(2) values to ascertain whether employee ratings of the OPM-MU were adequate for aggregation to the workplace level. The mean multi-item rwg (calculated using a uniform null distribution) was .96, indicating a high level of within-group agreement (Bliese, 2000). The inter-rater reliability values of ICC

Fig. 1. Person-item threshold map for the OPM-MU.

T. Shea et al. / Safety Science 85 (2016) 293–304 Table 5 Correlations between OPM-MU and other perceptual measures.

OPM-MU Safety motivation Safety compliance Safety participation Safety control

OPM-MU

Motivation

Compliance

Participation

.38 .51 .45 .51

.65 .50 .60

.61 .67

.68

Note: All correlations p < .001.

(1) and ICC(2) were .14 and .97, respectively. Taken together, these results provide strong support for aggregation of the OPM-MU to the workplace level. Correlational analyses revealed negative associations between aggregated workplace scores on the OPM-MU at baseline and both LTIFR (rs = .30, p < .05) and MTIFR (rs = .38, p < .01) three months later. There were no statistically significant correlations between aggregated workplace scores on the OPM-MU at baseline and reported incidents (rs = .06, p > 05), reported near misses (rs = .02, p > .05) or reported hazards (rs = .02, p > .05) three months later. 4. Discussion Leading indicators of OHS are key to a proactive approach to OHS and the measurement and monitoring of OHS performance. Despite the apparent value of leading indicators, there has been very little development of academic research that focuses on the measurement of OHS leading indicators (Sinelnikov et al., 2015). Our review of measures in this field indicated that the OPM was the only scale of those identified that not only directly addressed the leading indicators construct, but also met practical criteria such as being an easy to administer measure that can be applied across different industries and at different levels of analysis (e.g., employee and workplace-level). With our amendments, we found that the OPM-MU is a measure of OHS leading indicators that is both theoretically sound and practical to administer. We conducted an evaluation of the OPM-MU through a rigorous psychometric testing process that aimed to examine the latent structure and psychometric properties of the OPM-MU, using exploratory factor and Rasch model analyses. The results of both the EFA and the Rasch analysis revealed good model fit and demonstrated that the OPM-MU is a unidimensional scale with good reliability. This study has also demonstrated that the OPMMU is a valid tool to use with multiple informants to measure employee perceptions of leading indicators in their workplace. This responds to recent calls for OHS researchers and practitioners to pay attention to employee perceptions and behaviors to assess and gauge the impact of organizational practices (Wachter and Yorio, 2014). The OPM-MU showed good inter-rater reliability and high levels of within-group agreement, providing strong support for aggregation to the workplace level. Hence, an advantage of the OPM-MU is that it can be used to compare the presence of leading indicators of workplace OHS both within and across organizations. This finding directly addresses Sinelnikov et al.’s (2015) call for the development and validation of an index of OHS leading indicators that could be used for benchmarking purposes. Our study has also responded to suggestions by previous scholars (e.g., Christian et al., 2009; O’Neill et al., 2013), to investigate the different relationships among OHS leading indicators and a range of lagging indicators. Hence, we also sought to test the validity of the OPM-MU by evaluating its relationship with other measures of OHS at both the employee and workplace level. The OPMMU was more strongly associated with measures of employee behaviors such as safety compliance, safety participation, and

301

safety control compared to employee perceptions such as their safety motivation. This pattern is consistent with expectations as the OPM-MU measures practices and behaviors and therefore stronger associations should be found with other measures of safety behavior, compared with measures of internal motivational states (i.e., safety motivation). This finding raises implications that could be explored in future research to build understanding of the varying relationships between OHS leading indicators and employee perceptions and behaviors (Wachter and Yorio, 2014). The OPM-MU was also associated with OHS outcomes as reported by employees but the pattern of correlations varied by incident type. Higher scores on the OPM-MU were correlated with fewer incidents that were not reported to management and fewer near misses; however, there was no evidence of a relationship between scores on the OPM-MU and incidents that were reported to management. This pattern may arise not only because reported OHS incidents are relatively rare events but also because the OPM-MU, as a leading indicator of OHS, might be reflective of an open and transparent safety climate. This interpretation is supported by the stronger negative associations with incidents that are not reported and near misses. Aggregated OPM-MU scores were associated with OHS incidents collected at the workplace level using a three-month time lag. Specifically, aggregated OPM-MU scores were associated with the lost-time and medical-treatment injury rates although not reported incidents, reported hazards or reported near-misses. In terms of practical significance, the statistically significant correlations of the OPM-MU with self-reported OHS outcomes (the number of OHS incidents that respondents experienced but did not report to management and self-reported near misses) are about double the point estimate between safety climate and selfreported accidents/injuries reported by Christian et al. (2009) in their meta-analysis (mean uncorrected r = .21 vs. .10). The correlations of the aggregated OPM-MU scores with LTIFR and MTIFR were also larger than those of the OPM-MU and self-reported OHS outcomes and similar in magnitude to those found between grouplevel safety climate and accidents/injuries (Christian et al., 2009). Overall, our findings support the predictive validity of the OPMMU and demonstrate the utility of the OPM-MU as a short and practical measure of leading indicators of OHS. In sum, this analysis shows that, with our adaptations, the OPM-MU performs well across several industries that vary in terms of the type of work performed and risk potential within the workplace. In practical terms, as a short measure of leading indicators of OHS, the OPM-MU can be used as part of an initial investigation of OHS within an organization as part of an employee attitude survey. Where necessary, this initial evaluation could be supplemented with a more detailed questionnaire that investigates more specific elements of OHS unique to the organization or industry. We acknowledge limitations of the study. First, the OPM-MU is a high-level, initial flag that can provide some information on the quality of OHS management systems within an organization. Where necessary, other measures (e.g., risk controls, quality controls) should also be considered so the OPM-MU can be part of a suite of tools that gives a comprehensive or more in-depth understanding of occupational health and safety. Second, we also acknowledge that self-reports are subject to bias. However, we have attempted to ameliorate the potential for any common methods bias with the inclusion of data collected from two different sources: OHS outcomes reported by OHS managers and employee OPM-MU scores aggregated to the workplace level. In interpreting our findings, it is also important to note that Christian et al. (2009) found little evidence of inflationary common methods bias for correlations of safety climate with self-reported accidents/injuries. If any bias does exist, their findings suggest in fact that self-reports of OHS incidents may slightly underestimate relationships with safety climate and related constructs.

302

T. Shea et al. / Safety Science 85 (2016) 293–304

Third, given the limited number of industries captured in this study, future work is needed in a wider range of industries to test the capacity of the OPM-MU as a benchmarking tool across all industrial contexts. Our study was also limited to medium-tolarge organizations; future research could investigate whether the OPM-MU is applicable in small business, which may have more informal OHS practices and procedures. Fourth, further testing of the OPM-MU in other countries would extend the validation of the OPM-MU beyond the Western context and add to our understanding of its uses and its benchmarking capacity. While we have delivered a rigorous validation study, testing the OPM-MU across multiple contexts such as industry, workplace size and cultural settings would add evidence to the OPM-MU’s validity and strengthen its utility as a benchmarking tool. Future work could also develop recommendations for the implementation of OHS leading indicators and the ongoing measurement of them in workplaces, which would rely on planning, education, connection with existing processes and practices, and implementation of a regular evaluation process (Hallowell et al., 2013; Sinelnikov et al., 2015).

original OPM as a promising new scale that provides parsimonious coverage of the characteristics of the leading indicators construct. Following several adaptations, we conducted a detailed psychometric analysis of the OPM-MU. The findings of this research show that the OPM-MU is a reliable and valid measure that could be used as an initial ‘flag’ of the leading indicators of OHS performance. The OPM-MU is a brief and easily administered tool that has demonstrated psychometric properties for use in workplaces and has the potential to be a benchmarking tool both within and across organizations.

Acknowledgements We acknowledge the support provided for this research project by the WorkSafe Victoria and the Institute for Safety Compensation and Recovery Research, Australia. We acknowledge Dr Benjamin Amick III, Institute for Work & Health, Ontario Canada, for his comments on the development of this research.

5. Conclusion Appendix A Following a comprehensive comparison of existing measures of OHS leading indicators reported in the literature, we identified the

The scales reviewed in this study can be found in the list below.

Authors

Scale name

Scale type

Abdullah et al. (2009) Amick et al. (2000) Bahari and Clarke (2013) Brondino et al. (2013) Chen and Chen (2012) Cheyne et al. (2003) Clarke (2006a) Cox and Cheyne (2000) Cui et al. (2013) DeJoy et al. (2004) Dı´az and Cabrera (1997) Fernández-Muñiz et al. (2009) Frazier et al. (2013) Gittleman et al. (2010) Glendon and Litherland (2001) Grabowski et al. (2007) Griffin and Neal (2000) Grote and Kunzler (2000) Hahn and Murphy (2008) Hon et al. (2013) Huang et al. (2006) Huang et al. (2012) IWH (2013) Keren et al. (2009) Kines et al. (2011) Mitchell (2000) Nja and Fjelltun (2010) Payne et al. (2009) Prussia et al. (2003) Rundmo (2000) Seo et al. (2004) Silva et al. (2004) Tang et al. (2011) Vinodkumar and Bhasi (2010) Vredenburgh (2002) Walker (2010)

Safety climate assessment scale Organizational policies & practices (OPP-19) Safety climate scale Integrated organizational safety climate Safety management system Attitudes to safety Safety climate Safety climate assessment tool Management commitment to safety Organizational safety climate Organizational safety climate Safety management system Safety culture Safety climate Safety climate Perceived safety Safety climate Operational safety Safety climate Safety climate index Safety climate Management commitment to safety Organizational performance metric Safety climate Nordic safety climate questionnaire Positive performance indicators Manager attitudes to health, environment & safety Process safety climate Safety efficacy Safety climate Safety climate OSCI: Safety climate questionnaire (4 scales) Organizational policies & practices (OPP-11) Safety management practices Management safety practices Employer obligations scale

Safety climate Safety climate Safety climate Safety climate Safety management Safety climate Safety climate Safety climate Safety climate Safety climate Safety climate Safety management Safety culture Safety climate Safety climate Leading indicators Safety climate Safety culture Safety climate Safety climate Safety climate Safety climate Leading indicators Safety climate Safety climate Positive performance indicators Safety management Safety climate Safety efficacy Safety climate Safety climate Safety climate Safety climate Safety management Safety culture Safety obligations

303

T. Shea et al. / Safety Science 85 (2016) 293–304 Appendix A (continued)

Authors

Scale name

Scale type

Williams et al. (2007) Wu (2008) Wu et al. (2008) Wu et al. (2010) Wu et al. (2010) Zohar and Luria (2005)

Organizational policies & practices (OPP-52) Safety leadership Safety leadership Operations manager safety leadership Employer safety leadership Safety climate

Safety climate OHS leadership OHS leadership OHS leadership OHS leadership Safety climate

References Abdullah, N.A., Spickett, J.T., Rumchev, K.B., Dhaliwal, S.S., 2009. Validity and reliability of the safety climate measurement in Malaysia. Int. Rev. Bus. Res. Pap. 5 (3), 111–141. American Educational Research Association, 1999. Standards for Educational and Psychological Testing. American Educational Research Association, Washington, DC. Amick III, B.C., Habeck, R.V., Hunt, A., Fossel, A.H., Chapin, A., Keller, R.B., Katz, J.N., 2000. Measuring the impact of organizational behaviors on work disability prevention and management. J. Occup. Rehabil. 10 (1), 21–38. Andrich, D., Sheridan, B., Luo, G., 2012. RUMM 2030. RUMM Laboratory, Perth. Bahari, S.F., Clarke, S., 2013. Cross-validation of an employee safety climate model in Malaysia. J. Saf. Res. 45, 1–6. Baker, J., Bowman, F., Erwin, G., Gorton, S., Hendershot, D., Leveson, N., Priest, S., Rosenthal, I., Tebo, P., Weigmann, D., Wilson, L.D., 2007. The Report of the BP US Refineries Independent Safety Review Panel. Retrieved April 2012 from: . Battaglia, M., Bianchi, L., Frey, M., Passetti, E., 2015. Sustainability reporting and corporate identity: action research evidence in an Italian retailing cooperative. Bus. Ethics: Eur. Rev. 24, 52–72. Bhakta, B., Tennant, A., Horton, M., Lawton, G., Andrich, D., 2005. Using item response theory to explore the psychometric properties of extended matching questions examination in undergraduate medical education. BMC Medical Education, 5, 9. Retrieved from: . Blair, E., O’Toole, M., 2010. Leading measures: enhancing safety climate & driving safety performance. Prof. Saf. J. 8 (1), 29–34. Bliese, P.D., 2000. Within-group agreement, non-independance and reliability: implications for data aggregation and analysis. In: Klein, K.J., Kozlowski, S.W.J. (Eds.), Multilevel Theory, Research, and Organizational Theory: Foundations, Extensions, and New Directions. Jossey-Bass, San Francisco, pp. 349–381. Bond, T.G., Fox, C.M., 2007. Applying the Rasch Model: Fundamental Measurement in the Human Sciences. Lawrence Erlbaum Associates, Mahwah, NJ. Brondino, M., Pasini, M., da Silva, S., 2013. Development and validation of an Integrated Organizational Safety Climate Questionnaire with multilevel confirmatory factor analysis. Qual. Quant. 47, 2191–2223. Carson, P.A., Snowden, D., 2010. Health, safety and environment metrics in loss prevention – part 1. Loss Prev. Bull. 212, 11–15. Chen, C.-F., Chen, S.-F., 2012. Scale development of safety management system evaluation for the airline industry. Accid. Anal. Prev. 47, 177–181. Cheyne, A., Tomás, J.M., Cox, S., Oliver, A., 2003. Perceptions of safety climate at different employment levels. Work Stress 17, 21–37. Choudhry, R.M., Fang, D., Mohamed, S., 2007. The nature of safety culture: a survey of the state-of-the-art. Saf. Sci. 45, 993–1012. Christian, M.S., Bradley, J.C., Wallace, J.C., Burke, M.J., 2009. Workplace safety: a meta-analysis of the roles of person and situation factors. J. Appl. Psychol. 94, 1103–1127. Clarke, S., 2006a. Safety climate in an automobile manufacturing plant. Pers. Rev. 35, 413–430. Clarke, S., 2006b. The relationship between safety climate and safety performance: a meta-analytic review. J. Occup. Health Psychol. 11, 315–327. Clarke, S., 2013. Safety leadership: a meta-analytic review of transformational and transactional leadership styles as antecedents of safety behaviours. J. Occup. Org. Psychol. 86, 22–49. Cox, S.J., Cheyne, A.J.T., 2000. Assessing safety culture in offshore environments. Saf. Sci. 34, 111–129. Cui, L., Fan, D., Fu, G., Zhu, C.J., 2013. An integrative model of organizational safety behavior. J. Saf. Res. 45, 37–46. Daniels, C., Marlow, P., 2005. Literature Review on the Reporting of Workplace Injury Trends. Retrieved June 2012 from: . DeJoy, D.M., Schaffer, B., Wilson, M., et al., 2004. Creating safer workplaces: assessing the determinants and role of safety climate. J. Saf. Res. 35, 81–90. DeVellis, R.F., 2012. Scale Development: Theory and Applications, third ed. SAGE Publications. Dı´az, R., Cabrera, D., 1997. Safety climate and attitude as evaluation measures of organizational safety. Accid. Anal. Prev. 29, 643–650. Dyreborg, J., 2009. The causal relation between lead and lag indicators. Saf. Sci. 47, 474–475. Embretson, S.E., Reise, S.P., 2000. Item Response Theory for Psychologists. Lawrence Erlbaum Associates, Mahwah, New Jersey.

Erikson, S.G., 2009. Performance indicators. Saf. Sci. 47, 468. Fernández-Muñiz, B., Montes-Peón, J.M., Vázquez-Ordás, C.J., 2009. Relation between occupational safety management and firm performance. Saf. Sci. 47, 980–991. Flin, R., Mearns, K., O’Connor, P., Bryden, R., 2000. Measuring safety climate: identifying the common features. Saf. Sci. 34, 177–192. Frazier, C.B., Ludwig, T.D., Whitaker, B., Roberts, D.S., 2013. A hierarchical factor analysis of a safety culture survey. J. Saf. Res. 45, 15–28. Geldart, S., Smith, C.A., Shannon, H.S., Lohfeld, L., 2010. Organizational practices and workplace health and safety: a cross-sectional study in manufacturing companies. Saf. Sci. 48, 562–569. Gerhart, B., Wright, P.M., McMahan, G.C., 2000. Measurement error in research on the human resources and firm performance relationship: further evidence and analysis. Pers. Psychol. 53, 855–872. Gittleman, J.L., Gardner, P.C., Haile, E., Sampson, J.M., Cigularov, K.P., Ermann, E.D., Chen, P.Y., 2010. CityCenter and Cosmopolitan Construction Projects, Las Vegas, Nevada: lessons learned from the use of multiple sources and mixed methods in a safety needs assessment. J. Saf. Res. 41, 263–281. Glendon, A.I., Litherland, D.K., 2001. Safety climate factors, group differences and safety behavior in road construction. Saf. Sci. 39, 157–188. Grabowski, M., Ayyalasomayajula, P., Merrick, J., Harrald, J.R., Roberts, K., 2007. Leading indicators of safety in virtual organizations. Saf. Sci. 45, 1013–1043. Griffin, M.A., Neal, A., 2000. Perceptions of safety at work: a framework for linking safety climate to safety performance, knowledge, and motivation. J. Occup. Health Psychol. 5, 347–358. Grote, G., Kunzler, C., 2000. Diagnosis of safety culture in safety management audits. Saf. Sci. 34, 131–150. Hahn, S.E., Murphy, L.R., 2008. A short scale for measuring safety climate. Saf. Sci. 46, 1047–1066. Hallowell, M., Hinze, J., Baud, K., Wehle, A., 2013. Proactive construction safety control: measuring, monitoring, and responding to safety leading indicators. J. Constr. Eng. Manage. 139, 04013010. Health and Safety Executive (HSE), 2005. Development and validation of the HMRI safety culture inspection toolkit. HSE Research Report 365. HSE Books, London. Retrieved July 2014 from: . Hinze, J., Thurman, S., Wehle, A., 2013. Leading indicators of construction safety performance. Saf. Sci. 51, 23–28. Hon, C., Chan, A., Yam, M., 2013. Determining safety climate factors in the repair, maintenance, minor alteration, and addition sector of Hong Kong. J. Constr. Eng. Manage. 139, 519–528. Hopkins, A., 2009. Thinking about process safety indicators. Saf. Sci. 47, 460–465. Horn, J.L., 1965. A rationale and test for the number of factors in factor analysis. Psychometrika 30, 179–185. Huang, Y.-H., Chen, P.Y., Krauss, A.D., Rogers, D.A., 2004. Quality of the execution of corporate safety policies and employee safety outcomes: assessing the moderating role of supervisor safety support and the mediating role of employee safety control. J. Bus. Psychol. 18, 483–506. Huang, Y.-H., Ho, M., Smith, G.G., Chen, P.Y., 2006. Safety climate and self-reported injury: assessing the mediating role of employee safety control. Accid. Anal. Prev. 38, 425–433. Huang, Y.-H., Verma, S.K., Chang, W.-R., Courtney, T.K., Lombardi, D.A., Brennan, M. J., Perry, M.J., 2012. Supervisor vs. employee safety perceptions and association with future injury in US limited-service restaurant workers. Accid. Anal. Prev. 47, 45–51. IBM Corp., 2012. IBM SPSS Statistics for Windows, Version 21.0. IBM Corp., Armonk, NY. IWH, 2011. Benchmarking Organizational Leading Indicators for the Prevention and Management of Injuries and Illnesses: Final Report. Institute for Work & Health, Toronto, Ontario. IWH, 2013. Developing Leading Indicators of Work Injury and Illness. Institute for Work & Health, Toronto, Ontario. Jiang, K., Takeuchi, R., Lepak, D.P., 2013. Where do we go from here? New perspectives on the black box in strategic human resource management research. J. Manage. Stud. 50, 1448–1480. Keren, N., Mills, T.R., Freeman, S.A., Shelley III, M.C., 2009. Can level of safety climate predict level of orientation toward safety in a decision making task? Saf. Sci. 47, 1312–1323. Kines, P., Lappalainen, J., Mikkelsen, K.L., Olsen, E., Pousette, A., Tharaldsen, J., Tómasson, K., Törner, M., 2011. Nordic Safety Climate Questionnaire (NOSACQ50): a new tool for diagnosing occupational safety climate. Int. J. Ind. Ergon. 41, 634–646. Kjellén, U., 2009. The safety measurement problem revisited. Saf. Sci. 47, 486–489.

304

T. Shea et al. / Safety Science 85 (2016) 293–304

Lebeau, M., Duguay, P., Boucher, A., 2014. Costs of occupational injuries and diseases in Quebec. J. Saf. Res. 50, 89–98. Liao, H., Toya, K., Lepak, D.P., Hong, Y., 2009. Do they see eye to eye? Management and employee perspectives of high-performance work systems and influence processes on service quality. J. Appl. Psychol. 94, 371–391. Lingard, H., Wakefield, R., Cashin, P., 2011. The development and testing of a hierarchical measure of project OHS performance. Eng. Constr. Archit. Manage. 18, 30–49. Mariscal, M.A., Herrero, S.G., Toca Otero, A., 2012. Assessing safety culture in the Spanish nuclear industry through the use of working groups. Saf. Sci. 50, 1237– 1246. Mitchell, R., 2000. Development of PPIs to monitor OHS performance in the Australian construction industry. J. Occup. Health Saf. 15, 325–331. Nahrgang, J.D., Morgeson, F.P., Hofmann, D.A., 2011. Safety at work: a meta-analytic investigation of the link between job demands, job resources, burnout, engagement, and safety outcomes. J. Appl. Psychol. 96, 71–94. Neal, A., Griffin, M.A., 2006. A study of the lagged relationships among safety climate, safety motivation, safety behavior, and accidents at the individual and group levels. J. Appl. Psychol. 91, 946–953. Nja, O., Fjelltun, S.H., 2010. Managers’ attitudes towards safety measures in the commercial transport sector. Saf. Sci. 48, 1073–1080. O’Neill, S., Martinov-Bennie, N., Cheung, A., 2013. Issues in the measurement and reporting of work health and safety performance: a review. Macquarie Lighthouse Press, Sydney, NSW, Australia. Pallant, J., Tennant, A., 2007. An introduction to the Rasch measurement model: an example using the Hospital Anxiety and Depression Scale HADS. Br. J. Clin. Psychol. 46, 1–18. Payne, S.C., Bergman, M.E., Beus, J.M., Rodríguez, J.M., Henning, J.B., 2009. Safety climate: leading or lagging indicator of safety outcomes? J. Loss Prev. Process Ind. 22, 735–739. Pidgeon, N.F., 1991. Safety culture and risk management in organizations. J. Cross Cult. Psychol. 22 (1), 129–140. Probst, T.M., Estrada, A.X., 2010. Accident under-reporting among employees: testing the moderating influence of psychological safety climate and supervisor enforcement of safety practices. Accid. Anal. Prev. 42, 1438–1444. Prussia, G.E., Brown, K.A., Willis, P.G., 2003. Mental models of safety: do managers and employees see eye to eye? J. Saf. Res. 34, 143–156. Purcell, J., Hutchinson, S., 2007. Front-line managers as agents in the HRMperformance causal chain: theory, analysis and evidence. Human Res. Manage. J. 17, 3–20. Reiman, T., Pietikäinen, E., 2012. Leading indicators of system safety – monitoring and driving the organizational safety potential. Saf. Sci. 50, 1993–2000. Rundmo, T., 2000. Safety climate, attitudes and risk perception in Norsk Hydro. Saf. Sci. 34, 47–59. Salzberger, T., Koller, M., 2013. Towards a new paradigm of measurement in marketing. J. Bus. Res. 66, 1307–1317. Seo, D.C., Torabi, M.R., Blair, E.H., Ellis, N.T., 2004. A cross-validation of safety climate scale using confirmatory factor analytic approach. J. Saf. Res. 35, 427– 445.

Silva, S., Lima, M.L., Baptista, C., 2004. OSCI: an organisational and safety climate inventory. Saf. Sci. 42, 205–220. Sinelnikov, S., Inouye, J., Kerper, S., 2015. Using leading indicators to measure occupational health and safety performance. Saf. Sci. 72, 240–248. Streiner, D.L., 2010. Measure for measure: new developments in measurement and item response theory. Can. J. Psychiatry 55, 180–186. Tabachnick, B.G., Fidell, L.S., 2000. Using Multivariate Statistics. Allyn and Bacon, Boston, MA. Tang, K., MacDermid, J.C., Amick III, B.C., Beaton, D.E., 2011. The 11-item workplace Organizational Policies and Practices Questionnaire (OPP-11): examination of its construct validity, factor structure, and predictive validity in injured workers with upper-limb disorders. Am. J. Ind. Med. 54, 834–846. Van Dyck, C., Dimitrova, N.G., de Korne, D.F., Hiddema, F., 2013. Walk the talk: leaders’ enacted priority of safety, incident reporting, and error management. Adv. Health Care Manage. 14, 95–117. Vinodkumar, M.N., Bhasi, M., 2010. Safety management practices and safety behaviour: assessing the mediating role of safety knowledge and motivation. Accid. Anal. Prev. 42, 2082–2093. Vredenburgh, A., 2002. Organizational safety: which management practices are most effective in reducing employee injury rates? J. Saf. Res. 33, 259–276. Wachter, J.K., Yorio, P.L., 2014. A system of safety management practices and worker engagement for reducing and preventing accidents: an empirical and theoretical investigation. Accid. Anal. Prev. 68, 117–130. Walker, A., 2010. The development and validation of a psychological contract of safety scale. J. Saf. Res. 41, 315–321. Wiegand, D.M., 2007. Exploring the role of emotional intelligence in behavior-based safety coaching. J. Saf. Res. 38, 391–398. Williams, R.M., Westmorland, M.G., Shannon, H.S., Amick III, B.C., 2007. Disability management practices in Ontario health care workplaces. J. Occup. Rehabil. 17 (1), 153–165. Worksafe Australia, 1990. Workplace Injury and Disease Recording Standard. Standards Australia, Sydney. Wright, P.M., Gardner, T.M., Moynihan, L.M., 2003. The impact of HR practices on the performance of business units. Human Res. Manage. J. 13, 21–36. Wu, T.-C., 2008. Safety leadership in the teaching laboratories of electrical and electronic engineering departments at Taiwanese universities. J. Saf. Res. 39, 599–607. Wu, T.-C., Chen, C.-H., Li, C.-C., 2008. A correlation among safety leadership, safety climate and safety performance. J. Loss Prev. Process Ind. 21, 307–318. Wu, T.-C., Lin, C.-H., Shiau, S.-Y., 2010. Predicting safety culture: the roles of employer, operations manager and safety professional. J. Saf. Res. 41, 423–431. Wurzelbacher, S., Jin, Y., 2011. A framework for evaluating OSH program effectiveness using leading and trailing metrics. J. Saf. Res. 42, 199–207. Zanko, M., Dawson, P., 2012. Occupational health and safety management in organizations: a review. Int. J. Manage. Rev. 14, 328–344. Zohar, D., 2010. Thirty years of safety climate research: reflections and future directions. Accid. Anal. Prev. 42, 1517–1522. Zohar, D., Luria, G., 2005. A multilevel model of safety climate: cross-level relationships between organization and group-level climates. J. Appl. Psychol. 90, 616–628.