Organizational Climate Metrics as Safety, Health and Environment Performance Indicators and an Aid to Relative Risk Ranking within Industry

ORGANIZATIONAL CLIMATE METRICS AS SAFETY, HEALTH AND ENVIRONMENT PERFORMANCE INDICATORS AND AN AID TO RELATIVE RISK RANKING WITHIN INDUSTRY M. Dodsworth1
, K. E. Connelly1, C. J. Ellett1 and P. Sharratt2 1

AstraZeneca, Hallen, Bristol, UK. School of Chemical Engineering and Analytical Science, The University of Manchester, Manchester, UK. 2

Abstract: The chemical, pharmaceutical and other related process industries are characterized by inherently hazardous processes and activities. To ensure that considered risk management decisions are made it is essential that organizations have the ability to rank the risk profiles of their assets and operations. Current industry risk ranking techniques are biased toward the assessment of the risk potential of the asset or operation. Methodologies used to assess these risks tend to be engineering-based and include, for example, hazard identification and event rate estimation techniques. Recent research has associated lagging safety performance indicators with metrics of organizational safety climate. Despite the evidence suggesting their potential usefulness, organizational climate metrics have not yet been exploited as a proactive safety, health and environmental performance indicator or as an aid to relative risk ranking. This paper summarizes research that successfully produced a statistical model of organizational climate and its relationship to site significant injury frequency rates, allowing the relative risk ranking of sites based upon organizational climate metrics. The responses to an industrial organizational survey are examined for a pharmaceutical company’s sites in the United Kingdom, Sweden and the United States. Projection to Latent Structures Analysis is performed on the survey responses. The resultant models are shown to be able to accurately model the site significant injury frequency rates. The organizational climate metrics that discriminate between the safety performance levels of different sites are identified. Keywords: organizational; climate; culture; safety; PLS; SIMCA; modelling; human factors.


Correspondence to: Dr M. Dodsworth, AstraZeneca, Severn Road, Hallen, Bristol, BS10 7ZE, UK. E-mail: mark.dodsworth@ astrazeneca.com DOI: 10.1205/psep06006 0957–5820/07/ $30.00 þ 0.00 Process Safety and Environmental Protection Trans IChemE, Part B, January 2007 # 2007 Institution of Chemical Engineers

INTRODUCTION Despite the desire to improve SHE performance, accident and incident rates for many commercial and industrial organizations have plateaued (Donald and Canter, 1993; Krause, 1994; Astrazeneca, 2006). Saari (1990) suggested that after a certain point, technology alone cannot achieve further improvements in safety. Instead, organizational and cultural factors become more important. Awareness regarding the importance of organizational culture and its relationship to positive SHE performance is growing (Fleming and Larder, 1999; Newton, 2001). Assessments have been made that attribute over 50% of industrial-based accidents to poor management, poor training and other psychological factors (Kantyka, 1977; HSE, 2002). Other studies within the chemical industry have shown that more accidents

are attributable to human issues and managerial deficiencies than to weaknesses in technical components (Powell and Canter, 1985; Reason, 1990; Layfield, 1986). The importance of addressing cultural aspects has been highlighted by recent well publicized major loss events such as Chernobyl (Group I.N.S.A., 1988), Piper Alpha (Cullen, 1990), the Kings Cross fire (Fennell, 1988) and the inquiry into the 1999 Ladbroke Grove rail accident (Cullen, 2001). The recognition that there is a relationship between organizational culture and safety performance has spawned an increased interest in identification of methods that allow measurement of organizational culture (Guldenmund, 2000; HSE, 2003; Silvia et al., 2004; Zohar, 2000). Although much work has been done regarding the identification and modelling of safety culture, much less research has been performed in correlating overall organizational 59

Vol 85 (B1) 59–69

60

DODSWORTH et al.

culture with outcome metrics such as lagging safety performance indicators. Despite their potential usefulness, organizational climate metrics have not yet been exploited by industry as a proactive measure of SHE performance or as an aid to relative risk ranking. The purpose of this research (Dodsworth, 2005) was to produce a statistical model that could be used to predict site significant injury frequency rates based upon organizational climate metrics.

ORGANIZATIONAL CULTURE AND ITS CORRELATION WITH SHE PERFORMANCE Schein (1985) summarizes culture as ‘The way in which a group of people solves problems and reconciles dilemmas’. Ostrom et al. (1993) define culture as: ‘. . . norms or patterns of perceptions, speech and even building design features that make the culture what it is’. Rentch’s (1990) definition of organizational culture refers to shared perceptions among members of an organization with regard to organizational policies, procedures and practices. James and Jones (1974) distinguished metrics of organizational climate that are based on the structural properties of organizations such as size, leadership, systems and organizational structure from perceptions held by employees about aspects of their organizational environment. Other definitions of safety culture are available (Glendon and Mckenna, 1995; Lee, 1996; Ostrom et al., 1993; Pidgeon, 1991). The related but distinct concept of safety climate has been defined as ‘The objective measurement of attitudes and perceptions toward occupational health and safety issues’ (Coyle et al., 1995). There are many other similar definitions for example, Cabrera et al. (1997), Coyle et al. (1995), DeDobbeleer and Beland (1991), Niskanen (1994), Williamson et al. (1997), Zohar (1980). Safety climate can be thought of as the observable manifestations of a safety culture. Guldenmund (2000) has argued that it is inappropriate to attempt to separate safety culture from general business management culture. Indeed, Apostolakis and Wu (1995) suggested that ‘When the subject is culture, we must question the wisdom of separating safety culture from the culture that exists with respect to normal plant operation and power production. The dependencies between them are much stronger because they are due to common work processes and organizational factors’. Ball and Scotney (1998) suggest a ‘composite’ holistic view of culture as being the product of organizational and personal factors. Cooper (2000) proposed that safety culture should be the dominant sub-component of an organization’s culture when the organization is operating within a high-risk industry sector. Flin et al. (2000) and Cox and Flin (1998) have suggested that one set of climate indicators may not adequately describe a safety culture across different businesses. Research by Coyle et al. (1995) has indicated that the organizational climate factors may not be stable over time. De Cock et al. (1986) have, however, shown that organizational culture has an element of stability over a period of at least 5 years. The majority of the above research recognizes that culture is a multi-dimensional construct. Being "multi-dimensional, no one attribute may adequately describe it, and researchers have had considerable scope to formulate and label their own dimensions. Because of this interpretative freedom, a number of researchers have put forward different methodologies by which safety culture can be measured (Jick,

1979; Denzin, 1978; Gadd, 2002). Although the number of methodologies by which safety culture can be measured is great, all of them share common features. The most common administration method is via a written set of questions. The questions are sometimes the result of group discussions or interviews, often referred to as focus groups, in which the researchers verbally explore the issues pertinent to that group or organization (see e.g., Coyle et al., 1995). The number of culture dimensions identified varies widely from two (DeDobbeleer and Beland, 1991) to 19 (Lee, 1996). Guldenmund (2000) performed an extensive review of the available research in the field of safety culture, summarizing about 150 safety culture factors found in the literature. Flin et al. (2000) performed a thematic analysis of 18 publicly available industrial-based safety climate surveys. Their analysis concluded that each culture dimension encountered within the surveys could be categorised into one of the five themes: management/supervision, safety system, risk, work pressure and competence. A set of climate factors may adequately represent elements of an organization’s safety culture. However, the same set of factors may not necessarily be able to accurately represent another organization’s safety culture. Flin et al. (2000) and Cox and Flin (1998) suggest that there are insufficient empirical data on the applicability of these common thematic labels across industry or across different cultures. Although safety culture has been stated to be an important contributory factor to accidents, little research has been done to establish a quantitative relationship between the two. Pidgeon (2004) wrote: ‘Some 10 years on from Chernobyl, the existing empirical attempts to study safety culture and its relationship to organizational outcomes have remained unsystematic, fragmented, and in particular under-specified in theoretical terms’. In his discussions regarding the evolution of the terms ‘safety culture’, Sorenson (2002) wrote: ‘Statistical evidence that unambiguously links safety culture with the safety of operations is surprisingly rare’. Industry’s enthusiasm to improve safety culture in the absence of statistical evidence linking it to organizational outcomes has been summarized by Sorensen (2002): ‘The proponents of safety culture as a determinant of operational safety in the nuclear power industry are relying, at least to some degree, on that indirect assumption [that relatively low accident plant must have a relatively good safety culture]’. In a study comparing culture factors within organizations, Coyle et al. (1995) wrote: ‘From a proactive viewpoint, it appears that low scores on the safety climate factors identified in the organizations studied here correlated highly with traditional indices such as lost time or accident rates. The relationship between safety climate analysis and other positive performance indicators of occupational safety and health has not been reported and is a major area for future research’. Studies attempting to correlate organizational culture with lagging SHE performance indicators are complicated by the potentially very different inherent SHE risk of the units encountered. This complication results from the Heinrich et al. (1980) model of accident causation. In the Heinrich et al. model, accidents are caused by the simultaneous occurrence of unsafe acts and unsafe conditions. In his study examining the relationship between accidents and organizational culture, Zohar (2000) attempted to take the inherent risk of a workgroup into consideration by including a risk factor that was based upon the subjective assessment

Trans IChemE, Part B, Process Safety and Environmental Protection, 2007, 85(B1): 59–69

ORGANIZATIONAL CLIMATE METRICS of the inherent risk in each area. Zohar failed to establish a correlation between perceived risk of a work area and the number of accidents occurring within it. In his literature review Zohar (1980) found that several organizational characteristics were able to discriminate those industrial sites with very good safety performance and those with bad safety performance. He found that the most common factors that distinguished superior safety performing sites was a strong management commitment to safety. Other, more obscure relationships were also found. For example, companies with good safety performance were characterized by personnel holding their Safety Officers in higher regard. This finding was also reported in the 1976 Accident Prevention Advisory Unit report (HSE, 1976). Ferguson et al. (1984) found a relationship between educational background and accidents. Leigh (1986) discovered a relationship between gender and accidents. Leveson and Hirchfield (1980) found a relationship between the occurrences of accidents and recent ‘life events’. Melamed et al. (1989) discovered a link between accidents and job satisfaction. Dwyer and Raftery (1991) have linked accidents with management reward for work rates and overtime. Cox and Cox (1991) linked perceptions of risks and/or attitudes toward safety to safety behaviour. Researching the links between safety climate factors and accidents, Coyle et al. (1995) concluded that safety climate factors correlated highly with traditional (lagging) indicators. Coyle et al’s experiment, however, did not quantify this correlation. Based upon a literature review of previous research, Zohar (1980) formulated a safety climate questionnaire. The questionnaire consisted of 49 questions designed to measure seven organizational climate dimensions: . . . . . . .

perceived management attitudes toward safety; perceived effects of safe conduct on promotion; perceived effects of safe conduct on social status; perceived organizational status of the Safety Officer; perceived importance and effectiveness of safety training; perceived risk level at the workplace; perceived effectiveness of enforcement versus guidance in promoting safety.

The questionnaire was administered to 20 factories in the chemical, metal fabrication, food processing and textile industry sectors in Israel. The questions measuring the factors were then aggregated to give a single ‘safety climate score’. A team of four judges assessed the perceived risk of each factory. Spearman rank correlation coefficients between the safety climate scores and the subjective perceived risk were calculated for five of the metal, four chemical and three food companies. Only the metal factory correlations were found to be above the level of statistical significance at the 95% confidence level. By administration of a climate survey, Lee and Harrison (2000) measured individuals’ attitudes in three nuclear power stations. Twenty-four climate factors resulting from the climate survey were correlated with one or more of nine self-reported accident criteria. In his paper, O’Toole (2002) proposed a link between employees’ perceptions of management’s commitment to safety and injury frequency rate; the correlation was, however, not quantified. Fleming et al. (1996) measured subjective risk perception in six offshore drilling rigs and examined how these perceptions related to lost time accident (LTA) frequency records

61

(number of LTAs  1 000 000/number of man hours worked) and available quantitative risk assessment (QRA) data. The risk perceptions of 622 workers across six different UK oil platforms were measured by the application and analysis of a 14-section questionnaire. The risk perceptions of the workers were plotted against the rank average LTA frequency for the preceding 2–3 years. Summarizing the data, Fleming et al. conclude that: ‘The installations are generally ranked in the same order according to respondents’ feelings of safety, as they are by the frequency of LTAs, although the degree of correlation was not found to be significant’. Fleming et al. also compared ‘feelings of safety’ with QRA data. All offshore installations working within UK territorial waters are required to have a temporary refuge (TR) for emergency use. Five of the six installations provided Fleming et al. with QRA data associated with TR impairment, i.e., an inability to use the refuges. Fleming et al. interpreted TR impairment as follows: ‘The TR failure value is a representative measure of how secure the platform is with regard to the reliability of protective systems on the installation’. Flemming et al. proposed that the TR QRA values represented: ‘. . . a measure of the cumulative failure of the systems, which takes both the probability and consequence of the events into consideration’. Comparison between the rankings indicated a high but nonsignificant correlation due to the small sample size. Torbjørn (1992) has examined the relationship between perceived risk, job stress and frequency of accidents and near misses. Perceived risk and job stress on eight Norwegian oil installations representing five different companies were evaluated by administering a question survey. The results of the questionnaire were analysed and compared with self-reported injury data. Although Torbjørn’s study reported correlations between organizational culture factors, no statistical correlations were calculated between any of the factors and injury data. Canter and Olearnik (1989), Canter and Donald (1990), Donald and Canter (1994) developed a 10 factor question set to measure safety climate, and administered it to 10 chemical plants in the UK. The results of the study showed that there was a: ‘. . . strong link between the 10 climate measures and the number of self-reported accidents’. Diaz and Cabrera (1997) administered a safety climate and attitude survey to two companies and one authority associated with a Spanish airport. The responses were used to assign a safety attitude, safety climate and safety level scale to each of the three groups. The safety level scale was calculated based upon the response to six questions that measured workers perceptions regarding: (1) their involvement in an accident in the previous 12 months and the likelihood of them being involved in accidents in the near future; (2) the level of safety involved with work tasks; (3) the compliance with safety standards; (4) the general level of safety of the operators. Diaz and Cabrera then plotted the resulting average scores for safety attitude, safety climate and safety level of the three companies. The resultant plot indicated that high safety-level scores correlated with high safetyclimate and safety-attitude scores. Although a correlation was demonstrated, the results of the analysis were not statistically significant because only three companies were involved in the survey. Research examining how safety climate indicators correlate with lagging safety performance indicators has been reported by the HSE (2003). In their research, safety climate surveys were performed on 13 North Sea oil-drilling platforms

Trans IChemE, Part B, Process Safety and Environmental Protection, 2007, 85(B1): 59– 69

62

DODSWORTH et al.

during 1998 and 1999. In 1998, 682 questionnaires from 10 installations were available. In 1999, 806 questionnaires from 13 installations were available. The values of safety climate factor scores were correlated with the following four lagging safety indicators: number of injuries requiring absence from work greater than three days; number of dangerous occurrences; number of visits to the rig medic for first aid; number of RIDDOR (HMSO, 1995) reportable incidents. In their conclusions the HSE (2003) state: ‘Assuming that the Offshore Safety Questionnaire does measure safety climate reliably, it would appear that dimensions of climate predictive of safety outcome in one time period do not retain their predictive power either between years or between accident types’. Silvia et al. (2004) examined the correlation between safety climate and safety outcomes. In their study an organizational safety climate questionnaire was administered to 930 individuals in fifteen Portuguese organizations in different sectors, including the chemical industry, public administration, electricity and health. The responses to the survey were used to provide measures of the following five safety climate dimensions: . . . . .

strength strength strength strength strength

of of of of of

organizational climate index (OCSI); safety climate index; safety as an organizational value index; organizational safety practices index; personal involvement with safety index.

Attempts to obtain accident rate, accident frequency rate and severity rate data were made for each of the 15 organizations. Definitions for the three rate criteria used by Silvia et al. (2004) are as follows. Accident rate: an instantaneous bodily defect so that the individual is physically or mentally, as determined by a competent medical authority, incapable to work on a scheduled day or shift, resulting in at least three days off the job (Chemical Industry Association, 1988). (It is noted that accident rate, as defined by Silvia et al., is actually the number of accidents that have occurred and therefore is not a rate.) Accident frequency rate: the amount of time lost due to injuries per million working hours. Severity rate: the number of workdays lost per million hours. After examination of the collected data, Silvia et al. (2004) found that not all of the 15 organizations were able to provide information that followed the above definitions. Seven organizations were able to provide accident rate data. Six organizations were able to provide accident frequency rate data and five organizations were able to provide severity rate data. Silvia et al. (2004) then went on to correlate the metrics of safety climate with the above three accident rate data. The resultant Spearman correlations are reproduced in Table 1. Numbers in parentheses indicate those results above the 95% significance level. In their conclusions Silvia et al. (2004) write: ‘. . . these results suggest

that OSCI has some capacity to predict and discriminate organizations with different accident levels’. Table 1 indicates statistically significant Spearman correlations between all five safety climate metrics and the accident rates. Silvia et al. do not provide information regarding how many personnel were in each of the organizations taking part in the study. The use of ‘accident rate’ as defined above by Silvia et al. is inappropriate unless the number of personnel in each organization is the same. Examining the Spearman correlations between the safety climate metrics with the accident frequency rates supports this conclusion. Column 3 of Table 1 indicates that only the ‘strength of safety practices’ correlates with accident frequency rates above the 95% confidence level. Zohar (2000) administered an organizational climate question survey to 534 production workers, divided into 53 work groups, in a metal-processing plant. He performed principal component analysis (PCA) (Eriksson, 1999) on the question responses. The results of the analysis indicated two principal components that Zohar labelled as ‘Supervisory Action’ and ‘Supervisor Expectation’. The subunit risk of each of the 53 workgroups was subjectively assessed and scored by each of the workgroup supervisors. The numbers of lost days due to injury and micro-accidents (minor accidents requiring first aid) were recorded for a period of 5 months following the administration of the survey. Zohar then went on to correlate subunit risk, supervisory action, supervisory expectation, injury rate (based upon micro-accidents divided by group size) and accidents (expressed as the number of lost days due to injury). Zohar failed to establish a significant relationship between supervisory action or expectation and the number of micro-accidents or number of lost days due to injury. In the final part of Zohar’s paper he performed least squares regression of supervisory action, and supervisory expectation with micro-accidents as the outcome variable. Zohar reported that his model accounted for 16% of the micro-accident variation seen within the responses. He went on to write: ‘It is evident that both climate subscales provided significant prediction of the micro-accident rate’. For Zohar to write the above is inappropriate as his model fails to account for 84% of the observed micro-accident variation. According to Sorensen (2002): ‘No performance indicators to gauge safety culture and its impact on safety of operations appear to have been identified and validated’. After reviewing organizational safety culture and climate research over the proceeding 20 years, Guldenmund (2000) concluded: ‘. . . the measurement of safety climate could be considered an alternative safety performance indicator . . . research should not be undertaken to develop “new” safety climate measurement instruments, but should rather focus on the

Table 1. Silvia et al. (2004) Spearman correlation between climate factor and accident data (parentheses indicate results above the 95% significance level). Climate factor Strength Strength Strength Strength Strength

of of of of of

organizational climate index safety climate index safety as an organizational value index organizational safety practices index personal involvement with safety index

Accident rate correlation coefficient

Frequency rate correlation coefficient

Severity rate correlation coefficient

(20.865) (20.955) (20.883) (20.883) (20.955)

20.31 20.77 20.77 (20.83) 20.77

20.30 20.60 20.60 20.70 20.60

Trans IChemE, Part B, Process Safety and Environmental Protection, 2007, 85(B1): 59–69

ORGANIZATIONAL CLIMATE METRICS validity of the construct and whether it indeed yields a robust indication of an organization’s safety performance’. In summary, the literature review indicated that existing research appears to have concentrated very much upon the social science aspects of organizational culture rather than any practical industrial applications. Although many hypotheses have been proposed, few researchers have attempted to build predictive models. The predictive power of the few models that have been produced is mostly poor. It is suggested that the poor predictive ability of these models is due to the failure of the researchers to appropriately address the multi-dimensional nature of organizational culture.

STATISTICAL ANALYSIS OF ORGANIZATIONAL CULTURE DATA Shannon et al. (1997) performed a review of the available literature that attempted to correlate organizational factors with injury rates. Their study found that the statistical techniques used by previous researchers included: . factor analysis followed by regression analysis (Habeck et al., 1991; Hunt et al., 1993; Tuohy and Simnard, 1993); . factor analysis followed by discriminant function analysis (Itabeck et al., 1988); . Univariate analysis using sign test; . Wilcoxon matched-pairs signed-ranked test (Cohen et al., 1975; Chew, 1988; Shafiai-Sahari, 1973); . Discriminant function analysis; . Principal component analysis; . Logistic regression analysis (Simnard et al., 1988); . t-tests (Mines safety and Health Administration, 1983). The application of these methods identified by Shannon et al. (1997) is flawed in that they are unable to simultaneously correlate several dimensions of organizational culture with an outcome variable. Statistical methodologies used to establish links between organizational culture with an outcome variable must be able to simultaneously model several dimensions of X (predictor) data and correlate it with the Y (predicted) block data. A desirable feature of the statistical methodology is that it should be able to provide information regarding the relative importance of the X block variables have in predicting the Y block data. Projection to latent structures (PLS) (Eriksson et al., 1999) and Artificial neural networks (ANNs) (Smith, 2004; Statsoft, 2004) are two techniques that are

63

capable of simultaneously correlating multivariate data with an outcome variable. ANNs are however not able to provide useful information regarding X block variable importance. PLS was therefore selected as the preferred technique for the experimental phase of this project. An overview of the PLS technique is detailed in Appendix 1 of this document.

EXPERIMENTAL The Company had carried out an 82 question organizational climate survey in 2002 (Focus 2002). The significant accident frequency rate data for 2002 and the mean responses to this survey were collected for nine (UK), 10 Swedish (SE) and five United States (US) sites. Each of the sites belonged to a multi-national pharmaceutical company. Activities at each of the sites varied but included research and development, marketing and the large scale manufacture of pharmaceuticals. In the survey, 7160 UK (56%), 8279 SE (74%) and 8289 US (72%) personnel participated (numbers in parentheses indicate the percentage of the total population who participated in the survey). The climate survey questions were compared with the five organizational climate themes described by Flin et al. (2000). The subjective comparison indicated that the climate survey contains attitudinal questions that address each of these five thematic factors suggesting that the survey data should be able to measure organizational culture. The above data were entered into the PLS software SIMCA Pþ (Version 10.0.4.0) (UMETRICS, 2004). Four separate PLS models were built, one for each national data set (UK, SE US) and one for the combined data set (UK-SEUS). The X block data for the PLS models UK, SE and US consisted of the average site responses to the Focus 2002 questions for each for the UK, SE and US sites, respectively. The Y block data consisted of the Significant Injury Frequency Rate (SIFR) data for each UK, SE and US site. The X block data for model UK-SE-US consisted of the site average responses to the Focus questions for all UK, SE and US sites. The Y block data consisted of the SIFR data for all UK, SE and US sites. The default mean centering and scaling option was selected within SIMCA Pþ for each model. The models were then optimized and refined. The resultant models were validated using the ‘response permutation validation’ option within SIMCA Pþ. The results of the modelling are summarized in Table 2. Figures 1–3 detail the actual versus model predicted SIFR best-fit line for the UK, SE

Table 2. Summary of the PLS modelling results. PLS model name

UK

SE

US

UK-SE-USa

The number of model principal components used The R2X value for the 1st principal component The R2X cumulative value for all of the model principal components The Q2 value for the 1st principal component The Q2 cumulative value for all of the model principal components The R2Y value for the 1st principal component. The R2Y cumulative value for all of the model principal components

2 0.76 0.87

2 0.82 0.89

1 0.72 0.72

— — —

0.89 0.92

0.36 0.54

0.90 0.90

— —

0.91 0.96

0.50 0.88

0.92 0.92

— —

a

SIMCA Pþ was unable to model PLS-UK-SE-US (as indicated by SIMCA Pþ being unable to formulate principal components).

Trans IChemE, Part B, Process Safety and Environmental Protection, 2007, 85(B1): 59– 69

64

DODSWORTH et al.

Figure 1. Actual versus predicted SIFR for the UK PLS model.

Figure 4. Score scatter plot for the SE model.

. factors other than organizational culture dominate or significantly affect SIFR performance at the SE sites; . the SIFR performance at the SE sites is driven by a population whose organizational culture is not represented by the mean Focus 2002 question responses; . the way in which the Focus 2002 questions were translated from English to Swedish introduced degrees of interpretive freedom that in turn gave rise to a greater response variance.

Figure 2. Actual versus predicted SIFR for the SE PLS model.

and US models. Figures 4 and 5 are example score and loading plots for the Swedish sites. All of the SIMCA Pþ ‘response permutation validation’ plots for all models produced were found to have Q 2 ordinate intercepts less than 0.05. All models produced were therefore proven to have some validity i.e., they predict SIFR performance significantly better than chance. Inspection of the actual versus predicted SIFR prediction plots indicates that all of the models produced are able to discriminate between those sites with good SIFR performance from those who have poorer SIFR performance. The ability of PLS to model the SE data appears to be poorer than UK and US data. There may be several possible explanations for this difference that include: . there may be inconsistencies in the threshold of SIFR reporting across the SE sites;

Figure 3. Actual versus predicted SIFR for the US PLS model.

The experimental work was unable to identify which, if any, of the above factors gave rise to the poorer predictive ability of the SE PLS model. It was shown that it was not possible to produce a combined UK, SE, US PLS model. A possible explanation of the inability to identify a set of questions that correlate within all three nations may be the dominance of national culture over organizational culture. If national culture dominates over organizational culture, the likelihood of finding a common set of questions that correlate with safety performance in several nations is decreased. This decreased likelihood will be a result of the question responses being influenced by national cultural differences rather than how sites in any one nation perceived a particular issue. It is suggested that the production of a single PLS UK, SE and US model may be possible after identification of a set of questions that correlate with SIFR performance in each of the nations. All of the models produced are able to account for at least 88% of the SIFR variation (as indicated by R2Ycum). The R2Y variation accounted for by the first component is 0.91 for the UK, 0.50 for SE and 0.92 for the US. Each of the

Figure 5. Loadings plot for the SE model.

Trans IChemE, Part B, Process Safety and Environmental Protection, 2007, 85(B1): 59–69

ORGANIZATIONAL CLIMATE METRICS principal components of the PLS models can be visualised as a dimension of organizational culture that is related to SIFR performance. One can therefore conclude that: . one dimension of organizational culture dominates SIFR performance at the Company’s UK and US sites (as indicated by an R2Y value of greater than 0.91 for the first component in each model); . two dimensions of organizational culture are related to SIFR performance at the Company’s SE sites (as indicated by a two principal component model with an R2Y for the first component of 0.55 and the second component cumulative R2Y value of 0.88); . the Focus 2002 survey is able to measure the organizational cultural constructs that dominate SIFR performance at the Company’s UK, SE and US sites. The Focus 2002 question responses and the number of responses retained in the final PLS models are dissimilar for the UK, SE and US nations. Several possible explanations for the dissimilarities are possible and include: Each of the nations may have different accident causation models, i.e., different factors are associated with the events leading to an accident. The Focus 2002 questions may be interpreted differently in each of the nations. Reasons for the interpretation differences may be due to cultural issues or the way in which the Focus 2002 question set was translated. Comparison of the model score scatter and loadings plots provide information regarding which questions discriminate sites of differing significant injury frequency performance. Figure 4 shows that the poorer SIFR performing sites SE1 and SE 9 (as indicated in Figure 2) are distinct from the other better SIFR performing Swedish sites. SE1 and SE9 are discriminated from the other Swedish sites as a result of the mean site responses from SE1 and SE9 being greater than the Swedish site average for those questions in the upper right quadrant and less than average response to the question in the lower left quadrant of Figure 5.

CONCLUSIONS The results of this research work represent the first time that several dimensions of organizational culture have been simultaneously correlated with a lagging SHE performance indicator. Because PLS has been shown able to account for about 90% of the SHE outcome variation, the application of PLS modelling of organizational climate metrics appears to correlate well with site SIFR performance. The practical application of a PLS organizational climate model as a proactive SHE performance indicatator is dependent upon the dynamics of organizational culture. The use of PLS modelled climate metrics as a proactive SHE indicator will be appropriate subject to confirmation that (1) organizational culture is stable over time and (2) the correlation between organizational culture and its relationship with SIFR is stable over time. The ability to model SIFR performance without consideration of site relative risk profile information suggests that, for the sites included within this study, the levels of risk its workers are exposed to do not significantly affect SIFR performance. This observation puts into question Heinrich’s (1980) classic model that specifies that accidents happen as a probabilistic function of the joint occurrence of unsafe acts and unsafe conditions. It is unlikely that Heinrich’s model is incorrect. More likely is that industrial risks are

65

already well controlled by engineering and procedural means. The observed relationship between organizational culture and SIFR performance is most likely to be related to the residual risk remaining after establishment of efficient engineering and procedural controls. The predictor X block of the PLS models used only the Focus 2002 question responses. The literature review indicated that several researchers in the field of organizational culture have advocated the ‘triangulation approach’ to measure organizational culture. Proponents of such an approach are of the opinion that measurement of several attributes allows a more accurate assessment of organizational culture compared to the measurement of a single attribute. The inclusion of other factors may therefore improve the already excellent predictive ability of the PLS models. The technique of PLS modelling is well able to deal with several predictor variables and would therefore be ideally suited to the inclusion of additional organizational culture attributes. The ‘triangulation approach’ may also be helpful in assessing an organizations overall SHE performance. Instead of using a single SHE performance metric, several SHE performance metrics could be measured simultaneously. This approach would give a better appreciation of the overall SHE performance of the organization or unit. The PLS models built as part of this research only used one lagging SHE performance indictor as a metric of site SHE performance. Appendix 1 explains that PLS is capable of dealing with several Y block (predicted) variables. One can therefore imagine creating a PLS model with many predictor and predicted variables. In such a model one could select an area in ‘n’ dimensional Y space that corresponded to desirable SHE performance. The PLS model could then be used to identify those predictor attributes that were associated with the desirable SHE performance. Taking this idea one stage further, the Y block could contain many business outputs such as productivity, down time, SHE performance, absenteeism, and so on to give an overall organizational performance indicator. It is recommended that further research work be carried out in this area to fully exploit the potential usefulness of PLS modelling. This research was based upon a large combined UK, SE and US data set of 23 728 responses. Although it is clearly advantageous to possess large data sets, they are not essential to make use of the PLS technique. The use of smaller data sets is practicable so long as the researcher takes into consideration the possible presence of sub-cultures. Modelling problems may be encountered if sample sizes are small (e.g., 10 individuals) and distinct sub-cultures are present within the group. Alternative modelling strategies could, however, be used, and include Principal Component Analysis pre-treatment of the data to identify the sub-cultures that could then be analysed separately. Nine UK, 10 SE and five US sites were modelled in this project. Industrialists need not have several sites to take advantage of PLS. Single site companies could collect organizational climate and SHE outcome data from several different functions and/or departments. The data could then be subjected to PLS to identify those organizational culture factors that relate to desirable and undesirable SHE outcomes. Industry should not restrict the application of PLS techniques to the linking of organizational culture with SHE outcomes. Conceptually, the techniques could be used to establish links with other business outcomes such as productivity, absenteeism, staff turnover and quality metrics. The number of potential applications

Trans IChemE, Part B, Process Safety and Environmental Protection, 2007, 85(B1): 59– 69

66

DODSWORTH et al.

and uses of PLS is vast. The contribution to knowledge summarized above, provides industry with the necessary confidence and information to enable it to implement local initiatives to exploit organizational climate metrics to improve SHE and potentially many other business outcomes.

REFERENCES Apostolakis, G. and Wu, J.S., 1995, A structured approach to the assessment of the quality culture in nuclear installations. in American Nuclear Society International Topical Meeting on Safety Culture in Nuclear Installations. Vienna. AstraZeneca, AstraZeneca Website—reporting performance, Accessed 25/06/2006 (www.astrazeneca.com). Ball, P.W. and Scotney, V., 1998, Approaches to Safety Culture Enhancement, prepared for British Nuclear Fuels Ltd. Cabrera, D.D., Isla, R. and Vilela, L.D., 1997, An evaluation of safety climate in ground handling activities. in Proceedings of the IASC97 International Aviation Safety Conference, Netherlands. Canter, D. and Donald, I., 1990, Proceedings of Culture Space History, in 11th International Conference of the International Association for the Study of People and their Physical Settings, Faculty of Architecture Press, Middle East Technical University, Ankara, Turkey. Canter, D. and Olearnik, H., 1989, Total Safety: A Strategy for Zero Accidents at British Steel Teesside Works (University of Surrey, Guilford, UK). Chemical Industry Association, 1988, Health, Safety and Environmental Reporting Guidelines (European Chemical Industry Council, Brussels). Chew, D., 1988, Effective occupational safety activities: findings in three Asian developing countries, International Labour Review, 127: 111–124. Cohen, A., Smith, M. and Cohen, H.H., 1975, Safety Programme Practices in High Versus Low Accident Rate Companies, HEW Publication No (NIOSH) 75–185 (National Institute For Occupational Safety and Health, Cincinnati, OH, USA). Cooper, M.D., 2000, Towards a model of safety culture, Safety Science, 36: 111 –136. Cox, S. and Cox, T., 1991, The structure of employee attitudes to safety: a European example, Work and Stress, 5: 93–106. Cox, S. and Flin, R., 1998, Safety culture. Philosopher’s stone or man of straw. Work and Stress, 12(3): 189– 201. Coyle, I.R., Sleeman, S.D. and Adams, N., 1995, Safety climate. Journal of Safety Research, 26(4): 247– 254. Cullen, W.D., 2001, The Ladbroke Grove Rail Inquiry Part 2 Report (HSE Books). Cullen, W.D., 1990, The Public Inquiry into the Piper Alpha Disaster. (Department of Energy, HMSO, London, UK). De Cock, G., Bouwen, R. and De Witte, K., 1986, Organisatieklimaat: Een opdracht voor het personeelsbeleid? Pranktisch Personeelsbeleid, Capita Selecta, 16: 1– 20. DeDobbeleer, N. and Beland, F., 1991, A safety climate measure for construction sites, Journal of Safety Research, 22: 97–103. Denzin, N.K., 1978, The Research Act (McGraw Hill, New York, USA). Diaz, R.I. and Cabrera, D.D., 1997, Safety Climate and Attitude as Evaluation Measures of Organisational Safety, Accident Analysis and Prevention, 5: 643–650. Dodsworth, M., 2005, Organisational climate metrics as a leading SHE performance indicator and an aid to relative risk ranking within industry, in School of Chemical Engineering and Analytical Science—Faculty of Engineering and Physical Sciences (UMIST, Manchester). Donald, I. and Canter, D., 1994, Employee attitudes and safety in the chemical industry, Journal of Loss Prevention in the Process Industry, 7(3): 203 –208. Donald, I. and Canter, D., 1993, Psychological factors and the accident plateau, Health and Safety Bulletin, 215: 5– 12. Dwyer, T. and Raftery, A.E., 1991, Industrial accidents are produced by social relations of work: A sociological theory of industrial accidents, Applied Ergonomics, 22: 167 –179. Eriksson, L., Johansson, E., Kettaneh-Wold, N. and Wold, S., 1999, Introduction to multi- and megavariate data analysis using projection methods (PCA & PLS). 21 June. 1999: UMETRICS AB.

Fennell, D., 1988, Investigation into King’s Cross Underground Fire. Department of Transport. (HMSO, London). Ferguson, J.L., McNally, M.S. and Both, R.F. 1984, Individual characteristics as predictors of accidental injuries in naval personnel, Accident Analysis and Prevention, 16: 47–54. Fleming, M., Flin, R., Mearns, K. and Gordon, R., 1996, Risk Perception by Offshore Workers on UK Oil and Gas Platforms, Safety Science, 22: 131–145. Fleming, M. and Larder, R., 1999, Safety Culture—The Way Forward, in The Chemical Engineer, 11 March. Flin, R., Mearns, K., O’Connor, P. and Bryden, R., 2000, Measuring safety climate: identifying the common features, Safety Science, 34: 177–192. Gadd, S., 2002, Safety Culture: A Review of the Literature, Health and Safety Laboratory. Glendon, A.I. and McKenna, E.F., 1995, Human Safety and Risk Management. (Chapman and Hall, London). Group, I.N.S.A., 1988, Basic Safety Principles for Nuclear Power Plants. (International Atomic Energy Agency, Vienna). Guldenmund, F.W., 2000, The nature of safety culture: a review of theory and research, Safety Science, 34: 215 –257. Habeck, R., Leahy, M.J. and Hunt, H.A., 1988, Disability Prevention and Management and Workers’ Compensation Claims (Upjohn Institute for Employment Research, Kalamazoo, MI, USA). Habeck, R., Leahy, M.J., Hunt, H.A., Chan, E. and Welch, E.M., 1991, Employer Factors Related To Workers’ Compensation Claims and Disability Management, Rehabilitation Counselling Bulletin, 34: 210– 226. Heinrich, H.W., Peterson, D. and Roos, N., 1980, Industrial Accident Prevention (McGraw-Hill, New York). HMSO, 1995, The Reporting of Injuries Diseases and Dangerous Occurrences Regulations, SI 3163. Hoskuldsson, A., 1998, The Heisenberg Modelling Procedure and Application to Non-Linear Modelling, Chemometrics and Intelligent Laboratory Systems, 44: 15– 30. HSE, Factoring the human into safety: Translating research into practice. Benchmarking human and organisational factors in offshore safety. Research report 059, Accessed 12/03/2003 (www.hse.gov.uk). HSE, 2002, Strategies to promote safe behaviour as part of a helath and safety management sytem. Research report 430/2002. HSE, 1976, Success and failure in accident prevention. Accident Prevention Advisory Unit. Hunt, H.A., Habeck, R.V., Van Tol, B. and Scully, S.M., 1993, Disability Prevention Among Michigan Employers. (Upjohn Institute Technical Report No 93-004. Upjohn Institute for Employment Research. Kalamazoo, MI, USA). Jackson, J.E., 1991, A User’s Guide to Principal Components. (John Wiley). James, L.R. and Jones, A.P., 1974, Organizational climate: A review of theory and research, Psychological Bulletin, 81: 1096–1112. Jick, T.D., 1979, Mixing qualitative and quantitative methods: triangulation in action, Administrative Science Quarterly, 24: 602 –611. Kantyka, T., 1977, Industry and Society. (TNO, The Hague). Krause, T.R., 1994, Continuous safety progress focuses on ‘upstream’ factors in analyses, Occupational Health and Safety, 63: 81. Layfield, F., 1986, Sizewell B Public Enquiry (HMSO, London). Lee, T. and Harrison, K., 2000, Assessing safety culture in nuclear power stations, Safety Science, 34: 61–97. Lee, T.R., 1996, Perceptions, attitudes and behaviour: the vital elements of a safety culture, Health and Safety Bulletin, October: 1– 15. Leigh, J.P., 1986, Individual job characteristics as predictors of industrial accidents, Accident Analysis and Prevention, 18: 209–216. Leveson, H. and Hirchfield, A.H., 1980, Industrial accidents and recent events, Journal of Occupational Medicine, 22: 53– 57. Melamed, S., Luz, J., Najenson, T., Jucha, E. and Green, M., 1989, Ergonomic stress levels, personal characteristics, accident occurrences and sickness absence among factory workers, Ergonomics, 32: 1101 –1110. Mines Safety and Health Administration, 1983, Factors Associated with Disabling Injuries in Underground Coal Mines (US Department of Labor. US Government Printing Office, Washington, DC). Newton, A., 2001, Compliance is Not Enough: Getting the Ethical culture right in your firm. Securities Institute. Integrity and Ethics Committee. (Printflow, London).

Trans IChemE, Part B, Process Safety and Environmental Protection, 2007, 85(B1): 59–69

ORGANIZATIONAL CLIMATE METRICS Niskanen, T., 1994, Safety climate in the road administration, Safety Science, 17: 237– 255. Ostrom, L., Wilhelmsen, C. and Kaplan, B., 1993, Assessing safety culture, Nuclear Safety, 34(2): 163– 172. O’Toole, M., 2002. The relationship between employees’ perceptions of safety and organisational culture, Journal of Safety Research, 3: 231– 243. Pidgeon, N.F., 1991. Safety culture and risk management in organizations, Journal of Cross-Cultural Psychology, 22(1): 129–140. Pidgeon, N.F., 2004. Safety culture: transferring theory and evidence from the major hazards industries, Accessed 15 June (http:// www.dft.gov. uk/stellent/groups/dft_rdsafety/documents/page/ dft_rdsafety_504575-06. hcsp). Powell, J. and Canter, D., 1985. Quantifying the human contribution to losses in the chemical industry, Journal of Environmental Psychology, 5(1): 37– 53. Reason, J., 1990, Human Error. (Cambridge University Press, New York). Rentch, J.R., 1990. Climate and Culture: Interaction and qualitative differences in organizational meanings, Journal of Applied Psychology, 75: 668–681. Saari, J., 1990. On strategies and methods in company safety work: From informational to motivational strategies, Journal of Occupational Accidents, 12: 107– 118. Schein, E., 1985, Organisational Culture and Leadership (JoneseyBass, San Francisco). Shafiai-Sahari, Y., 1973, Determinants of Occupational Injury Experience. (Michigan State University, East Lansing, MI). Shannon, H.S., Mayr, J. and Haines, T., 1997, Overview of the relationship between organisational and workplace factors and injury rates, Safety Science, 26: 201 –217. Silvia, S., Lima, L.M. and Baptista, C.S., 2004, OSCI: an organisational and safety climate inventory, Safety Science, 42: 205– 220. Simnard, M., Levesque, C. and Bouteiller, D., 1988, L’efficacite en gestion de la securite du travail: principaux resultants d’une recherche´ dans l’industrie manufactiere. (GRASP/ sst Universite de Montreal, Montreal). Smith, L., 2004, An introduction to neural networks, accessed 19/01/ 2004 (http://www.cs.stir.ac.uk/lss/NNIntro/InvSlides.html). Sorensen, J.N., 2002, Safety culture: a survey of the state-of-the-art, Reliability Engineering and Systems Safety, 76: 189–204. Statsoft, Electronic Text Book, accessed 19/01/2004 (http://www. statsoft.com/textbook/stathome.html). Torbjørn, R., 1992, Risk perception and safety on offshore petroleum platforms—Part II: P received risk, job stress and accidents, Safety Science, 15: 53–68. Tuohy, C. and Simnard, M., 1993, The impact of joint health and safety committees in Ontario and Quebec. Prepared for the Canadian Association of Administrators of Labour Law (available from Ontario Ministry of Labour, Toronto, Ontario). UMETRICS, UMETRICS home page, accessed 28/09/2004 (http:// www.umetrics.com/). Williamson, A.M., Feyer, A., Cairns, D. and Biancotti, D., 1997. The development of a measure of safety climate; the role of safety perceptions and attitudes, Safety Science, 25: 15– 27. Wold, S., Kettaneh-Wold, N. and Skagerberg, B., 1989, Non-linear PLS modelling. Chemometrics and Intelligent Laboratory Systems, 7: 53– 56. Zohar, D., 2000, A group-level model of safety climate: testing the effect of group climate on microaccidents in manufacturing Jobs, Journal of Applied Psychology, 85(4): 587– 596. Zohar, D., 1980, Safety climate in industrial organizations: Theoretical and applied implications, Journal of Applied Psychology, 65: 96–102. The manuscript was received 30 January 2006 and accepted for publication after revision 2 May 2006.

APPENDIX 1: OVERVIEW OF THE PROJECTION TO LATENT STRUCTURES TECHNIQUE Projection to latent structures (PLS) is a method that enables the identification of linear relationships between two separate data sets X (the predictor data) and Y (the

67

predicted data). PLS is best envisaged in graphical terms. Let us first consider the X data. Further consider a situation with n objects (e.g., people) and K variables that measure aspects of the objects (e.g., height, weight). We can plot the objects and variables in a table, the rows representing the n objects, the columns representing the K variables. A K-dimensional space is then drawn and each of the objects plotted within it to form a cloud of n points. It is then possible to draw a best-fit line through the cloud of points such that the mean-square distance of all points from the line is minimized. This best-fit line is known as the first principal component (PC1). The position of each object in K space can then be projected onto the principal component line. This projection gives a new coordinate in the new coordinate system. This projection is called the ‘score’. One principal component is not usually sufficient to adequately describe a data set and therefore a second component is required. The second principal component (PC2) is drawn through the mean point of the first principal component and orthogonal to it. The first and second principal components together now represent a plane in K-dimensional space. For ease of explanation assume K ¼ 3. Figure A1 (left) is a graphical representation of a two-component plane in three-dimensional space. The position of any point in three-dimensional space can be projected onto this two-dimensional plane. Visualization of this projection allows interpretation of the inter-relationships within the data. The projection is called a score plot. The score plot will be denoted by ‘t’. A plot of t(1) versus t(2) would therefore represent a score plot of the first and second principal components. Score plots may be plotted for any pair of principal components. Objects clustered together on a score plot are related to one another. Objects positioned close to the score axis indicate those that exhibit average behaviour for that particular axis principal component. Score plots are useful in identifying outliers in the data. Outliers have significant ‘power’ or ‘leverage’ to rotate the principal components towards them. Strong object outliers are detected using the Hotelling’s T2 identification tool (Jackson, 1991). Hotelling’s T2 may be viewed as the multivariate equivalent of the Student t-test and is a check for observations adhering to multivariate normality. Hotelling’s T2 is plotted on the score plot as an ellipse. The area of the ellipse corresponds to a confidence level of 95% or 99% or any other value assigned by the user. Data points outside the Hotelling’s ellipse are defined as strong outliers. Information regarding the importance of the variables within the PCA model is obtained by examination of the loadings plot. Loadings are denoted by w
c. A plot of w
c[1] versus w
c[2] represents a loadings plot of the first and second principal component. Loadings plots may be plotted for any two principal components. Each of the points plotted on a loadings plot represents a variable. The further away the variable is from the loadings plot origin, the more important, or influential the variable is in the model. Referring to Figure A1 (right), Eriksson et al. (1999) write: ‘Another way to understand the principal component loadings is to say that they express the orientation of the obtained model plane inside the Kdimensional variable space. The direction of PC1 in relation to the original variables is given by the cosine of the angles ax1, ax2 and ax3. Of course, another set of three loadings is needed to express the direction of PC2 in relation to the original variables. Hence, with two principal components and three original variables, six loading values (cosine of

Trans IChemE, Part B, Process Safety and Environmental Protection, 2007, 85(B1): 59– 69

68

DODSWORTH et al.

Figure A1. Left—A loadings plot projection. Right—an example of a Principal Component Analysis projection. From Eriksson et al. (1999).

angles) are needed to understand how the model plane is positioned in variable space’. So, if the projected PC1, PC2 principal component plane in Figure A1 (right) is drawn perpendicular to X3, variable X3 would be found at the centre of the loadings plot. Variable X3 would, as a result, have weak model influence. Variables X1 and X2 would, however, be found far from the loadings plot origin. The distance of X1 and X2 from the loadings plot origin is determined by the model. The creation of PCA projections is sensitive to the scale of the variables being modelled. Variables that are more than one order of magnitude greater than the other model variables have ‘strength’ to rotate the projected plane toward the variable in K-dimensional variable space. To ensure that all variables have an equal chance of contributing to the model the variables are standardized prior to calculation of the principal components. Generally two separate standardisation routines, called ‘scaling’ and ‘mean centring’, are preformed. Scaling is achieved by calculation of the standard deviation of each of the variables. Each variable is then multiplied by the inverse of the standard deviation. Multiplication of the variable values by the inverse of the standard deviation ensures that all of the variables are given equal variance. The process of giving equal variance is sometimes referred to as auto-scaling. The process of autoscaling prevents any one variable dominating over others due to its numerical range. Mean centering is achieved by calculation of the average value for each variable. The average values are then subtracted from each of the variables. The process of mean centring ensures that all of the variables are ‘centred’ around a mean value of zero. Further information regarding auto-scaling is given by Eriksson et al. (1999). Useful information can be obtained by simultaneous examination of the score and loading plots for any two principal components. Objects in the score plot are characterized by: above average responses to the variables in the same region of the loadings plot and below average responses to the variables in the region on the opposite side of the loadings plot origin. If the first two principal components are insufficient to adequately describe the data then a third principal component can be added to the model to improve its predictive ability. The third principal component is similar to the first two components in that it must be oriented towards the third largest variation direction within the data, be orthogonal to the other two, and be oriented to best fit the objects in the least squares sense. At the same time the X block data is being plotted in K dimensional space, the Y data block is similarly plotted in

M-dimensional space and represented by principal components. Each point in K-dimensional space has a corresponding point in M-dimensional space. The principal components in K and M-dimensional space are then oriented to best approximate the relationship between the X and Y data. The resultant model is represented as two best-fit lines in two different dimensional spaces, the association between the lines being representative of the associations within the X and Y data. Principal components in Y dimensional space are denoted by u. The term t1 therefore denotes the first principal component of X dimensional space, u1 denotes the first principal component of Y dimensional space. Figure A2 is a graphical representation of a two-component PLS model. The ability of PLS to model the data is measured by a goodness of fit metric. The goodness of fit metric is denoted by ‘R 2’. R 2 is a measure of the models ability to reproduce the data in the training set (Eriksson, 1999). R 2 varies from 0 to 1. A value of R 2 ¼ 0 indicates that the model does not fit the data at all. A value of R 2 ¼ 1 indicates that the model fits the data perfectly. The goodness of fit of the resultant model increases as the number of principal components is increased. If sufficient numbers of principal components are used then the goodness of fit will approach unity. Increasing the number of principal components to achieve unity goodness of fit is counterproductive in that the resultant model may be difficult to interpret due to the large number of variables. The predictive ability of the model is denoted by ‘Q 2’. Q 2 varies from 0 to 1. Q 2 is the fraction of the total variation of the data that can be explained by a principal component. A value of Q 2 ¼ 1 indicates that the model is able to predict the data perfectly. A value of Q 2 ¼ 0 indicates that the model is unable to predict the data. Q 2 differs from

Figure A2. A representation of a two-component PLS model, from Eriksson et al. (1999).

Trans IChemE, Part B, Process Safety and Environmental Protection, 2007, 85(B1): 59–69

ORGANIZATIONAL CLIMATE METRICS R 2 in that it does not approach unity as the number of principal components is increased. Values of R 2 and Q 2 are produced for the X and Y block data e.g., R 2X represents the model’s ability to account for the X block variation, R 2Y represents the model’s ability to account for the Y block variation. The optimal number of principal components can be defined by the calculation of Q 2. Q 2 increases with the number of components up to a maximum, beyond which the predictive ability of the model does not increase. The number of principal components required is calculated from the point at which Q 2 is maximized. The ability of the PCA to model R 2 and Q 2 with each successive component can be assessed by inspection of a model overview plot. The ordinate of the model overview plot represents the R 2 and Q 2 value for each component varying in all cases from between 0 and 1. The abscissa of the model overview plot is the number of principal components. The model overview plot provides cumulative R 2 and Q 2 values for each successive model principal component. The plot has two main uses: firstly, it allows the researcher to view how Q 2 varies with increasing number of principal components and therefore what the optimal number of components is, and secondly, it provides information regarding how much of the variation is accounted for by each component. How well the model is able to represent the data is dependent upon the association between the X and Y data. The stronger the relationship between the X and Y data blocks, the better the model will be able to represent the data. PLS is a parametric method. It assumes that both the X and Y data blocks are approximately normally distributed. According to Eriksson et al. (1999), PLS works best when the data are fairly symmetrically distributed and have a fairly constant error variance. The requirement for constant error variance means that variables that range more than a factor of 10 require logarithmic transformation prior to analysis. Non-normal distributions of either X and Y block require transformation prior to PLS

69

analysis. PLS also requires the relationship between X and Y data to be linear. PLS computer software packages, such as SIMCA Pþ (UMETRICS, 2004), have diagnostic tools that allow the identification and transformation of non-linear relationships between the X and Y data blocks. Further information regarding non-linear PLS modelling can be found in Hoskuldsson (1998) and Wold et al. (1989). SIMCA Pþ has a built-in validation function. When the response permutation validation function is initiated, the X block is left intact whilst the Y block is randomly re-ordered. A new PLS model is computed for the re-ordered data and corresponding values of Q 2 and R 2Y are calculated. Values of Q 2 and R 2Y from the original model are compared with those of the re-ordered model. If the original model resulted by chance then one would expect the re-ordered Q 2 and R 2Y values to be close to the original model Q 2 and R 2Y values. If the original model was valid, i.e. not created by chance, one would expect the values of Q 2 and R 2Y of the re-ordered models to be significantly less than the original model Q 2 and R 2Y values. This process of random shuffles of the Y block and recalculation of Q 2 and R 2Y is repeated a number of times (set by the user up to a value of 999). For each permutation the Pearson correlation coefficient is calculated between the original and the permuted Y data. If the process of re-ordering does not significantly perturb the model then the Pearson correlation coefficient will be close to unity. If the process of re-ordering the Y block gives rise to a large difference between the original and permuted value then the Pearson correlation coefficient will approach zero. Q 2 and R 2Y are plotted against the Pearson correlation coefficient for each of the re-ordered models and a best-fit line drawn through the set of Q 2 and R 2Y values. According to Eriksson et al. (1999) a valid model has a R 2Y ordinate intercept no greater than 0.3–0.4 and the Q 2 intercept should not exceed 0.05.

Trans IChemE, Part B, Process Safety and Environmental Protection, 2007, 85(B1): 59– 69