- Email: [email protected]
Quantitative assessment of occupational safety and health: Application of a general methodology to an Italian multi-utility company
Safety Science 72 (2015) 75–82
Contents lists available at ScienceDirect
Safety Science journal homepage: www.elsevier.com/locate/ssci
Quantitative assessment of occupational safety and health: Application of a general methodology to an Italian multi-utility company A. Saracino a, G. Antonioni a,⇑, G. Spadoni a, D. Guglielmi e, E. Dottori b, L. Flamigni c, M. Malagoli c, V. Pacini d a
DICAM, Dipartimento di Ingegneria Civile, Chimica, Ambientale e dei Materiali, Università di Bologna, via Terracini n.28, 40131 Bologna, Italia HERA, Group Director of Quality, Safety and Environment, V.le C. Berti Pichat 2/4 – 40127 Bologna, Italy c HERA, Quality, Safety and Environmental Management System, V.le C. Berti Pichat 2/4 – 40127 Bologna, Italy d NIER Ingegneria, Via Bonazzi, 2 – 40013 Castelmaggiore (Bo), Italy e Dipartimento di Scienze Dell’Educazione Giovanni Maria Bertin, Alma Mater Studiorum, Università di Bologna, Italy b
a r t i c l e
i n f o
Article history: Received 21 March 2014 Received in revised form 2 July 2014 Accepted 19 August 2014
Keywords: Occupational health and safety Occupational safety assessment Quantification Workplace safety
a b s t r a c t The core of safety practice is the occupational safety risk assessment, which is quite often a complex process since it requires to take into account parameters that are often difficult to quantify. The European Council Directive 89/391/EC is concerned with the introduction of measures to encourage improvements in the Occupational Health and Safety (OHS). Among other issues, it deals with risk assessment and preventive measures. In this framework a novel methodology named M.I.M.O.SA. (Methodology for the Implementation and Monitoring of Occupational SAfety) has been developed with the aim of quantify the occupational health and safety level of a company and thus of its OHS Management System (OHSMS). The methodology was born within a project where a multi-disciplinary team discussed the main topics involved in occupational safety and finally a global index, which takes into account, among others, both technical and organizational aspects, was defined. In this work after a brief description of the key concepts of this methodology, it has been applied to one activity of an Italian multiutility company in order to test its applicability to a real case study. Once the performance of the OHSMS was assessed, thanks to the quantification of the different aspects that concur to its global level, it has been possible to identify the priority of interventions for improvements, so that the management process becomes more effective and efficient. In addition, from this first application of the methodology some issues emerged that could be useful for its future improvement. Ó 2014 Elsevier Ltd. All rights reserved.
1. Introduction Prevention of occupational accidents is an important task of the OHSMS of a company, in fact the main aim of the ‘‘Framework Directive’’ of the European Union about OHS (EEC 391/1989) is to introduce measures to encourage improvements in the safety and health of workers at work and it applies to all sectors of activity. In recent years the European Occupational Safety and Health Agency (EU-OSHA), which collects, analyses and disseminates information related to occupational safety and health across the EU, along with six national safety and health organizations, participates in a consortium that has gathered together many risk assessment tools and methodologies available to help enterprises and organizations assess their health and safety risks. The choice of ⇑ Corresponding author at: DICAM, Dipartimento di Ingegneria Civile, Chimica, Ambientale e dei Materiali, Università di Bologna, via Terracini n.28, 40131 Bologna, Italia. http://dx.doi.org/10.1016/j.ssci.2014.08.007 0925-7535/Ó 2014 Elsevier Ltd. All rights reserved.
method will depend on workplace conditions, for example the number of workers, the type of work activities and equipment, the particular features of the workplace and any specific risks. Moreover a great number of European Standards are related to the importance of health safety at work. For instance OHSAS 18001 (BSI, 2007), is a British Standard for occupational health and safety management systems and it is widely seen as the world’s most recognized standard for Occupational Health and Safety Management Systems (OHSMSs). In general, the most common risk assessment tools are checklists, which are a useful tool to help identify hazards. Other kinds of risk assessment tools include: guides, guidance documents, handbooks, brochures, questionnaires, and ‘interactive tools’ (free interactive software, including downloadable applications which are usually sector-specific). These tools can be either generic or branch/risk-specific. Nevertheless only mainly sector-specific approaches for the quantification of occupational risk have been proposed (e.g., Murè and Demichela, 2009; Papadakis and Chalkidou, 2008;
76
A. Saracino et al. / Safety Science 72 (2015) 75–82
Table 1 MIMOSA key elements and themes with number of questions of the checklists and summary of the results for the case-study. Questions of checklist
Not applicable
Applicable
‘‘YES’’ answers
‘‘NO’’ answers
Partial answers
‘‘NO’’ + ‘‘partial’’
Key elements
Themes
Planning [P] acting [A]
Key element 1: leadership and consistency of targets
Organization and structure of responsibilities Direct involvement of the management Management of economic resources
P A
5 7
0 0
5 7
5 7
0 0
0 0
0 0
P A P A
6 6 4 3
0 0 0 2
6 6 4 1
4 3 1 1
0 0 0 0
2 3 3 0
2 3 3 0
Risk assessment
P A P A
13 15 2 5
0 0 0 0
13 15 2 5
12 13 2 5
1 1 0 0
0 1 0 0
1 2 0 0
P A P A P A P A P A P A P A P A P A
12 17 9 9 2 2 6 9 9 9 9 9 11 5 5 5 3 8
1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
11 17 9 9 2 2 6 9 9 9 9 9 11 5 5 5 3 8
11 17 6 8 2 2 6 9 9 9 9 9 11 5 2 2 3 7
0 0 0 0 0 0 0 0 0 0 0 0 0 0 3 3 0 0
0 0 3 1 0 0 0 0 0 0 0 0 0 0 0 0 0 1
0 0 3 1 0 0 0 0 0 0 0 0 0 0 3 3 0 1
P A P A P A P A
3 8 6 5 6 6 14 14
0 0 0 0 0 0 0 0
3 8 6 5 6 6 14 14
2 6 3 1 4 5 10 8
0 1 1 2 1 1 4 6
1 1 2 2 1 0 0 0
1 2 3 4 2 1 4 6
P A P A
5 9 16 17
0 1 0 0
5 8 16 17
5 7 4 5
0 0 9 10
0 1 3 2
0 1 12 12
Compliance with sector-specific formal requirements Formal validity of general requirements Reporting system
P A
12 15
3 3
9 12
9 12
0 0
0 0
0 0
P A P A
1 5 3 3
0 0 0 0
1 5 3 3
1 4 2 3
0 0 0 0
0 1 1 0
0 1 1 0
Human resources
P A P A P A A
45 50 4 6 5 5 5
0 0 0 0 0 0 3
45 50 4 6 5 5 2
36 36 4 6 4 4 2
3 8 0 0 1 1 0
6 6 0 0 0 0 0
9 14 0 0 1 1 0
Key element 2: orientation to risk reduction and people protection in compliance with the law
Measures of prevention and protection Education, training and communication Participation Risk monitoring Events (near misses) monitoring Health supervision Emergencies Contracts and subcontractors Safety levels improving Alertness at work
Key element 3: involvement, learning and development of individual culture
Safety climate Risk perception Open communication Rewarding system for safety
Key element 4: continuous improvement and innovation
Control system Comfort and development of human resources
Key element 5: formal and general compliance
Key element 6: social responsibility
Ethical and institutional aspects Voluntary certifications Environment
Aneziris et al., 2010; Pinto et al., 2011) and also those that are related to the assessment of the OHSMS of a company (e.g., Bellamy et al., 2008; Papadakis, 2000) are rather sector-specific. For this purpose, a methodology and a related index have then been developed within a specific project (FAM, 2012), where a number of actors with different background knowledge and experience were involved. On the basis of their experience, numerical values were assigned to checklists and indicators and thus a global index could be defined (Saracino et al., 2012). Despite the wide experience of the involved professionals, the methodology still
needs to be tested on a real situation in order to highlight its possible criticalities and its strengths and thus to improve it for future applications. 2. Fundamentals of the methodology M.I.M.O.SA. (Methodology for the Implementation and Monitoring of Occupational SAfety) is a methodology that has been introduced in order to allow the evaluation of the performance of a company concerning health and safety in the workplaces (FAM,
A. Saracino et al. / Safety Science 72 (2015) 75–82
77
Fig. 1. Tree structure of M.I.M.O.SA. methodology.
2012). In the methodology, built with the purpose of the self-assessment of the performance for both big and small scale enterprises, six so-called main ‘key elements’ are defined (see first column of Table 1). They allow building and implementing an appropriate and efficient OHSMS. Three categories of OHSMSs are introduced by MIMOSA. They derive from the legal requirements and from technical standards of different sectors (as mentioned in the introduction there can be different systems for different branches/activities), because the goals that each company expects to achieve through the implementation of a management system can be different from each other, and since OHSMSs can range from the simple compliance with law to continuous improvement. In short, the MIMOSA methodology identifies the following classes of management systems: – OHSMS derived from the Italian legislation on health and safety in the workplace (DL 81/2008). – OHSMS developed on the basis of national and international standards, developed by technical-scientific and regulatory agencies such as OHSAS 18001 (BSI, 2007), or guidelines for their implementation (BSI, 2008). – OHSMS derived from the Italian legislation on the liability of the companies for occupational safety (DL 231/2001). Starting from these three classes, MIMOSA methodology should be implemented under three different perspectives of analysis for the evaluation of the health and safety level of a company, namely: – The perspective of mandatory compliance with law. – The perspective of continuous improvement (e.g., OHSMS that has obtained or is trying to obtain the OHSAS 18001 certificate) once the compliance with law is guaranteed. – The perspective of crime prevention. 2.1. The structure of MIMOSA The tree structure of the methodology is presented in Fig. 1: at the top there are the key elements that are described through a
different number of themes, and then each theme is derived from the characterization of checklists and/or Key Performance Indicators (KPIs), which together form the ‘‘leaves’’ of the tree. Each key element is detailed through a number of specific themes and for each theme different checklists and KPIs have been developed for the assessment of that theme in a typical tree-structured hierarchy (see Fig. 1), in particular checklists are divided into planning (P) and acting questions (A). The quantification of each checklist and of each KPI is clearly related to a specific theme, as a consequence the tree structure of the M.I.M.O.SA. methodology is fully defined. The use of checklists and KPIs should guarantee that the assessment of the company’s performance in safety is achieved through an objective, though uncertain, analysis and owing to this peculiarity MIMOSA gives a help in directing improvement interventions.
2.2. Key elements and themes allocation The six key elements of the self-assessment methodology are (see also first column of Table 1): 1. Leadership and consistency of the targets. 2. Orientation to risk reduction and people protection, in compliance with the law. 3. Involvement, learning and education development of the personnel. 4. Continuous improvement and innovation. 5. Formal and general compliance. 6. Social responsibility. The first key element evidences that a proper leadership of the company communicates the importance of adequate requirements for health and safety in the workplace. The leadership exercised by the top management influences the behaviors of the members of the company. It consists in the deep interaction between employees/workers and/or executives/managers (or in general members in the higher company positions and with organizational responsibility).
78
A. Saracino et al. / Safety Science 72 (2015) 75–82
Table 2 Planning and acting checklists for the assessment of safety climate. Planning checklist 1 Has the company provided the appropriate investigative tools (such as surveys, interviews, and focus groups) about how the people perceive the value and the priority given to targets (e.g., ranking of goals, timetable and rate of production, continuous improvement) in relationship with safety in the various business activities? 2 If pertaining, is the discussion of safety issues scheduled in the agenda of the management meetings? 3 Are organizational interventions and initiatives planned by the company for the promotion and the active care of safety? (e.g., guidance actions for new recruits to facilitate their socialization with the safety regulations and procedures; mentoring and/or coaching initiatives by the appointed responsible for safety and by senior workers. . .) Acting checklist 1 Has the company implemented the appropriate investigative tools (such as surveys, interviews, and focus groups) about how the people perceive the value and the priority given to targets (e.g., ranking of goals, timetable and rate of production, continuous improvement) in relationship with safety in the various business activities? 2 Is the senior management directly and publicly involved in safety programs, with explicit information to the workforce of their commitment? (e.g., constant attendance at periodical meetings; direct and personal commitment in the risk-assessment procedures; drafting of safety procedures. . .) 3 Is the role of Safety Managera assigned to an executive manager? 4 Is the Safety Manager involved in meetings with the management where safety targets are planned? 5 Are the considerations and the suggestions of the Safety Manager regarded as key items during the management meetings 6 Are the proposals of the Safety Manager put in action? 7 Does the management require to the appointed persons in charge the improvement of safety? 8 Does the management carry out awareness campaigns of appointed persons in charge in order to encourage the involvement of all workers (e.g., guidance actions for new recruits, mentoring or coaching actions, etc.)? a
According to the definition of the UK Health and Safety Executive.
The second, the third and the fourth key elements take into account those activities that have the greatest impact on the level of health and safety (even without the formalization of a real model). In the second key element the attention is addressed to the requirements that can be important in determining features of the workplace, which have a great influence on the levels of health and safety of workers, so that the compliance with the law is substantial. The third key element emphasizes that all human resources operating in the enterprise play a fundamental role in the organization. The fourth key element highlights the importance of continuous improvement that is achieved through the well-known PDCA (Plan, Do, Check, Act) cycle, consisting of 4 continuous stages of action and verification (Deming, 1986). The fifth element is defined on the basis of some formalities required by the Italian law: they are particularly important if the target is to measure the effectiveness in exempting from the administrative liability. Finally, the sixth key element wants to underline the awareness of the companies about social issues and environmental sustainability (Saracino et al., 2012). The total number of themes in the MIMOSA system is 27, they are all listed in Table 1 (second column) and they are described in detailed elsewhere (FAM, 2012). Themes are partitioned into the six key elements as follows: three themes for key element 1, eleven themes for key element 2, four themes for key element 3, two themes for key element 4, three themes for key element 5, and four themes for key element 6. As can be easily seen the second key element has the highest number of themes and this will be important in the quantification process.
2.3. Quantification of KPIs, checklists and overall safety index The assessment procedure starts with the quantification of each checklist and of each KPI that are our operative aspects of the methodology. For the sake of brevity among them only two, though important, items are described in order to better explain the key aspects of the methodology. As an example of checklists, the two checklists (planning and acting), related to the evaluation of the theme ‘‘safety climate’’ (see Table 1), are described through the questions they contain, 3 planning questions and 8 acting questions, which are reported in Table 2. Safety climate is one of the themes that concur to the description and quantification of the key-element 3: ‘‘involvement, learning and development of individual culture’’ and moreover, all the questions, in both checklists, were applicable in the case-study described in the following. The standard format drawn up for each indicator is formed by the following seven fields: 1. definition of the indicator; 2. target of the indicator; 3. theme and key element to which it belongs; 4. quantification methodology; 5. reference indicator values; 6. laws, rules and other references; and 7. type of company. These fields must be filled in for each indicator. Points 1–3 and 6 are self-explanatory and they do not require other specific description. Points 5 and 7 define respectively which values are considered valid for a positive self-evaluation and for which types of companies (small, medium or big) the indicator can be used. Finally, the person in charge of filling in the form has only to calculate the indicator at point 5, on the basis of its description reported in point 4. As an example, in Table 3 is reported the format for
Table 3 KPI for the investigation on accidents. Indicator
Formal investigation on accidents
Definition
Number of accidents for which a formal investigation has been carried out (root cause), divided by the number of occurred accidents The aim is to facilitate the identification of the injury’s causes and to support the implementation of corrective actions Theme: measures of prevention and protection Key element: orientation to risk reduction and people protection in compliance with the law A = total number of accidents with formal investigation B = total number of accidents Safety threshold: not defined ISafety ¼ AB Not applicable Big and medium companies
Target Theme and key element of belonging Quantification methodology Reference indicator values Law, rules and other references Type of company
A. Saracino et al. / Safety Science 72 (2015) 75–82
one of the fourteen indicators used for the quantification of the theme ‘‘Measures of prevention and protection’’ belonging to the second key-element ‘‘orientation to risk reduction and people protection, in compliance with the law’’ (FAM, 2012), which was also used in the case-study. Once that each checklist and each KPI has been quantified, by properly weighting their values, a single numeric value can be obtained for each theme and thus in turn for each key element and finally for the company (or for a single activity of a company) as a possible representation of the overall performance of the company’s OSHMS. This value has been called IPESHE (Index of PErformance for Safety and HEalth). In other words the quantification of the methodology requires to evaluate the scores of each checklist and of each KPI in order to obtain, through a weighted sum, the score of each theme subsequently, through another weighted sum, each key element can be quantified and finally the value of the IPESHE index is obtained as shown in Eq. (1).
IPESHE ¼
N ke X
N t ðiÞ nKPI=cklðjÞ X X tj pk I k
i¼1
j¼1
xi
ð1Þ
k¼1
where Nke is the number of key elements (six), Nt(i) is the number of themes that contribute, with their relative weights tj, to key element i, whose weight is xi. Then nKPI/ckl(j) is the number of KPIs and/or checklists belonging to theme j and pk is the weight of each KPI or checklist k. Ik is the calculated value of a KPI or of a checklist (0 < Ik < 1). As a consequence, the IPESHE index will range between 0% and 100% and the greater will be its numerical value the greater the health and safety level for the workers of the company’s OHSMS. It is worth noting that the quantification is carried out by means of checklists and indicators, which are tools of common, though not exclusive, use tools, for the quantitative assessment of occupational risk through index methods (Kongsvik et al., 2010). 2.4. Weighting criteria As a first attempt, MIMOSA methodology considered two criteria for the calculation of IPESHE, called priority-criterion and equality-criterion respectively. They are intended to represent two different ways of evaluation. The equality-criterion assumes that all the indicators and the checklists give the same contribution to the theme that they belong (pk = 1/nKPI/ckl(j)), and thus the weight of each theme can be calculated through Eq. (2), as the ratio between the number of KPIs/checklists of that theme and the total number of KPIs/checklists of all the themes within a key-element. Finally also the weights of each key element, xi, are the same (i.e., 1/6).
nKPI=ckl ðjÞ t j ¼ N ðiÞ t X nKPI=ckl ðjÞ
ð2Þ
j¼1
These assumptions should guarantee that each aspect of the methodology has the same importance, regardless of its actual content, since it is required by the in-force law or by international standards. Thus, the index resulting from the application of these weights is intended to provide a measure of the formal safety obtained through the implementation of the OHSMS. In the priority-criterion, different weights, discussed on the basis of the expert judgment within the team involved in the project, are assumed for each key element, but still equal weights for each indicator/checklist within a theme (pk) are considered and thus the weight of each theme within a key element (tj) can be calculated again by means of Eq. (2). This second criterion should quantify the safety level of a company on the basis of the relative
79
importance of some key elements, and of their related themes and KPIs/checklists, with respect to others and thus it should provide a measure of the actual safety level that can be obtained by the company’s OHSMS. 3. Application of the methodology The methodology has been first tested through its application to an activity of an Italian company, which is responsible for the management of all road vehicles of the company including maintenance and reparation in full service mode throughout the complete life cycle. Thus the important health and safety issues in an automotive repair and maintenance service activity are exposures to chemicals, biological materials, vehicle exhaust, fatalities or injuries from contact with objects or equipment, especially struck by falling objects, transportation events, and fires and explosions. There are about 160 employees that works in several garages located within Italian region Emilia-Romagna. The OHSMS of the company has been certified in 2012 according to the BS OHSAS 18001:2007. Some adaptations have been introduced in order to better fit the methodology to the enterprise under investigation; in Section 3.1 the methods through which questions of the checklists and KPIs have been modified are described. Finally, the tools used, the assumptions introduced in the test, the results obtained and the limitations found in the application of MIMOSA are highlighted with the aim to put in evidence some possible improvements of the methodology, which can well overcome difficulties in its application. 3.1. Modifications to checklists In order to answer to the questions of the MIMOSA checklists, in this application two different approaches have been used. They have been called ‘‘direct interview’’ and ‘‘indirect interview by group membership’’. 43% of the questions were evaluated by means of the first approach and clearly the remaining 57% have been answered through the second approach. The two approaches are explained below: ‘‘Direct interview’’: the answers to the questions are given by a direct interview to different interlocutors, chosen on the basis of their expertise about the topic of the question. For instance, for checklists related to the theme ‘‘risk assessment’’ the Safety Manager has been selected, while for checklists related to the theme ‘‘direct involvement of the management’’ executives have been interviewed. ‘‘Indirect interview to group membership’’: the answers to the questions in the checklists were obtained indirectly by the evaluator, thanks to the deep knowledge of the group to which the company belongs. Indeed the group defines some elements of the management system (e.g., organizational procedures, policy of the company, initiatives of corporate social responsibility, etc.) that must be acquired also by the company to which they belong. These elements are defined at the corporate level and they are supported by documentation that is accessible to the evaluator (e.g., policy, budget of sustainability, etc.). The methodology requires only a YES/NO answer and does not consider the impossibility to answer, giving to some questions a null solution. An example is the question: ‘‘if the company is small (up to 50 employees), was an external subject put in charge of the surveillance institute?’’ which is clearly not applicable to a company with about 160 employees, as in the case under investigation. To overcome this problem, the specific answer ‘‘not applicable
80
A. Saracino et al. / Safety Science 72 (2015) 75–82
(NA)’’ is introduced for questions like the previous one and similar questions are excluded from the IPESHE calculation (rescaling relative weights). However, only 2.75% of the total number of questions was excluded due to a ‘‘NA’’ answer in this application. The summary of questions and answers is presented in Table 1, where the total number of ‘‘No’’, ‘‘YES’’ or ‘‘NA’’ answers for each theme (and for each key element) is reported. The set of all answers represents the raw output of the methodology and it is the basis for the calculation of the IPESHE index of the company, which, as previously mentioned, must reflect the level of health and safety of the company. 3.2. The use of indicators In order to assign a numerical value to each KPIs two different methods of calculation were used. Indicators were first subdivided into three categories (see Table 4), on the basis of the difficulties (or sometimes of the impossibility) observed to enumerate or to determine the factors involved in their quantification. The three categories of indicators were identified as follows: 1. Quantitative indicators: for this type of indicators, the value is obtained from the accurate calculation of the elements which concur to their definition, following the instructions on a form. 2. Semi-quantitative indicators: some elements required for the calculation are not managed in a systematic manner (e.g., the records of checks and maintenance operations related to equipment, fire prevention and PPE, etc.) but more frequently they are collected and kept locally in a paper-based archive. For example, the calculation of the indicator ‘‘fire-prevention/emergency’’ requires the knowledge of the number of inspections actually carried out on fire/emergency devices and of the number of expected inspections. Being managed locally, these data are not aggregated for the whole company and therefore they cannot be gathered globally. However, the checks on the fire
Table 4 Classification of indicators. Quantitative KPIs
Semi-quantitative KPIs Not applicable KPIs
Low level of criticality Medium level of criticality High level of criticality Ik = 1 unless irregularities Not applicable indicators
Easy interpretation Assumptions applied New interpretation Not completely quantitative Incompatible with activity under examination
protection measures and on emergency devices are subjected to periodic audits, as required by the certification OHSAS 18001:2007. For this reason, unless there are irregularities detected during the audit, it is assumed that the required checks are regularly carried out and the value of the indicator is 1. Note that, however, this simplification causes a sort of difficulty in the comparison of indicators, since the semi-quantitative ones are lower than 1 only if irregularities appear during the audits. 3. Not applicable indicators: some of the indicators (as it happened for checklists) cannot be applicable to every industrial or manufacturing activity (e.g., some indicators related to the observation of the behavior of the employees that shows a scarce commitment to the company). Furthermore each quantitative indicator has been classified according to the criticality level detected during its application (low–medium–high). The levels of criticality are reported in Table 4: a low level of criticality was assigned to those KPIs that were found to be easily understood by the surveyed, a medium level for those that required some interpretative assumptions and finally a high level of criticality to those that have been misunderstood and thus reinterpreted by the answerers. In this application about 37% of the indicators has not been valued, because they were not applicable and, in addition, 20 out of 54 indicators (again 37%) belongs to the class of quantitative indicators, which have been measured punctually (see Fig. 2) among them about a half showed a low level of criticality for their quantification while the remaining 50% showed a medium or high level of criticality (Fig. 2).
4. Results and discussions The application of MIMOSA methodology to the Italian company lead to two different values of IPESHE index when calculated on the basis of Eq. (1) and of two different weighting criteria as described in Section 2.4. When the equality criterion was selected an index of 81% was obtained, while when the priority criterion was adopted the index was 72%. The numerical results are also graphically depicted in Fig. 3, where also the contribution to the overall index of different key elements of the methodology can be seen. The results show that the OHSMS implemented by the company in this activity resulted in a good performance both when looking at the formal safety level (equality criterion) and when considering the greater importance of the first key-elements (priority criterion). Different considerations can be drawn from the application of these two criteria. In the case of the priority criterion, the highest
Fig. 2. Classification of the indicators.
A. Saracino et al. / Safety Science 72 (2015) 75–82
81
several checklists and KPIs, this high number of base elements greatly affects the contribution of this key-element in the overall value of the index if the equality criterion is applied. But if the priority criterion is selected, its importance in the IPESHE value decreases from about 67% to less than 30% (see Fig. 4) and the same reasoning can be applied to the other key elements as shown in Fig. 4. Thus more attention should be paid to these two keyelements since they largely contribute to the overall index even when trying to measure the actual performance, because results obtained in this case-study so far show that targets required by the law, or required in order to obtain the OHSAS 18001:2007 certification, have been reached, but some aspects related to the leadership of the management towards safety and to actively reducing the risk can still be improved in order to obtain the same score when applying different weighting criteria.
Fig. 3. Contributions of key elements to global IPESHE index.
5. Conclusions 5.1. Drawbacks and improvements of the methodology
Fig. 4. Relative importance of each key element for different criteria.
number of checklists and of indicators of some themes with respect to others bear only slightly down on the determination of the global index and, consequently, they play roles of limited importance whatever are their numbers in the key-elements; instead, if equality criterion is considered, a high importance is given to the number of indicators and checklists belonging to different themes and key-elements. In other words, the application of the two criteria has two different meanings: while the equality criterion is applied to mainly test the formal compliance against all the requirements provided by national (or other) in force law about OHS, the priority criterion could better attest the real safety level of a company, since it assumes decreasing weights for the key elements of the methodology (in the order they are listed in Table 1). As mentioned in Section 2.2, the requirements that have a greater impact on the protection of workers and thus mainly contribute to effective risk reduction, belong to the first key elements and thus these elements should have higher weights regardless of the number of checklists and/or indicators used for their quantification. In Fig. 3 it can be seen how the weights of the key elements affect the value of the global index in two different manners. For example the second key element (orientation of the company to risk reduction and people protection in compliance with the law) is described through 11 themes (see Table 1, second column) and thus through
The application of the methodology to an actual case study has highlighted some critical issues that require some investigation. First some difficulties have arisen form the only two possible answers (‘‘YES’’ or ‘‘NOT’’) to the questions of the checklists. This sort of on/off assessment penalizes a number of questions that can have partial positive or partial negative answer. Trying to take into account these cases, these questions have been tracked on the basis of the comments of the people interviewed and they resulted to be about 9% of the applicable questions. It should be noted that in the calculation of IPESHE index, answers that should have been evaluated only partially, were considered as negative. Thus one of the changes that could be introduced in the methodology is the possibility of intermediate answers (i.e., a ‘‘maybe’’ or better ‘‘more NO than yes’’ and ‘‘more YES than no’’). This would allow to obtain a more accurate value of the global index and also to take into account the possible uncertainty of the people interviewed. The second problem concerns checklists and indicators that can be considered not applicable to a given situation under analysis, because the non-applicability of the questions and indicators has important effects in the calculation of IPESHE, whose assigned weights depend on the number of checklists and KPIs belonging to different themes and key elements (at least if the equality criterion is selected, see Section 2.4). For the present study not applicable checklists and indicators were excluded in the IPESHE calculation, but this has highlighted the requirement for a customization and adaption of the methodology MIMOSA to a specific company. The customization makes the comparison among companies difficult, because each one of them can have a different number of not applicable items. Consequently, the methodology should investigate the criteria for the evaluation of these elements, through the deletion from some checklists of questions that do not have a general validity and eventually through the revision of the calculation method of some indicators. Another critical issue is the impossibility to verify the objectivity level of the answers to each question in a checklist, especially to those that were obtained by a direct interview. The questions, if not correctly formulated, can be misunderstood due to the subjectivity of the interlocutor or of the auditor. To overcome this criticality some evidences could be attached to the answers requiring further justification. These evidences can include for instance: – Internal audits, – Recordings, – Information that can be acquired from interviews and field surveys, etc.
82
A. Saracino et al. / Safety Science 72 (2015) 75–82
An evidence allow to assess the quality of an answer and to improve its objectivity level. For instance, in order to demonstrate that the correct answer to the question: ‘‘has been implemented the inspection of PPE?’’ is yes, a tailored checklist, developed by the category associations rather than by the company itself, though based on national or international guidelines, could be periodically applied. Clearly the auditor itself will have to judge in turn also the quality of the gathered evidences through a comparison e.g., with common standards and practices for that industrial activity. For this reason the methodology should also define criteria to adopt in order to give an objective judgment about the evidences, also because a company could supply a lot of evidences to support the same question. As outlined in Section 2, the methodology has been developed keeping in mind that three different perspectives can be adopted. Nevertheless attention should be paid to weighting criteria, possibly taking into account also evidences, in order to clearly evidence those checklists (or even single questions) and KPIs that mainly contribute to each perspective. Finally critical points about the approach of ‘‘direct interview’’, used to answer to some questions, have to be considered. As already mentioned, through this approach, different interlocutors of the company, identified on the basis of their expertise about the topic and to their role within the company, were asked to answer to the same questions. Thus the answers to these questions have been compared between each other and with some evidences, gathered by means of some field inspections, in order to crosscheck whether there could be inconsistency or not. Some inconsistencies actually appeared (e.g., for prevention and protection in activities where chemicals are handled, noise pollution is present and moving mechanical parts can harm the workers). This is primarily due to the level of subjectivity which is typical of these kind of surveys because different interlocutors (workers/employees and Safety Manager) can perceive differently the same question. When inconsistencies were found and there was no evidence to support one alternative, for the calculation of IPESHE the following ranking was assumed: (1) employer, (2) Safety Manager, (3) executive, and (4) staff (information systems and other personnel). However different results could be obtained if a different ranking had been assumed and also if other questions. Note that, extending all questions through interviews to several people and expanding the collection of evidences on the field to all the themes of the key elements, different results could be obtained, for this reason the methodology should provide in general more stringent criteria for the definition of the questions (and of the modality to their answers). 5.2. Final conclusions In this paper a first testing of the MIMOSA methodology was presented. This application helped to explain and clarify the way
the methodology is applied in order to assess the safety level of a company (or of a branch of it). The methodology was applied to the branch of a large multi-utility company operating in northern Italy, which manages the whole company’s fleet of road vehicles. This study shows that the methodology needs to be improved in some points, even if its results can be already used by the executives in order to manage and to improve the safety at work. The application is also intended to be a first guideline for the application of MIMOSA methodology and for suggesting some improvements to it. References Aneziris, O.N., Papazoglou, I.A., Kallianiotis, D., 2010. Occupational risk of tunneling construction. Saf. Sci. 48, 964–972. http://dx.doi.org/10.1016/j.ssci.2009.11.003. Bellamy, L.J., Geyer, T.A.W., Wilkinson, J., 2008. Development of a functional model which integrates human factors, safety management systems and wider organisational issues. Saf. Sci. 46 (3), 461–492. http://dx.doi.org/10.1016/ j.ssci.2006.08.019. British Standards Institution (BSI), 2007. Occupational health and safety management systems – requirements. OHSAS 18001:2007. British Standards Institution (BSI), 2008. Occupational health and safety management systems – guidelines for the implementation of OHSAS 18001:2007. OHSAS 18002:2008. Council Directive (EEC) 391/1989 of 12 June 1989 on the introduction of measures to encourage improvements in the safety and health of workers at work. OJ L183. Deming, W.E., 1986. Out of the Crisis. Cambridge University Press, UK. Fondazione Alma Mater (FAM), 2012. MIMOSA, Metodo per Implementare, Misurare e Organizzare la Sicurezza in Azienda. Il Mulino. In Italian. ISBN 978-88-15-24113-9. Kongsvik, T., Almklov, P., Fenstad, J., 2010. Organisational safety indicators: some conceptual considerations and a supplementary qualitative approach. Saf. Sci. 48 (10), 1402–1411. http://dx.doi.org/10.1016/j.ssci.2010.05.016. Legislative Decree (DL) 231/2001 of 8 June 2001. Disciplina della responsabilità amministrativa delle persone giuridiche, delle società e delle associazioni anche prive di personalità giuridica. GU 140. Legislative Decree (DL) 81/2008 of 9 April 2008. Attuazione dell’articolo 1 della legge 3 agosto 2007, n. 123, in materia di tutela della salute e della sicurezza nei luoghi di lavoro. GU 101. Murè, S., Demichela, M., 2009. Fuzzy application procedure (FAP) for the risk assessment of occupational accidents. J. Loss Prevent. Process Ind. 22, 593–599. http://dx.doi.org/10.1016/j.jlp.2009.05.007. Papadakis, G.A., Chalkidou, A.A., 2008. The exposure-damage approach in the quantification of occupational risk in workplaces involving dangerous substances. Saf. Sci. 46, 972–991. http://dx.doi.org/10.1016/j.ssci.2007.11.009. Papadakis, G.A., 2000. Assessment of requirements on safety management systems in EU regulations for the control of major hazard pipelines. J. Hazard. Mater. 78, 63–89. http://dx.doi.org/10.1016/S0304-3894(00)00217-X. Pinto, A., Nunes, I.L., Ribeiro, R.A., 2011. Occupational risk assessment in construction industry – overview and reflection. Saf. Sci. 49, 616–624. http:// dx.doi.org/10.1016/j.ssci.2011.01.003. Saracino, A., Curcuruto, M., Pacini, V., Spadoni, G., Guglielmi, D., Saccani, C., Bocci, V.M., Cimarelli, M., 2012a. IPESHE: an index for quantifying the performance for safety and health in a workplace. Chem. Eng. Trans. 26, 489–494. http:// dx.doi.org/10.3303/CET1226082. Saracino, A., Spadoni, G., Curcuruto, M., Guglielmi, D., Bocci, V.M., Cimarelli, M., Dottori, E., Violante, F.S., 2012b. A new model for evaluating occupational health and safety management systems (OHSMS). Chem. Eng. Trans. 26, 519– 524. http://dx.doi.org/10.3303/CET1226087.
Contents lists available at ScienceDirect
Safety Science journal homepage: www.elsevier.com/locate/ssci
Quantitative assessment of occupational safety and health: Application of a general methodology to an Italian multi-utility company A. Saracino a, G. Antonioni a,⇑, G. Spadoni a, D. Guglielmi e, E. Dottori b, L. Flamigni c, M. Malagoli c, V. Pacini d a
DICAM, Dipartimento di Ingegneria Civile, Chimica, Ambientale e dei Materiali, Università di Bologna, via Terracini n.28, 40131 Bologna, Italia HERA, Group Director of Quality, Safety and Environment, V.le C. Berti Pichat 2/4 – 40127 Bologna, Italy c HERA, Quality, Safety and Environmental Management System, V.le C. Berti Pichat 2/4 – 40127 Bologna, Italy d NIER Ingegneria, Via Bonazzi, 2 – 40013 Castelmaggiore (Bo), Italy e Dipartimento di Scienze Dell’Educazione Giovanni Maria Bertin, Alma Mater Studiorum, Università di Bologna, Italy b
a r t i c l e
i n f o
Article history: Received 21 March 2014 Received in revised form 2 July 2014 Accepted 19 August 2014
Keywords: Occupational health and safety Occupational safety assessment Quantification Workplace safety
a b s t r a c t The core of safety practice is the occupational safety risk assessment, which is quite often a complex process since it requires to take into account parameters that are often difficult to quantify. The European Council Directive 89/391/EC is concerned with the introduction of measures to encourage improvements in the Occupational Health and Safety (OHS). Among other issues, it deals with risk assessment and preventive measures. In this framework a novel methodology named M.I.M.O.SA. (Methodology for the Implementation and Monitoring of Occupational SAfety) has been developed with the aim of quantify the occupational health and safety level of a company and thus of its OHS Management System (OHSMS). The methodology was born within a project where a multi-disciplinary team discussed the main topics involved in occupational safety and finally a global index, which takes into account, among others, both technical and organizational aspects, was defined. In this work after a brief description of the key concepts of this methodology, it has been applied to one activity of an Italian multiutility company in order to test its applicability to a real case study. Once the performance of the OHSMS was assessed, thanks to the quantification of the different aspects that concur to its global level, it has been possible to identify the priority of interventions for improvements, so that the management process becomes more effective and efficient. In addition, from this first application of the methodology some issues emerged that could be useful for its future improvement. Ó 2014 Elsevier Ltd. All rights reserved.
1. Introduction Prevention of occupational accidents is an important task of the OHSMS of a company, in fact the main aim of the ‘‘Framework Directive’’ of the European Union about OHS (EEC 391/1989) is to introduce measures to encourage improvements in the safety and health of workers at work and it applies to all sectors of activity. In recent years the European Occupational Safety and Health Agency (EU-OSHA), which collects, analyses and disseminates information related to occupational safety and health across the EU, along with six national safety and health organizations, participates in a consortium that has gathered together many risk assessment tools and methodologies available to help enterprises and organizations assess their health and safety risks. The choice of ⇑ Corresponding author at: DICAM, Dipartimento di Ingegneria Civile, Chimica, Ambientale e dei Materiali, Università di Bologna, via Terracini n.28, 40131 Bologna, Italia. http://dx.doi.org/10.1016/j.ssci.2014.08.007 0925-7535/Ó 2014 Elsevier Ltd. All rights reserved.
method will depend on workplace conditions, for example the number of workers, the type of work activities and equipment, the particular features of the workplace and any specific risks. Moreover a great number of European Standards are related to the importance of health safety at work. For instance OHSAS 18001 (BSI, 2007), is a British Standard for occupational health and safety management systems and it is widely seen as the world’s most recognized standard for Occupational Health and Safety Management Systems (OHSMSs). In general, the most common risk assessment tools are checklists, which are a useful tool to help identify hazards. Other kinds of risk assessment tools include: guides, guidance documents, handbooks, brochures, questionnaires, and ‘interactive tools’ (free interactive software, including downloadable applications which are usually sector-specific). These tools can be either generic or branch/risk-specific. Nevertheless only mainly sector-specific approaches for the quantification of occupational risk have been proposed (e.g., Murè and Demichela, 2009; Papadakis and Chalkidou, 2008;
76
A. Saracino et al. / Safety Science 72 (2015) 75–82
Table 1 MIMOSA key elements and themes with number of questions of the checklists and summary of the results for the case-study. Questions of checklist
Not applicable
Applicable
‘‘YES’’ answers
‘‘NO’’ answers
Partial answers
‘‘NO’’ + ‘‘partial’’
Key elements
Themes
Planning [P] acting [A]
Key element 1: leadership and consistency of targets
Organization and structure of responsibilities Direct involvement of the management Management of economic resources
P A
5 7
0 0
5 7
5 7
0 0
0 0
0 0
P A P A
6 6 4 3
0 0 0 2
6 6 4 1
4 3 1 1
0 0 0 0
2 3 3 0
2 3 3 0
Risk assessment
P A P A
13 15 2 5
0 0 0 0
13 15 2 5
12 13 2 5
1 1 0 0
0 1 0 0
1 2 0 0
P A P A P A P A P A P A P A P A P A
12 17 9 9 2 2 6 9 9 9 9 9 11 5 5 5 3 8
1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
11 17 9 9 2 2 6 9 9 9 9 9 11 5 5 5 3 8
11 17 6 8 2 2 6 9 9 9 9 9 11 5 2 2 3 7
0 0 0 0 0 0 0 0 0 0 0 0 0 0 3 3 0 0
0 0 3 1 0 0 0 0 0 0 0 0 0 0 0 0 0 1
0 0 3 1 0 0 0 0 0 0 0 0 0 0 3 3 0 1
P A P A P A P A
3 8 6 5 6 6 14 14
0 0 0 0 0 0 0 0
3 8 6 5 6 6 14 14
2 6 3 1 4 5 10 8
0 1 1 2 1 1 4 6
1 1 2 2 1 0 0 0
1 2 3 4 2 1 4 6
P A P A
5 9 16 17
0 1 0 0
5 8 16 17
5 7 4 5
0 0 9 10
0 1 3 2
0 1 12 12
Compliance with sector-specific formal requirements Formal validity of general requirements Reporting system
P A
12 15
3 3
9 12
9 12
0 0
0 0
0 0
P A P A
1 5 3 3
0 0 0 0
1 5 3 3
1 4 2 3
0 0 0 0
0 1 1 0
0 1 1 0
Human resources
P A P A P A A
45 50 4 6 5 5 5
0 0 0 0 0 0 3
45 50 4 6 5 5 2
36 36 4 6 4 4 2
3 8 0 0 1 1 0
6 6 0 0 0 0 0
9 14 0 0 1 1 0
Key element 2: orientation to risk reduction and people protection in compliance with the law
Measures of prevention and protection Education, training and communication Participation Risk monitoring Events (near misses) monitoring Health supervision Emergencies Contracts and subcontractors Safety levels improving Alertness at work
Key element 3: involvement, learning and development of individual culture
Safety climate Risk perception Open communication Rewarding system for safety
Key element 4: continuous improvement and innovation
Control system Comfort and development of human resources
Key element 5: formal and general compliance
Key element 6: social responsibility
Ethical and institutional aspects Voluntary certifications Environment
Aneziris et al., 2010; Pinto et al., 2011) and also those that are related to the assessment of the OHSMS of a company (e.g., Bellamy et al., 2008; Papadakis, 2000) are rather sector-specific. For this purpose, a methodology and a related index have then been developed within a specific project (FAM, 2012), where a number of actors with different background knowledge and experience were involved. On the basis of their experience, numerical values were assigned to checklists and indicators and thus a global index could be defined (Saracino et al., 2012). Despite the wide experience of the involved professionals, the methodology still
needs to be tested on a real situation in order to highlight its possible criticalities and its strengths and thus to improve it for future applications. 2. Fundamentals of the methodology M.I.M.O.SA. (Methodology for the Implementation and Monitoring of Occupational SAfety) is a methodology that has been introduced in order to allow the evaluation of the performance of a company concerning health and safety in the workplaces (FAM,
A. Saracino et al. / Safety Science 72 (2015) 75–82
77
Fig. 1. Tree structure of M.I.M.O.SA. methodology.
2012). In the methodology, built with the purpose of the self-assessment of the performance for both big and small scale enterprises, six so-called main ‘key elements’ are defined (see first column of Table 1). They allow building and implementing an appropriate and efficient OHSMS. Three categories of OHSMSs are introduced by MIMOSA. They derive from the legal requirements and from technical standards of different sectors (as mentioned in the introduction there can be different systems for different branches/activities), because the goals that each company expects to achieve through the implementation of a management system can be different from each other, and since OHSMSs can range from the simple compliance with law to continuous improvement. In short, the MIMOSA methodology identifies the following classes of management systems: – OHSMS derived from the Italian legislation on health and safety in the workplace (DL 81/2008). – OHSMS developed on the basis of national and international standards, developed by technical-scientific and regulatory agencies such as OHSAS 18001 (BSI, 2007), or guidelines for their implementation (BSI, 2008). – OHSMS derived from the Italian legislation on the liability of the companies for occupational safety (DL 231/2001). Starting from these three classes, MIMOSA methodology should be implemented under three different perspectives of analysis for the evaluation of the health and safety level of a company, namely: – The perspective of mandatory compliance with law. – The perspective of continuous improvement (e.g., OHSMS that has obtained or is trying to obtain the OHSAS 18001 certificate) once the compliance with law is guaranteed. – The perspective of crime prevention. 2.1. The structure of MIMOSA The tree structure of the methodology is presented in Fig. 1: at the top there are the key elements that are described through a
different number of themes, and then each theme is derived from the characterization of checklists and/or Key Performance Indicators (KPIs), which together form the ‘‘leaves’’ of the tree. Each key element is detailed through a number of specific themes and for each theme different checklists and KPIs have been developed for the assessment of that theme in a typical tree-structured hierarchy (see Fig. 1), in particular checklists are divided into planning (P) and acting questions (A). The quantification of each checklist and of each KPI is clearly related to a specific theme, as a consequence the tree structure of the M.I.M.O.SA. methodology is fully defined. The use of checklists and KPIs should guarantee that the assessment of the company’s performance in safety is achieved through an objective, though uncertain, analysis and owing to this peculiarity MIMOSA gives a help in directing improvement interventions.
2.2. Key elements and themes allocation The six key elements of the self-assessment methodology are (see also first column of Table 1): 1. Leadership and consistency of the targets. 2. Orientation to risk reduction and people protection, in compliance with the law. 3. Involvement, learning and education development of the personnel. 4. Continuous improvement and innovation. 5. Formal and general compliance. 6. Social responsibility. The first key element evidences that a proper leadership of the company communicates the importance of adequate requirements for health and safety in the workplace. The leadership exercised by the top management influences the behaviors of the members of the company. It consists in the deep interaction between employees/workers and/or executives/managers (or in general members in the higher company positions and with organizational responsibility).
78
A. Saracino et al. / Safety Science 72 (2015) 75–82
Table 2 Planning and acting checklists for the assessment of safety climate. Planning checklist 1 Has the company provided the appropriate investigative tools (such as surveys, interviews, and focus groups) about how the people perceive the value and the priority given to targets (e.g., ranking of goals, timetable and rate of production, continuous improvement) in relationship with safety in the various business activities? 2 If pertaining, is the discussion of safety issues scheduled in the agenda of the management meetings? 3 Are organizational interventions and initiatives planned by the company for the promotion and the active care of safety? (e.g., guidance actions for new recruits to facilitate their socialization with the safety regulations and procedures; mentoring and/or coaching initiatives by the appointed responsible for safety and by senior workers. . .) Acting checklist 1 Has the company implemented the appropriate investigative tools (such as surveys, interviews, and focus groups) about how the people perceive the value and the priority given to targets (e.g., ranking of goals, timetable and rate of production, continuous improvement) in relationship with safety in the various business activities? 2 Is the senior management directly and publicly involved in safety programs, with explicit information to the workforce of their commitment? (e.g., constant attendance at periodical meetings; direct and personal commitment in the risk-assessment procedures; drafting of safety procedures. . .) 3 Is the role of Safety Managera assigned to an executive manager? 4 Is the Safety Manager involved in meetings with the management where safety targets are planned? 5 Are the considerations and the suggestions of the Safety Manager regarded as key items during the management meetings 6 Are the proposals of the Safety Manager put in action? 7 Does the management require to the appointed persons in charge the improvement of safety? 8 Does the management carry out awareness campaigns of appointed persons in charge in order to encourage the involvement of all workers (e.g., guidance actions for new recruits, mentoring or coaching actions, etc.)? a
According to the definition of the UK Health and Safety Executive.
The second, the third and the fourth key elements take into account those activities that have the greatest impact on the level of health and safety (even without the formalization of a real model). In the second key element the attention is addressed to the requirements that can be important in determining features of the workplace, which have a great influence on the levels of health and safety of workers, so that the compliance with the law is substantial. The third key element emphasizes that all human resources operating in the enterprise play a fundamental role in the organization. The fourth key element highlights the importance of continuous improvement that is achieved through the well-known PDCA (Plan, Do, Check, Act) cycle, consisting of 4 continuous stages of action and verification (Deming, 1986). The fifth element is defined on the basis of some formalities required by the Italian law: they are particularly important if the target is to measure the effectiveness in exempting from the administrative liability. Finally, the sixth key element wants to underline the awareness of the companies about social issues and environmental sustainability (Saracino et al., 2012). The total number of themes in the MIMOSA system is 27, they are all listed in Table 1 (second column) and they are described in detailed elsewhere (FAM, 2012). Themes are partitioned into the six key elements as follows: three themes for key element 1, eleven themes for key element 2, four themes for key element 3, two themes for key element 4, three themes for key element 5, and four themes for key element 6. As can be easily seen the second key element has the highest number of themes and this will be important in the quantification process.
2.3. Quantification of KPIs, checklists and overall safety index The assessment procedure starts with the quantification of each checklist and of each KPI that are our operative aspects of the methodology. For the sake of brevity among them only two, though important, items are described in order to better explain the key aspects of the methodology. As an example of checklists, the two checklists (planning and acting), related to the evaluation of the theme ‘‘safety climate’’ (see Table 1), are described through the questions they contain, 3 planning questions and 8 acting questions, which are reported in Table 2. Safety climate is one of the themes that concur to the description and quantification of the key-element 3: ‘‘involvement, learning and development of individual culture’’ and moreover, all the questions, in both checklists, were applicable in the case-study described in the following. The standard format drawn up for each indicator is formed by the following seven fields: 1. definition of the indicator; 2. target of the indicator; 3. theme and key element to which it belongs; 4. quantification methodology; 5. reference indicator values; 6. laws, rules and other references; and 7. type of company. These fields must be filled in for each indicator. Points 1–3 and 6 are self-explanatory and they do not require other specific description. Points 5 and 7 define respectively which values are considered valid for a positive self-evaluation and for which types of companies (small, medium or big) the indicator can be used. Finally, the person in charge of filling in the form has only to calculate the indicator at point 5, on the basis of its description reported in point 4. As an example, in Table 3 is reported the format for
Table 3 KPI for the investigation on accidents. Indicator
Formal investigation on accidents
Definition
Number of accidents for which a formal investigation has been carried out (root cause), divided by the number of occurred accidents The aim is to facilitate the identification of the injury’s causes and to support the implementation of corrective actions Theme: measures of prevention and protection Key element: orientation to risk reduction and people protection in compliance with the law A = total number of accidents with formal investigation B = total number of accidents Safety threshold: not defined ISafety ¼ AB Not applicable Big and medium companies
Target Theme and key element of belonging Quantification methodology Reference indicator values Law, rules and other references Type of company
A. Saracino et al. / Safety Science 72 (2015) 75–82
one of the fourteen indicators used for the quantification of the theme ‘‘Measures of prevention and protection’’ belonging to the second key-element ‘‘orientation to risk reduction and people protection, in compliance with the law’’ (FAM, 2012), which was also used in the case-study. Once that each checklist and each KPI has been quantified, by properly weighting their values, a single numeric value can be obtained for each theme and thus in turn for each key element and finally for the company (or for a single activity of a company) as a possible representation of the overall performance of the company’s OSHMS. This value has been called IPESHE (Index of PErformance for Safety and HEalth). In other words the quantification of the methodology requires to evaluate the scores of each checklist and of each KPI in order to obtain, through a weighted sum, the score of each theme subsequently, through another weighted sum, each key element can be quantified and finally the value of the IPESHE index is obtained as shown in Eq. (1).
IPESHE ¼
N ke X
N t ðiÞ nKPI=cklðjÞ X X tj pk I k
i¼1
j¼1
xi
ð1Þ
k¼1
where Nke is the number of key elements (six), Nt(i) is the number of themes that contribute, with their relative weights tj, to key element i, whose weight is xi. Then nKPI/ckl(j) is the number of KPIs and/or checklists belonging to theme j and pk is the weight of each KPI or checklist k. Ik is the calculated value of a KPI or of a checklist (0 < Ik < 1). As a consequence, the IPESHE index will range between 0% and 100% and the greater will be its numerical value the greater the health and safety level for the workers of the company’s OHSMS. It is worth noting that the quantification is carried out by means of checklists and indicators, which are tools of common, though not exclusive, use tools, for the quantitative assessment of occupational risk through index methods (Kongsvik et al., 2010). 2.4. Weighting criteria As a first attempt, MIMOSA methodology considered two criteria for the calculation of IPESHE, called priority-criterion and equality-criterion respectively. They are intended to represent two different ways of evaluation. The equality-criterion assumes that all the indicators and the checklists give the same contribution to the theme that they belong (pk = 1/nKPI/ckl(j)), and thus the weight of each theme can be calculated through Eq. (2), as the ratio between the number of KPIs/checklists of that theme and the total number of KPIs/checklists of all the themes within a key-element. Finally also the weights of each key element, xi, are the same (i.e., 1/6).
nKPI=ckl ðjÞ t j ¼ N ðiÞ t X nKPI=ckl ðjÞ
ð2Þ
j¼1
These assumptions should guarantee that each aspect of the methodology has the same importance, regardless of its actual content, since it is required by the in-force law or by international standards. Thus, the index resulting from the application of these weights is intended to provide a measure of the formal safety obtained through the implementation of the OHSMS. In the priority-criterion, different weights, discussed on the basis of the expert judgment within the team involved in the project, are assumed for each key element, but still equal weights for each indicator/checklist within a theme (pk) are considered and thus the weight of each theme within a key element (tj) can be calculated again by means of Eq. (2). This second criterion should quantify the safety level of a company on the basis of the relative
79
importance of some key elements, and of their related themes and KPIs/checklists, with respect to others and thus it should provide a measure of the actual safety level that can be obtained by the company’s OHSMS. 3. Application of the methodology The methodology has been first tested through its application to an activity of an Italian company, which is responsible for the management of all road vehicles of the company including maintenance and reparation in full service mode throughout the complete life cycle. Thus the important health and safety issues in an automotive repair and maintenance service activity are exposures to chemicals, biological materials, vehicle exhaust, fatalities or injuries from contact with objects or equipment, especially struck by falling objects, transportation events, and fires and explosions. There are about 160 employees that works in several garages located within Italian region Emilia-Romagna. The OHSMS of the company has been certified in 2012 according to the BS OHSAS 18001:2007. Some adaptations have been introduced in order to better fit the methodology to the enterprise under investigation; in Section 3.1 the methods through which questions of the checklists and KPIs have been modified are described. Finally, the tools used, the assumptions introduced in the test, the results obtained and the limitations found in the application of MIMOSA are highlighted with the aim to put in evidence some possible improvements of the methodology, which can well overcome difficulties in its application. 3.1. Modifications to checklists In order to answer to the questions of the MIMOSA checklists, in this application two different approaches have been used. They have been called ‘‘direct interview’’ and ‘‘indirect interview by group membership’’. 43% of the questions were evaluated by means of the first approach and clearly the remaining 57% have been answered through the second approach. The two approaches are explained below: ‘‘Direct interview’’: the answers to the questions are given by a direct interview to different interlocutors, chosen on the basis of their expertise about the topic of the question. For instance, for checklists related to the theme ‘‘risk assessment’’ the Safety Manager has been selected, while for checklists related to the theme ‘‘direct involvement of the management’’ executives have been interviewed. ‘‘Indirect interview to group membership’’: the answers to the questions in the checklists were obtained indirectly by the evaluator, thanks to the deep knowledge of the group to which the company belongs. Indeed the group defines some elements of the management system (e.g., organizational procedures, policy of the company, initiatives of corporate social responsibility, etc.) that must be acquired also by the company to which they belong. These elements are defined at the corporate level and they are supported by documentation that is accessible to the evaluator (e.g., policy, budget of sustainability, etc.). The methodology requires only a YES/NO answer and does not consider the impossibility to answer, giving to some questions a null solution. An example is the question: ‘‘if the company is small (up to 50 employees), was an external subject put in charge of the surveillance institute?’’ which is clearly not applicable to a company with about 160 employees, as in the case under investigation. To overcome this problem, the specific answer ‘‘not applicable
80
A. Saracino et al. / Safety Science 72 (2015) 75–82
(NA)’’ is introduced for questions like the previous one and similar questions are excluded from the IPESHE calculation (rescaling relative weights). However, only 2.75% of the total number of questions was excluded due to a ‘‘NA’’ answer in this application. The summary of questions and answers is presented in Table 1, where the total number of ‘‘No’’, ‘‘YES’’ or ‘‘NA’’ answers for each theme (and for each key element) is reported. The set of all answers represents the raw output of the methodology and it is the basis for the calculation of the IPESHE index of the company, which, as previously mentioned, must reflect the level of health and safety of the company. 3.2. The use of indicators In order to assign a numerical value to each KPIs two different methods of calculation were used. Indicators were first subdivided into three categories (see Table 4), on the basis of the difficulties (or sometimes of the impossibility) observed to enumerate or to determine the factors involved in their quantification. The three categories of indicators were identified as follows: 1. Quantitative indicators: for this type of indicators, the value is obtained from the accurate calculation of the elements which concur to their definition, following the instructions on a form. 2. Semi-quantitative indicators: some elements required for the calculation are not managed in a systematic manner (e.g., the records of checks and maintenance operations related to equipment, fire prevention and PPE, etc.) but more frequently they are collected and kept locally in a paper-based archive. For example, the calculation of the indicator ‘‘fire-prevention/emergency’’ requires the knowledge of the number of inspections actually carried out on fire/emergency devices and of the number of expected inspections. Being managed locally, these data are not aggregated for the whole company and therefore they cannot be gathered globally. However, the checks on the fire
Table 4 Classification of indicators. Quantitative KPIs
Semi-quantitative KPIs Not applicable KPIs
Low level of criticality Medium level of criticality High level of criticality Ik = 1 unless irregularities Not applicable indicators
Easy interpretation Assumptions applied New interpretation Not completely quantitative Incompatible with activity under examination
protection measures and on emergency devices are subjected to periodic audits, as required by the certification OHSAS 18001:2007. For this reason, unless there are irregularities detected during the audit, it is assumed that the required checks are regularly carried out and the value of the indicator is 1. Note that, however, this simplification causes a sort of difficulty in the comparison of indicators, since the semi-quantitative ones are lower than 1 only if irregularities appear during the audits. 3. Not applicable indicators: some of the indicators (as it happened for checklists) cannot be applicable to every industrial or manufacturing activity (e.g., some indicators related to the observation of the behavior of the employees that shows a scarce commitment to the company). Furthermore each quantitative indicator has been classified according to the criticality level detected during its application (low–medium–high). The levels of criticality are reported in Table 4: a low level of criticality was assigned to those KPIs that were found to be easily understood by the surveyed, a medium level for those that required some interpretative assumptions and finally a high level of criticality to those that have been misunderstood and thus reinterpreted by the answerers. In this application about 37% of the indicators has not been valued, because they were not applicable and, in addition, 20 out of 54 indicators (again 37%) belongs to the class of quantitative indicators, which have been measured punctually (see Fig. 2) among them about a half showed a low level of criticality for their quantification while the remaining 50% showed a medium or high level of criticality (Fig. 2).
4. Results and discussions The application of MIMOSA methodology to the Italian company lead to two different values of IPESHE index when calculated on the basis of Eq. (1) and of two different weighting criteria as described in Section 2.4. When the equality criterion was selected an index of 81% was obtained, while when the priority criterion was adopted the index was 72%. The numerical results are also graphically depicted in Fig. 3, where also the contribution to the overall index of different key elements of the methodology can be seen. The results show that the OHSMS implemented by the company in this activity resulted in a good performance both when looking at the formal safety level (equality criterion) and when considering the greater importance of the first key-elements (priority criterion). Different considerations can be drawn from the application of these two criteria. In the case of the priority criterion, the highest
Fig. 2. Classification of the indicators.
A. Saracino et al. / Safety Science 72 (2015) 75–82
81
several checklists and KPIs, this high number of base elements greatly affects the contribution of this key-element in the overall value of the index if the equality criterion is applied. But if the priority criterion is selected, its importance in the IPESHE value decreases from about 67% to less than 30% (see Fig. 4) and the same reasoning can be applied to the other key elements as shown in Fig. 4. Thus more attention should be paid to these two keyelements since they largely contribute to the overall index even when trying to measure the actual performance, because results obtained in this case-study so far show that targets required by the law, or required in order to obtain the OHSAS 18001:2007 certification, have been reached, but some aspects related to the leadership of the management towards safety and to actively reducing the risk can still be improved in order to obtain the same score when applying different weighting criteria.
Fig. 3. Contributions of key elements to global IPESHE index.
5. Conclusions 5.1. Drawbacks and improvements of the methodology
Fig. 4. Relative importance of each key element for different criteria.
number of checklists and of indicators of some themes with respect to others bear only slightly down on the determination of the global index and, consequently, they play roles of limited importance whatever are their numbers in the key-elements; instead, if equality criterion is considered, a high importance is given to the number of indicators and checklists belonging to different themes and key-elements. In other words, the application of the two criteria has two different meanings: while the equality criterion is applied to mainly test the formal compliance against all the requirements provided by national (or other) in force law about OHS, the priority criterion could better attest the real safety level of a company, since it assumes decreasing weights for the key elements of the methodology (in the order they are listed in Table 1). As mentioned in Section 2.2, the requirements that have a greater impact on the protection of workers and thus mainly contribute to effective risk reduction, belong to the first key elements and thus these elements should have higher weights regardless of the number of checklists and/or indicators used for their quantification. In Fig. 3 it can be seen how the weights of the key elements affect the value of the global index in two different manners. For example the second key element (orientation of the company to risk reduction and people protection in compliance with the law) is described through 11 themes (see Table 1, second column) and thus through
The application of the methodology to an actual case study has highlighted some critical issues that require some investigation. First some difficulties have arisen form the only two possible answers (‘‘YES’’ or ‘‘NOT’’) to the questions of the checklists. This sort of on/off assessment penalizes a number of questions that can have partial positive or partial negative answer. Trying to take into account these cases, these questions have been tracked on the basis of the comments of the people interviewed and they resulted to be about 9% of the applicable questions. It should be noted that in the calculation of IPESHE index, answers that should have been evaluated only partially, were considered as negative. Thus one of the changes that could be introduced in the methodology is the possibility of intermediate answers (i.e., a ‘‘maybe’’ or better ‘‘more NO than yes’’ and ‘‘more YES than no’’). This would allow to obtain a more accurate value of the global index and also to take into account the possible uncertainty of the people interviewed. The second problem concerns checklists and indicators that can be considered not applicable to a given situation under analysis, because the non-applicability of the questions and indicators has important effects in the calculation of IPESHE, whose assigned weights depend on the number of checklists and KPIs belonging to different themes and key elements (at least if the equality criterion is selected, see Section 2.4). For the present study not applicable checklists and indicators were excluded in the IPESHE calculation, but this has highlighted the requirement for a customization and adaption of the methodology MIMOSA to a specific company. The customization makes the comparison among companies difficult, because each one of them can have a different number of not applicable items. Consequently, the methodology should investigate the criteria for the evaluation of these elements, through the deletion from some checklists of questions that do not have a general validity and eventually through the revision of the calculation method of some indicators. Another critical issue is the impossibility to verify the objectivity level of the answers to each question in a checklist, especially to those that were obtained by a direct interview. The questions, if not correctly formulated, can be misunderstood due to the subjectivity of the interlocutor or of the auditor. To overcome this criticality some evidences could be attached to the answers requiring further justification. These evidences can include for instance: – Internal audits, – Recordings, – Information that can be acquired from interviews and field surveys, etc.
82
A. Saracino et al. / Safety Science 72 (2015) 75–82
An evidence allow to assess the quality of an answer and to improve its objectivity level. For instance, in order to demonstrate that the correct answer to the question: ‘‘has been implemented the inspection of PPE?’’ is yes, a tailored checklist, developed by the category associations rather than by the company itself, though based on national or international guidelines, could be periodically applied. Clearly the auditor itself will have to judge in turn also the quality of the gathered evidences through a comparison e.g., with common standards and practices for that industrial activity. For this reason the methodology should also define criteria to adopt in order to give an objective judgment about the evidences, also because a company could supply a lot of evidences to support the same question. As outlined in Section 2, the methodology has been developed keeping in mind that three different perspectives can be adopted. Nevertheless attention should be paid to weighting criteria, possibly taking into account also evidences, in order to clearly evidence those checklists (or even single questions) and KPIs that mainly contribute to each perspective. Finally critical points about the approach of ‘‘direct interview’’, used to answer to some questions, have to be considered. As already mentioned, through this approach, different interlocutors of the company, identified on the basis of their expertise about the topic and to their role within the company, were asked to answer to the same questions. Thus the answers to these questions have been compared between each other and with some evidences, gathered by means of some field inspections, in order to crosscheck whether there could be inconsistency or not. Some inconsistencies actually appeared (e.g., for prevention and protection in activities where chemicals are handled, noise pollution is present and moving mechanical parts can harm the workers). This is primarily due to the level of subjectivity which is typical of these kind of surveys because different interlocutors (workers/employees and Safety Manager) can perceive differently the same question. When inconsistencies were found and there was no evidence to support one alternative, for the calculation of IPESHE the following ranking was assumed: (1) employer, (2) Safety Manager, (3) executive, and (4) staff (information systems and other personnel). However different results could be obtained if a different ranking had been assumed and also if other questions. Note that, extending all questions through interviews to several people and expanding the collection of evidences on the field to all the themes of the key elements, different results could be obtained, for this reason the methodology should provide in general more stringent criteria for the definition of the questions (and of the modality to their answers). 5.2. Final conclusions In this paper a first testing of the MIMOSA methodology was presented. This application helped to explain and clarify the way
the methodology is applied in order to assess the safety level of a company (or of a branch of it). The methodology was applied to the branch of a large multi-utility company operating in northern Italy, which manages the whole company’s fleet of road vehicles. This study shows that the methodology needs to be improved in some points, even if its results can be already used by the executives in order to manage and to improve the safety at work. The application is also intended to be a first guideline for the application of MIMOSA methodology and for suggesting some improvements to it. References Aneziris, O.N., Papazoglou, I.A., Kallianiotis, D., 2010. Occupational risk of tunneling construction. Saf. Sci. 48, 964–972. http://dx.doi.org/10.1016/j.ssci.2009.11.003. Bellamy, L.J., Geyer, T.A.W., Wilkinson, J., 2008. Development of a functional model which integrates human factors, safety management systems and wider organisational issues. Saf. Sci. 46 (3), 461–492. http://dx.doi.org/10.1016/ j.ssci.2006.08.019. British Standards Institution (BSI), 2007. Occupational health and safety management systems – requirements. OHSAS 18001:2007. British Standards Institution (BSI), 2008. Occupational health and safety management systems – guidelines for the implementation of OHSAS 18001:2007. OHSAS 18002:2008. Council Directive (EEC) 391/1989 of 12 June 1989 on the introduction of measures to encourage improvements in the safety and health of workers at work. OJ L183. Deming, W.E., 1986. Out of the Crisis. Cambridge University Press, UK. Fondazione Alma Mater (FAM), 2012. MIMOSA, Metodo per Implementare, Misurare e Organizzare la Sicurezza in Azienda. Il Mulino. In Italian. ISBN 978-88-15-24113-9. Kongsvik, T., Almklov, P., Fenstad, J., 2010. Organisational safety indicators: some conceptual considerations and a supplementary qualitative approach. Saf. Sci. 48 (10), 1402–1411. http://dx.doi.org/10.1016/j.ssci.2010.05.016. Legislative Decree (DL) 231/2001 of 8 June 2001. Disciplina della responsabilità amministrativa delle persone giuridiche, delle società e delle associazioni anche prive di personalità giuridica. GU 140. Legislative Decree (DL) 81/2008 of 9 April 2008. Attuazione dell’articolo 1 della legge 3 agosto 2007, n. 123, in materia di tutela della salute e della sicurezza nei luoghi di lavoro. GU 101. Murè, S., Demichela, M., 2009. Fuzzy application procedure (FAP) for the risk assessment of occupational accidents. J. Loss Prevent. Process Ind. 22, 593–599. http://dx.doi.org/10.1016/j.jlp.2009.05.007. Papadakis, G.A., Chalkidou, A.A., 2008. The exposure-damage approach in the quantification of occupational risk in workplaces involving dangerous substances. Saf. Sci. 46, 972–991. http://dx.doi.org/10.1016/j.ssci.2007.11.009. Papadakis, G.A., 2000. Assessment of requirements on safety management systems in EU regulations for the control of major hazard pipelines. J. Hazard. Mater. 78, 63–89. http://dx.doi.org/10.1016/S0304-3894(00)00217-X. Pinto, A., Nunes, I.L., Ribeiro, R.A., 2011. Occupational risk assessment in construction industry – overview and reflection. Saf. Sci. 49, 616–624. http:// dx.doi.org/10.1016/j.ssci.2011.01.003. Saracino, A., Curcuruto, M., Pacini, V., Spadoni, G., Guglielmi, D., Saccani, C., Bocci, V.M., Cimarelli, M., 2012a. IPESHE: an index for quantifying the performance for safety and health in a workplace. Chem. Eng. Trans. 26, 489–494. http:// dx.doi.org/10.3303/CET1226082. Saracino, A., Spadoni, G., Curcuruto, M., Guglielmi, D., Bocci, V.M., Cimarelli, M., Dottori, E., Violante, F.S., 2012b. A new model for evaluating occupational health and safety management systems (OHSMS). Chem. Eng. Trans. 26, 519– 524. http://dx.doi.org/10.3303/CET1226087.