Metrological management evaluation based on ISO10012: an empirical study in ISO-14001-certified Spanish companies

Energy 35 (2010) 140–147

Contents lists available at ScienceDirect

Energy journal homepage: www.elsevier.com/locate/energy

Metrological management evaluation based on ISO10012: an empirical study in ISO-14001-certified Spanish companies ˜ uzuri b, Miguel Rivas a, Cristina Gonza´lez c Jaime Beltra´n a, *, Jesu´s Mun a

Andalusian Institute of Technology, C/Leonardo Da Vinci, 2, 41092 Sevilla, Spain University of Seville, Av/Ame´rico Vespucio, s/n. 41092 Sevilla, Spain c National University of Distance Education, C/Juan del Rosal, 2, 28040 Madrid, Spain b

a r t i c l e i n f o

a b s t r a c t

Article history: Received 11 December 2007 Received in revised form 1 September 2009 Accepted 7 September 2009 Available online 6 October 2009

Environmental management systems based on the ISO 14001 standard rely strongly on metrological measurement and confirmation processes to certify the extent to which organizations monitor and improve their environmental behavior. Nevertheless, the literature lacks in studies that assess the influence of these metrological processes on the performance of environmental management in organizations, even now that the international standard ISO 10012 is already available to establish requisites and guidelines for the development of a metrological management system that is compatible with any other standardized management system. This work seeks to assess that influence through the development of an evaluation model for metrological management, which is then validated through an experimental analysis of the results obtained from the application of an audit process in 11 Spanish companies, all ISO-14001-certified and operating in different industrial sectors. Ó 2009 Elsevier Ltd. All rights reserved.

Keywords: Environmental management systems Metrological confirmation Measurement management systems Evaluation Audit process ISO 14001 ISO 10012

1. Introduction Environmental management systems (EMSs) based on ISO 14001 [1] were developed to evaluate and improve the environmental behavior of organizations, thus establishing measurement, evaluation and monitoring processes for all relevant environmental aspects. Most of these environmental aspects, when imported to the management system, are translated to metrological requisites, that is, they are defined as quantitative variables that need to be measured to evaluate their compliance by comparing those measurements with the specifications previously established for the requisites. Seeking to ensure the fulfillment of metrological requisites and to guarantee all compliance declarations in their evaluation, the international standard ISO 10012 [2], Measurement Management Systems (MMS), was issued in April 2003. This standard defines a model both for the management of measurement processes and for the metrological confirmation of the equipment used to support and test the compliance with all the metrological requisites of the organization, which in this work we refer to as the EMS. Also, in order to facilitate the application, integration and evaluation of a MMS, the Spanish standard UNE 66180 [3], Guide for * Corresponding author. Tel.: þ34 954 468 010; fax: þ34 954 460 407. E-mail address: [email protected] (J. Beltra´n). 0360-5442/$ – see front matter Ó 2009 Elsevier Ltd. All rights reserved. doi:10.1016/j.energy.2009.09.004

the metrological management and assessment, was issued in February 2008. This standard defines and clarifies the concept of metrological management, and thus enlarges the scope of MMSs and improves their integration with other standardized management systems, like EMSs. This standard used some previous research works on the subject [4–7] as an input. The importance of appropriately managing all the processes designed to ensure compliance with metrological requisites in all the areas of the company seems evident, as shown by several recent works on quality management systems (QMSs) [6–9]. However, to the authors’ best knowledge there are no examples in the literature focused directly on their effect on EMSs, despite their being a key factor to ensure correct environmental behaviors in organizations. Several works do focus on the impact and results obtained from the application of EMSs from different points of view, like sustainable development [10], the growth of organizations [11], the improvement of performance in certain manufacturing sector and specific production processes [12–24], or assess their applicability in SMEs [15], from the point of views of social responsibility [16] or in relation to the improvement of stakeholder results [17]. On the other hand, some researchers have focused on the factors influencing the development of EMSs, either in terms of the obtention and maintenance of certifications [17–20] or analyzing some of their key factors: human resources [21], training and

J. Beltra´n et al. / Energy 35 (2010) 140–147

communication [22], integration with other management systems [11,23,24] or operational control [20]. This work seeks a double objective. Firstly, we wanted to prove the fact that MMSs can contribute to enhance the performance of EMSs, through the application of a metrological evaluation model and an audit process in parallel. Secondly, we aimed to demonstrate the existence of a direct correlation between both evaluation processes, evaluation model and audit process, so that further empirical tests could be performed in the future without needing to directly audit the companies, and simply relying on the application of the evaluation model. To reach that double objective, we tested the methodology in a group of 11 Spanish companies with certified environmental systems and operating in different industrial sectors. This experimental research was designed by taking into account other previous works on strategies for qualitative research [25–28] and keeping in mind that the application of the validation procedure to a small sample of companies can be beneficial in this type of work for the drawing of general conclusions [29]. The experimental data were obtained through the application of validated and standardized models [3–6] for monitoring and evaluating metrological management in organizations. The results obtained confirm, on one hand, that MMSs allow for direct and natural integration into EMSs, and, on the other, that organizations that already have an EMS and decide to implement a MMS will benefit from the appropriate metrological processes to guarantee compliance with all the requisites linked to the most relevant environmental issues. Along this line, the suitability of the evaluation tools available for metrological management [3,4] has also been confirmed for the case of companies with certified EMSs. 2. Qualitative analysis of the contributions of ISO 10012 to the metrological requirements of ISO 14001 The ISO 14001 standard, in its Section 4.5.1 Monitoring and Measurement, defines metrological requirements for EMSs, seeking to provide reliability to the results of all the measurement and verification processes in the organization. Hence, those requisites are addressed both at the monitoring of the measurement processes themselves and at the use of measurement systems and

141

equipment required to verify compliance in those measurement processes, thus fully converging with the scope and objectives of the MMSs based on the ISO 10012 standard. Moving further from the previous works that focused on the contribution of MMSs to QMSs [6–9], our research focused on the main metrological requirements of the ISO 14001 standard, especially those compiled in Section 4.5.1. We have established a relation between those requirements and the different elements contemplated in the ISO 10012 standard (see Table 1), which relation stems from the management of those aspects from the MMS’s point of view. From Table 1 it follows that all the metrological requirements of ISO 14001 can be addressed from the ISO 10012 standard, which thus provides solid guidance when facing the main metrological issues required by EMSs. Measurement management systems provide full control over all the metrological resources, both physical (measuring equipment) and information-related (procedures and software used in the measurement and metrological confirmation processes). Besides, the main operational requirements related to the measurement equipment used to verify compliance with environmental requisites are developed in the ISO 10012 standard. This development is followed by a wide description of all the activities necessary to effectively manage and carry out all the operative metrological processes (metrological confirmation and measurement processes), as well as all the involved supporting processes. The relations shown in Table 1 represent the systematic procedure to fully and efficiently follow all the metrological requirements set by the ISO 14001 standard. This can be achieved by applying all the principles, requisites and guidelines established in the ISO 10012 standard, which contains deeper knowledge, support and intervention capacity on all the metrological requirements established in the environmental management standard. A strong relation is then established between the metrological aspects contemplated in ISO 14001 and the guidelines contained in ISO 10012 to guarantee their fulfillment. In Appendix A we have extended that relation to all the other common management issues in both standards, in order to provide full integration between both systems. This integration is extremely relevant to guarantee the sustainable success of the global management system in the organization [23,24].

Table 1 Relation between the monitoring and measurement required in ISO 14001 and their treatment in ISO 10012. Metrological aspects required in ISO 14001 4.5.1 Monitoring and Measurement

Treatment in ISO 10012

Elements of the ISO 10012

IDENTIFY POINTS TO BE MEASURED. The key features of the operations and activities which may have a significant impact on the environment should be measured, in addition to the regular assessment of compliance with applicable environmental legislation and requirements (emissions, dumps, consumption, etc.).

To all effects, the requirements for measurement which demonstrate compliance with an environmental specification must be considered as metrological requirements of the measurement management system. When the scope of the measurement management system is analysed, guidelines are included on how to tackle the identification of metrological requirements. The design of the measurement processes in order to demonstrate compliance with metrological requirements is a key element of the measurement management system. The measurement processes must be appropriate to the requirement to be measured. Generally, the most direct action concerning measurement equipment is reflected in the management of the resources of the measurement management system and in the metrological confirmation of the measurement equipment. The ISO 10012:2003 standard clarifies these concepts in the framework of the metrological confirmation of measurement equipment, which is fully analysed in its content. It includes guidance on how to tackle the management of equipment and other resources of the measurement management system It includes guidance on the selection and control of records which demonstrate metrological confirmation

4 General requirements 4 General requirements 7.2.2 Design of the measurement process

CARRY OUT MONITORING AND MEASUREMENT

ACTION CONCERNING MEASUREMENT EQUIPMENT

a) It must be calibrated or verified

b) Calibration records

7.2 Measurement Process 7.3 Uncertainty of measurement and traceability

6 Resource management 7.1 Metrological confirmation of measurement equipment 7.1Metrological confirmation. 7.1.1 General points 7.3 Uncertainty of measurement and traceability 6.3 Material Resources 6.2.3 Records 7.1.4 Records of metrological confirmation

´ n et al. / Energy 35 (2010) 140–147 J. Beltra

142

Selection Selectionofofcompanies companies

Training Trainingofofcompanies companies

Autoevaluation Autoevaluationofof measurement measurementmanagement management system systemused usedby bycompanies companies

Monitoring Monitoring

Audit Auditplanning planning InInsitu situactivities activities •....... •....... ••External ExternalEvaluation Evaluation

Drafting Draftingofofreport report

Analysis Analysisofofthe theresults results

Review Reviewofofimprovement improvementplan plan

Analysis Analysisofofthe theresults results Fig. 1. Work plan for model application.

3. Experimental validation in 11 Spanish companies All organizations with a certified EMS must have completed the implementation process of all the metrological requisites contained in the ISO 14001 standard as an additional element in their management systems [18,20]. Nevertheless, we have not been able to find in the literature any analysis of these metrological aspects as key factors of EMSs, which led us to conclude that the ISO 14001 standard alone cannot guarantee that certified organizations reach a level of metrological management that is mature enough to comply successfully with all the metrological requirements involved. We then undertook this experimental validation to determine the level of maturity in metrological management reached by a group of 11 Spanish companies operating in different sectors with a certified EMS older than 2 years. This quantitative analysis on metrological management was based on the application of two diagnosis tools: an evaluation model [3,4] and an audit process [4,30], which had already been applied together to evaluate similar aspects in quality management systems [5]. Following previous qualitative research works on data acquisition methods through experimentation [25–28]. it is possible to conclude that this type of methods methods, even when based on small samples, can reach the standards required by experimental research. This is the case of our paper, where the sample of 11 chosen companies produced results that were sufficient for the objectives and scope of our research work. Fig. 1 shows the process followed to apply the aforementioned tools in the 11 companies. Fundamental aspects of this work scheme were the sample selection and the methodology and tools used for data acquisition, which are described in the following sections. 3.1. Sample selection Table 2 shows the selection of companies used for the experimental analysis, which represents several different sectors of

industrial activity. All the selected companies had at least a twoyear-old ISO 14001 certification, which guaranteed that all the metrological requirements for environmental issues had already been implemented and audited by certification firms. 3.2. Data acquisition methodology The acquisition of data in the selected companies was carried out using the evaluation model and the subsequent audit process that had been previously developed. 3.2.1. Evaluation model for measurement management systems Described in the UNE 66180 standard [3]. We designed, validated and standardised this model in previous research works [4,5]. It allows diagnosing, in a quantitative and systematic manner, the maturity level, globally or element-by-element, of the measurement management system, covering the requisites specified by the ISO 10012 standard and the guidelines contained in the UNE 66180 standard. The evaluation is based on a questionnaire related to all the elements in a MMS, assigning points to the maturity level reached by the element in question1, as well as identifying strong points and opportunities for improvement when detected by the evaluator. Table 3 summarises the main characteristics of this model for evaluating metrological management. The application of this model in the 11 companies selected for our research consisted of three stages: (1) training of the designated company staff on the ISO 10012 and UNE 66180 standards; (2) self-evaluation or internal evaluation of the company by the

1 The points assigned in the evaluation model are scaled from 1 to 5, as described in [3, 4], where 1 represents a maturity level with no formal approximation to compliance with the evaluated element, 2 a reactive and planned approximation to compliance, 3 effective and proactive compliance, 4 represents compliance with clear focus on measurement and continuous improvement, and 5 is the top excellence level in compliance with the evaluated aspect.

J. Beltra´n et al. / Energy 35 (2010) 140–147

143

Table 2 Companies selected for the experimental study. Code company

Activity

Sector

E1 E2 E3 E4 E5 E6 E7 E8 E9 E10 E11

Manufacturing and testing of carbon fiber structures Analysis, selection and testing of electrical, electronic and electromechanical components Transformation, distribution and marketing of horticultural products Agro-food laboratory for physical, chemical and microbiological analyses Technical inspection of vehicles Environmental monitoring, with sampling and laboratory equipment Laboratory for physical, chemical and microbiological analyses Production of aerosols and other different chemical and biological products Research, development and innovation Research laboratory for metallic materials Medium and light boilermaking, assembly, maintenance and repairs. Pipe design, prefabricating and installation.

Aerospace Aerospace Agro-industry Agro-industry Environment Environment Environment Chemical Chemical Metallic materials Metallic materials

trained staff; and (3) external evaluation carried out by the authors of this work. The data collection both for the internal and external evaluation used a web application [31] supported by the metrological management evaluation model. 3.2.2. Audit process for measurement management systems The ISO 10012 standard stresses the relevance of audits as the main tool available to evaluate the compliance of the measurement management system with the requisites set in that international standard. We sought to build a solidly founded audit process for metrological management, compatible with the existing standards, [30] experimentally tested through its application in a group of companies and coherent with the underlying evaluation model. We thus carried out the studies and developments documented in, [4] which resulted in this research work after the completion of MMS audits in the 11 selected companies, and whose quantitative value is related to the number of non-conformities with the requisites in the reference standard. The main characteristics of the audit process for metrological management applied in our work can be summarised as follows:  It represents a systematic, justified and contrasted process to evaluate the degree to which the requirements of the ISO 10012 standard are met.  It contributes to the improvement of the performance of the measurement management system, as a result of the audit findings.  It provides an external vision of the level of maturity of metrological management in the organisation, through the completion, in parallel to the audit, of the evaluation of the MMS according to the proposed model. Appendix B exemplifies the quantitative summary of an evaluation and audit process on metrological management. The summary form shown was the main document used for data analysis in this research. This form is part of a wider report issued after each evaluation and audit process containing the detailed description of every strong point, opportunity for improvement and non-conformity.

3.3. Results and discussion Table 4 shows the results obtained after the application in the 11 selected companies of the evaluation model for metrological management, both through internal and external evaluation, and of the audit process. The first analysis corresponds to the consistency and validity of the results obtained following this methodology as described in. [4] The methodology requires, by comparison with other widely used evaluation techniques for management systems, a gap lower than 10% between the results of the internal and external evaluations. Should the difference be larger, both of them would have to be repeated by a joint team with internal and external evaluators. In our case, the previous training stage represented an improvement with respect to other previous works, [4,5] and possibly thanks to it the results obtained by the internal evaluators were very close to the observations of the external team. Both evaluations produced consistent data in a single cycle of the evaluation process. Once the results were thus validated, we used those obtained in the external evaluation for subsequent analyses. These external evaluation results are compiled in Table 5. The main conclusions drawn from the analysis on an aggregate level are as follows:  Policies and strategies defined by companies within the framework of their EMSs do not take into account the results obtained by those companies in terms of the efficacy and efficiency of their metrological processes, which causes evaluation results to be comparatively low for chapter 5 of MMSs (Management Responsibility). Also, as a consequence, continuous improvement mechanisms are lacking in those processes, which in turn results in relatively low evaluations for chapter 8 (Analysis and Improvement).  EMSs need to strengthen all the aspects related to metrological infrastructure management, as highlighted in the results obtained for Section 6.3 (Material Resources).  The EMSs of all the analysed companies show great potential for improvement in their operative metrological processes. This is specially so for the metrological confirmation of the

Table 3 Evaluation model for a MMS. Objectives

Method of evaluation Evaluation criteria Perspective provided by the evaluation

-Measure the level of maturity, either globally or by components of the measurement management system. -Establish measurable objectives for improvement of metrological management. -Prioritise and establish detailed action plans concerning the requirements and directives of the reference standard. -Monitor the evolution of the level of maturity and of the impact of adopted actions for improvement. Questionnaire of specific questions for each component of the ISO 10012 standard accompanied by examples of evidence to guide the evaluation process. 5 levels of maturity according to the evidence provided by the organisation for the approach, deployment, results and improvement attributes in each question of the questionnaire. A detailed vision of the measurement management system related to the operations, processes, requirements and methods.

´ n et al. / Energy 35 (2010) 140–147 J. Beltra

144

Table 4 Results of the application on the evaluation model and audit process for metrological management systems. ELEMENTS OF THE ISO 10012

E1

E2

E3

E4

E5

E6

E7

E8

E9

E10

E11

IA EE NC IA EE NC IA EE NC IA EE NC IA EE NC IA EE NC IA EE NC IA EE NC IA EE NC IA EE NC IA EE NC 4 GENERAL REQUIRIMENTS 5 MANAGEMENT RESPONSIBILITY 5.1 Metrological function 5.2 Customer focus 5.3 Quality objectives 5.4 Management review 6 RESOURCE MANAGEMENT 6.1 Human resources 6.2 Information resources 6.3 Material resources 6.4 Outside suppliers 7 METROLOGICAL CONFIRMATION 7.1 Metrological confirmation 7.2 Measurement process 7.3 Measurement uncertainity and traceability 8 ANALISYS AND IMPROVEMENT 8.1 General 8.2 Auditing and monitoring 8.3 Control of nonconformities 8.4 Improvement

3,8 2,1 2,8 2,3 1,7 1,8 3,4 3,6 2,7 3,3 4,1 3,2 3,5 3,3 2,8

2,4 2,2 2,8 2,7 1,4 1,7 2,8 3,2 2,4 2,5 2,7 2,5 2,6 2,6 2,3

1 2

4,0 4,3 3,7 4,0 4,3

2,9 2,2 2,7 3,2 3,6

GLOBAL RESULTS

3,2 2,7 6

1 1 2

1 1 1

1

3,0 2,7 3,2 3,0 1,9 2,8 3,3 3,1 3,5 3,4 3,0 3,1 3,1 3,0 3,1

2,3 2,3 3,3 2,2 2,0 2,4 2,5 2,3 2,4 2,9 2,5 2,2 2,1 2,7 1,9

3,0 2,7 3,1 3,3 3,0

2,6 2,4 2,6 2,6 2,5

1

1 3 1 2 3 1 1 1

3,1 2,3 7

2,0 1,8 2,1 2,2 1,4 1,7 2,0 2,0 2,3 1,6 2,3 1,7 1,5 1,7 1,8

1,8 1,7 1,5 2,1 1,1 1,4 2,1 1,9 2,7 1,7 2,6 1,7 1,3 1,8 2,0

1 1

1,8 1,9 1,8 1,6 1,8

2,0 1 2,1 2,5 1 1,2 1,6

1 3 1 2

2 1 1

1,9 1,8 8

2,3 2,5 3,0 2,9 1,8 2,5 2,6 2,5 2,9 2,3 2,7 2,0 1,7 2,1 2,2

2,0 2,6 2,7 2,9 2,6 2,1 2,5 2,1 3,3 2,6 2,8 2,0 1,6 2,4 2,3

1 1

2,3 2,0 2,5 2,3 2,4

2,7 1 2,1 2,9 1 2,7 2,8

1 1

1 4 1 3

2,4 2,3 8

2,0 1,8 2,1 1,5 2,5 1,6 2,4 1,9 3,5 2,6 2,0 2,2 2,0 2,1 2,6

1,7 1,9 1,9 1,4 2,5 2,5 2,1 1,9 2,5 2,2 2,2 2,3 2,3 2,0 2,6

1 3 1 1

1,8 2,0 1,6 1,7 2,1

1,9 3 1,8 2,1 3 2,0 1,5

1 4 1 1 2 1 1

2,8 3,2 3,7 3,2 2,7 3,7 3,5 3,4 3,7 3,4 3,7 3,0 3,2 2,7 3,1

2,1 2,1 3,2 1,9 1,4 2,7 3,1 3,1 3,4 2,9 3,1 2,9 2,7 2,5 3,4

2,9 2,1 3,0 3,6 2,7

1 1

1

2,3 2,3 3,0 2,1 2,1 2,7 3,3 3,1 3,8 3,2 3,0 3,1 3,5 2,1 3,6

1,9 2,1 2,1 2,4 1,4 2,3 3,0 2,7 3,5 3,1 3,0 2,7 2,7 2,1 3,2

2,5 1 2,0 2,5 1 3,0 2,1

3,0 2,2 2,9 2,9 3,1

1

2 1

2,1 2,1 12 3,1 2,6 5

3 1 1 1

2,1 2,0 2,3 2,2 1,5 1,9 2,0 1,9 2,3 1,9 1,8 2,1 2,3 1,9 2,0

1,8 1,8 1,9 2,1 1,5 1,5 1,9 2,0 1,4 2,1 2,2 1,8 2,1 1,7 1,5

2,8 1 1,6 1 3,0 3,0 3,0

2,3 2,7 2,5 2,0 2,0

2 1 1 1

1

2,9 2,6 7

1 1

4 1 1 2

2,1 2,3 3,1 2,1 1,4 2,6 3,0 2,9 2,9 3,1 3,1 2,5 2,8 2,4 2,3

2,1 2,2 3,0 2,6 1,5 1,7 3,1 3,2 3,1 2,9 3,0 2,5 3,0 1,6 3,5

2,1 1 1,6 2,4 1 2,3 1,7

2,4 2,2 2,6 2,3 2,5

1 2 1 1

2,1 1,8 9

1 1

1

2,4 2,8 3,2 3,0 2,3 2,9 3,2 2,9 3,8 3,2 2,9 2,9 3,0 2,6 3,1

2,3 2,3 2,0 2,5 2,1 2,3 3,3 3,5 3,7 2,9 3,3 2,7 3,6 2,2 2,2

1,9 2 1,2 1,9 1 2,5 1 2,2

2,8 2,5 2,9 2,9 2,6

1 1 1

1

2,4 2,4 6

1 1

1

2,6 2,7 2,8 2,8 2,2 2,9 3,1 2,9 2,6 3,7 3,5 2,7 3,0 2,6 2,4

3,0 3,1 3,6 3,6 2,2 3,0 2,7 2,7 2,1 2,8 3,2 2,3 3,1 2,0 1,7

2,5 1 2,0 1 2,6 2,5 2,5

2,9 2,2 3,3 2,9 2,3

2,8 1 2,3 3,2 1 2,7 2,4

1

1

2,9 2,7 4

1 1

1 2 1 1 2 1 1

2,8 2,6 6

IA, internal application; EE, external evaluation; NC, number of non-conformities found in the audit process.

measuring equipment used for environmental issues, and for the lack of traceability shown by many of the related measurement processes, which yields low evaluation results for Section 7.1 (Metrological Confirmation) of ISO 10012.

3.3.1. Results of the audit process for measurement management systems The analysis that follows stems from the findings of the audit processes as shown on Table 6, together with the considerations and qualitative insight gained when applying the audit process in the 11 companies. The results show high correlation between the findings of the audits (failures to comply with the requisites of the ISO 10012 standard resulting in non-conformities) and the level of maturity

reached according to the evaluation of the measurement management system. This indicates high coherence and compatibility between the two diagnosis tools, and validates the evaluation model as a sound means to quantitatively assess measurement management in a company without need to complete the audit process, or prior to it. With respect to the results themselves, on an aggregate level we can conclude that:  Companies lack in metrological confirmation of their measuring equipment and in process monitoring for measuring environmental aspects. This results in a high number of failure evidences for metrological requisites on this subject, which results are of special relevance, as these requisites are also specified in the ISO 14001 standard.

Table 5 Results of the external evaluation for the 11 selected companies. ELEMENTS OF THE ISO 10012

E1

E2

E3

E4

E5

E6

E7

E8

E9

E10

EE

EE

EE

EE

EE

EE

EE

EE

EE

EE

E11 EE

4 GENERAL REQUIRIMENTS 5 MANAGEMENT RESPONSIBILITY 5.1 Metrological function 5.2 Customer focus 5.3 Quality objectives 5.4 Management review 6 RESOURCE MANAGEMENT 6.1 Human resources 6.2 Information resources 6.3 Material resources 6.4 Outside suppliers 7 METROLOGICAL CONFIRMATION 7.1 Metrological confirmation 7.2 Measurement process 7.3 Measurement uncertainity and traceability 8 ANALISYS AND IMPROVEMENT 8.1 General 8.2 Auditing and monitoring 8.3 Control of nonconformities 8.4 Improvement

2,4 2,2 2,8 2,7 1,4 1,7 2,8 3,2 2,4 2,5 2,7 2,5 2,6 2,6 2,3 2,9 2,2 2,7 3,2 3,6

2,3 2,3 3,3 2,2 2,0 2,4 2,5 2,3 2,4 2,9 2,5 2,2 2,1 2,7 1,9 2,6 2,4 2,6 2,6 2,5

1,8 1,7 1,5 2,1 1,1 1,4 2,1 1,9 2,7 1,7 2,6 1,7 1,3 1,8 2,0 2,0 2,1 2,5 1,2 1,6

2,0 2,6 2,7 2,9 2,6 2,1 2,5 2,1 3,3 2,6 2,8 2,0 1,6 2,4 2,3 2,7 2,1 2,9 2,7 2,8

1,7 1,9 1,9 1,4 2,5 2,5 2,1 1,9 2,5 2,2 2,2 2,3 2,3 2,0 2,6 1,9 1,8 2,1 2,0 1,5

2,1 2,1 3,2 1,9 1,4 2,7 3,1 3,1 3,4 2,9 3,1 2,9 2,7 2,5 3,4 2,5 2,0 2,5 3,0 2,1

1,9 2,1 2,1 2,4 1,4 2,3 3,0 2,7 3,5 3,1 3,0 2,7 2,7 2,1 3,2 2,8 1,6 3,0 3,0 3,0

1,8 1,8 1,9 2,1 1,5 1,5 1,9 2,0 1,4 2,1 2,2 1,8 2,1 1,7 1,5 2,1 1,6 2,4 2,3 1,7

2,1 2,2 3,0 2,6 1,5 1,7 3,1 3,2 3,1 2,9 3,0 2,5 3,0 1,6 3,5 1,9 1,2 1,9 2,5 2,2

2,3 2,3 2,0 2,5 2,1 2,3 3,3 3,5 3,7 2,9 3,3 2,7 3,6 2,2 2,2 2,5 2,0 2,6 2,5 2,5

3,0 3,1 3,6 3,6 2,2 3,0 2,7 2,7 2,1 2,8 3,2 2,3 3,1 2,0 1,7 2,8 2,3 3,2 2,7 2,4

GLOBAL RESULTS

2,7

2,3

1,8

2,3

2,1

2,6

2,6

1,8

2,4

2,7

2,6

J. Beltra´n et al. / Energy 35 (2010) 140–147

145

Table 6 Comparison of results between the evaluation and audit of metrological management in the 11 selected companies. ELEMENTS OF THE ISO 10012

4 5 6 7 8

GENERAL REQUIRIMENTS MANAGEMENT RESPONSIBILITY RESOURCE MANAGEMENT METROLOGICAL CONFIRMATION ANALISYS AND IMPROVEMENT

GLOBAL RESULTS

E1

E2

E3

E4

E5

E6

E7

E8

E9

E10

E11

EE

NC

EE

NC

EE

NC

EE

NC

EE

NC

EE

NC

EE

NC

EE

NC

EE

NC

EE

NC

EE

NC

2,4 2,2 2,8 2,5 2,9

1 2 2 1 -

2,3 2,3 2,5 2,2 2,6

1 3 3 -

1,8 1,7 2,1 1,7 2,0

1 1 3 2 1

2,0 2,6 2,5 2,0 2,7

1 1 1 4 1

1,7 1,9 2,1 2,3 1,9

1 3 4 1 3

2,1 2,1 3,1 2,9 2,5

1 1 2 1

1,9 2,1 3,0 2,7 2,8

2 1 3 1

1,8 1,8 1,9 1,8 2,1

1 1 2 4 1

2,1 2,2 3,1 2,5 1,9

1 1 1 1 2

2,3 2,3 3,3 2,7 2,5

1 1 1 1

3,0 3,1 2,7 2,3 2,8

1 1 2 2 1

2,7

6

2,3

7

1,8

8

2,3

8

2,1

12

2,6

5

2,6

7

1,8

9

2,4

6

2,7

4

2,6

6

 All the processes related to chapter 6 (Resource Management) also produce a high number of non-conformities with respect to the ISO 10012 standard, especially in section 6.3 (Material Resources). On the other hand, when applying the audit process, we observed that its efficiency relies strongly on the auditor’s competence. This competence must include knowledge on environmental management systems, metrological management and operative metrological processes, as well as expertise on the audit process itself, through an adequate planning and management of all the activities involved. 4. Conclusions and future work The results obtained in this research work show that organisations with certified EMSs have a great potential for improvement in their metrological processes. These processes guarantee compliance with all the environmental requirements that need to be measured and that ultimately evaluate the environmental behaviour of the organisation. We also found that this improvement in metrological performance can be attained through the integration of a MMS in the company’s EMS, given that the ISO 10012 was specifically designed for that integration, as shown by the analysis of the qualitative study. Companies should thus seek to incorporate measurement management systems by integrating them in their overall management processes, following the guidelines and recommendations of previous works on management system integration [23,24]. On the other hand, both diagnosis tools for metrological management (evaluation model and audit process) have proved reliable in the evaluation of the efficiency and the level of maturity of metrological management in companies with certified EMSs. We were thus able to confirm the validity of our evaluation model, observing high consistency between the results obtained by both models when evaluating measurement management systems from both perspectives. Our future research will focus on the formulation of more general quantitative conclusions, by increasing the sample size through the evaluation of a larger number of companies without the need to carry out the parallel audit process. As for the results obtained for the analysed companies, we can conclude that EMSs should take steps towards incorporating the results of metrological processes to the organisation’s policies and strategies. This includes an adequate management of the metrological infrastructure used for verification processes of environmental issues, as well as correct metrological equipment confirmation and metrological traceability for environmental measurement processes. Along this line, the ISO 10012 and UNE 66180 standards constitute the ideal framework to help EMSs improve these aspects.

contributions. We also thank the IAT (Andalusian Institute of Technology) its strong implication in the development and improvement of the metrological aspects of management systems, and the Andalusian Administration for the financial support of this research.

Appendix A. Relation between the requirements of the ISO 10012:2003 standard and those of the ISO 14001:2004 standard

Requisites ISO 10012

Requisites ISO 14001

4 GENERAL REQUIRIMENTS 5. MANAGEMENT RESPONSIBILITY 5.1 Metrological function

4.1 General Requirements

5.2 Customer focus 5.3 Quality objectives 5.4 Management review 6. RESOURCE MANAGEMENT 6.1 Human resources 6.2 Information resources

6.3 Material resources 6.4 Outside suppliers 7. METROLOGICAL CONFIRMATION AND REALIZATION OF MEASUREMENT PROCESSS 7.1 Metrological confirmation 7.2 Measurement process 7.3 Measurement uncertainty and traceability 8 MEASUREMENT MANAGEMENT SYSTEM ANALYSIS AND IMPROVEMENT 8.1 General 8.2.1 Auditing and monitoring. General 8.2.2 Customer satisfaction 8.2.3 Measurement management system audit. 8.2.4 Monitoring of the measurement management system 8.3 Control of nonconformities 8.4.1 General

Acknowledgements

8.4.2 Corrective action

The authors wish to thank the 11 companies that took part in the empirical analysis for their active participation and valuable

8.4.3 Preventive action

4.4.1 Resources, Responsibility and authority 4.4.3 Communication 4.3.1 Environment aspects 4.3.2 Legal requirements and others 4.2 Environmental Policy 4.3.3 Objectives and programs 4.6 Management review 4.4.2 Competence, training and awareness 4.4.4 Documentation 4.4.5 Control of documents 4.5.4 Control of records 4.4.6 Operational control 4.5.1 Monitoring and measuring 4.4. Operational control

4.5.1 Monitoring and Measuring 4.5.1 Monitoring and Measuring 4.5.1 Monitoring and Measuring

4.5.1 Monitoring and Measuring 4.5.1 Monitoring and Measuring 4.5.5 Internal audit 4.5.1 Monitoring and Measuring 4.5.3 Nonconformities, corrective action and preventive action 4.2 Environmental Policy 4.3.3 Objectives and programs 4.5.2 Nonconformities, corrective action and preventive action 4.5.2 Nonconformities, corrective action and preventive action

´ n et al. / Energy 35 (2010) 140–147 J. Beltra

146

Appendix B. Summary of the audit process for measurement management systems

ELEMENTS OF THE ISO 10012 4 General Requirements 5.1 Metrological function 5.2 Customer focus 5.3 Quality objectives 5.4 Management review 6.1 Resource management 6.2.1 Procedures 6.2 Information resources

6.2.2 Software

6.3 Material resources

6.3.1 Measuring equipment

6.2.3 Records 6.2.4 Identification

6.3.2 Environment

6.4 Outside suppliers 7.1.1 General

7.1 Metrological confirmation

7.1.2 Intervals between metrological confirmation 7.1.3 Equipment adjustment control 7.1.4 Records of the metrological confirmation process 7.2.1 General

7.2 Measurement process

7.2.2 Measurement process design

7.3 Measurement uncertainty and traceability

7.3.1 Measurement uncertainty

7.2.3 Realization of the measurement process 7.2.4 Records of measurement processes

7.3.2 Traceability

8.1 General 8.2.1 General 8.2 Auditing and monitoring

8.2.2 Customer satisfaction 8.2.3 Measurement management system audit 8.2.4 Monitoring of the measurement 8.3.1 Nonconforming measurement

8.3 Control of nonconformities

8.3.2 Nonconforming measurement proceses 8.3.3 Nonconforming measuring equipment 8.4.1 General

8.4 Improvement

8.4.2 Corrective action 8.4.3 Preventive action

GLOBAL RESULTS

(*) Non conformity. (**) Opportunity for improvement. (***) Strong Points.

NC (*)

EVALUATION OF THE MEASUREMENT MANAGEMENT SYSTEM (UNE 66180) External Internal OI SP Evaluation Evaluation (**) (***)

J. Beltra´n et al. / Energy 35 (2010) 140–147

References [1] ISO 14001:2004. Environmental management systems – requirements with guidance for use; 2004. [2] ISO 10012:2003. Measurement management systems – requirements for measurement processes and measuring equipment; 2003. [3] UNE 66180. Guide for the metrological management and assessment. AENOR; 2008. [4] Beltra´n J. Analysis of an autoevaluation model and audit process for measurement management systems (MMS) based on ISO 10012:2003 (Ana´lisis de un modelo de autoevaluacio´n y de un proceso de auditorı´a de los sistemas de gestio´n de las mediciones basados en la norma ISO 10012:2003): PhD thesis. Madrid: School of Industrial Engineering at the National University of Distance Education; 2006 (in Spanish). [5] Beltra´n J, Gonza´lez C, Domingo R. Results of the application of an evaluation model and audit process for measurement management systems based on ISO 10012:2003. In: The 2nd Manufacturing Engineering Society International Conference. Madrid; 2007. [6] Beltra´n J, Gonza´lez C, Rivas M. Ana´lisis de los requerimientos metrolo´gicos de la norma ISO 9001:2000 respecto a los establecidos para los procesos metrolo´gicos operativos de la norma ISO 10012:2003 Analysis of the metrological requirements of ISO 9001:2000 with respect to metrological operational processes of ISO 10012:2003. In: The 2nd Manufacturing Engineering Society International Conference. Madrid; 2007. [7] Beltra´n, J. La gestio´n de los procesos metrolo´gicos. Ana´lisis e integracio´n de un sistema de gestio´n de las mediciones (ISO 10012). AENOR. Madrid; 2004 (in Spanish). [8] Carbone P, Macii D, Petri D. Measurement uncertainty and metrological confirmation in quality-oriented organizations. Measurement Magazine 2003;34(4):263–71. [9] Gonza´lez C, Sebastia´n M.A. Considerations about the interrelation between the industrial metrology and quality systems. 5th IMEKO Syposium on dimensional metrology in production and quality control. Zaragoza, Spain, 1995; p. 29–37. [10] Sarmento M, Durao D, Duarte M. Evaluation of company effectiveness in implementing environmental strategies for a sustainable development. Energy 2007;32(6):920–6. [11] Casadesu´s M, Marimon F, Heras IISO. 14001 diffusion after the sucess of the ISO 9001 model. Journal of Cleaner Production 2008;16(16):1741–54. [12] Turk A. The benefits associated with ISO 14001 certification for construction firms: Turkish case. Journal of Cleaner Production 2008; doi:10.1016/ j.jclepro.2008.11.001. [13] Gonza´lez-Benito J, Gonza´lez-Benito O. Operations management practices linked to the adoption of ISO 14001: an empirical ana´lisis of Spanish manufacturers. International Journal of Production Economics 2008;113(1):60–73.

147

[14] Brouwer Martin AC, van Koppen CSA(Kris). The soul of the machine: continual improvement in ISO 14001. Journal of Cleaner Production 2008;16(4):450–7. [15] Bernardini M. Environmental impact evaluation using a cooperative model for implementing EMS (ISO 14001) in small and me´dium-sized enterprises. Journal of Cleaner Production 2008;16(14):1447–61. [16] Castka P, Balzarova M. The impact of ISO 9000 and ISO 14000 on standardisation of social responsibility- an inside perspective. International Journal of Production Economics 2008;113(1):74–87. [17] Gavronski I, Ferrer G, Laureano EISO. 14001 certification in Brazil: motivations and benefits. Journal of Cleaner Production 2008;16(1):87–94. [18] Balzarova M, Castka P. Underlying mechanism in the maintenance of ISO 14001 environmental management system. Journal of Cleaner Production 2008;16(18):1949–57. [19] Curkovic S, Sroufe R, Melnyk S. Identifying the factors which affect the decision to attain ISO 14000. Energy 2005;30(8):1387–407. [20] Marguglio BW. Environmental management systems. New York: M. Dekker; 1991. [21] Chiappetta C, Almada F, Seido M. Environmental management system and human resource practices: is there a link between them in four Brazilian companies. Journal of Cleaner Production 2008;16(17):1922–5. [22] Sammalisto K, Brorson T. Training and communication in the implementation of environmental management systems (ISO 14001): a case study at the University of Ga¨vle, Sweden. Journal of Cleaner Production 2008;16(3): 299–309. [23] Domingo, R. Study of integrated management systems (QMS, EMS and SHMS) (Estudio de la integracio´n de sistemas certificados de calidad, medio ambiente y seguridad): PhD thesis. Madrid: School of Industrial Engineering at the National University of Distance Education; 1999 (in Spanish). [24] Salomone R. Integrated management systems: experiencies in Italian organizations. Journal of Cleaner Production 2008;16(16):1786–806. [25] Glaser B, Strauss A. The discovery of grounded theory: strategies for qualitative research. Chicago: Aldine; 1967. [26] Miles M, Huberman A. Qualitative data analysis: a sourcebook of new methods. Newbury Park (CA): Sage Publications; 1994. [27] Cook T, Campbell D. Quasi-experimentationddesign and analysis issues for field settings. Boston: Houghton-Mifflin; 1979. [28] McCutcheon D, Meridith J. Conducting case study research in operations management. Journal of Operations Management 1993;11(3):239–56. [29] Eisenhardt K. Building theories from case study research. The Academy of Management Review 1989;14(4):532–50. [30] ISO 19011:2002. Guidelines for quality and/or environmental management systems auditing; 2002. [31] Beltra´n J. Evamed IAT; 2008. Available at: http://www.iat.es/simce/ evamed.