- Email: [email protected]
Human error risk management methodology for safety audit of a large railway organisation
ARTICLE IN PRESS
Applied Ergonomics 36 (2005) 709–718 www.elsevier.com/locate/apergo
Human error risk management methodology for safety audit of a large railway organisation P.C. Cacciabue European Commission-Joint Research Centre, Institute for the Protection and Security of the Citizen, Via E. Fermi, 1, I-21020 Ispra, Fermi, Italy Received 15 July 2004; accepted 4 April 2005
Abstract This paper considers the application of the human error risk management for engineering systems (HERMES) methodology for safety assessment studies. The application concerns the recurrent safety audits (RSA) of a large organisation in the domain of railway transportation systems. The objective of the study was the identification of the most relevant areas of intervention for improving safety and reliability of the service. The methodology has been applied to the whole organisation and its working processes. Specific attention was paid to train drivers. A number of critical indicators of safety and recurrent safety audit matrices were identified, which enabled the assessment of the safety level of the organisation and the generation of safety recommendations. The application of HERMES to this case study shows that the methodology is applicable in practice and can give valuable and significant results. r 2005 Elsevier Ltd. All rights reserved. Keywords: Rail human factors; Recurrent safety audit; Modelling human machine interaction; Field studies
1. Introduction The need to include human factors (HF) considerations in the design and safety assessment processes of technological systems is nowadays widely recognised by almost all stakeholders of technology, from end-users to providers and regulators. The critical role assigned to HF in design and safety assessment is further enhanced by the common sense appreciation that it is impossible to conceive a plant that is totally ‘‘human-error free’’, which must be considered an intrinsic characteristic of any technological system. Therefore, the improvement of the safety level of a system can only be achieved through the implementation of appropriate measures that exploit the enormous power of both human skills and automation for preventing or recovering from human errors, and for Corresponding author. Tel: +39 322 78 9869; fax:+39 322 78 5813
E-mail address: [email protected]. 0003-6870/$ - see front matter r 2005 Elsevier Ltd. All rights reserved. doi:10.1016/j.apergo.2005.04.005
mitigating the consequences of those errors that still occur and cannot be recovered. This represents a process of human error and accident management (HEAM). From an HF perspective, the constituent elements of complex technologies can be identified in the presence and interconnection of organisational processes and cultural traits, local working conditions, defences, barriers and safeguards, and personal and external factors (Reason, 1997). These elements have to be evaluated in order to examine the state of a system or organisation with respect to its safety conditions. Moreover, the intrinsic dynamic nature of any organisation demands that these elements are regularly evaluated with adequate approaches that emphasise the cognitive ergonomics aspects of human–machine interactions (HMI) (Wilson et al., 2001). For these reasons, recurrent safety audits (RSA) attempt to evaluate, at regular intervals, the safety state (level) of an organisation with respect to a variety of safety indicators and markers. The performance of a
ARTICLE IN PRESS P.C. Cacciabue / Applied Ergonomics 36 (2005) 709–718
710
RSA requires a methodological framework where different methods and approaches are combined and integrated for considering HMI. RSAs are essential key processes for preserving the integrity of organisations and for preventing and protecting from accidents. In this paper, a methodological reference framework, called human error risk management for engineering systems (HERMES), is presented which offers a ‘‘roadmap’’ for selecting and applying coherently and consistently HEAM approaches, including the performance of RSA (Cacciabue, 2004a). In particular, the points of view proposed by HERMES as basic reference conditions before and during any analysis or safety study are firstly reviewed. Then, the logical and temporal sequence of models and methods for performing an HMI study will be discussed, focusing on their integration within the overall design and safety assessment process. This represents the methodological framework HERMES. Finally, a specific application of HERMES for the safety audit of a large railway organisation will be discussed in detail.
2. Standpoints for performance of human machine studies A number of standpoints should be considered when studying a human–machine system (HMS). Five standpoints are to be set at the beginning of any study (Fig. 1). These are: definition of the goals of the HMS; concept and scope of the HMS under study; types of analyses, i.e., prospective and retrospective approaches; areas of application; and measures of safety levels. 1. Goals of the HMS under study and/or development. The first standpoint demands that the goals of the systems under study are clearly identified and constantly accounted for during the whole process. As an example, the defences, barriers and safeguards (DBS) represent all structures and components, either physical or social, that are designed, programmed, and inserted in the HMS with the objective of making more efficient and safe the management of a plant, in normal and emergency conditions. When studying or designing a STANDPOINT 1: Goals of “Human Machine System” Goals of HEAM • Prevention of accidents • Recovery from accidents • Containment of consequences STANDPOINT 2: Concept of “Human Machine System” • Models of Human Machine Interactions, as events embedded & dependent on socio-technical context and dynamic contingencies STANDPOINT 3: Prospective - retrospective approaches • Commonality of reference models of HMI
STANDPOINT 4: Areas of application • Deigns • Training • Accident investigation • Safety Assessment STANDPOINT 5: Measures safety levels • Safety critical indicators as guiding elements of safety assessment • Recurrent Safety Audit as means for dynamic safety assessment
• Data and parameters:outcome of retrospective & input to prospective methods
Fig. 1. Standpoints for HMS studies.
specific DBS, the analyst/designer should clearly identify whether the DBS under study aims at tackling one or all three objectives of HEAM, namely: prevention, recovery from human errors and containment of the consequences that result from their occurrence. This process defines and bounds limits and performances of the DBS. 2. Concept and scope of the HMS. According to this standpoint, it is essential to develop a clear understanding of human performances or behaviours and their dependence on specific dynamic context (contingencies) and socio-technical environment in which they are embedded. This leads to the development of a predictive model of the HMS under analysis, which describes in qualitative and, possibly, quantitative terms humans, machines and environments and their interactions. 3. Perspective and retrospective approaches. The models and methods necessary for the development of HEAM can be classified as retrospective and prospective studies. These are complementary to each other and contribute equally to the development and assessment of HEAM measures. They rest on common empirical and theoretical platforms, i.e., models and taxonomies of HMI. Consistency and complementarity between the two types of study are assured by the data and parameters, which are derived from retrospective studies and can be applied for predicting consequences of HMIs in prospective studies. The need to consider and harmonize retrospective and prospective analyses is the third standpoint for the analyst of HEAM measures. 4. Areas of application. Any implementation of human factors analysis can be framed in one of the four main areas of application, namely, design, training, safety assessment, and accident investigation. Each of these four areas of application encompasses specific types of assessment. The fourth standpoint for the development of effective HEAM measures lies in the appreciation that a variety of HF tools and approaches must be applied for the verification that adequate safety conditions exist and are maintained before and during the lifetime of a system. 5. Measures of safety levels. The fifth standpoint is related to the definition of appropriate measures of safety levels of a plant. In all types of analysis, and for all areas of application, it is essential that adequate indicators be identified that allow the estimation or measurement of the safety level of a system. In particular, the performance of recurrent safety audits requires that a number of indicators of safety (IoS) define the level of safety of a system/ organisation. Moreover, these IoS are structured in a safety matrix that combines each IoS with the specific objectives of the HEAM measure under scrutiny, with the frequency of audit, and with reference values of IoS defined by regulations and standards.
ARTICLE IN PRESS P.C. Cacciabue / Applied Ergonomics 36 (2005) 709–718
From the above discussion, it is clearly seen that these five standpoints are strongly related and overlap each other in many circumstances. This is quite normal, as the design or safety assessment of an HMS represents a complicated process, which is distributed over several areas of application and requires the integration of different types of study. In principle, even though there is not a hierarchical scale amongst these five standpoints, the definition of measures of safety can only be obtained when all other standpoints are coherently considered and combined in a holistic process of analysis and evaluation of a system or organisation.
procedure. The steps to be carried out in the application of HERMES are the following:
3. A methodological framework The methodology described in Fig. 2 aims at implementing a stepwise procedure that respects the logical correlation between the five standpoints discussed above and sets the sequential process of application of methods and models for the performance of design or assessment of HEAM measures from the HF perspective. The methodology is called human error risk management for engineering systems (HERMES) (Cacciabue, 2004a). HERMES must be supported by existing models and specific methods for performing each step of the
EVALUATION OF SOCIOTECHNICAL CONTEXT Ethnographic Studies Cognitive Task Analysis
Retrospective Analysis
Root Cause Analysis
THEORETICAL STAND Models and Taxonomies of HMI
Prospective Analysis
Evaluation: • Data, Influencing factors • Erroneous behaviour
ACCIDENT/INCIDENT INVESTIGATION
Identification: • Boundary conditions • Initial conditions
Identification: • Causes, effects, reasons • Parameters, markers
Evaluation: • Consequences • Hazards
711
Firstly, it is necessary to select a common theoretical platform for both retrospective and prospective types of analysis. In combination with this theoretical basis, data and parameters typical of the system are derived by ethnographic studies and cognitive task analysis. Iterations between these two crucial steps may be necessary in order to ensure that the theoretical model fits the reality and availability of data and information. Then a set of data, influencing factors and erroneous behaviours are evaluated by investigation of past accidents and incidents, based on root cause analysis (RCA). This process leads to the identification of parameters and markers of cognitive behaviour, as well as causes, effects and reasons for human behaviours. At this point, a complete prospective study can be carried out: o a body of data, influencing factors and erroneous behaviours can be evaluated from the retrospective analysis; o the analyst and designer must then apply their experience and creativity for identifying boundary and initial conditions; and o predictive safety studies can be performed for evaluating unwanted consequences and hazards. The outcome of this prospective analysis can then be utilised for design and safety assessment purposes, as well as for testing and training.
In this way, HMI methods, data and parameters derived from retrospective analyses may be consistently and coherently applied for design, safety assessment and accident investigation, as well as for tailored training. The HEREMES methodology has been applied to tackle human factors issues in all areas of application, namely design, safety assessment, accident analysis, and training. In particular, the possible implementation of the methodology for performing human risk assessment analysis has been presented elsewhere (Cacciabue, 2004b). In the following, the application of HERMES to the performance of safety audit is shown with particular reference to the domain of railway systems.
4. Application of HERMES for safety audit of a large railway organisation TRAINING
DESIGN & SAFETY ASSESSMENT
Fig. 2. Human error risk management for engineering systems (HERMES).
4.1. Problem statement and boundaries of study The top management of a large European railway company (called ERC for convenience in the remaining
ARTICLE IN PRESS 712
P.C. Cacciabue / Applied Ergonomics 36 (2005) 709–718
part of this paper) felt the need to carry out a study of the human factors and safety–related issues existing within the organisation, with particular attention to the population of train drivers. The objective of such study was the identification of the most relevant areas of intervention for improving safety and reliability of the service, and in general to ascertain the current state of and possible needs for human error and accident management measures. The need to develop improvements and preventive actions of safety is very often the reaction to an accident. This was the case also for the study presented here. However, the resulting measures, if well developed and designed, can be very ‘‘proactive’’ and should enable the consideration of a wide variety of safety improvements that go beyond the development of countermeasures solely focused on the causes and factors associated to the specific accident. The ERC organisation consisted of more than 100,000 employees, engaged every day in the management of the railway traffic, maintenance, and service in general. The train drivers represented more than 70% of the entire population of employees. The technology and means available at ERC consisted of a high number of ‘‘trains’’ presenting a wide variety of technological solutions on board, from the most modern fully automatic controlled, high-speed machines to very conventional locomotives based practically on full manual control. At the time of the study, the railway network covered almost 20,000 km over a vast territory and a variety of terrains. Approximately 10,000 trains per day ensured the movement of several hundred of thousands of passengers. The application of HERMES was limited to the identification of safety critical factors, or IoS, and the development of recurrent safety audit matrices (RSAMatrices) that serve the purpose of defining the existing level of safety within the organisation and defining the reference measures for future audits. 4.2. Application of HERMES The HERMES methodology was performed in three strongly related phases. A major goal of the study was to transfer to the safety specialists of the company the methodological know-how and information that would enable ERC to carry out future audit exercises and safety evaluations from within the organisation, as this is the most efficient approach to ensure continuity and consistency of safety checks and performance. Phase 1. This phase included the setting up of a team of human factors experts and working groups supporting the study, and covered the initial steps of HERMES: (a) acquisition of information and knowledge in the field about the working environments and practices of train driving (ethnographic studies);
(b) Study of work requirements by task and job analysis; and (c) Identification of theoretical models and techniques to apply. Phase 2. The second phase of work was dedicated to the extensive field assessment and thus to the collection of the most important sources of information and data through questionnaires and interviews. The analysis of all data collected aimed at identifying possible areas of concern. Moreover, the annual reports of ERC on incidents and accidents were made available and could be studied from a retrospective point of view. Phase 3. The third phase was totally dedicated to the core development of the safety audit and preparation of recommendations. The identification of safety indicators and matrices, as well as the recommendations on safety improvements were performed with reference to the results of phase 2, exploiting the experience and creativity of the HF analysts. 4.2.1. Phase 1: Ethnographic studies and theoretical stand This activity involved some staff members and managers at different levels within the organisation. The focus of this phase was to select a consistent number of train drivers, or the most representative depots, that would represent the reference populations of drivers and depots for further detailed interviews and data collection. Training procedures and books with norms, regulations and standards were also made available by ERC, as a part of theoretical documentation. Phase 1 consisted of five main steps, namely: (1) creation of a human factors team and steering committee; (2) preliminary selection of model and taxonomy; (3) preparation of ethnographic studies; (4) task analysis and initial ethnographic studies; and (5) adjustment of model and taxonomy. An essential outcome of this process was the identification of a model for representing the sociotechnical interactions, and the workflows and procedures applied within the organisation (Fig. 3). The model SHELL (Edwards, 1988) was selected as a reference for the analysis of the behaviour of train drivers and their working context and environment. This model is a consolidated framework and has been adopted in other domains strongly affected by human factors issues (ICAO, 1992, 1993). The chart of human factors relationships was particularly important for identifying an initial set of factors that affect performance and behaviour of drivers, called performance influencing factors (PIFs), and for preparing the guidelines for the interviews and field observations. The basic elements of the SHELL model represent the human being (Liveware, L), that plays a central role in a system made of other humans (L), their socio-technical
ARTICLE IN PRESS P.C. Cacciabue / Applied Ergonomics 36 (2005) 709–718
713
ERC
TD & his/her colleagues L-L
Relations with ERC Management L-E
Train driver & working context L-H
Train Driver TD
Support systems & resources L-S
L = Liveware S = Software H = Hardware E = Environment
Machine controller
Maintenance Distributor
Maintenance workshop
Family
Department Head Dispatcher Traffic Management Directorate
Traction engineer
Crew Train Controller
TD
Society
Partner TD
Production manager Area coordinator Unit Head
Quality manager
Line Instructor
Other TDs colleagues
Traffic managers
ERC Trainer
Maneuvers engineer
Unions
Fig. 3. SHELL model and workflows for the organisation ERC.
working environment (E), the hardware (H), and software (S) applied during the performance of operations and HMI. The train driver (or central Liveware element) covers all aspects associated with the individual characteristics and performances, including physical attributes, physiological, and psychological issues. The interactions between the central Liveware and the other elements of the model imply that:
Liveware– Liveware interface (L– L) covers communications, supervision, and checks with other persons collaborating with the train driver. Liveware– Hardware interface (L– H) refers to the interactions with the actual cabin instrumentation, equipment, and any supporting material which may be utilised to carry out a job or task. Liveware– Environment interface (L– E) covers all aspects associated with the driver interacting with the physical environment, as well as the social and company/management. Liveware– Software interface (L– S) accounts for indirect or non-tangible issues affecting the activity of drivers, such as training and procedures.
The next step focused on the definition of the tasks of a train driver. This required the combination of field observations with the study of supporting material provided by the organisation. In particular, the attention of the HF-team was dedicated to the evaluation of safety aspects, norms and standards contained in the reference books and training instructions. To identify critical tasks with respect to safety, a number of interviews with ERC Managers and ‘‘line instructors’’ were performed. The results were compared with the analysis of the reference books and training instructions. Priority was given to the practical aspects of performance of tasks rather than theoretical ones. The combination of the task analysis and SHELL model helped in refining important PIFs. The theoretical instrument selected to represent formally the tasks of train drivers was hierarchical task analysis (HTA) (Kirwan and Ainsworth, 1992). HTA allows description in a simple structured format (tree like), the sequence of tasks to be carried out by drivers. It starts by defining high level tasks, such as ‘‘train driving’’, and develops detailed specific activities, such as actions like ‘‘opening of doors on arrival at platform’’.
ARTICLE IN PRESS 714
P.C. Cacciabue / Applied Ergonomics 36 (2005) 709–718
4.2.2. Phase 2: Identification of data and parameters The work of data analysis was performed with the support of a selected number of experts of the company, including train drivers and managers. The activity involved substantial field studies, data collection, and analysis. It required the application of well-established tools and methods, such as software specifically dedicated to statistical analysis. It consisted of four main steps: (1) cab-rides and time-line analysis; (2) detailed interviews; (3) definition of an error analysis method; and (4) analysis of data collected from questionnaires and interviews. Cab-rides were carried out immediately before and after the first set of interviews and workshops. They helped the HF-team in refining the task analysis method and in preparing the study of the workload associated with the performance of the driver’s task, especially with respect to available times for performing the required duties. The theoretical instrument selected to represent graphically the time distribution of the work of drivers and of associated workload was the time line analysis (TLA) (Kirwan and Ainsworth, 1992). TLA allows representation, in a simple format, of the sequence of action carried out by drivers during the performance of tasks and can be utilised to compare the workload of the drivers during the performance of tasks and to compare the workload of the co-drivers present in the cabin and working together. TLA focuses on dynamic aspects. The second step was the performance of several interviews and collection of questionnaire data. Three main depots had been identified as representative of the organisation, and extensive data collection in these depots was carried out. More than 300 drivers were interviewed. The questionnaire was distributed to the entire population of the three depots, approximately 2500 drivers. Altogether, 710 questionnaires were colleted and the data were stored in a database. Step three consisted of the selection of a technique for representing error generation and management during the activity of a driver. This process was strongly influenced by the fact that the HF-team aimed at selecting a simple theoretical configuration of HMI. The goal was to favour the discussion with drivers for the identification of PIFs that should be associated with possible error types and error modes, in relation to the SHELL model, selected as a reference for representing the socio-technical factors existing within the company. With these objectives in mind, the theoretical framework THERP (technique for human error rate prediction) (Swain and Guttmann, 1983) was selected. THERP is normally utilised in human reliability assessment to calculate probabilities of human errors associated with the performance of tasks. However, in this case, only the basic theoretical structure and graphical representation of the method was utilised, leaving aside the quantifica-
tion part. The THERP ‘‘human error’’ structure is based on the development of binary alternatives at every action point during the development of a task. Each branch generated in this way is then followed until either the task is successfully or unsuccessfully completed, or a recovery point is reached and that specific branch is combined with another one. In step four, the combination of the results of the data collected with the questionnaires and the information retrieved from the interviews led to the identification of eight areas of influence and eight generic PIFs that may lead to different types of errors, such as mistakes due to lack of knowledge, violations, lapses, or simple slips that may endanger seriously the safe performance of the service (Reason, 1997) (Table 1). 4.2.3. Phase 3: Definition of IoS and RSA-Matrices for the safety audit This phase of work merged two essential components, namely the information and experience gained by the HF team during the field studies, and the outcome of task analyses and study of reports on incidents and accidents from past operating experience. The objective of this last phase of work was the development of the IoSs and RSA-Matrices to be applied for assessing the state of the organisation. This phase was conducted in two subsequent steps: (1) development of THERP-trees; and (2) generations of IoS and RSA-Matrices, and safety recommendations. In step 1, a variety of scenarios were studied, and the associated THERP-trees were developed. For each scenario, the success or failure of the mission was evaluated in consideration of PIFs, workloads, working processes, and expected performances of drivers. The second step focused on the generation of a set of IoSs and RSA-Matrices that would allow ERC to perform successive evaluations and internal audits of its own safety level. The IoSs were also the guiding elements for developing a number of safety recommendations. PIFs and possible types and modes of errors identified during the previous phases of work were considered for identifying relevant IoSs. In Table 2, the PIFs have been associated to possible causes/effects on drivers’ behaviour and consequently to corresponding families of IoSs. In summary, a total of 13 IoSs were utilised for performing the safety audit: two IoSs related to organisational processes (OP), two related to personal and external factors (PEF), five concerning local working conditions (LWC), and four to DBS. 4.3. Safety recommendations and results of safety audit 4.3.1. Safety recommendations Safety recommendations have been developed with reference to the PIFs and IoSs. Specific attention was
ARTICLE IN PRESS P.C. Cacciabue / Applied Ergonomics 36 (2005) 709–718
715
Table 1 Final set of PIFs affecting TD performance PIF
SHELL Reference—Error type
Error modes
Communication within ERC (PIF 1)
Serious problems encountered in the contacting managers for discussing rules and standards (L–E).
Errors and violations of rules
Communication means (PIF 2)
Inadequate communication technology between several actors, i.e, TDs with train crew controller, traffic manager, line instructors, etc (L–L).
Catastrophic errors in emergency conditions and high workloads
Technological interfaces (PIF 3)
Difficult ergonomics of signals in the cabin and inconsistent displacement of signals on the railway (L–H).
Possible occurrence of passing signals at danger
Maintenance of trains/railway (PIF 4)
Inadequate maintenance of trains and railway reduce reliability of system and increase difficulty of train driving (L–H).
Induced errors of traffic management and violations
Comfort of working contexts (PIF 5)
Inadequacy of working contexts and poor logistic of rest areas increase stress and workload on long distance shifts (L–E).
Errors due to stress
Roster and shifts planning (PIF 6)
Too little involvement of TDs in the definition of shifts; the ‘‘fixed couple’’ strategy may not contribute to safety (L–E).
Errors due to stress or complacency
Rules and regulations (PIF 7)
Too many complex rules and regulations, sometimes contrasting with each other (L–S).
Errors and violations of rules
Training methods and simulators (PIF 8)
Existing training is inadequate to cope with advanced automation and complexity of tasks (L–S).
Errors due to lack of knowledge and training
Table 2 Identification of IoSs with reference to PIFs PIF
Causes/effects
IoS
1
Serious problems encountered in contacting managers for discussing rules and standards Uncertainty about future—low morale Unions as unique channel for communicating with top management level
OP1 PEF1 OP2
Unwritten rules; reporting systems Mental conditions Role of unions vs. management
2
Obsolete technology for communication Inadequate maintenance of means Unclear rules for communication
LWC3 LWC3 DBS3
Quality of tools Maintenance of tools Policies, standards
3
Poor ergonomics of interfaces of train cabins Problems in understand/manage automation Inconsistency between signals on track/cabin
LWC1 LWC2 LWC4
Workplace design Automation Signals: track and cabin
4
Inadequate and insufficient maintenance of trains and tracks
DBS1 PEF2
Safety devices Conditions, stress
5
Obsolete technology for communication Poor comfort, rest areas, and long-owls Lack of development plans
LWC3 LWC1 PEF1
Quality of tools Workplaces Mental condition, morale
6
Heavy and too stiff TD shifts Inadequacy of TD team vs. safety
PEF2 LWC5
Physical conditions Job planning
7
Excess of rules and regulations
8
Inadequate training Insufficient expertise of trainers/instructors
DBS3 DBS4 DBS2 OP2
Policies, standards Procedures Training standards Human relationship
ARTICLE IN PRESS 716
P.C. Cacciabue / Applied Ergonomics 36 (2005) 709–718
dedicated to the identification of the areas of concern and definition of possible interventions or improvements. The safety recommendations started with some general considerations derived from the overall results of the investigation. In particular, the ERC was considered a company with a good overall safety record, given the amount of traffic sustained and accidents reported. Train drivers were in general proud of their role and dedicated to their job. The company was undergoing a process of renovation at organisational and technical levels that aimed at improving the service for the public, as well as the infrastructures and technological means (tracks, trains, communications). The burden of this process was mostly sustained by ERC personnel and management. However, the complexity and dimension of the activity rested too heavily on personal commitment and individual intervention, with little observation of the too many rules and regulations which, in some cases, were contradictory and inapplicable. Quite often, drivers were led to the application of unwritten rules and practices deemed necessary for performing the task. However, these practices were also perceived contrary to or bypassing the established safety procedures. A general revision of the policies and procedures applied within the company was therefore considered of primary importance, aiming at a more formal compliance with standard and emergency operating procedures (DBS3, DBS4). Moreover, the development and establishment of a confidential reporting system for incidents, near misses, and all other anomalies encountered while performing duties, was considered essential for improving communication with the management and policy makers of the company (OP1). The transformation process of the company caused uncertainty among staff about the future. Moreover, a certain number of problems in communications within the company aggravated the situation affecting the morale of personnel (PEF1). In particular, the presence of trade unions as the sole communication channel was not well accepted (OP2). The entire system for communicating with the personnel within ERC needed revision and improvement. Another important issue associated with the transformation of the company was the issue of job planning. As an example, the issue of the ‘‘number of train drivers’’ present in the cabin was considered very important to improve service while limiting stress and workload, IoS (LWC5, PEF2). A solution to this problem had to be studied and adequately planned, in order to balance the requirements of modern technology with drivers’ habits and expectations, in favour of safety and efficiency of performance.
From a more technical perspective, the effort of improving railway systems by introducing high-speed trains and automated controls was not associated with an equivalent improvement in the quality of means for communication (LWC3), via interfaces within and outside the cabin (LWC1, LWC2, LWC4). This problem was further aggravated by the fact that modern and traditional trains were very different in their layout and comfort. Therefore, cabins of different types of trains needed ergonomics redesign. Moreover, the overall signalling system needed revision in order to accommodate the co-existence trains of different technologies. The maintenance of trains and tracks was one of the areas that needed great amelioration and improvement. Too frequently, at least from the field observations, the existence of inadequate train or track conditions were compensated for by train driver actions, or even by circumventing rules and safety devices ‘‘in favour’’ of the service (DBS1), causing stressful mental and physical conditions (PEF2). This problem was considered of very high relevance and a strong recommendation was issued concerning the need to revise the whole system of maintenance. Finally another very important issue, linked to the introduction of automation and modern technology, was ‘‘training’’. Current standards and procedures for training drivers were considered insufficient and inadequate, both from the technical and human relations viewpoints (DBS2, OP2). Therefore, a recommendation to review the entire training process was issued. In particular, the introduction of new training courses dedicated to accident and emergency management from the human factors perspective was recommended, as is done in the aviation domain with crew resource management (Wiener et al., 1993). 4.3.2. RSA—Matrices From the set of 13 indicators of safety, four main RSA-Matrices were developed. Each RSA-Matrix focused on the IoSs associated with a basic element characterising the ERC, which were identified in (a) organisational processes, (b) personal and external factors, (c) local working conditions and (d) DBS (Fig. 4). An example of the RSA-Matrix relative to the DBS is shown in Table 3. Each IoS is associated with three elements: 1. Safety objectives of each IoS, i.e., prevention, recovery or protection of accidents/incidents (columns of RSA-Matrix), 2. Frequency (f) of performance of the safety audit, or evaluation of the specific IoS (cells of RSA-Matrix), 3. Type of approach for evaluating each IoS, e.g., field assessment (fld), medical checks (med), data analysis, questionnaire/interviews (int), and quality &
ARTICLE IN PRESS P.C. Cacciabue / Applied Ergonomics 36 (2005) 709–718
Organisational Processes
Personal and External Factors
Interviews /field analyses / workshops
717 Theories and Methods
Background material
First Interviews First Workshop Interview management
Workshop
Cabin Rides / Visits Depot
Acc. Statist. HTA Workload
Defences, Barriers, Safeguards
Fig. 4. Socio-technical elements of a human machine system.
Acc. Studies
Errors
Meet. Steer. Comm.
TLA
PIFs
Discussions with depot managers
Local Working Conditions
Rules & Proc.
PIFs
Cabin Rides / Visits depot
ERC
SHELL
Meet. Steer. Comm.
THERP
Recurrent Safety Audit • IoS • RSA-MATRICES
Fig. 5. Methods applied for the safety audit of a large railway company.
5. Summary and conclusions Table 3 ERC RSA-Matrix for the IoS DBS IoS—DBS: Defences, Barriers, Safeguards DBS1 ¼ Safety devices Quality & Engin. Checks
Prevention
Recovery
Containment
jeng
jeng
jeng
jint
jint
ffld
jfld
jeng
jeng
jeng
jeng
jeng
jint
jint
jint
DBS2 ¼ Training Questionnaire/ jint interviews Field assessment ffld DBS3 ¼ Policies, standards Quality & Engin. jeng Checks DBS4 ¼ Procedures, instructions, supervision Quality & Engin. Checks Questionnaire/ interviews
engineering checks (eng) (attributes of each cell of RSA-Matrix). In this way, it is possible to define what are the specific objectives associated with each IoS, i.e., whether its goals are of accident prevention, or recovery or protection or a combination of the three. Moreover, the frequency of performance of these assessments and of IoSs are defined, in order to ensure that safety standards are maintained throughout the life of a system. Finally, there is also definition of how to evaluate each specific IoS, by means of field studies, or engineering evaluation, medical analysis, etc.
This paper has considered the application of the HERMES (human error risk management for engineering systems) methodology for safety assessment studies. The application concerned the RSA of a large organisation in the domain of railway transportation systems. The objective of such a study was the identification of the most relevant areas of intervention for improving safety and reliability of the service and the identification of the critical indicators of safety. This case study showed that a substantial application of the HERMES methodology to large systems is feasible and can provide important and useful results. Fig. 5 shows the summary of methods and techniques utilised. The application of HERMES to a real railway working context and a large organisation showed that, in order to perform a consistent and accurate evaluation, a considerable effort of human factors analysis is needed. In particular, it is necessary to perform several and successive collections of data from field tests and carry out observations of real working contexts. Without this critical part of the work the application of the methodology may lead to trivial and possibly misleading results. References Cacciabue, P.C., 2004a. Guide to Applying Human Factors Methods. Springer, London, UK. Cacciabue, P.C., 2004b. Human error risk management for engineering systems: a methodology for design, safety assessment, accident investigation and training. Reliability Eng System Safety RE & SS 83, 229–240. Edwards, E., 1988. Introductory overview. In: Wiener, E.L., Nagel, D.C. (Eds.), Human Factors in Aviation. Academic Press, San Diego, CA, pp. 3–25.
ARTICLE IN PRESS 718
P.C. Cacciabue / Applied Ergonomics 36 (2005) 709–718
Kirwan, B., Ainsworth, L.K., 1992. Guide to Task Analysis. Taylor & Francis, London. ICAO, 1992. Circular 238-AN/143. Human Factor Digest No. 6: Ergonomics. ICAO, Montreal. ICAO, 1993. Circular 240-AN/144. Human Factor Digest No. 7: Investigation of Human Factors in Accidents and Incidents. ICAO, Montreal. Reason, J., 1997. Managing the Risks of Organisational Accidents. Ashgate, Aldershot, UK.
Swain, A. D., Guttmann, H. E., 1983. Handbook on human reliability analysis with emphasis on nuclear power plant application. NUREG/CR-1278. Wiener, E.L., Kanki, B.G., Helmreich, R.L. (Eds.), 1993. Cockpit Resource Management. Academic Press, San Diego, CA. Wilson, J.R., Cordiner, L., Nichols, S., Norton, L., Bristol, N., Clarke, T., Roberts, S., 2001. On the right track: systematic implementation of ergonomics in railway network control. Int J Cognition Technol Work, IJ-CTW 3 (4).
Applied Ergonomics 36 (2005) 709–718 www.elsevier.com/locate/apergo
Human error risk management methodology for safety audit of a large railway organisation P.C. Cacciabue European Commission-Joint Research Centre, Institute for the Protection and Security of the Citizen, Via E. Fermi, 1, I-21020 Ispra, Fermi, Italy Received 15 July 2004; accepted 4 April 2005
Abstract This paper considers the application of the human error risk management for engineering systems (HERMES) methodology for safety assessment studies. The application concerns the recurrent safety audits (RSA) of a large organisation in the domain of railway transportation systems. The objective of the study was the identification of the most relevant areas of intervention for improving safety and reliability of the service. The methodology has been applied to the whole organisation and its working processes. Specific attention was paid to train drivers. A number of critical indicators of safety and recurrent safety audit matrices were identified, which enabled the assessment of the safety level of the organisation and the generation of safety recommendations. The application of HERMES to this case study shows that the methodology is applicable in practice and can give valuable and significant results. r 2005 Elsevier Ltd. All rights reserved. Keywords: Rail human factors; Recurrent safety audit; Modelling human machine interaction; Field studies
1. Introduction The need to include human factors (HF) considerations in the design and safety assessment processes of technological systems is nowadays widely recognised by almost all stakeholders of technology, from end-users to providers and regulators. The critical role assigned to HF in design and safety assessment is further enhanced by the common sense appreciation that it is impossible to conceive a plant that is totally ‘‘human-error free’’, which must be considered an intrinsic characteristic of any technological system. Therefore, the improvement of the safety level of a system can only be achieved through the implementation of appropriate measures that exploit the enormous power of both human skills and automation for preventing or recovering from human errors, and for Corresponding author. Tel: +39 322 78 9869; fax:+39 322 78 5813
E-mail address: [email protected]. 0003-6870/$ - see front matter r 2005 Elsevier Ltd. All rights reserved. doi:10.1016/j.apergo.2005.04.005
mitigating the consequences of those errors that still occur and cannot be recovered. This represents a process of human error and accident management (HEAM). From an HF perspective, the constituent elements of complex technologies can be identified in the presence and interconnection of organisational processes and cultural traits, local working conditions, defences, barriers and safeguards, and personal and external factors (Reason, 1997). These elements have to be evaluated in order to examine the state of a system or organisation with respect to its safety conditions. Moreover, the intrinsic dynamic nature of any organisation demands that these elements are regularly evaluated with adequate approaches that emphasise the cognitive ergonomics aspects of human–machine interactions (HMI) (Wilson et al., 2001). For these reasons, recurrent safety audits (RSA) attempt to evaluate, at regular intervals, the safety state (level) of an organisation with respect to a variety of safety indicators and markers. The performance of a
ARTICLE IN PRESS P.C. Cacciabue / Applied Ergonomics 36 (2005) 709–718
710
RSA requires a methodological framework where different methods and approaches are combined and integrated for considering HMI. RSAs are essential key processes for preserving the integrity of organisations and for preventing and protecting from accidents. In this paper, a methodological reference framework, called human error risk management for engineering systems (HERMES), is presented which offers a ‘‘roadmap’’ for selecting and applying coherently and consistently HEAM approaches, including the performance of RSA (Cacciabue, 2004a). In particular, the points of view proposed by HERMES as basic reference conditions before and during any analysis or safety study are firstly reviewed. Then, the logical and temporal sequence of models and methods for performing an HMI study will be discussed, focusing on their integration within the overall design and safety assessment process. This represents the methodological framework HERMES. Finally, a specific application of HERMES for the safety audit of a large railway organisation will be discussed in detail.
2. Standpoints for performance of human machine studies A number of standpoints should be considered when studying a human–machine system (HMS). Five standpoints are to be set at the beginning of any study (Fig. 1). These are: definition of the goals of the HMS; concept and scope of the HMS under study; types of analyses, i.e., prospective and retrospective approaches; areas of application; and measures of safety levels. 1. Goals of the HMS under study and/or development. The first standpoint demands that the goals of the systems under study are clearly identified and constantly accounted for during the whole process. As an example, the defences, barriers and safeguards (DBS) represent all structures and components, either physical or social, that are designed, programmed, and inserted in the HMS with the objective of making more efficient and safe the management of a plant, in normal and emergency conditions. When studying or designing a STANDPOINT 1: Goals of “Human Machine System” Goals of HEAM • Prevention of accidents • Recovery from accidents • Containment of consequences STANDPOINT 2: Concept of “Human Machine System” • Models of Human Machine Interactions, as events embedded & dependent on socio-technical context and dynamic contingencies STANDPOINT 3: Prospective - retrospective approaches • Commonality of reference models of HMI
STANDPOINT 4: Areas of application • Deigns • Training • Accident investigation • Safety Assessment STANDPOINT 5: Measures safety levels • Safety critical indicators as guiding elements of safety assessment • Recurrent Safety Audit as means for dynamic safety assessment
• Data and parameters:outcome of retrospective & input to prospective methods
Fig. 1. Standpoints for HMS studies.
specific DBS, the analyst/designer should clearly identify whether the DBS under study aims at tackling one or all three objectives of HEAM, namely: prevention, recovery from human errors and containment of the consequences that result from their occurrence. This process defines and bounds limits and performances of the DBS. 2. Concept and scope of the HMS. According to this standpoint, it is essential to develop a clear understanding of human performances or behaviours and their dependence on specific dynamic context (contingencies) and socio-technical environment in which they are embedded. This leads to the development of a predictive model of the HMS under analysis, which describes in qualitative and, possibly, quantitative terms humans, machines and environments and their interactions. 3. Perspective and retrospective approaches. The models and methods necessary for the development of HEAM can be classified as retrospective and prospective studies. These are complementary to each other and contribute equally to the development and assessment of HEAM measures. They rest on common empirical and theoretical platforms, i.e., models and taxonomies of HMI. Consistency and complementarity between the two types of study are assured by the data and parameters, which are derived from retrospective studies and can be applied for predicting consequences of HMIs in prospective studies. The need to consider and harmonize retrospective and prospective analyses is the third standpoint for the analyst of HEAM measures. 4. Areas of application. Any implementation of human factors analysis can be framed in one of the four main areas of application, namely, design, training, safety assessment, and accident investigation. Each of these four areas of application encompasses specific types of assessment. The fourth standpoint for the development of effective HEAM measures lies in the appreciation that a variety of HF tools and approaches must be applied for the verification that adequate safety conditions exist and are maintained before and during the lifetime of a system. 5. Measures of safety levels. The fifth standpoint is related to the definition of appropriate measures of safety levels of a plant. In all types of analysis, and for all areas of application, it is essential that adequate indicators be identified that allow the estimation or measurement of the safety level of a system. In particular, the performance of recurrent safety audits requires that a number of indicators of safety (IoS) define the level of safety of a system/ organisation. Moreover, these IoS are structured in a safety matrix that combines each IoS with the specific objectives of the HEAM measure under scrutiny, with the frequency of audit, and with reference values of IoS defined by regulations and standards.
ARTICLE IN PRESS P.C. Cacciabue / Applied Ergonomics 36 (2005) 709–718
From the above discussion, it is clearly seen that these five standpoints are strongly related and overlap each other in many circumstances. This is quite normal, as the design or safety assessment of an HMS represents a complicated process, which is distributed over several areas of application and requires the integration of different types of study. In principle, even though there is not a hierarchical scale amongst these five standpoints, the definition of measures of safety can only be obtained when all other standpoints are coherently considered and combined in a holistic process of analysis and evaluation of a system or organisation.
procedure. The steps to be carried out in the application of HERMES are the following:
3. A methodological framework The methodology described in Fig. 2 aims at implementing a stepwise procedure that respects the logical correlation between the five standpoints discussed above and sets the sequential process of application of methods and models for the performance of design or assessment of HEAM measures from the HF perspective. The methodology is called human error risk management for engineering systems (HERMES) (Cacciabue, 2004a). HERMES must be supported by existing models and specific methods for performing each step of the
EVALUATION OF SOCIOTECHNICAL CONTEXT Ethnographic Studies Cognitive Task Analysis
Retrospective Analysis
Root Cause Analysis
THEORETICAL STAND Models and Taxonomies of HMI
Prospective Analysis
Evaluation: • Data, Influencing factors • Erroneous behaviour
ACCIDENT/INCIDENT INVESTIGATION
Identification: • Boundary conditions • Initial conditions
Identification: • Causes, effects, reasons • Parameters, markers
Evaluation: • Consequences • Hazards
711
Firstly, it is necessary to select a common theoretical platform for both retrospective and prospective types of analysis. In combination with this theoretical basis, data and parameters typical of the system are derived by ethnographic studies and cognitive task analysis. Iterations between these two crucial steps may be necessary in order to ensure that the theoretical model fits the reality and availability of data and information. Then a set of data, influencing factors and erroneous behaviours are evaluated by investigation of past accidents and incidents, based on root cause analysis (RCA). This process leads to the identification of parameters and markers of cognitive behaviour, as well as causes, effects and reasons for human behaviours. At this point, a complete prospective study can be carried out: o a body of data, influencing factors and erroneous behaviours can be evaluated from the retrospective analysis; o the analyst and designer must then apply their experience and creativity for identifying boundary and initial conditions; and o predictive safety studies can be performed for evaluating unwanted consequences and hazards. The outcome of this prospective analysis can then be utilised for design and safety assessment purposes, as well as for testing and training.
In this way, HMI methods, data and parameters derived from retrospective analyses may be consistently and coherently applied for design, safety assessment and accident investigation, as well as for tailored training. The HEREMES methodology has been applied to tackle human factors issues in all areas of application, namely design, safety assessment, accident analysis, and training. In particular, the possible implementation of the methodology for performing human risk assessment analysis has been presented elsewhere (Cacciabue, 2004b). In the following, the application of HERMES to the performance of safety audit is shown with particular reference to the domain of railway systems.
4. Application of HERMES for safety audit of a large railway organisation TRAINING
DESIGN & SAFETY ASSESSMENT
Fig. 2. Human error risk management for engineering systems (HERMES).
4.1. Problem statement and boundaries of study The top management of a large European railway company (called ERC for convenience in the remaining
ARTICLE IN PRESS 712
P.C. Cacciabue / Applied Ergonomics 36 (2005) 709–718
part of this paper) felt the need to carry out a study of the human factors and safety–related issues existing within the organisation, with particular attention to the population of train drivers. The objective of such study was the identification of the most relevant areas of intervention for improving safety and reliability of the service, and in general to ascertain the current state of and possible needs for human error and accident management measures. The need to develop improvements and preventive actions of safety is very often the reaction to an accident. This was the case also for the study presented here. However, the resulting measures, if well developed and designed, can be very ‘‘proactive’’ and should enable the consideration of a wide variety of safety improvements that go beyond the development of countermeasures solely focused on the causes and factors associated to the specific accident. The ERC organisation consisted of more than 100,000 employees, engaged every day in the management of the railway traffic, maintenance, and service in general. The train drivers represented more than 70% of the entire population of employees. The technology and means available at ERC consisted of a high number of ‘‘trains’’ presenting a wide variety of technological solutions on board, from the most modern fully automatic controlled, high-speed machines to very conventional locomotives based practically on full manual control. At the time of the study, the railway network covered almost 20,000 km over a vast territory and a variety of terrains. Approximately 10,000 trains per day ensured the movement of several hundred of thousands of passengers. The application of HERMES was limited to the identification of safety critical factors, or IoS, and the development of recurrent safety audit matrices (RSAMatrices) that serve the purpose of defining the existing level of safety within the organisation and defining the reference measures for future audits. 4.2. Application of HERMES The HERMES methodology was performed in three strongly related phases. A major goal of the study was to transfer to the safety specialists of the company the methodological know-how and information that would enable ERC to carry out future audit exercises and safety evaluations from within the organisation, as this is the most efficient approach to ensure continuity and consistency of safety checks and performance. Phase 1. This phase included the setting up of a team of human factors experts and working groups supporting the study, and covered the initial steps of HERMES: (a) acquisition of information and knowledge in the field about the working environments and practices of train driving (ethnographic studies);
(b) Study of work requirements by task and job analysis; and (c) Identification of theoretical models and techniques to apply. Phase 2. The second phase of work was dedicated to the extensive field assessment and thus to the collection of the most important sources of information and data through questionnaires and interviews. The analysis of all data collected aimed at identifying possible areas of concern. Moreover, the annual reports of ERC on incidents and accidents were made available and could be studied from a retrospective point of view. Phase 3. The third phase was totally dedicated to the core development of the safety audit and preparation of recommendations. The identification of safety indicators and matrices, as well as the recommendations on safety improvements were performed with reference to the results of phase 2, exploiting the experience and creativity of the HF analysts. 4.2.1. Phase 1: Ethnographic studies and theoretical stand This activity involved some staff members and managers at different levels within the organisation. The focus of this phase was to select a consistent number of train drivers, or the most representative depots, that would represent the reference populations of drivers and depots for further detailed interviews and data collection. Training procedures and books with norms, regulations and standards were also made available by ERC, as a part of theoretical documentation. Phase 1 consisted of five main steps, namely: (1) creation of a human factors team and steering committee; (2) preliminary selection of model and taxonomy; (3) preparation of ethnographic studies; (4) task analysis and initial ethnographic studies; and (5) adjustment of model and taxonomy. An essential outcome of this process was the identification of a model for representing the sociotechnical interactions, and the workflows and procedures applied within the organisation (Fig. 3). The model SHELL (Edwards, 1988) was selected as a reference for the analysis of the behaviour of train drivers and their working context and environment. This model is a consolidated framework and has been adopted in other domains strongly affected by human factors issues (ICAO, 1992, 1993). The chart of human factors relationships was particularly important for identifying an initial set of factors that affect performance and behaviour of drivers, called performance influencing factors (PIFs), and for preparing the guidelines for the interviews and field observations. The basic elements of the SHELL model represent the human being (Liveware, L), that plays a central role in a system made of other humans (L), their socio-technical
ARTICLE IN PRESS P.C. Cacciabue / Applied Ergonomics 36 (2005) 709–718
713
ERC
TD & his/her colleagues L-L
Relations with ERC Management L-E
Train driver & working context L-H
Train Driver TD
Support systems & resources L-S
L = Liveware S = Software H = Hardware E = Environment
Machine controller
Maintenance Distributor
Maintenance workshop
Family
Department Head Dispatcher Traffic Management Directorate
Traction engineer
Crew Train Controller
TD
Society
Partner TD
Production manager Area coordinator Unit Head
Quality manager
Line Instructor
Other TDs colleagues
Traffic managers
ERC Trainer
Maneuvers engineer
Unions
Fig. 3. SHELL model and workflows for the organisation ERC.
working environment (E), the hardware (H), and software (S) applied during the performance of operations and HMI. The train driver (or central Liveware element) covers all aspects associated with the individual characteristics and performances, including physical attributes, physiological, and psychological issues. The interactions between the central Liveware and the other elements of the model imply that:
Liveware– Liveware interface (L– L) covers communications, supervision, and checks with other persons collaborating with the train driver. Liveware– Hardware interface (L– H) refers to the interactions with the actual cabin instrumentation, equipment, and any supporting material which may be utilised to carry out a job or task. Liveware– Environment interface (L– E) covers all aspects associated with the driver interacting with the physical environment, as well as the social and company/management. Liveware– Software interface (L– S) accounts for indirect or non-tangible issues affecting the activity of drivers, such as training and procedures.
The next step focused on the definition of the tasks of a train driver. This required the combination of field observations with the study of supporting material provided by the organisation. In particular, the attention of the HF-team was dedicated to the evaluation of safety aspects, norms and standards contained in the reference books and training instructions. To identify critical tasks with respect to safety, a number of interviews with ERC Managers and ‘‘line instructors’’ were performed. The results were compared with the analysis of the reference books and training instructions. Priority was given to the practical aspects of performance of tasks rather than theoretical ones. The combination of the task analysis and SHELL model helped in refining important PIFs. The theoretical instrument selected to represent formally the tasks of train drivers was hierarchical task analysis (HTA) (Kirwan and Ainsworth, 1992). HTA allows description in a simple structured format (tree like), the sequence of tasks to be carried out by drivers. It starts by defining high level tasks, such as ‘‘train driving’’, and develops detailed specific activities, such as actions like ‘‘opening of doors on arrival at platform’’.
ARTICLE IN PRESS 714
P.C. Cacciabue / Applied Ergonomics 36 (2005) 709–718
4.2.2. Phase 2: Identification of data and parameters The work of data analysis was performed with the support of a selected number of experts of the company, including train drivers and managers. The activity involved substantial field studies, data collection, and analysis. It required the application of well-established tools and methods, such as software specifically dedicated to statistical analysis. It consisted of four main steps: (1) cab-rides and time-line analysis; (2) detailed interviews; (3) definition of an error analysis method; and (4) analysis of data collected from questionnaires and interviews. Cab-rides were carried out immediately before and after the first set of interviews and workshops. They helped the HF-team in refining the task analysis method and in preparing the study of the workload associated with the performance of the driver’s task, especially with respect to available times for performing the required duties. The theoretical instrument selected to represent graphically the time distribution of the work of drivers and of associated workload was the time line analysis (TLA) (Kirwan and Ainsworth, 1992). TLA allows representation, in a simple format, of the sequence of action carried out by drivers during the performance of tasks and can be utilised to compare the workload of the drivers during the performance of tasks and to compare the workload of the co-drivers present in the cabin and working together. TLA focuses on dynamic aspects. The second step was the performance of several interviews and collection of questionnaire data. Three main depots had been identified as representative of the organisation, and extensive data collection in these depots was carried out. More than 300 drivers were interviewed. The questionnaire was distributed to the entire population of the three depots, approximately 2500 drivers. Altogether, 710 questionnaires were colleted and the data were stored in a database. Step three consisted of the selection of a technique for representing error generation and management during the activity of a driver. This process was strongly influenced by the fact that the HF-team aimed at selecting a simple theoretical configuration of HMI. The goal was to favour the discussion with drivers for the identification of PIFs that should be associated with possible error types and error modes, in relation to the SHELL model, selected as a reference for representing the socio-technical factors existing within the company. With these objectives in mind, the theoretical framework THERP (technique for human error rate prediction) (Swain and Guttmann, 1983) was selected. THERP is normally utilised in human reliability assessment to calculate probabilities of human errors associated with the performance of tasks. However, in this case, only the basic theoretical structure and graphical representation of the method was utilised, leaving aside the quantifica-
tion part. The THERP ‘‘human error’’ structure is based on the development of binary alternatives at every action point during the development of a task. Each branch generated in this way is then followed until either the task is successfully or unsuccessfully completed, or a recovery point is reached and that specific branch is combined with another one. In step four, the combination of the results of the data collected with the questionnaires and the information retrieved from the interviews led to the identification of eight areas of influence and eight generic PIFs that may lead to different types of errors, such as mistakes due to lack of knowledge, violations, lapses, or simple slips that may endanger seriously the safe performance of the service (Reason, 1997) (Table 1). 4.2.3. Phase 3: Definition of IoS and RSA-Matrices for the safety audit This phase of work merged two essential components, namely the information and experience gained by the HF team during the field studies, and the outcome of task analyses and study of reports on incidents and accidents from past operating experience. The objective of this last phase of work was the development of the IoSs and RSA-Matrices to be applied for assessing the state of the organisation. This phase was conducted in two subsequent steps: (1) development of THERP-trees; and (2) generations of IoS and RSA-Matrices, and safety recommendations. In step 1, a variety of scenarios were studied, and the associated THERP-trees were developed. For each scenario, the success or failure of the mission was evaluated in consideration of PIFs, workloads, working processes, and expected performances of drivers. The second step focused on the generation of a set of IoSs and RSA-Matrices that would allow ERC to perform successive evaluations and internal audits of its own safety level. The IoSs were also the guiding elements for developing a number of safety recommendations. PIFs and possible types and modes of errors identified during the previous phases of work were considered for identifying relevant IoSs. In Table 2, the PIFs have been associated to possible causes/effects on drivers’ behaviour and consequently to corresponding families of IoSs. In summary, a total of 13 IoSs were utilised for performing the safety audit: two IoSs related to organisational processes (OP), two related to personal and external factors (PEF), five concerning local working conditions (LWC), and four to DBS. 4.3. Safety recommendations and results of safety audit 4.3.1. Safety recommendations Safety recommendations have been developed with reference to the PIFs and IoSs. Specific attention was
ARTICLE IN PRESS P.C. Cacciabue / Applied Ergonomics 36 (2005) 709–718
715
Table 1 Final set of PIFs affecting TD performance PIF
SHELL Reference—Error type
Error modes
Communication within ERC (PIF 1)
Serious problems encountered in the contacting managers for discussing rules and standards (L–E).
Errors and violations of rules
Communication means (PIF 2)
Inadequate communication technology between several actors, i.e, TDs with train crew controller, traffic manager, line instructors, etc (L–L).
Catastrophic errors in emergency conditions and high workloads
Technological interfaces (PIF 3)
Difficult ergonomics of signals in the cabin and inconsistent displacement of signals on the railway (L–H).
Possible occurrence of passing signals at danger
Maintenance of trains/railway (PIF 4)
Inadequate maintenance of trains and railway reduce reliability of system and increase difficulty of train driving (L–H).
Induced errors of traffic management and violations
Comfort of working contexts (PIF 5)
Inadequacy of working contexts and poor logistic of rest areas increase stress and workload on long distance shifts (L–E).
Errors due to stress
Roster and shifts planning (PIF 6)
Too little involvement of TDs in the definition of shifts; the ‘‘fixed couple’’ strategy may not contribute to safety (L–E).
Errors due to stress or complacency
Rules and regulations (PIF 7)
Too many complex rules and regulations, sometimes contrasting with each other (L–S).
Errors and violations of rules
Training methods and simulators (PIF 8)
Existing training is inadequate to cope with advanced automation and complexity of tasks (L–S).
Errors due to lack of knowledge and training
Table 2 Identification of IoSs with reference to PIFs PIF
Causes/effects
IoS
1
Serious problems encountered in contacting managers for discussing rules and standards Uncertainty about future—low morale Unions as unique channel for communicating with top management level
OP1 PEF1 OP2
Unwritten rules; reporting systems Mental conditions Role of unions vs. management
2
Obsolete technology for communication Inadequate maintenance of means Unclear rules for communication
LWC3 LWC3 DBS3
Quality of tools Maintenance of tools Policies, standards
3
Poor ergonomics of interfaces of train cabins Problems in understand/manage automation Inconsistency between signals on track/cabin
LWC1 LWC2 LWC4
Workplace design Automation Signals: track and cabin
4
Inadequate and insufficient maintenance of trains and tracks
DBS1 PEF2
Safety devices Conditions, stress
5
Obsolete technology for communication Poor comfort, rest areas, and long-owls Lack of development plans
LWC3 LWC1 PEF1
Quality of tools Workplaces Mental condition, morale
6
Heavy and too stiff TD shifts Inadequacy of TD team vs. safety
PEF2 LWC5
Physical conditions Job planning
7
Excess of rules and regulations
8
Inadequate training Insufficient expertise of trainers/instructors
DBS3 DBS4 DBS2 OP2
Policies, standards Procedures Training standards Human relationship
ARTICLE IN PRESS 716
P.C. Cacciabue / Applied Ergonomics 36 (2005) 709–718
dedicated to the identification of the areas of concern and definition of possible interventions or improvements. The safety recommendations started with some general considerations derived from the overall results of the investigation. In particular, the ERC was considered a company with a good overall safety record, given the amount of traffic sustained and accidents reported. Train drivers were in general proud of their role and dedicated to their job. The company was undergoing a process of renovation at organisational and technical levels that aimed at improving the service for the public, as well as the infrastructures and technological means (tracks, trains, communications). The burden of this process was mostly sustained by ERC personnel and management. However, the complexity and dimension of the activity rested too heavily on personal commitment and individual intervention, with little observation of the too many rules and regulations which, in some cases, were contradictory and inapplicable. Quite often, drivers were led to the application of unwritten rules and practices deemed necessary for performing the task. However, these practices were also perceived contrary to or bypassing the established safety procedures. A general revision of the policies and procedures applied within the company was therefore considered of primary importance, aiming at a more formal compliance with standard and emergency operating procedures (DBS3, DBS4). Moreover, the development and establishment of a confidential reporting system for incidents, near misses, and all other anomalies encountered while performing duties, was considered essential for improving communication with the management and policy makers of the company (OP1). The transformation process of the company caused uncertainty among staff about the future. Moreover, a certain number of problems in communications within the company aggravated the situation affecting the morale of personnel (PEF1). In particular, the presence of trade unions as the sole communication channel was not well accepted (OP2). The entire system for communicating with the personnel within ERC needed revision and improvement. Another important issue associated with the transformation of the company was the issue of job planning. As an example, the issue of the ‘‘number of train drivers’’ present in the cabin was considered very important to improve service while limiting stress and workload, IoS (LWC5, PEF2). A solution to this problem had to be studied and adequately planned, in order to balance the requirements of modern technology with drivers’ habits and expectations, in favour of safety and efficiency of performance.
From a more technical perspective, the effort of improving railway systems by introducing high-speed trains and automated controls was not associated with an equivalent improvement in the quality of means for communication (LWC3), via interfaces within and outside the cabin (LWC1, LWC2, LWC4). This problem was further aggravated by the fact that modern and traditional trains were very different in their layout and comfort. Therefore, cabins of different types of trains needed ergonomics redesign. Moreover, the overall signalling system needed revision in order to accommodate the co-existence trains of different technologies. The maintenance of trains and tracks was one of the areas that needed great amelioration and improvement. Too frequently, at least from the field observations, the existence of inadequate train or track conditions were compensated for by train driver actions, or even by circumventing rules and safety devices ‘‘in favour’’ of the service (DBS1), causing stressful mental and physical conditions (PEF2). This problem was considered of very high relevance and a strong recommendation was issued concerning the need to revise the whole system of maintenance. Finally another very important issue, linked to the introduction of automation and modern technology, was ‘‘training’’. Current standards and procedures for training drivers were considered insufficient and inadequate, both from the technical and human relations viewpoints (DBS2, OP2). Therefore, a recommendation to review the entire training process was issued. In particular, the introduction of new training courses dedicated to accident and emergency management from the human factors perspective was recommended, as is done in the aviation domain with crew resource management (Wiener et al., 1993). 4.3.2. RSA—Matrices From the set of 13 indicators of safety, four main RSA-Matrices were developed. Each RSA-Matrix focused on the IoSs associated with a basic element characterising the ERC, which were identified in (a) organisational processes, (b) personal and external factors, (c) local working conditions and (d) DBS (Fig. 4). An example of the RSA-Matrix relative to the DBS is shown in Table 3. Each IoS is associated with three elements: 1. Safety objectives of each IoS, i.e., prevention, recovery or protection of accidents/incidents (columns of RSA-Matrix), 2. Frequency (f) of performance of the safety audit, or evaluation of the specific IoS (cells of RSA-Matrix), 3. Type of approach for evaluating each IoS, e.g., field assessment (fld), medical checks (med), data analysis, questionnaire/interviews (int), and quality &
ARTICLE IN PRESS P.C. Cacciabue / Applied Ergonomics 36 (2005) 709–718
Organisational Processes
Personal and External Factors
Interviews /field analyses / workshops
717 Theories and Methods
Background material
First Interviews First Workshop Interview management
Workshop
Cabin Rides / Visits Depot
Acc. Statist. HTA Workload
Defences, Barriers, Safeguards
Fig. 4. Socio-technical elements of a human machine system.
Acc. Studies
Errors
Meet. Steer. Comm.
TLA
PIFs
Discussions with depot managers
Local Working Conditions
Rules & Proc.
PIFs
Cabin Rides / Visits depot
ERC
SHELL
Meet. Steer. Comm.
THERP
Recurrent Safety Audit • IoS • RSA-MATRICES
Fig. 5. Methods applied for the safety audit of a large railway company.
5. Summary and conclusions Table 3 ERC RSA-Matrix for the IoS DBS IoS—DBS: Defences, Barriers, Safeguards DBS1 ¼ Safety devices Quality & Engin. Checks
Prevention
Recovery
Containment
jeng
jeng
jeng
jint
jint
ffld
jfld
jeng
jeng
jeng
jeng
jeng
jint
jint
jint
DBS2 ¼ Training Questionnaire/ jint interviews Field assessment ffld DBS3 ¼ Policies, standards Quality & Engin. jeng Checks DBS4 ¼ Procedures, instructions, supervision Quality & Engin. Checks Questionnaire/ interviews
engineering checks (eng) (attributes of each cell of RSA-Matrix). In this way, it is possible to define what are the specific objectives associated with each IoS, i.e., whether its goals are of accident prevention, or recovery or protection or a combination of the three. Moreover, the frequency of performance of these assessments and of IoSs are defined, in order to ensure that safety standards are maintained throughout the life of a system. Finally, there is also definition of how to evaluate each specific IoS, by means of field studies, or engineering evaluation, medical analysis, etc.
This paper has considered the application of the HERMES (human error risk management for engineering systems) methodology for safety assessment studies. The application concerned the RSA of a large organisation in the domain of railway transportation systems. The objective of such a study was the identification of the most relevant areas of intervention for improving safety and reliability of the service and the identification of the critical indicators of safety. This case study showed that a substantial application of the HERMES methodology to large systems is feasible and can provide important and useful results. Fig. 5 shows the summary of methods and techniques utilised. The application of HERMES to a real railway working context and a large organisation showed that, in order to perform a consistent and accurate evaluation, a considerable effort of human factors analysis is needed. In particular, it is necessary to perform several and successive collections of data from field tests and carry out observations of real working contexts. Without this critical part of the work the application of the methodology may lead to trivial and possibly misleading results. References Cacciabue, P.C., 2004a. Guide to Applying Human Factors Methods. Springer, London, UK. Cacciabue, P.C., 2004b. Human error risk management for engineering systems: a methodology for design, safety assessment, accident investigation and training. Reliability Eng System Safety RE & SS 83, 229–240. Edwards, E., 1988. Introductory overview. In: Wiener, E.L., Nagel, D.C. (Eds.), Human Factors in Aviation. Academic Press, San Diego, CA, pp. 3–25.
ARTICLE IN PRESS 718
P.C. Cacciabue / Applied Ergonomics 36 (2005) 709–718
Kirwan, B., Ainsworth, L.K., 1992. Guide to Task Analysis. Taylor & Francis, London. ICAO, 1992. Circular 238-AN/143. Human Factor Digest No. 6: Ergonomics. ICAO, Montreal. ICAO, 1993. Circular 240-AN/144. Human Factor Digest No. 7: Investigation of Human Factors in Accidents and Incidents. ICAO, Montreal. Reason, J., 1997. Managing the Risks of Organisational Accidents. Ashgate, Aldershot, UK.
Swain, A. D., Guttmann, H. E., 1983. Handbook on human reliability analysis with emphasis on nuclear power plant application. NUREG/CR-1278. Wiener, E.L., Kanki, B.G., Helmreich, R.L. (Eds.), 1993. Cockpit Resource Management. Academic Press, San Diego, CA. Wilson, J.R., Cordiner, L., Nichols, S., Norton, L., Bristol, N., Clarke, T., Roberts, S., 2001. On the right track: systematic implementation of ergonomics in railway network control. Int J Cognition Technol Work, IJ-CTW 3 (4).