- Email: [email protected]
SOL – Safety through organizational learning: A method for event analysis
Safety Science 49 (2011) 27–31
Contents lists available at ScienceDirect
Safety Science journal homepage: www.elsevier.com/locate/ssci
SOL – Safety through organizational learning: A method for event analysis Babette Fahlbruch a,*, Markus Schöbel b a b
TuV NORD SysTec GmbH & Co. KG, Zimmerstrasse 23, 10969 Berlin, Germany Berlin Institute of Technology, 10587 Berlin, Germany
a r t i c l e
i n f o
Article history: Received 21 January 2010 Received in revised form 30 April 2010 Accepted 9 May 2010
Keywords: Event analysis Organizational learning Organizational factors
a b s t r a c t Under the guidance of Bernhard Wilpert a research group at the Berlin University of Technology developed an event analysis methodology called safety through organizational learning (SOL). The method has been adopted by the Swiss and German nuclear industries as standard procedure for their in-depth event analyses. SOL aims at facilitating organizational learning from events by supporting the process of analyzing events, ensuring its standardized conduct and mobilizing expert knowledge and creativity in the analysis. In this paper we provide a short description of SOL and its theoretical background. We summarize the empirical evidence and practical experience regarding SOL, which proves it to be a valid methodology that gives sufficient support to analysts. Finally, practical experiences and challenges for future research are discussed. Ó 2010 Elsevier Ltd. All rights reserved.
1. Introduction Nearly 20 years ago the influence of human factors on incidents in German nuclear power plants became a topic of increasing interest and significance. At this time practitioners from the nuclear power industry understood human factors mainly as human errors at the man–machine interface, whereas safety researchers had a quite different understanding that subsumed organizational and environmental aspects under the term human factors (e.g. Reason, 1990). In 1992 a research group at Berlin University of Technology started under the guidance of Bernhard Wilpert with the development of a new event analysis methodology called safety through organizational learning (SOL). The authors of the present article worked in this research group. Bernhard Wilpert’s central idea was to propagate a holistic socio-technical system approach for analysing events which implies that human factors do not only relate to the immediate man–machine interface, but comprise human actions on all system levels in as much as they contribute to the critical outcome of safety. According to the organizational learning literature (e.g. Argyris and Schoen, 1978), one important goal of event analysis is to draw lessons from an event to prevent future events. Therefore, adequate recommendations and safety measures must be implemented for all contributing factors identified. When the research group started the development of SOL, event reports in nuclear industries discussed technical failures and human errors as the main contributors to events. Organizational or inter-organizational
* Corresponding author. Tel.: +49 30 20177454; fax: +49 30 20177458. E-mail address: [email protected] (B. Fahlbruch). 0925-7535/$ - see front matter Ó 2010 Elsevier Ltd. All rights reserved. doi:10.1016/j.ssci.2010.05.004
factors were only considered on an informal basis and did not get due weight in the lessons learned. The group speculated about potential reasons and came to the conclusion that event analysts might not have adequate knowledge about organizational factors, and therefore their methods did not explicitly cover these factors. Moreover, the strong focus on apparent factors might result from shortcomings in causal reasoning. Consequently, the development of SOL should fulfill the following requirements: it should cover a broad range of human, organizational, and inter-organizational factors derived from theory and empirical data; it should be easy to use without expert knowledge in human factors psychology, because it should be applied by operators and supervisors in nuclear power plants; it should help to overcome well known shortcomings in human causal reasoning which could lead to truncated search in event analysis; it should support organizational learning from events; it should have empirical validation. In the following sections we will describe the development of the method, show how SOL was evaluated and close with open questions and challenges for the future.
2. The development of SOL – safety through organizational learning SOL was initially developed for the nuclear power industry (Becker et al., 1995; Wilpert et al., 1997). However, a version for
28
B. Fahlbruch, M. Schöbel / Safety Science 49 (2011) 27–31
the chemical industry exists (Wilpert et al., 1998) and a computer supported version was developed as well (Maimer et al., 1999). At the time SOL was developed, event analyses were still dominated by approaches characterized by attributions of fault to individual or/and technical actors (Benner, 1981a,b; Hendrick and Benner, 1987; Manuele, 1982; Shealy, 1979). Nevertheless, in high hazard industries a trend of systematic analyses supported by different methodologies could be observed (Fahlbruch and Wilpert, 1999). Unfortunately some of those methodologies were not completely published (e.g. the German human-factor concept of Vereinigung der Großkraftwerksbetreiber – VGB and the Human Performance Enhancement System of INPO – Institute of Nuclear Power Operations). For the development of SOL we reviewed the following methodologies: ASSET – Assessment of Safety Significant Events Teams (IAEA, 1991, 1994); Change Analysis (Bullock, 1981); HPES – Human Performance Enhancement System (INPO Bishop and LaRhette, 1988); MORT – Management Oversight and Risk Tree (Johnson, 1973, 1980); STEP – Sequentially Timed Event Plotting (Hendrick and Benner, 1987); TOR – Technique of Operations and Review (Weaver, 1973). The above methodologies are based on different accident causation models, e.g. Change Analysis and HPES had no explicit model, whereas MORT and STEP were based on explicitly formulated models. The methodologies vary in degree of standardization from general requirements for the process (Change Analysis) up to a set of attributes evaluated as adequate or less than adequate (MORT). Although ASSET, MORT and TOR explicitly consider organizational factors, inter-organizational factors are not included in any of the reviewed methods. None of the methods had explicit features for overcoming judgmental biases or shortcomings in causal reasoning. 2.1. Theoretical background of SOL The development of SOL was based on the assumption that an event analysis is a socially accepted reconstruction of a surprise, i.e. the identification of what happened and why. Therefore, it was important to provide not only a method, but also a scientific event causation model which had to be accepted by practitioners in the field. SOL builds upon Reason’s Swiss-Cheese-Model (1990) and the socio-technical system approach (e.g. Trist and Bamforth, 1951). It was Bernhard Wilpert’s theoretical contribution to combine both models in a socio-technical event causation model. According to this model, contributing factors stem from five sub-systems and their interaction: individual, team, organization, extra-organizational environment and technology. These sub-systems were further divided into categories (or contributing factors) which were derived from organizational theory as well as from an extensive analysis of published event reports in the nuclear power industry (Becker et al., 1995). In order to promote interdisciplinary work and to develop a method which would be accepted by practitioners and scientists, Bernhard Wilpert put together a team of experts consisting of psychologists and nuclear engineers who evaluated these factors and assigned each to a subsystem or its interaction. Thus an initial set of 19 contributing factors resulted, but practical feedback from the field and expert assessments led to a modified version with 21 potential contributing factors. Furthermore, the factors were differentiated according to their potential direct and indirect (or latent) influence on event causation (see Table 1).
Table 1 SOL categories of contributing factors. Direct factors
Indirect factors
Information Communication Working condition
Operation scheduling Group influence Rules, procedures & documents Organization & management
Responsibility Control & supervision Qualification
Experience feedback Maintenance
Safety principles Quality management
Regulatory bodies
Environmental influence
Personal performance Rule violation Technical component
Training
Technology
The quality of an event analysis method is not only influenced by its underlying theoretical model, but also depends on the conceptualization of the analysis process and the supporting features (Fahlbruch et al., 1998). Fahlbruch (2000, 2001) modeled the psychological process of analyzing events by drawing on psychological research on attributional processes and causal inferences. She identified various psychological factors which follow from characteristics of human information processing and general attribution processes and which may diminish the quality of an event analysis such as: premature hypotheses leading to truncated search for information, i.e. the first plausible ‘‘cause” is taken to explain what happened (Einhorn and Hogarth, 1986); difficulties to identify contributing factors which are remote in time and space from the actual event (Einhorn and Hogarth, 1986) which result in an over-weighting of so-called active errors; mono-causal thinking and satisfaction with only one contributing factor although there may have been multiple factors involved (Shaklee and Fischhoff, 1982); identification of contributing factors because of recent past events (mental availability of potentially contributing factors) (Tversky and Kahnemann, 1981); ignoring contributing factors which are not written down in the method (e.g. in a given checklist or fault-tree) (‘‘out of sight – out of mind”) (Fischhoff et al., 1978); concentration on contributions by human actors directly involved in the event (fundamental attribution error) (Ross, 1977). SOL was intended to be a standardized process of event analysis that would minimize the above factors and cognitive biases. 2.2. Description of SOL Analyzing events with SOL is conceptualized as a backward oriented problem-solving process (Fahlbruch, 2000; Fahlbruch and Wilpert, 1997). SOL operationalizes the concept of event analysis in a set of two standardized process steps: (1) the description of the actual event situation, and (2) the identification of contributing factors. For both steps guidelines were developed which support the event analyst. As the first step of the analysis, a situational description is constructed. The information needed for the description of the event is gathered by interviews and document analysis. A set of questions helps the analyst to ask the right questions in order to completely reconstruct the course of an event. Based on the STEP method (Hendrick and Benner, 1987) the collected information is broken down into a sequence of so-called event-building blocks, i.e. the
B. Fahlbruch, M. Schöbel / Safety Science 49 (2011) 27–31
event is decomposed into a sequence of single micro-events to clarify and illustrate what happened (Wilpert and Fahlbruch, 1998; Fahlbruch and Wilpert, 1999). For each event-building block the information is categorized according to the actor (human and technical actors), the action, the point in time of the action, the location (where the action takes place) and additional remarks. Thus, an event is determined by a sequence of singular actions by different actors. The starting point of an event (i.e. the first event-building block) is defined as the first deviation from a warranted course of action. These deviations are identified by contrasting actions against formal procedures and technical system design or against ‘‘normal” system performance based on the appraisal of an event analyst. The end point (i.e. the last event-building block) is defined as the recovery of a safe system state. The situational description illustrates only observable facts (what happened). Actions which were not shown as well as hypotheses about potential causes should not be incorporated into the situational description. Each event-building block is graphically ordered in a time-actordiagram which provides an overview of the recomposed event and serves as an important information source for the subsequent identification of contributing factors. The identification of contributing factors, i.e. the second step, is conducted in the following way: for each event-building block a separate analysis is conducted. A guideline with categories of possible contributing factors – the identification aid – supports this second step. The identification aid consists of categories of potential contributing factors which cover individual, technical, group, organizational and inter-organizational aspects to guarantee a sufficient scope of investigation. In order to support the identification of contributing factors each factor is assigned to a general question. For instance, the factor ‘‘working conditions” is transferred into the question ‘‘Could there have been an influence of the working conditions on the operator’s performance?”. For each factor several specific examples are given to support the analysts, e.g. for ‘‘working conditions” the examples are time pressure, noise, heat, lights or disturbances. Thus, the aid contains general questions related to possible contributing factors covering each of the five sub-systems in order to ensure the comprehensiveness of the analysis. Since it is assumed that an event analyst may not exclusively be a human factors specialist, the aid also gives illustrative examples of potential influences of contributing factors with the aim to stimulate creative problem solving processes. These examples are concrete enough to cover a broad range of potentially contributing factors but they are not meant to be exhaustive. To guarantee the comprehensiveness of the analysis all general questions are linked to others. These so-called cross-references are theoretically and empirically based. If one question is answered in the affirmative, the team is guided to answer another set of questions in order to identify other potentially contributing factors. Contributing factors are roughly divided into direct and indirect factors. The analysis process starts with the identification of direct factors which are linked to a couple of indirect factors due to the cross-references. For instance, if the direct factor ‘‘personal performance” is identified, a cross-reference to the indirect factor ‘‘training” is given. By these cross-references mono-causal thinking and over-weighting of active errors should be overcome. Finally, all identified contributing factors are added to the time-actor-matrix, thus successively completing the reconstruction of the event and its causes. Altogether SOL is assumed to overcome the above mentioned problems of analyzing events. The separation of information search and identification of contributing factors prevents premature hypotheses leading to a restricted information and causal search. The cross-references between potential contributing factors support the identification of factors remote in time and space (latent
29
failures), prevent a focus on solely individual contributions and avoid mono-causal thinking by the analysts. The identification of contributing factors for each single event-building block prevents premature hypotheses and the identification of contributing factors because of past accidents. Finally, the questions and examples of the identification aid support the analysts in identifying ‘‘out of sight” factors (e.g. factors that contribute by their absence). In sum, SOL ensures a sufficient scope of investigation by introducing 21 categories of possible contributing factors with more than 160 specific and illustrative examples. SOL provides degrees of freedom for the analyst’s problem solving, but at the same time standardizes the process of analysis. In the following section, the evidence to evaluate SOL will be discussed.
3. Evaluation of SOL The evaluation of SOL was conducted in two separate steps: (1) an empirical evaluation by conducting experiments with student samples and (2) expert judgments of actual use in the nuclear power and chemical industries. Causes of events can be identified only in hindsight and thus nobody knows the ‘‘true” causes. Therefore, the empirical evaluation was done for ‘‘constructed” cases for which the causes were determined a priori by a standardized analysis. Thus, four previously-analyzed event reports from the literature (e.g. the capsizing of the Herald of Free Enterprise, Clapham Junction railway accident) were transferred into a SOL analysis. The results served as standard solutions for the empirical evaluation. A set of empirical studies was conducted (Lauer, 1997; Ritz, 1998; Fahlbruch, 2000; Domeinski, 2004) in which student subjects analyzed events with SOL on the basis of short event descriptions and standardized information given in response to their questions (simulating an investigation). Altogether 146 subjects conducted 92 event analyses with SOL, either working individually or in small groups of three subjects. All received a short description of the accident and then could ask for more information to complete the situational description and the identification of contributing factors. Overall, the results showed that the SOL methodology may help to overcome the above mentioned biases (for a full description of results see Fahlbruch, 2000). Specifically, the subjects exhibited broad causal search, multi-causal thinking, and consideration of factors beyond individual errors. Breadth of causal search: SOL was designed to overcome restricted causal search due to premature hypotheses. As an indicator of broad causal search, we compared the number of correctly stated hypotheses after the description of the event (premature hypotheses) with the number of contributing factors correctly identified at the end of the analysis. The overall results showed that in an analysis with SOL significantly more correct factors were identified than premature hypotheses stated. This is supported by the study of Domeinski (2004) who compared the results of analyses conducted with SOL with results of analyses without an explicit method and showed that with SOL significantly more factors were identified. Multi-causal thinking: overall 98% of the subjects identified more than one contributing factor and more than 75% of the subjects identified five or more factors. Considering causes beyond individual human actions: we compared the number of identified factors related to individual human behavior (e.g. communication, personal performance, violation and qualification) to the number of other causes (e.g. working conditions, technical components). Among correctly identified contributing factors, work-system factors clearly
30
B. Fahlbruch, M. Schöbel / Safety Science 49 (2011) 27–31
outweigh single human action factors. Domeinski (2004) also found that with SOL about twice as many organizational factors were identified than without SOL. SOL was also evaluated for its applicability by international scientists and practitioners in the field. Practitioners from nuclear power plant judged that SOL leads to at least as good or even better results than conventional methods which were used in the German industry. SOL was judged as an analysis methodology which supports practitioners in NPPs and enhances systemic thinking and a questioning attitude. Meanwhile, the computerized version SOL–VE has been adopted by the Swiss and German nuclear power industries as the standard procedure for their in-depth event analyses. The implementation and iterative improvement of SOL was crucially supported by Bernhard Wilpert’s outstanding network building activities. He established several workshops with practitioners from nuclear power plants and organized scientific conferences on event analysis and organizational learning (e.g. Hale et al., 1997). Due to his membership on the German Reactor Safety Commission (RSK) he encouraged plant representatives as well as governmental officials to tackle ‘‘human” safety problems from a socio-technical system perspective.
4. Open questions and future challenges Considerable practical experience has accumulated since SOL was applied in the nuclear power industries. In the following section, two issues are discussed which consistently arise from the experiences of SOL users. Due to the broad scope of investigation of SOL, several factors are identified which do not necessarily contribute to an event, but describe more general organizational weaknesses of system performance. As a consequence, SOL users identify on average combinations of 20–30 contributing factors per event. In order to communicate the results of an event analysis, users asked for a weighting procedure that would assign relative weights to each of the contributing factors and their interactions. This would allow for an aggregation of contributing factors and facilitate communication of ‘‘lessons learned” from an event. Therefore, a weighting procedure for event-building blocks and corresponding contributing factors was developed (MTO, 2009). It is a decision-support tool based on the AHP approach by Saaty (1990) which generates a ‘‘contributing” factor ranking. However, this importance ranking bears the risk that contributing factors are dropped from the analysis and another ‘‘story” of the event is told (e.g. depending on the target group of the analysis). Due to cross-references, questions and examples in SOL, users are guided to conduct an extensive information search, i.e. a lot of documents are analyzed and interviews conducted. Thereby, hypotheses about potential organizational weaknesses are developed which could not be adequately covered by the method. This mainly refers to cultural factors, e.g. complacency or lack of a questioning attitude, risky decision-making or bad leadership skills. When SOL was developed, it was decided to exclude cultural aspects or ‘‘inside-factors” like attitudes or values because they are already translated into actions. Hence, the underlying socio-technical model does not explicitly integrate the concept of safety culture. However, analyzing events with SOL always provides cultural insights into an organization. Event analysts infer patterns of behavior from interviews and document analyses; this may lead to conclusions about implicit norms or shared values of members of a given organization. SOL does not offer support for structuring these observations. On the one hand, this is an adequate solution in that events are rare and therefore should not be used for culture
assessment. On the other hand, when organizations want to introduce change after an event, they need to develop safety models that go beyond behavior and structures; especially in cases where culture negatively affects the output of an event analysis itself (e.g. in ‘‘blame cultures” when interview partners highlight specific information about other’s error or important information is hidden). It remains a challenge for future research to balance both concerns.
5. Conclusion SOL facilitates organizational learning from events by supporting the process of analyzing events, ensuring its standardized conduct and mobilizing expert knowledge and creativity of the analysis (Becker et al., 1995; Fahlbruch et al., 1998; Miller et al., 1999; Wilpert et al., 1997). The method helps to avoid generally known biasing tendencies when events are analyzed. Based in sound socio-technical systems theory, the method guarantees a comprehensive analytic scope with contributing factors from a wide range of possible sub-systems of the focal organization. Ideally, analyzing events with SOL fosters an attitude of critical reflection on system performance in the whole organization. Due to Bernhard Wilpert’s vision and drive, SOL is a product of interdisciplinary work and has been adopted by the Swiss and German nuclear power industries as the standard procedure for their in-depth event analyses. The main challenges for future refinement of SOL refer to the aggregation of ‘‘lessons learned” from an event and the handling of cultural factors that emerge in the course of an event analysis.
References Argyris, Ch., Schoen, D., 1978. Organisational Learning: A Theory of Action Perspective. Addison-Wesley, Reading, MA. Becker, G., Hoffmann, S., Wilpert, B., Miller, R., Fahlbruch, B., Fank, M., Freitag, M., Giesa, H., Schleifer, L., 1995. Analyse der Ursachen von ‘‘menschlichen Fehlverhalten” beim Betrieb von Kernkraftwerken (BMU-1996-454). Bundesministerium für Umwelt, Naturschutz und Reaktorsicherheit, Bonn. Benner, L. 1981. Accident Perceptions: Their Implications for Accident Investigators. Isasi Forum, Spring 1981, pp. 13–17. Benner, L. 1981. Methodological biases which undermine accident investigations. In: Proceedings of International Society of Air Safety Investigators International Symposium. pp. 1–5. Bishop, J., LaRhette, R., 1988. Managing human performance – INPO’s human performance evaluation system. In: Human-Error-Avoidance-Techniques Conferences Proceedings. Warrendale, Pennsylvania, Society of Automotive Engineers, Inc. (SAE), Publication No. P-204. pp. 79–85. Bullock, M.G., 1981. Change Control and Analysis (SSDC-21), vol. 77. US Department of Energy, Washington DC. Domeinski, J., 2004. Organisationales Lernen in komplexen Systemen. Diplomarbeit an der Technischen Universität Berlin, Berlin. Einhorn, H.J., Hogarth, R.M., 1986. Judging probable cause. Psychol. Bull. 99 (1), 3– 19. Fahlbruch, B., 2000. Vom Unfall zu den Ursachen: Eine empirische Bewertung von Analyseverfahren. Dissertation an der Technischen Universität. Mensch & Buch Verlag, Berlin. Fahlbruch, B., 2001. Event analysis as a tool to enhance safety culture in nuclear power plants. In: Wilpert, B., Itoigawa, N. (Eds.), Safety Culture in Nuclear Power Op-erations. Taylor & Francis, London, pp. 241–251. Fahlbruch, B., Wilpert, B., 1997. Event analysis as problem solving process. In: Hale, A., Wilpert, B., Freitag, M. (Eds.), After the Event from Accident to Organisational Learning. Pergamon, Oxford, pp. 113–130. Fahlbruch, B., Wilpert, B., 1999. System safety – an emerging field for I/O psychologie. In: Cooper, C.L., Robertson, I.T. (Eds.), International Review of Industrial and Organizational Psychology, vol. 14. Wiley, Chichester, pp. 55–93. Fahlbruch, B., Miller, R., Wilpert, B. (1998). Das Lernen aus Ereignissen und BeinaheEreignissen. SOL – Sicherheit durch Organisationales Lernen. atw, vol. 43(11). pp. 699–703. Fischhoff, B., Slovic, P., Lichtenstein, S., 1978. Fault trees: sensitivity of estimated failure probabilities to problem representation. J. Exp. Psychol. Human Percept. Perform. 4 (2), 330–344. Hale, A., Wilpert, B., Freitag, M. (Eds.), 1997. After the Event from Accident to Organisational Learning. Pergamon, Oxford. Hendrick, K., Benner, L., 1987. Investigating Accidents with STEP. Dekker, New York.
B. Fahlbruch, M. Schöbel / Safety Science 49 (2011) 27–31 IAEA. 1991. ASSET Guidelines Revised 1991 Edition. Reference Material Prepared by the IAEA for Assessment of Safety Significant Events Teams (IAEA-TECDOC632). International Atomic Energy Agency, Vienna. IAEA, 1994. The Asset Services (J-8 TC-871.2). International Atomic Energy Agency, Vienna. Johnson, W., 1973. The management oversight and risk tree – MORT. US Atomic Energy Commission, Germantown, MD. Johnson, W., 1980. MORT Safety Assurance Systems (Occupational Safety and Health), Vol. 4. Marcel Dekker, New York. Lauer, J., 1997. Methodische Zugänge bei Ursacheanalysen. Diplomarbeit an der Technischen Universität Berlin, Berlin. Maimer, H., Baggen, R., Szameitat, S. 1999. Computerized Event Analysis and the SOL Concept. Paper to be Presented at ESREL’99. European Safety and Reliability Conference, Munich, September 13–17, 1999. Manuele, F.A., 1982. Accident investigation and analysis: an evaluative review. Prof. Saf. 82, 53–57. Miller, R., Wilpert, B., & Fahlbruch, B. 1999. Sicherheit durch Organisationales Lernen (SOL): Konzeption und Überprüfung eines Verfahrens zur Ereignisanalyse. In: Engel, K., Kociok, B., (Hrsg.), Viertes Expertengespräch ‘‘Mensch-MaschineWechselwirkung in Kernkraftwerken” (BfS-KT-22/99) (S. 96-105). Salzgitter: Bundes-amt für Strahlenschutz. MTO, 2009. A Weighting Procedure for SOL. Internal report. Reason, J., 1990. Human Error. Cambridge University Press, Cambridge. Ritz, F., 1998. Der Einfluß von Training auf die Analyse von Unfällen. Diplomarbeit an der Technischen Universität Berlin, Berlin.
31
Ross, L., 1977. The intuitive psychologist and his shortcomings: distortions in the attribution process. Adv. Exp. Soc. Psychol. 10, 174–220. Saaty, Th.L., 1990. Multicriteria Decision Making: The Analytic Hierarchy Process. RWS Publishing, Pittsburgh. Shaklee, H., Fischhoff, B., 1982. Strategies of information search in causal analysis. Mem. Cogn. 10 (6), 520–530. Shealy, J.E., 1979. Impact of theory of accident causation on intervention strategies. In: Proceedings of the Human Factors Society, 23rd Annual Meeting. pp. 225229. Trist, E.L., Bamforth, K.W., 1951. Some social and psychological consequences of the longwall method of coalgetting. Hum. Relat. 4, 3–38. Tversky, A., Kahnemann, D., 1981. The framing of decisions and the psychology of choice. Science 211, 453–458. Weaver, D.A., 1973. TOR analysis: a diagnostic tool. ASSE J. June, 24–29. Wilpert, B., Fahlbruch, B., 1998. Safety related interventions in inter-organisational fields. In: Hale, A., Baram, M. (Eds.), Safety Management: The Challenge of Change. Pergamon, Oxford, pp. 235–248. Wilpert, B., Maimer, H., Miller, R., Fahlbruch, B., Baggen, R., Gans, A., Leiber, I., Szameitat, S., Becker, G. 1997. Umsetzung und Erprobung von Vorschlägen zur Einbeziehung von Human Factors (HF) bei der Meldung und Ursachenanalyse in Kernkraftwerken (BMU-1998-505). Bundesministerium für Umwelt, Naturschutz und Reaktorsicherheit, Bonn. Wilpert, B., Miller, R., Geymüller, C., Uhlemann, F., Ninov, E., 1998. Analyse von sicherheitsrelevanten Ereignissen in verfahrenstechnischen Anlagen (UBA Texte 79/98). Umweltbundesamt, Berlin.
Contents lists available at ScienceDirect
Safety Science journal homepage: www.elsevier.com/locate/ssci
SOL – Safety through organizational learning: A method for event analysis Babette Fahlbruch a,*, Markus Schöbel b a b
TuV NORD SysTec GmbH & Co. KG, Zimmerstrasse 23, 10969 Berlin, Germany Berlin Institute of Technology, 10587 Berlin, Germany
a r t i c l e
i n f o
Article history: Received 21 January 2010 Received in revised form 30 April 2010 Accepted 9 May 2010
Keywords: Event analysis Organizational learning Organizational factors
a b s t r a c t Under the guidance of Bernhard Wilpert a research group at the Berlin University of Technology developed an event analysis methodology called safety through organizational learning (SOL). The method has been adopted by the Swiss and German nuclear industries as standard procedure for their in-depth event analyses. SOL aims at facilitating organizational learning from events by supporting the process of analyzing events, ensuring its standardized conduct and mobilizing expert knowledge and creativity in the analysis. In this paper we provide a short description of SOL and its theoretical background. We summarize the empirical evidence and practical experience regarding SOL, which proves it to be a valid methodology that gives sufficient support to analysts. Finally, practical experiences and challenges for future research are discussed. Ó 2010 Elsevier Ltd. All rights reserved.
1. Introduction Nearly 20 years ago the influence of human factors on incidents in German nuclear power plants became a topic of increasing interest and significance. At this time practitioners from the nuclear power industry understood human factors mainly as human errors at the man–machine interface, whereas safety researchers had a quite different understanding that subsumed organizational and environmental aspects under the term human factors (e.g. Reason, 1990). In 1992 a research group at Berlin University of Technology started under the guidance of Bernhard Wilpert with the development of a new event analysis methodology called safety through organizational learning (SOL). The authors of the present article worked in this research group. Bernhard Wilpert’s central idea was to propagate a holistic socio-technical system approach for analysing events which implies that human factors do not only relate to the immediate man–machine interface, but comprise human actions on all system levels in as much as they contribute to the critical outcome of safety. According to the organizational learning literature (e.g. Argyris and Schoen, 1978), one important goal of event analysis is to draw lessons from an event to prevent future events. Therefore, adequate recommendations and safety measures must be implemented for all contributing factors identified. When the research group started the development of SOL, event reports in nuclear industries discussed technical failures and human errors as the main contributors to events. Organizational or inter-organizational
* Corresponding author. Tel.: +49 30 20177454; fax: +49 30 20177458. E-mail address: [email protected] (B. Fahlbruch). 0925-7535/$ - see front matter Ó 2010 Elsevier Ltd. All rights reserved. doi:10.1016/j.ssci.2010.05.004
factors were only considered on an informal basis and did not get due weight in the lessons learned. The group speculated about potential reasons and came to the conclusion that event analysts might not have adequate knowledge about organizational factors, and therefore their methods did not explicitly cover these factors. Moreover, the strong focus on apparent factors might result from shortcomings in causal reasoning. Consequently, the development of SOL should fulfill the following requirements: it should cover a broad range of human, organizational, and inter-organizational factors derived from theory and empirical data; it should be easy to use without expert knowledge in human factors psychology, because it should be applied by operators and supervisors in nuclear power plants; it should help to overcome well known shortcomings in human causal reasoning which could lead to truncated search in event analysis; it should support organizational learning from events; it should have empirical validation. In the following sections we will describe the development of the method, show how SOL was evaluated and close with open questions and challenges for the future.
2. The development of SOL – safety through organizational learning SOL was initially developed for the nuclear power industry (Becker et al., 1995; Wilpert et al., 1997). However, a version for
28
B. Fahlbruch, M. Schöbel / Safety Science 49 (2011) 27–31
the chemical industry exists (Wilpert et al., 1998) and a computer supported version was developed as well (Maimer et al., 1999). At the time SOL was developed, event analyses were still dominated by approaches characterized by attributions of fault to individual or/and technical actors (Benner, 1981a,b; Hendrick and Benner, 1987; Manuele, 1982; Shealy, 1979). Nevertheless, in high hazard industries a trend of systematic analyses supported by different methodologies could be observed (Fahlbruch and Wilpert, 1999). Unfortunately some of those methodologies were not completely published (e.g. the German human-factor concept of Vereinigung der Großkraftwerksbetreiber – VGB and the Human Performance Enhancement System of INPO – Institute of Nuclear Power Operations). For the development of SOL we reviewed the following methodologies: ASSET – Assessment of Safety Significant Events Teams (IAEA, 1991, 1994); Change Analysis (Bullock, 1981); HPES – Human Performance Enhancement System (INPO Bishop and LaRhette, 1988); MORT – Management Oversight and Risk Tree (Johnson, 1973, 1980); STEP – Sequentially Timed Event Plotting (Hendrick and Benner, 1987); TOR – Technique of Operations and Review (Weaver, 1973). The above methodologies are based on different accident causation models, e.g. Change Analysis and HPES had no explicit model, whereas MORT and STEP were based on explicitly formulated models. The methodologies vary in degree of standardization from general requirements for the process (Change Analysis) up to a set of attributes evaluated as adequate or less than adequate (MORT). Although ASSET, MORT and TOR explicitly consider organizational factors, inter-organizational factors are not included in any of the reviewed methods. None of the methods had explicit features for overcoming judgmental biases or shortcomings in causal reasoning. 2.1. Theoretical background of SOL The development of SOL was based on the assumption that an event analysis is a socially accepted reconstruction of a surprise, i.e. the identification of what happened and why. Therefore, it was important to provide not only a method, but also a scientific event causation model which had to be accepted by practitioners in the field. SOL builds upon Reason’s Swiss-Cheese-Model (1990) and the socio-technical system approach (e.g. Trist and Bamforth, 1951). It was Bernhard Wilpert’s theoretical contribution to combine both models in a socio-technical event causation model. According to this model, contributing factors stem from five sub-systems and their interaction: individual, team, organization, extra-organizational environment and technology. These sub-systems were further divided into categories (or contributing factors) which were derived from organizational theory as well as from an extensive analysis of published event reports in the nuclear power industry (Becker et al., 1995). In order to promote interdisciplinary work and to develop a method which would be accepted by practitioners and scientists, Bernhard Wilpert put together a team of experts consisting of psychologists and nuclear engineers who evaluated these factors and assigned each to a subsystem or its interaction. Thus an initial set of 19 contributing factors resulted, but practical feedback from the field and expert assessments led to a modified version with 21 potential contributing factors. Furthermore, the factors were differentiated according to their potential direct and indirect (or latent) influence on event causation (see Table 1).
Table 1 SOL categories of contributing factors. Direct factors
Indirect factors
Information Communication Working condition
Operation scheduling Group influence Rules, procedures & documents Organization & management
Responsibility Control & supervision Qualification
Experience feedback Maintenance
Safety principles Quality management
Regulatory bodies
Environmental influence
Personal performance Rule violation Technical component
Training
Technology
The quality of an event analysis method is not only influenced by its underlying theoretical model, but also depends on the conceptualization of the analysis process and the supporting features (Fahlbruch et al., 1998). Fahlbruch (2000, 2001) modeled the psychological process of analyzing events by drawing on psychological research on attributional processes and causal inferences. She identified various psychological factors which follow from characteristics of human information processing and general attribution processes and which may diminish the quality of an event analysis such as: premature hypotheses leading to truncated search for information, i.e. the first plausible ‘‘cause” is taken to explain what happened (Einhorn and Hogarth, 1986); difficulties to identify contributing factors which are remote in time and space from the actual event (Einhorn and Hogarth, 1986) which result in an over-weighting of so-called active errors; mono-causal thinking and satisfaction with only one contributing factor although there may have been multiple factors involved (Shaklee and Fischhoff, 1982); identification of contributing factors because of recent past events (mental availability of potentially contributing factors) (Tversky and Kahnemann, 1981); ignoring contributing factors which are not written down in the method (e.g. in a given checklist or fault-tree) (‘‘out of sight – out of mind”) (Fischhoff et al., 1978); concentration on contributions by human actors directly involved in the event (fundamental attribution error) (Ross, 1977). SOL was intended to be a standardized process of event analysis that would minimize the above factors and cognitive biases. 2.2. Description of SOL Analyzing events with SOL is conceptualized as a backward oriented problem-solving process (Fahlbruch, 2000; Fahlbruch and Wilpert, 1997). SOL operationalizes the concept of event analysis in a set of two standardized process steps: (1) the description of the actual event situation, and (2) the identification of contributing factors. For both steps guidelines were developed which support the event analyst. As the first step of the analysis, a situational description is constructed. The information needed for the description of the event is gathered by interviews and document analysis. A set of questions helps the analyst to ask the right questions in order to completely reconstruct the course of an event. Based on the STEP method (Hendrick and Benner, 1987) the collected information is broken down into a sequence of so-called event-building blocks, i.e. the
B. Fahlbruch, M. Schöbel / Safety Science 49 (2011) 27–31
event is decomposed into a sequence of single micro-events to clarify and illustrate what happened (Wilpert and Fahlbruch, 1998; Fahlbruch and Wilpert, 1999). For each event-building block the information is categorized according to the actor (human and technical actors), the action, the point in time of the action, the location (where the action takes place) and additional remarks. Thus, an event is determined by a sequence of singular actions by different actors. The starting point of an event (i.e. the first event-building block) is defined as the first deviation from a warranted course of action. These deviations are identified by contrasting actions against formal procedures and technical system design or against ‘‘normal” system performance based on the appraisal of an event analyst. The end point (i.e. the last event-building block) is defined as the recovery of a safe system state. The situational description illustrates only observable facts (what happened). Actions which were not shown as well as hypotheses about potential causes should not be incorporated into the situational description. Each event-building block is graphically ordered in a time-actordiagram which provides an overview of the recomposed event and serves as an important information source for the subsequent identification of contributing factors. The identification of contributing factors, i.e. the second step, is conducted in the following way: for each event-building block a separate analysis is conducted. A guideline with categories of possible contributing factors – the identification aid – supports this second step. The identification aid consists of categories of potential contributing factors which cover individual, technical, group, organizational and inter-organizational aspects to guarantee a sufficient scope of investigation. In order to support the identification of contributing factors each factor is assigned to a general question. For instance, the factor ‘‘working conditions” is transferred into the question ‘‘Could there have been an influence of the working conditions on the operator’s performance?”. For each factor several specific examples are given to support the analysts, e.g. for ‘‘working conditions” the examples are time pressure, noise, heat, lights or disturbances. Thus, the aid contains general questions related to possible contributing factors covering each of the five sub-systems in order to ensure the comprehensiveness of the analysis. Since it is assumed that an event analyst may not exclusively be a human factors specialist, the aid also gives illustrative examples of potential influences of contributing factors with the aim to stimulate creative problem solving processes. These examples are concrete enough to cover a broad range of potentially contributing factors but they are not meant to be exhaustive. To guarantee the comprehensiveness of the analysis all general questions are linked to others. These so-called cross-references are theoretically and empirically based. If one question is answered in the affirmative, the team is guided to answer another set of questions in order to identify other potentially contributing factors. Contributing factors are roughly divided into direct and indirect factors. The analysis process starts with the identification of direct factors which are linked to a couple of indirect factors due to the cross-references. For instance, if the direct factor ‘‘personal performance” is identified, a cross-reference to the indirect factor ‘‘training” is given. By these cross-references mono-causal thinking and over-weighting of active errors should be overcome. Finally, all identified contributing factors are added to the time-actor-matrix, thus successively completing the reconstruction of the event and its causes. Altogether SOL is assumed to overcome the above mentioned problems of analyzing events. The separation of information search and identification of contributing factors prevents premature hypotheses leading to a restricted information and causal search. The cross-references between potential contributing factors support the identification of factors remote in time and space (latent
29
failures), prevent a focus on solely individual contributions and avoid mono-causal thinking by the analysts. The identification of contributing factors for each single event-building block prevents premature hypotheses and the identification of contributing factors because of past accidents. Finally, the questions and examples of the identification aid support the analysts in identifying ‘‘out of sight” factors (e.g. factors that contribute by their absence). In sum, SOL ensures a sufficient scope of investigation by introducing 21 categories of possible contributing factors with more than 160 specific and illustrative examples. SOL provides degrees of freedom for the analyst’s problem solving, but at the same time standardizes the process of analysis. In the following section, the evidence to evaluate SOL will be discussed.
3. Evaluation of SOL The evaluation of SOL was conducted in two separate steps: (1) an empirical evaluation by conducting experiments with student samples and (2) expert judgments of actual use in the nuclear power and chemical industries. Causes of events can be identified only in hindsight and thus nobody knows the ‘‘true” causes. Therefore, the empirical evaluation was done for ‘‘constructed” cases for which the causes were determined a priori by a standardized analysis. Thus, four previously-analyzed event reports from the literature (e.g. the capsizing of the Herald of Free Enterprise, Clapham Junction railway accident) were transferred into a SOL analysis. The results served as standard solutions for the empirical evaluation. A set of empirical studies was conducted (Lauer, 1997; Ritz, 1998; Fahlbruch, 2000; Domeinski, 2004) in which student subjects analyzed events with SOL on the basis of short event descriptions and standardized information given in response to their questions (simulating an investigation). Altogether 146 subjects conducted 92 event analyses with SOL, either working individually or in small groups of three subjects. All received a short description of the accident and then could ask for more information to complete the situational description and the identification of contributing factors. Overall, the results showed that the SOL methodology may help to overcome the above mentioned biases (for a full description of results see Fahlbruch, 2000). Specifically, the subjects exhibited broad causal search, multi-causal thinking, and consideration of factors beyond individual errors. Breadth of causal search: SOL was designed to overcome restricted causal search due to premature hypotheses. As an indicator of broad causal search, we compared the number of correctly stated hypotheses after the description of the event (premature hypotheses) with the number of contributing factors correctly identified at the end of the analysis. The overall results showed that in an analysis with SOL significantly more correct factors were identified than premature hypotheses stated. This is supported by the study of Domeinski (2004) who compared the results of analyses conducted with SOL with results of analyses without an explicit method and showed that with SOL significantly more factors were identified. Multi-causal thinking: overall 98% of the subjects identified more than one contributing factor and more than 75% of the subjects identified five or more factors. Considering causes beyond individual human actions: we compared the number of identified factors related to individual human behavior (e.g. communication, personal performance, violation and qualification) to the number of other causes (e.g. working conditions, technical components). Among correctly identified contributing factors, work-system factors clearly
30
B. Fahlbruch, M. Schöbel / Safety Science 49 (2011) 27–31
outweigh single human action factors. Domeinski (2004) also found that with SOL about twice as many organizational factors were identified than without SOL. SOL was also evaluated for its applicability by international scientists and practitioners in the field. Practitioners from nuclear power plant judged that SOL leads to at least as good or even better results than conventional methods which were used in the German industry. SOL was judged as an analysis methodology which supports practitioners in NPPs and enhances systemic thinking and a questioning attitude. Meanwhile, the computerized version SOL–VE has been adopted by the Swiss and German nuclear power industries as the standard procedure for their in-depth event analyses. The implementation and iterative improvement of SOL was crucially supported by Bernhard Wilpert’s outstanding network building activities. He established several workshops with practitioners from nuclear power plants and organized scientific conferences on event analysis and organizational learning (e.g. Hale et al., 1997). Due to his membership on the German Reactor Safety Commission (RSK) he encouraged plant representatives as well as governmental officials to tackle ‘‘human” safety problems from a socio-technical system perspective.
4. Open questions and future challenges Considerable practical experience has accumulated since SOL was applied in the nuclear power industries. In the following section, two issues are discussed which consistently arise from the experiences of SOL users. Due to the broad scope of investigation of SOL, several factors are identified which do not necessarily contribute to an event, but describe more general organizational weaknesses of system performance. As a consequence, SOL users identify on average combinations of 20–30 contributing factors per event. In order to communicate the results of an event analysis, users asked for a weighting procedure that would assign relative weights to each of the contributing factors and their interactions. This would allow for an aggregation of contributing factors and facilitate communication of ‘‘lessons learned” from an event. Therefore, a weighting procedure for event-building blocks and corresponding contributing factors was developed (MTO, 2009). It is a decision-support tool based on the AHP approach by Saaty (1990) which generates a ‘‘contributing” factor ranking. However, this importance ranking bears the risk that contributing factors are dropped from the analysis and another ‘‘story” of the event is told (e.g. depending on the target group of the analysis). Due to cross-references, questions and examples in SOL, users are guided to conduct an extensive information search, i.e. a lot of documents are analyzed and interviews conducted. Thereby, hypotheses about potential organizational weaknesses are developed which could not be adequately covered by the method. This mainly refers to cultural factors, e.g. complacency or lack of a questioning attitude, risky decision-making or bad leadership skills. When SOL was developed, it was decided to exclude cultural aspects or ‘‘inside-factors” like attitudes or values because they are already translated into actions. Hence, the underlying socio-technical model does not explicitly integrate the concept of safety culture. However, analyzing events with SOL always provides cultural insights into an organization. Event analysts infer patterns of behavior from interviews and document analyses; this may lead to conclusions about implicit norms or shared values of members of a given organization. SOL does not offer support for structuring these observations. On the one hand, this is an adequate solution in that events are rare and therefore should not be used for culture
assessment. On the other hand, when organizations want to introduce change after an event, they need to develop safety models that go beyond behavior and structures; especially in cases where culture negatively affects the output of an event analysis itself (e.g. in ‘‘blame cultures” when interview partners highlight specific information about other’s error or important information is hidden). It remains a challenge for future research to balance both concerns.
5. Conclusion SOL facilitates organizational learning from events by supporting the process of analyzing events, ensuring its standardized conduct and mobilizing expert knowledge and creativity of the analysis (Becker et al., 1995; Fahlbruch et al., 1998; Miller et al., 1999; Wilpert et al., 1997). The method helps to avoid generally known biasing tendencies when events are analyzed. Based in sound socio-technical systems theory, the method guarantees a comprehensive analytic scope with contributing factors from a wide range of possible sub-systems of the focal organization. Ideally, analyzing events with SOL fosters an attitude of critical reflection on system performance in the whole organization. Due to Bernhard Wilpert’s vision and drive, SOL is a product of interdisciplinary work and has been adopted by the Swiss and German nuclear power industries as the standard procedure for their in-depth event analyses. The main challenges for future refinement of SOL refer to the aggregation of ‘‘lessons learned” from an event and the handling of cultural factors that emerge in the course of an event analysis.
References Argyris, Ch., Schoen, D., 1978. Organisational Learning: A Theory of Action Perspective. Addison-Wesley, Reading, MA. Becker, G., Hoffmann, S., Wilpert, B., Miller, R., Fahlbruch, B., Fank, M., Freitag, M., Giesa, H., Schleifer, L., 1995. Analyse der Ursachen von ‘‘menschlichen Fehlverhalten” beim Betrieb von Kernkraftwerken (BMU-1996-454). Bundesministerium für Umwelt, Naturschutz und Reaktorsicherheit, Bonn. Benner, L. 1981. Accident Perceptions: Their Implications for Accident Investigators. Isasi Forum, Spring 1981, pp. 13–17. Benner, L. 1981. Methodological biases which undermine accident investigations. In: Proceedings of International Society of Air Safety Investigators International Symposium. pp. 1–5. Bishop, J., LaRhette, R., 1988. Managing human performance – INPO’s human performance evaluation system. In: Human-Error-Avoidance-Techniques Conferences Proceedings. Warrendale, Pennsylvania, Society of Automotive Engineers, Inc. (SAE), Publication No. P-204. pp. 79–85. Bullock, M.G., 1981. Change Control and Analysis (SSDC-21), vol. 77. US Department of Energy, Washington DC. Domeinski, J., 2004. Organisationales Lernen in komplexen Systemen. Diplomarbeit an der Technischen Universität Berlin, Berlin. Einhorn, H.J., Hogarth, R.M., 1986. Judging probable cause. Psychol. Bull. 99 (1), 3– 19. Fahlbruch, B., 2000. Vom Unfall zu den Ursachen: Eine empirische Bewertung von Analyseverfahren. Dissertation an der Technischen Universität. Mensch & Buch Verlag, Berlin. Fahlbruch, B., 2001. Event analysis as a tool to enhance safety culture in nuclear power plants. In: Wilpert, B., Itoigawa, N. (Eds.), Safety Culture in Nuclear Power Op-erations. Taylor & Francis, London, pp. 241–251. Fahlbruch, B., Wilpert, B., 1997. Event analysis as problem solving process. In: Hale, A., Wilpert, B., Freitag, M. (Eds.), After the Event from Accident to Organisational Learning. Pergamon, Oxford, pp. 113–130. Fahlbruch, B., Wilpert, B., 1999. System safety – an emerging field for I/O psychologie. In: Cooper, C.L., Robertson, I.T. (Eds.), International Review of Industrial and Organizational Psychology, vol. 14. Wiley, Chichester, pp. 55–93. Fahlbruch, B., Miller, R., Wilpert, B. (1998). Das Lernen aus Ereignissen und BeinaheEreignissen. SOL – Sicherheit durch Organisationales Lernen. atw, vol. 43(11). pp. 699–703. Fischhoff, B., Slovic, P., Lichtenstein, S., 1978. Fault trees: sensitivity of estimated failure probabilities to problem representation. J. Exp. Psychol. Human Percept. Perform. 4 (2), 330–344. Hale, A., Wilpert, B., Freitag, M. (Eds.), 1997. After the Event from Accident to Organisational Learning. Pergamon, Oxford. Hendrick, K., Benner, L., 1987. Investigating Accidents with STEP. Dekker, New York.
B. Fahlbruch, M. Schöbel / Safety Science 49 (2011) 27–31 IAEA. 1991. ASSET Guidelines Revised 1991 Edition. Reference Material Prepared by the IAEA for Assessment of Safety Significant Events Teams (IAEA-TECDOC632). International Atomic Energy Agency, Vienna. IAEA, 1994. The Asset Services (J-8 TC-871.2). International Atomic Energy Agency, Vienna. Johnson, W., 1973. The management oversight and risk tree – MORT. US Atomic Energy Commission, Germantown, MD. Johnson, W., 1980. MORT Safety Assurance Systems (Occupational Safety and Health), Vol. 4. Marcel Dekker, New York. Lauer, J., 1997. Methodische Zugänge bei Ursacheanalysen. Diplomarbeit an der Technischen Universität Berlin, Berlin. Maimer, H., Baggen, R., Szameitat, S. 1999. Computerized Event Analysis and the SOL Concept. Paper to be Presented at ESREL’99. European Safety and Reliability Conference, Munich, September 13–17, 1999. Manuele, F.A., 1982. Accident investigation and analysis: an evaluative review. Prof. Saf. 82, 53–57. Miller, R., Wilpert, B., & Fahlbruch, B. 1999. Sicherheit durch Organisationales Lernen (SOL): Konzeption und Überprüfung eines Verfahrens zur Ereignisanalyse. In: Engel, K., Kociok, B., (Hrsg.), Viertes Expertengespräch ‘‘Mensch-MaschineWechselwirkung in Kernkraftwerken” (BfS-KT-22/99) (S. 96-105). Salzgitter: Bundes-amt für Strahlenschutz. MTO, 2009. A Weighting Procedure for SOL. Internal report. Reason, J., 1990. Human Error. Cambridge University Press, Cambridge. Ritz, F., 1998. Der Einfluß von Training auf die Analyse von Unfällen. Diplomarbeit an der Technischen Universität Berlin, Berlin.
31
Ross, L., 1977. The intuitive psychologist and his shortcomings: distortions in the attribution process. Adv. Exp. Soc. Psychol. 10, 174–220. Saaty, Th.L., 1990. Multicriteria Decision Making: The Analytic Hierarchy Process. RWS Publishing, Pittsburgh. Shaklee, H., Fischhoff, B., 1982. Strategies of information search in causal analysis. Mem. Cogn. 10 (6), 520–530. Shealy, J.E., 1979. Impact of theory of accident causation on intervention strategies. In: Proceedings of the Human Factors Society, 23rd Annual Meeting. pp. 225229. Trist, E.L., Bamforth, K.W., 1951. Some social and psychological consequences of the longwall method of coalgetting. Hum. Relat. 4, 3–38. Tversky, A., Kahnemann, D., 1981. The framing of decisions and the psychology of choice. Science 211, 453–458. Weaver, D.A., 1973. TOR analysis: a diagnostic tool. ASSE J. June, 24–29. Wilpert, B., Fahlbruch, B., 1998. Safety related interventions in inter-organisational fields. In: Hale, A., Baram, M. (Eds.), Safety Management: The Challenge of Change. Pergamon, Oxford, pp. 235–248. Wilpert, B., Maimer, H., Miller, R., Fahlbruch, B., Baggen, R., Gans, A., Leiber, I., Szameitat, S., Becker, G. 1997. Umsetzung und Erprobung von Vorschlägen zur Einbeziehung von Human Factors (HF) bei der Meldung und Ursachenanalyse in Kernkraftwerken (BMU-1998-505). Bundesministerium für Umwelt, Naturschutz und Reaktorsicherheit, Bonn. Wilpert, B., Miller, R., Geymüller, C., Uhlemann, F., Ninov, E., 1998. Analyse von sicherheitsrelevanten Ereignissen in verfahrenstechnischen Anlagen (UBA Texte 79/98). Umweltbundesamt, Berlin.