- Email: [email protected]
The application of the loglinear model to quantify human errors
Reliability Engineering and System Safety 37 (1992) 157-165
The application of the Ioglinear model to quantify human errors Ya-Lih Lin & Sheue-Ling Hwang Department of Industrial Engineering, National Tsing-Hua University, Hsin-Chu, Taiwan (Received 14 March 1991; accepted 11 October 1991)
This paper concentrates on using a quantitative method, the loglinear model, to investigate human error under a state of emergency. The ioglinear model provides a systematic approach to find out the factors which lead to occurrences of human errors. The external performance shaping factor (PSF), internal PSF, and stressor PSF were found to be the statistically significant factors of human errors. In addition, the first-order interaction between external PSF and stressor PSF, as well as the second-order interaction of these three factors, were also verified as important parts of the human error model. Furthermore, the maximum likelihood which predicts human error probabilities could be obtained from the selected loglinear model. In contrast with the qualitative approach, two major quantitative approaches can be taken to characterize human errors: probabilistic and causal approaches. The probabilistic approach is typically pursued by those who are interested in the human reliability analysis ( H R A ) . 8-12 The causal approach can be used to trace the causes and contributing factors, but does not consider the mechanism of human error. 3 By using the concept of performance shaping factors (PSFs), 1° this study starts with such a cognitive approach to the causes of human errors. As mentioned previously, the cognitive approach to human error provides an operational tool which allows us to pay more attention to the human causes. The cognitive processes reveal what and how information is perceived, and what one decides to do about it. Although various cognitive mechanisms and causes for human errors were considered, it seems very difficult to quantify all the influences of the cognitive aspects of behavior and the interactions of PSFs on the behavioral processes. 1° Therefore, in order to measure the 'effects' of factors and causes contributing to human errors more accurately and objectively, we make an attempt to find out the quantitative function of the cognitive mechanism by employing the loglinear model. Loglinear models have been extensively used in the analysis of contingency tables, also applied in psychology, social sciences, psychiatry, etc. The major advantages obtained from use of these techniques are: (1) They provide a systematic approach to the analysis of multidimensional tables.
INTRODUCTION In emergency situations, human performance is always degraded by delaying responses, ignoring information or processing information incorrectly provided, so that mental workload is far overloaded than in normal operations. It has been reported that 7 0 % - 9 0 % of system failures are directly or indirectly due to human error, ~-3 and resulted in great damage to the systems or to human beings. Therefore, it is important to understand the mediating process or cognitive behavior of humans and investigate the potential factors of human error so that an incident in the emergency situation can be prevented. The current literature on human error in systems typically emphasize either a qualitative or a quantitative approach. In essence, the qualitative approach provides a structural framework for the system designer to recognize and evaluate the errors in the system. A variety of mental, cognitive or conceptual models have been proposed for human error. 3-7 From the qualitative models, the system designer develops a logical concept of what should be considered to prevent operator's errors, but cannot obtain a detailed procedure that designates the error characteristics of the operator. Moreover, one common deficiency of those mental models is the lack of a clear guideline for transferring the concept to the instructions of design.
Reliability Engineering and System Safety 0951-8320/92/$05.00 © 1992 Elsevier Science Publishers Ltd, England. 157
158
Ya-Lih Lin, Sheue-Ling Hwang
(2) They provide estimates of the magnitudes of effects of interest. (3) They allow the relative importance of different effects to be judged. The objective of this study is to find out the significant factors causing human errors so that the system designer can focus on these significant factors to reduce human errors more efficiently and effectively. This study presents how a loglinear model can be used to analyze the cognitive mechanism of human errors and identify those PSFs relevant to HRA. By the analysis of categorical data from the aspect of human cognition, the loglinear model of human errors was fitted and the PSFs with significant contribution to the occurrence of human errors were found. Also, we obtained the predicted failure probabilities of the combinations of the categorized levels of these three factors (the external, internal and stressor PSFs).
BACKGROUND A number of studies have been attributed to the application of mental models or cognitive models in the study of human error. One of the articulate discussions of the relationships between mental models and human error in the operation of engineering systems has been provided by Rasmussen. 4 In ordinary, familiar circumstances, the human operator appears to rely upon available heuristics and rules of operation. In other words, the operator's behavior is rule-based. However, in unusual situations for which rules do not apply, the human operator must reason at a knowledge-based level, using an understanding of the functioning of the system to determine an appropriate course of action. The skill-based, rule-based, knowledge-based classification (which we shall henceforth abbreviate to 'SRK') is a result of Rasmussen's long-standing concern with how to reduce human error in the control of complex systems.~3 The SRK distinction has contributed to the development of a taxonomy of human malfunction and a framework for cognitive task analysis. However, classification of behavior does not spell out the relationship between levels or categories, nor does it provide an explanation for why one level might be chosen over another. The previous models of human error could be viewed as frameworks in the assessment of human error in emergency situations. Swain and Guttmann ~° propose the concept of PSFs which might influence the likelihood of a human error in a given situation. A long series of arguments lead the authors to derive a nominal value for each error rate, as well as upper and lower uncertainty bounds (UCB). They suggested that these UCBs were used as the 5th and 95th
percentiles of a lognormal distribution for the error rate. After the milestone contribution of Swain and Guttmann who proposed the technique for human error rate prediction (THERP), there has been increasing concern about the role of human beings in the normal operation of nuclear, chemical and other industrial plants. THERP recognizes a host of problems which affect human reliability under the name of 'performance shaping factors'. Embrey, et al. 14 proposed the success likelihood index methodology (SLIM); the basic rationale underlying SLIM is that the likelihood of an error in a particular situation depends on the combined effects of a relatively small set of PSFs such as environment, state of current practice or skill, and time constraints. They claim that 'experimenal evidence suggests that the success likelihood index (SLI) is related to the logarithm of the success probability for a task'. Vestrucci ~5 suggested the logistic model in which SLI is related to the logarithm of ratio of the failure probability to success probability. A critique has been proposed by Apostolakis,16 and the major observations of this review are as follows: SLIM provides a highly structured approach for the derivation of human error rates under given conditions. However, the treatment of the weights and ratings in this model is internally inconsistent. In addition, the assumption that the weights of the performance shaping factors are independent of the task ratings may not accurately reflect some real world situations. Ujita 17 considered that human error causes were a function of task type. A two-dimensional array was displayed to perform a descriptive statement between error causes and task type. However, it lacks of further inference for the human error. After reviewing the above methods, this research focused on building a quantitative model and making an inference of failure probabilities concerning all the causes. Then an experiment was conducted to verify the feasibility of the loglinear model. The loglinear model also provided a systematic approach for the interpretation of human error causes and the prediction of the human error rates. Moreover, in terms of the findings of the failure probabilities, the system designer or trainer will have had some directions to follow to improve system reliability.
METHOD A cognitive model In modeling human performance for probabilistic risk assessment (PRA), it is necessary to consider those factors that have the most effect on performance.
The application of the loglinear model to quantify human errors
affecting performance
Iluman errors
Mechanisms
Factors v
of human malfunction
v r . . . . . . . . .
|
I Internal
I I
I
human
I
mode of malfunction
:| malfunction,
External
Incorrect
PSFs
mediating
l.ternal
(cognitive)
human
activities
inputs
PSFs Stressor
(Incorrect
& processes
PSFs
159
to system) L . . . . .
(hq)uts)
. ....
(Outputs)
Fig. 1. A cognitive model of human malfunction involving human errors as outputs.
Many factors affect human performance in a complex man-machine system. Some of these performance shaping factors (PSFs) are external to the person and others are internal. The external PSFs include the entire work environment, especially the equipment design, the written procedures, and oral instructions. The internal PSFs represent the individual characteristics--skills, motivations, and the expectations that influence one's performance. Psychological and physiological stresses result from a work environment in which the task demands for operators in the system do not conform to their capabilities. The cognitive model proposed here was based on previous studies l°'~s but a modification was made. Figure 1 represents a general framework of human error in systems such as aircraft, ships, power plants, or process control systems. To perform an H R A , one must identify those most relevant and influential PSFs in the model. One may consider the external, internal, and stressor PSFs as inputs (independent variables) of the cognitive mechanism which result in human errors. If these three variables occur simultaneously, then they form a three-dimensional contingency table. The frequencies of human error are treated as dependent variables (or responses). They are categorized with respect to three variables and the underlying levels (Table 1).~9 T h e ioglinear m o d e l
In contingency table analysis, the loglinear model was employed to assess the main effects and interaction effects of independent variables on a dependent variable. Also the maximum likelihood analysis of variance was used as the primary statistical tool to partition overall variability for human error data
Table 1. Three independent variables for the human errors and the underlying levels
Variables
Levels
A. External PSF
I. Situational characteristics II. Task and equipment characteristics III. Job and task instructions IV. Others
B. Internal PSF
I. Characteristics of people resulting from external influences II. Characteristics of people resulting from internal or other influences
C. Stressor PSF
I. Distraction of secondary task II. Psychological stressors III. Physiological stressors and others
analysis. For the three-dimensional tables, let Xij k be the observation in ith row, jth column, and kth layer of the table, and let misk be the corresponding expected value for that entry under some model. TM The general loglinear model is written as follows: log mij k = U +
Ul(i) ~1_ U2(j ) .~_ U3(k ) q_
U12(ij ) .~_ Ul3(ik )
"Jr"U23(jk) "Jr"U123(ijk) ,
(1)
where u is the grand mean of the logarithms of expected counts, 1
1
s
U=I~i~----lj~=l k=l~'d log m#~,
(2)
Ul(i) is the main effect of the logarithms of the expected counts at level i of the first variable, 1 s K u + Ul(i) = f ~ ~ ~ log mijk, ix j = l k = l
(3)
160
Ya-Lih Lin, Sheue-Ling Hwang
and, similarly, log mijk,
(4)
U'JrU3(k) = ~a i =El j E=1 log mijk,
(5)
u + u2(j) = ~-K
i=l k=l
1
~
J
In eqn (1), u12(ij), u13(ik), and U23(jk) are the first-order interactions (or the two-factor effects) of the logarithms of the expected counts, and u123(ijk) is the second-order interaction (or the three-factor effect) of the logarithms of the expected counts. Estimation of the interaction effects would be useful in identifying those categories responsible for any departure from independence. As in the usual A N O V A model, u~(o, u2(j), and u3(k) represent deviations from the grand mean u. E UI(i) = E U2(j) = E U3(k) = O, i j k
(6)
In addition to (6), we have
E U12(ij)= E U12('1)-~-E U13(ik)= E Ul3(ik) i j i k : E U23(jk) = E j
U 2 3 ( / k ) = 0,
(7)
assigned to use one of the experimental procedures under an emergency situation. All subjects were either industrial engineering or materials science engineering majors and have completed courses in physics and calculus. It is important to note that although the use of students as subjects is often considered as compromising to the credibility of applied research, this subject population was well-suited to the questions at hand. This is because operators in many systems (e.g., nuclear power plants) were required to complete a training program that was technically equivalent to that required for a bachelor's degree in engineering. Therefore, it was argued that these students had educational backgrounds comparable to actual operator trainees in some domains.
Apparatus This study was conducted in a laboratory equipped with an electron-beam evaporation system and a personal computer. The electron-beam evaporation system consists of three parts: vacuum pumping, deposition, and cooling subsystems. A graphic display for the system is shown in Fig. 2.
k
and
Experimental tasks u,23(,jk) = i
u,23(0 ]
) =
u,23,j
> = o.
(8)
k
Once we have estimated expected values under one of the loglinear models, we can check the goodness-of-fit of the model using the likelihood-ratio chi-square statistic: / Observed \
02= 2 E (Observed)logl
(9)
where the summations are over all cells in the table. If the model fitting is adequate and the total sample size is large, G z has an approximate X2 distribution with degrees of freedom given by the following formula: d.f. = # cells - # parameters fitted.
(10)
Experiment The human error data were collected from an experiment of the electron-beam evaporation system operated under emergency situations. The primary objective of this experiment was to verify the feasibility of the loglinear model, and search for the contributing factors and the functional form of human error mechanism.
Subjects Thirteen graduate and 16 undergraduate students of National Tsing Hua University were paid to take part in this experiment as subjects. They were randomly
Both primary and secondary tasks were simultaneously assigned to a subject who had acquired a systematic training in the operational principles and procedures for dealing with the emergency incidents of the electron-beam evaporation system. The primary task was to supervise the vacuum pumping process or the deposition process of the electron-beam evaporation system. Meanwhile, the secondary task was either to play a guessing game, or to operate an arithmetic test on an IBM PC/XT. Many studies showed that fatigue due to long working hours or highly concentrated work resulted in less attention to certain types of signals. The secondary task was used to increase the work load, and consequently increase the internal and external stress of the subjects.
Experimental procedures (1) Subjects were taught about the routine operation principles and the procedures to manipulate the electron-beam evaporation system. (2) To perform a supervisory task on the vacuum pumping and play a guessing game. Accidents such as cooling water supply breakdown or power supply breakdown occurred randomly. If an accident occurred, the subject then pressed a corresponding key. (3) To perform a supervisory task on the deposition part and an arithmetic operation. In addition, the same accidents and corresponding
The application of the loglinear model to quann'fy human errors
161
8~
11'
i
12
13
1 ri] L _ . . .
Components of vacuum pumping subsystem:
Components of deposition subsystem:
(1)
Main power-1
(8)
Main power-2
(2)
Rotary pump
(9)
Thickness monitor power supply
(4)
Heater
(10)
P~tameter setting board
(5)
Foreline valve
(12)
Voltage
(6)
Roughing valve
(13)
Ampere
(7)
Main valve or high vacuum valve
(14)
Shutter
Components of cooling subsystem: (3)
Cooling water valve--1
(11)
Cooling water valve-2 Fig. 2. A graphic display for an electron-beam evaporation system.
key-operations as those designated in procedure (2) were also assigned to the subject. (4) At the end of the experiment, the total number of occurrences of human errors were counted and the subject was asked to take a questionnaire.
Data collection During the experiment, the subject was asked to report what and why he/she was doing whenever he/she took an action, and both actions and answers
were recorded by the experimenter. After the experiment, the subject took a questionnaire (Table 2) to repeat and/or add some explanation of his/her actions and the experimenter could revise the answer of the previous record if necessary. The PSFs in the questionnaire were adapted from NUREG/CR-1278 for the electron-beam evaporation system. In taking the questionnaire, the subjects were asked to choose the most likely factor(s) which caused a specific error he/she made in the experiment. Table 2 is a sample questionnaire to which a subject has responded. In this case, the subject recalled the situation when
162
Ya-Lih Lin, Sheue-Ling Hwang
Table 2. The questionnaire which listed the related PSFs resulting in human error for the electron-beam evaporation system
A. External PSFs I. Situational characteristics: -Organizational structure (e.g. Authority, responsibility, communication channels). -Actions by supervisors. -Rewards, benefits. II. Task and equipment characteristics: -Perceptual requirements. -Motor requirements (speed, precision). -Control-display relationships. -Decision-making. -Complexity (information load). -Task criticality. -Long- and short-term memory. -Feedback (knowledge of results). -Man-machine interface factors (design of prime equipment, test equipment, job aids). III. Job and task instructions: -Procedures required (written or not written). -Written or oral communication. -Cautions and warnings. -Work methods. IV. Others B. Internal PSFs I. Characteristics of people resulting from external influences: -Previous training/experience. -State of current practice or skill. -Knowledge of required performance standards. II. Characteristics of people resulting from internal or other influences: -Personality and intelligence variables. -Motivation and attitudes. -Emotional state. -Attitudes based on influence of family and other outside persons or agencies. -Others. C. Stressor PSFs I. Distraction from secondary task II. Psychological stressors: -Suddenness of onset. -Duration of stress. -Task speed. -Task load. -Threats (of failure, loss of job). -Monotonous, degrading, or meaningless work. -Long, uneventful vigilance periods. -Distractions (Noise, glare, movement). III. Physiological stressors and others: -Duration of stress. -Fatigue. -Pain or discomfort. -Hunger or thirst. -Movement constriction. -Others. he/she made an error and chose the 'motor requirements', 'state of current practice or skill', and 'task speed' as the main factors. As shown in Table 2, the questionnaire consists of three parts; 1) External PSFs, 2) Internal PSFs, and 3)
Table 3. The contingency table of human error data
Variables
Levels'~
A. External PSF:
I
B. Internal PSF: C. Stressor: PSF:
I II III
II
I
II
1 14 5
1 7 4
I
1II II
I
IV II
I
II
27 6 10 12 8 10 71 32 25 14 12 14 15 19 11 4 11 9
a The levels for each variable are described in Table 1. Stressor PSFs. Each part contains 2 - 4 levels and there are several items under each level which can be selected. For example, if a subject selected 'motor requirements' under Level II, 'Task and Equipment', as shown In Table 2, this would be counted as one occurrence of External PSFs Level II, in Table 3.
RESULT
The frequencies of factors which caused some specific human errors are shown in Table 3. Within the framework of loglinear models, it is important to look at all possible sources of variation and estimate the parameter effects. From the maximum likelihood analysis of variance tables of the full or saturated model, it leads to the consideration of a limited number of loglinear models as part of a preliminary step in the model-building process, z~ A cursory examination suggested that both u13 (the interaction effect of external PSF and stressor PSF) and u123 (the interaction effect of external, internal, and stressor PSFs) must be definitely included in our model since their chi-square values were 14.52 and 12.88 (p < 0-05), respectively. The next step is to examine the maximum likelihood estimates for all parameter effects (u-terms) in the full or saturated model. Techniques such as maximum likelihood and least squares may be used to estimate the parameters, and estimated parameter values may then be used in identifying which variables are of greatest importance in determining the values taken by the observations. 22 G o o d m a n 2~ showed that the standardized values have an asymptotically standard normal distribution, and might therefore be compared with the z-value for any particular probability level ( t x = 0 . 0 5 or 0.01). By the above techniques, we suggest that a model including both interaction terms u~3 and u123 might provide an adequate fit for these data. Consequently, such a model is: log mqk = U + U~ti) + U2(j) + U3(k) + Ul3(ik) q'- U123(ijk)
(11)
The maximum likelihood analysis of variance tables and the estimated parameter effects based on the
The application of the loglinear model to quantify human
Table 4. Maximum likeBhood analysis of variance for the model specified in eqn (11) fitted to the data in Table 3
Source
DF
Chi-square
p-value
Ul U2 U3 U13 u123 Likelihood ratio
3 1 2 6 6 5
61"87 14"35 44"84 13"66 15-25 5.96
0"0000~ 0"0002~ 0"0000" 0"0337b 0"0184 b
0-3100
° p<0.01 b p<0.05 selected model in eqn (11) are shown in Tables 4 and 5, respectively. The value of G 2 is not significant (p = 0.31) and implies an acceptable fit of the model. In addition, applying formula (9) to each model, a summary of the value of G 2 and probability level for each of the possible loglinear models to the data in Table 5. Maximum likelihood estimates of the parameters for the model specified in eqn (11) fitted to the data in Table 3 Parameter
ulo) u1<2) u1(3) u2o) U30)
U3(2) ula(m
U13(12) u13(21)
U13(22) U13(31) /213(32) U123(111) U123012) U123(211)
Ul23(212) U123(311) U123(312)
Estimate
-1.0153 0-839 2 0.1309 0.227 1 -0.4761 0.652 4 -0.807 2 0.403 9 0.009 06 O"120 7 0-464 7 -0.115 5 --0'006 15 0"042 9 0"347 7 0" 107 4 --0"316 4 0"011 8
Standard error
Chi-square
0.2092 O. 106 9 0.1234 0.059 9 0.1423 0.098 5 0'377 5 0-236 0 0.177 6 O"127 3 0-190 8 0-151 6 0"229 7 0" 162 0 0"138 7 0"096 2 0"167 9 0"132 9
23.56 61.63 1.12 14.35 11.19 43.84 4.57 2-93 0.00 0'90 5-93 0-58 0"00 0"07 6"29 1"25 3'55 0"01
a p<0.01
b p<0.05
p-value
0.000 0" 0.000 lY 0.2890 0.000 2a 0.0008 a 0-000 0~ 0.032 5 b 0.087 0 0.959 3 0"342 8 0.014 9 b 0.446 1 0"978 6 0"791 0 0"012 2 0'264 3 0"059 5 0"929 3
163
errors
Table 3 is obtained (Table 6). Table 6 indicates that models ( a ) - ( e ) do not provide an adequate fit for these data. In regard to model (g), it is plausible to be a good fit since the value of G 2 is nonsignificant, so it is a candidate for model selection. But the contribution of ux2-effect is not significant (chi-square v a l u e = 5 . 5 0 , p > 0 . 0 5 ) . For the principles of parsimony and simplicity, model (g) is not adoptable. Therefore, we might conclude that the model in eqn (11) or model (f) of Table 6 provides an adequate fit for the data. In other words, a model which includes a first-order interaction between an external PSF and a stressor PSF, and a second-order interaction among an external PSF, an internal PSF, and a stressor PSF was chosen which fits the data reasonably well. Furthermore, the maximum likelihood predicted value for response (i.e. human error) probabilities can be calculated (Table 7). These failure probabilities can be used to predict the occurrence of the human errors.
DISCUSSION AND CONCLUSION
There are some important results of this study. First, the loglinear model provides a systematic approach to obtaining an adequate human error model from an aspect of human cognition. Secondly, an appropriate model in which some PSFs are found to contribute significantly to the reduction of the number of human errors was selected. In addition, the first-order interaction (u13) between external PSF ( u 0 and stressor PSF (u3), as well as the second-order interaction of these three factors (u123), are verified as important parts of the human error model. Moreover, the results of Table 5 indicate that there is a positive association between the job and task instructions of external PSF and distraction of secondary task of stressor PSF [/~13(31) = 0.4647, p < 0.05], and a positive association among the task and equipment characteristics of external PSF, characteristics of people resulting from external influences and distraction of secondary task [/~123(211) ~---0.3477, p < 0.05]. Thirdly, the most important thing for a human error model is
Table 6. Likelihood ratio chi-square values for some Ioglinear models applied to the data in Table 3
Model (a) (b) (c) (d) (e) (f) (g)
log mqk = u + ul(o + u2t/) + u3(r) log mOk = U + UI(O + U2(j)+ U3(r) + U12(ij) log m,ik = U + Ut(o + U2(i) + U~(X) + Ut3(~k) log m q k = U + Ul(i) + U2(i) + U3(K) + Ul2(ii) "~- U13(ik) log m~ik = u + ulfo + U2(l) + u3(x) + ut2(e) + U130k) + U23(ik) log rail k = u + Ill(,) + U2(1) + U3(K) + Ula(ik) "~- U123(ijk) log m0k = u + ul(o + u2(/) + u3(r) + u12(ii)+ u13(ik)+ U123(qk),
"p
D.F.
G2
p-value
17 14 11 8 6
37"50 31"18 21"91 15"59 13"78 5.96 0"43
0.002 9a 0.005 2a 0.025 1b 0-048 7b 0.032 20 0-3100 0.807 1
5
2
Ya-Lih Lin, Sheue-Ling Hwang
164
Table 7. Maximum likelihood predicted values for response probabilities (or predicted human error probabilities) °
Observed
Predicted
Function number
Human error probability
Standard error
Human error probability
Standard error
Residual
P123(111) P123o12) Pl23(113) P123(121) Pl23(122) Pt23(123) P123(211)
0.002 924 0-040 936 0.014 620 0"002 924 0"020 468 0"011 696 0.078 947 0"207 602 0"043 860 0"017 544 0"093 567 0"055 556 0"029 240 0"073 099 0"032 164 0"035 088 0"040 936 0"011 696 0"023 392 0"035 088 0"032 164 0"029 240 0"040 936 0"026 316
0.002 920 0.010 714 0.006 490 0"002 920 0"007 657 0"005 814 0.014 581 0"021 932 0"011 073 0"007 099 0"015 748 0"012 386 0"009 110 0"014 075 0"009 541 0"009 950 0"010 714 0"005 814 0"008 173 0"009 950 0"009 541 0"009 110 0"010 714 0"008 656
0-003 560 0.038 797 0.015 633 0"002 288 0"022 607 0"010 683 0-073 280 O"199 160 0"038 569 0"023 212 0" 102 010 0"060 846 0"029 301 0"070 386 0"032 603 0"035 026 0"043 649 0"011 257 0"031 559 0"040 480 0"040 708 0"021 073 0"035 543 0"017 772
0.002 598 0.009 627 0-005 838 0"001 748 0"006 946 0"004 472 0"013 359 0"021 152 0"009 366 0"006 797 0"015 774 0"012 099 0"008 136 0"013 151 0"008 861 0"009 048 0"010 182 0"004 341 0"008 544 0"009 870 0"009 889 0"006 629 0"009 169 0"005 885
-0.000 636 0.002 139 -0.001 013 0"000 636 --0"002 139 0"001 013 0.005 668 0"008 442 0"005 291 --0"005 668 --0"008 443 --0"005 290 --0"000 061 0"002 713 --0"000 439 0"000 062 --0"002 713 0"000 439 --0"008 167 --0"005 392 --0"008 544 0"008 167 0"005 393 0"008 544
P123(zt2) P123(213) p123(221) p123(222) P123(223) P123(3!1) P123(312) P123(313) P123(321~ P123(322) P123(323) P123(411) P123(412) P123(413) Pl23(421) Pt23(422)
P123(423)
ap123(ijk ) denotes the joint probability that human error was attributed to the i, j, kth
levels of variables 1, 2, 3, respectively.
to predict failure probabilities which result from the combinations of h u m a n error causes. From the results of Table 7, the estimated probability of h u m a n error under the combination of task and equipment characteristics, characteristics of people resulting from external influences, and psychological stressor (P123(212)) is 0-199. Similarly, among task and equipment characteristics, characteristics of people resulting f r o m internal or other influences and psychological stressor, the probability (Px23~222)) is 0.102. Obviously, for the failure probability, it is worthwhile to improve the internal PSF under task and equipment characteristics and psychological stressor. In this study, the h u m a n error data are collected from an experiment and are analyzed through a loglinear model to find a best fit model. In addition, the likelihood of error inducing the combined effects of three PSFs are estimated based on the optimal model. In the real system, the h u m a n error data can be collected from accident reports, and the contingency table (see Table 3) may be derived by experienced engineers and h u m a n factor specialists. Then, the systematic procedure of quantifying h u m a n errors developed in this study may be beneficial to
search for significant factors and to predict the likelihood of h u m a n error occurring in the system.
ACKNOWLEDGMENT
The financial support by National Science Council of Taiwan, with the project n u m b e r # N S C 80-0415E0007-10, is gratefully acknowledged.
REFERENCES
1. Rubinstein, E., Three Mile Island and the future of nuclear power. IEEE Spectrum, 16 (1979) 30-111. 2. Danaher, J. W., Human error in ATC system operations. Human Factors, 22 (1980) 535-45. 3. Rouse, W. B., & Rouse, S. H., Analysis and classification of human error. IEEE Trans. Systems, Man, and Cybernetics, SMC-13 (1983) 539-49. 4. Rasmussen, J., Skills, rules, and knowledge; signals, signs and symbols, and other distinctions in human performance models. IEEE Trans. Systems, Man, and Cybernetics, SMC-13, 1983 May/Jun, pp. 257-66. 5. Rasmussen, J., Cognitive control and human error mechanisms. In New Technology and Human Error, ed.
The application of the loglinear model to quantify human errors
6.
7.
8. 9. 10.
J. Rasmussen, K. Duncan & J. Leplat. John Wiley & Sons, Chichester, 1987, pp. 53-61. Reason, J., Generic error-modeling system (GEMS): A cognitive framework for locating common human error forms. In New Technology and Human Error, ed. J. Rasmussen, K. Duncan & J. Leplat. John Wiley & Sons, Chichester, 1987, pp. 63-83. Taylor, J. R., Using cognitive models to make plants safer: experimental and practical studies. In Tasks, Errors and Mental Models ed. L. P. Goodstein, H. B. Andersen & S. E. Olsen. Ris~ National Laboratory, Denmark, Taylor & Francis Ltd., London 1988, pp. 233-9. Meister, D., A critical review of human performance reliability predictive methods, IEEE Trans. Reliability, R-22 (1973) 116-23. Lees, F. P., Quantification of man-machine system reliability in process control. IEEE Trans. Reliability, R-22 (1973) 124-31. Swain, A. D., & Guttmann, H. E., Handbook of
Human Reliability Analysis with Emphasis on Nuclear Power Plant Applications. US Nuclear Regulatory Commission, NUREG/CR-1278, Washington, DC, 1983. 11. Adams, J. A., Issues in human reliability. Human Factors, 24 (1982) 1-10. 12. Wreathall, J., Operator Action Trees: An Approach to Quantifying Operator Error Probability during Accident Sequences. Report No. 4159, NUS Corporation, Gaithersburg, Maryland, USA, 1982. 13. Sanderson, P. M., & Harwood, K., The skills, rules and knowledge classification: A discussion of its emergence and nature. In Tasks, Errors and Mental Models, ed. L. P. Goodstein, H. B. Andersen & S. E. Olsen, Ris~
14,
15. 16. 17. 18. 19.
20. 21.
22.
165
National Laboratory, Denmark. Taylor & Francis Ltd, London, 1988,pp. 21-34. Embrey, D. E., Humphreys, P. C., Rosa, E. A., Kirwan, B., & Rea, K., SLIM-MAUD: An Approach to Assessing Human Error Probabilities Using Structured Expert Judgment. Report No. NUREG/CR-3518, US Nuclear Regulatory Commission, Washington, DC, 1983. Vestrucci, P., The logistic model for assessing human error probabilities using the SLIM method. Reliability Engineering and System Safety, 21 (1988) 189-96. Apostolakis, G. E., A critique of recent models for human error rate assessment, Reliability Engineering and System Safety, 22 (1988) 201-17. Ujita, H., Human error classification and analysis in nuclear power plants, Journal of Nuclear Science and Technology, 22(6) (1985) 496-8. Carnino, A., An EDF perspective on human factors. Reliability Engineering and System Safety, 24 (1989) 95-111. Embrey, D. E., The Use of Performance Shaping Factors and Quantified Expert Judgment in the Evaluation of Human Reliability: An Initial Appraisal, Report No. NUREG/CR-2986, Brookhaven National Laboratory, Upton, New York, 1983. Fienberg, S. E., The Analysis of Cross-Classified Categorical Data, The MIT Press, Boston, Massachusetts USA, 1981. Goodman, L. A., The analysis of multidimensional contingency tables: Stepwise procedures and direct estimation methods for building models for multiple classification, Technometrics, 13 (1971) 33-61. Everitt, B. S., The Analysis of Contingency Tables, Ch. 5, Chapman and Hall, London, 1977.
The application of the Ioglinear model to quantify human errors Ya-Lih Lin & Sheue-Ling Hwang Department of Industrial Engineering, National Tsing-Hua University, Hsin-Chu, Taiwan (Received 14 March 1991; accepted 11 October 1991)
This paper concentrates on using a quantitative method, the loglinear model, to investigate human error under a state of emergency. The ioglinear model provides a systematic approach to find out the factors which lead to occurrences of human errors. The external performance shaping factor (PSF), internal PSF, and stressor PSF were found to be the statistically significant factors of human errors. In addition, the first-order interaction between external PSF and stressor PSF, as well as the second-order interaction of these three factors, were also verified as important parts of the human error model. Furthermore, the maximum likelihood which predicts human error probabilities could be obtained from the selected loglinear model. In contrast with the qualitative approach, two major quantitative approaches can be taken to characterize human errors: probabilistic and causal approaches. The probabilistic approach is typically pursued by those who are interested in the human reliability analysis ( H R A ) . 8-12 The causal approach can be used to trace the causes and contributing factors, but does not consider the mechanism of human error. 3 By using the concept of performance shaping factors (PSFs), 1° this study starts with such a cognitive approach to the causes of human errors. As mentioned previously, the cognitive approach to human error provides an operational tool which allows us to pay more attention to the human causes. The cognitive processes reveal what and how information is perceived, and what one decides to do about it. Although various cognitive mechanisms and causes for human errors were considered, it seems very difficult to quantify all the influences of the cognitive aspects of behavior and the interactions of PSFs on the behavioral processes. 1° Therefore, in order to measure the 'effects' of factors and causes contributing to human errors more accurately and objectively, we make an attempt to find out the quantitative function of the cognitive mechanism by employing the loglinear model. Loglinear models have been extensively used in the analysis of contingency tables, also applied in psychology, social sciences, psychiatry, etc. The major advantages obtained from use of these techniques are: (1) They provide a systematic approach to the analysis of multidimensional tables.
INTRODUCTION In emergency situations, human performance is always degraded by delaying responses, ignoring information or processing information incorrectly provided, so that mental workload is far overloaded than in normal operations. It has been reported that 7 0 % - 9 0 % of system failures are directly or indirectly due to human error, ~-3 and resulted in great damage to the systems or to human beings. Therefore, it is important to understand the mediating process or cognitive behavior of humans and investigate the potential factors of human error so that an incident in the emergency situation can be prevented. The current literature on human error in systems typically emphasize either a qualitative or a quantitative approach. In essence, the qualitative approach provides a structural framework for the system designer to recognize and evaluate the errors in the system. A variety of mental, cognitive or conceptual models have been proposed for human error. 3-7 From the qualitative models, the system designer develops a logical concept of what should be considered to prevent operator's errors, but cannot obtain a detailed procedure that designates the error characteristics of the operator. Moreover, one common deficiency of those mental models is the lack of a clear guideline for transferring the concept to the instructions of design.
Reliability Engineering and System Safety 0951-8320/92/$05.00 © 1992 Elsevier Science Publishers Ltd, England. 157
158
Ya-Lih Lin, Sheue-Ling Hwang
(2) They provide estimates of the magnitudes of effects of interest. (3) They allow the relative importance of different effects to be judged. The objective of this study is to find out the significant factors causing human errors so that the system designer can focus on these significant factors to reduce human errors more efficiently and effectively. This study presents how a loglinear model can be used to analyze the cognitive mechanism of human errors and identify those PSFs relevant to HRA. By the analysis of categorical data from the aspect of human cognition, the loglinear model of human errors was fitted and the PSFs with significant contribution to the occurrence of human errors were found. Also, we obtained the predicted failure probabilities of the combinations of the categorized levels of these three factors (the external, internal and stressor PSFs).
BACKGROUND A number of studies have been attributed to the application of mental models or cognitive models in the study of human error. One of the articulate discussions of the relationships between mental models and human error in the operation of engineering systems has been provided by Rasmussen. 4 In ordinary, familiar circumstances, the human operator appears to rely upon available heuristics and rules of operation. In other words, the operator's behavior is rule-based. However, in unusual situations for which rules do not apply, the human operator must reason at a knowledge-based level, using an understanding of the functioning of the system to determine an appropriate course of action. The skill-based, rule-based, knowledge-based classification (which we shall henceforth abbreviate to 'SRK') is a result of Rasmussen's long-standing concern with how to reduce human error in the control of complex systems.~3 The SRK distinction has contributed to the development of a taxonomy of human malfunction and a framework for cognitive task analysis. However, classification of behavior does not spell out the relationship between levels or categories, nor does it provide an explanation for why one level might be chosen over another. The previous models of human error could be viewed as frameworks in the assessment of human error in emergency situations. Swain and Guttmann ~° propose the concept of PSFs which might influence the likelihood of a human error in a given situation. A long series of arguments lead the authors to derive a nominal value for each error rate, as well as upper and lower uncertainty bounds (UCB). They suggested that these UCBs were used as the 5th and 95th
percentiles of a lognormal distribution for the error rate. After the milestone contribution of Swain and Guttmann who proposed the technique for human error rate prediction (THERP), there has been increasing concern about the role of human beings in the normal operation of nuclear, chemical and other industrial plants. THERP recognizes a host of problems which affect human reliability under the name of 'performance shaping factors'. Embrey, et al. 14 proposed the success likelihood index methodology (SLIM); the basic rationale underlying SLIM is that the likelihood of an error in a particular situation depends on the combined effects of a relatively small set of PSFs such as environment, state of current practice or skill, and time constraints. They claim that 'experimenal evidence suggests that the success likelihood index (SLI) is related to the logarithm of the success probability for a task'. Vestrucci ~5 suggested the logistic model in which SLI is related to the logarithm of ratio of the failure probability to success probability. A critique has been proposed by Apostolakis,16 and the major observations of this review are as follows: SLIM provides a highly structured approach for the derivation of human error rates under given conditions. However, the treatment of the weights and ratings in this model is internally inconsistent. In addition, the assumption that the weights of the performance shaping factors are independent of the task ratings may not accurately reflect some real world situations. Ujita 17 considered that human error causes were a function of task type. A two-dimensional array was displayed to perform a descriptive statement between error causes and task type. However, it lacks of further inference for the human error. After reviewing the above methods, this research focused on building a quantitative model and making an inference of failure probabilities concerning all the causes. Then an experiment was conducted to verify the feasibility of the loglinear model. The loglinear model also provided a systematic approach for the interpretation of human error causes and the prediction of the human error rates. Moreover, in terms of the findings of the failure probabilities, the system designer or trainer will have had some directions to follow to improve system reliability.
METHOD A cognitive model In modeling human performance for probabilistic risk assessment (PRA), it is necessary to consider those factors that have the most effect on performance.
The application of the loglinear model to quantify human errors
affecting performance
Iluman errors
Mechanisms
Factors v
of human malfunction
v r . . . . . . . . .
|
I Internal
I I
I
human
I
mode of malfunction
:| malfunction,
External
Incorrect
PSFs
mediating
l.ternal
(cognitive)
human
activities
inputs
PSFs Stressor
(Incorrect
& processes
PSFs
159
to system) L . . . . .
(hq)uts)
. ....
(Outputs)
Fig. 1. A cognitive model of human malfunction involving human errors as outputs.
Many factors affect human performance in a complex man-machine system. Some of these performance shaping factors (PSFs) are external to the person and others are internal. The external PSFs include the entire work environment, especially the equipment design, the written procedures, and oral instructions. The internal PSFs represent the individual characteristics--skills, motivations, and the expectations that influence one's performance. Psychological and physiological stresses result from a work environment in which the task demands for operators in the system do not conform to their capabilities. The cognitive model proposed here was based on previous studies l°'~s but a modification was made. Figure 1 represents a general framework of human error in systems such as aircraft, ships, power plants, or process control systems. To perform an H R A , one must identify those most relevant and influential PSFs in the model. One may consider the external, internal, and stressor PSFs as inputs (independent variables) of the cognitive mechanism which result in human errors. If these three variables occur simultaneously, then they form a three-dimensional contingency table. The frequencies of human error are treated as dependent variables (or responses). They are categorized with respect to three variables and the underlying levels (Table 1).~9 T h e ioglinear m o d e l
In contingency table analysis, the loglinear model was employed to assess the main effects and interaction effects of independent variables on a dependent variable. Also the maximum likelihood analysis of variance was used as the primary statistical tool to partition overall variability for human error data
Table 1. Three independent variables for the human errors and the underlying levels
Variables
Levels
A. External PSF
I. Situational characteristics II. Task and equipment characteristics III. Job and task instructions IV. Others
B. Internal PSF
I. Characteristics of people resulting from external influences II. Characteristics of people resulting from internal or other influences
C. Stressor PSF
I. Distraction of secondary task II. Psychological stressors III. Physiological stressors and others
analysis. For the three-dimensional tables, let Xij k be the observation in ith row, jth column, and kth layer of the table, and let misk be the corresponding expected value for that entry under some model. TM The general loglinear model is written as follows: log mij k = U +
Ul(i) ~1_ U2(j ) .~_ U3(k ) q_
U12(ij ) .~_ Ul3(ik )
"Jr"U23(jk) "Jr"U123(ijk) ,
(1)
where u is the grand mean of the logarithms of expected counts, 1
1
s
U=I~i~----lj~=l k=l~'d log m#~,
(2)
Ul(i) is the main effect of the logarithms of the expected counts at level i of the first variable, 1 s K u + Ul(i) = f ~ ~ ~ log mijk, ix j = l k = l
(3)
160
Ya-Lih Lin, Sheue-Ling Hwang
and, similarly, log mijk,
(4)
U'JrU3(k) = ~a i =El j E=1 log mijk,
(5)
u + u2(j) = ~-K
i=l k=l
1
~
J
In eqn (1), u12(ij), u13(ik), and U23(jk) are the first-order interactions (or the two-factor effects) of the logarithms of the expected counts, and u123(ijk) is the second-order interaction (or the three-factor effect) of the logarithms of the expected counts. Estimation of the interaction effects would be useful in identifying those categories responsible for any departure from independence. As in the usual A N O V A model, u~(o, u2(j), and u3(k) represent deviations from the grand mean u. E UI(i) = E U2(j) = E U3(k) = O, i j k
(6)
In addition to (6), we have
E U12(ij)= E U12('1)-~-E U13(ik)= E Ul3(ik) i j i k : E U23(jk) = E j
U 2 3 ( / k ) = 0,
(7)
assigned to use one of the experimental procedures under an emergency situation. All subjects were either industrial engineering or materials science engineering majors and have completed courses in physics and calculus. It is important to note that although the use of students as subjects is often considered as compromising to the credibility of applied research, this subject population was well-suited to the questions at hand. This is because operators in many systems (e.g., nuclear power plants) were required to complete a training program that was technically equivalent to that required for a bachelor's degree in engineering. Therefore, it was argued that these students had educational backgrounds comparable to actual operator trainees in some domains.
Apparatus This study was conducted in a laboratory equipped with an electron-beam evaporation system and a personal computer. The electron-beam evaporation system consists of three parts: vacuum pumping, deposition, and cooling subsystems. A graphic display for the system is shown in Fig. 2.
k
and
Experimental tasks u,23(,jk) = i
u,23(0 ]
) =
u,23,j
> = o.
(8)
k
Once we have estimated expected values under one of the loglinear models, we can check the goodness-of-fit of the model using the likelihood-ratio chi-square statistic: / Observed \
02= 2 E (Observed)logl
(9)
where the summations are over all cells in the table. If the model fitting is adequate and the total sample size is large, G z has an approximate X2 distribution with degrees of freedom given by the following formula: d.f. = # cells - # parameters fitted.
(10)
Experiment The human error data were collected from an experiment of the electron-beam evaporation system operated under emergency situations. The primary objective of this experiment was to verify the feasibility of the loglinear model, and search for the contributing factors and the functional form of human error mechanism.
Subjects Thirteen graduate and 16 undergraduate students of National Tsing Hua University were paid to take part in this experiment as subjects. They were randomly
Both primary and secondary tasks were simultaneously assigned to a subject who had acquired a systematic training in the operational principles and procedures for dealing with the emergency incidents of the electron-beam evaporation system. The primary task was to supervise the vacuum pumping process or the deposition process of the electron-beam evaporation system. Meanwhile, the secondary task was either to play a guessing game, or to operate an arithmetic test on an IBM PC/XT. Many studies showed that fatigue due to long working hours or highly concentrated work resulted in less attention to certain types of signals. The secondary task was used to increase the work load, and consequently increase the internal and external stress of the subjects.
Experimental procedures (1) Subjects were taught about the routine operation principles and the procedures to manipulate the electron-beam evaporation system. (2) To perform a supervisory task on the vacuum pumping and play a guessing game. Accidents such as cooling water supply breakdown or power supply breakdown occurred randomly. If an accident occurred, the subject then pressed a corresponding key. (3) To perform a supervisory task on the deposition part and an arithmetic operation. In addition, the same accidents and corresponding
The application of the loglinear model to quann'fy human errors
161
8~
11'
i
12
13
1 ri] L _ . . .
Components of vacuum pumping subsystem:
Components of deposition subsystem:
(1)
Main power-1
(8)
Main power-2
(2)
Rotary pump
(9)
Thickness monitor power supply
(4)
Heater
(10)
P~tameter setting board
(5)
Foreline valve
(12)
Voltage
(6)
Roughing valve
(13)
Ampere
(7)
Main valve or high vacuum valve
(14)
Shutter
Components of cooling subsystem: (3)
Cooling water valve--1
(11)
Cooling water valve-2 Fig. 2. A graphic display for an electron-beam evaporation system.
key-operations as those designated in procedure (2) were also assigned to the subject. (4) At the end of the experiment, the total number of occurrences of human errors were counted and the subject was asked to take a questionnaire.
Data collection During the experiment, the subject was asked to report what and why he/she was doing whenever he/she took an action, and both actions and answers
were recorded by the experimenter. After the experiment, the subject took a questionnaire (Table 2) to repeat and/or add some explanation of his/her actions and the experimenter could revise the answer of the previous record if necessary. The PSFs in the questionnaire were adapted from NUREG/CR-1278 for the electron-beam evaporation system. In taking the questionnaire, the subjects were asked to choose the most likely factor(s) which caused a specific error he/she made in the experiment. Table 2 is a sample questionnaire to which a subject has responded. In this case, the subject recalled the situation when
162
Ya-Lih Lin, Sheue-Ling Hwang
Table 2. The questionnaire which listed the related PSFs resulting in human error for the electron-beam evaporation system
A. External PSFs I. Situational characteristics: -Organizational structure (e.g. Authority, responsibility, communication channels). -Actions by supervisors. -Rewards, benefits. II. Task and equipment characteristics: -Perceptual requirements. -Motor requirements (speed, precision). -Control-display relationships. -Decision-making. -Complexity (information load). -Task criticality. -Long- and short-term memory. -Feedback (knowledge of results). -Man-machine interface factors (design of prime equipment, test equipment, job aids). III. Job and task instructions: -Procedures required (written or not written). -Written or oral communication. -Cautions and warnings. -Work methods. IV. Others B. Internal PSFs I. Characteristics of people resulting from external influences: -Previous training/experience. -State of current practice or skill. -Knowledge of required performance standards. II. Characteristics of people resulting from internal or other influences: -Personality and intelligence variables. -Motivation and attitudes. -Emotional state. -Attitudes based on influence of family and other outside persons or agencies. -Others. C. Stressor PSFs I. Distraction from secondary task II. Psychological stressors: -Suddenness of onset. -Duration of stress. -Task speed. -Task load. -Threats (of failure, loss of job). -Monotonous, degrading, or meaningless work. -Long, uneventful vigilance periods. -Distractions (Noise, glare, movement). III. Physiological stressors and others: -Duration of stress. -Fatigue. -Pain or discomfort. -Hunger or thirst. -Movement constriction. -Others. he/she made an error and chose the 'motor requirements', 'state of current practice or skill', and 'task speed' as the main factors. As shown in Table 2, the questionnaire consists of three parts; 1) External PSFs, 2) Internal PSFs, and 3)
Table 3. The contingency table of human error data
Variables
Levels'~
A. External PSF:
I
B. Internal PSF: C. Stressor: PSF:
I II III
II
I
II
1 14 5
1 7 4
I
1II II
I
IV II
I
II
27 6 10 12 8 10 71 32 25 14 12 14 15 19 11 4 11 9
a The levels for each variable are described in Table 1. Stressor PSFs. Each part contains 2 - 4 levels and there are several items under each level which can be selected. For example, if a subject selected 'motor requirements' under Level II, 'Task and Equipment', as shown In Table 2, this would be counted as one occurrence of External PSFs Level II, in Table 3.
RESULT
The frequencies of factors which caused some specific human errors are shown in Table 3. Within the framework of loglinear models, it is important to look at all possible sources of variation and estimate the parameter effects. From the maximum likelihood analysis of variance tables of the full or saturated model, it leads to the consideration of a limited number of loglinear models as part of a preliminary step in the model-building process, z~ A cursory examination suggested that both u13 (the interaction effect of external PSF and stressor PSF) and u123 (the interaction effect of external, internal, and stressor PSFs) must be definitely included in our model since their chi-square values were 14.52 and 12.88 (p < 0-05), respectively. The next step is to examine the maximum likelihood estimates for all parameter effects (u-terms) in the full or saturated model. Techniques such as maximum likelihood and least squares may be used to estimate the parameters, and estimated parameter values may then be used in identifying which variables are of greatest importance in determining the values taken by the observations. 22 G o o d m a n 2~ showed that the standardized values have an asymptotically standard normal distribution, and might therefore be compared with the z-value for any particular probability level ( t x = 0 . 0 5 or 0.01). By the above techniques, we suggest that a model including both interaction terms u~3 and u123 might provide an adequate fit for these data. Consequently, such a model is: log mqk = U + U~ti) + U2(j) + U3(k) + Ul3(ik) q'- U123(ijk)
(11)
The maximum likelihood analysis of variance tables and the estimated parameter effects based on the
The application of the loglinear model to quantify human
Table 4. Maximum likeBhood analysis of variance for the model specified in eqn (11) fitted to the data in Table 3
Source
DF
Chi-square
p-value
Ul U2 U3 U13 u123 Likelihood ratio
3 1 2 6 6 5
61"87 14"35 44"84 13"66 15-25 5.96
0"0000~ 0"0002~ 0"0000" 0"0337b 0"0184 b
0-3100
° p<0.01 b p<0.05 selected model in eqn (11) are shown in Tables 4 and 5, respectively. The value of G 2 is not significant (p = 0.31) and implies an acceptable fit of the model. In addition, applying formula (9) to each model, a summary of the value of G 2 and probability level for each of the possible loglinear models to the data in Table 5. Maximum likelihood estimates of the parameters for the model specified in eqn (11) fitted to the data in Table 3 Parameter
ulo) u1<2) u1(3) u2o) U30)
U3(2) ula(m
U13(12) u13(21)
U13(22) U13(31) /213(32) U123(111) U123012) U123(211)
Ul23(212) U123(311) U123(312)
Estimate
-1.0153 0-839 2 0.1309 0.227 1 -0.4761 0.652 4 -0.807 2 0.403 9 0.009 06 O"120 7 0-464 7 -0.115 5 --0'006 15 0"042 9 0"347 7 0" 107 4 --0"316 4 0"011 8
Standard error
Chi-square
0.2092 O. 106 9 0.1234 0.059 9 0.1423 0.098 5 0'377 5 0-236 0 0.177 6 O"127 3 0-190 8 0-151 6 0"229 7 0" 162 0 0"138 7 0"096 2 0"167 9 0"132 9
23.56 61.63 1.12 14.35 11.19 43.84 4.57 2-93 0.00 0'90 5-93 0-58 0"00 0"07 6"29 1"25 3'55 0"01
a p<0.01
b p<0.05
p-value
0.000 0" 0.000 lY 0.2890 0.000 2a 0.0008 a 0-000 0~ 0.032 5 b 0.087 0 0.959 3 0"342 8 0.014 9 b 0.446 1 0"978 6 0"791 0 0"012 2 0'264 3 0"059 5 0"929 3
163
errors
Table 3 is obtained (Table 6). Table 6 indicates that models ( a ) - ( e ) do not provide an adequate fit for these data. In regard to model (g), it is plausible to be a good fit since the value of G 2 is nonsignificant, so it is a candidate for model selection. But the contribution of ux2-effect is not significant (chi-square v a l u e = 5 . 5 0 , p > 0 . 0 5 ) . For the principles of parsimony and simplicity, model (g) is not adoptable. Therefore, we might conclude that the model in eqn (11) or model (f) of Table 6 provides an adequate fit for the data. In other words, a model which includes a first-order interaction between an external PSF and a stressor PSF, and a second-order interaction among an external PSF, an internal PSF, and a stressor PSF was chosen which fits the data reasonably well. Furthermore, the maximum likelihood predicted value for response (i.e. human error) probabilities can be calculated (Table 7). These failure probabilities can be used to predict the occurrence of the human errors.
DISCUSSION AND CONCLUSION
There are some important results of this study. First, the loglinear model provides a systematic approach to obtaining an adequate human error model from an aspect of human cognition. Secondly, an appropriate model in which some PSFs are found to contribute significantly to the reduction of the number of human errors was selected. In addition, the first-order interaction (u13) between external PSF ( u 0 and stressor PSF (u3), as well as the second-order interaction of these three factors (u123), are verified as important parts of the human error model. Moreover, the results of Table 5 indicate that there is a positive association between the job and task instructions of external PSF and distraction of secondary task of stressor PSF [/~13(31) = 0.4647, p < 0.05], and a positive association among the task and equipment characteristics of external PSF, characteristics of people resulting from external influences and distraction of secondary task [/~123(211) ~---0.3477, p < 0.05]. Thirdly, the most important thing for a human error model is
Table 6. Likelihood ratio chi-square values for some Ioglinear models applied to the data in Table 3
Model (a) (b) (c) (d) (e) (f) (g)
log mqk = u + ul(o + u2t/) + u3(r) log mOk = U + UI(O + U2(j)+ U3(r) + U12(ij) log m,ik = U + Ut(o + U2(i) + U~(X) + Ut3(~k) log m q k = U + Ul(i) + U2(i) + U3(K) + Ul2(ii) "~- U13(ik) log m~ik = u + ulfo + U2(l) + u3(x) + ut2(e) + U130k) + U23(ik) log rail k = u + Ill(,) + U2(1) + U3(K) + Ula(ik) "~- U123(ijk) log m0k = u + ul(o + u2(/) + u3(r) + u12(ii)+ u13(ik)+ U123(qk),
"p
D.F.
G2
p-value
17 14 11 8 6
37"50 31"18 21"91 15"59 13"78 5.96 0"43
0.002 9a 0.005 2a 0.025 1b 0-048 7b 0.032 20 0-3100 0.807 1
5
2
Ya-Lih Lin, Sheue-Ling Hwang
164
Table 7. Maximum likelihood predicted values for response probabilities (or predicted human error probabilities) °
Observed
Predicted
Function number
Human error probability
Standard error
Human error probability
Standard error
Residual
P123(111) P123o12) Pl23(113) P123(121) Pl23(122) Pt23(123) P123(211)
0.002 924 0-040 936 0.014 620 0"002 924 0"020 468 0"011 696 0.078 947 0"207 602 0"043 860 0"017 544 0"093 567 0"055 556 0"029 240 0"073 099 0"032 164 0"035 088 0"040 936 0"011 696 0"023 392 0"035 088 0"032 164 0"029 240 0"040 936 0"026 316
0.002 920 0.010 714 0.006 490 0"002 920 0"007 657 0"005 814 0.014 581 0"021 932 0"011 073 0"007 099 0"015 748 0"012 386 0"009 110 0"014 075 0"009 541 0"009 950 0"010 714 0"005 814 0"008 173 0"009 950 0"009 541 0"009 110 0"010 714 0"008 656
0-003 560 0.038 797 0.015 633 0"002 288 0"022 607 0"010 683 0-073 280 O"199 160 0"038 569 0"023 212 0" 102 010 0"060 846 0"029 301 0"070 386 0"032 603 0"035 026 0"043 649 0"011 257 0"031 559 0"040 480 0"040 708 0"021 073 0"035 543 0"017 772
0.002 598 0.009 627 0-005 838 0"001 748 0"006 946 0"004 472 0"013 359 0"021 152 0"009 366 0"006 797 0"015 774 0"012 099 0"008 136 0"013 151 0"008 861 0"009 048 0"010 182 0"004 341 0"008 544 0"009 870 0"009 889 0"006 629 0"009 169 0"005 885
-0.000 636 0.002 139 -0.001 013 0"000 636 --0"002 139 0"001 013 0.005 668 0"008 442 0"005 291 --0"005 668 --0"008 443 --0"005 290 --0"000 061 0"002 713 --0"000 439 0"000 062 --0"002 713 0"000 439 --0"008 167 --0"005 392 --0"008 544 0"008 167 0"005 393 0"008 544
P123(zt2) P123(213) p123(221) p123(222) P123(223) P123(3!1) P123(312) P123(313) P123(321~ P123(322) P123(323) P123(411) P123(412) P123(413) Pl23(421) Pt23(422)
P123(423)
ap123(ijk ) denotes the joint probability that human error was attributed to the i, j, kth
levels of variables 1, 2, 3, respectively.
to predict failure probabilities which result from the combinations of h u m a n error causes. From the results of Table 7, the estimated probability of h u m a n error under the combination of task and equipment characteristics, characteristics of people resulting from external influences, and psychological stressor (P123(212)) is 0-199. Similarly, among task and equipment characteristics, characteristics of people resulting f r o m internal or other influences and psychological stressor, the probability (Px23~222)) is 0.102. Obviously, for the failure probability, it is worthwhile to improve the internal PSF under task and equipment characteristics and psychological stressor. In this study, the h u m a n error data are collected from an experiment and are analyzed through a loglinear model to find a best fit model. In addition, the likelihood of error inducing the combined effects of three PSFs are estimated based on the optimal model. In the real system, the h u m a n error data can be collected from accident reports, and the contingency table (see Table 3) may be derived by experienced engineers and h u m a n factor specialists. Then, the systematic procedure of quantifying h u m a n errors developed in this study may be beneficial to
search for significant factors and to predict the likelihood of h u m a n error occurring in the system.
ACKNOWLEDGMENT
The financial support by National Science Council of Taiwan, with the project n u m b e r # N S C 80-0415E0007-10, is gratefully acknowledged.
REFERENCES
1. Rubinstein, E., Three Mile Island and the future of nuclear power. IEEE Spectrum, 16 (1979) 30-111. 2. Danaher, J. W., Human error in ATC system operations. Human Factors, 22 (1980) 535-45. 3. Rouse, W. B., & Rouse, S. H., Analysis and classification of human error. IEEE Trans. Systems, Man, and Cybernetics, SMC-13 (1983) 539-49. 4. Rasmussen, J., Skills, rules, and knowledge; signals, signs and symbols, and other distinctions in human performance models. IEEE Trans. Systems, Man, and Cybernetics, SMC-13, 1983 May/Jun, pp. 257-66. 5. Rasmussen, J., Cognitive control and human error mechanisms. In New Technology and Human Error, ed.
The application of the loglinear model to quantify human errors
6.
7.
8. 9. 10.
J. Rasmussen, K. Duncan & J. Leplat. John Wiley & Sons, Chichester, 1987, pp. 53-61. Reason, J., Generic error-modeling system (GEMS): A cognitive framework for locating common human error forms. In New Technology and Human Error, ed. J. Rasmussen, K. Duncan & J. Leplat. John Wiley & Sons, Chichester, 1987, pp. 63-83. Taylor, J. R., Using cognitive models to make plants safer: experimental and practical studies. In Tasks, Errors and Mental Models ed. L. P. Goodstein, H. B. Andersen & S. E. Olsen. Ris~ National Laboratory, Denmark, Taylor & Francis Ltd., London 1988, pp. 233-9. Meister, D., A critical review of human performance reliability predictive methods, IEEE Trans. Reliability, R-22 (1973) 116-23. Lees, F. P., Quantification of man-machine system reliability in process control. IEEE Trans. Reliability, R-22 (1973) 124-31. Swain, A. D., & Guttmann, H. E., Handbook of
Human Reliability Analysis with Emphasis on Nuclear Power Plant Applications. US Nuclear Regulatory Commission, NUREG/CR-1278, Washington, DC, 1983. 11. Adams, J. A., Issues in human reliability. Human Factors, 24 (1982) 1-10. 12. Wreathall, J., Operator Action Trees: An Approach to Quantifying Operator Error Probability during Accident Sequences. Report No. 4159, NUS Corporation, Gaithersburg, Maryland, USA, 1982. 13. Sanderson, P. M., & Harwood, K., The skills, rules and knowledge classification: A discussion of its emergence and nature. In Tasks, Errors and Mental Models, ed. L. P. Goodstein, H. B. Andersen & S. E. Olsen, Ris~
14,
15. 16. 17. 18. 19.
20. 21.
22.
165
National Laboratory, Denmark. Taylor & Francis Ltd, London, 1988,pp. 21-34. Embrey, D. E., Humphreys, P. C., Rosa, E. A., Kirwan, B., & Rea, K., SLIM-MAUD: An Approach to Assessing Human Error Probabilities Using Structured Expert Judgment. Report No. NUREG/CR-3518, US Nuclear Regulatory Commission, Washington, DC, 1983. Vestrucci, P., The logistic model for assessing human error probabilities using the SLIM method. Reliability Engineering and System Safety, 21 (1988) 189-96. Apostolakis, G. E., A critique of recent models for human error rate assessment, Reliability Engineering and System Safety, 22 (1988) 201-17. Ujita, H., Human error classification and analysis in nuclear power plants, Journal of Nuclear Science and Technology, 22(6) (1985) 496-8. Carnino, A., An EDF perspective on human factors. Reliability Engineering and System Safety, 24 (1989) 95-111. Embrey, D. E., The Use of Performance Shaping Factors and Quantified Expert Judgment in the Evaluation of Human Reliability: An Initial Appraisal, Report No. NUREG/CR-2986, Brookhaven National Laboratory, Upton, New York, 1983. Fienberg, S. E., The Analysis of Cross-Classified Categorical Data, The MIT Press, Boston, Massachusetts USA, 1981. Goodman, L. A., The analysis of multidimensional contingency tables: Stepwise procedures and direct estimation methods for building models for multiple classification, Technometrics, 13 (1971) 33-61. Everitt, B. S., The Analysis of Contingency Tables, Ch. 5, Chapman and Hall, London, 1977.