- Email: [email protected]
The Effects of Participatory Mode on the detection of Dynamic System Failure
CopHight © I FAC \Ian-\Ia chine Svstems. Oulu. Finland. 19HH
THE EFFECTS OF PARTICIPATORY MODE ON THE DETECTION OF DYNAMIC SYSTEM FAILURE S. Sugiyama*, N. Yuhara** and S. Horiuchi** *IBM japan. Engineering SYSIf11LI, Tokyo, japan ""College of SCifllff ant! Technology, Xi/lOll L'lIi,'ersily. Tokyo, japan
ABSTRACT While some papers argue that the controller requires shorter detection time to detect system failures in a single-loop task, some others contend that in a multi-loop task, the monitor achieves shorter detection time. The present paper discusses a series of experi ments on the validity of the assumption that "such findings have come from the intermittency in which the human operator recieves visual information" (this intermittency of information corresponds to the numbers of control axes and instruments). The experiments included a subsidiary task to press the push button for making visual information intermittent. The interval ratio (IR), the ratio of the time for watching the subsidiary task to the total time period of each trial, was defined as an index for the intermittency of visual information. More specifically, IR;O% corresponds to an instance where a single-loop task with one instrument is being performed, and an increase in this ratio means an increase in the number of control axes or monitoring instruments. The experiments found that whatever the Significance or type of system failure might be, the controller achieved shorter detection time than the monitor at IR=O%, while at IR;30% to 40% or higher, the controller needed longer detection time. This finding indicates that the intermittency of visual information recieved by the human operator is a major factor of the difference between the controller and the monitor in detection time. Keywords
Human Engineering; Failure Detection; Manual Control; Monitoring Behavior; Workload.
INTRODUCTION
To ensure safe operation of a system, it is important to not only improve the reliability of system components themselves but also detect a system failure quickly if it occurs . Broadly there are two ways of detecting system failures. One is performed by a human operator assigned to monitor or control the system, and the other is done by a failure detection system using a computer. This paper deals with the first way of failure detection which relies on human operators. Even in the latter way, human operators have to check whether or not the failure detection system is working normally, and in addition, they have great adaptive ability to cope with failures other than those included in the scenario of likely failure modes. Therefore, particular emphasis should be plased on studies on failure detection by human operators.
Fie,1. MONITOR F ~-~
~--
u+
FiC.2. CONTROLLER
The identification of the r elationship between the human operator's participatory modes and failure detection characteristics is an important consideration in studying the construction of a manmachine system.
As shown in Figs. 1 and 2, the detection of system failures by operators may be broadly divided into two categories by the mode of their participation in system operation. The instances under these categories are:
To examine the effects of participatory modes on
failure detection time,Young(1969),Morizumi etal. (1983) conducted a failur e detection experiment on a single- loop system. Their paper says that the controller achieves shorter detection time than the monitor. Yuhara and Horiuchi(1986), after making a failure detection experiment on singleloop system, with the first- and second-order plant dynamics, also report that the controller needs shorter detection time. Based on the findings of a dul-axis pursuit tracking task, Wickens and Kessel (1979) contend that the monitor is better than the controller in the accuracy of
(1) Where the operator, working outside the control loop , monitors the p l ant output and detects system failures, if any (the operator working as a monitor); (2) where the operator performs control manipulation in the control loop and at the same time monitors the plant output, thus detecting system failures, if any (the operator working as a controller ).
2i9
280
S. Sugi\'ama, :'>J. Yuhara and S. Horiuchi
detection, while in detection time, the contrary is the case. With a fixed-base flight simulator, meanwhile, Ephrath and Curry(1977) conducted a multi-loop task simulating a large transport aircraft in the landing-approach flight envelope. They report that the monitor achieves shorter detection time than the controller. After studying why their respective experiments had shown conflicting results, Young and Ephrath (1981) found that detection time was affected by a number of factors including: 1. Participatory mode of the human operator; 2. Work load level; 3. Number of the instruments to be monitored; ard 4. Number of the axes to be controlled. They maintain that the monitor can probably achieve shorter detection time than the controller if high work load is involved in their task which, like flying an aircraft, requires the oprerator to perform multi-loop control while watChing multiple instruments. However this comment is based on the interpretation of the experimental results of Young(op.cit.) and Ephrath(op . cit .) in such a manner that they do not conflict with each other. Another notable point is that no paper has been published on the measurement of work load in Young's experiment(op.cit.) or on a systematic experiments to verify the conclusion noted above. With this in mind, we made an empirical study on why the findings of Young(op.cit.), Morizumi (op.cit.), Yuhara & Horiuchi(op.cit.) and Wickens & kessel(op.cit.) differed from those of Ephrath & Curry(op.cit.). The study was based on the working hypothesis described below: The experiments in Young(op.cit.), Morizumi et al. (op.cit.) and Yuhara & Horiuchi(op.cit.) dealt with the monitoring an instrument or control task in a single-input/sing le-output system. Accordingly the human operator continuously took in visual information from the instrument. In the experiment by Ephrath & Curry(op.cit.) which simulated the landing approach of an aircraft , on the other hand, the human operator scanned a good many instruments, including an attitude indicator, an altitude indicator and lIS indicator, and took in many information from these indicators. Therefore visual information from each of these indicators was intermittently recieved by the operator. We assume that this intermittency of visual information led to the discrepancy between the findings in Young (op.cit.), Morizumi et al. (op.cit.) and those in Ephrath & Curry(op.cit.). The intermittency rate of visual information indirectly reflects the factors in 3 and 4 above, i.e., the numbers of the instruments to be monitored and the axes to be controlled ; an increase in intermittency rate means an increase in the number of the axes to be controlled or the instruments to be monitored. The higher the rate of intermittency is, the more difficult it is to estimate the system characteristics as described by the original continuous information from the intermittent time-series data taken in by the operator. Presumably this leads to longer detection time. EXPERIMENTS Interval ratio IR, used as the intermittency rate of visual information from given instrument X, can be defined as follows: IR=
[I. Cu.lative Dwell Ti.e on X] XlOO[%] Trial Ti.e
(1)
Here IR=O% implies in a single-loop task with an instrument. An increase in IR value means an in-
crease in the number of the instruments to be moni tored or the axes to be controlled. This series of experiments used an instrument (a cathode ray tube) that could display information continually. To change this information into intermittent one which would be recieved by the human operator at varying intermittency rates, a subsidiary task was given him to forcedly move the fixation point of his gaze from time to time. It was a modified version of the subsidiary task used by Ephrath & Curry(op.cit.)(See Fig. 3). The device used for this task in our experiments had lights A and B installed one directly above the other, either of which would go on at any moment accompanied by a buzzer to attract the subject's attention. When hearing a buzzer, he was to find which light was on, and within a second from the moment the light went on, he was to press either of keys A and B which were provided for light A and B, respectively. Pressing the appropriate key would turn off both the light and the buzzer. The subject was also instructed that whenever hearing a buzzer for the subsidiary task, he should perform it immediately, temporarily suspending the failure detction task (hereinafter called the "main task"), if necessary, so that the task allocation between the main and subsidiary tasks would always be kept at the preset value. Therefore, if he pressed the wrong key or failed to press the right key within a second from the moment the light went on, the test was terminated at once and started anew. In tha t case , IR can be rewr i t ten as: IR
Cu.lative Dwell Ti.e on Subtask XlOO[%] Trial Ti.e
(2)
1/8 inch LED A
2 inch B
Fig .3. Subsidiary Task The values of IR could be set at any desirable value by changing the number of times the light was turned on in each trial (which lasted 75 seconds). This frequency of light switch-on to give the specified IR value was determined from a preliminary experiment . First, the subject ' s fixation time at given frequency of light switchon during the main task was measured by a nistamograph, and the period thus determined was given as the fixation time for the subsidiary task. During the preliminary experiment, the system was operated free of any failure. Then the value of IR at such frequency of light switch-on was calculated by equation (2). A calibration curve as shown in Fig.4 for the relationship between the fequency of light switch-on and IRs was obtained by performing several rounds of similar test at varying frquen cies of light switch-on. From this figure, the frequency of light switch-on which results in the desired IR value can be worked out. The figure also shows that for the monitor, the value of IR increases in proportion to the frequency of light switch-on up to IR=70% , while for the controller, IR value increases similarly up to IR=40%. Accordingly the subsidiary task used in this series of experiments can give the desired IR value whichever the paticipatory mode of the human operator may be. The with a subsidiary task given to the operator,
281
Detection of Dynamic System Failure
a failure detection task similar to the one described in Yuhara & Horiuchi(op.cit.) was conducted both in the monitor and =ntroller modes of participation at the specified IR values (10%, 20%, 30%, 40%, 50%, 60% and 70% for the monitor and 10%, 20%, 30% and 40% for the controller). The essential task of the human operator was to detect failure of the linear dynamic system as soon as possible. The normal operating state of the plant used in the experiments is as follows: Wn 2
K s2+2I:wns+wn 2
G(s)= K=\.O,
wn=4.0
(3)
ISTD. DEV.
EXPERIMENTAL RESULTS
~ MONITOR
~~
~
~
~~~ ~
17 33 50 66 83 No. of times the licht was turned on in each trial(75 sec). FiC.4. IR vs. Frequency of licht switch-on 0
The system failure under the experiments scenario was associated with step change in the gain or natural frequency of the plant dynamics. The gain and natural frequency of the plant after the occurrence of a failure are: W
Fai lure Fai I
n=4.0 n=4.0
wn=5.7 wn=6.9
wn=4.0 wn=4.0
wn=2.8 wn=2.2
W W
Movinc Line
Fixed Line
(rad/s)
4 CONTROLLER
Nor.al
Error
....... ··l· .. ·.. ·· ..
Fig.5. Display Format
INTERVAl.. RATIO
70 60 50 40 30 20 10 0
1:=0.7,
experiment. The mean value and variance of detection time were calculated using data from 10 times of correct detection in the 21st and subseqent trials. A well-trained subject was selected for the experiment.
Nor.al
Fai lure Fai I
K=1.0 K=\'O
K=2.0 K=0.5
K
Table.l. Failure Types i Significance The plant dynamics represented by equation (3) was simulated by a digital =mputer and its output was given as vertical displacement of a horizontal line moving on the CRT display as shown in Fig. 5. The forcing function was worked out by sharping a gaussian white noise by a low-pass filter with a cutoff frequency of 100Hz. The subject acting as the controller was instructed to monitor information from the instrument while controlling the system by a spring-loaded manipulator in such a manner as to minimize position error. When deciding that a system failure was detected, he was to press the push button. Meanwhile the monitor was told to perform only the detection task, and when deciding that a failure was detected, he should press the push button. The CRT display gain was set so that the horizontal line indicating error moves within ±2cm during the normal operating state of the plant. A system failure was made to occur at any moment between 10 seconds and 60 seconds after the beginning of each trial. If the failure mode continued for 15 seconds or more after the occurrance of a system failure, i.e., if the subject failed to detect the failure within this time period, the trial was =nsidered a "miss alarm." If the push button was pressed before any system failure occurred, on the contrary, the trial was considered a "false alarm." The subj ects went through some 20 warm-up trials for a given system failure to get familiarlized with the
To evaluate the subsidialy task method described above ,the coherency of the controller dynamics during a control task was calculated by a method based on the fitting of an autoregressive ( AR ) model. In this method, the controller's output c ( a displacement of the control manipurator) and error e on the display screen were given as vector X, and the following two-dimensional AR model was fitted by Whittle's algorism to time-series data on c and e obtained from a manual control experiment: p
X(k)= L: A(i)X(k-i)+£(k) i=1
(4)
Where p represents the order of the AR model, A is an 2*2 AR coefficient matrix, and S is a twodimensional white gaussian noise, the covariance of which is given by: E[£(k)£T(I)]=RSkl
(5)
If the AR model is well-fitted to the time series of test data c and of e, R becomes a diagonal matrix. In that case, spectrum matrix S(jw) representing the power spectra of c and e and the cross spectra between them can be expressed by: -I
S(jw)=t.lA(jw») R [r(jw~
-I
(6)
Where A(jw)=
P ' . 2).e
(7)
i=o
11 stands for sampling time and * for a complex conjugate. From this, coherency r2(jwJ representing the linearity of e and c can be determined by:
And
2
0" (w)=
(8)
Fig. 6 shows the relationship between IR and the coherency of the controller dynamics calculated from the experimental data by the method described above. The figure indicates that as IR increases, 1.0 r
- - IR= 0 %
o·gl o.sl 0.7~" 0.6 0.5 4 0 .31 0. 0.2 0.1
IR=lo IR=20 IR=30
---"'.
IR=40
L-----~~1------~~10 ~----~~10 0
FREQUE NC Y rad s
Fig.6. IR Value vs. Coherency of Controller Oynalics
% % % %
282
S. Sugiyama, N. Yuhara and S. Horiuchi
the linearity of the human operator declines regu-
laly. This means that the method used in this paper, i.e., the use of a subsidialy task to make information on the main task intermittent, is valid and reasonable. Figs. 7 through 10 show failure detection time at varying IRs in the type of system failure which invol ves sudden change in natural frequency W" , while Fig. 11 and Fig. 12 describe the relationship between failure detection time and IRs in another type of system failure that is associated with sudden change in gain K. As is apparent from these figure, in either type of system failure, both the monitor and controller requires longer mean detection time as the value of IR increase. This tendency is particularly notable with the controller. At IR=O, the controller achieves shorter detection time than the monitor, and this ==esponds well with the test findings given in Yuhara & Horiuchi (op.cit.). Two curves in the figure represent quadratic curves approximated by the least square method for the mean values of detection time required by the monitor and controller, respectively. The crossing of these two curves in the renge of IR=10-20% indicates that the controller and monitor are reversed in detection time at the intersection; in other words, the =ntroller requires longer detection time than the moni tor in an IR range above this point.
10 9 8
~MONITOR
.J CONTROL.LER m.MON I TOR (Yuh,r"Hori uchi OP. ci t. ) j CONTROLLER (Yuhor.&Horiuchi o•. cit.)
7
6
I STD.
5
DEV.
4
1 0L---0~--~1-0--~2-0---3~0---4~0--~5~0--~6~0--~7~0--INTERVAL RATIO"
Fig.IO. Detect i on T i.e w 2 X 0.3 Fa i lure ¥~~~Cn2N 10 9 8 7 6 5
iJiMONITOR
t CONTROLLER lIi MONI TOR (YuhonlHoriuchi OP. ci t.) j CONTROI..LER (Vuh ...lKo.riu
I STD.
DEV.
~ ""'+oli,.~:tf=T: l1
o
o
10
20
30
40
50
60
70
INTERVAL RATIO X DETECTION TIME sec
10
Fig.l!. Detection Ti.e KX2.0 Failure
11 MON I TQR
DETECTION TIME sec
JCON;rROL.LER
10 9 8
g, MONl TOR t CONTROI.,LER .m MONI TOR (Yuh,roiHoriuchi op.ci t.)
7
j CONTROLLER {YUho." .l Hor. iU.
6
I
5
/-
O'-·~(\·f. 1- •
: fll.c·+"~· :rltLft ·· INTERVAL RATIO X
Fig.1. Detection Ti.e w2X2.D Failure
1 0L---0~--~1-0--~2-0---3~0---4~O~-5~O~~6~O--~7~O~~ INTERVAL RATIO X
DETECTION TIME sec
10 9 8 7 6
5
Fig.12. Detection Ti.e K XO.5 Fai lure
~MONITOR J CONTROLLER mMONITOR (Yuh ..r"Horiuchi oP. c.it.) j CONTROLLER (Yuh.r.lHoriu
I STD.
DEV.
4
3
o
o
10
20
30
40
50
60
70
INTERVAL RATIO"
Fig.B. Detection Ti.e w 2 X3.D Fai lure DETECTION TIME sec
10 9 8 7
In order to clarify how the significance and type of system failure are related to IR which causes the reversion of the controller and monitor in detection time, the relationship between the significance of system failure ( the ratio of system parameters in normal working order to those during a failure) and IR which makes it unreasonable to reject the hypothesis that "the monitor requires longer detection time than the controller." was shown in Fig. 13. Whatever the significance or type of system failure may be, the lengths of detection time by the =ntroller and monitor are reversed significantly in an IR range of 30%-40%. 'This implies that the IRs causing such reversion is not appreciably affected by the significance or type of the system failure.
iJiMONITOR
Interval Ratio[XJ
J CONTROLLER lIiMONITOR (Yuh ..r.&Horiuchi .o p.
6
j CONTROLLER (Yuh.r~lHoriu
:
I STD.
DEV.
.
f::::.
0
~t
0
", , . . ,.,.-,--.:,,.,...
~ t! \='.fr··tt#F-+l
0L---0~~1-0--~2-0--3~0---4~0~~5~0~~6~0~~7~0~ INTERVAL RATIO X
fig.9. Detection Ti.e w 2 XO.5 Failure
"1 30
0 w Failure f::::. K Failure Of::::.
0
:r:
O.S 1.0 2.0 3.0 w/w" Normalized Significance K/K" of Failure Fig. 13. IR Value Causing Reversion (Sicnificance Level O.OS)
0.3
Detectioll of DYllamic S\stem Failllre
'The foregoing experimental findings are summarized in Fig. 14. 'The abscissa in this figure shows the
DTm
DTc
DTc"""
'i5Tm'<.:.:.:.-. ",t
o
t 2 t R /1 R(no.;n.l)
numerically , the interva 1 ratio (IR) was defined and an attempt was made to examine the relationship between the value of IR and detection time. The experimental findings may be summa riz ed as follows:
DT c : De\e~tion Ti.e (Controller) "DTm :Deteition lift (ftoni1.or)
.,2 ;( 0. 3
283
x6. ..
. "1". $' . KX2.0 ... 2 )(2. 0
J(x O. 5 3
4
5
Fig.14. Norlal ized Detection Tile normalized IRs used the each of IR of intersect i on of two curves in Figs. 8 through 13 as a fiducial point, while the ordinate represents the ratio of monitor/controller detection time . The figure indicates that whatever the significance or type of system failure may be, the ratio of monitor /control l er detection time shows the same tendency. The cause of the reversion of the monitor and controller in detection time may be explained as follows : While the controller is disadvantaged in the performance of failure detection by heigher work load given him, compared with the monitor , because he has to do con trol task in addi tion to moni toring, he has an advantage in that propriocepti ve information from the operat i on of t he control manipulator as well as visua l information from the display unit can be used to detect system failure. In other words, the con troller has, in addition to visual information, a second channel of information which concerns proprioception associated with control manipulation. Since it may be assumed that the operator's control strategy does not change until a system failure is detected, the effects of any change in the plant dynam ics due to a system failure are considered to appear in the motion of the manipulator. This implies that control manipu l ation indirectly reflects the occurrence of a system failure. According l y better information is accessible to the controller than to the monitor in detecting system failure. In a low IR range , the favorable effects of the avai la bility of proprioceptive information are greater tha n the adverse effects of intermittent visual informati on upon failure detection. As the values of IR increases , visual information is used less effectivel y because the controller's attention is focused on controlling the system with intermittent visual information. As a resul t, he needs many more sampl i ng data to detect system failure , and consequently , he requires longer detection time than the monitor. CONCLUSION This paper studies the relationship between the participatory mooe of human operator and failure detection t ime . Assuming that the fa ctor affecting the difference between the moni t or and controller in detection time was the intermittency of visua l information (which corresponds to the number of the axes to be controlled or the instruments to be monitored) taken in by the human operator , we tried to verify this hypothesis by experiments . In these experiments , vi sual information on main task was made intermittent by giving the subject a subsidiary task of press i ng push buttons. To present the intermittency of visual information
(1) As the value of IR increases, both the monitor and controller require longer detection time. This tendency is particularly notable with the controller. (2) At IR=O% , the controller requires shorter detection time than the monitor , and this corresponds well with the experimental r esults gi ven in Young(op.ci t. ) and Yuhara & Horiuchi (op . cit. ) • (3 ) Regardless of the parameters subj ect to a failure or the significance of the failure, the controller and moitor were reversed in the length of detection time in the range of IR=30-40%; in other words, the controller required longer detection time than the monitor in this and higher ranges of lIt Whatever the parameters involved in, or the significance of a system failure may be, such reversion of detection time had virtually a fixed tendency. This finding indicates that the inter mittency rate of v i sual information taken in by the human operator is a major factor for the reversion of the controller and monitor in detection time. REFERENCE
Curry , R. E. and Ephrath , A. R. (1 976 ). Monitoring and Control of Unreliable System, in " Moni toring Behavior and Supervisory Control", Sheridan, T. ~ and Johanssen, ~ ed., Plenum Press Ephrath, A. R. and Curry, R. c. (1977 ). Detection by Pilot of System Failure Duri ng Instruments Landing, IEEE Trans. on Systems Man and Cybernetics , SMC-7, 841 -848 Ephrath , A. R. and Young , L. R. (1981). Monitoring vs. Man-in-the-Loop Detecti on of Aircraft Contr ol Failure, in ''Human I:etection and Diagnosis of System Fail~Rusmussen , ~ and Rouse, ~ ~ ed ., Plenum Press Morizumi , N. , Goto, N. and Kimura , H. (1983 ). Experimental I nvestigation on I:etection of System Changes by Human Pi l ots, Kyudai Kougaku Shuuhou, Vo1.56 , No. 1, 71-77 , (in Japanese) Young, L. R. (1969 ). On Adaptive Manual Control, IEEE Trans. on Man- Machine Systems , MMS-1 0 , 292-331 Yuhara , N. and Horiuchi, S. (1986). Detect ion of System Failure by Human Operator (Part 1) The Case of Mon i tor, Journal of the Japan Society for Aeronautical and Space Sciences , vo1.34 , No. 387, (in Japanese) Yuhara, N. and Horiuchi , S. (1986). Detection of System Failure by Human Operator (Part 2) The Case of Human Controller, Jurnal of the Japan Society for Aeronautical and Space Sciences , Vo1.34 , No. 388 , (in Japanese) Wickens , C. D. and Kessel, C. (1 979 ). The Effects of Participatory Mode and Task Workload on The I:etection of Dynamic System Failure , IEEE Trans. on Systems , Man and Cybernetics , - SMC-9, 24-34 Wickens;-C. D. and Kessel, C. (1981). Failure Detection in Dynamic Systems , in "Human Detection and Diagnosis of System Fai~ Rusmussen , J. and Rouse , ~ ~ ed ., Plenum Press
THE EFFECTS OF PARTICIPATORY MODE ON THE DETECTION OF DYNAMIC SYSTEM FAILURE S. Sugiyama*, N. Yuhara** and S. Horiuchi** *IBM japan. Engineering SYSIf11LI, Tokyo, japan ""College of SCifllff ant! Technology, Xi/lOll L'lIi,'ersily. Tokyo, japan
ABSTRACT While some papers argue that the controller requires shorter detection time to detect system failures in a single-loop task, some others contend that in a multi-loop task, the monitor achieves shorter detection time. The present paper discusses a series of experi ments on the validity of the assumption that "such findings have come from the intermittency in which the human operator recieves visual information" (this intermittency of information corresponds to the numbers of control axes and instruments). The experiments included a subsidiary task to press the push button for making visual information intermittent. The interval ratio (IR), the ratio of the time for watching the subsidiary task to the total time period of each trial, was defined as an index for the intermittency of visual information. More specifically, IR;O% corresponds to an instance where a single-loop task with one instrument is being performed, and an increase in this ratio means an increase in the number of control axes or monitoring instruments. The experiments found that whatever the Significance or type of system failure might be, the controller achieved shorter detection time than the monitor at IR=O%, while at IR;30% to 40% or higher, the controller needed longer detection time. This finding indicates that the intermittency of visual information recieved by the human operator is a major factor of the difference between the controller and the monitor in detection time. Keywords
Human Engineering; Failure Detection; Manual Control; Monitoring Behavior; Workload.
INTRODUCTION
To ensure safe operation of a system, it is important to not only improve the reliability of system components themselves but also detect a system failure quickly if it occurs . Broadly there are two ways of detecting system failures. One is performed by a human operator assigned to monitor or control the system, and the other is done by a failure detection system using a computer. This paper deals with the first way of failure detection which relies on human operators. Even in the latter way, human operators have to check whether or not the failure detection system is working normally, and in addition, they have great adaptive ability to cope with failures other than those included in the scenario of likely failure modes. Therefore, particular emphasis should be plased on studies on failure detection by human operators.
Fie,1. MONITOR F ~-~
~--
u+
FiC.2. CONTROLLER
The identification of the r elationship between the human operator's participatory modes and failure detection characteristics is an important consideration in studying the construction of a manmachine system.
As shown in Figs. 1 and 2, the detection of system failures by operators may be broadly divided into two categories by the mode of their participation in system operation. The instances under these categories are:
To examine the effects of participatory modes on
failure detection time,Young(1969),Morizumi etal. (1983) conducted a failur e detection experiment on a single- loop system. Their paper says that the controller achieves shorter detection time than the monitor. Yuhara and Horiuchi(1986), after making a failure detection experiment on singleloop system, with the first- and second-order plant dynamics, also report that the controller needs shorter detection time. Based on the findings of a dul-axis pursuit tracking task, Wickens and Kessel (1979) contend that the monitor is better than the controller in the accuracy of
(1) Where the operator, working outside the control loop , monitors the p l ant output and detects system failures, if any (the operator working as a monitor); (2) where the operator performs control manipulation in the control loop and at the same time monitors the plant output, thus detecting system failures, if any (the operator working as a controller ).
2i9
280
S. Sugi\'ama, :'>J. Yuhara and S. Horiuchi
detection, while in detection time, the contrary is the case. With a fixed-base flight simulator, meanwhile, Ephrath and Curry(1977) conducted a multi-loop task simulating a large transport aircraft in the landing-approach flight envelope. They report that the monitor achieves shorter detection time than the controller. After studying why their respective experiments had shown conflicting results, Young and Ephrath (1981) found that detection time was affected by a number of factors including: 1. Participatory mode of the human operator; 2. Work load level; 3. Number of the instruments to be monitored; ard 4. Number of the axes to be controlled. They maintain that the monitor can probably achieve shorter detection time than the controller if high work load is involved in their task which, like flying an aircraft, requires the oprerator to perform multi-loop control while watChing multiple instruments. However this comment is based on the interpretation of the experimental results of Young(op.cit.) and Ephrath(op . cit .) in such a manner that they do not conflict with each other. Another notable point is that no paper has been published on the measurement of work load in Young's experiment(op.cit.) or on a systematic experiments to verify the conclusion noted above. With this in mind, we made an empirical study on why the findings of Young(op.cit.), Morizumi (op.cit.), Yuhara & Horiuchi(op.cit.) and Wickens & kessel(op.cit.) differed from those of Ephrath & Curry(op.cit.). The study was based on the working hypothesis described below: The experiments in Young(op.cit.), Morizumi et al. (op.cit.) and Yuhara & Horiuchi(op.cit.) dealt with the monitoring an instrument or control task in a single-input/sing le-output system. Accordingly the human operator continuously took in visual information from the instrument. In the experiment by Ephrath & Curry(op.cit.) which simulated the landing approach of an aircraft , on the other hand, the human operator scanned a good many instruments, including an attitude indicator, an altitude indicator and lIS indicator, and took in many information from these indicators. Therefore visual information from each of these indicators was intermittently recieved by the operator. We assume that this intermittency of visual information led to the discrepancy between the findings in Young (op.cit.), Morizumi et al. (op.cit.) and those in Ephrath & Curry(op.cit.). The intermittency rate of visual information indirectly reflects the factors in 3 and 4 above, i.e., the numbers of the instruments to be monitored and the axes to be controlled ; an increase in intermittency rate means an increase in the number of the axes to be controlled or the instruments to be monitored. The higher the rate of intermittency is, the more difficult it is to estimate the system characteristics as described by the original continuous information from the intermittent time-series data taken in by the operator. Presumably this leads to longer detection time. EXPERIMENTS Interval ratio IR, used as the intermittency rate of visual information from given instrument X, can be defined as follows: IR=
[I. Cu.lative Dwell Ti.e on X] XlOO[%] Trial Ti.e
(1)
Here IR=O% implies in a single-loop task with an instrument. An increase in IR value means an in-
crease in the number of the instruments to be moni tored or the axes to be controlled. This series of experiments used an instrument (a cathode ray tube) that could display information continually. To change this information into intermittent one which would be recieved by the human operator at varying intermittency rates, a subsidiary task was given him to forcedly move the fixation point of his gaze from time to time. It was a modified version of the subsidiary task used by Ephrath & Curry(op.cit.)(See Fig. 3). The device used for this task in our experiments had lights A and B installed one directly above the other, either of which would go on at any moment accompanied by a buzzer to attract the subject's attention. When hearing a buzzer, he was to find which light was on, and within a second from the moment the light went on, he was to press either of keys A and B which were provided for light A and B, respectively. Pressing the appropriate key would turn off both the light and the buzzer. The subject was also instructed that whenever hearing a buzzer for the subsidiary task, he should perform it immediately, temporarily suspending the failure detction task (hereinafter called the "main task"), if necessary, so that the task allocation between the main and subsidiary tasks would always be kept at the preset value. Therefore, if he pressed the wrong key or failed to press the right key within a second from the moment the light went on, the test was terminated at once and started anew. In tha t case , IR can be rewr i t ten as: IR
Cu.lative Dwell Ti.e on Subtask XlOO[%] Trial Ti.e
(2)
1/8 inch LED A
2 inch B
Fig .3. Subsidiary Task The values of IR could be set at any desirable value by changing the number of times the light was turned on in each trial (which lasted 75 seconds). This frequency of light switch-on to give the specified IR value was determined from a preliminary experiment . First, the subject ' s fixation time at given frequency of light switchon during the main task was measured by a nistamograph, and the period thus determined was given as the fixation time for the subsidiary task. During the preliminary experiment, the system was operated free of any failure. Then the value of IR at such frequency of light switch-on was calculated by equation (2). A calibration curve as shown in Fig.4 for the relationship between the fequency of light switch-on and IRs was obtained by performing several rounds of similar test at varying frquen cies of light switch-on. From this figure, the frequency of light switch-on which results in the desired IR value can be worked out. The figure also shows that for the monitor, the value of IR increases in proportion to the frequency of light switch-on up to IR=70% , while for the controller, IR value increases similarly up to IR=40%. Accordingly the subsidiary task used in this series of experiments can give the desired IR value whichever the paticipatory mode of the human operator may be. The with a subsidiary task given to the operator,
281
Detection of Dynamic System Failure
a failure detection task similar to the one described in Yuhara & Horiuchi(op.cit.) was conducted both in the monitor and =ntroller modes of participation at the specified IR values (10%, 20%, 30%, 40%, 50%, 60% and 70% for the monitor and 10%, 20%, 30% and 40% for the controller). The essential task of the human operator was to detect failure of the linear dynamic system as soon as possible. The normal operating state of the plant used in the experiments is as follows: Wn 2
K s2+2I:wns+wn 2
G(s)= K=\.O,
wn=4.0
(3)
ISTD. DEV.
EXPERIMENTAL RESULTS
~ MONITOR
~~
~
~
~~~ ~
17 33 50 66 83 No. of times the licht was turned on in each trial(75 sec). FiC.4. IR vs. Frequency of licht switch-on 0
The system failure under the experiments scenario was associated with step change in the gain or natural frequency of the plant dynamics. The gain and natural frequency of the plant after the occurrence of a failure are: W
Fai lure Fai I
n=4.0 n=4.0
wn=5.7 wn=6.9
wn=4.0 wn=4.0
wn=2.8 wn=2.2
W W
Movinc Line
Fixed Line
(rad/s)
4 CONTROLLER
Nor.al
Error
....... ··l· .. ·.. ·· ..
Fig.5. Display Format
INTERVAl.. RATIO
70 60 50 40 30 20 10 0
1:=0.7,
experiment. The mean value and variance of detection time were calculated using data from 10 times of correct detection in the 21st and subseqent trials. A well-trained subject was selected for the experiment.
Nor.al
Fai lure Fai I
K=1.0 K=\'O
K=2.0 K=0.5
K
Table.l. Failure Types i Significance The plant dynamics represented by equation (3) was simulated by a digital =mputer and its output was given as vertical displacement of a horizontal line moving on the CRT display as shown in Fig. 5. The forcing function was worked out by sharping a gaussian white noise by a low-pass filter with a cutoff frequency of 100Hz. The subject acting as the controller was instructed to monitor information from the instrument while controlling the system by a spring-loaded manipulator in such a manner as to minimize position error. When deciding that a system failure was detected, he was to press the push button. Meanwhile the monitor was told to perform only the detection task, and when deciding that a failure was detected, he should press the push button. The CRT display gain was set so that the horizontal line indicating error moves within ±2cm during the normal operating state of the plant. A system failure was made to occur at any moment between 10 seconds and 60 seconds after the beginning of each trial. If the failure mode continued for 15 seconds or more after the occurrance of a system failure, i.e., if the subject failed to detect the failure within this time period, the trial was =nsidered a "miss alarm." If the push button was pressed before any system failure occurred, on the contrary, the trial was considered a "false alarm." The subj ects went through some 20 warm-up trials for a given system failure to get familiarlized with the
To evaluate the subsidialy task method described above ,the coherency of the controller dynamics during a control task was calculated by a method based on the fitting of an autoregressive ( AR ) model. In this method, the controller's output c ( a displacement of the control manipurator) and error e on the display screen were given as vector X, and the following two-dimensional AR model was fitted by Whittle's algorism to time-series data on c and e obtained from a manual control experiment: p
X(k)= L: A(i)X(k-i)+£(k) i=1
(4)
Where p represents the order of the AR model, A is an 2*2 AR coefficient matrix, and S is a twodimensional white gaussian noise, the covariance of which is given by: E[£(k)£T(I)]=RSkl
(5)
If the AR model is well-fitted to the time series of test data c and of e, R becomes a diagonal matrix. In that case, spectrum matrix S(jw) representing the power spectra of c and e and the cross spectra between them can be expressed by: -I
S(jw)=t.lA(jw») R [r(jw~
-I
(6)
Where A(jw)=
P ' . 2).e
(7)
i=o
11 stands for sampling time and * for a complex conjugate. From this, coherency r2(jwJ representing the linearity of e and c can be determined by:
And
2
0" (w)=
(8)
Fig. 6 shows the relationship between IR and the coherency of the controller dynamics calculated from the experimental data by the method described above. The figure indicates that as IR increases, 1.0 r
- - IR= 0 %
o·gl o.sl 0.7~" 0.6 0.5 4 0 .31 0. 0.2 0.1
IR=lo IR=20 IR=30
---"'.
IR=40
L-----~~1------~~10 ~----~~10 0
FREQUE NC Y rad s
Fig.6. IR Value vs. Coherency of Controller Oynalics
% % % %
282
S. Sugiyama, N. Yuhara and S. Horiuchi
the linearity of the human operator declines regu-
laly. This means that the method used in this paper, i.e., the use of a subsidialy task to make information on the main task intermittent, is valid and reasonable. Figs. 7 through 10 show failure detection time at varying IRs in the type of system failure which invol ves sudden change in natural frequency W" , while Fig. 11 and Fig. 12 describe the relationship between failure detection time and IRs in another type of system failure that is associated with sudden change in gain K. As is apparent from these figure, in either type of system failure, both the monitor and controller requires longer mean detection time as the value of IR increase. This tendency is particularly notable with the controller. At IR=O, the controller achieves shorter detection time than the monitor, and this ==esponds well with the test findings given in Yuhara & Horiuchi (op.cit.). Two curves in the figure represent quadratic curves approximated by the least square method for the mean values of detection time required by the monitor and controller, respectively. The crossing of these two curves in the renge of IR=10-20% indicates that the controller and monitor are reversed in detection time at the intersection; in other words, the =ntroller requires longer detection time than the moni tor in an IR range above this point.
10 9 8
~MONITOR
.J CONTROL.LER m.MON I TOR (Yuh,r"Hori uchi OP. ci t. ) j CONTROLLER (Yuhor.&Horiuchi o•. cit.)
7
6
I STD.
5
DEV.
4
1 0L---0~--~1-0--~2-0---3~0---4~0--~5~0--~6~0--~7~0--INTERVAL RATIO"
Fig.IO. Detect i on T i.e w 2 X 0.3 Fa i lure ¥~~~Cn2N 10 9 8 7 6 5
iJiMONITOR
t CONTROLLER lIi MONI TOR (YuhonlHoriuchi OP. ci t.) j CONTROI..LER (Vuh ...lKo.riu
I STD.
DEV.
~ ""'+oli,.~:tf=T: l1
o
o
10
20
30
40
50
60
70
INTERVAL RATIO X DETECTION TIME sec
10
Fig.l!. Detection Ti.e KX2.0 Failure
11 MON I TQR
DETECTION TIME sec
JCON;rROL.LER
10 9 8
g, MONl TOR t CONTROI.,LER .m MONI TOR (Yuh,roiHoriuchi op.ci t.)
7
j CONTROLLER {YUho." .l Hor. iU.
6
I
5
/-
O'-·~(\·f. 1- •
: fll.c·+"~· :rltLft ·· INTERVAL RATIO X
Fig.1. Detection Ti.e w2X2.D Failure
1 0L---0~--~1-0--~2-0---3~0---4~O~-5~O~~6~O--~7~O~~ INTERVAL RATIO X
DETECTION TIME sec
10 9 8 7 6
5
Fig.12. Detection Ti.e K XO.5 Fai lure
~MONITOR J CONTROLLER mMONITOR (Yuh ..r"Horiuchi oP. c.it.) j CONTROLLER (Yuh.r.lHoriu
I STD.
DEV.
4
3
o
o
10
20
30
40
50
60
70
INTERVAL RATIO"
Fig.B. Detection Ti.e w 2 X3.D Fai lure DETECTION TIME sec
10 9 8 7
In order to clarify how the significance and type of system failure are related to IR which causes the reversion of the controller and monitor in detection time, the relationship between the significance of system failure ( the ratio of system parameters in normal working order to those during a failure) and IR which makes it unreasonable to reject the hypothesis that "the monitor requires longer detection time than the controller." was shown in Fig. 13. Whatever the significance or type of system failure may be, the lengths of detection time by the =ntroller and monitor are reversed significantly in an IR range of 30%-40%. 'This implies that the IRs causing such reversion is not appreciably affected by the significance or type of the system failure.
iJiMONITOR
Interval Ratio[XJ
J CONTROLLER lIiMONITOR (Yuh ..r.&Horiuchi .o p.
6
j CONTROLLER (Yuh.r~lHoriu
:
I STD.
DEV.
.
f::::.
0
~t
0
", , . . ,.,.-,--.:,,.,...
~ t! \='.fr··tt#F-+l
0L---0~~1-0--~2-0--3~0---4~0~~5~0~~6~0~~7~0~ INTERVAL RATIO X
fig.9. Detection Ti.e w 2 XO.5 Failure
"1 30
0 w Failure f::::. K Failure Of::::.
0
:r:
O.S 1.0 2.0 3.0 w/w" Normalized Significance K/K" of Failure Fig. 13. IR Value Causing Reversion (Sicnificance Level O.OS)
0.3
Detectioll of DYllamic S\stem Failllre
'The foregoing experimental findings are summarized in Fig. 14. 'The abscissa in this figure shows the
DTm
DTc
DTc"""
'i5Tm'<.:.:.:.-. ",t
o
t 2 t R /1 R(no.;n.l)
numerically , the interva 1 ratio (IR) was defined and an attempt was made to examine the relationship between the value of IR and detection time. The experimental findings may be summa riz ed as follows:
DT c : De\e~tion Ti.e (Controller) "DTm :Deteition lift (ftoni1.or)
.,2 ;( 0. 3
283
x6. ..
. "1". $' . KX2.0 ... 2 )(2. 0
J(x O. 5 3
4
5
Fig.14. Norlal ized Detection Tile normalized IRs used the each of IR of intersect i on of two curves in Figs. 8 through 13 as a fiducial point, while the ordinate represents the ratio of monitor/controller detection time . The figure indicates that whatever the significance or type of system failure may be, the ratio of monitor /control l er detection time shows the same tendency. The cause of the reversion of the monitor and controller in detection time may be explained as follows : While the controller is disadvantaged in the performance of failure detection by heigher work load given him, compared with the monitor , because he has to do con trol task in addi tion to moni toring, he has an advantage in that propriocepti ve information from the operat i on of t he control manipulator as well as visua l information from the display unit can be used to detect system failure. In other words, the con troller has, in addition to visual information, a second channel of information which concerns proprioception associated with control manipulation. Since it may be assumed that the operator's control strategy does not change until a system failure is detected, the effects of any change in the plant dynam ics due to a system failure are considered to appear in the motion of the manipulator. This implies that control manipu l ation indirectly reflects the occurrence of a system failure. According l y better information is accessible to the controller than to the monitor in detecting system failure. In a low IR range , the favorable effects of the avai la bility of proprioceptive information are greater tha n the adverse effects of intermittent visual informati on upon failure detection. As the values of IR increases , visual information is used less effectivel y because the controller's attention is focused on controlling the system with intermittent visual information. As a resul t, he needs many more sampl i ng data to detect system failure , and consequently , he requires longer detection time than the monitor. CONCLUSION This paper studies the relationship between the participatory mooe of human operator and failure detection t ime . Assuming that the fa ctor affecting the difference between the moni t or and controller in detection time was the intermittency of visua l information (which corresponds to the number of the axes to be controlled or the instruments to be monitored) taken in by the human operator , we tried to verify this hypothesis by experiments . In these experiments , vi sual information on main task was made intermittent by giving the subject a subsidiary task of press i ng push buttons. To present the intermittency of visual information
(1) As the value of IR increases, both the monitor and controller require longer detection time. This tendency is particularly notable with the controller. (2) At IR=O% , the controller requires shorter detection time than the monitor , and this corresponds well with the experimental r esults gi ven in Young(op.ci t. ) and Yuhara & Horiuchi (op . cit. ) • (3 ) Regardless of the parameters subj ect to a failure or the significance of the failure, the controller and moitor were reversed in the length of detection time in the range of IR=30-40%; in other words, the controller required longer detection time than the monitor in this and higher ranges of lIt Whatever the parameters involved in, or the significance of a system failure may be, such reversion of detection time had virtually a fixed tendency. This finding indicates that the inter mittency rate of v i sual information taken in by the human operator is a major factor for the reversion of the controller and monitor in detection time. REFERENCE
Curry , R. E. and Ephrath , A. R. (1 976 ). Monitoring and Control of Unreliable System, in " Moni toring Behavior and Supervisory Control", Sheridan, T. ~ and Johanssen, ~ ed., Plenum Press Ephrath, A. R. and Curry, R. c. (1977 ). Detection by Pilot of System Failure Duri ng Instruments Landing, IEEE Trans. on Systems Man and Cybernetics , SMC-7, 841 -848 Ephrath , A. R. and Young , L. R. (1981). Monitoring vs. Man-in-the-Loop Detecti on of Aircraft Contr ol Failure, in ''Human I:etection and Diagnosis of System Fail~Rusmussen , ~ and Rouse, ~ ~ ed ., Plenum Press Morizumi , N. , Goto, N. and Kimura , H. (1983 ). Experimental I nvestigation on I:etection of System Changes by Human Pi l ots, Kyudai Kougaku Shuuhou, Vo1.56 , No. 1, 71-77 , (in Japanese) Young, L. R. (1969 ). On Adaptive Manual Control, IEEE Trans. on Man- Machine Systems , MMS-1 0 , 292-331 Yuhara , N. and Horiuchi, S. (1986). Detect ion of System Failure by Human Operator (Part 1) The Case of Mon i tor, Journal of the Japan Society for Aeronautical and Space Sciences , vo1.34 , No. 387, (in Japanese) Yuhara, N. and Horiuchi , S. (1986). Detection of System Failure by Human Operator (Part 2) The Case of Human Controller, Jurnal of the Japan Society for Aeronautical and Space Sciences , Vo1.34 , No. 388 , (in Japanese) Wickens , C. D. and Kessel, C. (1 979 ). The Effects of Participatory Mode and Task Workload on The I:etection of Dynamic System Failure , IEEE Trans. on Systems , Man and Cybernetics , - SMC-9, 24-34 Wickens;-C. D. and Kessel, C. (1981). Failure Detection in Dynamic Systems , in "Human Detection and Diagnosis of System Fai~ Rusmussen , J. and Rouse , ~ ~ ed ., Plenum Press