Human display monitoring and failure detection: Control theoretic models and experiments

0005-1098/83 $3.00 + 0.00 Pergamon Press Ltd. © 1983 International Federation of Automatic Control

Automatica, Vol. 19, No. 6, pp. 711 718, 1983 Printed in Great Britain.

Brief Paper

Human Display Monitoring and Failure Detection: Control Theoretic Models and Experiments* WILLI

STEIN t and PAUL

H. W E W E R I N K E : ~

Key Words--Man-machine systems; human factors; vehicle control; process control; human operator models; display monitoring; decision making; failure detection.

Abstract--Two newly developed control theoretic models for human display monitoring and decision making are presented that use the information structure of the well-known optimal control model of human response. Experimental paradigms with six dominant task variables (i.e. number of displayed processes, bandwidths, event probabilities, field of view, process couplings, and failure couplings), deduced from vehicle and process control situations, are the basis for extensive validation studies including eye-movement recordings. The broad coverage of the paradigms and the high degree of data/model correspondence provide the predictive potential for the analysis, design, and evaluation of man-machine systems. The relation of these models to existing prediction schemes is outlined.

Simon, 1981). An elementary classification of design methods results from identifying conventional, analytical, and modelbased approaches. Conventional approaches are supposed to be dominated by heuristics (Spillers, 1977), so that decomposing a problem (Himmelblau, 1973; Naylor, 1981) and synthesizing for a solution are primarily based on an heuristic taxonomy; in the context of man-machine systems, a taxonomy of tasks or human performance is required (Meister, 1976; Rouse, 1981; Companion and Corso, 1982). Employing model-based approaches has theoretical and practical advantages that stem, e.g. from the coherent incorporation of analytical elements. Design approaches based on human operator models originate in the systematic display design procedure for manual control situations (Clement, Jex and Graham, 1968; Clement, McRuer and Klein, 1972). This early approach utilizes frequency domain models of the human operator in combination with a rationale of visual scanning and an empirical metric of workload. Several design approaches with a more elaborated formal structure that can be applied to various aspects of manual control situations are based on the optimal control model (OCM) of haman response (Curry, Kleinman and Hoffman, 1977; Baron and Levison, 1977; Hess, 1977, 1981; Schmidt, 1979; Johannsen and Govindaraj, 1980). The advantages of the OCM-approaches stem from (1) the framework composed of modules for separate human functions (e.g. visual perception, central processing, motor response), (2) the flexible information structure suited for multivariable, multiple process and/or multitask situations, (3) the unique performance/workload or performance/attention metric, (4) the comparably high level of model validation, and (5) the underlying, normative modeling perspective (Baron and Levison, 1980). The OCM-approaches are highly developed and seem to be very attractive, but they are restricted to manual control situations. This paper aims at similar design approaches for the field of monitoring and decision making. Figure 1 represents monitoring and decision-making situations given in the fields of vehicle control and process control. The machine system illustrated incorporates plant dynamics as well as automatic control systems and may involve any given number of displayed processes. A task analysis of actual situations yields various information and decision structures to be investigated in the context of monitoring and decision making. This paper focuses on tolerance-band monitoring (TBM) and failure detection (FD) that are considered to be components of a model-based taxonomy of human performance. TBM-tasks, although somewhat differing with respect to the response required, have been studied by Senders (1964); Carbonell (1966); Smallwood (1967) and served as a basis for related models. An OCM-extension for TBM-tasks has been published by Levison (1971) and Levison and Tanner (1971). Research on various types of FD-tasks has been summarized by Moray (1980, 1981 ) and Rouse (1983). A FD-model, based on the information structure of the optimal control model (OCM), has been developed and validated by Gai and Curry (1976). Models of monitoring and decision making, based on the OCMinformation structure and only partly validated, have been proposed by Kleinman and Curry (1977). The TBM- and FDmodels applied in this paper have been developed and extensively validated by Wewerinke (1976, 1977a, 1981a, b, 19831; the two

1. Introduction

THE INCREASEDuse of computers in the control of large dynamic systems such as industrial plants, ships, and aircraft is changing the human's role in supervisory control rather than replacing him, so that monitoring and decision making are inevitable tasks in modern man-machine systems. Although this trend is quite clear (Sheridan and Johannsen, 1976), it is still not particularly easy to determine appropriate allocations of tasks between humans and computers. The purpose of this paper is to provide an approach, based on models of display monitoring and decision making, that is useful for the analysis, design, and evaluation of man-machine systems. The development of systematic methods for the analysis, design, and evaluation of man-machine systems seems to be an extraordinarily long-term goal. As suspected earlier, modelbased methods are gaining utility in these applications (Obermayer, 1964; Pew and Baron, 1983). Human factors engineering might parallel other scientific fields in so far as analysis procedures are more elaborated than design procedures (Spillers, 1974, 1977). At the heart of any design or synthesis procedure is analysis--in the current context this is an analysis of human operator performance for a given set of conditions. Design-methodological aspects of man-machine systems (Knowles and co-workers, 1969; Meister, 1971, 1976; McCormick, 1976; Moraal and Kraiss, 1981; Rouse, 1981; Patzak, 1982) may be viewed as constituent parts of an arising and yet uncoherent design theory (Zwicky and Wilson, 1967; Spillers, 1974, 1977; Jones, 1980; Sata and Warman, 1981; * Received 10 December 1982; revised 8 April 1983; revised 28 June 1983. The original version of this paper was presented at the 1FACflFIP/IFORS/IEA Conference on Analysis, Design, and Evaluation of Man-Machine Systems which was held in BadenBaden, F.R.G. during September 1982. The published proceedings of this IFAC meeting may be ordered from Pergamon Press Ltd, Headington Hill Hall, Oxford OX3 0BW, U.K. This paper was recommended for publication in revised form by editor A. Sage. $ Research Institute for Human Engineering (FAT), Koenigstrasse 2, D-5307 Wachtberg-Werthhoven, Federal Republic of Germany. National Aerospace Laboratory NLR, Anthony Fokkerweg 2, 1059 CM Amsterdam, The Netherlands. 711

712

Brief Paper MACHINE SYSTEM r .................. i zyl(t)

|I

i

Wllt},=,

":

AI

t

r"]

I

o o o

~1

I I I ~

o

III~

o

I '°

I

HUMAN OPERATOR

B

•

'1,

_=

Zym(t) I

I

,

*m"~',

~

I I

Am

I

•

;

I

L . . . . . . . . . . . . . . . .

..J

wi(t) Input Noise (White Gaussian Process) Yi (t) Disl~tay Variable (Gauss-Mar kay Process) Zyi(t) Failure Variable ( Deterministic Process)

A i Display B

Keyboard

FIG. 1. Monitoring and decision making: experimental situation. related models are OCM-extensions, too, suited for multivariable situations regardless of the dynamic characteristics.

2. Tasks and models Tolerance-band monitoring (TBM) involves observing a stochastic process ydt) (see Figs 1 and 2) in respect to exceeding the explicitly indicated tolerance band [b~i,b=i] so that the information for the human operator can be represented by the indicator variable hi(t)

fH ° [H~

i f b , ~< ydH ~< b.i else.

(1)

The human has to duplicate the binary process hi(t) e {H/°, H) } by generating the binary process u(t) ~ {D°, D l } ; i.e. the intervals of exceeded tolerance bands, H~, and the intervals related to the depressed keyboard, D', have to be synchronized. In the case ofm display variables, the response u(t) = D 1 represents toleranceband exeeedings of one, or up to m, display variables ydt), i = 1,..., m. The TBM-performance metric involves the decision error Pe = Pra + P~,s and the error ratio Re = Pra/Pm,, where Pfa and Pros are defined as time fractions of the incorrect responses (H/OD 1) and (Hit DO), respectively. Hence, Pf, and Pros denote the probabilities of false alarm and missed tolerance-band exceedings. The binary variables hdt) and u(t) are alternating renewal processes constituting a class of point processes (Cox and Lewis, 1966), where the variables hdt) are generated by the Gauss-Markov processes ydt). Thus, performing a TBM-task can be characterized as generating a sequence of independent

binary decisions, where each individual decision can potentially be based on a single observation testing a pair of hypotheses (Sage and Melsa, 1971). Failure detection (FD) involves observing a stochastic process yi(t) for the potential occurrence of abnormal events, where an event is defined as a change in the statistics of the displayed process that may be composed of changes in mean, standard deviation, and dynamic properties. The FD-performance metric contains speed and accuracy data with related trade-offs. The detection time Td = (ta - tf) denotes the interval between failure detection td and failure occurrence t:. The accuracy of failure detection is described by the false alarm probability PF and the miss probability PM. The time functions zxi(t) superimposed on stationary stochastic processes xdt) are used to investigate the human response to changes in mean (see Figs 1 and 2), where we restrict the class of zxdt) to elementary deterministic time functions (e.g. step and ramp functions) here. Optimally detecting an event or failure with a given accuracy requires sequential observations, Le. the number ofsubseq uent observations used as input to a decision is not fixed, but greater than one (Sage and Melsa, 1971). Thus, detecting a failure by observing a stochastic process ydt) is a binary task that consists in testing a pair of hypotheses. FD-tasks with m displayed processes ydt), i = 1,...,m simultaneously admit m independent events or respectively composite events having redundant information. The following assumptions concerning human perception and central processing parallel those of the optimal control model (OCM) of human response (Kleinman, Baron and Levison, 1971). In TBM- and FD-tasks we assume that the human

~ i.

zxCt)

q-r] I Machine System

]

I

[

1

Human Operator FiG. 2. Model of display monitoring and decision making.

Brief Paper observes an automatically controlled dynamic system, which is driven by white Gaussian processes w(t) with covariance W £(t) = Ax(t) + Ew(t).

(2)

J

The system is assumed linear and time-invariant, with noise shaping states and automatic control system dynamics included in the system state description. The displayed variables y(t) involve combinations of the system states x(t) and, in case of failure detection, a superimposed deterministic time function y(t) = C x ( t ) + z r ( t )

(3)

zy(t) = Cz~(t)

(4)

where zAt ) represents a specifiable abnormal event or a system failure to be detected. We assume that if a quantity y~(t) is displayed explicitly to the human, he can extract its rate of change, f/(t). Thus, y(t) having 2m components contains both position information y/(t), i = 1..... m and rate information

.Pi(t) = ym+i(t),

i = 1,...,m

v~(O, (6)

where yp/(t) is the perceived information upon which further processing is based. Frequently, the relatively invariant time delay • ~ 0.2 s is negligible in observation tasks, so that the predictor part of the model may be dropped. The covariance V~ of white Gaussian noise vi(t) scales with variance ori2 of display variable yi(t) and is given by 2 0 Vii = ~oyiPri/fri

('7)

where Pro/is the reference value of the observation noise/signal ratio that relates to the level of full attention, f r / = 1. Hence

Py/ = Prol/fr/= Vi/(rta2rl)

(8)

is the effective noise/signal ratio associated with variable yi(t). The attention sharing hypothesis of the OCM (Baron and Levison, 1977) assumes that the fractions 0 <<.Jy/<~ 1, Ji.i =Jr/, applied to the variables y/(t), i = 1.... , m obey the constraint m

frl = fro ~< 1

Accounting for the thresholds associated with the perception of a displayed process y/(t) (Baron and Levison, 1977), the describing function gain N(') of a threshold element is inserted in (7), so that the reference ra.tio Pro~ is made a function of the appropriate standard deviation trr~ and the threshold value at,

(9)

i=1

where fro denotes the effective level of attention directed to the entire monitoring task. Here the attentional fractions are assumed to be bounded in position-rate pairs (J?/= Jy0. Losses due to visual scanning, for instance, reduce the effective level of attention to fro < 1. In many tasks the reference value of the observation noise/signal ratio, Pro, equals the baseline value Pro --- 0.01 = - 2 0 d B (Kleinman, Baron and Levison, 1971), if specific physical conditions (i.e. zero-mean of y/(t), high resolution displays with zero-reference indication) and idealized viewing conditions (i.e. foveal viewing, full attention directed to the displayed process yi(t), negligible threshold and saturation effects) are given. Some task dependent values of Proiother than - 2 0 dB have been found (Baron and Levison, 1980). Unlike the above assumption of bounded attentional fractions, some investigations indicate trade-offs within each pair of f~./and ]i.i. Therefore, the attention sharing hypothesis [see (9)] has been modified

pO/ = pyo/N2OTyl, ari)

(11)

N(ayi, ayl) = erfc (ayl/(ayi~/2)).

(12)

According to the describing function gain N(-), the effects of perceptual thresholds are not negligible for ari < 3ari. Position thresholds ay, and rate thresholds a~/have to be considered, since we assumed position and rate perception [see (5)]. Thus, the effects of ay/depend on standard deviation trr/of the displayed process y/(t); the effects of a~/depend on standard deviation trr/ and bandwidth tool of y/(t), since tr:~, ~ oojtry/. The threshold values used in this study are (Wewerinke, 1981a)

(5)

where yi(t) and 3)i(t) are uncorrelated. The perceptual submodel reflects inherent limitations of time delay z and observation noise

ypi(t) = yi(t - "c) + vi(t - ~), i = 1..... 2m

713

ari = 0.1 deg visual arc

(13)

a~/= 0.2 deg visual arc/s.

(14)

Concerning the perceptual processes of display monitoring tasks, fundamental differences between TBM- and FD-tasks have to be stated. The perceptual submodel of the OCM [see (6)-(8)] assumes that the human operator estimates displayed analogous quantities y/(t) with respect to given reference indications. Perception in FD-tasks can be represented by this kind of statistical estimation (Sage and Melsa, 1971), but perception in TBM-tasks consists of reading binary-valued quantities h/(t) (i.e. the conditions for reliably perceiving the uncorrupted information h/(t) are given). As a consequence for modeling TBMtasks, the reference value of the observation noise/signal ratio, pyo, depends on event probability P(H~)[P(H~) is the probability of y/(t) exceeding tolerance band [bu, buil] and is generally smaller than in FD-tasks. Based on the above baseline value Pro = -20d1~, an empirical approach to the reference ratio of TBM-tasks is given by

Pro = Pr0K(%, bm b,/)

where K(.) is approximated by (1 - P(H~)). Thus, Pr°i = - 2 3 dB is a typical value for TBM-tasks having P(H~)= 0.5. These findings are based on extensive TBM-investigations including eye movement recordings (Stein, 1981). Following the assumptions of the OCM, the perceived information yp(t) is processed by an optimal observer (i.e. a Kalman-filter cascaded with an optimal predictor) that generates a best estimate i(t) related to x(t) of the observed system; i.e. if the error covariance matrix X = E{ Ix(t) - •(t)] [x(t) - ~(t)] T}

Pyo = - 2 3 d B

(16)

then ~(t) minimizes Tr {X}, where the trace Tr {.} indicates the sum of the diagonal elements of Z. The optimal observer yields i(t) as well as Z, so that estimate ~(t) and other variables can be derived (the matrices A, E, and C, given by the experimental situation under consideration, are an essential part of the model). Consequently, the estimator/predictor portion of the model [a mathematical description is given by Kleinman, Baron and Levison (1971 )] generates all the information necessary for both, optimal control and optimal decision-making. Following Fig. 2, the TBM-version of the decision-making model (Wewerinke, 1977a) involves an optimal Bayesian decision rule (Sage and Melsa, 1971 ). Thus, generating the binary response process u(t) is considered under the assumption of maximizing the expected utility by single-observation decisions, p(Hll~x) D 1 if ~ ~> K~

u(t) = (fr, +J~,)~< 1,

(15)

P(H lax)

(17}

D O else

(10)

i=I

so that bounded attentional fractions, J~/=fr/, can be considered as a special case (Wewerinke, 1977b, 1981a). Thus, using (10) requires changing the values offr i and PrO/[see (7)] by a factor of 1/2 and 2, respectively.

where P(H~ax), i = 0,1 is the probability that hypothesis H i is true, given information atx = (in(t), Z) by the observer, K~ denotes the decision threshold Ku

=

Uo/U1

= (Uoo

-- Ulo)/(U1

! -

Uol )

(18)

714

Brief Paper

where Uij is the utility of responding u(0 = D i, if h(t) = H ~ is given. The adjustable parameters of the TBM-model include the reference ratio P°i, an attention index f~ for each of the displayed processes, and the decision threshold K,. The decision error P~ = Pf, + Pm~, reflecting aspects of human decision performance, can be matched by adjusting pyo and j~. The error ratio R~ = Pf,/P,~ is a function of K, and reflects aspects of human decision strategy primarily. Following Fig. 2, the FD-version of the decision-making model presented (Wewerinke, 1981a, 1983) involves a sequential decision rule based on a generalized likelihood ratio test (Sage and Melsa, 1971). Frequently, the predictor portion of the model can be dropped if human time delay r _~ 0.2 s is small compared with detection time To (e.g. z/Td < 0.05 in this study). Like the FD-model of Gai and Curry (1976), the innovations process of the optimal observer nlt) = Cx(t) + z~.(y) + v(t) - ~(t)

(19)

is used as input to the decision mechanism. The right-hand side of (19) can be described as difference between human perception [see (6)1 and human expectation. A distinguishing mark of the FD-model presented is based on the assumption that human short-term memory plays an essential role in FD-tasks. Thus, an explicit representation of short-term memory operations is realized by the moving average of the innovations process, t fi(t) =

I/L m

f

n(t)dt

(20)

t - T~m

where the value of the time constant employed, T,m = 4s, is supported by available data (Sheridan and Ferrell, 1974). Further details of the sequential decision mechanism are given by the FD-example below. In case of ramp failures zy(t), the expected detection time is given by - 2 In (PF)

E{T~} ~ E,{fir(t) N ~fi(t)l

(21)

T A B L E 1. E X P E R I M E N T A L FACTORS O F M O N I T O R I N G A N D DECISION MAKING

N.

Number of displayed processes

tDoi

Bandwidth of ith process

3

P(H:)

4

G.

Event probability of ith process Field of view

1

5

Ppij

Level of process couplings

6

pf,~

Level of failure couplings

Based on the factors of the experimental pardigms, questions concerning the analysis, design, and evaluation of man machine systems can be considered on different levels of specification: 1. Information level. 2. Display level. 3. Environmental and/or physical level. The information level is represented by the matrices A, E, C, and W of the dynamic system under consideration [see (2) and (3)] and relates to all factors of Table 1 excepting field of view G,. The display level primarily refers to observation matrix C, event probability P(H~), and level of failure couplings, pj~j. The attributes of the display devices applied (e.g., electromechanical versus electronic, level of scaling versus level of integration, monochromatic versus color) affect the perceptual thresholds a~.~ and the observtion noise v~(t) [see (11 ) and (7)]. The physical level of display monitoring has connexion with the workspace area of the human operator, the arrangement of displays and controls, and the field of view G~ that affects human attention sharing in a high degree. The performance/workload metric of the human operator models presented facilitates taking into account the factors and levels of the experimental paradigms. The experimental paradigms are deduced from vehicle and process control tasks based on the assumption that 12) and (3) can sufficiently be approximated by a set of coupled second-order systems )?i(t) + 2(~iO)oi~'i(t)+ ~o~,iyi(t) = wdt)x/(4(ie)/~i)

where E~{.} indicates the expected value with regard to time and the ensemble of processes. The innovations process n(t) is stationary zero-mean white Gaussian with covariance N during unfailed system operation. Any bias failure zy(t), e.g. a step or ramp function, results in a bias of n(t), whereas covariance N is not affected by bias failures. The FD-accuracy of the human operator is represented by false alarm probability PF and miss probability PM [e.g. PF = PM = 0.05 ; see numerator of (21 ) ] and depends on individual factors, instructions and training. Miss probabilities tend towards zero (e.g. PM = 0.0 in (21)], if reliably detecting failures is facilitated by sufficiently high intensities of the failure variables zy(t), e.g. ramp and other unrestrictedly increasing functions. Data and model consistently show a considerable speed/accuracy tradeoff (i.e. Td versus PF and PM) indicating the stochastic aspects of decision-making (Sheridan and Ferrell, 1974): decreasing error probability P~ or Pm involves increasing detection time Tn in a given task situation. Additionally, the FD-model presented has the advantage that (1) it can be applied to any multiple-process situation (i.e. tasks comprising N, displayed processes) and (2) the expected detection time [see (21) ] can be made a function of the attentional fractions fyi, i ~ 1. . . . . N, [i.e. n(t) a n d N are affected byf~i according to (7) and (19)]. Hence, allocating the attentional fractions f~i to the N, displayed processes can be optimized resulting in the minimal expected detection time T*. 3. Experimental paradigms Two coherent and partly overlapping experimental paradigms of human display monitoring (i.e. a TBM- and a FD-paradigm) have been studied and have served as a basis for validating human operator models (Stein, 1981; Stein and Wewerinke, 1983; Wewerinke, 1981a, 1983). As shown in Table l, the paradigms comprise factors extracted from vehicle and process control tasks that include, e.g. different levels of task complexity, slowly and fast responding systems, and various failure situations.

(22)

where normalizing with x/(4~e~o31) causes variances of wi(t) and y~(t) to have the same values. Assuming a steady state of the dynamic system, the covariances W, X, and ¥ of the variables w(t), x(t), and y(t) obey the linear variance equation AX + XA r + EWE ~ = 0

(23)

Y = CXC 1

(24)

where the diagonal elements of covariance matrix Y equal a~ denoting the variance of the displayed processes yi(tk i = 1,...,N~. Thus, the displayed processes of the dynamic system under consideration are modelled as stationary secondorder Gauss-Markov processes y~(t) that are characterized by process bandwidths cool,damping ratios ~i = ~ = 1/,/2, variances a~, and process couplings pp~j. Here

Pp~J = Y,J/x/( ~ l:jjt

I25)

is the correlation coefficient of the displayed processes y~(t) and )~(t), where Y,j, i,j = 1. . . . . N , denote the elements of covariance matrix Y. Considering displayed processes yi(t) having different spectral characteristics, the power spectral density functions of y~(t) are preferably transformed to equivalent rectangles (Bendat and Piersol, 1971 ). It should be noted that process bandwidth ~o~ is the natural frequency of a second-order system, the equivalent rectangular bandwidth of which is given by Bi = ~)0i~z`'.:(2)/4.

126)

In case of TBM-tasks, the event probability P(H]) = 1 - P(H°,) of a displayed process ydt) depends on probability density function p(yi(t)) and tolerance band lb,, bui] [see (1)]. The failure couplings Pfli of a FD-task are defined as ratio of the related bias

Brief Paper failures zyfft) and zyj(t) [see (3)]. Restricting to ramp failures, the failure couplings are given by Pfij = zyi(t)/zyj(t) = c,~/c,j

(27)

where c , and c,j denote the slopes of the ramp functions, presuming Ic,d ~< Ic,jl; otherwise PsJ~ is used instead ofpli j. Here, a rump function initiated at t = t / i s defined by

ri(t-ty)=

~0, if t < t/ /(t-t/)c,i, ift1>t/

715 0.5.

-SdB ]

P, 0./,'

- 14dB

0.3.

-20dB

Pyi

0.2¸ (28) 0.1.

Data and model consistently show that human FDperformance in case of ramp failures is governed by ratio cri/tryi, where c,~ denotes the slope of ramp ri(t - t/) and tryl denotes the standard deviation of the displayed process y~(t). Hence, the slope c,o = 0.1arl per second is used to define the standard ramp r o ( t - tl). Taking into account others than ramp failures, appropriate measures of coupling effects have to be formulated. Failure couplings in vehicle and process control result from a system failure affecting more than a single displayed variable. In any case, failure couplings comprise redundant information and might facilitate FD-tasks. 4. Results and interpretation Human performance in display monitoring is shown in Figs 3-8. Both empirical and model results presented are based on the factors of the experimental paradigms shown in Table 1. An extended description of TBM-investigations has been given by Stein (1981); FD-investigations have been reported in detail by Graaff ( 1981) and Wewerinke (1981a, b). An overview of empirical TBM-results is given in Fig. 3. The decision error Pe is shown as a function of process bandwidth tOoi, number of displayed processes N°, and field of view G=. According to Pe = Pf~ + Pm~, the decision error represents cumulated time fractions of false alarm (Pf=) and missed tolerance-band exceedance (Pm~). The N~ displayed processes yi(t) are uncorrelated and homogeneous; i.e. all displayed processes y~(t), i = 1. . . . . N= of a particular task have equal process bandwidths (O9o~ = 0902 . . . . ) and equal probabilities (P(H ~) = P(H~) . . . . ), where P(H 1) = 0.5 is the combined event probability of the N= processes. The tasks investigated involve N~ = 1, 2, or 4 processes in the case of the small field of view (5 x 5 degrees of visual arc), while No = 4* processes refers to the big field of view (34 × 34 degrees of visual arc). The P~-curves originate at zero, increase monotonically with process bandwidth COo~,and tend asymptotically to the maximum value P~ = P(H ~), where event probability P(H ~) is an independent variable of TBM-tasks. Like process bandwidth ~Ooi, both number of processes, N~, and field of view, G~, monotonically affect decision error Pc. The results of the TBM-model shown in Fig. 4 are in farranging correspondence with the data of Fig. 3. Both empirical and model results are based on the assumption that N= uncorrelated homogeneous processes with combined event 0.5 0.t. 0.3

Na 0.2 0.l

Q5

1.0

1.5

2~0 2.5 =o, [rod/s]

field of view: S by5 degrees(Na=/,~:3/, by3/, degrees)

FIG. 3. Empirical decision error Pe vs bandwidth moi.

0

0

os

i

i.5

~

Woi

zs

[rQdIs]

FIG. 4. Decision error P~ vs bandwidth o~°~.

probability P(H ~) = 0.5 are monitored. Human time delay is assumed to be constant, ~ = 0.2. Effects of perceptual thresholds [see (13) and (14)] are negligible within the TBM-paradigm investigated, as sensitivity studies have shown. Monitoring performance in terms of decision error P~ is primarily a function of observation noise/signal ratio Pyi that is given by Pyi = P°Jfri in accordance with the attention sharing hypothesis [see (8) and (9)]. The reference ratio P° i = - 2 3 d B is typical with event probability P(H 1) = 0.5 [see (15)]. The effects of factor N= on decision error Pe are represented by the model in such a way that the fractions of human attention, fr~, devoted to the displayed processes y~(t), i = 1. . . . . N°, obey the attentional constraint ~frl = fro ~< 1 i

[see (9)]. The effects of factor G= (i.e. size of field of view) are represented in terms of attentional losses, so that the effective level of attention decreases (i.e. fro < 1). Thus, monitoring a single process y~(t) with full attention (i.e. fri = 1) results in the equivalent observation noise ratio Py~ = - 23 dB. Monitoring N= homogeneous processes with uniformly allocated attention results in attentional fractions frl = 1/N°, i = 1. . . . . No; e.g. the equivalent observation noise ratio is given by Py~ = - 2 0 d B in the case of N~ = 2 processes and by Pyl = - 17 dB in the case of N= = 4 processes. According to Fig. 4, the relationship between decision error Pe and bandwidth co°i is intrinsically nonlinear, where the degree of nonlinearity increases with observation noise/signal ratio Pyl. A nearly linear Pe/~Ool relation is given, if a single process is monitored with full attention. The error ratio Re = Pt=/Pm, (not illustrated here) represents aspects of human decision strategy in TBM-tasks and is affected by the task difficulty (e.g. number of processes N= and field of view G=). Based on the assumptions of the TBM-model, error ratio Re is primarily a function of decision threshold K~ [see (17)]. Nearly equivalence of error components (i.e. Pea ~ Pros) is given in less demanding tasks, whereas Pms ~- 2Pt.a is a typical value of highly demanding tasks; Pro, responds with greater sensitivity than Pfa, if the factors N= or Ga increase. Attentional losses have to be taken into account explicitly in display monitoring tasks, if the visual scan traffic of the human operator exceeds certain levels. Visual scan traffic is affected by many factors and task characteristics. The number of displayed processes (N=) and the size of the field of view {G=) belong to the general factors of display monitoring tasks that are relevant to visual scan traffic; specific factors of TBM-tasks are the process bandwidth cool and the event probability P(H~). The dimensions ofattentional losses might be undervalued in general. Indications can be deduced from the TBM-results below. The effective level of attention decreases from fy0 = 1.0 to fro = 0.5 [see (9)] due to visual scanning required by an extended field of view, if N~ = 2 uncorrelated homogeneous processes yl(t) and y2(t) are monitored; the experimental situation involves the process bandwidth ~ool = O9o2= 1 rad s-1, the field of view G~ with 34 degrees of visual arc, and the combined event probability

716

Brief Paper

P(H ~) = 0.5. This TBM-example can be illustrated by Fig. 4, so that the observation noise/signal ratio Pr~ = - 17 dB refers to the situation with the extended field of view and Pyl = - 20 dB refers to the small field of view. The attentional losses can be explained in such a way that each saccadic movement of human eyes is accompanied by a phase of perceptual inhibition (Volkmann, Schick and Riggs, 1968). Given a field of view G, with 34 degrees of visual arc, a saccadic transition takes about 80ms and the corresponding perceptual inhibition might amount to 200ms. Figure 5 shows the empirical decision error P~ of TBM-tasks as a function of event probability P(H ~) and process bandwidth o9oi. The experimental situation involves N, = 4 uncorrelated homogeneous processes and the field of view G, having 5 × 5 degrees of visual arc. The maximum of decision error P~ is given at event probability P(H ~) =0.5. The aspects of symmetry included in the P~-curves are in relation with the binary nature of the TBM-task and can be interpreted in such a way that the human operator selects between two monitoring and decision making strategies, since H°/H~ as well as D°/D ~ are exclusive states of a task [see (1)]; i.e. (1) using H ° as a basic state and responding to H/~ (in the case ofP(H~ ) ~< P(H °) preferably) or (2) using H~ as a basic state and responding to H ° [in the case of P(H °) <~ P(H~) preferably ]. Aspects of vigilance and rare events (Sheridan and Ferrell, 1974) are beyond the scope of this study, although the relatively low event probability P(H ~) = 0.125 is included in the paradigm. It can be assumed that the well-trained and well-motivated human operator optimally allocates his attention to the N~ displayed processes of a TBM-task. Optimal allocation of attention has been shown both theoretically and experimentally in different categories of human operator tasks, e.g. in closedloop manual control tasks (Kleinman, 1976; Wewerinke, 1977b) and in failure detection tasks (Wewerinke, 1981a). Indication of attentional optimality in TBM-tasks are given by experiments involving N a inhomogeneous displayed processes yi(t), i = 1. . . . . N~; i.e. the processes of a particular task have different process bandwidths (~Oo~# ~ o 2 # - . , ) and different event probabilities [P(H~) -¢ P(H~) ~ ...]. The results show that the partial decision errors P~ = Pf~ + Pm~i [P~i represents the constituents of decision error P~ related to the displayed process y~(t)] are highly correlated with the product of the corresponding factors, ~oo~P(H ~). Like the partial decision errors P~i, the dwell fractions fdl [Jd~ represents the time fraction of fixating the displayed process yi(t) during a task] are highly correlated with the product of factors, ogoiP(H~), as corresponding eye-movement studies have shown. Hence, it is assumed that the optimally selected fractions of attention, J~*, i = 1,.., N,, allocated to the displayed processes y~(t), i = 1. . . . . N~, minimize the cost functional N~

J, = 1IN a ~ f(P~,/P(H~)).

(29)

Both data and model of TBM-situations consistently show that the effects of process couplings Ppu on decision error P, (not illustrated here) are relatively small. The effects might be negligible mostly, since Ppu > 0.5 is very unusual in actual TBM0.5

situations. Similarly small effects of process couplings Ppu are given in FD-situations (see Fig. 8). TBM-tasks involve subsequently performed independent binary decisions. Due to the reliably displayed uncorrupted binary information, any particular binary decision of a TBM-task can be performed on the basis of a single observation; i.e. the perceptual process of TBM-tasks does not involve either estimating the displayed variables y~(t) or grouping subsequent observations into samples. Hence, the baseline value of the observation noise/signal ratio comes to Py0 = - 2 3 dB in TBM-tasks. A characteristic aspect of tolerance-band monitoring is revealed by the fact that several dependent variables of TBM-tasks (e.g. decision error P,. and its components Pra and P,,~, attentional fractions J~,, and dwell fractions fnl of visual fixation) either scale or increase monotonically with process bandwidth ~ool.Thus, one can assume that the instruments are sampled to best reconstruct the displayed processes yi(t), i = 1..... N~ in TBM-situations. Human performance in failure detection tasks is shown by model results shown in the Figs 6 8. The FD-model presented, although appropriate for any type of bias failure, is applied to superimposed ramp failures zyi(t) = ri(t - t:) having slope c,~. The standard ramp r o ( t - t : ) , used in the majority of experiments, has the slope C,o = 0.1ay~, so that the bias failure equals the standard deviation of the displayed process yAt) at (t - t.r) = 10s, i.e. 10s after failure occurrence. In accordance with the experimental study, a fixed FD-accuracy of the model is assumed that includes the error components Pv = 0.05 {i.e. false alarm probability) and PM = 0.0 (i.e. miss probability). The model results considered are based on the assumption that the perceptual thresholds of the human operator are negligible [i.e. a~.i = ay~ = 0; see (13) and (14)]. The predictor portion of the model is dropped, since human time delay r is small compared with detection time Tn. Extensive FD-investigations have shown that (1) display monitoring with respect to failure detection can be regarded, unlike tolerance-band monitoring, as an estimation task, (2) the attention sharing hypothesis of the optimal control model [see (8) and (9)] can be applied, and (3) the baseline value of the observation noise/signal ratio, Pro = - 2 0 d B , is appropriate to FD-tasks. Attentional losses due to visual scanning are neglected, so that the constraint goes to Z/;,~ =I;,o : 1. i

Hence, the effective noise/signal ratio Pyi of FD-results below is given by Prl = P~i = Pyo/J'y~, wherej~i is the fraction of attention allocated to the displayed variable yi(t). Figure 6 shows failure detection time Tdas a function of process bandwidth ~ooi and observation noise/signal ratio Py~ = P~.o/./~, where the reference ratio is given by Pyo = - 20 dB. The Td-curve with indication Pyl = 20 dB relates to a single-process FD-task performed with full attention (i.e. J~i = 1). FD-performance is, unlike TBM-performance of Fig. 4, a non-monotonic function of process bandwidth cool.In a first approximation one can assume that detection time Tn is independent of process bandwidth coo~,as data and model consistently show. Due to neglecting perceptual threshold a~ and ar~, the model results of Fig. 6 might -tlde

-14dB

Ta [s] -17dB

0.4

Pyi -20dB L W~,=1.5

0.3 0.2

........

O.f

.'--

0

0.25

0.5

035

p(H)) 1.0

FIG. 5. Empirical decision error P~ vs event probability P(H~).

0

0.5

1.0

1.5

2.0

25

woi [tad Is] FIG. 6. Detection time Ta vs bandwidth t;Jo~.

Brief Paper underestimate the detection time Ta of actual F D situations. Detection time Td might be underestimated especially in the range of small bandwidths o9o+ (e.g. O)oi < I rad s - 1 in Fig. 6), since the standard deviation of rate information ~+(t) follows try+ ~ 09o+trri, where tooi and trr~ denote bandwidth and standard deviation of the displayed process y~(t). In general, the effects of position and rate thresholds, at+ and a~i are not negligible in the cases of try~ < 3ar~ andtooitTri < 3arl, respectively. The failure detection results of Fig. 7 represent monitoring a given number Na of uncorrelated displayed processes y~(t),i= 1. . . . . N , with equal process bandwidths, i.e. 09ol :=~o2 = - . . . The FD-situations considered involve process couplings pp~j = 0 as well as failure couplings PJv = 0, so that failures are represented by the ramp function zyi(t) = ro(t - tj.) superimposed on a single displayed process. Perceptual thresholds are neglected, i.e. a~.~= ay+ = 0. The level of full attention, Jr0 = 1, is uniformly allocated to the N, processes of a particular task, so that perceiving a single process is characterized by attentional fraction Jv~ = 1/N, and observation noise/signal ratio Prl = Pyo/Jvl. Consequently, the detection time Td monotonically increases with the number of displayed processes, N~. The effect of process bandwidth 09oi on detection time Td shown in Fig. 7 is reduced considerably, if perceptual thresholds ar~ and a,~ are included in the model. Figure 8 describes human failure detection time T~ as a function of failure couplings Pyv and process couplings pp~,/.The FD-situations considered involve the N~ = 2 correlated displayed processes y l ( t ) and y2(t) with process bandwidths O9ol = 09o2 = 0.25 rad s - 1, observation noise/signal ratios Pvl = P~.z = - 1 4 d B , and superimposed ramp functions zr l ( t ) = ro( t -- ts) and eyz( t ) = P sisro(t - ts). Thus, human failure detection performance is improved by redundancy due to both process couplings Ppv and failure couplings Pfij. These FDresults are consistently supported by both data and model. The effects of process couplings p~(~ on detection time Td might be negligible in actual FD-situations, since pp~j > 0.5 is very unusual; in TBM-tasks, the effects of process couplings Ppo on decision error P~ are similarly small. The failure detection tasks presented involve independent decisions based on sequential observations; i.e. making a FD-decision on the basis of noisy information with a specified level of accuracy in minimal time requires a sample of n subsequent observations; the number n is not fixed and depends, e.g. on the informational content of each observation. The perceptual process of FD-tasks involves estimating the analogously displayed variables y,.(t), i = 1. . . . . N~. Thus, like manual control tasks, the reference value of observation noise/signal ratio is given by Pro = - 20 dB. A characteristic aspect of the failure detection tasks investigated is revealed by the fact that several dependent variables (e.g. detection time Td, attentional fractions fyl, and dwell fractions fdl of visual fixation) are nearly independent of process bandwidth COo~.One can assume that the instruments are not sampled to best reconstruct the displayed processes y~(t),i = 1 , . . . , N , in FD-situations. As a first approximation, human attention in the FD-tasks above is uniformly allocated according tofr i = fro/N,, i = 1. . . . . N~. A more accurate representation of the detection process has to take into account that the 1.0

20

/

#/!

To [sl

/ /

0.25 ] J

//

/ ~

/P

~ "el

/

0.1

/ i I

0.5 /P

.,,¢" i IIJ

.~t t

"@

+.4.... ~

--

0.01

1C

717

r,:,t+l,+

0.+

12" 1110-

T o

o'.2

o'.4

o'.s

o'.a

;.o

P+ii

FIG. 8. Detection time Td vs failure couplings Pfij. occurrence of a failure implies an inherently unstationary situation affecting the visual scanning process, the attentional allocation and other dependent variables (Moray, 1981; Wewerinke, 1981a). The explanation of the nearly equal attention is given by the FD-model: varying the attentional allocation shows that the expected detection times Td as a function of the attentional fractionsfri, i = 1,..., Na have broad minima. 5. Conclusions

In this paper we presented two multivariable models for the field of human display monitoring and decision making that correspond with the tasks called tolerance-band monitoring (TBM) and failure detection (FD). The models are based on modern control, estimation, and decision theory; they consist of two stages (i.e. a linear estimator and a decision mechanism) and are similar in principle to the model of the human operator as an optimal controller (OCM). The models have been previously applied to realistic problems (Graaff, 1981 ; Wewerinke, 1981 a); in the laboratory-type tasks reported, the models are applied to experimental paradigms studying six factors that represent fairly general aspects of vehicle control and process control. The models presented provide a unified framework for the field of monitoring and decision making that integrates essential parts of previous investigations and models. The model based approach outlined yields several significant byproducts that relate to the analysis, design, and evaluation of man-machine systems and a better understanding of human operator behavior. First, the problems of human performance can be considered on different levels of specification, i.e. (1) the information, (2) the display, and (3) the environmental and physical level. Second, the performance/workload or performance/attention metric for monitoring and decision making is needed to compare human performance at different levels of automation adequately. Third, identifying the different types of information and decision structure (e.g. tolerance-band monitoring and failure detection) is required to decompose complex performance and to develop a model-based task taxonomy. We do not claim that the work on the models of toleranceband monitoring and failure detection is finished, although a considerable level of data/model correspondence has been obtained that involves broad ranges of several essential factors representing vehicle and process control situations. Several improvements and extensions of both the models and the paradigms are desired that should relate to additional types of system failures (e.g. changes in bandwidth, standard deviation and other characteristics of displayed processes), the field of attentional allocation and visual scanning, and situations with combinations of tolerance-band monitoring and failure detection (i.e. decisions based on sequential and non-sequential observations). The paper illustrates that extending structural and quantitative understanding of human operator behavior is substantially based on the interaction of mathematical models and empirical studies within the context of experimental paradigms. References

oi

0 Na

FIG. 7. Detection time Td VS number of processes N,. AUT I~:6-I

Baron, S. and W. H. Levison (1977). Display analysis with the optimal control model of the human operator. H u m a n Factors, 19, 437. Baron, S. and W. H. Levison (1980). The optimal control model. Proceedings I E E E Conference on Cybernetics and Society.

718

Brief Paper

Bendat, J. S. and A. G. Piersol (1971 ). Random Data: Analysis and Measurement Procedures. John Wiley, New York. Carbonell, J. R. (1966). A Queueing model of many-instrument visual sampling. IEEE Trans Human Fact. Electron., HFE-7, 157. Clement, W. F., H. R. Jex and D. Graham (1968). A manual control display theory applied to instrument landings of a jet transport. IEEE Trans Man--Machine Syst., MMS-9, 93. Clement, W. F., D. T. McRuer and R. H. Klein (1972). Systematic manual control display design. Proceedings Conference on Guidance and Control Displays. AGARD-CP96, Advisory Group for Aerospace Research and Development, Neuilly sur Seine, France. Companion, M. A. and G. M. Corso (1982). Task taxonomies: a general review and evaluation. Int. J. Man--Machine Studies, 17, 459. Cox, D. R. and P. Lewis (1966). The Statistical Analysis of Series of Events. Methuen/Wiley, London. Curry, R. E., D. L. Kleinman and W. C. Hoffman (1977). A design procedure for control/display systems, Human Factors, 19, 421. Gai, E. G. and R. E. Curry (1976). A model of the human observer in failure detection tasks. IEEE Trans Syst., Man and Cybern., SMC-6, 85. Graaff, R. C. v.d. (1981). An experimental analysis of human monitoring behavior in multivariable failure detection tasks. Report TR 81063 U, National Aerospace Laboratory NLR, Amsterdam, The Netherlands. Hess, R. A. (1977). Analytical display design for flight tasks conducted under instrument meteorological conditions. IEEE Trans Syst., Man and Cybern., SMC-7, 453. Hess, R. A. (1981). Aircraft control-display analysis and design using the optimal control model of the human pilot. IEEE Trans Syst., Man and Cybern., SMC-I1,465. Himmelblau, D. M. (Ed.) (1973). Decomposition of Large Scale Problems. North-Holland, Amsterdam. Johannsen, G. and T. Govindaraj (1980). Optimal control model predictions of system performance and attention allocation and their experimental validation in a display design study. IEEE Trans Syst., Man and Cybern., SMC-10, 249. Jones, J. C. (1980). Design Methods. John Wiley, Chichester. Kleinman, D. L., S. Baron and W. H. Levison (1971). A control theoretic models lor human operator display monitoring Trans Aut. Control, AC-16, 824. Kleinman, D. L. (1976). Solving the optimal attention allocation problem in manual control. IEEE Trans Aut. Control, AC-21, 821. Kleinman, D. L. and R. E. Curry (1977l. Some new control theoretic models for human operator display monitoring. IEEE Trans Syst., Man and Cybern., SMC-7, 778. Knowles, W. B., W. J. Burger, M. B. Mitchell, D. T. Hanifan and J. W. Wulfeck (1969). Models, measures, and judgements in system design. Human Factors, !1, 577. Levison, W. H. (197l). A control-theory model for human decision making. Proceedings Seventh Annual Conference on Manual Control, NASA SP-281, National Aeronautics and Space Administration, Washington, DC. Levison, W. R. and R. B. Tanner (1971). A control-theory model for human decision-making. Report NASA CR-1953, National Aeronautics and Space Administration, Washington, DC. McCormick, E. J. (1976). Human Factors in Engineering and Design. McGraw-Hill, New York. Meister, D. (1971). Human Factors: Theory and Practice. John Wiley, New York. Meister, D. (1976). Behavioral Foundatiolls ~)1'System Development. John Wiley, New York. M oraal, J. and K.-F. Kraiss (Eds) (1981). Manned Systems Design: Methods, Equipment, and Applications. Plenum Press, NewYork. Moray, N. (1980). Human information processing and supervisory control. Report Man-machine Systems Laboratory, Department of Mechanical Engineering, Massachusetts Institute of Technology, Cambridge. Moray, N. (1981 ). The role of attention in the detection of errors and the diagnosis of failures in man-machine systems. In W. B. Rouse and J. Rasmussen (Eds.), Human Detection and Diagnosis of System Failures. Plenum Press, New York. Naylor, A. W. (1981). On decomposition theory: generalized dependence. IEEE Trans Syst., Man and Cybern., SMC-II, 699.

Obermayer, R. W. (1964). Simulation, models, and games: sources of measurement. Human Factors, 6, 607. Patzak, G. (1982). Systemtechnik- Planung komplexer innovativer Systeme. Springer, Berlin. Pew, R. W. and S. Baron (1983). Perspectives on human performance modelling. In G. Johannsen and J. E. Rijnsdorp (Eds.), Analysis, Design, and Evaluation of Man--Machine Systems. Pergamon Press, Oxford. Rouse, W. B. (1981). Human-computer interaction in the control of dynamic systems. Computing Surt~eys, 13, 72. Rouse, W. B. (1983). Models of human problem solving: detection, diagnosis, and compensation for system failures. In G. Johannsen and J. E. Rijnsdorp (Eds.), Analysis, Design, and Evaluation of Man Machine Systems. Pergamon Press, Oxford. Sage, A. P. and J. L. Melsa (1971). Estimation Theory with Applications to Communications and Control. McGraw-Hill, New York. Sata, T. and E. Warman (Eds.) (1981). Man-Machine Communication in CAD!CAM. North Holland, Amsterdam. Schmldt, D. K. (1979). Optimal flight control synthesis via pilot modelling. AIAA J. Guidance and Control, 2, 308. Senders, J. W. (1964). The human operator as a monitor and controller of multidegree of freedom systems. IEEE Trans Human Fact. Electron., FIFE-5, 2. Sheridan, T. B. and W. R. Ferrell (1974). Man-Machine Systems: Information, Control, and Decision Models ¢~f' Human PerJbrmance. MIT Press, Cambridge. Sheridan, T. B. and G. Johannsen (Eds.) (1976). Monitoring Behavior and Supervisory Control. Plenum Press, New York. Simon, H. A. (1981). The Sciences ~?!the Art![7"cial. MIT Press, Cambridge. Smallwood, R. D. (1967). Internal models and the human instrument monitor. IEEE Trans Human Fact. Electron., HFE8, 181. Spillers, W. R. (Ed,) (1974). Basic Questions of Design Theory. North-Holland, Amsterdam. Spillers, W. R. (1977). Design theory. IEEE Trans Syst,, Man and Cybern., SMC-7, 201. Stein, W. (1981). A monitoring and decision making paradigm: experiments and human operator modeling. Proceedings First European Conference on Human Decision Making and Manual Control, Delft University of Technology, The Netherlands. Stein, W. and P. H~ Wewerinke (1983). Model-based prediction of human performance with respect to monitoring and failure detection. In G. Johannsen and J. E. Rijnsdorp (Eds.), Analysis, Design, and Evaluation of Man-Machine Systems. Pergamon Press, Oxford. Volkmann, F. C., A. M. L. Schick and L. A. Riggs (1968). Time course of visual inhibition during voluntary saccades. J. Optical Soc. Am., 58, 562. Wewerinke, P. H. (1976). Human control and monitoring-models and experiments. Proceedings Twelfth Annual Conference on Manual Control, NASA TM X-73, 170, National Aeronautics and Space Administration, Washington, DC. Wewerinke, P. H. (1977a). Human monitoring and control behavior: models and experiments. Report TR77010 U, National Aerospace Laboratory NLR, Amsterdam, The Netherlands. Wewerinke, P. H. (1977b). Performance and workload analysis of in-flight helicopter missions. Report MP77013 U, National Aerospace Laboratory NLR, Amsterdam, The Netherlands. Wewerinke, P. H. (1981a). A model of the human decision maker observing a dynamic system. Report TR81062 L, National Aerospace Laboratory NLR, Amsterdam, The Netherlands. Wewerinke, P. H. (198lb). A model of the human observer and decision maker. Proceedings Seventeenth Annual ConJerence on Manual Control, JPL 81-95, California Institute of Technology, Pasadena. Wewerinke, P. H. (1983). Model of the human observer and decision maker--theory and validation. In G. Johannsen and J. E. Rijnsdorp (Eds.), Analysis, Design, and Evaluation of Man-Machine Systems. Pergamon Press, Oxford. Zwicky, F. and A. G. Wilson (Eds.) (1967). New Methods o! Thought and Procedure. Springer, New York.