- Email: [email protected]
Intuitive perceptual judgements and the efficiency of observed decision strategies
Acta Psychologka 29 (1969) 367-375; 0 Not to be reproduced
North-Holland Publishing Co., Amswdam
in any form without
IN’FUITJLVE PERCEPTUAL
written permission
from the publisher
JUDGEMBNTS
AND T DlX!ISIC?N STRATIXWS”
ICIEN
G. LOWE Department of Psychology, University of Hull, England
ABSTRACT
An attempt is ma& to examine k detaiI some of the pre&ctions from statistical decision theorize of perceptual julgements. Is most sensory threshold experiments, the stat&M par-meters of the stirnulLs situation cannot be measured directly. Ideal performance cannot be spe&fi& im these situations, and consequently no precise measure of the observer’s ofEciency PCS be obtti~!. ..! perceptual task is &crib&, invo:vmg intuitive statistical decisions, in which these parameters could be measured. indices of observer efficiency were determined in relation to an ideal statistical decision strategy. The observed discrepancy seemed to indicate that observers were using a sub-optimal decision strategy, based on independent observations. It k suggested that the seIection of this strategy may have ibeen determined by the relative complexity of the task, which was increased by uncertainty regarding signal rr:tensity.
A number of recent treatments of perceptual problems have sought explanations in terms of models erived fl-(Dlnstatistical decision theory @.g. GREEN, 1960; SWETS, ANNERarid BIRDSALL, 1961). These theories all assume that the brain, when faced with uncertti ,information, makes an elaiborate caEcula.tionwhich is identical with, or equivalent to, an optimum statistical decision system. For example, GREGORY and CANE (1955) halve suggested that visual threshold judgements are diflerentiaj discriminations analogous to a form of statistical t-test. The justifications for using the statistical decision model as a description of human perceptual decisions seem to be the following: (i) the results of many sensory threshold c:xperiments can be fitted by a normal distribution, or by some derivation of iL; (ii) over a certa.in range? absolute and incremental visual thresholds ckrease by a factor \‘x, where x is the area or duration of the ~tiim~uhts; (iii) Merent thresholds increase by a factor Vy, where 4’is an increase *
I am grateful to Professor
prehminary
experiments
relating
(1. T Wo:va.r% for many to this work. 367
of thn ideas and
368
G. LOWE
in background intensity. (There are the ‘square-root’ laws typically observed in visual threshold experiments.) Alternatively, one can treat the statistical model not as a model of how the brain actually makes perceptual decisions, but rathe:r as a system which provides a criteriou of ideal performance, against which any actual performance or mechanism can be measured. On this view, the experiments mentioned above cannot be regarded as evidence for a statistical decision model of perception. One can only infer indirectly from the existence of the ‘square-root’ laws tha.t whatever mechanism is operating is reasonably efficient.. Unfortunately, one cannot measure directly the efficiency of these perceptual judgements, since the statistical parameters of the stimulus situation cannot be measured exactly. A situation was therefore devised in which the relevant parameters could be specified precisely. The experimental task involved judgements about parameters of a random Gaussian distribution of dots displayed sequentially on an oscilloscope screen. By the use of such a system, the following conditions were satisfied: (i) both noise (N) and signal (S) values were normally di,stributed; (ii) (JN and us could be measured exactly and were, in fact, equal in this situation; (iii) signal sample sLi.ze (equivalent to stimulus area or duration in visual threshold experiments) wait-~ s-d precisdy (n or numbm of observations); (iv) there were none of the problems involved in the sensory transformation of stimulus information: measures were applied to the exact stimulus &uation presented to the subject. In other words, the investigation was concerned directly with the actual decision processes involved in the task. Consequently. ideal performance could be specified in all respects, enabling direct measures of efficiency to be obtained from a comparison of ideal and observer performance, The present experiment investigated the effects of increasing signal sample size, which is analogous to lincretig stimulus area or duration in a visual tieshold situation. For an ideal stat&t&l de&ion strategy, one would exI~ect a square-root re&ationship, i.e. the 50 % de&&on thres?Lold should be inversely proportional to the square-root of the signal sample s:ize (n). Moreover, since n is specified precisely, the ideal statistical decision strategy should take the form
EFFICIENCY
OF .INTUITIVE PERCEPTUAL
JUDGEMENTS
369
where zs is the mean level during the sequence of dct,ts in a stimulus trial, illNis the mean noise level, and z is some critical value related to the observer’s decision criterion. This ideal strategy treats the stimulus as a sample of n dependent observations and makes an estimate of the mean level. If one assumes that the decision criterion remains fairly constant, then detection thresholds for s&,naIs of thi3 kind should decrease as V/n increases. An alternative statistical strategy, ,which is less than ideal and which would lead to less than optimum performance, treats the stimuius sample as n independent observations. This is a!lready d&carding information present in the experimental situation. On this strategy, the observer would have to perform up to n independent tests until a significant difference was found in one of them. Thus the probability of detecting a signal of sample size n is equal to the probability that at least one of the n observations is significantly different: P,s=
r-(1
-Pp
(2)
where P, = probability of detecting a signal of sample size 1 (one observation or dot, in this case). erformancc with that of an ideal By comparing the observer’s statistical decision strategy, one would have a measure of the efficiency of observer performance. The experiment was directly concerned with the effects of signal intensity and sample size on intuitive statistical decisions, and thus such efficiency measures v;ould probably indicate the kind of decision strategy or process used by the observer in this situation. METHOD
The experimental set-up involved a random sequence of dots appearing on a cathode-ray oscilloscope (CRO) screen, forming a sequential Gaussian distribution with a measurable mean and standard deviation (a). A shit in the mean level of this distribution during a specified temporal interval constituted a signal. Signal sample size was determined by n, the number of dots appearing in a given temporal interval or sample sequence. An input to a display oscilloscope from a random signal generator produced the random Gaussian waveform along the Y-axis. In conjunction with a device for modulating the trace brightness, the square
370
G. LOWE
wave output from a low frt;\;uency signal generator, set at 4 cps, was also fed into the CRO. At e ich pulse from the LF generator only a single small illuminated dot could bbeseen on the screen at a position along the Y-axis determined by the variable output from the random signal generator. Consequently, a discrete, random sequence of dots appeared on the CR0 screen at a rate of four per second throughout the experimental sessions. Mean levels and (fN(wticih was q\ual to as) were measured by a standard voltmeter and a rms meter. The observation intervi {number of dots, n) on each stimulus trial was indicated by an equivalent sequence of auditory ‘pips’ through a loudspeaker. This, of course, also indicated signal sample size. Signal ‘intensity’ consisted of an upward shit of the mean level of the random distribution of dots along the Y-axis. This was achieved by appByi.ngto the CR0 input a bias voltage, directly related to the mean value and standard deviation (or rmsV) of the distribution. In (Tunits, signal intensity values were 0.5, 1.Q, and 2.0. There were 40 trials at each of the three intensity values for each sample size. Blank trials (B) were randomly interspersed with signal trials ;lS), and P(S) =: P(B) = 0.5. Signal sample size remained constant over a given block of trials, with n = 1: IZ= 4, and r2 = 16. Stimulus presentation, randomized intensity values and signal sampl: size were controlled by means of a programmed output from a tape-reader, driven by an amplified pulse output from the LF generator. Fourteen university studcnts served as observers, and their task was to decide, on each trial, whether a given sample or sequence of dots came from the ‘noise’ or normal random distribution (N), or from the deviated or signal distribution (S). Except for signal presentations, the N distribution was diplayed throughout (the experimental sessions, and for 10 minutes in a preliminary adaptation period. There were 12 blocks of 60 trials altogether for each observer. RESVLTS An ideal statistical decision system will show a square-root relation-
ship between the 50 9% dctection threshold and signal sample size. Observd thresholds were 1.O, 0.75 and 0.62 for JZ‘-L:1, 4 and 16 respectively. Although there is the expected decrease in detection threshold as II increases, the decrease is considerably less than that predicted on the basis of ,the square-root laws. Mowever, one important featAre of the data is that the FA (false alarm) rtite was s@nifican@ly
EFFICIENCY
371
OF .INTUITIVE PERCEPTUAL JUlXiEMENTS
reduced with larger signal samlsle sizes. The square-root laws are baseId on the assumption that the FA rate remains relatively constant; therefore the data were adjusted to a common FA rate (fig. 1). The adjusted data reveal detection thresholds of 1.025, 0.45 and 0.27, which closely approximate a square-root relationship. This may be one indication of efficient performance by the observers. WY) 099
-95
.“O
40 *LO
-20
-05 0
03
1-O SlGNAL
2-o
INTENSITY
(in (runitsl
Fig. I. Probabiiity o:E a deteotion response as a function of signal intensity and signal sample size. Straight lines represent best-fitting Iii through the average observed data points of 1.4 subjeots, adjusted to a common FA rate. 50 % detect,ion threshold values are indicated.
Ideal performance was also calculated on a statistical decision theory ba&, given, precise information rsgarding ON, signal mean deviation from mean noise level (SlignaIintensity), signal sample size Land&smed FA rate, which indicates the criterion or decision point along rhe deviation (si’gnal‘irrtensity) axis. Such idearl prfmnce will be given by P(YJS~)=l-p~z(l-P(
Ypv))-81)
(3)
using tables of the norm81 distribution, where P(Y/S,) is the predicted probability of detecting a signal of a given intensity, P(Y/N) is the observed FA rate, and p and z are references to normal distribution
372
G. LOWE
parameters. When signal salmple size, ~2~is greater than 1, then the standard error of the estimate of mean level of signal intensity is given by oN /dn. The SEE m&hod of predkting ideal performance cm then be used with (~~/dn
taZaiag,mw vdw
i-lcCax*!m
srn~~k size.
Fig. 2 shows ideal predicted performance on the basis of this ideal strategy and observed performance for each signal sample size. The r bserved trend is similar to that predicted, but not so close as to expect a very high degree of eficiency. One meesure of efficiency may be determined by a comparison of predicted and observed detection probabilities. Table 1 indicates that observer efficiency in this situation is quite reasonable, with an overall average efficiency index of 0.74 (Fp). WI
-80
'LO l20 n=4 *OS +0
[ -5
, 1-o
I
2-o t, SIGNAL
I -5
I 1-o
INTENSITY
I 2-o
I 1, -5
I 1-o
1
2-o
lin (r units)
Probability of a de@ction reqxmw as a function of signal intensity and points represent the average obswved performame of 14 subjects. Straight lii indicate predicted performance on the basis of an ideal strategy (heavy lines) and an ‘indqenknt observation’ strategy (thin fines). For signal sample size n = 1, both strategies are identical. Fig. 2.
signal s;ampk size (n). The data
alternative measure of efficiency is to compare detectability vahres (8) for ideal and observed performance. Table 1 shows that such An
EFFICIENCY OF JNTUITIVE PERCEPTUAL JUIXZMENTS
373
effitiency va!llues(F#) are cmly ate. ver, in or& ‘to inltepre‘t the si@cance of these values and to relate them to Fp, equivalent: measures of percentage decrement from ideal detection, % D(P(Y/s)), can be obtained from l-F, and are also indicated in table 1. Thus, an, average overall Fdt, of 0.46 is equivalent St0a overall nt of about 25 %.
Average e,iRkiency val~tes for the &served an ided statistid
lxx=foma dmiskn
of 14 subjects, based on
strategy.
Signal inten& 0.5 Fp 1 4 16 Av.
1.
Faj %D
.80 .46 .78 .67 .53 .42 .70 -52
20 22 47 30
2.0
Average
FP
Far ‘$(*D
FP Fat %D
FP Fk
%D
.68 .70 .66 .67
.39 .55 .34 .43
.71 .43 .87 .48 .99 .40 .86 .44
.73 .78 .Y3 .75
27 22 27 25
32 30 34 32
29 13 1 14
.43 .57 .39 .46
This degree of ine@iciency may be due to one of two things: either (a) the observer is using an ideal strategy but various factors (e.g. limited attention and memory, uncertainty about signal parameters, etc.) may ce; or (6) the observer may be reduce the efficiency of ‘his perfo using an alternative strategy which is less than ideal, and which results in less than optimum performance. For instance3 a strategy which treats the signal sample size as II independerzt aFbservations would show a performance decrement when sample size is greater than 1. If one applies the independence formula (2) directly, this would result in greatly increased FA rates as n increases. However, it seems reasonable to assume that the observer attempts to keep his FA rate roughly constant throughout all conditions. In effect, the observer raises his criterion (C) as n increases, thus reducing the probability that any single dot. or observation will exceed C. If one accepts the observed FA rate as an indication of the setting of C in each condition, then one can estimate derived probab&litia of &&&ion for single ob!xxvabions. Consequently, applying fmmu,Ia (2) #tc+thaw wU dtotermtmtepedioted ~probabiies of detection for multiple observation conditionb (n = 4, n = 16). Predicted performance, on the basis td this independent observation strategy is shown in fig. 2.
374 TABLB 2
A uxnpariso~ of the average performan!oe of 14 0tWer;Ws with predicted performance M qxxn a subqtimal ‘indepeaden~ observation’ strategy. RP and Rdlrefer to ratio values of gherved and predioted performance in relation to ~dekction probabilities (P) an8 dote&hiGty (&).
Signal Me
sim (R) 1 4 16 Av.
.80 1.0 .98 .93
.46 .97 .97 .80
20 (i 2 7
.68 .84 .80 .77
.39 .77 .73 .63
32 16 20 23
.71 .87 .99 .86
.43 29 .63 13 .78* 1 -68 14
,.73 .90 .92 .85
.43 .79 .83 .70
27 10 8 15
* Based on an estimated vahe of d’, since the predicted P(Y/sj, was pester than 0.999.
Predicted and observed pe~ormances are compared in table 2, which gives ratios for d’ and P (detection probabilities), together with percentage decrernen.t. Observer perfomzlnce shows a much closer fit to this independent observation strs;.tegythan to the ideal statistical decision model, when fz is greater than 1, (The two models give identical predictions, of course, for the n = 1 condition.) In fact, there is an overall decrement of only 15 % , as compared with 25 % on the ideal decision strategy. l3ISCUSSION
The results wrn to indicate two things: (1) Observers do not, generally, adopt an ideal statistical decision stratem in tasks of this sort. One reason for this may be that the task was complicated, to some extent, by the random presentation of signal intensities. In any case, this would probably calntribute to an unstable critetion, thereby reducing efficiency. (2) It seems likely that observers use a suboptimal stl ategy based on i observations. there still remains some degree of inefIicienqy (i.e. 15 % more than 60 % of this decrement is due (see table 2). This wo&d indicate &at an ion & the major portion of the in&ciency might ve.ry well be
EFFICIENCY OF JNTUITIVE IPERCEPTUALJUDGEMENTS1
37’5
related to
the special circumstances of this particular task conditiol.1. One possibility is the temporal uncertainty of the signal in the ‘1 dot’ condition. Since dots were appearing on the CR0 screen at a rate of 4 per set, observers may have been uncertain about exactly which dot was specified by the auditory tone or ‘pip’. Such temporal uncertainty, which has been shown to affect auditory detection (EGAN et al., 196%) and visual detection
(LOWE, 1967), would, of course, be very much less inthen= 4 condition, and almost ne&gible in the n = 16 conditioa, with longer sequences of dots. It is concluded that observers in an intuitive statistical decision tasik seem to adopt a sub-optimal decision strategy based on indeper dent observations. Uncertainty regarding signal parameters (intensity ar,d temporal specification) affects observer efficiency and may cause observers to adopt sub-optimal strategies. However, under more spec%c conditions, recent experiments have shown that human observers are capable of using optimal observational proccsc?s in signal detection tasks, particularly after extended periods of training (e.g. SWETS and BIRDSALL, 1967; BRAZEAL and BOOTH, 1966).
BRAZEAL,E. H. Jr. and T. L. BOOTH,1966. Operator noise in a discrete signal ddection task. IEEE Trans. on uman Factors in Electronics, Vol. HFE-7, 164-173. EGAN, J. P., G. Z. GREENBERGand A. I. SCHULMAX, 1961. l&xva.l of time WIcertainty in auditory detection. J. acoust. Sot. Amer. 33, 771-77:B. GREEN, D. M., 1960. Psycho-acoustics and detection theory. J. acoust. Sot. Amer. 32, 1189-1203. GREGORY,R. L. and V. CANE, 1955. A statistical informatioln theory of visual thresholds. Nature, 176, 1272. LOWE, G., 1967. Interval of time uncertainty in visiair detection. Percept. & i Psyuhophys. 2, 278-280. WETS, J. A. and T. G. BIRDSAU, 1967. Deferred decisions in human sign detection: A preliminary experiment.. Percept. & Psychophys. 15-28. , W. P. TANNER,Jr. zu-d T. G. BIRDSALL,1961. Decisdon processes in pe,rception. Psychd. Rev. 68, 301-340.
North-Holland Publishing Co., Amswdam
in any form without
IN’FUITJLVE PERCEPTUAL
written permission
from the publisher
JUDGEMBNTS
AND T DlX!ISIC?N STRATIXWS”
ICIEN
G. LOWE Department of Psychology, University of Hull, England
ABSTRACT
An attempt is ma& to examine k detaiI some of the pre&ctions from statistical decision theorize of perceptual julgements. Is most sensory threshold experiments, the stat&M par-meters of the stirnulLs situation cannot be measured directly. Ideal performance cannot be spe&fi& im these situations, and consequently no precise measure of the observer’s ofEciency PCS be obtti~!. ..! perceptual task is &crib&, invo:vmg intuitive statistical decisions, in which these parameters could be measured. indices of observer efficiency were determined in relation to an ideal statistical decision strategy. The observed discrepancy seemed to indicate that observers were using a sub-optimal decision strategy, based on independent observations. It k suggested that the seIection of this strategy may have ibeen determined by the relative complexity of the task, which was increased by uncertainty regarding signal rr:tensity.
A number of recent treatments of perceptual problems have sought explanations in terms of models erived fl-(Dlnstatistical decision theory @.g. GREEN, 1960; SWETS, ANNERarid BIRDSALL, 1961). These theories all assume that the brain, when faced with uncertti ,information, makes an elaiborate caEcula.tionwhich is identical with, or equivalent to, an optimum statistical decision system. For example, GREGORY and CANE (1955) halve suggested that visual threshold judgements are diflerentiaj discriminations analogous to a form of statistical t-test. The justifications for using the statistical decision model as a description of human perceptual decisions seem to be the following: (i) the results of many sensory threshold c:xperiments can be fitted by a normal distribution, or by some derivation of iL; (ii) over a certa.in range? absolute and incremental visual thresholds ckrease by a factor \‘x, where x is the area or duration of the ~tiim~uhts; (iii) Merent thresholds increase by a factor Vy, where 4’is an increase *
I am grateful to Professor
prehminary
experiments
relating
(1. T Wo:va.r% for many to this work. 367
of thn ideas and
368
G. LOWE
in background intensity. (There are the ‘square-root’ laws typically observed in visual threshold experiments.) Alternatively, one can treat the statistical model not as a model of how the brain actually makes perceptual decisions, but rathe:r as a system which provides a criteriou of ideal performance, against which any actual performance or mechanism can be measured. On this view, the experiments mentioned above cannot be regarded as evidence for a statistical decision model of perception. One can only infer indirectly from the existence of the ‘square-root’ laws tha.t whatever mechanism is operating is reasonably efficient.. Unfortunately, one cannot measure directly the efficiency of these perceptual judgements, since the statistical parameters of the stimulus situation cannot be measured exactly. A situation was therefore devised in which the relevant parameters could be specified precisely. The experimental task involved judgements about parameters of a random Gaussian distribution of dots displayed sequentially on an oscilloscope screen. By the use of such a system, the following conditions were satisfied: (i) both noise (N) and signal (S) values were normally di,stributed; (ii) (JN and us could be measured exactly and were, in fact, equal in this situation; (iii) signal sample sLi.ze (equivalent to stimulus area or duration in visual threshold experiments) wait-~ s-d precisdy (n or numbm of observations); (iv) there were none of the problems involved in the sensory transformation of stimulus information: measures were applied to the exact stimulus &uation presented to the subject. In other words, the investigation was concerned directly with the actual decision processes involved in the task. Consequently. ideal performance could be specified in all respects, enabling direct measures of efficiency to be obtained from a comparison of ideal and observer performance, The present experiment investigated the effects of increasing signal sample size, which is analogous to lincretig stimulus area or duration in a visual tieshold situation. For an ideal stat&t&l de&ion strategy, one would exI~ect a square-root re&ationship, i.e. the 50 % de&&on thres?Lold should be inversely proportional to the square-root of the signal sample s:ize (n). Moreover, since n is specified precisely, the ideal statistical decision strategy should take the form
EFFICIENCY
OF .INTUITIVE PERCEPTUAL
JUDGEMENTS
369
where zs is the mean level during the sequence of dct,ts in a stimulus trial, illNis the mean noise level, and z is some critical value related to the observer’s decision criterion. This ideal strategy treats the stimulus as a sample of n dependent observations and makes an estimate of the mean level. If one assumes that the decision criterion remains fairly constant, then detection thresholds for s&,naIs of thi3 kind should decrease as V/n increases. An alternative statistical strategy, ,which is less than ideal and which would lead to less than optimum performance, treats the stimuius sample as n independent observations. This is a!lready d&carding information present in the experimental situation. On this strategy, the observer would have to perform up to n independent tests until a significant difference was found in one of them. Thus the probability of detecting a signal of sample size n is equal to the probability that at least one of the n observations is significantly different: P,s=
r-(1
-Pp
(2)
where P, = probability of detecting a signal of sample size 1 (one observation or dot, in this case). erformancc with that of an ideal By comparing the observer’s statistical decision strategy, one would have a measure of the efficiency of observer performance. The experiment was directly concerned with the effects of signal intensity and sample size on intuitive statistical decisions, and thus such efficiency measures v;ould probably indicate the kind of decision strategy or process used by the observer in this situation. METHOD
The experimental set-up involved a random sequence of dots appearing on a cathode-ray oscilloscope (CRO) screen, forming a sequential Gaussian distribution with a measurable mean and standard deviation (a). A shit in the mean level of this distribution during a specified temporal interval constituted a signal. Signal sample size was determined by n, the number of dots appearing in a given temporal interval or sample sequence. An input to a display oscilloscope from a random signal generator produced the random Gaussian waveform along the Y-axis. In conjunction with a device for modulating the trace brightness, the square
370
G. LOWE
wave output from a low frt;\;uency signal generator, set at 4 cps, was also fed into the CRO. At e ich pulse from the LF generator only a single small illuminated dot could bbeseen on the screen at a position along the Y-axis determined by the variable output from the random signal generator. Consequently, a discrete, random sequence of dots appeared on the CR0 screen at a rate of four per second throughout the experimental sessions. Mean levels and (fN(wticih was q\ual to as) were measured by a standard voltmeter and a rms meter. The observation intervi {number of dots, n) on each stimulus trial was indicated by an equivalent sequence of auditory ‘pips’ through a loudspeaker. This, of course, also indicated signal sample size. Signal ‘intensity’ consisted of an upward shit of the mean level of the random distribution of dots along the Y-axis. This was achieved by appByi.ngto the CR0 input a bias voltage, directly related to the mean value and standard deviation (or rmsV) of the distribution. In (Tunits, signal intensity values were 0.5, 1.Q, and 2.0. There were 40 trials at each of the three intensity values for each sample size. Blank trials (B) were randomly interspersed with signal trials ;lS), and P(S) =: P(B) = 0.5. Signal sample size remained constant over a given block of trials, with n = 1: IZ= 4, and r2 = 16. Stimulus presentation, randomized intensity values and signal sampl: size were controlled by means of a programmed output from a tape-reader, driven by an amplified pulse output from the LF generator. Fourteen university studcnts served as observers, and their task was to decide, on each trial, whether a given sample or sequence of dots came from the ‘noise’ or normal random distribution (N), or from the deviated or signal distribution (S). Except for signal presentations, the N distribution was diplayed throughout (the experimental sessions, and for 10 minutes in a preliminary adaptation period. There were 12 blocks of 60 trials altogether for each observer. RESVLTS An ideal statistical decision system will show a square-root relation-
ship between the 50 9% dctection threshold and signal sample size. Observd thresholds were 1.O, 0.75 and 0.62 for JZ‘-L:1, 4 and 16 respectively. Although there is the expected decrease in detection threshold as II increases, the decrease is considerably less than that predicted on the basis of ,the square-root laws. Mowever, one important featAre of the data is that the FA (false alarm) rtite was s@nifican@ly
EFFICIENCY
371
OF .INTUITIVE PERCEPTUAL JUlXiEMENTS
reduced with larger signal samlsle sizes. The square-root laws are baseId on the assumption that the FA rate remains relatively constant; therefore the data were adjusted to a common FA rate (fig. 1). The adjusted data reveal detection thresholds of 1.025, 0.45 and 0.27, which closely approximate a square-root relationship. This may be one indication of efficient performance by the observers. WY) 099
-95
.“O
40 *LO
-20
-05 0
03
1-O SlGNAL
2-o
INTENSITY
(in (runitsl
Fig. I. Probabiiity o:E a deteotion response as a function of signal intensity and signal sample size. Straight lines represent best-fitting Iii through the average observed data points of 1.4 subjeots, adjusted to a common FA rate. 50 % detect,ion threshold values are indicated.
Ideal performance was also calculated on a statistical decision theory ba&, given, precise information rsgarding ON, signal mean deviation from mean noise level (SlignaIintensity), signal sample size Land&smed FA rate, which indicates the criterion or decision point along rhe deviation (si’gnal‘irrtensity) axis. Such idearl prfmnce will be given by P(YJS~)=l-p~z(l-P(
Ypv))-81)
(3)
using tables of the norm81 distribution, where P(Y/S,) is the predicted probability of detecting a signal of a given intensity, P(Y/N) is the observed FA rate, and p and z are references to normal distribution
372
G. LOWE
parameters. When signal salmple size, ~2~is greater than 1, then the standard error of the estimate of mean level of signal intensity is given by oN /dn. The SEE m&hod of predkting ideal performance cm then be used with (~~/dn
taZaiag,mw vdw
i-lcCax*!m
srn~~k size.
Fig. 2 shows ideal predicted performance on the basis of this ideal strategy and observed performance for each signal sample size. The r bserved trend is similar to that predicted, but not so close as to expect a very high degree of eficiency. One meesure of efficiency may be determined by a comparison of predicted and observed detection probabilities. Table 1 indicates that observer efficiency in this situation is quite reasonable, with an overall average efficiency index of 0.74 (Fp). WI
-80
'LO l20 n=4 *OS +0
[ -5
, 1-o
I
2-o t, SIGNAL
I -5
I 1-o
INTENSITY
I 2-o
I 1, -5
I 1-o
1
2-o
lin (r units)
Probability of a de@ction reqxmw as a function of signal intensity and points represent the average obswved performame of 14 subjects. Straight lii indicate predicted performance on the basis of an ideal strategy (heavy lines) and an ‘indqenknt observation’ strategy (thin fines). For signal sample size n = 1, both strategies are identical. Fig. 2.
signal s;ampk size (n). The data
alternative measure of efficiency is to compare detectability vahres (8) for ideal and observed performance. Table 1 shows that such An
EFFICIENCY OF JNTUITIVE PERCEPTUAL JUIXZMENTS
373
effitiency va!llues(F#) are cmly ate. ver, in or& ‘to inltepre‘t the si@cance of these values and to relate them to Fp, equivalent: measures of percentage decrement from ideal detection, % D(P(Y/s)), can be obtained from l-F, and are also indicated in table 1. Thus, an, average overall Fdt, of 0.46 is equivalent St0a overall nt of about 25 %.
Average e,iRkiency val~tes for the &served an ided statistid
lxx=foma dmiskn
of 14 subjects, based on
strategy.
Signal inten& 0.5 Fp 1 4 16 Av.
1.
Faj %D
.80 .46 .78 .67 .53 .42 .70 -52
20 22 47 30
2.0
Average
FP
Far ‘$(*D
FP Fat %D
FP Fk
%D
.68 .70 .66 .67
.39 .55 .34 .43
.71 .43 .87 .48 .99 .40 .86 .44
.73 .78 .Y3 .75
27 22 27 25
32 30 34 32
29 13 1 14
.43 .57 .39 .46
This degree of ine@iciency may be due to one of two things: either (a) the observer is using an ideal strategy but various factors (e.g. limited attention and memory, uncertainty about signal parameters, etc.) may ce; or (6) the observer may be reduce the efficiency of ‘his perfo using an alternative strategy which is less than ideal, and which results in less than optimum performance. For instance3 a strategy which treats the signal sample size as II independerzt aFbservations would show a performance decrement when sample size is greater than 1. If one applies the independence formula (2) directly, this would result in greatly increased FA rates as n increases. However, it seems reasonable to assume that the observer attempts to keep his FA rate roughly constant throughout all conditions. In effect, the observer raises his criterion (C) as n increases, thus reducing the probability that any single dot. or observation will exceed C. If one accepts the observed FA rate as an indication of the setting of C in each condition, then one can estimate derived probab&litia of &&&ion for single ob!xxvabions. Consequently, applying fmmu,Ia (2) #tc+thaw wU dtotermtmtepedioted ~probabiies of detection for multiple observation conditionb (n = 4, n = 16). Predicted performance, on the basis td this independent observation strategy is shown in fig. 2.
374 TABLB 2
A uxnpariso~ of the average performan!oe of 14 0tWer;Ws with predicted performance M qxxn a subqtimal ‘indepeaden~ observation’ strategy. RP and Rdlrefer to ratio values of gherved and predioted performance in relation to ~dekction probabilities (P) an8 dote&hiGty (&).
Signal Me
sim (R) 1 4 16 Av.
.80 1.0 .98 .93
.46 .97 .97 .80
20 (i 2 7
.68 .84 .80 .77
.39 .77 .73 .63
32 16 20 23
.71 .87 .99 .86
.43 29 .63 13 .78* 1 -68 14
,.73 .90 .92 .85
.43 .79 .83 .70
27 10 8 15
* Based on an estimated vahe of d’, since the predicted P(Y/sj, was pester than 0.999.
Predicted and observed pe~ormances are compared in table 2, which gives ratios for d’ and P (detection probabilities), together with percentage decrernen.t. Observer perfomzlnce shows a much closer fit to this independent observation strs;.tegythan to the ideal statistical decision model, when fz is greater than 1, (The two models give identical predictions, of course, for the n = 1 condition.) In fact, there is an overall decrement of only 15 % , as compared with 25 % on the ideal decision strategy. l3ISCUSSION
The results wrn to indicate two things: (1) Observers do not, generally, adopt an ideal statistical decision stratem in tasks of this sort. One reason for this may be that the task was complicated, to some extent, by the random presentation of signal intensities. In any case, this would probably calntribute to an unstable critetion, thereby reducing efficiency. (2) It seems likely that observers use a suboptimal stl ategy based on i observations. there still remains some degree of inefIicienqy (i.e. 15 % more than 60 % of this decrement is due (see table 2). This wo&d indicate &at an ion & the major portion of the in&ciency might ve.ry well be
EFFICIENCY OF JNTUITIVE IPERCEPTUALJUDGEMENTS1
37’5
related to
the special circumstances of this particular task conditiol.1. One possibility is the temporal uncertainty of the signal in the ‘1 dot’ condition. Since dots were appearing on the CR0 screen at a rate of 4 per set, observers may have been uncertain about exactly which dot was specified by the auditory tone or ‘pip’. Such temporal uncertainty, which has been shown to affect auditory detection (EGAN et al., 196%) and visual detection
(LOWE, 1967), would, of course, be very much less inthen= 4 condition, and almost ne&gible in the n = 16 conditioa, with longer sequences of dots. It is concluded that observers in an intuitive statistical decision tasik seem to adopt a sub-optimal decision strategy based on indeper dent observations. Uncertainty regarding signal parameters (intensity ar,d temporal specification) affects observer efficiency and may cause observers to adopt sub-optimal strategies. However, under more spec%c conditions, recent experiments have shown that human observers are capable of using optimal observational proccsc?s in signal detection tasks, particularly after extended periods of training (e.g. SWETS and BIRDSALL, 1967; BRAZEAL and BOOTH, 1966).
BRAZEAL,E. H. Jr. and T. L. BOOTH,1966. Operator noise in a discrete signal ddection task. IEEE Trans. on uman Factors in Electronics, Vol. HFE-7, 164-173. EGAN, J. P., G. Z. GREENBERGand A. I. SCHULMAX, 1961. l&xva.l of time WIcertainty in auditory detection. J. acoust. Sot. Amer. 33, 771-77:B. GREEN, D. M., 1960. Psycho-acoustics and detection theory. J. acoust. Sot. Amer. 32, 1189-1203. GREGORY,R. L. and V. CANE, 1955. A statistical informatioln theory of visual thresholds. Nature, 176, 1272. LOWE, G., 1967. Interval of time uncertainty in visiair detection. Percept. & i Psyuhophys. 2, 278-280. WETS, J. A. and T. G. BIRDSAU, 1967. Deferred decisions in human sign detection: A preliminary experiment.. Percept. & Psychophys. 15-28. , W. P. TANNER,Jr. zu-d T. G. BIRDSALL,1961. Decisdon processes in pe,rception. Psychd. Rev. 68, 301-340.