Individual differences in signal detection

Acta Psychofogica 34 (1970) 3S- 50; 0 North-Holland Pubkshing Cmnpany Not to be reproduced in any form without written permission from the publisher

INDWIDUAL

DIFFERENCES

IN SIGNAL

DETECTION

J. G, INGHAM* Medical Research Council, Liandough Hospital, Penarfh, Glartorgan, U.K.

ABSTRACT Can we study individual differences in sensitivity and response bi,as, using indices like the n’ and+8 of signal detectability theory? To do this, it would be desirable to be able to estimate them independently and consistently. Sensitivity, measured by d’, is not influenced by variations in p provided the variance of the noise distribution is equal to that of the signal-plus-noise distribution, but no such deduction can be made for 4. The intra-individual variances can be assumed to be approximately equal in the investigation described, and d’ may therefore be estimated without contamination by& It cannot be assumed, however, that B remains stable for different levels of d’; in fact it can be shown to depend upon the decision strategy adopted by the subject. The consistency and orthogonality of indiccs of sensitivity and bias have been e.xamined in an auditory detection task. Reasonably consistent estimates of d’ we:re obtained from as few as forty responses. The bias index 4 was less satisfactory but another index related to it was more promising. The latter was orthogonal to d’, under certain conditions, in the two-dimensional space describing inter-individual variation in detection behaviour.

There is a substantial literature dealing with i,rdividual differences in sensory thresholds and associations between thresholds and personality variables or clinical states. The earlier work was reviewed by HUNT(1944) and more recent appraisals have been made by ZUBIN (1957) and C&ANGER (1960). Examples of stimuli which have been studied in this way include visual sensitivity, critical flicker frequency, auditory sensitivity and pain sensitivity. It is now widely recognized, following the work of Tanner and Swets, Lute and others (GREENand SWETS, 1966; SWETS,1964; LL’CE,1963) that sensory perception of stimuli which are so indistinct that they are not always detected by the subject, cannot be adequately described by a single number (e.g. the fifty per cent threshoid). Whether the subject detects the stimulus or not dependc partly upon the degree to which it can be discriminated from its background of irrelevant information (internal l

I am indebted to Sister/Tutor- V. Morgan and the nurses who served as subjects;

also to my colleagues far mzu~y invaluable discussions. 39

40

J.

G.

INGHAM

and external to the su0ject) and partly upon a ‘decision process which is con&d of as being independent of the information on which it is based. Two numbers are required to describe both components of the process. The former, when ‘weare investigating variations between stimuli applied to the same individual, can be called ‘discriminability’. When we are investigating variations between individual subjects, however, using the same stimulus, ‘sensitivity’ is a more appropriate term though the index used may be identical. The number used to describe the decision component tells us about variations in the subject’s willingness to give a positive response on the basis of inadequate evidence. He may tend towards giving many positive responses or towards giving many negative responses and an appropriate term for this is ‘bias’. The threshold as measured by classical psychophysical methods, involves both components. A correlation between threshold and some other variable can be attributed to either or both components. The kind of interpretation one might place upon a correlation with sensitivity is clearly very different from the interpretation one might place upon a correlation with bias. It would be an obvious advantage if, in work on individual differences in stimulus thresholds, the two components could be estimated separatel:f. A familiar mode:1used in signal detection theory is illustrated in fig. 1. One of the dithculties that arises when we attempt to apply this model to ’ - investigation o,f individual differences concerns the orthogonality of rndlces of detection and bias. In a situation to which the model is applicable, the detecta0ility index presents no problem because d’ (the distance between the mzans of the two distributions in standard deviation units) remains the stirno:whatever the position of the criterion A along the horizontal axis. It is therefore possible to compare two individuals with respect to d” despite differences in bias. When we try to compare two individuals for bias, however, the situation is not so simple. The extent to which bias indices remain the same for different levels of d’ depends upon the decision strategy adopted by the individual. If he knows the value or cost of each correct or incorrect response a trained observer can, to a certain extent, ad.just his criterion to maximise the total value of a set of observations. In such circumstances it can be shown that )I is the same whatever the value of d’. LJnder other conditions the observer may ~ISOseek what has been termed a ‘Ncyman-Pearson’ objective, that is he may fix A with reference to the N distribution. If he does this, thenJ will decrease with increasing d’ as the SN distribution moves to the right in

INDIVIDUAL

DIFFERENCES

IN

SIGNALDETECTION

41

fig. 1. Presumably he might equally well fix A with reference to the SN distribution when J will increase with increasing d’. It szems likely that any of these strategies apply only under rather special experimental conditions using highly trained observers. It is of interest, however, to explore experimental situations which are less structured, leaving the subject himself to determine his own strategy, and to see how people do in fact behave under such circumstances. Until this has been done it IS unwise to assume that there is no index of bias which remains invariant over different values of d’.

.

33

0

+

w

_h&

Fig. 1. The horizontal axis represents that internal process in the organism which changesin the presence of the signal and therefore carries the information on which the decision ‘present’ or ‘absent’ is based. A sampling of the prozss at difierent instants pnerates distribution N in the absence of the stimulus and distribution SN in the presence of the stimulus. The subjwt decides that the signal is present when the internal process is to the right of A. Thlis version of the model assumes Gaussian distributions with equal variances. 8 = distance between the means = z,, - zan c = 2 x distance of A from point of intersection = z,, + q,,,; J = I,/&.

The position of A along the horizontal axis can also be indicated a.s a normal deviate (zn) from the mean of N or a normal deviate (zsn)from :the mean of the SN distribution. The detection index d’ is equal to the difference between these two deviates (zn - z& and it is worth consideriing the possibility of using the sum (zn + zen) as an index of bias. OLDHAM (1962) has pointed out that under certain conditions this sum, which will be called c, and the difference, which in this context is d’, are orthogonal. The index c is twice the distance, expressed in standard deviation units,

42

J. G, INGHAM

between the point at which the two distributions cross and the criterion

line d. For any given d” there is a one-to-one relationship between c and fl, but, when d’ changes, c and J change in diRerent ways. We have described one condition under whichjis cons’tant for different values of d’. Can we now state the conditions under which c is constant for different values of d’? A standard way of showing the results of a signal detection experiment (one subject and one signal inltensity) is to plot rn against zsn. The linearity and slope of such a plot are used to test two of the assumptions of the detectability model, namely the normality of both distributions and the equality of the variances. If we allow d’ to vary we can still plot the results for one individual onto the same diagram, thus obtaining a set of parallel straight lines. The diRerent values along each line are produced by altering bias levels, which may be done experimentally by changing the pay-off matrices, or by using different ins.tructions, or by asking the subject to rate each response according to the degree of confidence that he feels in the response. All the points can be considered as lying along a set of parallel lines where the parameter d’ can take any value, depending upon the stimulus intensity. The points can equally be considered as lying along alUotherset of parallel lines, at right angles to the first,

where the parameter c can take any value depending upon the response bias. So to express the results in terms of d’ and c is simply to rotate the axes through 45” from zn and zena Mow let us suppose it to be possible to do an experiment, still using a single individual as subject, in which b!y some appropriate process of sampling (from different bias instructions and different stimulus intensities) a bivariate normal distribution could be generated. The results of such an experiment are illustrated in fig. 2 in which the ellipse has been formed by joining together points of equal probability density. The principal axis of the ellipse has been drawn at 45” to z,, and zrstr. It will be seen that under these circumstances d’ and c are orthogonal (OLDHAM, 1962) If we draw a line linking,itogetherall points of a certain value of d’, the distribution of c is the same whicht:?/ervalue of d’ ‘wetake. Similarly for different values of c the distribution of d’ is the same.

INDIVIDUAL

DIFFERENCES

IN

SIGNAL

DETECTION

43

Fig. 2. The solid ellipse is f&cd by joining points of equal probability density in a bivariate normal distribution in which rQ = c++~~.The dotted ellipse represents a distribution in which ~2 c crS,,3.

Another way of saying that the principal axis of the ellipse is at 45” is to say tlhat the variance along the 2, axis (an2)equals the varia ice along the tall aKis (aSn2). The dotted ellipse illustrates the effect of changing these variances, which is to rotate the principal axis of the eilipse so that it is no longer at 45’. In these circumstances it can be seen that the mean value of c varies with d’. We now see that the condition for the orthogonaity of c and d’is that the two variances gsn2 and Q,,~are equal. It is important to remember that these variances are not those of fig. 1 where d’ is constant, but that they include a component due to variation in n’. If such an experiment were to be done on a number of individuals, each individual would generate a bivariate distribution like that in fig. 2 but the ellipses would themselves be scattered along both dimensions and this would introduce inter-individual components (an’2 and CT’~,~~) into the variances. It is suggested that the investigation to be described can be interpreted as one in which each individual is represented by a single point selected at random from his own individual bivariate distribution. If c7,12 =

Gm

2

and if also d’,2 = CT’& then the total variances will also be equal, including both inter-indijiiduai and in&a-individual components.

44

J.

G.

INGHAM

It is recognized that a possible weakness in the above argument is that the slopes of individual ROC curves may vary from one subject to another and, in general, may differ from unity. MARKOWITZand SWFTS (l%7) have recently shown that 21rating method apyiied to several different detection situations, produced ROC curves whose slopes decreased with increasing signal strengths., For a one-interval simple detection situatlt-g, however, like the one used in the present investigation, the slope did not depart greatly from unit.y for values of d’ up to about 2.0. Although the assumption of unit slopes for individual ROC curves may not be strictly justified, it is an approximation that will probabiy serve for the purpose of the present investigation. The objects of the enquiry were (1) to see whether individual differences in 11or c were correlated with those in d’ and (2) to see whether the three in&ices could be measured consistently both within a single session and on different sessions.

,1 six-category rating method was used, with knowledge of results during a preli,minary practice period. Subjects y;vereseated in a sound-insulated chamber and the stimulus, a pure tone of 1000 cps, with a duration of 1 set, was presented to one ear through a headphone. The stimulus was generated by means of a Dawe trznsistor oscillator type 421 and the amplitude was controlled by means ol’ ,an attenuator. The duration of the stimulus was controlled by an elc%tronic timer activating a reed switch inserted between the oscillator and the attenuator. The subject was required to indicate whether she had detected the stimulus during a period of fifteen seconds from the previous response until the occurrence of a light signal. During SO”JO of these time intervais the stimulus occurred at a random time from five to thirteen seconds from the previous response. Each group of four successive time intervals contained two stimuli, in random order, but the subjects were not aware of this grouping. Subjects were instructed to make a response every time the light signal appeared and they were to indicate whether they had heard the stimulus at a.ny time during the previous fifteen seconds (since the previous response). The response was a two-stage operation. First they were instructed to press one of two keys, the first ‘if you think you might have heard it’ and the second ‘if you think you didn’t hear it’. In the second stage subjects wtere asked to press one of three keys tc indicate how confident they were that their response was

INDIVIDUAL DIFFERENCES

IN SIGNAL DETECTION

45

correct. One key indicated that they were sure, the second that they were almost sure and the third that they were not sure. There were thus six possible responses: (1) present: sure (3) present: not sure (5) absent: almost sure

(2) present: almost sure (4) absent: not sure (6) absent: sure

In a preliminary procedure thresholds were measured using the method of limits. Three ascending and, three descending thresholds were recorded. The stimulus intensity was then set at 1 db above the average threshold and there was a period of practice using the rating method. The aim during this practice period uas to obtain a preliminary estimate of d”. The stimulus intensity was therefore adjusted as necessary to achieve this object with a series of ten presentations of the stimulus and ten presentations with no stimulus. Another light signal was used to indicate to the subject whenever she made a mistake. The practice series was continued until there had been a run of twenty responses using the same stimulus level which gave a distributi,on of responses in the different categories which could be used for calcu’ating d’. From this result a final adjustment of stimulus intensity was madie, aimed at producing a final value of n’ of about 1.5. This adjustment was based upon an experimentally determined function relating d’ with stirrulus intensity (EGAN et al., 1959). Using this intensity, and after a short rest period, the main series of observations was started. This consisted of forty presentations of the stimulus and forty intervals without the stimulus. There was random allocation of times and stimulus conditions during the first half of each main series and this was repeated during the second half. The whole procedure was repeated on the following day. There were twenty-two female subjects, most of them student mxses aged eighteen. RESULTS Independence

of bius and discriminatior~

Each series of observations was divided into two halves and dete’ction indices were calculated for each half separately. As a sample, table 1 shciws the number of responses,in each category given by trrle individual in one half session. Models like that in fig. 1 can be fitted to data of this kind if we assume that each response category corresponds to a different position of the criterion along the horizontal axis. Taking any line

3. G. INGHAM

46

,scparating two response categories, say 2 and 3 in table 1, all categories to the left of the line can be combined. Similarly, those to the right of the line can be combined, the result being a 2 x 2 contingency table from which the proportions of hits and false positives, or m‘isses and correct negatives, can be calculated. From these proportions the corresponding values of L”,,and zen can be found in tables of the normal distribution (in practice, it is more convenient to use probits). TABLE1 Frequency of responsesin each category. Example from one subjectin one half-session. Response 1

Sound 2 3

SignaJpresent 8 Signalabsent 0

5 4

1 3

4

No sound 5 6

4 7

2 5

0 I

To see if the indices d’ and e are orthogonal in the inter-individual space itisnecessary to find the ratio ssz2/sn2(where sn2 and sen2are the variances of the observed z values.) These ratios are shown in table 2 for each half of each session. Three things are clear from this tab!e: (1) The ratio an2/sD2is greater than one during the first half of session I indicating tha,t d’ and c are not orthogonal. (2’)The ratio tends to decline in the later series of observations. (3)id’ and c are virtually orthogonal in the first half of session II for all response categories except one. TABLE 2

Varianceratios &/s& 1 Session I

Ist half 2nd half !kssion II 1st half 2nd half

1.30 1.04* 1.14* 2.62.

Responsecategory 4 2 3 2.18 1.74’ 0.93 1.91”

2.97 2.10 0.81 1.59

4.82 3.19 1.03 1.41

5 5.25 3.22 2.19’ 1.63’

*It’<: 20. It was not always possible to calculate values for all response categories.

An example of what the variance ratios mean in terms of correlations between indices is shown in table 3. These figures are for response category 3. They confirm the orthogonality of d’ and c in the first half of

INDIVIDUAL DIFFERENCES IN SIGNAL DETECTION

47

session II and show that there is a correlation between c/’ and J on both occasions. TABLE3 Correlations between indices. Response category 3 4

2 Session I 1st half Sessian II 1st half Consistency

within

d’,c d’,J d’, c d’,J

-0.38 d-o.30 +0.05 -t 0.27

-0.58 -0.78 -t 0.10 $0.15

4.66 -0.87 4.02 -4I.65

sessions

Product moment correlations between indices obtained in the first and second halves of each session are shown in table 4. Each value of d’ was averaged over all response categories whereas values of,8 and c”were those for response category 3. Had it been possible to use the same stimulus intensity for all subiects, then the inter-individual variability of d’ would have been greater. Presumably the variability of d’ between half-sessions would have remained the same, however, so that the intra-session reliability coefficients would have been higher than those stated in table 4. It can, therefore, be said that the figures for d’ underestimate the true intra-session reliability. TABLE4 Correlations between half sessions. Session 1

d’ pl C

-+0.86 -i-O.55 t-O.82

II +0.67 3-0.71 3-0.71

These results for intra-session consistency provide rxsonably good evidence that d’ and c are suffciently stable over s5ort periods to make feasible their use in studies cf individual differences. Some doubt remains about J because ot’ the rather low figure for session I. Consistency

bet ween sessions

was often found to be necessary during initial practice in session II, to use a different stimulus intensity from that found appropriate in It

J. G. INGRAM

48

session I, in order to bring the value of d’ closer to the same arbitrary level for all subjects. The effect was to reduce inter-individual variability, to increase variability between sessions and in consequence, to reduce the correlation between sessions. The correlations between d” values for different sessions are shown in the first column of table 5 and are very low. On the assumption that the function of d’ against signal-to-noi% ratio is a straight line passing through the origin (c.f. EGANet al., 1959) it is possible to calculate what d’ would have been using the same signal level for both sessions. Correlations between the corrected values (d’tarr,) are shown in the second column of table 5. Although the assumption on which the correction is based may be suspect, the fact that d’COBr. seems more stable than d’ from session to session suggests that the experimenter was indeed reducing inter-session consistency by adjusting the stimulus intensity on session II. The correlations for both d’ and d’corr. are of course, underestimates for the same reasons that those relating to d’ in table 4 were said to be underestimated. TABLE 5

Correlations between sessions. d 1st Half 2nd Half

--0.04 +0.22

d’corr. +0.62 +0.(52

j?

c

+0.17 -i-O.44

+ 0.45 -t-0.59

.Also in table 5 are the correlations between sessions fat the two bias indices.- These figures are clearly less than those for d’corr. and it is not possible to argue in this instance that they underestimate the true values. It should be remembered, however, that values of# and c were those for response category 3 only. They are thus based upon less experimental information than are the d’ values which were averaged over as many response categories as possible, DISCUS?iION It

is impossible to know how much is contributed by variations within subjects to the overall variances used in calculating the ratios of table 2. The inter-individual variances, however, have been reduced by setting the stimulus intensity at a level appropriate to each individual’s sensitivity an,d it may he supposed that the variance ratios do reflect intra-individual variance ratios to quite a large extent. lif this is accepted, then the two main trends shown in table 2 can be explained.

INDIVIDUAL

DIFFERENCES

IN SIGNAL

DETECTION

49

The subject requires experience of the stimulus conditions, and knowledge of whether her responses are right or wrong, to generate the distributions N and SN. Knowledge of the Ncondition will already have been acquired before the experiment because everyone has experience of listening in quiet backgrounds for faint sounds that may or may not be there. Thus, from the beginning, the subject already has the information required for fixing the criterion with reference to N. She does not have the information required for fixing the criterion with reference to SN however. She will rarely, if ever, have heard that particular sound before. One implication of this is that Zn will not vary as d’ changes in one individual, in the early stages of observation, On the other hand zsn will vary with d’. The total variance sRtl2in a group of individuals will therefore include a component which is absent from sllz. As the experiment proceeds, however, the subject gains information concerning the SN distribution and is able to take this into account in fixing her criterion level. The result is that the ratio of the two variances changes, Ssn2tending to become relatively smaller in the later observations. Column I in table 2 corresponds to the positive response ‘I am sure the sound is present’ with all other responses regarded as negative. If the subject assumes that all positive responses at this level of confidence are correct (as indeed most of them are) then she will soon acquire the information necessary to fix this criterion with reference to SN. This is the equivalent of providing the subject with knowledge of results throughout the whole session instead of only during the initial practice series. On the other hand, for the response ‘I am sure the sound is absent’ the subject will acquire no additional information about SN. In general, therefore, subjects will acquire earlier, the information required to fix the criterion in relation to SN, for those criteria corresponding to the left hand columns in table 2. The ratios in table 2 will become smaller in the left hand columns at an earlier stage in the observations. If this suggestion is correct, then Zn may be invariant over different values of d’ in the early stages of observation, before the subiect has acquired enough information to generate the SN distribution. Here is one way, therefore, to seek a bias index uncontaminated with d’. What is required now is more information about variations within individuals. The interpretation which has been suggested could be investigated by using several different signal intensities for each individual. If the interpretation is correct, then without knowledge of results (required to gencrate the SN distri ution) Zn should remain relativelly constant for dif-

50

J. G.

INGMAM

ferent signal intensities whereas zan should reflect intensity changes rather closely. With knowledge of results on the other hand, zn, as well as Zen, shsuld change with intensity.

(,l) Meaningful estimates of auditory sensitivity and bias have been obtained using previously untrained subjects, in sessions lasting about one hour including preliminary observations and practice. (2) As an index of individual differences in sensitivity, d’ corrected for variations in signal intensity, was reasonably consistent both within and between sessions. (3) As an index of individual diF’r:nces in bias, 1 showed too much variation between sessions and even within sessions was not very consistent. A more consistent index of ‘bias was c, the sum of the normal deviates zn and zan. (4) Neither of the bias indices examined was orthogonal to d’ throughout both sessions. However, c approximated more closely to this than J and was virtually orthogonal to d’ during the first half of session II.

REFERENCES

EGAN, J. P., A. I. SCHULMAN and G. Z. G~ERO,

1959. Operating cilitracteristics determined by binary decisions arrd by ratings. J. acoust. Sot: Amer. 31, 76l3-773. GRANGER, G. W., 1960. Abnormalities of wsory perception. In: H. J. Eysenck ted.), Handbook of abnormal psychology. London: Pitman. GREEN,D. M. and J; A. SWETS,1966. Signal detection theory and psychophysics. New York: Wiley. HUNT, 1. McV., 1944. Personality and the tehaviour disorders. New York: Ronald Press. LUCE,R. D., 1959. Individual choice behaviour. New York: Wiley. t 1963. Detection and reccJgnition. In: R. 0. Lute, R. R. Bush and E. Gaksnter (eds.), Handbook of mat;?ematical psychology. New York: Wiley. MARKOWTTZ, 1. and J. A. SWETS,1967. Fact&s affecting the slope of empirical ROC curves: Comparison of binary andqating responses. Percept. & Psychophys. t r, 91-97. OIDELAM, P. D., 1%2. A note on the analysij of repeated measurements of the same subws. J. &on. Dis. 1$969-977. SWEIS. J. A. (ed.), 1964. Signal detection an:d recognition by human observers. NEW York: Wiley. ZUBIN~J., 1957. Experimental abnormal psyc~~ology. New York: Columbia University