An application of sequential analysis to observer-based psychophysics

An application of sequential analysis to observer-based psychophysics

INFANT BEHAVIOR AND DEVELOPMENT 15, 271-277 (1992) BRIEF REPORT An Application of Sequential Analysis to Observer-Based Psychophysics HOWARD ...

468KB Sizes 2 Downloads 32 Views

INFANT

BEHAVIOR

AND

DEVELOPMENT

15,

271-277

(1992)

BRIEF REPORT

An Application of Sequential Analysis to Observer-Based Psychophysics HOWARD

S. HOFFMAN

Bryn Mawr

College

Sequential analysis, a statistical technique developed during World War II, is shown to hove direct applications to certain observer-based psychophysical procedures. This article shows how o particular sequential onalysis procedure (the sequential probability ratio test] can be used to collect and analyze the data generated during abserver-based psychophysics. It also provides on indication of the savings in numbers of observations (about 50%) that can be expected when sequential analysis is used in this way.

statistical

analysis

observer

infant judgment

psychophysics methodology

Olsho, Koch, Halpin, and Carter (1987) have described a variety of observerbased psychophysical procedures and have provided examples of their use in the auditory assessment of young children. This article presents an application of sequential analysis to observer-based psychophysics that can, in principle, reduce the number of observations in half with no loss in statistical rigor. Consider the psychophysical problem of determining whether or not an infant can hear a given tone. With an observer-based psychophysical procedure, the tone in question would be presented to the infant on some trials, and it would be withheld on others. Alternatively, the tone might be presented in one of two locations on each of a number of trials. In either case, an observer (who is naive to the nature of the trials) would watch the infant’s reactions and, depending upon what had been seen, report a judgment as to the stimulus condition on each trial. With this procedure, the infant is said to have heard (or at least reacted to) the tone when it can be concluded that the observer’s judgments were correct on more trials than would be expected on the basis of chance.

the

The work author. Correspondence

of Psychology,

on

which

this and

Dalton

article

requests Hall,

was based for reprints

Bryn

Mawr

was supported should College,

by NIH

be sent Bryn

Grant

to Howard

Mawr,

PA

HD

10511

S. Hoffman,

directed

by

Department

19010.

271

272

HOFFMAN

The usual way to do this is to conduct a post-hoc test of the statistical hypothesis that the observer’s performance represents a random sample from a population in which the probability of a correct report on any given trial is .5. As will be shown, sequential analysis can be used to simplify and shorten this procedure while maintaining the same level of statistical rigor that otherwise would have occurred. With the most commonly used statistical procedures, the sample size (N) must be established prior to the collection of data. Sequential analysis, on the other hand, does not require that sample size be determined before the collection of data begins. With a sequential analysis, observations (i.e., the observers’ reports) are collected one at a time. After each observation, the investigator makes certain calculations and then decides to do one of three things: 1. reject the statistical hypothesis tested, 2. accept that hypothesis, or 3. collect an additional observation. The particular sequential analysis procedure that is directly applicable to observer-based psychophysics is called the sequential probability ratio test. It is one of several sequential tests that were devised by Abraham Wald and his. colleagues when Wald was Professor of Mathematics at Columbia University. Much of this work occurred during World War II in the early 1940s. The theory of sequential analysis was later elaborated by Wald in a book that was published posthumously (Wald, 1947). More recently, several of its applications have been described in the well-known statistical text by Dixon and Massey ( 1969). To illustrate the application of sequential analysis to observer-based psychophysics, a brief experiment was performed in the author’s laboratory. A sequential probability ratio test was used to determine whether or not each of several student volunteers was reacting to a tone. In previous work (Marsh, Hoffman, & Stitt, 1978), it had been found that with both infants and adults, the presentation of a mild tone just prior to an eyeblink-eliciting event can inhibit the blink that would otherwise occur. The procedures for the present illustration were derived from that study. On each of a series of trials a miniature solenoid delivered a controlled eyeblink-eliciting tap to a subject’s glabella (the flat region of skin between the eyebrows). On half the trials (randomly selected), a brief tone was presented 100 ms prior to the tap; on the rest, the tap was presented alone. An observer, who was naive to the order of conditions, was required to visually assess the subject’s reactions and decide, on each trial, whether or not a tone had been presented. The events on each trial were determined by an investigator who was positioned out of the view of both the subject and the observer. The investigator operated the specialized electronic equipment that delivered the tones and

OBSERVER-BASED

PSYCHOPHYSICS

273

taps, and determined the sequence of trial types. The observer sat in front of the subject and initiated each trial by pressing a silent hand-held switch. To avoid acoustic cues that might enable the observer to determine whether or not a tone was presented on a given trial, tones were delivered to the subject through close-fitting earphones, and the observer wore sound-attenuating hearing protectors. To insure that, except for the presence or absence of a tone, trials would be identical, taps were always delivered 100 ms after the observer closed the hand-held switch, whether a tone was presented or not. The observer’s task was to watch each elicited eyeblink and (based on its perceived amplitude) decide whether or not a tone had been presented. A small blink would imply that a tone had been presented, whereas a large blink would imply that the tap had occurred without a prior reflex-inhibiting tone. In addition to setting up the condition for each trial, the investigator recorded the observer’s decisions and, depending upon whether a given decision was correct or incorrect, entered it appropriately on the chart shown in Figure 1 (p. 274). This chart is a basic feature of the sequential probability ratio test, and its parameters are especially suited to examples of observer-based psychophysics that are similar to the illustration described here. The chart enables an investigator to rapidly record data as it comes in and to conduct the sequential analysis on-line. It frees the investigator of the task of performing calculations and is applicable to any procedure that requires an observer to use a subject’s reactions to discriminate between two types of trials. It is included here for the convenience of those investigators who might wish to duplicate it and assess its applicability to their own research. With the chart in Figure 1, the result of each observation is recorded as a point that is one unit up, if the observer’s decision is correct, or as a point that is one unit to the right, if the observer’s decision is incorrect. The observer’s decisions are entered as they occur, and the procedure terminates when a point contacts or crosses one of the two slanted parallel lines in the chart. These lines define the critical regions for a sequential probability ratio test of the statistical hypothesis (HO) that p = 5, against the alternative hypothesis (H,) that p = .75. The hypothesis tested (HO) asserts that each of the observer’s decisions represents a randomly selected item from a population in which the probability of a correct decision is 5. This is equivalent to asserting that the observer is operating at a chance level. The alternative hypothesis (H,) asserts that each of the observer’s decisions is a randomly selected item from a population in which the probability of a correct decision is .75. This is equivalent to asserting that the perceived amplitudes of the subjects’ blinks have enabled the observer to function at a better than chance level. The specific value, HI = .75, was chosen because a number of preliminary calculations showed that with HO = .5, the sequential probability ratio test

274

HOFFMAN

0 Figure 1. A chart in an observer-based

that can be used psychophysical

5

10

15 20 MISSES

25

30

to conduct a sequential probability procedure (see text for detoils).

35 ratio

test of the data

obtoined

would be more efficient than with any other value of HI between .5 and 1.0. Besides, .75 would correspond to the way that the psychophysical threshold is defined when the chance level is 5. With the lines placed as in Figure 1, H,, will be rejected and H, will be accepted with (Y = .Ol when a point contacts or crosses Line 1. In this context, a is the probability that the test will reject the null hypothesis that p = .5, when in fact that hypothesis is true. Alternatively, if a point contacts or crosses Line 2, H0 will be accepted, and H, will be rejected with p = .05. In this context, f3 = (1 - Power) = the probability that the hypothesis tested (H,) will be accepted when the alternative hypothesis (H,) is true. The formulae from which the parameters of the lines in Figure 1 were calculated are provided by Dixon and Massey (1969). It is noteworthy that the values of (Yand p in Figure 1 represent a balance between the risks and potential effects of the two kinds of errors that are possible in the present circumstances. On the one hand, the test could lead to the decision that a subject has processed the tone, when in fact the subject was unable to do so. To minimize this risk, (Y was set equal to .Ol. On the other hand, the test could wrongly lead to the conclusion that the subject was unable to react to the tone. Because (depending upon the population tested

OBSERVER-BASED

PSYCHOPHYSICS

275

and the circumstances of the test) this could imply that a subject was deaf, and because the prudent course of action in such circumstances would be to retest the subject, p was set to only .05. Figure 2 (p. 276) shows the results of several test runs. For each of the test runs, the tones had a frequency of 1000 Hz, a duration of 50 ms, and a rise/fall time of 10 ms. For the tests described by Charts a, b, and c, tone intensity was 80 dB (re: .0002 dynes/cm*). For Chart d, tone intensity was 40 dB (re: .0002 dynes/cm*). These examples indicate that the sequential probability ratio test provided a rapid and efficient way to decide if a given subject was reacting to the tones. In Charts a, b, and c, the point crosses (or contacts) Line 1. Accordingly, the hypothesis that the observer was operating at a chance level is rejected with CL= .Ol. For this reason it can be concluded that on these runs, the subjects reacted to the tones. This, it should be noted, is a reasonable conclusion. Although the tone used in these runs was itself too weak to elicit an observable (or measurable) overt reaction, they were nonetheless able to visibly inhibit the eyeblink to the tap that would otherwise have occurred. In Chart d, the point crosses Line 2, indicating that the observer’s judgments were no better than would be expected on the basis of chance. More specifically, the sequence of observer judgments was sufficiently in error to permit acceptance of the statistical hypothesis that they represent a random sequence from a population in which the probability of a correct response was 5. This, incidently, seems a reasonable conclusion, because the intensity of the tones (40 dB) was probably near, if not below, the subject’s threshold for hearing in the open room where the run was conducted. As a result, the tones were probably too weak to appreciably inhibit the responses to the taps. Figure 3 (p. 277) shows the operating characteristic for the sequential probability ratio test provided by the chart in Figure 1. The function in Figure 3 is based on calculations from formulae that are provided in Dixon and Massey (1969). As suggested in the earlier discussions and as shown in Figure 3, the test depicted in Figure 1 seems especially well suited to the circumstances that are encountered during observer-based psychophysics. When the true probability is 5, the probability that the test will accept the statistical hypothesis tested (i.e., p = .5) is equal to 1 - (Y = .99. As the true value of p increases toward 1, the probability of accepting Ho (and rejecting Hi) drops rapidly until it is .05 (when the true probability is .75). Thereafter, it drops slowly to 0. This clearly is the way a statistical test should operate in observer-based psychophysics. It should enable the investigator to reject the null hypothesis when that hypothesis is incorrect, and it should lead to acceptance of the null hypothesis (and rejection of the alternative hypothesis) when the null hypothesis is true. As was demonstrated by the results of the empirical tests shown in Figure 2, the sequential probability ratio test properly deals with the null hypothesis and, moreover, it does so in relatively few observations. In assessing the

HOFFMAN

0

5

10

15

20

23

50

i5

0

5

lb

MISSES

15

2’0

2’5

3b

2il

2‘5

3.0

MISSES

d

40

10

0 0

5

10

15

26

‘Iti

30

3s

0

5

MISSES Figure 2. Some empirical tests of the application of sequential analysis physics. In each case, the test ended when a point entered a rejection two diagonal lines.

10

15 MISSES

to observer-based region by crossing

psychoone of the

3‘5

OBSERVER-BASED

.5

Figure

3. The operating

characteristic

PSYCHOPHYSICS

277

.6 .7 .8 .9 TRUE VALUE OF P of the

sequential

probability

ratio

1.0

test shown

in Figure

1

savings engendered by the sequential probability ratio test, Wald (1947) showed that, at a ,given level of (Y and p, the average sample size for the sequential probability ratio test is approximately one half that of the best available test of a normally distributed variate (the 2 test for a proportion) that requires a fixed sample size. More specifically, Wald showed that when compared to the 2 test and with the parameters used here (CY= .Ol, l3 = .05), the sequential probability ratio test yields a 47% saving in sample size when the alternative hypothesis (H,) is true, and it yields a 63% saving in sample size when the null hypothesis (H,) is true. These considerations plus the ease of its use suggest that the sequential probability ratio test provided by Figure 1 is likely to have wide applications to observer-based psychophysics. Obviously, its use need not be restricted to audition. It can also be used in studies of vision or of any other sensory modality. REFERENCES Dixon,

W.J., & McGraw-Hill.

Marsh,

R.R., objective L.W., acoustic

Olsho, Wald,

A.

(1947).

Massy,

F.J.

(1969).

lnrroducrion

to sfafistical

analysis

(3rd

ed.).

New

York:

Hoffman, H.S., & Stitt, C.L. (1978). Reflex inhibition audiometry: A new technique. Actu Otofuryngologicu, 85. 336341. Koch, E.G., Halpin, CF., & Carter, E.A. (1987). An observer-based psychoprocedure for use with young infants. Developmental Psychology, 23, 627-640.

Sequential

analysis.

New

York:

Wiley. 11 October

1990;

Revised

14 May

1991

H