Eye Movements and Response Times for the Detection of Line Orientation During Visual Search

EYE MOVEMENTS A N D RESPONSE TIMES FOR THE DETECTION OF LINE ORIENTATION DURING VISUAL SEARCH S. Mannan, K.H. Ruddock and J.R. Wright Dept. Physics (Biophysics), Imperial College, L o n d o n SW7 2BZ, UK.

Abstract We have examined visual search performance made in response to a line target which differs in orientation from the reference or distractor elements. The eye movements made by the subject in the course of the experiments were monitored continuously with an infra-red device. Search performance was assessed by measurement of the manual response time, T1/2', required to register detection of the target, and of the ocular response time, TO, taken to foveate the target. We also measured the number of saccades made in achieving fixation of the target. Our choice of visual stimulus was guided by the empirical model for pre-attentive detection of line orientation, derived from their experimental data by Foster and Ward (1991 a,b). The principal aim of the experiments was to test the applicability of their model under stimulus conditions which differed markedly from those used to obtain their experimental data. We show that our observations are not consistent with the predictions of the model and we examine the reasons for the divergence between the results of our experimental study and the performance of the empirical model.

Keywords Visual search, Line orientation, Pre-attentive vision, Parallel processing.

Introduction Visual search experiments are designed to investigate detection of a target in the presence of multiple non-target elements, k n o w n variously as distractors or reference elements. Detectability depends on the nature of the target and on the nature and n u m b e r of the reference elements, and is usually measured in terms of the response time required for signal detection. In simple search tasks, with the target distinguished by a single parametric difference from a set of identical reference elements, response time is independent of the n u m b e r of distractors, and is achieved by parallel processing of the stimulus (Neisser, 1963; Egeth et al., 1972; Shiffrin and Gardner, 1972; Treisman, 1988). U n d e r other conditions, including conjunction of different parameters, such as colour and orientation, limitingly small differences between the target and the reference elements, and the presence of more than one class of reference elements, parallel search is frequently not observed (Treisman and Gelade, 1980; Egeth et al., 1984; Treisman and Souther, 1985; Javadnia and Ruddock, 1988; Alkhateeb et al., 1990 a, b). In such cases, response times increase as the n u m b e r of reference elements increases, giving rise to so-called serial search. The dichotomy between serial and parallel search has been modelled in

Eye Movement Research/J.M. Findlay et al. (Editors) 9 1995 Elsevier Science B.V. All rights reserved.

338

S. Mannan, K.H. Ruddock & J.R. Wright

terms of successive stages of visual processing, involving initial automatic detection of changes in a given image feature, such as colour, and subsequent localisation and conjunction of different features, such as colour and orientation, through which image specification is achieved (Treisman, 1988; Treisman and Gormican, 1988; Cave and Wolfe, 1990). The distinction between parallel and serial search is not, however, always clear cut (Egeth et al., 1984; Kleiss and Lane, 1986) and Duncan and Humpheys (1989) have analysed visual search without invoking this classification of responses. Other authors have attempted to identify the properties of the mechanisms involved in target detection. Binello et al. (1993a) have determined, experimentally, the parametric band-widths, discrimination capacities and specificities of visual channels involved in several search tasks. Foster and Ward (1991a,b) have derived a two filter model which predicts their experimental observations on pre-attentive detection of line orientation. Their proposed detection mechanisms (Fig. lb) are tuned to near vertical and near horizontal directions, and both have band-widths at half maximum sensitivity of approximately 30 ~ Alkhateeb et al. (1990a) performed experiments to determine response times for detection of line orientation in the presence of heterogeneous reference elements. Their data are illustrated by Fig. 2, in which manual response times, T1/2, for detection of a target line, embedded in reference elements orientated in one of two alternative directions, are plotted against the number of reference elements, N. Values are given for three different pairs of reference orientations, and in each case the target element was orientated so that its angular direction lay approximately midway between those of the two classes of reference elements. Comparison values are given for fields containing only one of the two classes of reference elements. The T1/2 values for measurements with a single class of reference elements are always independent of N, that is, parallel processing occurs. The introduction of a second class of reference elements orthogonal to the first has little effect on the response patterns, except that T~/2 is increased slightly. With a smaller angular difference between the two classes of reference elements, however, T~/2 values increase as N increases and become significantly greater that those measured with either class of reference elements alone. These results demonstrate that line orientation detection mechanisms have sensitivity ranges extending some 30 ~ either side of the target orientation. The broad-band characteristics of Foster and Ward's (1991a) filters are, therefore, consistent with the experimental data obtained by Alkhateeb et al. (1990a), even though the latter's experimental techniques were very different from Foster and Ward's. Although line orientation is a particularly efficient stimulus for the generation of parallel search, similar responses are observed with other stimulus parameters such as orientation of 2-D elements, magnification and flicker frequency (Alkhateeb et al., 1990a; Binello et al. 1993a). Foster and Ward's model may, therefore, have broad applicability in the interpretation of parallel, or pre-attentive, visual search. The principal aim of the study to be described was to test their model against the results of visual search experiments involving the detection of line orientation. Response times provide a restricted description of visual search performance and in order to investigate more closely performance in the search experiments, we have traced the eye-movements made by our subjects during their observation of the

Detection of line orientation during visual search

339

search stimuli. In a previous study, we showed that the ocular search time, that is, the time taken to achieve fixation of the target, is closely correlated with the traditional, manual, search time recorded by pressing a response button (Binello et al., 1993b). Other characteristics of the eye movements, such as the number of saccades made in response to a given target are, however, dependent on the nature of the stimulus and reveal more complex response patterns. We include in our results data for the ocular response times and the number of saccadic eye movements made in response to the visual stimulus. METHODS

Equipment Stimuli were generated on a high resolution, Hewlett-Packard display screen (1280 x 1024 pixels), viewed from 1.5m to give a field approximately 13 deg. horizontally by 10 deg. vertically. The display was controlled by a Hewlett-Packard processor (Vectra RS/25c) which also stored and processed the eye movement signals. The latter were measured with an infra-red eye tracking device, with a 20ms temporal resolution limit, the P-scan system (Barbur et al.,1987).

Stimulus patterns The stimulus patterns consisted of a single target embedded in a selected number, N, of reference elements which were either identical or were divided into two subgroups of N / 2 identical elements. The screen locations of all elements were randomised between each presentation, with the restriction that all elements were non-overlapping. The elements were green (C.I.E. 2 ~ chromaticity co-ordinates x =0.293; y=0.607; luminance 79 cd m -2 ), and were presented against a dark background (luminance 0.1 cd m -2 ). The combinations of target and reference elements examined in these experiments are illustrated in Fig. l a. Each experiment comprised 100 stimulus presentations, 50 with a target and 50 without, presented in a random sequence. The stimuli were selected with reference to the response characteristics of the filters described by Foster and Ward (1991a), as is illustrated in Fig. lb. According to Foster and Ward's (1991a) model, the line orientations corresponding to the maximum filter responses are +4 ~ and -83 ~ and those corresponding to equal filter responses are +58 ~ and -50 ~ (see Fig. 1). In order to test the validity of the model under our altered experimental conditions (that is, longer stimulus durations), we investigated the properties of line orientation discrimination tasks in which these key orientations were used for background and target line elements. We set up paradigms where the model predicted clear differences between data for experiments using one and two classes of background elements, or predicted no difference, and compared our results to the model's predictions.

Experimental Procedure During the experimental measurements the subject's head was held steady by clamping it at the temples and supporting it with a chin-rest on a fixed mount. Each presentation commenced with the subject's eye fixated for ls on a small cross at the centre of the screen, and during this time, the position of the eye was calibrated. The

340

S. Mannan, K.H. Ruddock & J.R. Wright

cross disappeared and 60 ms later the stimulus was presented for a fixed time. The subject was instructed to press a response button as soon as the target was detected and, additionally, was asked to fixate as quickly as possible on the target and to maintain fixation until the stimulus disappeared. The central cross then re-appeared and was fixated prior to the presentation of the next stimulus. The duration of each stimulus presentation depended on the difficulty of the search task and was set long enough to ensure that detection could be achieved without causing discomfort for the subject, who had to suppress blinks during stimulus presentation. The distance between the centre of rotation of the eyeball and the pupil centre was calibrated at the start of each experiment.

Data analysis The eye movements were analysed off-line by computer, using an algorithm written to identify fixations and to extract other features of the traces. Fixations were defined as a period of 60 ms or greater during which the trace was localised to an area <0.5 ~ Traces including blinks and saccades between two consecutive fixations which overlapped spatially were both excluded from the analysis. Occasionally, the subject failed to maintain fixation on the target and returned gaze to the centre of the screen before the stimulus disappeared, and these final saccades were also discounted. The ocular response time, To, was defined as the mean time from stimulus onset required to fixate the target to within 0.5 ~. Response times for detection of the t a r g e t , T1/2, was the mean time required to press the response button. The spread of the values given in the figures correspond to + 2 standard errors. The dead-time for manual responses was just under 200 ms. Any experiment in which there occurred greater that 10% erroneous responses (false positives and negatives) was discarded.

Subjects Data are given for two female subjects, both aged 24 years, who are authors. Both had normal 6/5 Snellen visual acuity and some preliminary data (Fig. 2), taken from Alkhateeb et al. (1990a), are given for another female subject who also possessed normal visual acuity.

Results Data for two classes of reference elements orientated at +58 ~ and -50 ~ to the vertical, both of which provide equal input to the two filters of Foster and Ward's (1991a) model (Fig. lb), are given in Fig.3 together with data for a single class of reference elements, orientated at-50 ~ The manual response times, T1/2, the ocular response times,T0, and the probabilities of fixating the target with fewer than two, or fewer than three saccades are all plotted against N, and values are given for two subjects. Each subject responds similarly, with longer response times for the stimuli which contain two classes of reference elements, and correspondingly lower probabilities of target fixation of a given number of saccades. The differences in the response times recorded with two stimulus configurations for subject JW are, in most cases, not significant. Data for a target orientated at-50 ~ and for two classes of

341

Detection of line orientation during visual search

a~

Target

Reference

/

Target

Reference

\

/

/

N

/

/

\

/

\ Fig. 4

Fig. 3

/

bw

1.0

p

o~

%

rj) I %

I %

0.0

~

I ~

t

% 9 .~'

i

_ L .,p l/~

9O

0

%

~r

/

-90 e (deg.)

Figure I a) The combinations of reference and target elements used in the experiments, data for which are given in Figures 3 and 4, as noted. When two classes of reference elements were used, there were equal numbers of each. The lines were 1 o in length. b) The relative sensitivities, S, of the two filters proposed by Foster and Ward (1991a), expressed as a function of q, the angular orientation of the line elements, q is measured in degrees from the vertical, positive clockwise and negative anticlockwise. The lines above the sensitivity functions show the orientations of lines used in our experiments, one each at the angles for peak response of each filter, and two at the angles at which the filter responses intersect, to give equal responses.

S. Mannan, K.H. Ruddock & J.R. Wright

342

reference elements, one orientated along each of the directions for peak response of Foster and Ward's (1991a) two filters (+4 ~ and -83~ are given in Fig.4, together with comparison data for only the +4 ~ reference elements. The response times for the mixed reference elements are greater than those for the single class, and the probabilities for detection with a given number of saccades are lower, but the differences between the two types of stimuli are less marked than for the data of Fig. 3. Also plotted in Fig.4 are data measured with the same target, but with a single class of reference elements, orientated at +58 ~, so that it provides equal input to the two filters. These are generally similar to those obtained with reference elements orientated at +4 ~.

1.0-a

'

'

~ .~0

,-

..... ,

/b,

I

'-

'

_o-

~

I

I

I

8

,

,

i

'F

,

;--

4

0

0

I

C

20

40 N

~T4"'MI

0

-" "I

" " - I" -'~ I

2 0 4 0

N

Figure 2 Response times, T1/2, for detection of a line target plotted against the number of reference elements, N. a). Target orientation 45 ~," reference orientations 0 ~ and 90 ~ (open circles), or 0 ~ (full circles) b). Target orientation 30 ~,"reference orientations 0 ~ and 60 ~ (open circles), or 0 ~ (full circles) c). Target orientation 22 ~; reference orientations 0 ~and 40 ~ (open circles) or 0 ~ (full circles). Subject W A (after Alkhateeb et al., 1990a).

343

Detection of line orientation during visual search

8M 0.8

I

JW

I

I

I

I

I

I

I

8.

0.

0.8 A v

D0

0.4, ~

0.0

b

R

.__

_.@

I

I

I

m

I

I

I

I

I

I

I

I

I

I

I

I

I

I

I

!

I

I

I

I

I

I

!

I

I

I

I

I

b

0.6

A

0.4

0.0

0.6

_

O2 V

/ 0.4

0

/O

o

. - - ...-@

/

/

_ t~..~--~ I

_

I

I A

:1..0

d A

o

0.0

-o/

O3

v

V

0.8

0.4

0

I EO

i 40

N

0

I 20

I 40

N

Figure 3 Response times T1/2 and To and the probabilities P(<2) and P(<3) of fixating the target in fewer than two saccades and fewer than three saccades respectively, plotted against N, the number of reference elements. Open circles; target orientation +4 ~ reference orientation +58 ~ and-50~ full circles; target orientation +4 o, reference orientation -50 ~ Error bars show + 2 standard errors. Data for SM (left column) and JW (right column)

S. Mannan, K.H. Ruddock & J.R. Wright

344

8M 0.8

I

I

I

I

a

I

I

i

I

r~

0.4

0.8

I

I

I

I

I

I

I

I

I

I

I

I

I

I

I

I

b

b A

0.4

0.8

i

I

I

I

i

I

I

I

I

I

I

I

I

I

I

I

I

I

e A

v D,,

0.6

/

-

i

/

i

/--

t

/

0.4-

,/

..........--o I

I

I

I

I

I

iii

v

0.6 /

0.4

0

i

i

20

40 N

~

0

i

20 H

Figure 4 As Fig. 3, but for different line orientations. Open circles; target orientation - 50 ~ reference orientations +4 ~and-83 o. Full circles; target orientation -50 o, reference orientation +4 ~ dotted lines, target orientation -50 ~ reference orientation +58 o.

345

Detection of line orientation during visual search

SM 0.4

a

i

i

I

I

I

I

I

I

I

I

I

I

I

I

I

0.0

0.4

b

~~ o.o <1 I

0.4

I

I -

c

~N a ~

-

-

C

0.2 o~ V

v

0.0

-

v

-0.;~

/

i :

"-

-$

_

i/// ,,,,.

/

I

I

/

I

I

0.4

0.~ ~'

~"

0.0

i/.-.-

-0.2

_

-0.4 I

0

I

20

40

N

i

0

I..

20

40

N

Figure 5 Difference values taken between the two sets of data points in Fig. 3 (open circles), and between the open and full circles of Fig. 4 (full circles) and between the full circles and the points joined by the dotted lines of Fig. 4 (triangles).

346

S. Mannan, K.H. Ruddock & J.R. Wright

Discussion

The data of Fig. 2, taken from Alkhateeb et al. (1990a), demonstrate that the presence of reference elements orientated orthogonally to each other (vertically and horizontally) has relatively little effect on manual response times compared to values for vertical reference elements alone. When two classes of reference elements differ in their orientations by 60 ~ or 40 ~, however, response times increase significantly and search becomes serial rather than parallel. Alkhateeb et al. argued that the detection mechanisms for line orientation must respond over an angular range of at least --+30 ~ around the direction of the target orientation, in order to be influenced by the presence of two class of reference elements which differ in orientation by 60 ~. This compares well with the + 30 ~ half-widths of Foster and Ward's (1991a) filters, which suggests that the same mechanisms may be active in both detection tasks, despite the many differences in stimulus conditions. In the new experiments reported by us, the orientations of the line elements were chosen to be +4 ~ or -83 ~, such that they stimulate only one or other of the two filters, or + 58 ~ and -50 ~ such that they stimulate both filters equally. Thus, the two classes of reference elements used to obtain the data of Figure 3 both provide equal inputs to the two filters and should be entirely equivalent to each other. According to the model, substitution of one class of reference elements for the other should not affect the responses, but experimentally, response times are longer with the two classes of reference elements than with one (Fig. 3a,b). Correspondingly, the probabilities of achieving target fixation with less than three saccades are greater for the single class of reference elements (Fig. 3c,d). The elements of the mixed reference field used to obtain data presented in Fig. 4 (open circles) are orientated so that they provide input to only one or other of the filters, whereas the target stimulates both equally. Removal of one of the two classes of references elements should, therefore, facilitate target detection by the filter which no longer receives an input from the reference elements. The experimental data show the expected effects (Fig. 4) as the response times for the mixed reference elements (Fig.4a,b open circles) are greater than those obtained when one class is removed (Fig. 4, a.b., full circles) and the probabilities of detection within a given number of saccades are fewer (Fig. 4c,d). For SM, however, the differences (Fig. 5, full circles) are smaller those found in the previous experiment and for JW, the differences in the manual response times, T 1/2 are consistently greater than the corresponding values found in the previous experiment. The final comparison is made between two sets of data for the same target (orientated at 50~ with the reference elements orientated either at +4 ~ (full circles, Figure 4) or at +58 ~ (dotted lines, Figure 4). The model predicts that detection of the target should be easier in the former case because one of the filters receives inputs from the target, but not from the reference elements. In practice, the differences between the two sets of data (Fig. 5, crosses) are inconsistent, and often zero. In summary, two of the three experiments used to test the model yield results contrary to its predictions, there being consistent differences between results for a pair of stimulus configurations for which there should be no difference (Fig. 3, Fig. 5 open circles) and no consistent differences between those for a pair which should yield them (Fig. 4, full circles and dotted lines; Fig. 5 crosses). Our results show that

Detection of line orientation during visual search

347

detection is always more difficult with two classes of reference elements than with one, regardless of relative orientations. This is not predicted by Foster and Ward's model, but is consistent with the subjective impression that a reference field containing two classes of elements is more 'cluttered', thereby rendering target detection more difficult. Foster and Ward (1991a) used a 20 ~ x 20 ~ field, with the target restricted to an annulus between 3 ~ and 8 ~ from the central fixation, and stimuli were presented for 40ms, followed by 60 ms inter stimulus interval and a 500 ms masking field. The target in our experiments, however, could lie anywhere within the 13 ~ x 10 ~ field, and the stimulus was visible for a fixed period of some 2s to 3s. The former arrangement elicits strictly pre-attentive vision whereas in the latter, the subject is aware of the content of the visual images. We believe that as a consequence, mechanisms other than those involved in the early, pre-attentive stages of vision contribute to the responses we observe, and cause our data to diverge from those predicted on the basis of Foster and Ward's model.

Acknowledgement We are particularly grateful to Professor D.H. Foster, of the University of Keele, for his advice on the design of the experiments, and for general discussions. This work was supported by a grant awarded to KHR by the Defence Research Agency (No C B / R A E / 9 / 4 / 2 0 3 7 / 3 8 4 / R A R D E ) , and JW was supported by a Prize Research Studentship awarded by The Wellcome Trust.

References Alkhateeb W.F., Morris R.J. and Ruddock K.H. (1990a) Effects of stimulus complexity on simple spatial discriminations. Spatial Vision 5, 129-141. Alkhateeb W.F., Morland A.B., Ruddock K.H. and Savage C.J. (1990b). Spatial, colour and contrast response characteristics of mechanisms which mediate discrimination of pattern orientation and magnification. Spatial Vision 5, 143-157. Barbur J.L., Forsyth P.M. and Thompson D. (1987). A new system for the simultaneous measurement of pupil size and two-dimensional eye-movements. Clin. Vision Sci. 2, 131-142. Binello A, Mannan S. and Ruddock K.H. (1993a). Control of visual search by different stimulus parameters: the hierarchical organisation of processing. Visual Search 3 (ed. A. Gale and K. Carr). In press. Binello A., Mannan S. and Ruddock K.H. (1993b) Eye movements in visual search performed with multi-element fields. Visual Search 3 (ed A. Gale and K. Carr) In press. Cave K.R. and Wolfe J.M. (1990) Modeling the role of parallel processing and visual search. Cognitive Psychol. 22, 225-271. Duncan J. and Humphreys G.W. (1989). Visual search and stimulus similarity. Psychol. Rev. 96, 433-458.

348

S. Mannan, K.H. Ruddock & J.R. Wright

Egeth H., Jonides J.and Wall S. (1972). Parallel processing of multielement displays. Cog. Psychol. 3, 674-698. Egeth H., Virzi R.A. and Garbart H. (1984). Searching for conjunctively defined targets. J. Exp. Psychol. Human Percept. Perform. 10, 32-39. Foster D.H. and Ward P.A. (1991a) Asymmetries in oriented-line detection indicate two orthogonal filters in early vision. Proc. Roy. Soc. London B, 243, 75-81. Foster D.H. and Ward P.A. (1991b). Horizontal-vertical filters in early vision predict anomalous line orientation identification frequencies. Proc. Roy. Soc. London B, 243, 83-86. Javadnia, A. and Ruddock, K.H. (1988). The limits of parallel processing in the visual discrimination of orientation and magnification. Spatial Vision 3, 97-114. Kleiss, J.A. and Lane, D.M. (1986). Locus and persistence of capacity limitations in visual information processing. J. Exp. Psychol: Human Percept. Perform. 12, 200210. Neisser, U. (1963). Decision time without reaction-time. Experiments in visual scanning. Am. J. Psychol. 76, 376-385. Shiffrin, R.M. and Gardner, G.T. (1972). Visual processing capacity and attentional control. J. Exp. Psychol., 93, 72-83. Treisman. A. (1988). Features and objects. The fourteenth Bartlett memorial leture. Q.J. Exp. Psychol. 40A, 201-237. Treisman A. and Gelade, G. (1980). A feature-integration theory of attention. Cog. Psychol. 12, 97-136. Treisman A. and Gormican, S. (1988). Feature analysis in early vision; Evidence from search asymmetries. Psychol. Rev. 95, 15-48. Treisman A. and Souther, J. (1985). Search asymmetry; A diagnostic for preattentive processing of separable features. J. Exp. Psychol. Gen. 114, 285-310.