- Email: [email protected]
The role of eye movements in motion detection
V&ionRes. Vol. 27, No. 5, pp. 141-754, 1907 Printed in Great Britain.All rights reserved
0042-6989/87 S3.00 + 0.00 Copyright Q 1987 Pcrgamon JournalsLtd
THE ROLE OF EYE MOVEMENTS
IN MOTION
DETECTION
ULKER TULUNAY-KEESEYand JAMBSN. VERHOEVE Department of Ophthalmology, 569 Waisman Center, University of Wisconsin, Madison, WI 53705, U.S.A. (Received 9 April 1986; in revisedform 15 October 1986)
Abstract-The roles of small eye movements of fixation, and of different kinds of background in motion detection were studied. Minimum detectable displacement for a luminous line oscillating either in a blank field or in the presence of three types of background was measured under two viewing conditions: normal, when eye movements generate normal movements of the image on the retina, and stabilized when these retinal image movements were nearly eliminated. It was demonstrated that eye movements enhance motion detection for a sinusoidally moving target when the target is superimposed on a patterned background; they are detrimental when there is no background. In addition, it was found that the function relating threshold amplitude to frequency of movement is band-pass when the image is stabilii or when the bar moves on a blank field, and is more low-pass when both the background and the test target are subject to the effects of eye movements. Motion detection
Eye movements
Image stabilization
INTRODUCTION
The human visual system detects motion with exquisite accuracy; lateral displacements as small as 5-10 set arc which are well within the intercone distance, can be apprehended under many conditions (Tyler and Torres, 1972; Legge and Campbell, 1981; Nakayama and Tyler, 1978, 1981; Nalcayama, 1981, 1985; McKee and Nakayama, 1984). Whether this hyper-acuity is based on a mechanism which infers motion through a change in position, or through velocity detection, has been frequently debated (Leibowitz, 1955; and see Nakayama, 1985 for a review). In the presence of position cues, the amplitude of the minimum detectable displacement decreases for oscillating bars, suggesting that a position detection mechanism may also play an important role in motion detection (Tyler and Torres, 1972; Legge and Campbell, 198 1). Conversely, minimal displacement sensitivity is better than spatial resolution (Legge and Campbell, 1981), it varies with velocity of displacement (Tyler and Torres, 1972; Nakayama and Tyler, 1978; McKee and Nakayama, 1984) and motion sensitivity remains impervious to variables that degrade positional acuities (Nakayama, 1981; McKee and Nakayama, 1984); all of which suggest that motion may be detected by a mechanism funda-
mentally different from that responsible for spatial acuities. Still another aspect of motion detection which is different from position detection is its relation to eye movements. Stabilizing the retinal image against the effects of eye movements does not change vernier acuity (Keesey, 1960), neither does imposed motion of velocities as high as 3 deg/sec (Westheimer and McKee, 1975). Conversely, there is speculation (Tyler and Torres, 1972; Legge and Campbell, 1981; Nakayama, 1981 and Buckingham and Whitaker, 1985) as well as indirect evidence (Nakayama, 198 1) that motion detection lacks such tolerance to retinal image motion. Eye movements which occur during fixation convert the image of a steady target into an image that moves on the retina. Any motion superimposed on the target has to be detected over this natural, eye-movement engendered retinal motion of the target image. This is a strong argument for the adverse influence of retinal motion on sensitivity to target motion. But, efforts to control for eye movements in motion detection studies have been limited to controlling the length of target presentation during which different amount of eye movements occur (Nakayama, 1981). The present study was designed to determine directly the role of normal retinal image motion 747
ULICER TULUNAY-KEESEY
748
in motion detection. Retinal image movements were minimized by optical stabilization described previously (Keesey, 1960), and motion sensitivity was compared under stabilized and normal conditions. The effect of position cues was assessed as well, by determining motion thresholds on backgrounds with differing cues. A sinusoidally moving line was chosen as the target following a paradigm first used by Tyler and Torres (1972) and Nakayama and Tyler (1978). This target is advantageous in dehneating motion judgements made on the basis of velocity alone from judgements based on position cues. Sinusoids with inversely related amplitude vs frequency components have a common peak velocity. If sensitivity to motion is limited by a threshold velocity, then the psychophysi~l function relating threshold amplitude to frequency of sinusoidal oscillation should have a slope of - 1 (see Nakayama and Tyler, 1978). If on the other hand, a motion detection depends on amplitude of displacement alone, threshold amplitude should be independent of frequency, and the slope should be close to zero. The argument is that in the first case a motion mechanism, and in the second case a position detection mechanism subserves motion detection. METHODS
stimulus, a 1” long, 3 min of arc wide, luminous, vertically oriented line, was oscillated sinusoidally in the horizontal direction with frequencies of 0.5, 1,2,4, 6, 8, 12 and 16 Hz on four different backgrounds. One was a blank background, the second was also blank but contained four thin reference lines placed in an implied cross-hair configuration with a 1.5” free central area. The third and fourth backgrounds contained patterns upon which the moving line was superimposed. one was a resolvable static random dot pattern, and the other a 7 c/deg static vertically oriented cosine grating. The contrast of either of the patterns was 50%. The average luminance was 15 cd/m*, the line luminance was always 30 cd/m2. There were three experimental conditions: (1) normal test and stabilized back~ound, (2) stabilized test and normal background, and (3) normal test and normal background. Motion was superimposed only on the test line under either the normal or the stabilized viewing condition. The optical lever method of stabilization (Keesey, 1960) afforded simultaneous viewing of The
and
JAMES
N. VERHOEVE
normal and stable targets, but the condition where a stabilized line moved against a stabilized background could not be obtained. Motion was superimposed on an otherwise stabilized target by oscillating the target image reflected from a small galvanometer mirror mounted very close to the eye. The accuracy of image stabilization was measured by an afterimage tracking procedure first described by Riggs and Schick (1968). and was found to be 6 to 15 arc set for the subjects who participated in the experiment. During the experiment the line and the background were always present. In a given trial which was 5 set in duration and marked by auditory signals, the line either moved with a given amplitude or was steady. The observer indicated his decision by a button press. A staircase method with four reversals was used to obtain amplitude thresholds for each frequency, no feedback was given. An average of 20 iterations produced such a threshold. Each data point represents the average of three estimates. To prevent fading, the subjects were instructed to move their heads slightly in between trials. They started the trials after ensuring that the whole target, i.e. the background and the test line, was clearly visible, and that stabilization was good. The adequacy of stabilization before each trial was assessed by fixating at various parts of the field. If it was good the line moved to the point of fixation. In the case where the back~ound was a blank without references, the subjects made use of the faint outline of one of the lenses. However, after the initial adjustment at the start of the session, no further adjustments were found to be necessary to maintain good stabilization. One experienced (R.P.), and one inexperienced observer (M.P.), served as subjects. A full set of data was obtained from subject R.P.; subject M.P. provided a check on selected points. RESULTS
The results from both subjects are similar: in general, the amplitude necessary to just detect motion decreases as a function of frequency of oscillation to a broad minimum, increases in frequency above 8-10 Hz serve to increase the threshold ampitude. Both the absolute values of the threshold and the shape of the motion sensitivity function, particularly over the low frequencies are affected by both the background and eye movements.
749
Eye movements in motion detection STABILIZED
NORMAL 30
I
.3
I
3
IO
FREQUENCY (Hz )
.3
I
3
IO
FREQUENCY (Hz 1
Fig. 1. Threshold amplitude to detect sinusoidal motion of a thin luminous line as a function of frequency. Xs denote blank field; squares, references; circles, random dot backgrounds; and triangles, grating background. Panels A and B for subject R.P. and M.P.; normal viewing. Panels C and D stabilized viewing for R.P. and M.P. respectively.
The data were analysed in subsets, straight lines were fit to log threshold amplitude as a function of log frequency for frequencies 4 Hz and above 6 Hz. These values were chosen as cut-off points for fitting, because visual inspection showed that between these values there was either a minimum or the start of a broad transition. Table 1 contains the slopes and the estimates of the goodness-of-fit for all the data. Eflect of background
Figure 1 summarizes the effect of the background. Panels A for subject R.P. and B for subject M.P. the results obtained under normal conditions when images of both the target and the background move normally on the retina, Panels C and D show the effect of background when either the line target or the background is stabilized. An inspection of panels A and B suggest that when the image moves normally on the retina, the blank field yields the highest threshold values for the low frequencies. Threshold amplitudes converge to approximately the same minimum value of 4-5 arc set for all backgrounds for subject M.P. But for subject R.P. the minimum values vary between 5 and 2 arc
set according to the pattern of the background, the blank field yielding the highest, and the random dot pattern the lowest values. Low frequency attenuation is steep for both subjects when the line moves in a blank field; the slope of the function between 0.5 and 4 Hz, is -0.86 and -0.77 for R.P. and M.P. respectively. The presence of reference lines reduces this slope to between -0.3 and -0.5 for both subjects. The structured background has a similar effect of reducing the slope over the low frequencies, producing a low-pass shape function (Table 1). As pointed out in the Introduction, a slope of - 1 for a sinusoidally moving stimulus such as the one used here implies that detection of movement is based on velocity alone. Therefore, a slope of -0.8 or higher, as obtained for a line moving in a blank field without any references, may imply of a maximum velocity in the target. This reciprocal relation between threshold amplitude of displacement and frequency of displacement does not hold however, when reference lines are provided in the otherwise blank field, or when the moving target is superimposed on a structured, patterned background. In these cases, slopes of -0.5 or lower may imply that
750
ULKER TULUNAY-KEFSEYand JAMES N. VERHOEVE
judgment of motion becomes less dependent on the detection of a constant maximum velocity, and relatively more dependent on the detection of the amplitude of displacement. Motion sensitivity for high frequencies (> 6 Hz) of oscillation does not seem to reflect the nature of the background, whether it is blank or patterned. According to M.P.‘s data, threshold displacement amplitudes to detect motion increase more steeply over high frequencies when the line oscillates in a blank field than on patterned backgrounds; but R.P.‘s data do not support this observation, threshold amplitudes increase the same way regardless of the background. In addition, a straight line fit to the data for frequencies above 6 Hz is not good (see Table 1). This is due to a relatively flat middle portion of the curve, the broad minimum, which in some cases extends to 10Hz. Does this relative independence of thresholds from frequency mean that motion judgments depend on amplitude alone at these high frequencies? * Tyler and Torres (1972) had suggested that motion sensitivity to high frequency oscillations of 10 Hz or higher may be determined in a fashion similar to flicker detection. Indeed, they report that at these high frequencies of oscillation the target appears as a flickering bar of variable width. In our case, subjectively, the bar also appeared to be changing width rather than moving at oscillation frequencies above 12 Hz. It should be noted that the observers were asked, in an informal session, to distinguish a stationary but flickering line from a line oscillating at the same frequency. An oscillating line of frequencies up to 10 Hz was always correctly identified as moving when the threshold amplitude for movement was reached, and a flickering line could always be identified as stationary but flickering at modulation amplitudes where flicker could be detected. However, at oscillation frequencies of 12 Hz and above the criteria of distinction from flicker was a change in width rather than movement. If motion sensitivity to high frequencies of oscillation is governed by the flicker detection mechanism as proposed here, then insensitivity to high frequency oscillations may be due to the visual system’s attenuation of sensitivity to high frequency flicker independent of the spatial parameters of the field. The flat portion of the motion sensitivity curves, i.e. the broad minimum between 4 and 10 Hz may represent a transition zone between a motion detecting mechanism
Eye
751
movements in motion detection
based either on velocity or amplitude, and the flicker detection mechanism. An observation which would favor this interpretation is that high frequency flicker sensitivity attenuation starts around between 5 and 10 Hz in small fovea1 fields and is independent of the spatial structure (Keesey, 1970). Panels C and D of Fig. 1 illustrate the effect of the background when the line target was stabilized but motion was superposed on it. Similar results were obtained when the background was stabilized and the line target was subject to the effect of eye movements. It appears that the main effect of the background, that of increasing motion sensitivity for low frequencies of oscillation, is lost under image stabilization. In addition, eliminating the normal retinal movements of the target (or background) image, results in an increase of low frequency attenuation, especially for the structured backgrounds. As an extreme example, for subject R.P., in normal view, the random dot background condition yields a slope of -0.47; stabilizing the line increases the slope to - 1.14. (Table 1) With the exception of two cases, the slopes for the low frequency portion of the curves all background conditions vary between -0.7 and - 1.14. This would indicate that
under stabilization the visual system detects motion mainly on the basis of velocity regardless of the background. The exceptions are for subject R.P., for the blank field with and without references. Here the slopes over the low frequency range are about -0.5, due to aberrant points for 4 Hz (blank with references) and possibly 1 Hz (blank no references). If these points are ignored, the slopes over the low frequencies become - 1 and -0.7 respectively. As it is under normal viewing, high frequency attention under image stabilization is not related to the pattern of the background in any particular way. EfSect of eye movements In Fig. 2 (A-D) the data were replotted to emphasize the effect of eye movements in motion detection. Here, a direct comparison of amplitude thresholds in normal and stabilized vision is provided. Panels A and B show data obtained with the patterned backgrounds. In R.P.‘s case, panel A, the background was stabilized, and the moving line was subject to the effect of eye movements. In M.P.‘s case, panel B, the target line was stabilized and the background image moved normally on the retina. Under normal conditions, eye movements
MP I
“““’
I
.3
I
3
FREOUENCY (Hz
IO
1
.3
I
3
FREOUENCY
IO
(Hz)
Fig. 2. (A, B) Threshold amplitude for motion detection under normal (solid symbols and lines) and stabilized (open symbols, dotted lines) viewing conditions; target superimposed on the grating (triangles) and random dot (circles) background. For R.P. and M.P. respectively. (C, D) Threshold amplitude for motion detection for normal (solid symbols and linea) and stabilized (open symbols and dashed lines) vision, target moves in blank field. For R.P. and M.P. respectively.
ULKER TULUNAY-KEESEY and JAMES N. VERHOEW
152
impart retinal motion whih is common both to the background and the moving target. These graphs emphasize that this common motion has the effect of decreasing the amplitude necessary to detect low frequency oscillations, as well as decreasing the slope relating amplitude to frequency, producing a low-pass type function. The question arises of whether eye movements remain facilitatory when they do not impart common motion to the background and the target, i.e. when imposed target motion has to be detected over the retinal motion of the target image. In panels C and D data are presented comparing the threshold amplitude functions obtained on blank backgrounds with the stabilized and the normally moving lines. It would appear that for both subjects, when there is no background, i.e. when eye movements cannot impart common motion to the background and the target, eye movements are not facilitatory, in fact they are detrimental for motion detection. In addition, contrary to the case with structured background, presence of retinal image motion does not alter the shape of the curve. The slopes of these curves over the low frequency range range from -0.7 to -0.9 for both subjects (Table 1; compare NormalBlank NR to stabilized Blank NR). Again, the exception is RP’s stabilized condition, where the aberrant point at 1 Hz decreases the slope. Between 0.5 and 2 Hz the slope is -0.7. In Fig. 3 the results are plotted in terms of velocity needed for motion detection (V = 2piAS, where A = amplitude and f = frequency of the sine wave). The random dot pattern and the blank background condi-
tions with the normally moving and stabilized lines are shown for R.P. in panel A; the blank and grating background with the normally moving and stabilized lines are shown for M.P. in panel B. Under conditions of normal image motion, velocity (log) at which motion is detected increases linearly with (log) frequency with a slope of 0.9 for both subjects. Stabilizing the test target (or the background) against the effects of eye movements or removing the background, causes a change in this slope reducing it close to 0 for frequencies below 2 Hz. This indicates that a minimum velocity, independent of frequency has to be reached for motion to be detected under these conditions. Note however, that the slope remains 0.9 for frequencies higher than 4 Hz for all backgrounds and viewing conditions, emphasizing the possibility of two mechanisms subserving motion detection (Nakayama, 1985). One would operate for frequencies below 2 Hz, and be influenced by both the background and image motion, the other would operate above 4 Hz, and be independent of the background and image motion. There are two ancillary observations: (1) both subjects reported that the patterned background did not disappear during the five seconds the judgement was made, (2) both subjects reported that when either the line or the background was stabilized, the line seemed to be moving in three dimensional space. SUMMARY AND DISCUSSION
In this study, the roles of small eye movements and of backgrounds were evaluated on
A _
1000
51
r ; -
---( .--a 4
blonh norm blonh stab. qrotinq stab. qrattnq norm
B ---.s ---.a
blank norm blank rtob. mndam dot stab.
< : :: 2 c
100
J !3 5
FREQUENCY (Hz)
Fig. 3. Velocity (set arc/se@ to detect sinusoidal motion. Solid symbols and lines for normal, open symbols and dotted lines for stabilized vision. Xs depict for blank background; circles, random dot; and triangles the grating background. Panel A: subject R.P.; Panel B, Subject M.P.
Eye movements in the
displacement thresholds of a sinusoldally moving luminous line. It was found that both factors, the background and the small movements of the eye affect motion detection profoundly. The effect of background is manifested in two ways: (1) in the presence of a background, such as random dots or gratings, threshold amplitude of displacement to detect motion is reduced relative to the situation where the line oscillates in a blank background. This is true for low frequencies of oscillation up to about 4 Hz. (2) Threshold amplitude is reciprocally related to these low frequencies (slopes close to - 1) when the target oscillates in a blank field, but is proportional to frequency (slopes close to -0.5) over the same range when it moves on a structured, patterned field. It is as if motion detection is achieved by a system which responds to maximum velocity when motion takes place in a blank field, but that motion detection is achieved by a mixture of position and velocity sensitive systems when motion takes place in an environment that supplies relative position cues. Background has essentially no effect for sensitivity to high frequencies of oscillation. Eye movements affect motion detection in two, seemingly contradictory, ways: (1) when image motion engendered by eye movements is eliminated the threshold reducing, sensitivity enhancing effect of the patterned background is also eliminated. In other words, retinal image motion has a facilitatory effect for low frequencies when the target moves on structured backgrounds. (2) Conversely, over the same frequency range, stabilization yields the lower thresholds, i.e. retinal image motion has a detrimental effect when the target moves on a blank background. Under image stabilization, motion sensitivity curves remain band-pass; the slope of the functions relating amplitude to frequency of oscillation below 4 Hz is close to - 1 regardless of the background, blank or patterned. Considering that the presence of retinal image motion reduces the slope to -0.5, it is as if retinal motion forces a transition from a velocity to a position detection mechanism for detecting motion. Although any number of current models of motion detection may be used to analyze the results, none of them formally take into consideration the effect of small eye movements, especially in a way that would account for their contradictory effect in the presence and absence
motion detection
753
of backgrounds. Many tif the current models of motion detection depend on the comparison of the output of band-limited spatio-temporal filters early in the visual system (van Santen and Sperling, 1984; Watson and Ahumada, 1984; Adelson and Bergen, 1983, see also Regan and Beverly, 1981). To account for our results a neural network which acts as a differential amplifier and which precedes the spatiotemporal detection filter is proposed. In this model the spatio-temporal composition of the incoming stimulus is modified by the fixational eye movements. In a motion detection situation, it is proposed that one of the inputs to the comparator is analogous to the signal generated by the patterned background, and the other to the signal generated by the moving line. When both the background and the bar target are subject to the effects of eye movements, both inputs have a common input supplied by eye movements, and this common mode signal is rejected, effectively amplifying only the signal associated by the imposed sinusoidal motion of the line. Since the energy of eye movements is concentrated below 3 Hz, (St. Cyr and Fender, 1969; Tulunay-Keesey and Baker, 1982), targets moving sinusoidally with lower frequencies are especially affected because they are within the operating range of the system. This may also explained why eye movements, their presence or absence, does not affect minimum amplitude displacement for lines oscillating with frequencies exceeding 4 Hz. In the case of the stabilized line moving on a normal background, or the normal line moving on a blank field, the eye movement-generated signals are uncorrelated, are treated as noise, and the movement signal has to be increased in amplitude in order to be detected. Motion sensitivity to high frequencies of oscillation which seems to be independent of the background or image motion, may simply reflect attenuation exercised by the spatio-temporal filter. This model would predict that if both the target and the background were stabilized, and sinusoidal motion su~~rnpos~ on the bar alone, motion detection function would be of the low-pass type obtained when the neither the background or the target was stabilized. One of the important implications of this model is that in situations where there is a patterned background, it is not necessary to postulate a positional mechanism for the accuracy of motion detection over low frequencies. It can still be argued that motion detection is a
754
ULKER TULUNAY-KEESEYand Jams
special case of relative position acuity; patterned backgrounds yield the lower displacement amplitude thresholds; and stabilized conditions simply remove position cues through image fading. This would explain the facilitatory effect of image motion in the presence of the background, but not its detrimental effect on blank fields. In addition, the high contrast patterned structured backgrounds did not fade during the 5 set the judgements were made. One of the subjective phenomena is worth reiterating: in cases where the line target was stabilized either on a blank field or the structured background, the subjects reported a very strong impression of motion in depth, emphasizing the importance of uncorrelated signals in monocular depth perception (Nakayama, 1985). Note: While this paper was being revised a paper by Bender and Davidson appeared in Bruin Research (1986). They report that the neurons in the superficial layers of the superior colliculus in the monkey are nonselective for target direction and speed when the target moves through any empty visual field. However, the same cells were sensitive to target direction and speed relative to a textured moving background; the response was suppressed maximally when the direction and speed of the target matched that of the background. This is analogous to the normal viewing condition in our experimental situation, when the background and the target move with the same speed and direction due to small eye movements. The suppression of the cells response would be analogous to the “common mode rejection” principle proposed here. It would appear the response of the system to eye movement generated common image motion is suppressed (rejected) effectively amplifying the response to the imposed target motion. Acknowledgemenu-The work was completed during the tenure of an NE1 grant and partially supported by NSF 84-18827. Core support of the Waisman Center from NICHD 2P30 HD03352 is gratefully acknowledged. I am thankful for the many hours M.P. and R.P. spent as subjects and the valuable discussions with Dr J. Dannemiller. Informed consent was obtained from the two subjects after the nature and possible consequences of the procedures were fully explained.
N. VERHOEW
REFERENCES Adelson E. H. and Bergen J. R. (1985) Spatiotemporal energy models for the perception of motion. J. opr. Sot. Am. AL, 284-299. Bender D. B. and Davidson, R. M. (1986) Global visual processing in the monkey superior colliculus. Brain Res. 381, 372-375. Buckingham T. and Whitaker D. (1985) The influence of luminance on displacement thresholds for continuous oscillatory movement. Vision Res. 25, 1675-1677. Keesey U. (1960) Effects of involuntary eye movements on visual acuity. J. opt. Sot. Am. SO, 769-774. Keese.y U. (1970) Variables determining flicker sensitivity in small fields. J. opf. Sot. Am. 60, 390-398. Legge G. and Campbell F. W. (1981) Displacement detection in human vision. Vision Res. 21, 205-214. Leibowitz H. B. (1955) The relation between the rate threshold for the perception of movement and luminance for various durations of exposure. J. exp. Psycho/. 49, 209-214. McKee S. P. and Nakayama K. (1984) The detection of motion in the peripheral visual field. Vision Res. 24, 25-32. Nakayama K. (1985) Biological image motion processing: a review. Vision Rex 25, 625-660. Nakayama K. (1981) Differential motion hyperacuity under conditions of common image motion. Vision Res. 21, 1475-1482. Nakayama K. and Tyler C. W. (1981) Psychophysical isolation of movement sensitivity by removal of familar position cues. Vision Res. 21, 427-433. Nakayama K. and Tyler C. W. (1978) Relative motion induced between stationary lines. Vision Res. 18, 1663-1668. Regan D. and Beverly K. I. (1981) Motion sensitivity measured by a psychophysical linearizing technique. J. opt. Sot. Am. 71, 958-965. Riggs L. A. and Schick A. M. L. (1968) Accuracy of image stabilization achieved with a plane mirror on a tightly fitting contact lens. Vision Res. 8, 159-169. StCyr G. J. and Fender D. H. (1969) Non-linear&s of the human oculomotor system: Gain. Vision Res. 9, 1235-1246. Tulunau-Keesey U. and Baker E. (1982) Eye movements: Stabilized and normal viewing. Inuesr. Ophthol. visual Sci., Suppl. 82, 49. Tyler C. W. and Torres J. (1972) Frequency response characteristics for sinusoidal movement in the fovea and periphery. Percept. Psychophys. 12, 232-236. van Santen J. P. H. and Sperling G. (1985) Elaborated Reichardt detectors. J. opt. Sot. Am. AZ, 300-321. Watson A. B. and Ahumada A. J. (1985) Model of human motion sensing. 1. opt. Sot. Am. A2, 322-342. Westheimer G. and McKee S. P. (1975) Visual acuity in the presence of retinal image motion. J. opt. Sot. Am. 66, 847-850.
0042-6989/87 S3.00 + 0.00 Copyright Q 1987 Pcrgamon JournalsLtd
THE ROLE OF EYE MOVEMENTS
IN MOTION
DETECTION
ULKER TULUNAY-KEESEYand JAMBSN. VERHOEVE Department of Ophthalmology, 569 Waisman Center, University of Wisconsin, Madison, WI 53705, U.S.A. (Received 9 April 1986; in revisedform 15 October 1986)
Abstract-The roles of small eye movements of fixation, and of different kinds of background in motion detection were studied. Minimum detectable displacement for a luminous line oscillating either in a blank field or in the presence of three types of background was measured under two viewing conditions: normal, when eye movements generate normal movements of the image on the retina, and stabilized when these retinal image movements were nearly eliminated. It was demonstrated that eye movements enhance motion detection for a sinusoidally moving target when the target is superimposed on a patterned background; they are detrimental when there is no background. In addition, it was found that the function relating threshold amplitude to frequency of movement is band-pass when the image is stabilii or when the bar moves on a blank field, and is more low-pass when both the background and the test target are subject to the effects of eye movements. Motion detection
Eye movements
Image stabilization
INTRODUCTION
The human visual system detects motion with exquisite accuracy; lateral displacements as small as 5-10 set arc which are well within the intercone distance, can be apprehended under many conditions (Tyler and Torres, 1972; Legge and Campbell, 1981; Nakayama and Tyler, 1978, 1981; Nalcayama, 1981, 1985; McKee and Nakayama, 1984). Whether this hyper-acuity is based on a mechanism which infers motion through a change in position, or through velocity detection, has been frequently debated (Leibowitz, 1955; and see Nakayama, 1985 for a review). In the presence of position cues, the amplitude of the minimum detectable displacement decreases for oscillating bars, suggesting that a position detection mechanism may also play an important role in motion detection (Tyler and Torres, 1972; Legge and Campbell, 198 1). Conversely, minimal displacement sensitivity is better than spatial resolution (Legge and Campbell, 1981), it varies with velocity of displacement (Tyler and Torres, 1972; Nakayama and Tyler, 1978; McKee and Nakayama, 1984) and motion sensitivity remains impervious to variables that degrade positional acuities (Nakayama, 1981; McKee and Nakayama, 1984); all of which suggest that motion may be detected by a mechanism funda-
mentally different from that responsible for spatial acuities. Still another aspect of motion detection which is different from position detection is its relation to eye movements. Stabilizing the retinal image against the effects of eye movements does not change vernier acuity (Keesey, 1960), neither does imposed motion of velocities as high as 3 deg/sec (Westheimer and McKee, 1975). Conversely, there is speculation (Tyler and Torres, 1972; Legge and Campbell, 1981; Nakayama, 1981 and Buckingham and Whitaker, 1985) as well as indirect evidence (Nakayama, 198 1) that motion detection lacks such tolerance to retinal image motion. Eye movements which occur during fixation convert the image of a steady target into an image that moves on the retina. Any motion superimposed on the target has to be detected over this natural, eye-movement engendered retinal motion of the target image. This is a strong argument for the adverse influence of retinal motion on sensitivity to target motion. But, efforts to control for eye movements in motion detection studies have been limited to controlling the length of target presentation during which different amount of eye movements occur (Nakayama, 1981). The present study was designed to determine directly the role of normal retinal image motion 747
ULICER TULUNAY-KEESEY
748
in motion detection. Retinal image movements were minimized by optical stabilization described previously (Keesey, 1960), and motion sensitivity was compared under stabilized and normal conditions. The effect of position cues was assessed as well, by determining motion thresholds on backgrounds with differing cues. A sinusoidally moving line was chosen as the target following a paradigm first used by Tyler and Torres (1972) and Nakayama and Tyler (1978). This target is advantageous in dehneating motion judgements made on the basis of velocity alone from judgements based on position cues. Sinusoids with inversely related amplitude vs frequency components have a common peak velocity. If sensitivity to motion is limited by a threshold velocity, then the psychophysi~l function relating threshold amplitude to frequency of sinusoidal oscillation should have a slope of - 1 (see Nakayama and Tyler, 1978). If on the other hand, a motion detection depends on amplitude of displacement alone, threshold amplitude should be independent of frequency, and the slope should be close to zero. The argument is that in the first case a motion mechanism, and in the second case a position detection mechanism subserves motion detection. METHODS
stimulus, a 1” long, 3 min of arc wide, luminous, vertically oriented line, was oscillated sinusoidally in the horizontal direction with frequencies of 0.5, 1,2,4, 6, 8, 12 and 16 Hz on four different backgrounds. One was a blank background, the second was also blank but contained four thin reference lines placed in an implied cross-hair configuration with a 1.5” free central area. The third and fourth backgrounds contained patterns upon which the moving line was superimposed. one was a resolvable static random dot pattern, and the other a 7 c/deg static vertically oriented cosine grating. The contrast of either of the patterns was 50%. The average luminance was 15 cd/m*, the line luminance was always 30 cd/m2. There were three experimental conditions: (1) normal test and stabilized back~ound, (2) stabilized test and normal background, and (3) normal test and normal background. Motion was superimposed only on the test line under either the normal or the stabilized viewing condition. The optical lever method of stabilization (Keesey, 1960) afforded simultaneous viewing of The
and
JAMES
N. VERHOEVE
normal and stable targets, but the condition where a stabilized line moved against a stabilized background could not be obtained. Motion was superimposed on an otherwise stabilized target by oscillating the target image reflected from a small galvanometer mirror mounted very close to the eye. The accuracy of image stabilization was measured by an afterimage tracking procedure first described by Riggs and Schick (1968). and was found to be 6 to 15 arc set for the subjects who participated in the experiment. During the experiment the line and the background were always present. In a given trial which was 5 set in duration and marked by auditory signals, the line either moved with a given amplitude or was steady. The observer indicated his decision by a button press. A staircase method with four reversals was used to obtain amplitude thresholds for each frequency, no feedback was given. An average of 20 iterations produced such a threshold. Each data point represents the average of three estimates. To prevent fading, the subjects were instructed to move their heads slightly in between trials. They started the trials after ensuring that the whole target, i.e. the background and the test line, was clearly visible, and that stabilization was good. The adequacy of stabilization before each trial was assessed by fixating at various parts of the field. If it was good the line moved to the point of fixation. In the case where the back~ound was a blank without references, the subjects made use of the faint outline of one of the lenses. However, after the initial adjustment at the start of the session, no further adjustments were found to be necessary to maintain good stabilization. One experienced (R.P.), and one inexperienced observer (M.P.), served as subjects. A full set of data was obtained from subject R.P.; subject M.P. provided a check on selected points. RESULTS
The results from both subjects are similar: in general, the amplitude necessary to just detect motion decreases as a function of frequency of oscillation to a broad minimum, increases in frequency above 8-10 Hz serve to increase the threshold ampitude. Both the absolute values of the threshold and the shape of the motion sensitivity function, particularly over the low frequencies are affected by both the background and eye movements.
749
Eye movements in motion detection STABILIZED
NORMAL 30
I
.3
I
3
IO
FREQUENCY (Hz )
.3
I
3
IO
FREQUENCY (Hz 1
Fig. 1. Threshold amplitude to detect sinusoidal motion of a thin luminous line as a function of frequency. Xs denote blank field; squares, references; circles, random dot backgrounds; and triangles, grating background. Panels A and B for subject R.P. and M.P.; normal viewing. Panels C and D stabilized viewing for R.P. and M.P. respectively.
The data were analysed in subsets, straight lines were fit to log threshold amplitude as a function of log frequency for frequencies 4 Hz and above 6 Hz. These values were chosen as cut-off points for fitting, because visual inspection showed that between these values there was either a minimum or the start of a broad transition. Table 1 contains the slopes and the estimates of the goodness-of-fit for all the data. Eflect of background
Figure 1 summarizes the effect of the background. Panels A for subject R.P. and B for subject M.P. the results obtained under normal conditions when images of both the target and the background move normally on the retina, Panels C and D show the effect of background when either the line target or the background is stabilized. An inspection of panels A and B suggest that when the image moves normally on the retina, the blank field yields the highest threshold values for the low frequencies. Threshold amplitudes converge to approximately the same minimum value of 4-5 arc set for all backgrounds for subject M.P. But for subject R.P. the minimum values vary between 5 and 2 arc
set according to the pattern of the background, the blank field yielding the highest, and the random dot pattern the lowest values. Low frequency attenuation is steep for both subjects when the line moves in a blank field; the slope of the function between 0.5 and 4 Hz, is -0.86 and -0.77 for R.P. and M.P. respectively. The presence of reference lines reduces this slope to between -0.3 and -0.5 for both subjects. The structured background has a similar effect of reducing the slope over the low frequencies, producing a low-pass shape function (Table 1). As pointed out in the Introduction, a slope of - 1 for a sinusoidally moving stimulus such as the one used here implies that detection of movement is based on velocity alone. Therefore, a slope of -0.8 or higher, as obtained for a line moving in a blank field without any references, may imply of a maximum velocity in the target. This reciprocal relation between threshold amplitude of displacement and frequency of displacement does not hold however, when reference lines are provided in the otherwise blank field, or when the moving target is superimposed on a structured, patterned background. In these cases, slopes of -0.5 or lower may imply that
750
ULKER TULUNAY-KEFSEYand JAMES N. VERHOEVE
judgment of motion becomes less dependent on the detection of a constant maximum velocity, and relatively more dependent on the detection of the amplitude of displacement. Motion sensitivity for high frequencies (> 6 Hz) of oscillation does not seem to reflect the nature of the background, whether it is blank or patterned. According to M.P.‘s data, threshold displacement amplitudes to detect motion increase more steeply over high frequencies when the line oscillates in a blank field than on patterned backgrounds; but R.P.‘s data do not support this observation, threshold amplitudes increase the same way regardless of the background. In addition, a straight line fit to the data for frequencies above 6 Hz is not good (see Table 1). This is due to a relatively flat middle portion of the curve, the broad minimum, which in some cases extends to 10Hz. Does this relative independence of thresholds from frequency mean that motion judgments depend on amplitude alone at these high frequencies? * Tyler and Torres (1972) had suggested that motion sensitivity to high frequency oscillations of 10 Hz or higher may be determined in a fashion similar to flicker detection. Indeed, they report that at these high frequencies of oscillation the target appears as a flickering bar of variable width. In our case, subjectively, the bar also appeared to be changing width rather than moving at oscillation frequencies above 12 Hz. It should be noted that the observers were asked, in an informal session, to distinguish a stationary but flickering line from a line oscillating at the same frequency. An oscillating line of frequencies up to 10 Hz was always correctly identified as moving when the threshold amplitude for movement was reached, and a flickering line could always be identified as stationary but flickering at modulation amplitudes where flicker could be detected. However, at oscillation frequencies of 12 Hz and above the criteria of distinction from flicker was a change in width rather than movement. If motion sensitivity to high frequencies of oscillation is governed by the flicker detection mechanism as proposed here, then insensitivity to high frequency oscillations may be due to the visual system’s attenuation of sensitivity to high frequency flicker independent of the spatial parameters of the field. The flat portion of the motion sensitivity curves, i.e. the broad minimum between 4 and 10 Hz may represent a transition zone between a motion detecting mechanism
Eye
751
movements in motion detection
based either on velocity or amplitude, and the flicker detection mechanism. An observation which would favor this interpretation is that high frequency flicker sensitivity attenuation starts around between 5 and 10 Hz in small fovea1 fields and is independent of the spatial structure (Keesey, 1970). Panels C and D of Fig. 1 illustrate the effect of the background when the line target was stabilized but motion was superposed on it. Similar results were obtained when the background was stabilized and the line target was subject to the effect of eye movements. It appears that the main effect of the background, that of increasing motion sensitivity for low frequencies of oscillation, is lost under image stabilization. In addition, eliminating the normal retinal movements of the target (or background) image, results in an increase of low frequency attenuation, especially for the structured backgrounds. As an extreme example, for subject R.P., in normal view, the random dot background condition yields a slope of -0.47; stabilizing the line increases the slope to - 1.14. (Table 1) With the exception of two cases, the slopes for the low frequency portion of the curves all background conditions vary between -0.7 and - 1.14. This would indicate that
under stabilization the visual system detects motion mainly on the basis of velocity regardless of the background. The exceptions are for subject R.P., for the blank field with and without references. Here the slopes over the low frequency range are about -0.5, due to aberrant points for 4 Hz (blank with references) and possibly 1 Hz (blank no references). If these points are ignored, the slopes over the low frequencies become - 1 and -0.7 respectively. As it is under normal viewing, high frequency attention under image stabilization is not related to the pattern of the background in any particular way. EfSect of eye movements In Fig. 2 (A-D) the data were replotted to emphasize the effect of eye movements in motion detection. Here, a direct comparison of amplitude thresholds in normal and stabilized vision is provided. Panels A and B show data obtained with the patterned backgrounds. In R.P.‘s case, panel A, the background was stabilized, and the moving line was subject to the effect of eye movements. In M.P.‘s case, panel B, the target line was stabilized and the background image moved normally on the retina. Under normal conditions, eye movements
MP I
“““’
I
.3
I
3
FREOUENCY (Hz
IO
1
.3
I
3
FREOUENCY
IO
(Hz)
Fig. 2. (A, B) Threshold amplitude for motion detection under normal (solid symbols and lines) and stabilized (open symbols, dotted lines) viewing conditions; target superimposed on the grating (triangles) and random dot (circles) background. For R.P. and M.P. respectively. (C, D) Threshold amplitude for motion detection for normal (solid symbols and linea) and stabilized (open symbols and dashed lines) vision, target moves in blank field. For R.P. and M.P. respectively.
ULKER TULUNAY-KEESEY and JAMES N. VERHOEW
152
impart retinal motion whih is common both to the background and the moving target. These graphs emphasize that this common motion has the effect of decreasing the amplitude necessary to detect low frequency oscillations, as well as decreasing the slope relating amplitude to frequency, producing a low-pass type function. The question arises of whether eye movements remain facilitatory when they do not impart common motion to the background and the target, i.e. when imposed target motion has to be detected over the retinal motion of the target image. In panels C and D data are presented comparing the threshold amplitude functions obtained on blank backgrounds with the stabilized and the normally moving lines. It would appear that for both subjects, when there is no background, i.e. when eye movements cannot impart common motion to the background and the target, eye movements are not facilitatory, in fact they are detrimental for motion detection. In addition, contrary to the case with structured background, presence of retinal image motion does not alter the shape of the curve. The slopes of these curves over the low frequency range range from -0.7 to -0.9 for both subjects (Table 1; compare NormalBlank NR to stabilized Blank NR). Again, the exception is RP’s stabilized condition, where the aberrant point at 1 Hz decreases the slope. Between 0.5 and 2 Hz the slope is -0.7. In Fig. 3 the results are plotted in terms of velocity needed for motion detection (V = 2piAS, where A = amplitude and f = frequency of the sine wave). The random dot pattern and the blank background condi-
tions with the normally moving and stabilized lines are shown for R.P. in panel A; the blank and grating background with the normally moving and stabilized lines are shown for M.P. in panel B. Under conditions of normal image motion, velocity (log) at which motion is detected increases linearly with (log) frequency with a slope of 0.9 for both subjects. Stabilizing the test target (or the background) against the effects of eye movements or removing the background, causes a change in this slope reducing it close to 0 for frequencies below 2 Hz. This indicates that a minimum velocity, independent of frequency has to be reached for motion to be detected under these conditions. Note however, that the slope remains 0.9 for frequencies higher than 4 Hz for all backgrounds and viewing conditions, emphasizing the possibility of two mechanisms subserving motion detection (Nakayama, 1985). One would operate for frequencies below 2 Hz, and be influenced by both the background and image motion, the other would operate above 4 Hz, and be independent of the background and image motion. There are two ancillary observations: (1) both subjects reported that the patterned background did not disappear during the five seconds the judgement was made, (2) both subjects reported that when either the line or the background was stabilized, the line seemed to be moving in three dimensional space. SUMMARY AND DISCUSSION
In this study, the roles of small eye movements and of backgrounds were evaluated on
A _
1000
51
r ; -
---( .--a 4
blonh norm blonh stab. qrotinq stab. qrattnq norm
B ---.s ---.a
blank norm blank rtob. mndam dot stab.
< : :: 2 c
100
J !3 5
FREQUENCY (Hz)
Fig. 3. Velocity (set arc/se@ to detect sinusoidal motion. Solid symbols and lines for normal, open symbols and dotted lines for stabilized vision. Xs depict for blank background; circles, random dot; and triangles the grating background. Panel A: subject R.P.; Panel B, Subject M.P.
Eye movements in the
displacement thresholds of a sinusoldally moving luminous line. It was found that both factors, the background and the small movements of the eye affect motion detection profoundly. The effect of background is manifested in two ways: (1) in the presence of a background, such as random dots or gratings, threshold amplitude of displacement to detect motion is reduced relative to the situation where the line oscillates in a blank background. This is true for low frequencies of oscillation up to about 4 Hz. (2) Threshold amplitude is reciprocally related to these low frequencies (slopes close to - 1) when the target oscillates in a blank field, but is proportional to frequency (slopes close to -0.5) over the same range when it moves on a structured, patterned field. It is as if motion detection is achieved by a system which responds to maximum velocity when motion takes place in a blank field, but that motion detection is achieved by a mixture of position and velocity sensitive systems when motion takes place in an environment that supplies relative position cues. Background has essentially no effect for sensitivity to high frequencies of oscillation. Eye movements affect motion detection in two, seemingly contradictory, ways: (1) when image motion engendered by eye movements is eliminated the threshold reducing, sensitivity enhancing effect of the patterned background is also eliminated. In other words, retinal image motion has a facilitatory effect for low frequencies when the target moves on structured backgrounds. (2) Conversely, over the same frequency range, stabilization yields the lower thresholds, i.e. retinal image motion has a detrimental effect when the target moves on a blank background. Under image stabilization, motion sensitivity curves remain band-pass; the slope of the functions relating amplitude to frequency of oscillation below 4 Hz is close to - 1 regardless of the background, blank or patterned. Considering that the presence of retinal image motion reduces the slope to -0.5, it is as if retinal motion forces a transition from a velocity to a position detection mechanism for detecting motion. Although any number of current models of motion detection may be used to analyze the results, none of them formally take into consideration the effect of small eye movements, especially in a way that would account for their contradictory effect in the presence and absence
motion detection
753
of backgrounds. Many tif the current models of motion detection depend on the comparison of the output of band-limited spatio-temporal filters early in the visual system (van Santen and Sperling, 1984; Watson and Ahumada, 1984; Adelson and Bergen, 1983, see also Regan and Beverly, 1981). To account for our results a neural network which acts as a differential amplifier and which precedes the spatiotemporal detection filter is proposed. In this model the spatio-temporal composition of the incoming stimulus is modified by the fixational eye movements. In a motion detection situation, it is proposed that one of the inputs to the comparator is analogous to the signal generated by the patterned background, and the other to the signal generated by the moving line. When both the background and the bar target are subject to the effects of eye movements, both inputs have a common input supplied by eye movements, and this common mode signal is rejected, effectively amplifying only the signal associated by the imposed sinusoidal motion of the line. Since the energy of eye movements is concentrated below 3 Hz, (St. Cyr and Fender, 1969; Tulunay-Keesey and Baker, 1982), targets moving sinusoidally with lower frequencies are especially affected because they are within the operating range of the system. This may also explained why eye movements, their presence or absence, does not affect minimum amplitude displacement for lines oscillating with frequencies exceeding 4 Hz. In the case of the stabilized line moving on a normal background, or the normal line moving on a blank field, the eye movement-generated signals are uncorrelated, are treated as noise, and the movement signal has to be increased in amplitude in order to be detected. Motion sensitivity to high frequencies of oscillation which seems to be independent of the background or image motion, may simply reflect attenuation exercised by the spatio-temporal filter. This model would predict that if both the target and the background were stabilized, and sinusoidal motion su~~rnpos~ on the bar alone, motion detection function would be of the low-pass type obtained when the neither the background or the target was stabilized. One of the important implications of this model is that in situations where there is a patterned background, it is not necessary to postulate a positional mechanism for the accuracy of motion detection over low frequencies. It can still be argued that motion detection is a
754
ULKER TULUNAY-KEESEYand Jams
special case of relative position acuity; patterned backgrounds yield the lower displacement amplitude thresholds; and stabilized conditions simply remove position cues through image fading. This would explain the facilitatory effect of image motion in the presence of the background, but not its detrimental effect on blank fields. In addition, the high contrast patterned structured backgrounds did not fade during the 5 set the judgements were made. One of the subjective phenomena is worth reiterating: in cases where the line target was stabilized either on a blank field or the structured background, the subjects reported a very strong impression of motion in depth, emphasizing the importance of uncorrelated signals in monocular depth perception (Nakayama, 1985). Note: While this paper was being revised a paper by Bender and Davidson appeared in Bruin Research (1986). They report that the neurons in the superficial layers of the superior colliculus in the monkey are nonselective for target direction and speed when the target moves through any empty visual field. However, the same cells were sensitive to target direction and speed relative to a textured moving background; the response was suppressed maximally when the direction and speed of the target matched that of the background. This is analogous to the normal viewing condition in our experimental situation, when the background and the target move with the same speed and direction due to small eye movements. The suppression of the cells response would be analogous to the “common mode rejection” principle proposed here. It would appear the response of the system to eye movement generated common image motion is suppressed (rejected) effectively amplifying the response to the imposed target motion. Acknowledgemenu-The work was completed during the tenure of an NE1 grant and partially supported by NSF 84-18827. Core support of the Waisman Center from NICHD 2P30 HD03352 is gratefully acknowledged. I am thankful for the many hours M.P. and R.P. spent as subjects and the valuable discussions with Dr J. Dannemiller. Informed consent was obtained from the two subjects after the nature and possible consequences of the procedures were fully explained.
N. VERHOEW
REFERENCES Adelson E. H. and Bergen J. R. (1985) Spatiotemporal energy models for the perception of motion. J. opr. Sot. Am. AL, 284-299. Bender D. B. and Davidson, R. M. (1986) Global visual processing in the monkey superior colliculus. Brain Res. 381, 372-375. Buckingham T. and Whitaker D. (1985) The influence of luminance on displacement thresholds for continuous oscillatory movement. Vision Res. 25, 1675-1677. Keesey U. (1960) Effects of involuntary eye movements on visual acuity. J. opt. Sot. Am. SO, 769-774. Keese.y U. (1970) Variables determining flicker sensitivity in small fields. J. opf. Sot. Am. 60, 390-398. Legge G. and Campbell F. W. (1981) Displacement detection in human vision. Vision Res. 21, 205-214. Leibowitz H. B. (1955) The relation between the rate threshold for the perception of movement and luminance for various durations of exposure. J. exp. Psycho/. 49, 209-214. McKee S. P. and Nakayama K. (1984) The detection of motion in the peripheral visual field. Vision Res. 24, 25-32. Nakayama K. (1985) Biological image motion processing: a review. Vision Rex 25, 625-660. Nakayama K. (1981) Differential motion hyperacuity under conditions of common image motion. Vision Res. 21, 1475-1482. Nakayama K. and Tyler C. W. (1981) Psychophysical isolation of movement sensitivity by removal of familar position cues. Vision Res. 21, 427-433. Nakayama K. and Tyler C. W. (1978) Relative motion induced between stationary lines. Vision Res. 18, 1663-1668. Regan D. and Beverly K. I. (1981) Motion sensitivity measured by a psychophysical linearizing technique. J. opt. Sot. Am. 71, 958-965. Riggs L. A. and Schick A. M. L. (1968) Accuracy of image stabilization achieved with a plane mirror on a tightly fitting contact lens. Vision Res. 8, 159-169. StCyr G. J. and Fender D. H. (1969) Non-linear&s of the human oculomotor system: Gain. Vision Res. 9, 1235-1246. Tulunau-Keesey U. and Baker E. (1982) Eye movements: Stabilized and normal viewing. Inuesr. Ophthol. visual Sci., Suppl. 82, 49. Tyler C. W. and Torres J. (1972) Frequency response characteristics for sinusoidal movement in the fovea and periphery. Percept. Psychophys. 12, 232-236. van Santen J. P. H. and Sperling G. (1985) Elaborated Reichardt detectors. J. opt. Sot. Am. AZ, 300-321. Watson A. B. and Ahumada A. J. (1985) Model of human motion sensing. 1. opt. Sot. Am. A2, 322-342. Westheimer G. and McKee S. P. (1975) Visual acuity in the presence of retinal image motion. J. opt. Sot. Am. 66, 847-850.