- Email: [email protected]
Temporal structure of the apparent motion perception: a magnetoencephalographic study
Neuroscience Research 48 (2004) 111–118
Temporal structure of the apparent motion perception: a magnetoencephalographic study Tetsuo Kubota a,b , Yoshiki Kaneoke a,∗ , Koichi Maruyama a,b , Kazuyoshi Watanabe b , Ryusuke Kakigi a a
Department of Integrative Physiology, National Institute for Physiological Sciences, Myodaiji-cho, Okazaki 444-8585, Japan b Department of Pediatrics, Nagoya University School of Medicine, 65 Tsurumai-cho, Showa-ku, Nagoya 466-0065, Japan Received 25 April 2003; accepted 2 October 2003
Abstract Humans perceive motion when numerous small dots pattern is followed by one of the same pattern but with all the dots shifted a little in one direction. When the amount of shift exceeds a level humans no more perceive motion even though physical visual information does not change. Using this stimulus, we addressed to elucidate the temporal structure of the neural activity related to this apparent motion perception. The magnetic responses to the random-dot patterns with various amounts of shift were measured while the subjects were performing a direction discrimination task. A significant magnetic response amplitude change occurred with three distinct peaks when the response inducing apparent motion was compared with those inducing no motion without change in the response latencies. The major difference occurred at about 110, 140, 210 ms after the stimulus onset. The response origin was always within the occipitotemporal area. The results indicate that the neural activity for the perception of apparent motion can be measured by MEG that occur at least 110 ms after the stimulus onset possibly in the human MT+. Three distinct peaks in the response difference may represent the sequential multiple neural process proposed theoretically though further study is necessary to prove. © 2003 Elsevier Ireland Ltd and The Japan Neuroscience Society. All rights reserved. Keywords: Apparent motion; Human visual system; Magnetoencephalography; Random-dot pattern; Response latency
1. Introduction Apparent motion is the perception of motion induced by two successive stimuli of proper displacement and timing. Because there is no motion in the stimuli, the observer’s brain generates the image of motion by interpolating the first and second stimuli. The visually induced apparent motion may be generated by the same neural system as that for the detection of real motion, because neurons in the monkey MT/V5 have been shown to respond to both real and apparent motions (Mikami et al., 1986b). Furthermore, there appears to be cortical regions around the occipitotemporal area in the human brain exclusively sensitive to various visual motions including apparent motions (Kaneoke et al., 1997; Zihl et al., 1983; Zeki et al., 1993). Recent human imaging study (Muckli et al., 2002) revealed that the human ∗ Corresponding author. Tel.: +81-564-55-7766; fax: +81-564-52-7913. E-mail address: [email protected] (Y. Kaneoke).
MT/V5+ is exclusively related to the generation of the image of the apparent motion. Previous human electrophysiological studies have been done to elucidate how apparent motions are generated in the human brain and how a presented visual information is differentiated from blinking or flash of an object. All of those studies have suggested the role of the so-called dorsal pathway (Ungerleider and Haxby, 1994) in the generation of the motion image (Gallichio and Andreassi, 1982; Manning and Mazzucchelli, 1992; Tobimatsu et al., 1995; Uusitalo et al., 1997; Kawamoto et al., 1997). However, detail time course of the neural process for the apparent motion perception remains to be elucidated. In this study, we intended to investigate in detail the temporal structure of the neural responses to visual stimuli that induce apparent motion using magnetoencephalography (MEG). Although MEG was initially developed with the hope of determining the sources of the responses (Hughes, 1983), it is not always possible because the signals measured by MEG do not have information about their origins in
0168-0102/$ – see front matter © 2003 Elsevier Ireland Ltd and The Japan Neuroscience Society. All rights reserved. doi:10.1016/j.neures.2003.10.006
112
T. Kubota et al. / Neuroscience Research 48 (2004) 111–118
themselves (Hamalainen et al., 1993). MEG, however, still has advantages over electroencephalography (EEG), which also measures the electrical activity of the brain in milliseconds, because the magnetic fields MEG measures are less affected by the cerebrospinal fluid and other tissues than are the electrical fields measured by EEG (Hari and Lounasmaa, 1989; Hamalainen et al., 1993). Thus, there must be some response components seen only in the MEG waveforms. The response latency and amplitude change in the waveforms of MEG should have important information of the brain function as many EEG and evoked potential studies have shown (McCarthy and Donchin, 1981; Thorpe et al., 1996). Apparent motion can be induced visually by a sequence of either distinct objects or random-dot patterns. One can vary the quality of the perceived apparent motions with the stimulus conditions such as stimulus onset asynchrony, the interstimulus interval, and the spatial distance between two objects (Kahneman and Wolman, 1970; Anstis et al., 1985). Motion from random-dot patterns can only be seen when the shift of the pattern is sufficiently short (Braddick, 1974; Chang and Julesz, 1983; Barlow and Tripathy, 1997). The amount of the pattern shift will not influence directly the waveform of the magnetic response to this stimulus because the physical stimulus conditions (such as global luminance and contrast, spatiotemporal frequency, and texture) do not vary with the shift. Thus, if there is any change in the measured magnetic response, it must be related to the neural process for the interpretation of the visual information and not to the primary sensory response. We used here two successive random-dot patterns with two amounts of shift to investigate the neural activity related to the perception of apparent motion.
2. Methods 2.1. Subjects Six healthy right-handed colleagues (one woman and five men, 27–43 years old) participated in this study. All subjects had normal or corrected to normal visual acuity. They were fully informed and were experienced subjects of the MEG experiments. The study was approved by the ethical committee for the human study in this institute. 2.2. Visual stimulus The visual stimulus used in this study consisted of three frames. In Frame 1, stationary random dots were presented for 1–2 s. The subsequently presented Frame 2 consisted of the same stationary dots, but the whole dots’ locations in Frame 1 were shifted either left or right at fixed distances so that the observer would perceive apparent motion of the dots’ pattern if the distance was short enough. We used three amounts of shift for the dots, 0.17, 0.99, and 1.50◦ in visual angle. The amounts of shift were determined so that
one condition (0.17◦ ) evokes vivid apparent motion perception and the other (1.5◦ ) evokes no motion according to our preliminary experiment. Frame 2 was shown for 0.5 s and followed by Frame 3 for 2 s, in which no dot was shown. Frame 3 was necessary for the subjects to blink and to prepare for the next stimulus. By this stimulus presentation, we could measure the magnetic response evoked that was related only to the change in the frame (from Frames 1 to 2). Because the onset of Frame 1 was long time before the frame change, we did not need to consider the onset response of Frame 1. One experimental session consisted of randomly presented two stimuli, in each of which Frames 1, 2, and 3 were projected sequentially. In one experimental session, 120 epochs for each shift were collected. The direction (left or right) and the distance of the pattern shift were chosen at random for each epoch. The dot pattern in Frame 1 was newly presented for each epoch. The pattern in Frame 2 at the opposite side of the shift was not a wraparound of Frame 1, but rather the area was replaced by the new dot pattern so as not to evoke the perception of motion at the direction opposite to the shift. The stimulus created by the PC was projected on a screen in a magnetically shielded room from outside through a small window using a Liquid Crystal Display (LCD) projector (LP-9200, SANYO). The stimuli (presented on the left visual field) subtended 10.3◦ × 10.3◦ visual angle with a dot size of 0.13◦ × 0.13◦ and a density of 10% at the viewing distance of 2 m. The mean distance between the nearest dots (from center to center) was 0.464◦ . The luminance of the dots and the background was 2.5 and 1.0 cd/m2 , respectively. 2.3. Magnetic response measurement We used a 37-channel neuromagnetometer (Magnes; BTi, San Diego, CA, USA) to record the magnetic responses from the right occipitoparietotemporal area of the subject’s brain (Kaneoke et al., 1997). Each subject lay on his or her right side on a bed in the dimly lit magnetically shielded room and was asked to gaze at the fixation point, which was 1.0◦ offset from the middle point of the edge of the visual stimulus. The stimulus was presented on the left visual field because the waveforms of the MEG responses were simpler than the response waveforms for both hemifields according to our preliminary study. (The response sources of complex waveforms are often difficult to estimate.) The subjects practiced the timing of their blinking beforehand. The center of the MEG sensors was aimed at the right lateral occipital scalp. The measurement area was more than two thirds of the hemisphere, which should cover nearly the entire cortical regions specifically related to the visual motion process (Dupont et al., 1994). Magnetic response data of 50 ms before and 500 ms after the onset of Frame 2 were amplified, filtered (0.1–800 Hz) and digitized at a sampling rate of 2083.3 Hz until all epochs of data were collected. Each epoch’s data were averaged separately for each amount of shift and the baseline for each channel was corrected at the
T. Kubota et al. / Neuroscience Research 48 (2004) 111–118
mean level of 50 ms before the onset of Frame 2. MEG responses that drifted more than ±1500 femtotesla (fT) were discarded to reject artifacts related to blink and eye movement, and the data were filtered in the range of 1.0–50 Hz for further analysis. Assessment of the global magnetic field strength at each time (t) and the onset and peak latencies of the first response was carried out based on the root mean square (rms) values across MEG data: rms(t) = the 37 channels of the averaged ( x(i, t)2 /37)0.5 , where x(i, t)2 /37 is the square mean of the 37 channels’ magnetic field strength (x(i, t), i = 1–37) at time t (Kawakami et al., 2002). The onset latency was systematically determined as the time when the rms value exceeded 30 fT. The single current dipole (ECD) model was used to estimate the locations of the cortical activities that produced the magnetic fields (Sarvas, 1987). We made two criteria for the application of the ECD model (Kaneoke et al., 1997). First, the estimated ECD must be stable for at least 10 ms within 20 ms around the peak. Second, the correlation coefficient between the recorded magnetic field and the values expected from the ECD model must be more than 0.95. The location of the estimated dipole during that period was calculated for each magnetic response. The patterns of the magnetic field during the responses were also investigated to assess the distribution of the neural activities responsible for the magnetic responses.
113
0.17
0.99
1.50˚
100fT 100
100
(fT) 100
50
50
50
0
0
0
500
0
500
0
200ms
0
500 (ms)
Fig. 1. Averaged waveforms and rms values of the magnetic responses with the shifts of 0.17, 0.99, and 1.50◦ from one subject (S1). Waveforms from 37 channels were overlaid at the mean baseline before the stimulus onset (time 0). The first component was always the largest and was followed by several components. The rms value at the peak of the first component for 0.17◦ is the largest but the peak latencies were similar.
two-factor ANOVA (subject × amount of the shift) revealed that the effect of the shift was significant (P < 0.05). All the data for each subject are shown in Table 1. The results of the direction discrimination task are shown in Fig. 2. The percent correct response for the stimulus at the shift of 0.17◦ was nearly 100% and decreased as the amount of shift increased (P < 0.01, two-factor ANOVA,
2.4. Psychophysical examination Magnetic Responses
Fig. 1 shows the waveforms and rms values of the magnetic response for each shift from subject S1. As seen, the first component was the largest and was followed by several components. The peak latency of the first component did not change with the amount of the shift but the peak amplitude was inversely related to the shift. Fig. 2 shows the mean (and ±S.E.M) latency and amplitude change with the shift among all six subjects. The mean latency was around 165 ms and there was no significant difference among the three amounts of shift (P > 0.05) by two-factor ANOVA (subject×amount of the shift). In contrast, the peak amplitude was inversely related to the shift though the relation was not linear. The
(fT)
Amplitude
160
180
140 100 0.17
0.99
1.50˚
130
100
70
0.17
0.99
1.50˚
Direction discrimination Percent correct
3. Results
(ms) 220
Latency
The subjects performed the direction discrimination task during the magnetic response measurements. The subjects discriminated the direction of the dots’ pattern shift when the frame changed from Frames 1 to 2 and pushed the corresponding switch as soon as possible with the right hand, so that the neural activity related to the hand motion would not interfere with the visual response from the right hemisphere. There was no feedback during the experiment. The rate of correct responses was calculated for each amount of shift after all epochs were done.
(%) 100 90 80 70 60
50 0.17
0.99
1.50˚
Displacement Fig. 2. Changes in the magnetic responses and the human perception. Changes in the peak rms latency and amplitude of the magnetic response with the amount of shift are shown on top. The result of the direction discrimination task is also shown as the mean percent correct response (bottom). All the data are shown as mean (±S.E.M.) across the six subjects. Although there is no significant change in the latency with the amount of shift, the amplitude and the correct response rate were inversely related to the shift (P < 0.05, two-factor ANOVA). They were highly correlated with each other (P < 0.05).
114
T. Kubota et al. / Neuroscience Research 48 (2004) 111–118
Table 1 Peak latency and peak amplitude of the MEG response and percent correct response for each subject Shift (◦ )
Subject
Mean
S.E.M
S1
S2
S3
S4
S5
S6
155.0 152.2 149.8
188.2 166.6 167.0
171.8 171.4 172.3
193.9 182.9 185.8
119.5 131.5 136.8
147.8 191.5 189.1
162.7 166.0 166.8
11.3 8.8 8.3
Peak amplitude (fT) 0.17 114.6 0.99 93.7 1.50 93.1
143.5 104.1 94.0
206.2 126.4 127.9
116.9 108.2 114.3
81.2 73.2 66.5
91.5 73.7 74.4
125.7 96.6 95.1
18.4 8.,T 9.5
Correct response rate (%) 0.17 100 0.99 46.7 1.50 43.3
94.1 64.2 68.3
100 65.0 57.5
99.2 41.7 50
99.2 60 49.2
100 65 57.5
98.2 55.8 53.5
0.95 3.9 3.5
Peak latency (ms) 0.17 0.99 1.50
subject x amount of the shift). The mean percent correct response for the largest shift (1.50◦ ) was around 50%, indicating that the subject did not perceive global motion of the dot pattern. This psychometric function correlated with the mean magnetic response amplitude (Pearson’s correlation coefficient = 0.999, P < 0.05). We examined the time course of the amplitude differences of the magnetic responses using rms values. Fig. 3A shows the time courses of the rms values of the responses to the shifts of 0.17 and 1.50◦ from subject S2. The large amplitude change started at 140 ms for this subject. The time course of
the differences in the rms values between the two responses is shown in Fig. 3B. The differences were normalized by the mean and S.D. of the differences 50 ms before and after the onset of Frame 2, having assumed that only a measurement error caused the difference in the amplitude at this period. There were two peak differences that exceeded the 3.29 S.D. level (corresponding to P = 0.001) shown by the broken line. The first peak reached this level at 106 ms and the second at 150 ms. The rms differences between the responses for the two shifts were calculated for each subject and normalized. The
All subjects
Subject 2 (A) (fT) 160
(C)
0.17˚ 1.50˚
120
10
80
5
40
0
0 0
100
200
(B) (SD) 15
300
400
500 (ms)
[0.17]-[1.50]
10
-5 0
100
200
(SD) 15
300
400
500 (ms)
[0.17]-[1.50]
10
5
5
0
0
-5 0
[0.17]-[0.99]
(SD) 15
100
200
300
400
500 (ms)
-5 0
100
200
300
400
500
Fig. 3. Time course of the difference of the responses between the two stimulus conditions. (A) Time courses of the rms values for the responses with the shifts of 0.17 and 1.50◦ from subject S2. A marked amplitude difference started at 140 ms. (B) The difference in the rms values between the shifts of 0.17 and 1.50◦ were calculated and normalized for one subject, S2. The values are shown as deviations from the mean (0). The broken line shows ±3.29 S.D. level, which corresponds to P = 0.001. Two distinct peaks (indicated by the shaded areas) are seen and the first peak reached this level at 106 ms and the second at 150 ms. (C) The mean normalized differences across all the six subjects between each two shifts. Three distinct peaks (shaded areas) are seen in the differences between 0.17 and 0.99◦ ([0.17] − [0.99]) and between the shifts of 0.17 and 1.50◦ ([0.17] − [1.50]). There is no significant deviation (i.e., within ±1.96 S.D.) in the difference between 0.99 and 1.50◦ . The dotted lines with open and closed triangles show the mean onset latency (100 ± 4 ms) and the mean peak latency (165 ± 7 ms) of the MEG response for shift of 0.17◦ across all the subjects’ data, respectively.
T. Kubota et al. / Neuroscience Research 48 (2004) 111–118
115
Fig. 4. The cortical source and the distribution of the magnetic response. The estimated sources of the responses to the three shifts overlaid on the twoand three-dimensional MRI for subject S1 (left). The three-dimensional brain image is viewed from right posterior. The locations are close together and are about 10 mm posterior to the upper limb of the inferior temporal sulcus. At the right side of the figure, the patterns of the magnetic fields at the peak responses are shown as contour maps for subject S4, whose responses did not meet the criteria to estimate the sources using the single dipole model. All the maps are plotted from minimal to maximal values to present the relative distribution of the amplitudes in 37 channels for each response. Note that the patterns are similar regardless of the difference of the shifts. The correlation coefficients (Pearson) between each pair of maps are shown by the numbers between them. The time of dipole estimation was done around the peak response (see Section 2) and the contour maps were shown for the peak response latency (shown by each map).
averaged differences across all the subjects are shown in Fig. 3C. Three distinct peaks are seen in the difference values between the responses for the shift of 0.17 and 1.50◦ and in the differences between 0.17 and 0.99◦ . The first peak reached the 3.29 S.D. level at 110 and 109 ms and the second at 140 and 137 ms. The timing of the first difference peak is just after the MEG response onset and the second difference occurred around the peak latency of the response. The third peak appeared at about 210 ms, about 44 ms after the mean peak response latency. There was no significant difference between the responses for 0.99 and 1.50◦ (i.e., within ±1.96 S.D.). All but one subject’s responses were successfully used to estimate the source of the first components. Fig. 4 shows the estimated locations for the three responses in subject S1. The area corresponds to the locations estimated for the responses to various visual motions in our previous studies (Kaneoke et al., 1997; Bundo et al., 2000; Kawakami et al., 2002). The estimated locations for the responses from the other four subjects were also located in the occipitoparietotemporal area within 10 mm from the nearest upper limb of the inferior temporal sulcus. Although the response sources for subject S4 could not be estimated using the single dipole model, the magnetic field distributions for the first components were quite similar, as shown in Fig. 4.
4. Discussion The visual stimulus used in this study consisted of two successive random-dot patterns, the second of which was
the same as the first one but shifted either to the right or left. Our subjects could discriminate the direction of the shift nearly perfectly when the amount of the shift was 0.17◦ , but the mean correct response rates decreased to the chance level (50%) when the shifts was 1.50◦ (Fig. 1). The results correspond to those of previous works (Braddick, 1974; Barlow and Tripathy, 1997). The magnetic response amplitude measured during the direction discrimination task was inversely related to the shift as were the percent correct response rates (Fig. 1). This amplitude change must be related to the neural process that integrates visual information for the generation of apparent motion. This is because there was no fundamental difference in the stimulus conditions among the shifts of 0.17, 0.99 and 1.50◦ . That is, the global luminance and contrast was always the same because the frames consisted of the randomly placed dots with the same luminance, number, and size. Thus, the spatial frequency of the frame is always the same. The temporal frequency must also have been the same for both shifts. In this study, the temporal frequency at the time of the frame change was related to the number of pixels whose luminance changed with the dots’ locations. The probability for a certain bright pixel in Frame 1 to turn dark in Frame 2 would not change with the amount of the shift because dots of the same size and number were randomly placed for all the frames and the minimum shift of 0.17◦ corresponded to the 4 pixels that were larger than the dot’s size (3 × 3 pixels). This indicates that the mean luminance of the frame is always the same and its temporal frequency corresponds to the refresh rate of the projector (60 Hz). Thus, the amplitude change must represent the neural process for the
116
T. Kubota et al. / Neuroscience Research 48 (2004) 111–118
interpretation of the visual information after the primary sensory response. Prior to the MEG response peak, significant differences in the rms amplitude between the shifts of 0.17◦ and the other shifts were found from about 110 ms after the onset of Frame 2 (see Fig. 3). In our previous study, we found that the peak rms amplitude of the response was related to the psychometric function (quality of the apparent motion) (Kawakami et al., 2000). The two successive lights with varied intervals induced apparent motion. Because the waveforms of the magnetic responses changed with the stimulus condition (i.e., the interstimulus interval), we could not evaluate in detail the differences in the rms. This would be the reason why we did not find the amplitude difference related to the psychometric function earlier than the response peak in our previous study (Kawakami et al., 2000). The three distinct peaks in the time course of the rms differences between the response for 0.17 and the other shifts (0.99 and 1.50◦ ) found in the present study (see Fig. 3C) may be related to sequential multiple neural processes (Fig. 5). For the perception of apparent motion in response to the two successive random-dot patterns, the observer must first generate images of the local motions after the basic visual information process (detection of dots). The two possible pairs of dots, the one in Frame 1 and the other in Frame 2, induce the local motions of the dots. The smaller the shift of the random-dot pattern is, the larger the possible number of pairs of dots that induce apparent motion because of the lower correspondence noise (Barlow and Tripathy, 1997). These motion signals must be integrated in the later process in a large spatial scale to produce the coherent motion of all the dots (Williams and Sekuler, 1984). Possibly, the neural process for the step (detection of local motions, step II in Fig. 5) starts before the first rms difference because this step must be done for all the shifts of the dots even though they do not evoke global motions. This process may occur in the primary visual cortex (V1) because the neurons have smaller classical receptive fields and respond to the smaller displacement in an apparent motion stimulus than the neurons in the extrastriate area (Mikami et al., 1986b).
The later neural process for the perception of the apparent motion must be related to the integration of the local motion signals (that depend on the correspondence) and the calculation of the global motion direction (steps III and IV in Fig. 5). Because this process must vary with the correspondence of the paired dots, the neural activity has to depend on the amount of the shift in our visual stimulus. Thus, the rms difference detected in the present study should be related to the neural activity for these processes. The rms difference started about 110 ms. This may be related to the neural activity of human putative MT/V5 and adjacent areas, though the involvement of the neural activity in the occipital lobe cannot be fully excluded as suggested by the previous evoked potential study (Manning and Mazzucchelli, 1992). First, the difference started just after the onset of the first detectable response, the estimated source of which anatomically corresponded to that area in five of six subjects as in our previous studies (Kaneoke et al., 1997; Bundo et al., 2000; Kawakami et al., 2002; Maruyama et al., 2002). Second, the response latency corresponded to the latency measured directly from that cortical area (Ulbert et al., 2001). Third, this area in human and monkey is known to be related to the perception of apparent motion (Mikami et al., 1986a; Uusitalo et al., 1997; Newsome et al., 1986; Muckli et al., 2002). It should be noted that the neurons in this area also respond well to the simple luminance change like flash (Tootell et al., 1995) but the response magnitude is usually smaller than the response to the motion stimuli (Mikami et al., 1986b; Kawakami et al., 2000) as found in our present study. Thus, the stimulus attribute whether it is motion or no motion is represented in the response magnitude of the MT/V5 neurons, though the underlying mechanism is not yet elucidated. The role of the cortical areas higher than MT/V5 must also be important for the perception of the apparent motion. As seen in Fig. 3C, the larger rms difference occurs after the peak latency of the MEG response, the origin of which is estimated around MT/V5. The parietal area such as posterior end of the superior temporal sulcus is known to be related to the various motion perception (Puce et al., 1998; Pelphrey
III I
II
disappearance of dots in frame 1 appearance of dots in frame 2
local apparent motion or not
IV
integration of local motion
motion of the dots pattern
integration of non motion (blinking)
blinking of the dots pattern
Fig. 5. Possible neural process for the perception of apparent motion or blink of the dots pattern. The process can be divided into four steps (I–IV). Step I is for the primary visual information acquisition induced in the retina. Spatiotemporal local correspondence is measured at step II (possibly in V1) for the two possible pairs of dots, the one in Frame 1 and the other in Frame 2. Some pairs induce local apparent motion of the dot but the others do not. Such local information is sent to the higher cortical area (probably MT or V5 and the adjacent area) to be spatially integrated (step III). Spatially integrated local motion information competes with the spatially integrated information of dots pairs that did not induce apparent motion at step IV, resulting in the perceptual experience (global motion or blinking or the dot pattern) depending on the strength of the local motion information. The same neural process occurs for both stimuli that induce apparent motion and non-motion in our present study because there is no difference in the physical stimulus conditions. Thus, the rms difference in the MEG responses is considered to be caused by the neural process at steps III and IV.
T. Kubota et al. / Neuroscience Research 48 (2004) 111–118
et al., 2003) and the neural activity in the area could be measured by MEG (Bundo et al., 2000; Ahlfors et al., 1999). The other important evidence revealed by the present study is that the response onsets and the peak latencies for all the stimulus conditions were similar. The results support the view that the neural process for the perception of apparent motion and the process for the perception of non-motion (blink of the dot pattern in this study) occur in parallel and do not occur sequentially (Kawakami et al., 2002; Muckli et al., 2002). As previous psychophysical studies have shown that the perceptual experience (motion or not) changes with the amount of shift (Sato, 1998), the two processes must interact with each other to determine the final perceptual experience. Thus, we consider that the information of the spatially integrated the local motion must interact (compete) with the information of the spatially integrated non-motion (blink) as shown in Fig. 5 (step IV) and the result evokes the perceptual experience.
5. Conclusion We measured the magnetic responses to two successive random-dot patterns with two amounts of shift while the subjects were performing a direction discrimination task. A significant amplitude change in the magnetic response from the extrastriate area started as early as 110 ms with subsequent two distinct peaks, when the response inducing apparent motion was compared with that inducing no motion. We consider that such difference in the amplitude represents the multiple sequential process for the perception of apparent motion competing with the other process for the non-motion (blink) perception. Further study is necessary to elucidate what neural process is related to each distinct peak in the difference of the response amplitude.
Acknowledgements We thank Mr. O. Nagata and Mr. Y. Takeshima for technical assistance.
References Ahlfors, S.P., Simpson, G.V., Dale, A.M., Belliveau, J.W., Liu, A.K., Korvenoja, A., Virtanen, J., Huotilainen, M., Tootell, R.B.H., Aronen, H.J., Ilmoniemi, R.J., 1999. Spatiotemporal activity of a cortical network for processing visual motion revealed by MEG and fMRI. J. Neurophysiol. 82, 2545–2555. Anstis, S., Giaschi, D., Cogan, A.I., 1985. Adaptation to apparent motion. Vision Res. 25, 1051–1062. Barlow, H., Tripathy, S.P., 1997. Correspondence noise and signal pooling in the direction of coherent visual motion. J. Neurosci. 17, 7954–7966. Braddick, O., 1974. A short-range process in apparent motion. Vision Res. 14, 519–527. Bundo, M., Kaneoke, Y., Inao, S., Yoshida, J., Nakamura, A., Kakigi, R., 2000. Human visual motion areas determined individually by
117
magnetoencephalography and 3D magnetic resonance imaging. Hum. Brain Mapp. 11, 33–45. Chang, J.J., Julesz, B., 1983. Displacement limits for spatial frequency filtered random-dot cinematograms in apparent motion. Vision Res. 23, 1379–1385. Dupont, P., Orban, G.A., De Bruyn, B., Verbruggen, A., Mortelmans, L., 1994. Many areas in the human brain respond to visual motion. J. Neurophysiol. 72, 1420–1424. Gallichio, J.A., Andreassi, J.L., 1982. Visual evoked potentials under varied velocities of continuous and discrete apparent motion. Int. J. Neurosci. 17, 169–177. Hamalainen, M., Hari, R., Ilmoniemi, R.J., Knuutila, J., Lounasmaa, O.V., 1993. Magnetoencephalography—theory, instrumentation, and applications to noninvasive studies of the working human brain. Rev. Mod. Phys. 65, 413–497. Hari, R., Lounasmaa, O.V., 1989. Recording and interpretation of cerebral magnetic fields. Science 244, 432–436. Hughes, J.R., 1983. Magnetoencephalography—update and future hopes. Am. J. EEG Technol. 23, 179–189. Kahneman, D., Wolman, R.E., 1970. Stroboscopic motion: effects of duration and interval. Percept. Psychophys. 8, 161–164. Kaneoke, Y., Bundou, M., Koyama, S., Suzuki, H., Kakigi, R., 1997. Human cortical area responding to stimuli in apparent motion. NeuroReport 8, 677–682. Kawakami, O., Kaneoke, Y., Kakigi, R., 2000. Perception of apparent motion is related to the neural activity in the human extrastriate cortex as measured by magnetoencephalography. Neurosci. Lett. 285, 135– 138. Kawakami, O., Kaneoke, Y., Maruyama, K., Kakigi, R., Okada, T., Sadato, N., Yonekura, Y., 2002. Visual detection of motion speed in humans: spatiotemporal analysis by fMRI and MEG. Hum. Brain Mapp. 16, 104–118. Kawamoto, M., Yoshino, A., Suzuki, A., Ichinowatari, N., 1997. Human brain potentials observed using the line-motion method: the neurophysiological correlates of visual illusory motion perception. Neurosci. Lett. 231, 49–52. Manning, M.L., Mazzucchelli, T., 1992. Electrical responses to short-range kinematogram displays: an occipital lobe global motion process in humans? Vision Res. 32, 447–451. Maruyama, K., Kaneoke, Y., Watanabe, K., Kakigi, R., 2002. Human cortical responses to coherent and incoherent motion as measured by magnetoencephalography. Neurosci. Res. 44, 193–203. McCarthy, G., Donchin, E., 1981. A metric of thought: a comparison of P300 latency and reaction time. Science 211, 77–80. Mikami, A., Newsome, W.T., Wurtz, R.H., 1986a. Motion selectivity in macaque visual cortex. Part I. Mechanisms of direction and speed selectivity in extrastriate area MT. J. Neurophysiol. 55, 1308– 1327. Mikami, A., Newsome, W.T., Wurtz, R.H., 1986b. Motion selectivity in macaque visual cortex. Part II. Spatiotemporal range of directional interactions in MT and V1. J. Neurophysiol. 55, 1328– 1339. Muckli, L., Kriegeskorte, N., Lanfermann, H., Zanella, F.E., Singer, W., Goebel, R., 2002. Apparent motion: event-related functional magnetic resonance imaging of perceptual switches and States. J. Neurosci. 22, RC219. Newsome, W.T., Mikami, A., Wurtz, R.H., 1986. Motion selectivity in macaque visual cortex. Part III. Psychophysics and physiology of apparent motion. J. Neurophysiol. 55, 1340–1351. Pelphrey, K.A., Mitchell, T.V., McKeown, M.J., Goldstein, J., Allison, T., McCarthy, G., 2003. Brain activity evoked by the perception of human walking: controlling for meaningful coherent motion. J. Neurosci. 23, 6819–6825. Puce, A., Allison, T., Bentin, S., Gore, J.C., McCarthy, G., 1998. Temporal cortex activation in humans viewing eye and mouth movements. J. Neurosci. 18, 2188–2199.
118
T. Kubota et al. / Neuroscience Research 48 (2004) 111–118
Sarvas, J., 1987. Basic mathematical and electromagnetic concepts of the biomagnetic inverse problem. Physiol. Med. Biol. 32, 11–22. Sato, T., 1998. Dmax : relations to low- and high-level motion processes. In: Watanabe, T. (Ed.), High-Level Motion Processing: Computational, Neurobiological, and Psychophysical Perspectives. MIT Press, Cambridge, pp. 115–151. Thorpe, S., Fize, D., Marlot, C., 1996. Speed of processing in the human visual system. Nature 381, 520–522. Tobimatsu, S., Tomoda, H., Kato, M., 1995. Parvocellular and magnocellular contributions to visual evoked potentials in humans: stimulation with chromatic and achromatic gratings and apparent motion. J. Neurol. Sci. 134, 73–82. Tootell, R.B., Reppas, J.B., Kwong, K.K., Malach, R., Born, R.T., Brady, T.J., Rosen, B.R., Belliveau, J.W., 1995. Functional analysis of human MT and related visual cortical areas using magnetic resonance imaging. J. Neurosci. 15, 3215–3230.
Ulbert, I., Karmos, G., Heit, G., Halgren, E., 2001. Early discrimination of coherent versus incoherent motion by multiunit and synaptic activity in human putative MT+. Hum. Brain Mapp. 13, 226– 238. Ungerleider, L.G., Haxby, J.V., 1994. ‘What’ and ‘where’ in the human brain. Curr. Opin. Neurobiol. 4, 157–165. Uusitalo, M.A., Jousmaki, V., Hari, R., 1997. Activation trace lifetime of human cortical responses evoked by apparent visual motion. Neurosci. Lett. 224, 45–48. Williams, D.W., Sekuler, R., 1984. Coherent global motion percepts from stochastic local motions. Vision Res. 24, 55–62. Zeki, S., Watson, J.D.G., Frackowiak, R.S.J., 1993. Going beyond the information given: the relation of illusory visual motion to brain activity. Proc. R. Soc. Lond. B. Biol. Sci. 252, 215–222. Zihl, J., von Cramon, D., Mai, N., 1983. Selective disturbance of movement vision after bilateral brain damage. Brain 106, 313–340.
Temporal structure of the apparent motion perception: a magnetoencephalographic study Tetsuo Kubota a,b , Yoshiki Kaneoke a,∗ , Koichi Maruyama a,b , Kazuyoshi Watanabe b , Ryusuke Kakigi a a
Department of Integrative Physiology, National Institute for Physiological Sciences, Myodaiji-cho, Okazaki 444-8585, Japan b Department of Pediatrics, Nagoya University School of Medicine, 65 Tsurumai-cho, Showa-ku, Nagoya 466-0065, Japan Received 25 April 2003; accepted 2 October 2003
Abstract Humans perceive motion when numerous small dots pattern is followed by one of the same pattern but with all the dots shifted a little in one direction. When the amount of shift exceeds a level humans no more perceive motion even though physical visual information does not change. Using this stimulus, we addressed to elucidate the temporal structure of the neural activity related to this apparent motion perception. The magnetic responses to the random-dot patterns with various amounts of shift were measured while the subjects were performing a direction discrimination task. A significant magnetic response amplitude change occurred with three distinct peaks when the response inducing apparent motion was compared with those inducing no motion without change in the response latencies. The major difference occurred at about 110, 140, 210 ms after the stimulus onset. The response origin was always within the occipitotemporal area. The results indicate that the neural activity for the perception of apparent motion can be measured by MEG that occur at least 110 ms after the stimulus onset possibly in the human MT+. Three distinct peaks in the response difference may represent the sequential multiple neural process proposed theoretically though further study is necessary to prove. © 2003 Elsevier Ireland Ltd and The Japan Neuroscience Society. All rights reserved. Keywords: Apparent motion; Human visual system; Magnetoencephalography; Random-dot pattern; Response latency
1. Introduction Apparent motion is the perception of motion induced by two successive stimuli of proper displacement and timing. Because there is no motion in the stimuli, the observer’s brain generates the image of motion by interpolating the first and second stimuli. The visually induced apparent motion may be generated by the same neural system as that for the detection of real motion, because neurons in the monkey MT/V5 have been shown to respond to both real and apparent motions (Mikami et al., 1986b). Furthermore, there appears to be cortical regions around the occipitotemporal area in the human brain exclusively sensitive to various visual motions including apparent motions (Kaneoke et al., 1997; Zihl et al., 1983; Zeki et al., 1993). Recent human imaging study (Muckli et al., 2002) revealed that the human ∗ Corresponding author. Tel.: +81-564-55-7766; fax: +81-564-52-7913. E-mail address: [email protected] (Y. Kaneoke).
MT/V5+ is exclusively related to the generation of the image of the apparent motion. Previous human electrophysiological studies have been done to elucidate how apparent motions are generated in the human brain and how a presented visual information is differentiated from blinking or flash of an object. All of those studies have suggested the role of the so-called dorsal pathway (Ungerleider and Haxby, 1994) in the generation of the motion image (Gallichio and Andreassi, 1982; Manning and Mazzucchelli, 1992; Tobimatsu et al., 1995; Uusitalo et al., 1997; Kawamoto et al., 1997). However, detail time course of the neural process for the apparent motion perception remains to be elucidated. In this study, we intended to investigate in detail the temporal structure of the neural responses to visual stimuli that induce apparent motion using magnetoencephalography (MEG). Although MEG was initially developed with the hope of determining the sources of the responses (Hughes, 1983), it is not always possible because the signals measured by MEG do not have information about their origins in
0168-0102/$ – see front matter © 2003 Elsevier Ireland Ltd and The Japan Neuroscience Society. All rights reserved. doi:10.1016/j.neures.2003.10.006
112
T. Kubota et al. / Neuroscience Research 48 (2004) 111–118
themselves (Hamalainen et al., 1993). MEG, however, still has advantages over electroencephalography (EEG), which also measures the electrical activity of the brain in milliseconds, because the magnetic fields MEG measures are less affected by the cerebrospinal fluid and other tissues than are the electrical fields measured by EEG (Hari and Lounasmaa, 1989; Hamalainen et al., 1993). Thus, there must be some response components seen only in the MEG waveforms. The response latency and amplitude change in the waveforms of MEG should have important information of the brain function as many EEG and evoked potential studies have shown (McCarthy and Donchin, 1981; Thorpe et al., 1996). Apparent motion can be induced visually by a sequence of either distinct objects or random-dot patterns. One can vary the quality of the perceived apparent motions with the stimulus conditions such as stimulus onset asynchrony, the interstimulus interval, and the spatial distance between two objects (Kahneman and Wolman, 1970; Anstis et al., 1985). Motion from random-dot patterns can only be seen when the shift of the pattern is sufficiently short (Braddick, 1974; Chang and Julesz, 1983; Barlow and Tripathy, 1997). The amount of the pattern shift will not influence directly the waveform of the magnetic response to this stimulus because the physical stimulus conditions (such as global luminance and contrast, spatiotemporal frequency, and texture) do not vary with the shift. Thus, if there is any change in the measured magnetic response, it must be related to the neural process for the interpretation of the visual information and not to the primary sensory response. We used here two successive random-dot patterns with two amounts of shift to investigate the neural activity related to the perception of apparent motion.
2. Methods 2.1. Subjects Six healthy right-handed colleagues (one woman and five men, 27–43 years old) participated in this study. All subjects had normal or corrected to normal visual acuity. They were fully informed and were experienced subjects of the MEG experiments. The study was approved by the ethical committee for the human study in this institute. 2.2. Visual stimulus The visual stimulus used in this study consisted of three frames. In Frame 1, stationary random dots were presented for 1–2 s. The subsequently presented Frame 2 consisted of the same stationary dots, but the whole dots’ locations in Frame 1 were shifted either left or right at fixed distances so that the observer would perceive apparent motion of the dots’ pattern if the distance was short enough. We used three amounts of shift for the dots, 0.17, 0.99, and 1.50◦ in visual angle. The amounts of shift were determined so that
one condition (0.17◦ ) evokes vivid apparent motion perception and the other (1.5◦ ) evokes no motion according to our preliminary experiment. Frame 2 was shown for 0.5 s and followed by Frame 3 for 2 s, in which no dot was shown. Frame 3 was necessary for the subjects to blink and to prepare for the next stimulus. By this stimulus presentation, we could measure the magnetic response evoked that was related only to the change in the frame (from Frames 1 to 2). Because the onset of Frame 1 was long time before the frame change, we did not need to consider the onset response of Frame 1. One experimental session consisted of randomly presented two stimuli, in each of which Frames 1, 2, and 3 were projected sequentially. In one experimental session, 120 epochs for each shift were collected. The direction (left or right) and the distance of the pattern shift were chosen at random for each epoch. The dot pattern in Frame 1 was newly presented for each epoch. The pattern in Frame 2 at the opposite side of the shift was not a wraparound of Frame 1, but rather the area was replaced by the new dot pattern so as not to evoke the perception of motion at the direction opposite to the shift. The stimulus created by the PC was projected on a screen in a magnetically shielded room from outside through a small window using a Liquid Crystal Display (LCD) projector (LP-9200, SANYO). The stimuli (presented on the left visual field) subtended 10.3◦ × 10.3◦ visual angle with a dot size of 0.13◦ × 0.13◦ and a density of 10% at the viewing distance of 2 m. The mean distance between the nearest dots (from center to center) was 0.464◦ . The luminance of the dots and the background was 2.5 and 1.0 cd/m2 , respectively. 2.3. Magnetic response measurement We used a 37-channel neuromagnetometer (Magnes; BTi, San Diego, CA, USA) to record the magnetic responses from the right occipitoparietotemporal area of the subject’s brain (Kaneoke et al., 1997). Each subject lay on his or her right side on a bed in the dimly lit magnetically shielded room and was asked to gaze at the fixation point, which was 1.0◦ offset from the middle point of the edge of the visual stimulus. The stimulus was presented on the left visual field because the waveforms of the MEG responses were simpler than the response waveforms for both hemifields according to our preliminary study. (The response sources of complex waveforms are often difficult to estimate.) The subjects practiced the timing of their blinking beforehand. The center of the MEG sensors was aimed at the right lateral occipital scalp. The measurement area was more than two thirds of the hemisphere, which should cover nearly the entire cortical regions specifically related to the visual motion process (Dupont et al., 1994). Magnetic response data of 50 ms before and 500 ms after the onset of Frame 2 were amplified, filtered (0.1–800 Hz) and digitized at a sampling rate of 2083.3 Hz until all epochs of data were collected. Each epoch’s data were averaged separately for each amount of shift and the baseline for each channel was corrected at the
T. Kubota et al. / Neuroscience Research 48 (2004) 111–118
mean level of 50 ms before the onset of Frame 2. MEG responses that drifted more than ±1500 femtotesla (fT) were discarded to reject artifacts related to blink and eye movement, and the data were filtered in the range of 1.0–50 Hz for further analysis. Assessment of the global magnetic field strength at each time (t) and the onset and peak latencies of the first response was carried out based on the root mean square (rms) values across MEG data: rms(t) = the 37 channels of the averaged ( x(i, t)2 /37)0.5 , where x(i, t)2 /37 is the square mean of the 37 channels’ magnetic field strength (x(i, t), i = 1–37) at time t (Kawakami et al., 2002). The onset latency was systematically determined as the time when the rms value exceeded 30 fT. The single current dipole (ECD) model was used to estimate the locations of the cortical activities that produced the magnetic fields (Sarvas, 1987). We made two criteria for the application of the ECD model (Kaneoke et al., 1997). First, the estimated ECD must be stable for at least 10 ms within 20 ms around the peak. Second, the correlation coefficient between the recorded magnetic field and the values expected from the ECD model must be more than 0.95. The location of the estimated dipole during that period was calculated for each magnetic response. The patterns of the magnetic field during the responses were also investigated to assess the distribution of the neural activities responsible for the magnetic responses.
113
0.17
0.99
1.50˚
100fT 100
100
(fT) 100
50
50
50
0
0
0
500
0
500
0
200ms
0
500 (ms)
Fig. 1. Averaged waveforms and rms values of the magnetic responses with the shifts of 0.17, 0.99, and 1.50◦ from one subject (S1). Waveforms from 37 channels were overlaid at the mean baseline before the stimulus onset (time 0). The first component was always the largest and was followed by several components. The rms value at the peak of the first component for 0.17◦ is the largest but the peak latencies were similar.
two-factor ANOVA (subject × amount of the shift) revealed that the effect of the shift was significant (P < 0.05). All the data for each subject are shown in Table 1. The results of the direction discrimination task are shown in Fig. 2. The percent correct response for the stimulus at the shift of 0.17◦ was nearly 100% and decreased as the amount of shift increased (P < 0.01, two-factor ANOVA,
2.4. Psychophysical examination Magnetic Responses
Fig. 1 shows the waveforms and rms values of the magnetic response for each shift from subject S1. As seen, the first component was the largest and was followed by several components. The peak latency of the first component did not change with the amount of the shift but the peak amplitude was inversely related to the shift. Fig. 2 shows the mean (and ±S.E.M) latency and amplitude change with the shift among all six subjects. The mean latency was around 165 ms and there was no significant difference among the three amounts of shift (P > 0.05) by two-factor ANOVA (subject×amount of the shift). In contrast, the peak amplitude was inversely related to the shift though the relation was not linear. The
(fT)
Amplitude
160
180
140 100 0.17
0.99
1.50˚
130
100
70
0.17
0.99
1.50˚
Direction discrimination Percent correct
3. Results
(ms) 220
Latency
The subjects performed the direction discrimination task during the magnetic response measurements. The subjects discriminated the direction of the dots’ pattern shift when the frame changed from Frames 1 to 2 and pushed the corresponding switch as soon as possible with the right hand, so that the neural activity related to the hand motion would not interfere with the visual response from the right hemisphere. There was no feedback during the experiment. The rate of correct responses was calculated for each amount of shift after all epochs were done.
(%) 100 90 80 70 60
50 0.17
0.99
1.50˚
Displacement Fig. 2. Changes in the magnetic responses and the human perception. Changes in the peak rms latency and amplitude of the magnetic response with the amount of shift are shown on top. The result of the direction discrimination task is also shown as the mean percent correct response (bottom). All the data are shown as mean (±S.E.M.) across the six subjects. Although there is no significant change in the latency with the amount of shift, the amplitude and the correct response rate were inversely related to the shift (P < 0.05, two-factor ANOVA). They were highly correlated with each other (P < 0.05).
114
T. Kubota et al. / Neuroscience Research 48 (2004) 111–118
Table 1 Peak latency and peak amplitude of the MEG response and percent correct response for each subject Shift (◦ )
Subject
Mean
S.E.M
S1
S2
S3
S4
S5
S6
155.0 152.2 149.8
188.2 166.6 167.0
171.8 171.4 172.3
193.9 182.9 185.8
119.5 131.5 136.8
147.8 191.5 189.1
162.7 166.0 166.8
11.3 8.8 8.3
Peak amplitude (fT) 0.17 114.6 0.99 93.7 1.50 93.1
143.5 104.1 94.0
206.2 126.4 127.9
116.9 108.2 114.3
81.2 73.2 66.5
91.5 73.7 74.4
125.7 96.6 95.1
18.4 8.,T 9.5
Correct response rate (%) 0.17 100 0.99 46.7 1.50 43.3
94.1 64.2 68.3
100 65.0 57.5
99.2 41.7 50
99.2 60 49.2
100 65 57.5
98.2 55.8 53.5
0.95 3.9 3.5
Peak latency (ms) 0.17 0.99 1.50
subject x amount of the shift). The mean percent correct response for the largest shift (1.50◦ ) was around 50%, indicating that the subject did not perceive global motion of the dot pattern. This psychometric function correlated with the mean magnetic response amplitude (Pearson’s correlation coefficient = 0.999, P < 0.05). We examined the time course of the amplitude differences of the magnetic responses using rms values. Fig. 3A shows the time courses of the rms values of the responses to the shifts of 0.17 and 1.50◦ from subject S2. The large amplitude change started at 140 ms for this subject. The time course of
the differences in the rms values between the two responses is shown in Fig. 3B. The differences were normalized by the mean and S.D. of the differences 50 ms before and after the onset of Frame 2, having assumed that only a measurement error caused the difference in the amplitude at this period. There were two peak differences that exceeded the 3.29 S.D. level (corresponding to P = 0.001) shown by the broken line. The first peak reached this level at 106 ms and the second at 150 ms. The rms differences between the responses for the two shifts were calculated for each subject and normalized. The
All subjects
Subject 2 (A) (fT) 160
(C)
0.17˚ 1.50˚
120
10
80
5
40
0
0 0
100
200
(B) (SD) 15
300
400
500 (ms)
[0.17]-[1.50]
10
-5 0
100
200
(SD) 15
300
400
500 (ms)
[0.17]-[1.50]
10
5
5
0
0
-5 0
[0.17]-[0.99]
(SD) 15
100
200
300
400
500 (ms)
-5 0
100
200
300
400
500
Fig. 3. Time course of the difference of the responses between the two stimulus conditions. (A) Time courses of the rms values for the responses with the shifts of 0.17 and 1.50◦ from subject S2. A marked amplitude difference started at 140 ms. (B) The difference in the rms values between the shifts of 0.17 and 1.50◦ were calculated and normalized for one subject, S2. The values are shown as deviations from the mean (0). The broken line shows ±3.29 S.D. level, which corresponds to P = 0.001. Two distinct peaks (indicated by the shaded areas) are seen and the first peak reached this level at 106 ms and the second at 150 ms. (C) The mean normalized differences across all the six subjects between each two shifts. Three distinct peaks (shaded areas) are seen in the differences between 0.17 and 0.99◦ ([0.17] − [0.99]) and between the shifts of 0.17 and 1.50◦ ([0.17] − [1.50]). There is no significant deviation (i.e., within ±1.96 S.D.) in the difference between 0.99 and 1.50◦ . The dotted lines with open and closed triangles show the mean onset latency (100 ± 4 ms) and the mean peak latency (165 ± 7 ms) of the MEG response for shift of 0.17◦ across all the subjects’ data, respectively.
T. Kubota et al. / Neuroscience Research 48 (2004) 111–118
115
Fig. 4. The cortical source and the distribution of the magnetic response. The estimated sources of the responses to the three shifts overlaid on the twoand three-dimensional MRI for subject S1 (left). The three-dimensional brain image is viewed from right posterior. The locations are close together and are about 10 mm posterior to the upper limb of the inferior temporal sulcus. At the right side of the figure, the patterns of the magnetic fields at the peak responses are shown as contour maps for subject S4, whose responses did not meet the criteria to estimate the sources using the single dipole model. All the maps are plotted from minimal to maximal values to present the relative distribution of the amplitudes in 37 channels for each response. Note that the patterns are similar regardless of the difference of the shifts. The correlation coefficients (Pearson) between each pair of maps are shown by the numbers between them. The time of dipole estimation was done around the peak response (see Section 2) and the contour maps were shown for the peak response latency (shown by each map).
averaged differences across all the subjects are shown in Fig. 3C. Three distinct peaks are seen in the difference values between the responses for the shift of 0.17 and 1.50◦ and in the differences between 0.17 and 0.99◦ . The first peak reached the 3.29 S.D. level at 110 and 109 ms and the second at 140 and 137 ms. The timing of the first difference peak is just after the MEG response onset and the second difference occurred around the peak latency of the response. The third peak appeared at about 210 ms, about 44 ms after the mean peak response latency. There was no significant difference between the responses for 0.99 and 1.50◦ (i.e., within ±1.96 S.D.). All but one subject’s responses were successfully used to estimate the source of the first components. Fig. 4 shows the estimated locations for the three responses in subject S1. The area corresponds to the locations estimated for the responses to various visual motions in our previous studies (Kaneoke et al., 1997; Bundo et al., 2000; Kawakami et al., 2002). The estimated locations for the responses from the other four subjects were also located in the occipitoparietotemporal area within 10 mm from the nearest upper limb of the inferior temporal sulcus. Although the response sources for subject S4 could not be estimated using the single dipole model, the magnetic field distributions for the first components were quite similar, as shown in Fig. 4.
4. Discussion The visual stimulus used in this study consisted of two successive random-dot patterns, the second of which was
the same as the first one but shifted either to the right or left. Our subjects could discriminate the direction of the shift nearly perfectly when the amount of the shift was 0.17◦ , but the mean correct response rates decreased to the chance level (50%) when the shifts was 1.50◦ (Fig. 1). The results correspond to those of previous works (Braddick, 1974; Barlow and Tripathy, 1997). The magnetic response amplitude measured during the direction discrimination task was inversely related to the shift as were the percent correct response rates (Fig. 1). This amplitude change must be related to the neural process that integrates visual information for the generation of apparent motion. This is because there was no fundamental difference in the stimulus conditions among the shifts of 0.17, 0.99 and 1.50◦ . That is, the global luminance and contrast was always the same because the frames consisted of the randomly placed dots with the same luminance, number, and size. Thus, the spatial frequency of the frame is always the same. The temporal frequency must also have been the same for both shifts. In this study, the temporal frequency at the time of the frame change was related to the number of pixels whose luminance changed with the dots’ locations. The probability for a certain bright pixel in Frame 1 to turn dark in Frame 2 would not change with the amount of the shift because dots of the same size and number were randomly placed for all the frames and the minimum shift of 0.17◦ corresponded to the 4 pixels that were larger than the dot’s size (3 × 3 pixels). This indicates that the mean luminance of the frame is always the same and its temporal frequency corresponds to the refresh rate of the projector (60 Hz). Thus, the amplitude change must represent the neural process for the
116
T. Kubota et al. / Neuroscience Research 48 (2004) 111–118
interpretation of the visual information after the primary sensory response. Prior to the MEG response peak, significant differences in the rms amplitude between the shifts of 0.17◦ and the other shifts were found from about 110 ms after the onset of Frame 2 (see Fig. 3). In our previous study, we found that the peak rms amplitude of the response was related to the psychometric function (quality of the apparent motion) (Kawakami et al., 2000). The two successive lights with varied intervals induced apparent motion. Because the waveforms of the magnetic responses changed with the stimulus condition (i.e., the interstimulus interval), we could not evaluate in detail the differences in the rms. This would be the reason why we did not find the amplitude difference related to the psychometric function earlier than the response peak in our previous study (Kawakami et al., 2000). The three distinct peaks in the time course of the rms differences between the response for 0.17 and the other shifts (0.99 and 1.50◦ ) found in the present study (see Fig. 3C) may be related to sequential multiple neural processes (Fig. 5). For the perception of apparent motion in response to the two successive random-dot patterns, the observer must first generate images of the local motions after the basic visual information process (detection of dots). The two possible pairs of dots, the one in Frame 1 and the other in Frame 2, induce the local motions of the dots. The smaller the shift of the random-dot pattern is, the larger the possible number of pairs of dots that induce apparent motion because of the lower correspondence noise (Barlow and Tripathy, 1997). These motion signals must be integrated in the later process in a large spatial scale to produce the coherent motion of all the dots (Williams and Sekuler, 1984). Possibly, the neural process for the step (detection of local motions, step II in Fig. 5) starts before the first rms difference because this step must be done for all the shifts of the dots even though they do not evoke global motions. This process may occur in the primary visual cortex (V1) because the neurons have smaller classical receptive fields and respond to the smaller displacement in an apparent motion stimulus than the neurons in the extrastriate area (Mikami et al., 1986b).
The later neural process for the perception of the apparent motion must be related to the integration of the local motion signals (that depend on the correspondence) and the calculation of the global motion direction (steps III and IV in Fig. 5). Because this process must vary with the correspondence of the paired dots, the neural activity has to depend on the amount of the shift in our visual stimulus. Thus, the rms difference detected in the present study should be related to the neural activity for these processes. The rms difference started about 110 ms. This may be related to the neural activity of human putative MT/V5 and adjacent areas, though the involvement of the neural activity in the occipital lobe cannot be fully excluded as suggested by the previous evoked potential study (Manning and Mazzucchelli, 1992). First, the difference started just after the onset of the first detectable response, the estimated source of which anatomically corresponded to that area in five of six subjects as in our previous studies (Kaneoke et al., 1997; Bundo et al., 2000; Kawakami et al., 2002; Maruyama et al., 2002). Second, the response latency corresponded to the latency measured directly from that cortical area (Ulbert et al., 2001). Third, this area in human and monkey is known to be related to the perception of apparent motion (Mikami et al., 1986a; Uusitalo et al., 1997; Newsome et al., 1986; Muckli et al., 2002). It should be noted that the neurons in this area also respond well to the simple luminance change like flash (Tootell et al., 1995) but the response magnitude is usually smaller than the response to the motion stimuli (Mikami et al., 1986b; Kawakami et al., 2000) as found in our present study. Thus, the stimulus attribute whether it is motion or no motion is represented in the response magnitude of the MT/V5 neurons, though the underlying mechanism is not yet elucidated. The role of the cortical areas higher than MT/V5 must also be important for the perception of the apparent motion. As seen in Fig. 3C, the larger rms difference occurs after the peak latency of the MEG response, the origin of which is estimated around MT/V5. The parietal area such as posterior end of the superior temporal sulcus is known to be related to the various motion perception (Puce et al., 1998; Pelphrey
III I
II
disappearance of dots in frame 1 appearance of dots in frame 2
local apparent motion or not
IV
integration of local motion
motion of the dots pattern
integration of non motion (blinking)
blinking of the dots pattern
Fig. 5. Possible neural process for the perception of apparent motion or blink of the dots pattern. The process can be divided into four steps (I–IV). Step I is for the primary visual information acquisition induced in the retina. Spatiotemporal local correspondence is measured at step II (possibly in V1) for the two possible pairs of dots, the one in Frame 1 and the other in Frame 2. Some pairs induce local apparent motion of the dot but the others do not. Such local information is sent to the higher cortical area (probably MT or V5 and the adjacent area) to be spatially integrated (step III). Spatially integrated local motion information competes with the spatially integrated information of dots pairs that did not induce apparent motion at step IV, resulting in the perceptual experience (global motion or blinking or the dot pattern) depending on the strength of the local motion information. The same neural process occurs for both stimuli that induce apparent motion and non-motion in our present study because there is no difference in the physical stimulus conditions. Thus, the rms difference in the MEG responses is considered to be caused by the neural process at steps III and IV.
T. Kubota et al. / Neuroscience Research 48 (2004) 111–118
et al., 2003) and the neural activity in the area could be measured by MEG (Bundo et al., 2000; Ahlfors et al., 1999). The other important evidence revealed by the present study is that the response onsets and the peak latencies for all the stimulus conditions were similar. The results support the view that the neural process for the perception of apparent motion and the process for the perception of non-motion (blink of the dot pattern in this study) occur in parallel and do not occur sequentially (Kawakami et al., 2002; Muckli et al., 2002). As previous psychophysical studies have shown that the perceptual experience (motion or not) changes with the amount of shift (Sato, 1998), the two processes must interact with each other to determine the final perceptual experience. Thus, we consider that the information of the spatially integrated the local motion must interact (compete) with the information of the spatially integrated non-motion (blink) as shown in Fig. 5 (step IV) and the result evokes the perceptual experience.
5. Conclusion We measured the magnetic responses to two successive random-dot patterns with two amounts of shift while the subjects were performing a direction discrimination task. A significant amplitude change in the magnetic response from the extrastriate area started as early as 110 ms with subsequent two distinct peaks, when the response inducing apparent motion was compared with that inducing no motion. We consider that such difference in the amplitude represents the multiple sequential process for the perception of apparent motion competing with the other process for the non-motion (blink) perception. Further study is necessary to elucidate what neural process is related to each distinct peak in the difference of the response amplitude.
Acknowledgements We thank Mr. O. Nagata and Mr. Y. Takeshima for technical assistance.
References Ahlfors, S.P., Simpson, G.V., Dale, A.M., Belliveau, J.W., Liu, A.K., Korvenoja, A., Virtanen, J., Huotilainen, M., Tootell, R.B.H., Aronen, H.J., Ilmoniemi, R.J., 1999. Spatiotemporal activity of a cortical network for processing visual motion revealed by MEG and fMRI. J. Neurophysiol. 82, 2545–2555. Anstis, S., Giaschi, D., Cogan, A.I., 1985. Adaptation to apparent motion. Vision Res. 25, 1051–1062. Barlow, H., Tripathy, S.P., 1997. Correspondence noise and signal pooling in the direction of coherent visual motion. J. Neurosci. 17, 7954–7966. Braddick, O., 1974. A short-range process in apparent motion. Vision Res. 14, 519–527. Bundo, M., Kaneoke, Y., Inao, S., Yoshida, J., Nakamura, A., Kakigi, R., 2000. Human visual motion areas determined individually by
117
magnetoencephalography and 3D magnetic resonance imaging. Hum. Brain Mapp. 11, 33–45. Chang, J.J., Julesz, B., 1983. Displacement limits for spatial frequency filtered random-dot cinematograms in apparent motion. Vision Res. 23, 1379–1385. Dupont, P., Orban, G.A., De Bruyn, B., Verbruggen, A., Mortelmans, L., 1994. Many areas in the human brain respond to visual motion. J. Neurophysiol. 72, 1420–1424. Gallichio, J.A., Andreassi, J.L., 1982. Visual evoked potentials under varied velocities of continuous and discrete apparent motion. Int. J. Neurosci. 17, 169–177. Hamalainen, M., Hari, R., Ilmoniemi, R.J., Knuutila, J., Lounasmaa, O.V., 1993. Magnetoencephalography—theory, instrumentation, and applications to noninvasive studies of the working human brain. Rev. Mod. Phys. 65, 413–497. Hari, R., Lounasmaa, O.V., 1989. Recording and interpretation of cerebral magnetic fields. Science 244, 432–436. Hughes, J.R., 1983. Magnetoencephalography—update and future hopes. Am. J. EEG Technol. 23, 179–189. Kahneman, D., Wolman, R.E., 1970. Stroboscopic motion: effects of duration and interval. Percept. Psychophys. 8, 161–164. Kaneoke, Y., Bundou, M., Koyama, S., Suzuki, H., Kakigi, R., 1997. Human cortical area responding to stimuli in apparent motion. NeuroReport 8, 677–682. Kawakami, O., Kaneoke, Y., Kakigi, R., 2000. Perception of apparent motion is related to the neural activity in the human extrastriate cortex as measured by magnetoencephalography. Neurosci. Lett. 285, 135– 138. Kawakami, O., Kaneoke, Y., Maruyama, K., Kakigi, R., Okada, T., Sadato, N., Yonekura, Y., 2002. Visual detection of motion speed in humans: spatiotemporal analysis by fMRI and MEG. Hum. Brain Mapp. 16, 104–118. Kawamoto, M., Yoshino, A., Suzuki, A., Ichinowatari, N., 1997. Human brain potentials observed using the line-motion method: the neurophysiological correlates of visual illusory motion perception. Neurosci. Lett. 231, 49–52. Manning, M.L., Mazzucchelli, T., 1992. Electrical responses to short-range kinematogram displays: an occipital lobe global motion process in humans? Vision Res. 32, 447–451. Maruyama, K., Kaneoke, Y., Watanabe, K., Kakigi, R., 2002. Human cortical responses to coherent and incoherent motion as measured by magnetoencephalography. Neurosci. Res. 44, 193–203. McCarthy, G., Donchin, E., 1981. A metric of thought: a comparison of P300 latency and reaction time. Science 211, 77–80. Mikami, A., Newsome, W.T., Wurtz, R.H., 1986a. Motion selectivity in macaque visual cortex. Part I. Mechanisms of direction and speed selectivity in extrastriate area MT. J. Neurophysiol. 55, 1308– 1327. Mikami, A., Newsome, W.T., Wurtz, R.H., 1986b. Motion selectivity in macaque visual cortex. Part II. Spatiotemporal range of directional interactions in MT and V1. J. Neurophysiol. 55, 1328– 1339. Muckli, L., Kriegeskorte, N., Lanfermann, H., Zanella, F.E., Singer, W., Goebel, R., 2002. Apparent motion: event-related functional magnetic resonance imaging of perceptual switches and States. J. Neurosci. 22, RC219. Newsome, W.T., Mikami, A., Wurtz, R.H., 1986. Motion selectivity in macaque visual cortex. Part III. Psychophysics and physiology of apparent motion. J. Neurophysiol. 55, 1340–1351. Pelphrey, K.A., Mitchell, T.V., McKeown, M.J., Goldstein, J., Allison, T., McCarthy, G., 2003. Brain activity evoked by the perception of human walking: controlling for meaningful coherent motion. J. Neurosci. 23, 6819–6825. Puce, A., Allison, T., Bentin, S., Gore, J.C., McCarthy, G., 1998. Temporal cortex activation in humans viewing eye and mouth movements. J. Neurosci. 18, 2188–2199.
118
T. Kubota et al. / Neuroscience Research 48 (2004) 111–118
Sarvas, J., 1987. Basic mathematical and electromagnetic concepts of the biomagnetic inverse problem. Physiol. Med. Biol. 32, 11–22. Sato, T., 1998. Dmax : relations to low- and high-level motion processes. In: Watanabe, T. (Ed.), High-Level Motion Processing: Computational, Neurobiological, and Psychophysical Perspectives. MIT Press, Cambridge, pp. 115–151. Thorpe, S., Fize, D., Marlot, C., 1996. Speed of processing in the human visual system. Nature 381, 520–522. Tobimatsu, S., Tomoda, H., Kato, M., 1995. Parvocellular and magnocellular contributions to visual evoked potentials in humans: stimulation with chromatic and achromatic gratings and apparent motion. J. Neurol. Sci. 134, 73–82. Tootell, R.B., Reppas, J.B., Kwong, K.K., Malach, R., Born, R.T., Brady, T.J., Rosen, B.R., Belliveau, J.W., 1995. Functional analysis of human MT and related visual cortical areas using magnetic resonance imaging. J. Neurosci. 15, 3215–3230.
Ulbert, I., Karmos, G., Heit, G., Halgren, E., 2001. Early discrimination of coherent versus incoherent motion by multiunit and synaptic activity in human putative MT+. Hum. Brain Mapp. 13, 226– 238. Ungerleider, L.G., Haxby, J.V., 1994. ‘What’ and ‘where’ in the human brain. Curr. Opin. Neurobiol. 4, 157–165. Uusitalo, M.A., Jousmaki, V., Hari, R., 1997. Activation trace lifetime of human cortical responses evoked by apparent visual motion. Neurosci. Lett. 224, 45–48. Williams, D.W., Sekuler, R., 1984. Coherent global motion percepts from stochastic local motions. Vision Res. 24, 55–62. Zeki, S., Watson, J.D.G., Frackowiak, R.S.J., 1993. Going beyond the information given: the relation of illusory visual motion to brain activity. Proc. R. Soc. Lond. B. Biol. Sci. 252, 215–222. Zihl, J., von Cramon, D., Mai, N., 1983. Selective disturbance of movement vision after bilateral brain damage. Brain 106, 313–340.