- Email: [email protected]
Depth perception from moving cast shadow in macaque monkey
Behavioural Brain Research 288 (2015) 63–70
Contents lists available at ScienceDirect
Behavioural Brain Research journal homepage: www.elsevier.com/locate/bbr
Research report
Depth perception from moving cast shadow in macaque monkey Saneyuki Mizutani a,b,c , Nobuo Usui b,c , Takanori Yokota a,c , Masato Taira b,c , Narumi Katsuyama b,c,∗ a b c
Department of Neurology and Neurological Science, Graduate School of Medical and Dental Sciences, Tokyo Medical and Dental University, Tokyo, Japan Department of Cognitive Neurobiology, Graduate School of Medical and Dental Sciences, Tokyo Medical and Dental University, Tokyo, Japan Center for Brain Integration Research, Tokyo Medical and Dental University, Tokyo, Japan
h i g h l i g h t s • • • • •
We examined whether monkey can perceive motion in depth induced by cast shadow. A motion illusion induced by moving cast shadow was used. Monkey could discriminate motion in depth induced by binocular disparity. The illusion was intervened as a catch trial during the discrimination by disparity. Monkey could discriminate the illusionary motion in depth induced by cast shadow.
a r t i c l e
i n f o
Article history: Received 30 January 2015 Received in revised form 2 April 2015 Accepted 4 April 2015 Available online 13 April 2015 Keywords: Motion in depth Cast shadow Macaque monkey Disparity Depth cue
a b s t r a c t In the present study, we investigate whether the macaque monkey can perceive motion in depth using a moving cast shadow. To accomplish this, we conducted two experiments. In the first experiment, an adult Japanese monkey was trained in a motion discrimination task in depth by binocular disparity. A square was presented on the display so that it appeared with a binocular disparity of 0.12 degrees (initial position), and moved toward (approaching) or away from (receding) the monkey for 1 s. The monkey was trained to discriminate the approaching and receding motion of the square by GO/delayed GOtype responses. The monkey showed a significantly high accuracy rate in the task, and the performance was maintained when the position, color, and shape of the moving object were changed. In the next experiment, the change in the disparity was gradually decreased in the motion discrimination task. The results showed that the performance of the monkey declined as the distance of the approaching and receding motion of the square decreased from the initial position. However, when a moving cast shadow was added to the stimulus, the monkey responded to the motion in depth induced by the cast shadow in the same way as by binocular disparity; the reward was delivered randomly or given in all trials to prevent the learning of the 2D motion of the shadow in the frontal plane. These results suggest that the macaque monkey can perceive motion in depth using a moving cast shadow as well as using binocular disparity. © 2015 Elsevier B.V. All rights reserved.
1. Introduction A shadow cast on a background surface by an object is called a cast shadow. Although the shape, transparency, and blur of cast shadows in our daily scenes vary based on the relative position between the light source and the objects casting them, they
∗ Corresponding author at: Department of Cognitive Neurobiology, Tokyo Medical and Dental University, 1-5-45, Yushima, Bunkyo-ku, Tokyo 113-8549, Japan. Tel.: +81 3 5803 5445; fax: +81 3 5803 0165. E-mail address: [email protected] (N. Katsuyama). http://dx.doi.org/10.1016/j.bbr.2015.04.005 0166-4328/© 2015 Elsevier B.V. All rights reserved.
serve as a powerful monocular depth cue in spatial vision. This is well demonstrated by a motion illusion named ‘square-overcheckerboard’ (SOC) provided by Kersten et al. [1]. Fig. 1A illustrates shots from the movie. There is a square at the center with its shadow cast on the background at the right-bottom corner. When the cast shadow moves away from and toward the square in the frontal plane, observers experience a strong percept that the square approaches and recedes from them, respectively, whereas the size and position of the square are actually constant during the movie. Thus, cast shadows can provide salient depth cues for the perception of object position in three-dimensional (3D) space [1–3]. The effect of cast shadows as monocular depth cues is more prominent
64
S. Mizutani et al. / Behavioural Brain Research 288 (2015) 63–70
Fig. 1. A: Motion of the cast shadow. The shadow was presented at the initial position in all trials (center), and moved either away from (approaching, right) or toward (receding, left) the square in the AP and RC trials, respectively. In the DD + SCS conditions in session 4, the cast shadow did not move from the initial position. B: Basic configuration of visual stimulus. In session 1, the square was also presented to the right of, above and below the FP at the same distance from the FP. C: Motion of the square defined by binocular disparity. It was presented initially at 0.12 degrees of the uncrossed disparity. This corresponded to a location 25 mm in front of the background (initial position). The square moved either 0.14 degrees toward the monkey (approaching) or 0.12 degrees toward the background (receding) in the AP and RC trials, respectively. In the rDD condition, the motion of the square from the initial position was reduced to 0.021, 0.016, 0.010, and 0.0052 degrees (corresponding to an estimated distance of 4, 3, 2, and 1 mm). In the SD condition, the square did not move from the initial position. deg.: degrees.
in the movie, because it becomes easier to link objects to their cast shadows in a dynamic scene than in a static image [3]. As a cast shadow moves away from the object casting it, it can provide ambiguous information about the spatial layout of the object. Nevertheless, we can precisely infer the spatial position of the object by using the cast shadow. Previous studies hypothesized that to infer the spatial positions of an object in 3D space from its cast shadow, the visual system uses the a priori constraint that the light source is above the observers and is stable [2,3]. If this hypothesis is correct, it is expected that non-human primates who have an anatomically and functionally similar visual system to humans may also take advantage of cast shadows for spatial vision. However, only a few studies have investigated this topic so far [4–6]. In the present study, we investigated whether adult macaque monkeys can perceive motion in depth induced by cast shadows. For this purpose, we conducted two experiments. In the first experiment, we trained an adult macaque in a motion discrimination task in depth by binocular disparity. In the next experiment, we presented a moving cast shadow as part of the visual stimulus and examined the responses of the monkey to the apparent motion in depth induced by the shadow. Our results suggest that the adult macaque monkey can indeed perceive the motion in depth induced by cast shadows as well as by binocular disparity.
2. Methods and materials 2.1. Materials We used a female Japanese monkey (Macaca fuscata). This monkey was 4 years and 5 months old when it was first presented with visual stimuli containing cast shadows. Throughout the experiments, the monkey was treated in accordance with the NIH Guide for Care and Use of Laboratory Animals. All animal experiments were approved by the Institutional Animal Care and Use Committee
of Tokyo Medical and Dental University (approval number: 0150187A). 2.2. Experimental setup All stimuli in the study were shown using software for 3D presentation (Omega Space ver. 3.1, Solidray Institute, Yokohama, Kanagawa, Japan). A liquid crystal display (LCD-3D231XBR, I-O DATA, Kanazawa, Ishikawa, Japan) was placed at 0.6 m in front of the monkey at eye level, and modified versions of the SOC movie were presented in a 265 × 155 mm (width × height) window attached to the front of the display. The monkey was trained to sit in a primate chair with its head fixed. It wore liquid crystal shutters (NVIDIA 3D Vision® , NVIDIA, Santa Clara, CA, USA) for stereo vision, operating at 120 Hz, so 60 frames/s of stimulus were presented to each eye. We implanted scleral search coils (DNI instrument, Newark, DE, USA) for monitoring eye position and vergence. A trial was aborted immediately when the eye position exceeded the limit of 1◦ from the fixation point (in this article, we describe the visual angle and binocular disparity with◦ and degrees, respectively, for clarity). The original images for the movie frames used in this study were created using drawing software (Illustrator CS5, Adobe systems incorporated, San Jose, CA, USA) and converted into movies by hand-made programs written in Matlab (MathWorks, Natick, MA, USA). The original spatial resolution of each movie was 72 pixels/inch and the frame rate of the movies was 27 frames/second. Binocular disparity was calculated on the basis of an interocular distance of 0.03 m for the monkey in the present study. 2.3. Experiments 2.3.1. Experiment 1: motion discrimination in depth by disparity We first trained the monkey in a GO/delayed GO (dGO)-type discrimination task of the motion in depth defined by disparity. A trial began with presentation of a checkerboard (13.27 × 16.03◦ in
S. Mizutani et al. / Behavioural Brain Research 288 (2015) 63–70
A 100
B
%
100
60 40 20
AP
RC
Left
40
0
Right
C %
D 100
60 40
Left
Right
Up
Down
%
80 Accuracy rate
80 Accuracy rate
60
20
0
100
%
80 Accuracy rate
Accuracy rate
80
65
60 40
20
20
0
0
Fig. 2. Results from the motion discrimination task in depth by binocular disparity in Experiment 1 (DD condition). A, Left: Accuracy rate when the green square was presented to the left of the FP in the AP and RC trials. Right: Accuracy rate of the response to the leftward and rightward motion of the retinal images of the square. Accuracy rate of the responses to the change in the position (B), color (C), and the shape of the moving object (D). The responses to the AP and RC trials are merged in B, C, and D. The broken line indicates chance level.
height × width) as background on the display (Fig. 2B). After a variable duration of 1000–1200 ms, a red circle (diameter = 0.24◦ ) was presented at the center of the checkerboard (fixation point, FP). The monkey had to fixate on the FP and hold a lever within 4000 ms after the presentation of the FP. After a variable duration of 300–500 ms following fixation, a green square (2.82 × 2.82◦ in width × height) appeared 3.05◦ to the left of the FP with binocular disparity of 0.12 degrees (Fig. 2B). The simulated position of the square in depth was 25 mm in front of the background. This position is referred to as the initial position of the square (Fig. 1C). After a variable duration (1000–1200 ms), the square ‘moved’ 0.14 degrees toward the observer (approaching, AP) or 0.12 degrees toward the background (receding, RC) for 1 s, and disappeared at the foremost or innermost positions, respectively (Fig. 1C). This change in the disparity corresponded to a motion of 25 mm toward the observer and toward the background from the initial position of the square. In the following, this is referred to as the dynamic disparity (DD) condition (Table 1). The binocular disparity was the only cue for the motion of the square in depth in this condition. The size of the square was constant throughout the experiments. The velocity of the square changed sinusoidally along the trajectory. The monkey was rewarded by a drop of juice if it released the lever when the color of the FP turned from red to green in the AP trials (GO response). However, it had to keep holding the lever when the FP was green (1000–1500 ms) and was rewarded if it released the lever when the FP turned off in the RC trials (dGO response). Once the monkey achieved an accuracy rate of > 90%, we changed the properties of the moving object in the AP and RC trials. First, we presented the green square at 3.05◦ to the right of, above, and below the FP in the AP and RC trials. Next, we changed the color of the square to cyan, red, blue, orange, yellow, and magenta. Finally, we changed the shape of the moving object to a circle, triangle, annulus, star, and an uppercase letter E. We gave the moving object various combinations of these properties to confirm that the monkey performed the task based on the motion in depth only. At the end of the experiment,
Table 1 Depth cues and reward delivery in the discrimination tasks. Experiment 1 Depth cue
Reward
DD
+
Experiment 2 Depth cue
Reward
Depth cue
Reward
Session 1 DD rDD SD
+ + §
Session 4 DD DD+SCS DD+SCS SD+MCS
+ + § §
Session 2 DD rDD + MCS Session 3 DD SD + MCS
+ ¶ + ¶
DD: dynamic disparity, MCS: moving cast shadow, rDD: reduced dynamic disparity, SCS: static cast shadow, SD: static disparity, SS: static shadow. +: reward was delivered for correct responses, §: reward was delivered randomly, irrespective of the responses, ¶: reward was given for all responses.
we presented the movie frame either to one (the left or right) or both eyes to confirm that the monkey did not respond to the horizontal motion of the square in the frontal plane. 2.3.2. Experiment 2: motion discrimination in depth by adding moving cast shadow In this experiment, we examined whether the monkey could discriminate the motion in depth using a cast shadow. The following visual stimuli were used (Table 1). (1) DD: this was the same condition as used in Experiment 1. As described above, the only cue for the motion in depth in this condition was binocular disparity. (2) reduced DD (rDD): the change in the disparity from the initial position of the square was reduced to 0.021, 0.016, 0.010, and 0.0052 degrees. These values were equivalent to a motion of 4, 3, 2, and 1 mm forward and backward from the initial position of the square in the AP and RC trials, respectively. The cue for the motion in
66
S. Mizutani et al. / Behavioural Brain Research 288 (2015) 63–70
depth in this condition was binocular disparity, although the effect weakened as the disparity reduced. (3) static disparity (SD): the square was presented at the initial position for 1 s and then disappeared without motion. Although binocular disparity was present in this condition, it provided no cues for the motion in depth, for it was static. The reward was delivered randomly, irrespective of the response. (4) rDD + MCS: a moving cast shadow (MCS) was added to the rDD condition. After the fixation, a cast shadow was presented at the right-bottom corner of the square at 0.86◦ in the center-tocenter distance from the square. This position is referred to as the initial position of the cast shadow (Fig. 1A). The shadow moved away from and toward the square on an oblique trajectory for 1 s in the AP and RC trials, respectively, and disappeared with the square. The right-most position of the cast shadow in the AP trials was 1.72◦ in the center-to-center distance from the square. The body of the shadow was transparent and the border was blurred so that the apparent motion of the square in depth by the cast shadow was maximized [1,2]. The velocity, transparency, and blur of the border of the shadow changed sinusoidally, and the transparency and blur became maximal at the right-most position. The apparent motion direction of the square induced by the cast shadow and binocular disparity were congruent. Both cast shadow and binocular disparity could be cues for motion in depth in this condition. However, it was expected that the depth cue provided by the cast shadow would exceed that by binocular disparity as the disparity was reduced. To prevent the learning of the 2D motion of the shadow in the frontal plane, the reward was given in all trials irrespective of the response. (5) SD + MCS: the moving cast shadow was added to the SD condition. Since the binocular disparity was static, the moving cast shadow was the only cue for the motion of the square in depth in this condition. As in the rDD + MCS, the reward was delivered in all trials. These conditions were presented to the monkey in a block (Table 1). In session 1, the conditions DD, rDD, and SD were presented in a block to examine whether the performance of the monkey declined as the motion in depth induced by binocular disparity was reduced. The 3 conditions were equally intermingled in a randomized order. In session 2, we examined the responses of the monkey when the moving cast shadow was presented under the condition in which the motion of the square in depth induced by binocular disparity became ambiguous. One of the rDD + MCS with different disparities (0.021, 0.016, 0.010, and 0.0052 degrees) was presented randomly as a catch condition with the DD in a block at a ratio of DD:rDD + MCS = 4:1. In session 3, we examined the performance of the monkey when the moving cast shadow was presented under the condition in which the square did not move from the initial position induced by binocular disparity. The SD + MCS was presented randomly as a catch condition with the DD in a block at a ratio of DD:SD + MCS = 4:1. This session was performed only after the completion of session 2, in which the disparity changed by up to 0.0052 degrees. In sessions 2 and 3, the test was stopped when the number of trials in the catch condition (the rDD + MCS and SD + MCS in sessions 2 and 3, respectively) reached 10 trials for each of the AP and RC trials to prevent the monkey familiarizing itself with the moving cast shadow (about 100 trials for a session in total). In session 4, we examined whether the monkey was conditioned to the 2D motion of the moving cast shadow in session 2 and 3. For this purpose, the monkey was re-trained in the DD condition with the responses switched: the GO response to the RC and the dGO to the AP trials. Then the SD + MCS was tested again with the DD condition as in session 3. In this session, the SD + MCS condition was presented in 10% of all trials, and the reward was delivered randomly irrespective of the responses of the monkey. Moreover, to prevent the monkey from attending excessively to the cast shadow presented in the SD + MCS, a static cast shadow (SCS) was presented in some trials of the DD condition. This condition is referred to as
the DD + SCS condition (Table 1). The cast shadow was presented at the initial position of the shadow and did not move as the binocular disparity changed. The disparity and static shadow disappeared after 1 s. Thus, in this condition, the cue for motion of the square in depth was binocular disparity. The reward was delivered for correct responses in the half of the SD + MCS trials and was given randomly irrespective of the responses in the other half of the condition. The ratio of the DD, SD + MCS, and the two DD + SCS conditions was 7:1:1:1 (Table 1). Six blocks were conducted on different days. A block was stopped when the number of trials reached 10 of each of the AP and RC trials in the SD + MCS condition (about 200 trials for the 4 conditions). 2.4. Statistical analysis We used a one-tailed binomial test implemented in the statistical program R (http://cran.r-project.org) to evaluate the success accuracy rate with respect to chance level. P values < 0.01 were considered statistically significant. 3. Results 3.1. Experiment 1: motion discrimination in depth by binocular disparity Fig. 2 indicates the results of Experiment 1. The monkey showed a high accuracy rate (> 90%) in the discrimination of the approaching and receding motion of the square induced by binocular disparity (Fig. 2A, left). However, the performance declined to chance level when the movie frame either to the left or right eye was presented to both eyes (Fig. 2A, right). The high accuracy rate was maintained for the change in the location (Fig. 2B) and color (Fig. 2C) of the square and the shape of the moving object (Fig. 2D). The accuracy rate presented in this figure was significantly higher than chance level except in Fig. 2A, right. 3.2. Experiment 2: motion discrimination in depth by adding a moving cast shadow In session 1, we randomly presented the DD, rDD, and SD conditions (Fig. 3, S1). The monkey showed an accuracy rate that was significantly higher (> 90%) than chance level in the DD conditions. However, performance declined as the motion of the square in depth induced by binocular disparity decreased (rDD, from top to bottom in Fig. 3, S1). The accuracy rate in the AP trials was not significantly different from chance level when the disparity was less than 0.016 degrees, whereas in the RC trials, the accuracy rate was significantly higher than chance level over all disparities. Performance was generally around chance level when the square did not move from the initial position (SD) In session 2, we added the moving cast shadow to the rDD condition (rDD + MCS in Fig. 3, S2). The monkey responded to the AP and RC trials in the rDD + MCS condition with GO and dGO responses, respectively, as in the DD condition. The accuracy rate was significantly higher than chance level for both the AP and RC trials. A reward was given in all trials, irrespective of the response in the rDD + MCS condition. In session 3, we presented the moving cast shadow in the SD condition (SD + MCS in Fig. 3, S3). The monkey showed the same responses in the SD + MCS as in the DD trials, and the accuracy rate was also significantly higher than chance level for both the AP and RC trials. The number of trials tested, the accuracy rate, and the p value obtained via binomial tests for each block are shown in the supplemental Table 1. In session 4, the monkey was re-trained with the responses to the AP and RC trials reversed, and tested in the SD + MCS condition
S. Mizutani et al. / Behavioural Brain Research 288 (2015) 63–70
67
Fig. 3. Results from sessions 1–3 in Experiment 2. Performance in the AP and RC trials are shown in the left and right columns, respectively. Accuracy rates of each condition in sessions 1–3 are aligned from top to bottom in the order of the binocular disparity presented in the rDD condition. The amount of binocular disparity change is indicated at the left side of the graphs, and the simulated distance of the square with respect to the initial position is shown in parentheses. The dotted line indicates the chance level. S1, 2, and 3 at the bottom indicate session 1, 2, and 3, respectively. AP: approaching, RD: receding, deg.: degrees, for the other abbreviations, refer to the footnote in Table 1.
again. The accuracy rate of the DD and SD + MCS conditions from the six blocks are summed and plotted in Fig. 4. The monkey responded correctly to the AP and RC trials with the dGO and GO responses in the SD + MCS condition, respectively, as well as in the DD. The accuracy rate was significantly higher than chance level in both
the AP and RC trials. The accuracy rate in the DD + SCS condition was as high as that in the DD condition, regardless of whether the reward was given randomly or for correct responses only. The number of trials and the accuracy rate in each block are indicated in supplemental Table 2.
68
S. Mizutani et al. / Behavioural Brain Research 288 (2015) 63–70
AP 100
%
RC 100 80
Acuuracy rate
Acuuracy rate
80 60 40 20 0
%
60 40 20
DD
SD+ MCS
0
DD
SD+ MCS
Fig. 4. Results from session 4 in Experiment 2. Accuracy rates for the AP and RC trials in the DD and SD + MCS conditions are shown. AP: approaching, RD: receding, for the other abbreviations, refer to the footnote in Table 1. The dotted line indicates the chance level.
4. Discussion The aim of this study was to examine whether the macaque monkey could perceive motion in depth induced by a moving cast shadow. To do this, we conducted two experiments in the present study. In the first experiment, we trained a macaque monkey in a motion discrimination task in depth defined by binocular disparity with a GO/dGO-type response. In the second experiment, we presented a moving cast shadow in the visual stimulus in which the motion in depth induced by binocular disparity was ambiguous, and tested the responses of the monkey. We hypothesized that if the monkey could perceive the apparent motion in depth by moving cast shadow, it should show the same responses in motion discrimination by the moving cast shadow as well as by binocular disparity. 4.1. Motion discrimination in depth by binocular disparity In Experiment 1, an adult macaque monkey was presented with a square that appeared to move toward (approaching) or away from (receding) the animal by binocular disparity and was trained to respond to the approaching and receding motion of the square with GO and dGO responses, respectively (DD condition). The square was presented at 0.12 degrees (simulated at 25 mm in front of the background) before moving. This was to make the monkey perform the task based on the motion of the square, not the position of the square where it was presented initially in a trial. For this reason, the square was always initially presented at 0.12 degrees in all trials of this study, and moved forward or backward from this position. In the present study, the size of the square was kept constant in the AP and RC trials. It is well known that an object moving toward or away from an observer with its retinal size constant appears to shrink or expand, respectively [7]. This size change might have an effect on the motion of the square in depth. Therefore, two of the authors independently tested the apparent size change by watching a square moving in depth with a variety of disparities in the preliminary experiment, and confirmed that the size change was negligible when the square was presented at less than 0.26 degrees (50 mm in front of the background). Thus, the most forward position of the square in the AP trials was determined to be 0.26 degrees, and the initial position of the square was defined to be half of this (0.12 degrees, 25 mm in front of the background). We found that the monkey could discriminate the approaching and receding motion of the square induced by binocular disparity at a high accuracy rate. When the movie frame for either the right or left eye was presented to both eyes, the performance declined
to chance level, indicating that the monkey did not respond to the horizontal motion of the retinal images of the square. To confirm further that the monkey responded solely to the motion in depth in the task, we changed the properties of the moving object, such as its position, color, and shape. The square was presented to the left of, to the right of, above, and below the FP. The square was replaced with geometric figures such as a circle, triangle, star, annulus, and an uppercase letter E. Nevertheless, the monkey showed a high accuracy rate in the discrimination of the motion of an object with a single, or a combination of the different properties. This result indicates that the monkey did not respond to the local features of the moving object, but to the motion of the object in depth. Previous studies have revealed that the macaque monkey can discriminate static 3D surface structures defined by binocular disparity [8–11], but no study has tested macaque monkeys behaviorally in a motion discrimination task in depth. The present results indicate that the macaque monkey can discriminate the motion of a surface in depth as well as static 3D structures using binocular disparity.
4.2. Motion discrimination in depth by adding moving cast shadow In the next experiment, we examined whether cast shadow helps the monkey to perceive motion in depth. In session 1, we reduced the binocular disparity gradually (rDD condition) and examined the responses of the monkey. The reduction of the disparity resulted in a decrease of the motion of the square from the initial position in both the AP and RC trials. As expected, the performance of the monkey declined as the disparity was reduced, and the motion of the square became invisible. In session 2, we presented the moving cast shadow in the rDD (rDD + MCS condition). We hypothesized that if the monkey could perceive the motion in depth by the moving cast shadow, it should show the same responses to the AP and RC trials in the rDD + MCS as in the DD, when the motion of the square was invisible. The result showed that this was true – the monkey responded to the AP and RC trials with the GO and dGO responses, respectively. Similarly, the monkey showed the same responses to the AP and RC trials in the SD + MCS condition in session 3 with a significantly higher accuracy rate than chance level. The responses in the rDD + MCS and SD + MCS conditions were not a result of learning the 2D motion of the cast shadow in the frontal plane, because the reward was delivered in all trials of these conditions. In the SD + MCS condition, the cast shadow was the only depth cue for the motion of the square. In this condition, the square was presented at the initial position, and there was no motion induced by binocular disparity. Instead, the cast shadow moved in the frontal plane. Nevertheless, the monkey made the same responses as in the DD condition. These observations suggest that the monkey could perceive the approaching and receding motion of the square induced by the moving cast shadow. To confirm further that the monkey was not conditioned to the 2D motion of the cast shadow in sessions 2 and 3, we re-trained the monkey in the DD condition with the responses to the AP and RC trials reversed, and examined the response of the monkey to the SD + MCS condition again in session 4. After the re-training, the monkey responded to the AP and RC trials in the DD with the dGO and GO responses, respectively. Moreover, it showed the same responses to the AP and RC trials in the SD + MCS condition, although the reward was randomly delivered in this condition. The accuracy rate was significantly higher than chance level. These results excluded the possibility that the monkey learned the 2D motion of the cast shadow in the frontal plane through sessions 2 and 3. All these results in Experiment 2 suggest that the monkey can perceive the motion of the square in depth induced by a cast shadow as well as by binocular disparity.
S. Mizutani et al. / Behavioural Brain Research 288 (2015) 63–70
69
Recently, depth perception in non-human primates by monocular cues is beginning to be studied behaviorally. For example, the macaque monkey can discriminate surface orientations defined by perspective [9] and texture gradient [10], and the 3D structure of surfaces (convex and concave) by shading [11]. An object placed at a farther position in a linear perspective background appears to be larger than an object placed at a nearer position by size constancy. Using this illusion, it has also been suggested that cast shadows affect the perception of object size and spatial configurations of objects in chimpanzees [4]. A recent study [5] investigated the response of infant macaques to another motion illusion by cast shadow referred to as “ball-in-a-box”. In this illusion, a small ball in an open box moves in an oblique trajectory. When the cast shadow of the ball moves along the same trajectory as the ball, the ball appears to roll on the bottom in depth (depth event). Conversely, when the shadow moves away from the ball in the horizontal trajectory, the ball seems to float in the frontal plane (up event). Although the ball moves in the same trajectory in both events, the perceived motion of the ball in 3D space changes based on the shadow’s trajectory [2]. Imura et al. examined the responses of infant macaques to the ball-in-a-box movie by using the preferential looking technique. This technique uses the nature of animals that look at a novel, unfamiliar visual stimulus for longer than the familiar one. They familiarized infant macaques aged 7–24 weeks with the depth event of the “ball-in-a-box” movie and presented the depth and up events to them to examine how long they looked at the movie of each event. In this case, the up event was the novel stimulus to the monkeys that were familiarized with the depth event. They found that the infant monkeys looked at the up event significantly longer than the depth event of the “ball-in-a-box”. This result suggests that the perceived impression of the monkeys was different in the movies of the two events. However, it is still unclear that they really perceive the motion of moving in depth and up in the movies. To overcome this problem, we examined the response of an adult macaque monkey in the motion discrimination in depth induced by a cast shadow. Although the visual stimulus used in the present study was different from the one used by Imura et al. [5], our results showed direct evidence that the macaque monkey can perceive motion in depth using moving cast shadows.
located at separated sites in the visual field by single cortical neurons in the early visual areas. Therefore, the involvement of other cortical areas can be suggested. Indeed, in our previous study using the fMRI technique, activation was observed in the right posterior parietal cortex during observation of the ‘ball-in-a-box’ movie [17]. Moreover, a clinical study reported that a patient who has lesions in large cortical areas extending from the upper calcarine fissure to the occipito-parietal sulcus but spared early visual areas including V1, V2, V3, and hMT/V5 bilaterally showed severe deficit in distinguishing the motion of the ball in the depth and up events of the ‘ball-in-a-box’ illusion [18]. Castiello et al. showed that the posterior superior temporal cortex was commonly damaged in hemispatial neglect patients who were not aware of cast shadows presented in the neglected hemifield [19]. Although the exact cortical areas mentioned above are different, all these studies suggest that higher-order areas may be involved in motion perception in depth by cast shadows. The most direct evidence of the cortical mechanisms of motion perception in depth by cast shadows will be obtained by recording neuronal activities from monkey visual cortex. Previous studies have found neurons selective to the motion of disparity in depth in the extrastriate visual areas including MT [20,21]. However, only a few studies have so far investigated the neuronal responses to the motion of a surface in depth. Sakata et al. found that neurons in area 7a are selective to the approaching and receding motion of a disc presented in front of monkeys [22]. Recently, Sanada and DeAngelis revealed that neurons in MT are selective to the motion in depth of a surface, and this selectivity is driven by a difference in interocular image velocities on the two retinae [23]. Previous studies have revealed that neurons in areas CIP and MT selectively respond to surface orientation defined by both texture gradient and binocular disparity [10,24], indicating that information from binocular and monocular depth cues is integrated in the activity of identical cortical neurons. These findings suggest that motion in depth by cast shadows may be encoded by neurons selective to the motion of a surface induced by binocular depth cues in 7a and MT described above. This is a future problem to be solved, and the present result provides an important step toward it.
4.3. Cortical mechanisms underlying motion perception in depth by cast shadow
5. Conclusion
Little is known so far of the details of the cortical mechanisms underlying motion perception in depth by cast shadows. However, the case of shape perception from shading may serve as a useful reference to elucidate this issue, because it has been pointed out that shape perception from shading and motion perception in depth may share an a priori constraint about light source (light is above the observer and static) during processing [2,3,12,13]. Previous studies using MEG [14], fMRI [15], and single unit recording from macaque monkey [16] have suggested that the early visual cortex may be involved in shape perception from shading. Considering the rapid process of both shape from shading and motion perception in depth by cast shadows, it is possible that the latter is also processed in the early visual cortex. However, Mamassian et al. proposed that at least three stages may be involved in the process of motion perception in depth induced by cast shadows: the segmentation of the cast shadow from the background, the linking of the shadow and an object casting it, and inferring the spatial layout of the object from the relative position of the cast shadow, the object, and the (intrinsic) light source [3]. This model suggests that the integration of information from separated positions in the visual field is necessary in processing motion perception in depth by cast shadows. Considering the small receptive field of neurons, it may be difficult to encode both the object and its cast shadow
In the present study, we showed that a macaque monkey could discriminate the approaching and receding motion of a surface induced by binocular disparity. When the change in the binocular disparity decreased, the performance of the monkey in the discrimination task declined. However, when a moving cast shadow was presented in the visual stimulus, the monkey responded to the approaching and receding motion of the surface induced by the cast shadow in the same way as the binocular disparity, even though the reward was delivered randomly or in all trials to prevent the learning of the 2D motion of the cast shadow in the frontal plane. These observations suggest that the macaque monkey can perceive motion in depth by cast shadows as well as by binocular disparity.
Conflict of interest The authors declare no conflict of interest.
Acknowledgements The Japanese monkey used in the present study was provided through the National BioResource Project “Japanese Monkeys” of the Ministry of Education, Culture, Sports, Science and Technology (MEXT). This research was supported by a Grant-in-Aid for
70
S. Mizutani et al. / Behavioural Brain Research 288 (2015) 63–70
Scientific Research (C) (23500382 to NK and 24500377 to MT) by the MEXT. Appendix A. Supplementary data Supplementary data associated with this article can be found, in the online version, at http://dx.doi.org/10.1016/j.bbr.2015.04.005 References [1] Kersten D, Knill DC, Mamassian P, Bülthoff I. Illusory motion from shadows. Nature 1996;379:31. [2] Kersten D, Mamassian P, Knill DC. Moving cast shadows induce apparent motion in depth. Perception 1997;26:171–92. [3] Mamassian P, Knill DC, Kersten D. The perception of cast shadows. Trends Cogn Sci 1998;2:288–95. [4] Imura T, Tomonaga M. Moving shadows contribute to the corridor illusion in a chimpanzee (Pan troglodytes). J Comp Psychol 2009;123:280–6. [5] Imura T, Adachi I, Hattori Y, Tomonaga M. Perception of the motion trajectory of objects from moving cast shadows in infant Japanese macaques (Macaca fuscata). Dev Sci 2013;16:227–33. [6] Tomonaga M, Imura T. Pacman in the sky with shadows: the effect of cast shadows on the perceptual completion of occluded figures by chimpanzees and humans. Behav Brain Funct 2010;6:38. [7] Coren S. A size contrast illusion without physical size differences. Am J Psychol 1971;84:565–6. [8] Uka T, Tanaka H, Kato M, Fujita I. Behavioral evidence for visual perception of 3-dimensional surface structures in monkeys. Vision Res 1999;39:2399–410. [9] Tsutsui K, Jiang M, Yara K, Sakata H, Taira M. Integration of perspective and disparity cues in surface-orientation-selective neurons of area CIP. J Neurophysiol 2001;86:2856–67.
[10] Tsutsui K, Sakata H, Naganuma T, Taira M. Neural correlates for perception of 3D surface orientation from texture gradient. Science 2002;298:409–12. [11] Zhang Y, Weiner VS, Slocum WM, Schiller PH. Depth from shading and disparity in humans and monkeys. Vis Neurosci 2007;24:207–15. [12] Ramachandran VS. Perception of shape from shading. Nature 1988;331:163–6. [13] Kleffner DA, Ramachandran VS. On the perception of shape from shading. Percept Psychophys 1992;52:18–36. [14] Mamassian P, Jentzsch I, Bacon BA, Schweinberger SR. Neural correlates of shape from shading. Neuroreport 2003;14:971–5. [15] Humphrey GK, Goodale MA, Bowen CV, Gati JS, Vilis T, Rutt BK, et al. Differences in perceived shape from shading correlate with activity in early visual areas. Curr Biol 1997;7:144–7. [16] Lee TS, Yang CF, Romero RD, Mumford D. Neural activity in early visual cortex reflects behavioral experience and higher-order perceptual saliency. Nat Neurosci 2002;5:589–97. [17] Katsuyama N, Usui N, Nose I, Taira M. Perception of object motion in threedimensional space induced by cast shadows. Neuroimage 2011;54:485–94. [18] Michel F, Henaff MA. Seeing without the occipito-parietal cortex: Simultagnosia as a shrinkage of the attentional visual field. Behav Neurol 2004;15:3–13. [19] Castiello U, Lusher D, Burton C, Disler P. Shadows in the brain. J Cogn Neurosci 2003;15:862–72. [20] Poggio GF, Talbot WH. Mechanisms of static and dynamic stereopsis in foveal cortex of the rhesus monkey. J Physiol 1981;315:469–92. [21] Maunsell JH, Van Essen DC. Functional properties of neurons in middle temporal visual area of the macaque monkey. II. Binocular interactions and sensitivity to binocular disparity. J Neurophysiol 1983;49:1148–67. [22] Sakata H, Shibutani H, Kawano K, Harrington TL. Neural mechanisms of space vision in the parietal association cortex of the monkey. Vision Res 1985;25:453–63. [23] Sanada TM, DeAngelis GC. Neural representation of motion-in-depth in area MT. J Neurosci 2014;34:15508–21. [24] Nguyenkim JD, DeAngelis GC. Disparity-based coding of three-dimensional surface orientation by macaque middle temporal neurons. J Neurosci 2003;23:7117–28.
Contents lists available at ScienceDirect
Behavioural Brain Research journal homepage: www.elsevier.com/locate/bbr
Research report
Depth perception from moving cast shadow in macaque monkey Saneyuki Mizutani a,b,c , Nobuo Usui b,c , Takanori Yokota a,c , Masato Taira b,c , Narumi Katsuyama b,c,∗ a b c
Department of Neurology and Neurological Science, Graduate School of Medical and Dental Sciences, Tokyo Medical and Dental University, Tokyo, Japan Department of Cognitive Neurobiology, Graduate School of Medical and Dental Sciences, Tokyo Medical and Dental University, Tokyo, Japan Center for Brain Integration Research, Tokyo Medical and Dental University, Tokyo, Japan
h i g h l i g h t s • • • • •
We examined whether monkey can perceive motion in depth induced by cast shadow. A motion illusion induced by moving cast shadow was used. Monkey could discriminate motion in depth induced by binocular disparity. The illusion was intervened as a catch trial during the discrimination by disparity. Monkey could discriminate the illusionary motion in depth induced by cast shadow.
a r t i c l e
i n f o
Article history: Received 30 January 2015 Received in revised form 2 April 2015 Accepted 4 April 2015 Available online 13 April 2015 Keywords: Motion in depth Cast shadow Macaque monkey Disparity Depth cue
a b s t r a c t In the present study, we investigate whether the macaque monkey can perceive motion in depth using a moving cast shadow. To accomplish this, we conducted two experiments. In the first experiment, an adult Japanese monkey was trained in a motion discrimination task in depth by binocular disparity. A square was presented on the display so that it appeared with a binocular disparity of 0.12 degrees (initial position), and moved toward (approaching) or away from (receding) the monkey for 1 s. The monkey was trained to discriminate the approaching and receding motion of the square by GO/delayed GOtype responses. The monkey showed a significantly high accuracy rate in the task, and the performance was maintained when the position, color, and shape of the moving object were changed. In the next experiment, the change in the disparity was gradually decreased in the motion discrimination task. The results showed that the performance of the monkey declined as the distance of the approaching and receding motion of the square decreased from the initial position. However, when a moving cast shadow was added to the stimulus, the monkey responded to the motion in depth induced by the cast shadow in the same way as by binocular disparity; the reward was delivered randomly or given in all trials to prevent the learning of the 2D motion of the shadow in the frontal plane. These results suggest that the macaque monkey can perceive motion in depth using a moving cast shadow as well as using binocular disparity. © 2015 Elsevier B.V. All rights reserved.
1. Introduction A shadow cast on a background surface by an object is called a cast shadow. Although the shape, transparency, and blur of cast shadows in our daily scenes vary based on the relative position between the light source and the objects casting them, they
∗ Corresponding author at: Department of Cognitive Neurobiology, Tokyo Medical and Dental University, 1-5-45, Yushima, Bunkyo-ku, Tokyo 113-8549, Japan. Tel.: +81 3 5803 5445; fax: +81 3 5803 0165. E-mail address: [email protected] (N. Katsuyama). http://dx.doi.org/10.1016/j.bbr.2015.04.005 0166-4328/© 2015 Elsevier B.V. All rights reserved.
serve as a powerful monocular depth cue in spatial vision. This is well demonstrated by a motion illusion named ‘square-overcheckerboard’ (SOC) provided by Kersten et al. [1]. Fig. 1A illustrates shots from the movie. There is a square at the center with its shadow cast on the background at the right-bottom corner. When the cast shadow moves away from and toward the square in the frontal plane, observers experience a strong percept that the square approaches and recedes from them, respectively, whereas the size and position of the square are actually constant during the movie. Thus, cast shadows can provide salient depth cues for the perception of object position in three-dimensional (3D) space [1–3]. The effect of cast shadows as monocular depth cues is more prominent
64
S. Mizutani et al. / Behavioural Brain Research 288 (2015) 63–70
Fig. 1. A: Motion of the cast shadow. The shadow was presented at the initial position in all trials (center), and moved either away from (approaching, right) or toward (receding, left) the square in the AP and RC trials, respectively. In the DD + SCS conditions in session 4, the cast shadow did not move from the initial position. B: Basic configuration of visual stimulus. In session 1, the square was also presented to the right of, above and below the FP at the same distance from the FP. C: Motion of the square defined by binocular disparity. It was presented initially at 0.12 degrees of the uncrossed disparity. This corresponded to a location 25 mm in front of the background (initial position). The square moved either 0.14 degrees toward the monkey (approaching) or 0.12 degrees toward the background (receding) in the AP and RC trials, respectively. In the rDD condition, the motion of the square from the initial position was reduced to 0.021, 0.016, 0.010, and 0.0052 degrees (corresponding to an estimated distance of 4, 3, 2, and 1 mm). In the SD condition, the square did not move from the initial position. deg.: degrees.
in the movie, because it becomes easier to link objects to their cast shadows in a dynamic scene than in a static image [3]. As a cast shadow moves away from the object casting it, it can provide ambiguous information about the spatial layout of the object. Nevertheless, we can precisely infer the spatial position of the object by using the cast shadow. Previous studies hypothesized that to infer the spatial positions of an object in 3D space from its cast shadow, the visual system uses the a priori constraint that the light source is above the observers and is stable [2,3]. If this hypothesis is correct, it is expected that non-human primates who have an anatomically and functionally similar visual system to humans may also take advantage of cast shadows for spatial vision. However, only a few studies have investigated this topic so far [4–6]. In the present study, we investigated whether adult macaque monkeys can perceive motion in depth induced by cast shadows. For this purpose, we conducted two experiments. In the first experiment, we trained an adult macaque in a motion discrimination task in depth by binocular disparity. In the next experiment, we presented a moving cast shadow as part of the visual stimulus and examined the responses of the monkey to the apparent motion in depth induced by the shadow. Our results suggest that the adult macaque monkey can indeed perceive the motion in depth induced by cast shadows as well as by binocular disparity.
2. Methods and materials 2.1. Materials We used a female Japanese monkey (Macaca fuscata). This monkey was 4 years and 5 months old when it was first presented with visual stimuli containing cast shadows. Throughout the experiments, the monkey was treated in accordance with the NIH Guide for Care and Use of Laboratory Animals. All animal experiments were approved by the Institutional Animal Care and Use Committee
of Tokyo Medical and Dental University (approval number: 0150187A). 2.2. Experimental setup All stimuli in the study were shown using software for 3D presentation (Omega Space ver. 3.1, Solidray Institute, Yokohama, Kanagawa, Japan). A liquid crystal display (LCD-3D231XBR, I-O DATA, Kanazawa, Ishikawa, Japan) was placed at 0.6 m in front of the monkey at eye level, and modified versions of the SOC movie were presented in a 265 × 155 mm (width × height) window attached to the front of the display. The monkey was trained to sit in a primate chair with its head fixed. It wore liquid crystal shutters (NVIDIA 3D Vision® , NVIDIA, Santa Clara, CA, USA) for stereo vision, operating at 120 Hz, so 60 frames/s of stimulus were presented to each eye. We implanted scleral search coils (DNI instrument, Newark, DE, USA) for monitoring eye position and vergence. A trial was aborted immediately when the eye position exceeded the limit of 1◦ from the fixation point (in this article, we describe the visual angle and binocular disparity with◦ and degrees, respectively, for clarity). The original images for the movie frames used in this study were created using drawing software (Illustrator CS5, Adobe systems incorporated, San Jose, CA, USA) and converted into movies by hand-made programs written in Matlab (MathWorks, Natick, MA, USA). The original spatial resolution of each movie was 72 pixels/inch and the frame rate of the movies was 27 frames/second. Binocular disparity was calculated on the basis of an interocular distance of 0.03 m for the monkey in the present study. 2.3. Experiments 2.3.1. Experiment 1: motion discrimination in depth by disparity We first trained the monkey in a GO/delayed GO (dGO)-type discrimination task of the motion in depth defined by disparity. A trial began with presentation of a checkerboard (13.27 × 16.03◦ in
S. Mizutani et al. / Behavioural Brain Research 288 (2015) 63–70
A 100
B
%
100
60 40 20
AP
RC
Left
40
0
Right
C %
D 100
60 40
Left
Right
Up
Down
%
80 Accuracy rate
80 Accuracy rate
60
20
0
100
%
80 Accuracy rate
Accuracy rate
80
65
60 40
20
20
0
0
Fig. 2. Results from the motion discrimination task in depth by binocular disparity in Experiment 1 (DD condition). A, Left: Accuracy rate when the green square was presented to the left of the FP in the AP and RC trials. Right: Accuracy rate of the response to the leftward and rightward motion of the retinal images of the square. Accuracy rate of the responses to the change in the position (B), color (C), and the shape of the moving object (D). The responses to the AP and RC trials are merged in B, C, and D. The broken line indicates chance level.
height × width) as background on the display (Fig. 2B). After a variable duration of 1000–1200 ms, a red circle (diameter = 0.24◦ ) was presented at the center of the checkerboard (fixation point, FP). The monkey had to fixate on the FP and hold a lever within 4000 ms after the presentation of the FP. After a variable duration of 300–500 ms following fixation, a green square (2.82 × 2.82◦ in width × height) appeared 3.05◦ to the left of the FP with binocular disparity of 0.12 degrees (Fig. 2B). The simulated position of the square in depth was 25 mm in front of the background. This position is referred to as the initial position of the square (Fig. 1C). After a variable duration (1000–1200 ms), the square ‘moved’ 0.14 degrees toward the observer (approaching, AP) or 0.12 degrees toward the background (receding, RC) for 1 s, and disappeared at the foremost or innermost positions, respectively (Fig. 1C). This change in the disparity corresponded to a motion of 25 mm toward the observer and toward the background from the initial position of the square. In the following, this is referred to as the dynamic disparity (DD) condition (Table 1). The binocular disparity was the only cue for the motion of the square in depth in this condition. The size of the square was constant throughout the experiments. The velocity of the square changed sinusoidally along the trajectory. The monkey was rewarded by a drop of juice if it released the lever when the color of the FP turned from red to green in the AP trials (GO response). However, it had to keep holding the lever when the FP was green (1000–1500 ms) and was rewarded if it released the lever when the FP turned off in the RC trials (dGO response). Once the monkey achieved an accuracy rate of > 90%, we changed the properties of the moving object in the AP and RC trials. First, we presented the green square at 3.05◦ to the right of, above, and below the FP in the AP and RC trials. Next, we changed the color of the square to cyan, red, blue, orange, yellow, and magenta. Finally, we changed the shape of the moving object to a circle, triangle, annulus, star, and an uppercase letter E. We gave the moving object various combinations of these properties to confirm that the monkey performed the task based on the motion in depth only. At the end of the experiment,
Table 1 Depth cues and reward delivery in the discrimination tasks. Experiment 1 Depth cue
Reward
DD
+
Experiment 2 Depth cue
Reward
Depth cue
Reward
Session 1 DD rDD SD
+ + §
Session 4 DD DD+SCS DD+SCS SD+MCS
+ + § §
Session 2 DD rDD + MCS Session 3 DD SD + MCS
+ ¶ + ¶
DD: dynamic disparity, MCS: moving cast shadow, rDD: reduced dynamic disparity, SCS: static cast shadow, SD: static disparity, SS: static shadow. +: reward was delivered for correct responses, §: reward was delivered randomly, irrespective of the responses, ¶: reward was given for all responses.
we presented the movie frame either to one (the left or right) or both eyes to confirm that the monkey did not respond to the horizontal motion of the square in the frontal plane. 2.3.2. Experiment 2: motion discrimination in depth by adding moving cast shadow In this experiment, we examined whether the monkey could discriminate the motion in depth using a cast shadow. The following visual stimuli were used (Table 1). (1) DD: this was the same condition as used in Experiment 1. As described above, the only cue for the motion in depth in this condition was binocular disparity. (2) reduced DD (rDD): the change in the disparity from the initial position of the square was reduced to 0.021, 0.016, 0.010, and 0.0052 degrees. These values were equivalent to a motion of 4, 3, 2, and 1 mm forward and backward from the initial position of the square in the AP and RC trials, respectively. The cue for the motion in
66
S. Mizutani et al. / Behavioural Brain Research 288 (2015) 63–70
depth in this condition was binocular disparity, although the effect weakened as the disparity reduced. (3) static disparity (SD): the square was presented at the initial position for 1 s and then disappeared without motion. Although binocular disparity was present in this condition, it provided no cues for the motion in depth, for it was static. The reward was delivered randomly, irrespective of the response. (4) rDD + MCS: a moving cast shadow (MCS) was added to the rDD condition. After the fixation, a cast shadow was presented at the right-bottom corner of the square at 0.86◦ in the center-tocenter distance from the square. This position is referred to as the initial position of the cast shadow (Fig. 1A). The shadow moved away from and toward the square on an oblique trajectory for 1 s in the AP and RC trials, respectively, and disappeared with the square. The right-most position of the cast shadow in the AP trials was 1.72◦ in the center-to-center distance from the square. The body of the shadow was transparent and the border was blurred so that the apparent motion of the square in depth by the cast shadow was maximized [1,2]. The velocity, transparency, and blur of the border of the shadow changed sinusoidally, and the transparency and blur became maximal at the right-most position. The apparent motion direction of the square induced by the cast shadow and binocular disparity were congruent. Both cast shadow and binocular disparity could be cues for motion in depth in this condition. However, it was expected that the depth cue provided by the cast shadow would exceed that by binocular disparity as the disparity was reduced. To prevent the learning of the 2D motion of the shadow in the frontal plane, the reward was given in all trials irrespective of the response. (5) SD + MCS: the moving cast shadow was added to the SD condition. Since the binocular disparity was static, the moving cast shadow was the only cue for the motion of the square in depth in this condition. As in the rDD + MCS, the reward was delivered in all trials. These conditions were presented to the monkey in a block (Table 1). In session 1, the conditions DD, rDD, and SD were presented in a block to examine whether the performance of the monkey declined as the motion in depth induced by binocular disparity was reduced. The 3 conditions were equally intermingled in a randomized order. In session 2, we examined the responses of the monkey when the moving cast shadow was presented under the condition in which the motion of the square in depth induced by binocular disparity became ambiguous. One of the rDD + MCS with different disparities (0.021, 0.016, 0.010, and 0.0052 degrees) was presented randomly as a catch condition with the DD in a block at a ratio of DD:rDD + MCS = 4:1. In session 3, we examined the performance of the monkey when the moving cast shadow was presented under the condition in which the square did not move from the initial position induced by binocular disparity. The SD + MCS was presented randomly as a catch condition with the DD in a block at a ratio of DD:SD + MCS = 4:1. This session was performed only after the completion of session 2, in which the disparity changed by up to 0.0052 degrees. In sessions 2 and 3, the test was stopped when the number of trials in the catch condition (the rDD + MCS and SD + MCS in sessions 2 and 3, respectively) reached 10 trials for each of the AP and RC trials to prevent the monkey familiarizing itself with the moving cast shadow (about 100 trials for a session in total). In session 4, we examined whether the monkey was conditioned to the 2D motion of the moving cast shadow in session 2 and 3. For this purpose, the monkey was re-trained in the DD condition with the responses switched: the GO response to the RC and the dGO to the AP trials. Then the SD + MCS was tested again with the DD condition as in session 3. In this session, the SD + MCS condition was presented in 10% of all trials, and the reward was delivered randomly irrespective of the responses of the monkey. Moreover, to prevent the monkey from attending excessively to the cast shadow presented in the SD + MCS, a static cast shadow (SCS) was presented in some trials of the DD condition. This condition is referred to as
the DD + SCS condition (Table 1). The cast shadow was presented at the initial position of the shadow and did not move as the binocular disparity changed. The disparity and static shadow disappeared after 1 s. Thus, in this condition, the cue for motion of the square in depth was binocular disparity. The reward was delivered for correct responses in the half of the SD + MCS trials and was given randomly irrespective of the responses in the other half of the condition. The ratio of the DD, SD + MCS, and the two DD + SCS conditions was 7:1:1:1 (Table 1). Six blocks were conducted on different days. A block was stopped when the number of trials reached 10 of each of the AP and RC trials in the SD + MCS condition (about 200 trials for the 4 conditions). 2.4. Statistical analysis We used a one-tailed binomial test implemented in the statistical program R (http://cran.r-project.org) to evaluate the success accuracy rate with respect to chance level. P values < 0.01 were considered statistically significant. 3. Results 3.1. Experiment 1: motion discrimination in depth by binocular disparity Fig. 2 indicates the results of Experiment 1. The monkey showed a high accuracy rate (> 90%) in the discrimination of the approaching and receding motion of the square induced by binocular disparity (Fig. 2A, left). However, the performance declined to chance level when the movie frame either to the left or right eye was presented to both eyes (Fig. 2A, right). The high accuracy rate was maintained for the change in the location (Fig. 2B) and color (Fig. 2C) of the square and the shape of the moving object (Fig. 2D). The accuracy rate presented in this figure was significantly higher than chance level except in Fig. 2A, right. 3.2. Experiment 2: motion discrimination in depth by adding a moving cast shadow In session 1, we randomly presented the DD, rDD, and SD conditions (Fig. 3, S1). The monkey showed an accuracy rate that was significantly higher (> 90%) than chance level in the DD conditions. However, performance declined as the motion of the square in depth induced by binocular disparity decreased (rDD, from top to bottom in Fig. 3, S1). The accuracy rate in the AP trials was not significantly different from chance level when the disparity was less than 0.016 degrees, whereas in the RC trials, the accuracy rate was significantly higher than chance level over all disparities. Performance was generally around chance level when the square did not move from the initial position (SD) In session 2, we added the moving cast shadow to the rDD condition (rDD + MCS in Fig. 3, S2). The monkey responded to the AP and RC trials in the rDD + MCS condition with GO and dGO responses, respectively, as in the DD condition. The accuracy rate was significantly higher than chance level for both the AP and RC trials. A reward was given in all trials, irrespective of the response in the rDD + MCS condition. In session 3, we presented the moving cast shadow in the SD condition (SD + MCS in Fig. 3, S3). The monkey showed the same responses in the SD + MCS as in the DD trials, and the accuracy rate was also significantly higher than chance level for both the AP and RC trials. The number of trials tested, the accuracy rate, and the p value obtained via binomial tests for each block are shown in the supplemental Table 1. In session 4, the monkey was re-trained with the responses to the AP and RC trials reversed, and tested in the SD + MCS condition
S. Mizutani et al. / Behavioural Brain Research 288 (2015) 63–70
67
Fig. 3. Results from sessions 1–3 in Experiment 2. Performance in the AP and RC trials are shown in the left and right columns, respectively. Accuracy rates of each condition in sessions 1–3 are aligned from top to bottom in the order of the binocular disparity presented in the rDD condition. The amount of binocular disparity change is indicated at the left side of the graphs, and the simulated distance of the square with respect to the initial position is shown in parentheses. The dotted line indicates the chance level. S1, 2, and 3 at the bottom indicate session 1, 2, and 3, respectively. AP: approaching, RD: receding, deg.: degrees, for the other abbreviations, refer to the footnote in Table 1.
again. The accuracy rate of the DD and SD + MCS conditions from the six blocks are summed and plotted in Fig. 4. The monkey responded correctly to the AP and RC trials with the dGO and GO responses in the SD + MCS condition, respectively, as well as in the DD. The accuracy rate was significantly higher than chance level in both
the AP and RC trials. The accuracy rate in the DD + SCS condition was as high as that in the DD condition, regardless of whether the reward was given randomly or for correct responses only. The number of trials and the accuracy rate in each block are indicated in supplemental Table 2.
68
S. Mizutani et al. / Behavioural Brain Research 288 (2015) 63–70
AP 100
%
RC 100 80
Acuuracy rate
Acuuracy rate
80 60 40 20 0
%
60 40 20
DD
SD+ MCS
0
DD
SD+ MCS
Fig. 4. Results from session 4 in Experiment 2. Accuracy rates for the AP and RC trials in the DD and SD + MCS conditions are shown. AP: approaching, RD: receding, for the other abbreviations, refer to the footnote in Table 1. The dotted line indicates the chance level.
4. Discussion The aim of this study was to examine whether the macaque monkey could perceive motion in depth induced by a moving cast shadow. To do this, we conducted two experiments in the present study. In the first experiment, we trained a macaque monkey in a motion discrimination task in depth defined by binocular disparity with a GO/dGO-type response. In the second experiment, we presented a moving cast shadow in the visual stimulus in which the motion in depth induced by binocular disparity was ambiguous, and tested the responses of the monkey. We hypothesized that if the monkey could perceive the apparent motion in depth by moving cast shadow, it should show the same responses in motion discrimination by the moving cast shadow as well as by binocular disparity. 4.1. Motion discrimination in depth by binocular disparity In Experiment 1, an adult macaque monkey was presented with a square that appeared to move toward (approaching) or away from (receding) the animal by binocular disparity and was trained to respond to the approaching and receding motion of the square with GO and dGO responses, respectively (DD condition). The square was presented at 0.12 degrees (simulated at 25 mm in front of the background) before moving. This was to make the monkey perform the task based on the motion of the square, not the position of the square where it was presented initially in a trial. For this reason, the square was always initially presented at 0.12 degrees in all trials of this study, and moved forward or backward from this position. In the present study, the size of the square was kept constant in the AP and RC trials. It is well known that an object moving toward or away from an observer with its retinal size constant appears to shrink or expand, respectively [7]. This size change might have an effect on the motion of the square in depth. Therefore, two of the authors independently tested the apparent size change by watching a square moving in depth with a variety of disparities in the preliminary experiment, and confirmed that the size change was negligible when the square was presented at less than 0.26 degrees (50 mm in front of the background). Thus, the most forward position of the square in the AP trials was determined to be 0.26 degrees, and the initial position of the square was defined to be half of this (0.12 degrees, 25 mm in front of the background). We found that the monkey could discriminate the approaching and receding motion of the square induced by binocular disparity at a high accuracy rate. When the movie frame for either the right or left eye was presented to both eyes, the performance declined
to chance level, indicating that the monkey did not respond to the horizontal motion of the retinal images of the square. To confirm further that the monkey responded solely to the motion in depth in the task, we changed the properties of the moving object, such as its position, color, and shape. The square was presented to the left of, to the right of, above, and below the FP. The square was replaced with geometric figures such as a circle, triangle, star, annulus, and an uppercase letter E. Nevertheless, the monkey showed a high accuracy rate in the discrimination of the motion of an object with a single, or a combination of the different properties. This result indicates that the monkey did not respond to the local features of the moving object, but to the motion of the object in depth. Previous studies have revealed that the macaque monkey can discriminate static 3D surface structures defined by binocular disparity [8–11], but no study has tested macaque monkeys behaviorally in a motion discrimination task in depth. The present results indicate that the macaque monkey can discriminate the motion of a surface in depth as well as static 3D structures using binocular disparity.
4.2. Motion discrimination in depth by adding moving cast shadow In the next experiment, we examined whether cast shadow helps the monkey to perceive motion in depth. In session 1, we reduced the binocular disparity gradually (rDD condition) and examined the responses of the monkey. The reduction of the disparity resulted in a decrease of the motion of the square from the initial position in both the AP and RC trials. As expected, the performance of the monkey declined as the disparity was reduced, and the motion of the square became invisible. In session 2, we presented the moving cast shadow in the rDD (rDD + MCS condition). We hypothesized that if the monkey could perceive the motion in depth by the moving cast shadow, it should show the same responses to the AP and RC trials in the rDD + MCS as in the DD, when the motion of the square was invisible. The result showed that this was true – the monkey responded to the AP and RC trials with the GO and dGO responses, respectively. Similarly, the monkey showed the same responses to the AP and RC trials in the SD + MCS condition in session 3 with a significantly higher accuracy rate than chance level. The responses in the rDD + MCS and SD + MCS conditions were not a result of learning the 2D motion of the cast shadow in the frontal plane, because the reward was delivered in all trials of these conditions. In the SD + MCS condition, the cast shadow was the only depth cue for the motion of the square. In this condition, the square was presented at the initial position, and there was no motion induced by binocular disparity. Instead, the cast shadow moved in the frontal plane. Nevertheless, the monkey made the same responses as in the DD condition. These observations suggest that the monkey could perceive the approaching and receding motion of the square induced by the moving cast shadow. To confirm further that the monkey was not conditioned to the 2D motion of the cast shadow in sessions 2 and 3, we re-trained the monkey in the DD condition with the responses to the AP and RC trials reversed, and examined the response of the monkey to the SD + MCS condition again in session 4. After the re-training, the monkey responded to the AP and RC trials in the DD with the dGO and GO responses, respectively. Moreover, it showed the same responses to the AP and RC trials in the SD + MCS condition, although the reward was randomly delivered in this condition. The accuracy rate was significantly higher than chance level. These results excluded the possibility that the monkey learned the 2D motion of the cast shadow in the frontal plane through sessions 2 and 3. All these results in Experiment 2 suggest that the monkey can perceive the motion of the square in depth induced by a cast shadow as well as by binocular disparity.
S. Mizutani et al. / Behavioural Brain Research 288 (2015) 63–70
69
Recently, depth perception in non-human primates by monocular cues is beginning to be studied behaviorally. For example, the macaque monkey can discriminate surface orientations defined by perspective [9] and texture gradient [10], and the 3D structure of surfaces (convex and concave) by shading [11]. An object placed at a farther position in a linear perspective background appears to be larger than an object placed at a nearer position by size constancy. Using this illusion, it has also been suggested that cast shadows affect the perception of object size and spatial configurations of objects in chimpanzees [4]. A recent study [5] investigated the response of infant macaques to another motion illusion by cast shadow referred to as “ball-in-a-box”. In this illusion, a small ball in an open box moves in an oblique trajectory. When the cast shadow of the ball moves along the same trajectory as the ball, the ball appears to roll on the bottom in depth (depth event). Conversely, when the shadow moves away from the ball in the horizontal trajectory, the ball seems to float in the frontal plane (up event). Although the ball moves in the same trajectory in both events, the perceived motion of the ball in 3D space changes based on the shadow’s trajectory [2]. Imura et al. examined the responses of infant macaques to the ball-in-a-box movie by using the preferential looking technique. This technique uses the nature of animals that look at a novel, unfamiliar visual stimulus for longer than the familiar one. They familiarized infant macaques aged 7–24 weeks with the depth event of the “ball-in-a-box” movie and presented the depth and up events to them to examine how long they looked at the movie of each event. In this case, the up event was the novel stimulus to the monkeys that were familiarized with the depth event. They found that the infant monkeys looked at the up event significantly longer than the depth event of the “ball-in-a-box”. This result suggests that the perceived impression of the monkeys was different in the movies of the two events. However, it is still unclear that they really perceive the motion of moving in depth and up in the movies. To overcome this problem, we examined the response of an adult macaque monkey in the motion discrimination in depth induced by a cast shadow. Although the visual stimulus used in the present study was different from the one used by Imura et al. [5], our results showed direct evidence that the macaque monkey can perceive motion in depth using moving cast shadows.
located at separated sites in the visual field by single cortical neurons in the early visual areas. Therefore, the involvement of other cortical areas can be suggested. Indeed, in our previous study using the fMRI technique, activation was observed in the right posterior parietal cortex during observation of the ‘ball-in-a-box’ movie [17]. Moreover, a clinical study reported that a patient who has lesions in large cortical areas extending from the upper calcarine fissure to the occipito-parietal sulcus but spared early visual areas including V1, V2, V3, and hMT/V5 bilaterally showed severe deficit in distinguishing the motion of the ball in the depth and up events of the ‘ball-in-a-box’ illusion [18]. Castiello et al. showed that the posterior superior temporal cortex was commonly damaged in hemispatial neglect patients who were not aware of cast shadows presented in the neglected hemifield [19]. Although the exact cortical areas mentioned above are different, all these studies suggest that higher-order areas may be involved in motion perception in depth by cast shadows. The most direct evidence of the cortical mechanisms of motion perception in depth by cast shadows will be obtained by recording neuronal activities from monkey visual cortex. Previous studies have found neurons selective to the motion of disparity in depth in the extrastriate visual areas including MT [20,21]. However, only a few studies have so far investigated the neuronal responses to the motion of a surface in depth. Sakata et al. found that neurons in area 7a are selective to the approaching and receding motion of a disc presented in front of monkeys [22]. Recently, Sanada and DeAngelis revealed that neurons in MT are selective to the motion in depth of a surface, and this selectivity is driven by a difference in interocular image velocities on the two retinae [23]. Previous studies have revealed that neurons in areas CIP and MT selectively respond to surface orientation defined by both texture gradient and binocular disparity [10,24], indicating that information from binocular and monocular depth cues is integrated in the activity of identical cortical neurons. These findings suggest that motion in depth by cast shadows may be encoded by neurons selective to the motion of a surface induced by binocular depth cues in 7a and MT described above. This is a future problem to be solved, and the present result provides an important step toward it.
4.3. Cortical mechanisms underlying motion perception in depth by cast shadow
5. Conclusion
Little is known so far of the details of the cortical mechanisms underlying motion perception in depth by cast shadows. However, the case of shape perception from shading may serve as a useful reference to elucidate this issue, because it has been pointed out that shape perception from shading and motion perception in depth may share an a priori constraint about light source (light is above the observer and static) during processing [2,3,12,13]. Previous studies using MEG [14], fMRI [15], and single unit recording from macaque monkey [16] have suggested that the early visual cortex may be involved in shape perception from shading. Considering the rapid process of both shape from shading and motion perception in depth by cast shadows, it is possible that the latter is also processed in the early visual cortex. However, Mamassian et al. proposed that at least three stages may be involved in the process of motion perception in depth induced by cast shadows: the segmentation of the cast shadow from the background, the linking of the shadow and an object casting it, and inferring the spatial layout of the object from the relative position of the cast shadow, the object, and the (intrinsic) light source [3]. This model suggests that the integration of information from separated positions in the visual field is necessary in processing motion perception in depth by cast shadows. Considering the small receptive field of neurons, it may be difficult to encode both the object and its cast shadow
In the present study, we showed that a macaque monkey could discriminate the approaching and receding motion of a surface induced by binocular disparity. When the change in the binocular disparity decreased, the performance of the monkey in the discrimination task declined. However, when a moving cast shadow was presented in the visual stimulus, the monkey responded to the approaching and receding motion of the surface induced by the cast shadow in the same way as the binocular disparity, even though the reward was delivered randomly or in all trials to prevent the learning of the 2D motion of the cast shadow in the frontal plane. These observations suggest that the macaque monkey can perceive motion in depth by cast shadows as well as by binocular disparity.
Conflict of interest The authors declare no conflict of interest.
Acknowledgements The Japanese monkey used in the present study was provided through the National BioResource Project “Japanese Monkeys” of the Ministry of Education, Culture, Sports, Science and Technology (MEXT). This research was supported by a Grant-in-Aid for
70
S. Mizutani et al. / Behavioural Brain Research 288 (2015) 63–70
Scientific Research (C) (23500382 to NK and 24500377 to MT) by the MEXT. Appendix A. Supplementary data Supplementary data associated with this article can be found, in the online version, at http://dx.doi.org/10.1016/j.bbr.2015.04.005 References [1] Kersten D, Knill DC, Mamassian P, Bülthoff I. Illusory motion from shadows. Nature 1996;379:31. [2] Kersten D, Mamassian P, Knill DC. Moving cast shadows induce apparent motion in depth. Perception 1997;26:171–92. [3] Mamassian P, Knill DC, Kersten D. The perception of cast shadows. Trends Cogn Sci 1998;2:288–95. [4] Imura T, Tomonaga M. Moving shadows contribute to the corridor illusion in a chimpanzee (Pan troglodytes). J Comp Psychol 2009;123:280–6. [5] Imura T, Adachi I, Hattori Y, Tomonaga M. Perception of the motion trajectory of objects from moving cast shadows in infant Japanese macaques (Macaca fuscata). Dev Sci 2013;16:227–33. [6] Tomonaga M, Imura T. Pacman in the sky with shadows: the effect of cast shadows on the perceptual completion of occluded figures by chimpanzees and humans. Behav Brain Funct 2010;6:38. [7] Coren S. A size contrast illusion without physical size differences. Am J Psychol 1971;84:565–6. [8] Uka T, Tanaka H, Kato M, Fujita I. Behavioral evidence for visual perception of 3-dimensional surface structures in monkeys. Vision Res 1999;39:2399–410. [9] Tsutsui K, Jiang M, Yara K, Sakata H, Taira M. Integration of perspective and disparity cues in surface-orientation-selective neurons of area CIP. J Neurophysiol 2001;86:2856–67.
[10] Tsutsui K, Sakata H, Naganuma T, Taira M. Neural correlates for perception of 3D surface orientation from texture gradient. Science 2002;298:409–12. [11] Zhang Y, Weiner VS, Slocum WM, Schiller PH. Depth from shading and disparity in humans and monkeys. Vis Neurosci 2007;24:207–15. [12] Ramachandran VS. Perception of shape from shading. Nature 1988;331:163–6. [13] Kleffner DA, Ramachandran VS. On the perception of shape from shading. Percept Psychophys 1992;52:18–36. [14] Mamassian P, Jentzsch I, Bacon BA, Schweinberger SR. Neural correlates of shape from shading. Neuroreport 2003;14:971–5. [15] Humphrey GK, Goodale MA, Bowen CV, Gati JS, Vilis T, Rutt BK, et al. Differences in perceived shape from shading correlate with activity in early visual areas. Curr Biol 1997;7:144–7. [16] Lee TS, Yang CF, Romero RD, Mumford D. Neural activity in early visual cortex reflects behavioral experience and higher-order perceptual saliency. Nat Neurosci 2002;5:589–97. [17] Katsuyama N, Usui N, Nose I, Taira M. Perception of object motion in threedimensional space induced by cast shadows. Neuroimage 2011;54:485–94. [18] Michel F, Henaff MA. Seeing without the occipito-parietal cortex: Simultagnosia as a shrinkage of the attentional visual field. Behav Neurol 2004;15:3–13. [19] Castiello U, Lusher D, Burton C, Disler P. Shadows in the brain. J Cogn Neurosci 2003;15:862–72. [20] Poggio GF, Talbot WH. Mechanisms of static and dynamic stereopsis in foveal cortex of the rhesus monkey. J Physiol 1981;315:469–92. [21] Maunsell JH, Van Essen DC. Functional properties of neurons in middle temporal visual area of the macaque monkey. II. Binocular interactions and sensitivity to binocular disparity. J Neurophysiol 1983;49:1148–67. [22] Sakata H, Shibutani H, Kawano K, Harrington TL. Neural mechanisms of space vision in the parietal association cortex of the monkey. Vision Res 1985;25:453–63. [23] Sanada TM, DeAngelis GC. Neural representation of motion-in-depth in area MT. J Neurosci 2014;34:15508–21. [24] Nguyenkim JD, DeAngelis GC. Disparity-based coding of three-dimensional surface orientation by macaque middle temporal neurons. J Neurosci 2003;23:7117–28.