- Email: [email protected]
Visual mental imagery: What the head's eye tells the mind's eye
BR A I N R ES E A RC H 1 3 6 7 ( 2 01 1 ) 2 8 7 –2 97
available at www.sciencedirect.com
www.elsevier.com/locate/brainres
Research Report
Visual mental imagery: What the head's eye tells the mind's eye Clémence Bourlon a,b,c,d,⁎, Bastien Oliviero a , Nicolas Wattiez a,b , Pierre Pouget a,b , Paolo Bartolomeo a,b,d,e,f a Centre de Recherche de l'Institut du Cerveau et de la Moelle Epinière, Inserm UMR S975, CNRS 7225, Pavillon Claude Bernard, Hôpital Pitié Salpêtrière (AP-HP), Paris, France b UPMC Université Paris 06, France c Service de Rééducation et de Réadaptation Fonctionnelle, Clinique Les Trois Soleils, Boissise le Roi, France d Service de Médecine et de Réadaptation, Hôpital National de Saint Maurice, France e Fédération de Neurologie, Hôpital Pitié-Salpêtrière (AP-HP) Paris, France f Department of Psychology, Catholic University, Milan, Italy
A R T I C LE I N FO
AB S T R A C T
Article history:
The demonstration of an implication of attentional/eye gaze systems in visual mental
Accepted 12 October 2010
imagery might help to understand why some patients with visual neglect, who suffer from
Available online 19 October 2010
severe attentional deficits, also show neglect for mental images. When normal participants generate mental images of previously explored visual scenes, their oculomotor behavior
Keywords:
resembles that used during visual exploration. However, this could be a case of encoding
Mental imagery
specificity, whereby the probability of retrieving an event increases if some information
Eye movement
encoded with the event (in this case its spatial location) is present at retrieval. In the present
Neglect
study, normal participants were invited to conjure up a mental image of the map of France
Attention
and to say whether auditorily presented towns or regions were situated left or right of Paris. A perceptual version of the task was administered after the imaginal condition. Thus, in the imaginal condition participants had to retrieve information from long-term memory. Vocal response times and, unbeknownst to participants, also eye movements were recorded. Participants tended to produce similar eye movements on the imaginal and on the perceptual conditions of the task. We concluded that some mechanisms involved in spontaneous oculomotor behavior may be shared in exploration of visuospatial mental images. Deficits of these common processes participating in the oculomotor exploration might contribute to imaginal neglect. © 2010 Elsevier B.V. All rights reserved.
1.
Introduction
The mechanisms of visual mental imagery (VMI; the faculty whereby we can imagine or “visualize” objects and places in their absence) have fostered intense research and debate during the last decades. One major question of interest concerns whether, and to what extent, VMI shares functional processes
and neural substrates with visual perception (Pylyshyn, 2003; Kosslyn et al., 2006; Bartolomeo, 2008). According to an influential model, visual mental images are “displayed” on a visual buffer consisting of topographically organized areas in the occipital lobe, similar to visual percepts (Kosslyn et al., 2006). However, reports of patients with occipital damage and perceptual deficits but preserved VMI challenge strong versions
⁎ Corresponding author. INSERM UMR S975, Pavillon Claude Bernard, Hôpital Salpêtrière, 47 bd de l'Hôpital, F-75013 Paris, France. E-mail address: [email protected] (C. Bourlon). 0006-8993/$ – see front matter © 2010 Elsevier B.V. All rights reserved. doi:10.1016/j.brainres.2010.10.039
288
BR A I N R ES E A RC H 1 3 6 7 ( 2 01 1 ) 2 8 7 –29 7
of perceptual/imagery equivalence, and suggests that VMI implicates visual processes at higher levels of integration than those performed by primary visual cortex (Bartolomeo, 2002). For example, mechanisms related to visuospatial attention might be activated during spatial forms of VMI (Thomas, 1999; Bartolomeo and Chokron, 2002a,b). Consistent with this idea, some patients with right hemisphere damage and visual neglect (i.e., lack of awareness for left-sided visual items) also display signs of neglect in the representational, or imaginal space. When describing known places from memory, these patients “forget” to mention items laying on the left side of their mental images, regardless of the (imaginary) vantage point (Bisiach and Luzzatti, 1978). An important question raised by these patterns of performance is whether asymmetries in orienting of attention, which are a component deficit in perceptual neglect (Bartolomeo and Chokron, 2002a,b), also play a role in imaginal neglect (Bartolomeo et al., 1994, 2005; Bourlon et al., 2008, 2010). The study of eye movements may help addressing these issues, because eye movements are in most common situations a behavioral correlate of orienting of spatial attention. Although, in the context of the laboratory, attention can be oriented “covertly,” or without eye movements (Posner, 1980), in more ecological situation saccades are usually accompanied by corresponding attentional shifts (Shepherd et al., 1986; Hoffman and Subramaniam, 1995). Thus, if attention is important in spatial VMI, then the production of eye movements should most commonly accompany imaginal tasks. Empirical support for this prediction came from the study of Brandt and Stark (1997), who asked normal participants to visually explore and then revisualize from memory irregularly-checkered diagrams while their eye movements were recorded. For any given picture, sequences of fixations and saccades were closely correlated with those recorded while viewing the same item. A further step was made by Laeng and Teodorescu (2002). Drawing on theoretical positions postulating that eye movements may guide the sequential synthesis of the different spatial components of mental images (Neisser, 1967; Hebb, 1968), Laeng and Teodorescu proposed that eye movements are not merely epiphenomenal, but play a functional role in VMI. In two experiments, participants saw irregular checkerboards or color pictures of fish and formed mental images of these items. Different groups of participants either were free to make eye movements or maintained central fixation during the tasks. The strength of relatedness of eye scanpaths between imagery and perception predicted performance accuracy in VMI. Furthermore, participants free to explore during perception, but maintaining fixation during VMI, had impaired performance in recalling the visual patterns of fish images. Laeng and Teodorescu concluded that reenacting motor sequences of eye movements is useful to achieve vivid mental images. These findings are important in suggesting a functional role for eye movements in VMI. However, a common feature of these studies on eye movements in VMI is the fact that participants viewed patterned visual items before performing the VMI task. As a consequence, their results do not directly explore the role of eye movements when participants retrieve visual mental images from long-term memory, as it is the case with description from memory of known places or similar tests of imaginal neglect. In fact, improvement of memory retrieval
when looking at the empty location where an event has previously occurred (Ferreira et al., 2008) may well be an instance of encoding specificity (Tulving and Thomson, 1973); the probability of retrieving an event increases if some information encoded with the event (in this case its spatial location) is present at retrieval. Additionally, as Laeng and Teodorescu acknowledge, their task design is not immune from the classical objection of participants acting according to the experimenters’ expectancies in the VMI condition (Pylyshyn, 1981; Intons-Peterson, 1983). The two methodological problems described above were addressed by Spivey and Geng (2001, Exp. 1). They recorded eye movements while subjects imagined verbally described scenes. No visual display was used and participants did not know that their eye movements were being recorded at the time. Participants produced more horizontal or vertical saccades when the description included, respectively, horizontal or vertical dimensions. In the present study, we used a task paradigm close to a typical test of imaginal neglect, the mental exploration from memory of a map of France (Rode and Perenin, 1994; Rode et al., 2007). We asked French participants to mentally visualize a map of France. Participants subsequently heard the names of French towns or regions and had to state whether each item was left or right of Paris (Bartolomeo et al., 2005), while their vocal response times (RTs) and eye movements were recorded. A perceptual version of the task was also used (Bourlon et al., 2008), in which participants saw the outline of the map and the same geographical locations appeared as dots within the map. However, all participants performed the perceptual condition after the VMI condition, so that a simple “spillover” of eye movements from visual exploration to VMI revisualization could be excluded. Moreover, we used an eye tracker which does not require any helmet or chin-rest, and participants were entirely unaware that their eye movements were being recorded. As a consequence, it is highly unlikely that they were trying to satisfy the experimenter's expectancies. Finally, we used two VMI conditions before the visual condition. In the “VMI without map” task, participants saw a blank screen when they imagined the map of France. The other condition involved the presentation of an outline of the map of France on the screen during all the experiment (“VMI with map” condition). This last condition was intended to explore the influence of the presence of a visual input on eye movements during VMI. The visual task was always performed after the two VMI conditions, to prevent recent visual experiences from influencing performance. To avoid any effects on performance of the order of the VMI conditions, half of the participants performed first the VMI task with map, then the VMI without map (Map First group, MF); the other half had the VMI tasks in the opposite order (Empty Screen First group, EMF).
2.
Results
2.1.
Response time and accuracy of responses
Statistical analyses were implemented with Statistica software (Statsoft, Inc.). A repeated-measure analysis of variance
BR A I N R ES E A RC H 1 3 6 7 ( 2 01 1 ) 2 8 7 –2 97
(ANOVA) was conducted on mean correct RTs with group, task and side position of the stimulus as factors. A further ANOVA was applied to the arcsin-transformed proportions of correct responses with the same factors. The side of stimulus presentation did not influence response times (RTs) (mean ± SD in ms, left, 1136 ± 294; right, 1127 ms ± 346), nor did the group (MF group, 1154 ± 254; ESF group = 1127 ± 346) or the task condition (VMI with map outline, 1193 ± 404; VMI without map, 1190 ms ± 383; perceptual task, 1129 ms ± 300). However, the group interacted with the task condition (F (2, 28) = 4.03, p = 0.029), because participants were 160 ms slower on the first task condition (i.e., VMI without map for MF group and VMI with map for ESF group) than on the second (Fig. 1A), probably as a consequence of practice. Accuracy was also similar for the two groups (mean number of hits± SD, MF group, 17.77± 1.96; ESF group, 18.02± 2.38) and the two sides of stimulus presentation (left, 17.60± 2.28; right, 18.18± 2.04). A reliable effect of task condition was observed (F (2, 28) = 81.62, p = 0.0001): Perceived stimuli (19.78 ± 0.55) were better detected than imagined stimuli (VMI with map, 16.66± 2.12; VMI without map, 17.25± 2.00). No interaction reached significance (Fig. 1B).
2.2.
Eye movements
The data were log-transformed and submitted to repeated measures ANOVAs. We wanted to test the hypothesis that eye
289
movements are important for VMI even for stimuli recollected from long-term memory, in the absence of any task-related perceptual activity immediately before the experimental session. Thus, we planned specific comparisons to explore eye movements in the VMI condition with a completely empty screen (i.e., without the map outline), for those participants who performed this condition first (ESF group).
2.2.1.
Fixations
The total number of fixations for each stimulus location was measured according to their spatial position. There were fewer fixations for the perceptual condition than for VMI conditions (mean number of fixations ± SD, VMI with map, 32.05 ± 16.96; VMI without map, 28.59 ± 13.81; perceptual task, 22.47 ± 12.33) (F (2, 28) = 4.87, p = 0.015). There was an interaction between the side of stimulus location and the side of fixation (F (1, 14) = 60.07, p = 0.0001), reflecting the fact that participants tended to produce more fixations on the side of the stimulus, independent of group and condition (Fig. 2). Importantly, the planned comparison revealed that this interaction was reliable even for the condition without map performed as first condition, i.e. when participants had not seen any map before performing the test (F (1, 7) = 13.67, p = 0.0077). A potential concern in the interpretation of the oculomotor bias shown by participants on the imaginal task is that it could have been influenced by the subsequent verbal response. Both the imaginal and exploratory task required a left vs right
Fig. 1 – A. Correct response times for left-sided items (hatched bars) and right-sided items (empty bars) on the RT tasks. B. Percentage of correct responses for left-sided items (hatched bars) and right-sided items (empty bars) on the RT tasks. Error bars denote 95% confidence intervals. MF, map first group; ESF, empty screen first group.
290
BR A I N R ES E A RC H 1 3 6 7 ( 2 01 1 ) 2 8 7 –29 7
decision; the response code adopted in the task could have affected or increased, especially in the imaginal task, the oculomotor bias of participants. To address this issue, we determined the first and the last fixation during the decision period, before response production. If subjects tended to look at the (imagined) stimulus location soon after the stimulus presentation, but not just before the verbal response, then an influence of the response on eye movement pattern would be less likely. Consistent with this prediction, subjects tended to make their first fixation towards the side of the stimulus (interaction between side of first fixation and side of stimulus, F (1, 14) = 8.72, p = 0.010). However, the corresponding interaction between side of last fixation and stimulus side was far from significance (F < 1). This suggests that participants first “explored” the side of the imaginary stimulus and then produced the relative verbal response. Moreover, the analysis of the longest fixation during the decision period revealed no significant interaction between the side of the stimulus and the side of the fixation (F (1, 14) = 1.67, p = 0.22). Finally, no correlations emerged with RTs (VMI with map, r = −0.26, p = 0.313; VMI without map, r = −0.21, p = 0.426; perceptual task, r = 0.49, p = 0.053). Concerning accuracy, no significant correlation emerged for the perceptual task (r = −0.30). However, interestingly, accuracy for the imaginal tasks did correlate with the number of fixations (VMI with map, r = 0.52, p = 0.039; VMI without map, r = 0.69, p = 0.003). This suggests that fixations were relatively useless for the perceptual task, where the detection of salient peripheral targets presumably did not require prolonged foveal processing, but they did help improving performance on the imaginal tasks.
2.2.2.
Saccades
The perceptual condition evoked fewer saccades than the two imaginal conditions (mean VMI with map, 30.53 ± 25.18; mean VMI without map = 26.62 ± 20.50; mean perceptual condition, 20.23 ± 17.77), (F (2, 28) = 3.69, p = 0.037). As noticed for the number of fixations participants tended to produce more saccades towards the side of stimulus presentation, again independent of the presence of a map or of the order of conditions (F (1, 14) = 130.93, p = 0.0001) (Figs. 2 and 3). Importantly, as it was the case for fixations, even those participants who had not seen the map outline before the VMI task made more saccades towards the side of the imagined stimulus, F (1, 7) = 45.99, p = 0.00026. Subjects tended to produce their first saccade towards the direction of the stimulus (interaction between first saccade and stimulus side, F (1, 14)= 7.06, p = 0.018). To establish whether not only the direction, but also the metrics of the eye movements depended on the location of imagined stimuli, we computed the correlations between the amplitude of the first saccade and the real distances between Paris and the peripheral targets. As expected, there was a reliable correlation for the perceptual task (r = 0.36, p = 0.046); however, no correlation emerged for the imaginal tasks, either with map (r = 0.25, p = 0.838) or without visible map (r = −0.07, p = 0.704). This result may depend on the lack of a fixation point in the VMI condition, so that subjects did not always start exploration from the center of the screen. In contrast to first saccades, and analogously to fixation patterns, last saccades were not significantly related to the stimulus side (F (1, 14) = 2.22, p = 0.158).
Finally, there were no reliable correlations between the number of saccades and RTs or accuracy of responses (RTs: VMI with map condition, r = 0.25; VMI without map condition, r = 0.26; perceptual task, r = 0.37; accuracy: VMI with map condition, r = −0.31; VMI without map condition, r = 0.04; perceptual task, r = −0.13; all p ns).
3.
Discussion
Human visual experience is not the result of a passive display of external images on the visual system, but a dynamic process during which the eyes continually sample the environment (Findlay and Gilchrist, 2003). In ecological settings, organisms orient and turn their head and gaze towards relevant stimuli (Sokolov, 1963), in order to align these stimuli with the part of the sensory surface with highest resolution (e.g., the retinal fovea). This allows better perceptual processing of the stimuli, for example their identification as useful or as dangerous objects. Orienting movements are thus a typical form of “embodied” cognition (Ballard et al., 1997), that is a process in which body movements are necessary to the processing of information. Thus, bodily orienting movements contribute to optimize processing resources, e.g. the segregation of mechanisms dedicated to simple detection from resources performing more complex identification tasks. Orienting behaviors reminiscent of exploratory saccades emerge even in very simple artificial organisms evolving following a genetic algorithm (Bartolomeo et al., 2002; Di Ferdinando et al., 2007). Patients with left neglect demonstrate severe dysfunctions of orienting behavior; their attention and gaze is often compulsorily captured by right-sided visual stimuli (Gainotti et al., 1991). However, in VMI tasks there are no visual stimuli by definition, and yet some neglect patients do seem to neglect the left side of their mental images. A hypothesis proposed to account for these surprising patterns of performance is to postulate that attentional processes, which are typically impaired in visual neglect (Bartolomeo and Chokron, 2002a,b), are also important to VMI (Thomas, 1999; Bartolomeo and Chokron, 2002a,b). Two previous studies (Brandt and Stark, 1997; Laeng and Teodorescu, 2002) produced evidence consistent with this proposal, by showing that normal participants tend to produce similar eye movements when exploring a visual display and when subsequently imagining it. In addition, as mentioned in Introduction Spivey and Geng (2001, Exp. 1) found that direction-specific (i.e., horizontal or vertical) eye movements were produced when participants listened to short stories with prominent spatial details describing same direction. However, in this study stories with “leftward” or “rightward” contents produced eye movements on the horizontal direction, but without definite preference for, respectively, the left or the right side; participants produced a similar amount of left- and rightdirected saccades irrespective of the story, presumably because they had to re-center their gaze after having produced a saccade. Our results, similar to those of Spivey and Geng (2001), show that a “plausible” oculomotor behavior occurred even when participants generated visual mental images when facing an empty screen, in the absence of any previous visual
A
Left fixations Right fixations
VMI with map
60
60
60
50
50
50
40
40
30
30
20
20
10
10
40 30 20 10 0
Left-sided stimuli
Right-sided stimuli
Left-sided stimuli
Right-sided stimuli
0
ESF group
MF group
Left-sided stimuli
Right-sided stimuli
Left-sided stimuli
MF group
B
VMI with map 80
Total number of saccades
Perceptual task 70
Left-directed saccades Right-directed saccades
70 60 50
Right-sided stimuli
0
ESF group
Left-sided stimuli
Right-sided stimuli
Left-sided stimuli
VMI without map
Right-sided stimuli
ESF group
MF group
Perceptual task
80
80
70
70
60
60
50
50
40
40
30
30
20
20
10
10
40 30 20 10 0
Left-sided stimuli
Right-sided stimuli
MF group
Left-sided stimuli
Right-sided stimuli
MF group
0
Left-sided stimuli
Right-sided stimuli
MF group
Left-sided stimuli
Right-sided stimuli
ESF group
0
Left-sided stimuli
Right-sided stimuli
MF group
Left-sided stimuli
BR A I N R ES E A RC H 1 3 6 7 ( 2 01 1 ) 2 8 7 –2 97
Total number of fixations
VMI without map 70
70
Right-sided stimuli
ESF group
Fig. 2 – Total number of left and right fixations (panel A) and of left- and right-directed saccades (panel B) for left- and right-sided stimuli averaged across subjects. MF, map first group; ESF, empty screen first group. Error bars denote 95% confidence intervals.
291
292
BR A I N R ES E A RC H 1 3 6 7 ( 2 01 1 ) 2 8 7 –29 7
Fig. 3 – Representative scanpaths and fixations (grey circles) for one left-sided and one right-sided stimulus (represented by a dot) during (A) the VMI task without map, (B) the VMI task with map and (C) the perceptual condition. Note that no map was presented on condition (A), and only the map outline was displayed on condition (B).
BR A I N R ES E A RC H 1 3 6 7 ( 2 01 1 ) 2 8 7 –2 97
perceptual activity related to the task. Moreover, the only interaction we found was between the first fixation/saccade and the stimulus side. There was no significant interaction of the stimulus side with the last fixation or with the longest fixation. This pattern of results suggests a link between eye movements and mental image exploration, rather than with decision or response stages. Consistent with this hypothesis, accuracy on the imaginal tasks correlated with the number of fixations produced. Thus, “re-enacting” ocular exploration in VMI was not simply a rehearsal of previous exploration of a visual display, but occurred even for imagined stimuli coming from long-term memory. In addition to the Spivey and Geng data, our results demonstrated that saccades mirrored not only the direction of mental exploration, but also the side of exploration. In the present results, left-sided imagined items tended to evoke leftward saccades, and right-sided items rightward saccades. Although the present results by no means demonstrate that eye movements play a functional role in VMI (but see Laeng and Teodorescu, 2002), it is tempting to relate them to the traditional assessment of imaginal neglect, based on description of known places from long-term memory, in the absence of any task-related perceptual activity (Bisiach and Luzzatti, 1978). Converging evidence linking attention and eye movements to imaginal neglect comes from the report of improved retrieval patients of left-sided, previously unmentioned objects when the patients' gaze is turned towards the left, either by turning the head (Meador et al., 1987), or as a result of vestibular stimulation (Rode and Perenin, 1994). Although the relationships between VMI and dreaming activity are by no means clear, it is also important to note that neglect patients may show impaired leftward REMs (Doricchi et al., 1993). In one patient, nystagmoid REMs were consistent with the corresponding dreamed scenes (a train running leftward) (Doricchi et al., 2006). However, a direct correspondence between visual and imaginal neglect is not consistent with reports of double dissociations between these patterns of impairment (Guariglia et al., 1993; Bartolomeo et al., 1994; Coslett 1997; Beschin et al., 2000; Ortigue et al., 2001). Moreover, if seen and imagined objects acted in a similar way on patients' attention, then the presence/absence of visual stimuli should modulate the severity of imaginal neglect (see Chokron et al., 2004), but this is apparently not the case (Rode et al., 2007, 2010). Damage to distinct mechanisms of spatial attention might thus be implicated in different forms of neglect. For example, deficits of exogenous or endogenous attention might, respectively, contribute to signs of perceptual or imaginal neglect (Bartolomeo et al., 1994; Bourlon et al., 2010; Rode et al., 2010). Our participants had to conjure up visual mental images based only on verbal description. Other studies have shown that eye movements may be consistent with verbal descriptions (Demarais and Cohen, 1998; Spivey and Geng, 2001; Johansson et al., 2005, 2006), suggesting that in the absence of visual input, subjects construct a mental model of an object or a scene from linguistic input (Bower and Morrow, 1990). It has been proposed that the “external world” may be used to remember a scene more than the putative construction of an internal representation of the contents of that scene (Altmann, 2004; Hoover and Richardson, 2008; Richardson et al., 2009). According to this
293
hypothesis, eye movements during a recall period of a scene reflect an orientation in the external world to obtain information about the scene as “its own memory” (see also O'Regan, 1992). There would be no internal image, but a “mind's eye” whose movements would be associated with spontaneous perception and cognitive processes underlying mental representation. Another position holds that “looking at nothing” facilitates retrieval by activating detailed internal memory representations of the event, created and stored by the visual system (Ferreira et al., 2008). Alternative models of memory retrieval might take into account the hypothesis that memory does not only retrieve information from the past, but it reconstructs partially novel representations based on limited internal knowledge, with occasional support from bodily actions such as “looking at nothing.” The relationships between visual mental imagery and perception have been the subject of a long-standing debate (Pylyshyn, 1973, 2003; Kosslyn, 1994; Bartolomeo, 2002; Kosslyn et al., 2006). A traditional objection to studies relating similarities between visual and imaginal behavior has been that participants might have tried to fulfill the experimenters’ expectations (Pylyshyn, 1981; Intons-Peterson, 1983). The present results were obtained with participants who did not know that their eye movements were registered,1 also thanks to an inconspicuous tracking device, thus avoiding this objection. Using a hidden eye movement capture system, subjects remained naïve to the purposes of the experiment and so most likely did not simply adopt an “imaginal behavior” similar to the visual behavior. In this sense, the present results confirmed, with a different experimental setting, the findings by Spivey and Geng (2001). The perceptual condition evoked fewer saccades than the VMI conditions. This may depend on the fact that the perceptual condition employed relatively salient stimuli, which could in principle be detected without the need of foveating. Alternatively, fixations may have been easier on clearly defined perceptual items than on the less determined imagined items, which might thus have evoked more saccades. In conclusion, the present findings add to our knowledge of the processes implicated in VMI, by demonstrating that participants tend to produce spontaneous eye movements coherent with the location of imagined stimuli. Deficits of these processes of mental exploration might contribute to imaginal neglect. The use of similar experimental paradigms employing eye tracking in patients with imaginal neglect is promising for investigating the neural mechanisms of this mysterious condition.
4.
Experimental procedures
4.1.
Participants
16 healthy subjects consented to participate in this study. All were right-handed, free from neurological or visual pathology
1 At debriefing, no participant claimed to have guessed the real purpose of the experiment.
294
BR A I N R ES E A RC H 1 3 6 7 ( 2 01 1 ) 2 8 7 –29 7
and reported normal vision. In order to counterbalance the order of the task presentation, participants were divided into two ageand education-matched groups; the map first (MF) group (mean age ± SD, 29.56 ± 7.86 years; education level, 12.89± 1.45) and the empty screen first (ESF) group (age, 29.78 ± 7.64 years; education level, 12.70± 1.25).
4.2.
Apparatus
Eye movements were recorded with a Tobii 1750 eye tracker system (Tobii Technology). The eye tracker is composed of a 17” TFT TV screen with two hidden cameras. One major interest of this system is that it allows to monitor and to record eye position with no visible “tracking devices” that may affect the subject behavior. Subjects were told before the experiment that pupil diameter would be examined during the session. The position of both eyes were sampled online at 50 Hz and recorded for offline analysis using dedicated software (ClearView software) and homemade scripts. The experimental displays were controlled and synchronized with eye movement signal by a PC running MeyeParadigm software (e(ye)Brain Inc). To insure proper analysis, the losses of eye movement signal were checked offline for each individual session (Fig. 4A).
4.3.
Experimental protocol
Vocal RTs for the geographical tasks (Bartolomeo et al., 2005; Bourlon et al., 2008) were analyzed using standard criteria with a vocal responses threshold criterion. For all experiments, participants were comfortably seated in front of the computer screen with the two hidden camera placed at a distance of ~60 cm.
4.3.1.
interval was set to 5 s, thus allowing the participant to respond before the next trial starts. Stimuli were given in a random sequence, preceded by six additional practice items, referring to three left locations and three right locations. Responses to practice items were discarded from the analysis. In order to minimize any effect of practice, each target was only presented once.
Imaginal RT task
Following previously described procedures (Bartolomeo et al., 2005; Bourlon et al., 2008), 20 pairs of geographical locations (names of towns and regions of France) were selected. Each pair consisted of items located east or west of the city of Paris. Care was taken to choose cities or regions with approximately equal number of citizens across the two halves of the map. At the beginning of each trial, participants were asked to imagine a map of France. Afterwards, depending on the experimental conditions of the visual mental imagery task, a map of France (VMI with map condition) or a blank page (VMI without map condition) was displayed. Finally, the word “Paris” was played in the subject's headphone, followed 300 ms later by the auditory presentation of the word of a second French town or region (e.g., “Bordeaux”). After the presentation of the second auditory stimulus the participants were instructed to say “gauche” (“left”) if that stimulus referred to a city or region west from Paris, or “droite” (“right”) if that second stimulus indicated a city or region east from Paris. The intertrial
4.3.2.
Perceptual RT task
In the empty map condition, a white map of France was displayed on a black background. A black dot (10-mm in diameter) appeared at the map location corresponding to Paris. After 300 ms, a lateralized target replaced the central dot. The visual targets consisted of red dots 10-mm in diameter, which were displayed one at a time, at the locations corresponding on the map to the corresponding imaginal items. As for the imaginal task, subjects had to respond by saying “gauche” or “droite,” depending on whether the red dot appeared left or right of the black dot. Targets remained on until response or for a maximum of 5 s.
4.4.
Data analysis
As previously described, participants were divided into two groups depending on the order of presentation the imaginal conditions with or without map. The MF group received first the task with map, then the task without map and finally the perceptual task; the ESF group performed first the task without map, then the task with map and finally the perceptual task. The perceptual condition was always administered after the imaginal conditions. Vocal response times (RTs) and accuracy were recorded. A microphone connected to the computer digitized the naming time and ran a signal detection algorithm (using an adaptive threshold) which permit to measure reaction time with a 7-ms precision level. Eye movements were analyzed by estimating the total number of saccades and the total of number of fixation for movements to the right and to the left side of each stimulus. Eye movement recording started at stimulus presentation and ended at the onset of the vocal response (Fig. 4B). The periods of fixation were defined as the time intervals when no saccades occurred with amplitude larger than 30 pixels (0.76°) and a minimum duration of 100 ms (Fig. 4C). All the periods of fixation were identified offline, by using the ClearView software. Saccade parameters such as direction, velocity and amplitude were also extracted offline. Given the purposes of the present study, interest was restricted to the lateral direction of saccades relative to the central fixation cue. Saccades less than 10 mm (0.95°) in amplitude (corresponding to the diameter of the perceptual stimuli) were not considered for the analysis.
Fig. 4 – A. Example of gazepoint for the two eyes. When the camera lost the signal (arrow), the corresponding data were excluded from analysis. B. Procedure to determine the location (right or left) of the eyes on the screen after hearing the stimulus. C. Algorithm for filter fixations.
BR A I N R ES E A RC H 1 3 6 7 ( 2 01 1 ) 2 8 7 –2 97
295
296
BR A I N R ES E A RC H 1 3 6 7 ( 2 01 1 ) 2 8 7 –29 7
Acknowledgments We are grateful to C. Fouchard, M. Martinez, A. Bordier, A. Thierry, A. Chatenet, L. Lehenaff and V. Happel for help with data collection and to G. Dalla Barba and S. Rivaud for helpful discussion.
REFERENCES
Altmann, G.T., 2004. Language-mediated eye movements in the absence of a visual world: the “blank screen paradigm”. Cognition 93 (2), B79–B87. Ballard, D.H., Hayhoe, M.M., Pook, P.K., Rao, R.P.N., 1997. Deictic codes for the embodiment of cognition. Behav. Brain Sci. 20 (4), 723–767. Bartolomeo, P., 2002. The relationship between visual perception and visual mental imagery: a reappraisal of the neuropsychological evidence. Cortex 38 (3), 357–378. Bartolomeo, P., 2008. The neural correlates of visual mental imagery: an ongoing debate. Cortex 44 (2), 107–108. Bartolomeo, P., Chokron, S., 2002a. Can we change our vantage point to explore imaginal neglect? (Commentary on Pylyshyn: Mental imagery: In search of a theory). Behav. Brain Sci. 25 (2), 184–185. Bartolomeo, P., Chokron, S., 2002b. Orienting of attention in left unilateral neglect. Neurosci. Biobehav. Rev. 26 (2), 217–234. Bartolomeo, P., D'Erme, P., Gainotti, G., 1994. The relationship between visuospatial and representational neglect. Neurology 44, 1710–1714. Bartolomeo, P., Pagliarini, L., Parisi, D., 2002. Emergence of orienting behavior in ecological neural networks. Neural Process. Lett. 15 (1), 69–76. Bartolomeo, P., Bachoud-Levi, A.-C., Azouvi, P., Chokron, S., 2005. Time to imagine space: a chronometric exploration of representational neglect. Neuropsychologia 43 (9), 1249–1257. Beschin, N., Basso, A., Della Sala, S., 2000. Perceiving left and imagining right: dissociation in neglect. Cortex 36 (3), 401–414. Bisiach, E., Luzzatti, C., 1978. Unilateral neglect of representational space. Cortex 14, 129–133. Bourlon, C., Pradat-Diehl, P., Duret, C., Azouvi, P., Bartolomeo, P., 2008. Seeing and imagining the “same” objects in unilateral neglect. Neuropsychologia 46, 2602–2606. Bourlon, C., Duret, C., Pradat-Diehl, P., Azouvi, P., Loeper-Jeny, C., Merat-Blanchard, M., Levy, C., Chokron, S. and Bartolomeo, P., 2010. Vocal response times to real and imagined stimuli in spatial neglect: A group study and single-case report. Cortex. doi:10.1016/j.cortex.2010.03.004. Bower, G.H., Morrow, D.G., 1990. Mental models in narrative comprehension. Science 247 (4938), 44–48. Brandt, S.A., Stark, L.W., 1997. Spontaneous eye movements during visual imagery reflect the content of the visual scene. J. Cogn. Neurosci. 9 (1), 27–38. Chokron, S., Colliot, P., Bartolomeo, P., 2004. The role of vision on spatial representations. Cortex 40 (2), 281–290. Coslett, H.B., 1997. Neglect in vision and visual imagery: a double dissociation. Brain 120, 1163–1171. Demarais, A.M., Cohen, B.H., 1998. Evidence for image-scanning eye movements during transitive inference. Biol. Psychol. 49 (3), 229–247. Di Ferdinando, A., Parisi, D., Bartolomeo, P., 2007. Modeling orienting behavior and its disorders with “ecological” neural networks. J. Cogn. Neurosci. 19 (6), 1033–1049. Doricchi, F., Guariglia, C., Paolucci, S., Pizzamiglio, L., 1993. Disturbances of the rapid eye movements (REMs) of REM sleep in patients with unilateral attentional neglect: clue for the
understanding of the functional meaning of REMs. Electroencephalogr. Clin. Neurophysiol. 87 (3), 105–116. Doricchi, F., Iaria, G., Silvetti, M., Figliozzi, F., Siegler, I., 2006. The “ways” we look at dreams: evidence from unilateral spatial neglect (with an evolutionary account of dream bizarreness). Exp. Brain Res. 178 (4), 450–461. Ferreira, F., Apel, J., Henderson, J.M., 2008. Taking a new look at looking at nothing. Trends Cogn. Sci. 12 (11), 405–410. Findlay, J.M., Gilchrist, I.D., 2003. Active vision: the psychology of looking and seeing. Oxford University Press, Oxford. Gainotti, G., D'Erme, P., Bartolomeo, P., 1991. Early orientation of attention toward the half space ipsilateral to the lesion in patients with unilateral brain damage. J. Neurol. Neurosurg. Psychiatry 54, 1082–1089. Guariglia, C., Padovani, A., Pantano, P., Pizzamiglio, L., 1993. Unilateral neglect restricted to visual imagery. Nature 364, 235–237. Hebb, D.O., 1968. Concerning imagery. Psychol. Rev. 75 (6), 466–477. Hoffman, J.E., Subramaniam, B., 1995. The role of visual attention in saccadic eye movements. Percept. Psychophys. 57 (6), 787–795. Hoover, M.A., Richardson, D.C., 2008. When facts go down the rabbit hole: contrasting features and objecthood as indexes to memory. Cognition 108 (2), 533–542. Intons-Peterson, M.J., 1983. Imagery paradigms: how vulnerable are they to experimenters' expectations? J. Exp. Psychol. Hum. Percept. Perform. 9 (3), 394–412. Johansson, R., Holsanova, J., Holmqvist, K., 2005. What do eye movements reveal about mental imagery ? Evidence from visual and verbal elicitations. In: Bara, B.G., Barsalou, L., Bucciarelli, M. (Eds.), Proceedings of the 27th Annual Conference of the cognitive science society. Erlbaum, pp. 1054–1059. Johansson, R., Holsanova, J., Holmqvist, K., 2006. Pictures and spoken descriptions elicit similar eye movement during mental imagery, both in light and in complete darkness. Cogn. Sci. 30, 1053–1079. Kosslyn, S.M., 1994. Image and Brain: The Resolution of the Imagery Debate. The MIT Press, Cambridge, Mass. Kosslyn, S.M., Thompson, W.L., Ganis, G., 2006. The Case for Mental Imagery. Oxford University Press, New York. Laeng, B., Teodorescu, D.-S., 2002. Eye scanpaths during visual imagery reenact those of perception of the same visual scene. Cogn. Sci. 26, 207–231. Meador, K.J., Loring, D.W., Bowers, D., Heilman, K.M., 1987. Remote memory and neglect syndrome. Neurology 37, 522–526. Neisser, U., 1967. Cognitive Psychology. Appleton-Century-Crofts, New York. O'Regan, J.K., 1992. Solving the “real” mysteries of visual perception: the world as an outside memory. Can. J. Psychol. 46 (3), 461–488. Ortigue, S., Viaud-Delmon, I., Annoni, J.M., Landis, T., Michel, C., Blanke, O., Vuilleumier, P., Mayer, E., 2001. Pure representational neglect after right thalamic lesion. Ann. Neurol. 50 (3), 401–404. Posner, M.I., 1980. Orienting of attention. Q. J. Exp. Psychol. 32, 3–25. Pylyshyn, Z.W., 1973. What the mind's eye tells the mind's brain: a critique of mental imagery. Psychol. Bull. 80 (1), 1–24. Pylyshyn, Z.W., 1981. The imagery debate: analogue media versus tacit knowledge. Psychol. Rev. 88, 16–45. Pylyshyn, Z.W., 2003. Seeing and visualizing : it's not what you think. MIT Press, Cambridge, Mass. Richardson, D.C., Altmann, G.T., Spivey, M.J., Hoover, M.A., 2009. Much ado about eye movements to nothing: a response to Ferreira et al.: Taking a new look at looking at nothing. Trends Cogn. Sci. 13 (6), 235–236. Rode, G., Perenin, M.T., 1994. Temporary remission of representational hemineglect through vestibular stimulation. NeuroReport 5, 869–872. Rode, G., Revol, P., Rossetti, Y., Boisson, D., Bartolomeo, P., 2007. Looking while imagining: the influence of visual input on representational neglect. Neurology 68 (6), 432–437.
BR A I N R ES E A RC H 1 3 6 7 ( 2 01 1 ) 2 8 7 –2 97
Rode, G., Cotton, F., Revol, P., Jacquin-Courtois, S., Rossetti, Y., Bartolomeo, P., 2010. Representation and disconnection in imaginal neglect. Neuropsychologia 48 (10), 2903–2911. Shepherd, M., Findlay, J.M., Hockey, R.J., 1986. The relationship between eye movements and spatial attention. Q. J. Exp. Psychol. 38A, 475–791. Sokolov, E.N., 1963. Higher nervous functions: the orienting reflex. Annu. Rev. Physiol. 25, 545–580.
297
Spivey, M.J., Geng, J.J., 2001. Oculomotor mechanisms activated by imagery and memory: eye movements to absent objects. Psychol. Res. 65 (4), 235–241. Thomas, N.J.T., 1999. Are theories of imagery theories of imagination? An active perception approach to conscious mental content. Cogn. Sci. 23 (2), 207–245. Tulving, E., Thomson, D.M., 1973. Encoding specificity and retrieval processes in episodic memory. Psychol. Rev. 80 (5), 352–373.
available at www.sciencedirect.com
www.elsevier.com/locate/brainres
Research Report
Visual mental imagery: What the head's eye tells the mind's eye Clémence Bourlon a,b,c,d,⁎, Bastien Oliviero a , Nicolas Wattiez a,b , Pierre Pouget a,b , Paolo Bartolomeo a,b,d,e,f a Centre de Recherche de l'Institut du Cerveau et de la Moelle Epinière, Inserm UMR S975, CNRS 7225, Pavillon Claude Bernard, Hôpital Pitié Salpêtrière (AP-HP), Paris, France b UPMC Université Paris 06, France c Service de Rééducation et de Réadaptation Fonctionnelle, Clinique Les Trois Soleils, Boissise le Roi, France d Service de Médecine et de Réadaptation, Hôpital National de Saint Maurice, France e Fédération de Neurologie, Hôpital Pitié-Salpêtrière (AP-HP) Paris, France f Department of Psychology, Catholic University, Milan, Italy
A R T I C LE I N FO
AB S T R A C T
Article history:
The demonstration of an implication of attentional/eye gaze systems in visual mental
Accepted 12 October 2010
imagery might help to understand why some patients with visual neglect, who suffer from
Available online 19 October 2010
severe attentional deficits, also show neglect for mental images. When normal participants generate mental images of previously explored visual scenes, their oculomotor behavior
Keywords:
resembles that used during visual exploration. However, this could be a case of encoding
Mental imagery
specificity, whereby the probability of retrieving an event increases if some information
Eye movement
encoded with the event (in this case its spatial location) is present at retrieval. In the present
Neglect
study, normal participants were invited to conjure up a mental image of the map of France
Attention
and to say whether auditorily presented towns or regions were situated left or right of Paris. A perceptual version of the task was administered after the imaginal condition. Thus, in the imaginal condition participants had to retrieve information from long-term memory. Vocal response times and, unbeknownst to participants, also eye movements were recorded. Participants tended to produce similar eye movements on the imaginal and on the perceptual conditions of the task. We concluded that some mechanisms involved in spontaneous oculomotor behavior may be shared in exploration of visuospatial mental images. Deficits of these common processes participating in the oculomotor exploration might contribute to imaginal neglect. © 2010 Elsevier B.V. All rights reserved.
1.
Introduction
The mechanisms of visual mental imagery (VMI; the faculty whereby we can imagine or “visualize” objects and places in their absence) have fostered intense research and debate during the last decades. One major question of interest concerns whether, and to what extent, VMI shares functional processes
and neural substrates with visual perception (Pylyshyn, 2003; Kosslyn et al., 2006; Bartolomeo, 2008). According to an influential model, visual mental images are “displayed” on a visual buffer consisting of topographically organized areas in the occipital lobe, similar to visual percepts (Kosslyn et al., 2006). However, reports of patients with occipital damage and perceptual deficits but preserved VMI challenge strong versions
⁎ Corresponding author. INSERM UMR S975, Pavillon Claude Bernard, Hôpital Salpêtrière, 47 bd de l'Hôpital, F-75013 Paris, France. E-mail address: [email protected] (C. Bourlon). 0006-8993/$ – see front matter © 2010 Elsevier B.V. All rights reserved. doi:10.1016/j.brainres.2010.10.039
288
BR A I N R ES E A RC H 1 3 6 7 ( 2 01 1 ) 2 8 7 –29 7
of perceptual/imagery equivalence, and suggests that VMI implicates visual processes at higher levels of integration than those performed by primary visual cortex (Bartolomeo, 2002). For example, mechanisms related to visuospatial attention might be activated during spatial forms of VMI (Thomas, 1999; Bartolomeo and Chokron, 2002a,b). Consistent with this idea, some patients with right hemisphere damage and visual neglect (i.e., lack of awareness for left-sided visual items) also display signs of neglect in the representational, or imaginal space. When describing known places from memory, these patients “forget” to mention items laying on the left side of their mental images, regardless of the (imaginary) vantage point (Bisiach and Luzzatti, 1978). An important question raised by these patterns of performance is whether asymmetries in orienting of attention, which are a component deficit in perceptual neglect (Bartolomeo and Chokron, 2002a,b), also play a role in imaginal neglect (Bartolomeo et al., 1994, 2005; Bourlon et al., 2008, 2010). The study of eye movements may help addressing these issues, because eye movements are in most common situations a behavioral correlate of orienting of spatial attention. Although, in the context of the laboratory, attention can be oriented “covertly,” or without eye movements (Posner, 1980), in more ecological situation saccades are usually accompanied by corresponding attentional shifts (Shepherd et al., 1986; Hoffman and Subramaniam, 1995). Thus, if attention is important in spatial VMI, then the production of eye movements should most commonly accompany imaginal tasks. Empirical support for this prediction came from the study of Brandt and Stark (1997), who asked normal participants to visually explore and then revisualize from memory irregularly-checkered diagrams while their eye movements were recorded. For any given picture, sequences of fixations and saccades were closely correlated with those recorded while viewing the same item. A further step was made by Laeng and Teodorescu (2002). Drawing on theoretical positions postulating that eye movements may guide the sequential synthesis of the different spatial components of mental images (Neisser, 1967; Hebb, 1968), Laeng and Teodorescu proposed that eye movements are not merely epiphenomenal, but play a functional role in VMI. In two experiments, participants saw irregular checkerboards or color pictures of fish and formed mental images of these items. Different groups of participants either were free to make eye movements or maintained central fixation during the tasks. The strength of relatedness of eye scanpaths between imagery and perception predicted performance accuracy in VMI. Furthermore, participants free to explore during perception, but maintaining fixation during VMI, had impaired performance in recalling the visual patterns of fish images. Laeng and Teodorescu concluded that reenacting motor sequences of eye movements is useful to achieve vivid mental images. These findings are important in suggesting a functional role for eye movements in VMI. However, a common feature of these studies on eye movements in VMI is the fact that participants viewed patterned visual items before performing the VMI task. As a consequence, their results do not directly explore the role of eye movements when participants retrieve visual mental images from long-term memory, as it is the case with description from memory of known places or similar tests of imaginal neglect. In fact, improvement of memory retrieval
when looking at the empty location where an event has previously occurred (Ferreira et al., 2008) may well be an instance of encoding specificity (Tulving and Thomson, 1973); the probability of retrieving an event increases if some information encoded with the event (in this case its spatial location) is present at retrieval. Additionally, as Laeng and Teodorescu acknowledge, their task design is not immune from the classical objection of participants acting according to the experimenters’ expectancies in the VMI condition (Pylyshyn, 1981; Intons-Peterson, 1983). The two methodological problems described above were addressed by Spivey and Geng (2001, Exp. 1). They recorded eye movements while subjects imagined verbally described scenes. No visual display was used and participants did not know that their eye movements were being recorded at the time. Participants produced more horizontal or vertical saccades when the description included, respectively, horizontal or vertical dimensions. In the present study, we used a task paradigm close to a typical test of imaginal neglect, the mental exploration from memory of a map of France (Rode and Perenin, 1994; Rode et al., 2007). We asked French participants to mentally visualize a map of France. Participants subsequently heard the names of French towns or regions and had to state whether each item was left or right of Paris (Bartolomeo et al., 2005), while their vocal response times (RTs) and eye movements were recorded. A perceptual version of the task was also used (Bourlon et al., 2008), in which participants saw the outline of the map and the same geographical locations appeared as dots within the map. However, all participants performed the perceptual condition after the VMI condition, so that a simple “spillover” of eye movements from visual exploration to VMI revisualization could be excluded. Moreover, we used an eye tracker which does not require any helmet or chin-rest, and participants were entirely unaware that their eye movements were being recorded. As a consequence, it is highly unlikely that they were trying to satisfy the experimenter's expectancies. Finally, we used two VMI conditions before the visual condition. In the “VMI without map” task, participants saw a blank screen when they imagined the map of France. The other condition involved the presentation of an outline of the map of France on the screen during all the experiment (“VMI with map” condition). This last condition was intended to explore the influence of the presence of a visual input on eye movements during VMI. The visual task was always performed after the two VMI conditions, to prevent recent visual experiences from influencing performance. To avoid any effects on performance of the order of the VMI conditions, half of the participants performed first the VMI task with map, then the VMI without map (Map First group, MF); the other half had the VMI tasks in the opposite order (Empty Screen First group, EMF).
2.
Results
2.1.
Response time and accuracy of responses
Statistical analyses were implemented with Statistica software (Statsoft, Inc.). A repeated-measure analysis of variance
BR A I N R ES E A RC H 1 3 6 7 ( 2 01 1 ) 2 8 7 –2 97
(ANOVA) was conducted on mean correct RTs with group, task and side position of the stimulus as factors. A further ANOVA was applied to the arcsin-transformed proportions of correct responses with the same factors. The side of stimulus presentation did not influence response times (RTs) (mean ± SD in ms, left, 1136 ± 294; right, 1127 ms ± 346), nor did the group (MF group, 1154 ± 254; ESF group = 1127 ± 346) or the task condition (VMI with map outline, 1193 ± 404; VMI without map, 1190 ms ± 383; perceptual task, 1129 ms ± 300). However, the group interacted with the task condition (F (2, 28) = 4.03, p = 0.029), because participants were 160 ms slower on the first task condition (i.e., VMI without map for MF group and VMI with map for ESF group) than on the second (Fig. 1A), probably as a consequence of practice. Accuracy was also similar for the two groups (mean number of hits± SD, MF group, 17.77± 1.96; ESF group, 18.02± 2.38) and the two sides of stimulus presentation (left, 17.60± 2.28; right, 18.18± 2.04). A reliable effect of task condition was observed (F (2, 28) = 81.62, p = 0.0001): Perceived stimuli (19.78 ± 0.55) were better detected than imagined stimuli (VMI with map, 16.66± 2.12; VMI without map, 17.25± 2.00). No interaction reached significance (Fig. 1B).
2.2.
Eye movements
The data were log-transformed and submitted to repeated measures ANOVAs. We wanted to test the hypothesis that eye
289
movements are important for VMI even for stimuli recollected from long-term memory, in the absence of any task-related perceptual activity immediately before the experimental session. Thus, we planned specific comparisons to explore eye movements in the VMI condition with a completely empty screen (i.e., without the map outline), for those participants who performed this condition first (ESF group).
2.2.1.
Fixations
The total number of fixations for each stimulus location was measured according to their spatial position. There were fewer fixations for the perceptual condition than for VMI conditions (mean number of fixations ± SD, VMI with map, 32.05 ± 16.96; VMI without map, 28.59 ± 13.81; perceptual task, 22.47 ± 12.33) (F (2, 28) = 4.87, p = 0.015). There was an interaction between the side of stimulus location and the side of fixation (F (1, 14) = 60.07, p = 0.0001), reflecting the fact that participants tended to produce more fixations on the side of the stimulus, independent of group and condition (Fig. 2). Importantly, the planned comparison revealed that this interaction was reliable even for the condition without map performed as first condition, i.e. when participants had not seen any map before performing the test (F (1, 7) = 13.67, p = 0.0077). A potential concern in the interpretation of the oculomotor bias shown by participants on the imaginal task is that it could have been influenced by the subsequent verbal response. Both the imaginal and exploratory task required a left vs right
Fig. 1 – A. Correct response times for left-sided items (hatched bars) and right-sided items (empty bars) on the RT tasks. B. Percentage of correct responses for left-sided items (hatched bars) and right-sided items (empty bars) on the RT tasks. Error bars denote 95% confidence intervals. MF, map first group; ESF, empty screen first group.
290
BR A I N R ES E A RC H 1 3 6 7 ( 2 01 1 ) 2 8 7 –29 7
decision; the response code adopted in the task could have affected or increased, especially in the imaginal task, the oculomotor bias of participants. To address this issue, we determined the first and the last fixation during the decision period, before response production. If subjects tended to look at the (imagined) stimulus location soon after the stimulus presentation, but not just before the verbal response, then an influence of the response on eye movement pattern would be less likely. Consistent with this prediction, subjects tended to make their first fixation towards the side of the stimulus (interaction between side of first fixation and side of stimulus, F (1, 14) = 8.72, p = 0.010). However, the corresponding interaction between side of last fixation and stimulus side was far from significance (F < 1). This suggests that participants first “explored” the side of the imaginary stimulus and then produced the relative verbal response. Moreover, the analysis of the longest fixation during the decision period revealed no significant interaction between the side of the stimulus and the side of the fixation (F (1, 14) = 1.67, p = 0.22). Finally, no correlations emerged with RTs (VMI with map, r = −0.26, p = 0.313; VMI without map, r = −0.21, p = 0.426; perceptual task, r = 0.49, p = 0.053). Concerning accuracy, no significant correlation emerged for the perceptual task (r = −0.30). However, interestingly, accuracy for the imaginal tasks did correlate with the number of fixations (VMI with map, r = 0.52, p = 0.039; VMI without map, r = 0.69, p = 0.003). This suggests that fixations were relatively useless for the perceptual task, where the detection of salient peripheral targets presumably did not require prolonged foveal processing, but they did help improving performance on the imaginal tasks.
2.2.2.
Saccades
The perceptual condition evoked fewer saccades than the two imaginal conditions (mean VMI with map, 30.53 ± 25.18; mean VMI without map = 26.62 ± 20.50; mean perceptual condition, 20.23 ± 17.77), (F (2, 28) = 3.69, p = 0.037). As noticed for the number of fixations participants tended to produce more saccades towards the side of stimulus presentation, again independent of the presence of a map or of the order of conditions (F (1, 14) = 130.93, p = 0.0001) (Figs. 2 and 3). Importantly, as it was the case for fixations, even those participants who had not seen the map outline before the VMI task made more saccades towards the side of the imagined stimulus, F (1, 7) = 45.99, p = 0.00026. Subjects tended to produce their first saccade towards the direction of the stimulus (interaction between first saccade and stimulus side, F (1, 14)= 7.06, p = 0.018). To establish whether not only the direction, but also the metrics of the eye movements depended on the location of imagined stimuli, we computed the correlations between the amplitude of the first saccade and the real distances between Paris and the peripheral targets. As expected, there was a reliable correlation for the perceptual task (r = 0.36, p = 0.046); however, no correlation emerged for the imaginal tasks, either with map (r = 0.25, p = 0.838) or without visible map (r = −0.07, p = 0.704). This result may depend on the lack of a fixation point in the VMI condition, so that subjects did not always start exploration from the center of the screen. In contrast to first saccades, and analogously to fixation patterns, last saccades were not significantly related to the stimulus side (F (1, 14) = 2.22, p = 0.158).
Finally, there were no reliable correlations between the number of saccades and RTs or accuracy of responses (RTs: VMI with map condition, r = 0.25; VMI without map condition, r = 0.26; perceptual task, r = 0.37; accuracy: VMI with map condition, r = −0.31; VMI without map condition, r = 0.04; perceptual task, r = −0.13; all p ns).
3.
Discussion
Human visual experience is not the result of a passive display of external images on the visual system, but a dynamic process during which the eyes continually sample the environment (Findlay and Gilchrist, 2003). In ecological settings, organisms orient and turn their head and gaze towards relevant stimuli (Sokolov, 1963), in order to align these stimuli with the part of the sensory surface with highest resolution (e.g., the retinal fovea). This allows better perceptual processing of the stimuli, for example their identification as useful or as dangerous objects. Orienting movements are thus a typical form of “embodied” cognition (Ballard et al., 1997), that is a process in which body movements are necessary to the processing of information. Thus, bodily orienting movements contribute to optimize processing resources, e.g. the segregation of mechanisms dedicated to simple detection from resources performing more complex identification tasks. Orienting behaviors reminiscent of exploratory saccades emerge even in very simple artificial organisms evolving following a genetic algorithm (Bartolomeo et al., 2002; Di Ferdinando et al., 2007). Patients with left neglect demonstrate severe dysfunctions of orienting behavior; their attention and gaze is often compulsorily captured by right-sided visual stimuli (Gainotti et al., 1991). However, in VMI tasks there are no visual stimuli by definition, and yet some neglect patients do seem to neglect the left side of their mental images. A hypothesis proposed to account for these surprising patterns of performance is to postulate that attentional processes, which are typically impaired in visual neglect (Bartolomeo and Chokron, 2002a,b), are also important to VMI (Thomas, 1999; Bartolomeo and Chokron, 2002a,b). Two previous studies (Brandt and Stark, 1997; Laeng and Teodorescu, 2002) produced evidence consistent with this proposal, by showing that normal participants tend to produce similar eye movements when exploring a visual display and when subsequently imagining it. In addition, as mentioned in Introduction Spivey and Geng (2001, Exp. 1) found that direction-specific (i.e., horizontal or vertical) eye movements were produced when participants listened to short stories with prominent spatial details describing same direction. However, in this study stories with “leftward” or “rightward” contents produced eye movements on the horizontal direction, but without definite preference for, respectively, the left or the right side; participants produced a similar amount of left- and rightdirected saccades irrespective of the story, presumably because they had to re-center their gaze after having produced a saccade. Our results, similar to those of Spivey and Geng (2001), show that a “plausible” oculomotor behavior occurred even when participants generated visual mental images when facing an empty screen, in the absence of any previous visual
A
Left fixations Right fixations
VMI with map
60
60
60
50
50
50
40
40
30
30
20
20
10
10
40 30 20 10 0
Left-sided stimuli
Right-sided stimuli
Left-sided stimuli
Right-sided stimuli
0
ESF group
MF group
Left-sided stimuli
Right-sided stimuli
Left-sided stimuli
MF group
B
VMI with map 80
Total number of saccades
Perceptual task 70
Left-directed saccades Right-directed saccades
70 60 50
Right-sided stimuli
0
ESF group
Left-sided stimuli
Right-sided stimuli
Left-sided stimuli
VMI without map
Right-sided stimuli
ESF group
MF group
Perceptual task
80
80
70
70
60
60
50
50
40
40
30
30
20
20
10
10
40 30 20 10 0
Left-sided stimuli
Right-sided stimuli
MF group
Left-sided stimuli
Right-sided stimuli
MF group
0
Left-sided stimuli
Right-sided stimuli
MF group
Left-sided stimuli
Right-sided stimuli
ESF group
0
Left-sided stimuli
Right-sided stimuli
MF group
Left-sided stimuli
BR A I N R ES E A RC H 1 3 6 7 ( 2 01 1 ) 2 8 7 –2 97
Total number of fixations
VMI without map 70
70
Right-sided stimuli
ESF group
Fig. 2 – Total number of left and right fixations (panel A) and of left- and right-directed saccades (panel B) for left- and right-sided stimuli averaged across subjects. MF, map first group; ESF, empty screen first group. Error bars denote 95% confidence intervals.
291
292
BR A I N R ES E A RC H 1 3 6 7 ( 2 01 1 ) 2 8 7 –29 7
Fig. 3 – Representative scanpaths and fixations (grey circles) for one left-sided and one right-sided stimulus (represented by a dot) during (A) the VMI task without map, (B) the VMI task with map and (C) the perceptual condition. Note that no map was presented on condition (A), and only the map outline was displayed on condition (B).
BR A I N R ES E A RC H 1 3 6 7 ( 2 01 1 ) 2 8 7 –2 97
perceptual activity related to the task. Moreover, the only interaction we found was between the first fixation/saccade and the stimulus side. There was no significant interaction of the stimulus side with the last fixation or with the longest fixation. This pattern of results suggests a link between eye movements and mental image exploration, rather than with decision or response stages. Consistent with this hypothesis, accuracy on the imaginal tasks correlated with the number of fixations produced. Thus, “re-enacting” ocular exploration in VMI was not simply a rehearsal of previous exploration of a visual display, but occurred even for imagined stimuli coming from long-term memory. In addition to the Spivey and Geng data, our results demonstrated that saccades mirrored not only the direction of mental exploration, but also the side of exploration. In the present results, left-sided imagined items tended to evoke leftward saccades, and right-sided items rightward saccades. Although the present results by no means demonstrate that eye movements play a functional role in VMI (but see Laeng and Teodorescu, 2002), it is tempting to relate them to the traditional assessment of imaginal neglect, based on description of known places from long-term memory, in the absence of any task-related perceptual activity (Bisiach and Luzzatti, 1978). Converging evidence linking attention and eye movements to imaginal neglect comes from the report of improved retrieval patients of left-sided, previously unmentioned objects when the patients' gaze is turned towards the left, either by turning the head (Meador et al., 1987), or as a result of vestibular stimulation (Rode and Perenin, 1994). Although the relationships between VMI and dreaming activity are by no means clear, it is also important to note that neglect patients may show impaired leftward REMs (Doricchi et al., 1993). In one patient, nystagmoid REMs were consistent with the corresponding dreamed scenes (a train running leftward) (Doricchi et al., 2006). However, a direct correspondence between visual and imaginal neglect is not consistent with reports of double dissociations between these patterns of impairment (Guariglia et al., 1993; Bartolomeo et al., 1994; Coslett 1997; Beschin et al., 2000; Ortigue et al., 2001). Moreover, if seen and imagined objects acted in a similar way on patients' attention, then the presence/absence of visual stimuli should modulate the severity of imaginal neglect (see Chokron et al., 2004), but this is apparently not the case (Rode et al., 2007, 2010). Damage to distinct mechanisms of spatial attention might thus be implicated in different forms of neglect. For example, deficits of exogenous or endogenous attention might, respectively, contribute to signs of perceptual or imaginal neglect (Bartolomeo et al., 1994; Bourlon et al., 2010; Rode et al., 2010). Our participants had to conjure up visual mental images based only on verbal description. Other studies have shown that eye movements may be consistent with verbal descriptions (Demarais and Cohen, 1998; Spivey and Geng, 2001; Johansson et al., 2005, 2006), suggesting that in the absence of visual input, subjects construct a mental model of an object or a scene from linguistic input (Bower and Morrow, 1990). It has been proposed that the “external world” may be used to remember a scene more than the putative construction of an internal representation of the contents of that scene (Altmann, 2004; Hoover and Richardson, 2008; Richardson et al., 2009). According to this
293
hypothesis, eye movements during a recall period of a scene reflect an orientation in the external world to obtain information about the scene as “its own memory” (see also O'Regan, 1992). There would be no internal image, but a “mind's eye” whose movements would be associated with spontaneous perception and cognitive processes underlying mental representation. Another position holds that “looking at nothing” facilitates retrieval by activating detailed internal memory representations of the event, created and stored by the visual system (Ferreira et al., 2008). Alternative models of memory retrieval might take into account the hypothesis that memory does not only retrieve information from the past, but it reconstructs partially novel representations based on limited internal knowledge, with occasional support from bodily actions such as “looking at nothing.” The relationships between visual mental imagery and perception have been the subject of a long-standing debate (Pylyshyn, 1973, 2003; Kosslyn, 1994; Bartolomeo, 2002; Kosslyn et al., 2006). A traditional objection to studies relating similarities between visual and imaginal behavior has been that participants might have tried to fulfill the experimenters’ expectations (Pylyshyn, 1981; Intons-Peterson, 1983). The present results were obtained with participants who did not know that their eye movements were registered,1 also thanks to an inconspicuous tracking device, thus avoiding this objection. Using a hidden eye movement capture system, subjects remained naïve to the purposes of the experiment and so most likely did not simply adopt an “imaginal behavior” similar to the visual behavior. In this sense, the present results confirmed, with a different experimental setting, the findings by Spivey and Geng (2001). The perceptual condition evoked fewer saccades than the VMI conditions. This may depend on the fact that the perceptual condition employed relatively salient stimuli, which could in principle be detected without the need of foveating. Alternatively, fixations may have been easier on clearly defined perceptual items than on the less determined imagined items, which might thus have evoked more saccades. In conclusion, the present findings add to our knowledge of the processes implicated in VMI, by demonstrating that participants tend to produce spontaneous eye movements coherent with the location of imagined stimuli. Deficits of these processes of mental exploration might contribute to imaginal neglect. The use of similar experimental paradigms employing eye tracking in patients with imaginal neglect is promising for investigating the neural mechanisms of this mysterious condition.
4.
Experimental procedures
4.1.
Participants
16 healthy subjects consented to participate in this study. All were right-handed, free from neurological or visual pathology
1 At debriefing, no participant claimed to have guessed the real purpose of the experiment.
294
BR A I N R ES E A RC H 1 3 6 7 ( 2 01 1 ) 2 8 7 –29 7
and reported normal vision. In order to counterbalance the order of the task presentation, participants were divided into two ageand education-matched groups; the map first (MF) group (mean age ± SD, 29.56 ± 7.86 years; education level, 12.89± 1.45) and the empty screen first (ESF) group (age, 29.78 ± 7.64 years; education level, 12.70± 1.25).
4.2.
Apparatus
Eye movements were recorded with a Tobii 1750 eye tracker system (Tobii Technology). The eye tracker is composed of a 17” TFT TV screen with two hidden cameras. One major interest of this system is that it allows to monitor and to record eye position with no visible “tracking devices” that may affect the subject behavior. Subjects were told before the experiment that pupil diameter would be examined during the session. The position of both eyes were sampled online at 50 Hz and recorded for offline analysis using dedicated software (ClearView software) and homemade scripts. The experimental displays were controlled and synchronized with eye movement signal by a PC running MeyeParadigm software (e(ye)Brain Inc). To insure proper analysis, the losses of eye movement signal were checked offline for each individual session (Fig. 4A).
4.3.
Experimental protocol
Vocal RTs for the geographical tasks (Bartolomeo et al., 2005; Bourlon et al., 2008) were analyzed using standard criteria with a vocal responses threshold criterion. For all experiments, participants were comfortably seated in front of the computer screen with the two hidden camera placed at a distance of ~60 cm.
4.3.1.
interval was set to 5 s, thus allowing the participant to respond before the next trial starts. Stimuli were given in a random sequence, preceded by six additional practice items, referring to three left locations and three right locations. Responses to practice items were discarded from the analysis. In order to minimize any effect of practice, each target was only presented once.
Imaginal RT task
Following previously described procedures (Bartolomeo et al., 2005; Bourlon et al., 2008), 20 pairs of geographical locations (names of towns and regions of France) were selected. Each pair consisted of items located east or west of the city of Paris. Care was taken to choose cities or regions with approximately equal number of citizens across the two halves of the map. At the beginning of each trial, participants were asked to imagine a map of France. Afterwards, depending on the experimental conditions of the visual mental imagery task, a map of France (VMI with map condition) or a blank page (VMI without map condition) was displayed. Finally, the word “Paris” was played in the subject's headphone, followed 300 ms later by the auditory presentation of the word of a second French town or region (e.g., “Bordeaux”). After the presentation of the second auditory stimulus the participants were instructed to say “gauche” (“left”) if that stimulus referred to a city or region west from Paris, or “droite” (“right”) if that second stimulus indicated a city or region east from Paris. The intertrial
4.3.2.
Perceptual RT task
In the empty map condition, a white map of France was displayed on a black background. A black dot (10-mm in diameter) appeared at the map location corresponding to Paris. After 300 ms, a lateralized target replaced the central dot. The visual targets consisted of red dots 10-mm in diameter, which were displayed one at a time, at the locations corresponding on the map to the corresponding imaginal items. As for the imaginal task, subjects had to respond by saying “gauche” or “droite,” depending on whether the red dot appeared left or right of the black dot. Targets remained on until response or for a maximum of 5 s.
4.4.
Data analysis
As previously described, participants were divided into two groups depending on the order of presentation the imaginal conditions with or without map. The MF group received first the task with map, then the task without map and finally the perceptual task; the ESF group performed first the task without map, then the task with map and finally the perceptual task. The perceptual condition was always administered after the imaginal conditions. Vocal response times (RTs) and accuracy were recorded. A microphone connected to the computer digitized the naming time and ran a signal detection algorithm (using an adaptive threshold) which permit to measure reaction time with a 7-ms precision level. Eye movements were analyzed by estimating the total number of saccades and the total of number of fixation for movements to the right and to the left side of each stimulus. Eye movement recording started at stimulus presentation and ended at the onset of the vocal response (Fig. 4B). The periods of fixation were defined as the time intervals when no saccades occurred with amplitude larger than 30 pixels (0.76°) and a minimum duration of 100 ms (Fig. 4C). All the periods of fixation were identified offline, by using the ClearView software. Saccade parameters such as direction, velocity and amplitude were also extracted offline. Given the purposes of the present study, interest was restricted to the lateral direction of saccades relative to the central fixation cue. Saccades less than 10 mm (0.95°) in amplitude (corresponding to the diameter of the perceptual stimuli) were not considered for the analysis.
Fig. 4 – A. Example of gazepoint for the two eyes. When the camera lost the signal (arrow), the corresponding data were excluded from analysis. B. Procedure to determine the location (right or left) of the eyes on the screen after hearing the stimulus. C. Algorithm for filter fixations.
BR A I N R ES E A RC H 1 3 6 7 ( 2 01 1 ) 2 8 7 –2 97
295
296
BR A I N R ES E A RC H 1 3 6 7 ( 2 01 1 ) 2 8 7 –29 7
Acknowledgments We are grateful to C. Fouchard, M. Martinez, A. Bordier, A. Thierry, A. Chatenet, L. Lehenaff and V. Happel for help with data collection and to G. Dalla Barba and S. Rivaud for helpful discussion.
REFERENCES
Altmann, G.T., 2004. Language-mediated eye movements in the absence of a visual world: the “blank screen paradigm”. Cognition 93 (2), B79–B87. Ballard, D.H., Hayhoe, M.M., Pook, P.K., Rao, R.P.N., 1997. Deictic codes for the embodiment of cognition. Behav. Brain Sci. 20 (4), 723–767. Bartolomeo, P., 2002. The relationship between visual perception and visual mental imagery: a reappraisal of the neuropsychological evidence. Cortex 38 (3), 357–378. Bartolomeo, P., 2008. The neural correlates of visual mental imagery: an ongoing debate. Cortex 44 (2), 107–108. Bartolomeo, P., Chokron, S., 2002a. Can we change our vantage point to explore imaginal neglect? (Commentary on Pylyshyn: Mental imagery: In search of a theory). Behav. Brain Sci. 25 (2), 184–185. Bartolomeo, P., Chokron, S., 2002b. Orienting of attention in left unilateral neglect. Neurosci. Biobehav. Rev. 26 (2), 217–234. Bartolomeo, P., D'Erme, P., Gainotti, G., 1994. The relationship between visuospatial and representational neglect. Neurology 44, 1710–1714. Bartolomeo, P., Pagliarini, L., Parisi, D., 2002. Emergence of orienting behavior in ecological neural networks. Neural Process. Lett. 15 (1), 69–76. Bartolomeo, P., Bachoud-Levi, A.-C., Azouvi, P., Chokron, S., 2005. Time to imagine space: a chronometric exploration of representational neglect. Neuropsychologia 43 (9), 1249–1257. Beschin, N., Basso, A., Della Sala, S., 2000. Perceiving left and imagining right: dissociation in neglect. Cortex 36 (3), 401–414. Bisiach, E., Luzzatti, C., 1978. Unilateral neglect of representational space. Cortex 14, 129–133. Bourlon, C., Pradat-Diehl, P., Duret, C., Azouvi, P., Bartolomeo, P., 2008. Seeing and imagining the “same” objects in unilateral neglect. Neuropsychologia 46, 2602–2606. Bourlon, C., Duret, C., Pradat-Diehl, P., Azouvi, P., Loeper-Jeny, C., Merat-Blanchard, M., Levy, C., Chokron, S. and Bartolomeo, P., 2010. Vocal response times to real and imagined stimuli in spatial neglect: A group study and single-case report. Cortex. doi:10.1016/j.cortex.2010.03.004. Bower, G.H., Morrow, D.G., 1990. Mental models in narrative comprehension. Science 247 (4938), 44–48. Brandt, S.A., Stark, L.W., 1997. Spontaneous eye movements during visual imagery reflect the content of the visual scene. J. Cogn. Neurosci. 9 (1), 27–38. Chokron, S., Colliot, P., Bartolomeo, P., 2004. The role of vision on spatial representations. Cortex 40 (2), 281–290. Coslett, H.B., 1997. Neglect in vision and visual imagery: a double dissociation. Brain 120, 1163–1171. Demarais, A.M., Cohen, B.H., 1998. Evidence for image-scanning eye movements during transitive inference. Biol. Psychol. 49 (3), 229–247. Di Ferdinando, A., Parisi, D., Bartolomeo, P., 2007. Modeling orienting behavior and its disorders with “ecological” neural networks. J. Cogn. Neurosci. 19 (6), 1033–1049. Doricchi, F., Guariglia, C., Paolucci, S., Pizzamiglio, L., 1993. Disturbances of the rapid eye movements (REMs) of REM sleep in patients with unilateral attentional neglect: clue for the
understanding of the functional meaning of REMs. Electroencephalogr. Clin. Neurophysiol. 87 (3), 105–116. Doricchi, F., Iaria, G., Silvetti, M., Figliozzi, F., Siegler, I., 2006. The “ways” we look at dreams: evidence from unilateral spatial neglect (with an evolutionary account of dream bizarreness). Exp. Brain Res. 178 (4), 450–461. Ferreira, F., Apel, J., Henderson, J.M., 2008. Taking a new look at looking at nothing. Trends Cogn. Sci. 12 (11), 405–410. Findlay, J.M., Gilchrist, I.D., 2003. Active vision: the psychology of looking and seeing. Oxford University Press, Oxford. Gainotti, G., D'Erme, P., Bartolomeo, P., 1991. Early orientation of attention toward the half space ipsilateral to the lesion in patients with unilateral brain damage. J. Neurol. Neurosurg. Psychiatry 54, 1082–1089. Guariglia, C., Padovani, A., Pantano, P., Pizzamiglio, L., 1993. Unilateral neglect restricted to visual imagery. Nature 364, 235–237. Hebb, D.O., 1968. Concerning imagery. Psychol. Rev. 75 (6), 466–477. Hoffman, J.E., Subramaniam, B., 1995. The role of visual attention in saccadic eye movements. Percept. Psychophys. 57 (6), 787–795. Hoover, M.A., Richardson, D.C., 2008. When facts go down the rabbit hole: contrasting features and objecthood as indexes to memory. Cognition 108 (2), 533–542. Intons-Peterson, M.J., 1983. Imagery paradigms: how vulnerable are they to experimenters' expectations? J. Exp. Psychol. Hum. Percept. Perform. 9 (3), 394–412. Johansson, R., Holsanova, J., Holmqvist, K., 2005. What do eye movements reveal about mental imagery ? Evidence from visual and verbal elicitations. In: Bara, B.G., Barsalou, L., Bucciarelli, M. (Eds.), Proceedings of the 27th Annual Conference of the cognitive science society. Erlbaum, pp. 1054–1059. Johansson, R., Holsanova, J., Holmqvist, K., 2006. Pictures and spoken descriptions elicit similar eye movement during mental imagery, both in light and in complete darkness. Cogn. Sci. 30, 1053–1079. Kosslyn, S.M., 1994. Image and Brain: The Resolution of the Imagery Debate. The MIT Press, Cambridge, Mass. Kosslyn, S.M., Thompson, W.L., Ganis, G., 2006. The Case for Mental Imagery. Oxford University Press, New York. Laeng, B., Teodorescu, D.-S., 2002. Eye scanpaths during visual imagery reenact those of perception of the same visual scene. Cogn. Sci. 26, 207–231. Meador, K.J., Loring, D.W., Bowers, D., Heilman, K.M., 1987. Remote memory and neglect syndrome. Neurology 37, 522–526. Neisser, U., 1967. Cognitive Psychology. Appleton-Century-Crofts, New York. O'Regan, J.K., 1992. Solving the “real” mysteries of visual perception: the world as an outside memory. Can. J. Psychol. 46 (3), 461–488. Ortigue, S., Viaud-Delmon, I., Annoni, J.M., Landis, T., Michel, C., Blanke, O., Vuilleumier, P., Mayer, E., 2001. Pure representational neglect after right thalamic lesion. Ann. Neurol. 50 (3), 401–404. Posner, M.I., 1980. Orienting of attention. Q. J. Exp. Psychol. 32, 3–25. Pylyshyn, Z.W., 1973. What the mind's eye tells the mind's brain: a critique of mental imagery. Psychol. Bull. 80 (1), 1–24. Pylyshyn, Z.W., 1981. The imagery debate: analogue media versus tacit knowledge. Psychol. Rev. 88, 16–45. Pylyshyn, Z.W., 2003. Seeing and visualizing : it's not what you think. MIT Press, Cambridge, Mass. Richardson, D.C., Altmann, G.T., Spivey, M.J., Hoover, M.A., 2009. Much ado about eye movements to nothing: a response to Ferreira et al.: Taking a new look at looking at nothing. Trends Cogn. Sci. 13 (6), 235–236. Rode, G., Perenin, M.T., 1994. Temporary remission of representational hemineglect through vestibular stimulation. NeuroReport 5, 869–872. Rode, G., Revol, P., Rossetti, Y., Boisson, D., Bartolomeo, P., 2007. Looking while imagining: the influence of visual input on representational neglect. Neurology 68 (6), 432–437.
BR A I N R ES E A RC H 1 3 6 7 ( 2 01 1 ) 2 8 7 –2 97
Rode, G., Cotton, F., Revol, P., Jacquin-Courtois, S., Rossetti, Y., Bartolomeo, P., 2010. Representation and disconnection in imaginal neglect. Neuropsychologia 48 (10), 2903–2911. Shepherd, M., Findlay, J.M., Hockey, R.J., 1986. The relationship between eye movements and spatial attention. Q. J. Exp. Psychol. 38A, 475–791. Sokolov, E.N., 1963. Higher nervous functions: the orienting reflex. Annu. Rev. Physiol. 25, 545–580.
297
Spivey, M.J., Geng, J.J., 2001. Oculomotor mechanisms activated by imagery and memory: eye movements to absent objects. Psychol. Res. 65 (4), 235–241. Thomas, N.J.T., 1999. Are theories of imagery theories of imagination? An active perception approach to conscious mental content. Cogn. Sci. 23 (2), 207–245. Tulving, E., Thomson, D.M., 1973. Encoding specificity and retrieval processes in episodic memory. Psychol. Rev. 80 (5), 352–373.