Visual search disorders beyond pure sensory failure in patients with acute homonymous visual field defects

Neuropsychologia 47 (2009) 2704–2711

Contents lists available at ScienceDirect

Neuropsychologia journal homepage: www.elsevier.com/locate/neuropsychologia

Visual search disorders beyond pure sensory failure in patients with acute homonymous visual field defects Björn Machner a,∗ , Andreas Sprenger a , Detlef Kömpf a , Thurid Sander a , Wolfgang Heide b , Hubert Kimmig a , Christoph Helmchen a a b

Department of Neurology, University Lübeck, D-23538 Lübeck, Germany Department of Neurology, General Hospital Celle, D-29223 Celle, Germany

a r t i c l e

i n f o

Article history: Received 6 October 2008 Received in revised form 18 May 2009 Accepted 22 May 2009 Available online 3 June 2009 Keywords: Visual search Homonymous visual field defects Spatial memory Saccades Gaze-contingent displays Simulated field defect

a b s t r a c t Patients with homonymous visual field defects (HVFD) are often crucially disabled during self-guided visual exploration of their natural environment. Abnormal visual search may be related to the sensory deficit, deficient spatial orientation or compensatory eye movements. We tested the hypothesis that visual search in HVFD is purely determined by the visual–sensory deficit by comparing nine patients with HVFD due to occipital stroke in an acute stage to nine healthy subjects with technically simulated “virtual” homonymous visual field defects (vHVFD) and to nine controls with normal visual fields. The simulated gaze-contingent visual field defects in vHVFD subjects were individually matched to the patients’ HVFD with respect to their size and side. Eye movements were recorded while subjects searched for targets among distractors and indicated target detection by clicks. All patients, in particular those with lesions involving the inferior occipito-temporal (fusiform) gyrus, but also those with small lesions restricted to the visual cortex, showed longer search durations than vHVFD subjects. This was tightly related to the higher number of fixations and particularly “re-fixations” (repeated scanning of fixated items). Working memory across saccades during the search was intact (no increased “re-clicks”). Scanpath strategies were similar in patients and vHVFD subjects. For both groups amplitude and frequency of saccades did not differ between the hemifields. In HVFD patients with acute occipital brain lesions, visual input failure does not fully account for abnormal visual search. It might either result from disconnections of the primary visual cortex to associated occipital and temporal brain areas or reflect an early stage of compensatory eye movements which differ from chronic HVFD patients. © 2009 Elsevier Ltd. All rights reserved.

1. Introduction Patients with homonymous visual field defects (HVFD) are often heavily impaired in everyday life activities. They may collide with other people or objects like shopping trolleys or door frames and often suffer from reading difficulties. Homonymous hemianopia describes a HVFD, in which half of the visual field is blind. If the HVFD is restricted to only a quarter of the visual field, it is referred to as quadrantanopia (Kolb & Whishaw, 1996). Most common causes of HVFD are unilateral infarction of the posterior cerebral artery, traumatic brain injury and tumours. The visual field defects persist in the majority (>80%) of occipital stroke patients (Brandt, Thie, Caplan, & Hacke, 1995).

∗ Corresponding author at: Department of Neurology, University of Lübeck, Ratzeburger Allee 160, D-23538 Lübeck, Germany. Tel.: +49 451 500 3709; fax: +49 451 500 2489. E-mail address: [email protected] (B. Machner). 0028-3932/$ – see front matter © 2009 Elsevier Ltd. All rights reserved. doi:10.1016/j.neuropsychologia.2009.05.016

Despite of persistent HVFD patients partially adopt to the difficulties during the course of the disease. They change the way how they explore a stationary or moving target in their visual environment, e.g. by the use of compensatory eye and headmovements (Zangemeister, Meienberg, Stark, & Hoyt, 1982). Thus, their visual search behaviour might be changed first by the lesion (acute stage) and subsequently by compensational efforts (chronic stage). Visual search behaviour can best be described by eye movement recordings (Mort & Kennard, 2003). They give insights into cognitive functions like attention, strategic planning and spatial memory (Mort & Kennard, 2003). In previous studies, eye movements in HVFD patients were largely recorded in a chronic stage during reading (McDonald, Spitsyna, Shillcock, Wise, & Leff, 2006), dotcounting (Zihl, 1995) or deliberately scanning of stationary scenes (Pambakian et al., 2000; Zangemeister & Utz, 2002). However, little is known about search behaviour in HVFD patients during an explorative visual search for targets among distractors within the first few weeks following the acute stroke.

B. Machner et al. / Neuropsychologia 47 (2009) 2704–2711 Table 1 Patients’ clinical data.

2705

Case

Age (years)

Sex

Lesion

H1

60

F

Left PCA

9

+

H2

48

M

Right PCA

8

+

H3

73

M

Right oICB

13

−

H4

70

M

Right oICB

10

−

H5

29

F

Left PCA

12

−

and they showed no further neurological signs of motor or sensory loss besides the stated visual field defects. The vHVFD group consisted of 9 healthy adults (mean age 58.7 ± 15.1, range 28–77). Each vHVFD subject was matched to one patient by technically creating a visual field defect identical in size, side of visual field defect and macular sparing. Areas of both “absolute” and “relative” visual field defects were matched as complete HVFD (black mask) in the virtual HVFD subjects. The control group consisted of 9 healthy adults (mean age 53.4 ± 14.1, range 37–76), each of them assigned to one matching pair of one “virtual” and one patient as described before. Mean age did not differ significantly between all groups. All participants gave their informed written consent, according to the Declaration of Helsinki. Search coil experiments in human subjects have been approved by the ethics committee of University Luebeck.

H6

59

M

Right PCA

16

+

2.2. Stimuli and tasks

H7

72

M

Left PCA

8

−

H8

55

M

Right PCA

11

+

H9

71

M

Left PCA

30

+

Time since symptom onset (days)

Field defect

Macular sparing

F female; M male; PCA infarction of the posterior cerebral artery; oICB occipital intracerebral bleeding.

It has been a matter of debate whether scanning behaviour in chronic HVFD patients is primarily generated by the visual field defect (Tant, Cornelissen, Kooijman, & Brouwer, 2002) or additional higher order brain damage (Zihl, 1995). The aim of our study was to test whether patients’ visual search abnormalities in the acute stage of occipital stroke can be explained exclusively by visual–sensory deprivation in the affected hemifield. The early time of examination was chosen since we expected that compensatory eye movement strategies have not been established yet. Therefore we examined scanning behaviour during a highly demanding visual search task (Machner, Sprenger, Kömpf, & Heide, 2005; Sprenger, Kömpf, & Heide, 2002) in acute hemianopic and quadrantanopic patients and compared them to healthy agematched subjects with technically simulated, gaze-contingent visual field defects (virtual HVFD). We hypothesized that visual search behaviour in HVFD patients is not just determined by a pure visual–sensory deficit and therefore expected additional deficits of spatial and temporal scan integration when comparing patients to vHVFD and healthy control subjects. 2. Methods 2.1. Subjects All 9 patients (mean age 59.7 ± 14.4, range 29–73) were in-patients of the Department of Neurology/University Hospital Lübeck (Table 1). Unilateral occipital brain damage due to infarction/bleeding has been confirmed by MRI brain scan. For analysis, patients’ lesions were plotted on normalized brain scans (Fig. 1a) using SPM2 (Statistical Parametric Mapping, http://www.fil.ion.ucl.ac.uk/spm, Welcome Department of Imaging Neuroscience, London, UK) and MRIcro (Mort et al., 2003; Rorden & Brett, 2000). Patients with multiple brain lesions were excluded from the study, as were patients under current use of psychoactive or sedating medications. Patients underwent extensive clinical neuropsychological examination using subtests from the “NET” neglect battery, the adapted German version of the “Behavioural Inattention Test (BIT)” (Wilson, Cockburn, & Halligan, 1987) including clock drawing, figure copying, star cancellation and reading. Patients’ results were within normal limits without any signs of spatial hemineglect. The extent of the individual visual field defect was assessed with the automated static Octopus 500EZ perimeter (Interzeag AG, Switzerland). Its background luminance was 1.27 cd/m2 and maximum stimuli luminance was 318 cd/m2 , providing a measurement range of 0–47 dB. Standard stimulus size was Goldmann III (25.9 , about 0.5◦ visual angle). Using the 2-niveau-test at 132 test points “absolute” and “relative” visual field defects were detected (supplementary e-Figure e1). If there was a loss of the differential luminance sensitivity >6 dB compared to age-related normal thresholds, a “relative” visual field defect (reduced sensitivity) was recorded. If there was no sensitivity at maximum luminance, an “absolute” visual field defect (0 dB, complete loss of sensitivity) was detected. The homonymous visual field defects of our patients were either hemianopic or quadrantanopic (Table 1, supplementary e-Figure e1). All patients had a visual acuity above 0.7, normal colour perception

Subjects sat in a dark room in front of a Sony Multiscan 17se II monitor. Its border was covered by a black paper frame in order to reduce extra-display visual information. Visual stimuli (Fig. 2) were presented at a 100 Hz refresh rate, generated by the Visual Stimulus Generator (VSG 2/4, Cambridge Research Systems Ltd.). The eye-to-screen distance was 60 cm. Visual features of the stimulus items were colour and form. Colour was defined by CIE coordinates (red 0.601, 0.322; green 0.218, 0.579; blue 0.147, 0.07) as used in a previous study (Zelinsky, 1996). The objects’ form feature as either a triangle, a square or a circle, each subtending 0.5◦ of the visual angle. The objects were presented at a luminance of 5 cd/m2 on a dark background (<0.1 cd/m2 ) in a dark room. Each stimulus display consisted of 40, 60, or 80 items, among which 0, 1, 4, or 8 targets had to be detected. The items were distributed randomly across the whole screen, covering a visual field area of 32◦ × 24◦ , with a distance of at least 1◦ between neighbouring items. The task was to search for all targets defined by a specified feature such as colour or form, or a conjunction of colour and form. For the testing an original set of 84 different displays was created, each of them was presented only once so that the whole session consisted of a maximum of 84 single trials. The experiment always started with a “basic stimulus subset” consisting of 21 trials covering the 3 different tasks, 2 different numbers of targets (0, 4) and the 3 set sizes as well as 3 extra trials of conjunction search with 1 or 8 targets. After completion of this subset additional trials were randomly presented by the stimulus software. The “basic stimulus subset” was introduced in order to provide a minimum number of comparable basic conditions for analysis, because we expected some patients not being able to complete all trials. The subjects were instructed to search for all targets as accurately and quickly as possible. The vHVFD subjects were informed about their “forthcoming” visual field defect. Upon detecting a target, subjects had to press a button (left mouse click). After finishing the search in each trial, subjects had to press another button and the stimulus disappeared (right mouse click). 2.3. Recording procedure and simulation of “virtual” HVFD Subjects’ eye movements were recorded in a scleral search coil system (CNC Engineering, Seattle). After performing 12 test trials the search coil was inserted into the anaesthetized dominant eye. No more than 30 min after its insertion, the search coil was removed. Subjects’ head was not fixed but rested comfortably on a chin rest and subjects were instructed not to move their head. Occasional small head-movements were recorded and used for offset correction. Eye movement data were digitized with a sampling rate of 500 Hz and stored in a binary format on the recording PC. Visual stimuli and the subjects’ responses (per mouse click) were stored on the stimulus PC. The communication between the PCs guaranteed exact timing for joining both data files in the off-line analysis. In order to achieve highly accurate eye position data, a 9-point calibration was performed at the beginning and the end of the test run. Small non-linear distortions were corrected by means of a neural network algorithm using a parametric self-organizing map (PSOM) (Pomplun, Velichkovsky, & Ritter, 1994). The spatial accuracy of recorded eye position with respect to the stimulus display was always better than 0.4◦ , usually 0.15◦ . Virtual HVFD was simulated by a moving window which completely blanked one part of the visual field depending on the current gaze position (“gaze-contingent displays”), i.e. the virtual hemianopic/quadrantanopic field defect moved with the eyes. The stimulus program responsible for the “moving window” was written in C++ using fast algorithms. The window was a black mask that moved in case of an eye position displacement of more than 0.5◦ compared to the last three samples. Taking all conceivable system delays into account, a screen refresh after an eye movement occurred at least after the next monitor cycle, i.e. in less than 20 ms. This rapid screen update prevented vHHQ subjects from perceiving stimuli in the blanked area after a new saccade. 2.4. Analysis of data and statistics Data were analyzed by an interactive program which read, calibrated and presented every single picture and the consecutive scanpath of each subject (Sprenger

2706

B. Machner et al. / Neuropsychologia 47 (2009) 2704–2711

Fig. 1. Overlapping, subtraction and mask lesion analyses. The overlap map (a) shows the degree of involvement of each voxel in the lesions of 9 HVFD patients after acute occipital brain damage (lesions flipped to one side). The range of the rainbow colour scale derives from the absolute number of patients, each colour represents a defined number of patients’ lesions in this area. By subtracting lesions of the mildly (group B, FSD < 3.5, n = 5) from the severely impaired patients (group A, FSD > 3.5, n = 4) the subtraction map (b) identifies lesion sites that are more common than others for a defined behavioural impairment, here the increased factor of search duration (FSD). Each percentage bar in the colour panel (b) represents 20% increments, so the lightest blue represents 81–100% of group B while the middle purple bar designates regions where there is an identical percentage of subjects of group A and B (0% difference). The subtraction map (b) and the subsequent mask analysis (c), which identifies the region that is unique to group A, reveal mesio-ventral parts of the temporal lobe corresponding to the fusiform gyrus, that are damaged in severely impaired but spared in mildly impaired patients. (For interpretation of the references to colour in this figure legend, the reader is referred to the web version of the article.)

et al., 2002). Each scanpath was inspected separately, while at the same time fixations and clicks on targets were checked manually. While fixations were rejected if they were outside the stimulus screen, a manual assignment of a click to an attended target was necessary if the program failed to do so, e.g. in case the gaze was moved away from the previously fixated target before the subject pressed the button. SPSS 15 (SPSS Inc., Chicago, IL) was used for statistics. Usually, the design of the study would have required a rather complex 3 × 4 × 3 × 3 ANOVA (three different quantities of items, four different quantities of targets, three different tasks, three groups of subjects). However, different numbers of targets and the changing set size were intentionally chosen to prevent learning effects during the trials as well as an early cancellation of the search. Subjects had to explore the whole screen before finishing their search which guaranteed “complete” scanning behaviour for further comparative scanpaths analyses. Depending on their state the patients performed between 27 and 82 trials per 30 min session, thus the “basic stimulus subset” was performed by all the patients. Only those trials from the individually matched vHVFD subjects and controls were further analyzed that had been completed by the patients. Due to this match in a triple-wise manner (one virtual, one patient and one control), statistics were conducted based on the same number and type of trials within each of the three subject groups. Since we were interested in the group differences and the influence of the task rather than influences of the target number or set size, the data were aggregated and further analyzed depending on the factors “group” and “task” in a 3 × 3 ANOVA with subsequent post hoc tests using the Bonferroni correction. For mean comparison of two groups or conditions t-tests were performed (d = mean difference). Data reported in the text are given in mean ± standard error of mean.

scanning patterns. The subjects’ scanpaths were assigned to either a circular, line-wise, column-wise, 8-shaped or chaotic pattern (Fig. 2). This classification derived from the experience with scanpath patterns elicited by the same stimulus in previous studies (Machner et al., 2005; Sprenger et al., 2002). The initial ratings of both investigators matched in 78%. In a post hoc joint discussion there was no final agreement on a distinct scanning pattern in only 2% of the trials and these were excluded from further statistics. 3 × 3 ANOVA on the rate of structured search (pictures with geometric scanpaths/all pictures × 100) revealed a significant influence only for the factor “task” (F(2,80) = 5.1, p < 0.01) but not for “group”. The average rate of pictures with structured scanpaths did not differ significantly between controls, patients or virtuals, neither in colour (controls: 55% ± 6/patients: 49% ± 11/virtual HH: 70% ± 11) nor in conjunction (77% ± 6/67% ± 11/75% ± 11) nor in form search (83% ± 7/82% ± 11/86% ± 11). The difference of structured scanpaths between colour and form search reached level of significance only in the control group (d = 28% ± 9, p < 0.05). Subanalysis of structured scanpaths, e.g. line-wise versus column-wise scanpath strategies, revealed no significant impact on search duration. 3.2. Search duration

3. Results 3.1. Scanpaths and strategies In order to assess search strategies qualitatively, two investigators (BM and AS) visually analyzed each scanpath for geometric

The “search duration” was measured from the onset of the stimulus to the moment when subjects pressed the right button indicating that their search was finished. 3 × 3 ANOVA on search duration revealed a significant influence for the factor “group” (F(2,80) = 19.7, p < 0.001) and “task”

B. Machner et al. / Neuropsychologia 47 (2009) 2704–2711

2707

Fig. 2. Scanpaths in colour (a–c) and form (d) search task. (a) 72-year-old patient with right homonymous hemianopia; (b) 67-year-old healthy subject with a virtual right sided homonymous hemianopia; (c) 52-year-old healthy control subject. While hemianopic patient (a) and virtual hemianopic subject (b) show similar strategic, column-wise search patterns, the control subject uses colour pop-out for detecting the targets. (d) 70-year-old patient with left homonymous hemianopia. Note the various re-fixations of items and targets in the patient’s scanpath while searching for a target with a defined form (triangle). However, the scanpath seems to follow a systematic line-wise pattern. Targets are red items (n = 4) in (a–c) and triangles (n = 4) in (d). White lines indicate saccades and circles fixations along the scanpath, the longer the fixation duration the larger the circles’ radius. (For interpretation of the references to colour in this figure legend, the reader is referred to the web version of the article.)

(F(2,80) = 8.1, p < 0.01). Patients showed longer search durations than controls (d = 16.7 s ± 2.7, p < 0.001, Fig. 3) and virtuals (d = 10.7 s ± 2.7, p < 0.001). There was a trend for longer search durations in virtuals than in controls (d = 5.9 s ± 2.7; p < 0.1). Within each group searching for targets with a defined colour was significantly faster than searching in the form task (p < 0.05).

to group A. The subtraction map in Fig. 1b shows a large area of overlapping lesions of both groups related to the uniform vascular supply of the posterior cerebral artery. However, both subtraction and mask analysis revealed mesio-ventral parts of the temporal

3.3. Influence of lesion and patients’ characteristics on search duration The factor of search duration (FSD) is a single value defined for each subject, by relating the individual search duration in each display to the mean search duration of all controls in the same display (Machner et al., 2005). There was no correlation between lesion volume and FSD in the patients. A median split analysis was performed by dividing the HVFD patients into 2 sub-groups (mildly impaired patients: FSD < 3.5, n = 5; severely impaired patients: FSD > 3.5, n = 4). The mean lesion volume did not differ significantly between the subgroups. Accordingly, patient H3 had a prolonged search duration (FSD = 4.1) but only a small brain lesion restricted to the visual cortex. In order to define a crucial brain area which might be involved in the severely impaired patients (FSD > 3.5) but spared in the others we performed subtraction (Fig. 1b) and mask (Fig. 1c) analyses using the MRIcro software (Rorden, Karnath, & Bonilha, 2007). In the subtraction analysis lesions of the mildly impaired patients (group B) were subtracted from the lesions of the severely impaired patients (group A). The mask analysis identifies regions that are unique

Fig. 3. Search duration for each search task. Colour search is reflected by shorter search durations in all subjects. HVFD patients required longer search durations than vHVFD subjects (“virtuals”).

2708

B. Machner et al. / Neuropsychologia 47 (2009) 2704–2711

Table 2 Mean eye movement parameters for each study group.

Search duration [s] Item re-fixations [%] Saccades [n] Saccadic amplitude [◦ ] Fixation duration [ms]

Controls

Virtuals

Patients

7 (2) 38 (2) 29 (7) 6.3 (0.3) 230 (6)

13 (2) 20 (2)a 43 (7) 5.7 (0.3) 263 (6)a

24 (2)a,b 61 (2)a,b 89 (7)a,b 5.1 (0.3)a 243 (6)b

a Significant difference (p < 0.05) compared to controls, b significant difference (p < 0.05) compared to virtuals. Post hoc t-tests; Bonferroni corrected. Data given in mean (sem).

lobe, i.e. the fusiform gyrus, that were affected in at least half of the severely impaired but spared in the mildly impaired patients (Fig. 1b/c). Subjects’ age correlated significantly with the FSD (n = 27, r = 0.49, p < 0.01). Taking into account the influence of patient’s age, right HVFD patients (FSD = 5.2 ± 0.9) had a significantly longer search duration than left HVFD patients (FSD = 2.3 ± 0.9, covariate age, p < 0.05). This difference was not seen between “left” and “right” virtuals. There was no significant correlation between patients’ macular sparing or size of field defect and their FSD. 3.4. Saccade parameters 3 × 3 ANOVA on the number of fixations showed a significant influence for the factor “group” (F(2,80) = 18.0, p < 0.001) as well as “task” (F(2,80) = 7.7, p < 0.001). Colour search required less saccades than form search in each group (p < 0.05). Subjects’ search duration correlated significantly with their absolute number of fixations (n = 27, r = 0.98, p < 0.001). Patients made more saccades (89 ± 7) than controls (29 ± 7, p < 0.001) and virtuals (43 ± 7, p < 0.001). Controls’ mean saccadic amplitude in the colour task was significantly larger than in the form task (d = 3.1◦ ± 0.4, p < 0.001). This was also true for patients (d = 2.5◦ ± 0.8, p < 0.05) but not for virtuals (d = 0.9◦ ± 0.7, p > 0.05). Patients’ mean saccadic amplitude was significantly smaller than in controls (d = 1.2◦ ± 0.4, p < 0.01) but not than in virtuals (d = 0.6 ± 0.4, p > 0.05). 3 × 3 ANOVA on fixation duration revealed a significant influence for the factor “group” (F(2,80) = 8.1, p < 0.01). Virtuals’ mean fixation duration (263 ms ± 6) was longer (p < 0.05) than in controls (230 ms ± 6) and patients (243 ms ± 6). 3.5. Item re-fixations Items, including distracters and targets, were marked as “fixated” when the fixation was located within a radius of 2◦ , or for longer fixations (>250 ms) up to 4◦ . A fixation of at least one item was taken as an “item fixation”. An “item re-fixation” was identified when the time and space interval between two fixations of the same item was longer than 1 s and more than 5◦ . Thus small saccades for correction did not contaminate the number of “real” re-fixations, namely fixations re-visiting previously searched locations. The “rate of item re-fixations [%]” was defined by “item re-fixations/item fixations × 100” in each picture. 3 × 3 ANOVA on the rate of item re-fixations revealed a significant influence for the factors “group” (F(2,80) = 86.2, p < 0.001) and “task” (F(2,80) = 20.3, p < 0.001). Patients showed a higher rate of re-fixations than controls (d = 23.3 ± 3.2, p < 0.001, Fig. 4). Virtuals’ rates of re-fixations were not just lower than in the patients (d = 41.3 ±3.2, p < 0.001) but also than in the controls (d = 18.0 ± 3.2, p < 0.001). A summary and mean comparison of the major eye movement parameters are given for the three subject groups in Table 2.

Fig. 4. Rate of item re-fixations [%]. While patients show higher rates of re-fixations than controls, healthy subjects with a virtual homonymous visual field defect refixate less than patients and even controls.

3.6. Click response All subjects searched very accurately and successfully, marking almost all targets (clicks) and hardly clicking twice for one target (re-clicks). The click rate (correct clicks/number of targets × 100) in the 4-target trials was above 90% in all groups. The re-click rate (reclicks/number of clicks × 100) was very low, not exceeding 3.5% in each group. 3.7. Analysis of saccades in and towards the blind visual field When patients as well as virtuals fixated the centre of an image, they were able to see only the part of the picture corresponding to their intact visual field. As the homonymous visual field defect is retinotopic, the “blind” part of the picture corresponding to the blind visual field changes with each new saccade. We therefore analyzed parameters of saccades made into the blind or seeing direction, which are referred to as saccades towards the affected or intact side. Furthermore we analyzed parameters of saccades which landed in the absolute left or right half of the screen, which at central fixation corresponds to a blind and a visible hemifield. These saccades are referred to as saccades in the affected or unaffected hemifield. In both groups, patients and virtuals, there were no significant differences between saccades which landed in the left or right half of the screen concerning total number, amplitude or fixation duration. Furthermore no significant differences were found when comparing rates of item re-fixations between the two absolute hemifields. For all three groups numbers of saccades towards either side were evenly distributed. Furthermore there was no significant difference between these saccades with respect to saccadic amplitude, fixation duration or rates of item re-fixations. 3.8. Quantifying the “pop-out” effect of colour The feature “colour” is known to attract the focus of attention even over large distances—this is called the pop-out effect (Treisman & Gelade, 1980; Wolfe, Cave, & Franzel, 1989). In the control group the pop-out effect of the feature “colour” was reflected by significantly (p < 0.01) larger amplitudes of saccades towards intended targets in the colour task than in the other two tasks

B. Machner et al. / Neuropsychologia 47 (2009) 2704–2711

2709

nificantly larger during the colour task. Irrespective of the search task, fixations of our HVFD patients were equally distributed across the whole screen. In particular there was no direction-specific bias, i.e. saccades directed towards the affected hemifield did not differ concerning frequency and amplitude from those directed towards the intact hemifield. 4.2. Virtual versus real HVFD

Fig. 5. Mean amplitude of target saccades. The pop-out effect of the targets’ colour attracts the focus of attention even over large distances. Therefore large-amplitude saccades towards the targets (‘target saccades’) occurred only in colour search task as to be seen in healthy control subjects. As expected vHVFD subjects (virtuals) did not show significantly larger saccades in colour search due to the complete blanking of parts of the visual field. HVFD patients had larger target saccades during colour search, maybe indicating preserved colour pop-out and reflecting “blind sight”.

(11.1◦ ± 0.5 versus 8.7◦ ± 0.5 and 8.0◦ ± 0.5, Fig. 5). In virtuals, the amplitudes of target saccades did not differ between colour (8.8◦ ± 0.8) and form search (7.9◦ ± 0.8). The patients showed significantly larger target saccades in the colour than in the form task (9.7◦ ± 1.0 versus 6.6◦ ± 0.9, p < 0.05). However, their separately analyzed target saccades towards the blind visual field were not significantly larger in the colour than in the form task (d = 3.1◦ ± 1.5, n = 9, p = 0.06). 4. Discussion We compared eye movements of patients with homonymous visual field defects due to acute occipital stroke to both healthy controls and subjects with a technically simulated homonymous visual field defect (“virtual” HVFD) during an exploratory visual search task. We found striking differences not only between our acute HVFD patients and previously described chronic HVFD patients but also between our patients and vHVFD subjects. This virtual model of pure visual–sensory loss and the visual search paradigm with non-semantic targets and distractors allowed us to specify features of visual search disorders in patients with acute HVFD beyond a pure sensory deficit. 4.1. HVFD patients versus controls Patients had longer search durations than healthy controls. This was related to a higher number and smaller amplitude of saccades in both hemifields, and to a higher number of “re-fixations”, i.e. fixations which were directed to previously fixated items. Nevertheless, patients’ scanpaths were not chaotic but followed a defined geometric pattern similar to those of healthy controls. Likewise, our patients seemed to benefit from the pop-out effect of colour (Sprenger et al., 2002) since search duration was reduced and saccades towards intended targets were larger in the colour task. However, we cannot prove the concept of “blindsight” of very salient features like colour (Stoerig & Cowey, 1997; Weiskrantz, Warrington, Sanders, & Marshall, 1974) in our patients, since those saccades towards targets in the blind hemifield alone were not sig-

Following the hypothesis of HVFD as a pure visual input failure, vHVFD subjects were expected to present an identical or at least very similar search performance when compared to “real” HVFD patients, as proposed by previous studies (Tant et al., 2002; Zangemeister & Utz, 2002). Stating the powerful and reliable model of our virtual HVFD, scanpaths and some saccade parameters in vHVFD subjects resembled those of HVFD patients, whose search duration was prolonged and saccadic amplitudes decreased as compared to healthy controls. Both groups, HVFD patients and vHVFD subjects, did not show a fixation bias towards the affected side. However, our data still cast doubt on the view of hemianopic scanning behaviour as a consequence of pure sensory failure since there were also striking differences between vHVFD subjects and HVFD patients. While patients’ mean single fixation duration was significantly lower, the search duration and the number of fixations were twice and the percentage of re-fixations even three times as high in the patient as in the vHVFD group. How can we explain the severely prolonged search duration in our patients, given that the pop-out-effect was partially preserved and that strategic search strategies and fixation distributions towards both hemifields resembled those of the vHVFD subjects? A key may be the high number of re-fixations in our patients. These re-fixations cannot be explained by a pure visual input failure, since vHHQ subjects with identical visual field defects showed very low re-fixation rates. What do the re-fixations tell us about HHQ patients’ higher cortical functions? First, considering re-fixations as an indicator of limited functional memory (Gilchrist & Harvey, 2000), our patients may have had a spatial working memory deficit. In a previous study in neglect patients the “re-clicks” during a visual search paradigm were found to be more reliable indicators than re-fixations for misjudging old items as new and the “re-click rate” correlated well with neglect severity (Mannan et al., 2005). Furthermore, “re-clicks” as a possible indicator for a spatial working memory deficit across saccades were found to be increased in a well studied neglect patient during a visual search task (Husain et al., 2001) who accordingly revealed spatial working memory deficits in established neuropsychological testing, i.e. the computerized CANTAB test battery (Owen, Downes, Sahakian, Polkey, & Robbins, 1990). In contrast, our patients did not show increased re-clicks. Thus, at least their working memory across saccades during this specific search task seemed to be sufficient to remember the target locations already visited and clicked upon. However, we cannot definitely rule out a general spatial working memory deficit in our patients because the neuropsychological testing of our study was not focused on such deficits. Second, do the re-fixations represent a deficient “inhibition of return”? Intact inhibition of return refers to direct attention away from a previously fixated object and avoid immediate re-fixations (Klein, 1988; Posner, Rafal, Choate, & Vaughan, 1985). This effect in a sequence of saccades has been called “inhibition of saccadic return”, a short-lived effect (about 1 s) which is strongest for previously fixated objects in a radius of about 4◦ and seems to be reset after each saccade (Hooge & Frens, 2000). However, the impact of inhibition of return on a free visual search task is a matter of debate (Gilchrist & Harvey, 2000; Hooge, Over, van Wezel, & Frens, 2005; Klein & MacInnes, 1999). As the re-fixations in our study are characterized

2710

B. Machner et al. / Neuropsychologia 47 (2009) 2704–2711

by time and space intervals >1 s and >5◦ , a possible impairment of inhibition of return is unlikely to explain the high rate of re-fixations in our patients. Third, a lack of strategic planning reflected in chaotic scanpaths would lead to multiple re-fixations. A real strategic deficit was not found, but it remains a matter of debate, whether our patients’ “second-look-strategy” by scanning the original scanpath again, and thus producing lots of re-fixations, can still be classified as intact strategy. This “second look” might have been used by the patients in order to compensate for the missing visual input at a single fixation due to the scotoma. In contrast, the vHVFD subjects seemed to cope with the scotoma partially by an increase of the single fixation duration in order to have a sufficiently long visual processing time. That may have prevented a second look in vHVFD subjects as supported by their low number of re-fixations. 4.3. HVFD scanning behaviour is influenced by age and site of the brain lesion Our findings are consistent with a previous study on HVFD scanning behaviour that showed increased search duration and high re-fixation frequency particularly in those patients with an “impaired spatial organization and integration of visual scanning due to additional brain damage of the posterior thalamus or parietooccipital brain areas” (Zihl, 1995). The damage to the latter brain regions may cause spatial neglect and disturb exploratory visual search tremendously (Sprenger et al., 2002). While parietal brain areas were spared in our patients, mask and subtraction lesion analyses revealed mesio-ventral parts of the temporal lobe, in particular the fusiform gyrus (gyrus occipito-temporalis), that were damaged in at least half of the severely impaired patients but spared in the mildly impaired patients. These occipito-temporal regions, in particular V4 within the fusiform gyrus, are known to be involved in object, form and colour recognition. They are part of the ventral stream processing the “what” of the visual information in contrast to the “where” processed via the dorsal stream in parietal and superior temporal brain areas (Goodale, 1993; Ungerleider & Haxby, 1994; Ungerleider & Mishkin, 1982). Hence, deficits in object and colour recognition might have been partially responsible for the various re-fixations and subsequently higher search duration of our severely impaired hemianopic patients. Although these findings are in accordance with previous studies with larger patient samples (Zihl, 1995), our lesion-symptom mapping derived from a small number of patients and should therefore be interpreted with caution. Damage to intra- and interhemispheric connecting fibres of the occipital lobe might be a reason for “non-topographic” effects like reduced accuracy and increased processing time (Ringo, Doty, Demeter, & Simard, 1994; Rizzo & Robin, 1996), applying in particular for the search disturbances of HVFD patients with small lesions restricted to the occipital lobe. It cannot be inferred from our data that the lesion’s size per se accounted for the degree of visual search impairments. Rather the lesion’s side might have had an additional impact, since our right HVFD patients had longer search durations than left HVFD patients. In fact, visual search has been previously reported to be more affected in right hemianopic than in left hemianopic patients (Hildebrandt, Giesselmann, & Sachsenheimer, 1999; Zihl, 1995). One major difference between our study and previous studies is the age of the brain lesion, i.e. the short period of time between lesion onset and eye movement recordings. While our patients were measured in an acute stage within the first few weeks following their stroke, previous studies examined HVFD patients in their chronic stage (months/years) for persisting impairments like increased search durations and a higher number of fixations as well as the development of different compensatory eye

(and head) movements (Gassel & Williams, 1963; Ishiai, Furukawa, & Tsukagoshi, 1987; Pambakian et al., 2000; Tant et al., 2002; Zangemeister et al., 1982; Zangemeister, Oechsner, & Freksa, 1995; Zihl, 1995). HVFD patients who are long-term adapted to their visual field loss (>6 months) were shown to spend even more time searching in the affected than in the intact hemifield reflecting an “overcompensation” (Ishiai et al., 1987; Pambakian et al., 2000). In contrast, there was no hemifield bias in our HVFD patients’ fixations, suggesting that probably due to their acute stage of stroke an “overcompensation” has not taken place yet. We also did not observe large overshooting (hypermetric) saccades towards the affected side followed by step-wise hypometric saccades towards the intact side as previously reported in chronic hemianopic patients (Meienberg, Zangemeister, Rosenberg, Hoyt, & Stark, 1981). In contrast, the amplitudes of saccades towards both directions did not differ in our patients, and types and frequency of geometric search strategies were similar to our healthy controls. There is one related study (Tant et al., 2002) using a simulated homonymous hemianopia in healthy subjects (“sHH” condition). Their search behaviour during a dot-counting task was compared to a control condition (no simulation) and to chronic hemianopic patients (HH). Some of their findings were different from ours: they reported (i) prolonged search durations in sHH more than in HH, (ii) a biased hemifield distribution of fixations with “more fixations in the ipsilateral hemispace” in both sHH and HH and (iii) smaller saccadic amplitudes in the “ipsilateral direction” (i.e. “into the blind hemifield”) than in the “contralateral direction” in both HH and sHH subjects. In accordance with our data “fixation durations” were longer in the sHH group than in their HH patients. The authors concluded that they “evidenced clear parallels between simulated and real HH suggesting that hemianopic scanning behaviour is primarily visually elicited, namely by the visual field defect, and not by additional brain damage” (Tant et al., 2002). In contrast to the latter study, we performed lesion-symptom mapping (Rorden et al., 2007) and analyzed re-fixations and search strategies. We could assess the strong influence of the feature colour on visual search while in the previous study a black and white dot paradigm was applied. The most striking difference between both studies, however, is not the experimental paradigm but the acute stage of stroke lesions in our patients. While the patients in our study were recorded within one month after the stroke (median 11 days), patients in the latter study were examined at a chronic stage (32/80 months for left/right HH). Thus, the different findings in the patients of both studies may not be contradictory but reflect the different stages of stroke (acute versus chronic) and compensational efforts. We cannot exclude, that the incomplete visual field defects (quadrantanopia) in some of our patients contributed to the differences observed. However, there were also patients with incomplete visual field defects in the previous study although their simulated HH was always complete. In contrast, we matched HVFD patients and vHVFD subjects with the same visual field defects. The border areas of “relative” visual field defects in our patients were matched with absolute defects (black mask) in our virtual HVFD subjects. Thus, the visual input failure of our vHVFD subjects was at least the same or even slightly larger than in the patients. Therefore, visual field areas of reduced sensitivity in our patients could not have been responsible for their greater deficits in the visual search task as compared to vHVFD subjects. In contrast to the sHH subjects in the previous study, there were no differences in fixation distribution and saccadic amplitudes between blind and seeing direction in our virtuals. That might be attributed to our stimulus paradigm. In fact, more than the dot-counting paradigm in previous studies (Tant et al., 2002; Zihl, 1995), our complex stimulus required and provoked a rather strict geometric scanning pattern driven by strong top-down influences, which might have prevented higher frequencies of small-amplitude

B. Machner et al. / Neuropsychologia 47 (2009) 2704–2711

saccades towards the scotoma. Finally, based on our additional eye movement analyses and lesion-symptom mapping we provide some evidence that acute occipito-temporal brain damage does impair the scanning behaviour in HVFD patients more than it could be expected from the pure visual field defect. 4.4. Concluding remarks We conclude that HVFD patients in an acute stage of stroke show visual search disturbances, which the pure visual input failure does not fully account for. They cannot be clearly attributed to the size of cerebral lesions, to deficits of working memory across saccades, search strategies or the pop-out effect of colour. We speculate that the increased search duration and multiple re-fixations may be rather due to an impairment of object recognition and directed visual attention, probably caused by damage to the neural network connecting the primary visual cortex to associated occipital and temporal brain areas. However, further studies applying lesionsymptom-mapping in larger patient samples are needed to confirm this finding. The re-fixations in our patients might also reflect an initial strategy to compensate for the visual field loss, in terms of having “a second look” to various parts of the display. In a more chronic stage, when lesions last for 10 months or more (Pambakian et al., 2000; Zihl, 1995), these excessive fixations become more and more restricted to the blind hemifield, obviously contributing to a more efficient compensatory eye movement strategy. Funding University Lübeck grants (FUL 23/98 to AS, E32-2008 to BM). Appendix A. Supplementary data Supplementary data associated with this article can be found, in the online version, at doi:10.1016/j.neuropsychologia.2009.05.016. References Brandt, T., Thie, A., Caplan, L. R., & Hacke, W. (1995). Infarkte im Versorgungsgebiet der A. cerebri posterior. Nervenarzt, 66, 267–274. Gassel, M. M., & Williams, D. (1963). Visual function in patients with homonymous hemianopia II. Oculomotor mechanisms. Brain, 86, 1–36. Gilchrist, I. D., & Harvey, M. (2000). Refixation frequency and memory mechanisms in visual search. Current Biology, 10, 1209–1212. Goodale, M. A. (1993). Visual pathways supporting perception and action in the primate cerebral cortex. Current Opinion in Neurobiology, 3, 578–585. Hildebrandt, H., Giesselmann, H., & Sachsenheimer, W. (1999). Visual search and visual target detection in patients with infarctions of the left or right posterior or the right middle brain artery. Journal of Clinical and Experimental Neuropsychology, 21, 94–107. Hooge, I. T., & Frens, M. A. (2000). Inhibition of saccade return (ISR): spatio-temporal properties of saccade programming. Vision Research, 40, 3415–3426. Hooge, I. T., Over, E. A., van Wezel, R. J., & Frens, M. A. (2005). Inhibition of return is not a foraging facilitator in saccadic search and free viewing. Vision Research, 45, 1901–1908. Husain, M., Mannan, S., Hodgson, T., Wojciulik, E., Driver, J., & Kennard, C. (2001). Impaired spatial working memory across saccades contributes to abnormal search in parietal neglect. Brain, 124, 941–952. Ishiai, S., Furukawa, T., & Tsukagoshi, H. (1987). Eye-fixation patterns in homonymous hemianopia and unilateral spatial neglect. Neuropsychologia, 25, 675– 679. Klein, R. M. (1988). Inhibitory tagging system facilitates visual search. Nature, 334, 430–431.

2711

Klein, R. M., & MacInnes, W. J. (1999). Inhibition of return is a foraging facilitator in visual search. Psychological Science, 10, 346–352. Kolb, B., & Whishaw, I. Q. (1996). Fundamentals of human neuropsychology (4th ed.). New York and Oxford: W.H. Freeman and Company. Machner, B., Sprenger, A., Kömpf, D., & Heide, W. (2005). Cerebellar infarction affects visual search. Neuroreport, 16, 1507–1511. Mannan, S. K., Mort, D. J., Hodgson, T. L., Driver, J., Kennard, C., & Husain, M. (2005). Revisiting previously searched locations in visual neglect: role of right parietal and frontal lesions in misjudging old locations as new. Journal of Cognitive Neuroscience, 17, 340–354. McDonald, S. A., Spitsyna, G., Shillcock, R. C., Wise, R. J., & Leff, A. P. (2006). Patients with hemianopic alexia adopt an inefficient eye movement strategy when reading text. Brain, 129, 158–167. Meienberg, O., Zangemeister, W. H., Rosenberg, M., Hoyt, W. F., & Stark, L. (1981). Saccadic eye movement strategies in patients with homonymous hemianopia. Annals of Neurology, 9, 537–544. Mort, D. J., & Kennard, C. (2003). Visual search and its disorders. Current Opinion in Neurology, 16, 51–57. Mort, D. J., Malhotra, P., Mannan, S. K., Rorden, C., Pambakian, A., Kennard, C., et al. (2003). The anatomy of visual neglect. Brain, 126, 1986–1997. Owen, A. M., Downes, J. J., Sahakian, B. J., Polkey, C. E., & Robbins, T. W. (1990). Planning and spatial working memory following frontal lobe lesions in man. Neuropsychologia, 28, 1021–1034. Pambakian, A. L., Wooding, D. S., Patel, N., Morland, A. B., Kennard, C., & Mannan, S. K. (2000). Scanning the visual world: a study of patients with homonymous hemianopia. Journal of Neurology, Neurosurgery, and Psychiatry, 69, 751–759. Pomplun, M., Velichkovsky, B., & Ritter, H. (1994). An artificial neural network for high precision eye movement tracking. In B. Nebel, & L. Dreschler-Fischer (Eds.), Lecture notes in artificial intelligence: AI-94 proceedings (1st ed., pp. 63–69). Berlin: Springer. Posner, M. I., Rafal, R. D., Choate, L. S., & Vaughan, J. (1985). Inhibition of return: neural basis and function. Cognitive Neuropsychology, 2, 211–228. Ringo, J. L., Doty, R. W., Demeter, S., & Simard, P. Y. (1994). Time is of the essence: a conjecture that hemispheric specialization arises from interhemispheric conduction delay. Cerebral Cortex, 4, 331–343. Rizzo, M., & Robin, D. A. (1996). Bilateral effects of unilateral visual cortex lesions in human. Brain, 119(Pt 3), 951–963. Rorden, C., & Brett, M. (2000). Stereotaxic display of brain lesions. Behavioural Neurology, 12, 191–200. Rorden, C., Karnath, H. O., & Bonilha, L. (2007). Improving lesion-symptom mapping. Journal of Cognitive Neuroscience, 19, 1081–1088. Sprenger, A., Kömpf, D., & Heide, W. (2002). Visual search in patients with left visual hemineglect. Progress in Brain Research, 140, 395–416. Stoerig, P., & Cowey, A. (1997). Blindsight in man and monkey. Brain, 120, 535–559. Tant, M. L., Cornelissen, F. W., Kooijman, A. C., & Brouwer, W. H. (2002). Hemianopic visual field defects elicit hemianopic scanning. Vision Research, 42, 1339–1348. Treisman, A. M., & Gelade, G. (1980). A feature-integration theory of attention. Cognitive Psychology, 12, 97–136. Ungerleider, L. G., & Haxby, J. V. (1994). ‘What’ and ‘where’ in the human brain. Current Opinion in Neurobiology, 4, 157–165. Ungerleider, L. G., & Mishkin, M. (1982). Two cortical visual systems. In D. J. Ingle, M. A. Goodale, & R. J. W. Mansfield (Eds.), Analysis of visual behavior. Cambridge, MA: MIT Press. Weiskrantz, L., Warrington, E. K., Sanders, M. D., & Marshall, J. (1974). Visual capacity in the hemianopic field following a restricted occipital ablation. Brain, 97, 709–728. Wilson, B., Cockburn, J., & Halligan, P. (1987). Development of a behavioral test of visuospatial neglect. Archives of Physical Medicine and Rehabilitation, 68, 98–102. Wolfe, J. M., Cave, K. R., & Franzel, S. L. (1989). Guided search: an alternative to the feature integration model for visual search. Journal of Experimental Psychology: Human, 15, 419–433. Zangemeister, W. H., Meienberg, O., Stark, L., & Hoyt, W. F. (1982). Eye-head coordination in homonymous hemianopia. Journal of Neurology, 226, 243–254. Zangemeister, W. H., Oechsner, U., & Freksa, C. (1995). Short-term adaptation of eye movements in patients with visual hemifield defects indicates high level control of human scanpath. Optometry and Vision Science, 72, 467–477. Zangemeister, W. H., & Utz, P. (2002). An increase in a virtual hemianopic field defect enhances the efficiency of secondary adaptive gaze strategies. Current Psychology of Cognition, 21, 281–303. Zelinsky, G. J. (1996). Using eye saccades to assess the selectivity of search movements. Vision Research, 36, 2177–2187. Zihl, J. (1995). Visual scanning behavior in patients with homonymous hemianopia. Neuropsychologia, 33, 287–303.