Time-resolved fMRI of mental rotation revisited-dissociating visual perception from mental rotation in female subjects

www.elsevier.com/locate/ynimg NeuroImage 32 (2006) 432 – 444

Time-resolved fMRI of mental rotation revisited-dissociating visual perception from mental rotation in female subjects Christine Ecker,a,* Michael J. Brammer,a Anthony S. David,b and Steven C. Williams c a

Brain Image Analysis Unit, PO Box 89, Centre for Neuroimaging Sciences (CNS), Institute of Psychiatry, KCL., DeCrespigny Park, London, SE5 8AF, UK Section of Cognitive Neuropsychiatry, Department of Psychological Medicine, Institute of Psychiatry, London, UK c Neuroimaging Research Group, Department of Neurology, Institute of Psychiatry, London, UK b

Received 7 December 2005; revised 24 February 2006; accepted 8 March 2006 Available online 2 May 2006

Functional neuroimaging studies have demonstrated that mental rotation paradigms activate a network of spatially distributed cortical areas rather than a discrete brain region. Although the neuroanatomical nodes of the rotation network are well established, their specific functional role is less well identified. It was the aim of the present study to dissociate network components involved in the visual perception of 3D cubic objects from regions involved in their mental spatial transformation. This was achieved by desynchronizing the time course of the perceptional process (i.e., stimulus duration) from the duration of the cognitive process (i.e., reaction times) and by comparing these with the temporal characteristics of the hemodynamic response functions (HRFs) in regions of interest. To minimize intersubject variability, an all-female subject group was chosen for this investigation. Time-resolved fMRI analysis revealed a significant increase in the full width at half maximum (FWHM) of the HRF with reaction times in the supplementary motor area (pre-SMA), in the bilateral premotor cortex (PMd-proper), and in the left parietal lobe (PP). The FWHM in visual system components such as the bilateral lateral occipital complex (LOC) and dorsal extrastriate visual areas (DE) was constant across trials and roughly equal to the stimulus duration. These findings suggest that visual system activation during mental rotation reflects visual perception and can be dissociated from other network components whose response characteristics indicates an involvement in the mental spatial transformation itself. D 2006 Elsevier Inc. All rights reserved.

Functional neuroimaging studies have demonstrated that the cognitive processes underlying mental rotation do not have a discrete neural correlate but are represented as a distributed neural system. This so-called rotation network comprises parietal regions (Carpenter et al., 1999; Harris et al., 2000; Iwaki et al., 1999), several areas of the motor system (e.g., Ganis et al., 2000; Kosslyn et al., 1998; Parsons et al., 1995; Vingerhoets et al., 2002), as well

* Corresponding author. E-mail address: [email protected] (C. Ecker). Available online on ScienceDirect (www.sciencedirect.com). 1053-8119/$ - see front matter D 2006 Elsevier Inc. All rights reserved. doi:10.1016/j.neuroimage.2006.03.031

as visual system components. Although the overall involvement of these neuro-anatomical nodes in mental rotation is well established, their specific functional role in the complex cognitive process is less well identified. The current study investigates the involvement of the visual system in the mental rotation of Shepard and Metzler (1971) objects. A number of previous studies have reported significant activation in several regions of the visual system using a variety of rotation paradigms. These regions are not limited only to early visual areas in the occipital lobe (Alivisatos and Petrides, 1997; Parsons et al., 1995; Vingerhoets et al., 2002) but also include higher visual areas in the dorsal and ventral processing streams. Mental rotation of 3D cubic structures, for example, has repeatedly been found to elicit activation in dorsal extrastriate visual regions (DE) located on the border between the parietal and occipital lobe (BA39/19) (Kosslyn et al., 1998). These regions are part of the dorsal processing stream, which is involved in spatial perception as well as in the visual guidance of movements towards objects in space (Fwhere_ features) (reviewed by Ungerleider and Haxby, 1994). Extrastriate activation including BA19/39 has also been suggested to correspond to the motion sensitive area V5/human MT (Cohen et al., 1996) or a satellite area of V5/MT (Barnes et al., 2000). Furthermore, several studies have reported activation in the inferior temporal lobe (ITp) during rotation of images depicting body parts (Parsons et al., 1995), 3D objects (Vanrie et al., 2002), and alpha-numeric characters (Alivisatos and Petrides, 1997). The ITp belongs to the ventral processing stream, which plays a crucial role in detecting visual features relevant to object identification (Fwhat_ features). Interestingly, lesions in these perceptual processing streams also lead to specific deficits in the ability to form visual mental images (Levine et al., 1985). Overall, the occurrence of visual system activation during mental rotation has attracted less attention than the activation in the motor system or parietal lobes. Because mental rotation is primarily a visual task, visual system activation is naturally expected. Early visual regions in particular have therefore predominantly been linked to the perception of the presented

C. Ecker et al. / NeuroImage 32 (2006) 432 – 444

objects (Alivisatos and Petrides, 1997; Parsons et al., 1995; Vingerhoets et al., 2002). On the other hand, mental rotation also requires the mental spatial transformation of the perceived objects and, thus, involves imagery. Consequently, the involvement of several of the activated regions (e.g., higher visual regions) might reflect visual mental imagery indicating an active involvement in the rotational process. Although studies investigating the neuroanatomical correlates of general visual imagery and perception are numerous (Ganis et al., 2004; Kosslyn et al., 1997, 2001), only few publications have as yet attempted to dissociate functional activation related to visual perception from activation related to mental rotation (i.e., computation of visuo-spatial transformation) using fMRI (Barnes et al., 2000). Notably, a study by de Lange et al. (2005) has previously addressed this issue by comparing cortical functional activation during visual (VI) and motor imagery (MI) tasks using a parametric analytical approach. By examining increases in the amplitude of the BOLD response as a function of angular disparity, the authors could discriminate between stimulusrelated, mental rotation-related, and response-related neural activity. Interestingly, activation in the motion-sensitive V5/MT showed rotation-modulated increase in activation during both imagery tasks, thus supporting Fdepictive_ theories of visual mental imagery (Kosslyn et al., 1996). Conventional imagery tasks involve the generation of visual images from long- or short-term memory and therefore focus on the generation and maintenance of mental images. Mental rotation, however, is a special imagery task in that it not only requires the generation and maintenance of the presented objects but also their active manipulation. The present study aimed to specifically address the question whether the functional activation in visual system components elicited by general rotation paradigms is more closely associated with the perception of the presented objects than with their mental spatial transformation. The answer to this question is of particular relevance for depictive theories of mental rotation, which reduce mental rotation tasks to a pure imagery task (i.e., rotation of a visual mental image analog to the rotation of physical images). In order to dissociate visual perception from rotation, a novel event-related experimental design was used. This design intended to desynchronize the time course of the perceptional process from the time course of the rotational process. This was achieved by keeping the stimulus duration constant for a period of time longer than the time required for the rotation. There is evidence that males and females differ in performance on mental rotation tasks with males displaying faster RTs and higher accuracy (Linn and Petersen, 1985; Voyer et al., 1995). These behavioral differences seem to be accompanied by different cortical patterns of activation during visuo-spatial tasks. Interestingly, female subjects seem to recruit an additional visual region in the inferior temporal/occipital lobe during mental rotation, which are not present in men (Jordan et al., 2002). To minimize the intersubject variability, the subject group most suitable for the present investigation was chosen, which was the subject group displaying longer RTs and the larger extent of visual system activation. Therefore, this study was based on an all-female subject group. To dissociate the specific time course of the hemodynamic response functions (HRF) in different ROIs, data were analyzed in a time-resolved fashion (Formisano and Goebel, 2003; Menon et al., 1998). This type of analysis is based on correlating the time course of a mental process with the time course of the HRF in ROIs. Here, the onset or latency of the response is taken as a

433

marker for the onset of the mental process, whereas the full width at half maximum (FWHM) indicates the duration of the mental process. Time-resolved fMRI was first applied to mental rotation paradigms by Richter et al. (1997, 2000) who demonstrated a significant positive correlation between the cognitive load and the FWHM of the HRF in the parietal lobe, pre-motor cortex and supplementary motor area. The authors did not, however, examine HRFs in the visual system. On the basis of the experimental paradigm, HRFs with three different temporal characteristics were expected: (1) HRFs with constant FWHM, which are roughly equal to the stimulus duration, thus being closely related to visual perception (e.g., visual regions); (2) HRFs whose FWHMs increases with RTs, thus being involved in the mental spatial transformations itself(e.g., in parietal regions, supplementary- and pre-motor areas); and (3) HRFs with constant FWHM unrelated to the stimulus duration or the phase of rotation (e.g., primary motor cortex).

Materials and methods Female subject group Ten right-handed female volunteers between 20 and 30 years of age were randomly chosen from the general population. All participants were in good general health without a history of neurological or psychiatric disorders and exhibited normal eyesight. Written consent was provided by all subjects. On the day before the scanning session, each participant was given full instructions and was trained with a set of practice trials for 15 min. This allowed them to become familiar with the transformation of the stimuli, as well as the control of the button box. Subjects were repeatedly instructed to use mental imagery as cognitive strategy for the transformation of the objects. The study was approved by the South London and Maudsley NHS Trust Ethics Committee. Experimental paradigm Stimuli The objects used in the mental rotation task were Shepard and Metzler (1971)-like structures reproduced with the use of the software package N3D Design1. These structures included 10 different three-dimensional objects of which half were mirror images (isomers) of existing figures. Each object consisted of ten solid cubes attached face-to-face to form a rigid arm-like structure with exactly three right-angled Felbows_. Each structure was presented in shapes of grey in front of a dark background. During the task, the stimuli were presented in pairs and displayed by a computer-controlled projector system on a screen. The screen was at right angles to the bore of the magnet and 3.4 m from the subjects’ eyes and was viewed through a prismatic mirror. The size of both images was approximately 60  91 cm on the screen. The 3D objects in each pair were either the same (same pair) or mirror images (different pair). In the same pair presentation, the two objects could be rotated into congruence with each other. In the presentation of a different pair, the two objects differed by a reflection, as well as a rotation in either x or z dimension and could not be rotated into congruence. As in the paradigm used by Richter

1

See http://www.n3d.com.au/HTML/index.html.

434

C. Ecker et al. / NeuroImage 32 (2006) 432 – 444

et al. (1997), object pairs were presented in four different experimental conditions according to the angular disparity between the objects: (1) 0- angular disparity (2 identical objects at the same orientation), (2) 20- angular disparity, (3) 60- angular disparity, and (4) 100- angular disparity. Task A total of 80 trials (20 trials per condition) were presented in random order with respect to the identical/mirror image condition and the angular disparity (0-, 20-, 60-, 100-). Half of the trials per condition were the same pair, and the other half of the trials were a different pair. Since each of the ten cubic structures was presented from two different starting positions in 3D space and rotated around two different axes, the number of available pairs (n = 40) exceeded the number of trials per condition. Thus, no object pair had to be presented twice. Once the object pair appeared on the screen, subjects were asked to decide whether the objects were the same or mirror images. Subjects were instructed to respond as quickly as possible while keeping errors to a minimum. As soon as a decision was reached, the subjects indicated their choice by pressing one of the two buttons on a keypad. If subjects thought that the presented pair belongs to the same pair category, a response was made using the index finger. If subjects thought that the presented pair belonged to the different pair category, a response was made using the middle finger. The nature of the response (same or mirror image) and RTs were recorded. Desynchronization of stimulus duration and reaction times Event-related fMRI with constant stimulus duration (SD) and constant interstimulus interval (ISI) was used in this study. In all trial conditions, the 3D objects were presented for a period of 10 s, followed by a fixation cross (baseline). To keep the SD constant, the objects did not disappear from the screen after subjects indicated their choice by button press. Instead, objects were presented for an additional period of Dt with Dt = 10 s RT(s). Subjects were instructed to avoid repeated object rotation by focusing their eye gaze on one of the presented objects. This experimental design was chosen in order to desynchronize RT and SD by a time equal to Dt. The desynchronization Dt was maximal at short RTs, as expected from trials with a low or no degree of angular disparity. Dt was minimal at long RTs, as expected from trials with a high degree of angular disparity. If subjects did not respond within 10 s after the stimulus onset, the response was recorded as a missing value, and the object pair was replaced by the fixation cross. Individual trials were separated by an ISI of 5 – 6 s, after which the HRF should have decayed to the baseline level (Kwong et al., 1992). Individual trials could thus be fully separated. In half of the trials (n = 40), the trial onset was delayed relative to the TR onset by 1 s (jittered stimulus presentation) in order to improve the characterization of the HRF. At a constant SD of 10 s and a variable ISI of either 5 or 6 s, each individual trial took thus either 15 or 16 s. The total time required for the presentation of all 80 trials was 21.34 min. An illustration of the trial presentation is shown in Fig. 1. A resting period of 2 min was included at the beginning and end of the task performance. This allowed subjects to become familiar with the experimental setting. Data acquisition Whole brain gradient-echo planar MR images were acquired using a 1.5 T GE Signa Neuro-optimized System (General Electric,

Fig. 1. Display and timings of individual events. In all trials, Shepard and Metzler figures were presented for a period of 10 s during which reaction times were recorded. This way, the time course of the perceptional process could be dissociated from the time course of the rotational process by Dt = 10 RT. Individual trials were separated by a 5- to 6-s fixation cross.

Milwaukee, WI, USA) fitted with 40 mT/m high-speed gradients at the Maudsley Hospital, London. Foam padding and a forehead strap were used to limit head motion. Daily quality assurance was carried out to ensure high signal to ghost ratio, high signal to noise ratio and excellent temporal stability using an automated quality control procedure (Simmons et al., 1999). A quadrature birdcage head coil was used for radio-frequency transmission and reception. At the beginning of each session, an inversion recovery EPI dataset was acquired at 43 near-axial 3 mm thick planes parallel to the AC-PC line: TE = 73 ms, TI (inversion time) = 180 ms, TR = 16 s, in-plane resolution = 1.72 mm, interslice gap = 0.3 mm. This higher resolution EPI dataset provided whole brain coverage and was later used to register the fMRI datasets acquired from each individual subject in standard stereotaxic space. After the acquisition of the structural images, 760 T2*-weighted images depicting BOLD contrast (Kwong et al., 1992; Ogawa et al., 1990) were acquired over 25.34 min at each of 25 near-axial, noncontiguous 5 mm thick planes parallel to the intercommissural (AC-PC) line: TE = 40 ms, TR = 2000 ms, theta = 80-, in-plane resolution = 3.75 mm, interslice gap 0.5 mm. Volumes 1 to 60 (0 – 119 s) were acquired during the first resting period, volumes 61 to 700 (120 – 1399 s) were acquired during the mental rotation task, and volumes 701 to 760 (1400 – 1519 s) were acquired during the second resting period. Data analysis Brain activation mapping Individual brain activation maps (IBAM). The data were first realigned (Bullmore et al., 1999) to minimize motion related artifacts and smoothed using a Gaussian filter (FWHM = 7.2 mm). In order to constrain the possible range of physiologically plausible BOLD responses, the constrained fitting procedure suggested by Friman et al. (2003) was adopted (Friman et al., 2003). Responses to the experimental paradigms were then detected by time-series analysis using gamma variate functions (peak responses weighted between 4 and 8 s) convolved with the experimental design to model the blood oxygen level-dependent response. A goodness-offit statistic and a measure of the mean power of neural response (the sum of squares [SSQ] ratio) was computed at each voxel. This was the ratio of the sum of squares of deviations from the mean intensity value due to the model (fitted time series) divided by the sum of squares due to the residuals (original time series minus

C. Ecker et al. / NeuroImage 32 (2006) 432 – 444

model time series). To sample the distribution of SSQ ratio under the null hypothesis that observed values of SSQ ratio were not determined by experimental design (with minimal assumptions), the time series at each voxel was permuted by a wavelet-based resampling method (Breakspear et al., 2004; Bullmore et al., 2001). This process was repeated 10 times at each voxel to produce the distribution of SSQ ratios under the null hypothesis. Voxels activated at any desired level of type I error can then be determined by obtaining the appropriate critical value of SSQ ratio from the null distribution. Individual brain activation maps were produced for each participant for each experimental condition vs. the baseline condition. Group activation maps (GBAM). To extend inference to the group level, the observed and randomized SSQ ratio maps were transformed into standard space (Talairach and Tournoux, 1988) by a 2-stage process (Brammer et al., 1997) using spatial transformations computed for each subject’s high-resolution structural scan. Once the statistic maps were in standard space, a group brain activation map was produced for each experimental condition by testing the median observed SSQ ratio over all subjects at each voxel in standard space (median values were used to minimize outlier effects), against a critical value of the permutation distribution for median SSQ ratio ascertained from the spatially transformed wavelet-permuted data (Bullmore et al., 2004). For greater sensitivity and to reduce the multiple comparison problem encountered in fMRI, hypothesis testing was carried out at the cluster level using methods developed by Bullmore et al. (1999). This method estimates the probability of occurrence of clusters under the null hypothesis using the distribution of median SSQ ratios computed from spatially transformed data obtained from wavelet permutation of the time series at each voxel. Image-wise expectation of the number of false positive clusters under the null hypothesis is set for each analysis at <1. Consequently, correction for multiple comparisons was implicit in the analysis, as thresholds were set on an image-wide, not a voxel-wise basis. Time-resolved fMRI analysis For time-resolved fMRI analysis, the movement-corrected fMRI time series were extracted from the main regions of interest (ROIs). ROIs were all regions of significant functional activation, which (1) appeared in any of the GBAMs, (2) were activated in at least 50% of the IBAMs, and (3) conformed with regions identified by previous publications. After selecting ROIs on the basis of the GBAMs, the time series were extracted from the individual IBAMs for each ROI at the Talairach coordinates indicated by the GBAMs. Each regional time series was then calculated as the average time series across all voxels in the selected clusters and contained 760 signal intensity values sampled at a frequency of 2 s. 80 impulse response functions consisting of signal intensities acquired during the first 16 s after the trial onset were then extracted from the time series across the whole experimental run. In order to determine a signal intensity value for the jittered events (stimulus onset asynchrony = 1 s), the time series underwent linear interpolation to the range of 1 s. A single response function for each of the four task conditions and for individual subjects was calculated by averaging 20 individual impulse responses per condition. For display purpose only, hemodynamic responses for each condition and ROI were also extracted on the basis of the 1st eigentimeseries

435

across all subjects (Buchel et al., 1999; Bullmore et al., 2000; Fletcher et al., 1999). The amplitude and FWHM of each HRF were then identified. This was done using an automated procedure taking into account the characteristic shape of the function (e.g., Kwong et al., 1992). The total time course of the response was subdivided into an onset phase (t = [0,. . .,3 s]) and a response phase (t = [4,. . .,16 s]), which included the response plateau and decay. Responses not displaying these characteristics were excluded from further analysis. The amplitude of the response was calculated by subtracting the minimal intensity value during the onset phase from the maximal intensity value during the response phase. In order to determine the FWHM in the range of 0.1 s, the average hemodynamic response was interpolated using cubic spline interpolation prior to the calculation of the parameters. Cubic spline interpolation and parameter calculation were performed with a customized script written in MATLAB R12. Finally, bivariate correlation analysis on a significance level <5% was carried out in order to identify ROIs whose HRFs increased in amplitude and FWHM as a function of reaction time measures recorded during each scanning session. By measuring the temporal characteristics of the cognitive process and the HRFs simultaneously, a direct relationship between the two measurers could be assessed. Since response parameters were calculated on the basis of the average HRF for each condition, only four data points were obtained of individual subjects. Consequently, regression analyses were carried out across subjects.

Results Behavioral results The mean RTs recorded during the scanning sessions and errors of response with the corresponding standard deviations are displayed as a function of angular disparity across all 10 subjects in Fig. 2. As expected, RTs and error rates increased monotonically with the angular disparity between the presented objects from 0- to 100-. The performed univariate one-way ANOVA revealed a significant main effect for angular disparity on RTs ( F = 68.99, df = 3, P < 0.001). As angular disparity increased, RT increased from 1.676 T 0.379 s at 0- disparity to 4.685 T 0.995 s at 100disparity. A trend analysis was also performed to test for linear as

Fig. 2. Mean reaction times (RTs) recorded during the scanning sessions and errors of response across all 10 female subjects. There was a significant linear increase in RT with the degree of angular disparity.

436

C. Ecker et al. / NeuroImage 32 (2006) 432 – 444

well as higher order polynomial trends between angular disparity and RT. Exclusively the linear trend for angular disparity reached statistical significance ( F = 108.924, df = 1, P < 0.001). No significant higher-order effects were found ( F quadratic = 0.674, df = 1, P < 0.5; F cubic = 0.3208, df = 1, P < 0.1). On average, subjects required approximately 30 ms for a rotation of 1-. Foci of group brain activation Consistent with previous publications, significant functional activation during the performance of the mental rotation was observed in predominantly three cortical systems: (1) the visual system, (2) the motor system, and (3) parietal regions. The group map of the 10 female subjects contrasting the 100- rotation conditions with the baseline is displayed in Fig. 3(A). In addition, the group map contrasting 60- with 100- of mental rotation is provided in Fig. 3(B). Group maps displaying differences in effect sizes between 20-, 60-, and 100- rotation with 0- rotation can be found in the Supplementary material. The coordinates of the main foci of activation in standard stereotaxic space as well as their statistical indices can be found in Table 1. A summary of the

individual contrast maps displaying significant activation in specific ROIs is given in Table 2. Visual system activation Significant visual system activation was observed in early visual areas, in the medial – occipital/inferior – temporal cortex (BA19/37), and in dorsal extrastriate visual areas (DE). Most of the early visual activation was observed in the calcarine fissure, its branches or accessory sulci (medial part of BA17/BA18). Voxels in the calcarine were predominantly activated during the 60- and 100- rotation and significantly increased in rCBF from 0- to 100- rotation (SSQ ratio = 0.0047, P = 5  10 4). Secondly, activation was observed in a cluster located on the medial-occipito/inferior – temporal junction (GOm/ITp). In the right hemisphere, the peak activation corresponded to BA37 located in the fusiform gyrus (GF), while the anterior part of the cluster extended into the inferior temporal gyrus (ITp). In the left hemisphere, the center of mass was located in the middle occipital gyrus (GOm) corresponding to BA19. As in the right hemisphere, the anterior part of this cluster extended into the ITp. In the visual literature, this region is also known as the lateral occipital complex (LOC), which begins in lateral occipital cortex

Fig. 3. Group activation map of ten right-handed female subjects during the performance of the mental rotation task for the contrast (A) 100- > baseline, and (B) 100- > 60-. The numbers above the transverse sections indicate the distance in mm from the transcallosal line. The right side of the brain is shown on the left side of each panel and vice versa. All contrasts were tested at a cluster-wise type 1 error probability of P < 0.01.

C. Ecker et al. / NeuroImage 32 (2006) 432 – 444

437

Table 1 Main foci of group brain activation for 10 female subjects during performance of the mental rotation task Contrast

Region definition

N

Hemisphere

Tal (x, y, z)

SSQ ratio

P

Effect size (%) 6

0- > Baseline 100- > 20100- > Baseline 100- > Baseline 100- > Baseline 100- > Baseline 60- > 060- > 0-

M1 PMd-proper PMd-proper Pre-SMA NC NC PMv PMv

9/10 7/10 9/10 7/10 5/10 5/10 5/10 9/10

L R L M R L R L

36, 22, 48 32, 11, 48 22, 11, 48 4, 4, 48 25, 19, 2 18, 19, 2 51, 7, 26 43, 11, 26

0.025 0.018 0.025 0.048 0.015 0.017 0.004 0.004

3  10 3  10 6 3  10 6 3  10 6 2.6  10 3  10 6 1.3  10 2.3  10

100100100100100-

Baseline Baseline Baseline Baseline Baseline

Early visual LOC LOC DE DE

8/10 5/10 8/10 5/10 8/10

M R L L R

36, 22, 48 43, 59, 13 40, 70, 7 22, 70, 20 29, 63, 20

0.018 0.037 0.015 0.036 0.019

3 3 2 3 3

100- > Baseline 100- > Baseline

Parietal ROI Parietal ROI

7/10 8/10

L R

22, 63, 31 29, 56, 31

0.015 0.012

2.6  10 1.4  10

> > > > >

    

10 10 10 10 10

5

3 3

–* 0.192 0.218 0.256 0.173 0.191 – –

6

0.247 0.273 0.296 0.279 0.252

6 5 6 6

5 4

0.197 0.2188

Contrast indicates the contrast map with the largest fit indices for a particular ROI. Note. L: left, R: right, N: number of subjects in whom each region was identified, M: medial, M1: primary motor cortex, PMd: pre-motor cortex, SMA: supplementary motor area, NC: caudate nucleus, LOC: lateral occipital complex, DE: dorsal extrastriate visual areas, PMv: ventral premotor areas. *Effect sizes could only be identified for the baseline contrasts.

and extends anteriorily and ventrally into inferior posterior temporal regions. Activation in the right LOC was found in all conditions requiring mental rotation (20-, 60-, 100-), whereas activation in the left LOC was observed exclusively during the 60- and 100- rotation. As in early visual regions, rCBF in the bilateral LOC increased significantly from 0- to 100- rotation. Furthermore, we found significant activation in dorsal extrastriate visual areas, which extended from the striate cortex to the intraparietal sulcus. As indicated in the group activation map, the DE activation occurred across several slices whose z coordinates ranged from z = 4 to 30 in standard stereotaxic space. The center of mass was located at the Talairach coordinates 22, 70, 20 in the left and 29, 63, 20 in the right hemisphere. In the visual literature, this location is also known as the occipital – parietal junction (OPJ),

which encompasses part of the superior occipital gyrus including the junction point of the intra-parietal and the transverse occipital sulci. 80% of all subjects displayed activation in the left DE, whereas only 50% of all subjects displayed activation in the right DE. Motor system activation Significant functional activation was observed in several components of the motor system. Firstly, there was significant activation in the primary motor cortex (M1 or BA4) located in the precentral gyrus of the left hemisphere, which was the contralateral hemisphere relative to the hand with which subjects performed the button press. Secondly, 70% of all subjects displayed significant activation in the medial part of BA6, which is also known as supplementary motor area (SMA). More

Table 2 Experimental contrasts displaying significant functional activation in ROIs Region definition

M1 PMd-proper Pre-SMA NC PMv Early Visual LOC DE Parietal ROI

Hemisphere

L L R M L R L R M L R L R L R

Contrast 0- > Baseline

20->Baseline

60- > Baseline

100- > Baseline

x

x x

x x

x x

20- > 0-

60- > 0-

100- > 0-

60- > 20-

x

x

x x x x x x

x

x x x

x x

x x x x x x x

x x x x x x x

x x

x x

x x x x x x x x x

x x x x x x x

x

x x x x

100- > 20-

100- > 60-

x x x x

x x x x x x

x x x x x x x

x x x x x x x

Note. L: left, R: right, M: medial, M1: primary motor cortex, PMd: pre-motor cortex, SMA: supplementary motor area, NC: caudate nucleus, LOC: lateral occipital complex, DE: dorsal extrastriate visual areas, PMv: ventral premotor areas, x: significant functional activation during contrast.

438

C. Ecker et al. / NeuroImage 32 (2006) 432 – 444

specifically, this focus of activation was located in the rostral part of the medial BA6 (i.e., rostral from the VCA line) and has also been referred to as pre-SMA. Activation in the pre-SMA was most prominent in GBAMs (group activation maps) contrasting 100rotation with the baseline. Thirdly, activation was found in ventral and dorsal premotor cortex (PM) located in the lateral part of BA6. The activation in the dorsal premotor area (PMd-proper) corresponded to the caudal subdivision, which is located posterior to the VCA line. The PMd-proper was observed in all contrasts requiring mental rotation. 70% of all subjects displayed significant activation in the right PMd-proper, and 90% of all subjects displayed activation in the left PMd-proper. The activation in the ventral premotor area (PMv) was located in the inferior frontal gyrus on the border between BA44 and BA6. In addition, we observed significant activation in the caudate nucleus (NC) of the basal ganglia. 90% of all subjects displayed activation in the left PMv, and only 50% of the subjects in the right PMv. A summary of all contrasts revealing activation in the described motor regions can be found in Table 2. Parietal activation Finally, the group activation map revealed significant bilateral activation in several subregions of the parietal lobe. These included the angular gyrus (Ga) corresponding to the most superior part of BA39, the inferior parietal lobe (LPi) corresponding to BA40, and parts of the superior parietal lobe (LPs). More specifically, the activation in the LPs was located in the intra-parietal sulcus corresponding to BA7. Parietal regions were significantly activated in all conditions requiring mental rotation. Although the parietal activation extended into different subregions of the parietal lobe, which might exhibit a differential functional involvement, all of these regions were considered a single ROI in further analysis. Significant functional activation was observed in 80% of all subjects in the right parietal ROI and in 70% of all subjects in the left parietal ROI. Results of the time-resolved fMRI analysis The results of the time-resolved fMRI analysis for all ROIs are summarized in Table 3. On the basis of the FWHM, HRFs could be

subdivided into one of the following categories: (1) responses whose FWHM were constant across conditions and unrelated to the stimulus duration, (2) responses whose FWHM were constant and roughly equal to the stimulus duration, and (3) responses whose FWHM increased with RTs. In all visual ROIs, a significant increase in FWHM with RTs was absent. Instead, HRFs were constantly elevated for a time roughly corresponding to the stimulus duration (10 s). Due to a non-characteristic shape, two HRFs were excluded from the analysis of the primary visual cortex (6.4%). A characteristic HRF was observed in 96% of all responses in the right DE and in 84.5% in the left DE. The average FWHM across conditions was 8.2 T 1.8 s (SD) in the left LOC, 8.9 T 2.1 s (SD) in the left DE and 8.8 T 1.4 s (SD) in the right DE. An exception was the primary visual cortex whose HRF was elevated for 6.12 T 2.87 s (SD). In addition, there was no significant correlation between the response amplitudes and RTs in any of the visual ROIs. The HRF in the right LOC did not exhibit a characteristic shape and was excluded from further analysis. Fig. 4 shows a typical HRF for the higher visual regions as well as the HRF for the early visual cortex. These results show that the time course of activation in visual regions is not directly related to the duration of the cognitive process (i.e., mental spatial transformation). Instead, the average width of the HRF directly corresponded to the stimulus duration, thus suggesting that visual system activation reflects visual perception. The FWHM increased significantly with RTs in several regions of the motor system as well as in the parietal ROI. A characteristic HRF was observed in 100% of all responses in the primary motor cortex and in the pre-SMA and in 90% of the HRFs in the left PMd-proper. Two HRFs were excluded from time-resolved analysis in the right PMd-proper (5.6%). The strongest increase was observed in the pre-SMA where the FWHM increased at the rate of 1.1 per 1 s RT (offset = 2.5 s) (see Fig. 5(A)). In the left PMd-proper, the FWHM increased significantly at the rate of 0.96 per 1 s RT (offset = 2.9 s), and at a rate of 1.06 per 1 s RT (offset = 2.98 s) in the right PMd-proper. Furthermore, there was a significant increase in FWHM and amplitude in the left PMv (90% of all HRFs included). Due to the small number of subjects displaying significant activation in the right PMv, a characteristic

Table 3 Correlations between subject’s reaction time measures and parameters of the hemodynamic response in regions of group brain activation during performance of the mental rotation task Region definition

Hemisphere

No of responses

M1 PMd-proper

L R L M R L R L M L R L R L R

36 26 36 28 14 16 – 29 30 35 – 27 18 27 29

Amplitudes Corr. with RTs

Pre-SMA NC PMv Early visual LOC DE Parietal ROI

*Significant correlations (P < 0.05).

0.129 0.378 0.365 0.610* 0.201 0.392 – 0.498* 0.221 0.22 – 0.217 0.117 0.526* 0.301

FWHM P <0.5 <0.1 <0.1 <0.001 <0.5 <0.2 – <0.01 <0.3 <0.3 – <0.3 <0.7 <0.01 <0.2

Corr. with RTs 0.226 0.559* 0.487* 0.704* 0.067 0.332 – 0.673* 0.04 0.139 – 0.03 0.125 0.399* 0.196

P <0.2 <0.01 <0.05 <0.001 <0.8 <0.1 – <0.001 <0.9 <0.5 – <0.9 <0.7 0.05 <0.5

C. Ecker et al. / NeuroImage 32 (2006) 432 – 444

439

Fig. 4. Response characteristics of the average hemodynamic responses (HRF) extracted from the 1st eigentimeseries in visual areas for each rotation condition (0-, 20-, 60-, 100-). Neither amplitude nor Full Width at Half Maximum (FWHM) of the HRF increased with reaction times. (A) Typical HRF in higher visual areas and (B) HRF in the primary visual cortex.

response could not be derived. The FWHM in the M1 or NC did not significantly increase with RTs. The HRF in M1 was elevated for 4.74 T 1.19 s in all conditions and displayed an increase in time to peak of approximately 1 s from 0- to 100- rotation (Fig. 5(B)). This increase in time to peak is consistent with the idea that M1 activation is related to the button press indicating the end of the rotational process. A significant increase in signal amplitude was observed in the pre-SMA exclusively. Finally, we found a significant increase in amplitude and FWHM of the hemodynamic response in the left but not in the right parietal ROI (Fig. 6). Due to a non-characteristic shape, one HRF was excluded from the analysis in the left parietal ROI (2.8%). 90.7% of all responses in the right parietal ROI displayed a characteristic response. In the left parietal lobe, the FWHM increased at a rate of 0.8 s per 1 s RT. Similar results were reported previously by Richter et al. (2000). The authors found an increase in FWHM at a rate of 0.9 s per 1 s RT in the pre-SMA, 0.96 s in the right PMd-proper, 1.06 s in the left PMd-proper and 0.65 s in the left parietal lobe.

These findings suggest that there is a 1:1 relationship between stimulus duration and the FWHM of the HRF, not only in response to sensory stimulation but also in relation to cognitive processing. In addition, the data support the hypothesis that the pre-SMA and PMd-proper as well as the left parietal lobe are likely to participate in the mental rotation itself (i.e., computation of the spatial transformation) rather than being associated with the perception of the presented objects or with the button press indicating the end of the cognitive process.

Discussion Using a novel event-related experimental design, this study enabled us to dissociate the time course for visual perception from the time course for the mental spatial transformation in female subjects. By comparing these with the temporal characteristics of the HRF in different components of the rotation network, we could dissociate regions involved in visual perception from regions more

440

C. Ecker et al. / NeuroImage 32 (2006) 432 – 444

Fig. 5. Response characteristics of the average hemodynamic responses (HRF) extracted from the 1st eigentimeseries in motor regions for each rotation condition (0-, 20-, 60-, 100-). There was a significant increase in FWHM of the HR in the (A) pre-SMA, the bilateral PMd, and in the left parietal ROI. (B) HRF in the primary motor cortex. Here, the FWHM was constant across conditions.

closely associated with the cognitive process underlying mental rotation. By means of time-resolved fMRI analysis, it was shown that in visual ROIs (i.e., bilateral LOC and DE), the FWHM of the HRF did not increase significantly with subject’s reaction time measures but was constant across conditions for a time roughly equal to the stimulus duration (mean FWHM ¨9 T 1 s, stimulus duration = 10 s). In contrast, as shown previously by Richter et al. (1997), the FWHM of the HRF in motor areas of the frontal lobe as well as in the left parietal lobe increased significantly with RT measures. These results suggest that visual system activation can be dissociated from other network components, whose time course of activation resembles the time course of the cognitive process rather than the stimulation paradigm. The presented data therefore provided strong evidence that visual regions do not actively participate in the mental spatial transformation itself. This has strong implications regarding the cognitive strategy used for the rotation. The results are not consistent with depictive theories of visual mental rotation, proposing that subjects use a visuo-spatial holistic strategy in

which objects are pictured in mind and then rotated continuously (Kosslyn et al., 1996). Rather, the results of the time-resolved fMRI analysis on regions in the motor system seem to support an egocentric strategy, where people imagine that they physically turn the objects thus involving motor processes (Kosslyn et al., 2001). This conclusion is supported by the finding that we did not observe significant activation in visual area V5/MT, which has been found to activate in response to real and apparent motion perception (Merchant et al., 2003). If mental images were indeed rotated in a holistic or Fgestalt_ fashion, the illusionary motion accompanying this rotation would be expected to activate V5/MT. Activation of V5/MT was, however, absent. Interestingly, the activation observed in the ventral processing stream was not restricted to inferior temporal regions (i.e., BA37), as was reported in previous publications. Instead, the activation was located at the junction between occipito/inferior – temporal cortex. This spread of activation is typical for the lateral occipital complex (LOC), which starts in lateral occipital regions and extends anteriorily and ventrally into inferior posterior temporal cortex. Previous neuroimaging

C. Ecker et al. / NeuroImage 32 (2006) 432 – 444

441

Fig. 6. Response characteristics of the average hemodynamic responses (HRF) extracted from the 1st eigentimeseries in the left and right parietal ROI for each rotation condition (0-, 20-, 60-, 100-). There was a significant increase in FWHM of the HR in the left parietal ROI (A), but not in the right parietal lobe (B).

studies have shown that the LOC is primarily involved in processing 3D structures (Gilaie-Dotan et al., 2002; Kanwisher et al., 1996; Welchman et al., 2005), which is in agreement with the visual stimuli used in the present study. Although one might feel tempted to relate the observed visual system activation exclusively to visual perception, the results do not exclude the possibility that visual areas are a necessary and integral part of the mental spatial transformation. In this study, the stimulus duration was held constant for a time longer than RTs. Theoretically, the HRF in visual regions could therefore be subdivided into two phases: an initial phase (t = 0, 1, . . . , RT seconds) and a later phase (t = RT, RT + 1, . . . , ¨10 s). One could argue that visual regions could potentially subserve both perception as well as mental rotation during the initial phase, whereas in the later phase only visual perception remains. This model either has to assume that individual neuronal populations can perform qualitatively different tasks at the same time – which is biologically highly implausible – or that proximate subsets of neurons are involved in different aspects of the task. In the latter case, one would expect a significant increase in amplitude with the level of task difficulty (i.e., longer RTs). Such an increase was, however,

absent in the investigated visual ROIs. It is therefore unlikely that visual areas perform the computation of the spatial transformation per se. The ultimate evidence that visual activation is exclusively related to perception could only be provided if the stimulus duration is held constant for a time shorter than RTs and if the HRF in visual regions decays earlier than in other components of the rotation network. This experimental setup would, however, depart from conventional mental rotation tasks and would require subjects to encode as well as to retrieve the presented objects from memory. An alternative model which highlights the importance of visual regions for mental rotation without assuming that the rotational process is localized in the visual system is the idea of a visual buffer. According to this model, a mental representation of the presented object is generated and continuously maintained in a visual buffer, where it is inspected during the actual image rotation (Farah, 1995; Kosslyn et al., 1995). The computation of the spatial transformation, however, would have to occur in a region outside the visual system. This model is in agreement with the data presented in so far as it does not suppose that visual regions perform a qualitatively different task during mental rotation as they perform during visual perception. In fact, there is now strong

442

C. Ecker et al. / NeuroImage 32 (2006) 432 – 444

evidence that visual imagery and perception draw on most (i.e., two-thirds) of the same neural machinery (Ganis et al., 2004; Kosslyn et al., 1997). This not only applies to higher order visual representations but also to low-level anatomical processes. For instance, disrupting neural activity in the primary visual cortex by transcranial magnetic stimulation (TMS) impairs the performance in both perception as well as imagery tasks (Kosslyn et al., 1999). Furthermore, it has recently been shown that even the imagination of visual objects induces retinotopically organized activation maps in early visual areas. Notably, these maps were identical to the maps elicited by the perception of the same objects (Klein et al., 2004; Slotnick et al., 2005). The role of early visual areas in mental rotation remains, as yet, unclear. In this study, a constant FWHM corresponding to the stimulus duration was observed in higher visual areas exclusively. In the striate cortex, however, the HR was constantly elevated for 6.12 T 2.87 s following the trial onset. This response time neither corresponded to the time course of perception nor to the time required for the spatial transformation. Among others, activation of early visual areas has been reported following saccadic eye movements, which might accompany the rotational process (Kimmig et al., 2001). One of the other possible explanations for the involvement of early visual areas could be the generation but not the maintenance of a mental object representation. Temporary activation in the calcarine fissure could therefore be evoked by the initial onset of the visual stimulus but subsides as soon as a mental representation is generated and maintained in higher visual areas. This hypothesis is consistent with previous publications, suggesting that sensory representations of objects are mediated by bottom – up mechanisms arising in early visual areas and maintained through a network of occipito-temporal parietal and frontal regions (Ishai et al., 2000, 2002; Mechelli et al., 2004). Interestingly, a significant increase in visual system activation was observed in some of the contrast maps, thus indicating hemodynamic changes in visual regions across different levels of task difficulty (e.g., Fig. 3(B)). This highlights a common methodological dilemma, which is the comparison between ROI analyses and mass-univariate approaches (i.e., GLM). Because GLM-type analyses are generally based on repeated measurement designs, variance due to intertrial variability can be considered. This makes such analyses much more powerful than ROI analyses in a statistical sense, which might explain the observed discrepancies. The reason why an ROI analysis was chosen over a massunivariate analysis was that the unsupervised voxel-by-voxel analysis of the HRFs is entirely dependent on the employed fitting function. However, none of the conventionally used fitting functions (e.g., gamma variate fitting function) provided an adequate fit to the observed HRFs. In addition, an automated procedure would always provide fit indices regardless of the existence of a characteristic response, thus resulting in meaningless response parameters. The underlying assumption of this study was that the mental spatial transformation does not take place in components of the visual system but in brain regions whose FWHM of the HRF increases significantly with RTs. The most prominent increase in FWHM with RTs was observed in pre-frontal motor regions (i.e., pre-SMA, bilateral PMd-proper, and PMv), thus providing further evidence for the hypothesis that the transformation of mental images could, at least partially, be guided by motor processes. Several previous publications have highlighted the contribution of the motor-system to mental rotation (Kosslyn et al., 2001; Parsons

et al., 1995; Vingerhoets et al., 2002; Wexler et al., 1998). Most of these investigations suggest that motor system activation is strategy-dependent and reflects the involvement of motor imagery (i.e., subjects rotate mental images in the same way as they physically would, thus using appropriate hand movements). Although the neural networks for overt and covert movement are partially overlapping, several motor regions seem to be engaged stronger during motor imagination, including bilateral PMd-proper, SMA, parietal areas, and the caudate nuclei (Gerardin et al., 2000). Significant activation in all of these regions was also observed in the present study. The strongest correlation between RTs and the FWHM of the HRF was observed in the pre-SMA. Although the pre-SMA is often regarded as a motor region, its functional specialization and anatomical connectivity actually resembles more a prefrontal area (Johansen-Berg et al., 2004; Picard and Strick, 2001). Unlike the SMA-proper (i.e., caudal portion of medial BA6), the pre-SMA does not project to the primary motor cortex nor to the spinal cord. Instead, it is highly interconnected with areas of the prefrontal cortex and with the reticular formations, thus suggesting a strong involvement of cognitive rather than motor processes. The specific role of the pre-SMA in mental rotation is, as yet, unclear. In general, the pre-SMA seems to be involved in establishing or retrieving visuo-motor associations such as visually cued changes between motor sequences (Sakai et al., 1999) and in updating or changing motor plans (Shima et al., 1996). Interestingly, the activation in the pre-SMA does not seem to be due to response preparation but appears to be detached from motor aspects of the task. For instance, Petit et al. (1998) demonstrated a sustained activity in the pre-SMA during a delay working memory task and concluded that pre-SMA activation does not reflect simple motor preparation, but rather a state of preparedness for selecting a motor response based on the information held on-line (Petit et al., 1998). Such a state of preparedness might also exist during mental rotation and its duration would correspond to RTs. Consequently, this could explain the strong correlation between RTs and FWHM in the pre-SMA. The functional role of the PMd-proper in mental rotation is as yet unclear. Traditionally, the pre-motor cortex is regarded as a Ftrue_ motor area, which is predominantly involved in the preparation and the generation of a motor response (Wise, 1985). More recent studies, however, suggest that the PMd contributes to the mental spatial transformation itself rather than being linked to the button press. Firstly, the activation of the PMd seems to be stimulus specific, thus supporting the motor imagery hypothesis. Vingerhoets et al. (2002) for instance demonstrated a significant increase in PMd activation during mental rotation of images depicting body parts as compared to images depicting tools. Secondly, Windischberger et al. (2003) used a Fuzzy Clustering Algorithm to demonstrate that while activation in M1 was related to the button press, activation in premotor areas was mainly linked to the spatial transformations. However, a recent study by Seurinck et al. (2005) suggests that the dorsolateral premotor cortex contributes to the mental rotation itself. Activation in the premotor cortex was observed in both a fixed- and selfpaced design, thus indicating that it is related to the transformation phase. There has also been some debate whether motor cortical activation during mental rotation tasks could reflect eye movements during task performance, which might also explain the relationship between RTs and the FWHM in motor regions. This

C. Ecker et al. / NeuroImage 32 (2006) 432 – 444

discussion is of particular interest for complex 3D stimuli such as the Shepard and Metzler objects, which require extensive recognition and comparison of objects and object parts (Carpenter et al., 1999). The hypothesis that motor system activation during MR is induced by eye movements has been addressed by Carpenter et al. (1999). The authors compared cortical activations during the mental rotation of Shepard and Metzler objects with a control condition requiring a systematic visual comparison of two 2D grids. Although the grid scanning condition required significantly more saccadic planning and execution than the rotation condition, the number of activated voxels in all ROIs (i.e., parietal lobes, inferior temporal regions and frontal motor regions) was higher in the rotation condition. These results suggest that motor regions are not merely involvement in processing eye movements but play an active role in the computation of the task (see also Vanrie et al., 2002). The specific contribution of eye movements to the results of the time-resolved analysis, however, needs to be identified by future research. Apart from the frontal pre-motor cortices, a significant increase in FWHM with RTs was also observed in the left but not right parietal ROI. The general evidence for a hemisphere-specific involvement of the parietal lobe in MR is inconclusive. Some previous publications have found that both hemispheres are more or less equally involved (Cohen et al., 1996; Jordan et al., 2001; Kosslyn et al., 1998; Tagaris et al., 1996), whereas other studies report unilateral activation of either the left or right parietal lobe (Alivisatos and Petrides, 1997; Harris et al., 2000). It has been argued that there is a shift from a right hemisphere dominance to a left hemisphere advantage with increasing practice (Voyer, 1995). Since activation in the right hemisphere was observed during the most difficult conditions requiring more practice, the reported results do not support this hypothesis. Furthermore, Corballis (1997) demonstrated that the left hemisphere is more strongly involved in MR of more complex stimuli (e.g., human faces). However, the perceived degree of complexity of the 3D cubic structures as compared to the complexity of human faces still needs to be determined. Finally, it is important to mention that this investigation was based on an all-female subject group. As was outlined in the introduction, there are gender-specific differences in performance on mental rotation tasks and in the associated patterns of cortical activation. The interpretation of the results should therefore be restricted to female subjects exclusively. Future research is, however, needed to examine the dissociation of perceptual processes and mental spatial transformations in a male sample.

Acknowledgments We gratefully acknowledge the support of the Neuroimaging Research Group, Department of Neurology, Institute of Psychiatry, London. We would also like to thank all participants who volunteered for this study and Chris Andrew for programming the paradigm.

Appendix A. Supplementary data Supplementary data associated with this article can be found, in the online version, at doi:10.1016/j.neuroimage.2006.03.031.

443

References Alivisatos, B., Petrides, M., 1997. Functional activation of the human brain during mental rotation. Neuropsychologia 35, 111 – 118. Barnes, J., Howard, R.J., Senior, C., Brammer, M., Bullmore, E.T., Simmons, A., Woodruff, P., David, A.S., 2000. Cortical activity during rotational and linear transformations. Neuropsychologia 38, 1148 – 1156. Brammer, M.J., Bullmore, E.T., Simmons, A., Williams, S.C., Grasby, P.M., Howard, R.J., Woodruff, P.W., Rabe-Hesketh, S., 1997. Generic brain activation mapping in functional magnetic resonance imaging: a nonparametric approach. Magn. Reson. Imaging 15, 763 – 770. Breakspear, M., Brammer, M.J., Bullmore, E.T., Das, P., Williams, L.M., 2004. Spatiotemporal wavelet resampling for functional neuroimaging data. Hum. Brain Mapp. 23, 1 – 25. Buchel, C., Coull, J.T., Friston, K.J., 1999. The predictive value of changes in effective connectivity for human learning. Science 283, 1538 – 1541. Bullmore, E.T., Suckling, J., Overmeyer, S., Rabe-Hesketh, S., Taylor, E., Brammer, M.J., 1999. Global, voxel, and cluster tests, by theory and permutation, for a difference between two groups of structural MR images of the brain. IEEE Trans. Med. Imaging 18, 32 – 42. Bullmore, E., Horwitz, B., Honey, G., Brammer, M., Williams, S., Sharma, T., 2000. How good is good enough in path analysis of fMRI data? NeuroImage 11, 289 – 301. Bullmore, E., Long, C., Suckling, J., Fadili, J., Calvert, G., Zelaya, F., Carpenter, T.A., Brammer, M., 2001. Colored noise and computational inference in neurophysiological (fMRI) time series analysis: resampling methods in time and wavelet domains. Hum. Brain Mapp. 12, 61 – 78. Bullmore, E., Fadili, J., Maxim, V., Sendur, L., Whitcher, B., Suckling, J., Brammer, M., Breakspear, M., 2004. Wavelets and functional magnetic resonance imaging of the human brain. NeuroImage 23 (Suppl. 1), S234 – S249. Carpenter, P.A., Just, M.A., Keller, T.A., Eddy, W., Thulborn, K., 1999. Graded functional activation in the visuospatial system with the amount of task demand. J. Cogn. Neurosci. 11, 9 – 24. Cohen, M.S., Kosslyn, S.M., Breiter, H.C., DiGirolamo, G.J., Thompson, W.L., Anderson, A.K., Brookheimer, S.Y., Rosen, B.R., Belliveau, J.W., 1996. Changes in cortical activity during mental rotation. A mapping study using functional MRI. Brain 119 (Pt. 1), 89 – 100. Corballis, M.C., 1997. Mental rotation and the right hemisphere. Brain Lang. 57, 100 – 121. de Lange, F.P., Hagoort, P., Toni, I., 2005. Neural topography and content of movement representations. J. Cogn. Neurosci. 17, 97 – 112. Farah, M.J., 1995. Current issues in the neuropsychology of image generation. Neuropsychologia 33, 1455 – 1471. Fletcher, P., Buchel, C., Josephs, O., Friston, K., Dolan, R., 1999. Learningrelated neuronal responses in prefrontal cortex studied with functional neuroimaging. Cereb. Cortex 9, 168 – 178. Formisano, E., Goebel, R., 2003. Tracking cognitive processes with functional MRI mental chronometry. Curr. Opin. Neurobiol. 13, 174 – 181. Friman, O., Borga, M., Lundberg, P., Knutsson, H., 2003. Adaptive analysis of fMRI data. NeuroImage 19, 837 – 845. Ganis, G., Keenan, J.P., Kosslyn, S.M., Pascual-Leone, A., 2000. Transcranial magnetic stimulation of primary motor cortex affects mental rotation. Cereb. Cortex 10, 175 – 180. Ganis, G., Thompson, W.L., Kosslyn, S.M., 2004. Brain areas underlying visual mental imagery and visual perception: an fMRI study. Brain Res. Cogn. Brain Res. 20, 226 – 241. Gerardin, E., Sirigu, A., Lehericy, S., Poline, J.B., Gaymard, B., Marsault, C., Agid, Y., Le Bihan, D., 2000. Partially overlapping neural networks for real and imagined hand movements. Cereb. Cortex 10, 1093 – 1104. Gilaie-Dotan, S., Ullman, S., Kushnir, T., Malach, R., 2002. Shapeselective stereo processing in human object-related visual areas. Hum. Brain Mapp. 15, 67 – 79. Harris, I.M., Egan, G.F., Sonkkila, C., Tochon-Danguy, H.J., Paxinos, G., Watson, J.D., 2000. Selective right parietal lobe activation during mental rotation: a parametric PET study. Brain 123 (Pt. 1), 65 – 73.

444

C. Ecker et al. / NeuroImage 32 (2006) 432 – 444

Ishai, A., Ungerleider, L.G., Haxby, J.V., 2000. Distributed neural systems for the generation of visual images. Neuron 28, 979 – 990. Ishai, A., Haxby, J.V., Ungerleider, L.G., 2002. Visual imagery of famous faces: effects of memory and attention revealed by fMRI. NeuroImage 17, 1729 – 1741. Iwaki, S., Ueno, S., Imada, T., Tonoike, M., 1999. Dynamic cortical activation in mental image processing revealed by biomagnetic measurement. NeuroReport 10, 1793 – 1797. Johansen-Berg, H., Behrens, T.E., Robson, M.D., Drobnjak, I., Rushworth, M.F., Brady, J.M., Smith, S.M., Higham, D.J., Matthews, P.M., 2004. Changes in connectivity profiles define functionally distinct regions in human medial frontal cortex. Proc. Natl. Acad. Sci. U. S. A. 101, 13335 – 13340. Jordan, K., Heinze, H.J., Lutz, K., Kanowski, M., Jancke, L., 2001. Cortical activations during the mental rotation of different visual objects. NeuroImage 13, 143 – 152. Jordan, K., Wustenberg, T., Heinze, H.J., Peters, M., Jancke, L., 2002. Women and men exhibit different cortical activation patterns during mental rotation tasks. Neuropsychologia 40, 2397 – 2408. Kanwisher, N., Chun, M.M., McDermott, J., Ledden, P.J., 1996. Functional imagining of human visual recognition. Brain Res. Cogn. Brain Res. 5, 55 – 67. Kimmig, H., Greenlee, M.W., Gondan, M., Schira, M., Kassubek, J., Mergner, T., 2001. Relationship between saccadic eye movements and cortical activity as measured by fMRI: quantitative and qualitative aspects. Exp. Brain Res. 141, 184 – 194. Klein, I., Dubois, J., Mangin, J.F., Kherif, F., Flandin, G., Poline, J.B., Denis, M., Kosslyn, S.M., Le Bihan, D., 2004. Retinotopic organization of visual mental images as revealed by functional magnetic resonance imaging. Brain Res. Cogn. Brain Res. 22, 26 – 31. Kosslyn, S.M., Maljkovic, V., Hamilton, S.E., Horwitz, G., Thompson, W.L., 1995. Two types of image generation: evidence for left and right hemisphere processes. Neuropsychologia 33, 1485 – 1510. Kosslyn, S.M., Shin, L.M., Thompson, W.L., McNally, R.J., Rauch, S.L., Pitman, R.K., Alpert, N.M., 1996. Neural effects of visualizing and perceiving aversive stimuli: a PET investigation. Neuroreport 7, 1569 – 1576. Kosslyn, S.M., Thompson, W.L., Alpert, N.M., 1997. Neural systems shared by visual imagery and visual perception: a positron emission tomography study. NeuroImage 6, 320 – 334. Kosslyn, S.M., DiGirolamo, G.J., Thompson, W.L., Alpert, N.M., 1998. Mental rotation of objects versus hands: neural mechanisms revealed by positron emission tomography. Psychophysiology 35, 151 – 161. Kosslyn, S.M., Pascual-Leone, A., Felician, O., Camposano, S., Keenan, J.P., Thompson, W.L., Ganis, G., Sukel, K.E., Alpert, N.M., 1999. The role of area 17 in visual imagery: convergent evidence from PET and rTMS. Science 284, 167 – 170. Kosslyn, S.M., Ganis, G., Thompson, W.L., 2001. Neural foundations of imagery. Nat. Rev., Neurosci. 2, 635 – 642. Kosslyn, S.M., Thompson, W.L., Wraga, M., Alpert, N.M., 2001. Imagining rotation by endogenous versus exogenous forces: distinct neural mechanisms. NeuroReport 12, 2519 – 2525. Kwong, K.K., Belliveau, J.W., Chesler, D.A., Goldberg, I.E., Weisskoff, R.M., Poncelet, B.P., Kennedy, D.N., Hoppel, B.E., Cohen, M.S., Turner, R., 1992. Dynamic magnetic resonance imaging of human brain activity during primary sensory stimulation. Proc. Natl. Acad. Sci. U. S. A. 89, 5675 – 5679. Levine, D.N., Warach, J., Farah, M., 1985. Two visual systems in mental imagery: dissociation of ‘‘what’’ and ‘‘where’’ in imagery disorders due to bilateral posterior cerebral lesions. Neurology 35, 1010 – 1018. Linn, M.C., Petersen, A.C., 1985. Emergence and characterization of sex differences in spatial ability: a meta-analysis. Child Dev. 56, 1479 – 1498. Mechelli, A., Price, C.J., Friston, K.J., Ishai, A., 2004. Where bottom – up meets top – down: neuronal interactions during perception and imagery. Cereb. Cortex 14, 1256 – 1265. Menon, R.S., Luknowsky, D.C., Gati, J.S., 1998. Mental chronometry using

latency-resolved functional MRI. Proc. Natl. Acad. Sci. U. S. A. 95, 10902 – 10907. Merchant, H., Battaglia-Mayer, A., Georgopoulos, A.P., 2003. Interception of real and apparent motion targets: psychophysics in humans and monkeys. Exp. Brain Res. 152, 106 – 112. Ogawa, S., Lee, T.M., Kay, A.R., Tank, D.W., 1990. Brain magnetic resonance imaging with contrast dependent on blood oxygenation. Proc. Natl. Acad. Sci. U. S. A. 87, 9868 – 9872. Parsons, L.M., Fox, P.T., Downs, J.H., Glass, T., Hirsch, T.B., Martin, C.C., Jerabek, P.A., Lancaster, J.L., 1995. Use of implicit motor imagery for visual shape discrimination as revealed by PET. Nature 375, 54 – 58. Petit, L., Courtney, S.M., Ungerleider, L.G., Haxby, J.V., 1998. Sustained activity in the medial wall during working memory delays. J. Neurosci. 18, 9429 – 9437. Picard, N., Strick, P.L., 2001. Imaging the premotor areas. Curr. Opin. Neurobiol. 11, 663 – 672. Richter, W., Ugurbil, K., Georgopoulos, A., Kim, S.G., 1997. Timeresolved fMRI of mental rotation. NeuroReport 8, 3697 – 3702. Richter, W., Somorjai, R., Summers, R., Jarmasz, M., Menon, R.S., Gati, J.S., Georgopoulos, A.P., Tegeler, C., Ugurbil, K., Kim, S.G., 2000. Motor area activity during mental rotation studied by time-resolved single-trial fMRI. J. Cogn. Neurosci. 12, 310 – 320. Sakai, K., Hikosaka, O., Miyauchi, S., Sasaki, Y., Fujimaki, N., Putz, B., 1999. Presupplementary motor area activation during sequence learning reflects visuo-motor association. J. Neurosci. 19, RC1. Seurinck, R., Vingerhoets, G., Vandemaele, P., Deblaere, K., Achten, E., 2005. Trial pacing in mental rotation tasks. NeuroImage. 25, 1187 – 1196. Shepard, R.N., Metzler, J., 1971. Mental rotation of three-dimensional objects. Science 171, 701 – 703. Shima, K., Mushiake, H., Saito, N., Tanji, J., 1996. Role for cells in the presupplementary motor area in updating motor plans. Proc. Natl. Acad. Sci. U. S. A. 93, 8694 – 8698. Simmons, A., Moore, E., Williams, S.C., 1999. Quality control for functional magnetic resonance imaging using automated data analysis and Shewhart charting. Magn. Reson. Med. 41, 1274 – 1278. Slotnick, S.D., Thompson, W.L., Kosslyn, S.M., 2005. Visual mental imagery induces retinotopically organized activation of early visual areas. Cereb. Cortex 15, 1570 – 1583. Tagaris, G.A., Kim, S.G., Strupp, J.P., Andersen, P., Ugurbil, K., Georgopoulos, A.P., 1996. Quantitative relations between parietal activation and performance in mental rotation. NeuroReport 7, 773 – 776. Talairach, J., Tournoux, P., 1988. Co-Planar Stereotaxic Atlas of a Human Brain. Ungerleider, L.G., Haxby, J.V., 1994. ‘‘What’’ and ‘‘where’’ in the human brain. Curr. Opin. Neurobiol. 4, 157 – 165. Vanrie, J., Beatse, E., Wagemans, J., Sunaert, S., Van Hecke, P., 2002. Mental rotation versus invariant features in object perception from different viewpoints: an fMRI study. Neuropsychologia 40, 917 – 930. Vingerhoets, G., de Lange, F.P., Vandemaele, P., Deblaere, K., Achten, E., 2002. Motor imagery in mental rotation: an fMRI study. NeuroImage 17, 1623 – 1633. Voyer, D., Voyer, S., Bryden, M.P., 1995. Magnitude of sex differences in spatial abilities: a meta-analysis and consideration of critical variables. Psychol. Bull. 117, 250 – 270. Voyer, D., 1995. Effect of practice on laterality in a mental rotation task. Brain Cogn. 29, 326 – 335. Welchman, A.E., Deubelius, A., Conrad, V., Bulthoff, H.H., Kourtzi, Z., 2005. 3D shape perception from combined depth cues in human visual cortex. Nat. Neurosci. 8, 820 – 827. Wexler, M., Kosslyn, S.M., Berthoz, A., 1998. Motor processes in mental rotation. Cognition 68, 77 – 94. Windischberger, C., Lamm, C., Bauer, H., Moser, E., 2003. Human motor cortex activity during mental rotation. NeuroImage 20, 225 – 232. Wise, S.P., 1985. The primate premotor cortex: past, present, and preparatory. Annu. Rev. Neurosci. 8, 1 – 19.