Orientation-specific contextual modulation of the fMRI BOLD response to luminance and chromatic gratings in human visual cortex

Vision Research 49 (2009) 1397–1405

Contents lists available at ScienceDirect

Vision Research journal homepage: www.elsevier.com/locate/visres

Orientation-specific contextual modulation of the fMRI BOLD response to luminance and chromatic gratings in human visual cortex J. Scott McDonald a, Kiley J. Seymour a, Mark M. Schira b, Branka Spehar b, Colin W.G. Clifford a,* a b

School of Psychology, The University of Sydney, Sydney, NSW 2006, Australia School of Psychology, UNSW, Sydney, NSW 2021, Australia

a r t i c l e

i n f o

Article history: Received 21 March 2008 Received in revised form 11 August 2008

Keywords: fMRI Inhibition Suppression Tilt illusion Centre–surround interactions

a b s t r a c t The responses of orientation-selective neurons in primate visual cortex can be profoundly affected by the presence and orientation of stimuli falling outside the classical receptive field. Our perception of the orientation of a line or grating also depends upon the context in which it is presented. For example, the perceived orientation of a grating embedded in a surround tends to be repelled from the predominant orientation of the surround. Here, we used fMRI to investigate the basis of orientation-specific surround effects in five functionally-defined regions of visual cortex: V1, V2, V3, V3A/LO1 and hV4. Test stimuli were luminance-modulated and isoluminant gratings that produced responses similar in magnitude. Less BOLD activation was evident in response to gratings with parallel versus orthogonal surrounds across all the regions of visual cortex investigated. When an isoluminant test grating was surrounded by a luminance-modulated inducer, the degree of orientation-specific contextual modulation was no larger for extrastriate areas than for V1, suggesting that the observed effects might originate entirely in V1. However, more orientation-specific modulation was evident in extrastriate cortex when both test and inducer were luminance-modulated gratings than when the test was isoluminant; this difference was significant in area V3. We suggest that the pattern of results in extrastriate cortex may reflect a refinement of the orientation-selectivity of surround suppression specific to the colour of the surround or, alternatively, processes underlying the segmentation of test and inducer by spatial phase or orientation when no colour cue is available. Ó 2009 Elsevier Ltd. All rights reserved.

1. Introduction We use functional MRI to investigate the substrates of orientation-specific contextual modulation in human vision. In each of two experiments, we compare the magnitude of the BOLD signal in response to stimulus blocks where the test and inducer are parallel (but 90° out of phase spatially) with blocks where test and inducer are perpendicular. The motivation for these experiments is described below. Contextual modulation is a fundamental property of visual processing with moment-to-moment relevance to our perception of attributes such as colour, lightness, contrast and orientation. Effects of image context have been investigated extensively using psychophysical and electrophysiological methods, and their functional basis has been much debated by theoreticians interested in the optimal coding of sensory information (for a review, see Schwartz, Hsu, & Dayan, 2007). Perhaps the ‘‘best studied test case” of contextual modulation is that of visual orientation processing

* Corresponding author. E-mail address: [email protected] (C.W.G. Clifford). 0042-6989/$ - see front matter Ó 2009 Elsevier Ltd. All rights reserved. doi:10.1016/j.visres.2008.12.014

(Schwartz et al., 2007), and it is for that reason that we focus on orientation-selective contextual modulation here. Visual neurons in cat and monkey cortex are excited by stimuli placed within their classical receptive fields (CRF). The area adjacent to the CRF, when stimulated alone, does not elicit a response from the neuron. However, this extra-classical receptive field (ECRF) region can profoundly modulate the neuron’s response to a stimulus within the CRF (Blakemore & Tobin, 1972; Chen, Kasamatsu, Polat, & Norcia, 2001; DeAngelis, Freeman, & Ohzawa, 1994; Jones, Grieve, Wang, & Sillito, 2001; Jones, Wang, & Sillito, 2002; Maffei & Fiorentini, 1976; Nelson & Frost, 1978, 1985; Walker, Ohzawa, & Freeman, 1999; Webb, Barraclough, Parker, & Derrington, 2003). For both cat and monkey, contextual modulation tends to be suppressive. The incidence of facilitation depends on the relative contrast of the stimuli in the CRF and ECRF, typically occurring only when the stimulus in the CRF has a low contrast (Polat, Mizobe, Pettet, Kasamatsu, & Norcia, 1998). Suppression predominates when the CRF is stimulated with high contrast stimuli. Of particular relevance to our study, the magnitude of suppression is strongly modulated by the relative orientations of the stimuli in the CRF and ECRF. Suppression tends to be greatest when the ECRF and CRF orientations are the same and is attenuated or

1398

J.S. McDonald et al. / Vision Research 49 (2009) 1397–1405

abolished when they are very different (Gulyas, Orban, Duysens, & Maes, 1987; Levitt and Lund, 1997; Li & Li, 1994; Li, Thier, & Wehrhahn, 2000; Nelson & Frost, 1978; Sillito et al, 1995; Cavanaugh, Bair, & Movshon, 2002). Indeed, at least two previous studies specifically identifying the surround orientations giving maximum and minimum suppression for each of a population of cells have found maximum suppression for surrounds near the cell’s preferred orientation and minima around the orthogonal orientation (Gilbert & Wiesel, 1990; Sengpiel, Sen, & Blakemore, 1997). Consequently, we have chosen to use inducing gratings oriented parallel and orthogonal to the test stimulus in order to investigate orientation-specific contextual modulation. Our hypothesis is that the effect of contextual modulation should be maximal for parallel and minimal for orthogonal test and inducing gratings. A direct comparison of the fMRI BOLD response to these stimuli is thus a measure of the orientation-specific component of contextual modulation in human visual cortex. Perceptually, similarly oriented surround gratings tend to produce the greatest effects on perceived orientation and contrast (Cannon & Fullenkamp, 1991; Gibson & Radner, 1937; Snowden & Hammett, 1998). The effect of an oriented surround on the perceived orientation of a central test stimulus, the tilt illusion (Fig. 1A), was first reported by Gibson and Radner (1937). The tilt illusion depends on the relative orientation of centre and surround (O’Toole & Wenderoth, 1977), indicating that it can be mediated no earlier in the visual processing hierarchy than the first site of significant orientation-selectivity, V1 (Hubel & Wiesel, 1962; Smith, Chino, Ridder, Kitagawa, & Langston, 1990; Xu, Ichida, Shostak, Bonds, & Casagrande, 2002). When centre and surround stimuli are presented in separate eyes (dichoptic viewing) the magnitude of the tilt illusion is reduced by around 20% (Forte & Clifford, 2005; Wade, 1980) compared with presentation to the same eye (monocular viewing). This incomplete interocular transfer indicates that the tilt illusion is mediated in part by neural mechanisms receiving input from only one eye. The only area of visual cortex known to contain a significant proportion of monocular neurons is V1 (Hubel & Wiesel, 1962), indicating that the tilt illusion is mediated at least in part within V1 itself. Thus, we predict that orientation-specific contextual modulation of the fMRI BOLD response will be evident as early as V1. The tilt illusion displays chromatic tuning, as do surround effects on perceived contrast (Singer & D’Zmura, 1994), such that the magnitude of the illusion is maximal for centre and surround modulated along the same axis of colour space. For centre and surround modulated along orthogonal axes of colour space, for example an L–M isolating (‘‘red–green”) central grating and a luminance-defined surround grating, the magnitude of the illusion drops by around 50% (Clifford, Pearson, Forte, & Spehar, 2003; Clifford, Spehar, Solomon, Martin, & Zaidi, 2003; Forte & Clifford, 2005). Thus, the illusion

Fig. 1. Contextual modulation of perceived orientation – the tilt illusion. (A) The orientation of a vertical central grating appears repelled away from that of the surround. (B) The illusion persists when a vertical test is presented in an annulus surrounded inside and out by an oriented inducer.

can be considered to involve colour-specific and colour-invariant components. When centre and surround stimuli are modulated along orthogonal axes of colour space, full interocular transfer of the tilt illusion is observed (Forte & Clifford, 2005). This indicates that the colour-invariant portion of the tilt illusion is purely binocular, and thus that the monocular component of the tilt illusion is colour-specific. The binocular component of the tilt illusion, however, is largely colour-invariant (Forte & Clifford, 2005). The observation that the monocular component of the tilt illusion is colour-specific implicates neurons in V1 coding both colour and orientation. Evidence for just such conjoint tuning of colour and orientation in human V1 comes from fMR adaptation (Engel, 2005) and a recent application of multivariate pattern analysis to fMRI data (Sumner, Anderson, Sylvester, Haynes, & Rees, 2008). Single neurons selective for both colour and orientation have also been reported in V1 of non-human primates (De Valois, Cottaris, Elfar, Mahon, & Wilson, 2000; Johnson, Hawken, & Shapley, 2001; Lennie, Krauskopf, & Sclar, 1990; Leventhal, Thompson, Liu, Zhou, & Ault, 1995; Thorell, De Valois, & Albrecht, 1984), as well as in areas V2 and V3 (Gegenfurtner, Kiper, & Fenstemaker, 1996; Gegenfurtner, Kiper, & Levitt, 1997). Thus, we predict that the orientation-selective component of contextual modulation in the fMRI BOLD response will be greater for test and inducer modulated along the same versus orthogonal axes of DKL colour space, and that this difference will be evident as early as V1. The tilt illusion is thought to be largely due to orientation-selective inhibition of neurons responding to the test grating (Fig. 2). A similar mechanism could potentially account for the reduction in perceived contrast (Olzak & Laurinen, 1999). Specifically, orientation-selective cortical neurons responding to the inducing grating inhibit similarly tuned neurons responding to the test (Blakemore & Tobin, 1972; Clifford, Wenderoth, & Spehar, 2000; Wenderoth & Johnstone, 1987). When test and inducer differ slightly in orientation (e.g. by 15°), this lateral inhibition biases the neural representation of the orientation of the test such that the population response (and hence the perceived orientation) is repelled away from the inducing orientation (Clifford et al., 2000; Gilbert & Wiesel, 1990). When test and inducer are parallel, the neuronal population response to the test is maximally inhibited. However, no orientation illusion is elicited because there is no asymmetry in the inhibition and hence no bias in the population response. Thus, whilst a difference in inducer-test orientation is necessary to elicit the perceptual tilt illusion, the underlying neuronal interactions are best studied physiologically using parallel gratings. Our use of a 90° shift in spatial phase between test and surround gratings in the parallel condition stems from our interest in the orientation-specific contextual modulation likely underlying the tilt illusion as distinct from phase-specific effects of contextual modulation with parallel gratings as are evident psychophysically in perceived contrast (Olzak & Laurinen, 1999) and electrophysiologically (Akasaki, Sato, Yoshimura, Ozeki, & Shimegi, 2002; DeAngelis et al., 1994; Xu, Shen, & Li, 2005). Relative spatial phase is not a meaningful parameter when test and surround are not parallel and hence is of no relevance in the tilt illusion, so we did not want to confound orientation- and phase-specific effects by using in-phase test and surround in the parallel condition. Whilst there is some evidence from single-cell studies to suggest that we might have observed a bigger effect if we had used in-phase test and surround in the parallel condition (Xu et al., 2005), our aim here was to isolate the effect of orientation rather than to observe the maximum effect. In psychophysical studies of contextual modulation it is conventional to position the test region at fixation, for example a circular test patch embedded in an annular inducer (Fig. 1A). Such a configuration has also been used to investigate orientation-specific contextual modulation in an fMRI study (Williams, Singh, & Smith, 2003). Williams et al. (2003) reported suppression of the response

J.S. McDonald et al. / Vision Research 49 (2009) 1397–1405

1399

Fig. 2. Relationship between neuronal population response and perceived orientation. (A) Hypothetical response of a population of orientation-selective neurons as a function of the neuronal peak orientation tuning in response to a vertical (0°) test stimulus. (B) Hypothetical effect of an oriented surround on the gain of the neurons responding to the test illustrated for surround oriented at 0° (solid line) or 15° (dotted line). Orientation-selective cortical neurons responding to the inducing grating inhibit similarly tuned neurons responding to the test. (C) Predicted neuronal population response to a test stimulus oriented at 0° in the presence of a 0° surround. The prediction is obtained by multiplying the unmodulated population response in (A) by the gain – solid line in (B). The overall population response is strongly reduced (compare area under population response curve with and without surround) but the effect on the response profile is symmetrical about 0° and hence the population continues to code the test orientation veridically – i.e. no tilt illusion. (D) Predicted neuronal population response to a test stimulus oriented at 0° in the presence of a 15° surround. The prediction is obtained by multiplying the unmodulated population response in (A) by the gain – dotted line in (B). The overall population response is not as strongly reduced as in (C) but the response profile is now biased away from the orientation of the surround and hence the population codes an orientation consistent with a repulsive tilt illusion. Thus, whilst a difference in inducer-test orientation is necessary to elicit the perceptual tilt illusion, the underlying neuronal interactions are best studied physiologically using parallel gratings.

to the test in retinotopic visual cortex in the presence of a parallel but not a perpendicular surround. However, foveal presentation of the test stimulus did not allow areas V1–V3 to be consistently distinguished, so quantitative data were only reported for a region-ofinterest encompassing all three areas. Delineation of the borders between retinotopic visual areas is easier in the periphery than in the fovea. Here, we are interested in tracking orientation-specific contextual modulation through the visual processing hierarchy, so we follow the stimulus configuration of Zenger-Landolt and Heeger (2003) in using an annular test region surrounded inside and out by the inducing stimulus (Fig. 3A). Psychophysical piloting with this stimulus revealed that the magnitude of the repulsive tilt illusion was of the same order (several degrees) as for foveal test presentation (compare the perceived orientation of the central grating in Fig. 1A and that of the annular grating in Fig. 1B: both are objectively vertical). Simultaneous presentation of stimuli has been shown to suppress the fMRI BOLD response across human visual cortex relative to sequential presentation (Kastner et al., 2001). Surround suppression of the response to collinear gratings has been specifically investigated in areas V1–V3 (Zenger-Landolt & Heeger, 2003). Both studies found that suppression was greater in extrastriate areas than in V1. When compared with behavioural data, Zenger-Landolt and Heeger (2003) found that the surround suppression observed in V1 was in good quantitative agreement with the psychophysical effect on perceived contrast. Our hypothesis about the link between contextual modulation at the neuronal level and its expression in the BOLD signal is that the contextual modulation we observe will be primarily inhibitory and thus that it will reduce the magnitude of the BOLD signal in the regions of the cortical maps being inhibited (Beck & Kastner, 2005, 2007; Kastner et al., 2001; Zenger-Landolt & Heeger, 2003). However, since the precise

relationship between neuronal inhibition and the BOLD signal remains uncertain (Attwell & Iadecola, 2002; Waldvogel et al., 2000), we conduct all statistical tests as two-tailed rather than assuming a priori that neuronal inhibition reduces the BOLD signal. One possible confound we sought to avoid was different BOLD responses elicited by different spatial orientations, the fMRI analogue of the psychophysical oblique effect (Furmanski & Engel, 2000). To ensure that any differences in the responses to parallel and orthogonal blocks were due to the relative orientation of test and surround rather than the absolute orientation, each block consisted of a sequence of grating stimuli presented at each of 16 spatial orientations (Fig. 3B). In this way, it was ensured that the distribution of orientations in both test and surround was the same for parallel and orthogonal conditions. Another important issue in using fMRI to study contextual modulation is isolation of the response to the test. Specifically, in comparing the response to a test stimulus presented in isolation with a test stimulus presented with a surround, there is a danger that a component of the response to the surround may be interpreted as a response to the test or that ‘‘haemodynamic stealing” by the surround might inflate the measured effect of contextual modulation (Zenger-Landolt & Heeger, 2003). Here, we avoided any such confound by comparing two conditions each of which consisted of test and surround stimuli. Thus, we are measuring the orientation-selective component of contextual modulation rather than the total magnitude of contextual modulation in any one condition. We concentrated our analysis on those voxels within five functionally-defined regions of visual cortex: V1, V2, V3, V3A/LO1 and hV4 that gave significant responses to the stimulus in the test annulus presented in isolation (see Section 2.2.1 for full details). We do not claim to have isolated only the response to the test region of the stimulus when presented with an abutting surround as

1400

J.S. McDonald et al. / Vision Research 49 (2009) 1397–1405

Fig. 3. Experimental stimuli. (A) The five different stimulus types used in the experiments, from left to right: test-only, inducer-only, orthogonal test-inducer, parallel testinducer, blank screen. All conditions also have a fixation marker. (B) Schematic example of the stimulus orientations used in a typical block, in this case with orthogonal testinducer.

voxel receptive field size in V1 at the eccentricity of the test annulus (2–3°) is of the order of 0.5° (standard deviation of fitted 2-D Gaussian: Dumoulin & Wandell, 2008; Kay, Naselaris, Prenger, & Gallant, 2008; Larsson & Heeger, 2006). However, we would expect lateral interactions between annulus and inducer to be reciprocal (i.e. if the test is orthogonal to the surround then the surround is orthogonal to the test), so voxels close to the boundary should show contextual modulation in the component of their response to the inducer as well as in the component of their response to the annulus. Failure to exclude all response to the surround would thus not be expected to bias the data as long as it was consistent across conditions. A final issue involves the border between test and surround. It is conceivable that the activity of neurons whose receptive fields include the border between test and surround might induce a different BOLD response for parallel and orthogonal surrounds regardless of any contextual modulation proper. For example, it might be that the salience of the border between test and surround differs between parallel and orthogonal stimulus blocks, or that segmentation processes are engaged differentially in the two conditions. Although we know of no direct evidence in support of this potential criticism, it is something for which we have attempted to control in our second experiment by using an isoluminant test grating and a luminance-modulated surround. In such a stimulus, the colour-luminance boundary provides a salient border and thus a strong cue to segmentation in both parallel and orthogonal stimulus blocks. 2. Methods and materials 2.1. Experimental procedures 2.1.1. Subjects and experimental sessions One female and three male subjects participated in the experiments. All were experienced psychophysical subjects and had normal or corrected to normal vision. Each subject participated in at least 16 sessions. Six sessions were required for each of Experiments 1 and 2 and at least four sessions were used to establish the retinotopic areas. 2.1.2. Scanner and experimental setup A Philips 3T scanner with a whole-head ‘‘birdcage” coil was used to perform the MRI. The stimuli were presented on a Fara-

day-shielded flat panel LCD Philips monitor size 35 cm  28 cm and resolution 1024  768. The monitor was calibrated so that pixel value was linearly related to luminance output. Subjects lay in the scanner and viewed the monitor through a mirror attached to the head coil. The viewing distance to the monitor, when seen through the mirror, was 158 cm. Behavioural responses were indicated through an MRI-compatible LU400-PAIR Lumina response pad (Cedrus Corporation, San Pedro, CA, USA). Whilst the subjects viewed the stimuli, a time series of 168 MRI volumes were collected using a T2*-sensitive, boustrophedon, field echo (i.e. gradient echo) echo planar imaging (FEEPI) pulse sequence. The echo time (TE) was 30 ms, repetition time (TR) 2000 ms, flip angle of 90°, field of view (FOV) 220 mm  87 mm  220 mm, in-plane resolution 1.7 mm  1.7 mm, slice thickness 3.0 mm. Twenty-nine slices were collected in an interleaved, ascending order, in the transverse plane and the scan covered most of the head including all of the occipital and parietal lobes. 2.1.3. Stimulus and task The stimuli were sinusoidal gratings (1 cycle/°) presented in blocks of 16 s. Each block consisted of 16 static images of 1 s duration. The stimuli were presented in a circular aperture with a radius of 6.0° surrounded by a uniform field at the mean luminance of the stimulus (57 Cd/m2). They consisted of an annular test region, which extended from 2.0° to 3.0° radius, and a surround inducer region which covered the remainder of the stimulus aperture inside and outside the test annulus. There were four different stimulus configurations (Fig. 3A): test-only, inducer-only, parallel test and inducer, and orthogonal test and inducer. Each 16-s block contained 16 different orientations of the stimuli, presented in pseudo-random order, such that every discrete stimulus orientation occurred once (Fig. 3B). The parallel and orthogonal blocks differed only in the relative orientation of test and inducer and not in the distribution of absolute orientations. Hence any ‘‘oblique effects”, whereby horizontal and vertical orientations generate greater BOLD response than diagonal orientations (Furmanski & Engel, 2000), were balanced between blocks. There were 21 blocks in each session, four blocks of each of the four stimulus configurations and five blank fixation periods. The blocks were ordered in a balanced design (each block type occurred an equal number of times before every other block type) and the first, last and every fifth block were blank fixation periods.

J.S. McDonald et al. / Vision Research 49 (2009) 1397–1405

Throughout all blocks, subjects’ attention was carefully controlled by requiring them to perform a demanding dimming task: detection of a brief luminance decrement of the fixation marker presented at the centre of the display. 2.1.4. Experiment 1 Both regions of the stimulus, test and inducer, were achromatic luminance-modulated gratings presented at full Michelson contrast. 2.1.5. Experiment 2 As for Experiment 1, the inducing regions of the stimulus contained full contrast achromatic luminance gratings. However, the test annulus was now a grating that was modulated along the L– M (red–green) isolating axis of DKL colour space (Derrington, Krauskopf, & Lennie, 1984). Isoluminance for the L–M axis was established separately for each subject prior to scanning using minimum motion (Anstis & Cavanagh, 1983) and minimum flicker techniques under viewing conditions matched to those of the scanner. Isoluminance was then confirmed in the scanner for each subject using only the minimum flicker technique. 2.2. Analysis procedures 2.2.1. Preprocessing and definition of regions of interest The fMRI data were processed with the BrainVoyagerQX 1.9 software package. The data were preprocessed to correct for slice time, head motion and linear trends. The functional scans were

1401

aligned to the anatomical data and transformed into Talairach coordinates at an effective in-plane resolution of 3.0 mm. For each subject the cortical surface was then segmented, smoothed and inflated so that we could identify the early retinotopic areas (V1, V2, V3, V3A/LO1 and hV4) as regions of interest (ROIs). Standard retinotopic mapping procedures were used to identify the visual areas. Subjects were presented with rotating wedge and expanding ring stimuli (Engel, Glover, & Wandell, 1997; Wandell et al ., 2007). Voxel-by-voxel analysis of the temporal profile of the response to these stimuli allowed us to visualise the mapping of the visual field onto an inflated representation of the cortical surface (illustrated for the left occipital lobe of one subject, KJS, in Fig. 4A). The early retinotopic areas were then delineated with reference to canonical data from fMRI of human visual cortex (Larsson & Heeger, 2006; Wandell et al., 2007). In defining our retinotopic regions of interest, we follow the nomenclature recommended by Wandell et al (2007). Three specific points are worth highlighting. First, our area V3 contains both dorsal V3 and its ventral counterpart, sometimes termed VP (e.g. Pitzalis et al., 2006). Second, since we were unable to separate areas V3A and LO1 with confidence in all subjects, we collapsed them into one large region lateral to V3d that we term V3A/LO1. Third, we have defined area hV4 as a hemifield map directly abutting the ventral portion of V3. We then isolated voxels where the test-only stimulus block evoked a strong BOLD signal to create a mask (Fig. 4B). To this end we created four reference functions, one for each stimulus type (fixation alone was not modelled), via convolution with a model

Fig. 4. Stages leading to definition of masked regions of interest (mROIs). (A) Example map on an inflated brain of peak response phase to a rotating wedge. Colours indicate position of the wedge in the visual field eliciting peak response. The key beneath the figure indicates which areas of the visual field correspond to which colours. Black lines on the main graphic indicate borders between visual regions determined manually. Data come from the left hemisphere of subject KJS. (B) Statistical map showing voxels where the response to the test-only HRF model was greater than baseline. From these voxels we extracted the response time course for masked regions of interest (mROIs) corresponding to each of the visual areas. (C) The same hemisphere with the ROIs labelled; gyri and sulci removed for clarity.

1402

J.S. McDonald et al. / Vision Research 49 (2009) 1397–1405

haemodynamic response function. These reference functions were used as the design matrix in a general linear model. The response to test-only stimulus blocks was contrasted against baseline. Serial correlations were corrected for by pre-whitening, assuming the correlations followed a first-order autoregressive AR(1) process. Only voxels with a significance p < 0.001 (Bonferroni corrected) were included in the mask. This mask was then combined with the retinotopically-defined ROIs to produce masked regions of interest (mROIs). 2.2.2. Comparison of conditions using percentage signal change For each subject we extracted fMRI responses by averaging data from all the voxels within each mROI. For each scan, we then averaged the signal across all blocks of the same type. The fMRI response in each condition was calculated as the percentage signal change (PSC) from fixation:

PSC ¼ 100 ðt  bÞ=b

ð1Þ

where t is the mean signal value across the block (offset by 2 TRs to allow for haemodynamic delay) and b is the baseline response to the blank, fixation-only blocks. The PSC was then averaged, for each subject in each mROI, across sessions. For each subject in each mROI, we defined an index of the orientation-selectivity of contextual modulation (m) as:

m ¼ ðPSCorth  PSCpara Þ=ðPSCorth þ PSCpara Þ

ð2Þ

where PSCorth is the per cent signal change for the condition where the test and inducing gratings are orthogonal and PSCpara is the per cent signal change for the condition where the test and inducing gratings are parallel. 3. Results Fig. 5A shows a scatter plot of the percentage signal change in the parallel versus orthogonal conditions for each of the regions (mROIs) V1, V2, V3, V3A/LO1, hV4 for each subject in each experiment. Two points are worth highlighting. First, the vast majority of data points (37 of 40) lie below the leading diagonal, indicating a consistently lower BOLD response in the parallel than the orthogonal condition. Second, the range of responses is similar for the two experiments (Experiment 1: luminance-modulated test – filled symbols; Experiment 2: isoluminant test – open symbols), indicating that any differences between the results are not due to differential sensitivity to the test gratings. The data for each region-of-interest, averaged across subjects for Experiments 1 and 2 separately, are shown in Fig. 5B. In each experiment, all regions followed the trend of showing greater per cent signal change in the orthogonal than parallel condition. Across the four subjects and two experiments, a repeated measures 2  2 ANOVA showed that differences in per cent signal change between the orthogonal and parallel conditions were significantly greater than zero within all regions of interest except V1, which narrowly escaped significance: V1  F(1,3) = 9.79 (p = 0.052); V2  F(1,3) = 11.44 (p = 0.043); V3  F(1,3) = 10.72 (p = 0.047); V3A/LO1  F(1,3) = 16.05 (p = 0.028); hV4  F(1,3) = 10.99 (p = 0.045). Comparing differences in per cent signal change between the orthogonal and parallel conditions for the luminance-modulated and chromatic test stimuli revealed a consistent trend across all regions for greater orientation-selectivity with the luminance-modulated test. This interaction was significant at p < 0.05, 2-tailed, only for area V3 (F(1,3) = 10.64, p = 0.047). Although effect sizes were similar or larger for regions V3A/LO1 and hV4 than for V3, the use of a repeated measures ANOVA revealed much smaller between subjects variability for V3 and hence only in this region was the orientationselectivity for Experiments 1 and 2 significantly different.

To gain an idea of the relative orientation-selectivity across visual areas we wanted a measure that took into account the overall magnitude of the BOLD signal change; this varied by almost a factor of two between V1, where per cent signal change was around 3%, and areas V3A/LO1 and hV4 (Fig. 5B). To this end we constructed an orientation-selectivity index as the difference between responses in the orthogonal and parallel conditions as a proportion of their sum (see Eq. (2)). Inspection of the resulting pattern of data (Fig. 5C) suggests that with the luminance-modulated test there is a trend for the orientation-selectivity of contextual modulation to increase as the cortical processing hierarchy is ascended. This qualitative impression is supported by post-hoc comparison of the orientation-selectivity index for the luminance-modulated test between areas V1 and hV4: t(3) = 4.54 (p < 0.05, 2-tailed). This is clearly not the case with the chromatic test, for which orientation-selectivity showed essentially no variation across visual areas. 4. Discussion Analyses of percentage signal change in predefined regions of interest revealed less BOLD activation in response to gratings with parallel versus orthogonal surrounds across all the retinotopic areas of visual cortex we investigated (Fig. 5B). The direction of the effect is consistent with greater inhibition of neuronal activity from parallel surrounds, as predicted on the basis of human psychophysical data and primate single-cell electrophysiology and consistent with the fMRI data reported by Williams et al. (2003). However, we cannot rule out the possibility that the observed pattern of results actually reflects less facilitation rather than greater suppression from parallel than perpendicular surrounds. Future experiments employing asynchronous presentation of test and surround are planned to disambiguate these possibilities. With the luminance-modulated test there was a trend for the orientation-selectivity of contextual modulation to increase as the cortical processing hierarchy is ascended (Fig. 5C). However, when an isoluminant test grating was surrounded by a luminance-modulated inducer, the degree of orientation-specific contextual modulation was no larger for extrastriate areas than for V1, suggesting that the observed effects might originate entirely in V1. Furthermore, the degree of orientation-selective contextual modulation was significantly greater in area V3 when test and surround both consisted of luminance-modulated gratings than when the test stimulus was an isoluminant (L–M isolating) grating. A similar although non-significant trend was evident in areas V3A/ LO1 and hV4. One possible interpretation of the results would be to relate the difference in the orientation-selectivity of contextual modulation of luminance and isoluminant gratings evident in area V3 to differences in the magnitude of psychophysical effects on perceived orientation and contrast. Specifically, the perceptual effect of a luminance-modulated surround on a luminance-modulated test is greater than its effect on an isoluminant test; here we found an analogous effect on the fMRI BOLD response in area V3. The pattern of results for a luminance-modulated test is clearly not consistent with the observed effect in V3 simply being inherited from lateral interactions within earlier visual areas. Instead, they suggest the operation of orientation-selective mechanisms of contextual modulation within V3 that enhance the observed orientation-selectivity of contextual modulation in V3 when test and surround are both luminance-defined but not when a luminancemodulated grating surrounds a chromatic test. Such a refinement of the orientation-selectivity of contextual modulation specific to the colour/luminance congruence of test and surround as the visual hierarchy is ascended would seem to

J.S. McDonald et al. / Vision Research 49 (2009) 1397–1405

1403

Fig. 5. Experimental data. (A) Scatter plot of percentage signal change (PSC) invoked by the parallel test-inducer as a function of PSC invoked by the orthogonal test-inducer. Symbol colours indicate the subject: red – CC, green – EA, blue – KJS, black – JSM. Filled symbols indicate achromatic test stimulus, empty symbols indicate chromatic test. Symbol shape indicates the visual area: circle – V1, square – V2, diamond – V3, 5-pointed star – V3a/LO1, 6-pointed star – V4v. Nearly all the points (37 of 40) lie below the line of equal PSC, indicating that parallel test-inducer stimulus elicited a smaller PSC than the orthogonal under nearly all circumstances. (B) – top. Percentage signal change for Experiment 1 (achromatic test) averaged across subjects and as a function of visual area. Dark bars indicate PSC invoked by orthogonal test-inducer stimuli. Light bars indicate PSC invoked by parallel test-inducer stimuli. (B) – bottom. Percentage signal change for Experiment 2 (chromatic test). Red bars indicate PSC invoked by orthogonal test-inducer stimuli. Green bars indicate PSC invoked by parallel test-inducer stimuli. The data indicate that the orthogonal stimuli consistently invoke a higher PSC than the parallel stimuli. (C) Comparison of orientation-selectivity index Eq. (2) across experiments and visual areas. The bars indicate the difference between orthogonal test-inducer PSC and parallel test-inducer PSC as a proportion of their sum, as a function of visual area. The light-dark bars indicate the value for achromatic stimuli and the red–green bars indicate the value for the chromatic stimuli. All error bars represent ±one standard error of the mean. (For interpretation of the references to colour in this figure legend, the reader is referred to the web version of this article.)

1404

J.S. McDonald et al. / Vision Research 49 (2009) 1397–1405

be at odds with the observation that the binocular, and thus presumably higher-level, component of the tilt illusion is essentially colour-invariant (Forte & Clifford, 2005). However, Gegenfurtner et al. (1997) have demonstrated that in macaque essentially all V3 neurons are orientation-selective and that their colour properties are much more like those observed in V1 than V2, cautioning against a simple hierarchical view of processing across the early visual areas. Furthermore, Solomon, Peirce, and Lennie (2004) have reported that surround suppression in V1 is primarily driven by luminance contrast and is largely insensitive to chromatic contrast, even in cells whose classical receptive field is strongly colour-selective. In V2, on the other hand, surround suppression is more strongly driven by chromatic modulation and there is a tendency for the surround to have the same chromatic signature as the classical receptive field. If V2 contains colour-selective mechanisms actively involved surround suppression, as suggested by Solomon et al. (2004), and does not simply inherit its surround properties in a feed-forward fashion from V1, then it is reasonable to speculate that other extrastriate areas such as V3 might also contain mechanisms capable of generating surround suppression with similar tuning to that of the classical receptive field. An alternative interpretation of the results is that the different patterns of orientation-selective contextual modulation for luminance-modulated and isoluminant test gratings reflect processes involved in segmenting the test region from the luminance-modulated surround. When both test and surround are luminance-modulated gratings, the only cue to segmentation between them is their relative phase (‘‘parallel” condition) or orientation (‘‘orthogonal” condition). However, when the test is an isoluminant grating, the colour-luminance boundary provides a strong cue to segmentation and a salient border in both parallel and orthogonal stimulus blocks. Thus, the greater levels of orientation-selective contextual modulation with the luminance-modulated test might reflect differences in segmentation by phase- versus orientation-difference. In this case, the orientation-specific contextual modulation underlying the perception of orientation in the tilt illusion might originate entirely in V1, as has been argued for orientation-specific adaptation to luminancemodulated patterns (Larsson, Landy, & Heeger, 2006). Support for this idea comes from Experiment 2, where it can be seen that orientation-selectivity for extrastriate areas is no larger than for V1 when an isoluminant test grating is used (Fig. 5C). However, psychophysical evidence has indicated that the tilt illusion is largest precisely when test and inducer are hardest to segment (Durant & Clifford, 2006), suggesting that a clear separation between processes affecting the perception of orientation and more general processes of image parsing may not exist. To date, a number of studies have employed the technique of fMR adaptation to investigate the processing of orientation in human visual cortex (Boynton & Finney, 2003; Engel, 2005; Fang, Murray, Kersten, & He, 2005; Larsson et al., 2006; Montaser-Kouhsari, Landy, Heeger, & Larsson, 2007). These adaptation studies have used not only stimuli defined by luminance or chromatic modulations but also second-order textures (Larsson et al., 2006) and subjective contours (Montaser-Kouhsari et al., 2007). The effects of adaptation can be considered as modulation by temporal context, as opposed to the modulation by spatial context studied here. Thus, natural extensions to the current work would include investigating the neural basis of contextual effects with second-order patterns (e.g. Wenderoth, Clifford, & Ma Wyatt, 2001). Acknowledgments Scanning was carried out at the Symbion Clinical Research Imaging Centre at the Prince of Wales Medical Research Institute in Sydney. We are grateful to Kirsten Moffat for expert technical

assistance. This research was funded by a grant to CC from The University of Sydney. References Akasaki, T., Sato, H., Yoshimura, Y., Ozeki, H., & Shimegi, S. (2002). Suppressive effects of receptive field surround on neuronal activity in the cat primary visual cortex. Neuroscience Research, 43, 207–220. Anstis, S. M., & Cavanagh, P. (1983). A minimum motion technique for judging equiluminance. In J. Mollon & R. T. Sharpe (Eds.), Color vision: Physiology and psychophysics (pp. 156–166). London, UK: Academic. Attwell, D., & Iadecola, C. (2002). The neural basis of functional brain imaging signals. Trends in Neurosciences, 25, 621–625. Beck, D. M., & Kastner, S. (2005). Stimulus context modulates competition in human extrastriate cortex. Nature Neuroscience, 8, 1110–1116. Beck, D. M., & Kastner, S. (2007). Stimulus similarity modulates competitive interactions in human visual cortex. Journal of Vision, 7(2)(19), 1–12. Blakemore, C., & Tobin, E. A. (1972). Lateral inhibition between orientation detectors in the cat’s visual cortex. Experimental Brain Research, 15, 439–440. Boynton, G. M., & Finney, E. M. (2003). Orientation-specific adaptation in human visual cortex. Journal of Neuroscience, 23, 8781–8787. Cannon, M. W., & Fullenkamp, S. C. (1991). Spatial interactions in apparent contrast: Inhibitory effects among grating patterns of different spatial frequencies, spatial positions and orientations. Vision Research, 31, 1985–1998. Cavanaugh, J. R., Bair, W., & Movshon, J. A. (2002). Nature and interaction of signals from the receptive field center and surround in macaque V1 neurons. Journal of Neurophysiology, 88, 2530–2546. Chen, C. C., Kasamatsu, T., Polat, U., & Norcia, A. M. (2001). Contrast response characteristics of long-range lateral interactions in cat striate cortex. Neuroreport, 12, 655–661. Clifford, C. W. G., Pearson, J., Forte, J. D., & Spehar, B. (2003). Colour and luminance selectivity of spatial and temporal interactions in orientation perception. Vision Research, 43, 2885–2893. Clifford, C. W. G., Spehar, B., Solomon, S. G., Martin, P. R., & Zaidi, Q. (2003). Interactions between colour and luminance in the perception of orientation. Journal of Vision, 3, 106–115. Clifford, C. W. G., Wenderoth, P., & Spehar, B. (2000). A functional angle on some aftereffects in cortical vision. Proceedings of the Royal Society of London. Series B: Biological Sciences, 267, 1705–1710. De Valois, R. L., Cottaris, N. P., Elfar, S. D., Mahon, L. E., & Wilson, J. A. (2000). Some transformations of color information from geniculate nucleus to striate cortex. Proceedings of the National Academy of Sciences of the United States of America, 97, 4997–5002. DeAngelis, G. C., Freeman, R. D., & Ohzawa, I. (1994). Length and width tuning of neurons in the cat’s primary visual cortex. Journal of Neurophysiology, 71, 347–374. Derrington, A. M., Krauskopf, J., & Lennie, P. (1984). Chromatic mechanisms in lateral geniculate nucleus of macaque. Journal of Physiology (London), 357, 241–265. Dumoulin, S. O., & Wandell, B. A. (2008). Population receptive field estimates in human visual cortex. Neuroimage, 39, 647–660. Durant, S., & Clifford, C. W. G. (2006). Dynamics of the influence of segmentation cues on orientation perception. Vision Research, 46, 2934–2940. Engel, S. A. (2005). Adaptation of oriented and unoriented color-selective neurons in human visual areas. Neuron, 45, 613–623. Engel, S. A., Glover, G. H., & Wandell, B. A. (1997). Retinotopic organization in human visual cortex and the spatial precision of functional MRI. Cerebral Cortex, 7, 181–192. Fang, F., Murray, S. O., Kersten, D., & He, S. (2005). Orientation-tuned fMRI adaptation in human visual cortex. Journal of Neurophysiology, 94, 4188–4195. Forte, J. D., & Clifford, C. W. G. (2005). Inter-ocular transfer of the tilt illusion shows that monocular orientation mechanisms are colour selective. Vision Research, 45, 2715–2721. Furmanski, C. S., & Engel, S. A. (2000). An oblique effect in human primary visual cortex. Nature Neuroscience, 3, 535–536. Gegenfurtner, K. R., Kiper, D. C., & Fenstemaker, S. B. (1996). Processing of color, form, and motion in macaque area V2. Visual Neuroscience, 13, 161–172. Gegenfurtner, K. R., Kiper, D. C., & Levitt, J. B. (1997). Functional properties of neurons in macaque area V3. Journal of Neurophysiology, 77, 1906–1923. Gibson, J. J., & Radner, M. (1937). Adaptation, after-effect, and contrast in the perception of tilted lines. I. Quantitative studies. Journal of Experimental Psychology, 20, 453–467. Gilbert, C. D., & Wiesel, T. N. (1990). The influence of contextual stimuli on the orientation selectivity of cells in the primary visual cortex of the cat. Vision Research, 30, 1689–1701. Gulyas, B., Orban, G. A., Duysens, J., & Maes, H. (1987). The suppressive influence of moving textured backgrounds on responses of cat striate neurons to moving bars. Journal of Neurophysiology, 57, 1767–1791. Hubel, D. H., & Wiesel, T. N. (1962). Receptive fields, binocular interaction and functional architecture in the cat’s visual cortex. Journal of Physiology, 160, 106–154. Johnson, E. N., Hawken, M. J., & Shapley, R. (2001). The spatial transformation of color in the primary visual cortex of the macaque monkey. Nature Neuroscience, 4, 409–416.

J.S. McDonald et al. / Vision Research 49 (2009) 1397–1405 Jones, H. E., Grieve, K. L., Wang, W., & Sillito, A. M. (2001). Surround suppression in primate V1. Journal of Neurophysiology, 86, 2011–2028. Jones, H. E., Wang, W., & Sillito, A. M. (2002). Spatial organization and magnitude of orientation contrast interactions in primate V1. Journal of Neurophysiology, 88, 2796–2808. Kastner, S., De Weerd, P., Pinsk, M. A., Elizondo, M. I., Desimone, R., & Ungerleider, L. G. (2001). Modulation of sensory suppression: Implications for receptive field sizes in the human visual cortex. Journal of Neurophysiology, 86, 1398–1411. Kay, K. N., Naselaris, T., Prenger, R. J., & Gallant, J. L. (2008). Identifying natural images from human brain activity. Nature, 452, 352–355. Larsson, J., & Heeger, D. J. (2006). Two retinotopic visual areas in human lateral occipital cortex. Journal of Neuroscience, 26, 13128–13142. Larsson, J., Landy, M. S., & Heeger, D. J. (2006). Orientation-selective adaptation to first- and second-order patterns in human visual cortex. Journal of Neurophysiology, 95, 862–881. Lennie, P., Krauskopf, J., & Sclar, G. (1990). Chromatic mechanisms in striate cortex of macaque. Journal of Neuroscience, 10, 649–669. Leventhal, A. G., Thompson, K. G., Liu, D., Zhou, Y., & Ault, S. J. (1995). Concomitant sensitivity to orientation, direction, and color of cells in layers 2, 3, and 4 of monkey striate cortex. Journal of Neuroscience, 15, 1808–1818. Levitt, J. B., & Lund, J. S. (1997). Contrast dependence of contextual effects in primate visual cortex. Nature, 387, 73–76. Li, C.-Y., & Li, W. (1994). Extensive integration field beyond the classical receptive field of cat’s striate cortical neurons: Classification and tuning properties. Vision Research, 34, 2337–2355. Li, W., Thier, P., & Wehrhahn, C. (2000). Contextual influence on orientation discrimination of humans and responses of neurons in V1 of alert monkeys. Journal of Neurophysiology, 83, 941–954. Maffei, L., & Fiorentini, A. (1976). The unresponsive regions of visual cortical receptive fields. Vision Research, 16, 1131–1140. Montaser-Kouhsari, L., Landy, M. S., Heeger, D. J., & Larsson, J. (2007). Orientationselective adaptation to illusory contours in human visual cortex. Journal of Neuroscience, 27, 2186–2195. Nelson, J. I., & Frost, B. J. (1978). Orientation-selective inhibition from beyond the classic visual receptive field. Brain Research, 139, 359–365. O’Toole, B., & Wenderoth, P. (1977). The tilt illusion: Repulsion and attraction effects in the oblique meridian. Vision Research, 17, 367–374. Olzak, L. A., & Laurinen, P. I. (1999). Multiple gain control processes in contrastcontrast phenomena. Vision Research, 39, 3983–3987. Pitzalis, S., Galletti, C., Huang, R. S., Patria, F., Committeri, G., Galati, G., et al. (2006). Wide-field retinotopy defines human cortical visual area V6. Journal of Neuroscience, 26, 7962–7973. Polat, U., Mizobe, K., Pettet, M., Kasamatsu, T., & Norcia, A. (1998). Collinear stimuli regulate visual responses depending on cell’s contrast threshold. Nature, 391, 580–584.

1405

Schwartz, O., Hsu, A., & Dayan, P. (2007). Space and time in visual context. Nature Reviews Neuroscience, 4, 522–535. Sengpiel, F., Sen, A., & Blakemore, C. (1997). Characteristics of surround inhibition in cat area 17. Experimental Brain Research, 116, 216–228. Singer, B., & D’Zmura, M. (1994). Color contrast induction. Vision Research, 34, 3111–3126. Smith, E. L., 3rd, Chino, Y. M., Ridder, W. H., 3rd, Kitagawa, K., & Langston, A. (1990). Orientation bias of neurons in the lateral geniculate nucleus of macaque monkeys. Visual Neuroscience, 5, 525–545. Snowden, R. J., & Hammett, S. T. (1998). The effects of surround contrast on contrast thresholds, perceived contrast and contrast discrimination. Vision Research, 38, 1935–1945. Solomon, S. G., Peirce, J. W., & Lennie, P. (2004). The impact of suppressive surrounds on chromatic properties of cortical neurons. Journal of Neuroscience, 24, 148–160. Sumner, P., Anderson, E. J., Sylvester, R., Haynes, J.-D., & Rees, G. (2008). Combined orientation and colour information in human V1 for both L–M and S-cone chromatic axes. Neuroimage, 39, 814–824. Thorell, L. G., De Valois, R. L., & Albrecht, D. G. (1984). Spatial mapping of monkey V1 cells with pure color and luminance stimuli. Vision Research, 24, 751–769. Wade, J. N. (1980). The influence of colour and contour rivalry on the magnitude of the tilt illusion. Vision Research, 20, 229–233. Waldvogel, D., van Gelderen, P., Muellbacher, W., Ziemann, U., Immisch, I., & Hallett, M. (2000). The relative metabolic demand of inhibition and excitation. Nature, 406, 995–998. Walker, G. A., Ohzawa, I., & Freeman, R. D. (1999). Asymmetric suppression outside the classical receptive field of the visual cortex. Journal of Neuroscience, 19, 10536–10553. Webb, B. S., Barraclough, N. E., Parker, A., & Derrington, A. M. (2003). Gain control from beyond the classical receptive field in primate primary visual cortex. Visual Neuroscience, 20, 221–230. Wenderoth, P., Clifford, C. W. G., & Ma Wyatt, A. (2001). Hierarchy of spatial interactions in the processing of contrast-defined contours. Journal of the Optical Society of America A, 18, 2190–2196. Wenderoth, P., & Johnstone, S. (1987). Possible neural substrates for orientation analysis and perception. Perception, 16, 693–709. Williams, C. B., Singh, K. D., & Smith, A. T. (2003). Surround modulation measured with functional MRI in the human visual cortex. Journal of Neurophysiology, 89, 525–533. Xu, X., Ichida, J., Shostak, Y., Bonds, A. B., & Casagrande, V. A. (2002). Are primate lateral geniculate nucleus (LGN) cells really sensitive to orientation or direction? Visual Neuroscience, 19, 97–108. Xu, W.-F., Shen, Z.-M., & Li, C.-Y. (2005). Spatial phase sensitivity of V1 neurons in alert monkey. Cerebral Cortex, 15, 1697–1702. Zenger-Landolt, B., & Heeger, D. J. (2003). Response suppression in V1 agrees with psychophysics of surround masking. Journal of Neuroscience, 23, 6884–6893.