- Email: [email protected]
Contextual Modulation outside of Awareness
Current Biology, Vol. 15, 574–578, March 29, 2005, ©2005 Elsevier Ltd All rights reserved. DOI 10.1016/j.cub.2005.01.055
Contextual Modulation outside of Awareness
Colin W.G. Clifford* and Justin A. Harris School of Psychology The University of Sydney Sydney, New South Wales 2006 Australia
Summary Contextual effects are ubiquitous in vision and reveal fundamental principles of sensory coding. Here, we demonstrate that an oriented surround grating can affect the perceived orientation of a central test grating [1] even when backward masking [2] of the surround prevents its orientation from being consciously perceived. The effect survives introduction of a gap between test and surround of over a degree even under masking, suggesting either that contextual information can effectively propagate across early visual cortex in the absence of awareness of the signaled context or that it can proceed undetected to higher processing levels at which such horizontal propagation may not be necessary. The effect under masking also shows partial interocular transfer, demonstrating processing of orientation by binocular neurons in visual cortex in the absence of conscious orientation perception. This pattern of results is consistent with the suggestion that simultaneous orientation contrast is mediated at multiple levels of the visual processing hierarchy [3–6], and it supports the view that propagation of signals to [7] and, possibly, back from [8–10] higher visual areas is necessary for conscious perception. Results and Discussion Previous studies suggest that, in visual cortex, a level of activity can be induced that is sufficient to generate adaptation but insufficient to produce a conscious percept [11–14]. Although this previous work can be thought of as mapping out an operating characteristic between neural activity (as gauged by adaptation) and consciousness, it is not sufficient to rule out the existence of a single process underlying adaptation and conscious perception. For example, our conscious perception could monitor activity in a given region of cortex as long as the rate of activity exceeded some baseline amount, whereas the activity threshold necessary to generate adaptation could be somewhat lower. Binocular rivalry has been used to demonstrate that intermittent suppression of an adapting stimulus does not always reduce its efficacy, suggesting that suppression from awareness is occurring at a site in the processing hierarchy after that mediating adaptation [15]. However, caution must be exercised in interpreting this result, which could simply reflect saturation of adaptation during the intermittent periods of stimulus domi*Correspondence: [email protected]
nance [16]. When binocular rivalry has been used to suppress a simultaneous inducing stimulus, results have been equivocal [17, 18]. Here, we demonstrate that the operating characteristic between neural activity (as gauged by simultaneous orientation contrast [1]) and conscious perception varies as a function of the conditions of stimulus presentation. Specifically, we show that an oriented surround grating affects the perceived orientation of a central test grating even when backward masking of the surround completely prevents its orientation from being consciously perceived. Perceptually, a vertical test grating appears repelled in orientation from that of a 15° surround grating (Figure 1A). The magnitude of this simultaneous orientation contrast for six subjects (two authors, C.C. and J.H., and four observers naive to the purposes of the study) in unmasked and masked conditions is shown in Figure 1B. The mean (± standard error) magnitude of simultaneous contrast across subjects in the unmasked and masked conditions was 3.50 ± 0.47 and 1.42 ± 0.37, respectively. Although the magnitude of the effect in the masked condition was significantly smaller than in the unmasked condition (t5 = 3.48, p < 0.01), it was still significantly greater than zero (t5 = 3.84, p < 0.01). In separate experimental runs, we measured perception of the surround in the masked and unmasked conditions. In the masked condition, with a randomly oriented center stimulus, subjects consistently performed at chance in a single-interval two-alternative forcedchoice judgment of the orientation of the surrounding stimulus (Figure 1C). Performance at chance level in this forced-choice task demonstrates that perceptual processing of the orientation of the surround was not merely subliminal, in the sense of being below some arbitrary response criterion—it was strictly unconscious. With a mask but a blank or horizontal center stimulus, performance was still close to chance (Figure 1D), leading us to conclude that the principal limit on perception is backward masking of the inducing stimulus [2] rather than response competition between the center and surround orientations [19]. Backward masking might operate either by interrupting processing of the inducing stimulus [2] or by disrupting recurrent interactions between higher and lower visual areas [8]. Primary visual cortex (V1) is the first stage of the visual processing hierarchy to contain cells selective for stimulus orientation [20]. Thus, the neural substrate of simultaneous orientation contrast must be cortical, involving interactions between neurons at or beyond the level of V1 [3]. It has been reported that the responses of individual neurons in V1 of monkeys are not correlated with perception in that they signal stereoscopic disparity rather than perceived depth [21], chromatic flicker beyond the perceptual resolution limit [22], and eye blinks [23], and only a minority of neurons in primary visual cortex show activity correlated with perception during binocular rivalry [24]. However, brain imaging studies of binocular rivalry in humans have found that modulations of hemodynamic activity in V1 track
Contextual Modulation outside of Awareness 575
Figure 1. Simultaneous Orientation Contrast (A) A surround grating oriented at or around 15° from the vertical causes an objectively vertical central stimulus to appear tilted in the opposite direction. In the actual experiments the relative sizes of center and surround were as illustrated, but the grating spatial frequency was much higher, such that the center contained 24 cycles of the grating and the surround contained 120. (B) Magnitude of simultaneous orientation contrast for six subjects in unmasked and masked conditions. (C) Percentage correct identification of surround orientation for the same six subjects in the presence of a randomly oriented central grating. (D) Percentage correct identification of surround orientation for three of the subjects with randomly oriented, horizontal, and blank central stimuli. Error bars in all figures indicate plus and minus one standard error of the mean.
perception [25, 26], consistent with the notion that perception monitors processing in early visual areas [27]. Recent studies from the first author’s laboratory have used manipulations of stimulus color to demonstrate that simultaneous orientation contrast is mediated by processing at multiple levels of visual cortex [4–6]. Orientation-specific interactions at the earliest, monocular level are selective for stimulus color, whereas processing at subsequent binocular levels is essentially cue invariant [6]. The net simultaneous orientation contrast effect presumably reflects an aggregate of these interactions across the various levels. If perception of orientation also monitors neural activity across these areas, then the effects of masking on orientation perception and simultaneous orientation contrast should be constrained by the operating characteristic relating these two measures in the unmasked condition. Specifically, the effect of masking on perception of the surround and on the magnitude of simultaneous orientation contrast would be to reduce effective stimulus strength without changing the operating characteristic relating the two measures. However, our results show that the effect of masking upon perception of the orientation of the surround is much greater than that predicted from its effect on the magnitude of simultaneous orientation contrast (Figure 2). This indicates that activity in early visual cortical areas may not contribute directly to the content of conscious perception. Instead, our results are consistent with the suggestion that simultaneous contrast is mediated at multiple levels of the visual processing hierarchy [3–6]. Under masking, the generally observed reduction in magnitude of simultaneous orientation contrast suggests that a component of the unmasked effect involves areas whose activity normally contributes to conscious perception of orientation. However, the existence of a simultaneous orientation contrast effect even when masking renders the orientation of the inducing grating invisible demonstrates the involve-
ment of orientation processing mechanisms whose activity is not consciously perceived. We carried out two further experiments to compare the effects of simultaneous contrast in masked and unmasked conditions. First, a gap was introduced between the center and surround stimulus. The magnitude of simultaneous contrast was found to drop off in a similar fashion in the two conditions (Figure 3). This pattern of results confirms that the effect observed under masking is qualitatively similar to the unmasked effect. Second, simultaneous orientation contrast was measured under dichoptic presentation of test and inducer followed by a binocular mask. Orientation contrast in the absence of awareness of the orientation of the inducing stimulus was again observed (Figure 4). Such interocular interactions implicate processing by binocular neurons that are first found in V1, demonstrating that certain network interactions in visual cortex are not of themselves sufficient to support visual awareness. It has been reported that the contextual modulation evident in the late (>100 ms) component of the responses of V1 neurons of awake monkeys is absent when the stimuli are not consciously perceived [28]. Dissociation between neuronal modulation and behavioral report is found only when the monkeys’ reward regime encourages them to adopt a conservative response criterion, consistent with the operation of a simple decision mechanism after initial sensory processing. In contrast, we find that a criterion-independent forced-choice procedure reveals marked contextual effects on the perceived orientation of a test stimulus in the complete absence of awareness of the inducing orientation. This is significant because it indicates that spatial interactions specific to stimulus orientation, and thus involving neural activity at a level as least as high as V1, can proceed in the absence of awareness. Our finding that the operating characteristic between neural activity (as gauged by simultaneous orientation con-
Current Biology 576
Figure 2. Simultaneous Orientation Contrast and Surround Identification at a Range of Surround Contrasts Each row depicts data from a single subject. Center contrasts were 25% for subject J.H., 50% for C.C., and 100% for C.M. Data from unmasked (filled circles) and masked (unfilled squares) conditions show the magnitude of simultaneous orientation contrast (left column) and percentage correct identification of surround orientation (middle column) as a function of surround contrast. The right-hand column shows “operating characteristics” for the two conditions obtained by plotting surround identification performance against measures of simultaneous orientation contrast from comparable conditions.
trast) and conscious perception differs markedly between masked and unmasked presentation conditions cannot be accounted for by a thresholding mechanism operating on the output of early sensory processing. Instead, it demonstrates that the activity of at least some of the neural processes mediating simultaneous orientation contrast is not accessible to conscious orientation perception. What is the essential difference between the pro-
cesses underlying conscious orientation perception and the processes mediating simultaneous orientation contrast? The difference could be an anatomical one, such that processing in visual areas higher than those mediating simultaneous orientation contrast under backward masking is required for conscious vision [7]. In this case, the effect of masking would be to disrupt processing disproportionately in those higher areas. Alternatively, the difference could be essentially tempoFigure 3. The Effect on Simultaneous Orientation Contrast of Introducing a Gap between Center and Surround Stimuli Data from two subjects for unmasked (filled circles) and masked (unfilled squares) conditions show the magnitude of simultaneous orientation contrast (left column) as a function of gap width. The right-hand column shows for each condition the data normalized to the value with no gap.
Contextual Modulation outside of Awareness 577
Figure 4. The Effect of Dichoptic Presentation on Simultaneous Orientation Contrast Data from three subjects for unmasked (left pairs of bars) and masked (right pairs of bars) conditions show the magnitude of simultaneous orientation contrast for binocular (black bars) and dichoptic (white bars) presentation. Figures below each pair of bars denote magnitude of dichoptic effect as a percentage of the corresponding binocular effect.
ral, such that masking truncates processing of the inducing stimulus. A temporal effect would be consistent with the observation that only a late (>100 ms) component of V1 activity correlates with perception [28]. Presentation of a mask 50 ms after stimulus onset could then interfere selectively with this late response component. These two alternatives are not mutually exclusive. For example, the late component of the V1 response might be the result of reentrant propagation of signals from higher visual areas [8]. Interactive models of visual awareness propose that consciousness is essentially a network property involving interactions between coalitions of neurons within or across cortical regions [8–10]. It is possible that effects of cortical adaptation in absence of awareness of the adapting stimulus [11–15] are generated intrinsically within the adapted neurons in the absence of interactions at the cortical level. In this case, it would be wrong to conclude that the adapted neurons are not an integral part of a fundamentally distributed neural correlate of consciousness [16]. Our results demonstrate that certain interactions within early cortical areas are not of themselves sufficient to support awareness. Instead, they are consistent with the view that propagation of signals to, and possibly back from, higher visual areas is necessary for conscious perception. Experimental Procedures Stimuli were generated with Matlab software to drive a VSG 2/5 graphics card (Cambridge Research Systems) and were displayed on a γ-corrected 21” Sony Trinitron GM 520 monitor (1024 × 768 resolution; 120 Hz refresh rate). The background was a uniform gray field with a luminance of 62.8 Cd/m2, and a central fixation marker was present at all times. The stimulus used to generate simultaneous orientation contrast consisted of a circular patch of test grating (diameter: 3°) surrounded by a concentric annulus (outer diameter: 15°) containing an inducing grating oriented 15° clockwise or counterclockwise from vertical. Both center and surround gratings had random spatial phase. The magnitude of the simultaneous orientation contrast effect was measured by instructing subjects to make a forcedchoice judgment of the perceived orientation of the center grating (clockwise or counterclockwise from vertical) while varying its orientation between trials. Trials containing clockwise and counterclockwise inducing gratings were randomly interleaved to avoid the build up of adaptation to any one surround orientation across trials. Interleaved adaptive psychophysical procedures [29] of 30 trials each were used to obtain estimates of perceived subjective vertical
for the clockwise and counterclockwise surround orientations. The magnitude of simultaneous orientation contrast was taken as half the difference between the two points of subjective vertical. Simultaneous orientation contrast was measured in two conditions: masked and unmasked. Center and surround stimuli were 8 cycles per degree sine wave gratings modulated in luminance around that of the background. They were ramped on and off together inside a temporal window of 50 ms total duration. In the masked condition, stimulus presentation was followed immediately by a random noise mask, covering the surround annulus but not the center, that remained on the screen until the subject indicated a response. Subjects were required to make a forced-choice judgment of the perceived orientation of the surround grating (clockwise or counterclockwise from vertical) in separate experimental runs to measure perception of the surround. The stimuli were identical to those used to measure simultaneous orientation contrast, except that the orientation of the center stimulus was randomly chosen every trial from a distribution uniform over the full 180° range. Randomization of the orientation of the center ensures that its perceived orientation cannot provide a cue to the orientation of the surround grating. For example, if the center stimulus is always vertical, then any simultaneous orientation contrast would tend to make it appear tilted away from the physical orientation of the surround, so subjects would be able to infer the orientation of the surround even when it was invisible.
Acknowledgments This work is supported by a Discovery Project Grant and Queen Elizabeth II Fellowship to C.C. from the Australian Research Council. We are grateful to Branka Spehar and Irina Harris for helpful discussions during the course of this work and to two anonymous reviewers for their constructive criticism of an earlier version of this manuscript. Received: November 22, 2004 Revised: January 13, 2005 Accepted: January 31, 2005 Published: March 29, 2005 References 1. Gibson, J.J., and Radner, M. (1937). Adaptation, after-effect, and contrast in the perception of tilted lines. I. Quantitative studies. J. Exp. Psychol. 20, 453–467. 2. Kahneman, D. (1968). Method, findings, and theory in studies of visual masking. Psychol. Bull. 70, 404–425. 3. Wenderoth, P., and Johnstone, S. (1987). Possible neural substrates for orientation analysis and perception. Perception 16, 693–709. 4. Clifford, C.W.G., Spehar, B., Solomon, S.G., Martin, P.R., and
Current Biology 578
5.
6.
7. 8.
9.
10.
11. 12.
13. 14.
15.
16.
17. 18. 19.
20.
21.
22.
23. 24.
25.
26.
27. 28.
29.
Zaidi, Q. (2003). Interactions between colour and luminance in the perception of orientation. J. Vis. 3, 106–115. Clifford, C.W.G., Pearson, J., Forte, J.D., and Spehar, B. (2003). Colour and luminance selectivity of spatial and temporal interactions in orientation perception. Vision Res. 43, 2885–2893. Forte, J.D., and Clifford, C.W.G. (2004). Interocular transfer of the tilt illusion shows that monocular orientation mechanisms are colour selective. J. Vis. 4, 785a. Crick, F., and Koch, C. (1995). Are we aware of neural activity in primary visual cortex? Nature 375, 121–123. Lamme, V.A., and Roelfsema, P.R. (2000). The distinct modes of vision offered by feedforward and recurrent processing. Trends Neurosci. 23, 571–579. Pascual-Leone, A., and Walsh, V. (2001). Fast backprojections from the motion to the primary visual area necessary for visual awareness. Science 292, 510–512. Hochstein, S., and Ahissar, M. (2002). View from the top: Hierarchies and reverse hierarchies in the visual system. Neuron 36, 791–804. He, S., Cavanagh, P., and Intriligator, J. (1996). Attentional resolution and the locus of visual awareness. Nature 383, 334–337. He, S., and MacLeod, D.I. (2001). Orientation-selective adaptation and tilt after-effect from invisible patterns. Nature 411, 473–476. Rajimehr, R. (2004). Unconscious orientation processing. Neuron 41, 663–673. Shady, S., MacLeod, D.I., and Fisher, H.S. (2004). Adaptation from invisible flicker. Proc. Natl. Acad. Sci. USA 101, 5170– 5173. Blake, R., and Fox, R. (1974). Adaptation to “invisible” gratings and the site of binocular rivalry suppression. Nature 249, 488– 490. Blake, R., and He, S. (2005). Adaptation as a tool for probing the neural correlates of visual awareness: Progress and precautions. In Fitting the Mind to the World: Aftereffects in HighLevel Vision, C.W.G. Clifford and G. Rhodes, eds. (Oxford: Oxford University Press), pp. 281–307. Rao, V.M. (1977). Tilt illusion during binocular rivalry. Vision Res. 17, 327–328. Wade, N.J. (1980). The influence of colour and contour rivalry on the magnitude of the tilt illusion. Vision Res. 20, 229–233. Evans, P.M., and Craig, J.C. (1992). Response competition: A major source of interference in a tactile identification task. Percept. Psychophys. 51, 199–206. Hubel, D.H., and Wiesel, T.N. (1962). Receptive fields, binocular interaction and functional architecture in the cat’s visual cortex. J. Physiol. 160, 106–154. Cumming, B.G., and Parker, A.J. (1997). Responses of primary visual cortical neurons to binocular disparity without depth perception. Nature 389, 280–283. Gur, M., and Snodderly, D.M. (1997). A dissociation between brain activity and perception: Chromatically opponent cortical neurons signal chromatic flicker that is not perceived. Vision Res. 37, 377–382. Gawne, T.J., and Martin, J.M. (2000). Activity of primate V1 cortical neurons during blinks. J. Neurophysiol. 84, 2691–2694. Leopold, D.A., and Logothetis, N.K. (1996). Activity changes in early visual cortex reflect monkeys’ percepts during binocular rivalry. Nature 379, 549–553. Polonsky, A., Blake, R., Braun, J., and Heeger, D.J. (2000). Neuronal activity in human primary visual cortex correlates with perception during binocular rivalry. Nat. Neurosci. 3, 1153– 1159. Tong, F., and Engel, S.A. (2001). Interocular rivalry revealed in the human cortical blind-spot representation. Nature 411, 195–199. Zeki, S., and Bartels, A. (1998). The asynchrony of consciousness. Proc. R. Soc. Lond. B. Biol. Sci. 265, 1583–1585. Super, H., Spekreijse, H., and Lamme, V.A. (2001). Two distinct modes of sensory processing observed in monkey primary visual cortex (V1). Nat. Neurosci. 4, 304–310. Kontsevich, L.L., and Tyler, C.W. (1999). Bayesian adaptive estimation of psychometric slope and threshold. Vision Res. 39, 2729–2737.
Contextual Modulation outside of Awareness
Colin W.G. Clifford* and Justin A. Harris School of Psychology The University of Sydney Sydney, New South Wales 2006 Australia
Summary Contextual effects are ubiquitous in vision and reveal fundamental principles of sensory coding. Here, we demonstrate that an oriented surround grating can affect the perceived orientation of a central test grating [1] even when backward masking [2] of the surround prevents its orientation from being consciously perceived. The effect survives introduction of a gap between test and surround of over a degree even under masking, suggesting either that contextual information can effectively propagate across early visual cortex in the absence of awareness of the signaled context or that it can proceed undetected to higher processing levels at which such horizontal propagation may not be necessary. The effect under masking also shows partial interocular transfer, demonstrating processing of orientation by binocular neurons in visual cortex in the absence of conscious orientation perception. This pattern of results is consistent with the suggestion that simultaneous orientation contrast is mediated at multiple levels of the visual processing hierarchy [3–6], and it supports the view that propagation of signals to [7] and, possibly, back from [8–10] higher visual areas is necessary for conscious perception. Results and Discussion Previous studies suggest that, in visual cortex, a level of activity can be induced that is sufficient to generate adaptation but insufficient to produce a conscious percept [11–14]. Although this previous work can be thought of as mapping out an operating characteristic between neural activity (as gauged by adaptation) and consciousness, it is not sufficient to rule out the existence of a single process underlying adaptation and conscious perception. For example, our conscious perception could monitor activity in a given region of cortex as long as the rate of activity exceeded some baseline amount, whereas the activity threshold necessary to generate adaptation could be somewhat lower. Binocular rivalry has been used to demonstrate that intermittent suppression of an adapting stimulus does not always reduce its efficacy, suggesting that suppression from awareness is occurring at a site in the processing hierarchy after that mediating adaptation [15]. However, caution must be exercised in interpreting this result, which could simply reflect saturation of adaptation during the intermittent periods of stimulus domi*Correspondence: [email protected]
nance [16]. When binocular rivalry has been used to suppress a simultaneous inducing stimulus, results have been equivocal [17, 18]. Here, we demonstrate that the operating characteristic between neural activity (as gauged by simultaneous orientation contrast [1]) and conscious perception varies as a function of the conditions of stimulus presentation. Specifically, we show that an oriented surround grating affects the perceived orientation of a central test grating even when backward masking of the surround completely prevents its orientation from being consciously perceived. Perceptually, a vertical test grating appears repelled in orientation from that of a 15° surround grating (Figure 1A). The magnitude of this simultaneous orientation contrast for six subjects (two authors, C.C. and J.H., and four observers naive to the purposes of the study) in unmasked and masked conditions is shown in Figure 1B. The mean (± standard error) magnitude of simultaneous contrast across subjects in the unmasked and masked conditions was 3.50 ± 0.47 and 1.42 ± 0.37, respectively. Although the magnitude of the effect in the masked condition was significantly smaller than in the unmasked condition (t5 = 3.48, p < 0.01), it was still significantly greater than zero (t5 = 3.84, p < 0.01). In separate experimental runs, we measured perception of the surround in the masked and unmasked conditions. In the masked condition, with a randomly oriented center stimulus, subjects consistently performed at chance in a single-interval two-alternative forcedchoice judgment of the orientation of the surrounding stimulus (Figure 1C). Performance at chance level in this forced-choice task demonstrates that perceptual processing of the orientation of the surround was not merely subliminal, in the sense of being below some arbitrary response criterion—it was strictly unconscious. With a mask but a blank or horizontal center stimulus, performance was still close to chance (Figure 1D), leading us to conclude that the principal limit on perception is backward masking of the inducing stimulus [2] rather than response competition between the center and surround orientations [19]. Backward masking might operate either by interrupting processing of the inducing stimulus [2] or by disrupting recurrent interactions between higher and lower visual areas [8]. Primary visual cortex (V1) is the first stage of the visual processing hierarchy to contain cells selective for stimulus orientation [20]. Thus, the neural substrate of simultaneous orientation contrast must be cortical, involving interactions between neurons at or beyond the level of V1 [3]. It has been reported that the responses of individual neurons in V1 of monkeys are not correlated with perception in that they signal stereoscopic disparity rather than perceived depth [21], chromatic flicker beyond the perceptual resolution limit [22], and eye blinks [23], and only a minority of neurons in primary visual cortex show activity correlated with perception during binocular rivalry [24]. However, brain imaging studies of binocular rivalry in humans have found that modulations of hemodynamic activity in V1 track
Contextual Modulation outside of Awareness 575
Figure 1. Simultaneous Orientation Contrast (A) A surround grating oriented at or around 15° from the vertical causes an objectively vertical central stimulus to appear tilted in the opposite direction. In the actual experiments the relative sizes of center and surround were as illustrated, but the grating spatial frequency was much higher, such that the center contained 24 cycles of the grating and the surround contained 120. (B) Magnitude of simultaneous orientation contrast for six subjects in unmasked and masked conditions. (C) Percentage correct identification of surround orientation for the same six subjects in the presence of a randomly oriented central grating. (D) Percentage correct identification of surround orientation for three of the subjects with randomly oriented, horizontal, and blank central stimuli. Error bars in all figures indicate plus and minus one standard error of the mean.
perception [25, 26], consistent with the notion that perception monitors processing in early visual areas [27]. Recent studies from the first author’s laboratory have used manipulations of stimulus color to demonstrate that simultaneous orientation contrast is mediated by processing at multiple levels of visual cortex [4–6]. Orientation-specific interactions at the earliest, monocular level are selective for stimulus color, whereas processing at subsequent binocular levels is essentially cue invariant [6]. The net simultaneous orientation contrast effect presumably reflects an aggregate of these interactions across the various levels. If perception of orientation also monitors neural activity across these areas, then the effects of masking on orientation perception and simultaneous orientation contrast should be constrained by the operating characteristic relating these two measures in the unmasked condition. Specifically, the effect of masking on perception of the surround and on the magnitude of simultaneous orientation contrast would be to reduce effective stimulus strength without changing the operating characteristic relating the two measures. However, our results show that the effect of masking upon perception of the orientation of the surround is much greater than that predicted from its effect on the magnitude of simultaneous orientation contrast (Figure 2). This indicates that activity in early visual cortical areas may not contribute directly to the content of conscious perception. Instead, our results are consistent with the suggestion that simultaneous contrast is mediated at multiple levels of the visual processing hierarchy [3–6]. Under masking, the generally observed reduction in magnitude of simultaneous orientation contrast suggests that a component of the unmasked effect involves areas whose activity normally contributes to conscious perception of orientation. However, the existence of a simultaneous orientation contrast effect even when masking renders the orientation of the inducing grating invisible demonstrates the involve-
ment of orientation processing mechanisms whose activity is not consciously perceived. We carried out two further experiments to compare the effects of simultaneous contrast in masked and unmasked conditions. First, a gap was introduced between the center and surround stimulus. The magnitude of simultaneous contrast was found to drop off in a similar fashion in the two conditions (Figure 3). This pattern of results confirms that the effect observed under masking is qualitatively similar to the unmasked effect. Second, simultaneous orientation contrast was measured under dichoptic presentation of test and inducer followed by a binocular mask. Orientation contrast in the absence of awareness of the orientation of the inducing stimulus was again observed (Figure 4). Such interocular interactions implicate processing by binocular neurons that are first found in V1, demonstrating that certain network interactions in visual cortex are not of themselves sufficient to support visual awareness. It has been reported that the contextual modulation evident in the late (>100 ms) component of the responses of V1 neurons of awake monkeys is absent when the stimuli are not consciously perceived [28]. Dissociation between neuronal modulation and behavioral report is found only when the monkeys’ reward regime encourages them to adopt a conservative response criterion, consistent with the operation of a simple decision mechanism after initial sensory processing. In contrast, we find that a criterion-independent forced-choice procedure reveals marked contextual effects on the perceived orientation of a test stimulus in the complete absence of awareness of the inducing orientation. This is significant because it indicates that spatial interactions specific to stimulus orientation, and thus involving neural activity at a level as least as high as V1, can proceed in the absence of awareness. Our finding that the operating characteristic between neural activity (as gauged by simultaneous orientation con-
Current Biology 576
Figure 2. Simultaneous Orientation Contrast and Surround Identification at a Range of Surround Contrasts Each row depicts data from a single subject. Center contrasts were 25% for subject J.H., 50% for C.C., and 100% for C.M. Data from unmasked (filled circles) and masked (unfilled squares) conditions show the magnitude of simultaneous orientation contrast (left column) and percentage correct identification of surround orientation (middle column) as a function of surround contrast. The right-hand column shows “operating characteristics” for the two conditions obtained by plotting surround identification performance against measures of simultaneous orientation contrast from comparable conditions.
trast) and conscious perception differs markedly between masked and unmasked presentation conditions cannot be accounted for by a thresholding mechanism operating on the output of early sensory processing. Instead, it demonstrates that the activity of at least some of the neural processes mediating simultaneous orientation contrast is not accessible to conscious orientation perception. What is the essential difference between the pro-
cesses underlying conscious orientation perception and the processes mediating simultaneous orientation contrast? The difference could be an anatomical one, such that processing in visual areas higher than those mediating simultaneous orientation contrast under backward masking is required for conscious vision [7]. In this case, the effect of masking would be to disrupt processing disproportionately in those higher areas. Alternatively, the difference could be essentially tempoFigure 3. The Effect on Simultaneous Orientation Contrast of Introducing a Gap between Center and Surround Stimuli Data from two subjects for unmasked (filled circles) and masked (unfilled squares) conditions show the magnitude of simultaneous orientation contrast (left column) as a function of gap width. The right-hand column shows for each condition the data normalized to the value with no gap.
Contextual Modulation outside of Awareness 577
Figure 4. The Effect of Dichoptic Presentation on Simultaneous Orientation Contrast Data from three subjects for unmasked (left pairs of bars) and masked (right pairs of bars) conditions show the magnitude of simultaneous orientation contrast for binocular (black bars) and dichoptic (white bars) presentation. Figures below each pair of bars denote magnitude of dichoptic effect as a percentage of the corresponding binocular effect.
ral, such that masking truncates processing of the inducing stimulus. A temporal effect would be consistent with the observation that only a late (>100 ms) component of V1 activity correlates with perception [28]. Presentation of a mask 50 ms after stimulus onset could then interfere selectively with this late response component. These two alternatives are not mutually exclusive. For example, the late component of the V1 response might be the result of reentrant propagation of signals from higher visual areas [8]. Interactive models of visual awareness propose that consciousness is essentially a network property involving interactions between coalitions of neurons within or across cortical regions [8–10]. It is possible that effects of cortical adaptation in absence of awareness of the adapting stimulus [11–15] are generated intrinsically within the adapted neurons in the absence of interactions at the cortical level. In this case, it would be wrong to conclude that the adapted neurons are not an integral part of a fundamentally distributed neural correlate of consciousness [16]. Our results demonstrate that certain interactions within early cortical areas are not of themselves sufficient to support awareness. Instead, they are consistent with the view that propagation of signals to, and possibly back from, higher visual areas is necessary for conscious perception. Experimental Procedures Stimuli were generated with Matlab software to drive a VSG 2/5 graphics card (Cambridge Research Systems) and were displayed on a γ-corrected 21” Sony Trinitron GM 520 monitor (1024 × 768 resolution; 120 Hz refresh rate). The background was a uniform gray field with a luminance of 62.8 Cd/m2, and a central fixation marker was present at all times. The stimulus used to generate simultaneous orientation contrast consisted of a circular patch of test grating (diameter: 3°) surrounded by a concentric annulus (outer diameter: 15°) containing an inducing grating oriented 15° clockwise or counterclockwise from vertical. Both center and surround gratings had random spatial phase. The magnitude of the simultaneous orientation contrast effect was measured by instructing subjects to make a forcedchoice judgment of the perceived orientation of the center grating (clockwise or counterclockwise from vertical) while varying its orientation between trials. Trials containing clockwise and counterclockwise inducing gratings were randomly interleaved to avoid the build up of adaptation to any one surround orientation across trials. Interleaved adaptive psychophysical procedures [29] of 30 trials each were used to obtain estimates of perceived subjective vertical
for the clockwise and counterclockwise surround orientations. The magnitude of simultaneous orientation contrast was taken as half the difference between the two points of subjective vertical. Simultaneous orientation contrast was measured in two conditions: masked and unmasked. Center and surround stimuli were 8 cycles per degree sine wave gratings modulated in luminance around that of the background. They were ramped on and off together inside a temporal window of 50 ms total duration. In the masked condition, stimulus presentation was followed immediately by a random noise mask, covering the surround annulus but not the center, that remained on the screen until the subject indicated a response. Subjects were required to make a forced-choice judgment of the perceived orientation of the surround grating (clockwise or counterclockwise from vertical) in separate experimental runs to measure perception of the surround. The stimuli were identical to those used to measure simultaneous orientation contrast, except that the orientation of the center stimulus was randomly chosen every trial from a distribution uniform over the full 180° range. Randomization of the orientation of the center ensures that its perceived orientation cannot provide a cue to the orientation of the surround grating. For example, if the center stimulus is always vertical, then any simultaneous orientation contrast would tend to make it appear tilted away from the physical orientation of the surround, so subjects would be able to infer the orientation of the surround even when it was invisible.
Acknowledgments This work is supported by a Discovery Project Grant and Queen Elizabeth II Fellowship to C.C. from the Australian Research Council. We are grateful to Branka Spehar and Irina Harris for helpful discussions during the course of this work and to two anonymous reviewers for their constructive criticism of an earlier version of this manuscript. Received: November 22, 2004 Revised: January 13, 2005 Accepted: January 31, 2005 Published: March 29, 2005 References 1. Gibson, J.J., and Radner, M. (1937). Adaptation, after-effect, and contrast in the perception of tilted lines. I. Quantitative studies. J. Exp. Psychol. 20, 453–467. 2. Kahneman, D. (1968). Method, findings, and theory in studies of visual masking. Psychol. Bull. 70, 404–425. 3. Wenderoth, P., and Johnstone, S. (1987). Possible neural substrates for orientation analysis and perception. Perception 16, 693–709. 4. Clifford, C.W.G., Spehar, B., Solomon, S.G., Martin, P.R., and
Current Biology 578
5.
6.
7. 8.
9.
10.
11. 12.
13. 14.
15.
16.
17. 18. 19.
20.
21.
22.
23. 24.
25.
26.
27. 28.
29.
Zaidi, Q. (2003). Interactions between colour and luminance in the perception of orientation. J. Vis. 3, 106–115. Clifford, C.W.G., Pearson, J., Forte, J.D., and Spehar, B. (2003). Colour and luminance selectivity of spatial and temporal interactions in orientation perception. Vision Res. 43, 2885–2893. Forte, J.D., and Clifford, C.W.G. (2004). Interocular transfer of the tilt illusion shows that monocular orientation mechanisms are colour selective. J. Vis. 4, 785a. Crick, F., and Koch, C. (1995). Are we aware of neural activity in primary visual cortex? Nature 375, 121–123. Lamme, V.A., and Roelfsema, P.R. (2000). The distinct modes of vision offered by feedforward and recurrent processing. Trends Neurosci. 23, 571–579. Pascual-Leone, A., and Walsh, V. (2001). Fast backprojections from the motion to the primary visual area necessary for visual awareness. Science 292, 510–512. Hochstein, S., and Ahissar, M. (2002). View from the top: Hierarchies and reverse hierarchies in the visual system. Neuron 36, 791–804. He, S., Cavanagh, P., and Intriligator, J. (1996). Attentional resolution and the locus of visual awareness. Nature 383, 334–337. He, S., and MacLeod, D.I. (2001). Orientation-selective adaptation and tilt after-effect from invisible patterns. Nature 411, 473–476. Rajimehr, R. (2004). Unconscious orientation processing. Neuron 41, 663–673. Shady, S., MacLeod, D.I., and Fisher, H.S. (2004). Adaptation from invisible flicker. Proc. Natl. Acad. Sci. USA 101, 5170– 5173. Blake, R., and Fox, R. (1974). Adaptation to “invisible” gratings and the site of binocular rivalry suppression. Nature 249, 488– 490. Blake, R., and He, S. (2005). Adaptation as a tool for probing the neural correlates of visual awareness: Progress and precautions. In Fitting the Mind to the World: Aftereffects in HighLevel Vision, C.W.G. Clifford and G. Rhodes, eds. (Oxford: Oxford University Press), pp. 281–307. Rao, V.M. (1977). Tilt illusion during binocular rivalry. Vision Res. 17, 327–328. Wade, N.J. (1980). The influence of colour and contour rivalry on the magnitude of the tilt illusion. Vision Res. 20, 229–233. Evans, P.M., and Craig, J.C. (1992). Response competition: A major source of interference in a tactile identification task. Percept. Psychophys. 51, 199–206. Hubel, D.H., and Wiesel, T.N. (1962). Receptive fields, binocular interaction and functional architecture in the cat’s visual cortex. J. Physiol. 160, 106–154. Cumming, B.G., and Parker, A.J. (1997). Responses of primary visual cortical neurons to binocular disparity without depth perception. Nature 389, 280–283. Gur, M., and Snodderly, D.M. (1997). A dissociation between brain activity and perception: Chromatically opponent cortical neurons signal chromatic flicker that is not perceived. Vision Res. 37, 377–382. Gawne, T.J., and Martin, J.M. (2000). Activity of primate V1 cortical neurons during blinks. J. Neurophysiol. 84, 2691–2694. Leopold, D.A., and Logothetis, N.K. (1996). Activity changes in early visual cortex reflect monkeys’ percepts during binocular rivalry. Nature 379, 549–553. Polonsky, A., Blake, R., Braun, J., and Heeger, D.J. (2000). Neuronal activity in human primary visual cortex correlates with perception during binocular rivalry. Nat. Neurosci. 3, 1153– 1159. Tong, F., and Engel, S.A. (2001). Interocular rivalry revealed in the human cortical blind-spot representation. Nature 411, 195–199. Zeki, S., and Bartels, A. (1998). The asynchrony of consciousness. Proc. R. Soc. Lond. B. Biol. Sci. 265, 1583–1585. Super, H., Spekreijse, H., and Lamme, V.A. (2001). Two distinct modes of sensory processing observed in monkey primary visual cortex (V1). Nat. Neurosci. 4, 304–310. Kontsevich, L.L., and Tyler, C.W. (1999). Bayesian adaptive estimation of psychometric slope and threshold. Vision Res. 39, 2729–2737.