- Email: [email protected]
fMRI studies of visual cortical activity during noise stimulation
Neurocomputing 26}27 (1999) 511}516
fMRI studies of visual cortical activity during noise stimulation夽 E. Simonotto *, F. Spano, M. Riani, A. Ferrari, F. Levrero, A. Pilot, P. Renzetti, R.C. Parodi, F. Sardanelli, P. Vitali, J. Twitty , F. Chiou-Tan, F. Moss Department of Physics and Astronomy, Center for Neurodynamics, University of Missouri at St. Louis, St. Louis, 8001 Natural Bridge Road, MO 63121, USA INFM-UdR Genova, Via Dodecaneso, 33, 16146 Genova, Italy Servizio di Fisica Medica, Ospedale S. Martino, Largo R. Benzi, 16132 Genova, Italy Istituto di Radiologia, Universita& di Genova, Largo R. Benzi, 16132 Genova, Italy Dipartimento di Scienze Motorie e Riabilitative, Universita& di Genova, Largo R. Benzi, 16132 Genova, Italy Department of Physical Medicine and Rehabilitation, Baylor College of Medicine, Houston, TX 77030, USA
Abstract Psychophysics experiments on the human visual system have established that the sensitivity for the detection of "ne detail in noise contaminated images can be quantitatively and repeatably measured using stochastic resonance as a tool. Optimal noise results in maximal sensitivity. Does this mean simply that the retina is averaging out the noise, or instead is the visual cortex involved in some level of computation? We report results of functional magnetic resonance imaging experiments which indicate the latter. 1999 Elsevier Science B.V. All rights reserved. Keywords: Stochastic resonance; Functional magnetic resonance imaging (fMRI); Noise; Visual cortex
1. Introduction The e!ects of noise on or within the brain have long been a fascinating topic [1]. Stochastic resonance (SR) has shown in several experiments that noise added to * Corresponding author. Tel.: #1-314-516-5015; fax: #1-314-516-6152. E-mail address: [email protected] (E. Simonotto) 夽 Supported in part by the U.S. O$ce of Naval Research, Physics Division. 0925-2312/99/$ - see front matter 1999 Elsevier Science B.V. All rights reserved. PII: S 0 9 2 5 - 2 3 1 2 ( 9 9 ) 0 0 0 4 2 - 9
512
E. Simonotto et al. / Neurocomputing 26}27 (1999) 511}516
a weak signal can enhance the evoked responses in the peripheral nervous system [3,7,2,14]. In these experiments some measure of the information content (e.g., signal-to-noise ratio) of the neural spike trains were computed o! line by standard algorithm, thus opening the question of whether the nervous system could implement similar processing to extract information from noise. This question suggests experiment with human subjects. We have chosen to experiment with the visual system, showing images processed with a SR "lter to human subjects. In all previous animal experiments showing SR, the noise enhanced information was measured on an a!erent neuron. The discharges were digitized and analyzed for their information content by computer. This raises a fundamental question: Does the animal actually perceive and make use of the enhanced information? Such a question can only be answered for animals with behavioral experiments. In contrast, the question can be addressed using psychophysical experiments with human subjects. We thus substitute, in place of the computer, the human visual system [11]. A previous neural network model, which assumed a bistable energy landscape contaminated with noise, "rst demonstrated SR in the interpretation of ambiguous "gures [9]. A recent experiment with the human tactile system has demonstrated SR using electrical, transcutaneous noise [10]. During our experiment we showed to healthy volunteers animations derived from the simplest paradigm of stochastic resonance: the threshold "lter. Following this paradigm we added Gaussian noise to gray scale images and then we "ltered with a threshold, thus having only black or white pixels on the screen. This general paradigm was followed in both psychophysical and functional magnetic resonance imaging (fMRI) experiments described below. After a description of these experiments and the presentation of some example results, we conclude with a short summary.
2. Psychophysical experiments The contrast sensitivity of humans observing noisy visual animations can be measured as a function, for example, of the standard deviation of the noise. Prior experiments have focused on noise as a limiting factor in a population of young and old observers [12]; contrast discrimination [8,6]; and in recognition e$ciency in luminance noise [13]. In our psychophysical experiments we have observed that, as expected in a stochastic resonance experiment, an optimal noise level exists for which the subject can perceive more details (or, more speci"cally, the contrast sensitivity threshold is lowest). The images shown to the subjects were mediated with dynamical noise, that is, the noise in every pixel added to the image gray scale was a determined function of time. Dynamical images can only be shown as animations. An example can be viewed at the URL http://neurodyn.umsl.edu/sr. Subjects were shown a pattern of vertical bars scaled for di!ering contrasts. Their sensitivity thresholds were measured as a function of noise intensity. Details have been presented previously [11]. We were able to "t the experimental results to a
E. Simonotto et al. / Neurocomputing 26}27 (1999) 511}516
513
Fig. 1. Subjects performance over noisy images. The quantity K measures the sensitivity of the subject to noisy stimulation.
mathematical function obtained from SR theory, using a single adjustable parameter. This parameter represents a quantitative measurement of the performance of the subject, that is, his or her sensitivity for detection and interpretation of a "ne detail in the noise contaminated scene. The results are depicted in Fig. 1. They show a good degree of reproducibility, with small #uctuations of the sensitivity parameter (10% over a few days or weeks; 20% after one year) for di!erent trials of the same subject. Di!erent subjects, however, displayed considerably di!erent sensitivities, up to a factor of two. In addition, we have made similar studies with noise of variable correlation time. In these we have found that the sensitivity is rapidly enhanced as the correlation time drops below about 40 ms in good agreement with, for example, the results of #ickerfusion experiments [5]. Dynamical noise results in better perceived image quality than static noise. The observed enhancements could be due to a simple averaging process like the fusion of frames in a movie. Or, in addition, the extraction of image features from the noise could be the result of some computational process occurring beyond the early vision level. The motivation for the fMRI experiments described next is to address this question. 3. fMRI experiments In order to study the aforementioned question, we are currently running functional MRI studies to locate the areas of the visual cortex which are active under the task of
Fig. 2. Comparison of the fMRI statistical maps of activity for one of the subjects for the four visual stimulations (no noise, low noise, optimal noise and high noise) described in the text. Statistical maps are reported in color over the anatomical substrate in grayscale.
514 E. Simonotto et al. / Neurocomputing 26}27 (1999) 511}516
E. Simonotto et al. / Neurocomputing 26}27 (1999) 511}516
515
processing noisy images. These were vertical gratings of 4 cycles/degree of visual "eld, contaminated by dynamical noise of varying intensity and threshold "ltered as described above. During each experiment we conduct four scanning sessions, three for three di!erent noise intensities and one control for the noiseless case. We used a block mode design for the stimulus presentations. During eachblock the subject was alternatively presented with the vertical bars or a control pattern (solid gray, same average luminosity as the vertical bars) for periods of 21 s. Echoplanar Image (EPI) scans are acquired on the 1.5 T Siemens Magnetom Vision scanner of the St. Martino Hospital, in Genova, Italy. We obtained 40 EPI scans per session, TE"66 ms, TR"0.96 s, each scan covers 18 near axial planes of 6 mm thickness. O!-line data analysis was performed using SPM [4]. Typical data for a single subject, are shown in Fig. 2, where the four panels span the range from no noise to high noise. We are scanning 18 planes, but the functional activity is con"ned to 12 planes; three of them shown in each of the panels. The functionally active areas are shown in color over an high-resolution anatomical scan in gray scale. The optimal noise level was chosen from the previously described psychophysical experiments. We note "rst that there is a systematic di!erence between the noiseless and noise added cases. Moreover, for four out of "ve subjects tested, the active volume (color) is a maximum for the optimal noise level. This is shown in Fig. 3 where we have plotted the active volume versus the noise intensity for the "ve subjects. For four subjects these graphs show a clear maximum at the optimal noise
Fig. 3. Total active volume for the "ve subjects in the fMRI study. All but one subject (Subject 1) show a maximum active volume for the optimal noise level. Incomplete data are available for Subject 3.
516
E. Simonotto et al. / Neurocomputing 26}27 (1999) 511}516
intensity. It is tempting to interpret this result as indicating that the visual cortex is indeed involved in some type of computation when the subject is attempting to extract information from a noisy image.
4. Summary To conclude, we have obtained psychophysical data that measure a subject's ability to extract "ne detail from a noise contaminated image. This data shows that enhanced ability occurs for an optimal noise intensity, and that this ability varies considerably from subject-to-subject. Our fMRI results indicate that the volume of activity passes through a maximum at the optimal noise intensity. New studies which hopefully will further illuminate this question are currently in progress.
References [1] W.R. Adey, Organization of brain tissue: is the brain a noisy processor?, Intern. J. Neurosci. 3 (1972) 271}284. [2] F.Y. Chiou-Tan, K.N. Magee, L.R. Robinson, M.R. Nelson, S.S. Tuel, T.A. Krouskop, F. Moss, Enhancement of subthreshold sensory nerve action potentials during muscle tension mediated noise, Intern. J. Bifurc. Chaos 7 (1996) 1389}1396. [3] J.K. Douglass, L. Wilkens, E. Pantazelou, F. Moss, Noise enhancement of information transfer in cray"sh mechanoreceptors by stochastic resonance, Nature 365 (1993) 337}340. [4] K.J. Friston, A.P. Holmes, K.J. Worsley, J.P. Poline, C.D. Frith, R.S.J. Frackowiak, Statistical parametric maps in functional imaging: A general linear approach, Human Brain Mapping 2 (1994) 189}210. [5] S. He, Sheng, D.I.A. MacLeod, Contrast-modulation #icker: dynamics and spatial resolution of the light adaptation process, Vision Res. 38 (1998) 985}1000. [6] G.E. Legge, D. Kersten, A.E. Burgess, Contrast discrimination in noise, J. Opt. Soc. Am. 4 (1987) 391}404. [7] J.E. Levin, J.P. Miller, Broadband neural encoding in the cricket cercal sensory system enhanced by stochastic resonance, Nature 380 (1996) 165}168. [8] D.G. Pelli, The quantum e$ciency of human vision, in: C. Blackemore (Ed.), Vision: Coding and e$ciency, Cambridge University Press, Cambridge, NY, 1990, pp. 3}24. [9] M. Riani, E. Simonotto, Stochastic resonance in the perceptual interpretation of ambiguous "gures: a neural network model, Phys. Rev. Lett. 72 (1994) 3120}3123. [10] K.A. Richardson, T.T. Imho!, P. Grigg, J.J. Collins, Using electrical noise to enhance the ability of humans to detect subthreshold mechanical cutaneous stimuli, Chaos 8 (1998) 599}603. [11] E. Simonotto, M. Riani, C. Seife, M. Roberts, J. Twitty, F. Moss, Visual perception of stochastic resonance, Phys. Rev. Lett. 78 (1997) 1186}1190. [12] F. Speranza, G. Moraglia, B.A. Schneider, Noise-limited detection in young and old observers, Perceptual and Motor Skills 84 (1997) 829,830. [13] B.S. Tjan, W.L. Braje, G.E. Legge, D. Kersten, Human e$ciency for recognizing 3-D objects in luminance noise, Vision Res. 35 (1995) 3053}3069. [14] K. Wiesenfeld, F. Moss, Stochastic resonance and the bene"ts of noise: From the ice ages to cray"sh and SQUIDs, Nature 373 (1995) 33}36.
fMRI studies of visual cortical activity during noise stimulation夽 E. Simonotto *, F. Spano, M. Riani, A. Ferrari, F. Levrero, A. Pilot, P. Renzetti, R.C. Parodi, F. Sardanelli, P. Vitali, J. Twitty , F. Chiou-Tan, F. Moss Department of Physics and Astronomy, Center for Neurodynamics, University of Missouri at St. Louis, St. Louis, 8001 Natural Bridge Road, MO 63121, USA INFM-UdR Genova, Via Dodecaneso, 33, 16146 Genova, Italy Servizio di Fisica Medica, Ospedale S. Martino, Largo R. Benzi, 16132 Genova, Italy Istituto di Radiologia, Universita& di Genova, Largo R. Benzi, 16132 Genova, Italy Dipartimento di Scienze Motorie e Riabilitative, Universita& di Genova, Largo R. Benzi, 16132 Genova, Italy Department of Physical Medicine and Rehabilitation, Baylor College of Medicine, Houston, TX 77030, USA
Abstract Psychophysics experiments on the human visual system have established that the sensitivity for the detection of "ne detail in noise contaminated images can be quantitatively and repeatably measured using stochastic resonance as a tool. Optimal noise results in maximal sensitivity. Does this mean simply that the retina is averaging out the noise, or instead is the visual cortex involved in some level of computation? We report results of functional magnetic resonance imaging experiments which indicate the latter. 1999 Elsevier Science B.V. All rights reserved. Keywords: Stochastic resonance; Functional magnetic resonance imaging (fMRI); Noise; Visual cortex
1. Introduction The e!ects of noise on or within the brain have long been a fascinating topic [1]. Stochastic resonance (SR) has shown in several experiments that noise added to * Corresponding author. Tel.: #1-314-516-5015; fax: #1-314-516-6152. E-mail address: [email protected] (E. Simonotto) 夽 Supported in part by the U.S. O$ce of Naval Research, Physics Division. 0925-2312/99/$ - see front matter 1999 Elsevier Science B.V. All rights reserved. PII: S 0 9 2 5 - 2 3 1 2 ( 9 9 ) 0 0 0 4 2 - 9
512
E. Simonotto et al. / Neurocomputing 26}27 (1999) 511}516
a weak signal can enhance the evoked responses in the peripheral nervous system [3,7,2,14]. In these experiments some measure of the information content (e.g., signal-to-noise ratio) of the neural spike trains were computed o! line by standard algorithm, thus opening the question of whether the nervous system could implement similar processing to extract information from noise. This question suggests experiment with human subjects. We have chosen to experiment with the visual system, showing images processed with a SR "lter to human subjects. In all previous animal experiments showing SR, the noise enhanced information was measured on an a!erent neuron. The discharges were digitized and analyzed for their information content by computer. This raises a fundamental question: Does the animal actually perceive and make use of the enhanced information? Such a question can only be answered for animals with behavioral experiments. In contrast, the question can be addressed using psychophysical experiments with human subjects. We thus substitute, in place of the computer, the human visual system [11]. A previous neural network model, which assumed a bistable energy landscape contaminated with noise, "rst demonstrated SR in the interpretation of ambiguous "gures [9]. A recent experiment with the human tactile system has demonstrated SR using electrical, transcutaneous noise [10]. During our experiment we showed to healthy volunteers animations derived from the simplest paradigm of stochastic resonance: the threshold "lter. Following this paradigm we added Gaussian noise to gray scale images and then we "ltered with a threshold, thus having only black or white pixels on the screen. This general paradigm was followed in both psychophysical and functional magnetic resonance imaging (fMRI) experiments described below. After a description of these experiments and the presentation of some example results, we conclude with a short summary.
2. Psychophysical experiments The contrast sensitivity of humans observing noisy visual animations can be measured as a function, for example, of the standard deviation of the noise. Prior experiments have focused on noise as a limiting factor in a population of young and old observers [12]; contrast discrimination [8,6]; and in recognition e$ciency in luminance noise [13]. In our psychophysical experiments we have observed that, as expected in a stochastic resonance experiment, an optimal noise level exists for which the subject can perceive more details (or, more speci"cally, the contrast sensitivity threshold is lowest). The images shown to the subjects were mediated with dynamical noise, that is, the noise in every pixel added to the image gray scale was a determined function of time. Dynamical images can only be shown as animations. An example can be viewed at the URL http://neurodyn.umsl.edu/sr. Subjects were shown a pattern of vertical bars scaled for di!ering contrasts. Their sensitivity thresholds were measured as a function of noise intensity. Details have been presented previously [11]. We were able to "t the experimental results to a
E. Simonotto et al. / Neurocomputing 26}27 (1999) 511}516
513
Fig. 1. Subjects performance over noisy images. The quantity K measures the sensitivity of the subject to noisy stimulation.
mathematical function obtained from SR theory, using a single adjustable parameter. This parameter represents a quantitative measurement of the performance of the subject, that is, his or her sensitivity for detection and interpretation of a "ne detail in the noise contaminated scene. The results are depicted in Fig. 1. They show a good degree of reproducibility, with small #uctuations of the sensitivity parameter (10% over a few days or weeks; 20% after one year) for di!erent trials of the same subject. Di!erent subjects, however, displayed considerably di!erent sensitivities, up to a factor of two. In addition, we have made similar studies with noise of variable correlation time. In these we have found that the sensitivity is rapidly enhanced as the correlation time drops below about 40 ms in good agreement with, for example, the results of #ickerfusion experiments [5]. Dynamical noise results in better perceived image quality than static noise. The observed enhancements could be due to a simple averaging process like the fusion of frames in a movie. Or, in addition, the extraction of image features from the noise could be the result of some computational process occurring beyond the early vision level. The motivation for the fMRI experiments described next is to address this question. 3. fMRI experiments In order to study the aforementioned question, we are currently running functional MRI studies to locate the areas of the visual cortex which are active under the task of
Fig. 2. Comparison of the fMRI statistical maps of activity for one of the subjects for the four visual stimulations (no noise, low noise, optimal noise and high noise) described in the text. Statistical maps are reported in color over the anatomical substrate in grayscale.
514 E. Simonotto et al. / Neurocomputing 26}27 (1999) 511}516
E. Simonotto et al. / Neurocomputing 26}27 (1999) 511}516
515
processing noisy images. These were vertical gratings of 4 cycles/degree of visual "eld, contaminated by dynamical noise of varying intensity and threshold "ltered as described above. During each experiment we conduct four scanning sessions, three for three di!erent noise intensities and one control for the noiseless case. We used a block mode design for the stimulus presentations. During eachblock the subject was alternatively presented with the vertical bars or a control pattern (solid gray, same average luminosity as the vertical bars) for periods of 21 s. Echoplanar Image (EPI) scans are acquired on the 1.5 T Siemens Magnetom Vision scanner of the St. Martino Hospital, in Genova, Italy. We obtained 40 EPI scans per session, TE"66 ms, TR"0.96 s, each scan covers 18 near axial planes of 6 mm thickness. O!-line data analysis was performed using SPM [4]. Typical data for a single subject, are shown in Fig. 2, where the four panels span the range from no noise to high noise. We are scanning 18 planes, but the functional activity is con"ned to 12 planes; three of them shown in each of the panels. The functionally active areas are shown in color over an high-resolution anatomical scan in gray scale. The optimal noise level was chosen from the previously described psychophysical experiments. We note "rst that there is a systematic di!erence between the noiseless and noise added cases. Moreover, for four out of "ve subjects tested, the active volume (color) is a maximum for the optimal noise level. This is shown in Fig. 3 where we have plotted the active volume versus the noise intensity for the "ve subjects. For four subjects these graphs show a clear maximum at the optimal noise
Fig. 3. Total active volume for the "ve subjects in the fMRI study. All but one subject (Subject 1) show a maximum active volume for the optimal noise level. Incomplete data are available for Subject 3.
516
E. Simonotto et al. / Neurocomputing 26}27 (1999) 511}516
intensity. It is tempting to interpret this result as indicating that the visual cortex is indeed involved in some type of computation when the subject is attempting to extract information from a noisy image.
4. Summary To conclude, we have obtained psychophysical data that measure a subject's ability to extract "ne detail from a noise contaminated image. This data shows that enhanced ability occurs for an optimal noise intensity, and that this ability varies considerably from subject-to-subject. Our fMRI results indicate that the volume of activity passes through a maximum at the optimal noise intensity. New studies which hopefully will further illuminate this question are currently in progress.
References [1] W.R. Adey, Organization of brain tissue: is the brain a noisy processor?, Intern. J. Neurosci. 3 (1972) 271}284. [2] F.Y. Chiou-Tan, K.N. Magee, L.R. Robinson, M.R. Nelson, S.S. Tuel, T.A. Krouskop, F. Moss, Enhancement of subthreshold sensory nerve action potentials during muscle tension mediated noise, Intern. J. Bifurc. Chaos 7 (1996) 1389}1396. [3] J.K. Douglass, L. Wilkens, E. Pantazelou, F. Moss, Noise enhancement of information transfer in cray"sh mechanoreceptors by stochastic resonance, Nature 365 (1993) 337}340. [4] K.J. Friston, A.P. Holmes, K.J. Worsley, J.P. Poline, C.D. Frith, R.S.J. Frackowiak, Statistical parametric maps in functional imaging: A general linear approach, Human Brain Mapping 2 (1994) 189}210. [5] S. He, Sheng, D.I.A. MacLeod, Contrast-modulation #icker: dynamics and spatial resolution of the light adaptation process, Vision Res. 38 (1998) 985}1000. [6] G.E. Legge, D. Kersten, A.E. Burgess, Contrast discrimination in noise, J. Opt. Soc. Am. 4 (1987) 391}404. [7] J.E. Levin, J.P. Miller, Broadband neural encoding in the cricket cercal sensory system enhanced by stochastic resonance, Nature 380 (1996) 165}168. [8] D.G. Pelli, The quantum e$ciency of human vision, in: C. Blackemore (Ed.), Vision: Coding and e$ciency, Cambridge University Press, Cambridge, NY, 1990, pp. 3}24. [9] M. Riani, E. Simonotto, Stochastic resonance in the perceptual interpretation of ambiguous "gures: a neural network model, Phys. Rev. Lett. 72 (1994) 3120}3123. [10] K.A. Richardson, T.T. Imho!, P. Grigg, J.J. Collins, Using electrical noise to enhance the ability of humans to detect subthreshold mechanical cutaneous stimuli, Chaos 8 (1998) 599}603. [11] E. Simonotto, M. Riani, C. Seife, M. Roberts, J. Twitty, F. Moss, Visual perception of stochastic resonance, Phys. Rev. Lett. 78 (1997) 1186}1190. [12] F. Speranza, G. Moraglia, B.A. Schneider, Noise-limited detection in young and old observers, Perceptual and Motor Skills 84 (1997) 829,830. [13] B.S. Tjan, W.L. Braje, G.E. Legge, D. Kersten, Human e$ciency for recognizing 3-D objects in luminance noise, Vision Res. 35 (1995) 3053}3069. [14] K. Wiesenfeld, F. Moss, Stochastic resonance and the bene"ts of noise: From the ice ages to cray"sh and SQUIDs, Nature 373 (1995) 33}36.