- Email: [email protected]
The role of blind humans’ visual cortex in auditory change detection
Neuroscience Letters 379 (2005) 127–131
The role of blind humans’ visual cortex in auditory change detection Teija Kujalaa,b,c,∗ , Matias J. Palvad , Oili Salonene , Paavo Alkuf , Minna Huotilainena,b,c , Antti J¨arvinenf , Risto N¨aa¨ t¨anenb,c b
a Helsinki Collegium for Advanced Studies, University of Helsinki, Finland Cognitive Brain Research Unit, Department of Psychology, P.O. Box 9, FIN-00014 University of Helsinki, Siltavuorenpenger 20 C, Finland c Helsinki Brain Research Center, Helsinki, Finland d Department of Bio- and Environmental Sciences, University of Helsinki, Finland e Department of Radiology, Helsinki University Central Hospital, Finland f Laboratory of Acoustics and Audio Signal Processing, Helsinki University of Technology, Espoo, Finland
Received 8 September 2004; received in revised form 23 November 2004; accepted 21 December 2004
Abstract Several studies using brain imaging have demonstrated occipital-cortex activation in blind individuals during tactile and auditory tasks, suggesting that the visual cortex deprived of its normal input has adopted a new role in information processing. So far, however, at what stages of information processing and to which perceptual sub-processes this applies remains unclear. We determined the auditory functions of this cortical region in early-blind humans by means of functional magnetic resonance imaging. We found that these areas were not activated by the mere presence of sound, but were involved in the attentive processing of changes in the auditory environment, which is important in detecting potentially dangerous or other important events in the surroundings, for example. © 2004 Elsevier Ireland Ltd. All rights reserved. Keywords: Blindness; Neural plasticity; Crossmodal; Auditory
A sensory-specific brain area may adopt a new function if it is deprived of its normal sensory input in deafness and blindness (for reviews, see [8,2,16]). For example, neurons in brain areas that normally process visual information become responsive to tactile [7] and auditory [18] stimuli in animals deprived of visual input after birth. Such cross-modal brain plasticity has also been shown in numerous studies in human subjects (see, e.g., [3,9,23,26,20,14,22,5]). Moreover, it has been found that the occipital cortex of blind individuals indeed has a functional role in processing non-visual information: transient interference of the neural functioning in their occipital cortex with trans-cranial magnetic stimulation impairs Braille reading [3]. Occipital-cortex activation in the blind has been demonstrated during many different tasks using diverse stimuli, suggesting that it might be involved in various non-visual func∗
Corresponding author. Tel.: +358 9 191 23760; fax: +358 9 191 24509. E-mail address: [email protected] (T. Kujala).
0304-3940/$ – see front matter © 2004 Elsevier Ireland Ltd. All rights reserved. doi:10.1016/j.neulet.2004.12.070
tions. For example, it is activated during Braille reading [23], tactile discrimination [22], semantic processing [14], auditory localization [26], target-tone detection [9], and speechlistening tasks [20]. In contrast, it has been proposed that non-visual stimulation involving no task does not activate their visual areas [8,5]. Therefore, attention towards the nonvisual stimuli seems to be a prerequisite for activating these brain areas. Auditory modality is essential for the blind in mediating important signals from the environment such as those indicating a possible danger. Being very dependent on this modality, the blind are superior to the sighted in various auditory functions. For example, they are able to use echoes in order to navigate in their surroundings [24], and they are more effective than the sighted in locating sound sources [17,10,11,21]. In order to fully use auditory modality as a means of detecting potentially important signals, one has to efficiently react to changes in the auditory environment. We addressed the role of blind humans’ occipital areas in the detection of
128
T. Kujala et al. / Neuroscience Letters 379 (2005) 127–131
auditory change and in attentional processing. Previous studies have shown posterior scalp topographies of evoked responses to deviant sounds in the blind when they attend to but not when they ignore the sounds [8,9,12]. It was hypothesized that selective attention to the sound features of the auditory stimuli is a prerequisite for occipital-cortex activation in the blind and, furthermore, it was predicted that sound changes in the attended sound stream are a necessary condition for activating these regions. The subjects were five early-blind (30–34 years old; 2 males) adults, with the causes of blindness being restricted to deficits of the retina, in other parts of the eye, or in the optic nerves. Two of the subjects were born blind because of an inherited deficit, two had retinopathy of prematurity, causing blindness by the age of 1 year, and the eyes of one subject were removed at the age of 9 months because of retinoblastoma. Four of them were able to perceive bright light, but none of them had any pattern vision. The evaluation of the brain anatomy, carried out by a neuroradiologist on the basis of magnetic resonance images, revealed that all of the structures in each cerebral lobe, including the occipital areas, were normal in macroanatomy. Five sighted (30–35 years old; 1 male) individuals served as controls. Informed consent was obtained from the subjects prior to the experiment, which was carried out according to the Declaration of Helsinki. Functional magnetic-resonance imaging (fMRI) scans were obtained using a 1.5-T MRI system in three sessions. The direction of the subject’s attention and the type of stimuli were varied in the sessions. The subject’s task in all the experimental blocks (activation periods) in each session was to detect and silently count rare target stimuli appearing once on average during each block. During the scanning, the stimuli were presented in the experimental blocks and turned off in the baseline blocks. During the baseline periods, the subject was instructed to relax. The order of the sessions was counterbalanced across the subjects. The sighted subjects kept their eyes closed during the scanning. In Session A we determined whether an attended monotonic auditory stimulation was sufficient for activating the occipital cortex of the blind, or whether changes needed to occur in the attended auditory stimulus stream. This session included two randomly presented conditions, one with repetitive standard stimuli (vowel /a/ as in car, p = 0.98; half of the experimental blocks) and the other with these standard stimuli (p = 0.8) and infrequently presented deviant stimuli (vowel /i/ as in pick, p = 0.18). An occasional, very rare (p = 0.02) increment in stimulus duration was used in both conditions as the target in order to ensure that the subjects selectively attended to the stimuli. In Session B we examined whether these standard and deviant stimuli activated the blind subjects’ visual areas if they selectively attended to a different type of sound than the repetitive and occasional vowel stimuli. The standard and deviant stimuli were identical to the /a/ and /i/ stimuli in Session A. However, completely new types of sounds, ones with a complex spectrotemporal pattern (“complex sound”),
were presented as rare targets. This was to direct the subject’s attention away from the sound features that existed in the continuous stimulation. The effect of the stimulus type on the activation pattern in the occipital lobe was examined in Session C. Half of the conditions contained the standard /a/ and deviant /i/ vowels, the remaining ones consisting of standard and deviant tone stimuli, which were approximately as complex as the vowel stimuli (the vowel and tone stimuli are described below). In both conditions, the task (detect stimulus-duration increments) and the probabilities of standard and deviant stimuli and of rare targets were the same as in Session A. The /a/ and /i/ stimuli were natural-sounding, so-called semi-synthetic vowels that were generated by combining two processes of human voice production: excitation from the vocal folds and the filtering effect of the vocal tract [1]. Excitation from the vocal folds was computed from the natural speech of a male speaker using automatic inverse-filtering techniques, and the vocal tract was modeled using digital all-pole filtering. The corresponding standard and deviant tones (Session C) were produced by bandpass-filtering these vowels with filters with centre frequencies adjusted to the strongest harmonic in the vicinity of the first formant of the corresponding vowel. The duration of the stimuli in each condition was 400 ms (including 10-ms rise and fall times), the stimulus-onset asynchrony being 500 ms. The stimuli were binaurally presented via an earphone system using EARLINK 3A ear inserts. The earphone system consisted of electrodynamic cone loudspeakers (Audax AT 080MO), and acoustic couplers from loudspeakers to variable diameter tubes with a total length of 4 m. The longitudal reflections of the tube were attenuated with foam rubber in the locations where the tube diameter changed. The frequency response of the earphone system was compensated with passive analog and digital filters. The digital-filtering system was realized on a Motorola 56002 EVM DSP-board. The system response was essentially flat from 100 Hz to 4 kHz when measured using a 2 cm3 artificial ear conforming to the IEC 711 standard. The sound-pressure level of the stimulus was adjusted to 80–90 dB, which clearly exceeded the gradient noise and was also attenuated by the ear inserts and hearing protectors. The head was fixed with a vacuum pillow. The head coil was filled with foam rubber in order to support the subject’s head and to further attenuate the gradient noise. Five localizer images (one transaxial, one coronal, and three sagittal) were acquired for slice positioning in the MRI scanning. T1- (anatomical) and T2- (functional) weighted images acquired with gradient-echo echo-planar imaging using 3D MPRAGE sequences were obtained with a Siemens 1.5T magnetic-resonance imaging device (Vision, Siemens, Erlangen, Germany). The echo time was 70 ms, repetition time 2 s, and flip angle 90◦ . A standard head coil was employed in the measurements. Sixteen axial slices (4-mm thickness and 4 mm × 4 mm in-plane resolution; 64 × 64 matrix) were obtained for the functional images, the most ventral slice
T. Kujala et al. / Neuroscience Letters 379 (2005) 127–131
Fig. 1. Averaged MR time course from one blind subject showing the responses in the conditions with standard and deviant sounds, the subject’s task being to detect rare target stimuli of longer duration. The hemodynamic responses are from an experimental block of Session C. T refers to tone stimuli and V to vowel stimuli. The red colour refers to activity measured in the medial occipital areas and the blue colour to that in the lateral occipital areas.
129
being positioned along the base of the skull (see Fig. 2). The slices extended from the basal parts of the brain up to the parietal lobe. Sixteen images were obtained during the first baseline block, which was followed by alternating experimental (12 images) and baseline (13 images) blocks, five and four of each, respectively, after which the subject’s response was obtained. Data from 10 experimental blocks were collected for each experimental condition. The head position changed by no more than 2 mm in any of the subjects during the experiment. A Kolmogorov-Smirnov test was applied on a pixel-by-pixel basis in the individual data analysis so that we could determine the activated occipital areas of the individual subjects as precisely as possible. This approach is helpful in determining the contribution of the primary and secondary areas to the processes of interest, for instance [2]. We found occipital activation in the blind subjects in conditions in which the standard and deviant stimuli were selectively attended (Figs. 1 and 2). In order to determine whether the hypothesis (that the occipital cortex of the blind is activated when sound streams contain changes and the features of the sounds are selectively attended) was supported by the data, we made the following comparisons. The number of activated occipital voxels (Table 1) was compared in con-
Fig. 2. The detected activation superimposed on an anatomical axial brain slice of one blind subject (Subject AT). The position of the slice is indicated at the bottom of the figure (marked with a red line). (a) The subject’s task was to detect very rare targets: stimulus-duration increments occurring in an otherwise monotonic stimulation containing /a/ vowels. (b) The stimulation consisted of repetitive standard /a/ stimuli and occasional deviant /i/ stimuli, the target stimulus being an infrequent duration increment of standard or deviant stimuli. (c) The standard and deviant stimuli were the same as in (b), but the target stimulus was an infrequent complex sound, which was very different from the standard and deviant stimuli. (d) The standard and deviant stimuli were tones with frequencies corresponding to the strongest harmonics of /a/ and /i/ in the vicinity of their first formants. The subject’s task was the same as in (b), i.e., to detect occasional stimulus-duration increments. Auditory cortices were activated in each condition. However, occipital areas were significantly activated only when the stimulation contained standard and deviant stimuli and the task directed the subject’s attention to the auditory stimulation (the stimulus-duration increments had to be detected). The figure also shows that, by and large, the same occipital areas were activated by the vowel and tone stimuli.
130
T. Kujala et al. / Neuroscience Letters 379 (2005) 127–131
Table 1 The number of significantly (p = 10−6 ) activated occipital-lobe pixels in the blind subjects in each condition Condition
S attend (Session A) S + D attend (Session A) S + D unattend (Session B) S + D vowels (Session C) S + D tones (Session C)
Subject AT
SL
TB
ST
TL
2 1 0 >30 >30
1 >10 0 0 >10
0 8 0 >30 >30
0 1 0 >10 3
0 >30 0 3 8
>10: over 10 activated pixels; >30: over 30 activated pixels; S: standard stimuli; D: deviant stimuli.
ditions including selectively-attended standard and deviant vowels and those including either selectively-attended standards (with no deviants) or standard and deviant vowels when the complex sound was the target. The data were collapsed as follows: (1) the conditions including standard and deviant vowels in Sessions A and C, and (2) the condition including standards without deviants in Session A, and that including standard and deviant vowels when the complex sound was the target in Session B. This comparison yielded a significant difference between the conditions (Wilcoxon Matched Pairs Test, Z = 2.02, p < 0.05). In addition, we compared the number of activated voxels between the different conditions, which revealed a significant effect (Friedman ANOVA, Chi-Square (d.f. = 4) = 13.21, p < 0.02). This analysis showed the following rank-order, starting with the condition with the largest number of activated voxels: selectively attended standard and deviant tones (Session C), average rank: 4.6; selectively attended standard and deviant vowels (Session C), 3.5; the same stimuli in Session A, 3.4; selectively attended standards (Session A), 2.1; and standard and deviant vowels when the complex sound was the target (Session B), 1.4. The occipital activation found during selective attention to features of sounds in the streams involving deviant sounds comprised the lateral and middle occipital gyri in the blind. Activity was also observed in the cuneus and lingual gyrus of some subjects. The calcarine sulcus was activated in three subjects by tone stimuli and in one subject by vowel stimuli. The comparison of the activations caused by vowels versus tones in Session C revealed that more wide-spread occipital activity was found for tones than vowels in four blind subjects, whereas in one blind subject the result was the opposite (Table 1). The results were as expected in three of the sighted subjects: no occipital activity was found in any of the conditions. However, in one of them, selectively attended stimulation including standard and deviant vowels (Session A) elicited clear occipital activity. This activity was also present in Session B, in which the target among the vowels was the complex sound. In another sighted subject, the attended monotonic stimulation (including only the vowel /a/ and the longer-duration vowel targets) in Session A elicited some occipital activity. Thus, the activation patterns were different in the two sighted
subjects compared with each other and compared with the blind subjects. Our results show that the occipital cortex of the blind processes auditory information under certain conditions. Occipital activity in blind humans was always observed in previous studies when attentional processing of the non-visual stimulation was required (for reviews, see [8,2,16]). More precisely, our results show that attentional processing per se is not sufficient for activating these areas, since they were activated only when occasional stimulus changes occurred in the sound sequences that were selectively attended. This is consistent with the results of previous studies using electroand magnetoencephalography showing that stimulus changes elicit activity in the posterior brain areas of the blind when the sound sequences are attended to but not when they are ignored [8,9,12]. Activity was found in several occipital areas of the blind subjects, even in the primary visual cortex in three out of five individuals. Regions of the lateral and middle occipital gyri were activated by both vowel and tone stimuli, and activity was also found in the cuneus and lingual gyrus in some subjects. Even the calcarine sulcus was activated (by tones in three subjects and by vowels in one). These results are in agreement with those of the majority of studies on early-blind human subjects, showing that the primary occipital areas and the regions close to them are activated by non-visual stimulation (see, e.g., [16,23,26,22]). According to a recent study, the primary occipital areas are involved in non-visual processes if the onset of blindness occurs before the age of 16 [22]. The tones activated larger occipital areas than the vowels in four out of five blind subjects. This result might suggest that stimulus novelty has an effect on the extent of occipital activity in the blind. The subjects were familiar with the vowels used in the present study, which were Finnish speech sounds, whereas they had never heard the tone stimuli, which were created so that they physically approximated the vowels but did not sound like speech. Thus, blind individuals’ occipital cortex might have a role in novelty detection. Somewhat unexpectedly, occipital activity was found in two sighted control subjects, too. There are three alternative explanations for this finding. First, the activity might reflect some visual imagery related to sounds in some conditions. Second, the subjects might have accidentally opened their eyes during the stimulation (even though they reported not having done so). Third, the activity might relate to the unmasking of some possible auditory input to the visual areas [4,19]. Support for such a possibility was found in a recent study showing that the occipital areas of sighted individuals were activated by auditory stimulation after they had been blindfolded for several days [15]. Furthermore, there is even evidence that a modality-specific brain area may be activated by stimuli of another modality even when there is no sensory deprivation [2]. Moreover, the occipital activity in the two sighted subjects of the present study did not follow the same pattern as in the blind subjects. Thus, according to these
T. Kujala et al. / Neuroscience Letters 379 (2005) 127–131
results, occipital activity in the sighted was neither stimulusor task-specific, nor did it reflect the same processes as in the blind subjects. The fact that the occipital activity was stimulus- and taskdependent across the blind subjects indicates that their occipital cortex has a specific role in auditory processing. These results, in agreement with those of previous studies [8,9,12], suggest that the occipital cortex of the blind processes sound changes in attended stimulus streams. Thus, this activity reflects higher-order processing rather than the early, automatic sound discrimination that occurs in the auditory cortex [25,13]. The activated occipital areas might belong to the network that also includes auditory-association areas, which were shown to be activated when (sighted) subjects detected targets in sound streams but not when the task involved no target detection [6]. Previous studies using behavioural measures have shown that the blind are more efficient than the sighted in detecting sound changes in an attended stream of sounds [10,12]. Efficiency in terms of rapid attention switching towards novel signals perceived via intact modalities is essential in compensating for lost visual modality in order to rapidly detect new and possibly warning or otherwise significant events in the environment. The neural population of the visual cortex, which, according to our study, processes unpredictable sound changes, might contribute to this enhanced efficiency in the blind.
[6]
[7]
[8] [9]
[10]
[11]
[12]
[13] [14]
[15] [16] [17] [18]
Acknowledgements This study was supported by the Academy of Finland and the University of Helsinki. We thank Satu Palva, M.A. for assisting with the data analysis, and Joan Nordlund, M.A. for the language check. References [1] P. Alku, Glottal wave analysis with pitch synchronous adaptive inverse filtering, Speech Commun. 11 (1992) 109–118. [2] D. Bavelier, H.J. Neville, Cross-modal plasticity: where and how? Nat. Rev. 3 (2002) 443–452. [3] L.G. Cohen, P. Celnik, A. Pascual-Leone, B. Corwell, L. Faiz, et al., Functional relevance of cross-modal plasticity in blind humans, Nature 389 (1997) 180–183. [4] A. Falchier, S. Clavagnier, P. Barone, H. Kennedy, Anatomical evidence of multimodal integration in primate striate cortex, J. Neurosci. 22 (2002) 5749–5759. [5] E.R. Gizewski, T. Gasser, A. de Greiff, A. Bohem, M. Forsting, Cross-modal plasticity for sensory and motor activa-
[19]
[20]
[21]
[22]
[23]
[24] [25]
[26]
131
tion patterns in blind subjects, NeuroImage 19 (2003) 968– 975. C.L. Grady, J.W. Van Meter, J.Ma. Maisog, P. Pietrini, J. Krasuski, J.P. Rauschecker, Attention-related modulation of activity in primary and secondary auditory cortes, NeuroReport 8 (1997) 2511–2516. J. Hyv¨arinen, S. Carlson, L. Hyv¨arinen, Early visual deprivation alters modality of neuronal responses in area 19 of monkey cortex, Neurosci. Lett. 26 (1981) 239–243. T. Kujala, K. Alho, R. N¨aa¨ t¨anen, Cross-modal reorganization of human cortical functions, Trends Neurosci. 23 (2000) 115–120. T. Kujala, M. Huotilainen, J. Sinkkonen, A.I. Ahonen, K. Alho, et al., Visual cortex activation in blind humans during sound discrimination, Neurosci. Lett. 183 (1995) 143–146. T. Kujala, A. Lehtokoski, K. Alho, J. Kekoni, R. N¨aa¨ t¨anen, Faster reaction times in the blind than sighted during bimodal dividedattention task, Acta Psychol. 96 (1997) 75–82. N. Lessard, M. Pare, F. Lepore, M. Lassonde, Early-blind human subjects localize sound sources better than sighted subjects, Nature 395 (1998) 278–280. M. Liotti, K. Ryder, M. Woldorff, Auditory attention in the congenitally blind: where, when and what gets reorganized? NeuroReport 9 (1998) 1007–1012. R. N¨aa¨ t¨anen, I. Winkler, The concept of auditory stimulus representation in cognitive neuroscience, Psychol. Bull. 6 (1999) 826–859. U. Noppeney, K.J. Friston, K. Price, Effects of visual deprivation on the organization of the semantic system, Brain 126 (2003) 1620–1627. A. Pascual-Leone, R. Hamilton, The metamodal organization of the brain, Prog. Brain Res. 134 (2001) 427–445. J. Rauschecker, Cortical map plasticity in animals and humans, Prog. Brain Res. 138 (2002) 73–88. J. Rauschecker, U. Kniepert, Auditory localization behaviour in visually deprived cats, Eur. J. Neurosci. 6 (1996) 149–160. J. Rauschecker, M. Korte, Auditory compensation for early blindness in cat cerebral cortex, J. Neurosci. 13 (1993) 4538–4548. K.S. Rockland, H. Ojima, Multisensory convergence in calcarine visual areas in macaque monkey, Int. J. Psychophysiol. 50 (2003) 19–26. B. R¨oder, O. Stock, S. Bien, F. R¨osler, Speech processing activates visual cortex in congenitally blind humans, Eur. J. Neurosci. 16 (2002) 930–936. B. R¨oder, W. Teder-S¨alej¨arvi, A. Sterr, F. Rosler, S.A. Hillyard, et al., Improved auditory spatial tuning in blind humans, Nature 400 (1999) 162–166. N. Sadato, T. Okada, M. Honda, Y. Yonekura, Critical period for cross-modal plasticity in blind humans: a functional MRI study, NeuroImage 16 (2002) 389–400. N. Sadato, A. Pascual-Leone, J. Grafman, V. Ibanez, M.-P. Deiber, et al., Activation of the primary visual cortex by braille reading in blind subjects, Nature 380 (1996) 526–528. E.R. Strelow, J.A. Brabyn, Locomotion of the blind controlled by natural sound cues, Perception 11 (1982) 635–640. H. Tiitinen, P. May, K. Reinikainen, R. N¨aa¨ t¨anen, Attentive novelty detection in humans is governed by pre-attentive sensory memory, Nature 327 (1994) 90–92. R. Weeks, B. Horwitz, A. Aziz-Sultan, B. Tian, C.M. Wessinger, et al., A positron emission tomographic study of auditory localization in the congenitally blind, J. Neurosci. 20 (2000) 2664–2672.
The role of blind humans’ visual cortex in auditory change detection Teija Kujalaa,b,c,∗ , Matias J. Palvad , Oili Salonene , Paavo Alkuf , Minna Huotilainena,b,c , Antti J¨arvinenf , Risto N¨aa¨ t¨anenb,c b
a Helsinki Collegium for Advanced Studies, University of Helsinki, Finland Cognitive Brain Research Unit, Department of Psychology, P.O. Box 9, FIN-00014 University of Helsinki, Siltavuorenpenger 20 C, Finland c Helsinki Brain Research Center, Helsinki, Finland d Department of Bio- and Environmental Sciences, University of Helsinki, Finland e Department of Radiology, Helsinki University Central Hospital, Finland f Laboratory of Acoustics and Audio Signal Processing, Helsinki University of Technology, Espoo, Finland
Received 8 September 2004; received in revised form 23 November 2004; accepted 21 December 2004
Abstract Several studies using brain imaging have demonstrated occipital-cortex activation in blind individuals during tactile and auditory tasks, suggesting that the visual cortex deprived of its normal input has adopted a new role in information processing. So far, however, at what stages of information processing and to which perceptual sub-processes this applies remains unclear. We determined the auditory functions of this cortical region in early-blind humans by means of functional magnetic resonance imaging. We found that these areas were not activated by the mere presence of sound, but were involved in the attentive processing of changes in the auditory environment, which is important in detecting potentially dangerous or other important events in the surroundings, for example. © 2004 Elsevier Ireland Ltd. All rights reserved. Keywords: Blindness; Neural plasticity; Crossmodal; Auditory
A sensory-specific brain area may adopt a new function if it is deprived of its normal sensory input in deafness and blindness (for reviews, see [8,2,16]). For example, neurons in brain areas that normally process visual information become responsive to tactile [7] and auditory [18] stimuli in animals deprived of visual input after birth. Such cross-modal brain plasticity has also been shown in numerous studies in human subjects (see, e.g., [3,9,23,26,20,14,22,5]). Moreover, it has been found that the occipital cortex of blind individuals indeed has a functional role in processing non-visual information: transient interference of the neural functioning in their occipital cortex with trans-cranial magnetic stimulation impairs Braille reading [3]. Occipital-cortex activation in the blind has been demonstrated during many different tasks using diverse stimuli, suggesting that it might be involved in various non-visual func∗
Corresponding author. Tel.: +358 9 191 23760; fax: +358 9 191 24509. E-mail address: [email protected] (T. Kujala).
0304-3940/$ – see front matter © 2004 Elsevier Ireland Ltd. All rights reserved. doi:10.1016/j.neulet.2004.12.070
tions. For example, it is activated during Braille reading [23], tactile discrimination [22], semantic processing [14], auditory localization [26], target-tone detection [9], and speechlistening tasks [20]. In contrast, it has been proposed that non-visual stimulation involving no task does not activate their visual areas [8,5]. Therefore, attention towards the nonvisual stimuli seems to be a prerequisite for activating these brain areas. Auditory modality is essential for the blind in mediating important signals from the environment such as those indicating a possible danger. Being very dependent on this modality, the blind are superior to the sighted in various auditory functions. For example, they are able to use echoes in order to navigate in their surroundings [24], and they are more effective than the sighted in locating sound sources [17,10,11,21]. In order to fully use auditory modality as a means of detecting potentially important signals, one has to efficiently react to changes in the auditory environment. We addressed the role of blind humans’ occipital areas in the detection of
128
T. Kujala et al. / Neuroscience Letters 379 (2005) 127–131
auditory change and in attentional processing. Previous studies have shown posterior scalp topographies of evoked responses to deviant sounds in the blind when they attend to but not when they ignore the sounds [8,9,12]. It was hypothesized that selective attention to the sound features of the auditory stimuli is a prerequisite for occipital-cortex activation in the blind and, furthermore, it was predicted that sound changes in the attended sound stream are a necessary condition for activating these regions. The subjects were five early-blind (30–34 years old; 2 males) adults, with the causes of blindness being restricted to deficits of the retina, in other parts of the eye, or in the optic nerves. Two of the subjects were born blind because of an inherited deficit, two had retinopathy of prematurity, causing blindness by the age of 1 year, and the eyes of one subject were removed at the age of 9 months because of retinoblastoma. Four of them were able to perceive bright light, but none of them had any pattern vision. The evaluation of the brain anatomy, carried out by a neuroradiologist on the basis of magnetic resonance images, revealed that all of the structures in each cerebral lobe, including the occipital areas, were normal in macroanatomy. Five sighted (30–35 years old; 1 male) individuals served as controls. Informed consent was obtained from the subjects prior to the experiment, which was carried out according to the Declaration of Helsinki. Functional magnetic-resonance imaging (fMRI) scans were obtained using a 1.5-T MRI system in three sessions. The direction of the subject’s attention and the type of stimuli were varied in the sessions. The subject’s task in all the experimental blocks (activation periods) in each session was to detect and silently count rare target stimuli appearing once on average during each block. During the scanning, the stimuli were presented in the experimental blocks and turned off in the baseline blocks. During the baseline periods, the subject was instructed to relax. The order of the sessions was counterbalanced across the subjects. The sighted subjects kept their eyes closed during the scanning. In Session A we determined whether an attended monotonic auditory stimulation was sufficient for activating the occipital cortex of the blind, or whether changes needed to occur in the attended auditory stimulus stream. This session included two randomly presented conditions, one with repetitive standard stimuli (vowel /a/ as in car, p = 0.98; half of the experimental blocks) and the other with these standard stimuli (p = 0.8) and infrequently presented deviant stimuli (vowel /i/ as in pick, p = 0.18). An occasional, very rare (p = 0.02) increment in stimulus duration was used in both conditions as the target in order to ensure that the subjects selectively attended to the stimuli. In Session B we examined whether these standard and deviant stimuli activated the blind subjects’ visual areas if they selectively attended to a different type of sound than the repetitive and occasional vowel stimuli. The standard and deviant stimuli were identical to the /a/ and /i/ stimuli in Session A. However, completely new types of sounds, ones with a complex spectrotemporal pattern (“complex sound”),
were presented as rare targets. This was to direct the subject’s attention away from the sound features that existed in the continuous stimulation. The effect of the stimulus type on the activation pattern in the occipital lobe was examined in Session C. Half of the conditions contained the standard /a/ and deviant /i/ vowels, the remaining ones consisting of standard and deviant tone stimuli, which were approximately as complex as the vowel stimuli (the vowel and tone stimuli are described below). In both conditions, the task (detect stimulus-duration increments) and the probabilities of standard and deviant stimuli and of rare targets were the same as in Session A. The /a/ and /i/ stimuli were natural-sounding, so-called semi-synthetic vowels that were generated by combining two processes of human voice production: excitation from the vocal folds and the filtering effect of the vocal tract [1]. Excitation from the vocal folds was computed from the natural speech of a male speaker using automatic inverse-filtering techniques, and the vocal tract was modeled using digital all-pole filtering. The corresponding standard and deviant tones (Session C) were produced by bandpass-filtering these vowels with filters with centre frequencies adjusted to the strongest harmonic in the vicinity of the first formant of the corresponding vowel. The duration of the stimuli in each condition was 400 ms (including 10-ms rise and fall times), the stimulus-onset asynchrony being 500 ms. The stimuli were binaurally presented via an earphone system using EARLINK 3A ear inserts. The earphone system consisted of electrodynamic cone loudspeakers (Audax AT 080MO), and acoustic couplers from loudspeakers to variable diameter tubes with a total length of 4 m. The longitudal reflections of the tube were attenuated with foam rubber in the locations where the tube diameter changed. The frequency response of the earphone system was compensated with passive analog and digital filters. The digital-filtering system was realized on a Motorola 56002 EVM DSP-board. The system response was essentially flat from 100 Hz to 4 kHz when measured using a 2 cm3 artificial ear conforming to the IEC 711 standard. The sound-pressure level of the stimulus was adjusted to 80–90 dB, which clearly exceeded the gradient noise and was also attenuated by the ear inserts and hearing protectors. The head was fixed with a vacuum pillow. The head coil was filled with foam rubber in order to support the subject’s head and to further attenuate the gradient noise. Five localizer images (one transaxial, one coronal, and three sagittal) were acquired for slice positioning in the MRI scanning. T1- (anatomical) and T2- (functional) weighted images acquired with gradient-echo echo-planar imaging using 3D MPRAGE sequences were obtained with a Siemens 1.5T magnetic-resonance imaging device (Vision, Siemens, Erlangen, Germany). The echo time was 70 ms, repetition time 2 s, and flip angle 90◦ . A standard head coil was employed in the measurements. Sixteen axial slices (4-mm thickness and 4 mm × 4 mm in-plane resolution; 64 × 64 matrix) were obtained for the functional images, the most ventral slice
T. Kujala et al. / Neuroscience Letters 379 (2005) 127–131
Fig. 1. Averaged MR time course from one blind subject showing the responses in the conditions with standard and deviant sounds, the subject’s task being to detect rare target stimuli of longer duration. The hemodynamic responses are from an experimental block of Session C. T refers to tone stimuli and V to vowel stimuli. The red colour refers to activity measured in the medial occipital areas and the blue colour to that in the lateral occipital areas.
129
being positioned along the base of the skull (see Fig. 2). The slices extended from the basal parts of the brain up to the parietal lobe. Sixteen images were obtained during the first baseline block, which was followed by alternating experimental (12 images) and baseline (13 images) blocks, five and four of each, respectively, after which the subject’s response was obtained. Data from 10 experimental blocks were collected for each experimental condition. The head position changed by no more than 2 mm in any of the subjects during the experiment. A Kolmogorov-Smirnov test was applied on a pixel-by-pixel basis in the individual data analysis so that we could determine the activated occipital areas of the individual subjects as precisely as possible. This approach is helpful in determining the contribution of the primary and secondary areas to the processes of interest, for instance [2]. We found occipital activation in the blind subjects in conditions in which the standard and deviant stimuli were selectively attended (Figs. 1 and 2). In order to determine whether the hypothesis (that the occipital cortex of the blind is activated when sound streams contain changes and the features of the sounds are selectively attended) was supported by the data, we made the following comparisons. The number of activated occipital voxels (Table 1) was compared in con-
Fig. 2. The detected activation superimposed on an anatomical axial brain slice of one blind subject (Subject AT). The position of the slice is indicated at the bottom of the figure (marked with a red line). (a) The subject’s task was to detect very rare targets: stimulus-duration increments occurring in an otherwise monotonic stimulation containing /a/ vowels. (b) The stimulation consisted of repetitive standard /a/ stimuli and occasional deviant /i/ stimuli, the target stimulus being an infrequent duration increment of standard or deviant stimuli. (c) The standard and deviant stimuli were the same as in (b), but the target stimulus was an infrequent complex sound, which was very different from the standard and deviant stimuli. (d) The standard and deviant stimuli were tones with frequencies corresponding to the strongest harmonics of /a/ and /i/ in the vicinity of their first formants. The subject’s task was the same as in (b), i.e., to detect occasional stimulus-duration increments. Auditory cortices were activated in each condition. However, occipital areas were significantly activated only when the stimulation contained standard and deviant stimuli and the task directed the subject’s attention to the auditory stimulation (the stimulus-duration increments had to be detected). The figure also shows that, by and large, the same occipital areas were activated by the vowel and tone stimuli.
130
T. Kujala et al. / Neuroscience Letters 379 (2005) 127–131
Table 1 The number of significantly (p = 10−6 ) activated occipital-lobe pixels in the blind subjects in each condition Condition
S attend (Session A) S + D attend (Session A) S + D unattend (Session B) S + D vowels (Session C) S + D tones (Session C)
Subject AT
SL
TB
ST
TL
2 1 0 >30 >30
1 >10 0 0 >10
0 8 0 >30 >30
0 1 0 >10 3
0 >30 0 3 8
>10: over 10 activated pixels; >30: over 30 activated pixels; S: standard stimuli; D: deviant stimuli.
ditions including selectively-attended standard and deviant vowels and those including either selectively-attended standards (with no deviants) or standard and deviant vowels when the complex sound was the target. The data were collapsed as follows: (1) the conditions including standard and deviant vowels in Sessions A and C, and (2) the condition including standards without deviants in Session A, and that including standard and deviant vowels when the complex sound was the target in Session B. This comparison yielded a significant difference between the conditions (Wilcoxon Matched Pairs Test, Z = 2.02, p < 0.05). In addition, we compared the number of activated voxels between the different conditions, which revealed a significant effect (Friedman ANOVA, Chi-Square (d.f. = 4) = 13.21, p < 0.02). This analysis showed the following rank-order, starting with the condition with the largest number of activated voxels: selectively attended standard and deviant tones (Session C), average rank: 4.6; selectively attended standard and deviant vowels (Session C), 3.5; the same stimuli in Session A, 3.4; selectively attended standards (Session A), 2.1; and standard and deviant vowels when the complex sound was the target (Session B), 1.4. The occipital activation found during selective attention to features of sounds in the streams involving deviant sounds comprised the lateral and middle occipital gyri in the blind. Activity was also observed in the cuneus and lingual gyrus of some subjects. The calcarine sulcus was activated in three subjects by tone stimuli and in one subject by vowel stimuli. The comparison of the activations caused by vowels versus tones in Session C revealed that more wide-spread occipital activity was found for tones than vowels in four blind subjects, whereas in one blind subject the result was the opposite (Table 1). The results were as expected in three of the sighted subjects: no occipital activity was found in any of the conditions. However, in one of them, selectively attended stimulation including standard and deviant vowels (Session A) elicited clear occipital activity. This activity was also present in Session B, in which the target among the vowels was the complex sound. In another sighted subject, the attended monotonic stimulation (including only the vowel /a/ and the longer-duration vowel targets) in Session A elicited some occipital activity. Thus, the activation patterns were different in the two sighted
subjects compared with each other and compared with the blind subjects. Our results show that the occipital cortex of the blind processes auditory information under certain conditions. Occipital activity in blind humans was always observed in previous studies when attentional processing of the non-visual stimulation was required (for reviews, see [8,2,16]). More precisely, our results show that attentional processing per se is not sufficient for activating these areas, since they were activated only when occasional stimulus changes occurred in the sound sequences that were selectively attended. This is consistent with the results of previous studies using electroand magnetoencephalography showing that stimulus changes elicit activity in the posterior brain areas of the blind when the sound sequences are attended to but not when they are ignored [8,9,12]. Activity was found in several occipital areas of the blind subjects, even in the primary visual cortex in three out of five individuals. Regions of the lateral and middle occipital gyri were activated by both vowel and tone stimuli, and activity was also found in the cuneus and lingual gyrus in some subjects. Even the calcarine sulcus was activated (by tones in three subjects and by vowels in one). These results are in agreement with those of the majority of studies on early-blind human subjects, showing that the primary occipital areas and the regions close to them are activated by non-visual stimulation (see, e.g., [16,23,26,22]). According to a recent study, the primary occipital areas are involved in non-visual processes if the onset of blindness occurs before the age of 16 [22]. The tones activated larger occipital areas than the vowels in four out of five blind subjects. This result might suggest that stimulus novelty has an effect on the extent of occipital activity in the blind. The subjects were familiar with the vowels used in the present study, which were Finnish speech sounds, whereas they had never heard the tone stimuli, which were created so that they physically approximated the vowels but did not sound like speech. Thus, blind individuals’ occipital cortex might have a role in novelty detection. Somewhat unexpectedly, occipital activity was found in two sighted control subjects, too. There are three alternative explanations for this finding. First, the activity might reflect some visual imagery related to sounds in some conditions. Second, the subjects might have accidentally opened their eyes during the stimulation (even though they reported not having done so). Third, the activity might relate to the unmasking of some possible auditory input to the visual areas [4,19]. Support for such a possibility was found in a recent study showing that the occipital areas of sighted individuals were activated by auditory stimulation after they had been blindfolded for several days [15]. Furthermore, there is even evidence that a modality-specific brain area may be activated by stimuli of another modality even when there is no sensory deprivation [2]. Moreover, the occipital activity in the two sighted subjects of the present study did not follow the same pattern as in the blind subjects. Thus, according to these
T. Kujala et al. / Neuroscience Letters 379 (2005) 127–131
results, occipital activity in the sighted was neither stimulusor task-specific, nor did it reflect the same processes as in the blind subjects. The fact that the occipital activity was stimulus- and taskdependent across the blind subjects indicates that their occipital cortex has a specific role in auditory processing. These results, in agreement with those of previous studies [8,9,12], suggest that the occipital cortex of the blind processes sound changes in attended stimulus streams. Thus, this activity reflects higher-order processing rather than the early, automatic sound discrimination that occurs in the auditory cortex [25,13]. The activated occipital areas might belong to the network that also includes auditory-association areas, which were shown to be activated when (sighted) subjects detected targets in sound streams but not when the task involved no target detection [6]. Previous studies using behavioural measures have shown that the blind are more efficient than the sighted in detecting sound changes in an attended stream of sounds [10,12]. Efficiency in terms of rapid attention switching towards novel signals perceived via intact modalities is essential in compensating for lost visual modality in order to rapidly detect new and possibly warning or otherwise significant events in the environment. The neural population of the visual cortex, which, according to our study, processes unpredictable sound changes, might contribute to this enhanced efficiency in the blind.
[6]
[7]
[8] [9]
[10]
[11]
[12]
[13] [14]
[15] [16] [17] [18]
Acknowledgements This study was supported by the Academy of Finland and the University of Helsinki. We thank Satu Palva, M.A. for assisting with the data analysis, and Joan Nordlund, M.A. for the language check. References [1] P. Alku, Glottal wave analysis with pitch synchronous adaptive inverse filtering, Speech Commun. 11 (1992) 109–118. [2] D. Bavelier, H.J. Neville, Cross-modal plasticity: where and how? Nat. Rev. 3 (2002) 443–452. [3] L.G. Cohen, P. Celnik, A. Pascual-Leone, B. Corwell, L. Faiz, et al., Functional relevance of cross-modal plasticity in blind humans, Nature 389 (1997) 180–183. [4] A. Falchier, S. Clavagnier, P. Barone, H. Kennedy, Anatomical evidence of multimodal integration in primate striate cortex, J. Neurosci. 22 (2002) 5749–5759. [5] E.R. Gizewski, T. Gasser, A. de Greiff, A. Bohem, M. Forsting, Cross-modal plasticity for sensory and motor activa-
[19]
[20]
[21]
[22]
[23]
[24] [25]
[26]
131
tion patterns in blind subjects, NeuroImage 19 (2003) 968– 975. C.L. Grady, J.W. Van Meter, J.Ma. Maisog, P. Pietrini, J. Krasuski, J.P. Rauschecker, Attention-related modulation of activity in primary and secondary auditory cortes, NeuroReport 8 (1997) 2511–2516. J. Hyv¨arinen, S. Carlson, L. Hyv¨arinen, Early visual deprivation alters modality of neuronal responses in area 19 of monkey cortex, Neurosci. Lett. 26 (1981) 239–243. T. Kujala, K. Alho, R. N¨aa¨ t¨anen, Cross-modal reorganization of human cortical functions, Trends Neurosci. 23 (2000) 115–120. T. Kujala, M. Huotilainen, J. Sinkkonen, A.I. Ahonen, K. Alho, et al., Visual cortex activation in blind humans during sound discrimination, Neurosci. Lett. 183 (1995) 143–146. T. Kujala, A. Lehtokoski, K. Alho, J. Kekoni, R. N¨aa¨ t¨anen, Faster reaction times in the blind than sighted during bimodal dividedattention task, Acta Psychol. 96 (1997) 75–82. N. Lessard, M. Pare, F. Lepore, M. Lassonde, Early-blind human subjects localize sound sources better than sighted subjects, Nature 395 (1998) 278–280. M. Liotti, K. Ryder, M. Woldorff, Auditory attention in the congenitally blind: where, when and what gets reorganized? NeuroReport 9 (1998) 1007–1012. R. N¨aa¨ t¨anen, I. Winkler, The concept of auditory stimulus representation in cognitive neuroscience, Psychol. Bull. 6 (1999) 826–859. U. Noppeney, K.J. Friston, K. Price, Effects of visual deprivation on the organization of the semantic system, Brain 126 (2003) 1620–1627. A. Pascual-Leone, R. Hamilton, The metamodal organization of the brain, Prog. Brain Res. 134 (2001) 427–445. J. Rauschecker, Cortical map plasticity in animals and humans, Prog. Brain Res. 138 (2002) 73–88. J. Rauschecker, U. Kniepert, Auditory localization behaviour in visually deprived cats, Eur. J. Neurosci. 6 (1996) 149–160. J. Rauschecker, M. Korte, Auditory compensation for early blindness in cat cerebral cortex, J. Neurosci. 13 (1993) 4538–4548. K.S. Rockland, H. Ojima, Multisensory convergence in calcarine visual areas in macaque monkey, Int. J. Psychophysiol. 50 (2003) 19–26. B. R¨oder, O. Stock, S. Bien, F. R¨osler, Speech processing activates visual cortex in congenitally blind humans, Eur. J. Neurosci. 16 (2002) 930–936. B. R¨oder, W. Teder-S¨alej¨arvi, A. Sterr, F. Rosler, S.A. Hillyard, et al., Improved auditory spatial tuning in blind humans, Nature 400 (1999) 162–166. N. Sadato, T. Okada, M. Honda, Y. Yonekura, Critical period for cross-modal plasticity in blind humans: a functional MRI study, NeuroImage 16 (2002) 389–400. N. Sadato, A. Pascual-Leone, J. Grafman, V. Ibanez, M.-P. Deiber, et al., Activation of the primary visual cortex by braille reading in blind subjects, Nature 380 (1996) 526–528. E.R. Strelow, J.A. Brabyn, Locomotion of the blind controlled by natural sound cues, Perception 11 (1982) 635–640. H. Tiitinen, P. May, K. Reinikainen, R. N¨aa¨ t¨anen, Attentive novelty detection in humans is governed by pre-attentive sensory memory, Nature 327 (1994) 90–92. R. Weeks, B. Horwitz, A. Aziz-Sultan, B. Tian, C.M. Wessinger, et al., A positron emission tomographic study of auditory localization in the congenitally blind, J. Neurosci. 20 (2000) 2664–2672.