Auditory Triggered Mental Imagery of Shape Involves Visual Association Areas in Early Blind Humans

NeuroImage 14, 129 –139 (2001) doi:10.1006/nimg.2001.0782, available online at http://www.idealibrary.com on

Auditory Triggered Mental Imagery of Shape Involves Visual Association Areas in Early Blind Humans Anne G. De Volder,* ,† Hinako Toyama,* ,‡ Yuichi Kimura,* Motohiro Kiyosawa,* ,§ Hideki Nakano, ¶ Annick Vanlierde,㛳 Marie-Chantal Wanet-Defalque,㛳 Masahiro Mishina,* ,** Keiichi Oda,* Kiichi Ishiwata,* and Michio Senda* *Positron Medical Center, Tokyo Metropolitan Institute of Gerontology, 35-2 Sakaecho, Itabashi-Ku, Tokyo 173-0015, Japan; 㛳Neural Rehabilitation Engineering Unit, †Positron Tomography Laboratory, University of Louvain, School of Medicine, B-1200 Brussels, Belgium; ‡National Institute of Radiological Sciences, Chiba, Japan; §Department of Ophthalmology, Tokyo Medical and Dental University, Tokyo, Japan; ¶Tsukuba University, Tokyo, Japan; and **The Second Department of Internal Medicine Nippon Medical School, Tokyo, Japan Received October 30, 2000; published online May 11, 2001

Previous neuroimaging studies identified a large network of cortical areas involved in visual imagery in the human brain, which includes occipitotemporal and visual associative areas. Here we test whether the same processes can be elicited by tactile and auditory experiences in subjects who became blind early in life. Using positron emission tomography, regional cerebral blood flow was assessed in six right-handed early blind and six age-matched control volunteers during three conditions: resting state, passive listening to noise sounds, and mental imagery task (imagery of object shape) triggered by the sound of familiar objects. Activation foci were found in occipitotemporal and visual association areas, particularly in the left fusiform gyrus (Brodmann areas 19-37), during mental imagery of shape by both groups. Since shape imagery by early blind subjects does involve similar visual structures as controls at an adult age, it indicates their developmental crossmodal reorganization to allow perceptual representation in the absence of vision. © 2001 Academic Press Key Words: functional imaging; human blindness; positron emission tomography; neural plasticity; visual cortex (extrastriate).

INTRODUCTION In humans a great variety of knowledge about the shape of objects in the environment is mediated by vision. Using this knowledge, sighted humans can elaborate mental images of a precise object without its actual presence. Neuropsychological studies have shown that both visual perception and visual imagery have common functional characteristics (e.g., subjects may confuse whether they have actually seen a stimulus or merely imagined seeing it; similar deficits in imagery and perception are observed following brain

damage, see Farah, 1988, and Kosslyn, 1994 for reviews). This led to the hypothesis that perception and imagery could share a similar processing network in the brain. However, visual imagery is not only considered a result of a bottom-up visual perception process used to build mental pictures. On the one hand, imagery is also a form of memory retrieval (nonverbal recall) and is involved in memory processes. For instance, positive effects of concrete nouns have been verified in paired-associate learning tasks, and this observation was interpreted as a facilitating effect due to visual imagery during these tasks (Paivio, 1995). On the other hand, imagery can also be regarded as participating in the additional top-down processes that are involved in complex visual perception tasks. For instance, according to Kosslyn and Sussman (1995), visual imagery is used to complete fragmented perceptual inputs or to match shapes during object recognition tasks. To perform visual imagery of familiar objects, knowledge referring to visual properties of these objects is required. For a sighted subject, this knowledge (visual semantics or sensory knowledge, e.g., the shape of a musical instrument) can be accessed by pictures and also indirectly by visual words (Caramazza, 1996). In subjects deprived of vision early in life (at birth or in the first years of life), the knowledge of the surrounding environment must be accessed by nonvisual modalities. Nevertheless, it has been proved that congenitally or early blind (EB) humans are also able to perform mental imagery tasks. For instance, Marmor and Zaback (1976) observed that EB subjects could mentally rotate tactually experienced shapes in a similar way as the sighted controls using the same visually presented shapes. In the study of Kerr (1983), the properties of mental images were studied both in sighted and EB subjects and, in both groups, a linear relationship was observed between the scanning time

129

1053-8119/01 $35.00 Copyright © 2001 by Academic Press All rights of reproduction in any form reserved.

130

DE VOLDER ET AL.

and the distance between two given objects in the mental images. Some EB subjects are also able to draw and to paint, using similar figurative representations as sighted controls to depict objects (Kennedy, 1982; Cornoldi et al., 1993). These results indicate that mental imagery would not exclusively depend on visual experience but could also be obtained using other sensory modalities. However, behavioral differences are observed as well, since the mental imagery tasks described above are performed at a significantly slower rate by EB subjects as compared with sighted controls (Marmor and Zaback, 1976; Kerr, 1983). Moreover, several investigators observed capacity limitations for mental representations (Cornoldi and De Beni, 1988; Cornoldi et al., 1989) and for visuospatial imagery tasks (Cornoldi et al., 1991; Vecchi et al., 1995; Vecchi, 1998) in EB subjects. For instance, EB subjects experienced more difficulty than sighted controls when the imagery task involved a high memory load (Cornoldi and De Beni, 1988; Cornoldi et al., 1989). Recent advances in neuroimaging techniques now allow noninvasive investigation of the functional state of the human brain. Regional cerebral blood flow (rCBF) measured with positron emission tomography (PET) provides an index of the local neural function related to synaptic activity. Previous neuroimaging studies have shown that common cerebral areas were involved in both perception and imagery tasks (Kosslyn et al., 1993, 1997; Roland and Gyluas, 1995; D’Esposito et al., 1997). In particular, the ventral visual pathway, including the occipitotemporal and inferior temporal cortex, was shown to be involved in visual imagery in sighted subjects (Kosslyn, 1994; Roland et al., 1995). However, this system is deprived of any visual afferences in the blind. This present study aims to clarify the neural structures involved in mental representation of the shape of familiar objects and to verify whether a similar system is recruited in subjects with early onset blindness, in whom internal representations of shape are only attainable through tactile and auditory experiences. The activation of the so-called visual areas in the blind subjects during shape imagery would indicate a functional reorganization of these neural structures during early development allowing perceptual representations from auditory and tactile inputs in the absence of vision. MATERIALS AND METHODS Subjects The PET studies were carried out on 6 male Japanese volunteers (mean age ⫾ SD: 24.3 ⫾ 2.4 years) who were affected by complete blindness (absence of light perception) as the result of bilateral ocular or optic nerve lesions, but who were otherwise neurologically normal. All these subjects were considered early blind

(EB) since they had very poor vision, if at all, at birth and lost sight completely including light sense under the age of 6 at latest, before completion of visual development. Accordingly, a potential visual experience can not be formally excluded in the early blind subjects during their first years after birth. A summary of their medical history is provided in Table 1. PET measurements in EB subjects were compared with those obtained in six age-matched blindfolded male volunteers (sighted controls (SC), mean age ⫾ SD: 22.7 ⫾ 0.8 years, P ⫽ 0.14) who present similar metabolic patterns as subjects with late-onset blindness according to previous studies (Phelps et al., 1981; Veraart et al., 1990; Buechel et al., 1998a). All subjects were righthanded. Informed consent was obtained before the PET study, the protocol of which had been approved by the Ethics Committee at the Tokyo Metropolitan Institute of Gerontology. Experimental Design The subjects were scanned during three conditions, each repeated twice in counterbalanced order across the subjects: (1) a resting state condition (REST); (2) a mental imagery task (IMAG) triggered by audition of familiar sounds; and (3) a control auditory task (CONT) using noise stimuli. Stimuli. Familiar sounds were recorded from sounding objects (n ⫽ 33) according to the following criteria: characteristic sound allowing easy identification of the object, maximal object size between 10 and 40 cm to allow easy tactile exploration, familiarity (everyday Japanese tools [e.g., toothbrush brushing, Japanese teapot (n ⫽ 25)] or objects known as Western tools in Japan [e.g., Western handbell ringing (n ⫽ 8)], and rather complex shape configuration allowing elaborate mental imagery. The 33 objects were selected out of a larger number of objects (n ⫽ 50) on the basis of results of behavioral testing in a separate group of 10 sighted volunteers. During this distinct testing, sounds and pictures of objects were presented for 5 s each. Immediately after this short familiarization, the subject’s ability to recognize the sounds was assessed and the sounds that were not recognized within 3.5 s were then rejected. For the remaining 33 objects, reaction times for sound recognition averaged 3 ⫾ 0.4 s. With a 5-s presentation time, all subjects depicted mental imagery of related objects and were able to provide, on average, 2.7 descriptive words related to each object, with 88 ⫾ 6% accuracy. Control sounds (n ⫽ 33) were created by mixing the frequencies of each meaningful sound as analyzed in Wavelab Demo 1.5. After reversing the sound files and adding echoes with a delay ranging from 100 to 400 ms, all sounds were made unrecognizable, although these noise stimuli were matched with the familiar sounds

PET STUDY OF MENTAL IMAGERY IN EARLY BLINDNESS

for frequency, duration, and intensity. Pretests performed on the same separate group of 10 sighted volunteers confirmed that these noise stimuli were indeed unrecognizable (accurately identified as unrelated to any object, with a mean performance of 94 ⫾ 4%). For the PET experiments, individual stimuli (familiar or control sounds) were associated in sequences of 17 sounds (each of 6-s duration with a 1-s silent break between sounds), using the SoundEdit 16 program (Macromedia Inc., San Francisco, CA). Training. Two 40-min training sessions were performed during which all 33 objects were named and presented one by one to the subject. Blind subjects were required to explore each object by touching it during one minute, while receiving the corresponding sound through earphones and to build up a precise mental representation of the object, representing it as a solitary stationary object with its longer side horizontally. The same instructions were provided to sighted controls who explored the objects visually and auditorily. Positron Emission Tomography Experimental conditions during PET scan. The mental imagery task (IMAG) required silent identification of each individual meaningful sound, provided by earphones, followed by mental retrieval of the internal representation of the related object. The subject was instructed to recall the shape of the object as precisely as possible, representing it with its longer side horizontally. Listening to the control sounds was used as a control auditory task (CONT), which was designed to allow subtracting away from IMAG data all activities related to auditory perception and processing. Unrecognizable sounds were used because they do not activate any semantic or episodic knowledge nor evoke a name or a picture and yet were psychophysically at least as complex as the stimuli used in the mental imagery task. The subjects also underwent basal resting scans (REST) without auditory stimulation, being instructed to relax without focusing their mind on any item. SC subjects were blindfolded. Data acquisition. Positron emission data were collected in Tokyo Metropolitan Institute of Gerontology with a Headtome-IV tomograph (Shimadzu), which allows imaging of 14 transaxial slices with a 6.5 mm center-to-center slice interval in two-dimensional (2-D) mode, with a spatial resolution of 7.5 mm full-widthat-half-maximum (FWHM) (Iida et al., 1989) and a slice thickness of 10 mm. A specially designed headholder with a restrictive inflated cushion was used to keep the head motionless in the canthomeatal orientation while allowing to wear earphones. Since PaCO2 is known to influence the cerebral blood flow (Knudsen et al., 1991; Shimosegawa et al., 1995), the end-tidal CO 2

131

concentration was continuously monitored with a gas analyzer (Respina, San-ei Co., Japan). Measurements were performed using a 5-s intravenous bolus injection of oxygen-15-labeled water (27 mCi, 1 GBq) following a protocol described previously (Kiyosawa et al., 1996). Beginning at tracer infusion, the related task was started and PET data were acquired simultaneously in a single 120-s frame. Emission images were reconstructed using filtered back projection with attenuation correction by transmission. The images of radioactivity averaged across the scan were used as an index of rCBF. The time interval between successive emission scans was 12 min, which allowed decay of residual radioactivity and was also used to assess the level of vigilance by asking the subject to discriminate whether five additional stimuli of the same type were or were not presented during the experimental sequence just before. For each subject, 3-D MRI anatomical data were also obtained in the bicommissural (AC–PC) orientation, using a 1.5 Tesla unit (General Electric Signa) and the Spoiled Grass (SPGR) technique (T-1-weighted images, TR ⫽ 13 ms, TE ⫽ 2.1 ms, flip angle ⫽ 20°, slice thickness ⫽ 1.3 mm). Data analysis. PET images were realigned to correct for interscan movements and coregistered to the subject’s MRI using AIR 3.0 (Woods et al., 1998a,b). The resulting matching brain images (MRI and coregistered PET) were spatially normalized, using SPM 96 (Wellcome Department of Cognitive Neurology), in the Talairach and Tournoux (1988) coordinate system with a cubic (2 ⫻ 2 ⫻ 2 mm) voxel size. PET images were further smoothed with an isotropic Gaussian filter (15-mm full-width at half maximum) and were corrected for differences in global activity by proportional scaling (Fox et al., 1988). Statistical parametric maps were computed on a voxel-by-voxel basis, using the general linear model (Friston et al., 1995). Within each group, a factorial design (Friston et al., 1995; Frackowiak et al., 1997) was used with two trial factors for condition (IMAG, CONT, REST) and for repetition (first and second scan), thereby generating Z maps, which tested main effects of the factors as well as interaction between two factors (Friston et al., 1995). Three main comparisons were made: (1) [(IMAG1⫹ IMAG2) ⫺ (CONT1⫹CONT2)], for the main effect of activating condition; (2) [(IMAG1⫹CONT1⫹REST1) ⫺ (IMAG2⫹CONT2⫹REST2)], testing the main effect of repetition; (3) ⫾ [(IMAG1-CONT1) ⫺ (IMAG2CONT2)], testing the interaction between repetition and condition factors in order to control for practicerelated changes in brain areas subtending shape knowledge (Raichle et al., 1994; Wagner et al., 1997). Differences due to blindness were similarly assessed using a group ⫻ condition interaction analysis. Common activation irrespective of subject group (EB or SC)

132

DE VOLDER ET AL.

TABLE 1 Profile of the Blind Subjects

Subjects

Age

Resting CBF in visual cortex (*)

1

24

184%

10 months

2

23

178%

Birth–4 years

3

24

178%

Birth–3 years

4

24

173%

Birth

5

22

176%

Birth–6 years

6

29

196%

Birth

Onset of blindness

Diagnosis Fulminant conjunctivitis, keratomalacia, and phtysis bulbi Congenital retinoblastoma (enucleated) Left eye enucleated when 1 year old Right eye enucleated when 4 years old Congenital glaucoma Left eye: visual acuity 0.01, enucleated when 3 years old; Right eye: buphtalmos Congenital glaucoma Bilateral phtysis bulbi after surgery (when 4 years) Congenital hypertrophic primary vitreous and bilateral phtysis bulbi. Left eye: blind since birth Right eye: cataract, lost light sense by 6 years old Congenital microphtalmia with leucoma cornea

Handedness (**)

Performance (***)

Right 8/10

60.0

Right 8/10

95.7

Right 9/10

78.3

Right 6/10

82.6

Right 6/10

82.6

Right 10/10

82.6

Note. (*) Pattern of cerebral blood flow at rest in visual cortex (% of whole brain mean value, at ⫺18, ⫺80, 0 (x,y,z) mm from AC, in BA 18). (**) Handedness scale (/10) according to Oldfield (1971). (***) Behavioral performance (percentage of correct answers) after PET (see text). As subjects 2, 3, and 5 had very poor vision from birth and lost their vision completely (including light perception) by the age of 3 to 6 years, they were considered early blind.

was assessed by conjunction analysis (Price et al., 1997). Statistical inference on the SPM {Z} was corrected for multiple comparisons using the theory of Gaussian fields (Friston et al., 1995). Only regions significantly activated at P ⬍ 0.05 (corrected for multiple comparisons) and P ⬍ 0.001 (uncorrected for multiple comparisons) were considered. Behavioral Assessment of Performance The accuracy of mental imagery was verified after the PET session as follows: while listening again to each individual meaningful sound, the subject was asked to retrieve the mental representation of the related object and to answer a series of questions on its shape attributes (e.g., “Are the holes of this object [scissors] equal in size ?”). RESULTS Behavioral Results Stable end tidal CO 2 was observed throughout the studies. All subjects gave 100% correct answers during assessment of vigilance (detection of individual sounds during the scans) for meaningful sounds and ⬎75% accuracy in the corresponding questioning for control sounds. The performance in mental imagery, as assessed by questions on shape attributes after the PET session, was satisfactory (Table 1) since the amount of details the subjects could retrieve on this trial averaged 80.3 ⫾ 11.6% (mean ⫾ SD) in EB group. Although the performance in SC group was slightly better

(89.1 ⫾ 5.3%), there was no significant difference between the two groups (P ⫽ 0.12, t test). PET Results As previously shown in EB subjects (Wanet-Defalque et al., 1988; Buechel et al., 1998a; Sadato et al., 1998; Breitenseher et al., 1998) the structural anatomy of occipital areas was found to be normal on MRI whereas high relative radioactivity was observed in the visual cortex at rest in each individual (i.e., mean regional activity: 181.0 ⫾ 8.1% of whole brain mean in EB subjects versus 157.6 ⫾ 9.7% in SC group, at ⫺18, ⫺80,0 (x,y,z) mm from AC, in BA 18, uncorrected P ⫽ 0.0006). Main effect of mental imagery versus auditory processing: Blind subjects. The brain regions observed to be involved in the mental imagery task (contrast 1: [(IMAG1⫹IMAG2) ⫺ (CONT1⫹CONT2)]) in EB group are presented in Fig. 1 and Table 2. Mental imagery by the blind selectively activated a large strip of lateral occipito-temporal areas including the posterior part of fusiform gyrus (BA 19 –37), more pronounced on the left side, and the posterior part of the inferotemporal gyrus (BA 37). The focus of activity also extended medially to the border between the fusiform gyrus, the lingual gyrus (BA 18) and the posterior part of the cerebellar vermis (Fig. 1). No activation was found in any prefrontal region nor in auditory cortex. A trend to activation was disclosed in the primary visual cortex, albeit to a nonsignificant level (P ⫽ 0.007, uncorrected for multiple comparisons, see Table 2). All significant foci in this contrast corresponded to real activations

PET STUDY OF MENTAL IMAGERY IN EARLY BLINDNESS

133

FIG. 1. Pattern of activation related to mental imagery of shape (IMAG) as contrasted to control auditory task (CONT) in early blind subjects. (a) The statistical parametric map for this comparison is superimposed on the axial, coronal, and sagittal sections of an individual normalized MRI. Only positive differences exceeding a threshold of P ⬍ 0.001 (uncorrected) are shown, according to the color scale. A bilateral activation of the ventral visual pathway is evident in this contrast. The intersection of the lines indicates a voxel in the left fusiform gyrus with a Z value of 5.0 (P ⬍ 0.05 corrected for multiple comparisons, coordinates with reference to the Talairach and Tournoux (1988) system). (b) The regional cerebral blood flow, expressed as the radioactivity (whole brain mean ⫽ 100) at this voxel, is plotted for each scan of each subject. Two scans are averaged as bars and broken lines are the average of six subjects. Difference between IMAG and CONT (⫹ 8.2%) is due to real activation because the blood flow in both conditions is well above the basal resting state (REST).

during mental imagery since the regional activity differences observed between IMAG and REST for these regions were always larger then or equal to the regional activity differences between IMAG and CONT (Fig. 1, Chart). The main repetition effect (contrast #2) was significant in widespread regions including those areas described above, confirming that the order of scanning had an influence on regional cerebral blood flow, although similar results were obtained for behavioral estimation of vigilance during scans. The image values at all activation foci voxels, with the exception of the left inferotemporal gyrus, were significantly decreased during the second scanning as compared with the first one. However, no significant change in these regions was detected when the interaction between condition and repetition was assessed (contrast 3: [(IMAG1CONT1) ⫺ (IMAG2-CONT2)]), even at a very low statistical threshold (i.e., height: P ⬍ 0.01; size: 20 voxels).

Therefore, we may consider that there was no influence of repetition on the observed difference in activation patterns between the two tasks and that the taskrelated differences appeared to remain stable over time. As an attempt to evaluate the effect of the onset time for total blindness, the six subjects were further categorized as very early onset (before 1 year of age, EB 1-4-6) and slightly later onset (EB 2-3-5) subgroups: both subgroups disclosed the same activation patterns, without any significant difference in the condition ⫻ subgroup interaction analysis (i.e., height: P ⬍ 0.01; size: 20 voxels). Main effect of mental imagery versus auditory processing: sighted controls. During the visual imagery task (contrast 1: meaningful minus control sounds), the blindfolded-sighted controls activated the same visual association areas as the EB subjects (Fig. 2), with two

134

DE VOLDER ET AL.

FIG. 2. Statistical parametric maps (SPM{Z}) as a maximum intensity projection showing the activations due to mental imagery relative to auditory processing in early blind subjects (top raw) and in blindfolded sighted controls (bottom raw). Views are from the right, behind and top. Only the voxels with larger regional activity during mental imagery as compared to the basal resting state were considered in this comparison. Voxels exceeding a threshold of P ⬍ 0.001 (uncorrected) are shown. A bilateral activation of visual association areas is present in both groups but more pronounced in EB subjects (top images), whereas SC subjects (bottom images) show additional foci in the right prefrontal cortex and in the left cingular gyrus. Coordinates are with reference to the Talairach and Tournoux (1988) system.

striking differences. First, in the SC group there was a significant activation in the right prefrontal cortex (medial frontal gyrus, BA 11) and in the left posterior cingular gyrus (BA 30), which was absent in the early blind subjects (Table 2). The second difference concerned the lower z scores found in all activation foci, including the fusiform gyrus, with the exception of the vermis. The maximally activated voxels, as indicated by the highest Z score within the fusiform gyri, were also more caudal and extended mainly to BA 20 rather than BA 19-37. Similarly as in the EB group, no significant activation was found in the primary visual cortex nor in auditory regions. When compared with the rest condition, all significant foci in IMAG minus CONT corresponded to real activations above the basal resting state. No significant change in these regions was detected when the interaction between condition and repetition was assessed (contrast 3), except for 35 voxels of the vermis where higher activity was found during the first IMAG trial only. Interaction: mental imagery activations in blind versus sighted subjects. A direct comparison of the differences in mental imagery activations (contrast 1, IMAG minus CONT) between the EB and SC groups showed a statistically significant group-by-condition interaction in the right inferior temporal gyrus (BA 19), in the left lingual gyrus (BA 18) and in a limited

part of the right fusiform gyrus (BA 18, 21 voxels), in the ventral visual pathway (Fig. 3; Table 3). This differential effect was due to activation in EB subjects during the mental imagery task whereas the sighted control group showed mostly decreased rCBF in these regions. On the other hand, larger activation was seen in the SC group as compared with the EB group in the right medial frontal gyrus only (BA 11), where decreased rCBF was found in the EB group during the mental imagery task. Consistently activated areas, irrespective of the group (EB or SC) and repetition (first or second trial) are shown in Fig. 3. In both groups, the mental imagery task, as contrasted to the control auditory task, activated consistently a large strip of visual association areas in the ventral pathway including especially the left fusiform gyrus (BA 37-19) and the cerebellar vermis. At a lower significant level, the right fusiform gyrus (BA 37) was also consistently activated in both EB and SC subjects (conjunction analysis, Table 3), irrespective of trial repetition. Auditory activations in blind versus sighted subjects. The comparison of the effects due to auditory processing (CONT), as compared to resting state, in EB relative to SC subjects revealed common activation in bilateral temporal regions (BA 41-42 and BA 21-22),

135

PET STUDY OF MENTAL IMAGERY IN EARLY BLINDNESS

TABLE 2 Activations Comparing Mental Imagery with Auditory Processing Coordinates Anatomical region

Side

x

y

z

z score

P value corrected

Uncorrected

L L R R R L R L R L

⫺44 ⫺58 ⫺4 30 34 5 ⫺5 42 ⫺42 62 ⫺62

⫺68 ⫺64 ⫺72 ⫺62 ⫺68 ⫺78 ⫺78 ⫺28 ⫺28 ⫺25 ⫺25

⫺12 2 ⫺12 ⫺14 4 12 12 10 10 3 3

5.01 3.00 3.88 4.35 3.72 2.43 1.84 ⫺1.93 ⫺1.94 ⫺1.96 0.42

0.002 n.s. n.s. 0.03 n.s. n.s. n.s. n.s. n.s. n.s. n.s.

3.00E-07 0.001 5.00E-05 7.00E-06 0.0001 0.007 0.030 0.027 0.026 0.025 0.339

R R L L R L R L R L

8 18 62 ⫺4 ⫺42 4 ⫺6 42 ⫺42 62 ⫺62

⫺72 34 ⫺46 ⫺46 ⫺62 ⫺78 ⫺78 ⫺28 ⫺28 ⫺25 ⫺25

⫺24 ⫺12 ⫺20 16 ⫺24 12 12 8 10 3 3

4.35 4.02 3.36 3.74 3.22 ⫺1.57 ⫺0.38 ⫺0.53 1.03 0.31 ⫺0.35

0.03 n.s. n.s. n.s. n.s. n.s. n.s. n.s. n.s. n.s. n.s.

7.00E-06 3.00E-05 0.0004 9.00E-05 0.0006 0.059 0.350 0.296 0.152 0.377 0.364

EB subjects Fusiform gyrus (BA 19) Infero-temporal gyrus (BA 37) Posterior part of vermis Fusiform gyrus (BA 37-19) Extension to Lingual gyrus (BA 18) Primary visual cortex (BA 17) (*) Heschl gyrus (BA 41-42) (*) Superior Temporal gyrus (BA 22) (*) Control subjects Vermis Medial frontal gyrus (BA 11) Infero-temporal-fusiform gyrus (BA 20) Posterior cingular gyrus (BA 30) Fusiform gyrus (BA 20) Primary visual cortex (BA 17) (*) Heschl gyrus (BA 41-42) (*) Superior Temporal gyrus (BA 22) (*)

Note. BA, Brodmann area; coordinates are according to Talairach and Tournoux (1988) and refer to the voxel with the highest z score within each area; (*) described here for reference though not significant.

extending slightly more posteriorly and caudally in EB than in the SC group.

retrieval of shape information and mental imagery of objects in early blind and sighted humans. The mental imagery task required the subjects to identify the trigger sound and to retrieve a representation of the “visual” attributes of the stimuli (shape, size, configuration), which had been previously accessed by haptic perception in EB subjects whereas

DISCUSSION This study shows a selective activation of occipitotemporal areas and of the cerebellar vermis during

TABLE 3 Group Comparisons for Activations during Mental Imagery as Compared with Auditory Processing Coordinates Anatomical region

Side

x

y

z

z score

P value corrected

Uncorrected

L

⫺68 ⫺72 ⫺60

⫺18 ⫺20 ⫺16

4.57 5.43 3.43

0.009 0.0002 n.s.

2.00E-06 3.00E-08 0.0003

Consistently activated foci (both groups) Fusiform gyrus (BA 37-19) Posterior part of vermis Fusiform gyrus (BA 37)

R

⫺40 4 18

EB subjects ⬎ Control subjects Infero-temporal gyrus (BA 19) Fusiform gyrus (BA 18) Lingual gyrus (BA 18)

R R L

34 22 ⫺22

⫺66 ⫺88 ⫺74

⫺2 16 ⫺2

4.04 3.22 3.69

n.s. n.s. n.s.

3.00E-05 0.0006 0.0001

Control subjects ⬎ EB subjects Medial frontal gyrus (BA 11)

R

16

28

⫺12

3.90

n.s.

5.00E-05

BA ⫽ Brodmann area; coordinates are according to Talairach and Tournoux (1988) and refer to the voxel with the highest z-score within each area.

136

DE VOLDER ET AL.

FIG. 3. Statistical parametric map (SPM{Z}) of conjunction analysis (Top) and interaction (EB ⬎ SC) analysis (Bottom) for the mental imagery task relative to auditory processing. Top images: consistently activated areas irrespective of the subject group (EB or SC) and repetition (first or second trial) are shown (hot metal color scale), superimposed on the axial, coronal and sagittal sections of an individual normalized MRI. The voxels in the left fusiform gyrus (BA 37-19), in the vermis and in the right fusiform gyrus, mesially, were commonly activated during mental imagery, relative to auditory processing, by early blind subjects and by sighted controls (P ⬍ 0.001, uncorrected). Cursor lines cross at (⫺40, ⫺68, ⫺18) [ z ⫽ 4.57, P ⫽ 0.009 with correction for multiple comparisons] in the left fusiform gyrus. Bottom images: brain areas with a larger activation in the EB group than in the SC one are shown in blue (P ⬍ 0.001, uncorrected). These areas are found around the commonly activated foci, as well as in the analogous regions in the opposite hemisphere.

these attributes were processed by vision in sighted subjects. For this task, each considered group used a specifically usual and familiar sensory modality as the entry level: a visual entry for the sighted controls and a tactual entry for the blind. Furthermore, a good performance was recorded in the behavioral testing after the PET session in both groups, which indicated that the subjects were similarly able to generate a precise mental representation of the objects during the scans. In addition, in order to restrict any verbal processing when accessing the object images, characteristic sounds were used rather than the name of objects during training and during scans. Accordingly, in both groups the task-related rCBF increases in language processing areas were rather limited (unsignificant trends in associative auditory cortex, BA 22) or absent (no rCBF changes in the inferior frontal cortex including the Broca area, BA 44).

With the exception of the vermis, only visual association areas (BA 19-37) were significantly involved in the specific operations related to the mental imagery by the blind. In sighted controls, the same visual association areas (fusiform gyrus and inferior temporal cortex) were activated during mental imagery although this activation was more caudal than in the blind and extended mainly to BA 20. The same brain regions have been previously found to be activated in sighted individuals during complex visual tasks such as attention to shapes and visual processing of objects (Haxby et al., 1991), visual form discrimination (Gulyas et al., 1994) and object identity retrieval (Moscovitch et al., 1995) or recognition (Malach et al., 1995), with a tendency for object processing to be left-lateralized (Farah, 1995). Consistent with the idea that imagery and perception have common neural substrates, imagery tasks in sighted subjects have yielded activa-

PET STUDY OF MENTAL IMAGERY IN EARLY BLINDNESS

tions in occipital regions, including the fusiform gyrus especially when the tasks involved activation of visual memories (Kosslyn et al., 1993; Roland et al., 1995; D’Esposito et al., 1997). The present results confirm this activation of higher order visual association cortex in a visual imagery task triggered by auditory signals. The primary visual cortex (BA 17) was not recruited by the task, neither in the SC group, nor in the EB group who showed only unsignificant activation trends during mental imagery of objects. Previous neuroimaging studies in sighted subjects have demonstrated that some types of mental imagery do not rely on primary visual areas (see Kosslyn et al., 1997 and Mellet et al., 1998 for reviews). According to Kosslyn (1994), the high resolution depictive representations (e.g., mental inspection of detailed patterns) are more likely to activate the primary visual cortex whereas low resolution figurative representations (e.g., global object shape) are not. In the present study, the required task could probably be considered a figurative imagery task that would provide a weak resolution representation since the mental image generation condition was based on a mental retrieval of global object shapes during the scan. Moreover, although the subjects were required to recall object shapes as precisely as possible, no questions were asked during the PET session. Therefore, it is likely that this mental imagery task would promote an activation of the inferior temporal cortex (BA 19-37) without recruitment of area 17, since only high resolution representations are presumed to activate the primary visual cortex. In the present study, auditory triggered mental imagery of object shape seems to involve the ventral pathway in EB and in SC subjects in the same way, although the activated areas tend to be more caudal in sighted controls who used vision in training. Since the imagery task involved a form of memory retrieval (nonverbal recall) this difference of entry modality, together with related differences in attentional load or working memory processes could hypothetically account for this slight inhomogeneity in visual association activation pattern as well as for the additional involvement of BA 11 in SC subjects. Differences in the cognitive processes that are involved during mental imagery have been found to affect notably the recruitment of non-visual areas in addition to the visual association cortex (see Mellet et al., 1998 for review). However, this hypothesis remains speculative since testing this point would require comparisons between EB and SC subjects having both explored the objects tactually (and this would need an extensive training of SC who are not experts in tactile exploration), as well as comparisons between tactile and visual exploration to trigger mental imagery in SC, which are behind the scope of this study. The results of this experiment support the hypothesis that mental imagery of objects recruits the ventral

137

visual pathway extending to occipitotemporal and inferior temporal cortex in sighted subjects. In total accordance, our findings indicate a similar involvement of the occipitotemporal cortex and a critical role of the ventral visual pathway in mental imagery in the early visually deprived subjects. According to the theory of neuronal group selection, as proposed by Edelman in 1978 (see Edelman (1993) for review), the functional organization of the brain is based, like in Darwin’s theory of natural selection, on competition at the synaptic level. In the developing brain, selection occurs during synapse formation and elimination, which results in enormous synaptic loss during brain maturation (Changeux and Danchin, 1976). To account for this adaptive selection, transient connections, such as auditory afferences to the visual cortex (Innocenti and Clarke, 1984) or corticocortical projections between primary and secondary visual areas (Berman, 1991), have been found in the early life, which are normally withdrawn during brain development but may persist in case of sensory deprivation (Berman, 1991). The “neural Darwinism” theory would predict that, when one sensory modality is lacking as in congenital blindness, the target structures would be taken over by the afferent inputs from other senses which would promote and control their functional maturation (Edelman, 1993). This view receives support both from crossmodal plasticity experiments in animal models and from functional imaging studies in man. On one hand, a reorganization of sensory representations with crossmodal expansion of nonvisual modalities into normally visual brain areas has been demonstrated, using electrode recordings, in early visually deprived animals (Rauschecker, 1995). These physiological modifications might be partly related to behavioral abilities of congenitally blind cats, who are for instance faster in learning tactile exploration of space or sound localization than blindfolded controls (Rauschecker and Korte, 1993). On the other hand, PET studies demonstrated that Braille reading and tactile discrimination tasks or auditory localization in EB humans caused increased blood flow in their occipital areas (Sadato et al., 1996; Buechel et al., 1998a; Cohen et al., 1999; Weeks et al., 2000). Cognitive tasks with visual components, such as visual linguistic processing (Buechel et al., 1998b), also activated visual association areas in the ventral pathway in EB subjects. In accordance with data obtained with evoked potentials (Uhl et al., 1994), the present study indicates that this is also the case for mental imagery. This convergence of results suggests that the inputs from auditory and tactile modalities are capable of promoting efficient functional development of the ventral visual pathway in the absence of vision. According to this view, the brain localization of functions is considered to be predetermined and reinforced during infancy as a function of utilization and based on appro-

138

DE VOLDER ET AL.

priate selection of available afferent stimuli. Since our subjects were congenitally deprived of adequate visual stimulation, reciprocal interactions between the other senses, the mental activity, and the association visual cortex must have contributed to the self-organization of these neuroanatomical structures as well as to the creation of shape knowledge (“visual semantics”) and imagery abilities. We hypothesize that auditory and tactile senses, at least partly, helped to create vision in the brain, acting as a natural substitutive system for vision during brain maturation and that this in turn enabled development of the specific visual functions for which these anatomical structures are responsible. ACKNOWLEDGMENTS A. G. De Volder gratefully acknowledges the Tokyo Metropolitan Institute of Gerontology for support through a research fellowship. Thanks are due to the volunteers, to the TMIG cyclotron staff for isotope preparation, to Ms. M. Ando for assistance with the subjects and to Dr. J. M. Bodart for software development. A.D.V. is research associate at the Belgian National Funds for Scientific Research.

REFERENCES Berman, N. E. J. 1991. Alterations of visual cortical connections in cats following early removal of retinal input. Dev. Brain Res. 63: 163–180. Breitenseher, M., Uhl, F., Prayer Wimberger, D., Deecke, L., Trattnig, S., and Kramer, J. 1998. Morphological dissociation between visual pathways and cortex: MRI of visually-deprived patients with congenital peripheral blindness. Neuroradiology 40: 424 – 427. Buechel, C., Price, C., Frackowiak, R. S. J., and Friston, K. 1998a. Different activation patterns in the visual cortex of late and congenitally blind subjects. Brain 121: 409 – 419. Buechel, C., Price, C., and Friston, K. 1998b. A multimodal language region in the ventral visual pathway. Nature 394: 274 –277. Caramazza, A. 1996. Pictures, words and the brain. Nature 383: 216 –217. Changeux, J. P., and Danchin, A. 1976. Selective stabilisation of developing synapses as a mechanism for the specification of neuronal networks. Nature 264: 705–712. Cohen, L. G., Weeks, R. A., Sadato, N., Celnik, P., Ishii, K., and Hallett, M. 1999. Period of susceptibility for cross-modal plasticity in the blind. Ann. Neurol. 45: 451– 460. Cornoldi, C., and De Beni, R. 1988. Weaknesses of imagery without visual experience: The case of total congenital blind using imaginal mnemonics. In Cognitive and Neuropsychological Approaches to Mental Imagery (M. Denis, J. Engelkamp, and J. T. E. Richardson, Eds.), Martins Nijhoff Publisher, Netherlands. Cornoldi, C., De Beni, R., Roncari, S., and Romano, S. 1989. The effects of imagery instructions on total congenital blind recall. Eur. J. Cogn. Psychol. 4: 321–331. Cornoldi, C., Cortesi, A., and Preti, D. 1991. Individual differences in the capacity limitations of visuospatial short-term memory: Research on sighted and totally congenitally blind people. Memory Cogn. 19: 459 – 468. Cornoldi, C., Bertuccelli, B., Rocchi, P., and Sbrana, B. 1993. Processing capacity limitations in pictorial and spatial representations in the totally congenitally blind. Cortex 29: 675– 689.

D’Esposito, M., Detre, J. A., Aguirre, G. K., Stallcup, M., Alstop, D. C., Tippet, L. J., and Farah, M. J. 1997. A functional MRI study of mental image generation. Neuropsychologia 35: 724 –730. Edelman, G. M. 1993. Neural Darwinism: selection and reentrant signaling in higher brain function. Neuron 10: 115–125. Farah, M. J. 1988. Is visual imagery really visual ? Overlooked evidence from neuropsychology. Psychol. Rev. 95: 307–317. Farah, M. J. 1995. The neural bases of mental imagery. In The Cognitive Neurosciences (M. S. Gazzaniga, Ed.), pp. 963–975. MIT Press, Cambridge. Fox, P. T., Mintun, M. A., Reiman, E. M., and Raichle, M. E. 1988. Enhanced detection of focal brain responses using intersubject averaging and change distribution analysis of subtracted PET images. J. Cereb. Blood Flow Metab. 8: 642– 653. Frackowiak, R. S. J., Friston, K., Frith, C. D., Dolan, R. J., and Mazziotta, J. C. 1997. Human Brain Function, Academic Press, London, UK. Friston, K. J., Holmes, A. P., Worsley, K. J., Poline, J. B., Frith, C. D., and Frackowiack, R. S. J. 1995. Statistical parametric maps in functional imaging: A general linear approach. Hum. Brain Mapp. 2: 189 –210. Gulyas, B., Heywood, C. A., Popplewell, D. A., Roland, P. E., and Cowey, A. 1994. Visual form discrimination from color or motion cues: Functional anatomy by positron emission tomography. Proc. Natl. Acad. Sci. USA 91: 9965–9969. Haxby, J. V., Grady, C. L., Horwitz, B., Ungerleider, L. G., Mishkin, M., Carson, R. E., Herscovitch, P., Schapiro, M. B., and Rapoport, S. I. 1991. Dissociation of object and spatial visual processing pathways in human extrastriate cortex. Proc. Natl. Acad. Sci. USA 88: 1621–1625. Iida, H., Miura, S., Kanno, I., Murakami, M., Yamamoto, S., and Amano, M. 1989. Design and evaluation of Headtome-IV, a wholebody positron emission tomograph. IEEE Trans. Nucl. Sci. NS-37: 1006 –1010. Innocenti, G. M., and Clarke, S. 1984. Bilateral transitory projection to visual areas from auditory cortex in kittens. Dev. Brain Res. 14: 143–148. Kennedy, J. 1982. Haptic pictures. In Tactual Perception (W. Schiff and E. Foulke, Eds.), Academic Press, New York. Kerr, N. H. 1983. The role of vision in “visual imagery” experiments: Evidence from the congenitally blind. J. Exp. Psychol. Gen. 112: 265–277. Kiyosawa, M., Inoue, C., Kawasaki, T., Tokoro, T., Ishii, K., Ohyama, M., Senda, M., and Soma, Y. 1996. Functional neuroanatomy of visual object naming: A PET study. Graefe’s Arch. Clin. Exp. Ophthalmol. 234: 110 –115. Knudsen, G. M., Paulson, O. B., and Hertz, M. M. 1991. Kinetic analysis of the human blood-brain barrier transport of lactate and its influence by hypercapnia. J. Cereb. Blood Flow Metab. 11: 581–586. Kosslyn, S. M., Alpert, N. M., Thompson, W. L., Maljovic, V., Weise, S. B., Chabris, C. F., Hamilton, S. E., Rauch, S. L., and Buonanno, F. S. 1993. Visual mental imagery activates topographically organized visual cortex: PET investigations. J. Cogn. Neurosci. 5: 263– 287. Kosslyn, S. M. 1994. Image and Brain: The Resolution of the Imagery Debate. MIT Press, Cambridge. Kosslyn, S. M., and Sussman, A. L. 1995. Roles of imagery in perception: Or, there is no such thing as immaculate perception. In The Cognitive Neurosciences (M. S. Gazzaniga, Ed.), pp. 1035– 1042. MIT Press, Cambridge. Kosslyn, S. M., Thompson, W. L., and Alpert, N. M. 1997. Neural systems shared by visual imagery and visual perception: A positron emission tomography study. NeuroImage 6: 320 –334.

PET STUDY OF MENTAL IMAGERY IN EARLY BLINDNESS Malach, R., Reppas, J. B., Benson, R. R., Kwong, K. K., Jiang, H., Kennedy, W. A., Ledden, P. J., Brady, T. J., Rosen, B. R., and Tootell, R. B. H. 1995. Object-related activity revealed by functional magnetic resonance imaging in human occipital cortex. Proc. Natl. Acad. Sci. USA 92: 8135– 8139. Marmor, G. S., and Zaback, L. A. 1976. Mental rotation by the blind: Does mental rotation depend on visual imagery? J. Exp. Psychol. Hum. Percept. 2: 515–521. Mellet, E., Petit, L., Mazoyer, B., Denis, M., and Tzourio, N. 1998. Review Reopening the mental imagery debate: Lessons from functional anatomy. NeuroImage 8: 129 –139. Moscovitch, M., Kapur, S., Kohler, S., and Houle, S. 1995. Distinct neural correlates of visual long-term memory for spatial location and object identity: A positron emission tomography study in humans. Proc. Natl. Acad. Sci. USA 92: 3712–3725. Oldfield, R. C. 1971. The assessment and analysis of handedness: The Edinburgh inventory. Neuropsychologia 9: 97–113. Paivio, A. 1995. Imagery and Memory. In The Cognitive Neurosciences (M. S. Gazzaniga, Ed.), pp. 977–986. MIT Press, Cambridge. Phelps, M. E., Mazziotta, J. C., Kuhl, D. E., Nuwer, M., Packwood, J., Metter, J., and Engel, J. 1981. Tomographic mapping of human cerebral metabolism: Visual stimulation and deprivation. Neurology 31: 517–529. Price, C. J., and Friston, K. J. 1997. Cognitive conjunction: A new approach to brain activation experiments. NeuroImage 5: 261–270. Raichle, M. E., Fiez, J. A., Videen, T. O., MacLeod, A. M. K., Pardo, J. V., Fox, P. T., and Petersen, S. E. 1994. Practice-related changes in human brain functional anatomy during nonmotor learning. Cerebral Cortex 4: 8 –26. Rauschecker, J. P., and Korte, M. 1993. Auditory compensation for early blindness in cat cerebral cortex. J. Neurosci. 13: 4538 – 4548. Rauschecker, J. P. 1995. Compensatory plasticity and sensory substitution in the cerebral cortex. Trends Neurosci. 18: 36 – 43. Roland, P. E., and Gulyas, B. 1995. Visual memory, visual imagery, and visual recognition of large field patterns by the human brain: Functional anatomy with positron emission tomography. Cerebral Cortex 1: 79 –93. Sadato, N., Pascual-Leone, A., Grafman, J., Ibanez, V., Deiber, M. P., Dold, G., and Hallett, M. 1996. Activation of the primary visual cortex by Braille reading in blind subjects. Nature 380: 526 –528.

139

Sadato, N., Pascual-Leone, A., Grafman, J., Deiber, M. P., Ibanez, V., and Hallett, M. 1998. Neural networks for Braille reading by the blind. Brain 121: 1213–1229. Shimosegawa, E., Kanno, I., Hatazawa, J., Fujita, H., Iida, H., Miura, S., Murakami, M., Inugami, A., Ogaa, T., and Uemara, K. 1995. Photic stimulation study of changing the arterial partial pressure level of carbon dioxide. J. Cereb. Blood Flow Metab. 15: 111–114. Talairach, J., and Tournoux, P. 1988. Co-Planar Stereotaxic Atlas of the Human Brain. Thieme Medical, New York. Uhl, F., Kretschmer, T., Lindinger, G., Goldenberg, G., Lang, W., Oder, W., and Deecke, L. 1994. Tactile mental imagery in sighted persons and in patients suffering from peripheral blindness early in life. Electroencephalogr. Clin. Neurophysiol. 91: 249 –255. Vecchi, T., Monticellai, M. L., and Cornoldi, C. 1995. Visuo-spatial working memory: Structures and variables affecting a capacity measure. Neuropsychologia 11: 1549 –1564. Vecchi, T. 1998. Visuo-spatial imagery in congenitally totally blind people. Memory 6: 91–102. Veraart, C., De Volder, A. G., Wanet-Defalque, M. C., Bol, A., Michel, C., and Goffinet, A. M. 1990. Glucose utilization in human visual cortex is respectively elevated and decreased in early versus late blindness. Brain Res. 510: 115–121. Wagner, A. D., Desmond, J. E., Demb, J. B., Glover, G. H., and Gabrieli, J. D. E. 1997. Semantic repetition priming for verbal and pictorial knowledge: A functional MRI study of left inferior prefrontal cortex. J. Cogn. Neurosci. 9: 714 –726. Wanet-Defalque, M. C., Veraart, C., De Volder, A., Metz, R., Michel, C., Dooms, G., and Goffinet, A. 1988. High metabolic activity in the visual cortex of early blind human subjects. Brain Res. 446: 369 – 373. Weeks, R., Horwitz, B., Aziz-Sultan, A., Tian, B., Wessinger, C. M., Cohen, L. G., Hallett, M., and Rauschecker, J. P. 2000. A positron emission tomographic study of auditory localization in the congenitally blind. J. Neurosci. 20: 2664 –2672. Woods, R. P., Grafton, S. T., Holmes, C. J., Cherry, S. R., and Mazziotta, J. C. 1998a. Automated Image Registration: I. General methods and intrasubject, intramodality validation. J. Comput. Assist. Tomogr. 22: 139 –152. Woods, R. P., Grafton, S. T., Watson, J. D. G., Sicotte, N. L., and Mazziotta, J. C. 1998b. Automated Image Registration: II. Intersubject validation of linear and nonlinear models. J. Comput. Assist. Tomogr. 22: 153–165.