- Email: [email protected]
Multisensory convergence in calcarine visual areas in macaque monkey
International Journal of Psychophysiology 50 (2003) 19–26
Multisensory convergence in calcarine visual areas in macaque monkey Kathleen S. Rockland*, Hisayuki Ojima Laboratory for Cortical Organization and Systematics, RIKEN Brain Science Institute, 2-1 Hirosawa, Wako-shi, Saitama 351-0198, Japan Received 30 September 2002; accepted 24 February 2003
Abstract The neural substrates of multisensory perception and integration are still obscure, especially at the cortical level. Alternative viewpoints emphasize (1) ‘bottom–up’ processes, where different modalities converge in higher order multisensory areas, or (2) ‘top–down’ projections from multimodal to unimodal areas. In this anatomic study, we use anterograde tracer injections in parietal (8 monkeys) and auditory (3 monkeys) association areas, and demonstrate direct projections to areas V1 and V2 in the calcarine fissure (i.e. the peripheral visual field representation). The laminar signature, with terminations in layers 1 andyor 6, could be consistent with feedback-type connections. A subset of connections from parietal areas, however, branch to both V1 and a ventral extrastriate area (TEO or TEp). Thus, the direct connections to early visual areas V1 and V2 may well operate in conjunction with polysynaptic pathways in a densely parallel network. 䊚 2003 Elsevier Science B.V. All rights reserved. Keywords: Auditory association; Binding; Collateral branching; Feedback; Parietal cortex
1. Introduction The importance of multisensory perception and integration is well-established. The particular neural substrates, however, have remained unclear, especially at the cortical level. According to one hypothesis, multisensory integration may be predominantly a ‘bottom–up’ process, where different modalities converge in higher order multimodal areas in the cerebral cortex. An alternative view *Corresponding author. Tel.: q81-48-467-6427; fax: q8148-467-6420. E-mail address: [email protected] (K.S. Rockland).
emphasizes instead the importance of feedback (‘top–down’) projections from multimodal to unimodal areas (Stein and Meredith, 1993; Driver and Spence, 1998, 2000; Pascual-Leone and Hamilton, 2001; Meredith, 2002; Molholm et al., 2002). In this article, we present evidence for direct anatomic connections to areas V1 and V2 in the macaque monkey from both auditory and parietal association cortices. These may be considered a type of top–down connection (but see Section 4), and may contribute to the efficacy of bimodal interactions, especially at short latencies (Fort et al., 2002; Molholm et al., 2002). Some of these
0167-8760/03/$ - see front matter 䊚 2003 Elsevier Science B.V. All rights reserved. doi:10.1016/S0167-8760(03)00121-1
20
K.S. Rockland, H. Ojima / International Journal of Psychophysiology 50 (2003) 19–26
results have been presented in abstract form (Rockland and Ojima, 2001, 2002). 2. Methods Results are based on extracellular injections of anterograde tracers in parietal (ns8) or auditory (ns3) association cortex of macaque monkeys (Macaca mulatta or Macaca fuscata). Animals were tranquilized with ketamine (11 mgykg) and maintained under deep anesthesia with Nembutal (25 mgykg sodium pentobarbital, delivered i.v.). Two monkeys, in which physiological mapping of auditory areas was carried out, were intubated and maintained on isoflurane anesthesia. All surgery was performed on deeply anesthetized animals under sterile conditions, according to guidelines specified in experimental protocols approved by the Experimental Animal Committee of the RIKEN Institute (or, for 4 animals, the University of Iowa). After craniotomy and durotomy, parietal areas were localized in relation to the intraparietal sulcus, and auditory areas in relation to the lateral and superior temporal sulci. In parietal areas, large injections of biotinylated dextran amine (BDA; Molecular Probes, Eugene, OR) were pressure injected through a 5-ml Hamilton syringe. Two to four injections, each 0.75 ml, were made of 10% BDA (in 0.0125 M phosphate buffered saline (PBS), pH 7.2; equal parts of 10 000 and 3000 MW). In auditory areas, smaller injections were made by iontophoresis (5–7 mA positive current, in a 7-s onyoff cycle). Paired injections of BDA and fluoro-ruby (FR, Molecular Probes) were made at two separate sites. For FR, a 10% solution (10 000 MW) was prepared in 0.1 M PBS. Physiological mapping of auditory areas was carried out with metal electrodes in two monkeys (case 248 and one not illustrated). The posterior portion of the superior temporal gyms (the lateral belt; Rauschecker et al., 1995) was mapped along the lateral fissure using band-pass noises with varied center frequencies. Animals were fixed via a head holder to make both ears free, in a soundproof room. Calibrated headphones (Beyerdynamics, Germany) were inserted into the external meati. Band-pass noises were generated by MalLab
system (Kaiser Instrument, California) controlled by a Macintosh computer (Ojima and Murakami, 2002), and applied binaurally through the headphones. Bursts of band-pass noises had a riseyfall time of 10 ms with a duration of 70 ms every 1.2 s. Extracellular recording was made from neuron clusters for the best responses by changing the intensity and the center frequency of the noises. Tonotopic maps were derived with respect to the center frequencies. Animals were given postsurgical doses of antibiotics and analgesics, recovered, and survived 18–29 days to allow for transport of the tracer substances. They were then re-anesthetized with Nembutal (75 mgykg) and perfused transcardially, in sequence, with 0.5 l of 0.9% saline mixed with 0.5% sodium nitrite, 2–4 l of 4% paraformaldehyde, and chilled phosphate buffer, with 10, 20 and 30% sucrose (0.5 l each). Brains were cut serially in the coronal plane by frozen microtomy (at 50 mm), and processed histologically. For BDA, tissue was reacted in avidin–biotin complex (ABC; Vector Labs, Burlingame, CA) followed by histochemistry for DAB. For double injections of BDA and FR, sections were first reacted with anti-FR (1:6000 Molecular Probes). Next followed an ABC–DAB reaction to visualize BDA (brown in color). Then, the FR reaction was completed by using a biotinylated secondary antibody and ABC– DAB reaction (made black by adding nickel ammonium sulfate). 3. Results 3.1. Parietal injection sites (Fig. 1) Large injections were placed in seven monkeys so as to distribute over the inferior parietal lobule. How best to subdivide this territory is still under investigation (Lewis and van Essen, 2000), and, as a neutral alternative, we will continue to use the traditional nomenclature, which distinguished broadly between a posterior field, PG, and an anterior field, PF (Bonin and Bailey, 1947). These are closely equivalent to areas 7a and 7b, respectively, in the nomenclature proposed by Vogt and Vogt (Pandya and Seltzer, 1982; Cavada and Goldman-Rakic, 1989). Areas PG and PF have
K.S. Rockland, H. Ojima / International Journal of Psychophysiology 50 (2003) 19–26
21
distinct connectivity patterns with specific areas in the superior temporal sulcus (STS) and prefrontal cortex, among other regions; and are generally considered as higher order parietal association areas (Cavada and Goldman-Rakic, 1989; Seltzer and Pandya, 1994). Physiological recording experiments have demonstrated neurons with bimodal response properties in both PG and PF. Area PG (7a) in particular is thought to combine visual, somatosensory, auditory and vestibular signals to represent the spatial location of stimuli. This may be important for specifying the targets of actions such as eye movements or reaching (Andersen, 1997; Colby and Goldberg, 1999). All seven of these injections, in area PF as well as PG, resulted in projections to the calcarine fissure (CF). One (P2) of the two injections in PF, however, produced projections only to area V2 and not V1. An eighth injection (P8) was located in area PE (also called area 5) in the superior parietal lobule. Area PE receives direct connections from early somatosensory cortex. In accord with its more medial location, it is associated with the trunk and limbs (as opposed to the face representation in area PG), and with visuomotor functions (Andersen, 1997 and the references therein). This injection also resulted in projections to areas V1 and V2 in the CF.
Fig. 1.
Fig. 1. (A) Schematic diagram of the lateral and medial surfaces of the monkey cerebral cortex. (For medial surface, ventral is at the top, and only posterior half is shown). Large architectonic subdivisions are according to the Bonin and Bailey (1947) brain map. Auditory association fields are indicated by horizontal lines; and parietal fields by dots. (B) Three auditory cases are illustrated, with paired injections of BDA (open circle) and FR (filled circle). All except one injection (a very small FR injection in M 238) resulted in terminations in areas V2 andyor V1. In case 248 physiological mapping was carried out, and tonotopic shifts were used to estimate the border between medial and caudal lateral belt areas (mLB and cLB). In case 250, the more posterior occipital lobe (beyond the line) was not processed. (C) Another eight monkeys received injections of BDA in different parts of parietal cortex. For convenience, these are shown on a single brain diagram. See (A) above for broad architectonic subdivisions of the parietal lobe. AMTS, anterior middle temporal sulcus; AS, arcuate sulcus; aLB, anterior lateral belt; CF, calcarine fissure; CS, central sulcus; IOS, inferior occipital sulcus; IPS, intraparietal sulcus; LS, lateral sulcus; M, monkey; PS, principal sulcus; STS, superior temporal sulcus; TMPS, temporal middle posterior sulcus.
22
K.S. Rockland, H. Ojima / International Journal of Psychophysiology 50 (2003) 19–26
Fig. 2. Photographs of coronal histology sections from cases P3 (parietal) and 248 (auditory). The second number indicates section sequence, with larger numbers being more anterior. Projections to area V1, V2, or prostriata are indicated by symbols. The dark area in P3 (Sections 353, 377) is part of the injection site. L, lateral; M, medial.
3.2. Auditory injection sites (Fig. 1) Six injections were placed in posterior auditory association cortex. The organization of the belt and parabelt areas is complex and, like parietal association cortex, still under investigation (see, for example, Hackett et al., 1998; Kaas and Hackett, 2000; Read et al., 2002). Our injections were localized to the caudolateral (CL) and middle lateral (ML) belt in the nomenclature of Rauschecker et al. (1995) and Hackett et al. (1998). The more posterior injections would be in area Tpt of Pandya and Sanides (1973). In accord with previous studies (Rauschecker et al., 1995; Romanski et al., 1999; Tian et al., 2001), physiological mapping in two monkeys confirmed the existence of neurons responding to auditory stimuli and of area-specific tonotopic maps. Three tonotopic subareas were found. For the most anterior and posterior subareas there was a progression of high-to-low frequency in an anteroposterior axis. A subarea in between had an opposite frequency progression (i.e. low to high). These areas anteroposteriorly corresponded to the anterolateral, ML and CL belt subareas. Of three injections in the caudolateral belt (CL, or Tpt) near the lateral fissure, all three produced
terminations in dorsal V2, and one (case 250) also had sparse projections to V1. One injection in the middle lateral belt (ML; case 248) projected to both V1 and V2, but only sparsely to V1. The injection in CL (Tpt) nearer the STS (case 250) similarly projected to V2 and, sparsely, V1. In summary, all five of these injections projected to dorsal area V2, and three additionally had sparse projections to area V1. Projections were in the CF. 3.3. Calcarine projection sites (Fig. 2) From all these injections, the projections are denser to area V2 than V1. In V1, the projections are dispersed, making it difficult to assess density or topography from different injections. The total number of projecting neurons, within an injection -3.0 mm in diameter, is likely to be much less than 100 and probably less than 50. This estimate is based on semi-quantitative assessment of labeled axons in the white matter subjacent to the target region in the CF. In contrast with denser projections, these bundles in single sections consisted of a ‘countable’ small number of axons, typically 10–20. For both parietal and auditory cases, projections were most consistently to the anterior 3.0 mm of
K.S. Rockland, H. Ojima / International Journal of Psychophysiology 50 (2003) 19–26
the CF (Figs. 2 and 4), the region corresponding to the monocular crescent and far periphery of the visual field. Some projections extended beyond area V1, to the zone corresponding to Prostriata (Morecraft et al., 2000). Sporadic projections continued posteriorly, typically over a total extent of 6.0–10.0 mm in V1. The projections in V2 were more concentrated over a 2.0–3.0 mm extent. For both V1 and V2, projections were mainly in the upper bank of the CF, corresponding to the lower visual hemifield (Fig. 2). For the parietal cases, terminations in area V1 were to layer 6 alone (cases P4 and P5), to layer 1 alone (cases PI, P3 and P7), or to layers 1 and 6 (case P6 and P8). For case P6, a few terminations occurred also in layer 4B. Auditory projections to area V1 terminated in layers 1 and 6. Terminations in V2, from both the parietal and auditory injections, were densest in layer 1, but involved subjacent layers as well. Occasional retrogradely filled neurons occurred in layers 3 and 5 of V2, but none were observed in area V1. Ten axons were serially reconstructed from three parietal cases (P5, P6 and P8). Seven of these were highly divergent, measuring 1.5–3.5 mm in the anteroposterior plane and 4.0–12.0 mm in the dorsoventral plane (data not shown). Three axons, however, were more local, measuring 1.0–1.5 mm. These three were more in the depth of the CF than its upper bank, and all had collateral branches to the lateral lip of the occipitotemporal sulcus (OTS), to what is likely to be part of area TEO or posterior TE (TEp) (Fig. 3). 4. Discussion In the primate, areas V1 and V2 are generally believed to be visually dedicated areas and to receive cortical connections only from visually related areas (but in rodent, see Sanderson et al., 1991). The present report shows that areas V1 and V2 receive projections from several parietal and auditory association areas (Fig. 4). This seems to be a dramatic endorsement of the contribution of ‘top–down’ processes in multisensory integration, although, as discussed below, this is probably only one component of the connectional network. The occurrence of retrogradely filled neurons implies a
23
Fig. 3. Camera lucida reconstruction of an axon anterogradely labeled by a BDA injection in case P5 (injection in area PG or 7a). One collateral, as shown, terminates in layer 6 of area V1. The approximate location is marked on the upper coronal section outline by the three Xs. (Dorsal-most X corresponds to the portion at section 218, the middle X to the portion at 211, and the ventral-most to the portion at 206. For convenience, all three points are mapped onto the outline for coronal section 211.) The blackened region in section 211 corresponds to part of the injection site. A second collateral (lower thick arrow) continues to a projection focus in the OTS (arrow in outline for section 299), approximately 4.0-mm anterior. The upper thick arrow denotes the main axon, which was followed toward the injection site until section 163 (almost 3.0-mm posterior). Numbers correspond to individual sections, larger numbers being anterior. Larger dashes designate the border between layer 6 and the white matter at sections 206, 214 and 218. Smaller dashes designate continuing (but not illustrated) portions of the main axon, which was traced to section 163 (posterior) and 299 (anterior). WM, white matter; OTS, occipitotemporal sulcus. See Fig. 1 for other abbreviations.
24
K.S. Rockland, H. Ojima / International Journal of Psychophysiology 50 (2003) 19–26
Fig. 4. Summary schematic indicates projections from several association regions (superior parietal lobule, dashed oval; inferior parietal lobule, hollow oval; and superior temporal gyrus, black ovals) to areas V1 and V2 in the CF. Cortical subdivisions are from the Bonin and Bailey (1947) brain map (see also Fig. 1).
partial reciprocity of connections from V2 (but not from V1). That these projections have been previously overlooked can be explained in part by the relative inaccessibility of the CF. In addition, the projections, especially to area V1, are both sparse and sporadic and can be easily missed unless the sections are closely scanned. Because of their sparsity, they probably could not be distinguished from background by the older techniques of WGAHRP or w3Hxamino acid autoradiography. As it is, close corroboration of our results is provided by the work of Falchier et al. (2002). After injecting retrograde tracers in different parts of area V1, these investigators observed retrogradely filled neurons in layer 6 of several auditory areas, and in the superior temporal polysensory (STP) region of the STS. In some discrepancy with our results, Falchier et al. report a few connections even to regions of central visual representation, and further report that the projections from auditory regions to more peripheral visual
representations are of ‘moderate’ density. On the basis of our own results, we characterized projections to V1 as sparse, although those to area V2 were denser. Other studies provide functional evidence for nonretinal processes in area V1, such as would be consistent with our anatomic results. Shams et al. (2002) recently reported that a single visual flash accompanied by two auditory beeps is perceived as two flashes, and that this sound-induced illusory effect is stronger in the lower visual field (corresponding to the upper bank of the CF) and in the peripheral field representation. There are also reports of neurons in area V1, and especially in extrastriate areas, that show constancy for orientation regardless of body tilt. These studies, using physiological recording in behaving monkeys, suggest that the early stages of visual cortical processing can be modified by vestibular andyor proprioceptive signals (Sauvan and Peterhans, 1999). The anatomic substrates of these phenomena are not known, but could at least partially involve the corticocortical pathways that we have demonstrated. On the basis of anatomic data alone, the functional significance of nonretinal connections to areas V1 and V2 is hard to determine. One clue is the preferential targeting of the peripheral visual field representations. The suggestion is that these connections might be part of a distributed system involving vestibular andyor proprioceptive signals in the context of motion perception or alerting responses, functions particularly associated with peripheral vision. (For relevant findings concerning peripheral vision in human, see Neville and Lawson, 1987.) The integration of different sensory inputs is also thought to subserve the elaboration of ego-, object-, or environment-centered reference frames, as opposed to sensory-based coordinates (Bottini et al., 2001). Another important factor is that the direct connections demonstrated here, may well operate in conjunction with polysynaptic pathways and loops, being part of a densely parallel connectivity network. The direct connections might contribute especially to the post-stimulus short latency early responses reported by recent ERP studies in
K.S. Rockland, H. Ojima / International Journal of Psychophysiology 50 (2003) 19–26
humans (Foxe and Simpson, 2001; Fort et al., 2002; Molholm et al., 2002). To some extent, the direct connections, terminating in ‘lower’ visual areas, can be considered as feedback connections. Moreover, as these originate from neurons in layer 6 (Falchier et al., 2002) and terminate in layers 1 andyor 6 (present study), the projections have the laminar profile of classical feedback or top–down connections (but see cautionary discussion in Rockland, 1997, 2002). An interesting and unexpected twist, however, is that some axons (3 of the 10 axons analysed from parietal areas) have branches in both area V1 and in the OTS (area TEO or TEp). This area can be considered visual association cortex, and probably projects back to V1 and V2 (Rockland and Drash, 1996), possibly even in the CF. Among many questions for continued investigation are how these putative nonretinal inputs are combined at the level of single neurons or clusters (e.g. Meredith, 2002). A particularly key question is whether the parent neurons are unimodal, dominated by auditory or parietal features, or whether they are themselves already multisensory responsive. This could result from convergent cortical connections (Schroeder and Foxe, 2002) andyor from subcortical connections, from the magnocellular medial geniculate nucleus (to auditory areas), or from medial pulvinar (to parietal areas). Acknowledgments We thank Adrian Knight for assistance with figure preparation, Michiko Fujisawa for assistance with manuscript preparation, and other lab members for excellent help with histological preparation. This work was supported by research funds from RIKEN Brain Science Institute. References Andersen, R.A., 1997. Multimodal integrating for the representation of space in the posterior parietal cortex. Phil. Trans. R. Soc.—Lond. B 352, 1421–1428. Bonin, G.von., Bailey, P., 1947. The Neocortex of Macaca Mulatta. The University of Illinois Press, Urbana. Bottini, G., Karnath, H.-O., Vallar, G., et al., 2001. Cerebral representations for egocentric space functional–anatomical
25
evidence from caloric vestibular stimulation and neck vibration. Brain 124, 1182–1196. Cavada, C., Goldman-Rakic, P.S., 1989. Posterior parietal cortex in rhesus monkey: I. Parcellation of areas based on distinctive limbic and sensory corticocortical connections. J. Comp. Neurol. 287, 393–421. Colby, C.L., Goldberg, M.E., 1999. Space and attention in parietal cortex. Annu. Rev. Neurosci. 22, 319–349. Driver, J., Spence, C., 1998. Attention and the crossmodal construction of space. Trends Cogn. Sci. 2, 254–262. Driver, J., Spence, C., 2000. Multisensory perception: beyond modularity and convergence. Curr. Biol. 10, R731–R735. Falchier, A., Clavagnier, S., Barone, P., Kennedy, H., 2002. Anatomical evidence of multimodal integration in primate striate cortex. J. Neurosci. 22, 5749–5759. Fort, A., Defpuech, C., Pernier, J., Giard, M.H., 2002. Dynamics of cortico-subcortical cross-modal operations involved in audio-visual object detection in humans. Cereb. Cortex 12, 1031–1039. Foxe, J.J., Simpson, G.V., 2001. Flow of activation from V1 to frontal cortex in humans. Exp. Brain Res. 142, 139–150. Hackett, T.A., Stepniewska, I., Kaas, J.H., 1998. Subdivisions of auditory cortex and ipsilateral cortical connections of the parabelt auditory cortex in macaque monkeys. J. Comp. Neurol. 394, 475–495. Kaas, J.H., Hackett, T.A., 2000. Subdivisions of auditory cortex and processing streams in primates. PNAS 97, 11793–11799. Lewis, J.W., van Essen, D.C., 2000. Mapping of architectonic subdivisions in the macaque monkey, with emphasis on parieto-occipital cortex. J. Comp. Neurol. 428, 79–111. Meredith, M.A., 2002. On the neuronal basis for multisensory convergence: a brief overview. Cogn. Brain Res. 14, 31–40. Molholm, S., Ritter, W., Murray, M.M., Javitt, D.C., Schroeder, C.E., Foxe, J.J., 2002. Multisensory auditory–visual interactions during early sensory processing in humans; a highdensity electrical mapping study. Cogn. Brain Res. 14, 115–128. Morecraft, R.J., Rockland, K.S., van Hoesen, G.W., 2000. Localization of area prostriata and its projection to the cingulate motor cortex in the rhesus monkey. Cereb. Cortex 10, 192–203. Neville, H.J., Lawson, D., 1987. Attention to central and peripheral visual space in a movement detection task: an even-related potential and behavioral study. II. Congenitally deaf adults. Brain Res. 405, 268–283. Ojima, H., Murakami, K., 2002. Intracellular characterization of suppressive responses in supragranular pyramidal neurons of cat primary auditory cortex in vivo. Cereb. Cortex 12, 1079–1091. Pandya, D.N., Sanides, P., 1973. Architectonic parcellation of the temporal operculum in rhesus monkey and its projection pattern. Z. Anat. Entwickle.-Gesch. 139, 127–161. Pandya, D.N., Seltzer, B., 1982. Intrinsic connections and architectonics of posterior parietal cortex in the rhesus monkey. J. Comp. Neurol. 204, 196–210.
26
K.S. Rockland, H. Ojima / International Journal of Psychophysiology 50 (2003) 19–26
Pascual-Leone, A., Hamilton, R., 2001. The metamodal organization of the brain. In: Casanova, C., Ptito, M. (Eds.), Progress in Brain Research, vol. 134. Elsevier Science B, Amsterdam, pp. 427–445. Rauschecker, J.P., Tian, B., Hauser, M., 1995. Processing of complex sounds in the macaque nonprimary auditory cortex. Science 268, 111–114. Read, H.L., Winer, J.A., Schreiner, C.E., 2002. Functional architecture of auditory cortex. Curr. Opin. Neurobiol. 12, 433–440. Rockland, K.S., 1997. Elements of cortical architecture. In: Peters, A., Rockland, K.S. (Eds.), Cerebral Cortex, vol. 10. Plenum Press, New York, pp. 243–294. Rockland, K.S., 2002. Visual cortical organization at the single axon level: a beginning. Neurosci. Res. 42, 155–166. Rockland, K.S., Drash, G.W., 1996. Collateralized divergent feedback connections that target multiple cortical areas. J. Comp. Neurol. 373, 529–548. Rockland, K.S., Ojima, H., 2001. Calcarine area V1 as a multimodal convergence area. Soc. Neurosci. 27, 1342 abstract. Rockland, K.S., Ojima, H., 2002. Multimodal convergence in calcarine visual areas in macaque monkey. International Multisensory Research Forum 3rd Ann. Mtg. Abstract.
Romanski, L.M., Tian, B., Fritz, J., Mishkin, M., GoldmanRakic, P.S., Rauschecker, J.P., 1999. Dual streams of auditory afferents target multiple domains in the primate prefrontal cortex. Nat. Neurosci. 2, 1131–1136. Sanderson, K.J., Dreher, B., Gayer, N., 1991. Prosencephalic connections of striate and extrastriate areas of rat visual cortex. Exp. Brain Res. 85, 324–334. Sauvan, X.M., Peterhans, E., 1999. Orientation constancy in neurons of monkey visual cortex. Vis. Cogn. 6, 43–54. Schroeder, C.E., Foxe, J., 2002. The timing and laminar profile of converging inputs to multisensory areas of the macaque neocortex. Cogn. Brain Res. 14, 187–198. Seltzer, B., Pandya, D.N., 1994. Parietal, temporal, and occipital projections to cortex of the superior temporal sulcus in the rhesus monkey: a retrograde tracer study. J. Comp. Neurol. 343, 445–463. Shams, L., Kamitani, Y., Shimojo, S., 2002. Visual illusion induced by second. Brain Res. Cogn. Brain Res. 14, 147–152. Stein, B.E., Meredith, M.A., 1993. The Merging of the Senses. MIT Press, Cambridge. Tian, B., Reser, D., Durham, A., Kustov, A., Rauschecker, J.P., 2001. Functional specialization in rhesus monkey auditory cortex. Science 292, 290–293.
Multisensory convergence in calcarine visual areas in macaque monkey Kathleen S. Rockland*, Hisayuki Ojima Laboratory for Cortical Organization and Systematics, RIKEN Brain Science Institute, 2-1 Hirosawa, Wako-shi, Saitama 351-0198, Japan Received 30 September 2002; accepted 24 February 2003
Abstract The neural substrates of multisensory perception and integration are still obscure, especially at the cortical level. Alternative viewpoints emphasize (1) ‘bottom–up’ processes, where different modalities converge in higher order multisensory areas, or (2) ‘top–down’ projections from multimodal to unimodal areas. In this anatomic study, we use anterograde tracer injections in parietal (8 monkeys) and auditory (3 monkeys) association areas, and demonstrate direct projections to areas V1 and V2 in the calcarine fissure (i.e. the peripheral visual field representation). The laminar signature, with terminations in layers 1 andyor 6, could be consistent with feedback-type connections. A subset of connections from parietal areas, however, branch to both V1 and a ventral extrastriate area (TEO or TEp). Thus, the direct connections to early visual areas V1 and V2 may well operate in conjunction with polysynaptic pathways in a densely parallel network. 䊚 2003 Elsevier Science B.V. All rights reserved. Keywords: Auditory association; Binding; Collateral branching; Feedback; Parietal cortex
1. Introduction The importance of multisensory perception and integration is well-established. The particular neural substrates, however, have remained unclear, especially at the cortical level. According to one hypothesis, multisensory integration may be predominantly a ‘bottom–up’ process, where different modalities converge in higher order multimodal areas in the cerebral cortex. An alternative view *Corresponding author. Tel.: q81-48-467-6427; fax: q8148-467-6420. E-mail address: [email protected] (K.S. Rockland).
emphasizes instead the importance of feedback (‘top–down’) projections from multimodal to unimodal areas (Stein and Meredith, 1993; Driver and Spence, 1998, 2000; Pascual-Leone and Hamilton, 2001; Meredith, 2002; Molholm et al., 2002). In this article, we present evidence for direct anatomic connections to areas V1 and V2 in the macaque monkey from both auditory and parietal association cortices. These may be considered a type of top–down connection (but see Section 4), and may contribute to the efficacy of bimodal interactions, especially at short latencies (Fort et al., 2002; Molholm et al., 2002). Some of these
0167-8760/03/$ - see front matter 䊚 2003 Elsevier Science B.V. All rights reserved. doi:10.1016/S0167-8760(03)00121-1
20
K.S. Rockland, H. Ojima / International Journal of Psychophysiology 50 (2003) 19–26
results have been presented in abstract form (Rockland and Ojima, 2001, 2002). 2. Methods Results are based on extracellular injections of anterograde tracers in parietal (ns8) or auditory (ns3) association cortex of macaque monkeys (Macaca mulatta or Macaca fuscata). Animals were tranquilized with ketamine (11 mgykg) and maintained under deep anesthesia with Nembutal (25 mgykg sodium pentobarbital, delivered i.v.). Two monkeys, in which physiological mapping of auditory areas was carried out, were intubated and maintained on isoflurane anesthesia. All surgery was performed on deeply anesthetized animals under sterile conditions, according to guidelines specified in experimental protocols approved by the Experimental Animal Committee of the RIKEN Institute (or, for 4 animals, the University of Iowa). After craniotomy and durotomy, parietal areas were localized in relation to the intraparietal sulcus, and auditory areas in relation to the lateral and superior temporal sulci. In parietal areas, large injections of biotinylated dextran amine (BDA; Molecular Probes, Eugene, OR) were pressure injected through a 5-ml Hamilton syringe. Two to four injections, each 0.75 ml, were made of 10% BDA (in 0.0125 M phosphate buffered saline (PBS), pH 7.2; equal parts of 10 000 and 3000 MW). In auditory areas, smaller injections were made by iontophoresis (5–7 mA positive current, in a 7-s onyoff cycle). Paired injections of BDA and fluoro-ruby (FR, Molecular Probes) were made at two separate sites. For FR, a 10% solution (10 000 MW) was prepared in 0.1 M PBS. Physiological mapping of auditory areas was carried out with metal electrodes in two monkeys (case 248 and one not illustrated). The posterior portion of the superior temporal gyms (the lateral belt; Rauschecker et al., 1995) was mapped along the lateral fissure using band-pass noises with varied center frequencies. Animals were fixed via a head holder to make both ears free, in a soundproof room. Calibrated headphones (Beyerdynamics, Germany) were inserted into the external meati. Band-pass noises were generated by MalLab
system (Kaiser Instrument, California) controlled by a Macintosh computer (Ojima and Murakami, 2002), and applied binaurally through the headphones. Bursts of band-pass noises had a riseyfall time of 10 ms with a duration of 70 ms every 1.2 s. Extracellular recording was made from neuron clusters for the best responses by changing the intensity and the center frequency of the noises. Tonotopic maps were derived with respect to the center frequencies. Animals were given postsurgical doses of antibiotics and analgesics, recovered, and survived 18–29 days to allow for transport of the tracer substances. They were then re-anesthetized with Nembutal (75 mgykg) and perfused transcardially, in sequence, with 0.5 l of 0.9% saline mixed with 0.5% sodium nitrite, 2–4 l of 4% paraformaldehyde, and chilled phosphate buffer, with 10, 20 and 30% sucrose (0.5 l each). Brains were cut serially in the coronal plane by frozen microtomy (at 50 mm), and processed histologically. For BDA, tissue was reacted in avidin–biotin complex (ABC; Vector Labs, Burlingame, CA) followed by histochemistry for DAB. For double injections of BDA and FR, sections were first reacted with anti-FR (1:6000 Molecular Probes). Next followed an ABC–DAB reaction to visualize BDA (brown in color). Then, the FR reaction was completed by using a biotinylated secondary antibody and ABC– DAB reaction (made black by adding nickel ammonium sulfate). 3. Results 3.1. Parietal injection sites (Fig. 1) Large injections were placed in seven monkeys so as to distribute over the inferior parietal lobule. How best to subdivide this territory is still under investigation (Lewis and van Essen, 2000), and, as a neutral alternative, we will continue to use the traditional nomenclature, which distinguished broadly between a posterior field, PG, and an anterior field, PF (Bonin and Bailey, 1947). These are closely equivalent to areas 7a and 7b, respectively, in the nomenclature proposed by Vogt and Vogt (Pandya and Seltzer, 1982; Cavada and Goldman-Rakic, 1989). Areas PG and PF have
K.S. Rockland, H. Ojima / International Journal of Psychophysiology 50 (2003) 19–26
21
distinct connectivity patterns with specific areas in the superior temporal sulcus (STS) and prefrontal cortex, among other regions; and are generally considered as higher order parietal association areas (Cavada and Goldman-Rakic, 1989; Seltzer and Pandya, 1994). Physiological recording experiments have demonstrated neurons with bimodal response properties in both PG and PF. Area PG (7a) in particular is thought to combine visual, somatosensory, auditory and vestibular signals to represent the spatial location of stimuli. This may be important for specifying the targets of actions such as eye movements or reaching (Andersen, 1997; Colby and Goldberg, 1999). All seven of these injections, in area PF as well as PG, resulted in projections to the calcarine fissure (CF). One (P2) of the two injections in PF, however, produced projections only to area V2 and not V1. An eighth injection (P8) was located in area PE (also called area 5) in the superior parietal lobule. Area PE receives direct connections from early somatosensory cortex. In accord with its more medial location, it is associated with the trunk and limbs (as opposed to the face representation in area PG), and with visuomotor functions (Andersen, 1997 and the references therein). This injection also resulted in projections to areas V1 and V2 in the CF.
Fig. 1.
Fig. 1. (A) Schematic diagram of the lateral and medial surfaces of the monkey cerebral cortex. (For medial surface, ventral is at the top, and only posterior half is shown). Large architectonic subdivisions are according to the Bonin and Bailey (1947) brain map. Auditory association fields are indicated by horizontal lines; and parietal fields by dots. (B) Three auditory cases are illustrated, with paired injections of BDA (open circle) and FR (filled circle). All except one injection (a very small FR injection in M 238) resulted in terminations in areas V2 andyor V1. In case 248 physiological mapping was carried out, and tonotopic shifts were used to estimate the border between medial and caudal lateral belt areas (mLB and cLB). In case 250, the more posterior occipital lobe (beyond the line) was not processed. (C) Another eight monkeys received injections of BDA in different parts of parietal cortex. For convenience, these are shown on a single brain diagram. See (A) above for broad architectonic subdivisions of the parietal lobe. AMTS, anterior middle temporal sulcus; AS, arcuate sulcus; aLB, anterior lateral belt; CF, calcarine fissure; CS, central sulcus; IOS, inferior occipital sulcus; IPS, intraparietal sulcus; LS, lateral sulcus; M, monkey; PS, principal sulcus; STS, superior temporal sulcus; TMPS, temporal middle posterior sulcus.
22
K.S. Rockland, H. Ojima / International Journal of Psychophysiology 50 (2003) 19–26
Fig. 2. Photographs of coronal histology sections from cases P3 (parietal) and 248 (auditory). The second number indicates section sequence, with larger numbers being more anterior. Projections to area V1, V2, or prostriata are indicated by symbols. The dark area in P3 (Sections 353, 377) is part of the injection site. L, lateral; M, medial.
3.2. Auditory injection sites (Fig. 1) Six injections were placed in posterior auditory association cortex. The organization of the belt and parabelt areas is complex and, like parietal association cortex, still under investigation (see, for example, Hackett et al., 1998; Kaas and Hackett, 2000; Read et al., 2002). Our injections were localized to the caudolateral (CL) and middle lateral (ML) belt in the nomenclature of Rauschecker et al. (1995) and Hackett et al. (1998). The more posterior injections would be in area Tpt of Pandya and Sanides (1973). In accord with previous studies (Rauschecker et al., 1995; Romanski et al., 1999; Tian et al., 2001), physiological mapping in two monkeys confirmed the existence of neurons responding to auditory stimuli and of area-specific tonotopic maps. Three tonotopic subareas were found. For the most anterior and posterior subareas there was a progression of high-to-low frequency in an anteroposterior axis. A subarea in between had an opposite frequency progression (i.e. low to high). These areas anteroposteriorly corresponded to the anterolateral, ML and CL belt subareas. Of three injections in the caudolateral belt (CL, or Tpt) near the lateral fissure, all three produced
terminations in dorsal V2, and one (case 250) also had sparse projections to V1. One injection in the middle lateral belt (ML; case 248) projected to both V1 and V2, but only sparsely to V1. The injection in CL (Tpt) nearer the STS (case 250) similarly projected to V2 and, sparsely, V1. In summary, all five of these injections projected to dorsal area V2, and three additionally had sparse projections to area V1. Projections were in the CF. 3.3. Calcarine projection sites (Fig. 2) From all these injections, the projections are denser to area V2 than V1. In V1, the projections are dispersed, making it difficult to assess density or topography from different injections. The total number of projecting neurons, within an injection -3.0 mm in diameter, is likely to be much less than 100 and probably less than 50. This estimate is based on semi-quantitative assessment of labeled axons in the white matter subjacent to the target region in the CF. In contrast with denser projections, these bundles in single sections consisted of a ‘countable’ small number of axons, typically 10–20. For both parietal and auditory cases, projections were most consistently to the anterior 3.0 mm of
K.S. Rockland, H. Ojima / International Journal of Psychophysiology 50 (2003) 19–26
the CF (Figs. 2 and 4), the region corresponding to the monocular crescent and far periphery of the visual field. Some projections extended beyond area V1, to the zone corresponding to Prostriata (Morecraft et al., 2000). Sporadic projections continued posteriorly, typically over a total extent of 6.0–10.0 mm in V1. The projections in V2 were more concentrated over a 2.0–3.0 mm extent. For both V1 and V2, projections were mainly in the upper bank of the CF, corresponding to the lower visual hemifield (Fig. 2). For the parietal cases, terminations in area V1 were to layer 6 alone (cases P4 and P5), to layer 1 alone (cases PI, P3 and P7), or to layers 1 and 6 (case P6 and P8). For case P6, a few terminations occurred also in layer 4B. Auditory projections to area V1 terminated in layers 1 and 6. Terminations in V2, from both the parietal and auditory injections, were densest in layer 1, but involved subjacent layers as well. Occasional retrogradely filled neurons occurred in layers 3 and 5 of V2, but none were observed in area V1. Ten axons were serially reconstructed from three parietal cases (P5, P6 and P8). Seven of these were highly divergent, measuring 1.5–3.5 mm in the anteroposterior plane and 4.0–12.0 mm in the dorsoventral plane (data not shown). Three axons, however, were more local, measuring 1.0–1.5 mm. These three were more in the depth of the CF than its upper bank, and all had collateral branches to the lateral lip of the occipitotemporal sulcus (OTS), to what is likely to be part of area TEO or posterior TE (TEp) (Fig. 3). 4. Discussion In the primate, areas V1 and V2 are generally believed to be visually dedicated areas and to receive cortical connections only from visually related areas (but in rodent, see Sanderson et al., 1991). The present report shows that areas V1 and V2 receive projections from several parietal and auditory association areas (Fig. 4). This seems to be a dramatic endorsement of the contribution of ‘top–down’ processes in multisensory integration, although, as discussed below, this is probably only one component of the connectional network. The occurrence of retrogradely filled neurons implies a
23
Fig. 3. Camera lucida reconstruction of an axon anterogradely labeled by a BDA injection in case P5 (injection in area PG or 7a). One collateral, as shown, terminates in layer 6 of area V1. The approximate location is marked on the upper coronal section outline by the three Xs. (Dorsal-most X corresponds to the portion at section 218, the middle X to the portion at 211, and the ventral-most to the portion at 206. For convenience, all three points are mapped onto the outline for coronal section 211.) The blackened region in section 211 corresponds to part of the injection site. A second collateral (lower thick arrow) continues to a projection focus in the OTS (arrow in outline for section 299), approximately 4.0-mm anterior. The upper thick arrow denotes the main axon, which was followed toward the injection site until section 163 (almost 3.0-mm posterior). Numbers correspond to individual sections, larger numbers being anterior. Larger dashes designate the border between layer 6 and the white matter at sections 206, 214 and 218. Smaller dashes designate continuing (but not illustrated) portions of the main axon, which was traced to section 163 (posterior) and 299 (anterior). WM, white matter; OTS, occipitotemporal sulcus. See Fig. 1 for other abbreviations.
24
K.S. Rockland, H. Ojima / International Journal of Psychophysiology 50 (2003) 19–26
Fig. 4. Summary schematic indicates projections from several association regions (superior parietal lobule, dashed oval; inferior parietal lobule, hollow oval; and superior temporal gyrus, black ovals) to areas V1 and V2 in the CF. Cortical subdivisions are from the Bonin and Bailey (1947) brain map (see also Fig. 1).
partial reciprocity of connections from V2 (but not from V1). That these projections have been previously overlooked can be explained in part by the relative inaccessibility of the CF. In addition, the projections, especially to area V1, are both sparse and sporadic and can be easily missed unless the sections are closely scanned. Because of their sparsity, they probably could not be distinguished from background by the older techniques of WGAHRP or w3Hxamino acid autoradiography. As it is, close corroboration of our results is provided by the work of Falchier et al. (2002). After injecting retrograde tracers in different parts of area V1, these investigators observed retrogradely filled neurons in layer 6 of several auditory areas, and in the superior temporal polysensory (STP) region of the STS. In some discrepancy with our results, Falchier et al. report a few connections even to regions of central visual representation, and further report that the projections from auditory regions to more peripheral visual
representations are of ‘moderate’ density. On the basis of our own results, we characterized projections to V1 as sparse, although those to area V2 were denser. Other studies provide functional evidence for nonretinal processes in area V1, such as would be consistent with our anatomic results. Shams et al. (2002) recently reported that a single visual flash accompanied by two auditory beeps is perceived as two flashes, and that this sound-induced illusory effect is stronger in the lower visual field (corresponding to the upper bank of the CF) and in the peripheral field representation. There are also reports of neurons in area V1, and especially in extrastriate areas, that show constancy for orientation regardless of body tilt. These studies, using physiological recording in behaving monkeys, suggest that the early stages of visual cortical processing can be modified by vestibular andyor proprioceptive signals (Sauvan and Peterhans, 1999). The anatomic substrates of these phenomena are not known, but could at least partially involve the corticocortical pathways that we have demonstrated. On the basis of anatomic data alone, the functional significance of nonretinal connections to areas V1 and V2 is hard to determine. One clue is the preferential targeting of the peripheral visual field representations. The suggestion is that these connections might be part of a distributed system involving vestibular andyor proprioceptive signals in the context of motion perception or alerting responses, functions particularly associated with peripheral vision. (For relevant findings concerning peripheral vision in human, see Neville and Lawson, 1987.) The integration of different sensory inputs is also thought to subserve the elaboration of ego-, object-, or environment-centered reference frames, as opposed to sensory-based coordinates (Bottini et al., 2001). Another important factor is that the direct connections demonstrated here, may well operate in conjunction with polysynaptic pathways and loops, being part of a densely parallel connectivity network. The direct connections might contribute especially to the post-stimulus short latency early responses reported by recent ERP studies in
K.S. Rockland, H. Ojima / International Journal of Psychophysiology 50 (2003) 19–26
humans (Foxe and Simpson, 2001; Fort et al., 2002; Molholm et al., 2002). To some extent, the direct connections, terminating in ‘lower’ visual areas, can be considered as feedback connections. Moreover, as these originate from neurons in layer 6 (Falchier et al., 2002) and terminate in layers 1 andyor 6 (present study), the projections have the laminar profile of classical feedback or top–down connections (but see cautionary discussion in Rockland, 1997, 2002). An interesting and unexpected twist, however, is that some axons (3 of the 10 axons analysed from parietal areas) have branches in both area V1 and in the OTS (area TEO or TEp). This area can be considered visual association cortex, and probably projects back to V1 and V2 (Rockland and Drash, 1996), possibly even in the CF. Among many questions for continued investigation are how these putative nonretinal inputs are combined at the level of single neurons or clusters (e.g. Meredith, 2002). A particularly key question is whether the parent neurons are unimodal, dominated by auditory or parietal features, or whether they are themselves already multisensory responsive. This could result from convergent cortical connections (Schroeder and Foxe, 2002) andyor from subcortical connections, from the magnocellular medial geniculate nucleus (to auditory areas), or from medial pulvinar (to parietal areas). Acknowledgments We thank Adrian Knight for assistance with figure preparation, Michiko Fujisawa for assistance with manuscript preparation, and other lab members for excellent help with histological preparation. This work was supported by research funds from RIKEN Brain Science Institute. References Andersen, R.A., 1997. Multimodal integrating for the representation of space in the posterior parietal cortex. Phil. Trans. R. Soc.—Lond. B 352, 1421–1428. Bonin, G.von., Bailey, P., 1947. The Neocortex of Macaca Mulatta. The University of Illinois Press, Urbana. Bottini, G., Karnath, H.-O., Vallar, G., et al., 2001. Cerebral representations for egocentric space functional–anatomical
25
evidence from caloric vestibular stimulation and neck vibration. Brain 124, 1182–1196. Cavada, C., Goldman-Rakic, P.S., 1989. Posterior parietal cortex in rhesus monkey: I. Parcellation of areas based on distinctive limbic and sensory corticocortical connections. J. Comp. Neurol. 287, 393–421. Colby, C.L., Goldberg, M.E., 1999. Space and attention in parietal cortex. Annu. Rev. Neurosci. 22, 319–349. Driver, J., Spence, C., 1998. Attention and the crossmodal construction of space. Trends Cogn. Sci. 2, 254–262. Driver, J., Spence, C., 2000. Multisensory perception: beyond modularity and convergence. Curr. Biol. 10, R731–R735. Falchier, A., Clavagnier, S., Barone, P., Kennedy, H., 2002. Anatomical evidence of multimodal integration in primate striate cortex. J. Neurosci. 22, 5749–5759. Fort, A., Defpuech, C., Pernier, J., Giard, M.H., 2002. Dynamics of cortico-subcortical cross-modal operations involved in audio-visual object detection in humans. Cereb. Cortex 12, 1031–1039. Foxe, J.J., Simpson, G.V., 2001. Flow of activation from V1 to frontal cortex in humans. Exp. Brain Res. 142, 139–150. Hackett, T.A., Stepniewska, I., Kaas, J.H., 1998. Subdivisions of auditory cortex and ipsilateral cortical connections of the parabelt auditory cortex in macaque monkeys. J. Comp. Neurol. 394, 475–495. Kaas, J.H., Hackett, T.A., 2000. Subdivisions of auditory cortex and processing streams in primates. PNAS 97, 11793–11799. Lewis, J.W., van Essen, D.C., 2000. Mapping of architectonic subdivisions in the macaque monkey, with emphasis on parieto-occipital cortex. J. Comp. Neurol. 428, 79–111. Meredith, M.A., 2002. On the neuronal basis for multisensory convergence: a brief overview. Cogn. Brain Res. 14, 31–40. Molholm, S., Ritter, W., Murray, M.M., Javitt, D.C., Schroeder, C.E., Foxe, J.J., 2002. Multisensory auditory–visual interactions during early sensory processing in humans; a highdensity electrical mapping study. Cogn. Brain Res. 14, 115–128. Morecraft, R.J., Rockland, K.S., van Hoesen, G.W., 2000. Localization of area prostriata and its projection to the cingulate motor cortex in the rhesus monkey. Cereb. Cortex 10, 192–203. Neville, H.J., Lawson, D., 1987. Attention to central and peripheral visual space in a movement detection task: an even-related potential and behavioral study. II. Congenitally deaf adults. Brain Res. 405, 268–283. Ojima, H., Murakami, K., 2002. Intracellular characterization of suppressive responses in supragranular pyramidal neurons of cat primary auditory cortex in vivo. Cereb. Cortex 12, 1079–1091. Pandya, D.N., Sanides, P., 1973. Architectonic parcellation of the temporal operculum in rhesus monkey and its projection pattern. Z. Anat. Entwickle.-Gesch. 139, 127–161. Pandya, D.N., Seltzer, B., 1982. Intrinsic connections and architectonics of posterior parietal cortex in the rhesus monkey. J. Comp. Neurol. 204, 196–210.
26
K.S. Rockland, H. Ojima / International Journal of Psychophysiology 50 (2003) 19–26
Pascual-Leone, A., Hamilton, R., 2001. The metamodal organization of the brain. In: Casanova, C., Ptito, M. (Eds.), Progress in Brain Research, vol. 134. Elsevier Science B, Amsterdam, pp. 427–445. Rauschecker, J.P., Tian, B., Hauser, M., 1995. Processing of complex sounds in the macaque nonprimary auditory cortex. Science 268, 111–114. Read, H.L., Winer, J.A., Schreiner, C.E., 2002. Functional architecture of auditory cortex. Curr. Opin. Neurobiol. 12, 433–440. Rockland, K.S., 1997. Elements of cortical architecture. In: Peters, A., Rockland, K.S. (Eds.), Cerebral Cortex, vol. 10. Plenum Press, New York, pp. 243–294. Rockland, K.S., 2002. Visual cortical organization at the single axon level: a beginning. Neurosci. Res. 42, 155–166. Rockland, K.S., Drash, G.W., 1996. Collateralized divergent feedback connections that target multiple cortical areas. J. Comp. Neurol. 373, 529–548. Rockland, K.S., Ojima, H., 2001. Calcarine area V1 as a multimodal convergence area. Soc. Neurosci. 27, 1342 abstract. Rockland, K.S., Ojima, H., 2002. Multimodal convergence in calcarine visual areas in macaque monkey. International Multisensory Research Forum 3rd Ann. Mtg. Abstract.
Romanski, L.M., Tian, B., Fritz, J., Mishkin, M., GoldmanRakic, P.S., Rauschecker, J.P., 1999. Dual streams of auditory afferents target multiple domains in the primate prefrontal cortex. Nat. Neurosci. 2, 1131–1136. Sanderson, K.J., Dreher, B., Gayer, N., 1991. Prosencephalic connections of striate and extrastriate areas of rat visual cortex. Exp. Brain Res. 85, 324–334. Sauvan, X.M., Peterhans, E., 1999. Orientation constancy in neurons of monkey visual cortex. Vis. Cogn. 6, 43–54. Schroeder, C.E., Foxe, J., 2002. The timing and laminar profile of converging inputs to multisensory areas of the macaque neocortex. Cogn. Brain Res. 14, 187–198. Seltzer, B., Pandya, D.N., 1994. Parietal, temporal, and occipital projections to cortex of the superior temporal sulcus in the rhesus monkey: a retrograde tracer study. J. Comp. Neurol. 343, 445–463. Shams, L., Kamitani, Y., Shimojo, S., 2002. Visual illusion induced by second. Brain Res. Cogn. Brain Res. 14, 147–152. Stein, B.E., Meredith, M.A., 1993. The Merging of the Senses. MIT Press, Cambridge. Tian, B., Reser, D., Durham, A., Kustov, A., Rauschecker, J.P., 2001. Functional specialization in rhesus monkey auditory cortex. Science 292, 290–293.