From brain synapses to systems for learning and memory: Object recognition, spatial navigation, timed conditioning, and movement control

BRES : 43922 pp: 1224ðcol:fig: : NILÞ Model7 brain research ] (]]]]) ]]]–]]] 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 ...

Download PDF

5MB Sizes 0 Downloads 46 Views

Report

Full Text

BRES : 43922

pp: 1224ðcol:fig: : NILÞ

Model7 brain research ] (]]]]) ]]]–]]]

121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 Q1 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170 171 172 173 174 175 176 177 178 179 180

Available online at www.sciencedirect.com

www.elsevier.com/locate/brainres

Research Report

From brain synapses to systems for learning and memory: Object recognition, spatial navigation, timed conditioning, and movement control$ Stephen Grossbergn Center for Adaptive Systems, Graduate Program in Cognitive and Neural Systems, Center for Computational Neuroscience and Neural Technology, Department of Mathematics, Boston University, Boston, MA 02215, United States

art i cle i nfo

ab st rac t

Article history:

This article provides an overview of neural models of synaptic learning and memory whose

Accepted 6 November 2014

expression in adaptive behavior depends critically on the circuits and systems in which the synapses are embedded. It reviews Adaptive Resonance Theory, or ART, models that use

Keywords:

excitatory matching and match-based learning to achieve fast category learning and whose

Learning

learned memories are dynamically stabilized by top-down expectations, attentional focusing,

Memory

and memory search. ART clariﬁes mechanistic relationships between consciousness, learning,

Adaptive resonance theory

expectation, attention, resonance, and synchrony. ART models are embedded in ARTSCAN

Attention

architectures that unify processes of invariant object category learning, recognition, spatial and

Category learning

object attention, predictive remapping, and eye movement search, and that clarify how

Predictive remapping

conscious object vision and recognition may fail during perceptual crowding and parietal

Eye movement

neglect. The generality of learned categories depends upon a vigilance process that is regulated

Spatial navigation

by acetylcholine via the nucleus basalis. Vigilance can get stuck at too high or too low values,

Grid cell

thereby causing learning problems in autism and medial temporal amnesia. Similar synaptic

Place cell

learning laws support qualitatively different behaviors: Invariant object category learning in the

Time cell

inferotemporal cortex; learning of grid cells and place cells in the entorhinal and hippocampal

Adaptively controlled conditioning

cortices during spatial navigation; and learning of time cells in the entorhinal–hippocampal

Autism

system during adaptively timed conditioning, including trace conditioning. Spatial and temporal

Medial temporal amnesia

processes through the medial and lateral entorhinal–hippocampal system seem to be carried

mGluR

out with homologous circuit designs. Variations of a shared laminar neocortical circuit design

Laminar cortical circuits

have modeled 3D vision, speech perception, and cognitive working memory and learning. A

3D vision

complementary kind of inhibitory matching and mismatch learning controls movement.

Speech perception

This article is part of a Special Issue entitled SI: Brain and Memory.

Cognitive working memory

& 2014 Elsevier B.V. All rights reserved.

☆ Invited review article for a special issue on: Brain and Memory: Old Arguments and New Perspectives Michel Baudry and Gary Lynch, Eds. Brain Research. n Correspondence to: Center for Adaptive Systems, Boston University, 677 Beacon Street, Boston, MA 02215, United States. Fax: þ1 617 353 7755. E-mail address: [email protected] URL: http://cns.bu.edu/ steve

http://dx.doi.org/10.1016/j.brainres.2014.11.018 0006-8993/& 2014 Elsevier B.V. All rights reserved.

Please cite this article as: Grossberg, S., From brain synapses to systems for learning and memory: Object recognition, spatial navigation, timed conditioning, and movement control. Brain Research (2014), http://dx.doi.org/10.1016/j.brainres.2014.11.018

181 182 183 184 185 186 187 188

BRES : 43922

2

189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248

brain research ] (]]]]) ]]]–]]]

1. Linking brain mechanisms to behavioral functions: Unity and complementarity Einstein famously said that “A scientiﬁc theory should be as simple as possible, but no simpler”. In the case of how brains learn and remember, the very phrase “the search for the engram” (Lashley, 1950, 1960; Thompson, 1976) invokes a simplicity that may be too simple to meet the adaptive demands that are placed on advanced brains by ever-changing and often unpredictable environments. Before the proper level of simplicity can be asserted with conviction, a linkage needs to be made between brain mechanisms of learning and memory and the behavioral functions that they realize. Lashley (1950, 1960) already realized that the substrates of learning and memory are distributed throughout many parts of the brain. However, being distributed does not necessarily imply being mechanistically different. The current article reviews the conclusion drawn from neural models of learning and memory that, at least when one links brain mechanisms to behavioral functions, it seems that there is no single engram. One reason for this is that different behavioral functions sometimes require computationally complementary brain mechanisms (Grossberg, 2000). It is argued below, for example, that brain mechanisms in the What cortical stream for learning categories for object recognition and spatial navigation are complementary to motor mechanisms in the Where cortical stream that control the movements needed to reach and manipulate these objects. Despite the need for complementarity, there seem to nonetheless be some remarkable unities in the brain mechanisms that underlie very different functions. These include the mechanisms that are used to represent objects in the inferotemporal and prefrontal cortices (Cao et al., 2011; Carpenter and Grossberg, 1987, 1993; Chang et al., 2014; Grossberg, 1980; Fazl et al., 2009; Foley et al., 2012) and space and time representations in the entorhinal–hippocampal system (Grossberg and Merrill, 1992, 1996; Grossberg and Schmajuk, 1989; Grossberg and Pilly, 2012; Pilly and Grossberg, 2012; Mhatre et al., 2012). The computational homology between spatial and temporal representations has inspired the term neural relativity (Gorchetchnikov and Grossberg, 2007; Grossberg and Pilly, 2012).

representations of objects and events in the world. This kind of learning enables humans and other sufﬁciently advanced animals to rapidly learn new facts without being forced to just as rapidly forget what they already know. Such a competence was invaluable in the dangerous world in which our ancestors evolved. It is also useful in our advanced societies today, since it enables us to conﬁdently go out into the world without fearing that, in learning to recognize new information, such as a face, we will suddenly forget other useful information, such as the faces of our family and friends. This is sometimes called the problem of catastrophic forgetting. Grossberg (1980) has called the problem whereby the brain learns quickly and stably without catastrophically forgetting its past knowledge the stability–plasticity dilemma. Solving this problem during perceptual and cognitive development and learning was one of the main motivations behind the discovery of Adaptive Resonance Theory, or ART. The stability–plasticity dilemma must be solved by every brain system that needs to rapidly and adaptively respond to the ﬂood of signals – the “blooming buzzing confusion” of James (1890) – that subserves even the most ordinary experiences. If the brain’s design is parsimonious, then similar design principles should operate in all brain systems that can rapidly learn yet stably remember an accumulating knowledge base in response to changing conditions throughout life. The discovery of such principles should clarify how the brain uniﬁes diverse sources of information into coherent moments of conscious experience. ART describes several of these principles and the neural mechanisms that realize them. Match-based learning solves the stability–plasticity dilemma and is the kind of learning used in ART. Match-based learning coexists with excitatory matching. Examples of excitatory matching occur when a learned top-down expectation is sufﬁciently well matched against a bottom-up input pattern. Such a match can support a resonant state wherein gain ampliﬁcation of the matched pattern, synchronization of the activities that are ampliﬁed, and attentional focusing occur. ART has predicted,

2. Learning and memory by complementary cortical streams for recognition and action Both perceptual/cognitive and spatial/motor processes undergo learning and memory. Neural models of these processes have proposed, and many experiments have supported, the hypothesis that perceptual/cognitive and spatial/motor processes often use different learning and memory laws to carry out their disparate behavioral functions.

2.1. Excitatory match learning vs. inhibitory mismatch learning As summarized in Fig. 1, perceptual/cognitive processes in the What ventral cortical processing stream often use excitatory matching and match-based learning to create predictive

Fig. 1 – Complementary What and Where cortical processing streams for spatially-invariant object recognition and spatially-variant spatial representation and action, respectively. Perceptual and recognition learning use topdown excitatory matching and match-based learning that achieves fast learning without catastrophic forgetting. Spatial and motor learning use inhibitory matching and mismatch-based learning that enable rapid adaptation to changing bodily parameters. IT¼ inferotemporal cortex, PPC¼posterior parietal cortex. See text for details. [Reprinted with permission from Grossberg (2009).]

Please cite this article as: Grossberg, S., From brain synapses to systems for learning and memory: Object recognition, spatial navigation, timed conditioning, and movement control. Brain Research (2014), http://dx.doi.org/10.1016/j.brainres.2014.11.018

249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299 300 301 302 303 304 305 306 307 308

BRES : 43922 brain research ] (]]]]) ]]]–]]]

309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340 341 342 343 344 345 346 347 348 349 350 351 352 353 354 355 356 357 358 359 360 361 362 363 364 365 366 367 368

moreover, that fast learning can occur only during such a resonant state – hence the term adaptive resonance – and that “all conscious states are resonant states”. ART hereby predicted a link between processes of Consciousness, Learning, Expectation, Attention, Resonance, and Synchrony, the so-called CLEARS processes. All current experimental evidence seems to be compatible with this family of predictions. See Grossberg (2007, 2012) for reviews. Although it is predicted that “all conscious states are resonant states”, it is not predicted that “all resonant states are conscious states”. Indeed, some resonant states, such as the storage of a sequence of events in working memory before rehearsal occurs (see Grossberg and Pearson, 2008 for a review), or the entorhinal– hippocampal resonances that may dynamically stabilize the learning of entrorhinal grid cells and hippocampal place cells (see Section 8), are not accessible to consciousness.

2.2.

Learning to be an expert in a changing body

Match learning, and by extension ART, does not describe the only kind of learning that the brain needs to accomplish autonomous adaptation to a changing world. If only for this reason, ART is not a theory about “everything”. There are just as essential, but complementary, spatial/motor processes in the Where dorsal cortical processing stream that often use inhibitory matching and mismatch-based learning (Fig. 1) to continually update spatial maps and sensory-motor gains as our bodily parameters change through time (Bullock and Grossberg, 1988; Bullock et al., 1998; Gaudiano and Grossberg, 1991; Georgopoulos et al., 1982, 1986). Indeed, we do not want the learned spatial representations and motor gains that controlled our infant bodies to be remembered and used to control our adult bodies. In this sense, catastrophic forgetting is a good property during spatial and motor learning. Inhibitory matching is illustrated by how we make an arm movement. To make such a movement, a representation of where the arm is now (its present position vector) is subtracted from a representation of where we want the arm to move (its target position vector), thereby computing a difference vector that represents the direction and distance of movement needed to attain the target. After moving to the target, the target and present positions agree, so the difference vector is zero. This sort of matching is thus inhibitory (Bullock and Grossberg, 1988). Neither type of matching and learning is sufﬁcient to design an adaptive autonomous agent, but each is necessary. By combining these two types of processes together, our brains can incrementally learn and stably remember perceptual and cognitive representations of a changing world, leading to a selfstabilizing front end that solves the stability–plasticity dilemma and enables us to become increasingly expert in understanding the world and predicting outcomes in the world. At the same time, our brains can adaptively update their representations of where objects are and how to act upon them using bodies whose parameters change continuously through time due to development, exercise, illness, and aging.

2.3.

Why procedural memories are not conscious

Brain systems that use inhibitory matching and mismatch learning cannot generate excitatory resonances. Hence, if “all

3

conscious states are resonant states”, then spatial and motor representations are not conscious. This way of thinking provides a mechanistic reason why declarative memories (or “learning that”), which are the sort of memories learned by ART, may be conscious, whereas procedural memories (or “learning how”), which are the sort of memories that control spatial orienting and action, are not conscious (Cohen and Squire 1980).

2.4. Spatially-invariant recognition vs. spatially localized action There is another basic reason why these complementary What and Where processes need to work together. The What stream attempts to learn spatially-invariant object categories, so that a combinatorial explosion does not occur wherein every view of every object at every position and distance needs to be represented by a different category. Indeed, learning in the What cortical stream leads to recognition categories that tend to be increasingly independent of object size and position at higher cortical levels. The anterior inferotemporal cortex exhibits such invariance (Bar et al., 2001; Sigala and Logothetis, 2002; Tanaka et al., 1991). Although how this occurs needs careful discussion (e.g., Zoccolan et al., 2007), such object invariance prevents a combinatorial explosion in memory of object representations that otherwise would need to be learned at every perceived size and position. Cao et al. (2011) and Grossberg et al. (2011) have used ART to simulate recent neurophysiological data about neurophysiologically observed properties of invariant category learning and recognition in inferotemporal cortex. In becoming spatially invariant, recognition categories lose information about how to direct action towards the locations in space where desired objects may be found. In contrast, the Where stream learns spatial maps that do enable us to locate such desired objects, as well as the movement gains that enable us to accurately act with respect to them. On the other hand, Where stream spatial processing gives up information about which objects are in those spatial locations. Interactions between the What and Where stream (“What–Where fusion”) overcome these complementary deﬁciencies to enable learned spatially-invariant object representations to control actions towards desired goals in space (e.g., Brown et al., 2004; Fazl et al., 2009; Grossberg, 2009), including actions that are learned by observing a teacher who experiences the world from a different perspective, by sharing joint attention during social Q4 cognition (Grossberg and Vladusich, 2011). In summary, because of their different types of matching and learning, perceptual and cognitive learning provide a self-stabilizing front end to control the more labile spatial and motor learning that enables changing bodies to effectively act upon recognized objects in the world.

2.5. Synaptic learning laws that support complementary computing Object, spatial, and temporal representations all use a family of learning laws whose variations are called gated steepest descent, normalized gated steepest descent, instar, outstar, fuzzy ART, and adaptive threshold learning laws. These variations of ART learning laws all support match-based

Please cite this article as: Grossberg, S., From brain synapses to systems for learning and memory: Object recognition, spatial navigation, timed conditioning, and movement control. Brain Research (2014), http://dx.doi.org/10.1016/j.brainres.2014.11.018

369 370 371 372 373 374 375 376 377 378 379 380 381 382 383 384 385 386 387 388 389 390 391 392 393 394 395 396 397 398 399 400 401 402 403 404 405 406 407 408 409 410 411 412 413 414 415 416 417 418 419 420 421 422 423 424 425 426 427 428

BRES : 43922

4

429 430 431 432 433 434 435 436 437 438 439 440 441 442 443 444 445 446 447 448 449 450 451 452 453 454 455 456 457 458 459 460 461 462 463 464 465 466 467 468 469 470 471 472 473 474 475 476 477 478 479 480 481 482 483 484 485 486 487 488

brain research ] (]]]]) ]]]–]]]

learning and excitatory matching. Indeed, when these synaptic learning laws are embedded into ART networks, their learning turns on when resonance occurs, and turns off when a big enough mismatch occurs between the predicted state (e.g., a learned top-down expectation) and the actual data (e.g., a bottom-up input pattern). The motor representations use variations of vector associative map, or VAM, learning laws that support mismatchbased learning and inhibitory matching laws (Gaudiano and Grossberg, 1991, 1992). In VAM learning laws, the difference vectors that control actions also serve as error signals that drive the learning process. Just as ART learning laws are triggered by a context-sensitive state, namely a resonance, VAM learning laws are modulated by whether or not the system is in a postural or movement state. Learning during a ﬁxed posture can adaptively change movement gains to help achieve desired target positions. Shutting off learning during movement prevents learning from undermining correctly calibrated gains. All these movement systems have GO and STOP, or burster and pauser, signals that can be used to control the modulatory signals that gate learning off and on.

3. Equations for short-term memory, mediumterm memory, and long-term memory Despite the need for specialization to accommodate requirements such as complementary computing, all of the neural learning laws that the author has introduced to model a wide range of behavioral functions, and that are used by many other modellers, can be characterized using a small set of equations for short-term memory, or STM; medium-term memory, or MTM; and long-term memory, or LTM). These laws are embedded within a somewhat larger number of modules or microcircuits (e.g., shunting on-center off-surround networks, gated dipole opponent processing networks, associative learning networks, spectral spacing and timing networks, difference vector networks, and the like) which, in turn, are specialized and assembled into modal architectures, where the term “modal” stands for modality (e.g., architectures for vision, audition, cognition, cognitive-emotional interactions, sensory-motor control, and the like). Modal architectures are less general than a Turing or von Neumann architecture for general computing, but far more general than a traditional AI algorithm. They are designed to be capable of general-purpose self-organizing processing of a particular modality of biological intelligence. Their particular specializations of the basic equations and modules have been selected over the millennia by evolutionary pressures. ART networks form part of such modal architectures. Modal architectures, in turn, embody new paradigms for brain computing, including Complementary Computing (Grossberg, 2000) and Laminar Computing (Grossberg, 1999). As noted above, Complementary Computing describes how the global brain is organized into complementary parallel processing streams whose interactions generate biologically intelligent behaviors. Laminar Computing describes how and why the cerebral cortex is organized into layered circuits whose specializations can support all forms of higher-order biological intelligence. ART networks are embodied in

laminar cortical circuitry for vision and visual object recognition; audition, speech, and language; cognitive information processing; and cognitive-emotional dynamics. Grossberg (1968c, 1969a) introduced laws of STM, MTM, and LTM that are used, with suitable variations and specializations, in many contemporary neural models, including ART architectures. One variant of them is:

3.1. STM: Short-term memory dynamics of fast cellular activation and inhibition dxi ¼ Axi þ ðBCxi Þ dt " n

#

"

#

n

Ii þ ∑ f k ðxk Þyk Dki zki ðE þ Fxi Þ Ji þ ∑ gk ðxk ÞY k Gki Zki : k¼1

k¼1

ð1Þ This equation describes the activities, or potentials, xi , of the ith cell (population) in a network of n interacting neurons. It includes both the Additive and Shunting models that were introduced in Grossberg (1968c, 1969a). In the shunting model, the parameters C a0 and F a0: The parameter E ¼ 0 when there is “silent” shunting inhibition, whereas Ea0 describes the case of hyperpolarizing shunting inhibition. In the Additive model, parameters C ¼ F ¼ 0: The excitatory interaction term Ii þ ∑nk ¼ 1 f k ðxk Þyk Dki zki describes an external input Ii plus the total excitatory feedback signal ∑nk ¼ 1 f k ðxk Þyk Dki zki that is a sum of signals from other populations via their output signals f k ðxk Þ. These output signals are often interpreted in terms of the spiking frequencies that are generated by potential xk . The term Dki is a constant connection strength between cell populations k and i, whereas terms yk and zki describe MTM and LTM variables, respectively. The inhibitory interaction term Ji þ ∑nk ¼ 1 gk ðxk ÞY k Dki Zki has a similar interpretation. Eq. (1) assumes “fast inhibition”; that is, inhibitory interneurons respond instantaneously to their inputs. Slower ﬁnite-rate inhibition, with activities Xi that are computed by inhibitory interneurons, uses an equation like (1) to describe the temporal evolution of the inhibitory activities. The output signals from these inhibitory interneurons provide the inhibitory feedback signals to the excitatory activities. With slow inhibition, the inhibitory feedback signals would be gk ðXk Þ instead of gk ðxk Þ:

3.2. MTM: Habituative transmitter gates and depressing synapses dyi ¼ HðK yi Þ Lf k ðxk Þyk : dt

ð2Þ

Eq. (2) describes how the strength yi of the habituative transmitter gate, or depressing synapse (Abbott et al., 1997), or dynamic synapse (Tsodyks, Pawelzik, and Markram, 1998), in the excitatory feedback term of (2) accumulates at a ﬁxed rate H to its maximum value K via term hðK yi Þ and is inactivated, habituated, or depressed via a mass action interaction between the feedback signal f k ðxk Þ and the gate concentration yk : The mass action term may be more complex than this in some situations; e.g., Gaudiano and Grossberg (1991, 1992). The habituative transmitter gate Yk in the inhibitory feedback

Please cite this article as: Grossberg, S., From brain synapses to systems for learning and memory: Object recognition, spatial navigation, timed conditioning, and movement control. Brain Research (2014), http://dx.doi.org/10.1016/j.brainres.2014.11.018

489 490 491 492 493 494 495 496 497 498 499 500 501 502 503 504 505 506 507 508 509 510 511 512 513 514 515 516 517 518 519 520 521 522 523 524 525 526 527 528 529 530 531 532 533 534 535 536 537 538 539 540 541 542 543 544 545 546 547 548

BRES : 43922

5

brain research ] (]]]]) ]]]–]]]

549 550 551 552 553 554 555 556 557 558 559 560 561 562 563 564 565 566 567 568 569 570 571 572 573 574 575 576 577 578 579 580 581 582 583 584 585 586 587 588 589 590 591 592 593 594 595 596 597 598 599 600 601 602 603 604 605 606 607 608

term of (1) obeys a similar equation. It should be noted that not all signals are habituative.

3.3. LTM: Gated steepest descent learning combines Hebbian and anti-Hebbian properties

dzij ¼ Mf i ðxi Þ hj ðxj Þ zij dt

kai

synaptic activity by other signals converging on the jth cell. Rewrite (5) as " # dzij ¼ Mf j ðxj Þ Ahi ðxi Þ zij ∑ hðxk Þ ; ð6Þ dt k and then as

ð3Þ

and dzij ¼ Mf j ðxj Þ hi ðxi Þ zij : dt

In (5), the term zij ∑ hðxk Þ describes a competition for

2

3

dzij 6 Ah ðx Þ 7 ¼ Mf j ðxj Þ∑ hðxk Þ4 i i zij 5: ∑ hðxk Þ dt k

ð7Þ

k

ð4Þ

Eq. (3) describes the outstar learning equation, by which an ith source cell can sample and learn a distributed spatial pattern of activation across a network of sampled cells. When the gating signal f i ðxi Þ is positive, the adaptive weights zij can learn about the activity-dependent signals hj ðxj Þ across the sampled network of cells. Then each zij converges via term hj ðxj Þ zij to a time-average of the signals hi ðxi Þ at a rate determined by f i ðxi Þ. As a result, zij can either increase or decrease through time while tracking hi ðxi Þ. This tracking operation is called steepest descent; hence the general name of such a learning law: gated steepest descent. When the gating, or sampling, signal f i ðxi Þ equals zero, then neither learning nor forgetting can occur. Eq. (4) describes another variant of learning by gated steepest descent; namely, the instar learning equation, by which the adaptive weights zij that abut the jth target cell can, when f j ðxj Þ is positive, sample and learn the distributed pattern of signals hi ðxi Þ that are passing through their synapse. The adaptive tuning of these weights converts term ∑nk ¼ 1 f k ðxk Þyk Dki zki in (1) into an adaptive ﬁlter whereby an input pattern can more effectively activate the cells that were able to sample it during prior learning. This is the learning equation that was used in the competitive learning and selforganizing map models in Grossberg (1976a, 1978), and later applied by Kohonen (1984). The catastrophically unstable memories of competitive learning and self-organizing maps in response to dense non-stationary series of input patterns led to the introduction of ART as a way to dynamically stabilize learned memories using matching and attentional focusing by top-down expectations upon bottom-up input patterns. There are many variations of these gated steepest descent equations that are been used in networks that support different behavioral functions; e.g., normalized gated steepest descent, doubly-gated learning, spike-timing dependent learning, etc. It should also be noted that not all connections are adaptive. Variations of the following normalized instar learning law have been used to learn maps of orientation and ocular dominance in cortical area V1 (Grossberg and Seitz, 2003; Grossberg and Williamson, 2001), grid cells and place cells in the entorhinal cortex and hippocampal cortex, respectively (Grossberg and Pilly, 2012, 2014; Mhatre et al., 2012), and recognition categories in the inferotemporal cortex (Carpenter and Grossberg, 1987, 1993): " # dzij ¼ Mf j ðxj Þ hi ðxi ÞðA zij Þ zij ∑ hðxk Þ : ð5Þ dt kai

The sum zi ¼ ∑ zij over all the weights zij in (7) that converge j onto the jth cell obeys: dzi ¼ Mf j ðxj Þ∑ hðxk Þ½A zi : dt k

ð8Þ

Eq. (8) shows that the total synaptic weight zi converges to A, and thus is normalized with a value that is independent of the total number of connections that abut the jth cell. This property is often called conservation of total synaptic weight. This sort of learning law is experimentally supported by data about the competition among developing axons that abut a target neuron for limited target-derived neurotrophic factor in order to survive (Cabelli et al., 1995, 1997; Purves, 1988), and about the conservation of total synaptic weight by balanced synaptic depression and potentiation (Royer and Pare, 2003).

4. When learning is not Hebbian: Combining LTP and LTD 4.1.

Gated steepest descent learning

As noted above, gated steepest descent learning weights can increase (long-term potentiation, or LTP) or decrease (long-term depression, or LTD) during the learning process. One reason for this assumption is that these weights learn about distributed patterns of inputs across a network, as was ﬁrst mathematically proved in Grossberg (1968a, 1968b), not just about a single input through a single connection between pairs of cells, as postulated by Hebb (1949). If the initial value of a weight that learns by gated steepest descent is smaller than the activity in a pattern that it is learning, then the weight will increase during learning (LTP). However, if the initial weight value is larger than the activity in a pattern that it is learning, then the weight will decrease during learning (LTD). Likewise, if two or more patterns are sampled sequentially in time during learning, one with larger activities and another with smaller activities at a given cell, then the corresponding adaptive weight may ﬁrst increase and then decrease to track the average of the activities that it samples through time. Reﬂecting this difference in the assumed functional units of learning, Hebb (1949) asserted that adaptive weights can only increase during learning whenever there is temporally contiguous pre- and post-synaptic pairing of inputs. Due to this property, weights could saturate at their maximum values as learning trials continue, assuming that all biological computations, including the computation of adaptive weights, have ﬁnite maximum values. They could then become useless for dealing with changes in environmental contingencies such as those that

Please cite this article as: Grossberg, S., From brain synapses to systems for learning and memory: Object recognition, spatial navigation, timed conditioning, and movement control. Brain Research (2014), http://dx.doi.org/10.1016/j.brainres.2014.11.018

609 610 611 612 613 614 615 616 617 618 619 620 621 622 623 624 625 626 627 628 629 630 631 632 633 634 635 636 637 638 639 640 641 642 643 644 645 646 647 648 649 650 651 652 653 654 655 656 657 658 659 660 661 662 663 664 665 666 667 668

BRES : 43922

6

669 670 671 672 673 674 675 676 677 678 679 680 681 682 683 684 685 686 687 688 689 690 691 692 693 694 695 696 697 698 699 700 701 702 703 704 705 706 707 708 709 710 711 712 713 714 715 716 717 718 719 720 721 722 723 724 725 726 727 728

brain research ] (]]]]) ]]]–]]]

might require a later reduction of weight. Such a reduction in weight can occur, for example, during reinforcement learning when a previously reinforced cognitive-emotional association, such as a conditioned reinforcer or incentive motivational adaptive weight, is later extinguished in favor of different reinforcing contingencies (Dranias et al., 2008; Grossberg, 1972, 1975; Kamin, 1969; Pavlov, 1927). Despite this basic problem of Hebbian learning, it has become commonplace for many experimental neuroscientists to reﬂexively label results about associative learning as being examples of Hebbian learning. This attribution does not sufﬁciently take into account what Hebb actually wrote. Outstars and instars were the ﬁrst mathematically deﬁned neural learning laws to combine both Hebbian (weight increase, LTP) and anti-Hebbian (weight decrease, LTD) properties, so that they can they track the patterns with which they are associated in time. These tracking properties were mathematically proved in a series of articles in the 1960s and early 1970s for increasingly complicated learning situations; e.g., for spatial pattern learning in outstars and recurrent networks (Grossberg, 1968b, 1969a), serial list learning (Grossberg, 1969c; Grossberg and Pepe, 1971), learning of arbitrarily complicated space-time patterns (Grossberg, 1969d) as in the case of birdsong (Hahnloser et al., 2002), and category learning in competitive learning and self-organizing map models (Grossberg, 1976a). Instars and outstars were joined at recurrent competitive networks in Grossberg (1976a) to show how to learn arbitrary maps from m-dimensional to n-dimensional vectors. In this universal recoding model, instars use their adaptive ﬁlters to enable the m-dimensional input vectors to selectively activate category cells as part of a competitive learning or selforganizing map model, after which the category cells read out the n-dimensional output vectors that they learn using outstar learning. This type of instar–outstar map learning scheme was called counter propagation by Hecht-Nielsen (1987). ART shows, in addition, how to learn maps whose adaptive weights are dynamically self-stabilizing, by using learned top-down expectations from the category cells that are matched against the m-dimensional input patterns. The top-down expectations also often use outstar learning; e.g., Carpenter and Grossberg (1987).

4.2. Monotonic weights during match-modulated category learning As noted above, if adaptive weights could only get larger or smaller through time whenever temporally contiguous preand post-synaptic inputs were paired, then they could become “stuck” at maximum or minimum values that may not adapt to changes in environmental contingencies. However, some category learning models do posit adaptive weights that can only change monotonically through time, such as fuzzy ART and fuzzy ARTMAP (Carpenter et al., 1991, 1992). Because of the monotonic change through time of these weights, they are guaranteed to converge to a limit. The problem is to ensure that the limiting values enable useful category recognition behaviors. Unlike the Hebb rule, this is ensured by control processes that prevent learning except when there is a good enough match between a currently active learned top-down expectation and bottom-up input

pattern. Temporally contiguous pre- and post-synaptic pairing of inputs is not sufﬁcient. See Section 6 for a discussion of how such matching happens.

5.

Neurophysiological data about LTP and LTD

Bliss and Lomo (1973) using the anesthetized rabbit preparation, and Bliss and Gardner-Medwin (1973) using the unanesthetized rabbit, provided the ﬁrst demonstration of a long-term potentiation (LTP) of synaptic efﬁcacy following tetanic stimulation of afferent ﬁbers in the perforant path to dentate gyrus synapse in the hippocampus. Long-tem depression (LTD) has also been reported in the hippocampus, including at area CA1 pyramidal cells, in pathways that can also induce LTP at higher rates of stimulation (e.g., Dudek and Bear, 1992; Levy et al., 1983; Levy and Desmond, 1985, Levy and Steward, 1983; Mulkey and Kalenka, 1992). Such LTP and LTD effects have also been reported in the cerebral cortex, including in the visual cortex of cats and rats (e.g., Artola and Singer, 1987, 1993; Fregnac et al., 1994; Kirkwood et al., 1995; Komatsu et al., 1981; Singer, 1983), among other structures in a literature that is, by now, vast. These results do not disclose, however, how shared synaptic properties may control different behavioral functions in different parts of the brain, or how and why particular combinations of functions are controlled by speciﬁc brain regions. For example, how do similar learning laws contribute to the learning of categories for visual object recognition by inferoretmporal and predfrontal cortex, as well as to spatial navigation via the entorhinal–hippocampal system? Why are both spatial navigation (space!) and adaptively timed conditioning (time!) represented in the medial and lateral streams, respectively, of the entorhinal–hippocampal system? This article reviews some of the network- and system-level interactions that are proposed to give rise to such behavioral properties.

6. Adaptive resonance theory: Fast category learning without catastrophic forgetting 6.1.

Solving the stability–plasticity dilemma

One emerging family of neural models controls how the visual cortex learns, recognizes, and remembers invariant visual object recognition categories and how this process is linked to object and spatial attention, predictive remapping, consciousness, and object search in the visual cortex. All these models build upon the category learning and memory capabilities of Adaptive Resonance Theory, or ART, models (Carpenter, 1997, 2003; Carpenter and Grossberg, 1987, 1993; Carpenter et al., 1991, 1992; Grossberg, 1976a, 1976b, 1978, 1980). All of the main ART predictions have received increasing support from psychological and neurobiological data since ART was introduced in Grossberg (1976a, 1976b), and ART is currently the cognitive and neural theory of recognition learning and memory with the broadest explanatory and predictive range. Successive developments of ART have explained and predicted increasingly large behavioral and neurobiological data bases, ranging from normal and abnormal aspects of human and animal perception and cognition, to the spiking and

Please cite this article as: Grossberg, S., From brain synapses to systems for learning and memory: Object recognition, spatial navigation, timed conditioning, and movement control. Brain Research (2014), http://dx.doi.org/10.1016/j.brainres.2014.11.018

729 730 731 732 733 734 735 736 737 738 739 740 741 742 743 744 745 746 747 748 749 750 751 752 753 754 755 756 757 758 759 760 761 762 763 764 765 766 767 768 769 770 771 772 773 774 775 776 777 778 779 780 781 782 783 784 785 786 787 788

BRES : 43922 brain research ] (]]]]) ]]]–]]]

789 790 791 792 793 794 795 796 797 798 799 800 801 802 803 804 805 806 807 808 809 810 811 812 813 814 815 816 817 818 819 820 821 822 823 824 825 826 827 828 829 830 831 832 833 834 835 836 837 838 839 840 841 842 843 844 845 846 847 848

oscillatory dynamics of hierarchically-organized laminar thalamocortical networks in multiple modalities. Indeed, some ART models explain and predict behavioral, anatomical, neurophysiological, biophysical, and even biochemical data. In this sense, they provide a growing set of examples capable of partially solving the classical mind/body problem. See Grossberg (2012) for a review of these developments.

6.2. Top-down matching, resonance, attention, and fast learning ART solves the stability–plasticity dilemma by proposing how top-down expectations focus object attention on salient combinations of visual features, and characterizes how attention may operate via a form of self-normalizing “biased competition” (Desimone, 1998). There is a convergence across models of how to mathematically instantiate the ART attentional circuit that have grown out of the results about the properties of shunting on-center off-surround networks that were originally proved in Grossberg (1973). For example, the “normalization model of attention” (Reynolds and Heeger, 2009) simulates several types of experiments on attention using the same equation for self-normalizing attention as the distributed ARTEXture (dARTEX) model (Bhatt et al., 2007, equation (A5)) used to simulate human psychophysical data about Orientation-Based Texture Segmentation (OBTS, BenShahar and Zucker, 2004). When a good enough match occurs between a top-down learned expectation, that is read out by a currently active recognition category, and a bottom-up input pattern, a synchronous resonant state emerges that embodies an attentional focus. This resonance is capable of driving fast learning of bottom-up recognition categories and top-down expectations within the attentional focus that can reﬁne previous knowledge; hence the name adaptive resonance. It has been mathematically proved that match learning within an ART model leads to stable memories of arbitrary events presented in any order (e.g., Carpenter and Grossberg, 1987, 1991). Simple learning laws, such as variants of instar and outstar learning, can solve the stability–plasticity dilemma when they are modulated by the results of such an attentional match. In particular, the adaptive weights in bottom-up pathways that help to learn recognition categories are often normalized instar or fuzzy ART laws (Eq. (5)), whereas the weights in top-down pathways that help to learn expectations are often outstar laws (Eq. (3)).

6.3.

Matching and synchronous oscillations

The resonance process is predicted to take place in the What cortical stream, notably in the sensory, temporal, and prefrontal cortices, where top-down expectations are matched against bottom-up inputs. When a top-down expectation achieves a good enough match with bottom-up data, the emerging attentional focus is often realized by oscillatory dynamics that synchronize the ﬁring properties of the resonating neurons. These match-sustained oscillations were predicted to occur in Grossberg (1976b), where they were called orderpreserving limit cycles to specify that the oscillating activities do not reverse their relative sizes through time, in keeping

7

with the idea that resonating features preserve their relative importance during the resonant event. Such oscillations are currently called synchronous oscillations (e.g., Eckhorn et al., 1988; Engel et al., 2001; Gray and Singer, 1989). Several subsequent modeling studies have simulated how these oscillations can explain data about such varied properties as fast synchronization by cooperative feature linking during perceptual grouping and attentional matching (Grossberg and Somers, 1991), perceptual framing, temporal order judgments, and stochastic resonance (Grossberg and Grunewald, 1997), and analog sensitivity of fast synchronization of perceptual grouping by the laminar circuits of visual cortex (Yazdanbakhsh and Grossberg, 2004). Of particular interest are simulations within realistic spiking laminar cortical circuits of faster gamma oscillations during a top-down attentional match and slower beta oscillations during a big enough mismatch, with the mismatch initiated within the deeper layers of the cortex (Grossberg and Versace, 2008). This gamma/beta prediction has received neurophysiological support from several labs in different parts of the brain (e.g., Berke et al., 2008; Buffalo et al., 2011; Buschman and Miller, 2009). The prediction is consistent with the possibility that other neural mechanisms can also cause such oscillatory frequencies.

6.4.

The cycle of resonance and reset

Match learning has a serious potential weakness: If a brain can only learn when there is a good enough match between bottom-up data and learned top-down expectations, then how can it ever learn anything that is really novel? When learning ﬁrst begins, and no learned recognition categories exist, then a novel input pattern can activate a potential future category that has the best match with it via the small, random initial weights in the bottom-up pathways. Then contrast enhancement via the rules of competitive learning and self-organizing maps (Grossberg, 1976a; Kohonen, 1984; von der Malsburg, 1973) can Q5 strengthen these bottom-up weights to ensure a better match with the input pattern in the future. However, this can occur only if, on the ﬁrst learning trial, the activated category can read out top-down signals that provide a good enough match with the novel input pattern to allow learning to proceed. In order for a top-down expectation to match the features that activate any new recognition category, all of its top-down adaptive weights initially have large values, which are pruned by the learning of a particular expectation to match the feature patterns that activate the category. After this ﬁrst category is learned, its large learned weights can compete with the small random weights in the bottom-up ﬁlter to interfere with learning about new objects and events. How is this problem overcome in a situation where the learning is unsupervised, and no external teacher is available to force the selection of a correct answer? ART proposes that this problem is solved by an interaction between complementary processes of resonance and reset that are predicted to control properties of attention and memory search, respectively. These complementary processes help our brains to balance between the complementary demands of processing the familiar and the unfamiliar, the expected and the unexpected. A big enough mismatch, which is computed within the attentional system, is proposed to activate a

Please cite this article as: Grossberg, S., From brain synapses to systems for learning and memory: Object recognition, spatial navigation, timed conditioning, and movement control. Brain Research (2014), http://dx.doi.org/10.1016/j.brainres.2014.11.018

849 850 851 852 853 854 855 856 857 858 859 860 861 862 863 864 865 866 867 868 869 870 871 872 873 874 875 876 877 878 879 880 881 882 883 884 885 886 887 888 889 890 891 892 893 894 895 896 897 898 899 900 901 902 903 904 905 906 907 908

BRES : 43922

8

909 910 911 912 913 914 915 916 917 918 919 920 921 922 923 924 925 926 927 928 929 930 931 932 933 934 935 936 937 938 939 940 941 942 943 944 945 946 947 948 949 950 951 952 953 954 955 956 957 958 959 960 961 962 963 964 965 966 967 968

brain research ] (]]]]) ]]]–]]]

complementary orienting system, which is activated by unexpected and unfamiliar events (Fig. 2b and c), and which includes the nonspeciﬁc thalamus and the hippocampal system. See Carpenter and Grossberg (1993) and Grossberg and Versace (2008) for consistent data. Output signals from the orienting system rapidly reset the recognition category that has been reading out the poorly matching top-down expectation. The cause of the mismatch is hereby removed, thereby freeing the system to activate a different recognition category (Fig. 2d). In this way, a reset event triggers memory search, or hypothesis testing, which automatically leads to the selection of a recognition category that can better match the input.

6.5. Memory consolidation and direct access to globally best match As sequences of inputs are practiced over learning trials, the search process eventually converges upon stable categories. It has been mathematically proved (e.g., Carpenter and Grossberg, 1987) that familiar inputs directly access the category whose prototype provides the globally best match, without undergoing any search, while unfamiliar inputs continue to activate the orienting subsystem to trigger memory searches for new or better-ﬁtting categories until they become familiar. In other words, ART provides a solution of the local minimum problem and clariﬁes how familiar objects can be rapidly recognized even as the total number of learned memories increases dramatically. The processes of search and category learning continue until the memory capacity, which can be chosen arbitrarily large, is fully utilized.

6.6.

Vigilance, acetylcholine, and nucleus basalis

A vigilance parameter controls how big a mismatch is needed to drive a memory search. Low vigilance permits the learning of general and abstract categories because it tolerates large mismatches between the currently active top-down expectation and the bottom-up input exemplar with which it is being matched. High vigilance forces learning of more speciﬁc and concrete categories, with individual input exemplars learning to activate their own categories in the limit of very high vigilance. Vigilance is computed within the orienting system of an ART model (Fig. 2b–d), where the total bottom-up excitation from all the active features in an input pattern I is reduced by the total inhibition from all the feature detectors that it activates (see F1 in Fig. 2). If the ratio of the total activity across the active features in F1 (that is, the “matched” features) to the total activity due to all the features in I is less than a vigilance parameter ρ (Fig. 2b), then a nonspeciﬁc reset, or arousal, wave is activated (Fig. 2c), which can drive the search for another category with which to classify the exemplar. This can be accomplished by letting ρ multiply the bottom-up inputs I to the orienting system; that is, ρ is the gain of the bottom-up inputs to the orienting system. The orienting system is activated when the total excitatory input ρI is greater than the total inhibition from the features Xn across top-down matching; that is, when F1 that survive ρI Xn 40, where : denotes the number of positive inputs 1 or matched features. Rewriting this inequality as ρ4Xn I

Fig. 2 – Search for a recognition code within an ART learning circuit: (a) Input pattern I is instated across feature detectors at level F1 as an activity pattern X, while it nonspeciﬁcally activates the orienting system A with gain ρ, which is called the vigilance parameter. Output signals from activity pattern X inhibits A and generates output pattern S. S is multiplied by learned adaptive weights to form the input pattern T. T activates category cells Y at level F2. (b) Y generates the topdown signals U which are multiplied by adaptive weights and added at F1 cells to form a prototype V that encodes the learned expectation of active F2 categories. If V mismatches I at F1, then a new STM activity pattern X* (the hatched pattern) is selected at F1. X* is active at I features that are conﬁrmed by V. Mismatched features (white area) are inhibited. When X changes to X*, total inhibition decreases from F1 to A. (c) If inhibition decreases sufﬁciently so that the total inhibition due to X* is less than the total excitation due to I multiplied by the vigilance parameter ρ, then A is activated and releases a nonspeciﬁc arousal burst to F2; that is, “novel events are arousing”. Arousal resets F2 by inhibiting Y. (d) After Y is inhibited, X is reinstated and Y stays inhibited as X activates a different activity pattern Y*. Search for better F2 category continues until a better matching or novel category is selected. When search ends, an attentive resonance triggers learning of the attended data. [Adapted with permission from Carpenter and Grossberg (1993).]

shows that the orienting system is activated whenever ρ is chosen higher than the ratio of active Xn matched features in F1 to total features in I. In other words, the vigilance parameter controls how bad a match can be before search for a new category is initiated. If the vigilance parameter is low, then many exemplars can all inﬂuence the learning of a

Please cite this article as: Grossberg, S., From brain synapses to systems for learning and memory: Object recognition, spatial navigation, timed conditioning, and movement control. Brain Research (2014), http://dx.doi.org/10.1016/j.brainres.2014.11.018

969 970 971 972 973 974 975 976 977 978 979 980 981 982 983 984 985 986 987 988 989 990 991 992 993 994 995 996 997 998 999 1000 1001 1002 1003 1004 1005 1006 1007 1008 1009 1010 1011 1012 1013 1014 1015 1016 1017 1018 1019 1020 1021 1022 1023 1024 1025 1026 1027 1028

BRES : 43922 brain research ] (]]]]) ]]]–]]]

1029 1030 1031 1032 1033 1034 1035 1036 1037 1038 1039 1040 1041 1042 1043 1044 1045 1046 1047 1048 1049 1050 1051 1052 1053 1054 1055 1056 1057 1058 1059 1060 1061 1062 1063 1064 1065 1066 1067 1068 1069 1070 1071 1072 1073 1074 1075 1076 1077 1078 1079 1080 1081 1082 1083 1084 1085 1086 1087 1088

shared prototype, which eliminates all the features that are not shared with all the exemplars, thereby leading to learning of an abstract category. If the vigilance parameter is high, then even a small difference between a new exemplar and a known prototype (e.g., F vs. E) can drive the search for a new category with which to represent F, thereby leading to learning of a more exemplar-like category. Vigilance can vary across learning trials. For example, during supervised learning, it may be controlled by a process of match tracking (Carpenter et al., 1991, 1992) whereby a predictive error (e.g., D is predicted in response to F) causes the vigilance parameter ρ to increase just enough to trigger reset and search for a better-matching category. Match tracking gives up the minimum amount of generalization in the learned categories to search for a better-matching category. It hereby “tracks” the degree of match between input exemplar and matched prototype, and thus increases vigilance by the minimum amount needed to trigger a reset and search for a new category. Match tracking hereby realizes a Minimax Learning Rule that conjointly maximizes category generality while it minimizes predictive error. Otherwise expressed, match tracking uses the least memory resources that can correct errors in classiﬁcation. ART predicts how vigilance may be controlled by acetycholine via the nucleus basalis of Meynert (Grossberg and Versace, 2008; Palma et al., 2012; Palma et al., 2012), thereby proposing a more precise interpretation of various data about the role of acetylcholine in cortical dynamics (e.g., Kraus et al., 1994; van Der Werf et al., 2002).

6.7.

Vigilance diseases: Autism and amnesia

Various mental disorders seem to include problems with vigilance control, leading to the hypothesis that it may be useful to classify the symptoms due to vigilance malfunctions as illustrations of “vigilance diseases”. Autism and medial temporal amnesia seem to be examples of such diseases. It should be clear, however, that a problem with vigilance is not the only problem that individuals with these mental disorders may have. Persistent high vigilance has been predicted to cause symptoms of hyperspeciﬁc category learning and attentional deﬁcits in some autistic individuals (Grossberg and Seidman, 2006). This prediction has been tested through psychophysical experiments done with high-functioning autistic individuals (Church et al., 2010; Vladusich et al., 2010). It is also consistent with reports of abnormal cholinergic activity in the parietal and frontal cortices of autistic individuals that is correlated with abnormalities in the nucleus basalis (Perry et al., 2001; Rubenstein and Merzenich, 2003). Persistent low vigilance has been predicted to cause some symptoms of medial temporal amnesia (Carpenter and Grossberg, 1993). A lesion of the hippocampus removes the orienting system from cortico-hippocampal interactions (Fig. 2). Memory search is hereby prevented, and the ability to learn new categories is impaired. A hippocampal lesion has the effect of causing vigilance to equal zero. Learning that occurs without mismatch-mediated reset and memory search can only form general categories.

9

Relevant data from amnesic individuals have been reported by Knowlton and Squire (1993), who showed that amnesic subjects and normal subjects perform equally well on easy categorization tasks, but amnesic subjects perform far worse on more demanding tasks. To explain these data, Knowlton and Squire (1993) posited two separate memory systems. However, Zaki et al. (2003) quantitatively ﬁt these data with a single exemplar-based model whose sensitivity parameter was chosen lower for amnesic than for normal subjects. This exemplar model is usually expressed in terms of formal algebraic equations. When the formal exemplar model is interpreted as a real-time dynamic process undergoing only locally deﬁned interactions, its operations include a top-down attentional matching process akin to that in ART. A low sensitivity parameter c in this exemplar model (see their Eq. (4)) then plays a role similar to that played by a low vigilance parameter ρ in an ART model (Amis et al., 2014).

7. Invariant object category learning, attention, predictive remapping, and search In order to achieve autonomous learning and memory in a mobile organism, ART models are embedded within larger architectures that can solve several fundamental problems that arise when searching a scene. The ﬁrst architecture of this type is called the ARTSCAN model (Fazl et al., 2009; Grossberg, 2009). ARTSCAN predicts how an observer can learn to recognize objects when seen from multiple perspectives, and thereby to learn view-invariant object categories while scanning a 2D scene with eye movements. As the eyes scan a scene, two successive eye movements may focus on different parts of the same object or on different objects. How does the brain avoid erroneously classifying views of different objects together under unsupervised learning conditions, even before the brain knows what the object is? ARTSCAN proposes how the brain controls eye movements that enable it to learn multiple view-speciﬁc categories of a given object and to associatively link them with view-invariant category representations of that object, and only that object. See Fig. 3 for a macrocircuit of the more comprehensive ARTSCAN Search model.

7.1. Attentional shrouds and Where-to-What modulation of invariant category learning To accomplish view-invariant object category learning during free scanning of a 2D scene, the ARTSCAN model predicts a critical role for spatial attentional signals from the Where cortical stream in modulating invariant category learning in the What cortical stream. Several studies have reported that the distribution of spatial attention can conﬁgure itself to ﬁt an object’s form. Form-ﬁtting spatial attention is sometimes called an attentional shroud (Tyler and Kontsevich, 1995). ARTSCAN explains how an object’s pre-attentively formed surface representation in prestriate cortical area V4 may induce such a formﬁtting attentional shroud in parietal cortex. In particular, feedback between the surface representation and the shroud are predicted to form a surface-shroud resonance that locks spatial attention on the object’s surface. I have predicted that such a

Please cite this article as: Grossberg, S., From brain synapses to systems for learning and memory: Object recognition, spatial navigation, timed conditioning, and movement control. Brain Research (2014), http://dx.doi.org/10.1016/j.brainres.2014.11.018

1089 1090 1091 1092 1093 1094 1095 1096 1097 1098 1099 1100 1101 1102 1103 1104 1105 1106 1107 1108 1109 1110 1111 1112 1113 1114 1115 1116 1117 1118 1119 1120 1121 1122 1123 1124 1125 1126 1127 1128 1129 1130 1131 1132 1133 1134 1135 1136 1137 1138 1139 1140 1141 1142 1143 1144 1145 1146 1147 1148

BRES : 43922

10

1149 1150 1151 1152 1153 1154 1155 1156 1157 1158 1159 1160 1161 1162 1163 1164 1165 1166 1167 1168 1169 1170 1171 1172 1173 1174 1175 1176 1177 1178 1179 1180 1181 1182 1183 1184 1185 1186 1187 1188 1189 1190 1191 1192 1193 1194 1195 1196 1197 1198 1199 1200 1201 1202 1203 1204 1205 1206 1207 1208

brain research ] (]]]]) ]]]–]]]

Fig. 3 – ARTSCAN Search diagram. The dashed boxes indicate the boundary and surface process. (a) Category learning. The arrows represent the excitatory cortical processes from the Where cortical stream to the What cortical stream whereby invariant category learning and recognition, and reinforcement learning, occur. The connections ending in circular disks indicate inhibitory connections. (b) One pathway for controlling top-down primed search for a valued object from the What to the Where cortical stream. The green arrows represent bottom-up image-driven processes and the blue arrows represent topdown processes from the What cortical stream to the Where cortical stream. ITa: anterior part of inferotemporal cortex, ITp: posterior part of inferotemporal cortex, PPC: posterior parietal cortex, LIP: lateral intraparietal cortex, LGN: lateral geniculate nucleus, ORB: orbitofrontal cortex, Amyg: amygdala, BG: basal ganglia, PFC: prefrontal cortex, SC: superior colliculus, V1 and V2: primary and secondary visual areas, V3 and V4: visual areas 3 and 4. [Reprinted with permission from Chang et al. (2014).] Please cite this article as: Grossberg, S., From brain synapses to systems for learning and memory: Object recognition, spatial navigation, timed conditioning, and movement control. Brain Research (2014), http://dx.doi.org/10.1016/j.brainres.2014.11.018

1209 1210 1211 1212 1213 1214 1215 1216 1217 1218 1219 1220 1221 1222 1223 1224 1225 1226 1227 1228 1229 1230 1231 1232 1233 1234 1235 1236 1237 1238 1239 1240 1241 1242 1243 1244 1245 1246 1247 1248 1249 1250 1251 1252 1253 1254 1255 1256 1257 1258 1259 1260 1261 1262 1263 1264 1265 1266 1267 1268

BRES : 43922 brain research ] (]]]]) ]]]–]]]

1269 1270 1271 1272 1273 1274 1275 1276 1277 1278 1279 1280 1281 1282 1283 1284 1285 1286 1287 1288 1289 1290 1291 1292 1293 1294 1295 1296 1297 1298 1299 1300 1301 1302 1303 1304 1305 1306 1307 1308 1309 1310 1311 1312 1313 1314 1315 1316 1317 1318 1319 1320 1321 1322 1323 1324 1325 1326 1327 1328

surface-shroud resonance supports conscious perception of the object’s visible surface qualia. After such a resonance locks attention upon a surface, it can also propagate top-down to lower cortical areas, such as V1 and V2, and bottom-up to higher cortical areas, such as prefrontal cortex, to synchronize them all to support the conscious percept, as discussed more completely in the next section. While this surface-shroud resonance remains active, it also has other important functions: First, it ensures that eye movements tend to end at locations on the object’s surface, thereby enabling different views of the same object to be sequentially explored, as conﬁrmed by Theeuwes et al. (2010). Second, it keeps the emerging view-invariant object category active while different views of the object are learned by viewspeciﬁc categories and associated with it. This latter function works as follows: As each viewspeciﬁc category is learned by the What stream, say in posterior inferotemporal cortex (ITp), it focuses object attention via a learned top-down expectation on the critical features in the visual cortex (e.g., in prestriate cortical area V4) that will be used to recognize that view and its variations in the future. Such a view-speciﬁc category and its learned top-down expectation are modeled by ART. When the ﬁrst such view-speciﬁc category is learned, it also activates a cell population at a higher cortical level, say anterior inferotemporal cortex (ITa), that will become the view-invariant object category (Fig. 3a). Suppose that the eyes or the object move sufﬁciently to expose a new view whose critical features are signiﬁcantly different from the critical features that are used to recognize the ﬁrst view. Then the ﬁrst view category is reset, or inhibited. This happens due to the mismatch of its learned top-down expectation, or prototype of attended critical features, with the newly incoming view information. This topdown prototype focuses object attention on the incoming visual information. Object attention hereby helps to control which view-speciﬁc categories are learned by determining when the currently active view-speciﬁc category should be reset, and a new view-speciﬁc category should be activated. Such a reset is controlled by the usual balance between resonance and reset in any ART model (Section 6.4). However, the view-invariant object category should not be reset every time a view-speciﬁc category is reset, or else it can never become view-invariant. This is what the attentional shroud accomplishes: It inhibits a tonically-active reset signal that would otherwise shut off the view-invariant category when each view-speciﬁc category is reset (Fig. 3a). As this process is iterated, and the eyes foveate a sequence of views on a single object’s surface through time, they trigger learning of a sequence of view-speciﬁc categories, and each of them is associatively linked through learning with the stillactive view-invariant category. When the eyes move off an object, its attentional shroud collapses in the Where stream, thereby transiently disinhibiting the parietal reset mechanism that shuts off the viewinvariant category in the What stream. This shroud-mediated reset mechanism between the Where and What cortical streams is different from the top-down mismatch-mediated reset within the What stream that resets the individual viewspeciﬁc categories.

11

When the eyes look at a different object, its shroud can form in the Where stream and a new view-speciﬁc category can be learned that can, in turn, activate the cells that will become a new view-invariant category in the What stream. Are there data that support these model predictions and that can be used to further test and develop the model? Chiu and Yantis (2009) have described rapid event-related fMRI experiments in humans showing that a spatial attention shift causes a domain-independent transient parietal burst that correlates with a change of categorization rules. This transient parietal signal is a marker against which further experimental tests of model mechanisms can be based. It supports the predicted sequence of V4-parietal surface-shroud collapse (shift of spatial attention), transient parietal burst (reset signal), and collapse of the currently active invariant object category in cortical area ITa (shift of categorization rules). The transient parietal burst is domain-independent because any object’s surface-shroud resonance can inhibit it, and the burst itself can inhibit any viewinvariant object category. These and related results (e.g., Cabeza et al., 2008; Corbetta et al., 2000; Yantis et al., 2002) are consistent with the model prediction of how different regions of the parietal cortex maintain sustained attention to a currently attended object (e.g., the shroud) and control transient attention switching (e.g., the reset burst) to a different object.

7.2. Conscious awareness of visual qualia: Crowding and parietal neglect The ARTSCAN model enables an answer to be proposed to the following fundamental question: What brain event subserves conscious awareness of visual qualia? The background leading to this proposal uniﬁes two streams of neural modeling. ART predicts that “all conscious states are resonant states” and the FACADE theory of 3D vision and ﬁgureground perception (e.g., Grossberg, 1994) predicts that “all consciously visible percepts are surface percepts”. Combining these two predictions raises the question: What sort of resonance supports conscious visible surface percepts? I have predicted that a surface-shroud resonance supports conscious percepts of an object’s visible surface qualia. After such a resonance locks attention upon a surface, it can also propagate top-down to lower visual areas where ﬁner visual features are represented, such as V1 and V2, and bottom-up to regions as high as prefrontal cortex, and can synchronize them all to support the conscious percept. A wide range of data are naturally explained by this prediction, such as data about crowding (Green and Bavelier, 2007; He et al., 1996; Intriligator and Cavanagh, 2001; Levi, 2008) and data about parietal neglect (Driver and Mattingley, 1998; Mesulam, 1999). Crowding probes the relationship between seeing and recognition (and thus learning and memory) by showing how seeing and recognition of individual objects can fail during certain conditions. ARTSCAN provides a deceptively simple explanation of crowding (Foley et al., 2012). It predicts that we consciously see an object only when its surface features are included within a surface-shroud resonance. When crowding occurs, however, such a resonance may not isolate an individual object. This can happen, for example, if the cortical magniﬁcation factor, among other factors, causes multiple, peripherally-viewed, object surfaces to all share a

Please cite this article as: Grossberg, S., From brain synapses to systems for learning and memory: Object recognition, spatial navigation, timed conditioning, and movement control. Brain Research (2014), http://dx.doi.org/10.1016/j.brainres.2014.11.018

1329 1330 1331 1332 1333 1334 1335 1336 1337 1338 1339 1340 1341 1342 1343 1344 1345 1346 1347 1348 1349 1350 1351 1352 1353 1354 1355 1356 1357 1358 1359 1360 1361 1362 1363 1364 1365 1366 1367 1368 1369 1370 1371 1372 1373 1374 1375 1376 1377 1378 1379 1380 1381 1382 1383 1384 1385 1386 1387 1388

BRES : 43922

12

1389 1390 1391 1392 1393 1394 1395 1396 1397 1398 1399 1400 1401 1402 1403 1404 1405 1406 1407 1408 1409 1410 1411 1412 1413 1414 1415 1416 1417 1418 1419 1420 1421 1422 1423 1424 1425 1426 1427 1428 1429 1430 1431 1432 1433 1434 1435 1436 1437 1438 1439 1440 1441 1442 1443 1444 1445 1446 1447 1448

brain research ] (]]]]) ]]]–]]]

single surface-shroud resonance. Since surface-shroud resonances create a link between conscious perception and the categories that we used to recognize objects, objects that share a single surface-shroud resonance cannot be individually recognized. Parietal neglect describes how a parietal lesion can cause a patient to consciously ignore the corresponding region of space. ART proposes that neglect may happen because, despite the fact that the lesion may leave the visual cortex intact, the corresponding surface-shroud resonance cannot form to support a conscious percept of the corresponding object surface.

7.3. Seeing and knowing via two kinds of attention and resonance Models like ARTSCAN illustrate how two different types of resonances may occur that support the difference between seeing and knowing. A surface-shroud resonance coordinates What and Where stream representations of surfaces and spatial attention, respectively. It supports conscious seeing of an object’s surface qualia. The attention that occurs during such a resonance is spatial attention. A feature-category resonance coordinates What stream representations of object features and their recognition categories, as modeled by ART. It supports knowing what the object is. The attention that occurs during such a resonance is object attention. During conscious awareness of an object, both kinds of resonances may be active, and even synchronous, to support seeing the object and knowing what it is.

7.4. Invariant object learning, recognition, predictive remapping, and search The pARTSCAN (positional ARTSCAN) model (Cao et al., 2011) builds upon ARTSCAN to propose how an observer can learn view-, size-, and positionally-invariant object categories in a 2D scene. pARTSCAN suggests a role for persistently ﬁring cells in the inferotemporal cortex (Brunel, 2003; Fuster and Jervey, 1981; Miyashita and Chang, 1988; Tomita et al., 1999) in this process. Such persistence represents a speciﬁc kind of short-term memory that is reset when spatial attention shifts from one object to another. The 3D ARTSCAN model (Grossberg et al., 2014) extends ARTSCAN to propose how the brain maintains the stability of binocularly fused representations of a 3D scene as the eyes ﬁxate different objects in the scene. Each eye movement requires a new binocular match of an object’s features, yet perceptual stability is maintained. This competence requires that predictive remapping occurs (Duhamel et al., 1992; Gottlieb et al., 2005; Mathot and Theeuwes, 2010; Melcher, 2007; Melcher, 2008–2009; Saygin and Sereno, 2008; Sommer and Wurtz, 2006; Tolias et al., 2001; Umeno and Goldberg, 1997) via gain ﬁelds (Andersen et al., 1985; Andersen and Mountcastle, 1983; Deneve and Pouget, 2003; Gancarz and Grossberg, 1999; Grossberg and Kuperstein, 1986; Pouget et al., 2003) to ensure that predictive eye position information enables the attentional shrouds and binocularly matches to be computed in head-centered coordinates that are stable as the eyes move.

The dARTSCAN (distributed ARTSCAN) model (Foley et al., 2012) clariﬁes why visual backgrounds do not become dark when spatial attention, via a surface-shroud resonance, is focused on a particular object, how Where stream motionactivated transient attentional components and What stream sustained attentional components interact (Corbetta et al., 2008; Corbetta and Shulman, 2002; Dosenbach et al., 2007, 2008; Egeth and Yantis, 1997), and how prefrontal priming interacts with parietal attention mechanisms to inﬂuence search efﬁciency. The ARTSCAN Search model (Fig. 3; Chang et al., 2014) builds upon pARTSCAN view-, size-, and positionallyinvariant object category learning and recognition via Where-to-What stream interactions. It can also learn to focus motivated attention upon such an object using a combination of conditioned reinforcer learning and incentive motivational learning during cognitive-emotional interactions (Grossberg, 1975; Grossberg and Seidman, 2006). Finally, it can search a 2D scene for a valued goal object using What-to-Where stream top-down ART expectations and basal ganglia volition (Fig. 3b), hereby proposing a neurobiologically-grounded solution of the Where’s Waldo problem. Combining all of these developments within a 3D ARTSCAN Search model enables learning of view-, size-, and positionally-invariant object categories of objects in a 3D scene, and search of such a scene to detect, attend, recognize, and look at a valued target object in it, without disrupting perceptual stability during the search.

8. Laminar cortical dynamics of vision, speech, and cognition How similar learning mechanisms may govern different functions is exempliﬁed at multiple organizational levels. For example, although all neocortex seems to share many basic anatomical features, notably its characteristic organization into six horizontal layers and sublaminae, their specializations in different cortical regions may support vision, or speech, or cognition. Figs. 4–6 summarize three examples of the LAMINART family of laminar cortical architectures: the 3D LAMINART model (Cao and Grossberg, 2005, 2012; Grossberg and Yazdanbakhsh, 2005) of 3D vision and ﬁgureground separation, the cARTWORD model of speech learning and perception (Grossberg and Kazerounian, 2011), and the LIST PARSE model of cognitive working memory and learned list unitization or chunking (Grossberg and Pearson, 2008). These models illustrate how functional diversity may arise even when the synaptic learning laws and the overall anatomical architecture share many features.

8.1. Laminar computing: A revolutionary new paradigm for intelligent computation These LAMINART models illustrate the computational paradigm of Laminar Computing (Grossberg, 2007, 2012) whose goal is to clarify how specializations of the characteristic bottom-up, top-down, and horizontal interactions in the cerebral cortex embody different types of biological intelligence. Laminar Computing proposes how this can happen, while explaining

Please cite this article as: Grossberg, S., From brain synapses to systems for learning and memory: Object recognition, spatial navigation, timed conditioning, and movement control. Brain Research (2014), http://dx.doi.org/10.1016/j.brainres.2014.11.018

1449 1450 1451 1452 1453 1454 1455 1456 1457 1458 1459 1460 1461 1462 1463 1464 1465 1466 1467 1468 1469 1470 1471 1472 1473 1474 1475 1476 1477 1478 1479 1480 1481 1482 1483 1484 1485 1486 1487 1488 1489 1490 1491 1492 1493 1494 1495 1496 1497 1498 1499 1500 1501 1502 1503 1504 1505 1506 1507 1508

BRES : 43922 brain research ] (]]]]) ]]]–]]]

1509 1510 1511 1512 1513 1514 1515 1516 1517 1518 1519 1520 1521 1522 1523 1524 1525 1526 1527 1528 1529 1530 1531 1532 1533 1534 1535 1536 1537 1538 1539 1540 1541 1542 1543 1544 1545 1546 1547 1548 1549 1550 1551 1552 1553 1554 1555 1556 1557 1558 1559 1560 1561 1562 1563 1564 1565 1566 1567 1568

13

up data-driven processing and top-down attentive hypothesisdriven processing.

8.2. Fast feedforward vs. slower feedback processing of unambiguous vs. ambiguous data When unambiguous information is processed, such as a familiar and well-rendered scene, a LAMINART architecture (Fig. 7) can quickly group the scene in a fast feedforward sweep of activation that passes directly through layer 4 to 2/3 and then on to layers 4 to 2/3 in subsequent cortical areas (Fig. 7c and e). This property clariﬁes how recognition can be so fast in response to unambiguous scenes; e.g., Thorpe et al. (1996). On the other hand, if there are multiple possible groupings in a scene, say in response to a complex textured scene, then competition among these possibilities due to inhibitory interactions in layers 4 and 2/3 (black cells and synapses in Fig. 7) can cause all cell activities to become smaller, because activities in the model’s competitive circuits are self-normalizing, or tend to conserve the total activity of the circuit. This property emerges from the architecture’s shunting on-center off-surround networks, which also enable input contrasts to be processed over a large dynamic range without saturation (Douglas et al., 1995; Grossberg, 1973, 1980; Heeger, 1992).

8.3. Trading certainty against speed: Real-time probabilities that run as fast as they can

Fig. 4 – (a) The 3D LAMINART model circuit diagram. The model consists of a (V1 Interblob)–(V2 Pale Stripe)-V4 boundary stream which computes 3D perceptual groupings, and a (V1 Blob)–(V2 Thin Stripe)-V4 surface stream which computes 3D surface representations of lightness, color, and depth. The two processing streams interact to overcome their complementary computational deﬁciencies (Grossberg, 1994) and create consistent 3D boundary and surface percepts. (b) A block diagram of the 3D LAMINART model. [Reproduced with permission from Cao and Grossberg (2005).]

how the laminar design of neocortex enables it to support (1) self-stabilizing development, learning, and memory that solve the stability–plasticity dilemma, (2) probabilitistic decisions that reconcile the demands of automatic data-driven bottom-up processing and task-selective attentive top-down processing, and (3) analog coherence, or the ability to unify the sensitivity of analog computing and the stability of digital computing. LAMINART hereby embodies the best properties of feedforward and feedback processing, digital and analog processing, and bottom-

Said in another way, these self-normalizing circuits carry out a type of real-time probability theory in which the amplitude of cell activity covaries with the certainty of the network’s selection, or decision, about a grouping. Amplitude covaries with processing speed and, due to the feedback interactions within these circuits, the coherence and synchronization of cell activations. Low activity slows down feedforward processing in the circuit because it takes longer for cell activities to exceed output thresholds and to activate subsequent cells above threshold. Network uncertainty is resolved through feedback (Fig. 7c and e): Active layer 2/3 grouping cells feedback signals to layers 6-then-4-then-2/3 to close a cortical feedback loop that contrast-enhances and ampliﬁes the winning grouping to a degree and at a rate that reﬂect the amount of statistical evidence for that grouping. As the winning grouping emerges, and weaker groupings are suppressed, the activities of the winning cells become more active and synchronous, and thus can resume transmitting this cortical decision to subsequent processing stages. In summary, a LAMINART circuit “runs as fast as it can” in response to the available evidence, trading certainty against speed: It operates in a fast feedforward mode when there is little uncertainty, and automatically engages a slower feedback mode when there is uncertainty. Feedback selects a winning decision that enables the circuit to speed up again, since activation amplitude, synchronization, and processing speed both increase with certainty. Such decision-making properties go beyond the Bayesian statistical approaches to biological intelligence that are so popular today.

Please cite this article as: Grossberg, S., From brain synapses to systems for learning and memory: Object recognition, spatial navigation, timed conditioning, and movement control. Brain Research (2014), http://dx.doi.org/10.1016/j.brainres.2014.11.018

1569 1570 1571 1572 1573 1574 1575 1576 1577 1578 1579 1580 1581 1582 1583 1584 1585 1586 1587 1588 1589 1590 1591 1592 1593 1594 1595 1596 1597 1598 1599 1600 1601 1602 1603 1604 1605 1606 1607 1608 1609 1610 1611 1612 1613 1614 1615 1616 1617 1618 1619 1620 1621 1622 1623 1624 1625 1626 1627 1628

BRES : 43922

14

1629 1630 1631 1632 1633 1634 1635 1636 1637 1638 1639 1640 1641 1642 1643 1644 1645 1646 1647 1648 1649 1650 1651 1652 1653 1654 1655 1656 1657 1658 1659 1660 1661 1662 1663 1664 1665 1666 1667 1668 1669 1670 1671 1672 1673 1674 1675 1676 1677 1678 1679 1680 1681 1682 1683 1684 1685 1686 1687 1688

brain research ] (]]]]) ]]]–]]]

computing and the sensitivity of analog computing. This is true because the feedback loop between layers 2/3-6-4-2/3 that selects or conﬁrms a winning grouping (Fig. 7c and e) has the property of analog coherence (Grossberg, 1999; Grossberg et al., 1997; Grossberg and Raizada, 2000; Yazdanbakhsh and Grossberg, 2004); namely, this feedback loop can synchronously choose and store a winning grouping, while suppressing losing groupings, without losing analog sensitivity to amplitude differences in the input pattern. Such synchronous storage in the feedback loop provides the stability of digital computing – for example, the feedback loop exhibits hysteresis that can preserve the stored pattern against external perturbations – while preserving the sensitivity of analog computation—so that it can weigh the evidence for a grouping and translate evidence into the graded activities of the winning grouping.

9. Spatial navigation: Entorhinal grid cell and hippocampal place cell learning 9.1.

Fig. 5 – The cARTWORD model describes a hierarchy of levels responsible for the processes involved in speech and language perception. Each level is organized into laminar cortical circuits, wherein deep layers (6 and 4) are responsible for processing and storing inputs, and superﬁcial layers (2/3) are proposed to group distributed patterns across these deeper layers into learned unitized representations. The lowest level is responsible for processing acoustic features (cell activities Fi and Ei ) and items (cell activities CiðIÞ ), whereas the higher level is responsible for storing of sequences of acoustic items in working memory (activities Yi and Xi ), and representing these stored sequences of these items as unitized, contextsensitive representations by list chunks (activities CðLÞ J ) in a network, called a masking ﬁeld, that is capable of selectively representing lists of variable length. [Reprinted with permission from Grossberg and Kazerounian (2011).]

8.4. Combining the stability of digital with the sensitivity of analog computing The LAMINART model also embodies a novel kind of hybrid computing that simultaneously realizes the stability of digital

Place cells and spatial navigation

Section 7 noted how an ART category learning circuit could be embedded in larger architectures that can autonomously carry out invariant object category learning as the eyes search a scene. Are similar laws of learning and memory used to accomplish behavioral competences that may seem to a casual observer to be totally unrelated? One afﬁrmative example concerns how spatial navigation is controlled. This example is particularly relevant to classical data about LTP and LTD because these data were recorded in the hippocampus, and the hippocampus and medial entorhinal cortex (MEC) play an important role in the control of spatial learning, memory, and navigation (Davis et al., 1992; Morris et al., 1982; Parron and Save, 2004). Place cells in the hippocampus ﬁre whenever a rat is in a speciﬁc localized region, or “place”, in an environment (O’Keefe and Dostrovsky, 1971). They can also exhibit multiple ﬁring ﬁelds when an animal navigates in large spaces (Fenton et al., 2008; Henriksen et al., 2010; Park et al., 2011). The ensemble of all place cells enables an animal to localize itself in an environment. Research on place cells has clariﬁed that they receive two kinds of inputs: one conveying information about the sensory, notably visual, context experienced from a given place, and the other from a navigational, or path integration, system that tracks relative position in the world by integrating angular and linear velocity self-movement estimates for instantaneous rotation and translation, respectively. An important open problem is to explain how sensory context and path integration information are combined to control navigation; e.g., Chen et al. (2013).

9.2.

Grid cells and path integration

What is the source of path integration inputs to place cells? More generally, how do place cells get learned? Grid cells in the superﬁcial layers of medial entorhinal cortex (MEC) provide inputs to hippocampal place cells, and the primary determinants of their ﬁring are path integration inputs (McNaughton et al., 2006). Grid cells are so called because each of them, unlike a place cell, ﬁres at multiple spatial

Please cite this article as: Grossberg, S., From brain synapses to systems for learning and memory: Object recognition, spatial navigation, timed conditioning, and movement control. Brain Research (2014), http://dx.doi.org/10.1016/j.brainres.2014.11.018

1689 1690 1691 1692 1693 1694 1695 1696 1697 1698 1699 1700 1701 1702 1703 1704 1705 1706 1707 1708 1709 1710 1711 1712 1713 1714 1715 1716 1717 1718 1719 1720 1721 1722 1723 1724 1725 1726 1727 1728 1729 1730 1731 1732 1733 1734 1735 1736 1737 1738 1739 1740 1741 1742 1743 1744 1745 1746 1747 1748

BRES : 43922 brain research ] (]]]]) ]]]–]]]

1749 1750 1751 1752 1753 1754 1755 1756 1757 1758 1759 1760 1761 1762 1763 1764 1765 1766 1767 1768 1769 1770 1771 1772 1773 1774 1775 1776 1777 1778 1779 1780 1781 1782 1783 1784 1785 1786 1787 1788 1789 1790 1791 1792 1793 1794 1795 1796 1797 1798 1799 1800 1801 1802 1803 1804 1805 1806 1807 1808

15

9.3. Both grid and place cells are learned from self-organizing maps that obey the same laws Despite their dramatically different appearance, the receptive ﬁelds of grid cells and place cells can be learned within successive processing stages in a hierarchy of self-organizing maps, wherein each processing stage obeys the same laws, including the normalized instar law of Eq. (5). This GridPlaceMap model has been developed with both rate-based and spiking neurons (Pilly and Grossberg, 2012, 2013); see Fig. 8. The difference in appearance of the cell receptive ﬁelds emerges from the different statistical properties of their input patterns (Fig. 9). The receptive ﬁelds that are learned by the GridPlaceMap model ﬁt neurophysiological data about grid and place cells and their development in juvenile rats (Langston et al., 2010; Wills et al., 2010). Due to the inputs that they receive from multiple scales of developing grid cells, the learned hippocampal place ﬁelds can represent much larger spaces than the grid cells, indeed spaces whose spatial scale may be the least common multiple of grid cell scales (Gorchetchnikov and Grossberg, 2007). These least common multiple scales are large enough to let place cells support navigational behaviors. In addition to both grid cell and place cell properties emerging from the same self-organizing map equations, albeit at different processing levels, at both of these processing levels, the self-organizing maps amplify and learn to categorize the most energetic and frequent co-activations of their inputs. Fig. 6 – Circuit diagram of the LIST PARSE model. An Item and Order working memory is realized by a recurrent shunting on-center off-surround network in layers 4 and 6 of the Cognitive Working Memory, which is assumed to occur in ventrolateral prefrontal cortex. The list chunks are learned in layer 2/3. Outputs from the Cognitive Working Memory to the Motor Working Memory interact with a Vector Integration To Endpoint (VITE) trajectory generator (Bullock and Grossberg, 1988; see Section 9), modulated by the basal ganglia, to perform sequences of variable length at variable speeds. Solid arrows indicate ﬁxed excitatory connections. Solid lines with hemi-disks indicate learned connections. Dashed arrows indicate ﬁxed inhibitory connections. Only 1-item chunks (C) and their feedback connections (M) within a single Cognitive Working Memory channel are shown, whereas the model uses chunks of various sizes in layer 2/3 and feedback from layer 2/3 to layer 5/6 of the cognitive working memory is broadly distributed. Also, only the excitatory projections from Cognitive Working Memory to the Motor Plan Field ðY-FÞ are shown. [Reprinted with permission from Grossberg and Pearson (2008).]

positions that form a regular hexagonal grid during navigation in an open ﬁeld (Hafting et al., 2005). Grid cells also exhibit a gradient of spatial scales along the dorsoventral axis of the MEC, with larger receptive ﬁelds towards the ventral end, and anatomically neighboring cells sharing similar grid spacings and orientations but having different spatial phases that are not topographically organized.

9.4. Top-down attention stabilizes grid and place cell memory In the GridPlaceMap model, place cells are spatial category cells that are activated by multiple scales of entrohinal grid cells in a self-organizing map. However, when Grossberg (1976a, 1978) introduced the modern laws for competitive learning and self-organizing maps, he proved that they can learn well in response to sparse input environments, indeed with Bayesian properties, but that they exhibit catastrophic forgetting in response to dense non-stationary environments. Grossberg (1976b) introduced ART as an enhanced model capable of dynamically stabilizing its category learning in response to arbitrary environments. If, in fact, grid and place cell learning occur in selforganizing maps, then, as in all self-organizing map models, one expects that grid cell and place cell learning are dynamically stabilized by ART top-down attentive matching mechanisms (Section 6). Anatomical and neurophysiological data from several labs support this hypothesis. The anatomy of the hippocampal system supports the possibility that such attentive feedback exists, since feedback pathways exist from the hippocampal CA1 region to the entorhinal cortex. Neurophysiological data also support the predicted role of attention in hippocampal learning. For example, Kentros et al. (2004) showed that “conditions that maximize place ﬁeld stability greatly increase orientation to novel cues. This suggests that storage and retrieval of place cells is modulated by a top-down cognitive process resembling attention and that place cells are neural correlates of spatial memory” (p. 283). It has similarly been proposed that

Please cite this article as: Grossberg, S., From brain synapses to systems for learning and memory: Object recognition, spatial navigation, timed conditioning, and movement control. Brain Research (2014), http://dx.doi.org/10.1016/j.brainres.2014.11.018

1809 1810 1811 1812 1813 1814 1815 1816 1817 1818 1819 1820 1821 1822 1823 1824 1825 1826 1827 1828 1829 1830 1831 1832 1833 1834 1835 1836 1837 1838 1839 1840 1841 1842 1843 1844 1845 1846 1847 1848 1849 1850 1851 1852 1853 1854 1855 1856 1857 1858 1859 1860 1861 1862 1863 1864 1865 1866 1867 1868

BRES : 43922

16

1869 1870 1871 1872 1873 1874 1875 1876 1877 1878 1879 1880 1881 1882 1883 1884 1885 1886 1887 1888 1889 1890 1891 1892 1893 1894 1895 1896 1897 1898 1899 1900 1901 1902 1903 1904 1905 1906 1907 1908 1909 1910 1911 1912 1913 1914 1915 1916 1917 1918 1919 1920 1921 1922 1923 1924 1925 1926 1927 1928

brain research ] (]]]]) ]]]–]]]

of properties that are predicted to arise from specializations of homologous circuits. The ﬁrst homology concerns the fact that path integration inputs to the model are sensitive to both angular velocity and linear velocity during movement. Angular velocity is processed by a well-known type of cell, a head-direction cell (Ranck, 1984; Taube et al., 1990). The GridPlaceMap model and its precursor (Mhatre et al., 2012) predicted the existence of a cell that is sensitive to linear velocity as well. It is called a stripe cell. Both head direction cells and stripe cells are predicted to exist within similar types of ring attractor circuits (e.g., Song and Wang, 2005), where they receive angular and linear velocity signals and process them into a form that inputs to model stripe cells. See Fig. 9 for a simulation of stripe cell responses. Recent experiments have reported cell properties that are consistent with those predicted by stripe cells (Krupic et al., 2012; Sargolini et al., 2006).

9.6. Neural relativity: Spectral timing and spectral spacing in the hippocampus

Fig. 7 – The LAMINART model clariﬁes how bottom-up, horizontal, and top-down interactions within and across cortical layers in V1 and V2 interblob and pale stripe regions, respectively, carry out bottom-up adaptive ﬁltering, horizontal grouping, and top-down attention. Similar interactions seem to occur in all six-layered cortices. See text for details. [Reprinted with permission from Raizada and Grossberg (2001).]

learning of place cell receptive ﬁelds reﬂects an “automatic recording of attended experience” (Morris and Frey, 1997, p. 1489). These experiments clarify that cognitive processes like attention play a role in hippocampal learning and memory stability, just as they do in learning object categories, and interact with NMDA receptors to mediate longlasting hippocampal place ﬁeld memory in novel environments (Kentros et al., 1998). Thus, the learning of grid cells and place cells in the medial entorhinal cortex and hippocampus may be viewed as part of a specialized ART system for learning spatial representations as an animal or human navigates its environment, just as the learning of invariant object categories in the inferotemporal cortex may be viewed as part of a specialized ART system for learning object representations as an animal or human scans its environment.

9.5. Unifying angular and linear acceleration, grid and place, and space and time The In addition to showing how both grid cells and place cells can be learned using the same equations, including learning laws, at different levels of a model hierarchy, the GridPlaceMap model also illustrates two other parsimonious and elegant sets

The second homology concerns the fact that the entorhinal– hippocampal system processes information about both space and time: In addition to supporting spatial navigation with grid and place cells, it can also support adaptively timed conditioning (Smith, 1968; Thompson et al., 1987). Why, from a mechanistic viewpoint, are both spatial and temporal representations processed in the same part of the brain? The fact of this convergence is consistent with data and hypotheses about a possible role of hippocampus in episodic learning and memory, since episodic memories typically combine both spatial and temporal information about particular autobiographical events; e.g., Eichenbaum and Lipton (2008). As noted in Section 8.3, multiple scales of entorhinal grid cells can cooperate in a self-organizing map to form place cell receptive ﬁelds. These multiple grid cell scales form along a dorsoventral spatial gradient in the entorhinal cortex such that grid cells have increasingly large spatial scales (that is, larger spatial intervals between activations in a hexagonal grid) in the ventral direction. Grid cells with several different spatial scales along the dorsoventral gradient can cooperate to learn place cells that can represent spaces much larger than those represented by individual grid cells (Grossberg and Pilly, 2012). In fact, place cells that are learned from these grid cells are capable of representing the lowest common multiple of the grid cell scales that activate them (Gorchetchnikov and Grossberg, 2007; Pilly and Grossberg, 2012). These place cell spatial scales are large enough to be useful in spatial navigation. In this way, a “spectrum” of small grid cell spatial scales can be combined in place cells to represent much larger and behaviorally relevant spaces. This homology has led to the name Spectral Spacing for the mechanism whereby grid cells give rise to place cells. With this background, we can begin to see a similarity in how the entorhinal–hippocampal system deals with both space and time. In the case of spatial representation, the issue is how to generate spatial representations on behaviorally relevant scales. The cooperation of multiple, but smaller, scales of entorhinal grid cells to learn hippocampal place cells accomplishes this. In the case of temporal representation, a

Please cite this article as: Grossberg, S., From brain synapses to systems for learning and memory: Object recognition, spatial navigation, timed conditioning, and movement control. Brain Research (2014), http://dx.doi.org/10.1016/j.brainres.2014.11.018

1929 1930 1931 1932 1933 1934 1935 1936 1937 1938 1939 1940 1941 1942 1943 1944 1945 1946 1947 1948 1949 1950 1951 1952 1953 1954 1955 1956 1957 1958 1959 1960 1961 1962 1963 1964 1965 1966 1967 1968 1969 1970 1971 1972 1973 1974 1975 1976 1977 1978 1979 1980 1981 1982 1983 1984 1985 1986 1987 1988

BRES : 43922

17

brain research ] (]]]]) ]]]–]]]

1989 1990 1991 1992 1993 1994 1995 1996 1997 1998 1999 2000 2001 2002 2003 2004 2005 2006 2007 2008 2009 2010 2011 2012 2013 2014 2015 2016 2017 2018 2019 2020 2021 2022 2023 2024 2025 2026 2027 2028 2029 2030 2031 2032 2033 2034 2035 2036 2037 2038 2039 2040 2041 2042 2043 2044 2045 2046 2047 2048

-

• • •

-

SOM

-

-

• • •

wjsk

SOM

Pk

-

-

-

• • •

+

• • •

• • •

-

-

Gjs

wdpsj

-

+

Sdps

v(t)

. Φ(t)

Ventral

Dorsal

Fig. 8 – The GridPlaceMap self-organizing map hierarchy of grid and place cell activation and learning: Stripe cells in either the parasubiculum (PaS) or the deeper layers of medial entorhinal cortex (MEC), self-organizing grid cells in layer II of MEC, and self-organizing place cells in hippocampal area CA3 learn to represent position in increasingly large spaces based on internally generated signals corresponding to translational and rotational movements during navigation. [Reprinted with permission from Pilly and Grossberg (2012).]

spectrum of small time scales can be combined to represent longer and behaviorally relevant durations of cell activation, as occur, for example, during trace conditioning (Moyer et al., 1990; Solomon et al., 1986). A model that accomplishes this is called the Spectral Timing model (Grossberg and Merrill, 1992, 1996; Grossberg and Schmajuk, 1989). The Spectral Timing model shows how large temporal intervals can be bridged by a spectrum of cells, or cell sites, with different reaction rates can learn to match the statistical distribution of expected delays in reinforcement. Although each of these cells, or cell sites, reacts relatively brieﬂy at different times, their population response as a whole can bridge a much longer time interval, ranging from hundreds of milliseconds to seconds, that can be tuned by learning to match temporal experimental contingencies. Spectrally-timed learning has Weber law properties such that larger inter-stimulus intervals between unconditioned and conditioned stimuli lead to learned response curves with broader variances, a property that is also called “scalar timing” (Gibbon, 1991; Roberts, 1981; Roberts et al., 1989; Smith, 1968). Hippocampal “time cells” with all the properties required to achieve spectral timing, including the Weber law, have been reported (MacDonald et al., 2011); in particular, “… the mean peak ﬁring rate for each time cell occurred at sequential moments, and the overlap among ﬁring periods from even these small ensembles of time cells bridges the

entire delay. Notably, the spread of the ﬁring period for each neuron increased with the peak ﬁring time ….” It remains to be shown whether the spectrum of time cells arises from a gradient in a single rate parameter, as is predicted by the Spectral Timing model. How Spectral Timing can bridge long temporal intervals using such a gradient has been modeled in terms of calcium dynamics in the metabotropic glutamate receptor (mGluR) system within the cerebellum (Fiala et al., 1996). The most parsimonious prediction is that a similar mechanism holds in all cases of spectral timing throughout the brain, including cerebellum, hippocampus, and basal ganglia (Brown et al., 1999; Fiala et al., 1996; Grossberg and Merrill, 1992, 1996; Grossberg and Schmajuk, 1989).

9.7.

Lateral stream for time and medial stream for space

The Spectral Timing model reﬂects the part of entorhinal– hippocampal dynamics that is devoted to representing objects and events, notably adaptively timed durations of these events, and includes lateral entorhinal cortex. The Spectral Spacing model reﬂects a complementary part of entorhinal– hippocampal dynamics that is devoted to representing spatial representations, and includes medial entrorhinal cortex. Both of these processing streams are joined in the hippocampus to support spatial navigation as well as episodic learning and memory (Eichenbaum and Lipton, 2008).

Please cite this article as: Grossberg, S., From brain synapses to systems for learning and memory: Object recognition, spatial navigation, timed conditioning, and movement control. Brain Research (2014), http://dx.doi.org/10.1016/j.brainres.2014.11.018

2049 2050 2051 2052 2053 2054 2055 2056 2057 2058 2059 2060 2061 2062 2063 2064 2065 2066 2067 2068 2069 2070 2071 2072 2073 2074 2075 2076 2077 2078 2079 2080 2081 2082 2083 2084 2085 2086 2087 2088 2089 2090 2091 2092 2093 2094 2095 2096 2097 2098 2099 2100 2101 2102 2103 2104 2105 2106 2107 2108

BRES : 43922

18

UNSMOOTHED RATE MAP

SMOOTHED RATE MAP

GRID CELL

STRIPE CELL

SPIKES ON TRAJECTORY

PLACE CELL

2109 2110 2111 2112 2113 2114 2115 2116 2117 2118 2119 2120 2121 2122 2123 2124 2125 2126 2127 2128 2129 2130 2131 2132 2133 2134 2135 2136 2137 2138 2139 2140 2141 2142 2143 2144 2145 2146 2147 2148 2149 2150 2151 2152 2153 2154 2155 2156 2157 2158 2159 2160 2161 2162 2163 2164 2165 2166 2167 2168

brain research ] (]]]]) ]]]–]]]

Fig. 9 – Spatial responses of representative (a) stripe cells, (b) grid cells, and (c) place cells. The ﬁrst column shows the spike locations (red dots) of the cells superimposed on the trajectory of the navigating model animal, or animat, during a trial. The second and third columns show the unsmoothed and smoothed spatial rate maps, respectively, of the cells. See Pilly and Grossberg (2013) for how spike recordings are converted into rate maps. [Reprinted with permission from Pilly and Grossberg (2013).] (For interpretation of the references to color in this ﬁgure legend, the reader is referred to the web version of this article.)

This heuristic homology between spatial and temporal representations in these parallel streams is supported by rigorous mathematical modeling and data simulations. Grossberg and Pilly (2012) developed the Spectral Spacing model to show that neural mechanisms thatenable a dorsoventral gradient of grid cell spatial scales to be learned are formally the same as mechanisms which enable a gradient of temporal scales to form in the Spectral Timing model. This claim was supported by quantitative simulations of challenging data about parametric neurophysiological properties of

grid cells along the dorsoventral gradient that could not be explained without this assumption. Thus, it may be that space and time are both in the hippocampus because they exploit shared computational mechanisms. The phrase “neural relativity” has been introduced to summarize this predicted property of the entorhinal–hippocampal system. Given the prediction and simulations of mGluR involvement in Spectral Timing, it is natural to ask if mGLuR plays a central role in controlling the dorsoventral gradient in Spectral Spacing as well.

Please cite this article as: Grossberg, S., From brain synapses to systems for learning and memory: Object recognition, spatial navigation, timed conditioning, and movement control. Brain Research (2014), http://dx.doi.org/10.1016/j.brainres.2014.11.018

2169 2170 2171 2172 2173 2174 2175 2176 2177 2178 2179 2180 2181 2182 2183 2184 2185 2186 2187 2188 2189 2190 2191 2192 2193 2194 2195 2196 2197 2198 2199 2200 2201 2202 2203 2204 2205 2206 2207 2208 2209 2210 2211 2212 2213 2214 2215 2216 2217 2218 2219 2220 2221 2222 2223 2224 2225 2226 2227 2228

BRES : 43922 brain research ] (]]]]) ]]]–]]]

2229 2230 2231 2232 2233 2234 2235 2236 2237 2238 2239 2240 2241 2242 2243 2244 2245 2246 2247 2248 2249 2250 2251 2252 2253 2254 2255 2256 2257 2258 2259 2260 2261 2262 2263 2264 2265 2266 2267 2268 2269 2270 2271 2272 2273 2274 2275 2276 2277 2278 2279 2280 2281 2282 2283 2284 2285 2286 2287 2288

19

10. Movement control: Inhibitory matching and mismatch learning 10.1. Complementary computing for object recognition and movement control ART-based learning of object representations in the What stream uses excitatory matching and match-based learning. Where stream learning for the control of movement, in contrast, often uses computationally complementary processes of inhibitory matching and mismatch-based learning (Fig. 1). Correspondingly, the What stream learns object representations that strive to become positionally-invariant, whereas the Where stream represents the positions and actions that enable recognized objects to be manipulated. Excitatory matching and match-based learning can solve the stability–plasticity dilemma. In contrast, inhibitory matching and mismatch-based learning can continually update the spatial maps and sensory-motor gains that are used to move as our bodily parameters change through time (Bullock and Grossberg, 1988; Bullock et al., 1998; Gaudiano and Grossberg, 1991; Georgopoulos et al., 1982, 1986). Indeed, we would be illserved by spatial and motor learning processes that solve the stability–plasticity dilemma, since we do not want the spatial representations and motor gains that were suitable for controlling our infant bodies to be remembered as we grow up and used to control our adult bodies. In this sense, catastrophic forgetting is a good property during spatial and motor learning.

10.2. vector

Target position, present position, and difference

As an example of inhibitory matching, consider how we make an arm movement. To make such a movement, a representation of where the arm is now (its present position vector) is subtracted from a representation of where we want the arm to move (its target position vector), thereby computing a difference vector that represents the direction and distance of movement needed to attain the target. After moving to the target, the target and present positions agree, so the difference vector is zero. In other words, this sort of matching is inhibitory (Bullock and Grossberg, 1988). The difference vector is volitionally gated, or multiplied, by a basal ganglia GO signal that determines when and how fast the movement will occur (Bullock and Grossberg, 1988; Bullock et al., 1998). Such a volitionally-gated difference vector computation is, for example, used in the Vector Integration To Endpoint, or VITE, model, and its variants, for computing arm movement trajectories (Bullock et al., 1998, 1993; Bullock and Grossberg, 1988). Let T denote the target position vector, P the present position vector, D the difference vector, and G the volitional GO signal (Fig. 10). Then: dD ¼ að D þ T PÞ dt

ð9Þ

and dP ¼ G½Dþ ; dt

ð10Þ

Fig. 10 – Vector Integration To Endpoint circuit (Bullock and Grossberg, 1988) for control of movement trajectories. T is the target position vector, P the outﬂow present position vector, D the difference vector, and G the volitional GO signal that multiplies, or gates, D. See text for details.

where ½Dþ ¼ maxðD; 0Þ. Eq. (9) says that D computes a timeaverage of the difference (T–P) between the target and present position vectors, and (10) says that the present position vector P integrates the difference vector D at a rate proportional to the GO signal G until P equals T, whence the movement stops.

10.3.

Difference vectors as error signals for learning

During motor learning, a difference vector can also generate error signals when the same target position and present position are encoded but not properly calibrated. These error signals activate a form of mismatch learning that eliminates the mismatch through time by recalibrating system maps and gains to be consistent. The following pair of equations illustrate how this can happen by replacing Eq. (9) by dD ¼ að D þ TZ PÞ dt

ð11Þ

and dZ ¼ bf ðTÞð cZdDÞ: dt

ð12Þ

In this learning process, the adaptive weight Z in Eq. (12) changes in response to a non-zero difference vector D at times when the target position T and present position P represent the same position, but are not properly calibrated. The weight Z increases if D is negative, and decreases if D is positive. Learning continues until the adaptive weight Z in Eq. (12), by acting as an adaptive gain that multiplies the target position T in Eq. (11), equals zero at the appropriate times.

Please cite this article as: Grossberg, S., From brain synapses to systems for learning and memory: Object recognition, spatial navigation, timed conditioning, and movement control. Brain Research (2014), http://dx.doi.org/10.1016/j.brainres.2014.11.018

2289 2290 2291 2292 2293 2294 2295 2296 2297 2298 2299 2300 2301 2302 2303 2304 2305 2306 2307 2308 2309 2310 2311 2312 2313 2314 2315 2316 2317 2318 2319 2320 2321 2322 2323 2324 2325 2326 2327 2328 2329 2330 2331 2332 2333 2334 2335 2336 2337 2338 2339 2340 2341 2342 2343 2344 2345 2346 2347 2348

BRES : 43922

20

2349 2350 2351 2352 2353 2354 2355 2356 2357 2358 2359 2360 2361 2362 2363 2364 2365 2366 2367 2368 2369 2370 2371 2372 2373 2374 2375 2376 2377 2378 2379 2380 2381 2382 2383 2384 2385 2386 2387 2388 2389 2390 2391 2392 2393 2394 2395 2396 2397 2398 2399 2400 2401 2402 2403 2404 2405 2406 2407 2408

brain research ] (]]]]) ]]]–]]]

Neural models explain and predict data about how mismatch learning may tune spatial representations and adaptive motor gains in basal ganglia, cerebellum, motor cortex, parietal cortex, and prefrontal cortex (Brown et al., 1999, 2004; Fiala et al., 1996; Grossberg and Paine, 2000; Guenther, 1995; Guenther et al., 1994). Models that carry out computation and learning by difference vectors are often called Adaptive Vector Integration To Endpoint (aVITE) or Vector Associative Map (VAM) models (Gaudiano and Grossberg, 1991, 1992).

10.4. Joining ART and VAM: Self-stabilizing expertise in evolving bodies In summary, perceptual/cognitive processes often use ART-like excitatory matching and match-based learning to create selfstabilizing representations of objects and events that enable us to gain increasing expertise as we learn about the world. Complementary spatial/motor processes often use VAM-like inhibitory matching and mismatch-based learning to continually update spatial maps and sensory-motor gains to compensate for bodily changes throughout life. Together (Fig. 1) these complementary predictive and learning mechanisms create a selfstabilizing perceptual/cognitive front end for intelligently manipulating the more labile spatial/motor processes which enable our changing bodies to act effectively upon a dynamic world.

Q2

Uncited references Gee et al. (2008), Grossberg (1969b), Hillyard et al. (1998), Ploran et al. (2007), Sears et al. (1994).

references

Abbott, L.F., Varela, K., Sen, K., Nelson, S.B., 1997. Synaptic depression and cortical gain control. Science 275, 220–223. Amis, G., Carpenter, G.A., Ersoy, B., Grossberg, S., 2014. Cortical Learning of Recognition Categories: Towards Resolving the Q6 Exemplar vs. Prototype Debate. Submitted for publication. Andersen, R.A., Essick, G.K., Siegel, R.M., 1985. Encoding of spatial location by posterior parietal neurons. Science 230, 456–458. Andersen, R.A., Mountcastle, V.B., 1983. The influence of the angle of gaze upon the excitability of the light-sensitive neurons of the posterior parietal cortex. J. Neurosci. 3, 532–548. Artola, A., Singer, W., 1987. Long-term potentiation and NMDA receptors in rat visual cortex. Nature 330, 649–652. Artola, A., Singer, W., 1993. Long-term depression of excitatory synaptic transmission and its relationship to long-term potentiation. Trends Neurosci. 16, 480–487. Bar, M., Tootell, R.B.H., Schacter, D.L., Greve, D.N., Fischl, B., Mendola, J.D., Rosen, B.R., Dale, A.M., 2001. Cortical mechanisms specific to explicit object recognition. Neuron 29, 529–535. Ben-Shahar, O., Zucker, S., 2004. Sensitivity to curvatures in orientation-based texture segmentation. Vis. Res. 44, 257–277. Berke, J.D., Hetrick, V., Breck, J., Green, R.W., 2008. Transient 23- to 30-Hz oscillations in mouse hippocampus during exploration of novel environments. Hippocampus 18, 519–529. Bhatt, R., Carpenter, G., Grossberg, S., 2007. Texture segregation by visual cortex: perceptual grouping, attention, and learning. Vis. Res. 47, 3173–3211.

Bliss, T.V.P., Lomo, T., 1973. Long-lasting potentiation of synaptic transmission in the dentate area of the anaesthetized rabbit following stimulation of the perforant path. J. Physiol. (London) 232, 331–356. Bliss, T.V.P., Gardner-Medwin, A.R., 1973. Long-lasting potentiation of synaptic transmission in the dentate area of the unanaesthetized rabbit following stimulation of the perforant path. J. Physiol. (London) 232, 357–374. Brown, J., Bullock, D., Grossberg, S., 1999. How the basal ganglia use parallel excitatory and inhibitory learning pathways to selectively respond to unexpected rewarding cues. J. Neurosci. 19, 10502–10511. Brown, J.W., Bullock, D., Grossberg, S., 2004. How laminar frontal cortex and basal ganglia circuits interact to control planned and reactive saccades. Neural Networks 17, 471–510. Brunel, N., 2003. Dynamics and plasticity of stimulus selective persistent activity in cortical network models. Cerebral Cortex 13, 1151–1161. Buffalo, E.A., Fries, P., Landman, R., Buschman, T.J., Desimone, R., Q7 2011. Laminar differences in gamma and alpha coherence in the ventral stream. Proc. Natl. Acad. Sci. U.S.A. 108, 11262–11267. Bullock, D., Cisek, P., Grossberg, S., 1998. Cortical networks for control of voluntary arm movements under variable force conditions. Cerebral Cortex 8, 48–62. Bullock, D., Grossberg, S., 1988. Neural dynamics of planned arm movements: emergent invariants and speed-accuracy properties during trajectory formation. Psychol. Rev. 95, 49–90. Bullock, D., Grossberg, S., Guenther, F.H., 1993. A self-organizing neural model of motor equivalent reaching and tool use by a multijoint arm. J. Cogn. Neurosci. 5, 408–435. Buschman, T.J., Miller, E.K., 2009. Serial, covert shifts of attention during visual search are reflected by the frontal eye fields and correlated with population oscillations. Neuron 63, 386–396. Cabelli, R.J., Hohn, A., Shatz, C.J., 1995. Inhibition of ocular dominance column formation by infusion of NT-4/5 or BDNF. Science 267, 1662–1666. Cabelli, R.J., Shelton, D.L., Segal, R.A., Shatz, C.J., 1997. Blockade of endogenous ligands of trkB inhibits formation of ocular dominance columns. Neuron 19, 63–76. Cabeza, R., Ciaramelli, E., Olson, I.R., Moscovitch, M., 2008. The parietal cortex and episodic memory: an attentional account. Nat. Rev. Neurosci. 9, 613–625. Cao, Y., Grossberg, S., 2005. A laminar cortical model of stereopsis and 3D surface perception: closure and da Vinci stereopsis. Spat. Vis. 18, 515–578. Cao, Y., Grossberg, S., 2012. Stereopsis and 3D surface perception by spiking neurons in laminar cortical circuits: a method of converting neural rate models into spiking models. Neural Networks 26, 75–98. Cao, Y., Grossberg, S., Markowitz, J., 2011. How does the brain rapidly learn and reorganize view- and positionally-invariant object representations in inferior temporal cortex?. Neural Networks 24, 1050–1061. Carpenter, G.A., 1997. Distributed learning, recognition, and prediction by ART and ARTMAP neural networks. Neural Networks 10, 1473–1494. Carpenter, G.A., 2003. Default ARTMAP. In: Proceedings of the International Joint Conference on Neural Networks (IJCNN’03), pp. 1396–1401. Carpenter, G.A., Grossberg, S., 1987. A massively parallel architecture for a self-organizing neural pattern recognition machine. Comput. Vis. Graphics Image Process. 37, 54–115. Carpenter, G.A., Grossberg, S., 1991. Pattern Recognition by SelfOrganizing Neural Networks. MIT Press, Cambridge, MA. Carpenter, G.A., Grossberg, S., 1993. Normal and amnesic learning, recognition, and memory by a neural model of

Please cite this article as: Grossberg, S., From brain synapses to systems for learning and memory: Object recognition, spatial navigation, timed conditioning, and movement control. Brain Research (2014), http://dx.doi.org/10.1016/j.brainres.2014.11.018

2409 2410 2411 2412 2413 2414 2415 2416 2417 2418 2419 2420 2421 2422 2423 2424 2425 2426 2427 2428 2429 2430 2431 2432 2433 2434 2435 2436 2437 2438 2439 2440 2441 2442 2443 2444 2445 2446 2447 2448 2449 2450 2451 2452 2453 2454 2455 2456 2457 2458 2459 2460 2461 2462 2463 2464 2465 2466 2467 2468

BRES : 43922 brain research ] (]]]]) ]]]–]]]

2469 2470 2471 2472 2473 2474 2475 2476 2477 2478 2479 2480 2481 2482 2483 2484 2485 2486 2487 2488 2489 2490 2491 2492 2493 2494 2495 2496 2497 2498 2499 2500 2501 2502 2503 2504 2505 2506 2507 2508 2509 2510 2511 2512 2513 2514 2515 2516 2517 2518 2519 2520 2521 2522 2523 2524 2525 2526 2527 2528

cortico-hippocampal interactions. Trends Neurosci. 16, 131–137. Carpenter, G.A., Grossberg, S., Markuzon, N., Reynolds, J.H., Rosen, D.B., 1992. Fuzzy ARTMAP: a neural network architecture for incremental supervised learning of analog multidimensional maps. IEEE Trans. Neural Networks 3, 698–713. Carpenter, G.A., Grossberg, S., Reynolds, J.H., 1991. ARTMAP: supervised real-time learning and classification of nonstationary data by a self-organizing neural network. Neural Networks 4, 565–588. Carpenter, G.A., Grossberg, S., Rosen, D.B., 1991. Fuzzy ART: fast stable learning and categorization of analog patterns by an adaptive resonance system. Neural Networks 4, 759–771. Chang, H.-C., Grossberg, S., Cao, Y., 2014. Where’s Waldo? How perceptual cognitive, and emotional brain processes cooperate during learning to categorize and find desired objects in a cluttered scene. Front. Integr. Neurosci.http: //dxdoi.org/10.3389/fnint.2014.0043 (http://journal.frontiersin. org/Journal/10.3389/fnint.2014.00043/full). Chen, C., King, J.A., Burgess, N., O’Keefe, J., 2013. How vision and movement combine in the hippocampal place code. Proc. Natl. Acad. Sci. U.S.A. 110, 378–383. Chiu, Y.C., Yantis, S., 2009. A domain-independent source of cognitive control for task sets: shifting spatial attention and switching categorization rules. J. Neurosci. 29, 3930–3938. Church, B.A., Krauss, M.S., Lopata, C., Toomey, J.A., Thomeer, M.L., Coutinho, M.V., Volker, M.A., Mercado, E. (2010). Atypical Q8 categorization in children with high-functioning autism spectrum disorder. Psychon. Bull. Rev., 17, 862–868. Cohen, N.J., Squire, L.R., 1980. Preserved learning and retention of a pattern-analyzing skill in amnesia: dissociation of knowing how and knowing that. Science 210, 207–210. Corbetta, M., Kincade, J.M., Ollinger, J.M., McAvoy, M.P., Shulman, G.L., 2000. Voluntary orienting is dissociated from target detection in human posterior parietal cortex. Nat. Neurosci. 3, 292–297. Corbetta, M., Patel, G., Shulman, G.G., 2008. The reorienting system of the human brain: from environment to theory of mind. Neuron 58, 306–324. Corbetta, M., Shulman, G.L., 2002. Control of goal-directed and stimulus-driven attention in the brain. Nat. Rev. Neurosci. 3, 201–215. Davis, S., Butcher, S.P., Morris, R.G., 1992. The NMDA receptor antagonist D-2-amino-5-phosphonopentanoate (D-AP5) impairs spatial learning and LTP in vivo at intracerebral concentrations comparable to those that block LTP in vivo. J. Neurosci. 12, 21–34. Deneve, S., Pouget, A., 2003. Basis functions for object-centered representations. Neuron 37, 347–359. Desimone, R., 1998. Visual attention mediated by biased competition in extrastriate visual cortex. Philos. Trans. R. Soc. London, Ser. B 353, 1245–1255. Dosenbach, N.U., Fair, D.A., Cohen, A.L., Schlaggar, B.L., Petersen, S.E., 2008. A dual-networks architecture of top-down control. Trends Cogn. Sci. 12, 99–105. Dosenbach, N.U.F., Fair, D.A., Miezin, F.M., Cohen, A.L., Wenger, K.K., Dosenbach, R.A.T., Fox, M.D., Snyder, A.Z., Vincent, J.L., Raichle, M.E., Schlagger, B.L., Petersen, S.E., 2007. Distinct brain networks for adaptive and stable task control in humans. Proc. Natl. Acad. Sci. U.S.A. 104, 11073–11078. Douglas, R.J., Koch, C., Mahowald, M., Martin, K.A.C., Suarez, H.H., 1995. Recurrent excitation in neocortical circuits. Science 269, 981–985. Dranias, M., Grossberg, S., Bullock, D., 2008. Dopaminergic and non-dopaminergic value systems in conditioning and outcome-specific revaluation. Brain Res. 1238, 239–287.

21

Driver, J., Mattingley, J.B., 1998. Parietal neglect and visual awareness. Nat. Neurosci. 1, 17–22. Dudek, S.M., Bear, M.F., 1992. Homosynaptic long-term depression in area VA1 of hippocampus and effects of N-methyl-D-aspartate receptor blockade. Proc. Natl. Acad. Sci. U.S.A. 89, 4363–4367. Duhamel, J.R., Colby, C.L., Goldberg, M.E., 1992. The updating of the representation of visual space in parietal cortex by intended eye movements. Science 255, 90–92. Eckhorn, R., Bauer, R., Jordan, W., Brosch, M., Kruse, W., Munk, M., Reitbock, H.J., 1988. Coherent oscillations: a mechanism of feature linking in the visual cortex? Biol. Cybern. 60, 121–130. Egeth, H.E., Yantis, S., 1997. Visual attention: control, representation, and time course. Annu. Rev. Psychol. 48, 269–297. Engel, A.K., Fries, P., Singer, W., 2001. Dynamics predictions: oscillations and synchrony in top-down processing. Nat. Rev. Neurosci. 2, 704–716. Fazl, A., Grossberg, S., Mingolla, E., 2009. View-invariant object category learning, recognition, and search: how spatial and object attention are coordinated using surface-based attentional shrouds. Cogn. Psychol. 58, 1–48. Fenton, A.A., Kao, H.-Y., Neymotin, S.A., Olypher, A., Vayntrub, Y., Lyton, W.W., Ludvig, N., 2008. Unmasking the CA1 ensemble place code by exposures to small and large environments: more place cells and multiple, irregularly arranged, and expanded place fields in the larger space. J. Neurosci. 28, 11250–11262. Fiala, J.C., Grossberg, S., Bullock, D., 1996. Metabotropic glutamate receptor activation in cerebellar Purkinje cells as substrate for adaptive timing of the classically conditioned eye blink response. J. Neurosci. 16, 3760–3774. Foley, N.C., Grossberg, S., Mingolla, E., 2012. Neural dynamics of object-based multifocal visual spatial attention and priming: object cueing, useful-field-of-view, and crowding. Cogn. Psychol. 65, 77–117. Fregnac, Y., Burke, J.P., Smith, D., Friedlander, M.J., 1994. Temporal covariance of pre- and postsynaptic activity regulates functional connectivity in the visual cortex. J. Neurophysiol. 71, 1403–1421. Fuster, J.M., Jervey, J.P., 1981. Inferotemporal neurons distinguish and retain behaviorally relevant features of visual stimuli. Science 212, 952–955. Gancarz, G., Grossberg, S., 1999. A neural model of saccadic eye movement control explains task-specific adaptation. Vis. Res. 39, 3123–3143. Gaudiano, P., Grossberg, S., 1991. Vector associative maps: unsupervised real-time error-based learning and control of movement trajectories. Neural Networks 4, 147–183. Gaudiano, P., Grossberg, S., 1992. Adaptive vector integration to endpoint: self-organizing neural circuits for control of planned movement trajectories. Hum. Mov. Sci. 11, 141–155. Gee, A.L., Ipata, A.E., Gottlieb, J., Bisley, J.W., Goldberg, M.E., 2008. Neural enhancement and pre-emptive perception: the genesis of attention and the attentional maintenance of the cortical salience map. Perception 37, 389–400. Georgopoulos, A.P., Kalaska, J.F., Caminiti, R., Massey, J.T., 1982. On the relations between the direction of two-dimensional arm movements and cell discharge in primate motor cortex. J. Neurosci. 2, 1527–1537. Georgopoulos, A.P., Schwartz, A.B., Kettner, R.E., 1986. Neuronal population coding of movement direction. Science 233, 1416–1419. Gibbon, J., 1991. The origins of scalar timing. Learn. Motiv. 22, 3–38. Gorchetchnikov, A., Grossberg, S., 2007. Space, time and learning in the hippocampus: how fine spatial and temporal scales are

Please cite this article as: Grossberg, S., From brain synapses to systems for learning and memory: Object recognition, spatial navigation, timed conditioning, and movement control. Brain Research (2014), http://dx.doi.org/10.1016/j.brainres.2014.11.018

2529 2530 2531 2532 2533 2534 2535 2536 2537 2538 2539 2540 2541 2542 2543 2544 2545 2546 2547 2548 2549 2550 2551 2552 2553 2554 2555 2556 2557 2558 2559 2560 2561 2562 2563 2564 2565 2566 2567 2568 2569 2570 2571 2572 2573 2574 2575 2576 2577 2578 2579 2580 2581 2582 2583 2584 2585 2586 2587 2588

BRES : 43922

22

2589 2590 2591 2592 2593 2594 2595 2596 2597 2598 2599 2600 2601 2602 2603 2604 2605 2606 2607 2608 2609 2610 2611 2612 2613 2614 2615 2616 2617 2618 2619 2620 2621 2622 2623 2624 2625 2626 2627 2628 2629 2630 2631 2632 2633 2634 2635 2636 2637 2638 2639 2640 2641 2642 2643 2644 2645 2646 2647 2648

brain research ] (]]]]) ]]]–]]]

expanded into population codes for behavioral control. Neural Networks 20, 182–193. Gottlieb, J., Kusunoki, M., Goldberg, M.E., 2005. Simultaneous representation of saccade targets and visual onsets in monkey lateral intraparietal area. Cerebral Cortex 15, 1198–1206. Gray, C.M., Singer, W., 1989. Stimulus-specific neuronal oscillations in orientation columns of cat visual cortex. Proc. Natl. Acad. Sci. U.S.A. 86, 1698–1702. Green, C.S., Bavelier, D., 2007. Action-video-game experience alters the spatial resolution of vision. Psychol. Sci. 18, 88–94. Grossberg, S., 1968a. Global ratio limit theorems for some nonlinear functional differential equations, II. Bull. Am. Math. Soc. 74, 101–105. Grossberg, S., 1968b. Some nonlinear networks capable of learning a spatial pattern of arbitrary complexity. Proc. Natl. Acad. Sci. U.S.A. 59, 368–372. Grossberg, S., 1968c. Some physiological and biochemical consequences of psychological postulates. Proc. Natl. Acad. Sci. U.S.A. 60, 758–765. Grossberg, S., 1969a. On learning and energy-entropy dependence in recurrent and nonrecurrent signed networks. J. Stat. Phys. 1, 319–350. Grossberg, S., 1969b. On the production and release of chemical transmitters and related topics in cellular control. J. Theor. Biol. 22, 325–364. Grossberg, S., 1969c. On the serial learning of lists. Math. Biosci. 4, 201–253. Grossberg, S., 1969d. Some networks that can learn, remember, and reproduce any number of complicated space-time patterns, I. J. Math. Mech. 19, 53–91. Grossberg, S., 1972. A neural theory of punishment and avoidance, II: Quantitative theory. Math. Biosci. 15, 253–285. Grossberg, S., 1973. Contour enhancement, short-term memory, and constancies in reverberating neural networks. Stud. Appl. Math. 52, 213–257. Grossberg, S., 1975. A neural model of attention, reinforcement, and discrimination learning. Int. Rev. Neurobiol. 18, 263–327. Grossberg, S., 1976a. Adaptive pattern classification and universal recoding, I: Parallel development and coding of neural feature detectors. Biol. Cybern. 23, 121–134. Grossberg, S., 1976b. Adaptive pattern classification and universal recoding, II: Feedback, expectation, olfaction, and illusions. Biol. Cybern. 23, 187–202. Grossberg, S., 1978. A theory of human memory: self-organization and performance of sensory-motor codes, maps, and plans. In: Rosen, R., Snell, F. (Eds.), Progress in Theoretical Biology, vol. 5. Academic Press, New York, NY, pp. 233–374. Grossberg, S., 1980. How does a brain build a cognitive code?. Psychol. Rev. 87, 1–51. Grossberg, S., 1994. 3D vision and figure-ground separation by visual cortex. Percept. Psychophys. 55, 48–120. Grossberg, S., 1999. How does the cerebral cortex work? Learning, attention and grouping by the laminar circuits of visual cortex. Spat. Vis. 12, 163–186. Grossberg, S., 2000. The complementary brain: unifying brain dynamics and modularity. Trends Cogn. Sci. 4, 233–246. Grossberg, S., 2007. Consciousness CLEARS the mind. Neural Networks 20, 1040–1053. Grossberg, S., 2009. Cortical and subcortical predictive dynamics and learning during perception, cognition, emotion and action. Philos. Trans. R. Soc. London, Ser. B 364, 1223–1234. Grossberg, S., 2012. Adaptive Resonance Theory: how a brain learns to consciously attend, learn, and recognize a changing world. Neural Networks 37, 1–47. Grossberg, S., Grunewald, A., 1997. Cortical synchronization and perceptual framing. J. Cogn. Neurosci. 9, 117–132. Grossberg, S., Kazerounian, S., 2011. Laminar cortical dynamics of conscious speech perception: a neural model of phonemic

restoration using subsequent context in noise. J. Acoust. Soc. Am. 130, 440–460. Grossberg, S., Kuperstein, M., 1986. Neural Dynamics of Adaptive Sensory-motor Control: Ballistic Eye Movements. Amsterdam, New York North-Holland. Grossberg, S., Markowitz, J., Cao, Y., 2011. On the road to invariant recognition: explaining tradeoff and morph properties of cells in inferotemporal cortex using multiple-scale task-sensitive attentive learning. Neural Networks 24, 1036–1049. Grossberg, S., Merrill, J.W.L., 1992. A neural network model of adaptively timed reinforcement learning and hippocampal dynamics. Cogn. Brain Res. 1, 3–38. Grossberg, S., Merrill, J.W.L., 1996. The hippocampus and cerebellum in adaptively timed learning, recognition, and movement. J. Cogn. Neurosci. 8, 257–277. Grossberg, S., Mingolla, E., Ross, W.D., 1997. Visual brain and visual perception: how does the cortex do perceptual grouping? Trends Neurosci 20, 106–111. Grossberg, S., Paine, R.W., 2000. A neural model of corticocerebellar interactions during attentive imitation and predictive learning of sequential handwriting movements. Neural Networks 13, 999–1046. Grossberg, S., Pearson, L., 2008. Laminar cortical dynamics of cognitive and motor working memory, sequence learning and performance: toward a unified theory of how the cerebral cortex works. Psychol. Rev. 115, 677–732. Grossberg, S., Pepe, J., 1971. Spiking threshold and overarousal effects in serial learning. J. Stat. Phys. 3, 95–125. Grossberg, S., Pilly, P.K., 2012. How entorhinal grid cells may learn multiple spatial scales from a dorsoventral gradient of cell response rates in a self-organizing map. PLoS Comput. Biol. 8 (10), 31002648, http://dx.doi.org/10.1371/journal.pcbi.1002648. Grossberg, S., Pilly, P.K., 2014. Coordinated learning of grid cell and place cell spatial and temporal properties: multiple scales, attention, and oscillations. Philos. Trans. R. Soc. London, Ser. B 369, 20120524. Grossberg, S., Raizada, R., 2000. Contrast-sensitive perceptual grouping and object-based attention in the laminar circuits of primary visual cortex. Vis. Res. 40, 1413–1432. Grossberg, S., Schmajuk, N.A., 1989. Neural dynamics of adaptive timing and temporal discrimination during associative learning. Neural Networks 2, 79–102. Grossberg, S., Seidman, D., 2006. Neural dynamics of autistic behaviors: cognitive, emotional, and timing substrates. Psychol. Rev. 113, 483–525. Grossberg, S., Seitz, A., 2003. Laminar development of receptive fields, maps, and columns in visual cortex: the coordinating role of the subplate. Cerebral Cortex 13, 852–863. Grossberg, S., Somers, D., 1991. Synchronized oscillations during cooperative feature linking in a cortical model of visual perception. Neural Networks 4, 453–466. Grossberg, S., Versace, M., 2008. Spikes, synchrony, and attentive learning by laminar thalamocortical circuits. Brain Res. 1218, 278–312. Grossberg, S., Vladusich, T., 2011. How do children learn to follow gaze, share joint attention, imitate their teachers, and use tools during social interactions? Neural Networks 23, 940–965 Grossberg, S., Williamson, J.R., 2001. A neural model of how horizontal and interlaminar connections of visual cortex develop into adult circuits that carry out perceptual groupings and learning. Cerebral Cortex 11, 37–58. Grossberg, S., Yazdanbakhsh, A., 2005. Laminar cortical dynamics of 3D surface perception: stratification, transparency, and neon color spreading. Vis. Res. 45, 1725–1743. Guenther, F.H., 1995. Speech sound acquisition, coarticulation, and rate effects in a neural network model of speech production. Psychol. Rev. 102, 594–621.

Please cite this article as: Grossberg, S., From brain synapses to systems for learning and memory: Object recognition, spatial navigation, timed conditioning, and movement control. Brain Research (2014), http://dx.doi.org/10.1016/j.brainres.2014.11.018

2649 2650 2651 2652 2653 2654 2655 2656 2657 2658 2659 2660 2661 2662 2663 2664 2665 2666 2667 2668 2669 2670 2671 2672 2673 2674 2675 2676 2677 2678 2679 2680 2681 2682 2683 2684 2685 2686 2687 2688 2689 2690 2691 2692 2693 2694 2695 2696 2697 2698 2699 2700 2701 2702 2703 2704 2705 2706 2707 2708

BRES : 43922 brain research ] (]]]]) ]]]–]]]

2709 2710 2711 2712 2713 2714 2715 2716 2717 2718 2719 2720 2721 2722 2723 2724 2725 2726 2727 2728 2729 2730 2731 2732 2733 2734 2735 2736 2737 2738 2739 2740 2741 2742 2743 2744 2745 2746 2747 2748 2749 2750 2751 2752 2753 2754 2755 2756 2757 2758 2759 2760 2761 2762 2763 2764 2765 2766 2767 2768

Guenther, F.H., Bullock, D., Greve, D., Grossberg, S., 1994. Neural representations for sensory-motor control, III: Learning a body-centered representation of 3-D target position. J. Cogn. Neurosci. 6, 341–358. Hafting, T., Fyhn, M., Molden, S., Moser, M.B., Moser, E.I., 2005. Microstructure of a spatial map in the entorhinal cortex. Nature 436, 801–806. Hahnloser, R.H.R., Kozhevnikov, A.A., Fee, M.S., 2002. An ultrasparse code underlies the generation of neural sequences in a songbird. Nature 419, 65–70. He, S., Cavanagh, P., Intriligator, J., 1996. Attentional resolution and the locus of visual awareness. Nature 383, 334–337. Hebb, D.O., 1949. The Organization of Behavior. Wiley, New York, NY. Hecht-Nielsen, R., 1987. Counterpropagation networks. Appl. Opt. 26, 4979–4983. Heeger, D.J., 1992. Normalization of cell responses in cat striate cortex. Visual Neurosci. 9, 181–197. Henriksen, E.J., Colgin, L.L., Barnes, C.A., Witter, M.P., Moser, M.B., Moser, E.L., 2010. Spatial representation along the proximodistal axis of CA1. Neuron 68, 127–137. Hillyard, S.A., Vogel, E.K., Luck, S.J., 1998. Sensory gain control (amplification) as a mechanism of selective attention: electrophysiological and neuroimaging evidence. Philos. Trans. R. Soc. London, Ser. B 353, 1257–1270. Intriligator, J., Cavanagh, P., 2001. The spatial resolution of visual attention. Cogn. Psychol. 43, 171–216. James, W., 1890. The Principles of Psychology. Harvard University Press, Cambridge, MA. Kamin, L.J., 1969. Predictability, surprise, attention and conditioning. In: Campbell, B.A., Church, R.M. (Eds.), Punishment and Aversive Behavior. Appleton-Century-Crofts, New York, NY. Kentros, C.G., Agniotri, N.T., Streater, S., Hawkins, R.D., Kandel, E. R., 2004. Increased attention to spatial context increases both place field stability and spatial memory. Neuron 42, 283–295. Kentros, C., Hargreaves, E., Hawkins, R.D., Kandel, E.R., Shapiro, M., Muller, R.V., 1998. Abolition of long-term stability of new hippocampal place cell maps by NMDA receptor blockade. Science 280, 2121–2126. Kirkwood, A., Lee, H.-K., Bear, M.F., 1995. Co-regulation of longterm potentiation and experience-dependent synaptic plasticity in visual cortex by age and experience. Nature 375, 328–331. Knowlton, B.J., Squire, L.R., 1993. The learning of categories: parallel brain systems for item memory and category knowledge. Science 262, 1747–1749. Kohonen, T., 1984. Self-organization and Associative Memory. Springer-Verlag, New York, NY. Komatsu, Y., Toyama, K., Maeda, J., Sakaguchi, H., 1981. Longterm potentiation investigated in a slice preparation of striate cortex of young kittens. Neurosci. Lett. 26, 269–274. Kraus, N., McGee, T., Littman, T., Nicol, T., King, C., 1994. Nonprimary auditory thalamic representation of acoustic change. J. Neurophysiol. 72, 1270–1277. Krupic, J., Burgess, N., O’Keefe, J., 2012. Neural representations of location composed of spatially periodic bands. Science 337, 853–857. Langston, R.F., Ainge, J.A., Couey, J.J., Canto, C.B., Bjerknes, T.L., Witter, M.P., Moser, E.I., Moser, M.B., 2010. Development of the spatial representation system in the rat. Science 328, 1576–1580. Lashley, K.S., 1950. In search of the engram. In: Society for Experimental Biology, Symposium 4. Physiological Mechanisms in Animal Behavior. Cambridge University Press, Cambridge, England2–31.

23

Lashley, K.S., 1960. In search of the engram. In: F.A., Beach, Hebb, D.O. (Eds.), The Neuropsychology of Lashley. McGraw-Hill, New York, NY, pp. 345–360. Levi, D.M., 2008. Crowding—an essential bottleneck for object recognition: a mini-review. Vis. Res. 48, 635–654. Levy, W.B., Brassel, S.E., Moore, S.D., 1983. Partial quantification of the associative synaptic learning rule of the dentate gyrus. Neuroscience 8, 799–808. Levy, W.B., Desmond, N.L., 1985. The rules of elemental synaptic plasticity. In: Levy, W.B., Anderson, J., Lehmkuhle, S. (Eds.), Synaptic Modification, Neuron Selectivity and Nervous System Organization. Erlbaum, Hillsdale, NJ, pp. 105–121In: Levy, W.B., Anderson, J., Lehmkuhle, S. (Eds.), Synaptic Modification, Neuron Selectivity and Nervous System Organization. Erlbaum, Hillsdale, NJ, pp. 105–121. Levy, W.B., Steward, O., 1983. Temporal contiguity requirements for long-term associative potentiation/depression in the hippocampus. Neuroscience 8, 791–797. MacDonald, C.J., Lepage, K.Q., Eden, U.T., Eichenbaum, H., 2011. Hippocampal “time cells” bridge the gap in memory for discontiguous events. Neuron 71, 737–749. Mathot, S., Theeuwes, J., 2010. Evidence for the predictive remapping of visual attention. Exp. Brain Res. 200, 117–122. McNaughton, B.L., Battaglia, F.P., Jensen, O., Moser, E.I., Moser, M.B., 2006. Path integration and the neural basis of the ‘cognitive map’. Nat. Rev. Neurosci. 7, 663–678. Melcher, D., 2007. Predictive remapping of visual features precedes saccadic eye movements. Nat. Neurosci. 10, 903–907. Melcher, D., 2008. Dynamic, object-based remapping of visual features in trans-saccadic perception. J. Vis. 8, 1–17. Melcher, D., 2009. Selective attention and the active remapping of object features in trans- saccadic perception. Vis. Res. 49, 1249–1255. Mesulam, M.-M., 1999. Spatial attention and neglect: parietal, frontal and cingulate contributions to the mental representation and attentional targeting of salient extrapersonal events. Philos. Trans. R. Soc. London, Ser. B 354, 1325–1346. Mhatre, H., Gorchetchnikov, A., Grossberg, S., 2012. Grid cell hexagonal patterns formed by fast self-organized learning within entorhinal cortex. Hippocampus 22, 320–334. Miyashita, Y., Chang, H.S., 1988. Neuronal correlate of pictorial short-term memory in the primate temporal cortex. Nature 331, 68–70. Morris, R.G.M., Frey, U., 1997. Hippocampal synaptic plasticity: role in spatial learning or the automatic recording of attended experience? Philos Trans. R. Soc. London, Ser. B 1360, 1469–1503. Morris, R.G.M., Garrud, P., Rawlins, J.N.P., O’Keefe, J., 1982. Place navigation impaired in rats with hippocampal lesions. Nature 297, 681–683. Moyer, J.R., Deyo, R.A., Disterhoft, J.F., 1990. Hippocampectomy disrupts trace eye-blink conditioning in rabbits. Behav. Neurosci. 104, 243–252. Mulkey, R.M., Malenda, R.C., 1992. Mechanisms underlying induction of homosynaptic long-term depression in area CA1 of the hippocampus. Neuron 9, 967–975. Palma, J., Grossberg, S., Versace, M., 2012. Persistence and storage of activity patterns in spiking recurrent cortical networks: modulation of sigmoid signals by after-hyperpolarization currents and acetylcholine. Front. Comput. Neurosci. 6, 42 (Doi: 10.3389.fncom.2012.00042). Palma, J., Versace, M., Grossberg, S., 2012. After-hyperpolarization currents and acetylcholine control sigmoid transfer functions in a spiking cortical model. J. Comput. Neurosci. 32, 253–280. Park, E.H., Dvorak, D., Fenton, A.A., 2011. Ensemble place codes in hippocampus: CA1, CA3, and dentate gyrus place cells have

Please cite this article as: Grossberg, S., From brain synapses to systems for learning and memory: Object recognition, spatial navigation, timed conditioning, and movement control. Brain Research (2014), http://dx.doi.org/10.1016/j.brainres.2014.11.018

2769 2770 2771 2772 2773 2774 2775 2776 2777 2778 2779 2780 2781 2782 2783 2784 2785 2786 2787 2788 2789 2790 2791 2792 2793 2794 2795 2796 2797 2798 2799 2800 2801 2802 2803 2804 2805 2806 2807 2808 2809 2810 2811 2812 2813 2814 2815 2816 2817 2818 2819 2820 2821 2822 2823 2824 2825 2826 2827 2828

BRES : 43922

24

2829 2830 2831 2832 2833 2834 2835 2836 2837 2838 2839 2840 2841 2842 2843 2844 2845 2846 2847 2848 2849 2850 2851 2852 2853 2854 2855 2856 2857 2858 2859 2860 2861 2862 2863 2864 2865 2866 2867 2868 2869 2870 2871 2872 2873 2874 2875 2876 2877 2878 2879 2880 2881 2882 2883 2884 2885 2886 2887 2888

brain research ] (]]]]) ]]]–]]]

multiple place fields in large environments. PLoS One 6, e22349. Parron, C., Save, E., 2004. Evidence for entorhinal and parietal cortices involvement in path integration in the rat. Exp. Brain Res. 159, 349–359. Pavlov, I.P., 1927. Conditioned Reflexes. Oxford University Press. Perry, E.K., Lee, M.L.W., Martin-Ruiz, C.M., Court, J.A., Volsen, S.G., Merrit, J., Folly, E., Iversen, P.E., Bauman, M.L., Perry, R.H., Wenk, G.L., 2001. Cholinergic activity in autism: abnormalities in the cerebral cortex and basal forebrain. Am. J. Psychiatry 158, 1058–1066. Pilly, P.K., Grossberg, S., 2012. How do spatial learning and memory occur in the brain? Coordinated learning of entorhinal grid cells and hippocampal place cells. J. Cogn. Neurosci. 24, 1031–1054. Pilly, P.K., Grossberg, S., 2013. Spiking neurons in a hierarchical self-organizing map model can learn to develop spatial and temporal properties of entorhinal grid cells and hippocampal place cells. PLoS One, http://dxdoi.org/10.1371/journal. pone.0060599. Ploran, E.J., Nelson, S.M., Velanova, K., Donaldson, D.I., Petersen, S.E., Wheeler, M.E., 2007. Evidence accumulation and the moment of recognition: dissociating perceptual recognition processes using fMRI. J. Neurosci. 27, 11912–11924. Pouget, A., Dayan, P., Zemel, R.S., 2003. Inference and computation with population codes. Annu. Rev. Neurosci. 26, 381–410. Purves, D., 1988. Body and Brain: A Trophic Theory of Neural Connections. Harvard University Press, Cambridge, MA. Raizada, R., Grossberg, S., 2001. Context-sensitive bindings by the laminar circuits of V1 and V2: a unified model of perceptual grouping, attention, and orientation contrast. Visual Cognit. 8, 431–466. Ranck Jr, J.B., 1984. Head-direction cells in the deep cell layers of dorsal presubiculum in freely moving rats. In: Proceedings of the Annual Conference of the Society for Neuroscience. Anaheim, CA. vol. 10, p. 599. Reynolds, J.H., Heeger, D.J., 2009. The normalization model of attention. Neuron 61, 168–185. Roberts, S., 1981. Isolation of an internal clock. J. Exp. Psychol. Anim. Behav. Processes 7, 242–268. Roberts, W.A., Cheng, K., Cohen, J.S., 1989. Timing light and tone signals in pigeons. J. Exp. Psychol. Anim. Behav. Processes 15, 23–25. Royer, S., Pare, D., 2003. Conservation of total synaptic weight through balanced synaptic depression and potentiation. Nature 422, 518–522. Rubenstein, J.L.R., Merzenich, M.M., 2003. Model of autism: increased ratio of excitation/inhibition in key neural systems. Genes Brain Behav. 2, 255–267. Sargolini, F., Fyhn, M., Hafting, T., McNaughton, B.L., Witter, M.P., Moser, M.-B., Moser, E.L., 2006. Conjunctive representation of position, direction, and velocity in entorhinal cortex. Science 312, 758–762. Saygin, A.P., Sereno, M.I., 2008. Retinotopy and attention in human occipital, temporal, parietal, and frontal cortex. Cerebral Cortex 18, 2158–2168. Sears, L.L., Finn, P.R., Steinmetz, J.E., 1994. Abnormal classical eyeblink conditioning in autism. J. Autism Dev. Disord. 24, 737–751. Sigala, N., Logothetis, N.K., 2002. Visual categorization shapes feature selectivity in the primate temporal cortex. Nature 415, 318–320. Singer, W., 1983. Neuronal activity as a shaping factor in the selforganization of neuron assemblies. In: Basar, E., Flohr, H., Haken, H., Mandell, A.J. (Eds.), Synergetics of the Brain. Springer-Verlag, New York, NY, pp. 89–101(1983). Smith, M.C., 1968. CS–US interval and US intensity in classical conditioning of the rabbit’s nictitating membrane response. J. Comp. Physiol. Psychol. 3, 679–687.

Solomon, P.R., Vander Schaaf, E.R., Thompson, R.F., 1986. Hippocampus and trace conditioning of the rabbit’s classically conditioned nictitating membrane response. Behav. Neurosci. 100, 729–744. Sommer, M.A., Wurtz, R.H., 2006. Influence of the thalamus on spatial vision processing in frontal cortex. Nature 444, 374–377. Song, P., Wang, X.J., 2005. Angular path integration by moving “hill of activity”: a spiking neuron model without recurrent excitation of the head-direction system. J. Neurosci. 25, 1002–1014. Tanaka, K., Saito, H., Fukada, Y., Moriya, M., 1991. Coding visual images of objects in the inferotemporal cortex of the macaque monkey. J. Neurophysiol. 66, 170–189. Taube, J.S., Muller, R.U., Ranck Jr, J.B., 1990. Head-direction cells recorded from the postsubiculum in freely moving rats. I. Description and quantitative analysis. J. Neurosci. 10, 420–435. Theeuwes, J., Mathot, S., Kingstone, A., 2010. Object-based eye movements: the eyes prefer to stay within the same object. Atten. Percept. Psychophys. 72, 597–601. Thompson, R.F., 1976. The search for the engram. Am. Psychol. 31, 209–227. Thompson, R.F., Clark, G.A., Donegan, N.H., Lavond, G.A., Lincoln, D.G., Maddon, J., Mamounas, L.A., Mauk, M.D., McCormick, D.A., 1987. Neuronal substrates of discrete, defensive conditioned reflexes, conditioned fear states, and their interactions in the rabbit. In: Gormenzano, I., Prokasy, W.F., Thompson, R.F. (Eds.), Classical Conditioning third ed Erlbaum Associates, Hillsdale, NJ, pp. 371–399. Thorpe, S., Fize, D., Marlot, C., 1996. Speed of processing in the human visual system. Nature 381, 520–522. Tolias, A.S., Moore, T., Smirnakis, S.M., Tehovnik, E.J., Siapas, A.G., Schiller, P.H., 2001. Eye movements modulate visual receptive fields of V4 neurons. Neuron 29, 757–767. Tomita, H., Ohbayashi, M., Nakahara, K., Hasegawa, I., Miyashita, Y., 1999. Top-down signal from prefrontal cortex in executive control of memory retrieval. Nature 401, 699–703. Tsodyks, M., Pawelzik, K., Markram, H., 1998. Neural networks with dynamic synapses. Neural Comput. 10, 821–835. Tyler, C.W., Kontsevich, L.L., 1995. Mechanisms of stereoscopic processing: stereoattention and surface perception in depth reconstruction. Perception 24, 127–153. Umeno, M.M., Goldberg, M.E., 1997. Spatial processing in the monkey frontal eye fields, I: Predictive visual responses. J. Neurophysiol. 78, 1373–1383. van Der Werf, Y.D., Witter, M.P., Groenewegen, H.J., 2002. The intralaminar and midline nuclei of the thalamus. Anatomical and functional evidence for participation in processes of arousal and awareness. Brain Res. 39, 107–140. Vladusich, T., Lafe, F., Kim, D.-S., Tager-Flusberg, H., Grossberg, S., 2010. Prototypical category learning in high-functioning autism. Autism Res. 3, 226–236. Wills, T.J., Cacucci, F., Burgess, N., O’Keefe, J., 2010. Development of the hippocampal cognitive map in preweanling rats. Science 328, 1573–1576. Yantis, S., Schwarzbach, J., Serences, J.T., Carlson, R.L., Steinmetz, M.A., Pekar, J.J., Courtney, S.M., 2002. Transient neural activity in human parietal cortex during spatial attention shifts. Nat. Neurosci. 5, 995–1002. Yazdanbakhsh, A., Grossberg, S., 2004. Fast synchronization of perceptual grouping in laminar visual cortical circuits. Neural Networks 17, 707–718. Zaki, S.R., Nosofsky, R.M., Jessup, N.M., Unversagt, F.W., 2003. Categorization and recognition performance of a memory impaired group: evidence for single-system models. J. Int. Neuropsychol. Soc. 9, 394–406. Zoccolan, D., Kouh, M., Poggio, T., DiCarlo, J.J., 2007. Trade-off between object selectivity and tolerance in monkey inferotemporal cortex. J. Neurosci. 27, 12292–12307.

Please cite this article as: Grossberg, S., From brain synapses to systems for learning and memory: Object recognition, spatial navigation, timed conditioning, and movement control. Brain Research (2014), http://dx.doi.org/10.1016/j.brainres.2014.11.018

2889 2890 2891 2892 2893 2894 2895 2896 2897 2898 2899 2900 2901 2902 2903 2904 2905 2906 2907 2908 2909 2910 2911 2912 2913 2914 2915 2916 2917 2918 2919 2920 2921 2922 2923 2924 2925 2926 2927 2928 2929 2930 2931 2932 2933 2934 2935 2936 2937 2938 2939 2940 2941 2942 2943 2944 2945 2946 2947 2948

From brain synapses to systems for learning and memory: Object recognition, spatial navigation, timed conditioning, and movement control

From brain synapses to systems for learning and memory: Object recognition, spatial navigation, timed conditioning, and movement control

Recommend Documents