Physiology and pathophysiology of cortico-basal ganglia–thalamocortical loops: Theoretical and practical aspects

Progress in Neuro-Psychopharmacology & Biological Psychiatry 26 (2002) 771 – 804

Review

Physiology and pathophysiology of cortico-basal ganglia–thalamocortical loops: Theoretical and practical aspects Konstantin V. Baeva,*, Karl A. Greenea,1, Frederick F. Marcianoa, Johan E.S. Samantab, Andrew G. Shettera, Kris A. Smitha, Mark A. Stacyb, Robert F. Spetzlera a

Division of Neurological Surgery, Barrow Neurological Institute, St. Joseph’s Hospital and Medical Center, 350 West Thomas Road, Phoenix, AZ 85013, USA b Division of Neurology, Barrow Neurological Institute, St. Joseph’s Hospital and Medical Center, 350 West Thomas Road, Phoenix, AZ 85013, USA

Abstract A new theoretical framework is used to analyze functions and pathophysiological processes of cortico-basal ganglia – thalamocortical loops and to demonstrate the hierarchical relationships between various loops. All hierarchical levels are built according to the same functional principle: Each loop is a neural optimal control system (NOCS) and includes a model of object behavior and an error distribution system. The latter includes dopaminergic neurons and is necessary to tune the model to a controlled object (CO). The regularities of pathophysiological processes in NOCSs are analyzed. Mechanisms of current functional neurosurgical procedures like lesioning and deep brain stimulation (DBS) of various basal ganglia structures and neurotransplantation are described based on proposed theoretical ideas. Parkinson’s disease (PD) is used to exemplify clinical applications of the proposed theory. Within the proposed theoretical framework, PD must be considered as a disease of the error distribution system. The proposed theoretical views have broad fundamental and clinical applications. D 2002 Elsevier Science Inc. All rights reserved. Keywords: Neural optimal control system; Cortico-basal ganglia – thalamocortical loops; Prefrontal loops; Limbic loop; Basal ganglia; Dopamine learning; Parkinson’s disease; Neurotransplantation; Deep brain stimulation; Functional neurosurgical procedures

1. Introduction In 1984, Graham Hoyle, a famous American neuroethologist, wrote the following: Unfortunately, in spite of an explosion of research activity in neuroscience in the 34 years since the Cambridge meeting, there has been little advance in its conceptual

underpinnings. The single general framework that has ever existed, the McCulloch – Pitts (1943) binomial model of neural function, had to be abandoned when intracellular recording revealed the widespread occurrence and importance of analog information processing and signaling. But the vacuum left behind has yet to be filled with even a tentative new model. Neuroscience came to be the art of the doable, with expediency ruling

Abbreviations: AD, anterodorsal thalamic nucleus; AV, anteroventral thalamic nucleus; CM, centromedian nucleus; CMAd, caudal cingulate motor area on the dorsal bank; CMAr, rostral cingulate motor area; CMAv, caudal cingulate motor area on the ventral bank; CO, controlled object; COMT, catechol-o-methyl transferase; CPG, central pattern generator; CS, conditioned stimulus; DBS, deep brain stimulation; DSCT, dorsal spino-cerebellar tract; GPe, external segment of glodus pallidus; GPi, internal segment of globus pallidus; LD, laterodorsal thalamic nucleus; DOPA, dihydroxyphenylalanine; MAO, monoamine oxidase; MC, motor cortex; MD, mediodorsal thalamic nucleus; MPTP, 1-methyl-4-phenyl-1,2,5,6-tetrahydropyridine; MRI, magnetic resonance imaging; NOCS, neural optimal control system; PD, Parkinson’s disease; PM, premotor area; SMA, supplementary motor area; SNc, substantia nigra pars compacta; SNr, substantia nigra pars reticulata; SOCT, spino-olivo-cerebellar tract; SRCT, spino-reticulo-cerebellar tract; STN, subthalamic nucleus; US, unconditioned stimulus; VAmc, nucleus ventralis anterior pars magnocellularis; VApc, nucleus ventralis anterior pars parvocellularis; Vc, ventralis caudalis nucleus; Vim, ventralis intermediate nucleus; VLo, nucleus ventralis lateralis pars oralis; VM, ventromedial thalamic nucleus; Voa, ventral oral anterior nucleus; Vop, ventral oral posterior nucleus; VSCT, ventral spino-cerebellar tract * Corresponding author. Tel.: +1-602-406-3624; fax: +1-602-406-4172. E-mail address: [email protected] (K.V. Baev). 1 Current address: NeuroSpine Center of Wisconsin, Appleton, WI, USA. 0278-5846/02/$ – see front matter D 2002 Elsevier Science Inc. All rights reserved. PII: S 0 2 7 8 - 5 8 4 6 ( 0 2 ) 0 0 2 0 1 - 4

772

K.V. Baev et al. / Progress in Neuro-Psychopharmacology & Biological Psychiatry 26 (2002) 771–804

the day, rather than a soundly based intellectual domain. Three generations of neuroscientists have now been trained without any link to a widely accepted general theory of neural circuit function and neural integration. They have been given to believe that they are engaged in a massive fact-finding operation guided only by the relative softness of the seams in the body unknown that happened to face their individual picks! Science without larger questions provides a dismal prospect to a truly inquiring mind. Of course, to those who would make careers out of providing random facts, nothing could be nicer, so varied and so complex are nervous systems. There is enough material to occupy armies of such persons for centuries. But without some strong delineations neuroscience will continue to explode into myriad fragments. We shall end up with masses of descriptive minutiae of many nervous systems without advancing our overall understanding of how they do the job for which they evolved. (Hoyle, 1984, p. 379)

Almost 20 years later, Hoyle’s words are still correct. Neuroscience has not yet become a sound theoretical domain. Most neuroscientists conduct research based on the traditional classic theory of neural circuit function with its serious limitations, and students are instructed based on old theoretical ideas. The need for a new neurobiological theory is enormous. Increasingly, neurobiologists are addressing functions of the highest brain levels, including such fundamental problems as the relationship between brain and mind and the mechanisms of thinking, consciousness, self-awareness, and so on — problems that traditionally belonged to the philosophical domain. Addressing these issues within the old neurobiological theoretical framework has failed to produce positive results, and new ideas are needed. Application fields also need a new theory. Developing new medical treatments is increasingly expensive and time consuming, and choosing a wrong strategy based on an inadequate theory wastes resources. Examples of such failed strategies are discussed in Sections 5 and 6. Furthermore, new medical treatments sometimes become mainstream even though their underlying mechanisms are unclear. Functional neurosurgical procedures for the treatment of movement disorders are typical examples. Partial lesioning of basal ganglia structures has been used successfully for decades to alleviate the symptoms of Parkinson’s disease (PD) and other motor disorders. These methods were discovered empirically and still await an adequate explanation of how they work. Recently, two new neurosurgical treatments were introduced — deep brain stimulation (DBS) and neurotransplantation. The former involves the same basal ganglia regions whose partial lesioning alleviates symptoms. Potentially, two features make DBS the most broadly used neurosurgical treatment for motor disorders. The first is its reversibility. The second is easy adjustability to a new situation by reprogramming the stimulator. Neurotransplantation is still a controversial

treatment. Miraculous results achieved by transplantation have been described, but detailed studies of this treatment are less encouraging. One theory that has tried to explain basal ganglia pathophysiology (Section 5.3) is popular in the medical field but remains almost unknown among basic scientists who study basal ganglia function. As originally proposed, this theory does not explain functions of the basal ganglia. In contrast, we strongly believe that it is impossible to explain pathophysiology without understanding the underlying function of an organ. The goal of this article is to introduce a broad audience to new theoretical ideas that can be used to explain various brain functions. The real power of the theory is its capability to explain functions and dysfunctions of the highest brain levels, including old problems such as the relationship between brain and mind, the definition of consciousness, and so on. The discussion focuses on the physiology and pathology of cortico-basal ganglia –thalamocortical loops, which are the major anatomical substrates of the highest levels of brain function. PD exemplifies how the theory can be applied clinically. The proposed theory offers original explanations for the mechanisms underlying functional neurosurgical procedures used to treat PD such as selective lesioning of basal ganglia nuclei, their chronic stimulation, and neurotransplantation. Any theory must satisfy a specific set of criteria. Before these criteria are developed, however, the term theory must be understood. According to The American Heritage Dictionary, third ed. (1994), a theory is a ‘‘systematically organized knowledge applicable in a relatively wide variety of circumstances, especially a system of assumptions, accepted principles, and rules of procedure devised to analyze, predict, or otherwise explain the nature or behavior of a specified set of phenomena’’. If this definition of theory is adapted to neurobiology, the set of criteria will include broad applicability (i.e., the capability to explain various neurobiological phenomena under normal and pathological conditions) and high predictive power. If systematically organized knowledge fails to meet these criteria, it cannot be considered a theory (e.g., theories that explain only pathological phenomena and not function (Section 6)). Requiring ‘‘strong’’ criteria allows two goals to be attained. First, a theory can be tested by appropriate experiments. Second, researchers can focus on the most important experimental and theoretical neurobiological achievements and exclude from consideration theoretical views that do not meet the criteria. In this way, the enormous number of theoretical papers that saturate neurobiology, views described in the form of theories, models, computer simulations, hypotheses, and philosophical arguments, can be filtered. The term theory should be reserved for the highest level of evaluation. Less ambitious propositions are usually published as models, hypotheses, and so on. The article material is organized as follows. First, limitations of the classic approaches used to study the brain are

K.V. Baev et al. / Progress in Neuro-Psychopharmacology & Biological Psychiatry 26 (2002) 771–804

presented. Experimental observations that form the foundation of the theory, basic theoretical conclusions, and straightforward logical derivations follow. Finally, some new notions are introduced. These steps are necessary to build the theory and to demonstrate its broad applicability to the functions and pathophysiology of cortico-basal ganglia – thalamocortical loops. The article is not comprehensive for several reasons. First, such a discussion would be voluminous, clumsy, and unreadable. Second, the presented theory is conceptual (i.e., it is about principles of brain construction) and describing functional principles does not require comprehensive citations. Only major experimental observations that reveal the utility and broad applicability of the proposed ideas are referenced; otherwise, the number of references would be excessive. Widely known experimental observations are mentioned without references. Nevertheless, all information needed to grasp the theory is included. The paper is written as a standalone article and develops previously published ideas (Baev, 1998; Baev and Shimansky, 1992; Baev et al., 1995).

773

mental models. Below, locomotor behavior is used to illustrate the common strategy for studying neural mechanisms of this behavior. According to traditional views, motor patterns for different body parts are produced by CPGs in response to simple tonic descending or afferent commands (Fig. 1). The term CPG, adopted from electrical engineering, reflects the capability of those neural centers to produce motor rhythm autonomously in the absence of peripheral feedback. The interaction of different generators is the basis for the coordination of different body parts. Generator activity is controlled by peripheral feedback and higher motor centers. A CPG produces an approximation of a necessary motor pattern, and numerous corrective reflexes and higher motor centers adjust this pattern to environmental conditions. There was a strong hope that understanding the organization of CPGs would bridge the gap between our knowledge of neural processes and behavior. Understanding the organization of CPGs was considered a prerequisite for understanding the control of automatic movement. The logic

2. Classic neurobiological theoretical views and their limitations Classic theoretical views demand that the nervous system be understood from a mechanistic perspective (i.e., biological neural networks must be explained in terms of neuronal interconnections, interactive nerve cells, transmitters, and cellular properties). Experimental analysis is directed toward obtaining this information, and a synthesis ‘‘recreates’’ the system — the model must mimic the behavior of the system as precisely as possible. This process is how the relationship between structure and function is viewed from the mechanistic approach. Within this framework, analysis and synthesis are ‘‘rather effective’’ when simple neural networks are studied. The following sections analyze the limitations of the mechanistic approach by using the neural control of inborn automatic behaviors in animals as an example. Without exclusion, the conclusions are applicable to other forms of behavior because the same approach was used to study the underlying neuronal processes. 2.1. Central pattern generators (CPGs) cannot be understood within the framework of classic theoretical views Of all inborn behaviors, automatic movements like locomotion, breathing, swallowing, chewing, and scratching are the most extensively studied, primarily for methodological reasons. Rudimentary experimental animal models allow scientists to control these behaviors and to study the functions of the parts of the brain that control these behaviors. Real or fictive locomotion or scratching in decerebrate animals and locomotor rhythm generated by isolated spinal cord or ganglion are typical of such experi-

Fig. 1. Traditional views of the system controlling inborn automatic behavior. CN: command neurons. CPG: central pattern generator for a given body part. M: motoneurons. EO: effector organ.

774

K.V. Baev et al. / Progress in Neuro-Psychopharmacology & Biological Psychiatry 26 (2002) 771–804

behind such an approach was simple. Knowledge of a CPG implied knowledge of a circuitry that produces a basic locomotor program. If obtained, such knowledge would help to explain various peripheral corrective reflexes and mechanisms of interaction of CPGs among themselves and with higher control centers. According to Getting (1986), the first scientist to apply the mechanistic strategy successfully to study CPGs in invertebrates (Getting, 1981, 1983a,b), investigating CPGs is ‘‘basically a ‘top-down’ approach starting with behavior and working down to a progressively more and more detailed description at the cellular and mechanistic levels’’. Getting distinguished eight basic steps in the strategy: (1) description of behavior; (2) characterization of motor pattern (i.e., efferent commands generated by CPG); (3) identification of motoneurons and interneurons (motoneurons are not considered components of CPGs); (4) identification of pattern-generating neurons (a change in the activity of a generator neuron must be accompanied by changes in rhythm pattern at all levels: interneuronal, motoneuronal, and behavioral levels); (5) mapping of synaptic connectivity between neurons; (6) characterization of cellular properties; (7) manipulation of network, synaptic, or cellular properties to identify the role of individual cells, synapses, or cellular properties in the generation of the overall motor pattern; and (8) reconstruction of pattern generator, motor output, and behavior. At this final step, the description may be qualitative and verbal or quantitative computer simulations. Numerous studies in different animal species have been based on this strategy. More exactly, it has been the only strategy used by various researchers. Detailed information about the organization of CPGs was obtained in some invertebrates. For example, the swimming generator in the sea slug includes only twelve neurons and appeared to be very simple (Getting, 1983a,b; Getting et al., 1980). Computer simulations based on neuronal interconnections and cellular and synaptic properties coincided well with the behavior of the real generator network (Getting, 1989). Much less detailed information was obtained in vertebrates, especially in higher vertebrates, where neural networks appeared to be too complex for this approach. However, all those studies led to the same conclusion: There is no single plan of generator organization. In other words, different schematic solutions evolved to build generators in various animal species. Even in invertebrate species, CPGs do not appear to operate by a single mechanism but rather by the interaction of many (Getting, 1986). Network, synaptic, and cellular mechanisms all provide generator activity. For most continuous behaviors, CPGs are associated with one or more neurons with pacemaker properties. In contrast, CPGs that are used rarely and produce only a few cycles rely on a balance of synaptic excitation and inhibition. In 1914, Brown proposed the first hypothesis of rhythm generation based on the observation that alternating flexion and extension movements can occur in a deafferented animal after spinalization. According to this hypothesis,

the locomotor program is the result of alternating activity of two mutually inhibiting half-centers, flexor and extensor half-centers (Fig. 2a). Even generators considered to be understood operate within the framework of Brown’s hypothesis; they have two mutually inhibiting half centers (Fig. 2b). That no single plan of generator organization could be found was a shocking result. It meant that the gap between our knowledge of neural processes and behavior was not bridged even in the case of inborn automatic behaviors. Not surprisingly, the mechanistic approach was even less successful in revealing the mechanisms underlying more complex learned behaviors, which usually involve the highest brain levels. The accumulated experimental observations demonstrate a variety of neural network schematic solutions (including synaptic, cellular, and molecular mechanisms) in different animal species and in different parts of the nervous system within the same species commensurate with the variety of life forms with a nervous system. Classic views fail to explain why biological neural networks are so different and not built according to one schematic plan. As we see, Graham Hoyle was right (Section 1). Neurobiology ended up ‘‘with masses of descriptive minutiae of many nervous systems without advancing our overall understanding of how they do the job for which they evolved’’. This situation evolved despite the enormous progress in experimental methods used to study the brain during the 1980s and 1990s. For example, imaging techniques such as positron emission tomography and functional magnetic resonance imaging (fMRI) became broadly used to reveal

Fig. 2. Conceptual models of central pattern generators. (a) Brown’s model. F and E: flexor and extensor half-centers. Mf and Me: flexor and extensor motoneurons. (b) A conceptual scheme of the organization of the locomotor generator in the Xenopus embryo (Roberts et al., 1986). E and C: excitatory and contralateral inhibitory interneurons, respectively. M: motoneurons.

K.V. Baev et al. / Progress in Neuro-Psychopharmacology & Biological Psychiatry 26 (2002) 771–804

discretely active brain regions and their sequential changes during different mental tasks in humans. Behavioral genetics started to provide data about the relationship between genes and behavior. Even old electrophysiological and morphological methods benefited considerably from developments in computer sciences. The history of other sciences (e.g., physics) has shown that experiments are important and necessary but not sufficient conditions for scientific progress. Theoretical research is needed to complement experimental investigations. Historically, however, neurobiology has emphasized experimental research based on the mechanistic approach. The prevailing idea has always been that an understanding of brain function would emerge if enough experiments were conducted. Consequently, almost all neurobiological data have been accumulated by conducting experiments within the classic mechanistic framework. Arguably, the mechanistic approach has not failed; perhaps, existing experimental methods are too weak to provide ‘‘complete’’ data about the structure and cellular activity of a complex biological neural network. What would happen if such ‘‘ultimate’’ methods were available and ‘‘complete’’ data were obtained? Would we then understand how the brain works? How useful could information about the connections and activity of millions and billions of cells be? These and other related questions remain unanswered within the mechanistic framework. It is increasingly clear that such detailed information is neither necessary nor heuristic. To understand how the brain works, our theoretical framework must change. If a theoretical approach does not work, everyone has a right to conclude that is incorrect. Any theory has its own inherent limitations as does every experimental method. Yet, a theory involves not only formulating a problem and planning research but asking the correct questions. Possibly, the mechanistic framework precluded asking the correct questions. Within the mechanistic framework, how networks that perform specific functions are constructed is the overarching question. To answer it, researchers have sought the schematic solutions used by various biological neural networks to control behavior. However, a different theoretical framework is needed to ask more productive questions. For example, why are biological neural networks built the way they are? Without pursuing the philosophical aspects of the mechanistic approach further, it is possible to conclude that this approach lacks the functional completeness needed to describe objects as complex as biological neural networks. The mechanistic approach possesses little if any generalization power. In the analysis of simple invertebrate nervous system, it created an illusion of ‘‘effectiveness’’. Consequently, research involving simple networks became popular. Eventually, however, this perspective leads us to ‘‘study what we can instead of studying what we should’’. The tacit hope is that knowledge of simple systems will clarify the function of complex systems. Historically, science has seldom worked this way; rather, studying a simple

775

object usually leads to simple knowledge. Perhaps, then, an alternative to the mechanistic approach is needed. Defining common principles of functional organization of various biological neural networks is a good starting point. Experimental observations and straightforward logical derivations are now used to build an alternative approach.

3. An alternative theoretical approach 3.1. Neuronal automatisms The brain can be viewed as a control device created by evolution, and the goal of neurobiology is to understand how it works. Based on the accumulated neurobiological data, it is possible to conclude that the capacity for automatic control is the principal feature of the nervous system. At birth, an animal possesses a vast variety of inborn automatisms and acquires new ones during its lifetime. Moreover, the more evolutionarily complex an animal is, the deeper is the significance of its spectrum of inborn and acquired automatisms. Disabling an animal by lesioning different brain regions abolishes the automatisms controlled by these parts. As a rule, the most evolutionarily recent areas of the brain control more complex automatisms. For example, a cortical lesion in mammals abolishes not only previously acquired automatisms, but the ability to produce new ones as well. Inborn automatisms, however, remain intact. To some degree, we have extended the concept of an automatism to include acquired skills compared to the definition usually used in neurobiology — to describe inborn behavior. The need for such a broad generalization of the automatism concept will become clear. Only a comprehensive understanding of automatisms permits the function of the brain to be analyzed from a unified perspective. Reflexes and program control in combination with feedback control are automatisms. The logical conclusion is that learning can be considered as the process involved in the formation of new automatisms. Given the utility of this more generalized definition of automatism, the major questions confronting neurobiology must now be reformulated. Instead of traditional questions that focus on the construction of different reflex arcs within the nervous system and on neural networks that generate programs for effector organs, a more fruitful and universally applicable question should be introduced: What is the basis for the automatisms of a nervous system and its different parts? As shown later, the answer to this question is the basis of our ability to overcome the limitations of the mechanistic approach and to formulate a more comprehensive theory of brain function. To solve the problem of neuronal automatisms, a new more general solution is needed for the problem of inborn automatisms. This new solution would reveal the principles underlying control of inborn automatisms and could be used to understand how acquired automatisms are controlled.

776

K.V. Baev et al. / Progress in Neuro-Psychopharmacology & Biological Psychiatry 26 (2002) 771–804

3.2. The essence of neuronal automatisms 3.2.1. Solution of CPG problem The spinal CPGs for locomotion and scratching interact with afferent flow arriving at the spinal cord via peripheral feedback circuits. These generators possess an internal model of the controlled object (CO)’s dynamics. This internal source of afferent information, model afferent flow, is treated by the CPG as a component of actual afferent flow (Baev and Shimansky, 1992). The internal model should be considered an additional internal ‘‘sensor’’ that complements other limb sensors. Its goal is to supply the control system with information about the most probable state of the controlled object at any given moment. After limb deafferentation, the control system relies solely on the information provided by the model. If the model information is adequate and no significant accidental perturbation occurs, the control system will produce the proper control action. Thus, ‘‘the generation of a rhythmic motor pattern’’ is a reverberation, not between generator flexor and extensor half-centers (Fig. 2), but rather within the loop of internal feedback that transfers model ‘‘sensory’’ signals (Fig. 3). Therefore, at least one principle exists for the functional construction of CPGs: In all animal species, CPGs include functional subunits such as an internal model of object behavior (i.e., the subsystem that generates expected afferent flow from the controlled object) and a controller (i.e., the subsystem that provides a governing set of commands — a control law — that directs the action of the recipient of these rules — the controlled object).

Fig. 3. Functional organization of central pattern generator. See text for explanations.

3.2.1.1. CPG is a learning system. Because the CPG includes a model of object behavior, this conclusion is inevitable. The model should be tuned to the object as precisely as possible for the controlling system to function properly. If the controlled object changes, the model must be retuned. The model is retuned during the intensive growth of body and limbs at certain developmental stages. Many animals are born with well-organized motor functions that develop during embryogenesis. The latter is characterized by embryonic motility, which already appears during the early stages of embryogenesis and undergoes several further stages. Perhaps the most detailed investigation was performed on chick embryos (Hamburger, 1963; Hamburger and Oppenheim, 1967). Such embryonic movements are characterized by random twitches that become progressively more complex during development. By trial and error, these movements evoke the afferent signals necessary to tune the internal model of the controlled object correctly. Because the solution of the CPG problem is functional, the term CPG must have only a functional meaning. In other words, is it necessary to reconsider the term CPG? Is it an anatomical or functional construct? The mechanistic strategy of investigating CPGs implies that the term generator designates a definite structure that scientists can study. In the case of complex neural networks, the term CPG has only a functional meaning. Otherwise, different structures must exist in one structure, like scratching and locomotor generators in the mammalian spinal cord network. The same spinal cord neurons must participate in generating scratching and locomotor rhythms. However, it is impossible to consider scratching and locomotion as different regimes of work of the same generator. Experimentally, scratching and locomotion are antagonistic movements that exclude each other. If the analyzed networks are multifunctional, the mechanistic approach is unlikely to be applicable. How would neuronal interconnections be correlated with function (i.e., how would the relationship between structure and function be approached)? One must conclude that the adopted term CPG, which possesses a structural meaning in electrical engineering, has fueled the mechanistic investigation of CPGs (i.e., neural control of inborn automatic behaviors). This research has been based on an incorrect assumption. If the term CPG has only a functional meaning and CPGs for scratching and locomotion are considered different regimes of the spinal motor control system, no contradictions exist. 3.2.2. Neural optimal control systems (NOCSs) The presence of an internal model in a neural control system suggests several profound conclusions. There are two major reasons for the controlling system to have a model of the object behavior. The first is the incomplete observability of the controlled object. The second is incomplete controllability; the controlling system cannot bring the controlled object into a new state in a single control step. The results of intermediate computations must be stored in a controlling system to perform recursive computations. Control theory

K.V. Baev et al. / Progress in Neuro-Psychopharmacology & Biological Psychiatry 26 (2002) 771–804

states that a controlling system must have a model of object behavior to control incompletely observable and incompletely controllable objects optimally. Therefore, neural control systems must be optimal control systems and must include a model of object behavior. The objects of NOCSs are incompletely observable and controllable. Although this statement is true for any level of the brain, it is best illustrated by the highest levels. For example, the visual system receives information from only a portion of the environment. A moving limb can encounter various external obstacles that are unaccounted for by a motor control system. Logically, then, a CPG is a regime of work of NOCS. The presence of a model within a NOCS provides certain advantages. The NOCS effectively uses model afferent flow to filter peripheral information to determine the current state of the controlled object. The complexity of interaction between model and actual afferent flows can vary significantly in different NOCSs. As mentioned, a NOCS considers model afferent flow as a component of real afferent flow (i.e., model and real afferent flows have parity rights). Parity interaction of different afferent components is organized in such way that a NOCS pays more attention to more intensive informational channels. In the spinal cord, presynaptic inhibition plays the role of such an attention mechanism (Baev and Shimansky, 1992). Due to this mechanism, the more intense channel, either model or real, evokes the largest postsynaptic response when compared to typical summation. In such situations, the NOCS considers low intensity or silent informational channels as unreliable sources of information. This characteristic explains why a NOCS that generates motor rhythm can function after partial or complete deafferentation. We call this feature holographic, which means that a system can function after its partial destruction. It is analogous to holography where a complete three-dimensional image of an object can be reproduced at a lower resolution after part of the photographic plate has been destroyed. This property is a fundamental and crucial property of any neural network. Interaction between the predictive output from the internal model and peripheral afferent signals has another important feature (Shimansky, 2000). A NOCS uses a special integration procedure to merge both signals to determine the current state of the controlled object with more precision than that based on either source separately. This process is labeled ‘‘summation of information precision’’ (Shimansky, 2000) and is performed by a special functional block, input integrator (Fig. 4). In the spinal cord, this integration mechanism is based on presynaptic inhibition. In other brain regions, complex synapses can play a similar role (e.g., cerebellar glomeruli or complex thalamic synapses). The other advantage of a NOCS having a model is the ability of the control system to compare model and actual afferent flows. This process enables the control system to receive important information such as mismatch or error signals (Fig. 4). A mismatch signal between real and model flows is necessary for learning to occur (see below).

777

Fig. 4. Functional organization of a generic NOCS. Informational and initiating signals are shown by single-line and double-line arrows, respectively. See text for explanation.

Any functional neural system involved in the expression of a discrete automatism sends controlling signals to a controlled object (effector organ) through its output. Two types of afferent signal inputs contain the information necessary to compute an output signal: (1) initiating signals and (2) signals containing the current informational context of the system as a whole. These two types of afferent signals have only one difference. As an automatism is realized, the system attempts to minimize initiating signals. Strictly speaking, it attempts to minimize the integral measure of initiating signals by using informational signals to compute the appropriate output. Initiating signals are analogous to ‘‘energetic’’ signals. This subdivision is relevant to the discussion of learning in NOCSs. Afferent flow can only be segregated into these two types of signals at the level of the recipient NOCS. Consequently, one NOCS can interpret the same control system differently than another NOCSs (i.e., either initiating or informational signals). Initiating and informational signals can be shortand long-lived depending on the control task. The signals are long when the controllability of the object is low and time is required to remove a corresponding initiating signal by sending complex control influences to the controlled object. Finally, there are subtypes of initiating signals. For instance, one starts an automatism. Another subtype, a mismatch between model and real flows (an error signal), goes to the model subsystem of a network and to the higher level (Fig. 4). Both must be minimized during control.

778

K.V. Baev et al. / Progress in Neuro-Psychopharmacology & Biological Psychiatry 26 (2002) 771–804

Therefore, a mismatch signal is the signal that initiates learning processes within the model (i.e., it initiates a learning automatism). The following describes each signal as precisely as possible to avoid confusion, especially when a mismatch signal is sent to another NOCSs. 3.2.3. Network computational principle Thanks to neurocomputing, neurobiology acquired the notion of ‘‘computation’’ in the 1980s and 1990s (see, for example, Hopfield, 1984, 1987, 1994; Hopfield and Tank, 1985; Hecht-Nielsen, 1990). The concept of computation is ideal for those who describe information processing in biological neural networks. According to this concept, biological neural networks approximate different mathematical functions. That is, they can perform computations over components of an input vector (input variables) and generate an output vector (output variables). To imagine how networks can perform computations, consider the following mathematical abstraction, the so-called three-layer unidirectional neural network (Fig. 5). The processing n elements of the first layer are fanout units that distribute the input x vector components to the processing elements of the second hidden layer. The processing elements of the hidden layer neither directly receive inputs from nor provide direct outputs to the external world. The transfer function of these units is similar to a linear-weighted sum. The m output processing elements of the third layer send signals to the external world, the output y vector. The transfer function of output units is highly nonlinear. Mathematically, such threelayer neural networks can implement any continuous mapping function if their synaptic weights are adjusted properly. The number of layers also may be greater than three. In neurocomputing, numerous learning algorithms have been developed that allow a neural network to implement any function of practical need by adjusting the network’s synaptic weights.

Fig. 5. Architecture of a three-layer neural network.

Two features are unique to neural networks. First, many mapping functions can be stored within one network. The quality of the performed approximations degrades slowly as the number of stored functions increases. Therefore, the very nature of the network computational principle consists in a remarkable capability to be multifunctional. Second, any network can continue to function after partial loss of its neural elements, the feature previously referred to as holographic. Biological neural networks are usually multilayered. They are not unidirectional and have numerous negative and positive feedback loops. Thus, the question emerges: How do these connections influence the computational power of biological neural networks? There is no answer yet, but the computational abilities of biological neural networks likely significantly surpass those of artificial ones. As mentioned, the presence of feedback loops in real neural networks can be interpreted as a necessary condition to improve afferent information processing (because they provide a substrate for the function of an internal model within the system) and to compute the output of a controlling system. The presence of feedback loops increases the calculating abilities of the network because recursion becomes possible when the function value for a given argument value can only be computed using its values for some other values of the argument. This procedure is important when it is impossible to move the object to a desired state during one control step. In some networks, recursive computation can continue until a desired result that satisfies a particular calculation criterion is reached. It is well known from the mathematical theory of algorithms that the class of recursive functions has maximum functional power. Therefore, from a computational perspective, real neural networks have more functional power than the unidirectional networks described above. The existence of complex life forms with advanced nervous systems also can be considered as evidence of the highly sophisticated computational abilities of biological neural networks. 3.2.4. Learning in biological neural networks Based on the notion of computation just introduced, the definition of neural automatism can be refined. The relationship between the terms automatism and computation is similar to the relationship between a goal and the means by which the goal is achieved. To create a new automatism or to improve an existing one, the computational abilities of each functional subdivision within a particular control system must be adjusted appropriately. After these adjustments, the corresponding neural networks can approximate the necessary functions (i.e., both the controller and the model function properly to control the specific automatism). Biological neural networks can use numerous learning strategies to adjust approximated functions. Moreover, not only synaptic weights can be changed during learning. Cellular properties also can be adjusted during learning. This section focuses on some temporal and spatial relation-

K.V. Baev et al. / Progress in Neuro-Psychopharmacology & Biological Psychiatry 26 (2002) 771–804

ships between initiating and informational signals during learning based on a trial-and-error strategy. The mechanisms responsible for changing network parameters are not discussed. Trial and error is the most universal strategy, and numerous other types of learning could derive from it during the course of evolution. Trial and error implies that a trial occurs first and is followed by an error signal (i.e., informational signals arrive in the controlling system earlier than initiating signals). Trial-and-error learning could be the foundation for conditioned responses. Classical conditioning is known as Pavlovian conditioning (Fig. 6). The most efficacious conditioning paradigm is simultaneous conditioning when a conditioned stimulus (CS) terminates at the end rather than at the beginning of an unconditioned stimulus (US). Recall that a NOCS tries to minimize initiating signals. The US is an initiating signal and the CS is informational context. The US evokes changes in network parameters. For example, synaptic weights of the inputs active before the US arrived are changed so that network starts computing a new function based on these adjustments. On the next control step, the network will compute a new function based on the informational context (CS) and a new control signal will be generated. If this new control signal does not reduce or remove the initiating signals, the adjustment process continues. It stops when the initiating signals are minimized.

Fig. 6. Types of classical conditioning. CS: conditioned stimulus. US: unconditioned stimulus.

779

Fig. 7. Minimization of the central influence of an unconditioned stimulus during classical conditioning. CS and US: conditioned and unconditioned stimuli, respectively. u: Control signal. m: Movement like eyelid closure. as: Aversive stimulus like an air puff to the eye. a and b: Beginning and end of learning, respectively.

Classical conditioning of the eyelid closure response is a typical example of this learning scheme with tone and air puff to the eye being the CS and US, respectively (Fig. 7). At the end of learning, eyelid closure is produced by the controlling system to minimize the initiating signals evoked by the air puff. A similar learning scheme can be used to tune the model to the controlled object. In this case, an error signal plays the role of the initiating signal. A trial-and-error strategy can be effective under several conditions. The first is the so-called credit assignment (Minsky, 1963; Barto et al., 1983). This problem consists of delivering an error signal to the appropriate neurons and hence to their corresponding synapses (spatial credit assignment) during a uniquely appropriate window in time (temporal credit assignment). The second, the informational context necessary for the computation of a controlling output must be present, or at least trace processes must be evoked by informational signals during computation of the controlling output. Otherwise, this type of learning will not work. Therefore, trace conditioning needs some form of memory mechanisms to be capable of solving a credit assignment problem. Proper learning requires changes in the synaptic weights of inputs active during a specific time window. At lower levels, the time window can be brief — tens or hundreds of milliseconds. Membrane parameters like the time constant could play a role in such short-term memory. At higher levels when the time between CS and US can be seconds, minutes, or hours, special memory mechanisms must be involved for this type of learning to work effectively. This type of learning can occur if initiating signals evoke fluctuations in the synaptic weights of neural networks. The stronger the signals are, the stronger are the fluctuations, and synaptic weights can increase or decrease. This process is usually defined as random search. The method of random search is universal because it gives a neural network the

780

K.V. Baev et al. / Progress in Neuro-Psychopharmacology & Biological Psychiatry 26 (2002) 771–804

ability to find a new decision when there are no preexisting algorithms to direct its output appropriately. Being universal, however, this method may be very slow when a search is performed in the hyperspace of a network’s parameters. Evolution could use several methods to accelerate this process. First, specific mechanisms could determine the gradient of an initiating signal. Second, system parameters could be adjusted in a dependent fashion (pattern adjustment). Third, incorrect decisions could be memorized (possible at highest levels). Finally, during evolution, systems based on random search methods could find a rule or a set of deterministic rules to adjust their computed functions for certain situations. For instance, an initiating signal could possess a sign — excitatory or inhibitory — and show the direction of any necessary synaptic changes. Any improvement in learning must be based on the specifics of a network, as well as its cellular and molecular mechanisms. Therefore, the more complex the strategy involved in the search for new decisions is, the more complex its mechanisms should be. 3.3. Interaction of NOCSs The brain includes numerous NOCSs. Analyzing the construction of a single NOCS reveals common principles of how NOCSs interact. Two basic types of interaction — coordination of various NOCSs located at the same hierarchical level and hierarchical interaction between separate NOCSs — are discussed below. Coordination is a problem at any hierarchical level. Functions of various NOCSs must be coordinated in space and time to achieve a control goal. A typical example is the interaction between different limbs during locomotion. Based on analysis of possible interactions between simple NOCSs forming a more complex one, two mechanisms are available to the system to solve coordination problems. First is the anatomical arrangement that results in mutual interconnections between all possible pairs of NOCSs. Each NOCS creates an internal model of signal inflow from other NOCSs because this inflow is a source of its afferent flow. Such constructions can be relatively efficient in simple systems with few NOCSs. In invertebrates, for example, the various ganglia that control limb movements are interconnected. In vertebrates, propriospinal neurons connect to various segments of the spinal cord. This first mechanism has a limited controlling capacity. It works well only for a simple coordination problem. This scheme becomes awkward when a large number of NOCSs need to be coordinated and other sensory modalities (e.g., visual, acoustic, and vestibular) are involved. Because each signal is sent to each participating NOCS, this solution is only found in the nervous systems of very simple animals. In nervous systems with a well-developed hierarchy, a second solution evolved. In this second mechanism, one coordinating central dispatcher receives information from all NOCSs and other sources, processes it, and sends corresponding commands

back to each NOCS. The dispatcher accumulates knowledge about spatiotemporal correlations between afferent signals and optimal control signals sent to each NOCS in a particular situation. The cerebellum, which evolved in vertebrates to control movements, is a typical example of such a central dispatcher. Its functions are described after the principles of hierarchical interaction among different NOCSs are considered. The nervous systems of almost all animal species are hierarchical. Only the simplest animals with a diffuse nervous system are excluded. In general, complex controlling systems, including technical, social, economic, and other systems, are hierarchical. Hierarchy is a frequently abused term. Colloquially, the word is used to express subordinate relationships. This meaning is correct but the term connotes far more: Why are complex control systems like the nervous system hierarchical? What are the major functional advantages of hierarchy in a control system? Consider the interaction between two NOCSs, one of which is hierarchically higher than the other. The lower level NOCS is a controlled object for the higher level NOCS, and the higher level NOCS has a model of its controlled object (i.e., a model of the lower NOCS). This model should not be confused with the model of the controlled object of the lower NOCS. The latter is included in the lower NOCS. Therefore, when moving from lower to higher levels, encoded parameters are generalized or abstracted within the hierarchy. The model of a higher level is qualitatively different from a lower level model. The parameters of higher levels change less frequently than the parameters of lower levels. In general, the hierarchy can be considered a consequence of an object-state hierarchy (Baev and Shimansky, 1992). For instance, a multijoint limb is a hierarchical object. Such a relationship between hierarchical levels explains why a simple command from a higher level can initiate a complex automatism controlled by a lower level. For example, in all studied animals, simple tonic activation of command neurons initiates complex automatic behaviors. Brainstem and spinal cord command areas are responsible for organized acts such as eating, gnawing, licking, lapping, sexual acts, aggression, and many other types of inborn behaviors. These command areas also activate corresponding automatisms that can coordinate the many subcomponents of behavioral acts by integrating the sensory feedback from each subcomponent. Thus, the higher level ‘‘imagines’’ the behavior of its object in a manner that differs from the way the lower level does. When lower automatisms are initiated by a command system, a trajectory at the lower level corresponds to a point within a given system state at this higher level of command neurons. The higher level controls more abstracted movement parameters, for example, intensity or speed of locomotion; the lower level controls the detailed activity of individual muscles. Consequently, a dynamic model of the lower level predicts the state of the object in the next

K.V. Baev et al. / Progress in Neuro-Psychopharmacology & Biological Psychiatry 26 (2002) 771–804

moment. The higher level model thus predicts in what state and when the transition of the object to its next state will occur. In other words, a cause – effect model of the controlled object predicts a series of object states. This prediction includes coding probability distributions for the time intervals after which these states must appear. The controlled object sends to its controlling system (i.e., to the lower NOCS) the same types of signals that the lower NOCS sends to a higher NOCS (Fig. 8a) because the lower NOCS is a controlled object for the higher NOCS. Other routes for initiating, mismatch, and informational signals also may exist in hierarchical biological neural networks (Fig. 8b). Their advantage is obvious. For instance, the control abilities of a lower level are significantly extended if it receives mismatch signals from higher levels because new minimization criteria are added. The higher level also can provide the lower level with an informational context, the type of which does not exist at the lower level (e.g., vestibular information for the spinal cord, visual and acoustic information for the cerebellum). The control level and its sensory detectors should match because the latter must describe the corresponding state space coordinates accurately. For instance, a higher motor control system can operate with a parameter such as the direction of locomotion, a parameter that does not exist at the level of a CPG and that is the result of hierarchical generalization of lower level parameters. The complexity and sophistication of sensory detectors can be considered from the same hierarchical perspective. Any descending influence on a specific system transferring ascending afferent flow can be considered a control influence of the control system on its controlled object. In such a system, a lower detector is fixed on a particular feature when a corresponding descending initiating signal arrives. A mismatch signal will ascend

Fig. 8. Information exchange between different hierarchical levels. is: Initiating signal. ms: Mismatch signal. u: Control signal. ic: Informational context. See text for explanations.

781

Fig. 9. Semantics of cerebellar connections. SOCT: spinoolivocerebellar tract. SRCT: spinoreticulocerebellar tract. VSCT: ventral spinocerebellar tract. DSCT: dorsal spinocerebellar tract. CO: controlled object. cfs, mfs: climbing and mossy fiber systems, respectively.

until it reaches a competent level capable of removing it from the system. Therefore, the principle of hierarchy significantly facilitates the resolution of any complex control problem for the whole control system. In a hierarchical system, each level becomes responsible for its own automatism. Any movement toward higher levels leads to new computational problems and is accompanied by the development of specific computational, learning, and memory mechanisms because state space becomes increasingly discrete and the working time interval increasingly extended. 3.3.1. The cerebellum and movement coordination Cerebellar function is described to help understand the functions of cortico-basal ganglia – thalamocortical loops. The same ideas underlie the highest levels of the brain. The cerebellum is a coordinating central dispatcher. Based on the proposed theory, the cerebellar coordination function can easily be conceptualized by using the semantics of its afferent inputs (Fig. 9). These semantics are based on data about the activity of the spino-cerebellar loops during locomotion and scratching (Arshavsky et al., 1986). The discussion assumes that readers are familiar with basic cerebellar anatomy. The cerebellum has two afferent inputs: the mossy and climbing fiber systems. Three ascending tracts bring information from the spinal cord to the cerebellum via the mossy fiber system: the dorsal spino-cerebellar tract (DSCT), the ventral spino-cerebellar tract (VSCT), and the

782

K.V. Baev et al. / Progress in Neuro-Psychopharmacology & Biological Psychiatry 26 (2002) 771–804

spino-reticulo-cerebellar tract (SRCT). The DSCT is rhythmically active during real locomotion and scratching. This rhythmic activity disappears after deafferentation or pharmacological neuromuscular paralysis when fictive movement is observed. The VSCT and SRCT are rhythmically active during both real and fictive rhythmic motor behavior. Only one ascending tract brings information to the cerebellum via the climbing fiber system — the spinoolivo-cerebellar tract (SOCT). The SOCT has a low level of activity and modulation during fictive and real unperturbed movements. During locomotion, complex spikes occur only in response to an unexpected perturbation of movement. When an animal learns to overstep an obstacle placed in its path, the intensity of the complex spikes that occur when the limb contacts the obstacle (the perturbation) is maximal at the beginning of the learning process. At the end of the learning process (i.e., no contact between limb and obstacle during locomotion), their intensity is minimal (Bloedel and Lou, 1987). Acoustic signals preceding the appearance of the perturbing bar by several tens of milliseconds can play the role of a CS. This stimulus alone can evoke corresponding changes in parameters of the locomotor cycle. The cerebellum sends information to the spinal cord via rubro-, reticulo-, and vestibulospinal tracts. These descending tracts are rhythmically active during real and fictive locomotion and scratching. This rhythmic activity almost disappears after cerebellectomy. Therefore, information from the VSCT and SRCT can be interpreted as information about the state of the model of the corresponding lower NOCS (i.e., information about how the lower NOCS ‘‘sees’’ the controlled object). Information arriving at the cerebellum via the DSCT can be regarded as actual peripheral afferent information unaccounted for by the internal model. The cerebellum also receives visual, auditory, and vestibular information from mossy fibers. The cerebellum optimally filters all the afferent information that it receives to obtain the most reliable information about the current state of its controlled object — in this case, the entire body of the animal and its surrounding environment. Based on these experimental data, the SOCT conveys information to the cerebellum through the climbing fiber system about errors or mismatches at the lower NOCSs. Mismatch signals occur when the content of model afferent flow fails to coincide with peripheral afferent flow. Thus, olivary neurons are activated during errors of execution of actual movements, during perturbations, and when unexpected afferent information is received (e.g., when a limb contacts an obstacle). The cerebellum is organized economically. It does not create its own model of object behavior because it does not need to do so. It uses models from other NOCS to do this job. Its complexity and size are the consequence of the following factors. The cerebellum needs an enormous memory capacity to store its acquired knowledge. It works in real time and processes enormous amounts of information to

perform coordination tasks. Why poorly coordinated movements are possible without the cerebellum also can be understood — the lower NOCSs still perform motor control within their limited capabilities. For the type of learning described in Section 3.2.4 to be effective, the error signal should reach the cerebellum after the informational signal, as has been shown experimentally. When a stimulus is applied to the spinal peripheral nerve, the mossy fiber information reaches Purkinje cells 10 – 15 ms earlier than climbing fiber information (Eccles et al., 1967). Therefore, the short-term memory mechanisms needed to permit learning (Section 3.2.4) should be present in the cerebellum. The cerebellum likely can memorize a brief prehistory (what happened before the error signal arrived — from tens to at most several hundreds of milliseconds). Any informational signal within this time frame can be used to learn how to avoid future mistakes (i.e., to avoid receiving an error signal). By using such informational signals, the cerebellar circuitry learns how to generate a controlling output that minimizes error signals. This process is the essence of cerebellar coordination function. This brief interval can be extended significantly if the cerebellum is provided the necessary informational and initiating signals. The cerebral cortex sends information to the cerebellum through both cerebellar afferent systems (i.e., it sends informational and initiating signals). The cerebellum is thus provided with new minimization criteria and necessary informational context, and it becomes possible to learn more complex motor tasks. The cerebral cortex also benefits from the cerebellum. From the cerebellum, the cortex receives the most precise information about the current trajectory of the controlled object. Because the cerebellum receives model and real afferent flow from other NOCSs, the cerebral cortex is also provided with information about any mismatch between these two signals (i.e., novelty in object behavior). This information may play a significant role in complex pattern recognition (i.e., when cognitive motor learning occurs). Based on the law of symmetry, we can conclude that the cerebral cortex also provides the cerebellum with model, expected, and real afferent information as do other NOCSs. 3.4. Nonneuronal automatisms The notion of automatisms can be generalized beyond neural networks. They could be molecular, genetic, biochemical, cellular, and even social. In all these cases, the control system is hierarchical, and each level is an optimal control system built of a network of interacting elements. Therefore, all the concepts described thus far are applicable to nonneuronal automatisms. This conclusion is consistent with the well-known fact that the same type of computation can be accomplished with different types of elements — electronic devices, mechanical devices, chemical reactions, and so on.

K.V. Baev et al. / Progress in Neuro-Psychopharmacology & Biological Psychiatry 26 (2002) 771–804

3.5. Ontogenetic development of neuronal automatisms Neurocomputing helps to clarify what must be achieved during the ontogenetic development of neuronal automatisms. Neurocomputing uses various learning algorithms to adjust network parameters, the synaptic weights. Synaptic weights are selected randomly at the beginning of a training session. Training often takes a long time and sometimes never converges. Lengthy training, however, is not a problem. Even though months of continuous training may be needed, once successful, the final network configuration easily can be copied to other systems. Therefore, the benefit can be significant. Biological neural networks evolved along similar lines. Successful network solutions were then transferred to future generations by genetic automatisms. A network must be genetically predetermined to compute specific classes of functions. Sources of initiating and informational signals (detectors) for each level also must appear during specific developmental stages. Otherwise, the system will be unable to learn and to create a necessary model of controlled object behavior and will lack the ability to perform proper control tasks. For any neuronal automatism, learning is a necessary developmental component because genetic information cannot account for the numerous environmental conditions that confront an animal. Without initial genetic structural approximations of the networks used to compute specific classes of functions, the process of learning would take too long. It would be normal for learning to fail to create essential automatisms during a lifetime, and the existence of complex biological systems would be impossible. The situation would be analogous to finding a new solution without the benefit of any previous knowledge of how a new solution was attained (i.e., without knowledge of how it evolved).

4. Cortico-basal ganglia– thalamocortical loops Several remarks are necessary before the functions of the highest brain levels are described. Two functional subdivisions of any given NOCS were described above. Anatomically, they may be inseparable. For instance, both the controller and the model can be within the same neural circuit in the simplest of control systems. For example, some CPGs include only one pacemaker neuron that performs both controlling and modeling functions. In more complex cases, as described below, these functional subdivisions can be more anatomically distinct because more complex neural networks are necessary. Why do CPGs or other neural control systems that perform similar functions in different animal species have different schematics? Different controlled objects need different models, control laws, and other components such as a system-integrating model and afferent flow, calculating error signals, and so on. Each functional block is optimally constructed based on the available set of elements.

783

A simple algorithm can be used to analyze any neural control system based on the proposed theory as shown by the example of the cerebellum. For each hierarchical level, both functional subsystems — controller and the model — and the controlled object must be identified, and the various types of afferent signals — informational and initiating — must be determined. Special attention should be paid to error distribution systems. Below, this algorithm is used to reach several important conclusions about the organization of the highest levels in the nervous system based on the theoretical ideals developed thus far. Surprisingly, these conclusions can be reached without detailed knowledge of the computational abilities of the underlying neural networks. The highest brain levels are typical examples of biological neural networks whose functions cannot be understood from the classic mechanistic perspective. Billions of neurons, cortical and subcortical, are included in a colossal neural network performing functions like perception, complex motor control, thinking, consciousness, and self-awareness. These functions remain a mystery and an attempt to map all connections among all neurons and to study neuronal activity during various functions (i.e., using the mechanistic approach described in Section 2) will fail to resolve them. Below, our theory is applied to functions of the cortico-basal ganglia – thalamocortical loops, a major anatomical substrate of the highest brain level. Early data suggested that the basal ganglia receive diverse inputs from the entire cerebral cortex and ‘‘funnel’’ these influences via the ventral thalamus to the motor cortex (Allen and Tsukahara, 1974; Evarts and Thach, 1969; Kemp and Powell, 1971). Later studies suggested that influences from the sensorimotor and association cortices were segregated through the basal ganglia –thalamocortical pathway (DeLong and Georgopoulos, 1981; DeLong et al., 1983). Two distinct loops were distinguished. First, a ‘‘motor’’ loop passing through the putamen receives input from the sensorimotor cortex, and its influences are transmitted to certain premotor areas. Second, an ‘‘association’’ (or ‘‘complex’’) loop passing through the caudate nucleus receives input from the association areas, and its influences are transmitted to portions of the prefrontal cortex. In recent physiological and anatomical studies, the concept of segregated basal ganglia – thalamocortical pathways has been developed further. Five basal ganglia – thalamocortical circuits have been distinguished (Alexander and Crutcher, 1990; Alexander et al., 1986): the motor (or skeletomotor), oculomotor, dorsolateral prefrontal, lateral orbitofrontal, and anterior cingulate (or limbic). Each basal ganglia– thalamocortical circuit receives its multiple corticostriate projections only from functionally related cortical areas. Moreover, each circuit is formed by partially overlapping corticostriate inputs that are progressively integrated as they pass through the pallidum and substantia nigra (pars reticulata) to the thalamus and from there to a distinct

784

K.V. Baev et al. / Progress in Neuro-Psychopharmacology & Biological Psychiatry 26 (2002) 771–804

neurons leads to PD. The first symptoms of the disease are usually expressed when only about 15% of the dopaminergic neurons remain. 4.1. Skeletomotor loop

Fig. 10. Generic cortico-basal ganglia – thalamocortical loop. Each loop receives outputs from several functionally related cortical areas (A, B, C). Projections from these cortical areas are progressively integrated within the loop. Each thalamic region (specific for a loop) projects to one of the cortical areas that feeds into the loop that forms the ‘‘closed-loop’’ portion of the circuit.

cortical area (Fig. 10). The target area likewise usually projects to the basal ganglia. Consequently, it was hypothesized that the characteristic feature of all basal ganglia – thalamocortical circuits is the combination of ‘‘open’’ and ‘‘closed’’ loops. The following supposition is based on the apparent uniformity of synaptic organization at corresponding levels of these loops and their parallel nature: Similar neuronal operations are performed at comparable stages of each of these five loops. This supposition is correct, but the operations performed by these loops are unclear. These loops contain important feedback that includes dopaminergic neurons of the substantia nigra pars compacta (SNc) and the adjacent mesocorticolimbic group. Early studies suggested that the substantia nigra projects to the striatum while adjacent tegmental dopaminergic neurons make mesocorticolimbic connections. However, the complex organization of the cell body subgroups — the one located in the substantia nigra pars compacta and the other in the ventral tegmental area — is no longer defined in terms of striatal or mesocorticolimbic projections. These subgroups are intermingled; some mesocorticolimbic projections originate in the substantia nigra and vice versa (Le Moal and Simon, 1991). Dopaminergic neurons receive inputs from various brain structures — from the cerebral cortex, putamen, nucleus caudatus, entopeduncular nucleus, dorsal nucleus of raphe, central nucleus of amygdala, bed nucleus of the stria terminalis, and other structures. The importance of this feedback is well known: The death of dopaminergic

The skeletomotor loop is much better studied than other loops. If it is understood, the theory is easily applied to other loops. The skeletomotor circuit largely projects to the putamen, which receives projections from the motor and somatosensory circuits. The putamen also receives projections from area 5, from lateral area 6 including the arcuate premotor area, and from the supplementary motor area. The most prominent projections of these cortical areas go to the putamen, but each projection encroaches slightly on neighboring regions of the caudate nucleus. The putamen sends topographically organized projections to discrete regions of the globus pallidus (e.g., ventrolateral two-thirds of both the internal and external segments) and to caudolateral portions of the pars reticulata of the substantia nigra. The internal pallidal regions and substantia nigra send topographic projections to specific thalamic nuclei, including the nucleus ventralis lateralis pars oralis (VLo), lateral part of nucleus ventralis anterior pars parvocellularis (VApc), lateral part of nucleus ventralis anterior pars magnocellularis (VAmc), and centromedian nucleus (CM). The motor circuit is closed by thalamocortical projections from VLo and lateral VAmc to the supplementary motor area (SMA), from lateral VApc (from VLo as well) to the premotor area (PM), and from VLo and CM to the motor cortex (MC). Cortical motor areas are involved in motor control, but how is unknown. The role of basal ganglia structures is even less clear. Results from recording from striatal neurons in different behavioral paradigms have been intriguing. Some neurons are activated during the expectation period before an externally imposed stimulus is presented. The activity of dopaminergic neurons is even more intriguing. Dopaminergic neurons of the pars compacta and mesencephalic tegmentum react to behavioral or environmental change (i.e., to unpredicted stimuli). If the stimulus can be predicted, the situation is different. For instance, in conditioning paradigms, dopaminergic neurons respond to an US at the beginning of learning trials. As the animal learns the task, the cells respond to a CS that cannot be predicted and do not respond to the US (Ljungberg et al., 1992). However, they will fire if the US is not presented at its previously predicted time interval. For example, if the stimulus appears earlier than expected, the neuron again fires. The anatomical and physiological data can easily be explained within the framework of the proposed theory. Closed neuronal networks are the substrate for the model (Fig. 11). Although a functional subdivision should not be completely identified with an anatomical subdivision, we may do so as a first approximation. Accordingly, the basal ganglia form a system that predicts the behavior of the controlled object, while the part of the cerebral cortex

K.V. Baev et al. / Progress in Neuro-Psychopharmacology & Biological Psychiatry 26 (2002) 771–804

Fig. 11. Structural and functional organization of the ‘‘skeletomotor’’ cortico-basal ganglia – thalamocortical loop. The loop is a control system that has a model of the controlled object. APA: arcuate premotor area. CO: controlled object. GPi: internal segment of globus pallidus. MC: motor cortex. PM: premotor cortex. SC: somatosensory cortex. SMA: supplementary motor area. SNr: substantia nigra pars reticulata. TN: thalamic nuclei.

included in the closed loop is considered the controller. At this hierarchical level, specific complex neural networks like the basal ganglia evolved to predict the behavior of the controlled object. The cortical areas included in the open loop can be conceptualized as detectors of complex features of the controlled object providing the informational context for the controlling system. The error distribution system within the skeletomotor cortico-basal ganglia– thalamocortical circuit includes the dopaminergic neurons of the substantia nigra pars compacta and mesencephalic tegmentum. As expected from this scheme, the error distribution system is not activated when the behavior of the controlled object is predicted correctly. The body of the animal and the environment are the controlled object for the skeletomotor loop. As part of the controlled object, the environment can be complex and includes static and dynamic objects. The latter can be passively and actively moving objects like other animals. An animal’s survival depends on how effectively its internal model predicts the behavior of such objects. A predator must have a model of its prey to hunt successfully, and prey must successfully predict the behavior of a predator to avoid being caught. A person driving a car must have models of surrounding objects — of the road, pedestrians, cars, and so on. Such objects are incompletely observable and incompletely controllable. According to the theory, the model uses the language of afferent signals that enter the system to predict the behavior of the object. For the spinal NOCS, it is the language of

785

peripheral afferents. At the level of the basal ganglia – thalamocortical circuit, the situation is different. Lower NOCSs are responsible for different motor automatisms, and the problem of basic movement coordination is solved at those lower levels. Various motor control levels, such as initiating systems of the brainstem, cerebellum, and even other cortical levels, are controlled objects for the corticobasal ganglia – thalamocortical loop. In addition, the cortical area to which the basal ganglia project their signals receives inputs from other cortical areas. Therefore, various complex cortical detectors (including complex ones) send signals to this area. These detectors determine what values are relevant to parameters describing the state of the object or its parts at any given moment. The model also predicts the behavior of these detectors. The skeletomotor cortico-basal ganglia – thalamocortical loop is a hierarchical system with numerous subloops subordinated to one another. The higher the subloop in the hierarchy, the more abstract are the parameters processed by the loop. The relative position of various objects, direction, and speed of an object’s movement are typical examples of variables encoded at the level of skeletomotor loop. Each subloop must receive corresponding afferent inputs to function properly. Consequently, all types of afferent information must be processed by different cortical regions before afferent signals arrive in the corresponding subloop. For instance, the information from the skeletomotor subloop that supplies the motor cortex does not intermingle directly with information from the cerebellum. Cerebellar projections go to the region located between the motor and sensorimotor cortices. The model for which the basal ganglia – thalamocortical circuit is the substrate is used in two ways. First, it is used during execution of a movement to predict the transition to a new state. Second, it is used during the planning of a movement that needs a full-scale model because afferent information about future states is lacking. It is hard to imagine this process without using a model. Moreover, a cause –effect model is unconstrained by real time and may function at rates faster than real time (e.g., rates necessary for rapid multistep planning). Using the model without the constraint of real time requires efferent and afferent channels to be turned off until a correct decision is found. At this level, functional deafferentation or deefferentation can be achieved with various inhibitory mechanisms. This model must allow very rapid tuning to an object. Environmental changes occur quickly, and the model must immediately account for any changes. Therefore, dopaminergic learning should happen quickly. Many strategies can be used to accelerate learning in biological networks. Very complex network, neurochemical, and genetic automatisms could underlie the rapid learning that occurs in the basal ganglia. Numerous mediators in the basal ganglia are indirect evidence of this contention. Rapid learning also must occur in the cortical regions included in the skeletomotor cortico-basal ganglia –thalamocortical loop.

786

K.V. Baev et al. / Progress in Neuro-Psychopharmacology & Biological Psychiatry 26 (2002) 771–804

This reasoning helps to explain the circuitry in the basal ganglia and suggests future research. As shown, the system at the higher level must ‘‘jump’’ from one state to another while performing controlling functions. The optimal (easiest) way to create such a system is to build it on the basis of pacemaker neurons or neurons possessing bistable properties. Circuit triggers also should be used to build the system. 4.2. Other loops The theory can now easily be applied to other corticobasal ganglia – thalamocortical loops. The skeletomotor loop provides a good example of the types of operations that cortico-basal ganglia – thalamocortical loops can perform. Based on this knowledge, the theory can be applied to other loops by determining the controlled object for each loop. For the oculomotor loop, this task is easy, while it is more challenging for the limbic loop. Analysis of the limbic system, however, clarifies which new functions were added to the highest brain levels during the evolution of the prefrontal loops. 4.2.1. Oculomotor loop An animal obtains a substantial portion of sensory information about the environment through vision. Visual information occupies a unique place among all the information that an animal receives from the environment through its sensory receptors. It provides an animal with the most detailed information about distant environmental objects. At each moment, however, the visual field covers only a small portion of the environment. This feature determines how the visual system can receive more complete information about environmental objects: Eye, head, and body movements are necessary components of the visual process. A complete picture of the environment results from synthesizing the separate visual information obtained during multistep scanning of the surroundings. Visual information processing needs a powerful controlling system. As a sensory system, it must include a hierarchy of detectors. This system includes visual detectors of various complexity, and this complexity progressively increases as information is transferred from visual receptors to cortical levels. A hierarchy of detectors is a prerequisite for object recognition. What happens to such a hierarchical system of detectors when an object moves in the visual field either by itself or because an animal scans the environment? Such a system loses its capacity for recognition unless changes in visual information are predicted, in other words, unless the visual controlling system possesses a model of object behavior and uses it to predict future positions of surrounding objects in the visual field. Such a controlling system can properly tune visual detectors during eye movements or autonomous object movement in the visual field, and it can easily distinguish between these two situations. It is impossible to imagine how, without the model, the

controlling system can detect a new object that appears in the visual field. The oculomotor cortico-basal ganglia – thalamocortical loop, its subordinate oculomotor and head movement automatisms, and the hierarchy of visual detectors are included in the system that controls visual scanning of the environment. The above reasoning shows why a complex model of object behavior is necessary at the level of oculomotor loop. It also helps to imagine the complexity of computational processes at this brain level and how fast learning processes must be for network parameters to be tuned properly. 4.2.2. Limbic loop The limbic (or anterior cingulate) loop is a part of the limbic system. To apply the theory to the limbic loop, we must briefly analyze functions of other parts of the limbic system, that is, their automatisms. The term limbic system usually designates the following anatomically interconnected structures: the hypothalamus, amygdala, hippocampus, septal area, and cingulate gyrus. From a functional perspective, combining these structures into one system, the limbic system, can be justified because ‘‘many, if not all, of the effects produced by stimulation and lesions of the extrahypothalamic limbic structures can be replicated by stimulation or lesions of the hypothalamus’’ (Isaacson, 1982, p. 2). The limbic system is hierarchical. The hypothalamus and the amygdala are the lowest levels, and the hippocampal formation is the next level. In the scientific literature, the term hippocampus is used to designate the structures consisting of the hippocampus and the dental area, while the term hippocampal formation is used to designate the hippocampus proper, the dentate gyrus, and the transitional cortical areas connected with both the hippocampus and neocortical areas. For a functional approach, only the second term makes sense. For simplicity, however, the first term is used to designate the hippocampal formation. Finally, the cingulate gyrus and its basal ganglia– thalamocortical loop are the highest hierarchical level of the limbic system. The limbic system is responsible for so many functions that it seems to many to be almost impossible to propose a reasonable theory that could explain its versatility. The limbic system controls activities essential for self-preservation, including emotional activities (e.g., feeding and aggression, behaviors often accompanied by emotions), activities essential for the preservation of the species (e.g., mating behavior, procreation, and care of young), visceral activities associated with both of the above and numerous other activities of the hypothalamus, and mechanisms for memory. The lowest hierarchical level of the limbic system, the hypothalamus, is often designated as a ‘‘head ganglion’’ of systems directed toward maintaining a favorable internal environment through the regulation of the internal organs and through the organism’s overall behavior. The principle of control at the hypothalamic level is well demonstrated by

K.V. Baev et al. / Progress in Neuro-Psychopharmacology & Biological Psychiatry 26 (2002) 771–804

the following experiments. Stimulation of some hypothalamic regions evokes changes in visceral functions such as heart rate and blood pressure. Other automatisms with a significant somatic component also can be evoked by electrical stimulation of various sites in the hypothalamus. These elicited behaviors are orientation reactions, locomotion, rage reactions, licking from a water bottle, eating a food pellet, drinking water from a dish, or a sexual or aggressive act. Such behavioral acts are usually described as actions that are not directed toward motivational alleviation (e.g., to obtain food or water). They seem to be patterns of behavior elicited without regard to motivational satisfaction. Based on these experimental observations and the principle of hierarchy (Section 3.3), the hypothalamus can be considered a region with numerous command NOCSs for subordinated visceral and somatic automatisms. Subordinated NOCSs may be located in the brainstem, spinal cord, other brain regions, and even in the hypothalamus. In the case of autonomic regulation, the principle of control is no different from somatic regulation. The only difference between somatic and autonomic automatisms is the speed of the control process. Autonomic automatisms are often much slower than somatic automatisms. For instance, the dynamics of blood pressure regulation are slower than those of fast limb movement. Hormonal and other types of humoral regulation such as osmotic regulation and seasonal changes exemplify even slower processes. The brain itself, as a part of the body, also needs a complex control system responsible for its trophic regulation. Analysis of hypothalamic-initiating signals clarifies the nature of motivation and emotion, phenomena that remain a mystery. These phenomena are a natural consequence of the abstraction of parameters in hierarchical controlling systems. According to the theory, a controlling system has many sources of initiating signals. This statement also is true for hypothalamic NOCSs, and one source of their initiating signals is the subordinated control systems. For the higher command system, such signals mean that the lower control systems have exhausted their controlling resources, are incapable of performing proper control, and require a controlling influence from a higher level. Initiating signals also can originate from specific detectors located at the level of hypothalamic NOCSs or from a higher system such as the neocortex. Due to initiating signals from higher levels, lower level automatisms may be included in more complex behavioral repertoires. In the system of osmotic regulation, for example, a detector of osmotic pressure generates a signal if the system cannot support the necessary parameters of its task without more water. In other words, the animal has to find water and drink it to restore osmotic balance. In actuality, this signal can initiate a complex series of behaviors. In the physiological literature, this type of signal is referred to as a motivational signal to emphasize that it evokes long-lasting complex behaviors that differ from the simple reaction of a lower level control system to an initiating signal (e.g., flexor

787

reflex evoked by noxious stimulation of an extremity). Formally, there is no difference in principle between the initiating signals of lower and higher levels. Regardless of the level involved, an initiating signal must be minimized during control. At higher levels, however, much more time may be needed to reach a control objective. By their nature, initiating signals persist longer at higher levels before they are minimized. The internal, usually seasonal, changes that initiate complex forms of mating behavior are good examples of extended initiating signals. Emotional, visceral, and somatic components can be executed at the level of the hypothalamus and the brainstem by using visceral and corresponding somatic automatisms. An example of aggression explains why emotional automatisms include both visceral and somatic components: Internal resources must be mobilized for the high level of physical activity necessary for a potential fight or an effective escape. Like any lower automatism, emotions can be incorporated in more complex behavioral repertoires generated by higher levels. In many cases, emotions therefore require higher control levels to create their corresponding feelings and to include the adequate expression of emotions in complex learned behaviors. Emotions should be considered a part of the evolutionary development of complex automatisms and corresponding detectors to make intra- and interspecies communication possible. Communication functions as a language of gestures, specific vocalizations or sounds, and signal molecules that reveals an individual’s intentions to another animal of the same or different species. Different gestures can express aggression, demonstrate peaceful intentions, signal danger, and so on. To understand this concept, some possibilities must first be considered. A sensor performing an integral measure of aversive input signals developed during the course of evolution. What does the presence of such a sensor mean for an animal? Potentially, the animal will be able to find a state that minimizes this aversive input signal. Second, a sensor that performs an integral measure of rewarding stimuli such as tasty food, some olfactory stimuli, comfortable surrounding ambient temperatures, and so on also evolved. The animal will thus perform different behaviors to maximize the activation of this sensor. The state in which this signal is maximal can be designated as pleasant. Stimulation of the anatomical region associated with activation of this sensor might result in pleasurable reactions. In principle, such detectors are no different from complex detectors responsible for the initiation of mating or other behavior. Activation of these detectors can be considered a result of hormonal influences on specific structures within the neural network that produce the necessary initiating signals. Numerous other specific detectors also must be involved in the control of mating behavior. The hypothalamus is a good example of hierarchical principles (Section 3.3). Any hierarchical network level must evolve new types of detectors to acquire new behavioral

788

K.V. Baev et al. / Progress in Neuro-Psychopharmacology & Biological Psychiatry 26 (2002) 771–804

abilities. At each hierarchical level, the type of control and its corresponding detectors also must match. The abstraction of parameters encoded at the appropriate hierarchical level is a prerequisite for such sophisticated improvements. In many cases, visceral automatisms must use somatic automatisms to minimize initiating signals ‘‘born’’ within visceral controlling systems. For instance, different motor automatisms must be activated to ‘‘remove’’ the feeling of hunger. An animal must activate locomotor automatisms to find, catch, and incorporate prey. In contrast, somatic automatisms do not need visceral automatisms to minimize their own initiating signals. Although somatic automatisms usually activate visceral automatisms (e.g., the increase in ventilation and heart rate that accompanies locomotor activity), the visceral automatisms are supportive but unnecessary to achieve the goal of the control. Another feature is related to cortical automatisms used by the limbic system. For example, the skeletomotor cortico-basal ganglia –thalamocortical loop possesses enough power to control the complex motor behavior of the animal by using lower motor automatisms. However, one aspect of control was glaringly absent — the external initiating inputs to this controlling system. The control repertoire of a system that lacks access to such initiating inputs is limited. Obviously, there should be a controlling system that should determine what the skeletomotor controlling system must do at each step of control. This controlling system is the cingulate corticobasal ganglia– thalamocortical loop. The surrounding cortical and subcortical automatisms are the controlled object for the limbic system. Through them, the limbic system controls an organism’s body and the environment. A designation of the body that includes visceral systems, the brain itself, and so on as a controlled object is understandable, but why is the environment considered a part of the controlled object? During control at the level of the hypothalamus, food, water, prey, and sexual partners are part of the controlled object. Social behavior requires the presence of other members of the same species to become a part of the limbic controlled object. An individual’s sexual activity must be synchronized with that of other individuals of the same species and with environmental changes (e.g., seasons). The environmental aspect of the controlled object may have a very low level of controllability, and considerable time is usually needed to minimize some initiating signals. Sometimes, the initiating signal is never minimized, for example, when an animal fails to catch its prey or to find a sexual partner. The hypothalamus became the principal motivating source for the evolution of the hippocampus, primarily to access a richer informational context from complex cortical detectors to better solve the problem of coordination and to better cope with space and time. Once evolved, the hippocampus began to play the role of a ‘‘visceral cerebellum’’. To understand this idea, consider the specificity of two other control functions performed by the limbic loop: coordination and long-range orientation in space and time.

In the brain, the problem of coordination exists wherever numerous automatisms need to be synchronized in space and time. Recall (Section 3.3) that this problem can be solved by interconnecting all possible pairs of controlling systems if relatively few systems need coordination. Alternatively, a coordination center (e.g., the cerebellum) must receive input from all control systems of lower levels. It needs a rich informational context from higher level detectors that can respond to discrete features unavailable to lower levels. Finally, it must send coordination signals back to all levels. The coordination problem also exists for the limbic system. Numerous hypothalamic and brainstem autonomic and somatic automatisms need to be coordinated. Some visceral automatisms are extremely sophisticated. Therefore, the necessity within the limbic system for a structure that plays the role of a coordinator for hypothalamic automatisms — or ‘‘limbic cerebellum’’ — becomes obvious. As will become clear, the hippocampal formation functions in this role. Compared with the cerebellum, however, the ‘‘limbic cerebellum’’ had to evolve a new ability. Because working time intervals are much longer at this level and changes in system states are much slower, a much longer prehistory had to be stored. Hence, the system must be able to memorize a long trajectory in its discrete working space, a series of states that precedes the arrival of a mismatch signal. In this case, the system becomes capable of generating control influence in advance to minimize error signals. The mechanism of memorizing these ‘‘state’’ sequences also can be used for orientation in space and time. An animal cannot perform purposeful movements in its surrounding environment and minimize corresponding initiating signals if it cannot solve navigational problems. The ability to cope with space varies significantly among species. To facilitate understanding of the new controlling abilities that must evolve at the level that controls the somatic and autonomic automatisms, consider the evolutionary scheme to improve navigation in space and to extend the range of time control. Evolutionarily, controlling systems would not develop separately because different automatisms develop in synchrony. For this discussion, however, assume that an animal already possesses somatic and autonomic automatisms but lacks an automatism for navigation. The starting point for such an animal is random searching using random movements in the environment. This search pattern can be considered exploration of the environment. In the first stage, an animal moves randomly in its environment without memorizing the trajectory of its movements and determines the gradient of a limited number of signals such as smell, light, sounds, signals from distant tactile receptors such as antennae, and so on. If the animal possesses a knowledge — whether genetically determined or acquired by classical conditioning — of correlations between the signals used to compute the gradient and mismatch signals, it can potentially find a rewarding place or avoid

K.V. Baev et al. / Progress in Neuro-Psychopharmacology & Biological Psychiatry 26 (2002) 771–804

dangerous regions in the future by moving along or against the previously determined gradient. The use of a gradient can facilitate coverage of a large territory. Such an organism will survive because it fits certain ecological niches. For many animal species, this is the only possible behavioral strategy. In the second stage, reality is more complex. Not all environmental spaces can be described by a gradient because the spatial topography is too complicated (e.g., a labyrinth). A more effective strategy consists of memorizing the sequence of states that preceded the minimization of an initiating signal. If the animal reaches one of the sequential states during the next epoch of random searching, the final state when the initiating signal was minimized can be found again by repeating the rest of the sequence. The controlling power of such a system depends on the length and number of stored sequences (i.e., on memory capacity). If its memory capacity is large enough, the animal can execute complicated behaviors. The only prerequisite for this improvement in navigational capabilities is the development of detectors describing space. In the third stage, the system acquires the ability to make temporal and spatial shortcuts (i.e., to optimize trajectories of movements in space). Theoretically, simple shortcuts are performed during the second stage. For instance, if the system repeats part of a sequence during exploration, the repeated part can later be excluded. A primitive or temporal shortcut is thereby formed. To make more complex shortcuts, the system must possess a complex model of the surrounding space in which all previous experience was accumulated. Such a model ‘‘knows’’ which space transitions are ‘‘permitted’’ and which are ‘‘prohibited’’. Therefore, the third stage needs some form of spatial representation, a model of space. In the fourth stage, the ability to work within an abstract space is acquired (Section 4.2.3). This capacity is the result of further abstraction of a network’s parameters and the use of earlier evolutionary features: the ability to memorize sequences of states and to optimize trajectories of ‘‘motion’’ in such spaces. Recall that control processes at the level of the limbic system develop slowly. Therefore, long intervals should be encoded and stored at this level. Different neural mechanisms can be used for this purpose. For instance, brief time intervals can be counted by using an integration function of a constant signal. To compute long intervals, the system can use circuit triggers or other circuitry and cellular mechanisms such as pacemaker neurons, specific membrane properties of neurons, and so on. Based on these general considerations, consider how each limbic hierarchical level adds new controlling abilities to the system by using the available surrounding visceral and somatic automatisms. Each level has its own control specificity and semantics of afferent and efferent signals. What hypothalamic hierarchical level is capable of computing if it has only its own initiating signals and informational context without interacting with the hippocampus, cingulate gyrus, and inputs from complex cortical detectors?

789

These initiating and informational afferent inputs to the hypothalamus come from lower automatisms and from complex brainstem detectors such as those necessary for visceral regulation, as well as for olfactory (from amygdala), visual, acoustic, vestibular, and tactile input. As a ganglionic structure, the hypothalamus has limited computational abilities. It can execute subordinated basic behavioral automatisms (e.g., feeding, avoidance, mating, expression of emotions) in response to initiating signals, and it can perform limited coordination functions similar to intraspinal coordination functions such as those necessary for crossextension of homologous limbs during walking. The abilities of the hypothalamus to cope with navigation in space also is limited. It can localize a source of light or sound and perform only simple spatiotemporal behaviors. At most, it can determine the gradient of signals from distant receptors by stimulating random locomotor movements to explore the environment. This process can help the animal to locate its prey, to avoid an enemy, or to find the borders of a marked territory if special signal molecules that trigger smell or other signaling methods are used to establish territorial boundaries. However, it is difficult to imagine a circuitry in the hypothalamus that can memorize sequential spatiotemporal parameters while an animal explores the environment. To understand the functions of the hippocampus, consider the analogy between this structure and the cerebellum. Cerebellar cortical structure is the result of the necessity of providing each cerebellar module with a rich informational context. The same is true for the hippocampus, which also has a cortical structure. In addition to the informational context available from the hypothalamus (i.e., information about the state of lower automatisms), the hippocampus receives information from various complex cortical detectors. The detectors provide the system with detailed information about the state of the body and the position of external objects in space and time. The presence of cortical detectors implies that they are tuned to specific features present and accessible in the afferent flow. Therefore, efferent projections from the hippocampal formation to widespread cortical regions have a logical explanation. The hippocampus tunes the diverse cortical detectors by using those connections. In turn, the control task for the hippocampus is determined by the initiating signals from the hypothalamus, brainstem, and cerebral cortex. The hippocampal error distribution system, which includes the septum, shares several functional similarities with the olivary system that serves the cerebellum. The septal nuclei project to the hippocampus and receive inputs from lower NOCSs, for instance, from the hypothalamus, amygdala, and brainstem nuclei. Like the cerebellum, this septum receives feedback from the hippocampus. Finally, like the cerebellar olive, the septum receives inputs from higher cortical levels such as the prefrontal cortex, and projections may be present from the cingulate gyrus to the

790

K.V. Baev et al. / Progress in Neuro-Psychopharmacology & Biological Psychiatry 26 (2002) 771–804

septum (Brodal, 1981). These higher cortical levels can create more complex minimization criteria. During learning, septal signals must be minimized. By using its available informational context, the hippocampal circuit learns to compute the controlling output necessary to minimize septal mismatch signals. Any novelty in the behavior of the object activates the septal neurons. The cerebellum loses its function if the inferior olivary complex is lesioned. After septal lesions, the effects are similar in the hippocampus. Behaviorally, animals with septal lesions that are trained in avoidance tasks appear similar to animals with hippocampal destruction. Both types of animals show an enhanced rate of acquisition of a twoway avoidance task and impairment in the acquisition of the one-way avoidance task. The changes in learning these tasks in animals with septal or hippocampal lesions are always relative to the performance of normal intact animals (Isaacson, 1982). Therefore, one can hypothesize that the hippocampus is the structure that memorizes temporal sequences of various states in order to perform its control function. Such states can include visceral states, as well as states encoding environmental spatiotemporal relationships. The hippocampal circuitry possesses recurrent connections, and this attribute is used in numerous network simulations. Such recurrent networks can memorize sequences and perform temporal shortcuts (Levy, 1989; Levy et al., 1995). The cerebellum receives model and real afferent information flow (Section 3.3). It is thus possible to propose that the hippocampus also receives these two types of information. If so, the hippocampus can compute mismatches between real and model afferent flow (i.e., react to novelty in afferent information). The cortex is provided by the cerebellum with the knowledge of movement trajectory. The same is true for the hippocampus. The latter sends its output signals to lower NOCSs, cortex, cingulate gyrus, and prefrontal cortex. From a formal perspective, the cerebral cortex is provided knowledge of any possible ‘‘movement’’ trajectory determined by the hippocampus. Subsequently, the cortical level can optimize this trajectory by using optimization criteria unavailable at the hippocampal level. The result of this optimization is the ability to perform spatial shortcuts. The foregoing discussion clarifies the hippocampal mechanisms used for recent memory (see below). Based on this conceptual description, complex network, cellular, and molecular automatisms must underlie the ability to memorize sequences. Hence, experimentally found, long-lasting changes in such networks (e.g., longterm potentiation) are easily explained within the limits of the proposed concept. The highest hierarchical level of the limbic system comprises the cingulate gyrus and its basal ganglia –thalamocortical loop. This loop passes through the ventral striatum and possesses powerful predictive abilities. The closed loop includes the anterior cingulate area. A crucial

feature of any cortico-basal ganglia– thalamocortical loop is mutual functional interconnections among cortical regions that send their signals to the loop. The same is true for the ‘‘limbic’’ loop. The cingulate gyrus has mutual functional interconnections with all cortical regions that participate in the ‘‘limbic’’ loop. Thus, the cingulate loop can determine the functional control tasks for subordinated automatisms, including various cortical detectors. Besides the information it receives directly from subordinated automatisms, the cingulate gyrus also receives a model of what it should expect to receive from the loop. Certain experimental findings show the complexity of subordinated detectors. In freely moving animals, singleunit recordings have revealed place cells in the hippocampal fields CA3 and CA1 (Burgess et al., 1995). Their firing is restricted to small regions of a rat’s environment referred to as place fields. The firing properties of place cells can be manipulated by changing the relative positions of pertinent environmental cues. In more complex spaces such as mazes, the firing of place cells depends on the animal’s location and direction of travel. Spatially correlated cell firing also has been identified in the entorhinal cortex and dorsal presubiculum. Compared to CA3 and CA1 place neurons, the place or directional correlates of these latter cells tend to be more complex. Several connecting regions of the premotor areas, including the caudal cingulate motor area on the dorsal bank (CMAd), the caudal cingulate motor area on the ventral bank (CMAv), and the rostral cingulate motor area (CMAr), are located in the cingulate sulcus. These cingulate motor areas and the supplementary motor area connect with the primary motor and sensory cortices (Darian-Smith et al., 1993). Different premotor areas are also interconnected. Therefore, the cingulate gyrus possesses all the necessary informational inputs and outputs to perform control tasks for cortical and subcortical motor and other automatisms (Fig. 12). Portions of this informational input come from

Fig. 12. Functional organization of the limbic system. See text for explanations.

K.V. Baev et al. / Progress in Neuro-Psychopharmacology & Biological Psychiatry 26 (2002) 771–804

the brainstem level through the mediodorsal thalamic nucleus (MD), anterodorsal thalamic nucleus (AD), ventromedial thalamic nucleus (VM), and laterodorsal thalamic nucleus (LD). Finally, the initiating inputs available to the cingulate gyrus must be identified. These inputs may emanate from the anteroventral thalamic nucleus (AV). This nucleus receives inputs from error distribution systems such as the septum; the mammillary bodies, structures connected with the septum and hippocampus; and the hippocampus itself. However, some initiating signals likely reach the cingulate gyrus through other thalamic nuclei or directly from the subiculum because these initiating signals might be the result of information preprocessing in the hippocampus. The prefrontal cortex is also a powerful source of initiating inputs for the cingulate gyrus. In summary, the cingulate gyrus can (i) establish and initiate control tasks for the hippocampus, (ii) create the necessary informational context for the hippocampus by tuning complex cortical detectors, and (iii) create new minimization criteria for the hippocampus by influencing it through the septum (i.e., the analog of the inferior olive). Again, the latter possibility is not excluded because the cingulate gyrus may have connections with the septum (see Brodal, 1981). The prefrontal cortex (Section 4.2.3) also sends signals to the septum. The complicated criteria computed by a fast-learning subsystem mediated by dopamine learning provide a powerful source of informational and mismatch signals that eventually expands novelty and other unique criteria for the control system. Therefore, the cingulate gyrus with the limbic loop and the initiating and informational inputs described above reached a new level of complexity and abstraction of encoded parameters. For the cingulate gyrus, the hippocampus is a portion of its controlled object. Together with the hippocampus, the cingulate gyrus can calculate spatial shortcuts, store information about potentially permissible and impermissible space transitions, compute transitions by using gradients, and so on. Such features show that the limbic system effectively copes with its spatial environment. When the cingulate gyrus with its limbic loop memorizes an optimal trajectory (i.e., a shortcut), the detailed knowledge of the sequence of states memorized by the hippocampus is unnecessary. This reasoning accounts for the role of the hippocampus in recent memory. The abilities to store a long prehistory and to make shortcuts qualitatively changed learning at the level of the limbic system. In the case of classical conditioning, information from memory can be used when the interval between the conditioned event and the arrival of an initiating signal is long. A learned control command can initiate complex subordinated automatisms. If an entire stored sequence becomes associated with an initiating signal, it also becomes the basis for operant learning. If the first event of the memorized sequence is encountered again, the system can slide to the desired state by repeating the same set of

791

consecutive states. Therefore, the latter situation can be considered primitive operant learning. The power of such learning depends on the length and number of the stored sequence and on the available automatisms that can minimize corresponding initiating signals if activated. More complex types of operant learning require temporal and spatial shortcuts like the ones evolutionarily complex animals perform. For instance, if an animal finds a place where it can avoid punishment, in the future, it will go directly to this place independent of its starting point. 4.2.3. Prefrontal loops The best way to understand the role of prefrontal loops is to understand the evolutionary need for this control level. The spectrum of available initiating and informational inputs used by the limbic system is the source of its computational and functional limitations. As described, minimization of initiating signals born within the hypothalamus is sufficient to meet the principal inborn needs of an organism. The same is true for the informational context from the complex detectors used to solve the problems of visceral control and navigation because they are tuned to particular object features that coincide with the needs of a given control task. Section 4.2.2 implied that cues present within a space to be explored do not change (i.e., the surrounding physical environment is static). Animals in a dynamic environment must have a knowledge of environmental dynamics in the form of the internal model. The more complete the model is, the more adaptive the animal’s behavior will be. The highest level control problems are not always solved by networks that function as reactive systems although it is the final stage of mastering an automatism in many situations. As mentioned, a complex control task needs recursive computations that cease once its specific criterion is satisfied. The creation of new minimization criteria requires an elaborate, full-scale model and depends on the quality and completeness of the model. In such a system, if the level of abstraction of the parameters became sophisticated enough to ‘‘move’’ abstract objects in an abstract state space, a new functional feature would exist: The process of elaborately detailed multistep planning would be possible (i.e., the capacity for thought). One way to achieve a new level of control abilities is the addition of hierarchical levels — prefrontal loops. These levels must be able to use hippocampal circuitry to store the results of the intermediate computations that transpire during learning. An additional hippocampal subdivision dedicated to serving the needs of this level also is needed. Eventually, to some degree, these new levels should be a hierarchical extension of the anterior cingulate gyrus — a system that uses rapid dopamine-mediated learning. From this evolutionary process, the following features were acquired: (a) an increase in the number of intermediate computational steps that might be necessary to minimize a visceral initiating signal and (b) the ability to

792

K.V. Baev et al. / Progress in Neuro-Psychopharmacology & Biological Psychiatry 26 (2002) 771–804

create new, more complex, and sophisticated classes of initiating and informational signals to direct the control tasks of lower levels. These new initiating signals can be conceptualized as higher-order derivatives of fundamental visceral initiating signals and as capable of initiating complex behaviors. The fundamental initiating signals themselves continue to localize their origins in visceral control systems. For instance, a human can exhibit complex forms of social behavior to satisfy his need for food, pleasure, and so on. Additional features were acquired during this expansive evolutionary process: (a) the ability to tune cortical and other detectors to any desirable criterion; (b) memory processes capable of storing the most precise and complete model of an environment’s most abstracted cause –effect relationships so that random searching is replaced by sorting all possible variants of a control task; (c) the capability of processing any environmental novelty at this level; and (d) the ability to function independently of real time so that this level can actively perform computations while lower levels control current behavior. In biological neural network computers, an effective description of environmental cause – effect relationships requires a perpetual ascension within the system as parameters are abstracted. This phenomenon was probably the basis for the evolution of language itself. Physically adding new hierarchical levels to create new abstractions has obvious limitations. In humans, language evolved to solve this problem. Language became a universal tool for creating new abstractions because new abstractions could be constructed by implementing new symbolic concepts or words. The state space at the level of the prefrontal loops is discrete. Theoretically, they can process objects such as words. From this point, the power of an intelligent biological network computer in the form of an individual or society began to rely heavily on external memory mechanisms. The capability for accumulating knowledge was otherwise limited by the human life span. Initially, the role of external memory was performed by storytelling in an individual’s social group. With the development of writing, the social group acquired an unlimited ability to store information in a collective external memory. From this point, the evolutionary process in human societies depended on the effectiveness of access to information contained within this collective memory.

5. Pathophysiology of NOCS If NOCSs are considered functional blocks of the brain, the pathophysiology of neurological disorders can be reinterpreted. Here, however, only general considerations about basic regularities of malfunctions in NOCS are presented. What types of dysfunction appear if pathology exists in different functional blocks of NOCS? The term pathology is used in its most general meaning and defines any abnormal

changes in the NOCS. These changes can be neural death or malfunction and abnormal synaptic transmission caused by trauma, infection, toxins, genetic defect, aging, and so on. Damage to informational context. In the case of a diffuse and mild destruction of informational channels, a system would function, but its resolution would be affected adversely. The precision of movements, for example, would decrease. Severe diffuse destruction may lead to the loss of the function because a large decrease in precision precludes normal function. Total deafferentation also causes the loss of function. NOCS is a learning system. In the absence of afferent flow and, consequently, error signals, an internal model eventually loses its capacity to predict future states of the controlled object. How fast it happens depends on the speed of learning processes in NOCS. If a specific informational input (e.g., visual, acoustic, from other parts of the body) is destroyed, the system loses its ability to predict the behavior of the object that corresponds to this type of informational context. For instance, if the part of the model describing the behavior of a specific body part receives no informational context from another body part, these parts cannot be coordinated. All other damage (between mild and total) to informational channels leads to intermediate dysfunction. When the system lacks part of the informational context that can be used to calculate the necessary controlling signal, the model is relearned (i.e., retuned to the new situation). Damage to initiating system. Such damage causes signals initiating a specific automatism to be lost partially or completely. The impairment of the corresponding automatism depends on the scale of the damage from partial to complete loss of function. At each control step under normal circumstances, an initiating system sets a target state for the controlled system. The target states cannot be set properly when parts of the signals do not arrive in the controlled system. If the controlled system is an internal model, the model loses the capability to describe properly the behavior of the controlled object. The modeling network does not slide down along the gradient of ‘‘error’’ signal to the optimal state corresponding to the minimum ‘‘error’’ signal because it does not receive the necessary error signals. The system will be in one of the local minima located far from the necessary global minimum (Fig. 13). If the model’s prediction approximates the real state, the controlling system likely functions well. If not, the controlling system behaves as if the controlled object had been perturbed by an external, unaccounted force. During the next step of control, it will try to adjust the state of the object. This search for a correct control can continue for a long time and cause specific symptoms, depending on the object controlled by the system (Section 5.1). The system cannot function when the prediction is wrong or even unreal (i.e., when the error distribution system is destroyed). For example, the lesion of the inferior olive that is a part of the error distribution system of the cerebellum leads to symptoms similar to those associated with a cerebellectomy.

K.V. Baev et al. / Progress in Neuro-Psychopharmacology & Biological Psychiatry 26 (2002) 771–804

Fig. 13. An example of an error surface profile. I: intensity of an error signal. S: state of the controlled object.

Theoretically, a neural network calculating mismatch signals could begin to function abnormally. Several scenarios are possible when these signals arrive in a modeling network. In one case, a new function for minimizing this abnormal initiating signal can be calculated by the model. Therefore, in the final stage of the learning process, the model network starts to calculate this incorrect function. The expression of pathological symptoms then depends on how far the predicted state varies from the actual state of the object. In another case, no function minimizes the mismatch signal. For instance, the comparing device generates an output signal without considering input signals. A persistent search of the system in its state space and therefore the destruction of the model result. With time, the neural network also may partially or completely reject such an input about error that it cannot minimize. Similar reasoning applies when a mismatch signal is used to tune a network responsible for control law. Pathologic afferent flow produced by any other subsystem of the brain begins to arrive in the normally functioning controlling system. If this pathologic flow is correlated with other afferent signals, the learning process will stop when the model starts to describe properly this pathologic afferent flow and its correlations (association) with other components of normal afferent flow. This process is called ‘‘mastering’’ pathology. In this case, any associative afferent flow can provoke pathological behavior. Damage to the controller or modeling network. Such damage leads to a loss of computational abilities that depend on the amount of damage to the systems. If the controller is damaged, the control signal is incorrect, and the controlled object will not move properly in its state space. If the modeling network is damaged, prediction of the object state will be wrong. At the same time, slight damage to a network results in slight changes in its computational capabilities. When a neural network loses its elements, it can still function but with less precision. We defined this feature as holographic (see Section 3.2.3).

793

Optimal filter does not work properly. When model flow prevails, that is, when actual afferent flow is treated by the system as an unreliable source of information, the malfunction can lead to an illusion at the lower level. At the highest levels, model prevalence can be the basis for delusions. In all of the abnormalities described above, the neural networks lose their computational abilities, and similar symptoms can result. However, the underlying reasons for the loss can be different. Only computer simulations and comprehensive theoretical analysis will reveal the mechanisms underlying the symptomatology of various neurological disorders, and only a general scheme is described here. This modeling, however, must be functional as it follows from the theory (Section 6). Because of its inherent limitations, mechanistic modeling will be ineffective. Functional modeling implies that the correct controlled object, functional blocks, and optimization criteria are defined for any neuronal control system that is analyzed. Any model or computer simulation remains a simplified version of the analyzed system and provides a simplified analogy of an actual neuronal system. Compensatory neural network mechanisms. The inherent holographic properties of biological neural networks and their capacity to learn are major mechanisms that should be considered when any neurological disorder is treated (Section 3.2.2). With time, the network adjusts itself to a partial loss of its elements by invoking learning mechanisms. 5.1. Clinical applications of the theory: PD PD was chosen as an example because it is one of the most studied neurodegenerative disorders. Clinical studies and experimental data from animal models of parkinsonism have shown that the death of dopaminergic neurons of the substantia nigra leads to PD. Akinesia, muscular rigidity, and tremor are the main symptoms (Greene et al., 1992; Montgomery et al., 1991). The pars compacta of the substantia nigra is part of the error distribution system (Section 4). Therefore, PD is the consequence of malfunction of learning processes in the basal ganglia, a disorder of the error distribution system. Consequently, the model incorrectly predicts the state of the object in these patients (Fig. 14). Diminution in the number of dopaminergic neurons, the output neurons of the error distribution system, is equivalent to a decrease in the resolution of the error distribution system. This decrease also can be viewed as a decrease in precision of error description. Imprecise error signals make it impossible for the modeling network to realize its computational potential and leads to incorrect predictions. In other words, the modeling network could have generated precise output if it had been provided more precise error signals. Symptoms of PD reflect on the decreased precision of the error signals. The controlling system treats incorrect predictions as if the controlled object were perturbed by an unaccounted external force. It tries to adjust the state of the object during

794

K.V. Baev et al. / Progress in Neuro-Psychopharmacology & Biological Psychiatry 26 (2002) 771–804

Fig. 14. Predicted and actual probability of the controlled object to be in a particular state under normal and pathological circumstances. If the model functions correctly, the prediction coincides with reality. With Parkinson’s disease, prediction and reality do not coincide because the model is incorrect.

the next step of control. The search for a correct control causes specific symptoms, depending on the object controlled by the system. The lower level skeletomotor subloop, for example, may control a position of a body part. In this case, this part of the body will demonstrate tremor. This situation looks like overregulation, in which the controlling system always misses the equilibrium point. If a higher motor control level is involved, however, large amplitude involuntary movements may be observed. More complex explanations are needed for bradykinesia. In this case, the prediction of the model differs significantly from the real state of the object. The model also may predict several states with equal probabilities. Therefore, the system needs more time to choose among these states. One of the simplest analogies is the interaction of antagonistic reflexes at the spinal level. When receptive fields for antagonistic reflexes are stimulated simultaneously, it is impossible to predict which reflex will be evoked. The latent period of the evoked reflex is usually much longer than in the normal situation. Moreover, sometimes, none of the reflexes appear. Severe akinesia results when the model predicts an absolutely unreal situation. The proposed theoretical view illustrates that dopaminergic neurons of the substantia nigra can die for several reasons and that parkinsonian-like symptoms can be observed without substantial loss of dopaminergic neurons. First, dopaminergic neurons can be destroyed directly, as during 1-methyl-4-phenyl-1,2,5,6-tetrahydropyridine (MPTP) intoxication or from genetic vulnerability of these neurons to harmful environmental agents. These neurons also can be damaged secondarily when molecular (intra- and extracellular) mechanisms of learning (molecular automatisms controlling learning) initiated by dopamine do not work effectively. This scenario can be associated with pronounced parkinsonian symptoms without a significant loss of dopaminergic neurons. The activity of dopaminergic neurons will be much higher

than normal because the intensity of mismatch signals will increase as a result of improper tuning of the model. Overloading can cause the dopaminergic neurons to die, but their death would be secondary. Finally, dopaminergic cells also can die secondarily if the neural network calculating mismatch signal functions incorrectly, and error signals are present even when the model functions correctly. Such initiating signals stimulate dopaminergic neurons to no purpose. PD also is associated with secondary damage to dopaminergic neurons because any inaccuracy in the model, which is the case in PD, increases the intensity of mismatch signals. The mechanism of levodopa therapy can be conceptualized based on these ideas. The therapy could reflect an increase in the gain of the error distribution system. If the resolution of the error distribution system decreases, the amplification of error signals has both advantages and disadvantages. The progression of PD is accompanied by a gradual decrease in the resolution power of the error distribution system and an increase in the dosage of medication. Hence, levodopa can be effective until the decrease in precision and corresponding increase in dosage fail to attain certain levels. At this level, patients exhibit medically induced symptoms (e.g., dyskinesias) because overamplified error signals cause large adjustments in the model. As a result, the system jumps over the global minimum while trying to minimize error signals (see Fig. 13). When this stage of PD is reached, symptoms can be alleviated by decreasing the resolution of the model network (see below). A rather disappointing conclusion can be made about levodopa therapy. This therapy functions much like instigating a horse that cannot run. In parkinsonian patients, the error distribution system is already overloaded. Acceleration of the process of deterioration of the error distribution system is the patient’s price for temporary motor improvement. From this perspective, drugs whose mechanism of action is based on monoamine oxidase (MAO) or catecholo-methyl transferase (COMT) inhibition or those that improve molecular mechanisms of learning seem more promising because they will not overload the molecular automatisms responsible for synthesizing dopamine. The latter automatisms also may be responsible for the periodic depletion of dopaminergic depots, which leads to additional fluctuations in the symptoms of patients with PD. 5.2. Mechanisms of functional neurosurgical procedures Functional neurosurgery has developed three major treatments for PD — partial lesioning of basal ganglia structures, DBS of certain structures, and neurotransplantation. Attempts to explain the mechanisms underlying these functional neurosurgical procedures based on traditional theoretical interpretations create more questions than answers. The mechanisms underlying the efficacy of these procedures, based on the proposed theory, follow.

K.V. Baev et al. / Progress in Neuro-Psychopharmacology & Biological Psychiatry 26 (2002) 771–804

5.2.1. Partial lesioning of basal ganglia structures Empirical observations generated by functional neurological surgery have unearthed two puzzling problems related to partial lesioning of the basal ganglia. First, why does partial lesioning of various basal ganglia structures alleviate parkinsonian symptoms? The globus pallidus pars interna (Gpi), subthalamic nucleus (STN), and ventral oral anterior and posterior (Voa and Vop) thalamic nuclei are well-known targets for partial lesions. Evidence suggests that lesions placed in other basal ganglia structures (e.g., caudate nucleus and putamen) also can improve symptoms. After a neurotransplantation procedure, about 40% of the improvement in parkinsonian symptoms in monkeys can be attributed to their lesion. During a neurotransplantation procedure, multiple implantation tracks usually are made in the caudate and putamen. The external segment of the globus pallidus (GPe) is not considered an effective target for the lesion. In some cases, however, lesions placed in the GPe instead of the Gpi have been associated with good results, at least for a while after surgery. We have observed several such cases. We performed a second pallidotomy on the same side about a year after patients underwent their first pallidotomy elsewhere in which the lesion was mistakenly placed in Gpe instead of Gpi. Therefore, although some lesions are more effective than others, partial lesioning of any basal ganglia structure alleviates parkinsonian symptoms. The other problem is related to the observation that lesions outside the basal ganglia – thalamocortical circuit also alleviate parkinsonian symptoms. Many neurosurgeons consider the ventralis intermediate nucleus (Vim), a good target for placing a lesion during thalamotomy. The Vim receives kinesthetic afferent inputs from contralateral body parts and from the cerebellum. Why partial lesioning of this nucleus reduces or even eliminates tremor in parkinsonian patients is unknown. Both of these puzzles led to an even more profound enigma: How can destroying part of a network improve its function? When the basal ganglia network generating model afferent flow is partially destroyed, the network’s precision decreases. The probability distribution of possible predicted

Fig. 15. Effect of placing a lesion in the basal ganglia in the case of Parkinson’s disease.

795

Fig. 16. Effect of placing a lesion in both the basal ganglia and the network processing real afferent flow.

object states becomes lower and wider (Fig. 15), and predicted states partly overlap with possible real states of the object. Consequently, the system no longer finds an error in its prediction and does not try to correct the object position in its state space. A positive clinical effect is obtained at the expense of a decrease in the resolution of the modeling network. The probability distribution of possible object states has been considered unimodal for simplicity, but in a real situation, this distribution can have numerous peaks. In addition, the resolution power of the error distribution system and the resolution power of the modeling network should match. The latter exceeds the first in parkinsonian patients, but the match is ‘‘restored’’ after partial lesioning. The decrease in resolution depends on the location of the lesion in the network. In Section 4, the cortico-basal ganglia– thalamocortical loop was compared to a funnel. A lesion in such a system will have the greatest effect if it is placed in the narrowest part of the funnel (i.e., near its output). In fact, Gpi, STN, and thalamic nuclei are the most effective places for lesions. The situation is symmetrical when the network processing real afferent flow (i.e., Vim) is lesioned. The predictions of the model remain the same. What changes is the probability distribution of the possible controlled object states generated by the system processing the real afferent flow. The two distributions start to overlap, alleviating parkinsonian symptoms (e.g., tremor). In reality, a thalamotomy usually involves Vim, Vop, and, possibly, even Voa. Involvement of both the model network and the network processing real afferent flow decreases the resolution of both systems (Fig. 16). The final outcome is superior to lesioning only the modeling network or the network processing real afferent flow. An additional mechanism can be involved in alleviating parkinsonian symptoms when the STN is lesioned. This mechanism is related to excitatory feedback from STN to Gpe (Fig. 17). If this positive feedback is organized so that each STN neuron excites Gpe neurons that project on neighboring STN neurons, then the functional meaning of the feedback is similar to recurrent collateral inhibition

796

K.V. Baev et al. / Progress in Neuro-Psychopharmacology & Biological Psychiatry 26 (2002) 771–804

Fig. 17. Accepted views of skeletomotor basal ganglia circuitry. Excitatory and inhibitory synapses are shown by arrows and black circles, respectively. GPe: external segment of globus pallidus. GPi: internal segment of globus pallidus. STN: subthalamic nucleus. SNc: substantia nigra pars compacta. SNr: substantia nigra pars reticulata. PPN: pedunculopontine nucleus. Th: thalamus.

(Fig. 18a). In fact, excitation of inhibitory neuron inhibits the target neuron for the inhibitory neuron. Conversely, inhibition of inhibitory neuron disinhibits the target neuron; the result is functionally equivalent to exciting the latter. As mentioned (Section 4), the basal ganglia network should include neurons with pacemaker properties; otherwise, a network that includes inhibition of inhibitory neurons cannot function. Collateral recurrent inhibition in sensory systems works like a focusing (contrasting) mechanism (Fig. 18a). Therefore, the positive feedback from STN to Gpe may reflect the implementation of a similar mechanism in the basal ganglia circuitry (Fig. 18b). This mechanism regulates selectivity or the network’s precision tuning and can be adjusted by the controlling system. The latter concept explains the existence of corticosubthalamic projections. A partial lesion of the STN renders this contrasting mechanism less effective. Partially lesioning the STN is probably the most effective site to improve parkinsonian symptoms because it involves two mechanisms: the general decrease in the resolution power of the system and the decrease in selectivity that follows the lesion. The connections of the pedunculopontine nucleus, which send excitatory connections to the Gpi and pars reticulata of the substantia nigra, are reminiscent of those of the STN. Therefore, partial lesions must be considered as a treatment based on the holographic properties of biological neural networks (Section 3.2.2). Less than 10% of the total volume of a specific structure is usually destroyed. For the decrease in resolution to be effective, the error distribution system must preserve some functional capability before any lesion is placed in the modeling network. In other words,

predictions of the model should approximate reality. Otherwise, the two distributions will not overlap after the lesion, and symptoms will not improve. Placing a lesion within a NOCS network improves symptoms, but does the controlling system function normally? The answer is no! After a lesion has been placed, the system is effectively tricked and finds fewer errors in its predictions. Normal function of the controlling system, however, is not restored. Therefore, partial lesions of various structures in PD should be considered as palliative, symptomatic interventions. They are not curative because they do not stop the fundamental degenerative process. However, they can slow the disease’s rate of progression because of the mechanism described in Section 5.1. Overloading the error distribution system decreases after lesions are placed. From a theoretical perspective, preventing further dopaminergic cell loss or restoring the structural integrity of the controlling system (if possible; see Section 5.2.3) would be the most effective form of treatment for PD. 5.2.2. Deep brain stimulation Recently, DBS has been developed as a method of treatment for PD and other neurological disorders. The method was implemented broadly, without understanding how it works. As a method of treating PD, DBS is even more puzzling than lesioning. High-frequency (above 100 Hz) stimulation of the same structures (Gpi, thalamus, and STN) whose lesioning successfully alleviates parkinsonian symptoms produces an identical effect. Two possible mechanisms could underlie the efficacy of DBS. First, stimulation might functionally block regions immediately adjacent to the electrode tip. The mechanism

Fig. 18. Modification of neuronal activity profile by a layer of neurons with recurrent connections. (a) Recurrent collateral inhibition. (b) recurrent excitation of inhibitory neurons. For simplicity, recurrent connections of only one neuron are shown. A: neuronal activity.

K.V. Baev et al. / Progress in Neuro-Psychopharmacology & Biological Psychiatry 26 (2002) 771–804

underlying DBS would be similar to that underlying lesioning. Second, in addition to blocking, activation may be involved. If so, influences spread to other brain regions via both fibers passing through the stimulated region and axons of neurons excited during stimulation. There are no clear data about the activation effects of DBS (Ashby, 2000), but concrete data suggest that activation components of DBS do exist. In the operating room, stimulating lesion targets has long been known to alleviate symptoms. This effect is rapid and may involve activation rather than neuronal inhibition. Long-lasting trains of stimuli alter the excitability of axons in complex ways. Axon excitability may increase, leading to spontaneous discharges, or it may decrease so that the axon is less responsive to electrical stimuli. These activating influences do not provide the system with meaningful information for signal processing. Thus, the second mechanism simply introduces noise into the network, and an obvious question appears: Why does adding noise to such a system improve its function? Although the results of chronic stimulation are similar to the clinical results of placing a lesion, neural tissue is not immediately destroyed. Turning off the source of current produces a reversible type of functional lesion. Furthermore, adding noise effectively helps the system to slide down the error surface to its global minimum, much like shaking an uneven sloping surface helps a ball slide down to the lowest point. The compound result of these two mechanisms is shown in Fig. 19. In the proposed theoretical framework, introducing noise itself to the system can decrease the resolution of the system. Because high precision is impossible with a noisy background, the noise forces the system to reduce its resolution power. Such is likely the case when chronic stimulation involves a system that processes real afferent flow (e.g., Vim). Obviously, dual action of DBS is key to improving this method by enhancing one action or another. In the case of chronic stimulation, more noise is introduced into the

Fig. 19. Changes in prediction under chronic stimulation of some basal ganglia nuclei — GPi: globus pallidus pars interna, STN: subthalamic nucleus, Voa: ventral oral anterior nucleus, and Vop: ventral oral posterior nucleus.

797

system as the frequency of stimulation decreases, and vice versa — an enhancement of the blocking effect accompanies the increase in stimulation frequency. An important conclusion about future strategies for DBS can be made based on this frequency dependency. For each stimulated structure, there should be an optimal ratio between how much neural tissue is blocked and how much noise is added to the controlling system to obtain the maximum therapeutical effect. This optimal ratio depends on the structure and stimulation parameters. Future theoretical, experimental, and clinical research should be dedicated to finding these ratios. A functional block in one structure and adding noise to another also might be a future strategy for DBS, but several electrodes and more sophisticated stimulators will be needed. 5.2.3. Transplantation When degenerative loss of nigrostriatal dopaminergic cells was discovered to be the etiological basis of PD, the possibility of substituting them with transplanted dopamineproducing cells — whether of neural, paraneural, or transfected cell origin (see Marciano and Greene, 1992) — was embraced. Within the framework of the proposed theory, transplantation should be considered a way to restore damaged function by replacing a whole automatism (organ transplantation) or by implanting new structural elements (tissue or cellular transplantation) in the malfunctioning system with the hope that they will be integrated so that normal function can be restored. In neurobiology, only tissue and cellular transplantation has been studied intensively, especially fetal neurotransplantation. Transplantation of the whole brain or its entire parts remains science fiction. Neurotransplantation studies and transplantation experimental therapies in neurological disorders have produced modest results. The miraculous results occasionally reported, for example, the successful treatment of PD by transplanting adrenal cells into striatum, are rarely confirmed and are excluded from this discussion. Numerous studies of brain stem cells renewed interest in neurotransplantation. The underlying simplistic assumption is that transplanted tissue or cells will establish correct connections with the surrounding neurons. As a result, the damaged system would be rewired completely or almost completely and lost function would be restored. To better understand the prospects and limitations of neurotransplantation, consider the following discussion of the phylo- and ontogenesis of NOCS. Neurodegenerative disorders are emphasized because these disorders are the major target of experimental neurotransplantation treatments. Each animal species has evolved to fit its ecological niche optimally. Evolutionary solutions to constructing an animal are usually the best or almost the best based on a set of possible automatisms inherent in a given species. Here, the term automatisms is used in its broadest sense to define cellular, molecular, and other types. If evolutionary solutions are not good enough for survival in a changing

798

K.V. Baev et al. / Progress in Neuro-Psychopharmacology & Biological Psychiatry 26 (2002) 771–804

environment and a species cannot adjust quickly, it becomes extinct. This statement applies to any NOCS that is also built to optimally match the control needs for corresponding objects. To accelerate the evolutionary process, nature evolved numerous mechanisms, including aging and death. More exactly, numerous mechanisms could be responsible for aging. They have evolved so that individual life expectancy optimally matches evolutionary needs. Life expectancy must be optimal for a species to reproduce; the evolutionary value of an individual life is limited. The concept of aging is best illustrated by analyzing biological automatisms (i.e., biological optimal control systems and mechanisms that can make these automatisms less efficient with time). Assume that we can create an artificial life form or its computer model and want to accelerate its evolution by applying the concept of death. Based on biological optimal control systems (Sections 3 and 5), there are numerous means to do so. One strategy, however, seems the most efficient: gradual programmed deterioration of initiating systems at a specific time after birth. The speed of this deterioration determines the creature’s life expectancy. All other means appear to be less effective and hence not optimal. Aging likely represents an optimal solution. Aging makes initiating systems vulnerable to unfavorable environmental influences. Initiating systems, especially error systems, are used to adjust the controlling system to a new situation. Individuals with less efficient initiating systems will die sooner than those with more efficient initiating systems. Similar reasoning is applicable to NOCS. These systems also are affected by aging. The most serious dysfunctions of NOCSs happens when their error distribution system is damaged. PD was conceptualized as a disease of the error distribution system. Another neurological degenerative disorder, Alzheimer’s disease, frequently affects the older population. Recent data suggest that this disease is also a disease of the error distribution system. In Alzheimer’s disease, the neurons of the nucleus basalis undergo substantial degeneration. Together with dopaminergic and norepinephrine neurons, these neurons broadcast prediction errors as global reinforcement or teaching signals to large number of postsynaptic structures (Schultz and Dickinson, 2000). How would NOCSs be built during ontogenesis? Genetic automatisms are used to store phylogenetic solutions, and ontogenesis should be viewed as executing these automatisms to build an individual, including individual NOCSs. Ontogenesis is a multistage process. Each automatism is started by a corresponding initiating signal(s). These automatisms have their own hierarchy and should be coordinated perfectly in space and time. This simple conceptual description clarifies an enormous complexity of ontogenetic development and the uniqueness of some initiating signals and automatisms for this process. Most of these stages occur only once during ontogenesis and are never repeated. For example, at early stages of the development of the nervous

system, specific neural cells should receive initiating signals from target cells to establish connections with them. At this early stage, these cells are usually located near each other and become much more distant as the animal grows. If these connections fail to become established in the appropriate time window, they will not be established in the future. Gene knockout experiments and numerous other studies of birth defects support this perspective. Building a NOCS by executing genetic automatisms is a process of structural optimization that is followed by learning. The latter is needed to optimize parameters of the neural network. Genetic information does not account for all the environmental variables that an animal can encounter during its development. It is hard to imagine that genetic memory has this capacity. Consequently, learning processes (i.e., optimization of parameters) are important for the development of NOCSs. Structural optimization of the network works like an initial structural approximation of the network that allows it to compute specific classes of functions. Structural optimization is the most complex problem associated with creating a neural control system and mostly occurs during embryogenesis and some time after birth (before maturity). However, some minor structural optimization changes can occur even in adults. Without structural optimization, NOCSs could not be built so quickly during development. Without genetic information, structural optimization would not occur as rapidly as it does during ontogenesis. Evolutionary time would be needed, and the lifespan of an individual would be insufficient to build complex NOCSs. Millions of years were needed to evolve corresponding genetic automatisms. Optimization of system parameters, theoretically, a quicker and easier process to achieve, mostly relies on learning processes. To facilitate discussion, the process of NOCS development is divided into structural optimization and optimization of parameters. In many cases, both optimization processes occur simultaneously and are inseparable in time. They can even share the same genetic automatisms. Embryonic motility (Section 3.5) is a good example of how these two types of optimization occur in the nervous system. Embryonic motility starts immediately after the connections between motoneurons and muscles are established and lasts until birth. Embryonic motility is necessary for the motor control system to develop correctly and should be considered as part of the learning process conducted primarily through trial and error. After birth motility persists until death and plays the same role that it played during embryogenesis. The genes for regenerating neural tissue are suppressed in higher vertebrates. Lower vertebrates like salamanders can regrow a lost extremity or regenerate a damaged spinal cord. The regenerating capabilities of some simpler animals are even more remarkable. Vertebrates higher on the evolutionary ladder do not possess this capability. Why? And how did the genes for regenerating neural tissue become suppressed? These questions have no answers yet. However,

K.V. Baev et al. / Progress in Neuro-Psychopharmacology & Biological Psychiatry 26 (2002) 771–804

logical conclusions can be based on the organizing principles of NOCSs in higher vertebrates. The more evolutionarily complex an animal is, the more it relies on learning while it copes with its environment (i.e., by using acquired automatisms). In the highest vertebrates, complex models of very complex controlled objects must be built within the highest NOCSs. Their construction usually requires considerable time and is the major reason why the interval between birth and maturity was significantly extended during the course of evolution. Even lower NOCSs become more controlled by higher ones. In humans, for example, biped locomotion or the use of hands is impossible without the involvement of the highest motor control levels, and the learning period is lengthy. Simpler animals can perform simpler locomotions like quadruped locomotion or swimming immediately after birth. This extended time is needed for structural and parameter optimization of higher level NOCS. In higher vertebrates, some stages of structural optimization in higher NOCSs can occur only after birth (i.e., after substantial development of lower NOCSs, which are controlled objects for the higher ones). The cerebellum is a typical example of how initial structural approximation can be achieved in the nervous system. Its network is hardwired after birth when lower automatisms have already formed. In the initial stages of cerebellar wiring, several climbing fibers connect with a single Purkinje cell. Later, only one climbing fiber input survives. The other climbing fibers whose signals could not be minimized likely were rejected. Experiments with visual deprivation are another excellent example of the timing of structural optimization in ontogenesis. A brief period of visual deprivation immediately after birth prevents cortical visual detectors from forming, and the animal will remain blind for the rest of its life. Consequently, the most likely reason for suppressing regenerating genes of neural tissue in higher vertebrates is the enormous complexity of their highest NOCSs, which undergo complex optimization processes during ontogenesis. Theoretically, it is easy to explain why the complexity of optimization processes becomes a prohibiting factor for the regeneration of the nervous system. If damage in the highest NOCS could be repaired in an adult higher vertebrate animal, the structure of the damaged NOCS and its connections with surrounding NOCSs could be repaired and parameters of these systems could be adjusted properly. To do so, neurons would have to multiply and grow. Each growing neuron must be provided with the correct sequence of initiating signals to establish the correct connections. Damaged and surrounding NOCSs would have to return to the initial stage of structural optimization and repeat the entire developmental process. All the knowledge accumulated within these NOCSs would be lost. The animal loses the automatisms that these NOCSs subserved. The development of new NOCS and

799

corresponding automatisms can take years. Who would care for such an animal in the wild? Because evolution is usually parsimonious, it is more reasonable to maintain a population of a species through reproduction mechanisms that already include caring for young. If such repairs could be performed in humans as a result of a breakthrough in medical technology, an individual would become a completely different individual with new experiences. There are two major reasons for a gene to become suppressed during the course of evolution. Genes either become ineffective or begin to contradict new, more effective genetic solutions. In higher vertebrates, genes of regeneration of the nervous system became incapable of repairing damage to complex higher NOCSs, which evolved as a result of new more complex genetic solutions. Even if genes for regenerating neural tissue are artificially activated in a vertebrate animal whose highest NOCSs was injured, these genes most likely could not repair these systems. During evolution, these genes were created to repair much simpler systems. Evolutionary circumstances did not favor the development of such genes for complex NOCSs. Neurotransplantation differs from the transplantation of entire organs (e.g., heart, liver, or kidney), in which a damaged automatism is replaced with a new one. In contrast, neurotransplantation repairs a damaged neuronal automatism by adding new neuronal elements. It is based on the assumption that these new elements ‘‘know’’ how to establish correct connections with surrounding neuronal structures. The above analysis shows that this assumption is incorrect. These cells do not ‘‘know’’ how to establish correct connections in an adult brain. During development, new neural cells appear only if the development of surrounding neurons is perfectly coordinated in time so that correct connections can be established. In an adult brain, a transplanted neural or stem cell will not receive the sequence of signals needed to establish the necessary connections. Therefore, a transplanted cell most likely can establish only aberrant connections with surrounding neurons. Ironically, the establishment of wrong connections with the surrounding neurons is most often mentioned as evidence that transplanted cells have been integrated with the surrounding neural tissue. Experimental facts like growth of the dendritic tree, new synapses, and the limited appearance of new neurons in an adult brain are considered strong support for neurotransplantation. These capabilities are necessary but not sufficient for successful neurotransplantation. The capability to heal a skin cut on a finger should not be identified with the capability to regrow an entire extremity. A network must possess complex, specific computational mechanisms to be able to incorporate new neuronal elements. Neurocomputing has shown that scaling up a neural network is not a simple matter. For example, to extend a trained neural network of 200 neurons to 201 neurons, an entire training session would be needed. The more complex the network is, the more complex these mechanisms should be. Genes of regeneration for neural

800

K.V. Baev et al. / Progress in Neuro-Psychopharmacology & Biological Psychiatry 26 (2002) 771–804

tissue were blocked during the evolution of higher vertebrates. Consequently, new neurons can appear in the adult brain in ontogenesis and become integrated by the surrounding neural tissue only in regions that have solved the above computational problem (i.e., computational problems related to incorporating a new neuron are not very complex). Theoretically, gene engineering could provide transplanted neurons with information about how to find target neurons in an adult brain. Possibly, genes of regeneration could be unblocked and improved in higher vertebrates. If these improvements became possible, science would have surpassed the creative achievements of evolution. Several clinical reports show that transplantation of fetal dopaminergic cells into the striatum improves parkinsonian symptoms. These cells are implanted in the striatum because the axons of dopaminergic neurons implanted in the substantia nigra pars compacta — the correct location for dopaminergic neurons — cannot reach the striatum in adult brains. These cells will not recreate the correct error distribution circuitry and will not reproduce the necessary temporal dynamics of local dopamine release. The question now becomes: How can these observations be explained by the proposed theory? In other words, why do symptoms improve by implanting embryonic neurons that do not make the correct connections with the surrounding neural tissue? Moreover, graft cells are implanted only in some parts of the striatal network. Several explanations are possible. First, partial damage to the network computing the model flow serves as a functional neurosurgical procedure (Section 5.2.1). A typical transplantation procedure consists of half a dozen tracks, and damage to the striatal network can be substantial. This mechanism definitely accounts for the immediate clinical transplantation effect (i.e., symptoms improve within the first 48 h of transplantation). This period for rewiring is brief and edema develops in the striatum. Second, dopaminergic cells liberate dopamine in the intrastriatal intercellular space in the absence of rewiring. Such a result would add noise to the system (Section 5.2.2). Third, the new circuits that result from sprouting are not connection specific. This explanation is equivalent to adding noise to the system. Such nonspecific implanted circuits will work like DBS (i.e., like ‘‘biological noise generators’’). Fourth, grafted cells initiate a process of repairing the entire system by a function-appropriate structural reorganization similar to that of early ontogenesis. However, it is highly unlikely that simple tissue implantation can initiate a process of dedifferentiation followed by differentiation. Fifth, embryonic grafted cells release trophic factors, which can improve the function of the existing elements of the error distribution system through humoral influences. The potential role of growth factors is discussed in Section 6. Finally, the positive result may be a placebo effect and reflects a ‘‘desired improvement’’. The results of implanting dopamine-producing cells of paraneural and transfected cell origin are also discussed in Section 6.

5.3. Other theories of PD: push –pull theory Push –pull theory (Wichmann et al., 2000) is popular among neurologists and neurosurgeons but much less accepted by basic scientists who study the physiology of the basal ganglia. The idea behind this theory is simple. Compared with normal animals, neurons in the STN and Gpi in MPTP-treated primates have higher discharge rates. The linear circuit model of the basal ganglia suggested that the loss of striatal dopamine decreases the activity of inhibitory striatal neurons projecting directly to the GPi and increases activity of inhibitory striatal neurons projecting to the Gpe (see Fig. 17). The increased inhibition of the Gpe permits more activity in the STN. This oversimplified explanation of the mechanisms underlying PD has been a guideline for the treatment of parkinsonian patients since the late 1980s. Treatment consisted of restoring balance between the direct and indirect pathways from the striatum to Gpi and SNr by pharmacological or surgical treatments. In recent years, however, some scientists have begun to consider this theory inadequate and limited (Levy et al., 1997; Parent and Cicchetti, 1998; Obeso et al., 2000). The authors of this theory state that ‘‘at present the physiologic functions of the basal ganglia remains unknown’’ (Wichmann et al., 2000). In other words, this theory does not meet the strong criteria for a theory outlined in Section 1. In addition, the theory cannot explain experimental findings of lesion studies of the basal ganglia in normal animals. Typically, such lesions have no or shortlived effects on skilled fine movements or evoke only mild bradykinesia (Wichmann et al., 2000). For this reason, the authors modestly call their push – pull theory ‘‘a model of the pathologic mechanisms underlying movement disorders of basal ganglia origin’’. These lesions studies, however, can easily be explained by the holographic properties of biological neural networks (Section 3).

6. Discussion Several topics need to be stressed. New treatment strategies rely on the predictive power of a theory. If its predictive power is low, the strategy will fail. Consequently, the theory proposed here can be compared with other theoretical views of current and future strategies for developing new treatments for PD and other disorders. The push –pull theory is simpler than the theory proposed here, but it is not the simplest one. The most simple conception of PD probably is that the striatum in parkinsonian patients lacks sufficient dopamine without trying to explain the function of the basal ganglia or the symptomatology of PD. Because simple theories lead to simple predictions, simple strategies emerge for medical treatment. Based on simple theories of PD and the fact that dopamine is a neuromodulator, simple treatments have been proposed.

K.V. Baev et al. / Progress in Neuro-Psychopharmacology & Biological Psychiatry 26 (2002) 771–804

One is to pump dopamine into the striatum. Another is similar, but the method of dopamine delivery is more sophisticated: Dopamine is delivered to the striatum by implanting dopamine-producing paraneural or genetically modified cells. Our theory makes no such predictions. Consequently, such treatments would not be pursued because there would be no local delivery of an error signal necessary for normal functioning of the control system. Not surprisingly, the idea of a pump, whether mechanical or molecular, appears to be an unsuccessful treatment strategy. Attempts to use neural growth factors to halt neurodegenerative disorders are also examples of failed treatment strategies developed based on a simple theoretical assumption: The function of neurons, including their ability to survive, should improve under the influence of growth factors. This assumption was based on the experimental finding that a growth factor can initiate the growth of neurons with membrane receptors specific to this factor. Methods of delivering growth factors similar to the ones used for dopamine delivery followed: pumps, injections, systemic applications, implantation of growth factors producing cells, and so on. From our theoretical perspective, growth factors administered this way cannot be effective. Growth factors are typical examples of signal molecules that evolved for intercellular communication. These molecules serve as initiating signals for structural neuronal automatisms. A target cell produces growth factor until contact between the growing cell (the cell that undergoes growth under the influence of the growth factor) and target cell is established. The production of growth factor then drops significantly (i.e., it is minimized; Moffett et al., 1996). Structural neuronal automatisms are a necessary component of the structural optimization processes that occur in a neural network. Consequently, only bizarre results can be achieved by administering growth factors without reproducing the natural dynamics of their concentration. The theory proposed here predicts the following about growth factors: If interactions of the marker-receptor type underlie all complex connections in the nervous system, the quantity of markers and corresponding receptors must not be fewer than the types of connections between neurons. There are indications that STN stimulation can halt epileptic seizures, (Vercuel et al., 1998), but there is no adequate explanation of this discovery within the traditional mechanistic framework. In contrast, our theory explains this observation and reveals regularities of the involvement of the cortex and its basal ganglia– thalamocortical loops in epileptic seizure activity. There are two major problems in epilepsy research — defining mechanisms of focal epileptic activity and understanding how this local activity spreads across cortical regions. In terms of the latter problem, we should recall that a mismatch signal from a lower hierarchical level gains access to higher levels by ascending the system until it encounters a level competent enough to minimize it. The

801

focus of seizure activity, a seizure generator, can then be conceptualized as follows. It generates an unpredictable signal for the system. The local seizure generator can be thought of as a source of perturbation to a controlling function similar to a movement perturbation. For the control system, this perturbation is a form of novelty. It is unpredictable or its level of predictability is low. The control system must exert more effort to perform an appropriate control task to accommodate this unpredictable signal because an additional computational load is placed on the learning systems involved in tuning both the controlling and model networks. If this load is high, eventually, the available learning automatisms within a given hierarchical level will be exhausted. The exhausted system cannot function properly and will make numerous errors while performing its controlling function. In its turn, the exhausted system would become a source of unpredictable perturbations for other hierarchical levels. The latter process leads to an increasingly pronounced spread of the mismatch signal beyond its original limits. The mismatch signal becomes a powerful source of an initiating signal that must be minimized by the entire control system. Eventually, all available levels become involved in this minimization process, and generalized seizures occur. Later, these initiating signals are minimized as the entire controlling system fatigues. The system memorizes the state necessary for minimizing this signal and the way to achieve it. Each time the local seizure generator becomes active, the system reaches this state with greater ease. Thus, the process can be considered the result of learning, which leads to the development of a seizure automatism. The logical extension of this hypothesis is that dopaminergic transmission should be involved in seizure activity, as has been shown experimentally (Ferraro et al., 1991; al-Tajir and Starr, 1991). This mechanism also is consistent with the finding that frequent seizures lead to dementia. Overactivity of the dopaminergic error distribution system during seizure episodes deletes information accumulated by the network during previous learning. Apparently, a neural network that incorporates rapid learning mechanisms is vulnerable to seizures. Perhaps that vulnerability is the evolutionary tradeoff for the brain’s capacity to learn rapidly. Mechanisms of STN stimulation were discussed in Section 5.2.2. A decrease in resolution power of numerous hierarchical cortico-basal ganglia –thalamocortical loops is a result of such stimulation. Under such circumstances, it becomes more difficult to propagate seizure activity across various hierarchical levels because the threshold for being involved in seizure activity for each hierarchical loop is higher during STN stimulation. Based on our theoretical approach, the strategies available to functional neurosurgery can be outlined. Contemporary strategies should follow several basic guidelines. The first and most crucial is to avoid destroying the substrates responsible for error distribution within a system. Destroying such a region is equivalent to destroying the whole

802

K.V. Baev et al. / Progress in Neuro-Psychopharmacology & Biological Psychiatry 26 (2002) 771–804

controlling system. For the second strategy, alleviating the symptoms of a particular disease, neurosurgical interventions should be limited to regions within a controlling system where lesioning or chronic stimulation effectively decreases its resolution power. Contemporary neurosurgical procedures rely on the holographic properties of the controlling system. There should be a reasonable limit to the loss of resolution power, and finding this limit needs additional theoretical and experimental research. Based on this perspective, any surgical procedure that removes a part of an organ (or any body part) eventually relies on the holographic properties of the operated system (operated organ). Because function is partially lost, the resolution power of the system decreases. This scenario is what contemporary functional neurosurgical procedures can achieve until radical treatments of neurological disorders are developed. These procedures are inherently symptomatic. The first priority of future neurosurgical strategies should be to provide mechanisms that halt the process(es) underlying neurological diseases. Second, these strategies should provide a means for replacing lost connections and appropriately rewiring systems whose dysfunction generates the symptoms of a particular disease. At present, medicine lacks the knowledge or technical capability to treat neurological disease in this manner. We therefore can conclude that modern mechanistic transplantation approaches lack this capacity. The first strategy seems more immediately attainable. If ever realized, it will surely lead to earlier diagnosis and more definitive treatments. The relationship between a neural structure and its physiological function is one of the most important general applications of the proposed theory. Almost all neuroscientific problems and questions need to be reformulated. After all, the classic mechanistic approach has failed to reveal this relationship for even the simplest forms of behavior like inborn neuronal automatic behaviors controlled by CPGs. Within the framework of the proposed theory, the relationship between structure and function can only be defined if the analysis starts from function and reveals the connections among function, computation, and structure. First, functional organization of a studied controlling system should be defined based on knowledge of the controlled object, control objectives, and optimization criteria. This functional description is a prerequisite for the second and third stages. In the second stage, the computational abilities of each functional block of the controlling system are determined. In turn, this information enables the types of neural networks capable of performing the necessary computations to be defined. The following observations should be considered while performing this analysis. The same type of computations can be performed by networks with different architectures. Network architectural solutions are significantly simplified if they are built of more complex neuronal elements. These observations explain why neural systems are so versatile among animal species.

Understanding the relationships between the brain and the mind, consciousness, self-awareness, and other psychological categories also can be characterized in terms of relationships between function and structure. These relationships can only be revealed by analyzing how highly abstracted models of an individual and the environment are created in the brain and how the brain uses these models to perform control. Long ago, psychologists concluded that the brain creates models of the environment. The proposed theory helps to understand these brain models. Another important application of the theory is related to the analysis of the methodological limitations of theoretical and experimental methods in neurobiology. For example, implementation of the above scheme of analyzing the relationships among function, computation, and neural network structures requires a broad spectrum of mathematical methods — beginning with analytical description and finishing with numerical methods of mathematical analysis. The latter are better known as computer simulations. Whatever mathematical method is used to describe a complex neural network, this description will be only an analogy of the actual processes occurring in the network. New branches of mathematics might improve such analogies. The synthesis of efficient artificial controlling systems based on biological principles could become a good testing ground for new theoretical ideas. The relationship between the macroscopic and microscopic parameters of a controlling system built of neuronal elements also can be addressed more appropriately within the framework of the proposed theory. In neurobiology, it is popular to study the activity of single neurons during various forms of animal behavior. This approach is based on a strong belief that behavior should correlate with the integral activity of a population of neurons. Some scientists use simple averaging procedures to obtain integral parameters of a neuronal population. Others use more complex mathematical operations over the population of studied neurons. Within the framework of the classic mechanistic theory, the inappropriateness of this approach for the study of complex neural networks is subtle. In the case of simple neural networks in primitive animals, a controlling system may include several neurons or even just one. The observability of such a system is high, and the correlation between the studied behavior and neuronal activity is strong. As the number of neurons in a controlling system increases, the observability of the system decreases. For example, connections between neurons and their synaptic weights are usually unknown for complex systems. Under such circumstances, attempts to reveal how behavioral parameters are encoded in the neural network are less promising. This approach was popular in the 1970s and 1980s when scientists started studying the problem of CPGs. Identifying such correlations was considered a necessary step to bridge the gap between neuronal activity and behavior. This approach has failed. How motor parameters are encoded by CPGs in complex animals is still unknown. Nevertheless,

K.V. Baev et al. / Progress in Neuro-Psychopharmacology & Biological Psychiatry 26 (2002) 771–804

the search for such correlations remains popular, and other forms of motor control, including learned forms of motor behavior, are studied this way. Similar ideas have been explored in Neurocomputing, with similar results. After a neural network is trained to perform its goal, its weights have no direct meaning. Underlying rules that may be implied from the neural network cannot be extracted. Biological neural networks are persistently changing because they are learning systems. This point should be considered by anyone who wants to correlate the micro- and macroparameters of a neural network.

7. Conclusion The proposed theory bridges the gap between behaviors known as inborn automatic behaviors and acquired forms of behavior. Consequently, the nervous system can finally be studied from a unified perspective. Both forms of behavior are controlled by learning NOCSs. The only difference consists in the speed of learning processes. The latter are faster in systems responsible for control of acquired forms of behavior. Life itself should be considered as the evolution of learning systems.

Acknowledgments We are very grateful to Shelley A. Kick, PhD, Senior Editor, and Judy Wilson, Editorial Assistant, Neuroscience Publications at Barrow Neurological Institute, for their tremendous help with preparing the manuscript.

References Alexander, G.E., Crutcher, M.D., 1990. Functional architecture of basal ganglia circuits: neural substrates of parallel processing. TINS 13, 266 – 271. Alexander, G.E., Delong, M.R., Strick, P.L., 1986. Parallel organization of functionally segregated circuits linking basal ganglia and cortex. Annu. Rev. Neurosci. 9, 357 – 381. Allen, G.I., Tsukahara, N., 1974. Cerebro-cerebellar communication system. Physiol. Rev. 54, 957 – 1006. al-Tajir, G., Starr, M.S., 1991. Anticonvulsant effect of striatal dopamine D2 receptor stimulation: dependence on cortical circuits? Neuroscience 43, 51 – 57. Arshavsky, Yu.I., Gelfand, I.M., Orlovsky, G.N., 1986. Cerebellum and Rhythmic Movements. Springer, Berlin. Ashby, P., 2000. What does stimulation in the brain actually do? In: Lozano, A.M. (Ed.), Movement Disorder Surgery. Prog. Neurol. Surg. 15, 236 – 245. Baev, K.V., 1998. Biological Neural Networks: Hierarchical Concept of Brain Function. Birkhauser, Boston. Baev, K.V., Shimansky, Yu.P., 1992. Principles of organization of neural systems controlling automatic movements in animals. Prog. Neurobiol. 39, 45 – 112. Baev, K.V., Greene, K.A., Marciano, F.F., Shetter, A.G., Lieberman, A.N., Spetzler, R.F., 1995. Novel heuristics of functional neural networks:

803

implications for future strategies in functional neurosurgery. Stereotact. Funct. Neurosurg. 65, 26 – 36. Barto, A.G., Sutton, R.S., Anderson, C.W., 1983. Neuronlike elements that can solve difficult learning control problems. IEEE Trans. Syst. Man. Cyber. 13, 835 – 846. Bloedel, J.R., Lou, J.-S., 1987. The relation between Purkinje cell spike responses and the action of the climbing fibre system in unconditioned and conditioned responses of the forelimb to perturbed locomotion. In: Glickstein, M., Yeo, Chr., Stein, J. (Eds.), Cerebellum and Neuronal Plasticity. Plenum, New York, pp. 261 – 276. Brodal, A., 1981. Neurological Anatomy. In Relation To Clinical Medicine. Oxford Univ. Press, New York. Brown, T.G., 1914. On the nature of fundamental activity of the nervous centres; together with an analysis of the conditioning of rhythmic activity in progression, and a theory of evolution of function in the nervous system. J. Physiol. 48, 18 – 46. Burgess, N., Recce, M., O’Keefe, J., 1995. Hippocampus: spatial models. In: Arbib, M. (Ed.), The Handbook of Brain Theory and Neural Networks. MIT Press, Cambridge, MA, pp. 468 – 472 (A Bradford Book). Darian-Smith, C., Darian-Smith, I., Burman, K., Ratcliffe, N., 1993. Ipsilateral cortical projections to areas 3A, 3B, and 4 in the Macaque monkey. J. Comp. Neurol. 335, 200 – 213. Delong, M.R., Georgopoulos, A.P., 1981. Motor functions of the basal ganglia. In: Brookhart, J.M., Mountcastle, V.B., Brooks, V.B. (Eds.), Handbook of Physiology, Sect 1, The Nervous System: vol. 2. Motor Control, Part 2. Am Physiol Soc, Bethesda, pp. 1017 – 1061. Delong, M.R., Georgopoulos, A.P., Crutcher, M.D., 1983. Cortico-basal ganglia relations and coding of motor performance. Exp. Brain Res. Suppl. 7, 30 – 40. Eccles, J., Ito, M., Szentagothai, J., 1967. The Cerebellum as a Neuronal Machine. Springer, Berlin. Evarts, E.V., Thach, W.T., 1969. Motor mechanism of the CNS: cerebrocerebellar interrelations. Annu. Rev. Physiol. 31, 451 – 498. Ferraro, G., Vella, N., Sardo, P., Caravaglios, G., Sabatino, M., La Grutta, V., 1991. Dopaminergic control of feline hippocampal epilepsy: a nigro hippocampal pathway. Neurosci. Lett. 123, 41 – 44. Getting, P.A., 1981. Mechanisms of pattern generation underlying swimming in Tritonia: I. Neuronal network formed by monosynaptic connections. J. Neurophysiol. 46, 65 – 79. Getting, P.A., 1983a. Mechanisms of pattern generation underlying swimming in Tritonia: II. Network reconstruction. J. Neurophysiol. 49, 1017 – 1035. Getting, P.A., 1983b. Mechanisms of pattern generation underlying swimming in Tritonia: III. Intrinsic and synaptic mechanisms for delayed excitation. J. Neurophysiol. 49, 1036 – 1050. Getting, P.A., 1986. Understanding central pattern generators: insights gained from the study of invertebrate systems. In: Grillner, S., Stein, P.S.G., Stuart, D.G., Fossberg, H., Herman, R. (Eds.), Neurobiology of Vertebrate Locomotion. MacMillan, London, pp. 231 – 244. Getting, P.A., 1989. Reconstruction of small neuronal networks. In: Koch, C.I., Segev, I. (Eds.), Methods in Neuronal Modeling. MIT Press, Cambridge, MA. Getting, P.A., Lennard, P.R., Hume, R.I., 1980. Central pattern generator mediating swimming in Tritonia: I. Identification and synaptic interactions. J. Neurophysiol. 44, 151 – 164. Greene, K.A., Marciano, F.F., Golfinos, J.G., Shetter, A.G., Lieberman, A.N., Spetzler, R.F., 1992. Pallidotomy in levodopa era. Adv. Clin. Neurosci. 2, 257 – 281. Hamburger, V., 1963. Some aspects of the embryology of behavior. Q. Rev. Biol. 38, 342 – 365. Hamburger, V., Oppenheim, R., 1967. Prehatching motility and hatching behavior in the chick. J. Exp. Zool. 166, 171 – 204. Hecht-Nielsen, R., 1990. Neurocomputing. Addison-Wesley Publishing, San Diego. Hopfield, J.J., 1984. Neurons with graded response have collective computational properties like those of two-state neurons. Proc. Natl. Acad. Sci. USA 81, 3088 – 3092.

804

K.V. Baev et al. / Progress in Neuro-Psychopharmacology & Biological Psychiatry 26 (2002) 771–804

Hopfield, J.J., 1987. Learning algorithms and probability distributions in feed-forward and feed-back networks. Proc. Natl. Acad. Sci. USA 84, 8429 – 8433. Hopfield, J.J., 1994. Neurons, dynamics and computation. Phys. Today 47, 40 – 46. Hopfield, J.J., Tank, D.W., 1985. ‘‘Neural’’ computation of decisions in optimization problems. Biol. Cybern. 52, 141 – 152. Hoyle, G., 1984. The scope of neuroethology. Behav. Brain Sci. 7, 367 – 412. Isaacson, R.L., 1982. The Limbic System. Plenum, New York. Kemp, J.M., Powell, T.P.S., 1971. The connections of the striatum and globus pallidus: synthesis and speculation. Philos. Trans. R. Soc. London, Ser. B 262, 441 – 457. Le Moal, M., Simon, H., 1991. Mesocorticolimbic dopaminergic network: functional and regulatory roles. Physiol. Rev. 71, 155 – 234. Levy, W., 1989. A computational approach to hippocampal function. In: Hawkins, R.D., Bower, G.H. (Eds.), Computational Models of Learning in Simple Neural Systems. Academic Press, San Diego, pp. 243 – 305. Levy, W.B., Wu, X.B., Baxter, R.A., 1995. A computational model of hippocampal region CA3 for flexible learning of temporal spatial associations. Soc. Neurosci. Abstr. 21, 1226. Levy, R., Hazrati, L.N., Herrero, M.T., Vila, M., Hassani, O.K., Mouroux, M., Ruberg, M., Asensi, H., Agid, Y., Feger, J., Obeso, J.A., Parent, A., Hirsch, E.C., 1997. Re-evaluation of the functional anatomy of the basal ganglia in normal and Parkinsonian states. Neuroscience 76, 335 – 343. Ljungberg, T., Apicella, P., Schultz, W., 1992. Responses of monkey dopamine neurons during learning of behavioral reactions. J. Neurophysiol. 67, 145 – 163. Marciano, F.F., Greene, K.A., 1992. Surgical management of Parkinson’s disease: Part I. Paraneural and neural tissue transplantation. Neurol. Forum 3, 1 – 7. Mcculloch, W., Pitts, W., 1943. A logical calculus of the ideas immanent in nervous activity. Bull. Math. Biophys. 5, 115 – 137.

Minsky, M.L., 1963. Steps towards artificial intelligence. In: Feigenbaum, E.A., Feldman, J. (Eds.), Computers and Thought. McGraw-Hill, New York, pp. 406 – 450. Moffett, J., Kratz, E., Florkiewicz, R., Stachowiak, M.K., 1996. Promoter regions involved in density-dependent regulation of basic fibroblast growth factor gene expression in human astrocytic cells. Proc. Natl. Acad. Sci. USA 93, 2470 – 2475. Montgomery, E.B., Gorman, D.S., Nuessen, J., 1991. Motor initiation versus execution in normal and Parkinson’s disease subjects. Neurology 41, 1469 – 1475. Obeso, J.A., Guridi, J., Rodriguez-Oroz, M.C., Macias, R., Rodriguez, M., Alvarez, L., Lopez, G., 2000. Functional models of the basal ganglia: where we are? In: Lozano, A.M. (Ed.), Movement Disorder Surgery. Karger, Basel. Prog. Neurol. Surg. 15, 58 – 77. Parent, A., Cicchetti, F., 1998. The current model of basal ganglia organization under scrutiny. Mov. Disord. 13, 199 – 203. Roberts, A., Soffe, S.R., Dale, N., 1986. Spinal interneurons and swimming in frog embryos. In: Grillner, S.S., Stein, P.S.G., Stuart, D.G., Fossberg, H., Herman, R. (Eds.), Neurobiology of Vertebrate Locomotion. MacMillan, London, pp. 279 – 306. Schultz, W., Dickinson, A., 2000. Neuronal coding of prediction errors. Annu. Rev. Neurosci. 23, 473 – 500. Shimansky, Yu.P., 2000. Spinal motor control system incorporates an internal model of limb dynamics. Biol. Cybern. 83, 379 – 389. The American Heritage Dictionary, third ed., 1994. Vercuel, L., Benazzouz, A., Deransart, C., Bressand, K., Marescaux, C., Depaulis, A., Benabid, A.L., 1998. High-frequency stimulation of the subthalamic nucleus suppresses absence seizures in the rat: comparison with neurotoxic lesions. Epilepsy Res. 31 (1), 39 – 46. Wichmann, T., Delong, M.R., Vitek, J.L., 2000. Pathophysiological considerations in basal ganglia surgery: role of the basal ganglia in hypokinetic and hyperkinetic movement disorders. In: Lozano, A.M. (Ed.), Movement Disorder Surgery. Karger, Basel. Prog. Neurol. Surg. 15, 31 – 57.