A central pattern generator for controlling sequential activation in a neural architecture for sentence processing

Author's Accepted Manuscript

A Central Pattern Generator for Controlling Sequential Activation in a Neural Architecture for Sentence Processing Djurre van Dijk, Frank van der Velde

www.elsevier.com/locate/neucom

PII: DOI: Reference:

S0925-2312(15)00860-7 http://dx.doi.org/10.1016/j.neucom.2014.12.113 NEUCOM15681

To appear in:

Neurocomputing

Received date: 30 June 2014 Revised date: 17 November 2014 Accepted date: 22 December 2014 Cite this article as: Djurre van Dijk, Frank van der Velde, A Central Pattern Generator for Controlling Sequential Activation in a Neural Architecture for Sentence Processing, Neurocomputing, http://dx.doi.org/10.1016/j.neucom.2014.12.113 This is a PDF file of an unedited manuscript that has been accepted for publication. As a service to our customers we are providing this early version of the manuscript. The manuscript will undergo copyediting, typesetting, and review of the resulting galley proof before it is published in its final citable form. Please note that during the production process errors may be discovered which could affect the content, and all legal disclaimers that apply to the journal pertain.

A Central Pattern Generator for Controlling Sequential Activation in a Neural Architecture for Sentence Processing Djurre van Dijk, Frank van der Velde1 1

Corresponding author

Technical Cognition, CTIT, University of Twente, P.O. Box 217, Enschede, 7500 AE, The Netherlands, IO, Leiden University, The Netherlands [email protected]

Abstract The neural architecture for sentence processing is a model of a neural ‘blackboard’ capable of temporarily storing both semantic and syntactic information. Retrieving information from the neural blackboard requires a sequence of activations that is controlled by a central pattern generator. We implement a central pattern generator that controls the sequence of activation. To ground the implementation in a biological context, the implementation is based on a model of the escape swim network of Tritonia diomedea, a marine mollusk. A central pattern generator is developed to meet the specifications required to successfully control the sequence of actions and activations needed to retrieve information from the neural blackboard in response to a question. The model is an existence proof for a biologically plausible implementation of a neural blackboard central pattern generator. The role of the central pattern generator in this neural architecture of sentence processing illustrates the potential relation between controlling movement processing and cognitive processing.

Keywords CPG, neural blackboard, sentence processing, sequential control

1

1. Introduction The role of Central Pattern Generators (CPGs) in motor control is well established (e.g., Grillner et al. 2005). Motor control with CPGs is found in organisms ranging from mollusks (e.g., Getting, 1989; Sakurai, Gunaratne, & Katz, 2014; Wu et al., 2014) to control of locomotion speed in humans (Dzeladini, van den Kieboom & Ijspeert, 2014). Based on the role of CPGs in motor control, CPG models have been used in controlling the motor behavior of robots (e.g., Yu et al. 2014), ranging from snake-like robots (e.g., Nor & Ma, 2014) to humanoid robots (e.g., Nassour, et al., 2014; Shahbazia et al. 2014). Here we want to investigate a role of CPGs in controlling higher level cognitive processing. The basis for such a role of CPGs is twofold. On the one hand, there are similarities between the microcircuits underlying CPGs in motor control and microcircuits observed in the neocortex (e.g., Yuste et al., 2005). On the other hand, there are functional similarities between motor control and control of higher level cognitive processing. In this respect, Llinás (2002) discussed the motor primacy in the organization of the brain and identified thinking as internalized movement. So, one would expect CPGs to play a role in this internalized movement as well. An illustration of how cognitive processing could be related to motion control is found in the notion of cell assemblies formulated by Hebb (1949). Hebb identified three organizational principles by which cognitive processing could be related to brain processing. The first one is the well-known principle of Hebbian learning. Connections between neurons are modified when these neurons are concurrently active in a given process. In this way, information based on experience is learned. The second principle is the cell assembly that can result from Hebbian learning. When a given process is repeated over time, the neurons involved in that process will be stronger connected to each other, forming an assembly of cells. As a result, the assembly can be reactivated when only a part of its neurons are activated by a stimulus, because the assembly structure ensures that the other neurons are activated as well. Assemblies could be local, but they can also be global, interconnecting groups of neurons in different parts of the brain (cortex). These more global assemblies can be expected to occur in higher level cognitive processing (e.g.,

2

concept formation) because different forms of information will be combined in these processes (e.g., Quian Quiroga, 2012). Hebb’s third principle is the “phase sequence” of assemblies that underlies thinking. In this view, a cognitive process typically results from a sequential activation of the assemblies (e.g., concepts) involved in the process (e.g., Buzsáki, 2010; Huyck & Passmore, 2013). This sequential activation can be controlled by a stimulus, but it can also be controlled internally, e.g., when the activation of one assembly initiates the activation of another. Here, one can already see the relation with motion control. In sequential activation of assemblies, one assembly needs to be inactivated when another is activated, just as one set of muscles needs to be inactivated when another set is activated in movement control. So, as in movement control, CPGs could play a role in the sequential activation of assemblies underlying higher level forms of cognitive processing. We will illustrate a role of CPGs in our neural architecture of language processing (van der Velde & de Kamps, 2006, 2010, 2011; van der Velde, 2014). In this architecture, words are indeed implemented as cell assemblies proposed by Hebb. Sentence structures are then formed by temporarily connecting cell assemblies representing words (or word assemblies for short) in a ‘Neural Blackboard Architecture’ (NBA). The NBA allows sequential processing to occur, as for example the process of answering a question when a sentence is stored in the NBA. So, when the sentence cat chases mouse is stored in the NBA, the question “What does the cat chase?” results in a sequential activation of assemblies in the NBA, finally resulting in the activation of the assembly for mouse as the answer to the question. This process was simulated in (van der Velde & de Kamps, 2006). In the simulation it was assumed that a CPG would control the sequential activation of assemblies, needed to produce the answer. However, the CPG was set by hand. Here we will develop and simulate a CPG that can be used to control this process. Before we discuss this CPG, we will first briefly outline the NBA and the role of control of sequential activation in this architecture.

3

2. Neural Blackboard Architecture for sentence structure The NBA described ibed by van der Velde and dde Kamps (2006) allows sentences to be processed and stored (temporarily) (temporarily). In the architecture, words are re encoded as neural ‘word’ assemblies (Pulvermüller Pulvermüller 1999). Such neural assemblies can be distributed over several parts of the brain. For example, assemblies referring to visible objects can be partially represented in the visual cortex and assemblies referring to verbs can be partially represented in the motor cortex. ‘Fire engine’ could be partially encoded by ‘red’ in the visual al cortex and by ‘loud’ in the auditory cortex. To o form a sentence structure, word assemblies are temporarily bound to a neural ‘structure’ assembly inn the blackboard (this can occur because each word assembly has a part that is connected to the NBA) NBA). Structure assemblies encode the relations between the word assemblies. Word assemblies can be simultaneously bound to several structure assemblies, allowing for unambiguous encoding of multiple instances of the same word.

Figure 1. Illustration of the neural n sentence structure of cat chases mouse inn the neural blackboard (based on van der Velde & de Kamps, 2006). (A)) Word assemblies for cat, chases and mouse are distributed over the cortex. (B)) Sentence structure is encoded with structure assemblies for nouns (N1, N2) and verbs (V1). A structure assembly consists of a main assembly and a number of sub-assemblies, sub assemblies, connected to the main assemblies by means of gating circuits. The labeled sub-assemblies sub assemblies represent the thematic roles of agent (a), and theme (t). Binding between assemblies assemblies is achieved with active memory circuits. (C)) A gating circuit based on disinhibition. The grey nodes are active.

4

Figure 1 illustrates the representation of the sentence cat chases mouse. As illustrated in figure 1A, the word assemblies for cat, chases and mouse are distributed over the brain. Figure 1B illustrates how structure assemblies of a specific type selectively bind word assemblies to form a sentence structure. So structure assemblies related to nouns (or noun assembles for short) bind to nouns and structure assemblies related to verbs (or verb assembles for short) bind to verbs. Furthermore, each structure assembly consists of a main assembly and subassemblies of a specific type. Subassemblies of the same type are used to bind structure assemblies to each other, in line with the structure of the sentence. In figure 1B the structure assemblies N1 (for cat) and V1 (for chases) are bound by their agent sub-assemblies. This binding thus encodes that cat is the agent of chases in this sentence. Several structure assemblies bound in this manner can encode the syntactic structure of a sentence. Binding and control of activation in the architecture result from the selective activation of gating and memory circuits. These gating and memory circuits control the flow of activation within and between structure assemblies. Figure 1C illustrates a gating circuit that controls the flow of activation between the assemblies X and Y. The circuit is based on disinhibition. If assembly X is active, both neurons ix and Xout will receive excitatory activation from this assembly. Xout will also receive inhibitory input from ix when ix becomes activated. This inhibition will prevent Xout from becoming active for as long as Xout and ix are both activated. However, when Ix is driven to activation by an external input (given by a control signal) it will inhibit ix. As a result, ix will no longer inhibit Xout and activation can flow from assembly X to assembly Y. The gating circuit illustrated in figure 1C also forms the basis for a memory circuit by which assemblies are bound. The only difference is that the external control signal that initiates the activation of Ix is replaced by a ‘delay’ assembly. The activity in the delay assembly is similar to the maintenance of activation found in experiments on working memory (e.g., Durstewitz et al., 2000). The delay assembly is activated in the processing of the sentence. As long as it remains active, the assemblies it connects are bound because activation can flow from one to the other (for further details see van der Velde and de Kamps, 2006). 5

Using assemblies, gating circuits and memory circuits, a representation of a sentence can be made in the NBA. When a word is processed, a structure assembly will be activated in the blackboard. The type of structure assembly activated depends on the processed word. For example, a processed noun will activate a noun (Ni) structure assembly and a verb (Vj) assembly will be activated when a verb is being processed. Which specific structure assembly is activated is irrelevant as long as that assembly is free - none of its memory circuits are activated - and it is of the correct type. When the sentence cat chases mouse (see figure 1B) is processed, the word assembly for cat is bound to N1 by the activation of the memory circuit that connects these assemblies. Chases is bound to V1 in a like manner. The binding between cat and chases is accomplished by binding the agent sub-assemblies of N1 and V1. To achieve this binding the two agent sub-assemblies have to be activated. Activating the gating circuits between the main assemblies N1 and V1 and their respective agent subassemblies allows activation to flow from the main assemblies to the sub-assemblies, thereby activating the sub-assemblies. Activation of the gating circuits is controlled by neural control circuits. Neural control circuits instantiate parsing operations based on the activation state of the neural blackboard and on which word assemblies are active (for details, see van der Velde and de Kamps, 2006; 2010). The subsequent binding between chases and mouse proceeds along similar lines. The model illustrated in figure 1 is not the only neural model of sentence processing. Alternative models are models based on dynamics systems (e.g., Gerth & beim Graben, 2009), reservoir computing

(e.g., Hinaut & Dominey, 2013) or

recurrent neural networks (e.g., Shi et al., 2013). But these models do not produce information (e.g., answering questions) based on the sentence structure. By contrast, in the model in figure 1 a sentence structure is created in which the word representations given by the neural assemblies remain grounded. As a result, these assemblies can be used to retrieve information from the sentence or sentences parts. Recently, Sagara and Hagiwara (2014) presented a neural model that can answer questions about sentences. But sentence processing in the model is based on a symbolic parser and sentences are represented as single nodes, associated to the nodes representing their words. Answers are derived from these associations, which do not 6

give the ability to take the sequential (syntactic) nature of the sentence into account. Below we outline a dynamic process by which information from a sentence representation as illustrated in figure 1 can be retrieved in a sequential manner, based on the sequential and syntactic representation of the sentence. A CPG is then needed to control the sequential activation occurring in this process.

2.1. Answering binding questions. Once a sentence is stored, the information contained in the sentence is available in the NBA. Retrieval of this information is accomplished by asking binding questions. For example, a binding question for the sentence cat chases mouse could be “What does the cat chase?”. To answer this question, the stored representation of cat chases

mouse has to be reactivated. A binding question gives information that can be used to arrive at the correct answer. The question “What does the cat chase?” will activate the word assemblies for cat and chase. It also asks for the theme of the verb in the sentence. Using this information, the answer mouse can be activated by the following sequence of actions: the activation of V1 (see figure 1B), followed by the activation of its theme subassembly, the activation of the theme sub-assembly of N2, followed by the activation of the main assembly N2, which then results in the activation of the assembly for

mouse. Control over this sequence of actions is exerted by a neural control circuit, which functions like a linear pattern generator (CPG). Van der Velde and de Kamps (2006) simulated the reactivation of the sentence structure required to answer the binding question “What does the cat chase?” when three sentences are simultaneously stored in the NBA architecture (see figure 2). The sentences stored are cat chases mouse, mouse chases cat and cat bites dog. Figure 2 illustrates that the word assemblies for cat, chases and mouse are the same in each of these sentence structures. This results from the fact that words are represented by word assemblies, with only one assembly for a given word. Sentences with similar words are distinguished because the word assemblies are bound to different structure assemblies. For example, in cat chases mouse, cat is bound to N1, chases to V1 and mouse to N2. In mouse chases cat, cat is bound to N4, chases to V2, and mouse to N3. And in cat bites dog, cat is bound to N5 (see figure 2). 7

Figure 2. Combined instantiation of the sentences cat chases mouse, mouse chases cat and cat bites dog in n the NBA. The multiple instantiations of cat, chases and mouse in different sentences (and different thematic roles) are distinguished by the different noun and verb structure assemblies to which they are bound. The double double-line connections illustrate the binding of word assemblies to structure assemblies in the NBA. Grey assemblies are activated by the question “What does the cat chase?” chase?”. Adapted d from van der Velde and de Kamps (2006).

The activation of the answer mouse via N2 (but not via N3) to the question “What does the cat chase?” depends on a competition in the NBA (see van der Velde & de Kamps, 2006).. In turn, this competition requires a sequential activation of the assemblies lies and subassemblies involved. The assumption is that the question posed activates a specific type of CPG, in this case a CPG for questions of the type “noun verb x?” (Dominey 1997). Activation of the CPG commences when the first neuron in the CPG's sequence is activated. The CPG in for this question consists of fifteen neurons, representing fifteen discrete time steps. Each neuron will be active for 25 ms. after which the next neuron is activated. The CPG sequence terminates terminates after the deactivation of the fifteenth neuron. At the start of answering the binding question “What does the cat chase?”, chase?” the only active assemblies are the delay assemblies of the memory circuits which encode the structure of the three sentences. As illustrated in figure 2, the binding question activates the cat and chases word assemblies assemblies. Via the memory circuits (see figure 1B) activation spreads from the cat word assembly to N1, N4 and N5 and from the chases 8

word assembly to V1 and V2, resulting in activation of these structure assemblies (figure 2). Activation of the theme sub-assemblies causes activation to spread from V1 to N2, which would correctly answer the binding question. However, cat (N1, N4 and N5) is also active due to direct activation by the binding question. Additionally, N4 receives activation from V2, via its theme sub-assembly, which also activates cat. This simultaneous activation of both mouse and cat results in an ambiguous or incorrect answer to the question.

Figure 3. The fifteen steps in the NBA CPG and the time-course for the sequence of actions, as controlled by the CPG, required to answer the binding question “noun verb x?”.

To arrive at an unambiguous answer to the question posed, the CPG control circuit has to complete a sequence of actions. Posing the question activates the word assemblies and theme sub-assemblies as before. To determine which of the two sentences containing chases holds the answer for the binding question, a winnertakes-all (WTA) competition takes place amongst the verb assemblies, from which V1 has to emerge as the winner. Activation of the first neuron of the CPG activates an inhibitory assembly of neurons connected to the verb main assemblies and thereby initiates competition amongst the verb assemblies. Activation of the inhibitory assembly lasts from step one to step five in the CPG sequence (see figure 3). Activation of the inhibitory population will, however, not produce a clear winner if there is no additional activation to bias the WTA competition in favor of one of the 9

verb assemblies. By the activation of the gating circuits connected to the agent subassemblies, in step one through four of the CPG sequence (figure 3), the activation of the noun assemblies decides which verb assembly remains active. Opening the agent gating circuits would allow activation to flow from N1 to V1, from N3 to V2 and from N5 to V3 (see figure 2). Activation flowing from N1 to V1 increases activation of the V1 assembly. Since N3 is not active, no additional activation flows to V2. This causes V2 to lose the competition with V1. Even though activation flows from N5 to V3, V3 itself is not active. V3 can therefore not match the activation level of V1 and loses the competition, leaving V1 as the only active verb assembly (see van der Velde & de Kamps, 2006). With only V1 active, activating the theme sub-assembly gating circuits activates N2 and with it mouse. This still results in an ambiguous answer, because N1, N4 and N5 are active as well. These assemblies have to be deactivated so that mouse is the sole answer to the question. This requires inhibition of both word and noun assemblies before activating the theme sub-assembly of V1. Inhibition of word assemblies occurs at time step five in the sequence (figure 3). Inhibition of noun assemblies is accomplished at step six. This results in deactivation of all word and noun assemblies, leaving only V1 active. With V1 as the sole active assembly, the theme gating circuits can be activated. During step eight to eleven the gating circuits between the verb main assemblies and the theme sub-assemblies are activated. This allows activation to flow from V1 to its theme sub-assembly and from this sub-assembly, through the binding memory circuit, to the theme sub-assembly of N2. During step thirteen to fifteen the gating circuits between the noun main assemblies and the theme sub-assemblies are activated. This allows activation to spread to N2 from its theme sub-assembly, thereby activating N2 and its bound word assembly mouse. At the end of the CPG controlled sequence the binding question is answered correctly by the activation of mouse.

10

2.2. Specifications for the required CPG The CPG illustrated in figure 3 was set by hand in the NBA simulation of answering the binding question in (van der Velde & de Kamps, 2006). Here, we aim to implement a neural model of this CPG to control the sequential activation in the NBA simulation. This goal is constrained by the requirement to find a biologically plausible implementation. The minimum specifications required of this CPG are derived from the sequence of actions described above (figure 3). Selection of the relevant CPG is assumed to be driven by the binding question and is considered a given. The moment the sequence is activated is assumed to be given by the moment the binding question is posed to the network. This leaves only the temporal characteristics of the control sequence unspecified. The CPG described above consists of fifteen neurons. Each of these neurons is activated once during the sequence and only one neuron is active at any given time (see figure 3). For the purpose of defining the CPG's temporal characteristics, an active neuron is defined as a neuron capable of causing selective inhibition of neural assemblies and activating gating circuits. Each neuron is active for a period of 25 ms, after which it becomes inactive and the next neuron in the sequence is activated. Providing an existence proof of a biologically plausible implementation of the NBA control neurons requires the successful implementation of a biologically plausible CPG capable of replicating the sequence of activations shown in figure 3, while respecting the temporal characteristics defined by the 25ms step duration.

3. Central Pattern Generator in Tritonia diomedea. The aim here is to develop a CPG that can regulate the binding process in the NBA when a question is answered. Instead of just designing a CPG on our own, we start from a biological model described in the literature. One such model, analyzed in detail, is the escape swim network of the marine mollusk (sea slug) Tritonia diomedea. This network has been well documented in scientific literature (Getting, Lennard & Hume, 1980; Getting, 1981; Getting 1983a; Getting 1983b; Getting & Dekin, 1985; Hume, Getting & Del Beccaro, 1982a; Hume & Getting, 1982b; Lennard, Getting & Hume, 1980).

11

Tritonia diomedea is a bottom dwelling marine mollusk. When exposed to a threat, for example contact with a predatory starfish, it flees by swimming away. When escape swimming is initiated, the animal shows a short duration withdrawal from the offending stimulus followed by alternating dorsal and ventral whole body flexations. These flexations make up the escape swim, which propels the mollusk upwards and away from the potential threat. The escape swim is a transient behaviour with a duration of twenty to sixty seconds (Getting 1981). The CPG driving the escape swim behaviour consists of two identical networks, one situated in each hemisphere. Three populations of neurons have been found that display rhythmic activity during escape swimming (Getting, et al. 1980). These are referred to as: the Dorsal Swim Interneurons (DSIs), the Ventral Swim Interneurons (VSIs) and the Cerebral 2 cells (C2s).

Figure 4. Progression of a single cycle of the pattern produced by the escape swim network of Tritonia diomedea. Excitatory synapses are displayed as T bars and inhibitory synapses as filled circles. Active neurons have fat and inhibited neurons dashed lines. The DRI is present in the network throughout the progression, but is only shown in A. The delay in activation caused by the A-current is represented by the grayed VSIs in F. See text for a full explanation.

A qualitative illustration of this CPG is given in figure 4 (quantitative analyses can be found in Getting, 1989). Here, a distinction is made between two VSI populations: VSI-A and VSI-B. The event that triggers pattern generation is activation of the 12

Dorsal Ramp Interneuron (DRI) by the sensory neurons (see figure 4A). Subsequent tonic spiking by the DRI causes the depolarization of the DSIs. When the DSIs are driven to their threshold, activation of the DSIs excites both the C2s and VSI-As, but inhibits the VSI-Bs (see figure 4B). Excitation from the DSIs also activates the dorsal pedal flexion neurons that initiate a dorsal whole body flexation in the animal. The activation of the C2s and VSI-As causes both groups to fire (see figure 4C). However, when the C2s start to fire they inhibit the VSI-As and quickly silence them. The synaptic weights are such that, when simultaneous excitated by the DSIs and inhibited by the C2s, inhibition is stronger and the VSI-As remain inactive (see figure 4D). Meanwhile, excitatory connections from the C2s to the VSI-Bs (see figure 4C) slowly lift its inhibition by the DSIs. The previously hyperpolarized VSI-Bs will depolarize (see figure 4D), which will eventually activate a current called the Acurrent. This current is activated when the cell’s membrane depolarizes and it slows the depolarization (Getting, 1989). At this point in the cycle several multicomponent connections switch sign (Getting 1983a). Most notably, the connection between the C2s and DSIs becomes inhibitory and the connection between the C2s and the VSI-As becomes excitatory (see figure 4E). This breaks the feedback loop between the DSIs and C2s. The first reason for this is that the reciprocal excitatory connection between the DSIs and the C2s is changed into an inhibitory connection. The second reason is the activation of the VSIAs, now driven by both the DSIs and C2s. The VSI-As inhibit the DSIs, deactivating one member of the feedback loop (see figure 4F). The VSI-Bs are now no longer inhibited by the DSIs and are driven to threshold by excitatory connections from both the C2s and VSI-As (see figure 4F). When the Acurrent deactivates, the VSI-Bs start to spike. The VSI-Bs inhibit both the DSI and C2, which fall silent when the VSI-Bs are active (see figure 4G). Spiking by the VSIBs also activates the ventral pedal flexion neurons that initiate the ventral whole body flexion in the animal. However, when both the DSIs and C2s become inactive, the VSI-As and VSI-Bs lose their sources of activation. The only active excitatory connections are the reciprocal connections between the VSIs (see figure 4G). The VSI-Bs will continue to

13

spike on the residual delayed excitation from the C2s. When that activation dissipates, the VSI-Bs stop spiking. When the VSI-Bs fall silent, the DSIs are released from inhibition and the network returns to its initial state (see figure 4H). If depolarization by DRI is still sufficient, the DSIs will once again start spiking, restarting the cycle (see figure 4A). Several multicomponent connections (see figure 5A) switch sign halfway through the cycle. The synaptic components that are active during the first half of the cycle (see figure 5B) shows reciprocal excitation amongst the synergists - the coactively firing DSIs and C2s - and inhibition of the antagonists - the inhibited VSI-As and VSI-Bs. The network is locked in this state unless the VSI-As or VSI-Bs overcome inhibition and reaches threshold. The second half of the cycle shows a different pattern of connectivity (see figure 5C). Connections between the C2s and DSIs become inhibitory and connections between the C2s and the VSI-As become excitatory, resulting in both the VSI-As and the VSI-Bs receiving only excitatory input. When these neurons reach threshold the firing pattern of this state is established. The pattern of connectivity is still reciprocal excitation amongst the synergists and inhibition of the antagonists. However, now the VSI-As and the VSI-Bs are the synergists and the C2s and DSIs are the antagonists.

Figure 5. Dynamic reorganization of the synaptic connectivity within part of the swim network during the course of a single swim cycle. The network (A) can be divided into two circuits depending on the initial (B) and secondary (C) effect of each of the multicomponent connections. Excitatory synapses are displayed as T bars and inhibitory synapses as filled circles. Active neurons have fat and inhibited neurons dashed lines. The delay in activation caused by the A-current is represented by the grayed VSI-Bs in C. Adapted from Getting (1983a).

In summary, the network moves from an inactive into its first state, when driven by DRI. The change in the pattern of connectivity resulting from multicomponent 14

connections with both excitatory and inhibitory elements causes the network to switch from its DRI enabled state into its second state. However, the second state is inherently unstable. This instability causes a return to either the inactive state or the first state, depending on the amount of activation that the network receives from DRI.

4. Development of the Neural Blackboard Central Pattern Generator In our development of the CPG for the NBA we started with the CPG illustrated above. In order to transform this CPG into the CPG needed for the NBA we made a number of choices. These choices were based on principles aimed at maintaining the biological plausibility of the resulting model, while working toward a model that used a minimum amount of elements. The development principles are:

•

The NBA CPG model uses the same leaky integrate and fire model as the escape swim network.

•

The NBA CPG model uses only elements shown to be necessary and sufficient for pattern generation.

•

Each element added to the NBA CPG model requires a justification.

o

To limit the number of free variables, the model is built by using multiple instances of a network module (the basic neural circuit).

•

Elements used in the NBA CPG use the parameter values used in the escape swim network, where possible.

o

Any departure from an escape swim network parameter value requires a justification.

o

Any departure from an escape swim network parameter value needs to be the minimum required to meet the justified goal.

Our exploration of the model showed that the mechanisms that are necessary and sufficient for pattern generation in the Tritonia Diomedia escape swim network are reciprocal inhibition between DSI and VSI combined with delayed excitation of VSI. These mechanisms are used as a starting point for the development of a biologically plausible NBA CPG. Exploration of the model also showed that the A-current, multicomponent synapses and spike frequency adaptation were not necessary. In fact,

15

the A-current slows down the process of pattern shifting, whereas we need to increase that process compared to the behavior of the Tritonia Diomedia escape swim network. The input to the CPG that triggers pattern generation is not specified in the NBA. It is assumed that posing the binding question to the network results in a short duration spike train. This burst excites the first neuron in the network, thereby starting pattern generation. The operationalization of the input signal for implementation is a pregenerated spike train that excites the first neuron via a synaptic connection. This is the same method used for the input signal that originates from DSI in the escape swim network. The input signal used is a 25ms 200Hz spike train with a total of six spikes. The output required to activate gating circuits or inhibitory neural assemblies is not specified. It is assumed that an activation of these elements can be achieved by an excitatory connection originating from a spiking neuron. Neither gating circuits nor inhibitory neural assemblies are present in the model. A spike train generated by a neuron in the CPG NBA model associated with specific gating circuits or inhibitory neural assemblies is taken to be sufficiency to represent activation of these elements, because they can be used to activate these elements in the NBA as presented in (van der Velde & de Kamps, 2006).

4.1. The basic neural circuit. The basic neural circuit of pattern generation in the NBA model includes the neuron that is currently active (N1) and the neuron that will be active in the next step (N2, see figure 6A). These neurons are based on the DSI and VSI neurons of the Tritonia Diomedia escape swim network, respectively. Connectivity between the neurons in based on the minimal elements required to implement reciprocal inhibition between N1 and N2 combined with delayed excitation of N2 by N1. Therefore, the basic circuit includes the Na neuron, which is based on the C2 neuron. Synaptic connections consist of an inhibitory connection from N1 to N2, an inhibitory connection from N2 to N1, an excitatory connection from N1 to Na and an excitatory connection from Na to N2 (see figure 6B). Pilot experiments showed that an additional connection, an inhibitory connection between N2 and Na, was required. The connection is based on the inhibitory connection between VSI and C2. This connection was required to compensate for the 16

excess excitation of Na. More details for this addition can be found the discussion of the networks synaptic parameters.

Figure 6. Basic neural circuit for the NBA CPG. A shows the neurons corresponding with step 1 (N1) and step 2 (N2) in the control sequence used to answer the binding question. B shows the connectivity of the basic neural circuit, and introduces an additional neuron, Na. C shows how the basic neural circuit is extended for use in pilot experiments, by adding an additional connection linked to a virtual neuron (Input). This neuron represents the pre-generated input spike train.

Changing from one state in the basic neural circuit to the other involves the following steps. When N1 spikes, its fast connection to Na drives that neuron to threshold. Meanwhile, its inhibitory connection depolarizes N2. When Na starts to spike, it in turn excites N2. However, due to N2 having been depolarized by N1, Na takes relatively long to drive N2 to threshold. As soon as N2 starts to spike, activation of both N1 and Na is inhibited by the connections from N2. With activation of N2, the basic neural circuit network has changed to the next state.

4.1.1. Determining Network Parameters To determine the network parameters a series of pilot experiments were run. These experiments used the basic neural circuit's architecture, extended by a pre-generated input spike train and a connection linking this input to N1 (see Figure 6C). The initial parameter values for the neurons and synapses in the basic neural circuit were the parameters used for the corresponding elements in the escape swim network. This excludes the initial parameter for the connection between Input and N1, which was set up to generate six spikes in N1 within a 25ms period. After each pilot experiment the output of the network was compared to the desired output. The desired output is defined as a 25 ms duration six spike burst by N1, immediately followed by a 25ms six spike burst by N2. Keeping the number of spikes in each state identical was required to ensure that the basic neural circuit module can 17

be repeated to form the full NBA CPG, without resulting in the loss or accumulation of activation. In the pilot experiments a single parameter was varied across a range with certain step size. A simulation of the basic neural circuit was run for each value in the resulting set. Analysis of the pilot experiment output guided the search for the parameter set that resulted in the desired input. The analysis was performed post-hoc and by hand after each experiment. Such an analysis would typically yield a parameter value that would best fit a specific characteristic of the desired output, i.e. the duration in a particular neuron or the number of spikes in a particular neuron. Based on the analysis, network parameters were modified one-by-one with the goal of achieving a specific improvement – a better fit on a specific characteristic – towards achieving generation the desired pattern. A large number of iterations were required to arrive at the final set of parameters. Several iterations let to situations where further improvement could not be achieved without modifying parameters set to achieve an earlier improvement. In such situations, the modifications of the guided search were rolled back to the point before these earlier improvement were achieved. This effectively resulted in a human guided traversal of a search tree. Since the goal of the model is to establish existence proof for a biologically plausible implementation of a neural blackboard CPG a more formalized method to determine the models parameter set was not required.

4.1.2. Neural Parameters The basic neural circuit is intended to be used as a module to build the complete NBA CPG network by iterating the pattern of the basic circuit (see figure 7). This iterative nature results in the neuron fulfilling the role of DSI in one step of pattern generation and the role of VSI in the next step. The neural parameters of the state neurons (N1, N2, etc) are based on VSI parameters. The choice to base the state neuron parameters on VSI, instead of on DSI, is due to role of VSI in the delayed excitation mechanism in the escape swim network. The model equations and parameters are given in the Appendix. Both membrane capacitance (Cn) and resistance (Rn) were modified to increase the responsiveness of the state neurons to input. Increased responsiveness was required 18

to allow the neuron to work with the smaller time scale the NBA CPG operates in. Spike bursts and intervals in the escape swim network have durations exceeding 110 ms, whereas the NBA CPG works with bursts and intervals of 25 ms. Increasing neurons' responsiveness is defined here as increasing the effect a single spike has on the post synaptic neuron. Decreasing the membrane capacitance results in an increased change of membrane potential, given the same current. Increasing membrane resistance results in less leakage, allowing a more rapid buildup of membrane potential. The values for these parameters are given in Appendix, table 1.

Figure 7. The NBA CPG network architecture, highlighting the repeated instances of the basic neural circuit architecture (within the dotted line).

The parameters of the delay neuron (Na, Nb, etc) are based on C2 parameters of the escape swim network. However, to allow these neurons to work within the time scale required, extensive modifications were required (Appendix, table 1). Membrane capacitance was decreased while membrane resistance was increased to increase the neuron's responsiveness. Membrane resistance was set to the resistance value of DSI, but membrane capacitance was reduced to one fourth of the lowest value seen in the escape swim network. To increase responsiveness further, threshold values were set closer to resting potential (Er). Reducing the threshold potential ( ∞ ) reduces the potential change required to trigger a spike. Reducing the threshold value after a spike ( 0 ) reduces the potential change required to trigger consecutive spikes in the neuron. For both state and delay neurons, the threshold's time constant (τθ ) was set to half the value used for VSI. This modification increases the spike frequency of these neurons. The increased spike output allows a neuron to drive connected neurons in a shorter time span, as required by the smaller time scale of the NBA CPG. With the A-current disabled and without the injected stabilizing current, determining the initial values for the neural variables becomes straightforward. The initial value for the time of the last action potential was maintained. Membrane potential was set equal to the neurons' resting potential. Threshold value was set equal to the neurons' steady-state threshold potential (see Appendix, table 2). 19

4.1.3. Afterpotential Parameters The IK after potential was implemented using a synaptic connection, as in the escape swim network. All neurons used the parameter set used for VSI, including the C2 based delay neurons (see Appendix, table 3). This choice was made because the VSI parameter set implements a rapid onset and short duration afterpotential that increases the firing rate of the neurons. The synapse implementing the IC afterpotential was disabled for all neurons. The IC afterpotential was used to implement spike frequency adaptation in the escape swim network. However, while IC has a minimal effect on the escape swim network, it confounds the NBA CPG network's behavior by introducing an inhibitory effect of a relatively large time scale.

4.1.4. Synaptic Parameters The synaptic parameters for the connections in the NBA CPG are based on connections in the escape swim network. The synaptic reversal potential (Esyn), which determines whether a connection is inhibitory or excitatory, are unchanged. However, ¹

while both temporal parameters (τo and τd) and maximum conductance ( g syn ) required a deviation from the escape swim network values for the network to generate the desired pattern. The onset time constant (τo) of all connections are reduced by a factor between 4 to 60, resulting in values between 5 and 15 ms. These modifications bring the onsets of the synaptic current (Isyn) within the time scale of the NBA CPG. Together with the modifications that increase responsiveness and spike frequency, this modification allows the network to produce a sequence of short duration spike bursts without quiescent intervals. The delay time onset values (τd) are scaled down to match the reduced τo values and meet the temporal characteristics of the NBA control sequence. ¹

The g syn values are used to balance the interactions of the post synaptic effects on the neurons (see Appendix, table 3). The parameters for the excitatory connections were set such that the post synaptic neuron received sufficient excitation to reach threshold at the desired moment, without causing the membrane potential exceeding the threshold value after a spike ( 0 ). Exceeding 0 results in a spike frequency in excess of the 500Hz limit, which jeopardizes the validity of the model. The N1 to Na connection is the faster of the two connections implementing the delayed excitation mechanism. The τo value is set equal to the IK value for this parameter, the lowest value for this parameter found in the escape swim network. The τd value is set such that the post synaptic conductance change caused by the first spike 20

in the burst has not decayed significantly before the post synaptic conductance change caused by the last spike in a burst takes effect. Together with a modification of the ¹

g syn value, this allows Na to quickly reach threshold potential. The Na to N2 connection is the slower of the two connections of the delayed

excitation mechanism. The τo value is set such that the contribution of both connections is similar to that observed in the escape swim network. Unlike the other excitatory connection, the τd value is set close to the τo value. This reduces the cumulative effect of consecutive post synaptic effects, preventing premature activation of N2. Both reaching threshold and not exceeding 0 proved impossible for both Na and N2, when only the excitatory connection parameters were modified. Inhibition is used to compensate for the excess excitation required by Na and N2 to reach threshold potential within the desired time frame. For N2, this inhibition was implemented using the inhibitory connection from N1 to N2. However, the Na neuron did not receive any inhibition connections. As noted above, this required the introduction of a new element in the model, the inhibitory connection between N2 and Na. The τo and τd values for these inhibitory connections are set such that the neurons are inhibited when the excess excitation would otherwise drive the spike frequency above 500 Hz. For the inhibitory connection to Na this is just after Na receives the last input from Na. Since the last spike of N1 coincides with the first spike of N2, inhibition of ¹

Na by N2 prevents Na from reaching the spike frequency limit. The g syn value for this connection is set such that Na stops firing shortly after N2 starts to fire. The inhibitory connection from N1 to N2 compensates for excess excitation that occurs as soon as N2 reaches threshold. To prevent high firing rates, N2 is inhibited before it receives excitation from Na. Excitation from Na needs to overcome the inhibition from N1, before N2 reaches threshold. The inhibitory connection from N1 to N2, together with the relatively slow excitation by the connection from Na, also ¹

implements the delayed excitation mechanism. The g syn value for this connection is ¹

set together with the g syn value of the excitatory connection from Na to N2 to ensure that N2 reaches threshold 26ms after N1 starts to fire. The parameters of the inhibitory connection from N2 to N1 are set such that N1 ceases to fire as soon as N2 becomes active. When several basic neural circuits modules are connected in series, the neuron representing the previous state of the network will fire after the desired period of activation. This is undesirable in relation to the temporal characteristics specified for the NBA control sequence. Any additional spikes would also exceed the six spikes per burst requirement for modular development of the NBA CPG. The τo is set to the lowest value for this parameter ¹

found in the escape swim network. Together with a relatively high g syn value, this results in rapid hyperpolarization of N1, when N2 becomes active. The τd is set such 21

that N1 will not be able to reach threshold potential before the excitatory post synaptic effect has decayed back to zero. 4.2. The NBA CPG

Figure 8. The NBA CPG network architecture. The neurons that implement the steps in the NBA control sequence have fat lines. The 'Input' neuron represents the pregenerated input signal.

The NBA CPG consists of 15 instances of the basic circuit (see figure 8). Each of these instance represents a state of the network. An additional instance is appended to the network. This instance, consisting of No and N16, provides N15 with the inhibition required to be deactivated after generating its 25ms spike train, but plays no role in the control sequence. To activate the network the pregenerated input spike train is added and connected to N1. The parameters of this connection are identical to those of the connection between Input and N1 used for the basic circuit.

4.2.1. Results The output of the NBA CPG model meets the specifications set for the NBA control sequence. Each of the state neurons fires a single burst during the sequence and only one state neuron spikes at any given time (see figure 9). Each state neuron is spikes for a period of 25 ms, after which spiking ceases and the next neuron in the sequence starts to spike. To verify that pattern generation of the NBA CPG is based on a combination of reciprocal inhibition and delayed excitation, several additional experiments were performed. 22

Figure 9. The input signal and spike train data from all fifteen state neurons representing network states. Generated using the NBA CPG model.

The inhibitory connection from N1 to N2 has the same function as the inhibitory ¹

connection on from DSI to VSI. Disabling the connection, by setting the g syn value for this connection to zero, resulted in increased spike frequency and early onset of spike bursts in the state neurons. The same effect observed when the inhibitory connection from DSI to VSI is disabled. The inhibitory connection from N2 to N1 is of lesser importance in the NBA CPG than the connection from VSI to DSI is in the escape swim network. Disabling the connection in the NBA CPG results in early onset onset of spike bursts in the state neurons, instead of disrupting pattern generation as observed in the escape swim network. This is due to the different input signals used in these models. In the linear NBA CPG the input signal is short and excitation propa propagates gates down the network. This removes the need to inhibit the previous network state for successful pattern generation. However, the inhibition connection proved to be vital to meet NBA CPG control sequence specification.

23

Figure 10. Delayed excitation by inhibition followed by excitation in both the 'Escape Escape Swim Network' (left) and the 'NBA NBA CPG CPG'' (right). The Escape Swim Network conductance changes shown are those of VSI in the control condition. The NBA CPG conductance changes shown are those of the N2 state neuron.

Disabling the additional inhibitory connection between N2 and Na resulted in increased spike burst duration, increased spike frequency and early onset of spike bursts in the state neurons. Disabling both the inhibitory connections from VSI to C2 has the same effect in the escape swim network. This shows that adding the connection between N2 and Na resulted in the desired effect. It also shows that the same effect can be observed in the escape swim network.

Figure 11. The he relative contributions to delayed excitation by the two excitatory connections in the basic neural circuit of the NBA CPG.

24

Analysis of conductance changes in the NBA CPG showed delayed excitation of the state neurons, implemented by an inhibitory effect followed by an excitatory effect (see figure 10, right). This is identical to the pattern observed in the escape swim network. The relative contributions to delay excitation of the connection between the current state neuron and the delay neuron and the connection between the delay neuron and the next state neuron matched those seen in the escape swim network. The connection between the current state neuron and the delay neuron was responsible for 32% of the delay, the connection between the delay neuron and the next state neuron was responsible for the remaining 68% (see figure 11).

4.3. Discussion The goal of this article was to give an existence proof of a biologically plausible implementation of the Neural Blackboard (NBA) Central Pattern Generator (CPG). The biological plausibility of the NBA CPG network is derived from its relation to the escape swim network. Therefore, biological plausibility depends on the measure of correspondence between the NBA CPG and the escape swim network. As far as the NBA CPG model architecture and pattern generation mechanisms are concerned, deviation from the escape swim network is both limited and justifiable. Network elements that did not contribute to pattern generation in the escape swim network were removed from the NBA CPG. Each of the elements present in the NBA CPG contributes either directly to pattern generation or is required to meet the specified temporal characteristics of the control sequence. Several parameter values used in the NBA CPG do deviate significantly from those seen in the escape swim network. The departure from escape swim model parameter values was required to allow the NBA CPG to function in the time scale required for control sequence, which differs significantly from the escape swim time scale. This concerns the following parameters: the membrane capacitance (Cn) of the delay neurons, the threshold value after a spike ( 0 ) of the delay neuron, the threshold's time constant (τθ) of both types of neuron and the temporal parameters (τo and τd) of several of the synaptic connections. The membrane capacitance parameter value for the delay neuron was low in comparison to the value in the escape swim network. While the scientific literature does not offer direct support for the membrane capacitance value used in the simulation, it does provide evidence for the improved responsiveness it achieves. Markram, Lubke and Frotscher (1997) and Feltmeyer et al. (1999) reported spike 25

latencies as small as

1.7 ± 0.9 ms and 0.92 ± 0.6 ms respectively, in the

somatosensory cortex of the developing rat. The 8ms spike latency of the simulated delay neuron thus falls well within biologically plausible boundaries. The values for the threshold value after a spike and the threshold's time constant were modified to increase spike frequency. Again, scientific literature does not offer direct support for the parameter values, but does provide evidence for the models spike frequency. Ranck (1973) defines a complex spike by a pyramidal cell in the dorsal hippocampus of a rat as a series of two to seven individual spikes with 1.5-6ms interspike intervals. This compares favorably with the average 4ms interspike interval of the simulated six spike bursts. The values used for the temporal parameters (τo and τd) of several of the synaptic connections were low in comparison to those used in the escape swim network. However, the values used in the model fall within the range of the values reported in the literature. Bannister and Thomson (2007) report EPSPs with 4-90% rise times in the range of 1 to 3ms in the neocortex of both cat and rat. Duration of these EPSP, measured by determining the EPSP width at half amplitude, are between 5 and 6ms. For IPSP, Lambert et al. (1996) report 4-90% rise time of about 2ms and a decay phase of up to 150ms. Given the results of the simulation, in combination with the corroborating physiological data, the presented model is considered to be an existence proof for a biologically plausible implementation of the NBA CPG. Further research could perhaps show how the parameters of a CPG for activation control in the neural blackboard architecture could be derived in a more general manner, e.g., using iteration methods (e.g., Consolini and Lini, 2014). However, the development of the CPGs for controlling activation would also depend on the development of the architecture itself. The research on this combined development could perhaps also indicate why the cyclic behavior of a CPG could be transferred into (more) linear behavior as demonstrated here. However, we do not exclude the possibility that a combined development of the architecture and activation control could also result in cyclic behavior of CPGs interacting with the activation that unfolds in the blackboard architecture. But given the current state of the architecture, the method we applied suffices for demonstrating the usefulness of activation control by a CPG. 26

5. Conclusions The presented NBA control circuit is based on the pattern generator responsible for the escape swim behavior of Tritonia Diomedia. The simulation gives bottom-up support to the claims that control of sequential processing required for language can be provided by neural mechanisms based on motor control architecture (Dominey 1997, van der Velde & de Kamps, 2011). These claims are based on theoretical similarities between motor and language control, and control of higher level cognitive processing in general (e.g., Llinás, 2002). Unlike cognitive systems such as sensory processing, which rely on parallel processing, both motor control and language require sequential processing. The physical constraints, imposed both by the environment and the bio-mechanics of the agent, force a sequential structure on movement. For example; for a successful escape swim Tritonia Diomedia must perform a dorsal flexation followed by a ventral flexation, or vice versa. Consecutive flexations of the same side will fail to lift the mollusk off the sea bed. The physical constraints that have to be dealt with in motor control are not unlike the syntactic constraints that have to be dealt with in language. As shown in van der Velde en de Kamps (2010), the binding of words in a sentence structure is dependent on syntactic properties of the word being processed. The feedforward network (FNN) trained in their experiments relies on three sources of input: the word type of the word being processed (e.g., noun or verb), feedback from the FFN and feedback from the NBA. Each type of input provides syntactic information that constraints how the word being processed is bound to the sentence representation in the NBA. Answering a question about sentences stored in the NBA likewise depends on syntactic constraints. A question initiates a search through the sentences stored in the NBA, constrained by the question's syntactic structure and the constituent words. The gating circuits in the NBA prevent a parallel search through sentences stored, because they block the free flow of activation required for a parallel processing. However, the gating circuits – and the selective activation they implement - are required to represent the syntactic structure of sentences. Given that parallel processing and syntactic structure are mutually exclusive on the NBA, cognitive 27

processes involving sentence structure require sequential processing in this architecture. Another similarity between language and motor control could be found in the nature of the representations involved. Motor control depends on the successful activation and deactivation of sets of muscles in line with the sequential order as needed for the motor behavior to be executed. One could say that these sets of muscles form a constraint for motor control in the sense that they are “in situ”, as given by the scheme of the body of which they are a part. The representations of words, as given by the word assemblies illustrated in figure 1, are forms of in situ representation as well (van der Velde & de Kamps, 2011). As neural assemblies, they are distributed over the brain as suggested by Hebb (1949), and a repeated activation of a specific word in a sentence structure would activate the same neural assembly (or part thereof). This notion is illustrated in figure 2 with the representation of sentence structures with repeated words. To distinguish between these sentences, a control of activation of word assemblies is needed, to ensure that the correct sentence structure is activated instead of other sentence structures in which these word assemblies are a part. This in turn requires a control of the sequence of activation in the NBA needed to execute a given action, such as answering a binding question. The role of neural assemblies in cognition has received considerable attention (e.g., Buzsáki, 2010; Huyck & Passmore, 2013). Neural models of cognitive processing other that language in which sequential activation of neural assemblies would occur could be models related to multimodal neuronal representations (Barsalou et al., 2003), models based on convergence zones (e.g., Damasio and Damasio, 1994), neural models based on dynamic routing strategies (e.g., Van Essen et al., 1994; Zylberberg et al., 2010), or neural models related to the ‘Global Workspace’ of cognition (Baars & Franklin, 2007; Baars, Franklin & Ramsoy, 2013; Wiggins, 2012). In these models neural activation is also selectively routed in the serial processing of information (as in the process of answering ‘who does what to whom’ queries illustrated here). So, one could expect that control of sequential activation of neural assemblies would be important for these models as well. Such control could be given by CPGs as we have demonstrated here.

28

Acknowledgements We thank two anonymous reviewers for their comments made on an earlier version of the manuscript. The work of the Frank van der Velde was funded by the project ConCreTe. The project ConCreTe acknowledges the financial support of the Future and Emerging Technologies (FET) programme within the Seventh Framework Programme for Research of the European Commission, under FET grant number 611733.

References. Baars, B. J. and Franklin, S. (2007).

An architectural model of conscious and

unconscious brain functions: Global Workspace Theory and IDA. Neural Networks 20, 955–961.

Baars, B. J., Franklin, S. & Ramsoy, T. Z. (2013). Global workspace dynamics: cortical “binding and propagation” enables conscious contents. Frontiers in Psychology, 4: 200.

Bannister, Peter A. & Thomson, Alex M. (2007). Dynamic properties of excitatory synaptic connections involving layer 4 pyramidal cells in adult rat and cat neocortex. Cerebral Cortex, 17 (9), 2190-2203.

Barsalou, L. W., Simmons, W. K., Barbey, A. K. and Wilson, C. D. (2003). Grounding conceptual knowledge in modality-specific systems. Trends in Cognitive Sciences, 7, 84-91. Buzsáki, G. (2010). Neural Syntax: Cell Assemblies, Synapsembles, and Readers. Neuron, 68, 362-85.

Consolini, L. & Lini, G. (2014). A Gauss-Newton Method for the Synthesis of Periodic Outputs With Central Pattern Generators IEEE Transactions on Neural Networks and Learning Systems, 25, 1394-1400.

Damasio, A. R. and Damasio, H. (1994). Cortical systems for retrieval of concrete knowledge: The convergence zone framework. In C. Koch and J. L. Davies (eds.) Large-scale neuronal theories of the brain (pp. 61-74). Cambridge, MA: MIT Press.

29

Dzeladini,F., van den Kieboom, J. & Ijspeert, A. (2014). The contribution of a central pattern generator in a reflex-based neuromuscular model. Frontiers in Human Neuroscience, 8: 371. doi: 10.3389/fnhum.2014.00371

Dominey, P.F. (1997). An Anatomically Structured Sensory-Motor Sequence Learning System Displays some General Linguistic Capacities. Brain and Language, 59, 55-75.

Durstewitz, D., Seamans, J.K. & Sejnowski, T.J. (2000). Neurocomputational Models of Working Memory. Nature Neuroscience, 3 (sup), 1184-1191. Feldmeyer, D., Egger, V., Lübke J., Sakmann B. (1999). Synaptic connections between excitatory layer 4 neurones in the ‘barrel field’ of rat somatosensory cortex. Journal of Physiology, 521, 169–190.

Gerth, S. & beim Graben, P. (2009). Unifying syntactic theory and sentence processing difficulty through a connectionist minimalist parser. Cognitive Neurodynamics, 3, 297-316.

Getting, P.A., Lennard, P.R. and Hume, R.I. (1980). Central Pattern Generator Mediating Swimming in Tritonia I. Identification and Synaptic Interactions. Journal of Neurophysiology, 41, 151-164.

Getting, P.A. (1981). Mechanisms of Pattern Generation Underlying Swimming in Tritonia. I. Neuronal Network Formed by Monosynaptic Connections. Journal of Neurophysiology, 46, 65-79.

Getting, P.A. (1983a). Mechanisms of Pattern Generation Underlying Swimming in Tritonia. II. Network Reconstruction. Journal of Neurophysiology, 49, 1017-1035. Getting, P.A. (1983b). Mechanisms of Pattern Generation Underlying Swimming in Tritonia. III. Intrinsic and Synaptic Mechanisms for Delayed Excitation. Journal of Neurophysiology, 49, 1036-1050.

Getting, P.A. and Dekin, M.S. (1985). Mechanisms of Pattern Generation Underlying Swimming in Tritonia. IV. Gating if Central Pattern Generator. Journal of Neurophysiology, 53, 466-480.

Getting, P.A. (1989). Reconstruction of small neural networks. In Koch, C. and Segev, I. (Eds.), Methods in Neural Modeling: From Synapses to Networks, 171-194. Cambridge: MIT Press.

30

Grillner, S., Markram, H., De Schutter, E., Silberberg, G., and LeBeau, F.E.N. (2005). Microcircuits in action – from CPGs to neocortex. Trends in Neurosciences, 28, 525533. Hebb, D. O. (1949). The organisation of behaviour. New York: Wiley. Hinaut, X, and Dominey, P.F. (2013). Real-time parallel processing of grammatical structure in the fronto-striatal system: A recurrent network simulation study using reservoir computing. PLoS ONE 8(2): e52946. doi:10.1371/journal.pone.0052946 Hume, R.I., Getting, P.A. and Del Beccaro, M.A. (1982a). Motor Organisation of Tritonia Swimming. I. Qualitative Analysis of Swim Behavior and Flexion Neuron Firing Patterns. Journal of Neurophysiology, 47, 60-74. Hume, R.I. and Getting, P.A. (1982b). Motor Organization of Tritonia Swimming. II Synaptic Drive to Flexion Neurons from Premotor Interneurons. Journal of Neurophysiology, 47, 75-90.

Huyck, C. R. & Passmore, P. J. (2013). A review of cell assemblies. Biological Cybernetics, 107, 263–288.

Lambert J.D., Fueta Y., Roepstorff A. & Andreasen M. (1996). Analysis of the kinetics of synaptic inhibition points to a reduction in GABA release in area CA1 of the genetically epileptic mouse, Epilepsy Research, 26, 15-23. Lennard, P.R., Getting, P.A., & Hume, R.I. (1980). Central Pattern Generation Mediating Swimming in Tritonia II. Initiation, Maintenance, and Termination. Journal of Neurophysiology, 44, 165-173.

Llinás, R. R. (2002). I of the Vortex: From Neurons to Self. Cambridge, MA: MIT Press. Markram H., Lubke J., Frotscher M., Roth A., Sakmann B. (1997). Physiology and anatomy of synaptic connections between thick tufted pyramidal neurones in the developing rat neocortex. Journal of Physiology, 500, 409–440. Nassour, J., Hénaff, P., Benouezdou, F. & Cheng, G. (2014). Multi-layered multipattern CPG for adaptive locomotion of humanoid robots. Biological Cybernetics, 108, 291-303.

Newman, J., Baars, B.J. & Cho, S.-B. (1997). A Neural Global Workspace Model for Conscious Attention. Neural Networks, 10, 1195-1206.

31

Nor, N. M. & Ma, S. G. (2014). A Simplified CPGs Network with Phase Oscillator Model for Locomotion Control of a Snake-like Robot. Journal of Intelligent & Robotic Systems, 75, 71-86.

Pulvermüller, F. (1999). Words in the Brains Language. Behavioral and Brain Sciences, 22, 253-336. Quian Quiroga, R. (2012). Concept cells: the building blocks

of declarative memory functions. Nature Reviews Neuroscience, 13, 587-597. Ranck, J.B.Jr. (1973). Studies on single neurons in dorsal hippocampual formation and septum in unrestrained rats. Experimental Neurology, 41, 461-555. Sagara, T. & Hagiwara, M. (2014). Natural language neural network and its application to question-answering system. Neurocomputing, 142, 201-208. Sakurai, A., Gunaratne, C.A. & Katz, P. S. (2014). Two interconnected kernels of reciprocally inhibitory interneurons underlie alternating left-right swim motor pattern generation in the mollusk Melibe leonina. Journal of Neurophysiology, 112, 13171328. Shahbazia, H., Jamshidib, K., Monadjemib, A. H. & Eslamib, H. (2014). Biologically inspired layered learning in humanoid robots. Knowledge-Based Systems, 57, 8-27. Shi Y. Wei-Qiang Zhang W-Q., Jia Liu J. & Michael T Johnson, M. T. (2013). RNN language model with word clustering and class-based output layer. EURASIP Journal on Audio, Speech, and Music Processing 2013:22.

van der Velde, F. (2014). Communication, concepts and grounding. Neural Networks, http://dx.doi.org/10.1016/j.neunet.2014.07.003 van der Velde, F & de Kamps, M. (2006). Neural blackboard architectures of combinatorial structures in cognition. Behavioral and Brain Sciences, 29-01, 37-70. van der Velde, F & de Kamps, M. (2010). Learning of control in a neural architecture of grounded language processing. Cognitive Systems Research, 11, 93-107. van der Velde, F and de Kamps, M. (2011). Compositional connectionist structures based on in situ grounded representations. Connection Science, 23, 97-107. Van Essen, D. C., Anderson, C. H. and Olshausen, B. A. (1994). Dynamic routing strategies in sensory, motor, and cognitive processing. In C. Koch and J. L. Davies (eds.) Large-scale neuronal theories of the brain (pp. 61-74). Cambridge, MA: MIT Press.

32

Wiggins, G. A. (2012). The Mind’s Chorus: Creativity before consciousness. Cognitive Computation, 4, 306–319.

Wu J.S., Wang N., Siniscalchi M.J., Perkins M.H., Zheng Y.T., Yu W., Chen S.A., Jia R.N., Gu J.W., Qian Y.Q., Ye Y., Vilim F.S., Cropper E.C., Weiss K.R. & Jing J. (2014). Complementary interactions between command-like interneurons that function to activate and specify motor programs. Journal of Neuroscience, 34, 651021. Yu, J. Z., Tan, M., Chen, J. & Zhang, J. W. (2014). A Survey on CPG-Inspired Control Models and System Implementation. IEEE Transactions on Neural Networks and Learning Systems, 25, 441-456.

Yuste, R. MacLean, J.N., Smith. J., and Lansner, A. (2005). The cortex as a central pattern generator. Nature Reviews Neuroscience, 6, 477-483. Zylberberg A., Fernandez Slezak D., Roelfsema P. R., Dehaene S., Sigman M. (2010). The brain’s router: a cortical network model of serial processing in the primate brain. PLoS Comp Biol 6: e1000765.

33

Appendix: Model and Network Parameters for the NBA CPG. The equations that describe the behavior of the neurons in the model are based on Getting (1989).

The membrane potential for a neuron is described by Cn = − I leak (Vm ) − I syn (Vm ) − I stim (t )

Here, Vm = membrane potential (mV) Cn = membrane capacitance (nF) Ileak = leak current (nA) Isyn = synaptic current (nA) Istim = externally applied current (nA)

Leakage current is given by I leak =

Vm − Er Rn

with Er = resting potential (mV) Rn = membrane resistance (MΩ)

Synaptic current is given by ¹

I syn = g syn . g syn (t ) (Vm − Esyn ) with ¹

g syn = maximum conductance for a given synapse (µS) Esyn = synaptic reverse potential (mV)

34

N

g syn ( t ) =

d d −o

∑(e

( ti − t )/ d

− e( ti −t )/ o

)

i =1

with

τo = onset time constant (msec) τd = decay time constant (msec) N = number of presynaptic spikes before time t ti = time of ith presynaptic spike (msec)

Threshold is given by

( t ) = ∞ + ( 0 − ∞ )e ( t ∞

x

− t )/

= steady-state threshold (mV)

= threshold value immediately after a spike (mV) tx = time of last spike (msec) τθ = threshold time constant (msec) 0

35

CV Authors Djurre van Dijk

Djurre van Dijk received his M.Sc. degree in Cognitive Psychology from Leiden University, the Netherlands, in 2010. His research interests included neural network simulation and biorobotics. He currently works at Triumph Studios as a video game producer."

Frank van der Velde

Prof. Frank van der Velde is chair of Technical Cognition at the department of Cognitive Psychology (CPE-GW) and the Centre for Telematics and Information Technology (CTIT) at the University of Twente. He obtained an MSc in Cognitive Psychology in 1982 (cum laude) and an MSc in Physics in 1989 (cum laude). He obtained a PhD in Cognitive Science in 1990. He has (co)-authored over 70 peerreviewed international publications, ranging from experimental psychology, modeling brain processes, neural dynamics to modeling (neuro) cognitive processing, and participated in national and European research projects. His research topics are focused on cognitive neuroscience and neuro-computation of visual attention, working memory, language and reasoning, and neural ‘blackboard’ architectures for high-level cognitive and productive processes, with the aim to implement these architectures in robotics, as with the iCub robot.

36

Table 1: Cellular Parameters Cell

Er

Cn

Rn

τθ

0

∞

State

-55

4.17

30

-43

-30

5

Delay

-50

1

30

-45

-11

5

Table 1 – Cellular parameters used in the NBA CPG. The characteristics of the neurons are described by: resting potential (Er in mV ), membrane capacitance (Cn in nF ), membrane resistance (Rn in MΩ), the steady-state threshold value ( ∞ in mV), the threshold value immediately after a spike ( 0 in mV) and the threshold's time constant (τθ ).

37

Table 2: Initial Values Cell

ts

θ (0)

Vm

State

-1100

-55

-43

Delay

-1100

-50

-45

Table 2 – The initial values of the cellular variables. The initialized values are: time of the last action potential (ts in ms), membrane potential (Vm in mV) and threshold potential (θ (t) in mV). Note that the initial value of the threshold potential equals the the steady-state threshold value of each neuron.

Table 3: Synaptic Parameters Pre

Post

Type

¹

g syn

Any

Any

IK

.8

Input

N1

EPSP

N1

Na

N1

Esyn

τo

τd

-80

5

6

.034

4

15

11

EPSP

.02

4

5

33

N2

IPSP

.015

-80

4

11

Na

N2

EPSP

.09

4

4

5

N2

N1

IPSP

.3

-80

5

11

N2

Na

IPSP

.005

-80

4

11

Table 3 – Synaptic parameters used in the NBA CPG. 'Pre' indicates the pre synaptic cell and 'Post' indicates the post synaptic cell. 'Type' states whether a connection is excitatory (EPSP), inhibitory (IPSP) or is used to implement an afterpotential current (IK). The numeric synaptic parameters are: ¹

maximum conductance ( g syn

in µS), synaptic reversal potential (Esyn in mV), onset time constant (τo

in ms) and delay time onset (τd in ms).

38