- Email: [email protected]
Sequence disambiguation and pattern completion by cooperation between autoassociative and heteroassociative memories of functionally divided hippocampal CA3
ARTICLE IN PRESS Neurocomputing 71 (2008) 3176– 3183
Contents lists available at ScienceDirect
Neurocomputing journal homepage: www.elsevier.com/locate/neucom
Sequence disambiguation and pattern completion by cooperation between autoassociative and heteroassociative memories of functionally divided hippocampal CA3 Toshikazu Samura a,, Motonobu Hattori b, Shun Ishizaki a a b
Graduate School of Media and Governance, Keio University, 5322 Endo, Fujisawa-shi, Kanagawa 252-8520, Japan Interdisciplinary Graduate School of Medicine and Engineering, University of Yamanashi, 4-3-11 Takeda, Kofu-shi, Yamanashi 400-8511, Japan
a r t i c l e in f o
a b s t r a c t
Available online 22 June 2008
The hippocampus memorizes event sequences and some events are shared by a few sequences. When a sequence is retrieved from the linked sequences, ambiguity of sequences becomes a serious problem because the shared events have some possible ways to retrieve other events. We have focused on location dependency elucidated from the hippocampus and suggested that CA3 is functionally divided into autoassociative and heteroassociative memories. Computer simulation results show that the functionally divided CA3 concurrently enables both sequence disambiguation and pattern completion. Consequently, it demonstrates that cooperation between both memories of CA3 brings out the abilities of sequence disambiguation and pattern completion in the hippocampus. & 2008 Elsevier B.V. All rights reserved.
Keywords: Functionally divided hippocampal CA3 Sequence disambiguation Pattern completion Autoassociative memory Heteroassociative memory
1. Introduction A promising hypothesis supporting many hippocampal processes has been suggested by Eichenbaum [6,5]. On the hypothesis, daily episodes are memorized as a relational network (previously called memory space) in the hippocampus. In the relational network, an episode is expressed by a sequence of events that are primitive elements of memory. Episodes are associated with each other by events shared among the episodes. For example, let us consider that the hippocampus memorizes two episodes, one is composed of the following events: A, B and C ðA ! B ! CÞ. The other is composed of the events: X, B and Y ðX ! B ! YÞ, where event B associates the two episodes. As a result, we can find out a new way from event X to event C by associating these episodes. Therefore, the relational network, that can flexibly associate episodes acquired, supports inference that derives new ways to deal with memory. Recent studies have suggested that the hippocampus relates to a flexible representation [17,9] used for inference. It follows that these suggestions support Eichenbaum’s hypothesis that leads to the inference. In the relational network, it is difficult to decide which pattern (C or Y) should be retrieved from event B. The ambiguity of sequences becomes a problem for retrieval. Thus, the sequence disambiguation is an essential function for retrieving the original episode from the relational network. Agster et al. have suggested that the hippocampus is required to accomplish sequence Corresponding author. Tel./fax: +81 466 48 6101.
E-mail address: [email protected] (T. Samura). 0925-2312/$ - see front matter & 2008 Elsevier B.V. All rights reserved. doi:10.1016/j.neucom.2008.04.026
disambiguation [1]. In the hippocampus, the CA3 region has unique recursive axons which are called recurrent collaterals (RCs). Because of its uniqueness, many researchers focused on it and proposed many computational models of CA3. Several of the models contribute to solving the sequence disambiguation problem by creating a code [14,8]. In those models, however, CA3 has been homogeneously modeled in spite of its heterogeneous anatomical properties [11]. Thus, we have suggested the importance of the model based on the heterogeneous properties that are elucidated from the location dependency of RCs and spike-timing dependent plasticity (STDP) [18]. On the basis of the location dependency, we have suggested that CA3 is functionally divided into autoassociative and heteroassociative memories [18]. In this paper, using computer simulations, we show that the functionally divided CA3 can disambiguate the overlapped sequences and enables pattern completion that is the already perceived ability of CA3 [16] to recall complete memory from incomplete set of cues. Consequently, it demonstrates that a cooperation mechanism between autoassociative and heteroassociative memories of the functionally divided CA3 brings out the abilities of the sequence disambiguation and pattern completion in the hippocampus.
2. Related works In this section, we review conventional CA3 models solving the sequence disambiguation problem. Levy et al. have focused on CA3 and proposed a simplified CA3 model composed of
ARTICLE IN PRESS T. Samura et al. / Neurocomputing 71 (2008) 3176–3183
McCulloch–Pitts neurons [15]. The model has the ability to create a local code for temporal context used for recognizing a subsequence of a large sequence. Since the model solves sequential problems including the sequence disambiguation with the context code, they have suggested the importance of the context for hippocampal functions. Moreover, they have proposed that fundamental properties to create the codes agree to the properties of CA3 [14]. In their study, McCulloch–Pitts neurons model spike rate, but not spike timing. However, STDP that is a rule of changing synaptic weights depending on spike timing exists in the hippocampus [3]. Hayashi et al. have proposed the hippocampal model [8] including CA3 and CA1 receiving from CA3. The synaptic weights of the model are updated by STDP. This model can also create context-like information by transforming temporal information to spatial information in CA3 and this context of CA3 leads to the sequence disambiguation in CA1. Hayashi’s model corresponds to the review that cooperation between CA3 and CA1 achieves the sequence disambiguation [13]. However, the spatial pattern completion function of CA3 is undescribed in this model. Furthermore, although heterogeneous anatomical properties that are the subregional local dependency of RCs have already been found in CA3, it was modeled homogeneously in their models. On the other hand, Samura et al. [18] have focused on the anatomical findings. Moreover, they have focused on the location dependency of STDP [20] and suggested that CA3 is functionally divided into autoassociative and heteroassociative memories. So, it seems plausible that the autoassociative memory leads to pattern completion and that the heteroassociative one which can reflect temporal information in synaptic weights leads to the transformation for creating a context. That is, the functionally divided CA3 may exist for the abilities of pattern completion and the creation of a context for the sequence disambiguation.
3177
3.2. Connections within CA3 CA3 neurons are connected recursively to other neurons by RCs. Fig. 2(a) shows the relationship between the location of a neuron and the projection zone of its RCs [11]. First, the RCs of CA3c neurons are limited to the area surrounding them. Second, the RCs of CA3b neurons are widely spread. Projection onto CA3c becomes a more temporal location than the position of its source and that onto CA3a becomes a more septal location. Finally, the RCs of CA3a neurons are limited to CA3a and CA3b. Projection onto CA3b becomes a more temporal location. In addition to the location dependency of the projection zones, the dendritic locations of RCs depend on where neurons receive RCs in the CA3 [11] (Fig. 2(b)). CA3c neurons receive RCs at a distance from a soma, while CA3a and CA3b receive RCs near a soma. Additionally, we review projection toward CA3 from other hippocampal regions [10]. Fig. 2(b) shows connections to CA3 from two regions: EC and DG. As shown in this figure, DG connects to all CA3 subregions and EC connects only to CA3a and CA3b. It was suggested that the connections from DG to CA3 contribute to memorization, while those from EC to CA3 are required for retrieval [19].
4. Physiological backgrounds of hippocampus 4.1. Spike-timing dependent plasticity A neural network memorizes information from the outside world by changing synaptic weights between neurons. In the hippocampus, STDP is observed as a rule of changing synaptic weights [3]. STDP determines the magnitude of a synaptic change and its polarity (potentiation or depression) according to the interval between pre- and postsynaptic spikes. Furthermore,
3. Anatomical backgrounds of hippocampus 3.1. Structure of hippocampus Fig. 1 shows the structure of the hippocampus. The hippocampus is divided into three regions: Dentate gyrus (DG), CA3 and CA1. Information is transmitted from DG to CA1 via CA3. Entorhinal cortex (EC) works as an interface between the cortex and the hippocampus and transmits information to all hippocampal regions. CA3 has RCs which recursively connect CA3 neurons to other CA3 neurons. Furthermore, CA3 is segmented into three subregions: CA3a, CA3b and CA3c (Fig. 1).
Fig. 1. Connections between the hippocampal regions.
Fig. 2. Connections within CA3. (a) Projection zone of RCs (circle: source neuron, ellipse: projection zone of the circled neuron in it). (b) Dendritic locations of RCs (inverse triangle: soma, forked line: dendrite, dashed line: RCs, chain line: external input).
ARTICLE IN PRESS 3178
T. Samura et al. / Neurocomputing 71 (2008) 3176–3183
recent studies suggested that STDP shows asymmetric profile or symmetric one depending on the density of inhibitory interneurons [20]. Symmetric profile STDP (SSTDP) is observed from a highdensity area near a soma (Fig. 3(a)), while asymmetric profile STDP (ASTDP) is observed from a low-density area distant from a soma (Fig. 3(b)). Therefore, the change of STDP profile correlates with a synaptic location on a dendrite. In other words, STDP shows the dendritic location dependency. Many experimental results of STDP are obtained from CA3–CA1 synapses, which are called Schaeffer collaterals (SCs). However, Debanne et al. observed STDP from RCs and suggested that RCs appear to be identical to SCs in both their basic properties and mechanisms of synaptic plasticity [3,4]. In addition, several inhibitory interneurons exist in the CA3 region [7], and they seem to be similar to those in the CA1 region. Thus, we apply the observations of SCs to RCs.
4.2. Theta phase precession The hippocampus receives inputs from the cortex via EC (Fig. 1). Then, theta phase precession, which emerges from EC, regulates the inputs to the hippocampus and contributes to memorization of sequences [21]. Theta phase precession is the phenomenon that the phase of neuronal firing gradually advances in each cycle of theta wave. In particular, a neuron always begins to fire at a certain phase and its firing phase advances to earlier phase in the next cycle. The firing of the neuron ceases, when the
phase advance of a neuron exceeds about one cycle of the theta wave.
5. Functional divisions of hippocampal CA3 From the integration of the location dependency, CA3c neurons receive RCs at distal dendrites and their plasticity becomes ASTDP, while CA3a and CA3b neurons receive RCs at proximal dendrites and show SSTDP. Here, we discuss a functional difference between the profiles. Under SSTDP, simultaneous firing leads to potentiation and time lag leads to depression (Fig. 3(a)). Thus, an instantaneous firing pattern of a network is mapped onto synaptic weights. This means that SSTDP suits autoassociative memory. On the other hand, ASTDP potentiates synapses when postsynaptic neurons fire after presynaptic firings. Conversely, if the firing order is reversed, synapses between them are depressed (Fig. 3(b)). Therefore, synaptic weights reflect the order of firing and ASTDP suits heteroassociative memory. Consequently, the CA3 region is functionally divided into two functions: CA3a and CA3b regions work as autoassociative memory, while CA3c region works as heteroassociative memory.
6. Hippocampal CA3 model 6.1. Neuron model A proposed hippocampal CA3 model consists of spiking neuron models modified from an integrate-and-fire neuron models which were used by August et al. [2]. The following equation shows the membrane potential of the jth neuron at time t.
tm
dV j ðtÞ ¼ Ij ðtÞ V j ðtÞ, dt
(1)
where Ij ðtÞ denotes synaptic current. tm denotes the membrane time constant which depends on the location of a neuron in CA3. The following equation shows the output of the jth neuron at time t, ( 1; V j ðtÞ4y; xj ðtÞ ¼ (2) 0; V j ðtÞpy: When the membrane potential exceeds threshold y, the cell fires and the membrane potential is reset to V init . The refractoriness of the jth neuron is realized by suspension of calculating Eq. (1) for time s after the firing. The input of the jth neuron through RCs at time t is defined as follows: X IRC wij ðtÞxi ðt dÞ, (3) j ðtÞ ¼ i
where wij ðtÞ denotes the synaptic weight between the ith neuron and the jth one at time t and d denotes the synaptic delay. The synaptic current of the jth neuron at time t is given by 2 ðIRC dIj ðtÞ Ij ðtÞ j ðtÞÞ ¼ RC , þ zj ðtÞ ts dt Ij ðtÞ þ Iinh ðtÞ
(4)
where Iinh ðtÞ means inhibitory synaptic current. zj ðtÞ ¼ f0; 1g denotes external input from EC and DG at time t and ts shows current time constant. The following equation means the inhibitory synaptic current at time t, Fig. 3. Profiles of STDP. A relative timing between pre- and postsynaptic neurons is given by subtracting presynaptic spike time and postsynaptic one. According to the timing and the profiles, the rate of synaptic change is determined. (a) Symmetric profile. (b) Asymmetric profile.
dIinh ðtÞ Iinh ðtÞ RC ¼ Iext , inh ðtÞ þ Iinh ðtÞ dt tinh
(5)
where Iext inh ðtÞ shows inhibition which arises from external inputs and IRC inh ðtÞ shows inhibition which arises from RCs. tinh denotes the
ARTICLE IN PRESS T. Samura et al. / Neurocomputing 71 (2008) 3176–3183
3179
time constant of inhibitory synaptic current. The external inhibition at time t is given as follows: Iext inh ðtÞ ¼
N ext ðtÞ , Rext firing
(6)
where Next ðtÞ means the number of neurons which receive external input at time t and Rext firing is the constant of external inhibition. The inhibition which arises from RCs at time t is given as follows: IRC inh ðtÞ ¼
N RC ðtÞ RRC firing
,
(7)
where N RC ðtÞ means the number of CA3 firing neurons at time t and RRC firing is the constant of RCs inhibition. Moreover, we introduced theta inhibition to the model. When sinð2pft pÞ becomes positive, Iinh ðtÞ is set to Ii . Therefore, a firable period and a nonfirable period interchange one after the other. 6.2. Synaptic plasticity Each synaptic weight is modified according to ASTDP or SSTDP. Spike interval Dt between the ith postsynaptic neuron and the jth presynaptic neuron is given by
Dtij ¼ ðT i T j Þ Z,
Fig. 4. Projection zones of CA3a and CA3b neurons in the model. In this figure, each cell corresponds to a neuron. Black cells project their RCs into gray areas surrounding them. Projection into dark gray cells is modified by SSTDP and projection into light gray cells is modified by ASTDP. H defines the area of projection.
(8)
where T i and T j denote the spike time of the ith postsynaptic neuron and that of the jth presynaptic one, respectively. Z ð40Þ is defined in consideration of the activity of receptor that underlies STDP. In this study, we employed the semi-nearest-neighbor manner for pairing spikes [12]. That is, for each presynaptic spike, only one preceding postsynaptic spike is considered and all earlier spikes are ignored. All postsynaptic spikes subsequent to the presynaptic spike are also considered. For each pre-/ postsynaptic spike pair, the synaptic weight is updated as follows:
Dwij ¼ 0:81ð1 C STDP ð0:12Dtij Þ2 Þeð0:12Dtij Þ wij ðt þ DtÞ ¼ mi Dwij wij ðtÞ, C , mi ¼ Pn j Dwij wij ðtÞ
2
=2
þ 1,
(9) (10) (11)
where C STDP shows the constant of STDP. When a synapse between the ith postsynaptic neuron and the jth presynaptic one is updated by ASTDP, the constant is defined by ( 0:01; DtX0; C STDP ¼ (12) 0:65; Dto0: When a synapse is updated by SSTDP, the constant is set to 0.65. C is a normalizing constant which depends on the locations of postsynaptic neurons in CA3. Owing to the coefficient mi , the sum of synaptic weights is conserved. 6.3. Structure of hippocampal CA3 model The proposed hippocampal CA3 model is composed of N spiking neuron models and each neuron has RCs. On the basis of the anatomical findings (Fig. 2(a)), we modeled the RCs between neurons as Fig. 4. CA3b neurons project RCs onto all subregions and CA3a neurons project RCs onto CA3a and CA3b. Then, in CA3a and CA3b, the RCs are modified by SSTDP, while the connections are modified by ASTDP in the CA3c. However, the CA3c-CA3c connections are so local that we omitted them.
Fig. 5. Input procedure following theta phase precession (letter: pattern, n: random pattern, !: transition).
6.4. Theta phase precession In this study, the events of episodes are represented by firing patterns. Thus, episodes are inputted to the hippocampus as sequences of firing patterns. In the hippocampus, sequences are transmitted to all hippocampal regions through EC, and then theta phase precession adjusts the sequences. We modeled the theta phase precession as shown in Fig. 5. First, we define one cycle of theta wave as 40 unit times. A new pattern is applied at the latest phase of each cycle and the pattern shows 10 unit time advance in each cycle. In this figure, we suppose that a sequence of patterns ð ! A ! B ! C ! D ! Þ is applied to the model. The sequence is adjusted as follows. The first pattern A of the sequence is applied to the model at the latest phase of the first cycle. Next, in the second cycle, pattern A is applied with 10 unit time advance and next pattern B is applied at the latest phase. Applying subsequent patterns in this manner, the phase advance of pattern A exceeds one cycle of the theta wave, and then pattern A disappears at the 6th cycle. The patterns (B–D) following pattern A are regulated similarly. Therefore, sequences are transformed into more than one cycle, in this case seven cycles, in this model. Moreover, 40 unit times are inserted as the nonfirable period between each cycle in order to divide them.
ARTICLE IN PRESS 3180
T. Samura et al. / Neurocomputing 71 (2008) 3176–3183
6.5. Communication within hippocampus Sequences adjusted by the theta phase precession are transmitted to the CA3 through two pathways (Fig. 2(b)). The switching of input pathways depends on the situations: memorization and retrieval [19]. According to the findings, sequences are inputted to all part of the CA3 from DG for the memorization and they are inputted only to CA3a and CA3b from EC for the retrieval. Since CA3c is the nearest to DG, it receives inputs from DG first. On the other hand, since CA3a is nearer to EC than CA3b, it receives inputs from EC earlier than CA3b. Here, we set input time lag between subregions at 1 unit time. Consequently, for the retrieval, sequences are processed first in CA3a and CA3b which work as autoassociative memory. After that the processed information is transmitted to CA3c.
7. Computer simulations 7.1. Conditions First of all, we set the parameters as shown in Table 1 and constructed the proposed model from 245 spiking neuron models. Next, we defined seven patterns (A–G). Each pattern is represented by the activation of 48 neurons and there is no overlap among them. As shown in Fig. 6, we defined two looped sequences on the basis of two conditions. The first condition is patternsensitive (PS) condition. Under the condition, subsequences I ðG ! A ! B ! C ! DÞ and II ðD ! E ! F ! C ! GÞ compose the looped sequence (Fig. 6(a)). The part of the subsequences just before overlapped pattern C consists of different patterns ðA ! B or E ! FÞ. The model should be sensitive to inputted
Table 1 Parameters for the simulation N ¼ 245; H ¼ 5, dCA3b!CA3c ¼ 8, ZCA3b!CA3c ¼ 15 dCA3b!CA3b ¼ 7, ZCA3b!CA3b ¼ 3, dCA3a!CA3b ¼ 8 ZCA3a!CA3b ¼ 2, dCA3a!CA3a ¼ 7, ZCA3a!CA3a ¼ 2
dCA3b!CA3a ¼ 13, ZCA3b!CA3a ¼ 4, tCA3b ¼ 20, tCA3a ¼ 20 m m tCA3c ¼ 5, y ¼ 0:047, V init ¼ 0:0, s ¼ 2, ts ¼ 2:5, tinh ¼ 2, RRC firing ¼ 0:5 m Rext firing ¼ 0:0001, f ¼ 12:5, C CA3b ; C CA3a ¼ 1:6, C CA3c ¼ 0:88, I i ¼ 30
patterns in order to solve this ambiguity. The other is ordersensitive (OS) condition. Under the condition, subsequences I ðE ! A ! B ! C ! DÞ and II ðD ! B ! A ! C ! EÞ compose the looped sequence (Fig. 6(b)). Both subsequences share the same patterns (A–C). Among them, the ambiguity appearing at pattern C may be the most difficult to solve, because the part of the sequences just before pattern C consists of the same pattern (A and B), and only the order of them is different from each other (A ! B or B ! A). Thus, the model should be sensitive to the order of inputs to solve the ambiguity. 7.2. Procedures We evaluated the proposed model in the two conditions as follows. Under each condition, first of all, the proposed model was initialized and the looped sequence of each condition was applied to the model five times (memorization period). After the memorization period, we evaluated the retrievability of a pattern in a subregion of CA3. This shows which pattern (retrieved pattern) is likely to be retrieved from patterns (source pattern) of CA3a or CA3b in a subregion. The retrievability of pattern z from pattern y is calculated by summing a direction cosine between a source pattern and the synaptic weights of each neuron that composes a retrieved pattern as follows: Rzy ¼ Sk
wk py , jwk jjpy j
(13)
where wk ¼ fwk1 ; wk2 ; ; wkN g denotes the synaptic weights from 3 all neurons of CA3a or CA3b to the kth neuron which composes pattern z in a subregion. wki shows the synaptic weight from the ith neuron of CA3a or CA3b to the kth neuron. py ¼ fpy1 ; py2 ; . . . ; pyN=3 g shows a source pattern in a subregion and then pyi means the activation (fire: 1, nonfire: 0) of the ith neuron of the subregion when pattern y is applied to the subregion. Following the memorization, the part of the subsequences until pattern C of each condition was applied to the model without the modification of the synaptic weights (retrieval period). During the retrieval period, the subsequences were applied only to CA3a and CA3b through the connections from EC. Then, we evaluated similarity between a certain pattern and the output of each subregion. The similarity shows what percentage of a pattern is included in the output of each subregion. The similarity between pattern z and the output of each subregion at time t is given as Sz ðtÞ ¼
xðtÞpz , jpz j2
(14)
where xðtÞ ¼ fx1 ðtÞ; x2 ðtÞ; . . . ; xN=3 ðtÞg denotes the output of all neurons in a subregion at time t. Then, we confirmed whether the model can distinguish pattern C of subsequences I and pattern C of subsequence II by using a difference between them. Under the OS condition, following the retrieval period, two subsequences were applied to the model again, but each pattern of subsequences lacks the activation of its seven neurons (completion period). Then, we evaluated the similarity and confirmed whether the model can recall complete pattern from incomplete inputs.
8. Computer simulation results 8.1. Memorization periods
Fig. 6. Two conditions emerging sequence ambiguity (letter: pattern, arrow: transition, dashed line: subsequence I, chain line: subsequence II). (a) Patternsensitive condition. (b) Order-sensitive condition.
Fig. 7 shows the retrievability of a pattern of a subregion from patterns of other subregions after the memorization period of each condition. As shown in Figs. 7(a)–(e), CA3a and CA3b work as autoassociative memory. On the other hand, CA3b–CA3c works as heteroassociative memory, because a pattern in CA3b leads the
ARTICLE IN PRESS T. Samura et al. / Neurocomputing 71 (2008) 3176–3183
3181
retrieval of the following patterns in each sequence (Figs. 7(c) and (f)). Consequently, we confirmed that the proposed model was functionally divided into autoassociative and heteroassociative memories.
8.2. Retrieval periods Figs. 8 and 9 show similarity during the retrieval period under the PS condition and the OS condition, respectively. At the beginning of the period, the first pattern of each subsequence was inputted to CA3a and CA3b. After that, next patterns were applied to them in the order of each subsequence at intervals of 10 unit time. Although each pattern was applied only once, as shown in these figures, they showed periodic activation in CA3a and CA3b. The periodic activation arose from the autoassociative connections between them. Then, we compared the similarity of CA3c output in two subsequences under the PS condition. As shown in Fig. 8(a), when pattern C of subsequence I was applied to the model, CA3c outputted pattern D (time: around 3562–3563). On the other hand, when pattern C of subsequence II was applied, CA3c outputted pattern G (Fig. 8(b), time: around 3722–3724). Next, we compared the similarity of CA3c outputs in two subsequences under the OS condition. As shown in Fig. 9(a), when pattern C of subsequence I was applied to the model, CA3c outputted pattern D (time: 3563). In contrast, when pattern C of sequence II was applied, CA3c outputted pattern E (Fig. 9(b), time: 3724). These results mean that the proposed model could generate different activities according to the differences between the subsequences in spite of the same pattern C under both conditions.
Fig. 7. Retrievability of patterns in each subregion under the PS condition ((a)–(c)) or the OS condition ((d)–(f)). Gray level of each cell means retrievability between a retrieved pattern and a source pattern.
Fig. 8. The similarity between the output of each subregion and original patterns during the retrieval period under the PS condition. Gray level of each cell means the similarity. (a) Subsequence I. (b) Subsequence II.
Fig. 9. The similarity between the output of each subregion and original patterns during the retrieval period under the OS condition. Gray level of each cell means the similarity. (a) Subsequence I. (b) Subsequence II.
ARTICLE IN PRESS 3182
T. Samura et al. / Neurocomputing 71 (2008) 3176–3183
8.3. Completion periods Fig. 10 shows the change of similarity when lacked sequence II was applied to the model under the OS condition. Therefore, the first patterns always lacked their activation. However, once patterns were inputted, the inputted patterns were periodically activated by autoassociative connections between CA3a and CA3b. As shown in these figures, the next patterns were complemented through the periodic activation in CA3a and CA3b. Thus, CA3a and CA3b realized pattern completion. Concurrently, with the pattern completion, we confirmed the generation of different activities according to the differences between the subsequences (Fig. 11). As shown in Fig. 11(a), when pattern C of lacked subsequence I was applied to the model, CA3c outputted pattern D (time: 3885). In contrast, when pattern C of lacked subsequence II was applied, CA3c outputted pattern E (Fig. 11(b), time: 4045).
9. Conclusions We have focused on the location dependency elucidated from anatomical findings of CA3 and the physiological findings of STDP, and then we have suggested that CA3 is functionally divided into two functions: autoassociative memory and heteroassociative memory. Moreover, we have shown that the functionally divided CA3 can generate a code for disambiguating sequences under A
B
C
D
1.0 0.9 0.8
Similarity
0.7 0.6 0.5 0.4 0.3 0.2 0.1 0.0 4000
4010
4020
4030
4040
4050
Time A
B
C
D
1.0 0.9 0.8
Similarity
0.7 0.6
Fig. 11. The similarity between the output of CA3c and original patterns when each lacked subsequence is applied during the completion period. (a) Subsequence I. (b) Subsequence II.
pattern-sensitive and order-sensitive conditions. In the proposed model, CA3a and CA3b periodically retrieved previously inputted patterns in an autoassociative fashion. This helps the model to buffer the difference between sequences, even if the difference comes from their components or the order of inputs. Then the information in the buffer was transmitted to CA3c through heteroassociative connections. As shown in the computer simulation results, the differences in the buffer caused the difference of pattern retrieved in CA3c. This corresponds to transforming temporal information to spatial information for creating the context-like information [8]. However, its mechanism differs from the conventional one in that a code, which dissociates the same pattern in sequences, was generated by the cooperation between autoassociative and heteroassociative memories. The code leads to sequence disambiguation in CA1. This cooperation between CA3 and CA1 to achieve the sequence disambiguation corresponds to Kesner’s review [13] as well. Furthermore, we have shown that incomplete inputs were complemented by autoassociative memory and a code for sequence disambiguation was created by heteroassociative memory at the same time. Namely, pattern completion [16] and the creation of a code were achieved simultaneously in CA3. Consequently, it demonstrates that the cooperation mechanism between autoassociative and heteroassociative memories of the functionally divided CA3 brings out the abilities of sequence disambiguation and pattern completion in the hippocampus.
0.5
References
0.4 0.3 0.2 0.1 0.0 4000
4010
4020
4030
4040
4050
Time Fig. 10. The similarity between the output of each subregion and patterns during the completion period. (a) CA3a. (b) CA3b.
[1] K.L. Agster, N.J. Fortin, H. Eichenbaum, The hippocampus and disambiguation of overlapping sequences, J. Neurosci. 22 (2002) 5760–5768. [2] D.A. August, W.B. Levy, Temporal sequence compression by an integrate-andfire model of hippocampal area CA3, J. Comput. Neurosci. 6 (1999) 71–90. [3] D. Debanne, B.H. Ga¨hwiler, S.M. Thompson, Long-term synaptic plasticity between pairs of individual CA3 pyramidal cells in rat hippocampal slice cultures, J. Physiol. 507 (1998) 237–247. [4] D. Debanne, N.C. Gue´rineau, B.H. Ga¨hwiler, S.M. Thompson, Physiology and pharmacology of unitary synaptic connections between pairs of cells in areas CA3 and CA1 of rat hippocampal slice cultures, J. Neurophysiol. 73 (1995) 1282–1294.
ARTICLE IN PRESS T. Samura et al. / Neurocomputing 71 (2008) 3176–3183
[5] H. Eichenbaum, Hippocampus: cognitive processes and neural representations that underlie declarative memory, Neuron 44 (2004) 109–120. [6] H. Eichenbaum, P. Dudchenko, E. Wood, M. Shapiro, H. Tanila, The hippocampus, memory, and place cells: is it spatial memory or a memory space?, Neuron 23 (1999) 209–226. [7] A.I. Gulya´s, R. Miles, N. Ha´jos, T.F. Freund, Precision and variability in postsynaptic target selection of inhibitory cells in the hippocampal CA3 region, Eur. J. Neurosci. 5 (1993) 1729–1751. [8] H. Hayashi, M. Yoshida, A memory model of the hippocampus for the foundations of brain-inspired information technology, Int. Congr. Ser. 1269 (2004) 109–112. [9] S. Heckers, M. Zalesak, A.P. Weiss, T. Ditman, D. Titone, Hippocampal activation during transitive inference in humans, Hippocampus 14 (2004) 153–162. [10] N. Ishizuka, W.M. Cowan, D.G. Amaral, A quantitative analysis of the dendritic organization of pyramidal cells in the rat hippocampus, J. Comp. Neurol. 362 (1995) 17–45. [11] N. Ishizuka, J. Weber, D.G. Amaral, Organization of intra-hippocampal projections originating from CA3 pyramidal cells in the rat, J. Comp. Neurol. 295 (1990) 580–623. [12] E.M. Izhikevich, N.S. Desai, Relating STDP to BCM, Neural Comput. 15 (2003) 1511–1523. [13] R.P. Kesner, I. Lee, P. Gilbert, A behavioral assessment of hippocampal function based on a subregional analysis, Rev. Neurosci. 15 (2004) 333–351. [14] W.B. Levy, A sequence predicting CA3 is a flexible associator that learns and uses context to solve hippocampal-like tasks, Hippocampus 6 (1996) 579–590. [15] W.B. Levy, X. Wu, R.A. Baxter, Unification of hippocampal function via computational/encoding considerations, Int. J. Neural Syst. 6 (1995) 71–80. [16] K. Nakazawa, M.C. Quirk, R.A. Chitwood, M. Watanabe, M.F. Yeckel, L.D. Sun, A. Kato, C.A. Carr, D. Johnston, M.A. Wilson, S. Tonegawa, Requirement for hippocampal CA3 NMDA receptors in associative memory recall, Science 297 (2002) 211–218. [17] A.R. Preston, Y. Shrager, N.M. Dudukovic, J.D.E. Gabrieli, Hippocampal contribution to the novel use of relational information in declarative memory, Hippocampus 14 (2004) 148–152. [18] T. Samura, M. Hattori, S. Ishizaki, Autoassociative and heteroassociative hippocampal CA3 model based on location dependencies derived from anatomical and physiological findings, Int. Congr. Ser. 1301 (2007) 140–143. [19] A. Treves, E.T. Rolls, Computational constraints suggest the need for two distinct input systems to the hippocampal CA3 network, Hippocampus 2 (1992) 189–199. [20] M. Tsukada, T. Aihara, Y. Kobayashi, H. Shimazaki, Spatial analysis of spiketiming-dependent LTP and LTD in the CA1 area of hippocampal slices using optical imaging, Hippocampus 15 (2005) 104–109.
3183
[21] Y. Yamaguchi, Theta phase coding and memory in the hippocampus, Seitai No Kagaku 55 (2004) 33–42. Toshikazu Samura received the bachelor degree in Engineering in 2003, and the master degree in Engineering in 2005, both from University of Yamanashi. He is now a Ph.D. student in Graduate School of Media and Governance at Keio University. He is engaged in research on neural networks, especially the information processing of the hippocampus.
Motonobu Hattori was born in Tokyo, Japan, on February 11, 1970. He received B.E., M.E. and Ph.D degrees in Electrical Engineering from Keio University in 1992, 1994 and 1997, respectively. In 1997, he became a research associate of University of Yamanashi, Kofu, Japan. Since 2000, he has been an Associate Professor. From 2003 to 2004, he was a visiting researcher at Carnegie Mellon University. In 1996, he received Niwa Memorial Award. His current research interest is in neural networks, reinforcement learning and evolutionary computation. He is a member of the IEEE, IEICE and IEE. Shun Ishizaki was born in 1947 in Tokyo, Japan. He graduated from the University of Tokyo in 1970. He received Ph.D. degree from the university. He joined Electro-technical Laboratory of MITI after he worked as a research associate at the university. He was a director of Machine Inference Section and then Natural Language Section of the laboratory. He joined Keio University in 1992 as a professor at Environmental Information Faculty. He was the president of Japanese Cognitive Science Society and also Japanese Natural Language Processing Association. He is now interested in neuroinformatics, natural language processing, machine learning and cognitive science researches.
Contents lists available at ScienceDirect
Neurocomputing journal homepage: www.elsevier.com/locate/neucom
Sequence disambiguation and pattern completion by cooperation between autoassociative and heteroassociative memories of functionally divided hippocampal CA3 Toshikazu Samura a,, Motonobu Hattori b, Shun Ishizaki a a b
Graduate School of Media and Governance, Keio University, 5322 Endo, Fujisawa-shi, Kanagawa 252-8520, Japan Interdisciplinary Graduate School of Medicine and Engineering, University of Yamanashi, 4-3-11 Takeda, Kofu-shi, Yamanashi 400-8511, Japan
a r t i c l e in f o
a b s t r a c t
Available online 22 June 2008
The hippocampus memorizes event sequences and some events are shared by a few sequences. When a sequence is retrieved from the linked sequences, ambiguity of sequences becomes a serious problem because the shared events have some possible ways to retrieve other events. We have focused on location dependency elucidated from the hippocampus and suggested that CA3 is functionally divided into autoassociative and heteroassociative memories. Computer simulation results show that the functionally divided CA3 concurrently enables both sequence disambiguation and pattern completion. Consequently, it demonstrates that cooperation between both memories of CA3 brings out the abilities of sequence disambiguation and pattern completion in the hippocampus. & 2008 Elsevier B.V. All rights reserved.
Keywords: Functionally divided hippocampal CA3 Sequence disambiguation Pattern completion Autoassociative memory Heteroassociative memory
1. Introduction A promising hypothesis supporting many hippocampal processes has been suggested by Eichenbaum [6,5]. On the hypothesis, daily episodes are memorized as a relational network (previously called memory space) in the hippocampus. In the relational network, an episode is expressed by a sequence of events that are primitive elements of memory. Episodes are associated with each other by events shared among the episodes. For example, let us consider that the hippocampus memorizes two episodes, one is composed of the following events: A, B and C ðA ! B ! CÞ. The other is composed of the events: X, B and Y ðX ! B ! YÞ, where event B associates the two episodes. As a result, we can find out a new way from event X to event C by associating these episodes. Therefore, the relational network, that can flexibly associate episodes acquired, supports inference that derives new ways to deal with memory. Recent studies have suggested that the hippocampus relates to a flexible representation [17,9] used for inference. It follows that these suggestions support Eichenbaum’s hypothesis that leads to the inference. In the relational network, it is difficult to decide which pattern (C or Y) should be retrieved from event B. The ambiguity of sequences becomes a problem for retrieval. Thus, the sequence disambiguation is an essential function for retrieving the original episode from the relational network. Agster et al. have suggested that the hippocampus is required to accomplish sequence Corresponding author. Tel./fax: +81 466 48 6101.
E-mail address: [email protected] (T. Samura). 0925-2312/$ - see front matter & 2008 Elsevier B.V. All rights reserved. doi:10.1016/j.neucom.2008.04.026
disambiguation [1]. In the hippocampus, the CA3 region has unique recursive axons which are called recurrent collaterals (RCs). Because of its uniqueness, many researchers focused on it and proposed many computational models of CA3. Several of the models contribute to solving the sequence disambiguation problem by creating a code [14,8]. In those models, however, CA3 has been homogeneously modeled in spite of its heterogeneous anatomical properties [11]. Thus, we have suggested the importance of the model based on the heterogeneous properties that are elucidated from the location dependency of RCs and spike-timing dependent plasticity (STDP) [18]. On the basis of the location dependency, we have suggested that CA3 is functionally divided into autoassociative and heteroassociative memories [18]. In this paper, using computer simulations, we show that the functionally divided CA3 can disambiguate the overlapped sequences and enables pattern completion that is the already perceived ability of CA3 [16] to recall complete memory from incomplete set of cues. Consequently, it demonstrates that a cooperation mechanism between autoassociative and heteroassociative memories of the functionally divided CA3 brings out the abilities of the sequence disambiguation and pattern completion in the hippocampus.
2. Related works In this section, we review conventional CA3 models solving the sequence disambiguation problem. Levy et al. have focused on CA3 and proposed a simplified CA3 model composed of
ARTICLE IN PRESS T. Samura et al. / Neurocomputing 71 (2008) 3176–3183
McCulloch–Pitts neurons [15]. The model has the ability to create a local code for temporal context used for recognizing a subsequence of a large sequence. Since the model solves sequential problems including the sequence disambiguation with the context code, they have suggested the importance of the context for hippocampal functions. Moreover, they have proposed that fundamental properties to create the codes agree to the properties of CA3 [14]. In their study, McCulloch–Pitts neurons model spike rate, but not spike timing. However, STDP that is a rule of changing synaptic weights depending on spike timing exists in the hippocampus [3]. Hayashi et al. have proposed the hippocampal model [8] including CA3 and CA1 receiving from CA3. The synaptic weights of the model are updated by STDP. This model can also create context-like information by transforming temporal information to spatial information in CA3 and this context of CA3 leads to the sequence disambiguation in CA1. Hayashi’s model corresponds to the review that cooperation between CA3 and CA1 achieves the sequence disambiguation [13]. However, the spatial pattern completion function of CA3 is undescribed in this model. Furthermore, although heterogeneous anatomical properties that are the subregional local dependency of RCs have already been found in CA3, it was modeled homogeneously in their models. On the other hand, Samura et al. [18] have focused on the anatomical findings. Moreover, they have focused on the location dependency of STDP [20] and suggested that CA3 is functionally divided into autoassociative and heteroassociative memories. So, it seems plausible that the autoassociative memory leads to pattern completion and that the heteroassociative one which can reflect temporal information in synaptic weights leads to the transformation for creating a context. That is, the functionally divided CA3 may exist for the abilities of pattern completion and the creation of a context for the sequence disambiguation.
3177
3.2. Connections within CA3 CA3 neurons are connected recursively to other neurons by RCs. Fig. 2(a) shows the relationship between the location of a neuron and the projection zone of its RCs [11]. First, the RCs of CA3c neurons are limited to the area surrounding them. Second, the RCs of CA3b neurons are widely spread. Projection onto CA3c becomes a more temporal location than the position of its source and that onto CA3a becomes a more septal location. Finally, the RCs of CA3a neurons are limited to CA3a and CA3b. Projection onto CA3b becomes a more temporal location. In addition to the location dependency of the projection zones, the dendritic locations of RCs depend on where neurons receive RCs in the CA3 [11] (Fig. 2(b)). CA3c neurons receive RCs at a distance from a soma, while CA3a and CA3b receive RCs near a soma. Additionally, we review projection toward CA3 from other hippocampal regions [10]. Fig. 2(b) shows connections to CA3 from two regions: EC and DG. As shown in this figure, DG connects to all CA3 subregions and EC connects only to CA3a and CA3b. It was suggested that the connections from DG to CA3 contribute to memorization, while those from EC to CA3 are required for retrieval [19].
4. Physiological backgrounds of hippocampus 4.1. Spike-timing dependent plasticity A neural network memorizes information from the outside world by changing synaptic weights between neurons. In the hippocampus, STDP is observed as a rule of changing synaptic weights [3]. STDP determines the magnitude of a synaptic change and its polarity (potentiation or depression) according to the interval between pre- and postsynaptic spikes. Furthermore,
3. Anatomical backgrounds of hippocampus 3.1. Structure of hippocampus Fig. 1 shows the structure of the hippocampus. The hippocampus is divided into three regions: Dentate gyrus (DG), CA3 and CA1. Information is transmitted from DG to CA1 via CA3. Entorhinal cortex (EC) works as an interface between the cortex and the hippocampus and transmits information to all hippocampal regions. CA3 has RCs which recursively connect CA3 neurons to other CA3 neurons. Furthermore, CA3 is segmented into three subregions: CA3a, CA3b and CA3c (Fig. 1).
Fig. 1. Connections between the hippocampal regions.
Fig. 2. Connections within CA3. (a) Projection zone of RCs (circle: source neuron, ellipse: projection zone of the circled neuron in it). (b) Dendritic locations of RCs (inverse triangle: soma, forked line: dendrite, dashed line: RCs, chain line: external input).
ARTICLE IN PRESS 3178
T. Samura et al. / Neurocomputing 71 (2008) 3176–3183
recent studies suggested that STDP shows asymmetric profile or symmetric one depending on the density of inhibitory interneurons [20]. Symmetric profile STDP (SSTDP) is observed from a highdensity area near a soma (Fig. 3(a)), while asymmetric profile STDP (ASTDP) is observed from a low-density area distant from a soma (Fig. 3(b)). Therefore, the change of STDP profile correlates with a synaptic location on a dendrite. In other words, STDP shows the dendritic location dependency. Many experimental results of STDP are obtained from CA3–CA1 synapses, which are called Schaeffer collaterals (SCs). However, Debanne et al. observed STDP from RCs and suggested that RCs appear to be identical to SCs in both their basic properties and mechanisms of synaptic plasticity [3,4]. In addition, several inhibitory interneurons exist in the CA3 region [7], and they seem to be similar to those in the CA1 region. Thus, we apply the observations of SCs to RCs.
4.2. Theta phase precession The hippocampus receives inputs from the cortex via EC (Fig. 1). Then, theta phase precession, which emerges from EC, regulates the inputs to the hippocampus and contributes to memorization of sequences [21]. Theta phase precession is the phenomenon that the phase of neuronal firing gradually advances in each cycle of theta wave. In particular, a neuron always begins to fire at a certain phase and its firing phase advances to earlier phase in the next cycle. The firing of the neuron ceases, when the
phase advance of a neuron exceeds about one cycle of the theta wave.
5. Functional divisions of hippocampal CA3 From the integration of the location dependency, CA3c neurons receive RCs at distal dendrites and their plasticity becomes ASTDP, while CA3a and CA3b neurons receive RCs at proximal dendrites and show SSTDP. Here, we discuss a functional difference between the profiles. Under SSTDP, simultaneous firing leads to potentiation and time lag leads to depression (Fig. 3(a)). Thus, an instantaneous firing pattern of a network is mapped onto synaptic weights. This means that SSTDP suits autoassociative memory. On the other hand, ASTDP potentiates synapses when postsynaptic neurons fire after presynaptic firings. Conversely, if the firing order is reversed, synapses between them are depressed (Fig. 3(b)). Therefore, synaptic weights reflect the order of firing and ASTDP suits heteroassociative memory. Consequently, the CA3 region is functionally divided into two functions: CA3a and CA3b regions work as autoassociative memory, while CA3c region works as heteroassociative memory.
6. Hippocampal CA3 model 6.1. Neuron model A proposed hippocampal CA3 model consists of spiking neuron models modified from an integrate-and-fire neuron models which were used by August et al. [2]. The following equation shows the membrane potential of the jth neuron at time t.
tm
dV j ðtÞ ¼ Ij ðtÞ V j ðtÞ, dt
(1)
where Ij ðtÞ denotes synaptic current. tm denotes the membrane time constant which depends on the location of a neuron in CA3. The following equation shows the output of the jth neuron at time t, ( 1; V j ðtÞ4y; xj ðtÞ ¼ (2) 0; V j ðtÞpy: When the membrane potential exceeds threshold y, the cell fires and the membrane potential is reset to V init . The refractoriness of the jth neuron is realized by suspension of calculating Eq. (1) for time s after the firing. The input of the jth neuron through RCs at time t is defined as follows: X IRC wij ðtÞxi ðt dÞ, (3) j ðtÞ ¼ i
where wij ðtÞ denotes the synaptic weight between the ith neuron and the jth one at time t and d denotes the synaptic delay. The synaptic current of the jth neuron at time t is given by 2 ðIRC dIj ðtÞ Ij ðtÞ j ðtÞÞ ¼ RC , þ zj ðtÞ ts dt Ij ðtÞ þ Iinh ðtÞ
(4)
where Iinh ðtÞ means inhibitory synaptic current. zj ðtÞ ¼ f0; 1g denotes external input from EC and DG at time t and ts shows current time constant. The following equation means the inhibitory synaptic current at time t, Fig. 3. Profiles of STDP. A relative timing between pre- and postsynaptic neurons is given by subtracting presynaptic spike time and postsynaptic one. According to the timing and the profiles, the rate of synaptic change is determined. (a) Symmetric profile. (b) Asymmetric profile.
dIinh ðtÞ Iinh ðtÞ RC ¼ Iext , inh ðtÞ þ Iinh ðtÞ dt tinh
(5)
where Iext inh ðtÞ shows inhibition which arises from external inputs and IRC inh ðtÞ shows inhibition which arises from RCs. tinh denotes the
ARTICLE IN PRESS T. Samura et al. / Neurocomputing 71 (2008) 3176–3183
3179
time constant of inhibitory synaptic current. The external inhibition at time t is given as follows: Iext inh ðtÞ ¼
N ext ðtÞ , Rext firing
(6)
where Next ðtÞ means the number of neurons which receive external input at time t and Rext firing is the constant of external inhibition. The inhibition which arises from RCs at time t is given as follows: IRC inh ðtÞ ¼
N RC ðtÞ RRC firing
,
(7)
where N RC ðtÞ means the number of CA3 firing neurons at time t and RRC firing is the constant of RCs inhibition. Moreover, we introduced theta inhibition to the model. When sinð2pft pÞ becomes positive, Iinh ðtÞ is set to Ii . Therefore, a firable period and a nonfirable period interchange one after the other. 6.2. Synaptic plasticity Each synaptic weight is modified according to ASTDP or SSTDP. Spike interval Dt between the ith postsynaptic neuron and the jth presynaptic neuron is given by
Dtij ¼ ðT i T j Þ Z,
Fig. 4. Projection zones of CA3a and CA3b neurons in the model. In this figure, each cell corresponds to a neuron. Black cells project their RCs into gray areas surrounding them. Projection into dark gray cells is modified by SSTDP and projection into light gray cells is modified by ASTDP. H defines the area of projection.
(8)
where T i and T j denote the spike time of the ith postsynaptic neuron and that of the jth presynaptic one, respectively. Z ð40Þ is defined in consideration of the activity of receptor that underlies STDP. In this study, we employed the semi-nearest-neighbor manner for pairing spikes [12]. That is, for each presynaptic spike, only one preceding postsynaptic spike is considered and all earlier spikes are ignored. All postsynaptic spikes subsequent to the presynaptic spike are also considered. For each pre-/ postsynaptic spike pair, the synaptic weight is updated as follows:
Dwij ¼ 0:81ð1 C STDP ð0:12Dtij Þ2 Þeð0:12Dtij Þ wij ðt þ DtÞ ¼ mi Dwij wij ðtÞ, C , mi ¼ Pn j Dwij wij ðtÞ
2
=2
þ 1,
(9) (10) (11)
where C STDP shows the constant of STDP. When a synapse between the ith postsynaptic neuron and the jth presynaptic one is updated by ASTDP, the constant is defined by ( 0:01; DtX0; C STDP ¼ (12) 0:65; Dto0: When a synapse is updated by SSTDP, the constant is set to 0.65. C is a normalizing constant which depends on the locations of postsynaptic neurons in CA3. Owing to the coefficient mi , the sum of synaptic weights is conserved. 6.3. Structure of hippocampal CA3 model The proposed hippocampal CA3 model is composed of N spiking neuron models and each neuron has RCs. On the basis of the anatomical findings (Fig. 2(a)), we modeled the RCs between neurons as Fig. 4. CA3b neurons project RCs onto all subregions and CA3a neurons project RCs onto CA3a and CA3b. Then, in CA3a and CA3b, the RCs are modified by SSTDP, while the connections are modified by ASTDP in the CA3c. However, the CA3c-CA3c connections are so local that we omitted them.
Fig. 5. Input procedure following theta phase precession (letter: pattern, n: random pattern, !: transition).
6.4. Theta phase precession In this study, the events of episodes are represented by firing patterns. Thus, episodes are inputted to the hippocampus as sequences of firing patterns. In the hippocampus, sequences are transmitted to all hippocampal regions through EC, and then theta phase precession adjusts the sequences. We modeled the theta phase precession as shown in Fig. 5. First, we define one cycle of theta wave as 40 unit times. A new pattern is applied at the latest phase of each cycle and the pattern shows 10 unit time advance in each cycle. In this figure, we suppose that a sequence of patterns ð ! A ! B ! C ! D ! Þ is applied to the model. The sequence is adjusted as follows. The first pattern A of the sequence is applied to the model at the latest phase of the first cycle. Next, in the second cycle, pattern A is applied with 10 unit time advance and next pattern B is applied at the latest phase. Applying subsequent patterns in this manner, the phase advance of pattern A exceeds one cycle of the theta wave, and then pattern A disappears at the 6th cycle. The patterns (B–D) following pattern A are regulated similarly. Therefore, sequences are transformed into more than one cycle, in this case seven cycles, in this model. Moreover, 40 unit times are inserted as the nonfirable period between each cycle in order to divide them.
ARTICLE IN PRESS 3180
T. Samura et al. / Neurocomputing 71 (2008) 3176–3183
6.5. Communication within hippocampus Sequences adjusted by the theta phase precession are transmitted to the CA3 through two pathways (Fig. 2(b)). The switching of input pathways depends on the situations: memorization and retrieval [19]. According to the findings, sequences are inputted to all part of the CA3 from DG for the memorization and they are inputted only to CA3a and CA3b from EC for the retrieval. Since CA3c is the nearest to DG, it receives inputs from DG first. On the other hand, since CA3a is nearer to EC than CA3b, it receives inputs from EC earlier than CA3b. Here, we set input time lag between subregions at 1 unit time. Consequently, for the retrieval, sequences are processed first in CA3a and CA3b which work as autoassociative memory. After that the processed information is transmitted to CA3c.
7. Computer simulations 7.1. Conditions First of all, we set the parameters as shown in Table 1 and constructed the proposed model from 245 spiking neuron models. Next, we defined seven patterns (A–G). Each pattern is represented by the activation of 48 neurons and there is no overlap among them. As shown in Fig. 6, we defined two looped sequences on the basis of two conditions. The first condition is patternsensitive (PS) condition. Under the condition, subsequences I ðG ! A ! B ! C ! DÞ and II ðD ! E ! F ! C ! GÞ compose the looped sequence (Fig. 6(a)). The part of the subsequences just before overlapped pattern C consists of different patterns ðA ! B or E ! FÞ. The model should be sensitive to inputted
Table 1 Parameters for the simulation N ¼ 245; H ¼ 5, dCA3b!CA3c ¼ 8, ZCA3b!CA3c ¼ 15 dCA3b!CA3b ¼ 7, ZCA3b!CA3b ¼ 3, dCA3a!CA3b ¼ 8 ZCA3a!CA3b ¼ 2, dCA3a!CA3a ¼ 7, ZCA3a!CA3a ¼ 2
dCA3b!CA3a ¼ 13, ZCA3b!CA3a ¼ 4, tCA3b ¼ 20, tCA3a ¼ 20 m m tCA3c ¼ 5, y ¼ 0:047, V init ¼ 0:0, s ¼ 2, ts ¼ 2:5, tinh ¼ 2, RRC firing ¼ 0:5 m Rext firing ¼ 0:0001, f ¼ 12:5, C CA3b ; C CA3a ¼ 1:6, C CA3c ¼ 0:88, I i ¼ 30
patterns in order to solve this ambiguity. The other is ordersensitive (OS) condition. Under the condition, subsequences I ðE ! A ! B ! C ! DÞ and II ðD ! B ! A ! C ! EÞ compose the looped sequence (Fig. 6(b)). Both subsequences share the same patterns (A–C). Among them, the ambiguity appearing at pattern C may be the most difficult to solve, because the part of the sequences just before pattern C consists of the same pattern (A and B), and only the order of them is different from each other (A ! B or B ! A). Thus, the model should be sensitive to the order of inputs to solve the ambiguity. 7.2. Procedures We evaluated the proposed model in the two conditions as follows. Under each condition, first of all, the proposed model was initialized and the looped sequence of each condition was applied to the model five times (memorization period). After the memorization period, we evaluated the retrievability of a pattern in a subregion of CA3. This shows which pattern (retrieved pattern) is likely to be retrieved from patterns (source pattern) of CA3a or CA3b in a subregion. The retrievability of pattern z from pattern y is calculated by summing a direction cosine between a source pattern and the synaptic weights of each neuron that composes a retrieved pattern as follows: Rzy ¼ Sk
wk py , jwk jjpy j
(13)
where wk ¼ fwk1 ; wk2 ; ; wkN g denotes the synaptic weights from 3 all neurons of CA3a or CA3b to the kth neuron which composes pattern z in a subregion. wki shows the synaptic weight from the ith neuron of CA3a or CA3b to the kth neuron. py ¼ fpy1 ; py2 ; . . . ; pyN=3 g shows a source pattern in a subregion and then pyi means the activation (fire: 1, nonfire: 0) of the ith neuron of the subregion when pattern y is applied to the subregion. Following the memorization, the part of the subsequences until pattern C of each condition was applied to the model without the modification of the synaptic weights (retrieval period). During the retrieval period, the subsequences were applied only to CA3a and CA3b through the connections from EC. Then, we evaluated similarity between a certain pattern and the output of each subregion. The similarity shows what percentage of a pattern is included in the output of each subregion. The similarity between pattern z and the output of each subregion at time t is given as Sz ðtÞ ¼
xðtÞpz , jpz j2
(14)
where xðtÞ ¼ fx1 ðtÞ; x2 ðtÞ; . . . ; xN=3 ðtÞg denotes the output of all neurons in a subregion at time t. Then, we confirmed whether the model can distinguish pattern C of subsequences I and pattern C of subsequence II by using a difference between them. Under the OS condition, following the retrieval period, two subsequences were applied to the model again, but each pattern of subsequences lacks the activation of its seven neurons (completion period). Then, we evaluated the similarity and confirmed whether the model can recall complete pattern from incomplete inputs.
8. Computer simulation results 8.1. Memorization periods
Fig. 6. Two conditions emerging sequence ambiguity (letter: pattern, arrow: transition, dashed line: subsequence I, chain line: subsequence II). (a) Patternsensitive condition. (b) Order-sensitive condition.
Fig. 7 shows the retrievability of a pattern of a subregion from patterns of other subregions after the memorization period of each condition. As shown in Figs. 7(a)–(e), CA3a and CA3b work as autoassociative memory. On the other hand, CA3b–CA3c works as heteroassociative memory, because a pattern in CA3b leads the
ARTICLE IN PRESS T. Samura et al. / Neurocomputing 71 (2008) 3176–3183
3181
retrieval of the following patterns in each sequence (Figs. 7(c) and (f)). Consequently, we confirmed that the proposed model was functionally divided into autoassociative and heteroassociative memories.
8.2. Retrieval periods Figs. 8 and 9 show similarity during the retrieval period under the PS condition and the OS condition, respectively. At the beginning of the period, the first pattern of each subsequence was inputted to CA3a and CA3b. After that, next patterns were applied to them in the order of each subsequence at intervals of 10 unit time. Although each pattern was applied only once, as shown in these figures, they showed periodic activation in CA3a and CA3b. The periodic activation arose from the autoassociative connections between them. Then, we compared the similarity of CA3c output in two subsequences under the PS condition. As shown in Fig. 8(a), when pattern C of subsequence I was applied to the model, CA3c outputted pattern D (time: around 3562–3563). On the other hand, when pattern C of subsequence II was applied, CA3c outputted pattern G (Fig. 8(b), time: around 3722–3724). Next, we compared the similarity of CA3c outputs in two subsequences under the OS condition. As shown in Fig. 9(a), when pattern C of subsequence I was applied to the model, CA3c outputted pattern D (time: 3563). In contrast, when pattern C of sequence II was applied, CA3c outputted pattern E (Fig. 9(b), time: 3724). These results mean that the proposed model could generate different activities according to the differences between the subsequences in spite of the same pattern C under both conditions.
Fig. 7. Retrievability of patterns in each subregion under the PS condition ((a)–(c)) or the OS condition ((d)–(f)). Gray level of each cell means retrievability between a retrieved pattern and a source pattern.
Fig. 8. The similarity between the output of each subregion and original patterns during the retrieval period under the PS condition. Gray level of each cell means the similarity. (a) Subsequence I. (b) Subsequence II.
Fig. 9. The similarity between the output of each subregion and original patterns during the retrieval period under the OS condition. Gray level of each cell means the similarity. (a) Subsequence I. (b) Subsequence II.
ARTICLE IN PRESS 3182
T. Samura et al. / Neurocomputing 71 (2008) 3176–3183
8.3. Completion periods Fig. 10 shows the change of similarity when lacked sequence II was applied to the model under the OS condition. Therefore, the first patterns always lacked their activation. However, once patterns were inputted, the inputted patterns were periodically activated by autoassociative connections between CA3a and CA3b. As shown in these figures, the next patterns were complemented through the periodic activation in CA3a and CA3b. Thus, CA3a and CA3b realized pattern completion. Concurrently, with the pattern completion, we confirmed the generation of different activities according to the differences between the subsequences (Fig. 11). As shown in Fig. 11(a), when pattern C of lacked subsequence I was applied to the model, CA3c outputted pattern D (time: 3885). In contrast, when pattern C of lacked subsequence II was applied, CA3c outputted pattern E (Fig. 11(b), time: 4045).
9. Conclusions We have focused on the location dependency elucidated from anatomical findings of CA3 and the physiological findings of STDP, and then we have suggested that CA3 is functionally divided into two functions: autoassociative memory and heteroassociative memory. Moreover, we have shown that the functionally divided CA3 can generate a code for disambiguating sequences under A
B
C
D
1.0 0.9 0.8
Similarity
0.7 0.6 0.5 0.4 0.3 0.2 0.1 0.0 4000
4010
4020
4030
4040
4050
Time A
B
C
D
1.0 0.9 0.8
Similarity
0.7 0.6
Fig. 11. The similarity between the output of CA3c and original patterns when each lacked subsequence is applied during the completion period. (a) Subsequence I. (b) Subsequence II.
pattern-sensitive and order-sensitive conditions. In the proposed model, CA3a and CA3b periodically retrieved previously inputted patterns in an autoassociative fashion. This helps the model to buffer the difference between sequences, even if the difference comes from their components or the order of inputs. Then the information in the buffer was transmitted to CA3c through heteroassociative connections. As shown in the computer simulation results, the differences in the buffer caused the difference of pattern retrieved in CA3c. This corresponds to transforming temporal information to spatial information for creating the context-like information [8]. However, its mechanism differs from the conventional one in that a code, which dissociates the same pattern in sequences, was generated by the cooperation between autoassociative and heteroassociative memories. The code leads to sequence disambiguation in CA1. This cooperation between CA3 and CA1 to achieve the sequence disambiguation corresponds to Kesner’s review [13] as well. Furthermore, we have shown that incomplete inputs were complemented by autoassociative memory and a code for sequence disambiguation was created by heteroassociative memory at the same time. Namely, pattern completion [16] and the creation of a code were achieved simultaneously in CA3. Consequently, it demonstrates that the cooperation mechanism between autoassociative and heteroassociative memories of the functionally divided CA3 brings out the abilities of sequence disambiguation and pattern completion in the hippocampus.
0.5
References
0.4 0.3 0.2 0.1 0.0 4000
4010
4020
4030
4040
4050
Time Fig. 10. The similarity between the output of each subregion and patterns during the completion period. (a) CA3a. (b) CA3b.
[1] K.L. Agster, N.J. Fortin, H. Eichenbaum, The hippocampus and disambiguation of overlapping sequences, J. Neurosci. 22 (2002) 5760–5768. [2] D.A. August, W.B. Levy, Temporal sequence compression by an integrate-andfire model of hippocampal area CA3, J. Comput. Neurosci. 6 (1999) 71–90. [3] D. Debanne, B.H. Ga¨hwiler, S.M. Thompson, Long-term synaptic plasticity between pairs of individual CA3 pyramidal cells in rat hippocampal slice cultures, J. Physiol. 507 (1998) 237–247. [4] D. Debanne, N.C. Gue´rineau, B.H. Ga¨hwiler, S.M. Thompson, Physiology and pharmacology of unitary synaptic connections between pairs of cells in areas CA3 and CA1 of rat hippocampal slice cultures, J. Neurophysiol. 73 (1995) 1282–1294.
ARTICLE IN PRESS T. Samura et al. / Neurocomputing 71 (2008) 3176–3183
[5] H. Eichenbaum, Hippocampus: cognitive processes and neural representations that underlie declarative memory, Neuron 44 (2004) 109–120. [6] H. Eichenbaum, P. Dudchenko, E. Wood, M. Shapiro, H. Tanila, The hippocampus, memory, and place cells: is it spatial memory or a memory space?, Neuron 23 (1999) 209–226. [7] A.I. Gulya´s, R. Miles, N. Ha´jos, T.F. Freund, Precision and variability in postsynaptic target selection of inhibitory cells in the hippocampal CA3 region, Eur. J. Neurosci. 5 (1993) 1729–1751. [8] H. Hayashi, M. Yoshida, A memory model of the hippocampus for the foundations of brain-inspired information technology, Int. Congr. Ser. 1269 (2004) 109–112. [9] S. Heckers, M. Zalesak, A.P. Weiss, T. Ditman, D. Titone, Hippocampal activation during transitive inference in humans, Hippocampus 14 (2004) 153–162. [10] N. Ishizuka, W.M. Cowan, D.G. Amaral, A quantitative analysis of the dendritic organization of pyramidal cells in the rat hippocampus, J. Comp. Neurol. 362 (1995) 17–45. [11] N. Ishizuka, J. Weber, D.G. Amaral, Organization of intra-hippocampal projections originating from CA3 pyramidal cells in the rat, J. Comp. Neurol. 295 (1990) 580–623. [12] E.M. Izhikevich, N.S. Desai, Relating STDP to BCM, Neural Comput. 15 (2003) 1511–1523. [13] R.P. Kesner, I. Lee, P. Gilbert, A behavioral assessment of hippocampal function based on a subregional analysis, Rev. Neurosci. 15 (2004) 333–351. [14] W.B. Levy, A sequence predicting CA3 is a flexible associator that learns and uses context to solve hippocampal-like tasks, Hippocampus 6 (1996) 579–590. [15] W.B. Levy, X. Wu, R.A. Baxter, Unification of hippocampal function via computational/encoding considerations, Int. J. Neural Syst. 6 (1995) 71–80. [16] K. Nakazawa, M.C. Quirk, R.A. Chitwood, M. Watanabe, M.F. Yeckel, L.D. Sun, A. Kato, C.A. Carr, D. Johnston, M.A. Wilson, S. Tonegawa, Requirement for hippocampal CA3 NMDA receptors in associative memory recall, Science 297 (2002) 211–218. [17] A.R. Preston, Y. Shrager, N.M. Dudukovic, J.D.E. Gabrieli, Hippocampal contribution to the novel use of relational information in declarative memory, Hippocampus 14 (2004) 148–152. [18] T. Samura, M. Hattori, S. Ishizaki, Autoassociative and heteroassociative hippocampal CA3 model based on location dependencies derived from anatomical and physiological findings, Int. Congr. Ser. 1301 (2007) 140–143. [19] A. Treves, E.T. Rolls, Computational constraints suggest the need for two distinct input systems to the hippocampal CA3 network, Hippocampus 2 (1992) 189–199. [20] M. Tsukada, T. Aihara, Y. Kobayashi, H. Shimazaki, Spatial analysis of spiketiming-dependent LTP and LTD in the CA1 area of hippocampal slices using optical imaging, Hippocampus 15 (2005) 104–109.
3183
[21] Y. Yamaguchi, Theta phase coding and memory in the hippocampus, Seitai No Kagaku 55 (2004) 33–42. Toshikazu Samura received the bachelor degree in Engineering in 2003, and the master degree in Engineering in 2005, both from University of Yamanashi. He is now a Ph.D. student in Graduate School of Media and Governance at Keio University. He is engaged in research on neural networks, especially the information processing of the hippocampus.
Motonobu Hattori was born in Tokyo, Japan, on February 11, 1970. He received B.E., M.E. and Ph.D degrees in Electrical Engineering from Keio University in 1992, 1994 and 1997, respectively. In 1997, he became a research associate of University of Yamanashi, Kofu, Japan. Since 2000, he has been an Associate Professor. From 2003 to 2004, he was a visiting researcher at Carnegie Mellon University. In 1996, he received Niwa Memorial Award. His current research interest is in neural networks, reinforcement learning and evolutionary computation. He is a member of the IEEE, IEICE and IEE. Shun Ishizaki was born in 1947 in Tokyo, Japan. He graduated from the University of Tokyo in 1970. He received Ph.D. degree from the university. He joined Electro-technical Laboratory of MITI after he worked as a research associate at the university. He was a director of Machine Inference Section and then Natural Language Section of the laboratory. He joined Keio University in 1992 as a professor at Environmental Information Faculty. He was the president of Japanese Cognitive Science Society and also Japanese Natural Language Processing Association. He is now interested in neuroinformatics, natural language processing, machine learning and cognitive science researches.