- Email: [email protected]
Spike-Timing-Dependent Plasticity Models
Spike-Timing-Dependent Plasticity Models
269
Spike-Timing-Dependent Plasticity Models J L van Hemmen, Technical University of Munich, Garching/Munich, Germany ã 2009 Elsevier Ltd. All rights reserved.
wondering where the ‘metabolic change’ might take place, but Hebb already postulated explicitly that all this happens in synapses connecting the pre- and postsynaptic neurons. So, we focus here on preand postsynaptic spikes.
Introduction Activity in a neuronal network is spatiotemporal. That is, it changes in space all over the network and in time for each specific neuron separately. How, then, can a network learn such detailed activity? Furthermore, what precisely does learning mean? In this article we focus on spiking neurons so that spatiotemporal activity for a network of size N means a collection of f spike times ti , where f indicates the spikes of neuron i and 1 i N labels the neurons. Learning then means that, as a result of and starting with a specific set of initial states, the network repeats the same spatiotemporal activity pattern as time proceeds. Of course, we implicitly assume ceteris paribus (the other things and thus all inputs from outside being equal). To understand learning, two fundamental principles are important. First, where does learning take place? The kind of learning we are studying here depends on the arrival of spikes and is strictly local, meaning that events that merely occur at or near a synapse where learning happens are important to the change of the synaptic strength (or efficacy). If this depends only on pre- and postsynaptic spikes, it is called Hebbian. Second, learning is, in general, slow with respect to the neuronal timescale (milliseconds); for instance, learning associated with sensory modalities takes days, weeks, or even longer. That is (to express it sloppily), most learning relies on repetition: practice makes perfect. The organism repeats the very same or similar patterns or sequences of patterns several times and, in so doing, trains the synapses. Hence, it is bound to be slow. Furthermore, for sensory systems this suggests that interesting phenomena in map formation also happen slowly. We define a map to be a neuronal representation of the outside sensory world. If learning is slow, then long-time correlations are picked up but short-time ones, such as white or light-colored noise, are averaged out. As for locality of learning, Donald Hebb’s classic The Organization of Behavior (1949, p. 62), contains the now famous neurophysiological postulate: “When an axon of cell A is near enough to excite a cell B and repeatedly or persistently takes part in firing it, some growth process or metabolic change takes place in one or both cells such that A’s efficiency, as one of the cells firing B, is increased”. We are then left
Learning a Spatiotemporal Pattern as Simply as Possible To make our argument as clear as possible, we simplify it for the moment as much as we can and discretize time by units Dt ¼ 1 ms, with the idea that a spike generically lasts 1 ms. Furthermore, we also discretize the states of our neurons i; 1 i N, through a state variable Si, assuming two values: Si ¼ 1 if a neuron fires a spike at a certain time and Si ¼ 1 if it does not (the negative sign signaling that nothing happens). This type of neuron is a classic one, described in 1943 by McCulloch and Pitts, and equipped with universal computational capabilities. To remember a pattern to come, given the network state at time t, we need the network’s past – the more of it the better – so as to predict the future state at time t þ Dt. How, then, can the network look backward in time to sample these data? That is both easy and natural – through delays. A neuronal network has lots of delays, most prominently the axonal ones. This is one of the key ideas behind the argument presented here. Interestingly, this idea is historical as well, and we therefore follow this line of thought for a while. We focus on an excitatory synapse connecting the presynaptic neuron j to the postsynaptic neuron i. Its strength is denoted by Jij. The axonal delay it takes a spike fired by j to arrive at a synapse on neuron i is Dax ij . Imprinting on the network a sequence of states SðtÞ ¼ fSi ðtÞ; 1 i Ng during a period 1 t Tl , where Tl will soon be specified, our synapse should contribute to the learning process in an appropriate way. We can now state without further ado (and then verify that this makes sense) that the simplest possible way for a synaptic change DJij is given by l 1X Si ðt þ tÞ½Sj ðt ax ij Þ þ 1 Tl t¼1
T
Jij ¼ ij
½1
This expression is to be added to the already existing synaptic efficacy Jij of a synapse connecting neuron j to neuron i. What does eqn [1] mean? At time t, a spike arrives at a synapse on neuron i, if at time t Dax ij neuron
270 Spike-Timing-Dependent Plasticity Models
j has fired. If it has not, then [Sj(t-Dax ij ) þ 1] ¼ 0, in agreement with the fact that neuronal networks are characterized by low activity so that what is not active should not count. Only if a spike arrives does the synapse change by 2ij according to the postsynaptic activity Si ðt þ tÞ at the time t þ Dt following the arrival of the presynaptic spike at time t. Because learning is slow, the factor ij is to be small, meaning in suitable units 0 < ij 1. Now comes the key point. We focus on an excitatory synapse where a spike arrives at time t. The argument of the sum in eqn [1] tells us that the synapse is to be strengthened by 2ij if neuron i fires at time t þ Dt, whereas it is to be weakened by 2ij if neuron i does not fire. That is, if as an excitatory synapse it does its job, telling i to fire and thus causing the postsynaptic neuron to fire at time t þ Dt, then the synapse is to be strengthened. If, on the other hand, at time t þ Dt the postsynaptic neuron does not fire, then the synapse does not do its job properly and it is to be weakened. “Those that come too late shall be punished.” As for the rationale, it all fits. Furthermore, we average over a time Tl, meaning we sum up, divide by Tl, and are done. How big or small, then, should Tl be? It is here that the adiabatic hypothesis 0 < ij 1 works to advantage – to great advantage, even. Because in real life learning is slow, which is equivalent to saying 0 < ij 1, we can choose, on the one hand, a Tl much bigger than all neuronal timescales (which are of the order of milliseconds) and, on the other hand, we can choose a Tl much smaller than the time needed to learn a specific task (which takes days or weeks). A gerbil needs 2 days and a barn owl needs 2 weeks to learn azimuthal sound localization – and for most humans learning to playing a piece by Chopin on the piano in 2 weeks is an optimistic estimate. Hence, we can imagine Tl to separate microscopic neuronal timescales (in milliseconds) from macroscopic learning times (in days and longer). Finally, why does eqn [1] work as an algorithm to store a spatiotemporal pattern? Figure 1 shows that it does work, but why does it? The key to understanding this is that the synapses use the very same pattern in conjunction with the very same wetware, that is, with the very same neuronal hardware, such as axons and synapses, and hence with the very same axonal delays Dax ij , both for learning and later on for replay. A closer look at Figure 1 readily reveals that the total pattern consists of the disjoint union of parts occurring at subsequent times. Both eqn [1] and the upshot of the physiological learning process are the sum of subsequent parts. If these, or different subsequent patterns, are strongly correlated – roughly
Figure 1 Motion of a ‘phase boundary’, a string of active neurons (black pixels), through a 20 20 storage layer. The system starts with a single point in the upper left-hand corner, and the string develops as time proceeds (top to bottom; first left, then right). During the motion, the number of black pixels varies between 1 and 20. From van Hemmen JL, Gerstner W, Herz AVM, Ku¨hn R, and Vaas M (1990) Encoding and decoding of patterns which are correlated in space and time. In: Dorffner G (ed.) Konnektionismus in Artificial Intelligence and Kognitionsforschung, pp. 153–162. Berlin: Springer.
speaking, strongly resembling one another – there is a correlation problem needing an outside solution that is also outside the present context (e.g., through unlearning).
Spike-Timing-Dependent Plasticity The inputs provided by many synapses decide what a neuron does, but, once it has fired, a local synaptic algorithm determines whether each of the synaptic efficacies increases or decreases. Also, without time discretization and when a synapse can remember more than the very last time step (i.e., over a period of time lasting longer than the time span Dt), there is a suitable generalization – the synaptic learning window, to which we now turn. This is a generalization of what we see in eqn [1], called (for good reason, as we will see) spike-timing-dependent plasticity (STDP). Each of the terms in eqn [2], below, has a
Spike-Timing-Dependent Plasticity Models
neurobiological origin. We designate the process they describe infinitesimal learning because the synaptic increments and decrements are small and, consequently, it takes quite a while before the organism builds up a noticeable effect. For the sake of definiteness, we now study the waxing and waning of synaptic strengths associated with a single neuron, which therefore need not carry a label (Figure 2). The 1 i N synapses stemming from the neurons that we originally labeled j but now relabel i provide their input at times tif . The firing times of the postsynaptic neuron are denoted by tn, where n is a label like f. Given the firing times, the change Ji ðtÞ: ¼ Ji ðtÞ Ji ðt Tl ) of the efficacy of synapse i (synaptic strength) during a learning session of duration Tl and ending at time t is governed by several factors: 2 X 6 X Ji ðtÞ ¼ 4 win þ wout tTl tn < t
f
tTl ti < t
þ
3
X
½2
7 Wðtif tn Þ5
tTl tif ;tn < t
which exhibit STDP in the literal sense of the words. Here the firing times tn of the postsynaptic neuron may, and in general will, depend on Ji; more precisely, they depend on all the Ji with 1 i N. We now focus on the individual terms of eqn [2] and treat them in turn. The prefactor 0 < 1 reminds us explicitly that learning is slow on a neuronal time scale. Throughout what follows, we refer to this condition as the ‘adiabatic hypothesis’. This holds in numerous biological situations and has been a mainstay of computational neuroscience ever since it was proposed. It may also play a beneficial role in an applied context. If the adiabatic hypothesis does not hold and is not small, a numerical implementation of
the learning rule (eqn [2]) is straightforward but an analytical treatment is not. Furthermore, each incoming spike and each action potential of the postsynaptic neuron change the synaptic efficacy in eqn [2] by win and wout, respectively. The last term in eqn [2] represents the ‘learning window’ W(s), which indicates the synaptic change in dependence on the time difference s ¼ tf tn f between an incoming spike ti and an outgoing spike n t . When the former precedes the latter, we have s < 0 , tif < tn , and the result is W(s) > 0, implying potentiation. This seems reasonable because N-methyl-D-aspartate (NMDA) receptors, which are important for long-term potentiation (LTP), need a strongly positive membrane voltage to become accessible by losing the Mg2þ ions that block their gate. A postsynaptic action potential induces a fast retrograde spike doing exactly this. Because the presynaptic spike arrived slightly earlier, the neurotransmitter is waiting to gain access, which is allowed after the Mg2þ ions are gone. The result is Ca2þ influx. On the other hand, if the incoming spike comes too late, then s > 0 and WðsÞ < 0, implying depression – which is in agreement with the rule “Those that come too late shall be punished”. In neurobiological terms, there is no neurotransmitter waiting to be admitted. The learning rule (eqn [2]) is a direct extension of eqn [1], its time-discrete predecessor. There is, meanwhile, extensive neurobiological evidence in favor of this time-resolved Hebbian learning. An illustration of what a learning window does is given in Figure 3 below. Once new (infinitesimal) learning algorithms have been discovered, we can simply adapt W accordingly. For instance, for inhibitory synapses infinitesimal growth processes have been found that can be described qualitatively by putting W: ¼ W in Figure 4, which figure and eqn [3] show a typical learning window for an excitatory synapse WðsÞ ¼
expðs=tsyn Þ½Aþ ð1 s=~tþ Þ þ A ð1 s=~t Þ for s 0; Aþ expðs=tþ Þ þ A expðs=t Þ for s > 0: f
S out
Output
J1
JN J2
Input
Ji Siin
Figure 2 Single neuron. We study the development of synaptic weights Ji (small filled circles, 1 i N) of a single neuron (large circle). The neuron receives input spike trains, denoted by Siin, and produces output spikes denoted by Sout.
271
½3
Here, as before, s ¼ ti tn is the time difference between the arrival of the presynaptic spike and the postsynaptic firing, is our small learning parameter, ~tþ : ¼ tsyn tþ =ðtsyn þ tþÞ, and ~t:¼ tsyn t =ðtsyn þ t Þ. Typical parameter values used in numerical simulations are ¼ 105, Aþ ¼ 1, A ¼ 1, tsyn ¼ 5 ms, tþ ¼ 1 ms, and t ¼ 20 ms. The philosophy behind eqn [3] and Figure 4 is straightforward. Neuronal spike generation happens at the axon hillock, where synaptic input is perceived through a postsynaptic potential of finite temporal width. Once a presynaptic spike has arrived at an excitatory synapse at time s, when W(s) is
272 Spike-Timing-Dependent Plasticity Models
approximately maximal, then the synapse’s influence is felt optimally at time s ¼ 0, when the postsynaptic neuron fires. This makes the finite width of W readily understandable. On the basis of S iin(t)
t i3
t i2
t i1
S out(t)
t1
t
t i4
t
t2
W(s) 0
s
0
Ji (t) w out
w in
0
w in + W(ti4 − t 2)
w in
w in
s
0
minimal assumptions in conjunction with the adiabatic hypothesis (0 < 1), we can now derive a general learning equation: Z 1 d Ji ¼ win iin þ wout out þ dsWðsÞCi ðs; tÞ ½4 dt 1 In this form the learning equation is easy to remember – the input rate vin i modifies the synaptic efficacy through win, the output rate nout does so through wout, and the Hebbian correlation function Ci favors or disfavors it through the learning window W. In eqn [4] the correlation function Ci is defined through
w out + W(ti3 − t 2) t
Figure 3 Schematic of Hebbian learning and spiking neurons. Shown in the bottom graph is the time course of the synaptic weight Ji(t), evoked through input and output spikes (upper graphs, vertical bars). An output spike (e.g., at time t1) induces the weight Ji to change by an amount wout, which is negative here. To show the effect of correlations between input and output spikes, the learning window W(s) (middle graphs) has been indicated around each output spike; s ¼ 0 matches the output spike times (vertical dashed lines). The three input spikes at times tif ¼ ti1 ; ti2 ; and ti3 (vertical dotted lines) increase Ji by an amount win each. There are no correlations between these input spikes and the output spike at time t1. This becomes clear once we look at them through the learning window W, centered at t1; the input spikes are too far away in time. The next output spike at t2, however, is close enough to the previous input spike at ti3 . The weight Ji is changed by wout < 0 plus the contribution Wðti3 t 2 Þ > 0, the sum of which is positive (arrowheads). Similarly, the input spike at time ti4 leads to a change w in þ Wðti4 t 2 Þ<0. From Kempter R, Gerstner W, and van Hemmen JL (1999) Hebbian learning and spiking neurons. Physical Review E 59: 4498–4514.
Ci ðs; tÞ: ¼
1 Tl
Z
t
tTl
0 out 0 dt0 hSin ðt Þi i ðt þ sÞS
½5
where Sin i is the input spike train (a flow of delta functions), Sout is its analog for the output spike train (cf. Figure 2), and the angle brackets denote an average over stochasticity in the neuronal dynamics with short-range correlations of the order of milliseconds. We now turn to two applications of STDP that show clearly why and how conjugate pre- and postsynaptic timing can together tune a collection of synapses on a neuron such as that in Figure 2. Barn Owl’s Laminar Nucleus
Our first example illustrates how STDP sampling singles out the right synapses. It is, in fact, what STDP had been made for. Azimuthal sound localiza-
2h W(s) h s (ms) −40
−20
s* 0
20
40
60
Change in EPSC amplitude (%)
100
−h a
80 60 40 20 0 −20 −40 −60
b
−100 −80 −60 −40 −20 0 20 40 60 Time of synaptic input (ms)
80 100
Figure 4 The learning window W: (a) in units of the learning parameter as a function of the delay s ¼ tif t n between the presynaptic spike arrival at synapse i at time tif and the postsynaptic firing at time tn; (b) experimentally obtained learning window of a cell in the rat hippocampus with EPSC as excitatory postsynaptic current. In (a), if W(s) is positive (negative) for some s, the synaptic efficacy Ji is increased (decreased). The increase of Ji is most efficient if a presynaptic spike arrives a few milliseconds before the postsynaptic neuron starts firing (vertical dashed line at s ¼ s*). For |s| ! 1 we have W(s) ! 0. The form of the learning window and parameter values are as described in eqn [3]. The similarity between (b) and (a) is evident. It is important to realize that the width of the learning window must be in agreement with the other neuronal time constants. In the auditory system, for instance, these are nearly two orders of magnitude smaller, so the learning window’s width scales accordingly. Panel (b) From Zhang LI, Tao HW, Holt CE, Harris WA, and Poo M (1998) A critical window for cooperation and competition among developing retinotectal synapses. Nature 395: 37–44.
Spike-Timing-Dependent Plasticity Models
tion is based on phase locking, a correlated neuronal response to cochlear input. In the barn owl, this happens up to 8 kHz. The animal’s laminar nucleus is the first place where signals from both ears come together, but the response is completely smeared out because the axonal delays from left and right ears have a broad distribution, approximately 1 ms wide (Figure 5(a)). This is the situation 3 weeks after hatching, when the head of the young barn owl is adult but the owl cannot perform sound localization yet. Two weeks later, the owl reaches a microsecond precision through an STDP tuning. The result of this is shown in Figures 5(b) and 5(c) for 2 and 5 kHz, respectively. Every axonal delay line preceding a synapse and equipped with the right delay is strengthened; the others are – so to speak – weeded 10
0 Delays per bin
a
b
10 T
0 10 T
273
out. Delay lines differing by one or several periods (here 500 or 200 ms) are effectively the same. The key mechanism involved in STDP is correlating incoming presynaptic and outgoing postsynaptic spikes. If the postsynaptic neuron fires, we often get a backpropagating spike guaranteeing a fast message telling the synapse, so to speak, that the neuron it is on has fired. In the auditory system, neurons are even more compact than usual, so synapses practically directly feel that their postsynaptic neuron has fired. Let us now imagine a laminar neuron, as in Figure 2. We start with a collection of synaptic efficacies with a uniform value and with an axonal delay line connected to each synapse providing it with spikes coming from the left or right ear; the wide delay distribution is shown in Figure 5(a). Independent of the other delay lines, each delay line is fed with an inhomogeneous periodic Poisson process with frequency n and angular frequency o ¼ 2pn as input, that is, a point process with rate function lðtÞ ¼ c þ sinðotÞ and c 1, a constant. If l(t) is small, the probability of getting an input spike is small, if it is large, the probability is large (although always l c þ 1), and cochlear input is always noisy. As we see from Figures 5(b) and 5(c), during a training period under the influence of external auditory input only few delay lines survive through a collective coupling of postsynaptic firing and well-timed arrival of presynaptic spikes. (In passing, we note that for the sake of convenience the asymptotic saturation of the synaptic efficacies was built in.) Place Cells in the Rat’s Hippocampus
c
0 1.5
2.5
3.5
Figure 5 Modeling the selection of synapses connected to axonal delay lines during synaptic pattern formation of the synaptic connectivity on an integrate-and-fire neuron as in Figure 2: (a) 3 weeks after hatching a young barn owl’s head is adult but the animal cannot perform azimuthal sound localization yet; in view of the broad (here Gaussian) distribution of delays, with a standard derivation of 0.5 ms, this is not surprising. The input through 300 axons from the left and 300 axons from the right ear is taken to be a periodic, inhomogeneous Poissonian input with frequency ¼ 1=T , where n is the best frequency stemming from the cochlea and T ¼ n1 is the corresponding period. After 2 more weeks the barn owl is able to localize a sound source and the resulting delay, i.e., synaptic distribution, is shown in (b) and (c) for best frequencies 2 and 5 kHz, respectively. We have obtained, so to speak, survival-of-the-fittest synapses with delays within in a narrow temporal domain or with corresponding delays shifted by one or more periods, which are identical as far as oscillations by period T are concerned. The rest are less fit and have been eliminated, a striking consequence of spike-timing-dependent plasticity. From Gerstner W, Kempter R, van Hemmen JL, and Wagner H (1996) A neuronal learning rule for sub-millisecond temporal coding. Nature 383: 76–78.
A completely analogous argument to the one for the barn owl auditory system may well hold for hippocampal place cells too. There is, however, a neat difference. The typical time constant in the auditory system is 0.1 ms; on the other hand, it is 10 ms in the hippocampus, that is, roughly two orders of magnitude more. We may therefore expect that the temporal width of the learning window W depends on the physiological structures it lives in. That is, the width of W in the hippocampus is, indeed, 50–100 ms.
Conclusion From a theoretical point of view, STDP is extremely appealing. Depending on the arrival time of a presynaptic spike at a synapse on the postsynaptic neuron, the synapse is strengthened or weakened depending on the firing time of the postsynaptic neuron. If the postsynaptic neuron fires at t ¼ 0 and the synapse is excitatory, then it is strengthened if it receives a presynaptic spike at a time t < 0 and thus tells the
274 Spike-Timing-Dependent Plasticity Models
neuron to fire before the latter actually fires. That is, it is doing its job properly, and thus its synaptic strength increases. If, on the other hand, the presynaptic spike arrives at t > 0, then it is too late because the postsynaptic neuron has already fired. “Those that come too late shall be punished,” and, hence, the synaptic strength decreases. All this is taken care of by the function W, the learning window in Figure 4. Clearly, if the arrival time of the presynaptic spike and the firing time of the postsynaptic neuron are too far apart, then nothing happens. What we have analyzed in this article is STDP as a stand-alone phenomenon, that is, in its full simplicity. Neurobiological reality is both more complicated and more fascinating than this, in that STDP automatically normalizes synaptic strengths and gives rise to a whole field of applications such as map formation. Let us now briefly consider these phenomena in turn. The arguments presented so far hold for a single pair of spikes, a presynaptic one and a postsynaptic one. It has turned out experimentally that synaptic modification may also depend on the spiking pattern, that is, on the interspike intervals within each neuron. In addition, the precise position of a synapse on the dendritic tree may play a role as well. Furthermore, STDP gives rise to a normalization of the synaptic efficacies so that they do not diverge as time proceeds. Finally, STDP allows for a comprehensive explanation of neuronal map formation, one of the most intricate, yet manageable problems in today’s neuroscience. As we have seen, a map is a neuronal representation of the outside sensory world. Although STDP itself is local, map formation can be explained in detail by means of STDP in conjunction with more globally operating mechanisms. In short, because the very same neuronal wetware with the very same delays is used both for learning and for replay, a reproduction (replay) of what is to be learned is possible. Because STDP is local, it cannot handle global correlations. In spite of that, STDP is a very potent mechanism for storing time-resolved spike data at a scale finer than a millisecond and in this way can explain a huge diversity of neuronal phenomena. As such, it exhibits the mathematical universality to explain neurobiological diversity. See also: Developmental Synaptic Plasticity: LTP, LTD, and Synapse Formation and Elimination; Hebbian Plasticity; Long-Term Potentiation and Long-Term Depression in Experience-Dependent Plasticity; SpikeTiming Dependent Plasticity (STDP); Synaptic Plasticity and Place Cell Formation; Synaptic Plasticity: Short-Term Mechanisms.
Further Reading Abbott LF and Regehr WG (2004) Synaptic computation. Nature 431: 796–803. Bi G and Poo M (2001) Synaptic modification by correlated activity: Hebb’s postulate revisited. Annual Review of Neuroscience 24: 139–166. Caillard O, Ben-Ari Y, and Gaiarsa J-L (1999) Mechanisms of induction and expression of long-term depression at GABAergic synapses in the neonatal rat hippocampus. Journal of Neuroscience 19: 7568–7577. Froemke RC, Poo M, and Dan Y (2005) Spike-timing-dependent synaptic plasticity depends on dendritic location. Nature 434: 221–225. Gerstner W and Abbott LF (1997) Learning navigational maps through potentiation and modulation of hippocampal place cells. Journal of Comparative Neuroscience 4: 79–94. Gerstner W, Kempter R, van Hemmen JL, and Wagner H (1996) A neuronal learning rule for sub-millisecond temporal coding. Nature 383: 76–78. Hebb DO (1949) The organization of behavior: A neuropsychological theory. Wiley: New York. Herz AVM, Sulzer B, Ku¨hn R, and van Hemmen JL (1989) Hebbian learning reconsidered: Representation of static and dynamic objects in associative neural nets. Biological Cybernetics 60: 457–467. Kempter R, Gerstner W, and van Hemmen JL (1999) Hebbian learning and spiking neurons. Physical Review E 59: 4498–4514. Kempter R, Gerstner W, and van Hemmen JL (2001) Intrinsic stabilization of output rates by spike-based Hebbian learning. Neural Computation 13: 2709–2741. Kempter R, Leibold C, Wagner H, and van Hemmen JL (2001) Formation of temporal feature maps by axonal propagation of synaptic learning. Proceedings of the National Academy of Sciences of the United States of America 98: 4166–4171. Koch C (1999) Biophysics of Computation. New York: Oxford University Press. Linden DJ (1999) The return of the spike: Postsynaptic action potentials and the induction of LTP and LTD. Neuron 22: 661–666. Lisman J and Spruston N (2005) Postsynaptic depolarization requirements for LTP and LTD: A critique of spike timing dependent plasticity. Nature Neuroscience 8: 839–841. Morrison A, Diesmann M, and Gerstner W (2008) Phenomenological models of synaptic plasticity based on spike timing. Biological Cybernetics 98: 459–478. Rubin JE, Gerkin RC, Bi G, and Chow CC (2005) Calcium time course as a signal for spike-timing-dependent plasticity. Journal of Neurophysiology 93: 2600–2613. Shon AP, Rao RPN, and Sejnowski TJ (2004) Motion detection and prediction through spike-timing dependent plasticity. Network: Computation in Neural Systems 15: 179–198. Song S, Miller KD, and Abbott LF (2000) Competitive Hebbian learning through spike-timing-dependent synaptic plasticity. Nature Neuroscience 3: 919–926. Stuart G, Spruston N, Sakmann B, and Ha¨usser M (1997) Action potential initiation and backpropagation in neurons of the mammalian CNS. Trends in Neuroscience 20: 125–131. van Hemmen JL (2001) Theory of synaptic plasticity. In: Moss F and Gielen S (eds.) Handbook of Biophysics, vol. 4, pp. 771–823. Amsterdam: Elsevier. van Hemmen JL, Gerstner W, Herz AVM, Ku¨hn R, and Vaas M (1990) Encoding and decoding of patterns which are correlated in space and time. In: Dorffner G (ed.) Konnektionismus in
Spike-Timing-Dependent Plasticity Models Artificial Intelligence and Kognitionsforschung, pp. 153–162. Berlin: Springer. Waters J, Schaefer A, and Sakmann B (2005) Backpropagating action potentials in neurons: Measurement, mechanisms and potential functions. Progress in Biophysics & Molecular Biology 87: 145–170.
275
Wenisch OG, Noll J, and van Hemmen JL (2005) Spontaneously emerging direction selectivity maps in visual cortex through STDP. Biological Cybernetics 93: 239–247. Zhang LI, Tao HW, Holt CE, Harris WA, and Poo M (1998) A critical window for cooperation and competition among developing retinotectal synapses. Nature 395: 37–44.
269
Spike-Timing-Dependent Plasticity Models J L van Hemmen, Technical University of Munich, Garching/Munich, Germany ã 2009 Elsevier Ltd. All rights reserved.
wondering where the ‘metabolic change’ might take place, but Hebb already postulated explicitly that all this happens in synapses connecting the pre- and postsynaptic neurons. So, we focus here on preand postsynaptic spikes.
Introduction Activity in a neuronal network is spatiotemporal. That is, it changes in space all over the network and in time for each specific neuron separately. How, then, can a network learn such detailed activity? Furthermore, what precisely does learning mean? In this article we focus on spiking neurons so that spatiotemporal activity for a network of size N means a collection of f spike times ti , where f indicates the spikes of neuron i and 1 i N labels the neurons. Learning then means that, as a result of and starting with a specific set of initial states, the network repeats the same spatiotemporal activity pattern as time proceeds. Of course, we implicitly assume ceteris paribus (the other things and thus all inputs from outside being equal). To understand learning, two fundamental principles are important. First, where does learning take place? The kind of learning we are studying here depends on the arrival of spikes and is strictly local, meaning that events that merely occur at or near a synapse where learning happens are important to the change of the synaptic strength (or efficacy). If this depends only on pre- and postsynaptic spikes, it is called Hebbian. Second, learning is, in general, slow with respect to the neuronal timescale (milliseconds); for instance, learning associated with sensory modalities takes days, weeks, or even longer. That is (to express it sloppily), most learning relies on repetition: practice makes perfect. The organism repeats the very same or similar patterns or sequences of patterns several times and, in so doing, trains the synapses. Hence, it is bound to be slow. Furthermore, for sensory systems this suggests that interesting phenomena in map formation also happen slowly. We define a map to be a neuronal representation of the outside sensory world. If learning is slow, then long-time correlations are picked up but short-time ones, such as white or light-colored noise, are averaged out. As for locality of learning, Donald Hebb’s classic The Organization of Behavior (1949, p. 62), contains the now famous neurophysiological postulate: “When an axon of cell A is near enough to excite a cell B and repeatedly or persistently takes part in firing it, some growth process or metabolic change takes place in one or both cells such that A’s efficiency, as one of the cells firing B, is increased”. We are then left
Learning a Spatiotemporal Pattern as Simply as Possible To make our argument as clear as possible, we simplify it for the moment as much as we can and discretize time by units Dt ¼ 1 ms, with the idea that a spike generically lasts 1 ms. Furthermore, we also discretize the states of our neurons i; 1 i N, through a state variable Si, assuming two values: Si ¼ 1 if a neuron fires a spike at a certain time and Si ¼ 1 if it does not (the negative sign signaling that nothing happens). This type of neuron is a classic one, described in 1943 by McCulloch and Pitts, and equipped with universal computational capabilities. To remember a pattern to come, given the network state at time t, we need the network’s past – the more of it the better – so as to predict the future state at time t þ Dt. How, then, can the network look backward in time to sample these data? That is both easy and natural – through delays. A neuronal network has lots of delays, most prominently the axonal ones. This is one of the key ideas behind the argument presented here. Interestingly, this idea is historical as well, and we therefore follow this line of thought for a while. We focus on an excitatory synapse connecting the presynaptic neuron j to the postsynaptic neuron i. Its strength is denoted by Jij. The axonal delay it takes a spike fired by j to arrive at a synapse on neuron i is Dax ij . Imprinting on the network a sequence of states SðtÞ ¼ fSi ðtÞ; 1 i Ng during a period 1 t Tl , where Tl will soon be specified, our synapse should contribute to the learning process in an appropriate way. We can now state without further ado (and then verify that this makes sense) that the simplest possible way for a synaptic change DJij is given by l 1X Si ðt þ tÞ½Sj ðt ax ij Þ þ 1 Tl t¼1
T
Jij ¼ ij
½1
This expression is to be added to the already existing synaptic efficacy Jij of a synapse connecting neuron j to neuron i. What does eqn [1] mean? At time t, a spike arrives at a synapse on neuron i, if at time t Dax ij neuron
270 Spike-Timing-Dependent Plasticity Models
j has fired. If it has not, then [Sj(t-Dax ij ) þ 1] ¼ 0, in agreement with the fact that neuronal networks are characterized by low activity so that what is not active should not count. Only if a spike arrives does the synapse change by 2ij according to the postsynaptic activity Si ðt þ tÞ at the time t þ Dt following the arrival of the presynaptic spike at time t. Because learning is slow, the factor ij is to be small, meaning in suitable units 0 < ij 1. Now comes the key point. We focus on an excitatory synapse where a spike arrives at time t. The argument of the sum in eqn [1] tells us that the synapse is to be strengthened by 2ij if neuron i fires at time t þ Dt, whereas it is to be weakened by 2ij if neuron i does not fire. That is, if as an excitatory synapse it does its job, telling i to fire and thus causing the postsynaptic neuron to fire at time t þ Dt, then the synapse is to be strengthened. If, on the other hand, at time t þ Dt the postsynaptic neuron does not fire, then the synapse does not do its job properly and it is to be weakened. “Those that come too late shall be punished.” As for the rationale, it all fits. Furthermore, we average over a time Tl, meaning we sum up, divide by Tl, and are done. How big or small, then, should Tl be? It is here that the adiabatic hypothesis 0 < ij 1 works to advantage – to great advantage, even. Because in real life learning is slow, which is equivalent to saying 0 < ij 1, we can choose, on the one hand, a Tl much bigger than all neuronal timescales (which are of the order of milliseconds) and, on the other hand, we can choose a Tl much smaller than the time needed to learn a specific task (which takes days or weeks). A gerbil needs 2 days and a barn owl needs 2 weeks to learn azimuthal sound localization – and for most humans learning to playing a piece by Chopin on the piano in 2 weeks is an optimistic estimate. Hence, we can imagine Tl to separate microscopic neuronal timescales (in milliseconds) from macroscopic learning times (in days and longer). Finally, why does eqn [1] work as an algorithm to store a spatiotemporal pattern? Figure 1 shows that it does work, but why does it? The key to understanding this is that the synapses use the very same pattern in conjunction with the very same wetware, that is, with the very same neuronal hardware, such as axons and synapses, and hence with the very same axonal delays Dax ij , both for learning and later on for replay. A closer look at Figure 1 readily reveals that the total pattern consists of the disjoint union of parts occurring at subsequent times. Both eqn [1] and the upshot of the physiological learning process are the sum of subsequent parts. If these, or different subsequent patterns, are strongly correlated – roughly
Figure 1 Motion of a ‘phase boundary’, a string of active neurons (black pixels), through a 20 20 storage layer. The system starts with a single point in the upper left-hand corner, and the string develops as time proceeds (top to bottom; first left, then right). During the motion, the number of black pixels varies between 1 and 20. From van Hemmen JL, Gerstner W, Herz AVM, Ku¨hn R, and Vaas M (1990) Encoding and decoding of patterns which are correlated in space and time. In: Dorffner G (ed.) Konnektionismus in Artificial Intelligence and Kognitionsforschung, pp. 153–162. Berlin: Springer.
speaking, strongly resembling one another – there is a correlation problem needing an outside solution that is also outside the present context (e.g., through unlearning).
Spike-Timing-Dependent Plasticity The inputs provided by many synapses decide what a neuron does, but, once it has fired, a local synaptic algorithm determines whether each of the synaptic efficacies increases or decreases. Also, without time discretization and when a synapse can remember more than the very last time step (i.e., over a period of time lasting longer than the time span Dt), there is a suitable generalization – the synaptic learning window, to which we now turn. This is a generalization of what we see in eqn [1], called (for good reason, as we will see) spike-timing-dependent plasticity (STDP). Each of the terms in eqn [2], below, has a
Spike-Timing-Dependent Plasticity Models
neurobiological origin. We designate the process they describe infinitesimal learning because the synaptic increments and decrements are small and, consequently, it takes quite a while before the organism builds up a noticeable effect. For the sake of definiteness, we now study the waxing and waning of synaptic strengths associated with a single neuron, which therefore need not carry a label (Figure 2). The 1 i N synapses stemming from the neurons that we originally labeled j but now relabel i provide their input at times tif . The firing times of the postsynaptic neuron are denoted by tn, where n is a label like f. Given the firing times, the change Ji ðtÞ: ¼ Ji ðtÞ Ji ðt Tl ) of the efficacy of synapse i (synaptic strength) during a learning session of duration Tl and ending at time t is governed by several factors: 2 X 6 X Ji ðtÞ ¼ 4 win þ wout tTl tn < t
f
tTl ti < t
þ
3
X
½2
7 Wðtif tn Þ5
tTl tif ;tn < t
which exhibit STDP in the literal sense of the words. Here the firing times tn of the postsynaptic neuron may, and in general will, depend on Ji; more precisely, they depend on all the Ji with 1 i N. We now focus on the individual terms of eqn [2] and treat them in turn. The prefactor 0 < 1 reminds us explicitly that learning is slow on a neuronal time scale. Throughout what follows, we refer to this condition as the ‘adiabatic hypothesis’. This holds in numerous biological situations and has been a mainstay of computational neuroscience ever since it was proposed. It may also play a beneficial role in an applied context. If the adiabatic hypothesis does not hold and is not small, a numerical implementation of
the learning rule (eqn [2]) is straightforward but an analytical treatment is not. Furthermore, each incoming spike and each action potential of the postsynaptic neuron change the synaptic efficacy in eqn [2] by win and wout, respectively. The last term in eqn [2] represents the ‘learning window’ W(s), which indicates the synaptic change in dependence on the time difference s ¼ tf tn f between an incoming spike ti and an outgoing spike n t . When the former precedes the latter, we have s < 0 , tif < tn , and the result is W(s) > 0, implying potentiation. This seems reasonable because N-methyl-D-aspartate (NMDA) receptors, which are important for long-term potentiation (LTP), need a strongly positive membrane voltage to become accessible by losing the Mg2þ ions that block their gate. A postsynaptic action potential induces a fast retrograde spike doing exactly this. Because the presynaptic spike arrived slightly earlier, the neurotransmitter is waiting to gain access, which is allowed after the Mg2þ ions are gone. The result is Ca2þ influx. On the other hand, if the incoming spike comes too late, then s > 0 and WðsÞ < 0, implying depression – which is in agreement with the rule “Those that come too late shall be punished”. In neurobiological terms, there is no neurotransmitter waiting to be admitted. The learning rule (eqn [2]) is a direct extension of eqn [1], its time-discrete predecessor. There is, meanwhile, extensive neurobiological evidence in favor of this time-resolved Hebbian learning. An illustration of what a learning window does is given in Figure 3 below. Once new (infinitesimal) learning algorithms have been discovered, we can simply adapt W accordingly. For instance, for inhibitory synapses infinitesimal growth processes have been found that can be described qualitatively by putting W: ¼ W in Figure 4, which figure and eqn [3] show a typical learning window for an excitatory synapse WðsÞ ¼
expðs=tsyn Þ½Aþ ð1 s=~tþ Þ þ A ð1 s=~t Þ for s 0; Aþ expðs=tþ Þ þ A expðs=t Þ for s > 0: f
S out
Output
J1
JN J2
Input
Ji Siin
Figure 2 Single neuron. We study the development of synaptic weights Ji (small filled circles, 1 i N) of a single neuron (large circle). The neuron receives input spike trains, denoted by Siin, and produces output spikes denoted by Sout.
271
½3
Here, as before, s ¼ ti tn is the time difference between the arrival of the presynaptic spike and the postsynaptic firing, is our small learning parameter, ~tþ : ¼ tsyn tþ =ðtsyn þ tþÞ, and ~t:¼ tsyn t =ðtsyn þ t Þ. Typical parameter values used in numerical simulations are ¼ 105, Aþ ¼ 1, A ¼ 1, tsyn ¼ 5 ms, tþ ¼ 1 ms, and t ¼ 20 ms. The philosophy behind eqn [3] and Figure 4 is straightforward. Neuronal spike generation happens at the axon hillock, where synaptic input is perceived through a postsynaptic potential of finite temporal width. Once a presynaptic spike has arrived at an excitatory synapse at time s, when W(s) is
272 Spike-Timing-Dependent Plasticity Models
approximately maximal, then the synapse’s influence is felt optimally at time s ¼ 0, when the postsynaptic neuron fires. This makes the finite width of W readily understandable. On the basis of S iin(t)
t i3
t i2
t i1
S out(t)
t1
t
t i4
t
t2
W(s) 0
s
0
Ji (t) w out
w in
0
w in + W(ti4 − t 2)
w in
w in
s
0
minimal assumptions in conjunction with the adiabatic hypothesis (0 < 1), we can now derive a general learning equation: Z 1 d Ji ¼ win iin þ wout out þ dsWðsÞCi ðs; tÞ ½4 dt 1 In this form the learning equation is easy to remember – the input rate vin i modifies the synaptic efficacy through win, the output rate nout does so through wout, and the Hebbian correlation function Ci favors or disfavors it through the learning window W. In eqn [4] the correlation function Ci is defined through
w out + W(ti3 − t 2) t
Figure 3 Schematic of Hebbian learning and spiking neurons. Shown in the bottom graph is the time course of the synaptic weight Ji(t), evoked through input and output spikes (upper graphs, vertical bars). An output spike (e.g., at time t1) induces the weight Ji to change by an amount wout, which is negative here. To show the effect of correlations between input and output spikes, the learning window W(s) (middle graphs) has been indicated around each output spike; s ¼ 0 matches the output spike times (vertical dashed lines). The three input spikes at times tif ¼ ti1 ; ti2 ; and ti3 (vertical dotted lines) increase Ji by an amount win each. There are no correlations between these input spikes and the output spike at time t1. This becomes clear once we look at them through the learning window W, centered at t1; the input spikes are too far away in time. The next output spike at t2, however, is close enough to the previous input spike at ti3 . The weight Ji is changed by wout < 0 plus the contribution Wðti3 t 2 Þ > 0, the sum of which is positive (arrowheads). Similarly, the input spike at time ti4 leads to a change w in þ Wðti4 t 2 Þ<0. From Kempter R, Gerstner W, and van Hemmen JL (1999) Hebbian learning and spiking neurons. Physical Review E 59: 4498–4514.
Ci ðs; tÞ: ¼
1 Tl
Z
t
tTl
0 out 0 dt0 hSin ðt Þi i ðt þ sÞS
½5
where Sin i is the input spike train (a flow of delta functions), Sout is its analog for the output spike train (cf. Figure 2), and the angle brackets denote an average over stochasticity in the neuronal dynamics with short-range correlations of the order of milliseconds. We now turn to two applications of STDP that show clearly why and how conjugate pre- and postsynaptic timing can together tune a collection of synapses on a neuron such as that in Figure 2. Barn Owl’s Laminar Nucleus
Our first example illustrates how STDP sampling singles out the right synapses. It is, in fact, what STDP had been made for. Azimuthal sound localiza-
2h W(s) h s (ms) −40
−20
s* 0
20
40
60
Change in EPSC amplitude (%)
100
−h a
80 60 40 20 0 −20 −40 −60
b
−100 −80 −60 −40 −20 0 20 40 60 Time of synaptic input (ms)
80 100
Figure 4 The learning window W: (a) in units of the learning parameter as a function of the delay s ¼ tif t n between the presynaptic spike arrival at synapse i at time tif and the postsynaptic firing at time tn; (b) experimentally obtained learning window of a cell in the rat hippocampus with EPSC as excitatory postsynaptic current. In (a), if W(s) is positive (negative) for some s, the synaptic efficacy Ji is increased (decreased). The increase of Ji is most efficient if a presynaptic spike arrives a few milliseconds before the postsynaptic neuron starts firing (vertical dashed line at s ¼ s*). For |s| ! 1 we have W(s) ! 0. The form of the learning window and parameter values are as described in eqn [3]. The similarity between (b) and (a) is evident. It is important to realize that the width of the learning window must be in agreement with the other neuronal time constants. In the auditory system, for instance, these are nearly two orders of magnitude smaller, so the learning window’s width scales accordingly. Panel (b) From Zhang LI, Tao HW, Holt CE, Harris WA, and Poo M (1998) A critical window for cooperation and competition among developing retinotectal synapses. Nature 395: 37–44.
Spike-Timing-Dependent Plasticity Models
tion is based on phase locking, a correlated neuronal response to cochlear input. In the barn owl, this happens up to 8 kHz. The animal’s laminar nucleus is the first place where signals from both ears come together, but the response is completely smeared out because the axonal delays from left and right ears have a broad distribution, approximately 1 ms wide (Figure 5(a)). This is the situation 3 weeks after hatching, when the head of the young barn owl is adult but the owl cannot perform sound localization yet. Two weeks later, the owl reaches a microsecond precision through an STDP tuning. The result of this is shown in Figures 5(b) and 5(c) for 2 and 5 kHz, respectively. Every axonal delay line preceding a synapse and equipped with the right delay is strengthened; the others are – so to speak – weeded 10
0 Delays per bin
a
b
10 T
0 10 T
273
out. Delay lines differing by one or several periods (here 500 or 200 ms) are effectively the same. The key mechanism involved in STDP is correlating incoming presynaptic and outgoing postsynaptic spikes. If the postsynaptic neuron fires, we often get a backpropagating spike guaranteeing a fast message telling the synapse, so to speak, that the neuron it is on has fired. In the auditory system, neurons are even more compact than usual, so synapses practically directly feel that their postsynaptic neuron has fired. Let us now imagine a laminar neuron, as in Figure 2. We start with a collection of synaptic efficacies with a uniform value and with an axonal delay line connected to each synapse providing it with spikes coming from the left or right ear; the wide delay distribution is shown in Figure 5(a). Independent of the other delay lines, each delay line is fed with an inhomogeneous periodic Poisson process with frequency n and angular frequency o ¼ 2pn as input, that is, a point process with rate function lðtÞ ¼ c þ sinðotÞ and c 1, a constant. If l(t) is small, the probability of getting an input spike is small, if it is large, the probability is large (although always l c þ 1), and cochlear input is always noisy. As we see from Figures 5(b) and 5(c), during a training period under the influence of external auditory input only few delay lines survive through a collective coupling of postsynaptic firing and well-timed arrival of presynaptic spikes. (In passing, we note that for the sake of convenience the asymptotic saturation of the synaptic efficacies was built in.) Place Cells in the Rat’s Hippocampus
c
0 1.5
2.5
3.5
Figure 5 Modeling the selection of synapses connected to axonal delay lines during synaptic pattern formation of the synaptic connectivity on an integrate-and-fire neuron as in Figure 2: (a) 3 weeks after hatching a young barn owl’s head is adult but the animal cannot perform azimuthal sound localization yet; in view of the broad (here Gaussian) distribution of delays, with a standard derivation of 0.5 ms, this is not surprising. The input through 300 axons from the left and 300 axons from the right ear is taken to be a periodic, inhomogeneous Poissonian input with frequency ¼ 1=T , where n is the best frequency stemming from the cochlea and T ¼ n1 is the corresponding period. After 2 more weeks the barn owl is able to localize a sound source and the resulting delay, i.e., synaptic distribution, is shown in (b) and (c) for best frequencies 2 and 5 kHz, respectively. We have obtained, so to speak, survival-of-the-fittest synapses with delays within in a narrow temporal domain or with corresponding delays shifted by one or more periods, which are identical as far as oscillations by period T are concerned. The rest are less fit and have been eliminated, a striking consequence of spike-timing-dependent plasticity. From Gerstner W, Kempter R, van Hemmen JL, and Wagner H (1996) A neuronal learning rule for sub-millisecond temporal coding. Nature 383: 76–78.
A completely analogous argument to the one for the barn owl auditory system may well hold for hippocampal place cells too. There is, however, a neat difference. The typical time constant in the auditory system is 0.1 ms; on the other hand, it is 10 ms in the hippocampus, that is, roughly two orders of magnitude more. We may therefore expect that the temporal width of the learning window W depends on the physiological structures it lives in. That is, the width of W in the hippocampus is, indeed, 50–100 ms.
Conclusion From a theoretical point of view, STDP is extremely appealing. Depending on the arrival time of a presynaptic spike at a synapse on the postsynaptic neuron, the synapse is strengthened or weakened depending on the firing time of the postsynaptic neuron. If the postsynaptic neuron fires at t ¼ 0 and the synapse is excitatory, then it is strengthened if it receives a presynaptic spike at a time t < 0 and thus tells the
274 Spike-Timing-Dependent Plasticity Models
neuron to fire before the latter actually fires. That is, it is doing its job properly, and thus its synaptic strength increases. If, on the other hand, the presynaptic spike arrives at t > 0, then it is too late because the postsynaptic neuron has already fired. “Those that come too late shall be punished,” and, hence, the synaptic strength decreases. All this is taken care of by the function W, the learning window in Figure 4. Clearly, if the arrival time of the presynaptic spike and the firing time of the postsynaptic neuron are too far apart, then nothing happens. What we have analyzed in this article is STDP as a stand-alone phenomenon, that is, in its full simplicity. Neurobiological reality is both more complicated and more fascinating than this, in that STDP automatically normalizes synaptic strengths and gives rise to a whole field of applications such as map formation. Let us now briefly consider these phenomena in turn. The arguments presented so far hold for a single pair of spikes, a presynaptic one and a postsynaptic one. It has turned out experimentally that synaptic modification may also depend on the spiking pattern, that is, on the interspike intervals within each neuron. In addition, the precise position of a synapse on the dendritic tree may play a role as well. Furthermore, STDP gives rise to a normalization of the synaptic efficacies so that they do not diverge as time proceeds. Finally, STDP allows for a comprehensive explanation of neuronal map formation, one of the most intricate, yet manageable problems in today’s neuroscience. As we have seen, a map is a neuronal representation of the outside sensory world. Although STDP itself is local, map formation can be explained in detail by means of STDP in conjunction with more globally operating mechanisms. In short, because the very same neuronal wetware with the very same delays is used both for learning and for replay, a reproduction (replay) of what is to be learned is possible. Because STDP is local, it cannot handle global correlations. In spite of that, STDP is a very potent mechanism for storing time-resolved spike data at a scale finer than a millisecond and in this way can explain a huge diversity of neuronal phenomena. As such, it exhibits the mathematical universality to explain neurobiological diversity. See also: Developmental Synaptic Plasticity: LTP, LTD, and Synapse Formation and Elimination; Hebbian Plasticity; Long-Term Potentiation and Long-Term Depression in Experience-Dependent Plasticity; SpikeTiming Dependent Plasticity (STDP); Synaptic Plasticity and Place Cell Formation; Synaptic Plasticity: Short-Term Mechanisms.
Further Reading Abbott LF and Regehr WG (2004) Synaptic computation. Nature 431: 796–803. Bi G and Poo M (2001) Synaptic modification by correlated activity: Hebb’s postulate revisited. Annual Review of Neuroscience 24: 139–166. Caillard O, Ben-Ari Y, and Gaiarsa J-L (1999) Mechanisms of induction and expression of long-term depression at GABAergic synapses in the neonatal rat hippocampus. Journal of Neuroscience 19: 7568–7577. Froemke RC, Poo M, and Dan Y (2005) Spike-timing-dependent synaptic plasticity depends on dendritic location. Nature 434: 221–225. Gerstner W and Abbott LF (1997) Learning navigational maps through potentiation and modulation of hippocampal place cells. Journal of Comparative Neuroscience 4: 79–94. Gerstner W, Kempter R, van Hemmen JL, and Wagner H (1996) A neuronal learning rule for sub-millisecond temporal coding. Nature 383: 76–78. Hebb DO (1949) The organization of behavior: A neuropsychological theory. Wiley: New York. Herz AVM, Sulzer B, Ku¨hn R, and van Hemmen JL (1989) Hebbian learning reconsidered: Representation of static and dynamic objects in associative neural nets. Biological Cybernetics 60: 457–467. Kempter R, Gerstner W, and van Hemmen JL (1999) Hebbian learning and spiking neurons. Physical Review E 59: 4498–4514. Kempter R, Gerstner W, and van Hemmen JL (2001) Intrinsic stabilization of output rates by spike-based Hebbian learning. Neural Computation 13: 2709–2741. Kempter R, Leibold C, Wagner H, and van Hemmen JL (2001) Formation of temporal feature maps by axonal propagation of synaptic learning. Proceedings of the National Academy of Sciences of the United States of America 98: 4166–4171. Koch C (1999) Biophysics of Computation. New York: Oxford University Press. Linden DJ (1999) The return of the spike: Postsynaptic action potentials and the induction of LTP and LTD. Neuron 22: 661–666. Lisman J and Spruston N (2005) Postsynaptic depolarization requirements for LTP and LTD: A critique of spike timing dependent plasticity. Nature Neuroscience 8: 839–841. Morrison A, Diesmann M, and Gerstner W (2008) Phenomenological models of synaptic plasticity based on spike timing. Biological Cybernetics 98: 459–478. Rubin JE, Gerkin RC, Bi G, and Chow CC (2005) Calcium time course as a signal for spike-timing-dependent plasticity. Journal of Neurophysiology 93: 2600–2613. Shon AP, Rao RPN, and Sejnowski TJ (2004) Motion detection and prediction through spike-timing dependent plasticity. Network: Computation in Neural Systems 15: 179–198. Song S, Miller KD, and Abbott LF (2000) Competitive Hebbian learning through spike-timing-dependent synaptic plasticity. Nature Neuroscience 3: 919–926. Stuart G, Spruston N, Sakmann B, and Ha¨usser M (1997) Action potential initiation and backpropagation in neurons of the mammalian CNS. Trends in Neuroscience 20: 125–131. van Hemmen JL (2001) Theory of synaptic plasticity. In: Moss F and Gielen S (eds.) Handbook of Biophysics, vol. 4, pp. 771–823. Amsterdam: Elsevier. van Hemmen JL, Gerstner W, Herz AVM, Ku¨hn R, and Vaas M (1990) Encoding and decoding of patterns which are correlated in space and time. In: Dorffner G (ed.) Konnektionismus in
Spike-Timing-Dependent Plasticity Models Artificial Intelligence and Kognitionsforschung, pp. 153–162. Berlin: Springer. Waters J, Schaefer A, and Sakmann B (2005) Backpropagating action potentials in neurons: Measurement, mechanisms and potential functions. Progress in Biophysics & Molecular Biology 87: 145–170.
275
Wenisch OG, Noll J, and van Hemmen JL (2005) Spontaneously emerging direction selectivity maps in visual cortex through STDP. Biological Cybernetics 93: 239–247. Zhang LI, Tao HW, Holt CE, Harris WA, and Poo M (1998) A critical window for cooperation and competition among developing retinotectal synapses. Nature 395: 37–44.