- Email: [email protected]
Synchronization properties of pulse-coupled resonate-and-fire neuron circuits and their application
International Congress Series 1301 (2007) 148 – 151
www.ics-elsevier.com
Synchronization properties of pulse-coupled resonate-and-fire neuron circuits and their application Kazuki Nakada a,⁎, Tetsuya Asai b , Hatsuo Hayashi a a
Kyushu Institute of Technology, Kitakyushu 808-0196, Japan b Hokkaido University, Sapporo 060-0814, Japan
Abstract. We address the synchronization properties of pulse-coupled resonate-and-fire neuron (RFN) circuits from theoretical and engineering points of view. In particular, we focus on the effects of noise on the synchronization properties in two globally pulse-coupled RFN circuits in view of neuromorphic information processing. The results indicate the possible contributions of the RFN circuits for constructing the large-scale integration of silicon spiking neural networks. © 2007 Published by Elsevier B.V. Keywords: Silicon spiking neural network; Resonate-and-fire neuron (RFN); Synchronization phenomena
1. Introduction Recent advances in neuroscience suggest that spiking neurons in the cortical area are classified into two categories: integrators and resonators [1]. These neurons are different in their sensitivity to the timing of stimulus, and therefore show different synchronization properties in pulse-coupled systems. It is interesting that the distribution of such neurons is layer-specific and depends on whether they are excitatory or inhibitory neurons from a point of view of information processing in the cortical area [2]. Functional networks of silicon spiking neurons have been progressing in the field of neuromorphic engineering, most of which are constructed from electronic analogs of integrators, such as the integrate-and-fire neuron (IFN) circuit. In spite of possible ⁎ Corresponding author. 2-4 Hibikino, Wakamatsu-ku, Kitakyshu, 808-0196, Japan. Tel./fax: +81 93 695 6095. E-mail address: [email protected] (K. Nakada). 0531-5131/ © 2007 Published by Elsevier B.V. doi:10.1016/j.ics.2006.12.020
K. Nakada et al. / International Congress Series 1301 (2007) 148–151
149
contributions, implementing an analog integrated circuit for resonators into the silicon spiking neural networks has rarely been considered. In this paper, we will first describe an analog integrated circuit for a resonate-and-fire neuron (RFN) model, which is the simplest resonator proposed by Izhikevich [1]. We will then describe curious synchronization phenomena in two pulse-coupled RFN circuits. We will further describe synchronization properties of a system of globally coupled RFN circuits with inhibitory coupling and its application for neuromorphic noise shaping. We will finally discuss the complementary roles of the RFN circuits in the IFN circuits in very large-scale integration (VLSI) of the silicon spiking neural networks. 2. Analog VLSI implementation of resonate-and-fire neuron model We have designed the RFN model as an analog integrated circuit [3]. The RFN model has second-order membrane dynamics, and thus it exhibits a damped subthreshold oscillation of membrane potential, resulting in coincidence detection, frequency preference, and post-inhibitory rebound [1]. We implemented such dynamic behavior into an analog integrated circuit using a standard CMOS technology. Fig. 1 shows the schematic of an RFN circuit that consists of a membrane circuit, a threshold-and-fire circuit, and excitatory and inhibitory synaptic circuits. The membrane circuit has the second-order membrane dynamics containing two state variables, Ui and Vi, and mimics the subthreshold oscillation of membrane potential of the RFN model. The circuit dynamics are described as follows: 2 dUi j Vi P ¼ −gðUi −Vrst Þ þ Iin þ IUi −Io exp C1 dt j þ 1 VT 2 dVi j Ui P ¼ Io exp C2 −IVi dt j þ 1 VT where the voltages Ui and Vi are state variables, C1 and C2 the capacitance, g the total of the conductance of transistors, M11 and M12, Vrst the reset voltage, κ the capacitive
Fig. 1. Schematic of the resonate-and-fire neuron (RFN) circuit. The circuit consists of a membrane circuit, a threshold-and-fire circuit, and excitatory and inhibitory synaptic circuits.
150
K. Nakada et al. / International Congress Series 1301 (2007) 148–151
coupling ratio from the gate to the channel, VT the thermal voltage, Io the pre-exponential P current, and Iin represents the summation of synaptic currents. The currents, IUi through P M8 and IVi through M9 are approximately described as follows: VDD−Ui P IUi ¼ aIUi 1 þ V E;p Vi P IVi ¼ bIVi 1 þ VE;n where IUi and IVi are bias currents, VDD the power-supply voltage, VE,n and VE,p the early voltages for an nMOS and pMOS FET, respectively, and α and β the fitting constants. The equilibrium point of the circuit, (Uo, Vo), can easily be obtained, and the stability of the point can also be analyzed by the Eigenvalues of the Jacobian matrix of the circuit. The membrane circuit exhibits damped subthreshold oscillation in response to an input when the equilibrium point is a focus. Inputs through synaptic circuits change the trajectory of the oscillation of the membrane circuit. If Vi exceeds a threshold voltage Vth, Ui is reset to the reset voltage Vrst, and then the threshold-and-fire circuit generates a spike, i.e., a pulse voltage Vpulse, which is fed into a delay-and-inverter circuit to shape the pulse voltage. Due to the second-order membrane dynamics, the RFN circuit has the sensitivity to the timing of inputs that it inherits from the RFN model. As a result, the circuit can act as a coincidence detector, and at the same time as a band-pass filter for a sequence of pulse inputs [3]. It is expected that pulse-coupled RFN circuits exhibit different behavior as compared with pulse-coupled IFN circuits due to the peculiar sensitivity to the timing of inputs of the RFN circuit. 3. Burst synchronization in two pulse-coupled RFN circuits We have investigated synchronization phenomena in two pulse-coupled RFN circuits [4]. So far the existence and stability of anti-phase synchronization states in a system of two pulse-coupled RFN models have been reported [5]. However, the influence of the afterspike reset on the synchronization states has not been considered. We found burst synchronization in out of phase and bifurcation phenomena depending on both the reset value and coupling strength in two pulse-coupled RFN circuits [4]. We theoretically analyzed the stability of such burst synchronization states in detail [6]. Focusing on the symmetry in the system, we used the firing time differences between two RFN models and constructed from 1D return maps corresponding to each synchronization state. Since the firing time differences should be equal on the synchronization states, their stability can be analyzed by calculating the gradient at a fixed point of the return map. Furthermore, we investigated the global stability of the system by using the firing time difference maps (FTDMs), each of which consists of the 1D return maps according to the domain of definition of the maps. The results indicate that the burst synchronization states were robust against noise and insensitive to the influence of device deviation, and pulse duration may enhance the stability of the burst synchronization [6].
K. Nakada et al. / International Congress Series 1301 (2007) 148–151
151
4. Neuromorphic noise shaping in globally coupled RFN circuits Biological sensory systems may utilize noise shaping, which is a circuit principle originally established for improving the dynamic range (DR) and the signal-to-noise ratio (SNR) in electronic AD converters, as well as in population rate coding for noisy sensory signals [7]. Mar et al. have shown that a system of coupled IFN models with global inhibition can perform noise shaping in a population rate coding, hereafter referred to as neuromorphic noise shaping [7]. Such a network has previously been implemented as an analog subthreshold CMOS circuit for ultralow-power AD converters [8]. The performance of the neuromorphic noise shaping depends on the correlation between the elements of the network, and the correlation properties of globally inhibitory-coupled RFN models are superior to that of coupled IFN models [9]. Based on this, we verified the performance of the noise shaping in a system of globally inhibitory-coupled RFN circuits with SPICE simulations. We assumed the additive Gaussian white noise on the threshold voltage to be dynamic noise. The results showed that the correlation between the inter-spike interval (ISI) of the system increased with the number of the elements, and the distribution of ISI approaches the Gaussian [10]. Furthermore, the noise spectrum of ISI of the system was reduced under mean firing frequency of a population of the coupled RFN circuits. Such properties are sufficient for neuromorphic noise shaping. 5. Summary Synchronization properties of the pulse-coupled RFN circuits we have considered here are very different from those of IFN circuits studied in previous works. Such differences may contribute to playing complementary roles in information processing in neuromorphic VLSI systems. Acknowledgements The author would like to thank Dr. Keiji Miura and Prof. Masato Okada for their helpful comments and suggestions. References [1] E.M. Izhikevich, Dynamical Systems in Neuroscience: The Geometry of Excitability and Bursting, The MIT press, 2006. [2] T. Tateno, H.P. Robinson, Rate coding and spike-time variability in cortical neurons with two types of threshold dynamics, J. Neurophysiol. 95 (2006) 2650–2663. [3] K. Nakada, T. Asai, H. Hayashi, Proc. Int. Symp. on Nonlinear Theory and its Applications, Bruges, Belgium, 2005, pp. 82–85. [4] K. Nakada, T. Asai, H. Hayashi, Proc. IFIP World Computer Congress, Santiago, Chile, 2006. [5] K. Miura, M. Okada, Phys. Rev., E Stat. Phys. Plasmas Fluids Relat. Interdiscip. Topics 70 (2004) 021914. [6] K. Nakada, K. Miura, and H. Hayashi, submitted for publication. [7] D.J. Mar, et al., Neurobiology 96 (18) (1999) 10450–10455. [8] A. Utagawa, et al., Proc. Int. Conf. on Cognitive and Neural Systems, Boston, USA, 2006, p. 53. [9] K. Miura, M. Okada, Prog. Theor. Phys., Suppl. 161 (2006) 255–259. [10] K. Nakada, K. Miura, M. Okada and H. Hayashi, submitted for publication.
www.ics-elsevier.com
Synchronization properties of pulse-coupled resonate-and-fire neuron circuits and their application Kazuki Nakada a,⁎, Tetsuya Asai b , Hatsuo Hayashi a a
Kyushu Institute of Technology, Kitakyushu 808-0196, Japan b Hokkaido University, Sapporo 060-0814, Japan
Abstract. We address the synchronization properties of pulse-coupled resonate-and-fire neuron (RFN) circuits from theoretical and engineering points of view. In particular, we focus on the effects of noise on the synchronization properties in two globally pulse-coupled RFN circuits in view of neuromorphic information processing. The results indicate the possible contributions of the RFN circuits for constructing the large-scale integration of silicon spiking neural networks. © 2007 Published by Elsevier B.V. Keywords: Silicon spiking neural network; Resonate-and-fire neuron (RFN); Synchronization phenomena
1. Introduction Recent advances in neuroscience suggest that spiking neurons in the cortical area are classified into two categories: integrators and resonators [1]. These neurons are different in their sensitivity to the timing of stimulus, and therefore show different synchronization properties in pulse-coupled systems. It is interesting that the distribution of such neurons is layer-specific and depends on whether they are excitatory or inhibitory neurons from a point of view of information processing in the cortical area [2]. Functional networks of silicon spiking neurons have been progressing in the field of neuromorphic engineering, most of which are constructed from electronic analogs of integrators, such as the integrate-and-fire neuron (IFN) circuit. In spite of possible ⁎ Corresponding author. 2-4 Hibikino, Wakamatsu-ku, Kitakyshu, 808-0196, Japan. Tel./fax: +81 93 695 6095. E-mail address: [email protected] (K. Nakada). 0531-5131/ © 2007 Published by Elsevier B.V. doi:10.1016/j.ics.2006.12.020
K. Nakada et al. / International Congress Series 1301 (2007) 148–151
149
contributions, implementing an analog integrated circuit for resonators into the silicon spiking neural networks has rarely been considered. In this paper, we will first describe an analog integrated circuit for a resonate-and-fire neuron (RFN) model, which is the simplest resonator proposed by Izhikevich [1]. We will then describe curious synchronization phenomena in two pulse-coupled RFN circuits. We will further describe synchronization properties of a system of globally coupled RFN circuits with inhibitory coupling and its application for neuromorphic noise shaping. We will finally discuss the complementary roles of the RFN circuits in the IFN circuits in very large-scale integration (VLSI) of the silicon spiking neural networks. 2. Analog VLSI implementation of resonate-and-fire neuron model We have designed the RFN model as an analog integrated circuit [3]. The RFN model has second-order membrane dynamics, and thus it exhibits a damped subthreshold oscillation of membrane potential, resulting in coincidence detection, frequency preference, and post-inhibitory rebound [1]. We implemented such dynamic behavior into an analog integrated circuit using a standard CMOS technology. Fig. 1 shows the schematic of an RFN circuit that consists of a membrane circuit, a threshold-and-fire circuit, and excitatory and inhibitory synaptic circuits. The membrane circuit has the second-order membrane dynamics containing two state variables, Ui and Vi, and mimics the subthreshold oscillation of membrane potential of the RFN model. The circuit dynamics are described as follows: 2 dUi j Vi P ¼ −gðUi −Vrst Þ þ Iin þ IUi −Io exp C1 dt j þ 1 VT 2 dVi j Ui P ¼ Io exp C2 −IVi dt j þ 1 VT where the voltages Ui and Vi are state variables, C1 and C2 the capacitance, g the total of the conductance of transistors, M11 and M12, Vrst the reset voltage, κ the capacitive
Fig. 1. Schematic of the resonate-and-fire neuron (RFN) circuit. The circuit consists of a membrane circuit, a threshold-and-fire circuit, and excitatory and inhibitory synaptic circuits.
150
K. Nakada et al. / International Congress Series 1301 (2007) 148–151
coupling ratio from the gate to the channel, VT the thermal voltage, Io the pre-exponential P current, and Iin represents the summation of synaptic currents. The currents, IUi through P M8 and IVi through M9 are approximately described as follows: VDD−Ui P IUi ¼ aIUi 1 þ V E;p Vi P IVi ¼ bIVi 1 þ VE;n where IUi and IVi are bias currents, VDD the power-supply voltage, VE,n and VE,p the early voltages for an nMOS and pMOS FET, respectively, and α and β the fitting constants. The equilibrium point of the circuit, (Uo, Vo), can easily be obtained, and the stability of the point can also be analyzed by the Eigenvalues of the Jacobian matrix of the circuit. The membrane circuit exhibits damped subthreshold oscillation in response to an input when the equilibrium point is a focus. Inputs through synaptic circuits change the trajectory of the oscillation of the membrane circuit. If Vi exceeds a threshold voltage Vth, Ui is reset to the reset voltage Vrst, and then the threshold-and-fire circuit generates a spike, i.e., a pulse voltage Vpulse, which is fed into a delay-and-inverter circuit to shape the pulse voltage. Due to the second-order membrane dynamics, the RFN circuit has the sensitivity to the timing of inputs that it inherits from the RFN model. As a result, the circuit can act as a coincidence detector, and at the same time as a band-pass filter for a sequence of pulse inputs [3]. It is expected that pulse-coupled RFN circuits exhibit different behavior as compared with pulse-coupled IFN circuits due to the peculiar sensitivity to the timing of inputs of the RFN circuit. 3. Burst synchronization in two pulse-coupled RFN circuits We have investigated synchronization phenomena in two pulse-coupled RFN circuits [4]. So far the existence and stability of anti-phase synchronization states in a system of two pulse-coupled RFN models have been reported [5]. However, the influence of the afterspike reset on the synchronization states has not been considered. We found burst synchronization in out of phase and bifurcation phenomena depending on both the reset value and coupling strength in two pulse-coupled RFN circuits [4]. We theoretically analyzed the stability of such burst synchronization states in detail [6]. Focusing on the symmetry in the system, we used the firing time differences between two RFN models and constructed from 1D return maps corresponding to each synchronization state. Since the firing time differences should be equal on the synchronization states, their stability can be analyzed by calculating the gradient at a fixed point of the return map. Furthermore, we investigated the global stability of the system by using the firing time difference maps (FTDMs), each of which consists of the 1D return maps according to the domain of definition of the maps. The results indicate that the burst synchronization states were robust against noise and insensitive to the influence of device deviation, and pulse duration may enhance the stability of the burst synchronization [6].
K. Nakada et al. / International Congress Series 1301 (2007) 148–151
151
4. Neuromorphic noise shaping in globally coupled RFN circuits Biological sensory systems may utilize noise shaping, which is a circuit principle originally established for improving the dynamic range (DR) and the signal-to-noise ratio (SNR) in electronic AD converters, as well as in population rate coding for noisy sensory signals [7]. Mar et al. have shown that a system of coupled IFN models with global inhibition can perform noise shaping in a population rate coding, hereafter referred to as neuromorphic noise shaping [7]. Such a network has previously been implemented as an analog subthreshold CMOS circuit for ultralow-power AD converters [8]. The performance of the neuromorphic noise shaping depends on the correlation between the elements of the network, and the correlation properties of globally inhibitory-coupled RFN models are superior to that of coupled IFN models [9]. Based on this, we verified the performance of the noise shaping in a system of globally inhibitory-coupled RFN circuits with SPICE simulations. We assumed the additive Gaussian white noise on the threshold voltage to be dynamic noise. The results showed that the correlation between the inter-spike interval (ISI) of the system increased with the number of the elements, and the distribution of ISI approaches the Gaussian [10]. Furthermore, the noise spectrum of ISI of the system was reduced under mean firing frequency of a population of the coupled RFN circuits. Such properties are sufficient for neuromorphic noise shaping. 5. Summary Synchronization properties of the pulse-coupled RFN circuits we have considered here are very different from those of IFN circuits studied in previous works. Such differences may contribute to playing complementary roles in information processing in neuromorphic VLSI systems. Acknowledgements The author would like to thank Dr. Keiji Miura and Prof. Masato Okada for their helpful comments and suggestions. References [1] E.M. Izhikevich, Dynamical Systems in Neuroscience: The Geometry of Excitability and Bursting, The MIT press, 2006. [2] T. Tateno, H.P. Robinson, Rate coding and spike-time variability in cortical neurons with two types of threshold dynamics, J. Neurophysiol. 95 (2006) 2650–2663. [3] K. Nakada, T. Asai, H. Hayashi, Proc. Int. Symp. on Nonlinear Theory and its Applications, Bruges, Belgium, 2005, pp. 82–85. [4] K. Nakada, T. Asai, H. Hayashi, Proc. IFIP World Computer Congress, Santiago, Chile, 2006. [5] K. Miura, M. Okada, Phys. Rev., E Stat. Phys. Plasmas Fluids Relat. Interdiscip. Topics 70 (2004) 021914. [6] K. Nakada, K. Miura, and H. Hayashi, submitted for publication. [7] D.J. Mar, et al., Neurobiology 96 (18) (1999) 10450–10455. [8] A. Utagawa, et al., Proc. Int. Conf. on Cognitive and Neural Systems, Boston, USA, 2006, p. 53. [9] K. Miura, M. Okada, Prog. Theor. Phys., Suppl. 161 (2006) 255–259. [10] K. Nakada, K. Miura, M. Okada and H. Hayashi, submitted for publication.