- Email: [email protected]
Analog perceptron circuit with DAC-based multiplier
INTEGRATION, the VLSI journal xxx (xxxx) xxx–xxx
Contents lists available at ScienceDirect
INTEGRATION, the VLSI journal journal homepage: www.elsevier.com/locate/vlsi
Analog perceptron circuit with DAC-based multiplier Yoritaka Ishiguchi, Daishi Isogai, Takuma Osawa, Shigetoshi Nakatake∗ The University of Kitakyushu, 1-1 Hibikino, Wakamatsu, Kitakyushu, Fukuoka, 808-0135, Japan
A B S T R A C T
This paper presents a perceptron circuit which can be implemented into a sensor analog front-end consistent with neural network-based machine learning. We introduce a DAC-based multiplier in the perceptron circuit, where the DAC is used as a programmable resistor. Compared with a traditional transconductor-based multiplier, the precision of our multiplier is formulated only by the digital codes, and it has a wide input range and a good temperature dependency. The simulation result demonstrates the DAC-based multiplier amplifies smoothly analog signal by the digital codes. Furthermore, we extend our perceptron model so as to deal with time series inputs and show a promising result by simulation. As one of an important future works, focusing on periodic signal inputs, we discuss a general architecture of perceptron circuit inspired by Fourier series.
1. Introduction In recent years, a machine learning has remarkably developed, and many cases of its social implementation have been reported. A neural network is one of fundamental architectures for the machine learning, and the VLSI (very large scale integrated circuit) implementation has been researched traditionally [1]. Recent trends of machine learning implementation are configuration on FPGAs (field programmable gate arrays) and acceleration by GPUs (graphics processing unit) [2,3]. On the other hand, a wireless sensor network is widely used to collect data in various area such as factories, farms, and cities. An advanced sensor node has a multi-input to compensate a sensing information by employing different sensor signals. For example [7], introduces an 8-channel EEG(Electroencephalogram)/electrode-tissue impedance acquisition system, where nine active electrodes and one back-end analog signal processor work being closely correlated. In Ref. [8], a pressure sensor and a microphone are combined to visualize the blood pressure measurement. In the compensation and combination of input signals, it is known that a machine learning technique is useful. Hence, a sensor node is requested to incorporate an architecture for neural network computing. Unlike GPU or FPGA-based neural network architectures, however, a sensor node must be low power and with a small hardware even sacrificing a large computation. Therefore, we attempt to embed a learning circuit mechanism into an analog front-end of a sensor node. This paper focuses on a perceptron which is an essential function of neural networks. Aiming a small embedding, we realize it by analog circuits introducing a DAC(digital-
∗
analog-converter)-based multiplier in the perceptron circuit. The DACbased multiplier is to amplify an analog input signal by a preset digital code. Compared with a traditional analog multiplier of transconductorbased, the precision of our multiplier is formulated only by the digital codes, and it has a wide input range and a good temperature dependency. The simulation result demonstrates a perceptron circuit with the DAC-based multiplier smoothly combines multiple analog signals amplifying by the digital codes. Furthermore, we introduce a phase shifter to our perceptron model to divide an input signal into time series inputs with the same phase delay. The simulation results convince us that the extended perceptron circuit is applicable to prediction algorithms such as a recurrent neural network and chaos pass filter. In addition, as one of future works, we discuss a general architecture of a perceptron circuit to represent any signal in the output based on Fourier series. The rest of the paper is organized as follows. Section 2 describes a perceptron circuit used in neural network systems. Section 3 introduces a DAC-based multiplier and describes the simulation results. Section 4 describes an idea and examples of time series input with phase shifters. In Section 5, we discuss a general architecture of our perceptron circuit focusing on periodic signals. Section 6 summarizes this work. 2. Perceptron circuit As described in Ref. [1], a perceptron is an essential function used in neural network systems. A typical model of the perceptron is illustrated in Fig. 1. The inputs are f1(t) …, fn(t), each of which is multiplied by a
Corresponding author. E-mail address: [email protected] (S. Nakatake).
https://doi.org/10.1016/j.vlsi.2018.05.010 Received 1 December 2017; Received in revised form 26 April 2018; Accepted 24 May 2018 0167-9260/ © 2018 Elsevier B.V. All rights reserved.
Please cite this article as: Ishiguchi, Y., INTEGRATION, the VLSI journal (2018), https://doi.org/10.1016/j.vlsi.2018.05.010
INTEGRATION, the VLSI journal xxx (xxxx) xxx–xxx
Y. Ishiguchi et al.
Fig. 1. A typical perceptron function.
Fig. 4. A programmable gain amplifier pattern (1).
determined by a ratio of the variable resistor and R2. However, this implementation also has a defect in the temperature dependency and the input range, which are described below.
3. DAC-based multiplier 3.1. DAC-based programmable gain amplifiers
weight w1, …, wn, and all weighted inputs are summed at the output. In general, realizing this perceptron function in an analog front-end of a sensor node, time division multiplexed input is used. Here, as shown in Fig. 2, each input of f1(t) …, fn(t) is controlled by switches S1 …, Sn exclusively, and a selected input is transmitted to the ADC. Plus, in an application of biological sensor systems, an input signal is so tiny, an input analog buffer is placed at each input. However, a VLSI implementation yields a variation of amplitudes among the input analog buffers due to a process-induced variation, and it degrades the precision of sensing values. To avoid degrading the precision of analog values, an analog multipliers and adder are used as shown in Fig. 3. In this figure, Vin1 and Vin2 are amplified by the weight signals, w1 and w2, respectively. The input is connected to the variable resistor which is realized by a transconductor of PMOS or NMOS transistor [6]. The weight value is
We adopt two types of programmable gain amplifiers introduced in Ref. [4] to realize a multiplier overcoming defects described above. The schematics of the programmable gain amplifiers are illustrated in Figs. 4 and 5 (called pattern (1) and (2), respectively). Both patterns basically configure the negative feedback of the opamp, but he resistance in the feedback is replaced by the DAC. In detail, the DAC is inserted at the input of the negative feedback circuit in pattern (1), while it is at the loop of the feedback in pattern (2). Changing of the DAC output current looks as if the resistance value were to change. We design these patterns and verify the function by the simulation. All circuits are designed with a model of 0.6 μm/5 V manufacturing process. The Vtn and Vtp of NMOS and PMOS are 0.7 V and 0.9 V, respectively. Plus, βn and βp are 1.0 × 10−4 A/V2, 0.4 × 10−4 A/V2. A typical two-stage opamp used in this paper is shown in Fig. 6, and the specification of the opamp is shown in Table 1. As well, the schematic of the DAC is illustrated in Fig. 7. In the simulation, we observe the changing of the output amplitude for pattern (1) and (2) when changing the DAC input codes from zero to the fullscale, and the results are shown in Figs. 8 and 9, respectively. In Fig. 8, we can observe the output amplified by N ⋅ Rb1. As well, the
Fig. 3. Analog multiplier and adder for perceptron.
Fig. 5. A programmable gain amplifier pattern (2).
Fig. 2. An example of analog front-end with time division multiplexed input.
2
INTEGRATION, the VLSI journal xxx (xxxx) xxx–xxx
Y. Ishiguchi et al.
Fig. 8. Changing of the output amplitude for each input code of DAC: circuit pattern (1).
Fig. 6. A schematic of OPAMP.
Table 1 A specification of our opamp. DC gain Cut-off freq. unity gain freq. phase margin DC offset
101.82 2.57 24.48 62.46 5.0
[dB] [kHz] [MHz] [deg.] [mV]
Fig. 9. Changing of the output amplitude for each input code of DAC: circuit pattern (2).
Fig. 10. The multiplier with a combination circuit pattern (1) and (2).
1.0 KHz frequency to the input Vref. Vout1 of the circuit pattern (1) shown in Fig. 4 is calculated as;
Vout1 = −
X ⋅Rb1 Vin, 2 N ⋅R d
where the decimal input code of the DAC is X, the resolution is N, the input resistance is Rd, and the DAC internal equivalent resistance from Vref to Iout is Rb1. Exchanging of the DAC and Rb1 in the circuit pattern (1), we can calculate Vout2 of the circuit pattern (2) shown in Fig. 5 as;
Fig. 7. Our current-type DAC. The circuit includes 4-to-16 decoder with the input A0 … A3 and the output D1 … D15, and it controls the switch S1⋯S15. In (b), either of Vref1 or Vref2 is used as an input, and the other is connected to VDD or GND.
Vout 2 = −
output shown in Fig. 9 is amplified by 1/N ⋅ Rb2. In addition, we realize a multiplier combining the circuit pattern (1) and (2) as illustrated in Fig. 10. The output waveforms are shown in Fig. 11 when we give a sine signal with the amplitudes of 5 mV and
2 N ⋅R d Vin X ⋅Rb 2
As a result, Vout of our multiplier circuit shown in Fig. 10,
Vout = −
X1 Vin, X2
where X1 and X2 are the decimal input codes of DAC1 and DAC2, 3
INTEGRATION, the VLSI journal xxx (xxxx) xxx–xxx
Y. Ishiguchi et al.
Fig. 11. The output waveform of the multiplier with a combination circuit pattern (1) and (2): input signal, 10× and 100× output signals.
respectively. Note that we can remove the influence of the DAC internal equivalent resistance from the gain. X1 According to Vout = − X 2 Vin , the simulation results show the output amplitude of 1×, 10×, and 100× are 4.98 mV, 49.40 mV, and 500.66 mV for the input signal of 5 mV, respectively. Notice that this multiplier has one analog input and the other of digital. The analog input is amplified by only digital codes, so that the precision is not affected by a process-induced variation. 3.2. Comparison with traditional analog multiplier Furthermore, we investigate the temperature dependency and input range compared with traditional analog multiplier shown in Fig. 3. Figs. 12 and 13 show the simulation results with respect to the temperature dependency and the input range, respectively. Our DAC-based multiplier demonstrates better performance in the both results. Fig. 13. Input range: (a) digital code, (b) gate voltage of PMOS, (c) gate voltage of NMOS.
3.3. Simulation of multiple inputs We design a perceptron circuit composed of DAC-based multipliers. The schematic is shown in Fig. 14, where three inputs are connected to DAC1, 2 and 3. The gains of multipliers are determined by ratios of X1/
X4, X2/X4 and X3/X4, respectively, and all multiplied values are summed at the output. To clarify a function of this circuit, we apply the simulation with the setting as; Vin1, Vin2 and Vin3 are sin signals of 5 mV, 10 mV and 15 mV amplitude at 1 kHz frequency. DAC1, DAC2 and DAC3 are set as the gains of the multipliers to be 3×, 2× and 1×, respectively. The simulation results are shown in Fig. 15(a). We can correctly observe 65 mV amplitude at the output signal. Next, we change the frequency of each input as; Vin1, Vin2 and Vin3 are sin signals of 10 kHz, 5 kHz and 1 kHz frequencies, respectively. The other settings as same as in Fig. 15(a). Fig. 15(b) demonstrates the simulation results. Our perceptron circuit can output a complicated waveform, so that it has a potential for complicated machine learning.
4. Time series inputs by phase shifting Fig. 12. Temperature dependency. The x-axis corresponds to the temperature (deg.), and the y-axis does to voltage (V).
We attempt to extend our perceptron model which can deal with 4
INTEGRATION, the VLSI journal xxx (xxxx) xxx–xxx
Y. Ishiguchi et al.
Fig. 16. Phase shift circuit.
4.1. Phase shift circuit We introduce a typical phase shift circuit to generate a delay between input signals, which is illustrated in Fig. 16. In this circuit, a phase delay between the input and the output is determined by the resistance values R1, R2, R3 and the capacitance value C1. Plus, these values change depending on the frequency of the input signal and the phase delay. We describe example values of these components to generate 90° phase delay at input difference frequencies in Table 2. In fact, 90° phase delay requests difference component values at difference frequencies. On the other hand, the same component values generate the same absolute phase delay. We notice the delay is formulated as;
Fig. 14. Our perceptron circuit for three inputs.
d = tan−1 ( −ωCR) − tan−1 (ωCR), where d is an absolute phase delay between the input and output signals.
4.2. Example of perceptron circuit with phase shifting We introduce phase shift circuits to our perceptron model to deal with time series inputs as illustrated in Fig. 17. For the simplicity, in this figure, we assume the number of inputs is 1, and the input is distributed passing phase shifters. As a result, we realize time series inputs to our perceptron circuit, where three input signals corresponding to f (t), f(t + d), and f(t + 2d), respectively. Next, we demonstrate the simulation results. Applying different input frequencies to our perceptron circuit of time series, we observe the output signals with all weights (DAC code) are set to be ‘0001’. The results are shown in Figs. 18 and 19, respectively. Note that we provide different component settings for different input frequencies. In both results, we can observe that the input signal phases are shifted by 90°. Furthermore, we verify the behavior of the proposed time series circuit for a random input signal containing different frequencies spectrum, which is the same as used in Fig. 15(b) in Section 3. The result is shown in Fig. 20. As seen in the figure, the phase delay is evenly generated among signals because our phase shifter can control the absolute delay. These results convince us that our perceptron circuit with time series is applicable to prediction algorithms such as a recurrent neural network and chaos pass filter.
Fig. 15. Our perceptron circuit for three inputs.
Table 2 Peripheral component values for 90° phase delay.
time series inputs like a time series prediction using a recurrent neural network. For the simplicity, we formulate a model of time series as follows;
fout (t ) = w1⋅f (t ) + w2⋅f (t + d ) + w3⋅f (t + 2d ), where t and d are time and delay variables, respectively. 5
Input frequency
R1 = R2 = R3
C1
1 kHz 10 kHz
1.55 MΩ 155 kΩ
100 pF 100 pF
INTEGRATION, the VLSI journal xxx (xxxx) xxx–xxx
Y. Ishiguchi et al.
Fig. 20. Simulation result for a random input signal.
Fig. 17. An example of DAC-based perceptron circuit with phase shifters.
Fig. 18. Simulation result for 1 kHz input signal.
Fig. 21. Vin1 = sin x, Vin2 = cos x, Vout = sin x + cos x =
2 sin(x − π /4) .
For input signals sin x (Vin1) and cos x (Vin2), the addition of signals is illustrated as Fig. 21, where the output (Vout) is 2 sin(x − π /4) , so that the output signal is shifted by π/4. Furthermore, assuming the addition of signals, sin x and q ⋅ cos x, the output signals are represented as Fig. 23 when changing the value q. Note that the amplitude and phase are changing according to q. We can understand this phase shifting from the following formula of trigonometric function.
p⋅sin x + q⋅cos x =
p2 + q2 sin(x + α ),
where sin α = q/ p2 + q2 and cos α = p / p2 + q2 . For example, if
Fig. 19. Simulation result for 10 kHz input signal.
5. Periodic signal and phase shifting Inspired by Fourier series,
a0 + 2
∞
∑ (ak cos kx + bk sin kx ), k=1
we propose a novel phase shifting mechanism for a periodic signal input based on our perceptron circuit.
Fig. 22. V in1 = sin x, V in2 = sin(π − x) = cos x. 6
INTEGRATION, the VLSI journal xxx (xxxx) xxx–xxx
Y. Ishiguchi et al.
Fig. 23. Phase shifting of an adding signal of sin x and q ⋅ cos x when changing q.
p = q = 1, then cos α = 2 /2 , and α = π/4. In other words, if we want to shift the signal by π/3, cos π /3 = 1/2 = p / p2 + q2 . As a result, we find p = 1 and q = 3 . Thus, we propose a phase shifting circuit shown in Fig. 22. In the figure, blocks ‘inv’ and ‘π/2’ are an inversion and a phase shift circuit, respectively. The former is the same as in Fig. 16 and the latter is easily realized by a negative feedback of an opamp. Applying the simulation to this circuit, the resultant waveforms are demonstrated in Fig. 24 (a) and (b). In the figure (a), p and q are set to 1 by DAC. On the other hand, in the figure (b), p and q are set to 1 and 4, respectively. As seen in these results, we can control the amplitude and the phase of the output signal by changing the DAC setting in our proposed circuit. Furthermore, based on Fourier series,
f (x ) ≃
a0 + 2
∞
∑ (ak cos kx + bk sin kx ), k=1
We provide a general architecture of a perceptron circuit to represent any signal in the output that be aware of Fourier series. The general form is illustrated in Fig. 25. However, we notice that we must control multiplying periods to realize this architecture. This is one of important our future works. 6. Conclusion This paper presents a perceptron circuit applicable to a sensor analog front-end. We adopt a DAC-based multiplier in the perceptron circuit, where the gain of the multiplier is determined by only digital codes. It contributes to the precision without degrading the processinduced variation. As well, compared with a traditional transconductorbased multiplier, it has a wide input range and a good temperature dependency. The simulation result demonstrates a perceptron circuit with the DAC-based multiplier can combine smoothly multiple analog signals amplifying by the digital codes. In addition, our perceptron circuit is extended to deal with time series inputs by using phase shifter, so that the circuit shows the possibility to be used for prediction
Fig. 24. Vout = p ⋅ sin x + q ⋅ cos x.
7
INTEGRATION, the VLSI journal xxx (xxxx) xxx–xxx
Y. Ishiguchi et al.
algorithms. Furthermore, we discuss the relation of periodic signal and phase shifting in our perceptron circuit. We provide a promising general architecture of perceptron circuit inspired by Fourier series. Appendix A. Supplementary data Supplementary data related to this article can be found at http://dx. doi.org/10.1016/j.vlsi.2018.05.010. References [1] S.M. Gowda, B.J. Sheu, J. Choi, Design and characterization of analog VLSI neural network modules, IEEE J. Solid State Circuit 28 (3) (1993) 301–313. [2] B.V. Essen, C. Macaraeg, M. Gokhale, R. Prenger, Accelerating a random forest classifier: multi-core, GP-GPU, or FPGA? IEEE 20th International Symposium on Field-programmable Custom Computing Machines, 2012, pp. 232–239. [3] Qi Yu, Chao Want, Xiang Ma, Xi Li, Xuehai Zhou, A deep learning prediction process accelerator based FPGA, IEEE/ACM 2015 International Symposium on Cluster, Cloud and Grid Computing, 2015, pp. 1159–1162. [4] John Wynne, Application Notes, AN-320A, ANALOG DEVICES. [6] Gunhee Han, Edgar Sanchez-Sinencio, CMOS transconductance multipliers: a tutorial, IEEE Trans. Circuits Syst. II: Analog Digit. Signal Process. 45 (12) (1998) 1550–1563. [7] J. Xu, S. Mitra, A. Matsumoto, S. Patki, C.V. Hoof, K.A.A. Makinwa, R.F. Yazicioglu, A wearable 8-channel active-electrode EEG/ETI acquisition system for body area networks, IEEE J. Solid State Circuit 49 (9) (2014) 2005–2016. [8] C.-L. Goh, S. Nakatake, A sensor-based data visualization system for training blood pressure measurement by auscultatory method, IEICE Trans. Info Syst. E99-D (4) (2016) 936–943.
Fig. 25. A perceptron circuit based on Fourier series.
8
Contents lists available at ScienceDirect
INTEGRATION, the VLSI journal journal homepage: www.elsevier.com/locate/vlsi
Analog perceptron circuit with DAC-based multiplier Yoritaka Ishiguchi, Daishi Isogai, Takuma Osawa, Shigetoshi Nakatake∗ The University of Kitakyushu, 1-1 Hibikino, Wakamatsu, Kitakyushu, Fukuoka, 808-0135, Japan
A B S T R A C T
This paper presents a perceptron circuit which can be implemented into a sensor analog front-end consistent with neural network-based machine learning. We introduce a DAC-based multiplier in the perceptron circuit, where the DAC is used as a programmable resistor. Compared with a traditional transconductor-based multiplier, the precision of our multiplier is formulated only by the digital codes, and it has a wide input range and a good temperature dependency. The simulation result demonstrates the DAC-based multiplier amplifies smoothly analog signal by the digital codes. Furthermore, we extend our perceptron model so as to deal with time series inputs and show a promising result by simulation. As one of an important future works, focusing on periodic signal inputs, we discuss a general architecture of perceptron circuit inspired by Fourier series.
1. Introduction In recent years, a machine learning has remarkably developed, and many cases of its social implementation have been reported. A neural network is one of fundamental architectures for the machine learning, and the VLSI (very large scale integrated circuit) implementation has been researched traditionally [1]. Recent trends of machine learning implementation are configuration on FPGAs (field programmable gate arrays) and acceleration by GPUs (graphics processing unit) [2,3]. On the other hand, a wireless sensor network is widely used to collect data in various area such as factories, farms, and cities. An advanced sensor node has a multi-input to compensate a sensing information by employing different sensor signals. For example [7], introduces an 8-channel EEG(Electroencephalogram)/electrode-tissue impedance acquisition system, where nine active electrodes and one back-end analog signal processor work being closely correlated. In Ref. [8], a pressure sensor and a microphone are combined to visualize the blood pressure measurement. In the compensation and combination of input signals, it is known that a machine learning technique is useful. Hence, a sensor node is requested to incorporate an architecture for neural network computing. Unlike GPU or FPGA-based neural network architectures, however, a sensor node must be low power and with a small hardware even sacrificing a large computation. Therefore, we attempt to embed a learning circuit mechanism into an analog front-end of a sensor node. This paper focuses on a perceptron which is an essential function of neural networks. Aiming a small embedding, we realize it by analog circuits introducing a DAC(digital-
∗
analog-converter)-based multiplier in the perceptron circuit. The DACbased multiplier is to amplify an analog input signal by a preset digital code. Compared with a traditional analog multiplier of transconductorbased, the precision of our multiplier is formulated only by the digital codes, and it has a wide input range and a good temperature dependency. The simulation result demonstrates a perceptron circuit with the DAC-based multiplier smoothly combines multiple analog signals amplifying by the digital codes. Furthermore, we introduce a phase shifter to our perceptron model to divide an input signal into time series inputs with the same phase delay. The simulation results convince us that the extended perceptron circuit is applicable to prediction algorithms such as a recurrent neural network and chaos pass filter. In addition, as one of future works, we discuss a general architecture of a perceptron circuit to represent any signal in the output based on Fourier series. The rest of the paper is organized as follows. Section 2 describes a perceptron circuit used in neural network systems. Section 3 introduces a DAC-based multiplier and describes the simulation results. Section 4 describes an idea and examples of time series input with phase shifters. In Section 5, we discuss a general architecture of our perceptron circuit focusing on periodic signals. Section 6 summarizes this work. 2. Perceptron circuit As described in Ref. [1], a perceptron is an essential function used in neural network systems. A typical model of the perceptron is illustrated in Fig. 1. The inputs are f1(t) …, fn(t), each of which is multiplied by a
Corresponding author. E-mail address: [email protected] (S. Nakatake).
https://doi.org/10.1016/j.vlsi.2018.05.010 Received 1 December 2017; Received in revised form 26 April 2018; Accepted 24 May 2018 0167-9260/ © 2018 Elsevier B.V. All rights reserved.
Please cite this article as: Ishiguchi, Y., INTEGRATION, the VLSI journal (2018), https://doi.org/10.1016/j.vlsi.2018.05.010
INTEGRATION, the VLSI journal xxx (xxxx) xxx–xxx
Y. Ishiguchi et al.
Fig. 1. A typical perceptron function.
Fig. 4. A programmable gain amplifier pattern (1).
determined by a ratio of the variable resistor and R2. However, this implementation also has a defect in the temperature dependency and the input range, which are described below.
3. DAC-based multiplier 3.1. DAC-based programmable gain amplifiers
weight w1, …, wn, and all weighted inputs are summed at the output. In general, realizing this perceptron function in an analog front-end of a sensor node, time division multiplexed input is used. Here, as shown in Fig. 2, each input of f1(t) …, fn(t) is controlled by switches S1 …, Sn exclusively, and a selected input is transmitted to the ADC. Plus, in an application of biological sensor systems, an input signal is so tiny, an input analog buffer is placed at each input. However, a VLSI implementation yields a variation of amplitudes among the input analog buffers due to a process-induced variation, and it degrades the precision of sensing values. To avoid degrading the precision of analog values, an analog multipliers and adder are used as shown in Fig. 3. In this figure, Vin1 and Vin2 are amplified by the weight signals, w1 and w2, respectively. The input is connected to the variable resistor which is realized by a transconductor of PMOS or NMOS transistor [6]. The weight value is
We adopt two types of programmable gain amplifiers introduced in Ref. [4] to realize a multiplier overcoming defects described above. The schematics of the programmable gain amplifiers are illustrated in Figs. 4 and 5 (called pattern (1) and (2), respectively). Both patterns basically configure the negative feedback of the opamp, but he resistance in the feedback is replaced by the DAC. In detail, the DAC is inserted at the input of the negative feedback circuit in pattern (1), while it is at the loop of the feedback in pattern (2). Changing of the DAC output current looks as if the resistance value were to change. We design these patterns and verify the function by the simulation. All circuits are designed with a model of 0.6 μm/5 V manufacturing process. The Vtn and Vtp of NMOS and PMOS are 0.7 V and 0.9 V, respectively. Plus, βn and βp are 1.0 × 10−4 A/V2, 0.4 × 10−4 A/V2. A typical two-stage opamp used in this paper is shown in Fig. 6, and the specification of the opamp is shown in Table 1. As well, the schematic of the DAC is illustrated in Fig. 7. In the simulation, we observe the changing of the output amplitude for pattern (1) and (2) when changing the DAC input codes from zero to the fullscale, and the results are shown in Figs. 8 and 9, respectively. In Fig. 8, we can observe the output amplified by N ⋅ Rb1. As well, the
Fig. 3. Analog multiplier and adder for perceptron.
Fig. 5. A programmable gain amplifier pattern (2).
Fig. 2. An example of analog front-end with time division multiplexed input.
2
INTEGRATION, the VLSI journal xxx (xxxx) xxx–xxx
Y. Ishiguchi et al.
Fig. 8. Changing of the output amplitude for each input code of DAC: circuit pattern (1).
Fig. 6. A schematic of OPAMP.
Table 1 A specification of our opamp. DC gain Cut-off freq. unity gain freq. phase margin DC offset
101.82 2.57 24.48 62.46 5.0
[dB] [kHz] [MHz] [deg.] [mV]
Fig. 9. Changing of the output amplitude for each input code of DAC: circuit pattern (2).
Fig. 10. The multiplier with a combination circuit pattern (1) and (2).
1.0 KHz frequency to the input Vref. Vout1 of the circuit pattern (1) shown in Fig. 4 is calculated as;
Vout1 = −
X ⋅Rb1 Vin, 2 N ⋅R d
where the decimal input code of the DAC is X, the resolution is N, the input resistance is Rd, and the DAC internal equivalent resistance from Vref to Iout is Rb1. Exchanging of the DAC and Rb1 in the circuit pattern (1), we can calculate Vout2 of the circuit pattern (2) shown in Fig. 5 as;
Fig. 7. Our current-type DAC. The circuit includes 4-to-16 decoder with the input A0 … A3 and the output D1 … D15, and it controls the switch S1⋯S15. In (b), either of Vref1 or Vref2 is used as an input, and the other is connected to VDD or GND.
Vout 2 = −
output shown in Fig. 9 is amplified by 1/N ⋅ Rb2. In addition, we realize a multiplier combining the circuit pattern (1) and (2) as illustrated in Fig. 10. The output waveforms are shown in Fig. 11 when we give a sine signal with the amplitudes of 5 mV and
2 N ⋅R d Vin X ⋅Rb 2
As a result, Vout of our multiplier circuit shown in Fig. 10,
Vout = −
X1 Vin, X2
where X1 and X2 are the decimal input codes of DAC1 and DAC2, 3
INTEGRATION, the VLSI journal xxx (xxxx) xxx–xxx
Y. Ishiguchi et al.
Fig. 11. The output waveform of the multiplier with a combination circuit pattern (1) and (2): input signal, 10× and 100× output signals.
respectively. Note that we can remove the influence of the DAC internal equivalent resistance from the gain. X1 According to Vout = − X 2 Vin , the simulation results show the output amplitude of 1×, 10×, and 100× are 4.98 mV, 49.40 mV, and 500.66 mV for the input signal of 5 mV, respectively. Notice that this multiplier has one analog input and the other of digital. The analog input is amplified by only digital codes, so that the precision is not affected by a process-induced variation. 3.2. Comparison with traditional analog multiplier Furthermore, we investigate the temperature dependency and input range compared with traditional analog multiplier shown in Fig. 3. Figs. 12 and 13 show the simulation results with respect to the temperature dependency and the input range, respectively. Our DAC-based multiplier demonstrates better performance in the both results. Fig. 13. Input range: (a) digital code, (b) gate voltage of PMOS, (c) gate voltage of NMOS.
3.3. Simulation of multiple inputs We design a perceptron circuit composed of DAC-based multipliers. The schematic is shown in Fig. 14, where three inputs are connected to DAC1, 2 and 3. The gains of multipliers are determined by ratios of X1/
X4, X2/X4 and X3/X4, respectively, and all multiplied values are summed at the output. To clarify a function of this circuit, we apply the simulation with the setting as; Vin1, Vin2 and Vin3 are sin signals of 5 mV, 10 mV and 15 mV amplitude at 1 kHz frequency. DAC1, DAC2 and DAC3 are set as the gains of the multipliers to be 3×, 2× and 1×, respectively. The simulation results are shown in Fig. 15(a). We can correctly observe 65 mV amplitude at the output signal. Next, we change the frequency of each input as; Vin1, Vin2 and Vin3 are sin signals of 10 kHz, 5 kHz and 1 kHz frequencies, respectively. The other settings as same as in Fig. 15(a). Fig. 15(b) demonstrates the simulation results. Our perceptron circuit can output a complicated waveform, so that it has a potential for complicated machine learning.
4. Time series inputs by phase shifting Fig. 12. Temperature dependency. The x-axis corresponds to the temperature (deg.), and the y-axis does to voltage (V).
We attempt to extend our perceptron model which can deal with 4
INTEGRATION, the VLSI journal xxx (xxxx) xxx–xxx
Y. Ishiguchi et al.
Fig. 16. Phase shift circuit.
4.1. Phase shift circuit We introduce a typical phase shift circuit to generate a delay between input signals, which is illustrated in Fig. 16. In this circuit, a phase delay between the input and the output is determined by the resistance values R1, R2, R3 and the capacitance value C1. Plus, these values change depending on the frequency of the input signal and the phase delay. We describe example values of these components to generate 90° phase delay at input difference frequencies in Table 2. In fact, 90° phase delay requests difference component values at difference frequencies. On the other hand, the same component values generate the same absolute phase delay. We notice the delay is formulated as;
Fig. 14. Our perceptron circuit for three inputs.
d = tan−1 ( −ωCR) − tan−1 (ωCR), where d is an absolute phase delay between the input and output signals.
4.2. Example of perceptron circuit with phase shifting We introduce phase shift circuits to our perceptron model to deal with time series inputs as illustrated in Fig. 17. For the simplicity, in this figure, we assume the number of inputs is 1, and the input is distributed passing phase shifters. As a result, we realize time series inputs to our perceptron circuit, where three input signals corresponding to f (t), f(t + d), and f(t + 2d), respectively. Next, we demonstrate the simulation results. Applying different input frequencies to our perceptron circuit of time series, we observe the output signals with all weights (DAC code) are set to be ‘0001’. The results are shown in Figs. 18 and 19, respectively. Note that we provide different component settings for different input frequencies. In both results, we can observe that the input signal phases are shifted by 90°. Furthermore, we verify the behavior of the proposed time series circuit for a random input signal containing different frequencies spectrum, which is the same as used in Fig. 15(b) in Section 3. The result is shown in Fig. 20. As seen in the figure, the phase delay is evenly generated among signals because our phase shifter can control the absolute delay. These results convince us that our perceptron circuit with time series is applicable to prediction algorithms such as a recurrent neural network and chaos pass filter.
Fig. 15. Our perceptron circuit for three inputs.
Table 2 Peripheral component values for 90° phase delay.
time series inputs like a time series prediction using a recurrent neural network. For the simplicity, we formulate a model of time series as follows;
fout (t ) = w1⋅f (t ) + w2⋅f (t + d ) + w3⋅f (t + 2d ), where t and d are time and delay variables, respectively. 5
Input frequency
R1 = R2 = R3
C1
1 kHz 10 kHz
1.55 MΩ 155 kΩ
100 pF 100 pF
INTEGRATION, the VLSI journal xxx (xxxx) xxx–xxx
Y. Ishiguchi et al.
Fig. 20. Simulation result for a random input signal.
Fig. 17. An example of DAC-based perceptron circuit with phase shifters.
Fig. 18. Simulation result for 1 kHz input signal.
Fig. 21. Vin1 = sin x, Vin2 = cos x, Vout = sin x + cos x =
2 sin(x − π /4) .
For input signals sin x (Vin1) and cos x (Vin2), the addition of signals is illustrated as Fig. 21, where the output (Vout) is 2 sin(x − π /4) , so that the output signal is shifted by π/4. Furthermore, assuming the addition of signals, sin x and q ⋅ cos x, the output signals are represented as Fig. 23 when changing the value q. Note that the amplitude and phase are changing according to q. We can understand this phase shifting from the following formula of trigonometric function.
p⋅sin x + q⋅cos x =
p2 + q2 sin(x + α ),
where sin α = q/ p2 + q2 and cos α = p / p2 + q2 . For example, if
Fig. 19. Simulation result for 10 kHz input signal.
5. Periodic signal and phase shifting Inspired by Fourier series,
a0 + 2
∞
∑ (ak cos kx + bk sin kx ), k=1
we propose a novel phase shifting mechanism for a periodic signal input based on our perceptron circuit.
Fig. 22. V in1 = sin x, V in2 = sin(π − x) = cos x. 6
INTEGRATION, the VLSI journal xxx (xxxx) xxx–xxx
Y. Ishiguchi et al.
Fig. 23. Phase shifting of an adding signal of sin x and q ⋅ cos x when changing q.
p = q = 1, then cos α = 2 /2 , and α = π/4. In other words, if we want to shift the signal by π/3, cos π /3 = 1/2 = p / p2 + q2 . As a result, we find p = 1 and q = 3 . Thus, we propose a phase shifting circuit shown in Fig. 22. In the figure, blocks ‘inv’ and ‘π/2’ are an inversion and a phase shift circuit, respectively. The former is the same as in Fig. 16 and the latter is easily realized by a negative feedback of an opamp. Applying the simulation to this circuit, the resultant waveforms are demonstrated in Fig. 24 (a) and (b). In the figure (a), p and q are set to 1 by DAC. On the other hand, in the figure (b), p and q are set to 1 and 4, respectively. As seen in these results, we can control the amplitude and the phase of the output signal by changing the DAC setting in our proposed circuit. Furthermore, based on Fourier series,
f (x ) ≃
a0 + 2
∞
∑ (ak cos kx + bk sin kx ), k=1
We provide a general architecture of a perceptron circuit to represent any signal in the output that be aware of Fourier series. The general form is illustrated in Fig. 25. However, we notice that we must control multiplying periods to realize this architecture. This is one of important our future works. 6. Conclusion This paper presents a perceptron circuit applicable to a sensor analog front-end. We adopt a DAC-based multiplier in the perceptron circuit, where the gain of the multiplier is determined by only digital codes. It contributes to the precision without degrading the processinduced variation. As well, compared with a traditional transconductorbased multiplier, it has a wide input range and a good temperature dependency. The simulation result demonstrates a perceptron circuit with the DAC-based multiplier can combine smoothly multiple analog signals amplifying by the digital codes. In addition, our perceptron circuit is extended to deal with time series inputs by using phase shifter, so that the circuit shows the possibility to be used for prediction
Fig. 24. Vout = p ⋅ sin x + q ⋅ cos x.
7
INTEGRATION, the VLSI journal xxx (xxxx) xxx–xxx
Y. Ishiguchi et al.
algorithms. Furthermore, we discuss the relation of periodic signal and phase shifting in our perceptron circuit. We provide a promising general architecture of perceptron circuit inspired by Fourier series. Appendix A. Supplementary data Supplementary data related to this article can be found at http://dx. doi.org/10.1016/j.vlsi.2018.05.010. References [1] S.M. Gowda, B.J. Sheu, J. Choi, Design and characterization of analog VLSI neural network modules, IEEE J. Solid State Circuit 28 (3) (1993) 301–313. [2] B.V. Essen, C. Macaraeg, M. Gokhale, R. Prenger, Accelerating a random forest classifier: multi-core, GP-GPU, or FPGA? IEEE 20th International Symposium on Field-programmable Custom Computing Machines, 2012, pp. 232–239. [3] Qi Yu, Chao Want, Xiang Ma, Xi Li, Xuehai Zhou, A deep learning prediction process accelerator based FPGA, IEEE/ACM 2015 International Symposium on Cluster, Cloud and Grid Computing, 2015, pp. 1159–1162. [4] John Wynne, Application Notes, AN-320A, ANALOG DEVICES. [6] Gunhee Han, Edgar Sanchez-Sinencio, CMOS transconductance multipliers: a tutorial, IEEE Trans. Circuits Syst. II: Analog Digit. Signal Process. 45 (12) (1998) 1550–1563. [7] J. Xu, S. Mitra, A. Matsumoto, S. Patki, C.V. Hoof, K.A.A. Makinwa, R.F. Yazicioglu, A wearable 8-channel active-electrode EEG/ETI acquisition system for body area networks, IEEE J. Solid State Circuit 49 (9) (2014) 2005–2016. [8] C.-L. Goh, S. Nakatake, A sensor-based data visualization system for training blood pressure measurement by auscultatory method, IEICE Trans. Info Syst. E99-D (4) (2016) 936–943.
Fig. 25. A perceptron circuit based on Fourier series.
8