- Email: [email protected]
Coupling enhances stochastic resonance in nonlinear dynamic elements driven by a sinusoid plus noise
cm ..__
__ 83
24 April 1995
PHYSICS
ELSEVIER
LETTERS
A
Physics Letters A 200 (1995) 283-288
Coupling enhances stochastic resonance in nonlinear dynamic elements driven by a sinusoid plus noise M.E. Inchiosa
‘, A.R. Bulsara 2
Naval Command, Control and Ocean Surveillance Center: Research, Development, Test and Evaluation Division, Materials Research Branch, San Diego, CA 92152-5000, USA Received
26 February
1995; accepted
for publication
26 February 1995
Communicated by C.R. Doering
Abstract We consider the response signal-to-noise ratio of a network of N globally and nonlinearly coupled overdamped nonlinear dynamic elements driven by a weak sinusoidal signal embedded in Gaussian white noise. In the “stochastic resonance” operating regime, the coupling significantly enhances the system’s collective response over that of a single element.
The enhancement of the output signal-to-noise ratio (SNR) in nonlinear dynamic systems via the “stochastic resonance” (SR) phenomenon has received considerable attention in the past decade (see Ref. [ 11 and for good overviews see also Ref. [ 21) . More recently, attention has been focussed on SR effects in the response of coupled bistable elements interacting via linear [ 31 or nonlinear [4] couplings, the latter having potential applicability in the neurosciences. However, the signal-processing potential of SR has received limited attention [ 5,6]. The SNR gain, defined as the ratio of the output SNR to the input SNR, has not been observed to exceed unity [7] for a single bistable element subject to Gaussian white noise when aliasing effects in sampling are properly accounted for [ 5,7,8]. However, for nonwhite noise this is not necessarily the case [ 9-1 l] 3 . In this Letter we con-
1 E-mail: [email protected]. 2 E-mail: [email protected]. 3 We note in passing that the noise considered by Li et al. [ 101 cannot truly be “monochromatic”, for it would then be a sine wave. Elsevier Science B.V. SSDIO375-9601(95)00198-O
sider the response, quantified by the output SNR, of a globally and nonlinearly coupled array of dynamic elements. One (“reference”) element is bistable, and the remaining (“bath”) elements are all either monostable or bistable. Although it appears to be impossible to achieve a gain greater than unity, we show that the coupling does significantly enhance the output SNR over that achieved by a single element. Our system is a variation of that considered in Ref. [ 41; we consider au ensemble of overdamped dynamic elements, each of which is subject to the same “input” signal, consisting of a sine wave in a Gaussian white noise background,
C,~i=-~+~l;,tanhq+qsinwt+~g(t), I j=l
(1) where ,$( t) is Gaussian white noise with mean zero and autocovariance (t(t)t(t + 7)) = a(~). We shall be concerned with the response of the reference element, i = 1. Systems of form ( 1) have been used to
284
M.E. Inchiosa, A.R. 3u~~ru/Physic~
describe connectionist-typeelectronic neural networks (see, e.g., Ref. [ 121) . In such networks, ui denotes the ith neuron’s activation function (membrane potential) , and Ci, Ri denote the neuronal input capacitance and trans-membrane resistance, with the coupling coefficients (synaptic efficacies) Jij usually determined via a “learning rule.” Here, we shall choose the couplings to maximize the output SNR. In general, Eq. ( 1) must be integrated numeric~ly. However, an an~ytic~ reduction of ( 1) to an equivalent one-body problem is possible under the following conditions [4,13] : f 1) The bath elements relax much more quickly than the reference element (“slaving principle” [ 141)) i.e. (RiCi) -’ (i > 1) must dominate the other terms in ( 1). In particular, ( R&) -’ > (RI Cl) -’ (i > 1). (2) The modulation frequency is much smaller than the relaxation rate of the unmodulated bath elements and the well relaxation rate of the unmodulated reference element (“adiabatic assumption”). A similar assumption is also a cornerstone of the adiabatic theory of stochastic resonance [ 1] . (3) The bath p~~eters satisfy DRi < ZC;( 1 - JiiRi) (i > 1) . This condition [ 11,131 guarantees the convergence of the steepest descents technique used to obtain (4) and (5)) and it ensures the monostability of the bath elements. (The reduced dynamics for the b&able-bath case are presented in an upcoming publication [ 111.) Given these conditions we can adiabatically eliminate the bath variables (z+r) from (1) and write down a closed equation for the variable ut [ 4,131,
(2) where U(ur,t)
= $auf-/3ln(coshui)
-Sut
sin(wt)
(3)
is the potential function that underpins the system dynamics. We have defined a = (RlCl)-“,
x JII
+c i>l
DRj
RJliJil
1 - JiiRi
’ - 2Ci(l
- JiiRi)
I
’ (4)
Leiters A 200 (1995) 283-288
DRi
’ - 2Ci( 1 - JiiRi)
I
* (5)
The dynamics are bistable for j? > a and S < &, with the potential (3) having a double-well shape with minima at ur M kc, c = ( P/cx) t~h~~/~), separated by a m~imum at 241M 0 (we ignore the small displacements in the extrema caused by the sinusoidal modulation) . 8, is the “deterministic switching threshold” for vanishingly small w: 8, = -cx~, + p tanh uc, where uc z ln[ (P/a)‘/’ f (P/cx - 1)‘/2]. Terms arising from intra-bath coupling are 0( RiCiRjCj/( RI (21)~) (i, j > 1) or higher and are negligible. The bifurcation properties of the reduced dynamics (2) are discussed elsewhere [ 4,111. Plots of the probability distribution function of ur for this [ 111 and a related [4] system help illustrate the shape of the effective potential. Here we consider the SNR, defined as the ratio of the signal power to the noise power in a frequency range of width Ao centered on the the signal frequency w. Under the aforementioned conditions we expect the reduced dynamics (2) to yield the dynamics of the reference element, in a coarse-grained sense. In this regime one can apply the adiabatic theory [ 1] to the reduced dynamics to calculate the SNR of ut(t). We have also computed the system’s output SNR via numerical integration of ( 1) using the modified Heun method [ 151. We use a small time step, At = (27r/o) /8192, in order to generate noise which has a flat spectrum out to a very high frequency. To avoid aliasing we maintain this sample rate throughout our computations. We convolve the time series with a Welch window before computing the power spectra. In our numerical work we consider two possibilities for the bath elements: for Jii < (RiCi) -’ (i > 3) the bath elements are monostable in isolation (i.e. in the absence of coupling), while for Jii > ( RiC’i) -’ (i > 1) they are bistable. The reference element is nlways bistable. Let us consider the case of monostable bath elements. Guided by (5) we choose Ji,i>r > 0 to increase the effective signal amplitude and thus the output SNR. Similarly, (4) implies we can choose Ji>>l,I < 0 to decrease j3, reducing the effective barrier between the
ME. Inchiosa, A.R. Bulsara /Physics
20
15 8 kz 10
5
0
\I
I
200
400
600
800
D Fig. 1. Monostable bath. Uppermost curve: curves (lowest to highest): Output SNR for 10. Main plot: analog output. Inset: binary and inset have the same coordinate ranges.) RI = 0.0186916, Ri>l = 0.001, Ci = 1, Jll
Input SNR. Lower N = 1, 2, 3, 6, 8, output. (Main plot Other parameters: = 216, & = 100
(i > l), h,i>l = 50, J{2,3 ,_.., lo},1 = (-50, -5% -48, -54, -46, -53, ~47)~ Jij = 0 (i, j > 1; i o = 1.22522, Aw = w/32.
-4%
-52,
f j), 4 = 8,
two potential wells. We have given each Ji> 1~1a unique value so that each bath oscillator evolves uniquely. However, setting all these coupling coefficients equal (or, equivalently, setting N = 2 and multiplying Jzt appropriately) gives very nearly the same output SNR. In Fig. 1 we show the output SNR (computed from the time series ~1 (t) ) for systems of one to ten elements, with the bath elements being monostable. For comparison we also plot the input SNR, with bw being the frequency res$q2/(2DAw/2r), olution of our power spectra. As we increase the number of elements, the output SNR at moderate noise levels also increases. This effect derives from our choice of couplings described earlier. The inset illustrates an even greater coupling effect: it shows the enhancement in the output SNR of a hypothetical transducer which has the same internal dynamics as our reference element but has a binary (rather than analog) output, sgn( ut (t) ). In all figures the output SNR’s noise dependence exhibits SR: At very low noise levels the effective dynamics are bistable, but the state point is confined to the bottom of a single well of the potential for many periods of the driving signal [ 1,161. The shape of the potential at the bottom of the wells is approximately
Letters A 200 (1995) 283-288
285
parabolic, i.e. that of a harmonic oscillator. Consequently, the response of the analog-output system is nearly linear, with the output SNR approaching the input SNR as D + 0. The binary output’s SNR goes to zero as D --f 0 because without help from noise, the sinusoidal input is too weak to cause inter-well hopping. With increasing noise, the output SNR goes through a minimum (for the analog output) followed by a maximum where the noise enhances inter-well transitions in phase with the driving signal (SR) [ 1,2]. Additional noise tends to minimize the potential barrier’s significance, returning us to the scenario of motion in the bottom of a single, approximately parabolic, potential well (albeit a much larger one). For the case of cj,t Jli not too large (e.g. for the N = 1, 2, and 3 curves of Fig. 1) numerical simulations and theory agree quantitatively. The output SNR for these cases is well quantified by the adiabatic theory [ 1] applied to the reduced dynamics, and the maximum of the output SNR occurs at a critical noise value approximately equal to the solution of the transcendental equation D = U(0) - U(c) , as predicted by the adiabatic theory [ 11. For larger values of Ci>t Jli (N = 8, lo), the simulations show that the output SNR approaches but does not exceed the input SNR. For these large amounts of coupling, the theoretically predicted output SNR curves still show qualitative agreement with simulations: they rise with N and retain the correct shape. However they are not quantitatively accurate: the predicted output SNR exceeds the input SNR at moderate noise strengths. The reduced dynamics do not apply quantitatively for such large amounts of coupling because the conditions required for their derivation are no longer satisfied. In fact, even the one-body adiabatic theory [ 1] can predict an output SNR exceeding the input SNR if one uses parameter values which violate the conditions on which its approximations are based. Fig. 2 compares the output SNR from systems of one to three elements, with the bath elements taken to be bistable in isolation. As in the monostable case, at moderate noise levels the output SNR increases with N, causing the SNR gain to approach unity. Fig. 3 illustrates the implications of systematically abandoning the slaving principle in a two element system by increasing Rz. We see that increasing R2 causes the SNR gain to approach unity at moderate noise levels.
ME. Inchiosa, A.R. Bulsara / Physics Letters A 200 (1995) 283-288
286
systematically alters the effective potential; however, the dominant effects are the result of the coupling, which reduces the effective potential barrier height so that even at moderate noise levels the potential tends to a parabola. The effective dynamics may be said to be “linearized” by a combination of the noise (an effect that has also been observed in a single bistable element [ 171) and the coupling. The effect is qualitatively similar to that observed at extremely low and high noise levels insofar as the dynamics tend to that of the simple harmonic oscillator and the output SNR approaches the input SNR. This is the central point of this Letter: for moderate input noise levels the SNR
20
15 3 g v) 10
5
0
Fig. 2. B&able bath. Uppermost curve: Input SNR. Lower curves (lowest to highest): Output SNR for N = 1, 2, 3. Main plot: analog output. Inset: binary output. (Main plot and inset have the same coordinate ranges.) Other parameters as in Fig. 1, except Jii = 1500 (i > 1). Note: sharp minima of the N = 2,3 curves not fully resolved by points plotted.
01,
I
0
200
400
600
800
D
Fig. 3. Abandoning the slaving principle by increasing the time constant of the bath element until it equals that of the reference element. N = 2. Uppermost curve: Input SNR. Lower curves (lowest to highest): Output SNR for Ra = 0.001, 0.00245, 0.004245, 0.008, 0.012, 0.0186916 (= RI). Main plot: analog output. Inset: binary output. (Main plot and inset have the same coordinate ranges.) Other parameters as in Fig. 1.
Observe from (2) -( 5) that for N > 1 the barrier height of the effective potential (3) as well as the locations fc of its wells are functions of the noise. Hence, at each point on the curves of Fig. 1 the noise
at the output of a nonlinear dynamic element can be made to approach the response of a linear system by incorporating the element into a coupled array. We have considered globally, nonlinearly coupled model neurons subject to “global” noise (same noise at each site of the array). Locally, linearly coupled Duffing oscillators subject to local noise have been considered in Refs. [ 18-201, and studies are underway to compare coupling and summing of outputs in systems subject to local noise. We have not found great differences attributable to the choice of Duffing versus neuron nonlinearity, nor global versus local coupling. However, linear versus nonlinear coupling and global versus local noise can make a great deal of difference in certain cases. For example, linear coupling of the form Jij (ui - Uj) has no effect on identical oscillators subject to global noise and identical initial conditions, since the oscillators evolve identically and the coupling terms vanish. Also, nonlinear coupling can completely remove oscillator nonlinearites for some configurations of identical oscillators subject to global noise: for a pair of identical oscillators (Eq. ( 1) with N = 2, Ct = C2, R1 = R2, .I11= 522) with couplings Jt2 = Jzt = -Jtt and identical initial conditions, the hyperbolic tangent terms identically cancel, giving the system a perfectly linear response to its input. The output SNRs of both coupled Duffing and coupled neuron systems exhibit a maximum as a function of coupling strength [ 11,18-201. The maximum comes at a nonzero value of the coupling, so the coupling strength can be used as a tuning parameter to optimize the output SNR for a given noise level. Also, when output SNR is maximized the coupling and noise cooperate to synchronize the oscillators [ 18-201. A nonlinear filter’s output SNR may exceed its in-
M.E. Imhiasa, A.R. Bulsara /Physics Letters A 200 (1995) 283-288
put SNR under certain conditions, The bandpass limiter circuit illustrates this effect 12111 lit em&b of a bandpass filter follows by a threshold (or “?nfi-
nite limiter”) circuit. The bandpass filter remove% all signals outside a narrow frequency band of interest, and the threshold circuit converts the analog output of the bandpass filter to a two-level output (+-U if the bandpass filter output is positive, -V if negative). This nonlinear device improves the SNR of a signal by tacitly assuming that the signal power exceeds the noise power. When the assumption holds, the SNR increases by several dB. However, when the assumption fails, the SNR decreases slightly. The fact that there is a positive SNR gain only above a certain threshold input SNR (the “~eshold effect”) is a universal phenomenon generic to all nonlinear processors (see, e.g., Ref. [ 221). When we drive one of our bistable nonlinear dynamic elements ( 1) with a sine wave plus noise which has most of its energy concentrated around the signal frequency, we should see behavior somewhat like the bandpass limiter. The limiting effect is due to the hyperbolic tangent function and does indeed result in a positive SNR gain, provided the input signal strength is large enough to ove~ome the threshold effect [ 111. Gang et al. have also observed a positive SNR gain for a different stoch~tic reson~ce circuit driven by bandlimited noise [ lo]. Similar gains have been predicted by W&mgi et al., for the driven Kramers equation at moderate to large friction [ 231. SNR is an important quantity in signal processing tasks such as signal detection and signal estimation. However, the SNR alone is not a complete measure of the performance of a signal processing system. For example, a nonlinear signal processor may output a signal which has infinite SNR but is useless because it has no co~elation with the input signal. What are good me~ures of pe~o~~ce~ For signal detection, one must consider detection statistics: probability of detection and probability of false alarm. Such statistics may be summarized in plots of detection probability versus false alarm probability, known as the receiver operating characteristic (ROC) . For signal estimation, relevant measures are mean square error or Bayesian tests [ 221. In Refs. [ 11,191 we have computed ROCs and found that the output SNR enhancement due to coupling reflects itself in improved signal detection statistics.
287
In summary, we have seen that the performance (measured via output SNR) of a nonlinear dynamic system or device can be improved by coupling the device in an array, with the coupling coefficients carefully selected. The effect of coupling elements appears to be to make the system as a whole respond more like a linear system (for the input signal and noise consi.dered here). Violating the slaving condition by increasing the time constants of the bath elements also appeam to “‘linearize” the system, with the SNR gain approaching unity, In all cases, the optimal choice of coupling coefficients Jti, Jit (i > 1) appears to be that which brings about a cooperative increase in the effective ~~l~~~de S and a decrease in the barrier height of the effective potential, i.e. a decrease in /3. Hence, even though the reduced description does not always provide a quantitatively accurate prediction of the response of ( 1) , it nonetheless provides guidance in selecting the optimal coupling coefficients. Can we apply coupling to practical systems? Nonlinear detectors of weak signals embedded in background noise exhibit SR: For example, an rf SQUID magnetometer operated in the hysteretic mode displays SR [24]. SR-like effects have also been observed in sensory neurons [ 251 and predicted in simple models of neural dyn~cs [26], and there has been speculation that noise may play a constructive role in the response of neural networks [ 271. Coupling such nonlinear dynamic elements may enhance their output SNR. From a signal-processing perspective, one notes that a linear filter is optimal for detecting a sine wave in Gaussian white noise [ 221. Thus, the linearizing effect of replacing a single nonlinear dynamic element by a coupled array may help systems relying on nonlinear detectors or transducers to approach optimal performance.
It is a pleasure to acknowledge conversations with T. Albert and G. Schmera (San Diego), P Hanggi (Augsburg) , F. Moss (St. Louis), E?Jung (Augsburg and Illinois), L. Gammaitoni and F. Marchesoni (Perugia) , L. Kiss (Uppsala) , S. Lowen (Columbia), K. Wiesenfeld (Atlanta), and D. Petracchi (Pisa) . Support from the Physics Division of the Office of Naval Research is also gratefully acknowledged.
288
ME. In&ma,
AR. BuEsara /Physics
References [ 11 B. McNamara
and K. Wiesenfeld, Phys. Rev. A 39 (1989) 4854; L. Gammaitoni, E. Menichaeiia-Saetta, S. Santucci and E M~chesoni, Fhys. Rev. Lett. 62 (1989) 349. [2] F. Moss, A. Bulsara and M. Sblesinger, in: Froc. NATO Advanced Research Workshop on Stochastic Resonance in Physics and Biology, J. Stat. Phys. 70, No. l/2 (1993); F! Jung, Phys. Rep. 234 (1994) 175; F. Moss, in: Some contemporary problems in statistical physics, ed. G. Weiss (SIAM, Philadelphia, PA, 1994). [3] l? Jung, U. Behn, E. Pantazelou and E Moss, Phys. Rev. A 46 (1991) Rl709; L. Kiss, Z. Gingl, Z. Marton, J. Kertesz, E Moss, G. Schmera and A. Bulsara, J. Stat. Phys. 70 (1993) 451; E. Pantazelou, F. Moss and D. Chialvo, in: Noise in physical systems and l/f fluctuations, eds. F! Handel and A. Chung (AIP press, New York, 1993). 141 A. Bulsara and G. Schmera, Phys. Rev. E 47 (1993) 3734; A. Bulsara, A. Marcn and G. Schmera, Biol. Cybern. 70 (1993) 145; A. Bulsara and A. Maren, in: Rethinking neural networks: Quantum fields and biological data, ed. K.H. Pribram (Erlbaum Associates, 1993). [5] L. Gammaitoni, E. Menichaella-Saetta, S. Santncci and E M~chesoni, Phys. Lctt. A 142 (1989) 59. [6] G. De-chun and H. Gang, Phys. Rev. A 46 ( 1992) 3243. [7] T. Albert, A. Bulsara, G. Schmera and M. Inchiosa, in: Conference record of the 27th Asilomar Conference on Signals, systems and computers, ed. A. Singh (IEEE Society Press, Los Alamitos, CA, 1993). [ 81 G. De-chun, H. Gang, W. Xia-dong, Y. Chun-yan,Q. Guangrong, L. Rong and D. Da-fn, Phys. Rev. E 8 (1993) 4862. [9] J. Grohs, S. Apanasevich, R Jung, H. 18ler, D. Burak and C. Klingshim, Phys. Rev. A 49 (1994) 2199; R Jung, Phys. Rev. E, to be published; Z. Gingl, L. Kiss and E Moss, to be published. [lo] R. Li, G. Hu, C. Yang, X. Weng, G. Qing and H. Zhu, Stochastic resonance in bistable systems subject to signal and mon~~omatic noise, to be pub~shed. [ 1 I ] M. Inchiosa and A. Bulsara, Periodically driven globally coupled nonlinear systems: Response and its enhancement via stochastic resonance, to be published. [ 121 D. Amit, Modelling brain function (Cambridge Univ. Press, Cambridge, 1989) ; G. Weisbuch, Complex systems dynamics (Addison-Wesley, Reading, MA, 1991); J. Hertz, A. Krogh and R. Palmer, Introduction to the theory of neural computation (Addison-Wesley, Reading, MA, 1991).
Letters A 200 CI995)2%3-288 [13] W. Schieve, A. Bulsara and G. Davis, Phys. Rev. A 43 (1991) 2613. [ 141 H. Haken, Synergetics (Springer, Berlin, 1978). [IS] T. C. Gard, Introduction to stochastic differential equations (Dekker, New York, 1988). t 161 V. Schnei~~n, l? Jung and R H&tggi, F’hys. Rev. L&t. 72 (1994) 2682; L. Gammaitoni, E Marchesoni, E. Menichaella-Saetta and S. Santucci, to be published; N. Stocks, to be published. [ 171 M.I. Dykman, D.G. Luchinsky and R. Maunella, in: Fluctuations and order: The new synthesis, ed. M. Millonos (Springer, Berlin, 1994). [ISI J. Lindner, B. Meadows, W. Ditto, M. Inchiosa and A. Bulsara, Array enhanced stochastic resonance, to be published. 1191 M. Inchiosa, A. Bulsara, J. Lindner, B. Meadows and W. Ditto, Array enhanced stochastic resonance: Implications for signal processing, in: Proc. 3rd Technical Conference on Nonlinear dyn~cs (chaos) and full spectrum processing, ed. R.A. Katz (American institute of Physics, New York, 1995). 1201 A. Neiman and L. Schimansky-Geier, Phys. Lett. A 197 (1995) 397. [21] W.B. Davenport Jr., J. Appl. Phys. 24 (1953) 720. [22] H. van Trees, Detection, estimation and modulation theory (Wiley, New York, 1978). 1231 R Hlnggi, R Jung, C. Zerbe and E Moss, J. Stat. Phys. 70 (1993) 25. [24] A. Hibbs, E. Jacobs, J. Bekkedahl, A. Bulsara and E Moss, in: Noise in physical systems and 1 /f fluctuations, eds. R Handel and A. Chung (AIP press, New York, 1993); J. Appl. Phys., to be published. [25] J. Douglass, L. Wilkens, E. Pantazelou and E Moss, Nature 365 (1993) 337; H.A. Braun, H. Wissing, K. Schafer and M.C. Hirsch, Nature 367 (1994) 270. [26] A. Bulsara, E. Jacobs, T. Zhou, F. Moss and L. Kiss, J. Theor. Biol. 152 (1991) 531; A. Longtin, A. Bulsara and E Moss, Phys. Rev. L&t. 67 (1991) 656; J. Maddox, Nature 352 (1991) 469; 369 (1994) 271; A. Longtin, J. Stat. Phys. 70 (1993) 309; A. Longtin, A. Bulsara, D. Pierson and E Moss, Biol. Cybem. 70 (1994) 569; K. Wiesenfeld, D. Pierson, E. Pantazelou and F. Moss, Phys. Rev. Len. 72 (1994) 2125; A. Bulsara, S. Lowen and D. Rees, Phys. Rev. E 49 (1994) 4989. [27] J. Buhmann and K. Schulten, Biol. Cybem. 61 (1987) 313; D. Gardner, Neural Netw. 2 (1989) 69.
__ 83
24 April 1995
PHYSICS
ELSEVIER
LETTERS
A
Physics Letters A 200 (1995) 283-288
Coupling enhances stochastic resonance in nonlinear dynamic elements driven by a sinusoid plus noise M.E. Inchiosa
‘, A.R. Bulsara 2
Naval Command, Control and Ocean Surveillance Center: Research, Development, Test and Evaluation Division, Materials Research Branch, San Diego, CA 92152-5000, USA Received
26 February
1995; accepted
for publication
26 February 1995
Communicated by C.R. Doering
Abstract We consider the response signal-to-noise ratio of a network of N globally and nonlinearly coupled overdamped nonlinear dynamic elements driven by a weak sinusoidal signal embedded in Gaussian white noise. In the “stochastic resonance” operating regime, the coupling significantly enhances the system’s collective response over that of a single element.
The enhancement of the output signal-to-noise ratio (SNR) in nonlinear dynamic systems via the “stochastic resonance” (SR) phenomenon has received considerable attention in the past decade (see Ref. [ 11 and for good overviews see also Ref. [ 21) . More recently, attention has been focussed on SR effects in the response of coupled bistable elements interacting via linear [ 31 or nonlinear [4] couplings, the latter having potential applicability in the neurosciences. However, the signal-processing potential of SR has received limited attention [ 5,6]. The SNR gain, defined as the ratio of the output SNR to the input SNR, has not been observed to exceed unity [7] for a single bistable element subject to Gaussian white noise when aliasing effects in sampling are properly accounted for [ 5,7,8]. However, for nonwhite noise this is not necessarily the case [ 9-1 l] 3 . In this Letter we con-
1 E-mail: [email protected]. 2 E-mail: [email protected]. 3 We note in passing that the noise considered by Li et al. [ 101 cannot truly be “monochromatic”, for it would then be a sine wave. Elsevier Science B.V. SSDIO375-9601(95)00198-O
sider the response, quantified by the output SNR, of a globally and nonlinearly coupled array of dynamic elements. One (“reference”) element is bistable, and the remaining (“bath”) elements are all either monostable or bistable. Although it appears to be impossible to achieve a gain greater than unity, we show that the coupling does significantly enhance the output SNR over that achieved by a single element. Our system is a variation of that considered in Ref. [ 41; we consider au ensemble of overdamped dynamic elements, each of which is subject to the same “input” signal, consisting of a sine wave in a Gaussian white noise background,
C,~i=-~+~l;,tanhq+qsinwt+~g(t), I j=l
(1) where ,$( t) is Gaussian white noise with mean zero and autocovariance (t(t)t(t + 7)) = a(~). We shall be concerned with the response of the reference element, i = 1. Systems of form ( 1) have been used to
284
M.E. Inchiosa, A.R. 3u~~ru/Physic~
describe connectionist-typeelectronic neural networks (see, e.g., Ref. [ 121) . In such networks, ui denotes the ith neuron’s activation function (membrane potential) , and Ci, Ri denote the neuronal input capacitance and trans-membrane resistance, with the coupling coefficients (synaptic efficacies) Jij usually determined via a “learning rule.” Here, we shall choose the couplings to maximize the output SNR. In general, Eq. ( 1) must be integrated numeric~ly. However, an an~ytic~ reduction of ( 1) to an equivalent one-body problem is possible under the following conditions [4,13] : f 1) The bath elements relax much more quickly than the reference element (“slaving principle” [ 141)) i.e. (RiCi) -’ (i > 1) must dominate the other terms in ( 1). In particular, ( R&) -’ > (RI Cl) -’ (i > 1). (2) The modulation frequency is much smaller than the relaxation rate of the unmodulated bath elements and the well relaxation rate of the unmodulated reference element (“adiabatic assumption”). A similar assumption is also a cornerstone of the adiabatic theory of stochastic resonance [ 1] . (3) The bath p~~eters satisfy DRi < ZC;( 1 - JiiRi) (i > 1) . This condition [ 11,131 guarantees the convergence of the steepest descents technique used to obtain (4) and (5)) and it ensures the monostability of the bath elements. (The reduced dynamics for the b&able-bath case are presented in an upcoming publication [ 111.) Given these conditions we can adiabatically eliminate the bath variables (z+r) from (1) and write down a closed equation for the variable ut [ 4,131,
(2) where U(ur,t)
= $auf-/3ln(coshui)
-Sut
sin(wt)
(3)
is the potential function that underpins the system dynamics. We have defined a = (RlCl)-“,
x JII
+c i>l
DRj
RJliJil
1 - JiiRi
’ - 2Ci(l
- JiiRi)
I
’ (4)
Leiters A 200 (1995) 283-288
DRi
’ - 2Ci( 1 - JiiRi)
I
* (5)
The dynamics are bistable for j? > a and S < &, with the potential (3) having a double-well shape with minima at ur M kc, c = ( P/cx) t~h~~/~), separated by a m~imum at 241M 0 (we ignore the small displacements in the extrema caused by the sinusoidal modulation) . 8, is the “deterministic switching threshold” for vanishingly small w: 8, = -cx~, + p tanh uc, where uc z ln[ (P/a)‘/’ f (P/cx - 1)‘/2]. Terms arising from intra-bath coupling are 0( RiCiRjCj/( RI (21)~) (i, j > 1) or higher and are negligible. The bifurcation properties of the reduced dynamics (2) are discussed elsewhere [ 4,111. Plots of the probability distribution function of ur for this [ 111 and a related [4] system help illustrate the shape of the effective potential. Here we consider the SNR, defined as the ratio of the signal power to the noise power in a frequency range of width Ao centered on the the signal frequency w. Under the aforementioned conditions we expect the reduced dynamics (2) to yield the dynamics of the reference element, in a coarse-grained sense. In this regime one can apply the adiabatic theory [ 1] to the reduced dynamics to calculate the SNR of ut(t). We have also computed the system’s output SNR via numerical integration of ( 1) using the modified Heun method [ 151. We use a small time step, At = (27r/o) /8192, in order to generate noise which has a flat spectrum out to a very high frequency. To avoid aliasing we maintain this sample rate throughout our computations. We convolve the time series with a Welch window before computing the power spectra. In our numerical work we consider two possibilities for the bath elements: for Jii < (RiCi) -’ (i > 3) the bath elements are monostable in isolation (i.e. in the absence of coupling), while for Jii > ( RiC’i) -’ (i > 1) they are bistable. The reference element is nlways bistable. Let us consider the case of monostable bath elements. Guided by (5) we choose Ji,i>r > 0 to increase the effective signal amplitude and thus the output SNR. Similarly, (4) implies we can choose Ji>>l,I < 0 to decrease j3, reducing the effective barrier between the
ME. Inchiosa, A.R. Bulsara /Physics
20
15 8 kz 10
5
0
\I
I
200
400
600
800
D Fig. 1. Monostable bath. Uppermost curve: curves (lowest to highest): Output SNR for 10. Main plot: analog output. Inset: binary and inset have the same coordinate ranges.) RI = 0.0186916, Ri>l = 0.001, Ci = 1, Jll
Input SNR. Lower N = 1, 2, 3, 6, 8, output. (Main plot Other parameters: = 216, & = 100
(i > l), h,i>l = 50, J{2,3 ,_.., lo},1 = (-50, -5% -48, -54, -46, -53, ~47)~ Jij = 0 (i, j > 1; i o = 1.22522, Aw = w/32.
-4%
-52,
f j), 4 = 8,
two potential wells. We have given each Ji> 1~1a unique value so that each bath oscillator evolves uniquely. However, setting all these coupling coefficients equal (or, equivalently, setting N = 2 and multiplying Jzt appropriately) gives very nearly the same output SNR. In Fig. 1 we show the output SNR (computed from the time series ~1 (t) ) for systems of one to ten elements, with the bath elements being monostable. For comparison we also plot the input SNR, with bw being the frequency res$q2/(2DAw/2r), olution of our power spectra. As we increase the number of elements, the output SNR at moderate noise levels also increases. This effect derives from our choice of couplings described earlier. The inset illustrates an even greater coupling effect: it shows the enhancement in the output SNR of a hypothetical transducer which has the same internal dynamics as our reference element but has a binary (rather than analog) output, sgn( ut (t) ). In all figures the output SNR’s noise dependence exhibits SR: At very low noise levels the effective dynamics are bistable, but the state point is confined to the bottom of a single well of the potential for many periods of the driving signal [ 1,161. The shape of the potential at the bottom of the wells is approximately
Letters A 200 (1995) 283-288
285
parabolic, i.e. that of a harmonic oscillator. Consequently, the response of the analog-output system is nearly linear, with the output SNR approaching the input SNR as D + 0. The binary output’s SNR goes to zero as D --f 0 because without help from noise, the sinusoidal input is too weak to cause inter-well hopping. With increasing noise, the output SNR goes through a minimum (for the analog output) followed by a maximum where the noise enhances inter-well transitions in phase with the driving signal (SR) [ 1,2]. Additional noise tends to minimize the potential barrier’s significance, returning us to the scenario of motion in the bottom of a single, approximately parabolic, potential well (albeit a much larger one). For the case of cj,t Jli not too large (e.g. for the N = 1, 2, and 3 curves of Fig. 1) numerical simulations and theory agree quantitatively. The output SNR for these cases is well quantified by the adiabatic theory [ 1] applied to the reduced dynamics, and the maximum of the output SNR occurs at a critical noise value approximately equal to the solution of the transcendental equation D = U(0) - U(c) , as predicted by the adiabatic theory [ 11. For larger values of Ci>t Jli (N = 8, lo), the simulations show that the output SNR approaches but does not exceed the input SNR. For these large amounts of coupling, the theoretically predicted output SNR curves still show qualitative agreement with simulations: they rise with N and retain the correct shape. However they are not quantitatively accurate: the predicted output SNR exceeds the input SNR at moderate noise strengths. The reduced dynamics do not apply quantitatively for such large amounts of coupling because the conditions required for their derivation are no longer satisfied. In fact, even the one-body adiabatic theory [ 1] can predict an output SNR exceeding the input SNR if one uses parameter values which violate the conditions on which its approximations are based. Fig. 2 compares the output SNR from systems of one to three elements, with the bath elements taken to be bistable in isolation. As in the monostable case, at moderate noise levels the output SNR increases with N, causing the SNR gain to approach unity. Fig. 3 illustrates the implications of systematically abandoning the slaving principle in a two element system by increasing Rz. We see that increasing R2 causes the SNR gain to approach unity at moderate noise levels.
ME. Inchiosa, A.R. Bulsara / Physics Letters A 200 (1995) 283-288
286
systematically alters the effective potential; however, the dominant effects are the result of the coupling, which reduces the effective potential barrier height so that even at moderate noise levels the potential tends to a parabola. The effective dynamics may be said to be “linearized” by a combination of the noise (an effect that has also been observed in a single bistable element [ 171) and the coupling. The effect is qualitatively similar to that observed at extremely low and high noise levels insofar as the dynamics tend to that of the simple harmonic oscillator and the output SNR approaches the input SNR. This is the central point of this Letter: for moderate input noise levels the SNR
20
15 3 g v) 10
5
0
Fig. 2. B&able bath. Uppermost curve: Input SNR. Lower curves (lowest to highest): Output SNR for N = 1, 2, 3. Main plot: analog output. Inset: binary output. (Main plot and inset have the same coordinate ranges.) Other parameters as in Fig. 1, except Jii = 1500 (i > 1). Note: sharp minima of the N = 2,3 curves not fully resolved by points plotted.
01,
I
0
200
400
600
800
D
Fig. 3. Abandoning the slaving principle by increasing the time constant of the bath element until it equals that of the reference element. N = 2. Uppermost curve: Input SNR. Lower curves (lowest to highest): Output SNR for Ra = 0.001, 0.00245, 0.004245, 0.008, 0.012, 0.0186916 (= RI). Main plot: analog output. Inset: binary output. (Main plot and inset have the same coordinate ranges.) Other parameters as in Fig. 1.
Observe from (2) -( 5) that for N > 1 the barrier height of the effective potential (3) as well as the locations fc of its wells are functions of the noise. Hence, at each point on the curves of Fig. 1 the noise
at the output of a nonlinear dynamic element can be made to approach the response of a linear system by incorporating the element into a coupled array. We have considered globally, nonlinearly coupled model neurons subject to “global” noise (same noise at each site of the array). Locally, linearly coupled Duffing oscillators subject to local noise have been considered in Refs. [ 18-201, and studies are underway to compare coupling and summing of outputs in systems subject to local noise. We have not found great differences attributable to the choice of Duffing versus neuron nonlinearity, nor global versus local coupling. However, linear versus nonlinear coupling and global versus local noise can make a great deal of difference in certain cases. For example, linear coupling of the form Jij (ui - Uj) has no effect on identical oscillators subject to global noise and identical initial conditions, since the oscillators evolve identically and the coupling terms vanish. Also, nonlinear coupling can completely remove oscillator nonlinearites for some configurations of identical oscillators subject to global noise: for a pair of identical oscillators (Eq. ( 1) with N = 2, Ct = C2, R1 = R2, .I11= 522) with couplings Jt2 = Jzt = -Jtt and identical initial conditions, the hyperbolic tangent terms identically cancel, giving the system a perfectly linear response to its input. The output SNRs of both coupled Duffing and coupled neuron systems exhibit a maximum as a function of coupling strength [ 11,18-201. The maximum comes at a nonzero value of the coupling, so the coupling strength can be used as a tuning parameter to optimize the output SNR for a given noise level. Also, when output SNR is maximized the coupling and noise cooperate to synchronize the oscillators [ 18-201. A nonlinear filter’s output SNR may exceed its in-
M.E. Imhiasa, A.R. Bulsara /Physics Letters A 200 (1995) 283-288
put SNR under certain conditions, The bandpass limiter circuit illustrates this effect 12111 lit em&b of a bandpass filter follows by a threshold (or “?nfi-
nite limiter”) circuit. The bandpass filter remove% all signals outside a narrow frequency band of interest, and the threshold circuit converts the analog output of the bandpass filter to a two-level output (+-U if the bandpass filter output is positive, -V if negative). This nonlinear device improves the SNR of a signal by tacitly assuming that the signal power exceeds the noise power. When the assumption holds, the SNR increases by several dB. However, when the assumption fails, the SNR decreases slightly. The fact that there is a positive SNR gain only above a certain threshold input SNR (the “~eshold effect”) is a universal phenomenon generic to all nonlinear processors (see, e.g., Ref. [ 221). When we drive one of our bistable nonlinear dynamic elements ( 1) with a sine wave plus noise which has most of its energy concentrated around the signal frequency, we should see behavior somewhat like the bandpass limiter. The limiting effect is due to the hyperbolic tangent function and does indeed result in a positive SNR gain, provided the input signal strength is large enough to ove~ome the threshold effect [ 111. Gang et al. have also observed a positive SNR gain for a different stoch~tic reson~ce circuit driven by bandlimited noise [ lo]. Similar gains have been predicted by W&mgi et al., for the driven Kramers equation at moderate to large friction [ 231. SNR is an important quantity in signal processing tasks such as signal detection and signal estimation. However, the SNR alone is not a complete measure of the performance of a signal processing system. For example, a nonlinear signal processor may output a signal which has infinite SNR but is useless because it has no co~elation with the input signal. What are good me~ures of pe~o~~ce~ For signal detection, one must consider detection statistics: probability of detection and probability of false alarm. Such statistics may be summarized in plots of detection probability versus false alarm probability, known as the receiver operating characteristic (ROC) . For signal estimation, relevant measures are mean square error or Bayesian tests [ 221. In Refs. [ 11,191 we have computed ROCs and found that the output SNR enhancement due to coupling reflects itself in improved signal detection statistics.
287
In summary, we have seen that the performance (measured via output SNR) of a nonlinear dynamic system or device can be improved by coupling the device in an array, with the coupling coefficients carefully selected. The effect of coupling elements appears to be to make the system as a whole respond more like a linear system (for the input signal and noise consi.dered here). Violating the slaving condition by increasing the time constants of the bath elements also appeam to “‘linearize” the system, with the SNR gain approaching unity, In all cases, the optimal choice of coupling coefficients Jti, Jit (i > 1) appears to be that which brings about a cooperative increase in the effective ~~l~~~de S and a decrease in the barrier height of the effective potential, i.e. a decrease in /3. Hence, even though the reduced description does not always provide a quantitatively accurate prediction of the response of ( 1) , it nonetheless provides guidance in selecting the optimal coupling coefficients. Can we apply coupling to practical systems? Nonlinear detectors of weak signals embedded in background noise exhibit SR: For example, an rf SQUID magnetometer operated in the hysteretic mode displays SR [24]. SR-like effects have also been observed in sensory neurons [ 251 and predicted in simple models of neural dyn~cs [26], and there has been speculation that noise may play a constructive role in the response of neural networks [ 271. Coupling such nonlinear dynamic elements may enhance their output SNR. From a signal-processing perspective, one notes that a linear filter is optimal for detecting a sine wave in Gaussian white noise [ 221. Thus, the linearizing effect of replacing a single nonlinear dynamic element by a coupled array may help systems relying on nonlinear detectors or transducers to approach optimal performance.
It is a pleasure to acknowledge conversations with T. Albert and G. Schmera (San Diego), P Hanggi (Augsburg) , F. Moss (St. Louis), E?Jung (Augsburg and Illinois), L. Gammaitoni and F. Marchesoni (Perugia) , L. Kiss (Uppsala) , S. Lowen (Columbia), K. Wiesenfeld (Atlanta), and D. Petracchi (Pisa) . Support from the Physics Division of the Office of Naval Research is also gratefully acknowledged.
288
ME. In&ma,
AR. BuEsara /Physics
References [ 11 B. McNamara
and K. Wiesenfeld, Phys. Rev. A 39 (1989) 4854; L. Gammaitoni, E. Menichaeiia-Saetta, S. Santucci and E M~chesoni, Fhys. Rev. Lett. 62 (1989) 349. [2] F. Moss, A. Bulsara and M. Sblesinger, in: Froc. NATO Advanced Research Workshop on Stochastic Resonance in Physics and Biology, J. Stat. Phys. 70, No. l/2 (1993); F! Jung, Phys. Rep. 234 (1994) 175; F. Moss, in: Some contemporary problems in statistical physics, ed. G. Weiss (SIAM, Philadelphia, PA, 1994). [3] l? Jung, U. Behn, E. Pantazelou and E Moss, Phys. Rev. A 46 (1991) Rl709; L. Kiss, Z. Gingl, Z. Marton, J. Kertesz, E Moss, G. Schmera and A. Bulsara, J. Stat. Phys. 70 (1993) 451; E. Pantazelou, F. Moss and D. Chialvo, in: Noise in physical systems and l/f fluctuations, eds. F! Handel and A. Chung (AIP press, New York, 1993). 141 A. Bulsara and G. Schmera, Phys. Rev. E 47 (1993) 3734; A. Bulsara, A. Marcn and G. Schmera, Biol. Cybern. 70 (1993) 145; A. Bulsara and A. Maren, in: Rethinking neural networks: Quantum fields and biological data, ed. K.H. Pribram (Erlbaum Associates, 1993). [5] L. Gammaitoni, E. Menichaella-Saetta, S. Santncci and E M~chesoni, Phys. Lctt. A 142 (1989) 59. [6] G. De-chun and H. Gang, Phys. Rev. A 46 ( 1992) 3243. [7] T. Albert, A. Bulsara, G. Schmera and M. Inchiosa, in: Conference record of the 27th Asilomar Conference on Signals, systems and computers, ed. A. Singh (IEEE Society Press, Los Alamitos, CA, 1993). [ 81 G. De-chun, H. Gang, W. Xia-dong, Y. Chun-yan,Q. Guangrong, L. Rong and D. Da-fn, Phys. Rev. E 8 (1993) 4862. [9] J. Grohs, S. Apanasevich, R Jung, H. 18ler, D. Burak and C. Klingshim, Phys. Rev. A 49 (1994) 2199; R Jung, Phys. Rev. E, to be published; Z. Gingl, L. Kiss and E Moss, to be published. [lo] R. Li, G. Hu, C. Yang, X. Weng, G. Qing and H. Zhu, Stochastic resonance in bistable systems subject to signal and mon~~omatic noise, to be pub~shed. [ 1 I ] M. Inchiosa and A. Bulsara, Periodically driven globally coupled nonlinear systems: Response and its enhancement via stochastic resonance, to be published. [ 121 D. Amit, Modelling brain function (Cambridge Univ. Press, Cambridge, 1989) ; G. Weisbuch, Complex systems dynamics (Addison-Wesley, Reading, MA, 1991); J. Hertz, A. Krogh and R. Palmer, Introduction to the theory of neural computation (Addison-Wesley, Reading, MA, 1991).
Letters A 200 CI995)2%3-288 [13] W. Schieve, A. Bulsara and G. Davis, Phys. Rev. A 43 (1991) 2613. [ 141 H. Haken, Synergetics (Springer, Berlin, 1978). [IS] T. C. Gard, Introduction to stochastic differential equations (Dekker, New York, 1988). t 161 V. Schnei~~n, l? Jung and R H&tggi, F’hys. Rev. L&t. 72 (1994) 2682; L. Gammaitoni, E Marchesoni, E. Menichaella-Saetta and S. Santucci, to be published; N. Stocks, to be published. [ 171 M.I. Dykman, D.G. Luchinsky and R. Maunella, in: Fluctuations and order: The new synthesis, ed. M. Millonos (Springer, Berlin, 1994). [ISI J. Lindner, B. Meadows, W. Ditto, M. Inchiosa and A. Bulsara, Array enhanced stochastic resonance, to be published. 1191 M. Inchiosa, A. Bulsara, J. Lindner, B. Meadows and W. Ditto, Array enhanced stochastic resonance: Implications for signal processing, in: Proc. 3rd Technical Conference on Nonlinear dyn~cs (chaos) and full spectrum processing, ed. R.A. Katz (American institute of Physics, New York, 1995). 1201 A. Neiman and L. Schimansky-Geier, Phys. Lett. A 197 (1995) 397. [21] W.B. Davenport Jr., J. Appl. Phys. 24 (1953) 720. [22] H. van Trees, Detection, estimation and modulation theory (Wiley, New York, 1978). 1231 R Hlnggi, R Jung, C. Zerbe and E Moss, J. Stat. Phys. 70 (1993) 25. [24] A. Hibbs, E. Jacobs, J. Bekkedahl, A. Bulsara and E Moss, in: Noise in physical systems and 1 /f fluctuations, eds. R Handel and A. Chung (AIP press, New York, 1993); J. Appl. Phys., to be published. [25] J. Douglass, L. Wilkens, E. Pantazelou and E Moss, Nature 365 (1993) 337; H.A. Braun, H. Wissing, K. Schafer and M.C. Hirsch, Nature 367 (1994) 270. [26] A. Bulsara, E. Jacobs, T. Zhou, F. Moss and L. Kiss, J. Theor. Biol. 152 (1991) 531; A. Longtin, A. Bulsara and E Moss, Phys. Rev. L&t. 67 (1991) 656; J. Maddox, Nature 352 (1991) 469; 369 (1994) 271; A. Longtin, J. Stat. Phys. 70 (1993) 309; A. Longtin, A. Bulsara, D. Pierson and E Moss, Biol. Cybem. 70 (1994) 569; K. Wiesenfeld, D. Pierson, E. Pantazelou and F. Moss, Phys. Rev. Len. 72 (1994) 2125; A. Bulsara, S. Lowen and D. Rees, Phys. Rev. E 49 (1994) 4989. [27] J. Buhmann and K. Schulten, Biol. Cybem. 61 (1987) 313; D. Gardner, Neural Netw. 2 (1989) 69.