- Email: [email protected]
Self-modeling in Hopfield Neural Networks with Continuous Activation Function
Available online at www.sciencedirect.com
This This This This This This
space is reserved for the Procedia header, space is reserved for the Procedia header, space is reserved for the Procedia header, space is reserved for the Procedia header, Procedia 123Procedia (2018) 573–578 space is Computer reservedScience for the header, space is reserved for the Procedia header,
ScienceDirect
do do do do do do
not not not not not not
use use use use use use
it it it it it it
Self-modeling inonHopfield Networks with BICA 2017 8th Annual International Conference BiologicallyNeural Inspired Cognitive Architectures,
1 1 1 1
Self-modeling in Hopfield Neural Networks with Self-modeling in Hopfield Neural Networks with Continuous Activation Function Self-modeling in Hopfield Neural Networks with Continuous Activation Function Self-modeling in Hopfield Neural Networks with Continuous Activation Function 1 12 Mario Zarco and Tom Froese Continuous Activation Function Self-modeling in Hopfield Neural Networks with 1 12 Continuous Activation Function Mario Zarco 1 and Tom Froese12 Zarco Froese Intituto de Investigaciones enMario Matem´ aticasActivation Aplicas y en Sistemas, Universidad Nacional Aut´ onoma 1 and Tom 12 Continuous Function Mario Zarco and Tom Froese de M´exico, 1Mexico City, Mexico 12
Intituto de Investigaciones enMario Matem´ aticas Aplicas y en Sistemas, Nacional Aut´ onoma Zarco and Tom Froese Universidad Intituto de Investigaciones en Matem´ aeticas Aplicas yCity, en Sistemas, Universidad Nacional Aut´ onoma [email protected] de M´ xico, Mexico Mexico 1 12 Matem´ aeticas Aplicas yCity, en Sistemas, Universidad Nacional Aut´ onoma 2Intituto de Investigaciones en Mario Zarco and Tom Froese de M´ xico, Mexico Mexico 1 Centro de de Investigaciones Ciencias de la Complejidad, Universidad Nacional Aut´ oUniversidad noma de M´eNacional xico, Mexico [email protected] Intituto en Matem´ aeticas Aplicas yCity, en Sistemas, Aut´ oCity, noma de M´ xico, Mexico Mexico [email protected] 2 MexicoCity, Universidad Nacional Aut´ o noma de M´ e xico, Mexico City, 1 2 Centro de Ciencias de la Complejidad, de M´ e xico, Mexico Mexico [email protected] Intituto en Matem´ aticas Aplicas y Nacional en Sistemas, oUniversidad Aut´ oCity, noma Centro de de Investigaciones Ciencias de la Complejidad, Universidad noma de M´eNacional xico, Mexico [email protected] Mexico Nacional Aut´ 2 [email protected] Centro de Ciencias de la Complejidad, Universidad Aut´ onoma de M´exico, Mexico City, de M´ e xico, Mexico City, Mexico Mexico 2 [email protected] Centro de Ciencias de la Complejidad, Universidad onoma de M´exico, Mexico City, Mexico Nacional Aut´ [email protected] [email protected] 2 Mexico Nacional Aut´ [email protected] Centro de Ciencias de la Complejidad, Universidad onoma de M´exico, Mexico City, Abstract [email protected] Mexico Hopfield different attractors of which most are local optima. It Abstractnetworks can exhibit many [email protected] Abstract
has been demonstrated combining states randomization Hebbian learning Hopfield networks can that exhibit many different attractors of and which most are local enlarges optima. the It Abstract Hopfield networks can exhibit many different attractors of and which most are local enlarges optima. It basin of attraction of globally optimal attractors. The procedure is called self-modeling and it has been demonstrated that combining states randomization Hebbian learning the Abstract Hopfield networks can that exhibit many different attractors of and which most are local enlarges optima. the It has been demonstrated combining states randomization Hebbian learning has been applied in symmetric Hopfield networks with discrete states and without self-recurrent basin of attraction of globally optimal attractors. The procedure is called self-modeling and it Hopfield networks can exhibit many different attractors of which most are local optima. It Abstract has been demonstrated that combining states randomization and Hebbian learning enlarges basin of attraction of globally optimal attractors. The procedure is called self-modeling andthe it connections. We are interested in knowing which topological constraints can be relaxed. So, been applied in symmetric Hopfield networks with discrete states and without self-recurrent has demonstrated that combining states randomization and Hebbian learning enlarges the Hopfield networks can exhibit many different attractors of which most are local optima. It basin of attraction of globally optimal attractors. Thediscrete procedure is called self-modeling and it has been applied in symmetric Hopfield networks with states and without self-recurrent the self-modeling process is tested asymmetric Hopfield networks states and connections. We in are interested in in knowing which topological constraints can be self-recurrent relaxed. So, basin of attraction of globally optimal attractors. The procedure is with called self-modeling and it has been demonstrated that combining states randomization and Hebbian learning enlarges the applied symmetric Hopfield networks with discrete states and continuous without connections. We are interested in in knowing which topological constraints can be relaxed. So, self-recurrent connections. The best results are obtained in networks with modular structure. the self-modeling process is tested asymmetric Hopfield networks with continuous states and has been applied in symmetric Hopfield networks with discrete states and without self-recurrent basin of attraction of globally optimal attractors. The procedure is called self-modeling and it connections. We process are interested in in knowing whichHopfield topological constraints can be relaxed. So, the self-modeling isThe tested asymmetric networks with continuous states and self-recurrent connections. best results are obtained in networks with modular structure. connections. We are interested in knowing which topological constraints can be relaxed. So, has been applied in symmetric Hopfield networks with discrete states and without self-recurrent the self-modeling processHopfield isby tested inLtd. asymmetric Hopfield with continuous states and Keywords: Self-modeling, neural network, Hebbian learning, continuous activation function self-recurrent connections. The best results innetworks networks with structure. © 2018 The Authors. Published Elsevier This are is anobtained open access article under the CCmodular BY-NC-ND license the self-modeling process isThe tested in asymmetric Hopfield with states and connections. We are interested in knowing which topological constraints can be relaxed. So, self-recurrent connections. best results are obtained innetworks networks withcontinuous modular structure. Keywords: Self-modeling, Hopfield neural network, Hebbian learning, continuous activation function (http://creativecommons.org/licenses/by-nc-nd/3.0/). Keywords: Self-modeling, Hopfield neural network, Hebbian learning, continuous activation function self-recurrent connections. The best results are obtained in networks with modular structure. the self-modeling process is tested in asymmetric Hopfield networks with continuous states and Peer-review responsibilityHopfield of the scientific the 8th Annual International Conference on Biologically Keywords: under Self-modeling, neuralcommittee network,ofHebbian learning, continuous activation function Inspired Cognitive Architectures self-recurrent connections. The best results are obtained in networks with modular Keywords: Self-modeling, Hopfield neural network, Hebbian learning, continuous activationstructure. function
1 Introduction Keywords: Self-modeling, Hopfield neural network, Hebbian learning, continuous activation function 1 Introduction 1 Introduction Hopfield neural network was first described by J. J. Hopfield in [2], and it is applied mainly in 1 Introduction two cases: associative memory [2] described and optimization [3]. In the case,itthe networkmainly learns in a Hopfield neural network was first by J. J. Hopfield in first [2], and is applied 1 Introduction Hopfield neural network waschanging first described by J.using J. Hopfield in first [2], and itthe is attractors applied mainly in set of training patterns by its weights a learning rule. The defined two cases: associative memory [2] and optimization [3]. In the case, network learns a Hopfield neural network was first by J. J. Hopfield in first [2], and is applied 1 Introduction two cases: associative memory [2] described and optimization [3]. In the case,itthe networkmainly learns in a
by the topology of thememory neural network correspond the training patterns, though there set of training patterns by weights a In learning rule. The attractors defined Hopfield neural network waschanging first by J.using J.toHopfield in first [2], and itthe iseven applied mainly in two cases: associative [2] described andits [3]. the case, network learns a set of training patterns by changing itsoptimization weights using a learning rule. The attractors defined exist spurious attractors which do not. In general, the latter kind of attractors are not desired, by the topology of the neural network correspond to the training patterns, even though there two cases: associative memory [2] and optimization [3]. In the first case, the network learns a Hopfield neural network was first described by J. J. Hopfield in [2], and it is applied mainly in set of training patterns by changing its weights using a learning rule. The attractors defined by the topology of the neural network correspond to the training patterns, even though there but they can be interpreted as a way of classification or generalization of a set of attractors exist spurious attractors which do not. In general, the latter kind of attractors are not desired, set of training patterns by changing its weights using a learning rule. The attractors defined two cases: associative [2] not. and optimization Intraining the first case, theeven network learns a by the topology of thememory neural network correspond to[3]. the patterns, there exist spurious attractors which do In classification general, the latter kind of attractors arethough desired, with similar the second case, the network weights define a The constraint satisfaction but they canpatterns. be interpreted as a way of or generalization of aeven set ofnot attractors by the topology of the In neural network correspond to the training patterns, though there set of training patterns by changing its weights using a learning rule. attractors defined exist spurious attractors which do not. In general, the latter kind of attractors are not desired, but they canpatterns. be interpreted a of way ofproblem. classification or generalization of a to setdetermine ofsatisfaction attractors problem between the the An energy function used the with similar In the as second case, the network weights define ais constraint exist spurious attractors which do not. In classification general, the latter kind of attractors arethough desired, by the topology of thecomponents neural network correspond to the training patterns, there but they canpatterns. be interpreted as a way of or generalization of aeven set ofnot attractors with similar In the second case, the network weights define a constraint satisfaction stability of the network, that is, if there exists such function, the network states will converge problem between the components of the problem. An energy function is used to determine the but they can be interpreted as a way of classification or generalization of a set of attractors exist spurious attractors do of not. Inproblem. general, the energy latter kind of attractors not desired, with similar patterns. Inwhich the second case, the network weights define ais constraint satisfaction problem between the components the An function used toare determine the into an attractor regardless of the initial conditions. The convergence of the network represents stability of the network, that is, if there exists such function, the network states will converge with similar patterns. In the second case, the network weights define a constraint satisfaction but they between can benetwork, interpreted as aifof way classification or generalization ofstates a to setdetermine of attractors problem the components theofproblem. Anfunction, energy function is used the stability of the that is, there exists such the network will converge the way that the activity ofofthe are coordinated sodefine as to constraints. into an attractor regardless theifcomponents initial conditions. convergence ofaissatisfy the network represents problem between the components ofthere the problem. AnThe energy function used tothe determine the with similar patterns. In the second case, the network weights constraint satisfaction stability of the network, that is, exists such function, the network states will converge into an attractor regardless ofthe theofcomponents initial conditions. Theconstraints convergence ofminimums the network represents Attractors are possible solutions the internal network and of the energy the way that the activity of are coordinated so as to satisfy the constraints. stability of the network, that is, if there exists such function, the network states will converge problem between the components of the conditions. problem. AnThe energy function used tothe determine the into an attractor regardless thecomponents initial convergence ofissatisfy the network represents the way that the activity ofofthe the are coordinated so asand to constraints. function. Thus, the value of energy isconditions. interpreted asconstraints the amount of constraints remain Attractors are possible solutions the internal network ofthat the energy into an attractor regardless ofthe theof initial The convergence ofminimums thestates network represents stability of the network, that is, if there exists such function, the network will converge the way that the activity of components are coordinated so as to satisfy the constraints. Attractors are possible solutions of the internal networkasconstraints and minimums of the remain energy unsatisfied. function. Thus, the value of energy isconditions. interpreted the amount of constraints the way that activity ofofthe the components are coordinated so asand to satisfy theofthat constraints. into an attractor regardless theof initial The convergence ofminimums the network represents Attractors arethe possible solutions the internal network constraints the energy function. Thus, the value of the energy is interpreted as the amount of constraints that remain unsatisfied. Attractors arethe possible solutions of the internal network constraints and minimums ofthat the remain energy the way that activity of the components are coordinated so as to satisfy the constraints. function. Thus, the value of the energy is interpreted as the amount of constraints unsatisfied. function. Thus, the value of the energy is interpreted as the amount of constraints that remain 1 Attractors are possible solutions of the internal network constraints and minimums of the energy unsatisfied. unsatisfied. 1 function. Thus, the value of the energy is interpreted as the amount of constraints that remain 1877-0509 © 2018 The Authors. Published by Elsevier Ltd. This is an open access article under the CC BY-NC-ND license 1 unsatisfied. 1 (http://creativecommons.org/licenses/by-nc-nd/3.0/). 1 Peer-review under responsibility of the scientific committee of the 8th Annual International Conference on Biologically Inspired Cognitive Architectures 1 10.1016/j.procs.2018.01.087
574
Self-modeling in Hopfield Neural Mario Networks .. Zarcowith et al.Continuous. / Procedia Computer Science 123 (2018) 573–578 Zarco and Froese
A Hopfield neural network can exhibit many different attractors, but only a small set of them are globally optimal. Watson, Buckley and Mills [8, 6] developed a process for enhancing the ability of a discrete time Hopfield neural network to find configuration that minimize constraints in tension, and therefore energy. The process is based on that the neural network forms an associative memory of its own attractors hence self-modeling its previous dynamic. Watson et al. used the following iterative algorithm: (1) the network states are initialized randomly, (2) the states are updated so as to the network converges into an attractor, and (3) after reaching the attractor, small changes in the weights are applied using Hebbian learning. The incremental Hebb rule strengthens connections which satisfy local constraints and weakens connections that do not. The self-modeling process increases the size of the basin of attraction of the more visited attractors due to the reinforcement done by Hebbian learning. Additionally, learning is able to cause a simple form of generalization producing attractors which are new combinations of similar patterns. These spurious attractors are possible solutions of the original optimization problem. Thus, the network can also converge into attractors which are unlikely to be visited and converge into new attractors formed through learning. This process allows the attractors outcompete each other in order to find one of the best solutions. The self-modeling process is grounded in two properties of the Hopfield neural network. First, there exists a positive correlation between the width and depth of a basin of attraction [5]. Second, associative memory allows generalizing over patterns learned. The reinforcement of an attractor at the same time can reinforce attractors with lower energy given that subpatterns that are common to many local optima can be common to superior optima. Thus, better attractors are reinforced more frequently due to learning, even if some of them have not been visited previously [8, 6]. We extend the self-modeling process using a discrete time Hopfield neural network with continuous activation function. This is not the first attempt of using a different activation function. A comparison between related works and the original one on self-modeling is presented in Table 1. In [7], a gene regulation network is presented using a discretized continuous-time recurrent neural network for defining the activation level of the network. In [9], a Hopfield neural network is emulated using a spiking neural network. The self-modeling process is applied in both works but with a different procedure such that in the first one the connections are modified according to a mutation function. The Hebbian changes to connections are consequence of the process since the network is evolved using a fitness function to evaluate the epistatic interactions. In the second one the Hebbian rule is used directly for changing the connections as in the original work and the work presented here. Reference [8, 6] [7] [9] This work
Activation Function Heaviside threshold function Sigmoid function (tanh(x/10)) Saturated linear function Sigmoid function ( 1+e2−x − 1)
States {−1, 1} (−1, 1) [0, 1] [−1, 1]
ωij
Weights (ωij ) −1 ≤ ωij ≤ 1 is not bounded −1 ≤ ωij ≤ 1 −1 ≤ ωij ≤ 1
Table 1: Differences between related works.
2 2.1
Methods Discrete-time Hopfield model
A Hopfield neural network is a fully-connected recurrent neural network, usually with symmetric connection matrix Ω(t), and without self-recurrent connections. The network consists of N 2
Self-modeling in Hopfield Neural Mario Networks .. Zarcowith et al.Continuous. / Procedia Computer Science 123 (2018) 573–578Zarco and Froese
discrete states si , with si ∈ {−1, 1}. The states are updated asynchronously according to the following equation: N si (t + 1) = θHT F ωij sj (t) , (1) j
where ωij is the weight between neuron i and j, with ωij ∈ [−1, 1], and θHT F is a Heaviside threshold function. So, if x > 0 then θHT F (x) = 1; else θHT F (x) = −1. The constraints on the connection matrix, i.e. ωij = ωji and ωii = 0, guarantee the existence of only fixed point attractors in the state space, which are minima of the energy function shown in equation (2), where H(t) represents the states configuration in t. N
E(H(t), Ω(t)) = −
2.2
1 ωij si (t)sj (t). 2 ij
(2)
Self-modeling in a Discrete-time Hopfield neural network
The self-modeling process is applied in a discrete-time Hopfield neural network constrained to ωij = ωji and ωii = 0 [6]. Initially, the states are randomized such that R = {−1|1}N , and the network is allowed to converge from a random configuration into an attractor. The process of convergence is called the relaxation of the network. Each relaxation last for τ time steps and the weights are changed either at every time step or at the end of the relaxation period according to the Hebbian rule: ωij (t + 1) = θLT F [ωij + δsi (t)sj (t)],
(3)
for all ωij , with i = j, where δ is the learning rate and θLT F is a linear threshold function. So, if x > 0 then θLT F (x) = 1; if x < 0 then θLT F (x) = −1; else θLT F (x) = x. The iterative process continues until the network converge into a single attractor from any initial configuration. The original energy, E 0 , is used to compute the degree to which a states configuration obtained by the self-modeling process successfully resolves the original constraints (a constraint is satisfied if ωij (t = 0)si sj > 0). This energy is calculated with the states configuration at the end of the relaxation period and the original weights, that is E 0 (H(t), Ω(t = 0)) = −
N
αij si (t)sj (t)
(4)
ij
where αij ≡ ωij (t = 0), and H(t) = s1 , ..., sN ∈ {−1, 1}n . If the process works properly, the energy of the attractors will be lower over time. That is, the energy of attractors reached during the process will be lower, not because any attractor is different from the original ones but because the basins of attraction of the better solutions will have come to dominate the state space. There exist three conditions for the process to work [6]: (C1) the initial dynamics of the system exhibit multiple point attractors; (C2) the states configuration are repeatedly relaxed from random initial conditions such that the system samples many different attractors on a timescale where connections change slowly; (C3) the system spends most of its time at attractors. As a consequence, there are two practical requirements: (R1) the learning rate must be small; (R2) the time of convergence into attractors during relaxation periods must be less than τ. 3
575
576
Self-modeling in Hopfield Neural Mario Networks .. Zarcowith et al.Continuous. / Procedia Computer Science 123 (2018) 573–578 Zarco and Froese
2.3
Discrete-time continuous-state Hopfield model
A discrete time, continuous state Hopfield neural network is a fully-connected recurrent network, usually with symmetric connection matrix Ω(t), and nonnegative self-recurrent connections. The network consists of N continuous states si , with si ∈ [−1, 1]. The states are updated asynchronously according to the following equation [4]: N si (t + 1) = σ ωij sj (t) , (5) j
where ωij is the weight between neuron i and j, with wij ∈ [−1, 1], and σ is a continuous function. It can be proved that the sigmoid function define by the next equation satisfies that σ is strictly increasing on an interval [a, b], with a = −∞, b = +∞, and limx→±∞ = ±1 [4]: 2 − 1. (6) 1 + ex The symmetricity constraint on the connection matrix guarantee the converge of the network into a fixed point attractor from any initial states configuration by asynchronous update. Attractors are minima of the energy function shown in equation (7), where H(t) represents the states configuration in t. σ(x) =
N
N
1 E(H(t), Ω(t)) = − ωij si (t)sj (t) + 2 ij i
2.4
si (t)
σ −1 (ξ)dξ.
(7)
0
Self-modeling in a Hopfield neural network with continuous activation function
A self-optimization process for a continuous-state version of the Hopfield neural network is useful given that “the non-convexity of the state space results in a high probability of falling into local minima” [1]. We use the following steps: (1) the states configuration is initially randomized such that R = [−1, 1]N , (2) the network is relaxed, during τ time steps, from a random configuration into an attractor of the state space defined by the modified weights, and (3) after finalizing the relaxation period, small changes in the weights are applied using the Hebbian rule: ωij (t + 1) = θLT F ωij (t) + δsri (t)srj (t)
(8)
for all ωij , where δ is the learning rate, θLT F is a linear threshold function as defined before, and sr (t) represents the attractors visited over the process which are not members of the original set defined by the original network weights. Ideally, the iterative process continues until the network converge into a single attractor from any initial configuration. As before, the original energy, E 0 , is calculated with the states configuration at the end of the relaxation period and the original weights, that is N
E 0 (H r (t), Ω(t = 0)) = −
N
1 αij sri (t)srj (t) + 2 ij i
sri (t)
σ −1 (ξ)dξ.
(9)
0
where αij ≡ ωij (t = 0), and H r (t) = sr1 , ..., srN ∈ [−1, 1]n . Finally, the three conditions, (C1), (C2), and (C3), and the two practical requirements, (R1), and (R2), explained before are still mandatory. 4
Self-modeling in Hopfield NeuralMario Networks .. Zarco with et al. Continuous. / Procedia Computer Science 123 (2018) 573–578 Zarco and Froese
3
Results
As in [6], we use two types of constraints for the connection matrix: (a) random constraints (RC): each initial connection αij = αji takes a value between [−1, 1] using a using a uniform random distribution, and (b) modular constraints (MC): each initial connection αij = αji takes the value |αij | = 1 if ki = kj ; else |αij | = 0.01. Here, k = 5, |αij | > 0 with probability 0.8, and there is not sign restriction on self-recurrence connections. We consider symmetric and asymmetric connection matrices. The asymmetric random matrix is computed adding noise to a symmetric random constraint matrix using a uniform random distribution between [−0.25, 0.25]. The asymmetric modular matrix is computed adding noise to a symmetric modular constraint matrix using a random distribution between [−0.01, −0.001] if |αij | < 0 or [0.001, 0.01] if |αij | > 0. The connection matrix is initialized using 50 different weights configurations per each type of constraint (symmetric RC, asymmetric RC, symmetric MC, and asymmetric MC), and the selfoptimization process was repeated for each one of them using 50 different initial random number seeds. We use N = 100, τ = 1000, δ = 0.00075 for the symmetric cases, and δ = 0.0001 for the asymmetric cases. The Figures 1 and 2 show the energy distribution and the satisfied constraints (αij si sj > 0), respectively, of 250, 000 relaxations for each of the four types of constraints. That is, the network was relaxed 100 times from 50 different initial states configuration using 50 different original weights configurations (before-learning), and from 50 different initial states configuration using 50 different modified weights configurations (after-learning).
Figure 1: Energy of attractors before and after self-modeling process when random constraints (left) and modular constraints (right) are used. Energy was measured using αij .
Figure 2: Satisfied constraints before and after self-modeling process when random constraints (left) and modular constraints (right) are used. Satisfied constraints were measured using αij .
5
577
578
Self-modeling in Hopfield NeuralMario Networks .. Zarco with et al. /Continuous. Procedia Computer Science 123 (2018) 573–578 Zarco and Froese
4
Conclusions
According to Figure 1, the energy of the attractors does not seem to be significantly smaller after the self-modeling process. This issue arises because applying Hebbian learning makes the attrators move towards the corners of the hypercube due to the fact that the activation function is not a hard-limit transfer function. So, small changes in weights increase the value of states which in turn increase the amount of learning, δsi (t)sj (t). Additionally, the integral term of equation (9) is always positive, and its value is maximum when the states of the evaluated configuration reach their maximum. Therefore, since the self-modeling process tends to saturate the states, the energy does not reflect if the process successfully resolves the original constraints because the energy not always decreases over time. Using this approach, the original energy will be different due to the fact that the attractors are different from the original ones. According to Figure 2, the number of satisfied constraints is better after applying the selfmodeling process to both random and modular constraints. The process works properly using symmetric and asymmetric weight matrices with self-recurrent connections. The best results are obtained when the connection matrix is structured (as can be seen in the graph of Figure 2), namely when it is not completely random, so that the process can exploit the correlation between local optima and superior optima. These promising results suggest that the selfmodeling process actually works as expected in Hopfield networks with sigmoid activation function. Future work should try to understand the role of the energy function and to address the problem of saturation. Acknowledgement.
This work was supported by UNAM-DGAPA-PAPIIT project IA104717.
References [1] Miguel Atencia Ruiz, Gonzalo Joya Caparr´ os, and Francisco Sandoval Hern´ andez. Two or three things that we (intend to) know about Hopfield and Tank networks. In 13th European Symposium on Artificial Neural Networks, 2005. [2] John J. Hopfield. Neural networks and physical systems with emergent collective computational abilities. Proceedings of the National Academy of Sciences of the United States of America, 79(8):2554– 2558, 1982. [3] John J. Hopfield and D. W. Tank. “Neural” computation of decisions in optimization problems. Biological Cybernetics, 52(3):141–152, 1985. [4] Pascal Koiran. Dynamics of discrete time, continuous state Hopfield networks. Neural Computation, 6(3):459–468, 1994. [5] Boris Kryzhanovsky and Vladimir Kryzhanovsky. Binary Optimization: On the Probability of a Local Minimum Detection in Random Search, pages 89–100. Springer Berlin Heidelberg, Berlin, Heidelberg, 2008. [6] Richard A. Watson, C. L. Buckley, and Rob Mills. Optimization in “self-modeling” complex adaptive systems. Complexity, 16(5):17–26, 2011. [7] Richard A. Watson, C. L. Buckley, Rob Mills, and Adam Davies. Associative memory in gene regulation networks. In Proceedings of the Twelfth International Conference on the Synthesis and Simulation of Living Systems (Artificial Life XII), pages 659–666. MIT Press, 2010. [8] Richard A. Watson, Rob Mills, and C. L. Buckley. Global adaptation in networks of selfish components: Emergent associative memory at the system scale. Artificial Life, 17(3):147–166, 2011. [9] Alexander Woodward, Tom Froese, and Takashi Ikegami. Neural coordination can be enhanced by occasional interruption of normal firing patterns: A self-optimizing spiking neural network model. Neural Networks, 62:39–46, 2015.
6
This This This This This This
space is reserved for the Procedia header, space is reserved for the Procedia header, space is reserved for the Procedia header, space is reserved for the Procedia header, Procedia 123Procedia (2018) 573–578 space is Computer reservedScience for the header, space is reserved for the Procedia header,
ScienceDirect
do do do do do do
not not not not not not
use use use use use use
it it it it it it
Self-modeling inonHopfield Networks with BICA 2017 8th Annual International Conference BiologicallyNeural Inspired Cognitive Architectures,
1 1 1 1
Self-modeling in Hopfield Neural Networks with Self-modeling in Hopfield Neural Networks with Continuous Activation Function Self-modeling in Hopfield Neural Networks with Continuous Activation Function Self-modeling in Hopfield Neural Networks with Continuous Activation Function 1 12 Mario Zarco and Tom Froese Continuous Activation Function Self-modeling in Hopfield Neural Networks with 1 12 Continuous Activation Function Mario Zarco 1 and Tom Froese12 Zarco Froese Intituto de Investigaciones enMario Matem´ aticasActivation Aplicas y en Sistemas, Universidad Nacional Aut´ onoma 1 and Tom 12 Continuous Function Mario Zarco and Tom Froese de M´exico, 1Mexico City, Mexico 12
Intituto de Investigaciones enMario Matem´ aticas Aplicas y en Sistemas, Nacional Aut´ onoma Zarco and Tom Froese Universidad Intituto de Investigaciones en Matem´ aeticas Aplicas yCity, en Sistemas, Universidad Nacional Aut´ onoma [email protected] de M´ xico, Mexico Mexico 1 12 Matem´ aeticas Aplicas yCity, en Sistemas, Universidad Nacional Aut´ onoma 2Intituto de Investigaciones en Mario Zarco and Tom Froese de M´ xico, Mexico Mexico 1 Centro de de Investigaciones Ciencias de la Complejidad, Universidad Nacional Aut´ oUniversidad noma de M´eNacional xico, Mexico [email protected] Intituto en Matem´ aeticas Aplicas yCity, en Sistemas, Aut´ oCity, noma de M´ xico, Mexico Mexico [email protected] 2 MexicoCity, Universidad Nacional Aut´ o noma de M´ e xico, Mexico City, 1 2 Centro de Ciencias de la Complejidad, de M´ e xico, Mexico Mexico [email protected] Intituto en Matem´ aticas Aplicas y Nacional en Sistemas, oUniversidad Aut´ oCity, noma Centro de de Investigaciones Ciencias de la Complejidad, Universidad noma de M´eNacional xico, Mexico [email protected] Mexico Nacional Aut´ 2 [email protected] Centro de Ciencias de la Complejidad, Universidad Aut´ onoma de M´exico, Mexico City, de M´ e xico, Mexico City, Mexico Mexico 2 [email protected] Centro de Ciencias de la Complejidad, Universidad onoma de M´exico, Mexico City, Mexico Nacional Aut´ [email protected] [email protected] 2 Mexico Nacional Aut´ [email protected] Centro de Ciencias de la Complejidad, Universidad onoma de M´exico, Mexico City, Abstract [email protected] Mexico Hopfield different attractors of which most are local optima. It Abstractnetworks can exhibit many [email protected] Abstract
has been demonstrated combining states randomization Hebbian learning Hopfield networks can that exhibit many different attractors of and which most are local enlarges optima. the It Abstract Hopfield networks can exhibit many different attractors of and which most are local enlarges optima. It basin of attraction of globally optimal attractors. The procedure is called self-modeling and it has been demonstrated that combining states randomization Hebbian learning the Abstract Hopfield networks can that exhibit many different attractors of and which most are local enlarges optima. the It has been demonstrated combining states randomization Hebbian learning has been applied in symmetric Hopfield networks with discrete states and without self-recurrent basin of attraction of globally optimal attractors. The procedure is called self-modeling and it Hopfield networks can exhibit many different attractors of which most are local optima. It Abstract has been demonstrated that combining states randomization and Hebbian learning enlarges basin of attraction of globally optimal attractors. The procedure is called self-modeling andthe it connections. We are interested in knowing which topological constraints can be relaxed. So, been applied in symmetric Hopfield networks with discrete states and without self-recurrent has demonstrated that combining states randomization and Hebbian learning enlarges the Hopfield networks can exhibit many different attractors of which most are local optima. It basin of attraction of globally optimal attractors. Thediscrete procedure is called self-modeling and it has been applied in symmetric Hopfield networks with states and without self-recurrent the self-modeling process is tested asymmetric Hopfield networks states and connections. We in are interested in in knowing which topological constraints can be self-recurrent relaxed. So, basin of attraction of globally optimal attractors. The procedure is with called self-modeling and it has been demonstrated that combining states randomization and Hebbian learning enlarges the applied symmetric Hopfield networks with discrete states and continuous without connections. We are interested in in knowing which topological constraints can be relaxed. So, self-recurrent connections. The best results are obtained in networks with modular structure. the self-modeling process is tested asymmetric Hopfield networks with continuous states and has been applied in symmetric Hopfield networks with discrete states and without self-recurrent basin of attraction of globally optimal attractors. The procedure is called self-modeling and it connections. We process are interested in in knowing whichHopfield topological constraints can be relaxed. So, the self-modeling isThe tested asymmetric networks with continuous states and self-recurrent connections. best results are obtained in networks with modular structure. connections. We are interested in knowing which topological constraints can be relaxed. So, has been applied in symmetric Hopfield networks with discrete states and without self-recurrent the self-modeling processHopfield isby tested inLtd. asymmetric Hopfield with continuous states and Keywords: Self-modeling, neural network, Hebbian learning, continuous activation function self-recurrent connections. The best results innetworks networks with structure. © 2018 The Authors. Published Elsevier This are is anobtained open access article under the CCmodular BY-NC-ND license the self-modeling process isThe tested in asymmetric Hopfield with states and connections. We are interested in knowing which topological constraints can be relaxed. So, self-recurrent connections. best results are obtained innetworks networks withcontinuous modular structure. Keywords: Self-modeling, Hopfield neural network, Hebbian learning, continuous activation function (http://creativecommons.org/licenses/by-nc-nd/3.0/). Keywords: Self-modeling, Hopfield neural network, Hebbian learning, continuous activation function self-recurrent connections. The best results are obtained in networks with modular structure. the self-modeling process is tested in asymmetric Hopfield networks with continuous states and Peer-review responsibilityHopfield of the scientific the 8th Annual International Conference on Biologically Keywords: under Self-modeling, neuralcommittee network,ofHebbian learning, continuous activation function Inspired Cognitive Architectures self-recurrent connections. The best results are obtained in networks with modular Keywords: Self-modeling, Hopfield neural network, Hebbian learning, continuous activationstructure. function
1 Introduction Keywords: Self-modeling, Hopfield neural network, Hebbian learning, continuous activation function 1 Introduction 1 Introduction Hopfield neural network was first described by J. J. Hopfield in [2], and it is applied mainly in 1 Introduction two cases: associative memory [2] described and optimization [3]. In the case,itthe networkmainly learns in a Hopfield neural network was first by J. J. Hopfield in first [2], and is applied 1 Introduction Hopfield neural network waschanging first described by J.using J. Hopfield in first [2], and itthe is attractors applied mainly in set of training patterns by its weights a learning rule. The defined two cases: associative memory [2] and optimization [3]. In the case, network learns a Hopfield neural network was first by J. J. Hopfield in first [2], and is applied 1 Introduction two cases: associative memory [2] described and optimization [3]. In the case,itthe networkmainly learns in a
by the topology of thememory neural network correspond the training patterns, though there set of training patterns by weights a In learning rule. The attractors defined Hopfield neural network waschanging first by J.using J.toHopfield in first [2], and itthe iseven applied mainly in two cases: associative [2] described andits [3]. the case, network learns a set of training patterns by changing itsoptimization weights using a learning rule. The attractors defined exist spurious attractors which do not. In general, the latter kind of attractors are not desired, by the topology of the neural network correspond to the training patterns, even though there two cases: associative memory [2] and optimization [3]. In the first case, the network learns a Hopfield neural network was first described by J. J. Hopfield in [2], and it is applied mainly in set of training patterns by changing its weights using a learning rule. The attractors defined by the topology of the neural network correspond to the training patterns, even though there but they can be interpreted as a way of classification or generalization of a set of attractors exist spurious attractors which do not. In general, the latter kind of attractors are not desired, set of training patterns by changing its weights using a learning rule. The attractors defined two cases: associative [2] not. and optimization Intraining the first case, theeven network learns a by the topology of thememory neural network correspond to[3]. the patterns, there exist spurious attractors which do In classification general, the latter kind of attractors arethough desired, with similar the second case, the network weights define a The constraint satisfaction but they canpatterns. be interpreted as a way of or generalization of aeven set ofnot attractors by the topology of the In neural network correspond to the training patterns, though there set of training patterns by changing its weights using a learning rule. attractors defined exist spurious attractors which do not. In general, the latter kind of attractors are not desired, but they canpatterns. be interpreted a of way ofproblem. classification or generalization of a to setdetermine ofsatisfaction attractors problem between the the An energy function used the with similar In the as second case, the network weights define ais constraint exist spurious attractors which do not. In classification general, the latter kind of attractors arethough desired, by the topology of thecomponents neural network correspond to the training patterns, there but they canpatterns. be interpreted as a way of or generalization of aeven set ofnot attractors with similar In the second case, the network weights define a constraint satisfaction stability of the network, that is, if there exists such function, the network states will converge problem between the components of the problem. An energy function is used to determine the but they can be interpreted as a way of classification or generalization of a set of attractors exist spurious attractors do of not. Inproblem. general, the energy latter kind of attractors not desired, with similar patterns. Inwhich the second case, the network weights define ais constraint satisfaction problem between the components the An function used toare determine the into an attractor regardless of the initial conditions. The convergence of the network represents stability of the network, that is, if there exists such function, the network states will converge with similar patterns. In the second case, the network weights define a constraint satisfaction but they between can benetwork, interpreted as aifof way classification or generalization ofstates a to setdetermine of attractors problem the components theofproblem. Anfunction, energy function is used the stability of the that is, there exists such the network will converge the way that the activity ofofthe are coordinated sodefine as to constraints. into an attractor regardless theifcomponents initial conditions. convergence ofaissatisfy the network represents problem between the components ofthere the problem. AnThe energy function used tothe determine the with similar patterns. In the second case, the network weights constraint satisfaction stability of the network, that is, exists such function, the network states will converge into an attractor regardless ofthe theofcomponents initial conditions. Theconstraints convergence ofminimums the network represents Attractors are possible solutions the internal network and of the energy the way that the activity of are coordinated so as to satisfy the constraints. stability of the network, that is, if there exists such function, the network states will converge problem between the components of the conditions. problem. AnThe energy function used tothe determine the into an attractor regardless thecomponents initial convergence ofissatisfy the network represents the way that the activity ofofthe the are coordinated so asand to constraints. function. Thus, the value of energy isconditions. interpreted asconstraints the amount of constraints remain Attractors are possible solutions the internal network ofthat the energy into an attractor regardless ofthe theof initial The convergence ofminimums thestates network represents stability of the network, that is, if there exists such function, the network will converge the way that the activity of components are coordinated so as to satisfy the constraints. Attractors are possible solutions of the internal networkasconstraints and minimums of the remain energy unsatisfied. function. Thus, the value of energy isconditions. interpreted the amount of constraints the way that activity ofofthe the components are coordinated so asand to satisfy theofthat constraints. into an attractor regardless theof initial The convergence ofminimums the network represents Attractors arethe possible solutions the internal network constraints the energy function. Thus, the value of the energy is interpreted as the amount of constraints that remain unsatisfied. Attractors arethe possible solutions of the internal network constraints and minimums ofthat the remain energy the way that activity of the components are coordinated so as to satisfy the constraints. function. Thus, the value of the energy is interpreted as the amount of constraints unsatisfied. function. Thus, the value of the energy is interpreted as the amount of constraints that remain 1 Attractors are possible solutions of the internal network constraints and minimums of the energy unsatisfied. unsatisfied. 1 function. Thus, the value of the energy is interpreted as the amount of constraints that remain 1877-0509 © 2018 The Authors. Published by Elsevier Ltd. This is an open access article under the CC BY-NC-ND license 1 unsatisfied. 1 (http://creativecommons.org/licenses/by-nc-nd/3.0/). 1 Peer-review under responsibility of the scientific committee of the 8th Annual International Conference on Biologically Inspired Cognitive Architectures 1 10.1016/j.procs.2018.01.087
574
Self-modeling in Hopfield Neural Mario Networks .. Zarcowith et al.Continuous. / Procedia Computer Science 123 (2018) 573–578 Zarco and Froese
A Hopfield neural network can exhibit many different attractors, but only a small set of them are globally optimal. Watson, Buckley and Mills [8, 6] developed a process for enhancing the ability of a discrete time Hopfield neural network to find configuration that minimize constraints in tension, and therefore energy. The process is based on that the neural network forms an associative memory of its own attractors hence self-modeling its previous dynamic. Watson et al. used the following iterative algorithm: (1) the network states are initialized randomly, (2) the states are updated so as to the network converges into an attractor, and (3) after reaching the attractor, small changes in the weights are applied using Hebbian learning. The incremental Hebb rule strengthens connections which satisfy local constraints and weakens connections that do not. The self-modeling process increases the size of the basin of attraction of the more visited attractors due to the reinforcement done by Hebbian learning. Additionally, learning is able to cause a simple form of generalization producing attractors which are new combinations of similar patterns. These spurious attractors are possible solutions of the original optimization problem. Thus, the network can also converge into attractors which are unlikely to be visited and converge into new attractors formed through learning. This process allows the attractors outcompete each other in order to find one of the best solutions. The self-modeling process is grounded in two properties of the Hopfield neural network. First, there exists a positive correlation between the width and depth of a basin of attraction [5]. Second, associative memory allows generalizing over patterns learned. The reinforcement of an attractor at the same time can reinforce attractors with lower energy given that subpatterns that are common to many local optima can be common to superior optima. Thus, better attractors are reinforced more frequently due to learning, even if some of them have not been visited previously [8, 6]. We extend the self-modeling process using a discrete time Hopfield neural network with continuous activation function. This is not the first attempt of using a different activation function. A comparison between related works and the original one on self-modeling is presented in Table 1. In [7], a gene regulation network is presented using a discretized continuous-time recurrent neural network for defining the activation level of the network. In [9], a Hopfield neural network is emulated using a spiking neural network. The self-modeling process is applied in both works but with a different procedure such that in the first one the connections are modified according to a mutation function. The Hebbian changes to connections are consequence of the process since the network is evolved using a fitness function to evaluate the epistatic interactions. In the second one the Hebbian rule is used directly for changing the connections as in the original work and the work presented here. Reference [8, 6] [7] [9] This work
Activation Function Heaviside threshold function Sigmoid function (tanh(x/10)) Saturated linear function Sigmoid function ( 1+e2−x − 1)
States {−1, 1} (−1, 1) [0, 1] [−1, 1]
ωij
Weights (ωij ) −1 ≤ ωij ≤ 1 is not bounded −1 ≤ ωij ≤ 1 −1 ≤ ωij ≤ 1
Table 1: Differences between related works.
2 2.1
Methods Discrete-time Hopfield model
A Hopfield neural network is a fully-connected recurrent neural network, usually with symmetric connection matrix Ω(t), and without self-recurrent connections. The network consists of N 2
Self-modeling in Hopfield Neural Mario Networks .. Zarcowith et al.Continuous. / Procedia Computer Science 123 (2018) 573–578Zarco and Froese
discrete states si , with si ∈ {−1, 1}. The states are updated asynchronously according to the following equation: N si (t + 1) = θHT F ωij sj (t) , (1) j
where ωij is the weight between neuron i and j, with ωij ∈ [−1, 1], and θHT F is a Heaviside threshold function. So, if x > 0 then θHT F (x) = 1; else θHT F (x) = −1. The constraints on the connection matrix, i.e. ωij = ωji and ωii = 0, guarantee the existence of only fixed point attractors in the state space, which are minima of the energy function shown in equation (2), where H(t) represents the states configuration in t. N
E(H(t), Ω(t)) = −
2.2
1 ωij si (t)sj (t). 2 ij
(2)
Self-modeling in a Discrete-time Hopfield neural network
The self-modeling process is applied in a discrete-time Hopfield neural network constrained to ωij = ωji and ωii = 0 [6]. Initially, the states are randomized such that R = {−1|1}N , and the network is allowed to converge from a random configuration into an attractor. The process of convergence is called the relaxation of the network. Each relaxation last for τ time steps and the weights are changed either at every time step or at the end of the relaxation period according to the Hebbian rule: ωij (t + 1) = θLT F [ωij + δsi (t)sj (t)],
(3)
for all ωij , with i = j, where δ is the learning rate and θLT F is a linear threshold function. So, if x > 0 then θLT F (x) = 1; if x < 0 then θLT F (x) = −1; else θLT F (x) = x. The iterative process continues until the network converge into a single attractor from any initial configuration. The original energy, E 0 , is used to compute the degree to which a states configuration obtained by the self-modeling process successfully resolves the original constraints (a constraint is satisfied if ωij (t = 0)si sj > 0). This energy is calculated with the states configuration at the end of the relaxation period and the original weights, that is E 0 (H(t), Ω(t = 0)) = −
N
αij si (t)sj (t)
(4)
ij
where αij ≡ ωij (t = 0), and H(t) = s1 , ..., sN ∈ {−1, 1}n . If the process works properly, the energy of the attractors will be lower over time. That is, the energy of attractors reached during the process will be lower, not because any attractor is different from the original ones but because the basins of attraction of the better solutions will have come to dominate the state space. There exist three conditions for the process to work [6]: (C1) the initial dynamics of the system exhibit multiple point attractors; (C2) the states configuration are repeatedly relaxed from random initial conditions such that the system samples many different attractors on a timescale where connections change slowly; (C3) the system spends most of its time at attractors. As a consequence, there are two practical requirements: (R1) the learning rate must be small; (R2) the time of convergence into attractors during relaxation periods must be less than τ. 3
575
576
Self-modeling in Hopfield Neural Mario Networks .. Zarcowith et al.Continuous. / Procedia Computer Science 123 (2018) 573–578 Zarco and Froese
2.3
Discrete-time continuous-state Hopfield model
A discrete time, continuous state Hopfield neural network is a fully-connected recurrent network, usually with symmetric connection matrix Ω(t), and nonnegative self-recurrent connections. The network consists of N continuous states si , with si ∈ [−1, 1]. The states are updated asynchronously according to the following equation [4]: N si (t + 1) = σ ωij sj (t) , (5) j
where ωij is the weight between neuron i and j, with wij ∈ [−1, 1], and σ is a continuous function. It can be proved that the sigmoid function define by the next equation satisfies that σ is strictly increasing on an interval [a, b], with a = −∞, b = +∞, and limx→±∞ = ±1 [4]: 2 − 1. (6) 1 + ex The symmetricity constraint on the connection matrix guarantee the converge of the network into a fixed point attractor from any initial states configuration by asynchronous update. Attractors are minima of the energy function shown in equation (7), where H(t) represents the states configuration in t. σ(x) =
N
N
1 E(H(t), Ω(t)) = − ωij si (t)sj (t) + 2 ij i
2.4
si (t)
σ −1 (ξ)dξ.
(7)
0
Self-modeling in a Hopfield neural network with continuous activation function
A self-optimization process for a continuous-state version of the Hopfield neural network is useful given that “the non-convexity of the state space results in a high probability of falling into local minima” [1]. We use the following steps: (1) the states configuration is initially randomized such that R = [−1, 1]N , (2) the network is relaxed, during τ time steps, from a random configuration into an attractor of the state space defined by the modified weights, and (3) after finalizing the relaxation period, small changes in the weights are applied using the Hebbian rule: ωij (t + 1) = θLT F ωij (t) + δsri (t)srj (t)
(8)
for all ωij , where δ is the learning rate, θLT F is a linear threshold function as defined before, and sr (t) represents the attractors visited over the process which are not members of the original set defined by the original network weights. Ideally, the iterative process continues until the network converge into a single attractor from any initial configuration. As before, the original energy, E 0 , is calculated with the states configuration at the end of the relaxation period and the original weights, that is N
E 0 (H r (t), Ω(t = 0)) = −
N
1 αij sri (t)srj (t) + 2 ij i
sri (t)
σ −1 (ξ)dξ.
(9)
0
where αij ≡ ωij (t = 0), and H r (t) = sr1 , ..., srN ∈ [−1, 1]n . Finally, the three conditions, (C1), (C2), and (C3), and the two practical requirements, (R1), and (R2), explained before are still mandatory. 4
Self-modeling in Hopfield NeuralMario Networks .. Zarco with et al. Continuous. / Procedia Computer Science 123 (2018) 573–578 Zarco and Froese
3
Results
As in [6], we use two types of constraints for the connection matrix: (a) random constraints (RC): each initial connection αij = αji takes a value between [−1, 1] using a using a uniform random distribution, and (b) modular constraints (MC): each initial connection αij = αji takes the value |αij | = 1 if ki = kj ; else |αij | = 0.01. Here, k = 5, |αij | > 0 with probability 0.8, and there is not sign restriction on self-recurrence connections. We consider symmetric and asymmetric connection matrices. The asymmetric random matrix is computed adding noise to a symmetric random constraint matrix using a uniform random distribution between [−0.25, 0.25]. The asymmetric modular matrix is computed adding noise to a symmetric modular constraint matrix using a random distribution between [−0.01, −0.001] if |αij | < 0 or [0.001, 0.01] if |αij | > 0. The connection matrix is initialized using 50 different weights configurations per each type of constraint (symmetric RC, asymmetric RC, symmetric MC, and asymmetric MC), and the selfoptimization process was repeated for each one of them using 50 different initial random number seeds. We use N = 100, τ = 1000, δ = 0.00075 for the symmetric cases, and δ = 0.0001 for the asymmetric cases. The Figures 1 and 2 show the energy distribution and the satisfied constraints (αij si sj > 0), respectively, of 250, 000 relaxations for each of the four types of constraints. That is, the network was relaxed 100 times from 50 different initial states configuration using 50 different original weights configurations (before-learning), and from 50 different initial states configuration using 50 different modified weights configurations (after-learning).
Figure 1: Energy of attractors before and after self-modeling process when random constraints (left) and modular constraints (right) are used. Energy was measured using αij .
Figure 2: Satisfied constraints before and after self-modeling process when random constraints (left) and modular constraints (right) are used. Satisfied constraints were measured using αij .
5
577
578
Self-modeling in Hopfield NeuralMario Networks .. Zarco with et al. /Continuous. Procedia Computer Science 123 (2018) 573–578 Zarco and Froese
4
Conclusions
According to Figure 1, the energy of the attractors does not seem to be significantly smaller after the self-modeling process. This issue arises because applying Hebbian learning makes the attrators move towards the corners of the hypercube due to the fact that the activation function is not a hard-limit transfer function. So, small changes in weights increase the value of states which in turn increase the amount of learning, δsi (t)sj (t). Additionally, the integral term of equation (9) is always positive, and its value is maximum when the states of the evaluated configuration reach their maximum. Therefore, since the self-modeling process tends to saturate the states, the energy does not reflect if the process successfully resolves the original constraints because the energy not always decreases over time. Using this approach, the original energy will be different due to the fact that the attractors are different from the original ones. According to Figure 2, the number of satisfied constraints is better after applying the selfmodeling process to both random and modular constraints. The process works properly using symmetric and asymmetric weight matrices with self-recurrent connections. The best results are obtained when the connection matrix is structured (as can be seen in the graph of Figure 2), namely when it is not completely random, so that the process can exploit the correlation between local optima and superior optima. These promising results suggest that the selfmodeling process actually works as expected in Hopfield networks with sigmoid activation function. Future work should try to understand the role of the energy function and to address the problem of saturation. Acknowledgement.
This work was supported by UNAM-DGAPA-PAPIIT project IA104717.
References [1] Miguel Atencia Ruiz, Gonzalo Joya Caparr´ os, and Francisco Sandoval Hern´ andez. Two or three things that we (intend to) know about Hopfield and Tank networks. In 13th European Symposium on Artificial Neural Networks, 2005. [2] John J. Hopfield. Neural networks and physical systems with emergent collective computational abilities. Proceedings of the National Academy of Sciences of the United States of America, 79(8):2554– 2558, 1982. [3] John J. Hopfield and D. W. Tank. “Neural” computation of decisions in optimization problems. Biological Cybernetics, 52(3):141–152, 1985. [4] Pascal Koiran. Dynamics of discrete time, continuous state Hopfield networks. Neural Computation, 6(3):459–468, 1994. [5] Boris Kryzhanovsky and Vladimir Kryzhanovsky. Binary Optimization: On the Probability of a Local Minimum Detection in Random Search, pages 89–100. Springer Berlin Heidelberg, Berlin, Heidelberg, 2008. [6] Richard A. Watson, C. L. Buckley, and Rob Mills. Optimization in “self-modeling” complex adaptive systems. Complexity, 16(5):17–26, 2011. [7] Richard A. Watson, C. L. Buckley, Rob Mills, and Adam Davies. Associative memory in gene regulation networks. In Proceedings of the Twelfth International Conference on the Synthesis and Simulation of Living Systems (Artificial Life XII), pages 659–666. MIT Press, 2010. [8] Richard A. Watson, Rob Mills, and C. L. Buckley. Global adaptation in networks of selfish components: Emergent associative memory at the system scale. Artificial Life, 17(3):147–166, 2011. [9] Alexander Woodward, Tom Froese, and Takashi Ikegami. Neural coordination can be enhanced by occasional interruption of normal firing patterns: A self-optimizing spiking neural network model. Neural Networks, 62:39–46, 2015.
6