- Email: [email protected]
Effective mutation rate for probabilistic evolutionary design of analogue electrical circuits
Applied Soft Computing 7 (2007) 1012–1018 www.elsevier.com/locate/asoc
Effective mutation rate for probabilistic evolutionary design of analogue electrical circuits Heinz Mu¨hlenbein a,1, Lyudmilla Zinchenko b,*, Victor Kureichik b,2, Thilo Mahnig a,1 b
a FhG-AiS Schloss Birlinghoven 53754 Sankt, Augustin, Germany Taganrog State University of Radio Engineering, l. Nekrasovsky, 44, GSP-17A, Taganrog 347928, Russia
Received 4 April 2003; accepted 11 November 2003 Available online 17 October 2006
Abstract The paper represents the approach to evolutionary analogue circuit design on the base of the univariate marginal distribution algorithm. In order to generate a new population the probability distribution is used instead of reproduction operators. It allows us to control evolvability of a population on mesoscopic level. Experimental results obtained have indicated that a high mutation rate increases the success rate, although computational costs are increased too. The effective mutation rate that supplies high success rate and small computational costs is examined for different weightings of the fitness function. # 2006 Published by Elsevier B.V. Keywords: Evolutionary design; Electronic circuits; Probabilistic models; Bayesian prior
1. Introduction Genetic algorithms are powerful tools that have been successfully applied to circuit design. Their main operators are selection and reproduction with variations. They work on a microscopic (individual) level. This restriction is often the cause for a premature convergence. Several approaches have been proposed in order to overcome this deficiency. In ref. [13], Mu¨hlenbein showed that genetic algorithms can be approximated by an algorithm using univariate marginal distribution only. Univariate marginal distribution algorithm (UMDA) works on mesoscopic level. The algorithm selects individuals on microscopic level, while it uses marginal distribution in order to generate a new population. Note that the marginal distribution is a macroscopic variable. Evolvability is controlled by the marginal distribution that changes during evolutionary process.
* Corresponding author. Present address: FhG-AiS Schloss Birlinghoven 53754 Sankt, Augustin, Germany. Tel.: +7 8634 37 16 94 (Russia)/ +49 2241 14 20 57 (Germany); fax: +7 8634 31 05 98 (Russia)/ +49 2241 14 23 84 (Germany). E-mail addresses: [email protected] (H. Mu¨hlenbein), [email protected] (L. Zinchenko), [email protected] (V. Kureichik), [email protected] (T. Mahnig). 1 Tel.: +49 2241 14 24 05; fax: +49 2241 14 28 89. 2 Tel.: +7 8634 37 16 94; fax: +7 8634 31 05 98. 1568-4946/$ – see front matter # 2006 Published by Elsevier B.V. doi:10.1016/j.asoc.2006.07.001
At each step the probability is evaluated using the best-selected individuals. Therefore, the average fitness function never decreases [13]. It allows us to accumulate the information about perspective regions of the fitness function landscape. Probabilistic models have been applied to different problems successfully. In this paper, we restrict our research to analogue circuit design. Unfortunately, it is impossible to use theoretical framework similar [14] for analysis of evolutionary circuit design because of its mathematical complexity. However, high population size leads to high computational costs, while small population can result in a premature convergence. Therefore, extensive experimental researches are required. In ref. [14], Mu¨hlenbein and Mahnig have extended UMDA by means of Bayesian prior. It reduces the requirement to an effective choice of a population size. However, the correct choice of the effective mutation rate turned out to be difficult for evolutionary circuit design. If we use the very high mutation rate then a search became random. If we use UMDA with low mutation rate then the algorithm is unable to overcome local optima. Therefore, we have to evaluate an effective mutation rate in order to find the equilibrium between the success rate and computational costs. We consider the successful runs after the best solution has been found only. Moreover, the effective mutation rate and a fitness landscape are dependent. Therefore, we have to extend our research to different weightings of the fitness function.
H. Mu¨hlenbein et al. / Applied Soft Computing 7 (2007) 1012–1018
The remainder of the paper is organized as follows. Section 2 summarizes recent trends in evolutionary circuit design briefly. Section 3 presents the binary genotype–phenotype mapping used. Experimental results obtained for different weightings of the fitness function are discussed in Section 4. It turns out that mutation not only makes UMDA more efficient in many cases, but also more robust in the sense that the choice of the correct parameters is of less importance. 2. Related works 2.1. Knowledge-based approaches There are several approaches to circuit design. Many researches are restricted to circuit creation based on expert knowledge [18,19]. Other methods consider circuit optimisation only [5,20–22]. However, a good technical decision lies within huge design space, which includes all possible circuits [23,24]. Its complexity is evaluated by the amount of elements that are used, the rules, that were introduced in order to generate a circuit and an application for that the circuit is being evolved. Evolutionary algorithms provide a non-heuristic investigation in such complex design spaces. However, if we consider the circuit creation and the circuit optimisation separately, then design solutions can be concentrated within a small part of fitness function landscape. Several approaches have been applied in order to combine circuit creation and circuit optimisation for filters design [6,16], transducers [7,8,10], etc. However, fixed pre-defined topologies restrict a possible technical decision. 2.2. Creative topologies approaches The use of genetic programming [4] introduced novel direction for circuit design that is called developmental synthesis. Unlike the methods mentioned above a circuit topology, types of components, its values are evolved from an embryonic circuit (a set of short circuits) to a full circuit. However, genetic programming approach generates circuits that are highly redundant as a result of knowledge absence about a problem solved. Because of the nature of the design technique the most part of circuits generated at the initial steps are faulty circuits. The use of transformation rules (cc-bot instructions) allows us to generate circuits with valid graphs [9]. However, a set of required instructions is increased significantly. Therefore, the application of the developmental synthesis is limited by problem complexity. A simulation of useful and faulty circuits results in the increase of computational costs. In order to enhance evolutionary design effectiveness starting points that are included in the initial population are used [2]. Computational costs are decreased significantly. This approach using the combination of evolutionary design and heuristics is effective. It is remarkable that preliminary knowledge reduces computational costs. However, stable attractors are possible around a starting point. Hence, local optima can be found when a landscape has saddle points in the initial area.
1013
2.3. Evolutionary probabilistic design The most important feature in evolutionary design is a choice of the search algorithm. The main requirements are its speed, flexibility and ability to investigate a solution space effectively. The most design steps are iterated in design flow. Therefore, an algorithm should offer a choice of good technical decisions that have to generate rapidly. Positive results have been obtained by means of combined genetic/annealing optimisation algorithm [5], hybrid genetic algorithm [3], etc. Our focus is on evolutionary probabilistic algorithms application. Evolutionary algorithms based on probabilistic models have been recognized as a new computing paradigm in evolutionary computation. They can be divided into two categories: univariate distribution algorithms, like population-based incremental learning algorithm and univariate marginal distribution algorithm and bivariate distribution algorithms. Several typical examples of probabilistic models are genetic quantum algorithm, ant colony algorithm, factorised distribution algorithm, compact genetic algorithm, etc. The probabilistic approach defines the direct connection between a genotype and its fitness. Therefore, a fitness function plays the crucial role for probabilistic evolutionary algorithms. Probabilistic algorithms have been applied successfully to a travelling salesman problem, the quadratic assignment problem, etc. Recently, ant colony algorithm has been used for analogue circuit design successfully [15]. At the initial step, a gain value of each cell can be chosen from 0 to 1. Each new generation restricts the potential change of the gain value. However, possible combinations of PALMO cells restrict circuit topologies. UMDA [13] is an evolutionary algorithm that combines mutation and recombination by using probabilistic distribution. The distribution is estimated from a set of selected points M. It is then used to generate new points for the next generation according to the probability p(X, t) in the population at the generation t pðX; tÞ ¼
n Y
psi ðxi ; tÞ;
i¼1
where n is equal to the genotype length. UMDA Step 1. Set t = 1. Generate N possible solutions randomly. Step 2. Select M < N solutions according to a selection method. Step 3. Compute marginal frequencies psi ðxi ; tÞof the selected set. Step 4. Generate N new solutions according to the marginal Q distribution pðX; t þ 1Þ ¼ ni¼1 psi ðxi ; tÞ: Set t = t + 1. Step 5. If termination criteria are not met, go to Step 2. Different selection methods have been investigated in ref. [12]. In this paper, our focus is on the truncation selection, where t = M/N is the amount of selected individuals.
H. Mu¨hlenbein et al. / Applied Soft Computing 7 (2007) 1012–1018
1014
In ref. [14], mutation has been introduced into UMDA by a concept called Bayesian prior. In practice, this means that the estimated marginal frequency psi ðxi ; tÞ is calculated as follows: psi ðxi ; tÞ ¼
mþr ; Nt þ 2r
where m is equal to the number of bits with value 1 in the selected parents in i loci, N the population size and r is the mutation rate. Experimental results for several test functions have been published in ref. [14]. In ref. [11], we have introduced the hierarchical technique for evolutionary circuit design. Thus, we were able to generate novel analogue circuits. The main difference between our approach and related works is a combination of hierarchical design at element, topology and parameter levels and UMDA. A comparison of different circuit representations has been published in ref. [17]. In this paper, we aim to evaluate the effective mutation rate. We examine the effective mutation rate for different weightings of the fitness function. It should be noted that we restrict our research to static fitness function schedule and binary genotype–phenotype mapping. 3. Evolutionary hierarchical design The hierarchical construction of design space is used to generate circuits with required properties. An overview of the proposed design technique has been published in ref. [11]. The approach is effective because of its direct genotype decoding to a system of equations. We use the modified nodal method likely to SPICE approach [11]. The growing technique is used during the evolutionary design. The embryonic circuit is shown in Fig. 1b. The evolutionary process is based on the topology of the complete graph. Adjacency matrix of the N-node complete graph is given as follows [1]: ai j ¼ 0; if i ¼ j A¼ : (1) ai j ¼ 1; if i 6¼ j
In order to reduce a number of incorrect solutions our design flow is described by means of separate sets analogously DNA. Therefore, each solution is represented by hierarchical set, including types of elements and parameters. The first level of the set is an amount of the complete graph nodes. The second level is defined by the correspondent sequence of the elements from the set ELEMENT = {S, R, L, C}, etc. We restrict our research to the two-terminal components only. Alleles for components’ type are S, R, L, C, that represent switches, resistors, inductors, capacitors, etc. The third level is parameter one. In our hierarchical coding the circuit is termed as a phenotype. The genotype is defined as a linear structure with a variable length: NEn. . .E2E1E0 Pk. . .P2P1P0, where N is the amount of complete graph nodes; En. . .E2E1E0 the part of the genotype that defines the evolution of elements; Pk. . .P2P1P0 the part of the genotype that describes parameter evolution (changes parameter elements without changes of topology); n the amount of the circuit elements; k is the amount of circuit element parameters. Notice, that when we use the complete graph topology, the amount of the circuit elements is given as follows: n¼
NðN þ 1Þ : 2
(2)
Analogue circuits are hard to manufacture. Only fixed parameters are possible for circuit manufacturing. Therefore, parameters can be changed as discrete values only. Therefore, the chromosome hierarchy structure is defined as the discrete set. Each element is determined by a type and a parameter. The first and the second bits of each group define a type. Each element can take on four different types, namely, switch (00), capacitance (01), inductance (10) and resistors (11). The next bits define a component sizing. The parameter of switch can take on two fixed values only. We consider that 0 means that a switch is turned off and 1 means that a switch is turned on. A
Fig. 1. Chromosome (a), embryonic circuit (b) and the circuit, created from it (c).
H. Mu¨hlenbein et al. / Applied Soft Computing 7 (2007) 1012–1018
state of switches defines a circuit topology. Parameters of inductances, resistors and capacitances can be chosen within the fixed set. The input of the circuit is always node 1 and the output is node 2. The branch 1 is the series connection of a voltage source and the source resistance always. The branch 2 is the load resistance. A circuit being evolved contains (n2) branches. Fig. 1 shows the chromosome of length 23 and the circuit created from it. Our embryonic circuit evolves according to design requirements. First, a user specifies the boundary limits for the amount of nodes. Second, the set EG is formed by two subsets {E1, E2}, where subset E1 includes elements that represent a structure of the initial solution; subset E2 includes elements that describe an evolutionary development. All elements of the subset E2 are set to 0 in the initial population, hence, the initial state of the set EG is described as follows: {E1, 0}. The initial population can be generated randomly, by means of user’s experience or a textbook. Each element of the set EG can change randomly during evolutionary design. Then the set P that defines parameter evolution changes automatically. There are two possible cases. If an element, which represents in E1, changes to 0, then the corresponding elements in set P are set to zero. In the second case, which represents a change of an element from 0 to any element from the set ELEMENT, the corresponding elements are obtained in the set P being element parameters. Restrictions are the fixed amount of switches defined by the user. Second, elements of the set P can be changed within the discrete set randomly. Restrictions are the variations of parameters within the fixed set. It should be noted that set P consists of two subsets {P1, P2}. Allowed constants, defining one possible phenotype form represent elements of the first subset. P2 is a subset of constants that defines a few forms of phenotype. Therefore, subset P2 contains parameters that can be changed. The relationship between constants of the subset and variables in the phenotype to be satisfied defines the variable domains. All constraints are defined by heuristics. Furthermore, the best solution of topological design level is used as an initial circuit for the following parametrical optimisation. At this step all bits correspond to element parameters, therefore, more flexible search within fixed topology is supplied. This type of connection between the genotype and the phenotype is the dynamic coding, that allows us to exploit evolution of genotype and phenotype simultaneously. Various forms of connection between genotype and phenotype are possible by means of P1 and P2 evolution. Subset P1 defines only one possible form, while subset P2 establishes the dynamic relation between a genotype and a phenotype. One form of genotype corresponds to a few forms of phenotype. Therefore, evolution at the phenotype level is possible for the dynamic coding. The use of the hierarchy for chromosome coding eliminates useless search and reduces the design space dimension. It should be noted that a fitness landscape has many local optima for circuit design [7]. Premature convergence can be overcome by means of huge population sizes [4,9], although large computation costs are required. We aim to remedy this deficiency by using the effective search algorithm.
1015
4. Experimental results In ref. [11], we have published the experimental results for the evolutionary circuit design that uses UMDA and elitism. However, the results were obtained for the single fitness landscape. In order to make a fair comparison, we have to compare actual simulation results for different weightings of the fitness function. We use the low pass filter design as the test benchmark. The starting point for the design of a filter is the specification by the user of the frequencies for the filter’s pass band and stop band. The experiment described here is based on ‘‘synthesis of low pass filter’’ [4]. We consider the filter design with a pass band below 1 kHz and a stop band above 2 kHz. It is acceptable if the output voltage in the pass band is from 970 mV to 1 V. It is acceptable if the output voltage in the stop band is from 0 V to 1 mV. The ratio in the pass band (1 V) to the maximum acceptable signal in the stop band (1 mV) is 60 dB, but there is a two-to-one ratio between the stop band frequency and the pass band frequency. In order to reduce computational costs the fitness is evaluated by means of the equation-based method [11]. The numerical frequency response evaluation and the modified nodal method are used. The fitness function evaluation technique used is similar to those described in ref. [11]. The fitness is evaluated in terms of the sum of the absolute weighted difference between the actual output voltage and the target one. These error values were summed across evaluation points Nc for the evaluation of the fitness: F¼
Nc X W½dð f i Þ; f i Þdð f i Þ; i¼0
dð f Þ ¼ jU goal ð8 f Þ U out ð f Þj; 1; if dð f Þ 0:03; f 1000 Hz; > > < w1 ; if dð f Þ 0:03; f 1000 Hz; W½dð f Þ; f ¼ 1; if dð f Þ 0:001; f 2000 Hz; > > : w2 ; if dð f Þ 0:001; f 2000 Hz: (3) where f is the frequency of the fitness evaluation, d( f) the absolute value of the difference between the output voltage and the target at the frequency f and W[d( f), f] is the penalty factor for the difference d( f) at the frequency f. We use the set of the circuits that can be generated by the topology of the complete three-node graph as the simplest test function. The best circuit is given in Fig. 1c. Amount of hits is equal to 86. The stopping criterion is the best solution finding. The graphs in Figs. 2 and 3 show how computational costs are changed when the population size and the truncation threshold are modified. Fig. 2 illustrates the case in which the population size is varying and t = 0.02. The case in which the truncation threshold t is varying and the population size N = 300 is shown in Fig. 3. It is obvious that the curves have approximately minimum, when rn ¼ 44:5; I Nt
(4)
1016
H. Mu¨hlenbein et al. / Applied Soft Computing 7 (2007) 1012–1018
Fig. 4. Performance comparison with different mutation rate for population size N = 300 and truncation selection with t = 0.02. FE is the number of evaluation before the best solution has been found.
Fig. 2. Performance comparison for several parameters. FE is the number of evaluation before the best solution has been found.
where I is the selection intensity that depends on a truncation threshold of t [13]. The results were obtained for the penalty coefficients w1 ¼ w2 ¼ 10: Furthermore, we evaluate success rates and computational costs for different weightings of the fitness function. Figs. 4–11 summarize the experimental results obtained for several penalty coefficients w1 , w2 . Figs. 4, 5, 8 and 9 show the results for varying population size N from 300 to 500 and t = 0.02. Figs. 6, 7, 10 and 11 illustrate the case, when the truncation threshold is increased to t = 0.04. The most beneficial result is that the mutation changes success rate in the same manner for different weightings of the fitness function. When the mutation rate is high, the number of function evaluations increases, as the probabilities are shifted towards 1/ 2 too much. When the mutation rate decreases, the success rate
Fig. 5. Performance comparison with different mutation rate for population size N = 300 and truncation selection with t = 0.02. Succ is the number of times, when the best solution was found in 100 runs.
Fig. 6. Performance comparison with different mutation rate for population size N = 300 and truncation selection with t = 0.04. FE is the number of evaluation before the best solution has been found.
decreases also, because it becomes too improbable to flip the remaining bits that are wrong. The choice a small population size leads to a small success rate, while the choice of big population size is a waste of function evaluations. It seems to us that both effects are overcome by using the effective mutation rate (4).
Fig. 3. Performance comparison for several parameters. FE is the number of evaluation before the best solution has been found.
Fig. 7. Performance comparison with different mutation rate for population size N = 300 and truncation selection with t = 0.04. Succ is the number of times, when the best solution was found in 100 runs.
H. Mu¨hlenbein et al. / Applied Soft Computing 7 (2007) 1012–1018
Fig. 8. Performance comparison with different mutation rate for population size N = 500 and truncation selection with t = 0.02. FE is the number of evaluation before the best solution has been found.
Fig. 9. Performance comparison with different mutation rate for population size N = 500 and truncation selection with t = 0.02. Succ is the number of times, when the best solution was found in 100 runs.
1017
Fig. 10. Performance comparison with different mutation rate for population size N = 500 and truncation selection with t = 0.04. FE is the number of evaluation before the best solution has been found.
Fig. 11. Performance comparison with different mutation rate for population size N = 500 and truncation selection with t = 0.04. Succ is the number of times, when the best solution was found in 100 runs.
Fig. 12. The best circuit (a) and the frequency response of the filter (b).
Moreover, the success rate is higher for the penalty coefficients w1 ¼ 10, w2 ¼ 15 and smaller truncation threshold. However, if truncation threshold is increased to 0.04, then higher success rate has been obtained for the penalty coefficients w1 ¼ 10, w2 ¼ 20. Therefore, success rate depends from truncation threshold and weightings of the fitness function. Furthermore, we use the effective mutation rate in order to design analogue cells for Field Programmable Analogue Arrays. The best circuit that was found for the four-node
graph is shown in Fig. 12a and b illustrates that the frequency response falls within the target specifications. We used PSPICE for the evaluation of the best circuit characteristic. 5. Conclusions Distribution algorithms control evolvability of a population on a mesoscopic level. Probability distributions are evaluated from the set of the best elements. Therefore, the information obtained at each step of evolutionary design is reserved for the
1018
H. Mu¨hlenbein et al. / Applied Soft Computing 7 (2007) 1012–1018
following generations. However, the evolutionary probabilistic circuit design approach requires the extensive research to ensure that the most effective design is exploited. The results have shown that distribution algorithms with mutation are both fast and requiring small computational costs. The effective mutation rate has shown to be better in computational complexity. The effective mutation rate was examined for three different weightings of the fitness function. Varying penalty coefficients change the fitness function landscape significantly. However, the behaviour of the univariate marginal distribution algorithm with the effective mutation rate is similar for different weightings of the fitness function. It should be noted that fitness function landscape and truncation threshold are crucial ingredients for high success rate. However, the effective mutation rate makes the correct choice of population size and truncation threshold is of less important. Acknowledgments This work was supported by INTAS (International Association for the promotion of co-operation with scientists from the New Independent States of the former Soviet Union) by grant No YSF 00-190. We would like to thank all members of the RWCP Group, FhG-AIS for their support, advice and criticism of this work. Lyudmilla Zinchenko and Victor Kureichik acknowledge support by Russian Fund of Basic Research (grant 04-0100174). References [1] N. Biggs, Algebraic Graph Theory, Cambridge, 1974. [2] C. Goh, Y. Li, GA automated design and synthesis of analog circuits with practical constraints, in: Proceedings IEEE International Conference of Evolution Computation, IEEE Press, 2001, pp. 170–177. [3] J.B. Grimbleby, Hybrid genetic algorithms for analogue network synthesis, in: Proceedings IEEE International Conference of Evolution Computation, IEEE Press, 1999, pp. 1781–1787. [4] J.R. Koza, F.H. Bennett, D. Andre, M.A. Keane, Genetic Programming III: Darwinian Invention and Problem Solving, Morgan Kaufmann, San Francisco, CA, 1999. [5] M. Krasnicki, R. Phelps, R.A. Rutenbar, L.R. Carley, Maelstrom: efficient simulation-based synthesis for custom analog cells, in: Proceeding ACM/ IEEE DAC, 1999. [6] W. Kruiskamp, D. Leenaetrs, Darwin: CMOS OpAmp Synthesis by means of Genetic Algorithms, in: Proceeding ACM/IEEE DAC, 1995. [7] V.M. Kureichik, L.A. Zinchenko, Evolutionary modelling with hierarchy in innovative computer-aided circuit design, IETE J. Res. 5 (2002) 361– 367.
[8] V.M. Kureichik, L.A. Zinchenko, Evolution design of electronic devices, in: Proceedings ICECS 2000, 2000, pp. 879–882. [9] J.D. Lohn, S.P. Colombano, A circuit representation technique for automated circuit design, IEEE Trans. Evol. Comput. 3 (1999) 205–219. [10] H. Mu¨hlenbein, V.M. Kureichik, T. Mahnig, L.A. Zinchenko, Evolutionary modelling with hierarchy and dynamic coding in computer-aided design, in: K.C. Giannakoglou, T. Fogarty (Eds.), Evolutionary Methods for Design, Optimization and Control with Applications to Industrial Problems, CIMNE, Spain, 2002, pp. 202–207. [11] H. Mu¨hlenbein, V.M. Kureichik, T. Mahnig, L.A. Zinchenko, Application of the univariate marginal distribution algorithm to analog circuit design, in: Proceedings of the Evolvable Hardware Conference, Computer Press, 2002, pp. 93–101. [12] H. Mu¨hlenbein, T. Mahnig, Comparing the adaptive boltzmann selection schedule SDS to truncation selection, in: Proceedings of the 3rd International Symposium on Adaptive Systems, 2001, pp. 121–128. [13] H. Mu¨hlenbein, The equation for response to selection and its use for prediction, Evol. Comput. 5 (1998) 303–346. [14] H. Mu¨hlenbein, T. Mahnig, Evolutionary computation and Wright’s equation, Theor. Comput. Sci. 287 (2002) 145–165. [15] M.R. Tamplin, A. Hamilton, Ant circuit world: an Ant algorithm MATHLAB toolbox for the design, visualization and analysis of analogue circuits, in: Proceedings of the International Conference on Evolvable Systems: From Biology to Hardware. Lecture Notes in Computer Science, Springer-Verlag, Berlin, Heidelberg, New York, 2001. [16] R.S. Zebulum, M.A. Pacheho, M. Vellasco, Comparison of different evolutionary methodologies applied to electronic filter design, in: Proceedings’1998 IEEE International Conference of Evolutionary Computation, IEEE Press, 1998, pp. 434–439. [17] L.A. Zinchenko, H. Muehlenbein, V.M. Kureichik, T. Mahnig, A Comparison of different circuit representations for evolutionary analog circuit design, in: Proceedings of the International Conference on Evolvable Systems: From Biology to Hardware. Lecture Notes in Computer Science, Springer-Verlag, Berlin, Heidelberg, New York, 2003. [18] F. El-Turkey, E.E. Perry, BLADES: an artificial intelligence approach to analog circuit design, IEEE Trans. Comput. Aided Des. Integr. Circuits Syst. 6 (1989) 680–692. [19] R. Harjani, R.A. Rutenbar, L.R. Carley, OASYS: a framework for analog circuit synthesis, IEEE Trans. Comput. Aided Des. Integr. Circuits Syst. 12 (1989) 1247–1266. [20] E.S. Ochotta, R.A. Rutenbar, L.R. Carley, Synthesis of high-performance analog circuits in ASTRX/OBLX, IEEE Trans. Comput. Aided Des. Integr. Circuits Syst. 3 (1996) 273–294. [21] G.G.E. Gielen, H.C.C. Walscharts, W.M.C. Sansen, Analog circuit design optimization based on symbolic simulation and simulated annealing, IEEE J. Solid State Circuits 3 (1990) 707–713. [22] R. Phelps, M. Krasnicki, R.A. Rutenbar, L.R. Carley, J.R. Hellums, ANACONDA: simulation based synthesis of analog circuits via stochastic pattern search, IEEE Trans. Comput. Aided Des. Integr. Circuits Syst. 6 (2000) 703–717. [23] Van der Plas, G. Debyser, F. Leyn, K. Lampaert, J. Vandenbussche, G.E. Gielen, W. Sansen, P. Vaselinovic, D. Leenaerts, AMGIE—a synthesis environment for CMOS analog integrated circuits, IEEE Trans. Comput. Aided Des. Integr. Circuits Syst. 9 (2001) 1037–1058. [24] H.Y. Koh, C.H. Sequin, P.R. Gray, OPASYN: a compiler for CMOS operational amplifiers, IEEE Trans. Comput. Aided Des. Integr. Circuits Syst. 2 (1990) 113–125.
Effective mutation rate for probabilistic evolutionary design of analogue electrical circuits Heinz Mu¨hlenbein a,1, Lyudmilla Zinchenko b,*, Victor Kureichik b,2, Thilo Mahnig a,1 b
a FhG-AiS Schloss Birlinghoven 53754 Sankt, Augustin, Germany Taganrog State University of Radio Engineering, l. Nekrasovsky, 44, GSP-17A, Taganrog 347928, Russia
Received 4 April 2003; accepted 11 November 2003 Available online 17 October 2006
Abstract The paper represents the approach to evolutionary analogue circuit design on the base of the univariate marginal distribution algorithm. In order to generate a new population the probability distribution is used instead of reproduction operators. It allows us to control evolvability of a population on mesoscopic level. Experimental results obtained have indicated that a high mutation rate increases the success rate, although computational costs are increased too. The effective mutation rate that supplies high success rate and small computational costs is examined for different weightings of the fitness function. # 2006 Published by Elsevier B.V. Keywords: Evolutionary design; Electronic circuits; Probabilistic models; Bayesian prior
1. Introduction Genetic algorithms are powerful tools that have been successfully applied to circuit design. Their main operators are selection and reproduction with variations. They work on a microscopic (individual) level. This restriction is often the cause for a premature convergence. Several approaches have been proposed in order to overcome this deficiency. In ref. [13], Mu¨hlenbein showed that genetic algorithms can be approximated by an algorithm using univariate marginal distribution only. Univariate marginal distribution algorithm (UMDA) works on mesoscopic level. The algorithm selects individuals on microscopic level, while it uses marginal distribution in order to generate a new population. Note that the marginal distribution is a macroscopic variable. Evolvability is controlled by the marginal distribution that changes during evolutionary process.
* Corresponding author. Present address: FhG-AiS Schloss Birlinghoven 53754 Sankt, Augustin, Germany. Tel.: +7 8634 37 16 94 (Russia)/ +49 2241 14 20 57 (Germany); fax: +7 8634 31 05 98 (Russia)/ +49 2241 14 23 84 (Germany). E-mail addresses: [email protected] (H. Mu¨hlenbein), [email protected] (L. Zinchenko), [email protected] (V. Kureichik), [email protected] (T. Mahnig). 1 Tel.: +49 2241 14 24 05; fax: +49 2241 14 28 89. 2 Tel.: +7 8634 37 16 94; fax: +7 8634 31 05 98. 1568-4946/$ – see front matter # 2006 Published by Elsevier B.V. doi:10.1016/j.asoc.2006.07.001
At each step the probability is evaluated using the best-selected individuals. Therefore, the average fitness function never decreases [13]. It allows us to accumulate the information about perspective regions of the fitness function landscape. Probabilistic models have been applied to different problems successfully. In this paper, we restrict our research to analogue circuit design. Unfortunately, it is impossible to use theoretical framework similar [14] for analysis of evolutionary circuit design because of its mathematical complexity. However, high population size leads to high computational costs, while small population can result in a premature convergence. Therefore, extensive experimental researches are required. In ref. [14], Mu¨hlenbein and Mahnig have extended UMDA by means of Bayesian prior. It reduces the requirement to an effective choice of a population size. However, the correct choice of the effective mutation rate turned out to be difficult for evolutionary circuit design. If we use the very high mutation rate then a search became random. If we use UMDA with low mutation rate then the algorithm is unable to overcome local optima. Therefore, we have to evaluate an effective mutation rate in order to find the equilibrium between the success rate and computational costs. We consider the successful runs after the best solution has been found only. Moreover, the effective mutation rate and a fitness landscape are dependent. Therefore, we have to extend our research to different weightings of the fitness function.
H. Mu¨hlenbein et al. / Applied Soft Computing 7 (2007) 1012–1018
The remainder of the paper is organized as follows. Section 2 summarizes recent trends in evolutionary circuit design briefly. Section 3 presents the binary genotype–phenotype mapping used. Experimental results obtained for different weightings of the fitness function are discussed in Section 4. It turns out that mutation not only makes UMDA more efficient in many cases, but also more robust in the sense that the choice of the correct parameters is of less importance. 2. Related works 2.1. Knowledge-based approaches There are several approaches to circuit design. Many researches are restricted to circuit creation based on expert knowledge [18,19]. Other methods consider circuit optimisation only [5,20–22]. However, a good technical decision lies within huge design space, which includes all possible circuits [23,24]. Its complexity is evaluated by the amount of elements that are used, the rules, that were introduced in order to generate a circuit and an application for that the circuit is being evolved. Evolutionary algorithms provide a non-heuristic investigation in such complex design spaces. However, if we consider the circuit creation and the circuit optimisation separately, then design solutions can be concentrated within a small part of fitness function landscape. Several approaches have been applied in order to combine circuit creation and circuit optimisation for filters design [6,16], transducers [7,8,10], etc. However, fixed pre-defined topologies restrict a possible technical decision. 2.2. Creative topologies approaches The use of genetic programming [4] introduced novel direction for circuit design that is called developmental synthesis. Unlike the methods mentioned above a circuit topology, types of components, its values are evolved from an embryonic circuit (a set of short circuits) to a full circuit. However, genetic programming approach generates circuits that are highly redundant as a result of knowledge absence about a problem solved. Because of the nature of the design technique the most part of circuits generated at the initial steps are faulty circuits. The use of transformation rules (cc-bot instructions) allows us to generate circuits with valid graphs [9]. However, a set of required instructions is increased significantly. Therefore, the application of the developmental synthesis is limited by problem complexity. A simulation of useful and faulty circuits results in the increase of computational costs. In order to enhance evolutionary design effectiveness starting points that are included in the initial population are used [2]. Computational costs are decreased significantly. This approach using the combination of evolutionary design and heuristics is effective. It is remarkable that preliminary knowledge reduces computational costs. However, stable attractors are possible around a starting point. Hence, local optima can be found when a landscape has saddle points in the initial area.
1013
2.3. Evolutionary probabilistic design The most important feature in evolutionary design is a choice of the search algorithm. The main requirements are its speed, flexibility and ability to investigate a solution space effectively. The most design steps are iterated in design flow. Therefore, an algorithm should offer a choice of good technical decisions that have to generate rapidly. Positive results have been obtained by means of combined genetic/annealing optimisation algorithm [5], hybrid genetic algorithm [3], etc. Our focus is on evolutionary probabilistic algorithms application. Evolutionary algorithms based on probabilistic models have been recognized as a new computing paradigm in evolutionary computation. They can be divided into two categories: univariate distribution algorithms, like population-based incremental learning algorithm and univariate marginal distribution algorithm and bivariate distribution algorithms. Several typical examples of probabilistic models are genetic quantum algorithm, ant colony algorithm, factorised distribution algorithm, compact genetic algorithm, etc. The probabilistic approach defines the direct connection between a genotype and its fitness. Therefore, a fitness function plays the crucial role for probabilistic evolutionary algorithms. Probabilistic algorithms have been applied successfully to a travelling salesman problem, the quadratic assignment problem, etc. Recently, ant colony algorithm has been used for analogue circuit design successfully [15]. At the initial step, a gain value of each cell can be chosen from 0 to 1. Each new generation restricts the potential change of the gain value. However, possible combinations of PALMO cells restrict circuit topologies. UMDA [13] is an evolutionary algorithm that combines mutation and recombination by using probabilistic distribution. The distribution is estimated from a set of selected points M. It is then used to generate new points for the next generation according to the probability p(X, t) in the population at the generation t pðX; tÞ ¼
n Y
psi ðxi ; tÞ;
i¼1
where n is equal to the genotype length. UMDA Step 1. Set t = 1. Generate N possible solutions randomly. Step 2. Select M < N solutions according to a selection method. Step 3. Compute marginal frequencies psi ðxi ; tÞof the selected set. Step 4. Generate N new solutions according to the marginal Q distribution pðX; t þ 1Þ ¼ ni¼1 psi ðxi ; tÞ: Set t = t + 1. Step 5. If termination criteria are not met, go to Step 2. Different selection methods have been investigated in ref. [12]. In this paper, our focus is on the truncation selection, where t = M/N is the amount of selected individuals.
H. Mu¨hlenbein et al. / Applied Soft Computing 7 (2007) 1012–1018
1014
In ref. [14], mutation has been introduced into UMDA by a concept called Bayesian prior. In practice, this means that the estimated marginal frequency psi ðxi ; tÞ is calculated as follows: psi ðxi ; tÞ ¼
mþr ; Nt þ 2r
where m is equal to the number of bits with value 1 in the selected parents in i loci, N the population size and r is the mutation rate. Experimental results for several test functions have been published in ref. [14]. In ref. [11], we have introduced the hierarchical technique for evolutionary circuit design. Thus, we were able to generate novel analogue circuits. The main difference between our approach and related works is a combination of hierarchical design at element, topology and parameter levels and UMDA. A comparison of different circuit representations has been published in ref. [17]. In this paper, we aim to evaluate the effective mutation rate. We examine the effective mutation rate for different weightings of the fitness function. It should be noted that we restrict our research to static fitness function schedule and binary genotype–phenotype mapping. 3. Evolutionary hierarchical design The hierarchical construction of design space is used to generate circuits with required properties. An overview of the proposed design technique has been published in ref. [11]. The approach is effective because of its direct genotype decoding to a system of equations. We use the modified nodal method likely to SPICE approach [11]. The growing technique is used during the evolutionary design. The embryonic circuit is shown in Fig. 1b. The evolutionary process is based on the topology of the complete graph. Adjacency matrix of the N-node complete graph is given as follows [1]: ai j ¼ 0; if i ¼ j A¼ : (1) ai j ¼ 1; if i 6¼ j
In order to reduce a number of incorrect solutions our design flow is described by means of separate sets analogously DNA. Therefore, each solution is represented by hierarchical set, including types of elements and parameters. The first level of the set is an amount of the complete graph nodes. The second level is defined by the correspondent sequence of the elements from the set ELEMENT = {S, R, L, C}, etc. We restrict our research to the two-terminal components only. Alleles for components’ type are S, R, L, C, that represent switches, resistors, inductors, capacitors, etc. The third level is parameter one. In our hierarchical coding the circuit is termed as a phenotype. The genotype is defined as a linear structure with a variable length: NEn. . .E2E1E0 Pk. . .P2P1P0, where N is the amount of complete graph nodes; En. . .E2E1E0 the part of the genotype that defines the evolution of elements; Pk. . .P2P1P0 the part of the genotype that describes parameter evolution (changes parameter elements without changes of topology); n the amount of the circuit elements; k is the amount of circuit element parameters. Notice, that when we use the complete graph topology, the amount of the circuit elements is given as follows: n¼
NðN þ 1Þ : 2
(2)
Analogue circuits are hard to manufacture. Only fixed parameters are possible for circuit manufacturing. Therefore, parameters can be changed as discrete values only. Therefore, the chromosome hierarchy structure is defined as the discrete set. Each element is determined by a type and a parameter. The first and the second bits of each group define a type. Each element can take on four different types, namely, switch (00), capacitance (01), inductance (10) and resistors (11). The next bits define a component sizing. The parameter of switch can take on two fixed values only. We consider that 0 means that a switch is turned off and 1 means that a switch is turned on. A
Fig. 1. Chromosome (a), embryonic circuit (b) and the circuit, created from it (c).
H. Mu¨hlenbein et al. / Applied Soft Computing 7 (2007) 1012–1018
state of switches defines a circuit topology. Parameters of inductances, resistors and capacitances can be chosen within the fixed set. The input of the circuit is always node 1 and the output is node 2. The branch 1 is the series connection of a voltage source and the source resistance always. The branch 2 is the load resistance. A circuit being evolved contains (n2) branches. Fig. 1 shows the chromosome of length 23 and the circuit created from it. Our embryonic circuit evolves according to design requirements. First, a user specifies the boundary limits for the amount of nodes. Second, the set EG is formed by two subsets {E1, E2}, where subset E1 includes elements that represent a structure of the initial solution; subset E2 includes elements that describe an evolutionary development. All elements of the subset E2 are set to 0 in the initial population, hence, the initial state of the set EG is described as follows: {E1, 0}. The initial population can be generated randomly, by means of user’s experience or a textbook. Each element of the set EG can change randomly during evolutionary design. Then the set P that defines parameter evolution changes automatically. There are two possible cases. If an element, which represents in E1, changes to 0, then the corresponding elements in set P are set to zero. In the second case, which represents a change of an element from 0 to any element from the set ELEMENT, the corresponding elements are obtained in the set P being element parameters. Restrictions are the fixed amount of switches defined by the user. Second, elements of the set P can be changed within the discrete set randomly. Restrictions are the variations of parameters within the fixed set. It should be noted that set P consists of two subsets {P1, P2}. Allowed constants, defining one possible phenotype form represent elements of the first subset. P2 is a subset of constants that defines a few forms of phenotype. Therefore, subset P2 contains parameters that can be changed. The relationship between constants of the subset and variables in the phenotype to be satisfied defines the variable domains. All constraints are defined by heuristics. Furthermore, the best solution of topological design level is used as an initial circuit for the following parametrical optimisation. At this step all bits correspond to element parameters, therefore, more flexible search within fixed topology is supplied. This type of connection between the genotype and the phenotype is the dynamic coding, that allows us to exploit evolution of genotype and phenotype simultaneously. Various forms of connection between genotype and phenotype are possible by means of P1 and P2 evolution. Subset P1 defines only one possible form, while subset P2 establishes the dynamic relation between a genotype and a phenotype. One form of genotype corresponds to a few forms of phenotype. Therefore, evolution at the phenotype level is possible for the dynamic coding. The use of the hierarchy for chromosome coding eliminates useless search and reduces the design space dimension. It should be noted that a fitness landscape has many local optima for circuit design [7]. Premature convergence can be overcome by means of huge population sizes [4,9], although large computation costs are required. We aim to remedy this deficiency by using the effective search algorithm.
1015
4. Experimental results In ref. [11], we have published the experimental results for the evolutionary circuit design that uses UMDA and elitism. However, the results were obtained for the single fitness landscape. In order to make a fair comparison, we have to compare actual simulation results for different weightings of the fitness function. We use the low pass filter design as the test benchmark. The starting point for the design of a filter is the specification by the user of the frequencies for the filter’s pass band and stop band. The experiment described here is based on ‘‘synthesis of low pass filter’’ [4]. We consider the filter design with a pass band below 1 kHz and a stop band above 2 kHz. It is acceptable if the output voltage in the pass band is from 970 mV to 1 V. It is acceptable if the output voltage in the stop band is from 0 V to 1 mV. The ratio in the pass band (1 V) to the maximum acceptable signal in the stop band (1 mV) is 60 dB, but there is a two-to-one ratio between the stop band frequency and the pass band frequency. In order to reduce computational costs the fitness is evaluated by means of the equation-based method [11]. The numerical frequency response evaluation and the modified nodal method are used. The fitness function evaluation technique used is similar to those described in ref. [11]. The fitness is evaluated in terms of the sum of the absolute weighted difference between the actual output voltage and the target one. These error values were summed across evaluation points Nc for the evaluation of the fitness: F¼
Nc X W½dð f i Þ; f i Þdð f i Þ; i¼0
dð f Þ ¼ jU goal ð8 f Þ U out ð f Þj; 1; if dð f Þ 0:03; f 1000 Hz; > > < w1 ; if dð f Þ 0:03; f 1000 Hz; W½dð f Þ; f ¼ 1; if dð f Þ 0:001; f 2000 Hz; > > : w2 ; if dð f Þ 0:001; f 2000 Hz: (3) where f is the frequency of the fitness evaluation, d( f) the absolute value of the difference between the output voltage and the target at the frequency f and W[d( f), f] is the penalty factor for the difference d( f) at the frequency f. We use the set of the circuits that can be generated by the topology of the complete three-node graph as the simplest test function. The best circuit is given in Fig. 1c. Amount of hits is equal to 86. The stopping criterion is the best solution finding. The graphs in Figs. 2 and 3 show how computational costs are changed when the population size and the truncation threshold are modified. Fig. 2 illustrates the case in which the population size is varying and t = 0.02. The case in which the truncation threshold t is varying and the population size N = 300 is shown in Fig. 3. It is obvious that the curves have approximately minimum, when rn ¼ 44:5; I Nt
(4)
1016
H. Mu¨hlenbein et al. / Applied Soft Computing 7 (2007) 1012–1018
Fig. 4. Performance comparison with different mutation rate for population size N = 300 and truncation selection with t = 0.02. FE is the number of evaluation before the best solution has been found.
Fig. 2. Performance comparison for several parameters. FE is the number of evaluation before the best solution has been found.
where I is the selection intensity that depends on a truncation threshold of t [13]. The results were obtained for the penalty coefficients w1 ¼ w2 ¼ 10: Furthermore, we evaluate success rates and computational costs for different weightings of the fitness function. Figs. 4–11 summarize the experimental results obtained for several penalty coefficients w1 , w2 . Figs. 4, 5, 8 and 9 show the results for varying population size N from 300 to 500 and t = 0.02. Figs. 6, 7, 10 and 11 illustrate the case, when the truncation threshold is increased to t = 0.04. The most beneficial result is that the mutation changes success rate in the same manner for different weightings of the fitness function. When the mutation rate is high, the number of function evaluations increases, as the probabilities are shifted towards 1/ 2 too much. When the mutation rate decreases, the success rate
Fig. 5. Performance comparison with different mutation rate for population size N = 300 and truncation selection with t = 0.02. Succ is the number of times, when the best solution was found in 100 runs.
Fig. 6. Performance comparison with different mutation rate for population size N = 300 and truncation selection with t = 0.04. FE is the number of evaluation before the best solution has been found.
decreases also, because it becomes too improbable to flip the remaining bits that are wrong. The choice a small population size leads to a small success rate, while the choice of big population size is a waste of function evaluations. It seems to us that both effects are overcome by using the effective mutation rate (4).
Fig. 3. Performance comparison for several parameters. FE is the number of evaluation before the best solution has been found.
Fig. 7. Performance comparison with different mutation rate for population size N = 300 and truncation selection with t = 0.04. Succ is the number of times, when the best solution was found in 100 runs.
H. Mu¨hlenbein et al. / Applied Soft Computing 7 (2007) 1012–1018
Fig. 8. Performance comparison with different mutation rate for population size N = 500 and truncation selection with t = 0.02. FE is the number of evaluation before the best solution has been found.
Fig. 9. Performance comparison with different mutation rate for population size N = 500 and truncation selection with t = 0.02. Succ is the number of times, when the best solution was found in 100 runs.
1017
Fig. 10. Performance comparison with different mutation rate for population size N = 500 and truncation selection with t = 0.04. FE is the number of evaluation before the best solution has been found.
Fig. 11. Performance comparison with different mutation rate for population size N = 500 and truncation selection with t = 0.04. Succ is the number of times, when the best solution was found in 100 runs.
Fig. 12. The best circuit (a) and the frequency response of the filter (b).
Moreover, the success rate is higher for the penalty coefficients w1 ¼ 10, w2 ¼ 15 and smaller truncation threshold. However, if truncation threshold is increased to 0.04, then higher success rate has been obtained for the penalty coefficients w1 ¼ 10, w2 ¼ 20. Therefore, success rate depends from truncation threshold and weightings of the fitness function. Furthermore, we use the effective mutation rate in order to design analogue cells for Field Programmable Analogue Arrays. The best circuit that was found for the four-node
graph is shown in Fig. 12a and b illustrates that the frequency response falls within the target specifications. We used PSPICE for the evaluation of the best circuit characteristic. 5. Conclusions Distribution algorithms control evolvability of a population on a mesoscopic level. Probability distributions are evaluated from the set of the best elements. Therefore, the information obtained at each step of evolutionary design is reserved for the
1018
H. Mu¨hlenbein et al. / Applied Soft Computing 7 (2007) 1012–1018
following generations. However, the evolutionary probabilistic circuit design approach requires the extensive research to ensure that the most effective design is exploited. The results have shown that distribution algorithms with mutation are both fast and requiring small computational costs. The effective mutation rate has shown to be better in computational complexity. The effective mutation rate was examined for three different weightings of the fitness function. Varying penalty coefficients change the fitness function landscape significantly. However, the behaviour of the univariate marginal distribution algorithm with the effective mutation rate is similar for different weightings of the fitness function. It should be noted that fitness function landscape and truncation threshold are crucial ingredients for high success rate. However, the effective mutation rate makes the correct choice of population size and truncation threshold is of less important. Acknowledgments This work was supported by INTAS (International Association for the promotion of co-operation with scientists from the New Independent States of the former Soviet Union) by grant No YSF 00-190. We would like to thank all members of the RWCP Group, FhG-AIS for their support, advice and criticism of this work. Lyudmilla Zinchenko and Victor Kureichik acknowledge support by Russian Fund of Basic Research (grant 04-0100174). References [1] N. Biggs, Algebraic Graph Theory, Cambridge, 1974. [2] C. Goh, Y. Li, GA automated design and synthesis of analog circuits with practical constraints, in: Proceedings IEEE International Conference of Evolution Computation, IEEE Press, 2001, pp. 170–177. [3] J.B. Grimbleby, Hybrid genetic algorithms for analogue network synthesis, in: Proceedings IEEE International Conference of Evolution Computation, IEEE Press, 1999, pp. 1781–1787. [4] J.R. Koza, F.H. Bennett, D. Andre, M.A. Keane, Genetic Programming III: Darwinian Invention and Problem Solving, Morgan Kaufmann, San Francisco, CA, 1999. [5] M. Krasnicki, R. Phelps, R.A. Rutenbar, L.R. Carley, Maelstrom: efficient simulation-based synthesis for custom analog cells, in: Proceeding ACM/ IEEE DAC, 1999. [6] W. Kruiskamp, D. Leenaetrs, Darwin: CMOS OpAmp Synthesis by means of Genetic Algorithms, in: Proceeding ACM/IEEE DAC, 1995. [7] V.M. Kureichik, L.A. Zinchenko, Evolutionary modelling with hierarchy in innovative computer-aided circuit design, IETE J. Res. 5 (2002) 361– 367.
[8] V.M. Kureichik, L.A. Zinchenko, Evolution design of electronic devices, in: Proceedings ICECS 2000, 2000, pp. 879–882. [9] J.D. Lohn, S.P. Colombano, A circuit representation technique for automated circuit design, IEEE Trans. Evol. Comput. 3 (1999) 205–219. [10] H. Mu¨hlenbein, V.M. Kureichik, T. Mahnig, L.A. Zinchenko, Evolutionary modelling with hierarchy and dynamic coding in computer-aided design, in: K.C. Giannakoglou, T. Fogarty (Eds.), Evolutionary Methods for Design, Optimization and Control with Applications to Industrial Problems, CIMNE, Spain, 2002, pp. 202–207. [11] H. Mu¨hlenbein, V.M. Kureichik, T. Mahnig, L.A. Zinchenko, Application of the univariate marginal distribution algorithm to analog circuit design, in: Proceedings of the Evolvable Hardware Conference, Computer Press, 2002, pp. 93–101. [12] H. Mu¨hlenbein, T. Mahnig, Comparing the adaptive boltzmann selection schedule SDS to truncation selection, in: Proceedings of the 3rd International Symposium on Adaptive Systems, 2001, pp. 121–128. [13] H. Mu¨hlenbein, The equation for response to selection and its use for prediction, Evol. Comput. 5 (1998) 303–346. [14] H. Mu¨hlenbein, T. Mahnig, Evolutionary computation and Wright’s equation, Theor. Comput. Sci. 287 (2002) 145–165. [15] M.R. Tamplin, A. Hamilton, Ant circuit world: an Ant algorithm MATHLAB toolbox for the design, visualization and analysis of analogue circuits, in: Proceedings of the International Conference on Evolvable Systems: From Biology to Hardware. Lecture Notes in Computer Science, Springer-Verlag, Berlin, Heidelberg, New York, 2001. [16] R.S. Zebulum, M.A. Pacheho, M. Vellasco, Comparison of different evolutionary methodologies applied to electronic filter design, in: Proceedings’1998 IEEE International Conference of Evolutionary Computation, IEEE Press, 1998, pp. 434–439. [17] L.A. Zinchenko, H. Muehlenbein, V.M. Kureichik, T. Mahnig, A Comparison of different circuit representations for evolutionary analog circuit design, in: Proceedings of the International Conference on Evolvable Systems: From Biology to Hardware. Lecture Notes in Computer Science, Springer-Verlag, Berlin, Heidelberg, New York, 2003. [18] F. El-Turkey, E.E. Perry, BLADES: an artificial intelligence approach to analog circuit design, IEEE Trans. Comput. Aided Des. Integr. Circuits Syst. 6 (1989) 680–692. [19] R. Harjani, R.A. Rutenbar, L.R. Carley, OASYS: a framework for analog circuit synthesis, IEEE Trans. Comput. Aided Des. Integr. Circuits Syst. 12 (1989) 1247–1266. [20] E.S. Ochotta, R.A. Rutenbar, L.R. Carley, Synthesis of high-performance analog circuits in ASTRX/OBLX, IEEE Trans. Comput. Aided Des. Integr. Circuits Syst. 3 (1996) 273–294. [21] G.G.E. Gielen, H.C.C. Walscharts, W.M.C. Sansen, Analog circuit design optimization based on symbolic simulation and simulated annealing, IEEE J. Solid State Circuits 3 (1990) 707–713. [22] R. Phelps, M. Krasnicki, R.A. Rutenbar, L.R. Carley, J.R. Hellums, ANACONDA: simulation based synthesis of analog circuits via stochastic pattern search, IEEE Trans. Comput. Aided Des. Integr. Circuits Syst. 6 (2000) 703–717. [23] Van der Plas, G. Debyser, F. Leyn, K. Lampaert, J. Vandenbussche, G.E. Gielen, W. Sansen, P. Vaselinovic, D. Leenaerts, AMGIE—a synthesis environment for CMOS analog integrated circuits, IEEE Trans. Comput. Aided Des. Integr. Circuits Syst. 9 (2001) 1037–1058. [24] H.Y. Koh, C.H. Sequin, P.R. Gray, OPASYN: a compiler for CMOS operational amplifiers, IEEE Trans. Comput. Aided Des. Integr. Circuits Syst. 2 (1990) 113–125.