- Email: [email protected]
Genetic algorithm approach for designing higher-order digital differentiators
Signal Processing 79 (1999) 175}186
Genetic algorithm approach for designing higher-order digital di!erentiators Shian-Tang Tzeng, Hung-Ching Lu* Department of Electrical Engineering, Tatung Institute of Technology, 40 Chungshan North Road, 3rd Sec., Taipei, 10451, Taiwan Received 15 June 1998; received in revised form 26 November 1998
Abstract An e!ect approach is proposed for designing higher-order digital di!erentiators by the genetic algorithm (GA) method. By minimizing a quadratic measure of the error in the frequency band, appropriate crossover, mutation, and selection operations are used to get the "lter coe$cients. This method is not only simple and fast but also optimal in the least-squares sense. Comparison to the well-known McClellan}Parks algorithm for minimax equiripple "lters shows that both are optimal in the sense of di!erent minimum norms of the error function, but much better performance is obtained with the proposed approach in most of the frequency band except in the narrow-band region near the cuto! edge. ( 1999 Elsevier Science B.V. All rights reserved. Zusammenfassung Ein wirkungsvoller Ansatz zum Entwurf digitaler Di!erenzierer hoK herer Ordnung unter Verwendung von Genetischen Algorithmen (GA) wird vorgestellt. Die Filterkoe$zienten werden durch Minimierung eines quadratischen Fehlerma{es innerhalb des Frequenzbandes ermittelt, wobei geeignete Operationen, insbesondere Kreuzung, Mutation und Selektion, verwendet werden. Diese Methode ist nicht nur einfach und schnell, sondern auch optimal in quadratischen Mittel. Ein Vergleich mit dem wohlbekannten McClellan}Parks Algorithmus zum Entwurf von Filtern mit gleichmaK {igem Frequenzgang nach dem Minimax-Prinzip zeigt, dass beide Filter optimal im Sinne verschiedener Normen bezogen auf die Fehlerfunktion sind, dass aber der vorgeschlagene Ansatz eine bessere LeistungsfaK higkeit in weiten Bereichen des interessierenden Frequenzbandes au{er im Schmalbandbereich in der NaK he der Durchla{raK nder besitzt. ( 1999 Elsevier Science B.V. All rights reserved. Re2 sume2 Nous proposons une meH thode e$cace de conception de di!eH rentiateurs numeH riques d'ordres eH leveH s par une meH thode utilisant des algorithmes geH neH tiques. En minimisant la mesure quadratique de l'erreur dans la bande de freH quence, des opeH rations approprieH es de croisement, mutation et seH lection sont utiliseH es pour obtenir les coe$cients du "ltre. Cette meH thode est non seulement simple et rapide, mais aussi optimale au sens des moindres carreH s. La comparaison avec 1'algorithme bien connu de McClellan}Parks pour des "ltres minmax a` oscillations eH gales montre que les deux sont optimaux au sens de di!eH rentes normes minimales de la fonction d'erreur, mais que des performances bien meilleures sont obtenues avec l'approche proposeH e dans la plupart des bandes de freH quences, sauf dans la bande eH troite pre`s du bord de coupure. ( 1999 Elsevier Science B.V. All rights reserved. Keywords: Higher-order di!erentiators; Genetic algorithm; McClellan}Parks algorithm
* Corresponding author. E-mail address: [email protected] (H.-C. Lu) 0165-1684/99/$ - see front matter ( 1999 Elsevier Science B.V. All rights reserved. PII: S 0 1 6 5 - 1 6 8 4 ( 9 9 ) 0 0 0 9 1 - 2
176
S.-T. Tzeng, H.-C. Lu / Signal Processing 79 (1999) 175}186
1. Introduction Conventionally, we often make use of the well-known McClellan}Parks program [4] to design the "rst-order di!erentiator. Rahenkamp and Vijaya Kumar have modi"ed the program for designing higherorder di!erentiators [10]. These higher-order digital di!erentiators are very useful for calculation of geometric moments [14] and for biological signal processing [15]. We have tested this modi"ed program for several numerical design examples and found that it often leads to very large deviation or fails to converge, especially for designing full-band higher-order di!erentiators. Vaidyanathan and Nguyen have introduced the eigen"lter approach for designing linear "nite-duration impulse response (FIR) digital "lters [13]. In this paper, we present a powerful design of higher-order di!erentiators by using the GA approach; this method is not only simple and fast but also optimal in the least-squares sense. The unique advantage of this method over the McClellan}Parks algorithm is that it is general enough to incorporate both time-domain and frequency-domain constraints. This method is based on the crossover, mutation, and selection operations of GA technique. After several generations, the algorithms converge to the best chromosome, which represents the optimal solution to the problem. Comparison to the McClellan}Parks algorithm for minimax equiripple "lters shows that both are optimal in the sense of di!erent minimum norms of the error function, but much better performance is obtained with our approach in most of the frequency band except in the narrow-band region near the cuto! edge. In general, we note that even-order di!erentiators can only be designed by symmetric impulse response sequences [9, cases 1 and 2]. On the contrary, odd-order di!erentiators can only be designed by antisymmetric impulse response sequences [9, cases 3 and 4]. This paper is organized as follows. The GA approach is described in Section 2, including crossover, mutation, and selection operations. In Section 3, we present the design of even-order di!erentiators; the odd-order case will be discussed in Section 4. Design examples and their frequency magnitude responses and error curves are shown in Section 5. At last, conclusions are given in Section 6.
2. Genetic algorithms The usual form of GA was described by Goldberg [3]. A chromosome is real-valued instead of binary bit strings in this paper. During each generation, the chromosomes are evaluated with some measures of "tness. According to the "tness values, a new generation is formed by selecting some of the parents and o!spring, and rejecting others so as to keep the population size constant. After several generations, the algorithms converge to the best chromosome, which represents the optimal solution to the problem. 2.1. Crossover operation Crossover is the main genetic operator. There are many papers to discuss the crossover operation [11,12,1,5,7,2]. With the above discussed genetic crossover operators, there is no guarantee that the o!spring are better than their parents. For direction-based crossover operator [6], problem-speci"c knowledge is introduced into genetic operation in order to produce improved o!spring. The operator generates a single o!spring B@ from two parents B1 and B2 according to the following rule: B@"r ) (B2 !B1 )#B2 ,
(1)
where r is a random number between 0 and 1. It also assumes that the parent B2 is not worse than B1 ; that is, "tness(B2 )*"tness(B1 ) for maximization problems and "tness(B2 ))"tness(B1 ) for minimization problems. In this paper, we will take full advantage of the latter case, minimization, to deal with the design of higher-order digital di!erentiators.
S.-T. Tzeng, H.-C. Lu / Signal Processing 79 (1999) 175}186
177
Fig. 1. GA approach design procedure.
2.2. Mutation operation Mutation is a background operator which produce spontaneous random changes in various chromosomes. A simple way to achieve mutation would be to randomly generate chromosomes. In this paper, the mutation rate is set to be 1% because of the low probability in nature. 2.3. Selection operation The principal behind GAs is essential Darwin natural selection. Selection procedure creates a new population for the next generation based on all parents and o!spring. Both parents and o!spring have the same chance of competing for survival. An evident advantage of this approach is that the GA performance can be improved.
178
S.-T. Tzeng, H.-C. Lu / Signal Processing 79 (1999) 175}186
There is a GA approach design procedure depicted in Fig. 1. In this paper, the population size is set to be 100, PS"100, for the sake of convenience.
3. Even-order di4erentiators design Ideal even-order di!erentiators are characterized by the following frequency response [10]:
G
D% (u)"
A B C
ju k , 2p
0)u)2pf1 ,
D
j(2p!u) k , 2p(1!f1 ))u)2p, ! 2p
(2)
where k is even and denotes the order of the di!erentiator and f1 is the highest frequency for which di!erentiating action is required. Obviously, D% (u) is a real-valued function and we can take advantage of cases 1 or 2 [4,9] design to approximate (2). However, while case 1 is suitable for designing the full-band or nonfull-band di!erentiators, case 2 is useful only for designing nonfull-band di!erentiators due to the inherent zero-folding frequency constraint [9]. Consider the typical transfer function of a FIR "lter of length N: N~1 H(Z)" + h(n)Z~n, n/0 where h(n) is the impulse response sequence with the even symmetric property h(n)"h(N!1!n).
(3)
(4)
Hence, the frequency response of H(Z) can be represented by H(e~+u)"e~+u(N~1)@2H (u), % where H (u) is real-valued and given by [9] %
G
H (u)" %
+(N~1)@2 b(n)cos(nu), n/0
(5)
case 1, N: odd, (6)
+N@2 b(n)cos(n!1/2)u, case 2, N: even, n/1
where b(0)"h((N!1)/2) and
G
b(n)"
2h
A A
B
N!1 N!1 !n , n"1, 2,2, , case 1, N: odd, 2 2
B
N 2h !n , 2
N n"1, 2,2, , 2
(7)
case 2, N: even.
Now, let us de"ne
G
B"
C C
b(0), b(1), 2, b
A BD A BD N!1 2
N b(1), b(2), 2, b 2
T
,
T , N: odd, (8) N: even,
S.-T. Tzeng, H.-C. Lu / Signal Processing 79 (1999) 175}186
179
and
G
C A C AB A B 1, cos u, 2, cos
C(u)"
N!1 u 2
BD A
T ,
u 3 N!1 cos , cos u , 2, cos u 2 2 2
N: odd,
BD
(9)
T ,
N: even,
where the superscript T is the vector transpose operation. Eq. (6) can thus be rewritten as H (u)"BTC(u)"CT(u)B , i% i i
(10)
where the subscript i stands for the ith chromosome. For obtaining the "tness, de"ne an objective error function
P
E" i
up
0
DD (u)!H (u)D2 du, % i%
(11)
where u "2pf and D (u) is the desired function we wish to approximate. After applying the GA technique 1 1 % as mentioned in Fig. 1 to get the optimal chromosome, B, it is easy to obtain the frequency response from (3) by means of (7).
4. Odd-order di4erentiators design Ideal odd-order di!erentiators are characterized by the following frequency response [10]:
G
D (u)" 0
A B C
ju k , 2p
0)u)2pf , 1
(12)
D
j(2p!u) k , 2p(1!f ))u)2p, ! 1 2p
where k is the odd and (12) is a pure imaginary function, so we can make use of cases 3 and 4 designs [9] to approximate it; while case 3 is only suitable for designing the nonfull-band di!erentiators due to the inherent zero-folding frequency constraint [9], case 4 is suitable for the full-band as well as nonfull-band di!erentiators. For case 3, the impulse response sequence has the odd symmetric property h(n)"!h(N!1!n), n"0, 1, 2,
N!3 , 2
h
A
B
N!1 "0, 2
(13)
and for case 4 h(n)"!h(N!1!n), n"0, 1, 2,
N !1. 2
(14)
Hence, the frequency response of H(Z) can be represented by H(e~+u)"e~+((N~1)@2)ue+(p@2)H (u), 0
(15)
180
S.-T. Tzeng, H.-C. Lu / Signal Processing 79 (1999) 175}186
where H (u) is real-valued and given by [9] 0
G
case 3, N: odd, +(N~1)@2 b(n)sin (nu), n/1 H (u)" 0 1 +N@2 b(n)sin n! u, case 4, N: even, n/1 2 where
G
b(n)"
De"ne
G
2h
A A
B
N!1 N!1 !n , n"1, 2,2, , N: odd, 2 2
B
N !n , 2h 2
C C
b(1), b(2),2, b
B"
and
A B
N n"1, 2,2, , 2
A BD A BD
T , N: odd,
T ,
N: even,
G
S(u)"
Then
(18)
C A BD C A B A B A BD sin (u), sin (2u), 2, sin
(17)
N: even.
N!1 2
N b(1), b(2),2, b 2
(16)
N!1 2u
T
u 3 N!1 sin , sin u , 2, sin 2 2 2u
N: odd,
,
T
(19) , N: even.
H (u)"BTS(u)"ST(u)B , i i i0
(20)
where B denotes the ith chromosome. De"ne the "tness function as follows: i
P
E" i
up
0
DD (u)!H (u)D2 du, 0 i0
(21)
in which D (u) is the desired function we wish to approximate. Similarly, it is easy to obtain the frequency 0 response after applying the GA technique.
5. Design examples Four even-order and odd-order di!erentiators have been designed for the full band or non-full band cases in this section. Example 1. Case 1 for even-order and odd length. A FIR full band second-order di!erentiator was designed with length N"25 and u "p. After applying the GA technique with 2000 generations, the optimal 1 chromosome (solution) is obtained in the 1190th generation. The best solution corresponding to the smallest
S.-T. Tzeng, H.-C. Lu / Signal Processing 79 (1999) 175}186
181
Fig. 2. Fitness.
"tness value at each generation is shown in Fig. 2. It takes about 25 s of CPU time based on an Intel Pentium II 300 system with MATLAB 4.2c software package. Fig. 3(a) and (b) show the frequency magnitude responses and the error curves, respectively. In Fig. 3(a), the solid line represents the frequency magnitude response by GA approach and the dotted line represents the frequency magnitude response by McClellan}Parks algorithm. It shows that our performance is much better and smoother in this case. In Fig. 3(b), the error curves are shown with solid line by GA approach and with dotted line by McClellan}Parks algorithm. Notice that the error is much smaller than that of McClellan}Parks algorithm in most of the frequency band except in the small frequency interval near the band edge, where the GA approach error exceeds the McClellan}Parks equiripple error bound. Example 2. Case 2 for even-order and even length. We have designed an even length nonfull-band fourth-order di!erentiator with N"32 and u "0.92p. In Fig. 4(a), the frequency magnitude responses are shown with 1 solid line by GA approach and with dotted line by McClellan}Parks algorithm. In Fig. 4(b), the solid line represents the error by GA approach and the dotted line represents the error by McClellan}Parks algorithm. It is shown that the performance is better by our proposed method. Example 3. Case 3 for odd-order and odd length. A non-full band FIR third-order di!erentiator was designed with length N"27 and u "0.88p. Fig. 5(a) and (b) show the frequency magnitude responses and the error 1 curves, respectively. In Fig. 5(b), we compare the error with the McClellan}Parks algorithm, the deviation is smaller in our case in most of the frequency band excep in the narrow region near the cuto! edge. Example 4. Case 4 for odd-order and even length. A full band "fth-order di!erentiator was designed. With the following designed parameters: N"32 and u "p, we can get the frequency magnitude responses and 1 the error curves shown in Fig. 6(a) and (b), respectively. In the "fth-order di!erentiator design case, the
182
S.-T. Tzeng, H.-C. Lu / Signal Processing 79 (1999) 175}186
Fig. 3. (a) Frequency magnitude responses of a full-band, case 1, second-order di!erentiator with length N"25 and u "p for the GA 1 approach (solid line) and McClellan-Parks algorithm (dotted line). (b) The error curves for the GA approach (solid line) and McClellan}Parks algorithm (dotted line).
McClellan}parks algorithm leads to very large deviation and the GA approach performance is much better and smoother. The performance of this full band di!erentiator is superior to that of McClellan}Parks algorithm.
S.-T. Tzeng, H.-C. Lu / Signal Processing 79 (1999) 175}186
183
Fig. 4. (a) Frequency magnitude response of a nonfull-band, case 2, fourth-order di!erentiator with length N"32 and u "0.92p for 1 the GA approach (solid line) and McClellan}Parks algorithm (dotted line). (b) The error curves for the GA approach (solid line) and McClellan}Parks algorithm (dotted line).
We have summary the performance of these four examples in Table 1 and also indicate in the table the eigen"lter [8] peak errors, the equiripple errors, and the small frequency interval ranges over the equiripple bound. It shows that our performance is much better.
184
S.-T. Tzeng, H.-C. Lu / Signal Processing 79 (1999) 175}186
Fig. 5. (a) Frequency magnitude response of a nonfull-band, case 3, third-order di!erentiator with length N"27 and u "0.88p for 1 the GA approach (solid line) and McClellan}Parks algorithm (dotted line). (b) The error curves for the GA approach (solid line) and McClellan}Parks algorithm (dotted line).
S.-T. Tzeng, H.-C. Lu / Signal Processing 79 (1999) 175}186
185
Fig. 6. (a) Frequency magnitude response of a full band, case 4, "fth-order di!erentiator with length N"32 and u "p for the GA 1 approach (solid line) and McClellan}Parks algorithm (dotted line). (b) The error curves for the GA approach (solid line) and McClellan}Parks algorithm (dotted line).
6. Conclusions In this paper, we have presented a new method for designing higher-order digital di!erentiators by the GA approach and have shown that this method is not only simple and fast but also optimal in the least-squares
186
S.-T. Tzeng, H.-C. Lu / Signal Processing 79 (1999) 175}186
Table 1 Higher-order digital di!erentiators design examples Di!erentiators
Second-order
Fourth-order
Third-order
Fifth-order
Filter length Type of impulse response u : Highest 1 di!erentiating frequency GA approach peak error Eigen"lter peak error McClellan}Parks algorithm equiripple error Small frequency interval: GA approach error over equiripple bound Illustrated "gure
25 case 1 p Full-band 6.1]10~3 8.1]10~3 3.7]10~3 0.99p &p Fig. 3
32 case 2 0.92p Nonfull-band 10.1]10~4 14.6]10~4 4.7]10~4 0.888p &0.92p Fig. 4
27 case 3 0.88p Nonfull-band 5.7]104 9.1]10~4 3.0]10~4 0.848p &0.88p Fig. 5
32 case 4 p Full-band 13.7]10~4 19.6]10~4 9.0]10~4 0.992p &p Fig. 6
sense. The powerful and broadly applicable stochastic search and optimization techniques are shown. It gives much better performance than the McClellan}Parks algorithm. Several numerical design examples are given to illustrate the e!ectiveness and usefulness of this proposed approach. Acknowledgements This work was supported by the National Science Council of Republic of China, under contract NSC-88-2213-E-036-018. The authors are deeply indebted to the reviewers for their interests, careful reviews of this manuscript, observations, and valuable suggestions. Responding to their helpful comments helped us improve this paper. References [1] [2] [3] [4] [5] [6] [7] [8] [9] [10] [11] [12] [13] [14] [15]
L. Eshelman, J. Scha!er, Real-coded genetic algorithms and interval schemata, Found. Genetic Algorithms 2 (1993) 187}202. M. Gen, R. Cheng, Genetic Algorithms and Engineering Design, Wiley, New York, 1997. D.E. Goldberg, Genetic Algorithms in Search Optimization, and Machine Learning, Addison-Wesley, Reading, MA, 1989. J.H. McClellan, T.W. Parks, L.R. Rabiner, A computer program for designing optimum FIR linear phase digital "lters, IEEE Trans. Audio Electroacoust. 21 (December 1973) 506}526. Z. Michalewicz, Genetic Algorithm#Data Structure"Evolution Programs, 2nd Edition, Spring, New York, 1994. Z. Michalewicz, T. Logan, S. Swaminathan, Evolutionary operations for continuous convex parameter spaces, in: Proceedings of the Third Annual Conference on Evolutionary Programming, 1994, pp. 84}97. H. Muhlenbein, D. Schlierkamp-Voosen, Predictive models for the breeder genetic algorithm I continuous parameter optimization, Evol. Comput 1 (1993) 25}49. S.C. Pei, J.J. Shyu, Eigen"lter design of higher-order digital di!erentiators, IEEE Trans. Acoust. Speech Signal Process. 37 (April 1989) 505}511. L.R. Rabiner, B. Gold, Theory and Application of Digital Signal Processing, Prentice-Hall, Englewood Cli!s, NJ, 1975, pp. 81}84. C.A. Rahenkamp, B.V.K. Vijaya Kumar, Modi"cations to the McClellan, Parks, and Rabiner computer program for designing higher order di!erentiating FIR "lters, IEEE Trans. Acoust. Speech Signal Process. 34 (December 1986) 1671}1674. W. Spears, K.D. Jong, On the virtues of parameterized uniform crossover, in: Proceedings of the Fourth International Conference on Genetic Algorithms, 1991, pp. 230}236. G. Syswerda, Uniform crossover in genetic algorithms, in: Proceedings of the Third International Conference on Genetic Algorithms, 1989, pp. 2}9. P.P. Vaidyanathan, T.Q. Nguyen, Eigen"lters: A new approach to least-squares FIR "lter design and applications including Nyquist "lters, IEEE Trans. Circuits Systems 34 (January 1987) 11}23. B.V.K. Vijaya Kumar, C.A. Rahenkamp, Calculation of geometric moments from Fourier plane intensities, Appl. Opt. 25 (March 1986) 997}1007. S. Usui, I. Amidror, Digital low-pass di!erentiation for biological signal processing, IEEE Trans. Biomed. Eng. BME-29 (October 1982) 686}693.
Genetic algorithm approach for designing higher-order digital di!erentiators Shian-Tang Tzeng, Hung-Ching Lu* Department of Electrical Engineering, Tatung Institute of Technology, 40 Chungshan North Road, 3rd Sec., Taipei, 10451, Taiwan Received 15 June 1998; received in revised form 26 November 1998
Abstract An e!ect approach is proposed for designing higher-order digital di!erentiators by the genetic algorithm (GA) method. By minimizing a quadratic measure of the error in the frequency band, appropriate crossover, mutation, and selection operations are used to get the "lter coe$cients. This method is not only simple and fast but also optimal in the least-squares sense. Comparison to the well-known McClellan}Parks algorithm for minimax equiripple "lters shows that both are optimal in the sense of di!erent minimum norms of the error function, but much better performance is obtained with the proposed approach in most of the frequency band except in the narrow-band region near the cuto! edge. ( 1999 Elsevier Science B.V. All rights reserved. Zusammenfassung Ein wirkungsvoller Ansatz zum Entwurf digitaler Di!erenzierer hoK herer Ordnung unter Verwendung von Genetischen Algorithmen (GA) wird vorgestellt. Die Filterkoe$zienten werden durch Minimierung eines quadratischen Fehlerma{es innerhalb des Frequenzbandes ermittelt, wobei geeignete Operationen, insbesondere Kreuzung, Mutation und Selektion, verwendet werden. Diese Methode ist nicht nur einfach und schnell, sondern auch optimal in quadratischen Mittel. Ein Vergleich mit dem wohlbekannten McClellan}Parks Algorithmus zum Entwurf von Filtern mit gleichmaK {igem Frequenzgang nach dem Minimax-Prinzip zeigt, dass beide Filter optimal im Sinne verschiedener Normen bezogen auf die Fehlerfunktion sind, dass aber der vorgeschlagene Ansatz eine bessere LeistungsfaK higkeit in weiten Bereichen des interessierenden Frequenzbandes au{er im Schmalbandbereich in der NaK he der Durchla{raK nder besitzt. ( 1999 Elsevier Science B.V. All rights reserved. Re2 sume2 Nous proposons une meH thode e$cace de conception de di!eH rentiateurs numeH riques d'ordres eH leveH s par une meH thode utilisant des algorithmes geH neH tiques. En minimisant la mesure quadratique de l'erreur dans la bande de freH quence, des opeH rations approprieH es de croisement, mutation et seH lection sont utiliseH es pour obtenir les coe$cients du "ltre. Cette meH thode est non seulement simple et rapide, mais aussi optimale au sens des moindres carreH s. La comparaison avec 1'algorithme bien connu de McClellan}Parks pour des "ltres minmax a` oscillations eH gales montre que les deux sont optimaux au sens de di!eH rentes normes minimales de la fonction d'erreur, mais que des performances bien meilleures sont obtenues avec l'approche proposeH e dans la plupart des bandes de freH quences, sauf dans la bande eH troite pre`s du bord de coupure. ( 1999 Elsevier Science B.V. All rights reserved. Keywords: Higher-order di!erentiators; Genetic algorithm; McClellan}Parks algorithm
* Corresponding author. E-mail address: [email protected] (H.-C. Lu) 0165-1684/99/$ - see front matter ( 1999 Elsevier Science B.V. All rights reserved. PII: S 0 1 6 5 - 1 6 8 4 ( 9 9 ) 0 0 0 9 1 - 2
176
S.-T. Tzeng, H.-C. Lu / Signal Processing 79 (1999) 175}186
1. Introduction Conventionally, we often make use of the well-known McClellan}Parks program [4] to design the "rst-order di!erentiator. Rahenkamp and Vijaya Kumar have modi"ed the program for designing higherorder di!erentiators [10]. These higher-order digital di!erentiators are very useful for calculation of geometric moments [14] and for biological signal processing [15]. We have tested this modi"ed program for several numerical design examples and found that it often leads to very large deviation or fails to converge, especially for designing full-band higher-order di!erentiators. Vaidyanathan and Nguyen have introduced the eigen"lter approach for designing linear "nite-duration impulse response (FIR) digital "lters [13]. In this paper, we present a powerful design of higher-order di!erentiators by using the GA approach; this method is not only simple and fast but also optimal in the least-squares sense. The unique advantage of this method over the McClellan}Parks algorithm is that it is general enough to incorporate both time-domain and frequency-domain constraints. This method is based on the crossover, mutation, and selection operations of GA technique. After several generations, the algorithms converge to the best chromosome, which represents the optimal solution to the problem. Comparison to the McClellan}Parks algorithm for minimax equiripple "lters shows that both are optimal in the sense of di!erent minimum norms of the error function, but much better performance is obtained with our approach in most of the frequency band except in the narrow-band region near the cuto! edge. In general, we note that even-order di!erentiators can only be designed by symmetric impulse response sequences [9, cases 1 and 2]. On the contrary, odd-order di!erentiators can only be designed by antisymmetric impulse response sequences [9, cases 3 and 4]. This paper is organized as follows. The GA approach is described in Section 2, including crossover, mutation, and selection operations. In Section 3, we present the design of even-order di!erentiators; the odd-order case will be discussed in Section 4. Design examples and their frequency magnitude responses and error curves are shown in Section 5. At last, conclusions are given in Section 6.
2. Genetic algorithms The usual form of GA was described by Goldberg [3]. A chromosome is real-valued instead of binary bit strings in this paper. During each generation, the chromosomes are evaluated with some measures of "tness. According to the "tness values, a new generation is formed by selecting some of the parents and o!spring, and rejecting others so as to keep the population size constant. After several generations, the algorithms converge to the best chromosome, which represents the optimal solution to the problem. 2.1. Crossover operation Crossover is the main genetic operator. There are many papers to discuss the crossover operation [11,12,1,5,7,2]. With the above discussed genetic crossover operators, there is no guarantee that the o!spring are better than their parents. For direction-based crossover operator [6], problem-speci"c knowledge is introduced into genetic operation in order to produce improved o!spring. The operator generates a single o!spring B@ from two parents B1 and B2 according to the following rule: B@"r ) (B2 !B1 )#B2 ,
(1)
where r is a random number between 0 and 1. It also assumes that the parent B2 is not worse than B1 ; that is, "tness(B2 )*"tness(B1 ) for maximization problems and "tness(B2 ))"tness(B1 ) for minimization problems. In this paper, we will take full advantage of the latter case, minimization, to deal with the design of higher-order digital di!erentiators.
S.-T. Tzeng, H.-C. Lu / Signal Processing 79 (1999) 175}186
177
Fig. 1. GA approach design procedure.
2.2. Mutation operation Mutation is a background operator which produce spontaneous random changes in various chromosomes. A simple way to achieve mutation would be to randomly generate chromosomes. In this paper, the mutation rate is set to be 1% because of the low probability in nature. 2.3. Selection operation The principal behind GAs is essential Darwin natural selection. Selection procedure creates a new population for the next generation based on all parents and o!spring. Both parents and o!spring have the same chance of competing for survival. An evident advantage of this approach is that the GA performance can be improved.
178
S.-T. Tzeng, H.-C. Lu / Signal Processing 79 (1999) 175}186
There is a GA approach design procedure depicted in Fig. 1. In this paper, the population size is set to be 100, PS"100, for the sake of convenience.
3. Even-order di4erentiators design Ideal even-order di!erentiators are characterized by the following frequency response [10]:
G
D% (u)"
A B C
ju k , 2p
0)u)2pf1 ,
D
j(2p!u) k , 2p(1!f1 ))u)2p, ! 2p
(2)
where k is even and denotes the order of the di!erentiator and f1 is the highest frequency for which di!erentiating action is required. Obviously, D% (u) is a real-valued function and we can take advantage of cases 1 or 2 [4,9] design to approximate (2). However, while case 1 is suitable for designing the full-band or nonfull-band di!erentiators, case 2 is useful only for designing nonfull-band di!erentiators due to the inherent zero-folding frequency constraint [9]. Consider the typical transfer function of a FIR "lter of length N: N~1 H(Z)" + h(n)Z~n, n/0 where h(n) is the impulse response sequence with the even symmetric property h(n)"h(N!1!n).
(3)
(4)
Hence, the frequency response of H(Z) can be represented by H(e~+u)"e~+u(N~1)@2H (u), % where H (u) is real-valued and given by [9] %
G
H (u)" %
+(N~1)@2 b(n)cos(nu), n/0
(5)
case 1, N: odd, (6)
+N@2 b(n)cos(n!1/2)u, case 2, N: even, n/1
where b(0)"h((N!1)/2) and
G
b(n)"
2h
A A
B
N!1 N!1 !n , n"1, 2,2, , case 1, N: odd, 2 2
B
N 2h !n , 2
N n"1, 2,2, , 2
(7)
case 2, N: even.
Now, let us de"ne
G
B"
C C
b(0), b(1), 2, b
A BD A BD N!1 2
N b(1), b(2), 2, b 2
T
,
T , N: odd, (8) N: even,
S.-T. Tzeng, H.-C. Lu / Signal Processing 79 (1999) 175}186
179
and
G
C A C AB A B 1, cos u, 2, cos
C(u)"
N!1 u 2
BD A
T ,
u 3 N!1 cos , cos u , 2, cos u 2 2 2
N: odd,
BD
(9)
T ,
N: even,
where the superscript T is the vector transpose operation. Eq. (6) can thus be rewritten as H (u)"BTC(u)"CT(u)B , i% i i
(10)
where the subscript i stands for the ith chromosome. For obtaining the "tness, de"ne an objective error function
P
E" i
up
0
DD (u)!H (u)D2 du, % i%
(11)
where u "2pf and D (u) is the desired function we wish to approximate. After applying the GA technique 1 1 % as mentioned in Fig. 1 to get the optimal chromosome, B, it is easy to obtain the frequency response from (3) by means of (7).
4. Odd-order di4erentiators design Ideal odd-order di!erentiators are characterized by the following frequency response [10]:
G
D (u)" 0
A B C
ju k , 2p
0)u)2pf , 1
(12)
D
j(2p!u) k , 2p(1!f ))u)2p, ! 1 2p
where k is the odd and (12) is a pure imaginary function, so we can make use of cases 3 and 4 designs [9] to approximate it; while case 3 is only suitable for designing the nonfull-band di!erentiators due to the inherent zero-folding frequency constraint [9], case 4 is suitable for the full-band as well as nonfull-band di!erentiators. For case 3, the impulse response sequence has the odd symmetric property h(n)"!h(N!1!n), n"0, 1, 2,
N!3 , 2
h
A
B
N!1 "0, 2
(13)
and for case 4 h(n)"!h(N!1!n), n"0, 1, 2,
N !1. 2
(14)
Hence, the frequency response of H(Z) can be represented by H(e~+u)"e~+((N~1)@2)ue+(p@2)H (u), 0
(15)
180
S.-T. Tzeng, H.-C. Lu / Signal Processing 79 (1999) 175}186
where H (u) is real-valued and given by [9] 0
G
case 3, N: odd, +(N~1)@2 b(n)sin (nu), n/1 H (u)" 0 1 +N@2 b(n)sin n! u, case 4, N: even, n/1 2 where
G
b(n)"
De"ne
G
2h
A A
B
N!1 N!1 !n , n"1, 2,2, , N: odd, 2 2
B
N !n , 2h 2
C C
b(1), b(2),2, b
B"
and
A B
N n"1, 2,2, , 2
A BD A BD
T , N: odd,
T ,
N: even,
G
S(u)"
Then
(18)
C A BD C A B A B A BD sin (u), sin (2u), 2, sin
(17)
N: even.
N!1 2
N b(1), b(2),2, b 2
(16)
N!1 2u
T
u 3 N!1 sin , sin u , 2, sin 2 2 2u
N: odd,
,
T
(19) , N: even.
H (u)"BTS(u)"ST(u)B , i i i0
(20)
where B denotes the ith chromosome. De"ne the "tness function as follows: i
P
E" i
up
0
DD (u)!H (u)D2 du, 0 i0
(21)
in which D (u) is the desired function we wish to approximate. Similarly, it is easy to obtain the frequency 0 response after applying the GA technique.
5. Design examples Four even-order and odd-order di!erentiators have been designed for the full band or non-full band cases in this section. Example 1. Case 1 for even-order and odd length. A FIR full band second-order di!erentiator was designed with length N"25 and u "p. After applying the GA technique with 2000 generations, the optimal 1 chromosome (solution) is obtained in the 1190th generation. The best solution corresponding to the smallest
S.-T. Tzeng, H.-C. Lu / Signal Processing 79 (1999) 175}186
181
Fig. 2. Fitness.
"tness value at each generation is shown in Fig. 2. It takes about 25 s of CPU time based on an Intel Pentium II 300 system with MATLAB 4.2c software package. Fig. 3(a) and (b) show the frequency magnitude responses and the error curves, respectively. In Fig. 3(a), the solid line represents the frequency magnitude response by GA approach and the dotted line represents the frequency magnitude response by McClellan}Parks algorithm. It shows that our performance is much better and smoother in this case. In Fig. 3(b), the error curves are shown with solid line by GA approach and with dotted line by McClellan}Parks algorithm. Notice that the error is much smaller than that of McClellan}Parks algorithm in most of the frequency band except in the small frequency interval near the band edge, where the GA approach error exceeds the McClellan}Parks equiripple error bound. Example 2. Case 2 for even-order and even length. We have designed an even length nonfull-band fourth-order di!erentiator with N"32 and u "0.92p. In Fig. 4(a), the frequency magnitude responses are shown with 1 solid line by GA approach and with dotted line by McClellan}Parks algorithm. In Fig. 4(b), the solid line represents the error by GA approach and the dotted line represents the error by McClellan}Parks algorithm. It is shown that the performance is better by our proposed method. Example 3. Case 3 for odd-order and odd length. A non-full band FIR third-order di!erentiator was designed with length N"27 and u "0.88p. Fig. 5(a) and (b) show the frequency magnitude responses and the error 1 curves, respectively. In Fig. 5(b), we compare the error with the McClellan}Parks algorithm, the deviation is smaller in our case in most of the frequency band excep in the narrow region near the cuto! edge. Example 4. Case 4 for odd-order and even length. A full band "fth-order di!erentiator was designed. With the following designed parameters: N"32 and u "p, we can get the frequency magnitude responses and 1 the error curves shown in Fig. 6(a) and (b), respectively. In the "fth-order di!erentiator design case, the
182
S.-T. Tzeng, H.-C. Lu / Signal Processing 79 (1999) 175}186
Fig. 3. (a) Frequency magnitude responses of a full-band, case 1, second-order di!erentiator with length N"25 and u "p for the GA 1 approach (solid line) and McClellan-Parks algorithm (dotted line). (b) The error curves for the GA approach (solid line) and McClellan}Parks algorithm (dotted line).
McClellan}parks algorithm leads to very large deviation and the GA approach performance is much better and smoother. The performance of this full band di!erentiator is superior to that of McClellan}Parks algorithm.
S.-T. Tzeng, H.-C. Lu / Signal Processing 79 (1999) 175}186
183
Fig. 4. (a) Frequency magnitude response of a nonfull-band, case 2, fourth-order di!erentiator with length N"32 and u "0.92p for 1 the GA approach (solid line) and McClellan}Parks algorithm (dotted line). (b) The error curves for the GA approach (solid line) and McClellan}Parks algorithm (dotted line).
We have summary the performance of these four examples in Table 1 and also indicate in the table the eigen"lter [8] peak errors, the equiripple errors, and the small frequency interval ranges over the equiripple bound. It shows that our performance is much better.
184
S.-T. Tzeng, H.-C. Lu / Signal Processing 79 (1999) 175}186
Fig. 5. (a) Frequency magnitude response of a nonfull-band, case 3, third-order di!erentiator with length N"27 and u "0.88p for 1 the GA approach (solid line) and McClellan}Parks algorithm (dotted line). (b) The error curves for the GA approach (solid line) and McClellan}Parks algorithm (dotted line).
S.-T. Tzeng, H.-C. Lu / Signal Processing 79 (1999) 175}186
185
Fig. 6. (a) Frequency magnitude response of a full band, case 4, "fth-order di!erentiator with length N"32 and u "p for the GA 1 approach (solid line) and McClellan}Parks algorithm (dotted line). (b) The error curves for the GA approach (solid line) and McClellan}Parks algorithm (dotted line).
6. Conclusions In this paper, we have presented a new method for designing higher-order digital di!erentiators by the GA approach and have shown that this method is not only simple and fast but also optimal in the least-squares
186
S.-T. Tzeng, H.-C. Lu / Signal Processing 79 (1999) 175}186
Table 1 Higher-order digital di!erentiators design examples Di!erentiators
Second-order
Fourth-order
Third-order
Fifth-order
Filter length Type of impulse response u : Highest 1 di!erentiating frequency GA approach peak error Eigen"lter peak error McClellan}Parks algorithm equiripple error Small frequency interval: GA approach error over equiripple bound Illustrated "gure
25 case 1 p Full-band 6.1]10~3 8.1]10~3 3.7]10~3 0.99p &p Fig. 3
32 case 2 0.92p Nonfull-band 10.1]10~4 14.6]10~4 4.7]10~4 0.888p &0.92p Fig. 4
27 case 3 0.88p Nonfull-band 5.7]104 9.1]10~4 3.0]10~4 0.848p &0.88p Fig. 5
32 case 4 p Full-band 13.7]10~4 19.6]10~4 9.0]10~4 0.992p &p Fig. 6
sense. The powerful and broadly applicable stochastic search and optimization techniques are shown. It gives much better performance than the McClellan}Parks algorithm. Several numerical design examples are given to illustrate the e!ectiveness and usefulness of this proposed approach. Acknowledgements This work was supported by the National Science Council of Republic of China, under contract NSC-88-2213-E-036-018. The authors are deeply indebted to the reviewers for their interests, careful reviews of this manuscript, observations, and valuable suggestions. Responding to their helpful comments helped us improve this paper. References [1] [2] [3] [4] [5] [6] [7] [8] [9] [10] [11] [12] [13] [14] [15]
L. Eshelman, J. Scha!er, Real-coded genetic algorithms and interval schemata, Found. Genetic Algorithms 2 (1993) 187}202. M. Gen, R. Cheng, Genetic Algorithms and Engineering Design, Wiley, New York, 1997. D.E. Goldberg, Genetic Algorithms in Search Optimization, and Machine Learning, Addison-Wesley, Reading, MA, 1989. J.H. McClellan, T.W. Parks, L.R. Rabiner, A computer program for designing optimum FIR linear phase digital "lters, IEEE Trans. Audio Electroacoust. 21 (December 1973) 506}526. Z. Michalewicz, Genetic Algorithm#Data Structure"Evolution Programs, 2nd Edition, Spring, New York, 1994. Z. Michalewicz, T. Logan, S. Swaminathan, Evolutionary operations for continuous convex parameter spaces, in: Proceedings of the Third Annual Conference on Evolutionary Programming, 1994, pp. 84}97. H. Muhlenbein, D. Schlierkamp-Voosen, Predictive models for the breeder genetic algorithm I continuous parameter optimization, Evol. Comput 1 (1993) 25}49. S.C. Pei, J.J. Shyu, Eigen"lter design of higher-order digital di!erentiators, IEEE Trans. Acoust. Speech Signal Process. 37 (April 1989) 505}511. L.R. Rabiner, B. Gold, Theory and Application of Digital Signal Processing, Prentice-Hall, Englewood Cli!s, NJ, 1975, pp. 81}84. C.A. Rahenkamp, B.V.K. Vijaya Kumar, Modi"cations to the McClellan, Parks, and Rabiner computer program for designing higher order di!erentiating FIR "lters, IEEE Trans. Acoust. Speech Signal Process. 34 (December 1986) 1671}1674. W. Spears, K.D. Jong, On the virtues of parameterized uniform crossover, in: Proceedings of the Fourth International Conference on Genetic Algorithms, 1991, pp. 230}236. G. Syswerda, Uniform crossover in genetic algorithms, in: Proceedings of the Third International Conference on Genetic Algorithms, 1989, pp. 2}9. P.P. Vaidyanathan, T.Q. Nguyen, Eigen"lters: A new approach to least-squares FIR "lter design and applications including Nyquist "lters, IEEE Trans. Circuits Systems 34 (January 1987) 11}23. B.V.K. Vijaya Kumar, C.A. Rahenkamp, Calculation of geometric moments from Fourier plane intensities, Appl. Opt. 25 (March 1986) 997}1007. S. Usui, I. Amidror, Digital low-pass di!erentiation for biological signal processing, IEEE Trans. Biomed. Eng. BME-29 (October 1982) 686}693.