Multiplierless Adaptive Filtering

Multiplierless Adaptive Filtering

Digital Signal Processing 12, 107–118 (2002) doi:10.1006/dspr.2001.0407, available online at http://www.idealibrary.com on Multiplierless Adaptive Fi...

213KB Sizes 2 Downloads 115 Views

Digital Signal Processing 12, 107–118 (2002) doi:10.1006/dspr.2001.0407, available online at http://www.idealibrary.com on

Multiplierless Adaptive Filtering 1 Tamal Bose,∗ Anand Venkatachalam,∗ and Ratchaneekorn Thamvichai† ∗ Electrical and Computer Engineering, Utah State University, Logan, Utah 84322-4120; and † Electrical and Computer Engineering, University of Colorado, Boulder, Colorado 80309-0425

Bose, T., Venkatachalam, A., and Thamvichai, R., Multiplierless Adaptive Filtering, Digital Signal Processing 12 (2002) 107–118. When digital filters are designed with power-of-2 coefficients, the multiplications can be implemented by simple shifting operations. For VLSI implementations, multiplierless filters are faster and more compact than filters with multipliers. In this paper, an algorithm for finding and updating the power-of-2 coefficients of an adaptive filter is designed. The new method uses the well-known Genetic Algorithm (GA) for this purpose. The GA is used in a unique way in order to reduce computations. Small blocks of data are used for the GA and only one new generation is produced per sample of data. This, coupled with the fact that the coefficients are powerof-2, yields a computational complexity of O(N) additions and no multiplications. The algorithm is investigated for applications in adaptive linear prediction and system identification. The results are very promising and illustrate the performance of the new algorithm.  2002 Elsevier Science (USA)

1. INTRODUCTION The use of an adaptive filter offers an attractive solution to the problem of processing signals that result from any operation of unknown statistics. Adaptive filtering is used for various applications such as linear prediction, channel equalization, adaptive noise and echo cancellation, adaptive modeling, and system identification [4, 5]. A wide variety of algorithms for the operation of adaptive filters has been developed in the past few decades. The choice of an algorithm is determined by various factors such as rate of convergence, robustness, computational requirements, numerical properties, and algorithmic structure. 1 This work was supported in part by NASA Grant NAG5-3997 and the Space Dynamics Laboratory.

107 1051-2004/02 $35.00  2002 Elsevier Science (USA) All rights reserved.

Digital Signal Processing Vol. 12, No. 1, January 2002

108

FIG. 1. Representation of an adaptive filter.

During the past decade, multiplierless filters have received widespread attention in the literature because of their high computational speed and lowcost implementation. A multiplierless filter uses power-of-2 coefficients, which require only shifters and adders for filtering the data. Several design methods for multiplierless filters have been also developed [6, 7]. However, to the best of our knowledge there has been no work done on multiplierless adaptive filters. In 1975, Holland [9] introduced an optimization procedure that mimics the process observed in natural evolution known as the Genetic Algorithm (GA). The potential use of GA in signal processing has been explored in [2, 8, 10, 11]. Recently the Genetic Algorithm was introduced for multimodal optimization in adaptive IIR filtering. The use of pure GA has its own disadvantages of slow convergence and high computational complexity. Another algorithm was developed as a hybrid of the gradient descent method and Genetic Algorithm for adaptive IIR filtering [1, 3]. In this paper we propose a new hybrid algorithm for an adaptive multiplierless FIR filter using the combination of the LMS algorithm and the Genetic Algorithm. The paper is organized as follows. In Section 2, we discuss adaptive filtering and the implementation of the LMS algorithm. Section 3 deals with the design procedure of the Genetic Algorithm. Section 4 describes the new algorithm in detail. The implementation of the new algorithm in the applications of linear prediction and system identification are given in Section 5. Concluding remarks are given in Section 6.

2. ADAPTIVE FILTERING The schematic representation of a general adaptive filter is shown in Fig. 1. Let d(n) be a given desired signal and x(n) be the input signal. The input x(n) is processed by a linear filter to produce the output y(n). The goal is to design the linear filter such that some algorithm function based on the error signal in the equation e(n) = d(n) − y(n) is minimized. For the well-known Wiener filtering problem, the objective function is E{e2 (n)}. The output signal y(n) can be calculated as y(n) = CnT Xn ,

(1)

Bose, Venkatachalam, and Thamvichai: Multiplierless Adaptive Filtering

109

where Cn = [cn (0), cn (1), . . . , cn (N − 1)]T

(2)

is the coefficient vector of an FIR filter at the nth iteration, and N is the filter length. The column vector Xn = [xn , xn−1 , xn−2 , . . . , xn−N+1 ]T

(3)

is the input sample vector to the adaptive filter at the nth iteration. The simplest algorithm for updating the coefficients Cn is the Least Mean Square (LMS) algorithm. The LMS algorithm [4] updates the filter coefficients as Cn+1 = Cn + µe(n)Xn ,

(4)

where µ > 0 is the step size. Our goal is to obtain power-of-2 coefficients for the filter at every iteration. The coefficient vector can be quantized into its lower bound and upper bound of power-of-2 values. It can be shown that simple quantization of the coefficients with the LMS does not work well at all. One way of finding the optimal coefficient vector is to form various combinations of the lower bound C˜ nl and the upper bound C˜ nu power-of-2 values. However, the computational complexity increases with filter size since we have 2N combinations of coefficient vectors for a given Cn . To overcome this hurdle, we use the GA to search for an optimal coefficient vector for filtering. The following section briefly explains the idea of using the GA to design multiplierless filters.

3. THE GENETIC ALGORITHM The Genetic Algorithm can be represented by a 5-tuple as GA = (λ, f, S, R, M),

(5)

where λ is the population size, f the fitness/objective function, S the selection operation, R the crossover operation, and M the mutation operation. The population of the GA is comprised of chromosomes where each chromosome contains the powers of 2 used to represent the filter coefficients. For example, the set of filter coefficients can be represented as Cn = {s0 .2e0 , s1 .2e1 , s2 .2e2 , . . . , sN−2 .2eN−2 , 0}, where si stands for the sign and {ei } is the set of integers representing the power. The corresponding chromosome used in the GA is k,n = {g0 , g1 , g2 , . . . , g(N−1) }, where each gi , called the gene of the chromosome, contains the sign bit si and the power-of-2 ei as shown in Fig. 2. The Genetic Algorithm produces new chromosomes (offsprings) based on the crossover operator {R}. The parameters defining the crossover operation are the probability of crossover (pc ) and the crossover point. Two chromosomes from the population based on the selection parameter S are subject to crossover operation to produce new offsprings. The new population comprising

Digital Signal Processing Vol. 12, No. 1, January 2002

110

FIG. 2. Representation of a gene.

the offsprings produced by crossover operation is used for further genetic operations. The crossover point can be typically one or more. Experimental results have shown that multiple crossover point is beneficial to the search process [12]. Mutation is a process by which a gene in the chromosome is replaced with a randomly generated gene with mutation probability pm . Let i,n denote the ith chromosome in the nth generation. Each chromosome is assigned a fitness value fi,n . In this algorithm we define the fitness function as n−1 

fi,n =

  T C˜ Xk − d(k), i,n

(6)

k=n−B−1 T is the power-of-2 coefficient vector obtained from the chromosome where C˜ i,n i,n and B is the block size of data over which we evaluate the filter and calculate the error.

4. ALGORITHM DESCRIPTION The basic idea of the new algorithm is to combine the GA and the LMS algorithm so as to provide a structured search during the adaptation period. The flow chart for the new algorithm is given in Fig. 3. It is composed of the following steps. Step 1: The LMS algorithm is used to guide the GA into a structured search. The LMS runs for the first few samples and updates the filter coefficients at every iteration. Step 2: The filter coefficients are quantized at a stage when the LMS starts converging. Two different sets of coefficients are formed by quantizing Cn to the lower bound C˜ nl and the upper bound C˜ nu , which are power-of-2 values. We then form a set of λ combinations of coefficients from C˜ nl and C˜ nu . Step 3: The GA forms its initial population of size λ. Step 4: The chromosomes of a population must be evaluated for their fitness values so that we can find the best chromosome in the population. For evaluating the fitness we take into consideration the absolute value of the difference between the desired sample and the corresponding filtered output for a blocksize of B. Consider Block I in Fig. 4. For this block, for the nth generation we find the fitness of each chromosome using Eq. (6). The coefficient vector C˜ i,n corresponding to one of the chromosomes, i,n , is used to filter the past data points to obtain the output y(n). Which particular chromosome is used will be explained shortly. Now, the sum of the absolute values of the difference between the filtered outputs and desired samples over Block B gives the fitness value fi,n for each chromosome i,n . Block II in Fig. 4 represents the data

Bose, Venkatachalam, and Thamvichai: Multiplierless Adaptive Filtering

111

FIG. 3. LMS–GA algorithm flow chart.

points considered for evaluating the fitness of the chromosomes for the (n + 1)th generation. Thus for every generation we slide our block by one data point. T X Since the coefficients of the vector C˜ i,n are power-of-2 values, the product C˜ i,n k reduces to simply N shifts and N additions. Note that for every chromosome, the fitness evaluation function requires BN additions and BN shifts. After the fitness evaluation, the chromosomes are arranged in an ascending order of their fitness value fi,n ; i.e., the first chromosome in the population has the lowest fitness value. Note that the lesser the fitness value, the better the chromosome.

FIG. 4.

Pictorial representation of the block over which a chromosome is evaluated.

Digital Signal Processing Vol. 12, No. 1, January 2002

112

Step 5: The best chromosome is extracted from the population and stored in memory. The chromosome remains until it is replaced with another one whose fitness value is better. Step 6: The best chromosome b,n stored in memory is used for filtering the data according to T Xn , y(n) = C˜ b,n

(7)

where C˜ b,n is the filter coefficient vector obtained from the best chromosome b,n at the nth generation. Step 7: In this step we have a new parameter called the error magnitude threshold. If the characteristics of the input signal or noise change with time, the error magnitude jumps to a higher value. Experiments have shown that the GA takes some time to adapt to this characteristic change in the signal. So we have a threshold value that determines whether the statistics of the signal or noise have changed. Based on this, we switch it back to the LMS and filter a few samples, thereby guiding the GA into a structured search. Step 8: In the crossover operation two chromosomes are selected according to the selection parameter S and are subject to multiple crossover. The multiple crosspoints are random integers between 0 and N . The chromosomes produced by the crossover are called the offsprings. Step 9: The offsprings produced are subject to mutation as follows. A random number is generated between 0 and 1. If this number is less than the mutation probability pm , then the offsprings are mutated. After mutation we go back to Step 4, where we evaluate the fitness of our offsprings. After determining the fitness values of these offsprings, we select our new population of size λ by picking the best chromosomes available in our old population and the offsprings.

4.1. Computational Complexity In Step 4 we have seen that for the evaluation of each chromosome we require BN additions and BN shifts. Since we have λ chromosomes, we have a total of λBN additions for evaluating the fitness in each generation. Adding these computations with the filtering of the output in Eq. (7), we have a total of λBN + N ∼ = λBN additions per sample. No multiplications are required at all.

5. APPLICATIONS OF ADAPTIVE FILTERING This section illustrates the implementation of the new algorithm in adaptive applications such as linear prediction and system identification.

5.1. Linear Prediction Consider the problem of predicting the value of a signal x(n) by a weighted linear combination of the past values x(n − 1), x(n − 2), . . . , x(n − p). The most frequently used structure is a transversal filter with coefficients adjusted to minimize some cost function based on the error between the output and the input. Let the signal of interest be s(n), which is corrupted by some noise

Bose, Venkatachalam, and Thamvichai: Multiplierless Adaptive Filtering

FIG. 5.

113

Linear prediction scheme.

sequence w(n). The classical model for the adaptive linear predictor is given in Fig. 5. The corrupted signal d(n) is delayed in order to form the input signal to the filter as x(n) = d(n − D).

(8)

If s(n) has strong periodic components, then a delay will not decorrelate the signal. However, the noise w(n) is assumed to be broadband and so the delay will decorrelate the noise component in d(n) from that in x(n). If w(n) is white, then a unit delay is sufficient to decorrelate it. The signal estimate sˆ (n) is obtained by an FIR filter as sˆ (n) = CnT Xn ,

(9)

where the filter coefficient vector is given by Eq. (2) at the nth iteration, and N is the filter length. The column vector Xn = [xn , xn−1 , xn−2 , . . . , xn−N+1 ]T

(10)

is the input data vector to the adaptive filter at the nth iteration. The nth output error sample is e(n) = d(n) − sˆ (n).

(11)

The LMS algorithm updates the filter coefficients according to Eq. (4). The LMS and GA parameters are set as µ = 0.0005, λ = 25, pm = 0.1, pc = 1.0, and B = 7. The genes of the chromosome are represented simply by an integer that represents the power-of-2 coefficients. The multiplierless filter coefficients lie in the set of {−20 , −2−1 , −2−2 , −2−3 , . . . , 2−7 , 0, 2−6 , 2−5 , . . . , 2−1 , 20 }. The genes are encoded in four bits as {0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15} for the corresponding power-of-2 value. The selection operator S is a random number between 0 and λ. E XAMPLE I. Let the input s(n) be a sinusoidal signal with a frequency of 25 kHz and sampling frequency of 500 kHz. An additive white Gaussian noise of power −13 dB is added to the signal. The adaptive filter is an FIR filter of

Digital Signal Processing Vol. 12, No. 1, January 2002

114

FIG. 6. MSE for Example I.

length N = 100. After filtering 1000 samples, the LMS algorithm gives it to the GA. The squared error of the system is plotted in Fig. 6 (ensemble averaged over 100 runs). Note that the mean square error (MSE) goes down to −12 dB, which is only 1 dB higher than expected. This is quite good given that the filter coefficients are all power of 2. The last 200 samples of the signal estimate sˆ (n) and the input to the adaptive filter x(n) are given in Fig. 7.

FIG. 7.

x(n) and sˆ(n) in Example I.

Bose, Venkatachalam, and Thamvichai: Multiplierless Adaptive Filtering

115

FIG. 8. MSE for Example II.

E XAMPLE II. The input signal consists of a sum of sinusoids of frequencies 5, 10, and 25 kHz. The sampling frequency is 500 kHz. An additive white Gaussian noise of −13 dB power is added to the signal. The FIR filter length is N = 100. The squared error in decibels (ensemble averaged over 100 runs) is shown in Fig. 8 and the last 200 points of the signal estimate and the input signal to the filter are shown in Fig. 9. In this case, the MSE does not go down to −13 dB since the input is not monochromatic. It is well known that the linear predictor performance deteriorates as the signal spectrum widens.

FIG. 9.

x(n) and sˆ(n) in Example II.

Digital Signal Processing Vol. 12, No. 1, January 2002

116

FIG. 10. Identification of unknown plant.

5.2. System Identification An adaptive filter can estimate the transfer function of an unknown plant, using the configuration shown in Fig. 10 [13]. The plant and the adaptive filter are both driven by the signal x(n). The desired signal d(n) is the plant output. The filter output y(n) is given by Eq. (1). E XAMPLE III. Let the input signal s(n) be a white noise and let the additive white Gaussian noise w(n) have a power of −10 dB. The adaptive filter given in Fig. 10 is an FIR with N = 10. The LMS and GA parameters are set as µ = 0.01, λ = 25, pm = 0.1, pc = 0.1, and B = 3. The squared error in decibels (ensemble averaged over 100 runs) is plotted in Fig. 11. We can see that the squared error reaches a level of −8 dB, which is 2 dB above the ideal value.

FIG. 11. MSE in decibels for Example III.

Bose, Venkatachalam, and Thamvichai: Multiplierless Adaptive Filtering

117

6. CONCLUSION A multiplierless adaptive filtering algorithm has been developed and presented in this paper. This algorithm uses the well-known Genetic Algorithm for finding and updating the power-of-2 coefficients of the adaptive filter. The LMS algorithm is also used in conjunction in order to “jump start” the GA. This is done in order to simplify the computational complexity. The new technique uses small blocks of data as input to the GA, which in turn only generates one new population per sample of data. Using this technique, the computational complexity is only O(N) additions and no multiplications. Simulation results show that the new algorithm has very good performance in linear prediction and system identification. The rate of convergence seems to be dictated by the population size λ and the block size B. Faster convergence can be obtained by increasing these parameters, but at the expense of increased computations.

REFERENCES 1. Ng, S. C., Chung, C. Y., Leung, S., and Luk, A., An evolutionary search algorithm for adaptive IIR equalizer. In Proc. IEEE International Symposium on Circuits and Systems, London UK, May 1994, Vol. 2, pp. 53–56. 2. Etter, D. M., Hicks, M. J., and Cho, K. H., Recursive adaptive filter design using an adaptive genetic algorithm. In Proc. IEEE International Conference on Acoustics, Speech, and Signal Processing, 1982, pp. 635–638. 3. Ng, S. C., Chung, C. Y., Leung, S., and Luk, A., Fast convergent genetic search for adaptive IIR filtering. In Proc. IEEE International Conference on Acoustics, Speech, and Signal Processing, Adelaide, South Australia, April 1994, Vol. III, pp. 105–108. 4. Haykin, S. S., Adaptive Filter Theory. Prentice-Hall, Englewood Cliffs, NJ, 1986. 5. Widrow, B. and Stearns, S. D., Adaptive Signal Processing. Prentice-Hall, Englewood Cliffs, NJ, 1985. 6. Thamvichai, R., Bose, T., and Etter, D. M., Design of multiplierless 2-D filters. In Proc. IEEE Intl. Symposium on Intelligent Signal Processing and Communication Systems, Nov. 2000. 7. Xue, P., Adaptive equalizer Using finite-bit power-of-2 quantizer. IEEE Trans. Acoustics Speech Signal Process. 34 (1986), 1603–1611. 8. Haupt, R. L. and Haupt, S. E., Practical Genetic Algorithms. Wiley, New York, 1998. 9. Holland, H., Adaption in Natural and Artificial Systems. Univ. of Michigan Press, Ann Arbor, 1975. 10. Goldberg, D. E., Genetic and evolutionary algorithms come of age. Commun. ACM 37 (1994), 113–119. 11. Mars, P., Chen, J. R., and Nambiar, R., Learning Algorithms, Theory and Applications in Signal Processing Control and Communications. CRC Press, Boca Raton, FL, 1996. 12. Syswerda, G., Uniform crossover in genetic algorithms. In Proc. of the 3rd Intl. Conf. on Genetic Algorithms. Morgan Kaufmann, San Mateo, CA, 1989. 13. Moon, T. K. and Stirling, W. C., Mathematical Methods and Algorithms for Signal Processing. Prentice-Hall, Upper Saddle River, NJ, 2000.

TAMAL BOSE received the Ph.D. in electrical engineering from Southern Illinois University in 1988. He is currently an associate professor at Utah State University at Logan. Prior appointments of Dr. Bose include teaching positions at the Citadel and at the University of Colorado. His research interests are in the areas of adaptive filtering algorithms, stability of multidimensional filters, and

118

Digital Signal Processing Vol. 12, No. 1, January 2002

channel equalization. Dr. Bose served as the associate editor for the IEEE Transactions on Signal Processing from 1992 to 1996. He is currently on the editorial board of the IEICE Transactions on Fundamentals of Electronics, Communications and Computer Sciences, Japan. Dr. Bose received the Researcher of the Year and Service Person of the Year awards at the University of Colorado at Denver and the Outstanding Achievement award at the Citadel. He also received two Exemplary Researcher awards from the Colorado Advanced Software Institute. Dr. Bose has published over 70 papers in the areas of signal processing and communications in technical journals and conference proceedings. Dr. Bose served on the organizing committees of several international conferences. He is a senior member of the IEEE. ANAND VENKATACHALAM is currently a masters student in electrical and computer engineering at Utah State University, Logan. He received his Bachelors degree in electrical and electronics engineering from Birla Institute of Technology and Science, Pilani, India in 1998. He worked with ABB Automation, Bangalore, India from 1998 to 2000. His research interests are in the areas of adaptive filtering algorithms and image processing. RATCHANEEKORN THAMVICHAI received her Bachelors degree in electrical engineering from Chulalongkorn University, Thailand in 1996 and the Masters degree in electrical engineering from Stanford University in 1998. Currently, she is pursuing her Ph.D. in electrical engineering at University of Colorado, Boulder. Her research interests include stability of periodic digital filters and image processing.