Using Bayesian inference for the design of FIR filters with signed power-of-two coefficients

Signal Processing 92 (2012) 2866–2873

Contents lists available at SciVerse ScienceDirect

Signal Processing journal homepage: www.elsevier.com/locate/sigpro

Using Bayesian inference for the design of FIR filters with signed power-of-two coefficients Chung-Yong Chan a,n, Paul M. Goggansb a b

Department of Electrical Engineering and Computer Science, University of Central Florida, Orlando, USA Department of Electrical Engineering, The University of Mississippi, USA

a r t i c l e in f o

abstract

Article history: Received 12 July 2011 Received in revised form 9 May 2012 Accepted 11 May 2012 Available online 22 May 2012

The design approach presented in this paper applies Bayesian inference to the design of finite impulse response (FIR) filters with signed power-of-two (SPoT) coefficients. Given a desired frequency magnitude response specified by upper and lower bounds in decibels, Bayesian parameter estimation and model selection are adapted to produce a distribution of potential designs, all of which perform at or close to the specified standard. In the process, having incorporated prior information such as the maximum acceptable number of SPoT terms and filter length, and the practical design requirement to use the fewest bits possible, the total number of bits, filter taps and SPoT terms, and the filter length required in a design are automatically determined. The produced design candidates have design complexity appropriate to the design specifications and requirements, as designs with higher design complexity than required are rendered less probable by the embedded Ockham’s razor. This innate ability is a prominent advantage that the newly developed framework possesses over many optimization based techniques as it leads to designs that require fewer SPoT terms and filter taps. Most importantly, it avoids the intricacy, arduousness and rigorousness involved in devising an appropriate scheme for balancing design performance against design complexity. & 2012 Elsevier B.V. All rights reserved.

Keywords: Finite impulse response (FIR) filter Signed power-of-two (SPoT) Inference-based design Bayesian method

1. Introduction A design process involves determining appropriate values for the design parameters based on prescribed design specifications and requirements. For a given design problem, the existence and uniqueness of solutions are not guaranteed. A design problem may have more than one solution such that different designs are capable of meeting the same design specifications and requirements. Alternatively, a design problem may have no solution, meaning that no design can be produced to meet the specified standard. A design problem shares these characteristics with inverse problems, and can be treated as a generalized inverse problem. In particular, this notion

n

Corresponding author. E-mail address: [email protected] (C.-Y. Chan).

0165-1684/$ - see front matter & 2012 Elsevier B.V. All rights reserved. http://dx.doi.org/10.1016/j.sigpro.2012.05.009

is applicable to the design of a linear phase finite impulse response (FIR) filter with signed power-of-two (SPoT) coefficients. Given a desired frequency magnitude response, which is characterized by design specifications such as the passband and stopband edge frequencies, the maximum passband ripple, and the minimum stopband attenuation, the goal is to produce a design that realizes the desired frequency response. To obtain a design, the values of all design parameters must be determined, including the total required number of SPoT terms and filter taps, the index of the filter taps, the filter length and the SPoT coefficient values. Depending on the design criteria and complexity, a filter design problem may have several sets of appropriate design parameter values, or none. Bayesian inference has been used extensively to solve generalized inverse problems in a wide variety of applications, and the tools and methods developed for Bayesian

C.-Y. Chan, P.M. Goggans / Signal Processing 92 (2012) 2866–2873

parameter estimation and model comparison can be adapted to solve filter design problems. The Bayesian inference framework for design has already been applied to two design applications: linear antenna arrays [1,2] and linear phase FIR filters with continuous coefficients [3]. Below, the developed framework is extended to design linear phase FIR filters with SPoT coefficients. The design of a linear phase finite impulse response (FIR) filter with signed power-of-two (SPoT) coefficients is commonly formulated as an optimization problem. The goal of the optimization approach is, in general, to minimize the difference between the achieved and desired frequency magnitude response. Many optimization techniques, such as hybrid genetic algorithm [4], polynomial-time algorithm [5], simulated annealing [6], discrete filled function [7], mixed integer linear programming [8] and others [9] have been successful; however, the optimization approach has the drawback that one or more design parameters, such as the number of SPoT terms and filter taps, and the filter length, are fixed in the design process. This limitation leads to filter designs with greater design complexity than necessary. The methods in [5,6,8] use a prescribed fixed filter length and prescribed number of SPoT terms per coefficient or entire filter, and consequently generate designs that have more SPoT terms than required. The method in [4] aims to minimize the required number of SPoT terms and filter taps by including these factors in the fitness function. Even though this author establishes a reasonable fitness function through a simulation study of a large number of filter designs, there is no fundamental principle for the fitness function. Consequently—as will emerge below—the resulting designs still involve more SPoT terms than necessary even though some improvement is attained over the methods in [5,6,8]. These comments show the enormous complexity involved in devising a method for designing filters with design complexity appropriate to the design requirements. This capability is innate in the Bayesian inference framework because it contains a principle of parsimony and quantitatively implements the Ockham’s razor principle, rendering filter designs with greater design complexity than required less probable. The embedded Ockham’s razor principle makes it superfluous to devise complicated and ultimately arbitrary schemes for balancing design performance against design complexity. Automatic determination of the design complexity, done properly, is equivalent to Bayesian model selection. The mechanism by which Ockham’s razor operates in Bayesian model selection is presented at length in, among other places, [10, Chapter 3; 11, Chapter 4; 12, Chapter 28]. In the framework of Bayesian inference, the solution to a filter design problem is encapsulated in the posterior probability distribution, which is a function of the design parameters. Bayes’ theorem, which is an immediate consequence of the sum and product rules of probability, states that the posterior is proportional to the product of the likelihood and the priors. The priors are probability distribution functions which encapsulate prior information about constraints on the design parameters, and the likelihood is obtained by assigning a probability distribution function to the error. In an inverse problem, the error

2867

is defined as the difference between the parametric model and the measured data. There are no data or measurements in a filter design problem, however. By exploiting the parallelism between inference and design problems, the error in a filter design problem can be identified as the difference between the desired and achieved frequency response. The posterior is approximated by a Monte Carlo method, as a closed-form solution cannot be obtained. In this method, a reasonable number of samples are drawn from the posterior using a Markov chain Monte Carlo method [13,14]. From the point of view of design, each posterior sample represents a potential design solution having specific values for the design parameters. As a result, the solution to a design problem consists of a number of design candidates rather than a single final design. Note that unlike many techniques in the optimization framework, the potential designs are all achieved without the aid of a set of continuous filter coefficients that has been predetermined to meet the desired frequency response. To obtain the final design, a designer has to select a single design candidate based on additional design criteria. This paper is organized as follows. Starting with a description of the parametric model, Section 2 presents the basic concepts of the Bayesian inference framework for design. The application of the Bayesian inference framework to two representative design examples is discussed in Section 3, along with a comparison of results. Section 4 concludes with some final reflections. 2. Bayesian inference framework Bayesian inference uses probability theory as extended logic for conducting scientific inference in a systematic and logically consistent way. This inferential approach has been used to solve generalized inverse problems in a multitude of scientific and engineering applications. In Bayesian inference, probability is a measure of a state of knowledge, and Bayes’ rule is used to update the prior state of knowledge in the light of new information. The resulting product of the overall prior and the likelihood which, normalized, gives the posterior distribution, updates the prior knowledge using new data in a way that is uniquely consistent. In model-based inference, the likelihood represents the degree of plausibility that the observed data is produced by the model under consideration, while the prior measures the degree of belief that the model of interest is the correct description before the data is observed. The likelihood is obtained via the assignment of a distribution to the difference between the parametric model and the data, while the priors are distributions assigned on the basis of background information about the model parameters. The posterior is calculated approximately by means of a Monte Carlo method, where an appropriate sampling method such as a Markov chain Monte Carlo method is used to draw a reasonable number of samples from it. (It is easy to multiply the overall prior and the likelihood but difficult to normalize the resulting product so as to give the posterior.) Further details of the theory and application of Bayesian inference and its application in problems of this sort can be found in a number of well written books [10–12,15,16].

2868

C.-Y. Chan, P.M. Goggans / Signal Processing 92 (2012) 2866–2873

where b denotes the maximum integer power. The parameters in Eq. (1) are determined, given values of the parameters in Eq. (2), as follows:

The inference framework for design is analogous to model-based Bayesian inference. In this framework, design specifications play the role of observed data, while a design described by a parametric model corresponds to the model of interest. The mathematical theory behind the Bayesian inference framework for design is presented in the following sections. The exposition begins with the descriptions of the parametric model used in a filter design problem, the specifications of the desired frequency response and the design goals. These topics are followed by an explanation of Bayes’ theorem, the assignment of probability distribution functions, the sampling of the posterior that produces a number of design candidates, and the final decision that gives a single design.

K ¼ max fT n g

ð4Þ

and ck ¼

X

72pn :

ð5Þ

n:T n ¼ k

The sum in Eq. (5) is necessary because the values of Tn for different values of n are not necessarily unique. Eq. (5) states that ck ¼0 if T n ak for n¼1 to N. The parameter N in Eq. (2) is related to the total number of SPoT terms used, J, by X J ¼ 2N 1, ð6Þ n:T n ¼ 0

2.1. The filter model, frequency response specifications and design goals

while the filter length is L ¼ 2 maxfT n g þ1:

This paper uses as examples linear phase FIR filters that have odd length, L, and symmetric impulse response. For this type of filter, the amplitude response is expressed as AðyÞ ¼

K X

ck cosðkyÞ

for 0 r y r p,

Fig. 1 illustrates the desired amplitude response in decibels of a low pass filter using upper and lower bounds at a total of M designated frequencies, ym . In Fig. 1, yp , ys , d, and b respectively denote the passband and stopband edge frequencies, the maximum passband ripple and the minimum stopband attenuation. Because of the discrete nature of the filter coefficients, the parametric model of interest is of the form  2 9AðyÞ9 gðyÞ ¼ 10 log10 for 0 r y r p,

ð1Þ

k¼0

where y is the dimensionless digital frequency, K ¼ ðL1Þ=2, and ck are the filter coefficients. For a filter with coefficients expressed as a sum of SPoT terms, the amplitude response in Eq. (1) can be rewritten as AðyÞ ¼

N X

72

pn

cosðT n yÞ,

O

where the amplitude response is normalized with respect to O, which is the average of the maximum and minimum passband values of AðyÞ. Given the filter model, the design goal is to determine the values for all of the design parameters, N and Xn ¼ fT n ,SPoTn g for n ¼1 to N, so as to produce a filter design with amplitude response that falls between the desired upper and lower limits.

ð2Þ

n¼1

where pn denotes the integer power and Tn takes on the values of the integer tap-position index, k in Eq. (1). In Eq. (1), k is not a parameter. The use of Tn as a parameter in Eq. (2) enables the automatic determination of used and unused taps. The use of SPoT terms constrains the filter coefficients, ck, to have a discrete set of values, which are expressed as a sum of one or more of the following SPoT terms:

2.2. Bayes’ Theorem

SPoTn 2 f1,21 ,22 ,23 , . . . , 2b ,2b , . . . ,23 ,22 ,21 ,1g,

20log10 (1+δ)

The crux of Bayesian inference is Bayes’ theorem. In the context of filter design, Bayes’ theorem can be expressed in

ð3Þ

0

g(θ) (dB)

20log10 (1−δ)

ð7Þ

β

0

θp

θs Frequency

Fig. 1. Design specifications for a low pass log filter.

π

C.-Y. Chan, P.M. Goggans / Signal Processing 92 (2012) 2866–2873

the form pðN,XN 9BU ,BL , r,EÞppðN,XN 9EÞpðBU ,BL 9N,XN , r,EÞ,

ð8Þ

where XN ¼ fT 1 ,SPoT1 ,T 2 ,SPoT2 , . . . ,T N ,SPoTN g are the SPoT term values and tap indexes of a FIR filter, while BU ¼ fBU ðy1 Þ,BU ðy2 Þ,BU ðy3 Þ, . . . ,BU ðyM Þg and BL ¼ fBL ðy1 Þ,BL ðy2 Þ, BL ðy3 Þ, . . . ,BL ðyM Þg respectively denote the upper and lower bounds of the design specifications at the frequency ym . The vector of positive quantities r ¼ fs1 , s2 , s3 , . . . , sM g indicates the required degree of compliance between the desired and achieved frequency response at all designated frequencies. The value at each frequency, sm , is specified by the designer based on the understanding that a smaller value produces a greater degree of compliance. A probability distribution function (pdf) can be of discrete, continuous or mixed form. A discrete pdf has parameters that can take only a discrete number of values, while a continuous pdf has parameters that are defined over a continuous interval. A mixed pdf has a combination of both the discrete and continuous pdfs. To simplify the notation, the symbol p is used to represent all three types of pdfs in this work. Additionally, the symbol E denoting the relevant background information about the design problem of interest is suppressed, since every pdf is conditioned on the background information. It is nevertheless important to keep in mind its existence in all pdfs that follow. Every term in Eq. (8) has a formal name. The term pðN,XN Þ is called the prior pdf because it is conditioned only on the background information at hand. Based on the background information, there is no basis for asserting any dependency between N and XN or between any two of the parameters of XN . All design parameters are therefore treated as logically independent. Upon applying the product rule and treating all design parameters as independent, the term pðN,XN Þ can be expanded as pðN,XN Þ ¼ pðNÞpðXN Þ ¼ pðNÞ

N Y

pðSPoTn ÞpðT n Þ:

ð9Þ

n¼1

The prior pdfs in Eq. (9) are all assigned by the designer based on the design requirements and/or the background information at hand, which could include the maximum acceptable total number of SPoT terms, filter length and integer power. The term pðBU ,BL 9N,XN , rÞ is called the likelihood function and its derivation will be discussed in the next section. Having assigned a pdf to all priors and obtained the likelihood function, the posterior pdf pðN,XN 9BU ,BL , rÞ can be approximated computationally using an appropriate sampling technique.

2.3. The likelihood The derivation of the likelihood begins with the assignment of a pdf to the error or discrepancy between the desired and realized frequency response. At every frequency point ym for 1 rm r M, the error is assigned heuristically a Gaussian pdf: pðBU ,BL 9N,XN , sm Þpexp½Q 2m =2s2m ,

ð10Þ

where 8 > < BL ðym Þgðym Þ Q m ¼ gðym ÞBU ðym Þ > : 0

2869

for gðym Þ oBL ðym Þ, for gðym Þ 4 BU ðym Þ,

ð11Þ

otherwise:

The value for each sm is assigned by the designer. The value assigned corresponds to the width of the error pdf for ym . Through the assignment of sm , a designer can specify, based on design preferences, a different degree of compliance between the desired and achieved frequency response for different regions. For instance, the values assigned to the sm in the passband region are smaller than in the stopband region. This is because the passband region is narrower and has less margin for error than the stopband region. Application of the product rule together with logical independence yields the following likelihood function: pðBU ,BL 9N,XN , rÞ ¼

M Y m¼1

pðBU ,BL 9N,XN , sm Þ "

pexp 

M X

# Q 2m =2 2m

s

,

ð12Þ

m¼1

where the summation term inside the square brackets denotes the total weighted squared error. The maximum number of SPoT terms allowed to represent a filter coefficient is not fixed. Consequently, for the SPoT samples that are drawn to represent a particular filter coefficient, four situations might arise: 1. Two SPoT terms, 2p1 and 2p2 , have opposite sign and identical power, p1 ¼ p2 , yielding an aggregate value of 2p1 2p2 ¼ 0. 2. Two SPoT terms, 2p1 and 2p2 , have the same sign and power, p1 ¼ p2 , yielding an aggregate value that can be expressed by a single SPoT term, 2p1 þ2p2 ¼ 2p1 þ 1 . 3. Two SPoT terms, 2p1 and 2p2 , have opposite sign and sequential powers, p2 p1 ¼ 1, yielding an aggregate value that can be expressed by one SPoT term, 2p1 2p2 ¼ 2p2 . 4. Two SPoT terms, 2p1 and 2p2 with p1 4 0, have the same sign and sequential powers, p2 p1 ¼ 1, yielding an aggregate value that can be expressed by another combination of two SPoT terms, 2p1 þ2p2 ¼ 2p1 þ 1 2p2 . The first three situations must be avoided in order to prevent trivial solutions. The fourth situation, which is detailed in [9], should be prevented in order to avoid a combination of three or more SPoT terms that have the same sign and sequential powers. The reason is that this combination can be replaced by another combination that has fewer SPoT terms. As explained in [9], a combination of two SPoT terms that have the same sign and sequential powers can always be replaced by another combination of two SPoT terms that do not have sequential powers. This assertion fails only if the value of a filter coefficient, ck, is of the form ck ¼

D X d¼0

2d

or

D X

2d for 1r D rb:

d¼0

The nature of the SPoT terms gives rise to a huge number of redundant combinations in representing a

2870

C.-Y. Chan, P.M. Goggans / Signal Processing 92 (2012) 2866–2873

filter coefficient value. This redundancy is undesirable and we believe that it adversely affects the sampling efficiency of the posterior. To overcome redundancy, we devise and implement a penalization scheme by modifying the likelihood in Eq. (12) as follows: " # M X pðBU ,BL 9N,XN , rÞpexp R2 =2s2R  Q 2m =2s2m , ð13Þ m¼1

where R denotes the number of times that one of the four situations is encountered, and similar to sm , sR is a quantity that has to be assigned a value. This value is not critical provided that it does not negate the importance of preventing trivial solutions at the expense of achieving the desired frequency response. A reasonable assignment is to specify a value comparable to sm . 2.4. Sampling of the posterior

2.5. Selecting the final design

The posterior distribution in Eq. (8), which depends on the design parameters N and XN , cannot be determined in closed form. The fix is to obtain a Monte Carlo approximation of the posterior distribution. In the Monte Carlo approximation, a reasonable number of samples are drawn from the posterior distribution using an appropriate sampling technique. In the present work, a program called BayeSys1 [17] which employs the MCMC method is used to draw W samples from the posterior distribution. In filter design, each sample drawn from the posterior distribution represents a design candidate or a potential solution to the design problem of interest. The samples drawn form an approximation to the posterior distribution, so that pðN,XN 9BU ,BL , rÞ 

W 1 X EðNNw ÞdðXN XwNw Þ, Ww¼1

ð14Þ

where Nw is the number of atoms of the wth sample, d is the Dirac delta function and ( 1 for N ¼ N w , ð15Þ EðNNw Þ ¼ 0 otherwise: Eq. (14) can be used to approximate the expected values of functions with respect to the posterior distribution. Use of this property yields the following approximations: /NS 

W 1 X Nw Ww¼1

ð16Þ

and pðN9BU ,BL , rÞ 

W 1 X EðNNw Þ, Ww¼1

samples drawn multiplied by the posterior probability for N, pðN9BU ,BL , rÞ. The Bayesian inference framework has the capability to design filters that have design complexity appropriate to the design requirements. This is because the Bayesian inference framework embodies the principle of parsimony, and quantitatively implements Ockham’s razor principle to render less probable those designs that have higher design complexity than required. Ockham’s razor that is implicit in the Bayesian inference framework ensures that a design with lower design complexity is preferred to one with higher design complexity, when both designs yield the same degree of compliance between the desired and achieved frequency response. This property, acting through the posterior probability for N, pðN9BU ,BL , rÞ, allows automatic determination of the total number of SPoT terms required in a design.

ð17Þ

The Bayesian inference portion of the design process ends with the drawing of the design candidates from the posterior distribution. The W design candidates are all sampled from regions of the posterior distribution where its values are close to or equal to the maximum. As a result, all W design candidates satisfy the prescribed design specifications and requirements with minimal error or none. If a problem has design requirements of immense complexity and adverse constraints on the values of the design parameters that render zero-error design unrealizable, then all W design candidates will have error close to the possible minimum error. All W design candidates have the potential to become the final design of a design problem; therefore, a designer must choose a single final design based on additional criteria. These criteria will depend on the specifics of the application for which a linear phase FIR filter is designed. The design examples presented here are intended primarily to illustrate the design approach. In the absence of any real application that provides additional criteria, we have chosen criteria that seem reasonable and exemplify typical rationales. In all of the examples presented in the next section, a total of W¼ 3000 design candidates are sampled from the posterior distribution, and the final design is selected using a multistep process. The selection process begins by first selecting the design candidates that have zero or minimum error. The next step is to narrow down the candidates to those designs that use the fewest number of SPoT terms. Then, the candidate that has the shortest filter length is chosen as the final design. 3. Results

which are respectively the expected value of N and its posterior probability distribution. Eq. (17) derives from pðN0 9BU ,BL , rÞ ¼ /NN0 S, where N0 is some value of N. Eq. (17) states that the number of samples drawn with N elements is approximately equal to the total number of

For demonstration and comparison purposes, two design examples from [4] are presented here. In both design problems, M¼101 equally spaced frequencies are used. The first problem has the following design specifications:

1 The open-source program BayeSys is available on the website http://www.inference.phy.cam.ac.uk/bayesys/.

In this design problem, the maximum number of bits allowed is specified to be b ¼10, and the following assignments

yp ¼ 0:3p, ys ¼ 0:5p, d ¼ 0:01, and b ¼ 40 dB:

C.-Y. Chan, P.M. Goggans / Signal Processing 92 (2012) 2866–2873

The values assigned to sm in the passband are substantially smaller than in the stopband because in the dB scale, the passband is significantly narrower than the stopband, for which the lower bound is (negatively) infinitely large. The prior distribution for SPoTn is of the form 8 > 23 for SPoTn ¼ 1, > > > pn > 3 > ð2 Þ for SPoTn ¼ 2pn where 0 o pn o b, > 8 > > > < 2b1 for SPoTn ¼ 2b , pðSPoTn Þ ¼ b1 > for SPoTn ¼ 2b , 2 > > > > pn 3 > ð2 Þ for SPoTn ¼ 2pn where 0 o pn o b, > 8 > > > : 23 for SPoT ¼ 1:

are made: pðNÞ ¼ U½1; 50, pðT n Þ ¼ U½0; 20,

sR ¼ 1=10, and (

1=50 dB

for 0 r ym o 0:5p,

1=2:5 dB

for 0:5p r ym r p:

Table 1 Estimated posterior probability for the number of SPoT terms for the low pass filter design problem with minimum stopband attenuation  40 dB. In this case, /NS ¼ 16:39. N

pðN9BU ,BL , rÞ

16 17 18 19 20

0.672000 0.268667 0.056333 0.002333 0.000667

n

This distribution, which renders SPoT terms having large pn values less probable, is used because the practical implementation of a SPoT term that has a smaller pn value requires fewer number of bits. Note that samples from the prior distribution for SPoTn can be generated by first drawing a sample from the continuous distribution, U½1; 1, and then rounding this sample value to the nearest possible value of a SPoT term. The results are summarized in Tables 1 and 2 and Fig. 2. Table 2 lists the parameter values of the final design. Using these parameter values, the frequency magnitude response at the 101 predefined frequency points is plotted in Fig. 2. This plot shows that the resulting frequency magnitude response fully complies with the design specifications. To evaluate the performance of the final design, the normalized

Table 2 Parameter values for the final design of the low pass filter with minimum stopband attenuation  40 dB. For this design, O ¼ 7:94 dB. k

ck

Table 3 Comparison of normalized peak ripples and filter design complexities between different methods.

0

20

1

20 21

2

21 þ 24

3

22 þ 25 þ 27

4

22 þ 24

6

23 25 27

7

24 27

8

24

9

4

2

11

25

Method

NPR (dB)

L

J

Word length (excluding sign bit)

Simulated annealing [6] Polynomial-time algorithm [5] Hybrid genetic algorithm [4] Bayesian inference framework

 41.30  41.35  40.19  40.15

27 23 23 23

54 46 46 35

9 8 7 7

0.9

1.0

0 −10 −20 g(θ) (dB)

sm ¼

2871

−30 −40 −50 −60

0

0.1

0.2

0.3

0.4

0.5 θ/π

0.6

0.7

0.8

Fig. 2. Log magnitude response versus frequency for the low pass filter with minimum stopband attenuation  40 dB.

2872

C.-Y. Chan, P.M. Goggans / Signal Processing 92 (2012) 2866–2873

peak ripple (NPR), defined as the ratio of the peak ripple to the mean value of the passband gain, was computed. The final design emerging from the Bayesian inference framework achieves a normalized peak ripple of 40.15 dB, which Table 4 Estimated posterior probability for the number of SPoT terms for the low pass filter design problem with minimum stopband attenuation  42 dB. In this case, /NS ¼ 28:86.

is comparable to the values obtained in [4–6]. More significantly, the final design produced comparable performance using 7 bits and a total of only J¼35 SPoT terms, which corresponds to a 23.9% saving in SPoT terms used, as shown in Table 3. In the second design problem considered here, the desired frequency response has a narrower transition band and smaller ripple. The design specifications for this problem are as follows:

N

pðN9BU ,BL , rÞ

yp ¼ 0:3p, ys ¼ 0:44p, b ¼ 42 dB and d ¼ 10b=20 :

28 29 30 31 32

0.195000 0.750667 0.052333 0.001667 0.000333

For this design problem, the maximum allowed number of bits and the prior assignments for all design parameters, N, Tn and SPoTn , are unchanged. The assignments for sm are as follows: ( 1=50 dB for 0 r ym o 0:44p, sm ¼ 1=2:5 dB for 0:44p r ym r p:

Table 5 Parameter values for the final design of the low pass filter with minimum stopband attenuation  42 dB. For this design, O ¼ 8:17 dB. k

For this second design example, the results are summarized in Tables 4 and 5 and Fig. 3. Fig. 3 shows the frequency magnitude response produced by the final design, for which the parameter values are listed in Table 5. The plot in Fig. 3 shows that the frequency magnitude response satisfies the design specifications. The final design achieves a normalized peak ripple of 42 dB using 8 bits, a total of J¼57 SPoT terms and a filter length of L¼39 taps. Although the linear phase FIR filter designed by the Bayesian inference framework has a longer filter length, it uses one fewer bit and 17.4% fewer SPoT terms, as shown in Table 6. If a shorter filter length is desired, an appropriate probability distribution can be assigned to Tn instead of the uniform distribution used here.

ck 0

20 25 27

1

20 þ 21

2

21 þ 24 þ 28

3

22 þ 24 27

4

21 þ 23

5

23 þ 26

6

23 þ 25

7

23 þ 25 þ 27

9

23 þ 27

10

24

11

25

12

24

13

26

14

2

15

25

17

6

2

19

27

4. Conclusion

5

The Bayesian inference framework for design has been applied to solve two linear phase FIR filter design problems, in which the filter coefficients are constrained to be a sum of SPoT terms. In both cases the inference

0 −10

g(θ) (dB)

−20 −30 −40 −50 −60 0

0.1

0.2

0.3

0.4

0.5 θ/π

0.6

0.7

0.8

0.9

1.0

Fig. 3. Log magnitude response versus frequency for the low pass filter with minimum stopband attenuation  42 dB.

C.-Y. Chan, P.M. Goggans / Signal Processing 92 (2012) 2866–2873

2873

Table 6 Comparison of normalized peak ripples and filter design complexities between different methods for the second design example. Method

NPR (dB)

L

J

Word length (excluding sign bit)

Simulated annealing [6] Polynomial-time algorithm [5] Hybrid genetic algorithm [4] Bayesian inference framework

 41.60  42.31  41.67  42.00

39 35 33 39

78 70 69 57

9 9 9 8

approach has successfully determined the number of bits, SPoT terms and filter taps, as well as the filter length, which satisfy the design specifications without the aid of a set of continuous filter coefficients predetermined to meet the desired frequency response. Compared to several methods in the literature, the resulting designs yield comparable performance using fewer SPoT terms. This is due to the Ockham’s razor principle intrinsic in the Bayesian approach, which makes it unnecessary to devise a scheme for balancing design performance against design complexity. The ability to automatically determine the appropriate design complexity becomes more valuable for more complicated design problems. In view of this unique ability, the Bayesian inference framework for design offers great potential in the automated design of digital filters. With further development, the approach presented here is likely to become a key design tool for the digital filter design community. References [1] C.-Y. Chan, P.M. Goggans, Using Bayesian inference for linear antenna array design, IEEE Transactions on Antennas and Propagation 59 (9) (2011) 3211–3217. [2] P.M. Goggans, C.Y. Chan, Antenna array design as inference, in: M.S. Lauretto, C.A.B. Pereira, J.M. Stern (Eds.), Bayesian Inference and Maximum Entropy Methods in Science and Engineering, American Institute of Physics, Sao Paulo, Brazil, 2008, pp. 294–300. [3] C.Y. Chan, P.M. Goggans, Using Bayesian inference for linear phase log FIR filter design, in: P.M. Goggans, C.Y. Chan (Eds.), Bayesian Inference and Maximum Entropy Methods in Science and Engineering, American Institute of Physics, Oxford, MS, 2009, pp. 329–335.

[4] L. Cen, A hybrid genetic algorithm for the design of FIR filters with SPoT coefficients, Signal Processing 87 (2007) 528–540. [5] D. Li, Y.C. Lim, Y. Lian, J. Song, A polynomial-time algorithm for designing FIR filters with power-of-two coefficients, IEEE Transactions on Signal Processing 50 (8) (2002) 1935–1941. [6] N. Benvenuto, M. Marchesi, A. Uncini, Applications of simulated annealing for the design of special digital filters, IEEE Transactions on Signal Processing 40 (2) (1992) 323–332. [7] Z.G. Feng, K.L. Teo, A discrete filled function method for the design of FIR filters with signed-powers-of-two coefficients, IEEE Transactions on Signal Processing 56 (1) (2008) 134–139. [8] O. Gustafsson, H. Johansson, L. Wanhammar, An MILP approach for the design of linear-phase FIR filters with minimum number of signed-power-of-two terms, European Conference on Circuit Theory and Design, vol. 2, Helsinki University of Technology, Espoo, Finland, 2001, pp. 217–220. [9] Y.C. Lim, R. Yang, D. Li, J. Song, Signed power-of-two term allocation scheme for the design of digital filters, IEEE Transactions on Circuits and Systems II: Analog and Digital Signal Processing 46 (5) (1999) 577–584. [10] P. Gregory, Bayesian Logical Data Analysis for the Physical Sciences: s A Comparative Approach with Mathematica Support, Cambridge University Press, 2005. [11] D.S. Sivia, J. Skilling, Data Analysis: A Bayesian Tutorial, 2nd edition, Oxford University Press, 2006. [12] D.J.C. MacKay, Information Theory, Inference, and Learning Algorithms, Cambridge University Press, 2003. [13] W.R. Gilks, S. Richardson, D.J. Spiegelhalter, Markov Chain Monte Carlo in Practice, Chapman and Hall, CRC, 1996. [14] P.J. Green, Reversible jump Markov chain Monte Carlo computation and Bayesian model determination, Biometrika 82 (4) (1995) 711–732. [15] E.T. Jaynes, Probability Theory: The Logic of Science, Cambridge University Press, 2003. [16] C.P. Robert, The Bayesian Choice: From Decision-Theoretic Foundations to Computational Implementation, 2nd edition, SpringerVerlag, New York, 2003. [17] J. Skilling, BayeSys and MassInf, Maximum Entropy Data Consultants Ltd, online manual at /http://www.inference.phy.cam.ac.uk/ bayesys/S, February 2004.