Steady-state mean-square error analysis of regularized normalized subband adaptive filters

Signal Processing 93 (2013) 2648–2652

Contents lists available at SciVerse ScienceDirect

Signal Processing journal homepage: www.elsevier.com/locate/sigpro

Fast communication

Steady-state mean-square error analysis of regularized normalized subband adaptive filters Jingen Ni n, Xiaoping Chen School of Electronic and Information Engineering, Soochow University, Suzhou 215006, China

a r t i c l e in f o

abstract

Article history: Received 5 January 2013 Received in revised form 20 March 2013 Accepted 24 March 2013 Available online 2 April 2013

The normalized subband adaptive filter (NSAF) has faster convergence rate than the normalized least-mean-square (NLMS) algorithm for colored input signals. Regularization of the NSAF is of importance in practical applications. In this paper, we analyze the steadystate mean-square error (MSE) of regularized NSAFs. The analysis is carried out based on the derivation of a variable regularization matrix NSAF (VRM-NSAF). Theoretical expressions for the steady-state MSE of two regularized NSAFs are derived under some assumptions. Simulation results are given to support the theoretical analysis. & 2013 Elsevier B.V. All rights reserved.

Keywords: Mean-square error Normalized subband adaptive filter Regularization

1. Introduction Subband adaptive filtering is widely used in applications such as system identification, network and acoustic echo cancellation, and active noise control, due to its capability of increasing convergence rate for colored input signals. In some early subband adaptive filters (SAFs), each subband employs an individual adaptive sub-filter in its own adaptation loop, and thus the convergence rate of such SAFs is decreased by aliasing and band-edge effects [1]. To solve this problem, Lee and Gan presented a normalized SAF (NSAF) from the principle of minimum disturbance [2], which can be viewed as a subband generalization of the normalized least mean square (NLMS) algorithm but exhibits faster convergence rate. The central idea of the NSAF is to use the subband input signals, normalized by their respective subband input signal variances, to update the tap-weight vector of the filter [3].

n

Corresponding author. Tel.: +86 182 6011 7775. E-mail addresses: [email protected], [email protected] (J. Ni), [email protected] (X. Chen). 0165-1684/$ - see front matter & 2013 Elsevier B.V. All rights reserved. http://dx.doi.org/10.1016/j.sigpro.2013.03.030

Steady-state performance analysis of adaptive filtering algorithms is very important [4–7]. The steady-state mean-square error (MSE) of the NSAF has been studied in [8,9], where the regularization parameter of the NSAF is set to zero for simplicity. In many applications, the regularization parameter for the NSAF cannot be ignored in order to avoid numerical difficulties or to obtain small MSE, such as in the network and acoustic echo cancellation. Besides, the regularization parameters of some other regularized NSAFs, such as the variable regularization matrix NSAF (VRM-NSAF) in [10], are relatively large in steady-state in order to obtain small steady-state MSE. Therefore, the steady-state MSE performance analysis in [8] is not suited for these regularized NSAF and the steadystate MSE performance of regularized NSAFs requires further study. This paper analyzes the steady-state MSE of the fixed regularized NSAF (FR-NSAF) and the VRM-NSAF. The MSE analysis is based on the derivation of the VRM-NSAF. The purpose of this paper is twofold. First, the MSE analysis results in [8] are suited for the NSAF without regularization parameter, but not suited for the FR-NSAF. The results of the steady-state MSE analysis here are suited for both cases. Second, the derivation of the VRM-NSAF is based on the largest decrease of the mean square deviation (MSD) of

J. Ni, X. Chen / Signal Processing 93 (2013) 2648–2652

the adaptive filter tap-weights, which can explain theoretically the fast convergence of the VRM-NSAF [10], but why the VRM-NSAF can obtain small misalignment is not proven. The results of the steady-state MSE analysis here can explain this reason and can, furthermore, show the impact of system parameters on the MSE performance. 2. Date model and regularized NSAFs Consider the desired response arising from the model dðnÞ ¼ uT ðnÞwo þ ηðnÞ;

ð1Þ

2649

the value of the step-size in the sense of controlling the tradeoff between the convergence rate and steady-state MSE. The equivalence of the use of a step-size and the use of a regularization parameter in the NLMS algorithm has been discussed in [11]. The NSAF is an subband extension of the NLMS algorithm, and therefore the conclusion about the equivalence in [11] can also be extended to the NSAF. Besides, simulation results showed that the FR-NSAF using a large regularization parameter and a unity step-size may obtain faster convergence rate than the one using a smaller regularization parameter and a smaller step-size with the same misalignment [10].

T

where u(n)¼[u(n),u(n−1),…,u(n−M+1)] denotes the input vector of length M, wo is the tap-weight vector of an unknown system to be estimated, and η(n) is the system noise. Fig. 1 shows the structure of the NSAF [2], where N is ^ the number of subbands, WðzÞ denotes the adaptive filter ^ whose tap-weight vector is wðkÞ of length M, di,D(k) and yi,D(k), i¼0, 1,…,N−1, are the decimated subband desired responses and outputs, respectively, Hi(z) and Gi(z) are the analysis and synthesis filters, respectively, and ↓N and ↑N represent N-fold decimation and interpolation, respectively. Here, n and k are used to indicate the original and decimated signals, respectively. Regularized NSAFs can be written as [2,10]: ui ðkÞ ^ þ 1Þ ¼ wðkÞ ^ wðk þ ∑ ei;D ðkÞ; 2 i ¼ 0 jjui ðkÞjj þ δi ðkÞ N−1

ð2Þ

where δi(k) is ith subband regularization parameter, and T

ui ðkÞ ¼ ½ui ðkNÞ; ui ðkN−1Þ; ⋯; ui ðkN−M þ 1Þ

ð3Þ

ei;D ðkÞ ¼ di;D ðkÞ−yi;D ðkÞ

ð4Þ

yi;D ðkÞ ¼ uTi ðkÞwo :

with Note that the step-size of the regularized NSAFs is set to unity in this paper. This is because increasing the value of the regularization parameter is equivalent to decreasing

3. Steady-state MSE analysis Define the MSD, subband system noise vector, and subband a priori error vector, respectively, as 2 ^ ; cðkÞ ¼ E½jjwo −wðkÞjj

ð5Þ

ηD ðkÞ ¼ ½η0;D ðkÞ; η1;D ðkÞ; ⋯; ηN−1;D ðkÞT ;

ð6Þ

and ea ðkÞ ¼ ½e0;a ðkÞ; e1;a ðkÞ; ⋯; eN−1;a ðkÞT ;

ð7Þ

where E[⋅] denotes the expectation operator, ηi,D(k) is the ith subband system noise, and ei,a(k) is the ith subband a priori error defined by ^ ei;a ðkÞ ¼ uTi ðkÞ½wo −wðkÞ:

ð8Þ

Assuming that η(n) is i.i.d. and statistically independent of the vectors ui(k), we have [10, Eq. (18)] ( ) N−1 jjui ðkÞjj2 e2i;D ðkÞ e2i;D ðkÞ−η2i;D ðkÞ : −2 cðkþ1Þ−cðkÞ ¼ ∑ E jjui ðkÞjj2 þ δi ðkÞ ½jjui ðkÞjj2 þ δi ðkÞ2 i¼0 ð9Þ This equation can also be obtained by using the energy conservation arguments [12]. According to Fig. 1, using (1), (4), and (8), we have [10]: ei;D ðkÞ ¼ ei;a ðkÞ þ ηi;D ðkÞ:

ð10Þ

With the assumption for η(n) before, (10) implies that E½e2i;D ðkÞ ¼ E½e2i;a ðkÞ þ E½η2i;D ðkÞ:

ð11Þ

Substituting (11) into (9) yields cðkþ1Þ−cðkÞ ( N−1

¼ ∑ E i¼0

−2

jjui ðkÞjj2 e2i;a ðkÞ 2

2

½jjui ðkÞjj þ δi ðkÞ ) e2i;a ðkÞ

þ

jjui ðkÞjj2 η2i;D ðkÞ ½jjui ðkÞjj2 þ δi ðkÞ2

jjui ðkÞjj2 þ δi ðkÞ ( ) N−1 −½jjui ðkÞjj2 þ 2δi ðkÞe2i;a ðkÞ þ jjui ðkÞjj2 η2i;D ðkÞ ¼ ∑ E : ½jjui ðkÞjj2 þ δi ðkÞ2 i¼0 ð12Þ Fig. 1. Block diagram of the NSAF proposed by Lee and Gan. This structure is referred to as multiband structure by the same authors in [1].

When an adaptive filter operates in steady state, the MSD satisfies c(k+1) ¼c(k), as k-∞. Thus, from (12)

2650

J. Ni, X. Chen / Signal Processing 93 (2013) 2648–2652

we get ( ) ( ) N−1 N−1 ½jjui ðkÞjj2 þ 2δi ðkÞe2i;a ðkÞ jjui ðkÞjj2 η2i;D ðkÞ E ¼ ∑ E ∑ ½jjui ðkÞjj2 þ δi ðkÞ2 ½jjui ðkÞjj2 þ δi ðkÞ2 i¼0 i¼0 ð13Þ as k-∞. Assume that the decimated subband signals are uncorrelated such that the variance of ei,a(k)in the ith subband is only due to the variance of ηi,D(k) in the same subband [1]. Then we may deduce that the ith terms of the summations on both sides of (13) correspond to each other as follows: ( ) ( ) ½jjui ðkÞjj2 þ 2δi ðkÞe2i;a ðkÞ jjui ðkÞjj2 η2i;D ðkÞ E ¼ E ð14Þ ½jjui ðkÞjj2 þ δi ðkÞ2 ½jjui ðkÞjj2 þ δi ðkÞ2

written as

" 2 # 2 2 2 1 N−1 2sui ðkÞsηi;D ðkÞ þ 2βsui ðkÞsηi;D ðkÞ MSEβ ¼ ∑ Ni¼0 s2ui ðkÞ þ 2βs2ui ðkÞ " 2 # 2 1 N−1 2sηi;D ðkÞ þ 2βsηi;D ðkÞ ¼ ∑ 1 þ 2β Ni¼0   1 1 N−1 ¼ 1þ  ∑ s2ηi;D ðkÞ: 1 þ 2β Ni¼0 2 Since s2η ¼ ð1=NÞ∑N−1 i ¼ 0 sηi;D ðkÞ [8], (22) reduces to   1 s2 : MSEβ ¼ 1 þ 1 þ 2β η

ð22Þ

ð23Þ

3.2. Steady-state MSE of the VRM-NSAF

as k-∞. For a high-order adaptive filter, the fluctuations of ||ui(k)||2 from one iteration to the next can be assumed to be small enough [8]. Then (14) can be approximately reduced as

In the VRM-NSAF, the regularization parameter δi(k) in (2) is expressed as

Ef½jjui ðkÞjj2 þ 2δi ðkÞe2i;a ðkÞg

δi ðkÞ ¼

Ef½jjui ðkÞjj2 þ δi ðkÞ2 g

¼

E½jjui ðkÞjj2 η2i;D ðkÞ Ef½jjui ðkÞjj2 þ δi ðkÞ2 g

ð15Þ

as k-∞. We can further simplify (15) as Ef½jjui ðkÞjj2 þ 2δi ðkÞe2i;a ðkÞg ¼ E½jjui ðkÞjj2 η2i;D ðkÞ

ð16Þ

as k-∞. Once more with the assumption for η(n) before, we have h i s2u ðkÞs2ηi;D ðkÞ E e2i;a ðkÞ ¼ 2 i ; sui ðkÞ þ 2δi ðkÞ

ð17Þ

where s2ui ðkÞ ¼ E½jjui ðkÞjj2  and s2ηi;D ðkÞ ¼ E½η2i;D ðkÞ. Using (17) in (11) gets s2ei;D ðkÞ ¼

2s2ui ðkÞs2ηi;D ðkÞ þ 2δi ðkÞs2ηi;D ðkÞ s2ui ðkÞ þ 2δi ðkÞ

ð18Þ

as k-∞, where s2ei;D ðkÞ ¼ E½e2i;D ðkÞ. In [8], it has been shown that the steady-state subband MSE and fullband MSE of regularized NSAFs are related by MSE ¼

1 N−1 2 ∑ s ðkÞ N i ¼ 0 ei;D

Substituting (18) into (19) gets " 2 # 2 2 1 N−1 2sui ðkÞsηi;D ðkÞ þ 2δi ðkÞsηi;D ðkÞ MSE ¼ : ∑ Ni¼0 s2ui ðkÞ þ 2δi ðkÞ

ð19Þ

ð20Þ

s2ui ðkÞ : 2 sei;D ðkÞ=s2ηi;D ðkÞ−1

ð24Þ

When the VRM-NSAF operates close to or at steady state, s2ei;D ðkÞ is close to the powers of the subband system noises, s2ηi;D . If (24) is used to estimate δi(k) in this phase, δi(k) may be very large, even negative, in some indices because of the fluctuations in estimating s2ei;D ðkÞ and sui ðkÞ. In order to avoid this problem, if δi(k)o0 or δi ðkÞ≥Q s2ui ðkÞ, then δi(k) is set to [10] δi ðkÞ ¼ Q s2ui ðkÞ

ð25Þ

where Q is a relatively large positive constant. Substituting (25) into (20), we get   1 s2 : ð26Þ MSEQ ¼ 1 þ 1 þ 2Q η From (26), we can see that if Q is greater than 100, the MSE of the VRM-NSAF is small enough. By comparing (23) and (26), we see that the expressions for the MSE of the FR-NSAF and VRM-NSAF have the same form. However, the principles of selecting the values of β and Q are different. The factor β in (23) is used to control the tradeoff between the convergence rate and steady-state MSE of the FR-NSAF, which should be a small value in order to obtain a relatively fast convergence rate, while Q in (26) is used to avoid the estimate fluctuations, which can be relatively large in order to obtain small misalignment. If Q and β are set to the same value, then the VRM-NSAF converges faster than the FR-NSAF with the same steady-state misalignment.

3.1. Steady-state MSE of the FR-NSAF

4. Experimental results

The regularization parameter of the NLMS algorithm is in general set to a value which is related to the input signal power [13]. In the FR-NSAF, we set the regularization parameter in the ith subband to

Consider a system identification problem. The system to be identified is an acoustic echo response, as depicted in Fig. 2, which is truncated to 512 taps. The length of the adaptive filter is also set to 512. The input signal is a white Gaussian sequence or an AR (1) process with a pole at 0.9. A white Gaussian noise is used as the system noise such that SNR¼20 dB or 30 dB. We assume sηi;D ðkÞ for i¼0,1,…,N−1 are known. Besides, the quantities sei;D ðkÞ and sui ðkÞ can be

δi ðkÞ ¼ βs2ui ðkÞ;

ð21Þ

where β is a positive constant. By substituting (21) into (20), the steady-state MSE of the FR-NSAF can be

J. Ni, X. Chen / Signal Processing 93 (2013) 2648–2652

2651

Fig. 2. Acoustic echo response to be identified, which is truncated to 512 taps.

Fig. 4. Steady-state MSE curves of the FR-NSAF for AR (1) input.

Fig. 3. Steady-state MSE curves of the FR-NSAF for white Gaussian input.

estimated via s2x ðkÞ ¼ αs2x ðk−1Þ þ ð1−αÞx2 ðkÞ, where α is a forgetting factor [10], which is set to 0.995 here. An eightband cosine-modulated filter bank is used for the NSAF [14]. Theoretical results are calculated using (23) and (26), and all the simulation results are obtained by averaging over 200 trials and then averaging over more than 1000 instantaneous square errors in the steady-state. Figs. 3 and 4 show the steady-state MSE of the FR-NSAF versus the value of β for the white input signal and the AR (1) input signal, respectively. It is seen that the theoretical results are very close to the simulated results, especially when the value of β is relatively large. For the FR-NSAF, β takes a tradeoff between the convergence rate and misalignment. To guarantee a reasonable convergence rate, the value of β should not be too large. We suggest that it range from 0.1 to 10. Figs. 5 and 6 show the steady-state MSE of the VRMNSAF. According to the VRM-NSAF in [10], the regularization parameters should be large in steady-state in order to obtain small misalignment. Thus, in the simulations the value of Q ranges from 1 to 100. It is also seen that the theoretical results are very close to the simulated results, especially when Q is small, although there is some discrepancy for Q410, which is caused by the estimate fluctuations of the quantities in (24). However, the MSE

Fig. 5. Steady-state Gaussian input.

MSE

curves

of

the

VRM-NSAF

for

white

Fig. 6. Steady-state MSE curves of the VRM-NSAF for AR (1) input.

2652

J. Ni, X. Chen / Signal Processing 93 (2013) 2648–2652

difference between the theoretical and simulation results is small. 5. Conclusion In this paper, the steady-state MSE of regularized NSAFs has been analyzed, which is carried out based on the derivation of the VRM-NSAF. Theoretical expressions for the steady-state MSE of the FR-NSAF and VRM-NSAF have been obtained, which are demonstrated by simulations. From the MSE closed form expressions and simulation results, we can conclude that the FR-NSAF can improve its misalignment by increasing the value of β at the cost of decreasing its convergence rate, and that the VRM-NSAF can obtain a very small misalignment and keep fast convergence rate if Q is greater than certain value, typical values of which range from 10 to 100. Acknowledgment This work was supported in part by the National Natural Science Foundation of China (NSFC) under Grant 61101217 and in part by the Natural Science Foundation of the Jiangsu Higher Education Institutions of China under Grant 11KJD510005. References [1] K.A. Lee, W.S. Gan, S.M. Kuo, Subband Adaptive Filtering: Theory and Implementation, Wiley, Chichester, UK, 2009.

[2] K.A. Lee, W.S. Gan, Improving convergence of the NLMS algorithm using constrained subband updates, IEEE Signal Processing Letters 11 (9) (2004) 736–739. [3] K.A. Lee, W.S. Gan, Inherent decorrelating and least perturbation properties of the normalized subband adaptive filter, IEEE Transactions on Signal Processing 54 (11) (2006) 4475–4480. [4] H.-C. Shin, A.H. Sayed, Mean-square performance of a family of affine projection algorithms, IEEE Transactions on Signal Processing 52 (1) (2004) 90–102. [5] L. Xiao, S. Boyd, S.-J. Kim, Distributed average consensus with leastmean-square deviation, Journal of Parallel and Distributed Computing 67 (2007) 33–46. [6] P.G. Park, C.H. Lee, J.W. Ko, Mean-square deviation analysis of affine projection algorithm, IEEE Transactions on Signal Processing 59 (12) (2011) 5789–5799. [7] J. Chen, A.H. Sayed, Diffusion adaptation strategies for distributed optimization and learning over networks, IEEE Transactions on Signal Processing 60 (8) (2012) 4289–4305. [8] K.A. Lee, W.S. Gan, S.M. Kuo, Mean-square performance analysis of the normalized subband adaptive filter, in: Proceedings of the Asilomar Conference on Signals, Systems and Computers, vol. 40 (2006) pp. 248–252. [9] W. Yin, A.S. Mehr, Stochastic analysis of the normalized subband adaptive filter algorithm, IEEE Transactions on Circuits and Systems I 58 (5) (2011) 1020–1033. [10] J. Ni, F. Li, A variable regularization matrix normalized subband adaptive filter, IEEE Signal Processing Letters 16 (2) (2009) 105–108. [11] J. Benesty, H. Rey, L.R. Vega, S. Tressens, A nonparametric VSS NLMS algorithm, IEEE Signal Processing Letters 13 (10) (2006) 581–584. [12] A.H. Sayed, Adaptive Filters, John Wiley & Sons, NJ, 2008. [13] J. Benesty, C. Paleologu, S. Ciochina, On regularization in adaptive filtering, IEEE Transactions on Speech and Audio Processing 19 (8) (2011) 1734–1742. [14] P.P. Vaidyanathan, Multirate Systems and Filter Banks, Prentice-Hall, Englewood Cliffs, NJ, 1993.