- Email: [email protected]
A note on “Performance analysis of the DCT–LMS adaptive filtering algorithm”
ARTICLE IN PRESS
Signal Processing 85 (2005) 1465–1467 www.elsevier.com/locate/sigpro
Short communication
A note on ‘‘Performance analysis of the DCT–LMS adaptive filtering algorithm’’ K. Mayyas Department of Electrical Engineering, Jordan University of Science and Technology, Irbid 221 10, Jordan Received 27 February 2004
Abstract We show that the performance analysis results of the proposed DCT–LMS algorithm in Kim and Wild [Signal Processing 80 (2000) 1629–1654] are flawed. By making an appropriate additional assumption, we illustrate how the integrity of these results can be maintained. r 2005 Elsevier B.V. All rights reserved. Keywords: Transform domain adaptive LMS algorithm; Variable step size
In [1], the weight error vector of the DCT–LMS algorithm is given by Eq. (26) as vðn þ 1Þ ¼ ½I lðnÞxðnÞxT ðnÞvðnÞ þ lðnÞz0 xðnÞ N 0 ðnÞ,
ð1Þ
where lðnÞ ¼ diag½m1 ðnÞ; m2 ðnÞ; . . . ; mN ðnÞ, and mi ðnÞ is proposed in Eq. (18) as 1 mi ðn þ 1Þ ¼ bmi ðnÞ þ gð1 bÞ . þ ð1=MÞxTi ðnÞxi ðnÞ (2) Using Eq. (1) above, [1] shows that mean weight convergence is guaranteed by 0oE½mi ðnÞ oð2=lmax Þ 8i, where lmax is the maximum eigenFax: +962 27095018.
E-mail address: [email protected].
value of Rxx ¼ TM Ruu TTM . This result is incorrect. Taking the expected value of Eq. (1) above and using the assumptions in [1], we get E½vðn þ 1Þ ¼ ðI E½lðnÞRxx ÞE½vðnÞ,
(3)
can be decomposed into where Rxx Rxx ¼ QKxx QT , where we define the rotated vector v ðnÞ ¼ QT vðnÞ. Then, Eq. (3) above can be rewritten as E½vðn þ 1Þ ¼ ðI QT E½lðnÞQKxx ÞE½vðnÞ.
(4)
Note from Eq. (4) above that the elements of E½vðn þ 1Þ are still coupled due to the fact that QT E½lðnÞQ is a nondiagonal matrix. Therefore, the bound derived in [1] in Eq. (28) is incorrect since it falsely assumes that QT E½lðnÞQ ¼ E½lðnÞ. Ref. [1] proceeds to study the mean-squared error of the algorithm and provides in Eq. (31) a
0165-1684/$ - see front matter r 2005 Elsevier B.V. All rights reserved. doi:10.1016/j.sigpro.2005.03.008
ARTICLE IN PRESS 1466
K. Mayyas / Signal Processing 85 (2005) 1465–1467
recursion for the weight error vector covariance matrix. Since E½lðnÞ is a diagonal matrix with different entries, Eq. (31) in [1] is incorrect and should have appeared as Cðn þ 1Þ ¼ CðnÞ E½lðnÞRxx CðnÞ CðnÞRxx E½lðnÞ þ zmin E½lðnÞRxx lðnÞ þ E½lðnÞxðnÞxT ðnÞvðnÞvT ðnÞ xðnÞxT ðnÞlðnÞ þ E½N 0 ðnÞN T0 ðnÞ.
ð5Þ
Eq. (33) in [1] is erroneous because it is derived from Eq. (31) (which was shown above to be incorrect) and also falsely assumes that QT E½lðnÞQ ¼ E½lðnÞ. As a result, the condition for mean-squared error convergence in Eq. (35), the steady-state excess mean-squared error zex ð1Þ in Eq. (44), misadjustment in Eq. (45), and results for the optimal convergence factor g in Eqs. (46) and (47) are all flawed. The correct version of Eq. (33) can be easily obtained from Eq. (5) above. However, the resultant expression cannot be used to proceed further in the mean-squared analysis because of the appearance of QT E½lðnÞQ in the expression thus rendering it useless. We should point out that another general issue concerning the analysis of adaptive algorithms with variable step size is the fact that the generic linear matrix difference equation in Eq. (4) above (or Eq. (27) in [1]) is time-variant. This is also the case for the variable step size algorithms in [2–5]. In [2–5], transient and steady-state analysis of these algorithms including obtaining sufficient conditions for the algorithm stability are performed using the standard analytical approach to the time-invariant linear matrix difference equation. Though this assumption is not accurate, it has been heavily used to simplify the stochastic analysis of variable step size adaptive algorithms [2–5], where results are shown to predict fairly well the actual behavior of the adaptive algorithm. In [6], a more accurate approach to the analysis of the adaptive algorithm with a simple variable step size equation is presented, and theories for sufficient conditions for stability of time-varying linear systems are investigated in [7,8] and the references therein.
1. Proposed correction We will show here that the integrity of the analytical results in [1] can be maintained if a new assumption is used in the analysis. We assume that all elements of xðnÞ ¼ TM uðnÞ are statistically uncorrelated (along with the assumptions in [1] that they are zero-mean and Gaussian) with s2xi ¼ E½x2i ðnÞ, where xi ðnÞ is the ith element of xðnÞ and i ¼ 0; 1; . . . ; N 1. This assumption is true for the optimum KLT transform and approximately true for the DCT for lowpass inputs [9]. We also follow [2–5] in applying the standard analysis of the time-invariant linear matrix difference equation to the variable step size DCT–LMS algorithm. In this case, it can be seen from Eq. (5) above that the diagonal elements of CðnÞ are governed by cii ðn þ 1Þ ¼ ð1 2E½mi ðnÞs2xi þ 2E½m2i ðnÞs4xi Þ cii ðnÞ ! N 1 X 2 2 2 sxj cjj ðnÞ þ s2N . þ E½mi ðnÞsxi zmin þ j¼0
ð6Þ
Note that owing to the new assumption, li ¼ s2xi and therefore Eq. (34) in [1] is identical to Eq. (6) above for i ¼ j. Thus, all expressions that come after Eq. (34) in [1] are now correct. Moreover, one can easily verify that conditions for mean convergence of the algorithm given by Eq. (28) in [1] are now true where lmax ¼ maxðli Þ0pipN1 ¼ maxðs2xi Þ0pipN1 . References [1] D.I. Kim, P. De Wild, Performance analysis of the DCT–LMS adaptive filtering algorithm, Signal Processing 80 (2000) 1629–1654. [2] D.I. Pazaitis, A.G. Constantinides, A novel kurtosis driven variable steps size adaptive algorithm, IEEE Trans. Signal Process. 43 (3) (March 1999) 864–872. [3] T. Aboulnasr, K. Mayyas, A robust variable step-size LMStype algorithm: analysis and simulations, IEEE Trans. Signal Process. 45 (3) (March 1997) 631–639. [4] V.J. Mathews, Z. Xie, A stochastic gradient adaptive filter with gradient adaptive step size, IEEE Trans. Signal Process. 41 (6) (June 1993) 2075–2087. [5] C.P. Kwong, E.W. Johnston, A variable step size LMS algorithm, IEEE Trans. Signal Process. 40 (7) (July 1992) 1633–1642.
ARTICLE IN PRESS K. Mayyas / Signal Processing 85 (2005) 1465–1467 [6] J. B Evans, P. Xue, B. Liu, Analysis and implementation of variable step size adaptive algorithms, IEEE Trans. Signal Process. 41 (8) (August 1993) 2517–2535. [7] P.H. Bauer, K. Premaratne, J. Duran, A necessary and sufficient condition for robust asymptotic stability of timevariant discrete systems, IEEE Trans. Automat. Control 38 (9) (September 1993) 1427–1430.
1467
[8] F. Mota, E. Kaszkurewicz, A. Bhaya, Robust stabilization of time-varying discrete interval systems, in: Proceedings of the 31st IEEE Conference Decision and Control, December 1992, pp. 341–346. [9] F. Beaufays, Transform-domain adaptive filters: an analytical approach, IEEE Trans. Signal Process. 43 (32) (1995) 422–431.
Signal Processing 85 (2005) 1465–1467 www.elsevier.com/locate/sigpro
Short communication
A note on ‘‘Performance analysis of the DCT–LMS adaptive filtering algorithm’’ K. Mayyas Department of Electrical Engineering, Jordan University of Science and Technology, Irbid 221 10, Jordan Received 27 February 2004
Abstract We show that the performance analysis results of the proposed DCT–LMS algorithm in Kim and Wild [Signal Processing 80 (2000) 1629–1654] are flawed. By making an appropriate additional assumption, we illustrate how the integrity of these results can be maintained. r 2005 Elsevier B.V. All rights reserved. Keywords: Transform domain adaptive LMS algorithm; Variable step size
In [1], the weight error vector of the DCT–LMS algorithm is given by Eq. (26) as vðn þ 1Þ ¼ ½I lðnÞxðnÞxT ðnÞvðnÞ þ lðnÞz0 xðnÞ N 0 ðnÞ,
ð1Þ
where lðnÞ ¼ diag½m1 ðnÞ; m2 ðnÞ; . . . ; mN ðnÞ, and mi ðnÞ is proposed in Eq. (18) as 1 mi ðn þ 1Þ ¼ bmi ðnÞ þ gð1 bÞ . þ ð1=MÞxTi ðnÞxi ðnÞ (2) Using Eq. (1) above, [1] shows that mean weight convergence is guaranteed by 0oE½mi ðnÞ oð2=lmax Þ 8i, where lmax is the maximum eigenFax: +962 27095018.
E-mail address: [email protected].
value of Rxx ¼ TM Ruu TTM . This result is incorrect. Taking the expected value of Eq. (1) above and using the assumptions in [1], we get E½vðn þ 1Þ ¼ ðI E½lðnÞRxx ÞE½vðnÞ,
(3)
can be decomposed into where Rxx Rxx ¼ QKxx QT , where we define the rotated vector v ðnÞ ¼ QT vðnÞ. Then, Eq. (3) above can be rewritten as E½vðn þ 1Þ ¼ ðI QT E½lðnÞQKxx ÞE½vðnÞ.
(4)
Note from Eq. (4) above that the elements of E½vðn þ 1Þ are still coupled due to the fact that QT E½lðnÞQ is a nondiagonal matrix. Therefore, the bound derived in [1] in Eq. (28) is incorrect since it falsely assumes that QT E½lðnÞQ ¼ E½lðnÞ. Ref. [1] proceeds to study the mean-squared error of the algorithm and provides in Eq. (31) a
0165-1684/$ - see front matter r 2005 Elsevier B.V. All rights reserved. doi:10.1016/j.sigpro.2005.03.008
ARTICLE IN PRESS 1466
K. Mayyas / Signal Processing 85 (2005) 1465–1467
recursion for the weight error vector covariance matrix. Since E½lðnÞ is a diagonal matrix with different entries, Eq. (31) in [1] is incorrect and should have appeared as Cðn þ 1Þ ¼ CðnÞ E½lðnÞRxx CðnÞ CðnÞRxx E½lðnÞ þ zmin E½lðnÞRxx lðnÞ þ E½lðnÞxðnÞxT ðnÞvðnÞvT ðnÞ xðnÞxT ðnÞlðnÞ þ E½N 0 ðnÞN T0 ðnÞ.
ð5Þ
Eq. (33) in [1] is erroneous because it is derived from Eq. (31) (which was shown above to be incorrect) and also falsely assumes that QT E½lðnÞQ ¼ E½lðnÞ. As a result, the condition for mean-squared error convergence in Eq. (35), the steady-state excess mean-squared error zex ð1Þ in Eq. (44), misadjustment in Eq. (45), and results for the optimal convergence factor g in Eqs. (46) and (47) are all flawed. The correct version of Eq. (33) can be easily obtained from Eq. (5) above. However, the resultant expression cannot be used to proceed further in the mean-squared analysis because of the appearance of QT E½lðnÞQ in the expression thus rendering it useless. We should point out that another general issue concerning the analysis of adaptive algorithms with variable step size is the fact that the generic linear matrix difference equation in Eq. (4) above (or Eq. (27) in [1]) is time-variant. This is also the case for the variable step size algorithms in [2–5]. In [2–5], transient and steady-state analysis of these algorithms including obtaining sufficient conditions for the algorithm stability are performed using the standard analytical approach to the time-invariant linear matrix difference equation. Though this assumption is not accurate, it has been heavily used to simplify the stochastic analysis of variable step size adaptive algorithms [2–5], where results are shown to predict fairly well the actual behavior of the adaptive algorithm. In [6], a more accurate approach to the analysis of the adaptive algorithm with a simple variable step size equation is presented, and theories for sufficient conditions for stability of time-varying linear systems are investigated in [7,8] and the references therein.
1. Proposed correction We will show here that the integrity of the analytical results in [1] can be maintained if a new assumption is used in the analysis. We assume that all elements of xðnÞ ¼ TM uðnÞ are statistically uncorrelated (along with the assumptions in [1] that they are zero-mean and Gaussian) with s2xi ¼ E½x2i ðnÞ, where xi ðnÞ is the ith element of xðnÞ and i ¼ 0; 1; . . . ; N 1. This assumption is true for the optimum KLT transform and approximately true for the DCT for lowpass inputs [9]. We also follow [2–5] in applying the standard analysis of the time-invariant linear matrix difference equation to the variable step size DCT–LMS algorithm. In this case, it can be seen from Eq. (5) above that the diagonal elements of CðnÞ are governed by cii ðn þ 1Þ ¼ ð1 2E½mi ðnÞs2xi þ 2E½m2i ðnÞs4xi Þ cii ðnÞ ! N 1 X 2 2 2 sxj cjj ðnÞ þ s2N . þ E½mi ðnÞsxi zmin þ j¼0
ð6Þ
Note that owing to the new assumption, li ¼ s2xi and therefore Eq. (34) in [1] is identical to Eq. (6) above for i ¼ j. Thus, all expressions that come after Eq. (34) in [1] are now correct. Moreover, one can easily verify that conditions for mean convergence of the algorithm given by Eq. (28) in [1] are now true where lmax ¼ maxðli Þ0pipN1 ¼ maxðs2xi Þ0pipN1 . References [1] D.I. Kim, P. De Wild, Performance analysis of the DCT–LMS adaptive filtering algorithm, Signal Processing 80 (2000) 1629–1654. [2] D.I. Pazaitis, A.G. Constantinides, A novel kurtosis driven variable steps size adaptive algorithm, IEEE Trans. Signal Process. 43 (3) (March 1999) 864–872. [3] T. Aboulnasr, K. Mayyas, A robust variable step-size LMStype algorithm: analysis and simulations, IEEE Trans. Signal Process. 45 (3) (March 1997) 631–639. [4] V.J. Mathews, Z. Xie, A stochastic gradient adaptive filter with gradient adaptive step size, IEEE Trans. Signal Process. 41 (6) (June 1993) 2075–2087. [5] C.P. Kwong, E.W. Johnston, A variable step size LMS algorithm, IEEE Trans. Signal Process. 40 (7) (July 1992) 1633–1642.
ARTICLE IN PRESS K. Mayyas / Signal Processing 85 (2005) 1465–1467 [6] J. B Evans, P. Xue, B. Liu, Analysis and implementation of variable step size adaptive algorithms, IEEE Trans. Signal Process. 41 (8) (August 1993) 2517–2535. [7] P.H. Bauer, K. Premaratne, J. Duran, A necessary and sufficient condition for robust asymptotic stability of timevariant discrete systems, IEEE Trans. Automat. Control 38 (9) (September 1993) 1427–1430.
1467
[8] F. Mota, E. Kaszkurewicz, A. Bhaya, Robust stabilization of time-varying discrete interval systems, in: Proceedings of the 31st IEEE Conference Decision and Control, December 1992, pp. 341–346. [9] F. Beaufays, Transform-domain adaptive filters: an analytical approach, IEEE Trans. Signal Process. 43 (32) (1995) 422–431.