Two-stage fractional least mean square identification algorithm for parameter estimation of CARMA systems

Signal Processing ] (]]]]) ]]]–]]]

1

Contents lists available at ScienceDirect

3

Signal Processing

5

journal homepage: www.elsevier.com/locate/sigpro

7 9 11

Two-stage fractional least mean square identification algorithm for parameter estimation of CARMA systems

13 15 Q1

Muhammad Asif Zahoor Raja a, Naveed Ishtiaq Chaudhary b

17

a

Q2

b

Department of Electrical Engineering, COMSATS Institute of Information Technology, Attock Campus, Attock, Pakistan Department of Electronic Engineering, International Islamic University, Islamabad, Pakistan

19 21 23 25

a r t i c l e i n f o

abstract

Article history: Received 29 December 2013 Received in revised form 15 June 2014 Accepted 17 June 2014

In the present study, single and two-stage least mean square (LMS) adaptive strategies based on fractional signal processing are developed for parameter estimation of controlled autoregressive moving average (CARMA) systems. The main idea is to use fractional LMS identification (FLMSI) and two-stage FLMSI (TS-FLMSI) algorithms for CARMA model that is decomposed into a system and noise models. The performance analyses for both proposed FLMSI and TS-FLMSI schemes are conducted based on adapting the prior known design parameters of the system and comparing the results with standard adaptive algorithms. The accuracy and convergence of the design schemes are verified and validated through the results of statistical analyses based on sufficient number of independent runs to adapt CARMA system. Comparative studies established the dominance of single and two-stage fractional adaptive algorithms over other counterpart in term of model accuracy and reliability in case of different scenarios based on variant signal to noise ratios and step size parameters. & 2014 Elsevier B.V. All rights reserved.

27 29 31

Keywords: Fractional signal processing Least square algorithms Two-stage identification Parameter estimation Fractional adaptive strategies

33 35 37 39 41 43 45 47 49 51 53

63 1. Introduction Need of parameter estimation has gained considerable importance in linear and non-linear systems models arising in signal processing, communication and control engineering [1–3] due to which research community has shown great interest in developing new system identification methods for these models including the iterative methods [4–6], the gradient based algorithms [7–10], the least-squares approaches [11–17], the multiinnovation techniques [18–20] and the hierarchical identification strategies [21–23] etc. Parameter estimation of control autoregressive moving average (CARMA) type systems has been carried out in various studies such as, Ding et al. adapted Hammerstein nonlinear autoregressive

55 57 59

E-mail addresses: [email protected], [email protected] (M.A.Z. Raja), [email protected], [email protected] (N.I. Chaudhary).

moving average with exogenous noise (ARMAX) systems [24], and also for output error (OE) and OE moving average (OE–MA) systems [25], Han et al. addressed effectively the multivariable CARMA systems [26], Zhang et al. estimated multivariable OE–MA systems [27], Bao et al. identified multivariable controlled ARMA systems [28]. Ding et al. provide iterative algorithm for controlled autoregressive autoregressive moving average (CARARMA) systems [29] and so on. Recently, two-stage least squares identification (TS–LSI) algorithm is used extensively for parameter estimation of different system models, for instance, Feng, Guoyu and Honghong have presented TS–LSI algorithms for Box–Jenkins, CARARMA, CARMA and OE type models, respectively [29–32]. However, recently introduced fractional adaptive strategies based on concepts of fractional signal processing (FSP) have great potential to be explored in these fields and aim of this study is to develop a new, accurate, reliable and effective computing framework or platform for these parameter

http://dx.doi.org/10.1016/j.sigpro.2014.06.015 0165-1684/& 2014 Elsevier B.V. All rights reserved.

61 Please cite this article as: M.A.Z. Raja, N.I. Chaudhary, Two-stage fractional least mean square identification algorithm for parameter estimation of CARMA systems, Signal Processing (2014), http://dx.doi.org/10.1016/j.sigpro.2014.06.015i

65 67 69 71 73 75 77 79 81 83 85

M.A.Z. Raja, N.I. Chaudhary / Signal Processing ] (]]]]) ]]]–]]]

2

1 3 5 7 9 11 13 15 17 19 21 23 25 27 29 31 33 35 37 39 41 43 45 47 49

estimation problems by exploiting the renowned strength of FSP. Adaptive algorithms based on FSP are recently developed by introducing the concepts of fractional order calculus in basic formulation of mathematical structure of the algorithm. The introductory material, subject term, theory, importance and applications of FSP can be seen in [33–34] and reference there in. However, the pioneer work of FSP belongs mainly to Ortigueira [35–36]. Tseng et al. applied FSP concepts for designing 1-D and 2-D FIR filters using fractional derivative constraints [37–39] and Wang et al. performed fractional zero phase filtering based on Riemann–Liouville integral [40]. Modern adaptive strategies based on FSP like F-LMS algorithm has been broadly used by the research community in diverse fields since its introduction in 2009 [41]. For example, dual channel speech enhancement, acoustic echo cancellation, performance analysis of Bessel beamformers [42–44] and F-LMS algorithm is effectively applied for identification of input nonlinear control auto regressive (INCAR) systems [45] and outperformed the well-known state of art KLMS and VLMS algorithms. The aim of this study is to investigate in FSP based adaptive algorithms and provide an accurate and reliable computing platform for effective identification of CARMA model with the help of single FLMSI and two stage FLMSI (TS–FLMSI) algorithms. In this paper, the design scheme based on FLMSI and TS–FLMSI algorithms are applied for parameter estimation of CARMA systems and comparative studies are made with least mean square identification LMSI and two-stage LMSI (TS–LMSI) algorithms. Adaptation procedure for these algorithms with different strategies is implemented in numerical experimentation and performance for different scheme is also analyzed by taking both low and high signal to noise ratios (SNRs). Reliable and effective inferences are drawn by determining the value of statistical parameters, i.e., the mean and standard deviation, based on sufficient large number of independent execution of each algorithm for all scenarios of CARMA model. Rest of the paper is organized as follows. In the next section detail description of system model based CARMA model is given. The necessary material about the proposed adaptive methods based on FSP is given in Section 3. In Section 4, results of detail computer simulations are presented for the problems along with discussion on findings. The evaluation for the adaptive schemes in terms of statistical parameter is presented in Section 5. Finally conclusions are provided along with future applications for these FSP based adaptive strategies to well known system identification problems.

51

v(t)

63

D(z) A(z)

65

u(t)

67

y(t)

B(z)

69

A(z)

71

Fig. 1. A system described by CARMA model.

73 here y(t) represents the output of system, x(t) is the input and v(t) is the exogenous noise, A(z), B(z) and D(z) are some know polynomials and given in term of unit backward shift operator z  1 ½z  1 yðtÞ ¼ yðt 1Þ; as AðzÞ ¼ 1 þ a1 z

1

þ a2 z

2

þ …þ ana z

75 77

 na

79

BðzÞ ¼ b1 z  1 þ b2 z  2 þb3 z  3 …þ bnb z  nb DðzÞ ¼ 1 þ d1 z  1 þ d2 z  2 þ … þdnd z  nd

81

Consider the system parameter vector, ΘS which is defined as for single stage CARMA model as

83

ΘS ¼ ½a1 ; a2 ; …; ana ; b1 ; b2 ; …; bnb ; d1 ; d2 ; …; dnd T ; A ℝna þ nb þ nd ;

85

ð2Þ and the corresponding information vector is given as 2 3  yðt 1Þ; yðt  2Þ; …;  yðt na Þ; T 6 xðt  1Þ; xðt  2Þ; …; xðt  n Þ; 7 ϕS ðtÞ ¼ 4 5 A ℝna þ nb þ nd b vðt  1Þ; vðt  2Þ; …; vðt  nd Þ ð3Þ The final model equation for single stage CARMA becomes T

yS ðtÞ ¼ ϕS ðtÞΘS þ vðtÞ;

ð4Þ

where ys(t) is the output of CARMA model for single stage adaptive algorithms. Now for two-stage CARMA system the parameter vector is decomposed in two parts as

ΘTS ¼ ½θ ϑT A ℝna þ nb þ nd ;

ð5Þ

87 89 91 93 95 97 99 101

for the translated vector

103

yTS ðtÞ ¼ φT ðtÞθ þ ψT ðtÞϑ þ vðtÞ

105

where θ is system parameter vector without noise and given as

107

θ ¼ ½a1 ; a2 ; …; ana ; b1 ; b2 ; …; bnb T A ℝna þ nb ;

109

while the noise parameter vector ϑ is given as

111

ϑ ¼ ½d1 ; d2 ; …; dnd T A ℝnd Accordingly, the corresponding information vectors are given as

113 115

53

2. System model: CARMA systems

55

59

In this section, the brief description of system model based on single and two-stage CARMA model is presented. The generic form of the CARMA model is shown in Fig. 1 and governing mathematical relation representing the CARMA system is given as [31]

Eq. (1) by using (5) and (6) is written as

121

61

AðzÞyðtÞ ¼ BðzÞxðtÞ þ DðzÞvðtÞ;

yTS ðtÞ ¼ φT ðtÞθ þ ψT ðtÞϑ þ vðtÞ:

123

57

ϕTS ðtÞ ¼ ½φðtÞ ψðtÞT ; A ℝna þ nb þ nd :

ð1Þ

"

φðtÞ ¼

 yðt  1Þ;  yðt  2Þ; …;  yðt na Þ; xðt 1Þ; xðt  2Þ; …; xðt  nb Þ

ψðtÞ ¼ ½vðt  1Þ; vðt  2Þ; …; vðt  nd ÞT A ℝnd

#T

117

A ℝna þ nb ð6Þ

Please cite this article as: M.A.Z. Raja, N.I. Chaudhary, Two-stage fractional least mean square identification algorithm for parameter estimation of CARMA systems, Signal Processing (2014), http://dx.doi.org/10.1016/j.sigpro.2014.06.015i

119

M.A.Z. Raja, N.I. Chaudhary / Signal Processing ] (]]]]) ]]]–]]]

3

1

yTS ðtÞ ¼ ϕTS ðtÞΘTS þ vðtÞ

3 5

Eq. (7) represents generic two stage parameters identification model for CARMA systems. For further information about the CARMA systems can be seen in [31].

7

3. Design methodology

9

In this section, brief introduction of the proposed single and two stage least mean squares (TS–LMS) techniques, as well as, single and two-stage fractional least mean squares (TS–FLMS) adaptive algorithms are given that are used for parameter estimation of CARMA model described in the last section.

ΘTS ðn þ1Þ ¼ ΘTS ðnÞ þ μ½ϕTTS ðnÞeðnÞ;

3.1. LMSI and TS–LMSI algorithms

Fractional calculus based modified version of LMS was introduced by Zahoor and Qureshi in the field of fractional signal processing in their work on system identification problem [41] and this algorithm is known as Fractional LMS (F-LMS) in the literature. In this study, aim is to formulate an updated version of F-LMS algorithm for parameter estimation of CARMA model. In mathematical construction of F-LMS algorithm, fractional derivative term is also incorporated in addition to usual first order derivative term in the cost function minimization problem. Hence, using cost function given in (8), weight update equation for kth tap in case of F-LMS algorithm is written as

11 13

T

ð7Þ

In case of TS–LMSI algorithm, parameter vector of the CARMA system is decomposed into two parts as given in (5) and two-stage output, yTSLMSI ðnÞ based on adapting both parts using (9) is written as T

69

and the weight update equation in this case is given as ð11Þ

23 25 27 29

The LMS is a well known adaptive algorithm introduced by Widrow and Hoff [46] and used even recently in many application of signal processing including channel equalization, system identification and adaptive beam forming. This algorithm is popular in the class of adaptive procedures due to its simplicity, efficiency and robustness. By taking e(n) as a difference between the desired response d(n) and estimated filter response, yðnÞ and defining the cost function for adaptive filter as JðnÞ ¼ E½jeðnÞj2  ¼ jeðnÞj2

73 75 3.2. FLMSI and TS–FLMSI algorithms

∂JðnÞ ∂f r JðnÞ wk ðn þ 1Þ ¼ wk ðnÞ  μ  μf r ∂wk ∂wf r

M1

2

¼ d ðnÞ  2dðnÞ ∑ wi ðnÞxðn  iÞ i¼0

31 33 35 37 39 41 43 45 47

ð12Þ

79 81 83 85 87 89 91

k

M1

M 1

þ ∑ wi ðnÞxðn iÞ ∑ wj ðnÞxðn  jÞ i¼0

ð8Þ

j¼0

where x(n) be the input of the model. Generally, the expectation in Eq. (8) is not computed. Instead, instantaneous estimate of that expectation is used. In order to find the relations for optimal weight vectors the derivative of the cost function J(n) with respect to M-tap weights vector w is taken and consequently weight updating formula for kth tap weight wk is given as ∂JðnÞ wk ðn þ 1Þ ¼ wk ðnÞ  μ ; k ¼ 0; 1; 2…M– 1; ∂wk where μ is the step size parameter. Simplifying the above weight updating equation to obtain governing the mathematical relations for standard LMS algorithm as wðn þ1Þ ¼ wðnÞ þ μ½xðnÞeðnÞ eðnÞ ¼ dðnÞ yðnÞ yðnÞ ¼ wH ðnÞxðnÞ

51

In case of CARMA model as given in (4), the output, yLMSI ðnÞ by LMSI algorithm using (9) is written as

53

yLMSI ðnÞ ¼ ϕS ðnÞΘS þ vðnÞ;

55 57

where ΘS is the parameter vector to be estimated, ϕS ðnÞ is the information vector and then the weight update equation for LMSI algorithm for single stage is given as

59

ΘS ðn þ 1Þ ¼ ΘS ðnÞ þ μ½ϕ

ð9Þ

T

T

T S ðnÞeðnÞ;

where fr shows the fractional order which is normally taken as real number between 0 and 1, and μf r is step size parameter related to fractional term. The fractional derivative [47] of order f r of a function xm is defined as Df r xm ¼

Г ðm þ 1Þ

xm  f r ; for m 4  1;

93 95 97

ð13Þ

99

where Df r represent the fraction order derivative operator. There are other definitions available in fractional calculus text books including famous definitions of Riemann–Liouville (RL) and Caputo for the function u(x) with order fr40. Fractional derivative in case of RL is given as [47] ( fr Rx nfr 1 dn Da;x uðxÞ ¼ Γ ðn1 f rÞ dx uðτÞdτ; n a ðx  τ Þ

101

Г ðm  f r þ 1Þ

x 4a;

103 105 107

n  1 o f r rn; 109

49

61

71

77

17

21

65 67

yTSLMSI ðnÞ ¼ ϕTS ðnÞΘTS þ vðnÞ;

15

19

63

ð10Þ

Here μ denotes the step size parameter that is normally used to control the convergence rate of the algorithm.

Similarly for Caputo fractional derivative is written as ( fr Rx Da;x uðxÞ ¼ Γ ðn1 f rÞ a ðx  τÞn  f r  1 uðnÞ ðτÞdτ; x 4a;

n  1 o f r rn

where for simplicity, Dνa;x , is given without subscripts as, Dfr, and represents the RL or Caputo fractional derivative operators of order fr, and n is a positive integer. The symbol Γ is a gamma function and is representing in integral form as Z 1 Γ ðzÞ ¼ t z  1 e  t dt 0

Equivalence of these definitions is well established in for some known function and also given in [47].

Please cite this article as: M.A.Z. Raja, N.I. Chaudhary, Two-stage fractional least mean square identification algorithm for parameter estimation of CARMA systems, Signal Processing (2014), http://dx.doi.org/10.1016/j.sigpro.2014.06.015i

111 113 115 117 119 121 123

M.A.Z. Raja, N.I. Chaudhary / Signal Processing ] (]]]]) ]]]–]]]

4

1 3 5 7 9 11 13 15

The fractional derivative term of (12) using (8) is written as

The output, yFLMSI ðnÞ by FLMSI based on (15) in case of CARMA system is given as

∂f r JðnÞ

yFLMSI ðnÞ ¼ ϕS ðnÞΘS þvðnÞ;

∂wfkr

¼  2ðeðnÞxðn kÞÞD wk ðnÞ

Using the definition of fractional derivative the above expression is written as ∂f r JðnÞ ∂wfkr

1 w1  f r ðnÞ; ¼ 2eðnÞxðn  kÞ Γ ð2  f rÞ k

then the final iterative formula for kth tap weight of F-LMS algorithm is written as [41]

17

# w1k  f r ðnÞ wk ðn þ 1Þ ¼ wk ðnÞ þ eðnÞxðn kÞ μ þ μf r ; Γ ð2  f rÞ

19

and accordingly in vector form it is given as

21

  w1  f r ðnÞ wðn þ 1Þ ¼ wðnÞ þeðnÞxðnÞ μ þ μf r Γ ð2 f rÞ

23 25 27

65

T

fr

"

ð14Þ

ð15Þ

The last two equations of set (9) together with (15) are governing equations of F-LMS algorithm and further detail for the derivation of F-LMS algorithm can be seen in [41,48].

and then the weight update equation for FLMSI algorithm for single stage is given as " # Θ1S  f r ðnÞ ΘS ðn þ 1Þ ¼ ΘS ðnÞ þ eðnÞϕS ðnÞ μ þ μf r ð16Þ Γ ð2  f rÞ Accordingly on a similar procedure as adapted for TS–LMSI algorithm, the weight update equation of TS–FLMSI method is formulated as " # Θ1TS f r ðnÞ ΘTS ðn þ 1Þ ¼ ΘTS ðnÞ þeðnÞϕTS ðnÞ μ þ μf r ð17Þ Γ ð2  f rÞ

69 71 73 75 77 79

^ is estimated vector of adaptive parameter for where Θ CARMA model by any of adaptive algorithms given in (10), (11), (16) and (17), the true or desired values are represented by Θ, and N ¼ na þ nb þ nd is number of adaptive parameters

85

81 83

87 89 91

31

System Identification in Signal Processing

33

Parameter Estimation of CARMA Systems

Input vector x(t), white noise v(t), Set desire parameter Θ, etc

93 95

Initialization

Problem

97

37 39

67

The performance of the design methodology is analyzed by defining the mean square error for single and twostages, respectively as ( 1 N ^ 1 N ^ ∑ jΘS  ΘS j2 ; ∑ jΘTS  ΘTS j2 δ¼ ð18Þ Ni¼1 Ni¼1

29

35

63

Fitness Calculations Single and Two Stage CARMA Model

99 101 103

41

System Modeling 43

105

ΘS, ΘTS for LMS

107

45 47

LMSI and TS-LMSI Techniques

ΘS, ΘTS for FLMS

FLMSI and TS-FLMSI Techniques

Update with step increment in Θ

109 111

49 51 53

Adaptive Strategies No

55 57 59 61

113

MSE≤ ε or Predefined Cycles Completed

115 117 119

Estimated Parameters Design Variables

Adaptive Algorithms

Yes

Fig. 2. The overall flowchart of proposed schemes for CARMA system.

Please cite this article as: M.A.Z. Raja, N.I. Chaudhary, Two-stage fractional least mean square identification algorithm for parameter estimation of CARMA systems, Signal Processing (2014), http://dx.doi.org/10.1016/j.sigpro.2014.06.015i

121 123

M.A.Z. Raja, N.I. Chaudhary / Signal Processing ] (]]]]) ]]]–]]]

1 3

in the system model. The mathematical relation given in (18) is called as a figure of merit or fitness function or objective function. Moreover, the graphical representation of whole design methodology is summarized in Fig. 2.

5

signal to noise ratios (SNRs), as well as, various step size m parameters.

65 4.1. Problem 1 67

5 7 9 11 13

63

4. Simulations and results In this section, results of simulations are presented by taking two examples of CARMA model using proposed adaptive algorithms, i.e., LMSI, TS–LMSI, FLMSI, and TS– FLMSI techniques. The parameter estimation for the system is evaluated by taking different low to high values for the

In this problem, the simulation studies are performed for the following CARMA system with design computing technique as [31] AðzÞyðtÞ ¼ BðzÞxðtÞ þDðzÞvðtÞ AðzÞ ¼ 1 þa1 z  1 þaz  2 ¼ 1 1:60z  1 þ 0:80z  2

69 71 73

BðzÞ ¼ b1 z  1 þb2 z  2 ¼ 0:40z  1 þ 0:30z  2 DðzÞ ¼ 1 þ d1 z  1 ¼ 1  0:64z  1

75

15

77

17

79

19

81

21

83

23

85

25

87

27

89

29

91

31

93

33

95

35

97

37

99

39

101

41

103

43

105

45

107

47

109

49

111

51

113

53

115

55

117

57

119

59

121

61

Fig. 3. Iterative adaptation of fitness function for each case of all proposed algorithms.

Please cite this article as: M.A.Z. Raja, N.I. Chaudhary, Two-stage fractional least mean square identification algorithm for parameter estimation of CARMA systems, Signal Processing (2014), http://dx.doi.org/10.1016/j.sigpro.2014.06.015i

123

M.A.Z. Raja, N.I. Chaudhary / Signal Processing ] (]]]]) ]]]–]]]

6

1

63

Table 1 Comparison of obtained results against true values of CARMA model for σ 2v ¼ 0:32 .

3

l

Method

5 7

65 67

a2

b1

b2

d1

LMS TS–LMS FLMS TS–FLMS

 1.5959  1.5972  1.5912  1.6000

0.7957 0.7973 0.7967 0.8000

0.3987 0.4004 0.3969 0.4000

0.2972 0.3027 0.2982 0.3001

 0.6448  0.6405  0.6509  0.6400

1.37E-05 4.53E-06 4.40E-05 1.32E-09

69

(10  03, 10  04)

LMS TS–LMS FLMS TS–LMS

 1.5990  1.5968  1.5996  1.6000

0.7989 0.7969 0.7993 0.8000

0.4007 0.3998 0.3992 0.4000

0.2991 0.3026 0.3014 0.3000

 0.6399  0.6404  0.6398  0.6400

7.44E-07 5.43E-06 6.48E-07 5.28E-10

73

(10  04, 10  05)

LMS TS–LMS FLMS TS–FLMS

 1.5999  1.6000  1.6004  1.6000

0.8001 0.8000 0.8002 0.8000

0.4002 0.4000 0.4001 0.4000

0.3001 0.3000 0.3001 0.3000

 0.6401  0.6405  0.6399  0.6400

1.72E-08 6.69E-06 4.67E-08 9.28E-10

 1.6000

0.8000

0.4000

0.3000

 0.6400

0

(10  02, 10  03)

13 15

MSE

a1

9 11

Optimal weights

17 True values

71

75 77 79 81

19

83

21 Table 2

23 25

Comparison of obtained results against true values of CARMA model for σ 2v ¼ 0:62 . l

Method

29 31

(10  03, 10  04)

33 35

(10  04, 10  05)

37 39

Optimal weights

MSE

a1

a2

b1

b2

d1

LMS TS-LMS FLMS TS-FLMS

 1.6206  1.5969  1.6334  1.6000

0.8088 0.7973 0.8079 0.8000

0.3852 0.3995 0.4216 0.4000

0.3101 0.3024 0.3163 0.3000

 0.6432  0.6384  0.6683  0.6390

1.66E-04 5.08E-06 5.42E-04 2.15E-07

LMS TS-LMS FLMS TS-FLMS

 1.6029  1.5967  1.5986  1.5999

0.8005 0.7967 0.7956 0.7999

0.3988 0.4000 0.3990 0.4000

0.3035 0.3025 0.3040 0.3001

 0.6435  0.6409  0.6365  0.6401

6.87E-06 5.81E-06 1.00E-05 5.74E-09

LMS TS-LMS FLMS TS-FLMS

 1.5994  1.6000  1.6011  1.6000

0.8004 0.8000 0.7982 0.8000

0.3997 0.4000 0.4002 0.4000

0.2997 0.3000 0.3024 0.3000

 0.6381  0.6400  0.6397  0.6400

8.78E-07 7.75E-06 2.09E-06 8.91E-10

 1.6000

0.8000

0.4000

0.3000

 0.6400

0

27 (10  02, 10  03)

85

True values

87 89 91 93 95 97 99 101 103

41 Table 3

43 45

Comparison of obtained results against true values of CARMA model for σ 2v ¼ 0:92 . l

Method

(10

, 10

 03

)

49 51

(10  03, 10  04)

53 55

(10  04, 10  05)

57 59

True values

Optimal weights

MSE

a1

a2

b1

b2

d1

LMS TS-LMS FLMS TS-FLMS

 1.6294  1.5960  1.6187  1.6000

0.8361 0.7966 0.8036 0.8000

0.4100 0.4006 0.4136 0.4000

0.3533 0.3035 0.2815 0.3000

 0.6467  0.6360  0.7300  0.6418

1.03E-03 1.12E-05 1.80E-03 6.19E-07

LMS TS-LMS FLMS TS-FLMS

 1.5950  1.5961  1.5852  1.6000

0.7900 0.7963 0.7859 0.8000

0.3917 0.4001 0.4005 0.4000

0.3053 0.3032 0.3048 0.3000

 0.6367  0.6379  0.6456  0.6396

4.67E-05 8.79E-06 9.41E-05 2.61E-08

LMS TS-LMS FLMS TS-FLMS

 1.5980  1.6000  1.6057  1.6000

0.7919 0.8000 0.7987 0.8000

0.4023 0.4000 0.4045 0.4000

0.3052 0.3000 0.3008 0.3000

 0.6305  0.6386  0.6403  0.6399

3.86E-05 9.09E-06 1.12E-05 1.83E-09

 1.6000

0.8000

0.4000

0.3000

 0.6400

0

47  02

105 107 109 111 113 115 117 119 121 123

61 Please cite this article as: M.A.Z. Raja, N.I. Chaudhary, Two-stage fractional least mean square identification algorithm for parameter estimation of CARMA systems, Signal Processing (2014), http://dx.doi.org/10.1016/j.sigpro.2014.06.015i

M.A.Z. Raja, N.I. Chaudhary / Signal Processing ] (]]]]) ]]]–]]]

1 3 5

ΘS ¼ ½a1 ; a2 ; b1 ; b2 ; d1 T ¼ ½  1:60; 0:80; 0:40; 0:30;  0:64T

respectively, as

ΘTS ¼ ½θT ϑT T

( 1 5 ^ δ¼ ∑ jΘS  ΘS j2 ; 5i¼1

7 ð19Þ

9 11 13 15

63 1 5 ^ ∑ jΘTS  ΘTS j2 5i¼1

ð20Þ

65 67

θ ¼ ½a1 ; a2 ; b1 ; b2 T ¼ ½ 1:60; 0:80; 0:40; 0:30T ϑ ¼ ½d1 T ¼  0:64

7

For experimentation, the input vector, x(t) is taken as uncorrelated persistent excitation signal having zero mean and unit variance, while the white noise vector, v(t) is taken with zero mean and variance, σ 2v . In case of true values given in (19), the fitness functions (18) for both single and two stage adaptive algorithms are given,

The proposed adaptive algorithms based on FLMSI and ^ TS–FLMSI are applied to find the optimal weight vector Θ for CARMA system using sufficient large number of iterations and these optimal parameters are also determined with LMSI and TS–LMSI algorithms for comparison. Three type of step size variation strategies are taken for each adaptive algorithm; in first case initially, m ¼10  2 is taken for faster convergence for ten percent of the total iterations and later on m ¼10  3 for the stability, while for second and third cases these parameters varies as

69 71 73 75 77

17

79

19

81

21

83

23

85

25

87

27

89

29

91

31

93

33

95

35

97

37

99

39

101

41

103

43

105

45

107

47

109

49

111

51

113

53

115

55

117

57

119

59

121

61

Fig. 4. Absolute error for each case of all proposed adaptive algorithms.

Please cite this article as: M.A.Z. Raja, N.I. Chaudhary, Two-stage fractional least mean square identification algorithm for parameter estimation of CARMA systems, Signal Processing (2014), http://dx.doi.org/10.1016/j.sigpro.2014.06.015i

123

8

1 3 5 7 9 11 13 15 17 19 21 23 25

M.A.Z. Raja, N.I. Chaudhary / Signal Processing ] (]]]]) ]]]–]]]

m ¼(10  3–10  4) and m ¼(10  4–10  5), respectively. The design schemes are evaluated in-depth for given CARMA system (19) by taking three different scenarios of low to high levels of noise variance, i.e., σ 2v ¼ 0.3, 0.6 and 0.9. The order of fractional derivative in both variants of F-LMS algorithm is set to be fr ¼0.5 with the step size strategies are mfr ¼(10  2–10  3), (10  3–10  4), and (10  4–10  5). The learning curves of each algorithm based on value of fitness δ are presented graphically in Fig. 3, for all three cases of step size strategy m, and for each noise variant σ 2v . It is observed that for not only the higher and lower values of SNR but also for each variation of step size m, parameter all four adaptive strategies provide convergent and accurate results but the precision of TS–FLMSI algorithm is much better than the rest of algorithm that is quite evident specially at later stages of adaptation, while the initial convergence rate is relatively better for FLMSI algorithm. Moreover, with the decrease in value of step size parameter, further enhancement in the stability is observed for all adaptive algorithms. In order to analyze the level of accuracy achieved in adapting the design parameters of CARMA model by LMSI, TS–LMSI, FLMSI and TS–FLMSI technique, the values of mean square error δ are calculated for each scheme and results are given in Tables 1–3 for σ 2v ¼ 0.32, 0.62 and 0.92, respectively, in case of all three variants of step size. It is

seen that on average TS–FLMSI, TS–LMSI, FLMSI and LMSI algorithms achieves the respective accuracy of the order 10–10, 10  6, 10  6, 10  7, for σ 2v ¼ 0.32, 10  9, 10  5, 10  6, 10  6, for σ 2v ¼ 0.62, and 10  8, 10  6, 10  4, 10  4, for σ 2v ¼ 0.92; which established the superior performance of TS–FLMS from rest. The absolute error (AE) is a measure of absolute deviation between the true and optimal parameters values obtained through the proposed design schemes. The values of AE are determined for each element of the design parameter and results are presented in Fig. 4 for all noise and step size variations. The results are plotted on semi-log scale in order to elaborate the small variations more evidently. The bigger length of bars in the graphs shows the smaller value of the AE, hence represents more accurate results. Length of bar giving the results of TS–FLMSI method is generally taller than that of LMSI, TS–LMSI and FLMSI algorithms which shows its dominance in accuracy. The effects of suitable choice of fractional order on the performance of the fractional adaptive algorithms are also analyzed. The decision of suitable fractional order is bit tricky, and no concrete scientific procedure could be given based on physics of the model. However, in simulations the value of fractional order is chosen by applications of the three clusters of fractional orders between 0 and 1 in F-LMS algorithms for parameter estimation of CARMA

63 65 67 69 71 73 75 77 79 81 83 85 87

27

89

29

91

31

93

33

95

35

97

37

99

39

101

41

103

43

105

45

107

47

109

49

111

51

113

53

115

55

117

57

119

59

121

61

Fig. 5. Results of statistical analysis for m A [10  02 and 10  03].

Please cite this article as: M.A.Z. Raja, N.I. Chaudhary, Two-stage fractional least mean square identification algorithm for parameter estimation of CARMA systems, Signal Processing (2014), http://dx.doi.org/10.1016/j.sigpro.2014.06.015i

123

M.A.Z. Raja, N.I. Chaudhary / Signal Processing ] (]]]]) ]]]–]]]

1 3 5 7 9 11

model; first cluster consists of lower values of fractional orders i.e. fr ¼0.1, 0.2 and 0.3, while in second cluster middle values are taken i.e., fr ¼0.4, 0.5 and 0.6, and for the third cluster fr ¼0.7, 0.8 and 0.9 are used. Parameter estimation of CARMA model is conduction for each cluster for noise variance¼ 0.52 in case of both step size strategies based on 10 independent runs and it is found that for the 50% of runs the best results are obtained for second cluster, while these values for first and third cluster are 20% and 30%, respectively. Therefore, it is reasonable to take the value of fractional order from middle cluster.

13 4.2. Comparative analyses 15 17 19 21

In this section comparative studies are presented for parameter estimation of CARMA model (19) through the results of statistical analysis based on sufficient large number of independent runs for each the adaptive strategies. The evaluation on the performance of each algorithm is carried out through the results of statistical analysis based on 100 independent runs for adaptation of parameters for

9

the CARMA model and result are shown graphically on semi-log scale in Figs. 5–7 in case of m¼ (10  2–10  3), m ¼(10  3–10  4), and m¼ (10  4–10  5), respectively, for σ 2v ¼ 0.32, σ 2v ¼ 0.52, and σ 2v ¼ 0.92. In these figures, the final fitness value as given in (20) is plotted against number of independent runs of each algorithm. It is seen from results that the average values of fitness δ for LMSI, TS–LMSI, FLMSI and TS–FLMSI are in the range of 10  2– 10  4, 10  3–10  5, 10  1–10  4 and 10  3–10  9, respectively in case of m A[10  2 and 10  3] and generally for lower values of SNR the performance of all four algorithms is degraded but still the best results are obtained by TS–FLMSI algorithm. The evaluation of the performance is also measured through statistical parameters, i.e., mean and standard deviation (STD), based on values of fitness δ in case of 100 runs of each algorithm and the results of these parameters are tabulated in Table 4 for all three step size strategies and noise variations. The best fitness value is defined as a minimum (MIN) value of δ achieved by the adaptive algorithm based on all independent runs and these values are also included in Table 4. The respective

63 65 67 69 71 73 75 77 79 81 83

23

85

25

87

27

89

29

91

31

93

33

95

35

97

37

99

39

101

41

103

43

105

45

107

47

109

49

111

51

113

53

115

55

117

57

119

59

121

61

Fig. 6. Results of statistical analysis for different noise variation.

Please cite this article as: M.A.Z. Raja, N.I. Chaudhary, Two-stage fractional least mean square identification algorithm for parameter estimation of CARMA systems, Signal Processing (2014), http://dx.doi.org/10.1016/j.sigpro.2014.06.015i

123

M.A.Z. Raja, N.I. Chaudhary / Signal Processing ] (]]]]) ]]]–]]]

10

1

63

3

65

5

67

7

69

9

71

11

73

13

75

15

77

17

79

19

81

21

83

23

85

25

87

27

89

29

91

31

93

33

95

35

97 Fig. 7. Results of statistical analysis for different noise variation.

37 39 41

l

Method

55

σ 2v ¼ 0:62

103

σ 2v ¼ 0:92

MEAN

STD

MIN

MEAN

STD

MIN

MEAN

STD

LMSI TS-LMSI FLMSI TS-FLMSI

1.37E-05 4.53E-06 4.40E-05 1.32E-09

9.64E-05 2.38E-05 6.78E-04 6.73E-06

4.08E-05 1.16E-05 8.47E-04 7.49E-06

1.66E-04 5.08E-06 5.42E-04 2.15E-07

1.22E-03 7.80E-05 4.50E-03 1.18E-04

5.37E-04 5.95E-05 2.31E-03 1.05E-04

1.03E-03 1.12E-05 1.80E-03 6.19E-07

5.27E-03 3.12E-04 1.96E-02 5.13E-04

2.11E-03 3.02E-04 1.16E-02 5.48E-04

)

LMSI TS-LMSI FLMSI TS-FLMSI

7.44E-07 5.43E-06 6.48E-07 5.28E-10

3.79E-06 1.90E-05 9.69E-06 8.57E-07

1.71E-06 5.69E-06 4.57E-06 8.58E-07

6.87E-06 5.81E-06 1.00E-05 5.74E-09

5.18E-05 2.62E-05 1.19E-04 8.15E-06

2.51E-05 9.82E-06 5.82E-05 8.40E-06

4.67E-05 8.79E-06 9.41E-05 2.61E-08

3.17E-04 5.43E-05 7.09E-04 6.77E-05

1.40E-04 3.33E-05 2.75E-04 6.66E-05

(10  04, 10  05)

LMSI TS-LMSI FLMSI TS-FLMSI

1.72E-08 6.69E-06 4.67E-08 9.28E-10

3.66E-07 1.86E-05 6.92E-07 5.79E-08

1.70E-07 5.61E-06 3.72E-07 5.98E-08

8.78E-07 7.75E-06 2.09E-06 8.91E-10

6.30E-06 2.02E-05 1.02E-05 1.12E-06

3.10E-06 5.39E-06 4.29E-06 1.17E-06

3.86E-05 9.09E-06 1.12E-05 1.83E-09

1.36E-04 2.41E-05 4.90E-05 4.55E-06

4.42E-05 7.85E-06 2.11E-05 5.70E-06

(10  02, 10  03)

(10

 03

, 10

 04

51 53

σ 2v ¼ 0:32 MIN

47 49

101

Table 4 Comparison of statistical parameters for proposed adaptive algorithms.

43 45

99

61

107 109 111 113 115 117 119

57 59

105

ranges for the average values of the best, mean and STD are around 10  7, 10  6, and 10  6 for LMSI, 10  6, 10  5, and 10  6 for TS–LMSI, 10  7, 10  6, and 10  6 for FLMSI, and

10  10, 10  7, and 10  7 for TS–FLMSI algorithms in case of σ 2v ¼ 0.32. Generally the small value of statistical parameters is obtained from all adaptive algorithms but the

Please cite this article as: M.A.Z. Raja, N.I. Chaudhary, Two-stage fractional least mean square identification algorithm for parameter estimation of CARMA systems, Signal Processing (2014), http://dx.doi.org/10.1016/j.sigpro.2014.06.015i

121 123

M.A.Z. Raja, N.I. Chaudhary / Signal Processing ] (]]]]) ]]]–]]]

1 3 5 7 9 11 13 15 17 19 21 23 25

lowest values belong mostly to TS–FLMSI technique which validates the consistent correctness of the approach for parameter estimation of the CARMA system. Computational complexity of each adaptive algorithm is given based on average execution time (AET) taken for finding the optimal weight of CARMA model. The values of AET along with STD are calculated for all proposed schemes based on 100 independent runs for each scenario and results are presented in Table 5. It is seen that the proposed algorithms take more time for smaller values of step size and relatively efficient for bigger values of step size parameter. It is also observed that FLMSI and TS–FLMSI algorithm takes a little longer than that of LMSI and TS–LMSI because of the fact that these algorithms are based on inherent complex fractional calculus structure but this aspect can be compensated due to their ability to provide most accurate and consistently convergent results. The experimentations in this paper are carried out on HP Pro-Book 4530 s, with a 2.30 GHz Core-i3 processor, 4.00 GB RAM, and running MATLAB version 2008a.

11

estimation of the CARMA system given as [49]

63

AðzÞyðtÞ ¼ BðzÞxðtÞ þDðzÞvðtÞ

65

AðzÞ ¼ 1 þa1 z  1 þaz  2 ¼ 1 1:15z  1 þ 0:40z  2

67

BðzÞ ¼ b1 z  1 þb2 z  2 ¼ 1:0z  1 þ 0:50z  2 DðzÞ ¼ 1 þ d1 z  1 þ d2 z  2 ¼ 1  1:0z  1 þ 0:20z  2

69

ΘS ¼ ½a1 ; a2 ; b1 ; b2 ; d1 ; d2 T

71

¼ ½ 1:15; 0:40; 1:0; 0:50; 1:0; 0:20T

ΘTS ¼ ½θT ϑT T θ ¼ ½a1 ; a2 ; b1 ; b2 T ¼ ½ 1:15; 0:40; 1:0; 0:50T ϑ ¼ ½d1 ; d2 T ¼ ½ 1:0; 0:20T

4.3. Problem 2 Simulation study is also performed for the proposed adaptive algorithms by taking another problem for parameter

73 ð21Þ

75

The adaptive algorithms based on LMSI, TS–LMSI, FLMSI and ^ for TS–FLMSI are applied to find the optimal weight vector Θ (21) on a similar pattern, settings and variations used in the last problem. The evaluation of the performance is analyzed only on the basis of statistical parameters, i.e., mean and STD, based on values of fitness δ of each algorithm and results are summarized in Table 6 for all three step size strategies and noise variations. It is inferred that fractional adaptive algorithms perform much better from the rest of algorithms for the relatively complex problem than the

77

31 33

Table 5 Comparison of computational analysis of proposed algorithms. l

σ 2v

39

LMS

TS-LMS

FLMS

TS-FLMS

MET

STD

MET

STD

MET

STD

0.32 0.62 0.92

0.0150 0.0152 0.0148

0.0006 0.0016 0.0004

0.0214 0.0209 0.0613

0.0040 0.0018 0.0044

0.0622 0.0620 0.0613

0.0042 0.0041 0.0044

0.0970 0.0978 0.0957

0.0039 0.0137 0.0034

(10  03, 10  04)

0.32 0.62 0.92

0.1504 0.1485 0.1516

0.0095 0.0062 0.0119

0.2120 0.2072 0.2118

0.0189 0.0089 0.0224

0.6175 0.6135 0.6261

0.0284 0.0192 0.0399

0.9697 0.9627 0.9866

0.0424 0.0405 0.0683

(10  04, 10  05)

0.32 0.62 0.92

1.5028 1.5147 1.4182

0.0752 0.0816 0.1737

2.1020 2.1036 1.9747

0.1226 0.0968 0.2390

6.2016 6.1981 5.8183

0.2325 0.2247 0.6923

9.7410 9.7010 9.1123

0.3287 0.3662 1.0733

(10  02, 10  03)

Table 6 Comparison of statistical parameters for proposed adaptive algorithms for problem 2. l

Method

 02

σ 2v ¼ 0:32

σ 2v ¼ 0:62

101 103

109

σ 2v ¼ 0:92

MIN

MEAN

STD

MIN

MEAN

STD

MIN

MEAN

STD

LMSI TS-LMSI FLMSI TS-FLMSI

1.27E-04 5.16E-05 1.03E-04 1.00E-05

8.67E-04 3.19E-04 3.90E-03 3.84E-05

9.33E-04 2.73E-04 9.59E-03 3.59E-05

2.78E-04 7.25E-05 8.29E-04 1.88E-05

2.11E-03 5.15E-04 4.49E-03 2.04E-04

1.42E-03 2.46E-04 3.27E-03 1.64E-04

5.47E-04 2.21E-04 2.70E-03 3.91E-04

1.05E-02 1.40E-03 3.06E-02 2.19E-03

5.83E-03 1.40E-03 2.07E-02 2.74E-03

(10  03, 10  04)

LMSI TS-LMSI FLMSI TS-FLMSI

1.48E-04 2.55E-05 1.32E-05 1.93E-06

7.62E-04 3.26E-04 3.42E-05 5.61E-06

4.45E-04 2.29E-04 2.89E-05 2.59E-06

2.05E-04 9.60E-05 2.13E-05 6.49E-06

1.52E-03 4.08E-04 2.67E-04 5.15E-05

1.06E-03 2.68E-04 1.55E-04 5.34E-05

1.51E-03 1.59E-04 4.57E-04 1.26E-05

4.55E-03 4.10E-04 1.22E-03 1.38E-04

2.20E-03 1.82E-04 4.22E-04 1.06E-04

LMSI TS-LMSI FLMSI TS-FLMSI

1.37E-04 8.27E-05 5.55E-06 4.00E-07

8.40E-04 2.24E-04 3.10E-05 2.65E-06

6.87E-04 1.15E-04 2.40E-05 1.88E-06

3.09E-04 8.51E-05 1.49E-05 1.84E-06

1.01E-03 4.43E-04 6.13E-05 7.05E-06

3.58E-04 2.48E-04 3.19E-05 5.42E-06

2.05E-03 8.10E-05 9.23E-05 3.32E-06

3.95E-03 2.17E-04 1.48E-04 2.82E-05

1.47E-03 9.24E-05 3.59E-05 2.36E-05

55 57 (10  04, 10  05)

61

99

107

)

(10

, 10

 03

53

59

97

105

49 51

87

95

43

47

85

93

STD

41

45

83

91

MET

35 37

81

89

27 29

79

Please cite this article as: M.A.Z. Raja, N.I. Chaudhary, Two-stage fractional least mean square identification algorithm for parameter estimation of CARMA systems, Signal Processing (2014), http://dx.doi.org/10.1016/j.sigpro.2014.06.015i

111 113 115 117 119 121 123

M.A.Z. Raja, N.I. Chaudhary / Signal Processing ] (]]]]) ]]]–]]]

12

1 3

CARMA system of the last problem. Moreover, comparing the fractional adaptive strategies, it is found that TS–FLMSI remains dominant from FLMSI for this problem as well.

5 7

5. Conclusion

9

Following conclusions are drawn based on results of simulations, presented in the last sections

11

 A single and two stage fractional LMS algorithms are

13 15 17



19 21 23 25 27



29 31 33 35



37 39 41 43 45 47 49



designed based on fractional signal processing approach and applied for parameters estimation CARMA systems; results show that these algorithms are accurate, reliable and effective. For all three case studies of step size μ strategies, it is observed that performance of all the four adaptive algorithms enhanced with decrease in the value of step size parameter but at the cost of higher number of iterations in the process. The proposed TS–FLMSI method provides relatively accurate results for all the step size variations than the rest and remained convergent to achieve more precise results with the increase in number of iterations, while the initial convergence rate of FLMSI is the best. The performance of proposed adaptive schemes is further evaluated by taking different values of noise variances in CARMA systems and it is observed that the design approaches are suitably handle both low and high SNRs but with slight decrease in performance for low SNR in term of accuracy and convergence, however, TS–FLMSI algorithm still gives the better results from the rest. The performance of each algorithm is verified and validated through the results of statistical analysis based on 100 independent runs for adaptation of parameters for the CARMA models and results show that the performance of TS–FLMSI algorithm is invariably better than that of LMSI, TS–LMSI and FLMSI algorithms in term of accuracy, convergence, values of statistical parameter of mean and standard deviation, in case of all variations of step size strategies and noise variances. Computation complexity of fractional adaptive strategies, i.e., FLMSI and TS–FLMSI algorithms, is always on the higher side due to their inherent complex mathematical foundations but that is also the reason of their brilliant performance in accurate modeling of stiff models like CARMA systems.

51 53 55 57 59 61

In future, this contribution may open new dimension for the research community to investigate in the field of fractional signal processing based adaptation strategies and develop the updated versions of these fractional and two-stage fractional LMS algorithms for parameter estimation of control autoregressive (CAR), ARMA, INCAR, CARAR, CARARMA, Hammerstein ARMAX, Box–Jenkins, OE, OEMA, OEAR, nonlinear Wiener models and many others which still remain unsolved.

Appendix A. Supporting information

63

Supplementary data associated with this article can be found in the online version at http://dx.doi.org/10.1016/ j.sigpro.2014.06.015.

65 67 69

References [1] Z.D. Feng, et al., Hierarchical gradient based and hierarchical least squares based iterative parameter identification for CARARMA systems, Signal Process 97 (2014) 31–39. [2] F.K.W. Chan, H.-C. So, W. Sun, Accurate estimation of common sinusoidal parameters in multiple channels, Signal Process 93 (4) (2013) 742–748. [3] F. Ding, G. Liu, X.P. Liu, Parameter estimation with scarce measurements, Automatica 47 (8) (2011) 1646–1655. [4] D. Wang, F. Ding, Least squares based and gradient based iterative identification for Wiener nonlinear systems, Signal Process 91 (5) (2011) 1182–1189. [5] F. Ding, Y. Liu, B. Bao, Gradient-based and least-squares-based iterative estimation algorithms for multi-input multi-output systems, Procee. Instit. Mech. Eng., Part I: J. Syst. Control Eng. 226 (1) (2012) 43–55. [6] Z. Zhang, J. Jia, R. Ding, Hierarchical least squares based iterative estimation algorithm for multivariable Box–Jenkins-like systems using the auxiliary model, Appl. Math. Comput. 218 (9) (2012) 5580–5587. [7] D. Wang, F. Ding, Performance analysis of the auxiliary models based multi-innovation stochastic gradient estimation algorithm for output error systems, Digit. Signal Process 20 (3) (2010) 750–762. [8] D. Wang, G. Yang, R. Ding, Gradient-based iterative parameter estimation for Box–Jenkins systems, Comput. Math. Appl. 60 (5) (2010) 1200–1208. [9] F. Ding, Y. Shi, T. Chen, Gradient-based identification methods for Hammerstein nonlinear ARMAX models, Nonlinear Dyn. 45 (1) (2006) 31–43. [10] Y. Liu, J. Sheng, R. Ding, Convergence of stochastic gradient estimation algorithm for multivariable ARX-like systems, Comput. Math. Appl. 59 (8) (2010) 2615–2627. [11] F. Ding, Decomposition based fast least squares algorithm for output error systems, Signal Process 93 (5) (2013) 1235–1242. [12] D.Q. Wang, Least squares-based recursive and iterative estimation for output error moving average systems using data filtering, IET Control Theory Appl. 5 (14) (2011) 1648–1657. [13] Y. Liu, D. Wang, F. Ding, Least squares based iterative algorithms for identifying Box–Jenkins models with finite measurement data, Digit. Signal Process 20 (5) (2010) 1458–1467. [14] J. Ding, F. Ding, The residual based extended least squares identification method for dual-rate systems, Comput. Mathem. Appl. 56 (6) (2008) 1479–1487. [15] Y. Xiao, et al., The residual based interactive least squares algorithms and simulation studies, Comput. Math. Appl. 58 (6) (2009) 1190–1197. [16] D. Wang, et al., Auxiliary model based recursive generalized least squares parameter estimation for Hammerstein OEAR systems, Math. Comput. Model. 52 (1) (2010) 309–317. [17] F. Ding, Y. Shi, T. Chen, Auxiliary model-based least-squares identification methods for Hammerstein output-error systems, Syst. Control Lett. 56 (5) (2007) 373–380. [18] Y. Liu, Y. Xiao, X. Zhao, Multi-innovation stochastic gradient algorithm for multiple-input single-output systems using the auxiliary model, Appl. Math. Comput. 215 (4) (2009) 1477–1483. [19] L. Han, F. Ding, Identification for multirate multi-input systems using the multi-innovation identification theory, Comput. Math. Appl. 57 (9) (2009) 1438–1449. [20] F. Ding, P.X. Liu, G. Liu, Auxiliary model based multi-innovation extended stochastic gradient parameter estimation with colored measurement noises, Signal Process 89 (10) (2009) 1883–1890. [21] J. Ding, et al., Hierarchical least squares identification for linear SISO systems with dual-rate sampled-data, Automatic Control, IEEE Transac. 56 (11) (2011) 2677–2683. [22] H. Han, et al., Hierarchical least-squares based iterative identification for multivariable systems with moving average noises, Math. Comput. Model. 51 (9) (2010) 1213–1220.

Please cite this article as: M.A.Z. Raja, N.I. Chaudhary, Two-stage fractional least mean square identification algorithm for parameter estimation of CARMA systems, Signal Processing (2014), http://dx.doi.org/10.1016/j.sigpro.2014.06.015i

71 73 75 77 79 81 83 85 87 89 91 93 95 97 99 101 103 105 107 109 111 113 115 117 119 121 123

M.A.Z. Raja, N.I. Chaudhary / Signal Processing ] (]]]]) ]]]–]]]

1 3 5 7 9 11 13 15 17 19 21 23 25 27 29 31

[23] L. Xiang, et al., Hierarchical least squares algorithms for single-input multiple-output systems based on the auxiliary model, Math. Comput. Model. 52 (5) (2010) 918–924. [24] F. Ding, T. Chen, Identification of Hammerstein nonlinear ARMAX systems, Automatica 41 (9) (2005) 1479–1489. [25] F. Ding, P.X. Liu, G. Liu, Gradient based and least-squares based iterative identification methods for OE and OEMA systems, Digit. Signal Process 20 (3) (2010) 664–677. [26] H. Han, et al., Hierarchical least-squares based iterative identification for multivariable systems with moving average noises, Math. Comput. Model. 51 (9) (2010) 1213–1220. [27] Z. Zhang, F. Ding, X. Liu, Hierarchical gradient based iterative parameter estimation algorithm for multivariable output error moving average systems, Comput. Math. Appl. 61 (3) (2011) 672–682. [28] B. Bao, et al., Least squares based iterative parameter estimation algorithm for multivariable controlled ARMA system modelling with finite measurement data, Math. Comput. Model. 53 (9) (2011) 1664–1669. [29] F. Ding, H. Duan, Two-stage parameter estimation algorithms for Box–Jenkins systems, IET, Signal Process 7 (8) (2013) 646–654. [30] F. Ding, Two-stage least squares based iterative estimation algorithm for CARARMA system modeling, Appl. Math. Model. 37 (7) (2013) 4798–4808. [31] G. Yao, R. Ding, Two-stage least squares based iterative identification algorithm for controlled autoregressive moving average (CARMA) systems, Comput. Math. Appl. 63 (5) (2012) 975–984. [32] H. Duan, J. Jia, R. Ding, Two-stage recursive least squares parameter estimation algorithm for output error models, Math. Comput. Model. 55 (3) (2012) 1151–1159. [33] M.D. Ortigueira, Introduction to fractional signal processing Part1: continuous-time systems, IEE Proc. Vis. Image Signal Process 147 (1) (2000) 62–70. [34] M.D. Ortigueira, Introduction to fractional signal processing. Part2: discrete-time systems, IEE Proc. Vis. Image Signal Process 147 (1) (2000) 71–78. [35] M.D. Ortigueira, J.A.T. Machado, Special issue on fractional signal processing and applications, Signal Process. 83 (11) (2003). [36] M.D. Ortigueira, J.A.T. Machado, Special section: fractional calculus applications in signals and systems, Signal Process. 86 (10) (2006).

13

[37] C.-C. Tseng, S.-L. Lee, Design of linear phase FIR filters using fractional derivative constraints, Signal Process. 92 (5) (2012) 1317–1327. [38] C.-C. Tseng, S.-L. Lee, Designs of two-dimensional linear phase FIR filters using fractional derivative constraints, Signal Process. 93 (5) (2013) 1141–1151. [39] C.-C. Tseng, S.-L. Lee, Designs of fractional derivative constrained 1-D and 2-D FIR filters in the complex domain, Signal Process. 95 (2014) 111–125. [40] J. Wang, et al., Fractional zero-phase filtering based on the Riemann–Liouville integral, Signal Process. 98 (2014) 150–157. [41] R.M.A. Zahoor, I.M. Qureshi, A modified least mean square algorithm using fractional derivative and its application to system identification, Eur. J. Sci. Res. 35 (1) (2009) 14–21. [42] G. Masoud, S.G. Osgouei, Dual-channel speech enhancement using normalized fractional least-mean-squares algorithm, In: Proceedings of the 19th Iranian Conference on Electrical Engineering (ICEE) 2011. [43] S.K. Dubey, N.K. Rout, FLMS algorithm for acoustic echo cancellation and its comparison with LMS, In: Proceedings of the 1st International Conference on IEEE Recent Advances in Information Technology (RAIT), 2012. [44] P. Akhtar, M. Yasin, Performance analysis of Bessel beamformer and LMS algorithm for smart antenna array in mobile communication system, Emerging Trends and Applications in Information Communication Technologies, 2012 pp. 52–61. [45] N.I. Chaudhary, M.A.Z. Raja, J.A. Khan, M.S. Aslam, Identification of input nonlinear control autoregressive systems using fractional signal processing approach, Sci. World J. 2013 (2013) 1–13. (ID 467276). [46] S. Haykin, Adaptive filter theory, 5th edn. Pearson Education, Limited, India, 2013 (revised). [47] I. Podlubny, Fractional Differential Equations, Academic Press, New York, 1999. [48] M.A.Z. Raja, Application of fractional calculus to engineering: a new computational approach, Diss. International Islamic University, Islamabad, 2011. [49] Q.M. David, K.J. Åström, J.M.C. Clark, A new algorithm for recursive estimation of parameters in controlled ARMA processes, Automatica 20 (6) (1984) 751–760.

Please cite this article as: M.A.Z. Raja, N.I. Chaudhary, Two-stage fractional least mean square identification algorithm for parameter estimation of CARMA systems, Signal Processing (2014), http://dx.doi.org/10.1016/j.sigpro.2014.06.015i

33 35 37 39 41 43 45 47 49 51 53 55 57 59 61 63