Time delay Chebyshev functional link artificial neural network

Communicated by Dr Q Wei

Accepted Manuscript

Time Delay Chebyshev Functional Link Artificial Neural Network Lu Lu, Yi Yu, Xiaomin Yang, Wei Wu PII: DOI: Reference:

S0925-2312(18)31247-5 https://doi.org/10.1016/j.neucom.2018.10.051 NEUCOM 20073

To appear in:

Neurocomputing

Received date: Revised date: Accepted date:

5 June 2018 17 September 2018 27 October 2018

Please cite this article as: Lu Lu, Yi Yu, Xiaomin Yang, Wei Wu, Time Delay Chebyshev Functional Link Artificial Neural Network, Neurocomputing (2018), doi: https://doi.org/10.1016/j.neucom.2018.10.051

This is a PDF file of an unedited manuscript that has been accepted for publication. As a service to our customers we are providing this early version of the manuscript. The manuscript will undergo copyediting, typesetting, and review of the resulting proof before it is published in its final form. Please note that during the production process errors may be discovered which could affect the content, and all legal disclaimers that apply to the journal pertain.

ACCEPTED MANUSCRIPT

Time Delay Chebyshev Functional Link Artificial Neural Network✩ Lu Lua , Yi Yub , Xiaomin Yanga∗ , Wei Wua a) School of Electronics and Information Engineering, Sichuan University, Chengdu, Sichuan 610065, China.

CR IP T

b) School of Information Engineering, Southwest University of Science and Technology, Mianyang 621010, China.

Abstract

In real applications, a time delay in the parameter update of the neural network is sometimes required. In this paper, motivated by the Chebyshev functional link artificial neural network

AN US

(CFLANN), a new structure based on the time delay adaptation is developed for nonlinear system identification. Particularly, we present a new CFLANN-delayed recursive least square (CFLANNDRLS), as an online learning algorithm for parameter adaptation in CFLANN. The CFLANNDRLS algorithm exploits the time delayed error signal with the gain vector delayed by D cycles to form the weight increment term, which provides potential implementation in the filter with pipelined structure. However, it suffers from the instability problems under imperfect network delay

M

estimate. To overcome this problem, we further propose a modified CFLANN-DRLS (CFLANNMDRLS) algorithm by including a compensation term to the error signal. We analyze the stability

ED

and convergence of the proposed algorithm. Simulations in nonlinear system identification contexts reveal that the newly proposed CFLANN-MDRLS algorithm can effectively compensate the time delay of system and it is even superior to the algorithm without delay in some cases.

PT

Keywords: Functional link artificial neural network (FLANN), Chebyshev polynomials, Time

CE

delay, Recursive algorithm, Nonlinear system identification.

1. Introduction

AC

System identification is the art and science of establishing a mathematical model for an unknown system through the input-output relationship, which plays a significant role in control theory [1, ✩ E-mail addresses: [email protected](L. Lu), yuyi [email protected](Y. Yu), [email protected](X. Yang), [email protected](W. Wu), (Corresponding author: Xiaomin Yang).

Preprint submitted to Elsevier

October 31, 2018

ACCEPTED MANUSCRIPT

2]. In previous studies, many approaches have been proposed in the context of linear system 5

identification [3, 4]. However, the real-world system is not always linear. For instance, different types of artificial noise in active noise control systems [5–7] and impulsive noises in communication systems [8, 9], can be described more accurately using nonlinear models. In these cases, linear

system identification technique into account. 10

CR IP T

system identification technique performs poorly. Therefore, it is reasonable to take the nonlinear

Several nonlinear approaches have been proposed to identify nonlinear dynamic systems [10–13]. Among these, the artificial neural network (ANN) has attracted more and more attention because of the learning ability and excellent approximating performance [13, 14]. Most of the ANN-based system identification techniques are based on the feedforward networks such as the radial basis function neural networks (RBFNNs) [15–17] and the multilayer perceptron (MLP) trained with backpropagation (BP) [18–21]. In [15], an effective procedure based on the RBFNN was proposed

AN US

15

to detect the harmonic amplitudes of the measured signal. However, the main bottleneck of the RBFNNs is their essentially static input-to-output maps, which reduces the ability of modeling nonlinear systems [22, 23]. The algorithm in [21] was developed for estimating carbon price of European Union Emissions Trading Scheme. Although the reliable performance of the MLP network model is achieved, such neural network does not consider the time delay in network and compensate

M

20

the delay for performance improvement.

As an effective alternative to the multilayer ANN, the functional link ANN (FLANN) was

ED

initially proposed on neural networks [24]. By removing the hidden layer, the FLANN becomes a single layer network. It employs the point-wise functional expansions of the current input pattern 25

and then it generates the output by linear combination. Previous studies have shown that the

PT

FLANN can simplify the learning algorithms and unify the architecture for all types of networks [25]. So far, a large number of FLANN algorithms have been proposed based upon different basis

CE

functions [5, 26–28]. By making use of the Fourier expansions, even mirror Fourier nonlinear (EMFN) algorithm was introduced in [27], which accelerates the convergence rate as compared with its Volterra expansion counterpart. The trigonometric polynomial is another popular expansion, which is an effective means for nonlinear active noise control (NANC) and provides a computational

AC

30

advantage over existing algorithms [5, 29]. Furthermore, many orthogonal basis functions, such as Hermite polynomial [30], Legendre polynomial [31], and Chebyshev polynomial [32] were also extensively investigated under the FLANN framework. Due to the merits of FLANN, it has been

ACCEPTED MANUSCRIPT

35

applied to the solution of various practical problems, such as nonlinear acoustic echo cancellation [26, 33], nonlinear channel equalization [34], and NANC [5, 29]. The Chebyshev polynomial expansions satisfy all the requirements of the Stone-Weierstrass approximation theorem, and therefore it has powerful nonlinear modeling capabilities as compared to the MLP and can be used for many practical applications [32, 35, 36]. These Chebyshev polynomials form an orthogonal basis, which converge faster than expansions in other sets of polynomials

CR IP T

40

[37, 38]. The Chebyshev functional link artificial neural network (CFLANN) combines the benefits of the Chebyshev polynomials and the FLANN, which is a well-known improvement to solve the nonlinear system identification problem. It has been proved that the CFLANN shares some similar characteristics to the EMFN algorithm and the Legendre nonlinear (LN) algorithm, and outper45

forms these algorithms in various environments [32]. At present, a vast number of CFLANN-based

AN US

algorithms have been proposed in diverse fields [35, 36, 39–41]. Particularly, in [36], a CFLANNbased recursive least square (CFLANN-RLS) algorithm was proposed for the ideal dynamic system identification in online mode. This algorithm obtains good identification accuracy with low computational cost, which makes it very well-suited for applications in the design of online adaptive 50

identification models.

M

In many practical situations, the updating step of the algorithm can be performed only after a fixed time delay. For instance, in many dynamic systems [42, 43], decision-directed adaptive channel equalizers [44] and implementation of the adaptive filter [45, 46], time delay widely exists. In [47], a

55

ED

practical identification algorithm for the time delayed linear system with incomplete measurement was proposed, which has a good quality of noise resistance. The algorithm in [48] provides a distortion correction method to identify the strongly nonlinear stiffness of the equivalent model

PT

and time delay in the absorber system. Following a different direction, the adaptive algorithm was proposed for system identification with time delay. Such research dates back to 1989 (delayed-

60

CE

least mean square, DLMS algorithm) [49], and so far several works of the DLMS algorithm have been developed [50–54]. But few algorithms aimed at enhancing the stability of DLMS algorithm were investigated. In [53], a modified leaky delayed least-mean-square (MLDLMS) algorithm was

AC

proposed under the imperfect system delay estimates. Nevertheless, this algorithm has the risk of instability in cases where the unknown system is nonlinear. Motivated by the advantages of CFLANN, we first consider the time delay problem in CFLANN

65

for nonlinear system identification. Before that, some time delay neural networks have been pro-

ACCEPTED MANUSCRIPT

posed for nonlinear system identification. However, these efforts rely on the priori information or have high computational complexity [23, 55], which are hard to implement. To facilitate its practical use, the delayed-recursive least sqaure (DRLS) algorithm is proposed as an online training algorithm in CFLANN, resulting in the CFLANN-DRLS algorithm. The proposed CFLANN-DRLS 70

is a recursive algorithm, which converges fast even when the eigenvalue spread of the input signal

CR IP T

correlation matrix is large. Besides, algorithms of the recursive type have excellent performance when implementing to time-varying environments [56]. To further enhance the performance of the CFLANN-DRLS algorithm, a modified CFLANN-DRLS (CFLANN-MDRLS) algorithm that does not need any a priori information about the noise statistical characteristics and nonlinear system, 75

is developed motivated by the method in [54]. This method makes it possible to estimate the time delay from the nonlinear system, thereby enabling the newly proposed CFLANN-MDRLS algo-

AN US

rithm to achieve an improved performance. In summary, our main contributions are depicted as the following points:

1) The CFLANN with time delay model is first proposed for nonlinear dynamic system identifi80

cation.

2) The CFLANN-DRLS algorithm is proposed for online adaptation of the parameter in CFLANN.

M

3) The CFLANN-MDRLS algorithm is developed for performance improvement. 4) The convergence analysis of the CFLANN-MDRLS algorithm is performed and supported by simulations.

The rest of the paper is organized as follows. In Section 2, we briefly introduce the nonlinear

ED

85

system model. In Section 3, we present the proposed algorithms in detail, including CFLANNDRLS and CFLANN-MDRLS algorithms. Then, the convergence analysis of the CFLANN-MDRLS

PT

algorithm is performed. Simulation results are shown in Section 4 to illustrate the effectiveness and advantages of the proposed algorithms. Finally, Section 5 presents the conclusions and future lines of research.

CE

90

AC

2. Problem formulation Fig. 1 shows the structure of nonlinear system identification based on the CFLANN with time

delay, where u(k) denotes the input signal, z(k) denotes the nonlinear system output, d(k) denotes the desired signal, v(k) is the additive noise, D is the time delay related to the nonlinear system ˆ is the time delay estimate, y(k) is the output of the FLANN, and e(k) is the (nonlinear ANN), D

ACCEPTED MANUSCRIPT

vk

u k

1RQOLQHDU6\VWHP

&)/$11

z k

Ȉ

d k

#

Ȉ e k

yk

DÖ

Adaptive algorithm

u k  DÖ

e k # D

CR IP T

D

Figure 1: Identification of nonlinear dynamic systems.

error signal between y(k) and d(k). Here, we consider the single-input-single-output (SISO) and multiple-input-multiple-output (MIMO) models, which are described by the nonlinear difference

AN US

equations in the following form:

SISO (Model 1) : d(k + 1) = f [d(k), d(k − 1), . . . , d(k − N + 1), u(k), u(k − 1), . . . , u(k − M + 1)] + v(k) MIMO (Model 2) : d1 (k + 1) = f1 (d1 (k), d2 (k)) + u1 (k) + v1 (k),

M

d2 (k + 1) = f2 (d1 (k)) + u2 (k) + v2 (k). Here N and M stand for the memory lengths of the SISO system (N ≥ M ), u(k) and d(k) denote

ED

the input and desired signals of the SISO system at the kth time instant, u1 (k) and u2 (k) denote the input signals of the MIMO system, d1 (k) and d2 (k) denote the desired signals of the MIMO system, and v1 (k) and v2 (k) denote the additive noises of the MIMO system, respectively. The

PT

error signal in SISO model at time instant k − D is defined as e(k − D) , d(k − D) − y(k − D).

(1)

ej (k − D) , dj (k − D) − yj (k − D),

AC

CE

The error signal in MIMO model SISO model at time instant k − D is defined as j = 1, 2

(2)

where ej (k − D) is the error signal of jth output and yj (k − D) is the jth output. The algorithms used for MIMO system identification can be easily obtained by integrating SISO algorithms in the MIMO model. For the sake of simplicity, the derivations and mean behavior analysis of the

95

algorithms in this work are based on the SISO model.

ACCEPTED MANUSCRIPT

In what follows, the Chebyshev neural network with the RLS algorithm is used to construct the nonlinear function f (·), f1 (·) and f2 (·) in order to approximate such mapping over compact sets. We assume that the system is bounded-input and bounded-output (BIBO) stable [57]. However, owing to the presence of time delay in the nonlinear system, parameter updating of the network algorithm 100

ˆ is also affected. For example, at time instant k, the incremental term ∆w(k) reduces to ∆w(k − D).

CR IP T

This will cause the online learning algorithm in ANN cannot track the change from the nonlinear ˆ may be different in practice. The imperfect estimation of time dynamic system. Besides, D and D delay will also deteriorate performance. Consequently, under the BIBO assumption, the stability of the ANN model cannot be assured. To guarantee the convergence of neural network model, a 105

time delay compensation scheme is warranted.

AN US

3. Time delay CFLANN 3.1. Chebyshev Basis Function

The ANN structure considered in time delay model is a single layer Chebyshev neural network based on the Chebyshev polynomials. The Chebyshev polynomials hold two-fold characteristics as follows. 1) Chebyshev polynomials are a family of orthogonal polynomials. 2) Chebyshev polyno-

M

mials converge faster than expansions in other sets of polynomials. The Chebyshev polynomials are given by a generating function [32]

ED

Tn+1 (x) = 2xTn (x) − Tn−1 (x)

(3)

where x ∈ [−1, 1] and n is the order of Chebyshev polynomials expansion. The close-form expression

AC

CE

PT

for Chebyshev polynomials of any order is expressed as [39] n

2 nX (n − i − 1)! Tn (x) = (−1)i (2x)n−2i . 2 i=0 i!(n − 2i)!

(4)

The first kind of Chebyshev polynomials are given by   T0 (x) = 1          T1 (x) = x      T2 (x) = 2x2 − 1     T3 (x) = 4x3 − 3x      T4 (x) = 8x4 − 8x2 + 1        T5 (x) = 16x5 − 20x3 + 5x       ..  .

CR IP T

ACCEPTED MANUSCRIPT

The Chebyshev functional expansion expression using the Chebyshev polynomials (increased

AN US

from 1 to S) is given as follow:

φ(Tn (uk )) = [1, T1 (u(k)), T1 (u(k − 1)), . . . , T1 (u(k − M )), T1 (u(k))T1 (u(k − 1)), . . .

, T1 (u(k − M + 1))T1 (u(k − M )),

M

T2 (u(k)), T2 (u(k − 1)), . . . , T2 (u(k − M )), . . . T

, TS (u(k)), TS (u(k − 1)), . . . , TS (u(k − M ))] T

is the input vector and (·)T is the transposition.

ED

where uk = [u(k), u(k − 1), . . . , u(u − M + 1)]

An alternative form of the Chebyshev polynomials based on the 1-dimensional basis functions with u(k), u(k − 1), . . . , u(k − M + 1) is wrote as [32]

PT

1, T1 (u(k)), T2 (u(k)), T3 (u(k)), . . . , TS (u(k)) 1, T1 (u(k − 1)), T2 (u(k − 1)), T3 (u(k − 1)), . . . , TS (u(k − 1))

CE

.. .

1, T1 (u(k − M + 1)), T2 (u(k − M + 1)), T3 (u(k − M + 1)), . . . , TS (u(k − M + 1))

AC

For example, if the order n = 0, Chebyshev polynomials is the constant 1; if the Chebyshev functional expansion of order n = 1, it corresponds to linear terms [1, u(k), u(k−1), . . . , u(u−M +1)].

ACCEPTED MANUSCRIPT

Consequently, φ(Tn (uk )) can be rewritten as the following when the order n = 3 [32]: φ(Tn (uk )) = [1, u(k), u(k − 1), . . . , u(u − M + 1), u(k)u(k − 1), . . . , u(k − M + 1)u(k − M ), 2u2 (k) − 1, . . . , 2u2 (k − M ) − 1]T .

CR IP T

When order n is higher, the nonlinear modeling capability is stronger. It is worth to mention that a large value of n carries heavy computational burdens. Hence, the order n of the Chebyshev functional expansion is set to 3 in this paper. In the case of n = 3, the length of Chebyshev functional expansion can be calculated as L = 4M + 4. The output of CFLANN can be expressed as

(5)

AN US

y(k) = wT (k)φ(Tn (uk ))

where w(k) is the weight vector of the neural network at time k. For the sake of simplicity, we utilize φ(k) = φ(Tn (uk )) in following sections. 110

3.2. CFLANN-DRLS algorithm

As state in Section 2, the adaptive algorithm can be used as an online algorithm for CFLANN model. In our development, a DRLS algorithm is proposed to online update the parameter of

M

ˆ is exactly identified CFLANN with time delay. We initially assume that the estimated time delay D as D. We later remove this assumption by estimating this time delay D in real time, as investigated

ED

in the next subsection. For the CFLANN-RLS algorithm [36], the main objective of identifications is to construct a suitable model generating an output y(k) which approximates the desired signal

PT

d(k) such that

ke(k)k2 = kd(k) − y(k)k2 < ξ

where ξ > 0 is a small value, and k·k2 is the l2 -norm. The objective in identification with CFLANN-

CE

DRLS algorithm is to

ke(k − D)k2 < ξ

AC

where

e(k − D) = d(k − D) − wT (k − D)φ(k − D).

(6)

The CFLANN-DRLS algorithm is computed by the following equation w(k) = P (k − D)r(k − D)

(7)

ACCEPTED MANUSCRIPT

where the weighted autocorrelation matrix (inverse autocorrelation matrix) P (k − D) and the weighted cross-correlation vector r(k − D) between the desired signal and the input vector are defined by

and r(k − D) ,

k−D X

k−D X

λk−D−i φ(i)φT (i)

λk−D−i φ(i)d(i)

i=1

(8)

i=1

CR IP T

P (k − D) , R−1 (k − D), R(k − D) ,

(9)

where 0  λ < 1 is the forgetting factor. Based on the Wiener solution, the autocorrelation matrix and the crosscorrelation vector in (7) can be updated by (8) and (9). Unfortunately, such algorithm is not a truly online approach. To online adapt the parameter in Chebyshev neural network,

AN US

P (k − D) and r(k − D) must be updated by recursive expression. According to the approach in [58], the following formulations are obtained:

R(k − D) = λR(k − D − 1) + φ(k − D)φT (k − D)

(10)

r(k − D) = λr(k − D − 1) + φ(k − D)d(k − D).

(11)

M

To avoid the inverse matrix operation, P (k − D) can be calculated by using Woodbury’s matrix inversion lemma [59]. This lemma states that for arbitrary positive definite matrices {F , G, H}, if

ED

the following relationship holds: F = G−1 + QH −1 QT , where Q is an arbitrary matrix, then, we have

F −1 = G − GQ(H + QT GQ)QT G.

obtain

PT

By representing F with R(k − D), G−1 with λR(k − D − 1), Q with φ(k − D), and H with 1, we

CE

P (k − D) = λ−1 P (k − D − 1) −

λ−2 P (k − D − 1)φ(k − D)φT (k − D)P (k − D − 1) . 1 + λ−1 φT (k − D)P (k − D − 1)φ(k − D)

(12)

AC

For simplicity, (12) can be rewritten as

P (k − D) = λ−1 P (k − D − 1) − λ−1 κ(k − D)φT (k − D)P (k − D − 1).

(13)

ACCEPTED MANUSCRIPT

By using the soft initialization, we have P (0) = δI, where δ is a small positive constant, and I represents an identity matrix. Thus, the delayed gain vector κ(k − D) is defined by κ(k − D) ,

P (k − D − 1)φ(k − D) . λ + φT (k − D)P (k − D − 1)φ(k − D)

(14)

κ(k − D) = λ−1 P (k − D − 1)φ(k − D)

CR IP T

Rearrange (14), we have

− λ−1 κ(k − D)κT (k − D)P (k − D − 1)φ(k − D) = [λ−1 P (k − D − 1)

(15)

− λ−1 κ(k − D)φT (k − D)P (k − D − 1)]φ(k − D).

AN US

According to (13), (15) can be simplified to κ(k − D) = P (k − D)φ(k − D). Then, substituting (11) into (7), we get

w(k) = P (k − D) [λr(k − D − 1) + φ(k − D)d(k − D)]

(16)

(17)

= λP (k − D)r(k − D − 1) + P (k − D)φ(k − D)d(k − D).

M

The inverse autocorrelation matrix of the first term in (17) can be replaced by (13) w(k) = P (k − D − 1)r(k − D − 1)

ED

− κ(k − D)φT (k − D)P (k − D − 1)r(k − D − 1) + P (k − D)φ(k − D)d(k − D)

(18)

PT

= w(k − 1)

CE

− κ(k − D)φT (k − D)w(k − 1) + P (k − D)φ(k − D)d(k − D).

Introducing (16) to (18), the weight update equation of the proposed CFLANN-DRLS algorithm

AC

is expressed as

w(k) = w(k − 1)

  + κ(k − D) d(k − D) − φT (k − D)w(k − 1) .

(19)

In this expression, φT (k − D)w(k − 1) can be regarded as an estimation of the desired signal at

time instant k − D. The term d(k − D) − φT (k − D)w(k − 1) represents a priori error ep (k − D) of

ACCEPTED MANUSCRIPT

the algorithm, which is hard to obtained in practical. To solve this problem, we replace the priori error by the instantaneous value of the error signal e(k − D) before adaptation (20)

w(k) = w(k − 1) + κ(k − D)e(k − D).

For D = 0, the proposed CFLANN-DRLS algorithm reduces to the CFLANN-RLS algorithm. The

CR IP T

conventional CFLANN-RLS algorithm does not support pipelined implementation for real-system. In contrast, the delayed type algorithms can be effectively applied in pipelined structure of the filter. We can decompose the second term of (20) into two parts: κ(k − D) is owing to the delay involved 115

in pipelining the weight vector adaptation, while e(k − D) is the delay introduced by the pipeline stages in finite impulse response (FIR) filtering. Note that the error signal becomes available after D cycles in the case of pipelined designs. This error is used with the gain vector delayed D cycles to

AN US

generate the weight-increment term. Based on such a decomposition of delay, the CFLANN-DRLS algorithm can be easily implemented to design the case of pipelined processing [60–62]. 120

Remark 1 : In many applications, such as active noise compensation, the time delay ‘D’ is caused by an ‘analog summation point’. Hence, there are few observations available to the CFLANN-DRLS algorithm at time instant k. In general, we only have access to the error signals e(k − D), output

M

signal y(k − D) and the previous input vector φ(k − D) of the Chebyshev neural network. In the proposed CFLANN-DRLS algorithm, each adaptation is done without any information learned from 125

the former D−1 updates. This may cause the poor convergence performance of the CFLANN-DRLS

ED

algorithm.

PT

3.3. Practical consideration: CFLANN-MDRLS algorithm

AC

CE

u k

vk 1RQOLQHDU6\VWHP

&)/$11

z k

Ȉ

d k

yk

#

Ȉ e k U k

DÖ

Adaptive algorithm

u k  DÖ

eÖk # D

Compensation algorithm

D

e k # D

Figure 2: Diagram of the CFLANN-MDRLS algorithm.

ACCEPTED MANUSCRIPT

By utilizing the information from Chebyshev network, we propose CFLANN-MDRLS algorithm based on the compensation scheme. The goal of this algorithm is to address the following questions: 130

- Time delay compensation: How to compensate the time delay under the imperfect system ˆ = delay estimate (D 6 D)? - Stability: What are the conditions of the CFLANN-MDRLS for convergence?

CR IP T

The structure of the CFLANN-MDRLS algorithm is shown in Fig. 2, where ρ(k) is the compensate term, and eˆ(k − D) is the compensated error signal, which can be defined as eˆ(k − D) , d(k − D) − wT (k − D)φ(k − D) − ρ(k).

(21)

Considering an imperfect estimate condition in the CFLANN, the error signal of the standard RLS algorithm is given by

AN US

ˆ − wT (k)φ(k − D). ˆ e(k) = d(k − D)

(22)

By enforcing (21) to be equal to (22), we have

ˆ − wT (k − D)φ(k − D) ρ(k) = d(n − D) − d(k − D) ˆ + wT (k)φ(k − D).

(23)

M

ˆ ˆ yields Supposing that wT (k − D)u(k − D) ≈ wT (k − D)u(k − D)

ED

h i ˆ φ(k − D). ˆ ρ(k) ≈ wT (k) − wT (k − D)

(24)

ˆ can be computed, and therefore (23) results in Using (20), wT (k) − wT (k − D)

PT

ρ(k)≈

ˆ D X j=1

ˆ − j)e(k − D ˆ − j)φ(k − D). ˆ κ(k − D

(25)

AC

CE

Combine (25) and (21), the compensated error signal is given by ˆ − wT (k − D)φ(k − D) eˆ(k − D) = d(k − D) −

ˆ D X j=1

ˆ − j)e(k − D ˆ − j)φ(k − D). ˆ κ(k − D

Remark 2 : The proposed compensate strategy does not need any priori information according to the noise and nonlinear system, at the price of moderate computational complexity. The proposed

135

algorithms have important application potential in many problems which involve time delay. A

ACCEPTED MANUSCRIPT

direct application of the new algorithms is the phoneme recognition [63]. The time delay model can discover acoustic-phonetic features and the temporal relationships between them. Moreover, this information is not blurred by temporal shifts in the input data. Another problem where the suggested algorithms can be employed is active noise/vibration control which is an important problem in applications such as the industrial equipment and adaptive control [53].

CR IP T

140

3.4. Mean behavior analysis

In this subsection, convergence analysis of the proposed algorithm is presented. It is assume that at time instant k, the input sequence u(k) is generated at random, and its autocorrelation function is ergodic, which can be expressed as 1 R(k) if k > M k

AN US

Φ≈

where R(k) is a time average correlation matrix of the input sequence u(k). This assumption implies that the time average can substitute for the ensemble average. Adding a regularization factor δλk−D I to R(k − D), we have k−D X

λk−D−i φ(i)φT (i) + δλk−D I.

i=1

M

R(k − D) = For λ = 1, we have

ED

R(k − D) =

k−D X

φ(i)φT (i) + R(0).

(26)

i=1

Next, introducing (1), (5) into (9), the vector r(k − D) can be written as

PT

r(k − D) =

k−D X

φ(i)φT (i)wo +

i=1

k−D X

φ(i)v(i)

(27)

i=1

CE

where wo is a vector containing the optimal coefficient values. Combining (26) and (27), we get r(k − D) = R(k − D)wo − R(0)wo +

k−D X

φ(i)v(i).

i=1

AC

Then, we rewrite (7) as w(k) = R−1 (k − D)R(k − D)wo − R−1 (k − D)R(0)wo + R−1 (k − D)

k−D X i=1

φ(i)φT (i)v(i).

(28)

ACCEPTED MANUSCRIPT

Taking expectations of both sides of (28) and referring to assumption 1 , we arrive at 1 −1 Φ R(0)wo k δ = wo − Φ−1 wo k δ = wo − K (if k > M ) k

E{w(k)} ≈ wo −

CR IP T

(29)

where K = Φ−1 wo is the average of cross-correlation vector between the desired signal d(k) and input sequence u(k). Therefore, according to the aforementioned analysis, the algorithm is convergent and stable. Due to using P (0) = δI to initialize the algorithm, the estimated value w(k) 145

is biased for k > M . However, when k  M (i.e. k → ∞), the estimator is unbiased and the

AN US

estimation error tends to zero.

4. Simulation results

In this section, we evaluate the performance of the proposed algorithms on some numerical simulations in the case of SISO system, MIMO system, and Box and Jenkins’ identification problem. To test the performance of the proposed algorithms, we compare them with the CFLANN-RLS

M

algorithm. Here, the CFLANN-RLS algorithm is used for nonlinear system without time delay, i.e., D = 0. It can be regarded as an ‘ideal state’ of the algorithm, that is to say, all the online learning algorithm and unknown nonlinear system are without effect of time delay D. By comparing with

ED

the CFLANN-RLS algorithm, we can see that the new algorithm is affected by the time delay and its effectiveness for time delay compensation. Identification performance will be measured with an

PT

estimate of the root-mean-squared error (RMSE) [17] between the real and the estimated system,

CE

at time instant k, the RMSE is calculated as the average of the system identification error v v u u k k u1 X u1 X RMSE(k) = t [d(q) − y(q)]2 = t e2 (q) (SISO model) 2k q=1 2k q=1 proceed analysis, we take advantage of the multiple linear regression model. Such model can describe most

AC

1 To

systems. The analysis based on nonlinear regression model is exactly difficult. We will focus on this analysis for future study.

ACCEPTED MANUSCRIPT

 s s k k  P P  1 1  [d1 (q) − y1 (q)]2 = 2k e21 (q)   RMSE1 (k) = 2k q=1 q=1 s s k k  P P  1 1  2  [d2 (q) − y2 (q)] = 2k e22 (q)  RMSE2 (k) = 2k q=1

(the first output of MIMO model) (the second output of MIMO model).

q=1

CR IP T

All the simulations are programmed with Matlab version 2017a, and are run on a PC with a clock

speed 3.4 GHz and 16 GB RAM, under a Windows 10 environment. The simulation results are 150

obtained by ensemble averaging over 100 independent runs.

0.35

CFLANN−DRLS (δ−1=0.005) CFLANN−DRLS (δ−1=0.01)

0.3

AN US

CFLANN−DRLS (δ−1=0.1) CFLANN−DRLS (δ−1=0.5)

0.25

CFLANN−DRLS (δ−1=1)

RMSE

CFLANN−DRLS (δ−1=10)

0.2

0.15

M

0.1

0

ED

0.05

0

200

400

600

800

1000

Discrete time, k

PT

Figure 3: RMSE performance of the CFLANN-DRLS algorithm for different δ.

CE

4.1. Example 1. System identification with white Gaussian noise input This example aims i) to show that the effect of δ on the algorithm performance; ii) to assess the identification performance of all algorithms in Gaussian scenarios. For this operating scenario,

AC

a SISO nonlinear system with L = 12 weight is considered. The desired signal of this system is obtained from a polynomial function [64] d(k) = u(k) + 0.3u3 (k) + 0.2u5 (k) + v(k),

ACCEPTED MANUSCRIPT

(a)

0 −1 200

300

400

500

600

700

800

900

1000

(b)

1 0 −1 100

200

300

400

500

600

700

CR IP T

Output & Error

Error

1

100

Output & Error

Estimated

800

900

1000

(c)

1 0 −1 100

200

300

AN US

Output & Error

Actual

400

500

600

700

800

900

1000

Discrete time, k

Figure 4: Identification of nonlinear system. (a) CFLANN-RLS, (b) CFLANN-DRLS, (c) CFLANN-MDRLS.

M

where u(k) is a zero-mean white Gaussian noise with variance 0.1. The noise signal is also a zeromean white Gaussian noise. The value of the forgetting factor used here is λ = 0.999. Fig. 3

155

ED

ˆ = 2 for different δ. The signal-to-noise shows the RMSE results obtained from simulation D = D ratio (SNR) is 20dB. Notice from this figure that all the selections can obtain a stable performance and δ −1 = 0.5 outperforms other selections. Hence, we choose δ −1 = 0.5 in this example. Fig. 4

PT

depicts the identification performance of three algorithms. As can be seen, all the algorithms can precisely model the nonlinear system, while the CFLANN-DRLS algorithm has a large error in initial ˆ = 2, confirming that the convergence stage. Fig. 5 illustrates the RMSEs of algorithms for D = D CFLANN-MDRLS algorithm can effectively compensate the time delay. To further demonstrate

CE

160

ˆ and SNRs, Fig. 6 depicts the representation of the performance of algorithms for different D, D

AC

the polynomial functions d(k) and the reconstructed functions when the input signal u(k) ∈ [−1, 1]. We can see that the reconstructed functions are close to their actual functions which are known a priori. Some mismatches are observed when u(k) closes to 1.

ACCEPTED MANUSCRIPT

0.25 CFLANN−RLS CFLANN−DRLS CFLANN−MDRLS 0.2

0.1

0.05

0

200

400

600

800

AN US

0

CR IP T

RMSE

0.15

1000

Discrete time, k

ˆ = 2, SNR=20dB. Figure 5: Learning curves for nonlinear system identification with D = D

165

4.2. Example 2. SISO system identification

M

In this example, the SISO discrete time system belongs to the Model 1 (N = 3, M = 2) and is described by [35]

ED

d(k + 1) = f [d(k), d(k − 1), d(k − 2), u(k), u(k − 1)] + v(k) where the unknown nonlinear function f is given by

PT

f [a1 , a2 , a3 , a4 , a5 ] =

(a1 a2 a3 a5 (a3 − 1) + a4 ) . 1 + a22 + a23

CE

This SISO system is identified by the following estimator ˆ + 1) = fˆ [d(k), d(k − 1), d(k − 2), u(k), u(k − 1)] d(k

AC

ˆ where fˆ(·) denotes an estimate of f (·) and d(k) is an estimate of d(k). The input signal u(k) is a sinusoidal signal which is generated as follow [35]:   sin 2πk
250 u(k) =  0.8 sin 2πk
+ 0.2 sin 250

2πk 25




for 0 < k ≤ 250 for k > 250.

ACCEPTED MANUSCRIPT

1

1

0

0

0

0.5

1

−1

1

1

0

0

(c)

−1 −1

Function, d(k)

−0.5

−0.5

0

0.5

−1 1

0

0

−1

(e) −0.5

0

0.5

Input signal, x(k)

0

0.5

−0.5

0

0.5

−1

−0.5

1

(f)

−1

1

1

(d)

−1 1

1

−1

(b) −0.5

CR IP T

Function, d(k)

−1

−1

(a)

−1

AN US

Function, d(k)

Original function Reconstructed function by CFLANN−RLS Reconstructed function by CFLANN−MDRLS

0

0.5

1

Input signal, x(k)

Figure 6: Representation of the polynomial function and the reconstructed function when the input u(k) ∈ [−1, 1].

ˆ = 2, SNR=20dB; (b) D = D ˆ = 2, SNR=30dB; (c) D = D ˆ = 5, SNR=20dB; (d) D = D ˆ = 5, SNR=40dB; (a) D = D

M

ˆ = 4, SNR=20dB; (f) D = 3, D ˆ = 2, SNR=40dB. (e) D = 5, D

The Gaussian signal with SNR=40dB is employed as the additive noise. For the CFLANN, the input

ED

signal expanded to L =12 terms using Chebyshev polynomials. We fix λ = 0.999 and δ −1 = 0.01 in three algorithms to provide the best choice in terms of convergence speed and steady state error,

170

PT

ˆ = 2. The weight vector in CFLANN is initialized to zero. Fig. 7 (a)and time delay D = D (c) shows the performance of identification model with the CFLANN-RLS, CFLANN-DRLS and CFLANN-MDRLS algorithms. As can be seen, all the outputs are rather close to the actual system

CE

output. However, the performance of the CFLANN-DRLS algorithm is very poor during interval 1 < k ≤ 40, when time delay appears in the system. In contrast, the CFLANN-MDRLS algorithm,

AC

owing to using compensation strategy, achieves improved performance at initial stage as compared 175

with the CFLANN-DRLS algorithm. We can see that the performance of the CFLANN-MDRLS algorithms is near to that of the CFLANN-RLS algorithm. To further show the performance, Fig. 8 shows RMSE learning curves of the algorithms. The CFLANN-MDRLS algorithm is superior to the CFLANN-DRLS algorithm, and slightly accelerates the convergence rate as compared to the

ACCEPTED MANUSCRIPT

Table 1: Averaged RMSEs and run time of the algorithms for Example 2. Averaged RMSE ˆ =2 ˆ =4 D=D D = 5, D

Run time (s) ˆ =2 D=D

ˆ =4 D = 5, D

and SNR=40dB

and SNR=25dB

and SNR=40dB

and SNR=25dB

CFLANN-RLS

0.0218

0.0463

0.1131

0.1673

CFLANN-DRLS

0.0577

3.1299(Unstable)

0.1176

0.1719

CFLANN-MDRLS

0.0214

0.0515

0.1193

0.1745

CR IP T

Algorithms

CFLANN-RLS algorithm. Next, we repeat the same simulations as above, but this time D = 5, 180

ˆ = 4 and SNR=25dB. Fig. 9 shows the RMSE learning curves of algorithms during adaptation. D Again, a satisfactory performance of the CFLANN-MDRLS algorithm is observed, which is due to the compensation scheme used in the model development. Table 1 shows the averaged RMSE and

AN US

average run execution time for the CFLANN-RLS and the proposed algorithms. It can be seen that the CFLANN-RLS algorithm obtains the shortest run time in all cases. The CFLANN-MDRLS 185

algorithm requires more run time for calculating the compensation scheme, which is the price paid for its superior performance, as investigated in Figs. 8 and 9.

0.8

Actual

0.4

−0.2

(a)

PT

−0.4

Error

ED

Output & Error

0.2 0

Estimated

M

0.6

−0.6

CE

−0.8

AC

−1

100

200

300

400

500

600

Discrete time, k

700

800

900

1000

ACCEPTED MANUSCRIPT

0.8 Actual

Estimated

Error

(b)

0.6 0.4

0

CR IP T

Output & Error

0.2

−0.2 −0.4 −0.6 −0.8

100

200

300

400

500

600

700

800

900

AN US

−1

1000

Discrete time, k

0.8

Actual

0.6

0

−0.4

PT

−0.6

(c)

ED

Output & Error

0.2

−0.2

Error

M

0.4

Estimated

−0.8

CE

−1

100

200

300

400

500

600

700

800

900

1000

Discrete time, k

AC

Figure 7: Identification of SISO discrete system. (a) CFLANN-RLS, (b) CFLANN-DRLS, (c) CFLANN-MDRLS.

ACCEPTED MANUSCRIPT

0.2 CFLANN−RLS CFLANN−DRLS CFLANN−MDRLS

0.18 0.16

0.025

0.14 0.02 0.015

0.1

0.01 20

0.08

40

60

80

CR IP T

RMSE

0.12

100

120

0.06 0.04 0.02 0

0

200

400

600

140

800

1000

AN US

Discrete time, k

Figure 8: Learning curves for SISO discrete system identification (D = 2, SNR=40dB).

0.12

CFLANN−RLS CFLANN−MDRLS

0.1

10

M 5

ED

0.08

RMSE

CFLANN−DRLS

0.06

0

200

400

600

800

1000

PT

0.04

0

0.02

CE

0

0

100

200

300

400

500

600

700

800

900

1000

Discrete time, k

AC

ˆ = 4, SNR=25dB). Figure 9: Learning curves for SISO discrete system identification (D = 5, D

4.3. Example 3: MIMO system identification In the third example, we consider a MIMO system described by the difference equation of Model

2. The two input two output nonlinear discrete time system is given by [36]         d2 (k) d1 (k + 1) u (k) v (k) 2   =  1+d1 (k)  +  1 + 1  d1 (k) d2 (k + 1) u (k) v (k) 2 2 2 1+d (k) 1

ACCEPTED MANUSCRIPT

Table 2: Averaged RMSEs and run time of the algorithms for Example 3. Averaged RMSE

Algorithms

Run time (s) Fig. 12

CFLANN-RLS (the first output y1 (k))

0.1209

0.1023

CFLANN-RLS (the second output y2 (k))

0.0948

0.0778

CFLANN-DRLS (the first output y1 (k))

6.4725e+06(Unstable)

1.3123e+03(Unstable)

CFLANN-DRLS (the second output y2 (k))

1.1000e+06(Unstable)

1.5823e+03(Unstable)

CFLANN-MDRLS (the first output y1 (k))

0.1293

0.1028

CFLANN-MDRLS (the second output y2 (k))

0.0886

0.0769

Fig. 11

Fig. 12

0.0659

0.0979

0.1066

0.1206

CR IP T

Fig. 11

0.1136

0.1652

where the estimator of d1 (k + 1) and d2 (k + 1) can be given as:   dˆ (k + 1) = f [d (k), d (k), u (k), u (k)] 1 1 1 2 1 2  dˆ (k + 1) = f [d (k), d (k), u (k), u (k)] 2

1

2

1

2

AN US

2

the first input u1 (k) and the second input u2 (k) are expressed as [36]:   u (k) = cos 2πk
1 100  u (k) = sin 2πk
. 2

100

A white Gaussian noise with zero mean and covariance of 9 × 10−4 is added to the true output 190

M

obtained from the MIMO system [36]. In the CFLANN, the 2-dimensional input vector is expanded by the Chebyshev polynomials upto L =16 terms (corrresponding to M = 3 memory length). The ˆ = 4. The parameter setting in three algorithms are the same as those of time delay is D = D

ED

Example 2. Fig. 10 (a)-(c) show the estimated output, the real output, and the error signal during iteration. We can see that the CFLANN-RLS algorithm achieves stable performance in the absence

195

PT

of time delay. When the neural network exists time delay, the CFLANN-DRLS algorithm fails to work for MIMO system identification. The proposed CFLANN-MDRLS algorithm can valid compensate the time delay and enjoys good convergence behavior. It has small fluctuation during

CE

50 < k ≤ 100, still has acceptable identification accuracy. Fig. 11 shows the learning curves of three algorithms. It can be observed that the CFLANN-DRLS algorithm diverges during iteration, and the CFLANN-MDRLS algorithm has similar performance to the CFLANN-RLS algorithm. Fig. 12 shows the RMSEs of algorithms when noise gets severe, i.e. low SNR. Similar results to

AC 200

Fig. 11 are observed in Fig. 12. It can also be seen from Table 2 that the proposed CFLANNMDRLS algorithm owns a smaller averaged RMSE compared with other algorithms. The run time of network is about 0.1, which is an affordable computation time.

ACCEPTED MANUSCRIPT

Actual

Estimated

Error

(a)

1 0.5 0

CR IP T

−0.5 −1 −1.5

Output & Error

2

0

50

100

150

200

250

0 −1 −2

300

(b)

1

0

50

100

AN US

Outpit & Error

1.5

150

200

250

300

Discrete time, k

Actual

0.5 0 −0.5 −1 −1.5

(c)

0

1

50

100

150

200

250

300

(d)

PT

Output & Error

2

Error

M

1

ED

Outpit & Error

1.5

Estimated

0

CE

−1

AC

−2

0

50

100

150

200

250

300

Discrete time, k

4.4. Example 4: Box and Jenkins’ identification problem

205

To test the performance of algorithms in a realistic application, the proposed algorithms are

applied to identify the data of Box and Jenkins’ gas furnace system. The data can be obtained from

ACCEPTED MANUSCRIPT

Actual

Estimated

Error

(e)

1 0.5 0

CR IP T

−0.5 −1

Output & Error

−1.5 0 2

50

100

150

200

250

(f)

1 0 −1 −2

300

0

50

AN US

Outpit & Error

1.5

100

150

200

250

300

Discrete time, k

Figure 10: Identification of MIMO discrete system. (a) the first output y1 (k) of CFLANN-RLS, (b) the second output y2 (k) of CFLANN-RLS (c) the first output y1 (k) of CFLANN-DRLS, (d) the second output y2 (k) of CFLANN-DRLS

M

(e) the first output y1 (k) of CFLANN-MDRLS (f) the second output y2 (k) of CFLANN-MDRLS.

0.4

6

15

0.35

0.2

CFLANN−DRLS(the first output) CFLANN−DRLS(the second output)

ED

0.25

10

5

PT

RMSE1 & RMSE2

0.3

x 10

0

0

50

100

150

200

250

300

0.15

CE

0.1

0.05

AC

0

CFLANN−RLS (the first output) CFLANN−RLS (the second output) CFLANN−MDRLS (the first output) CFLANN−MDRLS (the second output) 50

100

150

200

250

300

Discrete time, k ˆ = 4). Figure 11: Learning curves for MIMO discrete system identification (D = D

ACCEPTED MANUSCRIPT

0.35 3000 CFLANN−DRLS(the first output) CFLANN−DRLS(the second output)

0.3 2000

1000 0.2 0

0

50

100

150

200

0.15

0.1

0

CFLANN−RLS (the first output) CFLANN−RLS (the second output) CFLANN−MDRLS (the first output) CFLANN−MDRLS (the second output) 50

100

150

200

250

250

300

AN US

0.05

CR IP T

RMSE1 & RMSE2

0.25

300

Discrete time, k

ˆ = 2 and SNR=25dB). Figure 12: Learning curves for MIMO discrete system identification (D = D Actual

70

50

60 50 40

50

70

100

150

200

50

100

(a)

250

ED

70

PT

CO2 concentration

40

M

60

Estimated

(b)

150

200

250

(c)

CE

60 50

AC

40

Figure 13:

50

100

150

Box and Jenkins’ gas furnace identification model.

CFLANN-MDRLS.

200

250

Time Sample

(a) CFLANN-RLS, (b) CFLANN-DRLS, (c)

ACCEPTED MANUSCRIPT

60 50 1

x 10

CFLANN−DRLS

40

0.5

30

0

20

−0.5

10

−1

0

50

100

150

200

0 −10 −20

0

50

100

150

200

250

300

250

300

AN US

Time Sample

CR IP T

Error

CFLANN−RLS CFLANN−MDRLS

5

Figure 14: Error in Box and Jenkins’ gas furnace identification model with three algorithms

the site http://openmv.net/info/gas-furnace. The example consists of 296 input-output samples recorded with a sampling period of 9s. The gas combustion process has one variable, gas flow u(k), and one output variable, the concentration of CO2 , y(k). The desired signal d(k) is influenced by six variables: d(k − 1), d(k − 2), d(k − 3), u(k − 1), u(k − 2), u(k − 3). Fig. 13 plots the actual and

M

210

estimated values by using recursive online-learning algorithm. When comparing the CFLANN-RLS

ED

to the proposed algorithms with λ = 0.999 and δ −1 = 0.01, we see that the CFLANN-RLS algorithm (without time delay model) obtains a stable performance during 50 < k ≤ 296 and the CFLANNDRLS algorithm fails to work. The performance of the CFLANN-DRLS algorithm can be greatly improved by introducing compensation term (CFLANN-MDRLS). The proposed CFLANN-MDRLS

PT

215

ˆ = 4. To further demonstrate algorithm is as good as the CFLANN-RLS algorithm with D = D

CE

the improved performance of the CFLANN-MDRLS algorithm, Fig. 14 shows the error in Box and Jenkin’s gas furnace identification problem. Surprisingly, the CFLANN-MDRLS even achieves improved performance as compared to the CFLANN-RLS algorithm at initial convergence stage. Therefore, we can conclude that the CFLANN-MDRLS algorithm is an outstanding alternative to

AC

220

the CFLANN-RLS algorithm for nonlinear system identification with time delay.

ACCEPTED MANUSCRIPT

5. Conclusion In this paper, two CFLANN delayed-based RLS algorithms have been proposed for nonlinear system identification problem. By considering a time delay in the coefficient adaptation, the 225

CFLANN-DRLS algorithm has been proposed. However, it has a worse convergence performance

CR IP T

than the CFLANN-RLS algorithm (without time delay). To solve this problem, the CFLANNMDRLS algorithm has been developed by introducing compensation term in the error signal, which can achieve a better performance under the imperfect system delay estimate. In addition, we have analyzed the mean behavior of the CFLANN-MDRLS algorithm. Simulations on SISO, MIMO sys230

tem identification and Box and Jenkins’ gas furnace benchmark identification problems have been conducted to verify the effectiveness of the proposed algorithms. Our future works will concern

algorithm [66], and the Kalman filter [67].

Acknowledgments 235

AN US

with extending time delay model to the Levenberg-Marquardt algorithm [65], conjugate gradient

We thank Dr. Zongsheng Zheng (School of Electrical Engineering, Southwest Jiaotong Univer-

M

sity, China), he gives us many precious suggestions and discussions. Meanwhile, the authors would like to thank all the anonymous reviewers for their valuable comments and sugestions for improving the quality of this work.

Grant 61701327.

References

PT

240

ED

This work was partially supported by the National Science Foundation of P.R. China under

[1] T. S. D. Singh, A. Chatterjee, A comparative study of adaptation algorithms for nonlinear sys-

CE

tem identification based on second order Volterra and bilinear polynomial filters, Measurement 44 (10) (2011) 1915–1923. [2] L. Lu, H. Zhao, B. Chen, Collaborative adaptive Volterra filters for nonlinear system identifi-

AC

245

cation in α-stable noise environments, J. Franklin Inst. 353 (17) (2016) 4500–4525.

[3] Z. Zheng, H. Zhao, Memory improved proportionate M-estimate affine projection algorithm, Electron. Lett. 51 (6) (2015) 525–526.

ACCEPTED MANUSCRIPT

[4] Y. Yu, H. Zhao, B. Chen, Sparse normalized subband adaptive filter algorithm with l0 -norm 250

constraint, J. Franklin Inst. 353 (18) (2016) 5121–5136. [5] D. P. Das, G. Panda, Active mitigation of nonlinear noise processes using a novel filtered-s LMS algorithm, IEEE Trans. Speech Audio Process. 12 (3) (2004) 313–322.

CR IP T

[6] V. Patel, N. V. George, Nonlinear active noise control using spline adaptive filters, Appl. Acoust. 93 (2015) 38–43. 255

[7] L. Lu, H. Zhao, Adaptive Volterra filter with continuous lp -norm using a logarithmic cost for nonlinear active noise control, J. Sound Vib. 364 (2016) 14–29.

[8] A. Fermo, A. Carini, G. L. Sicuranza, Simplified Volterra filters for acoustic echo cancellation

AN US

in GSM receivers, in: 10th European Signal Processing Conference, 2000, pp. 1–4.

[9] E. E. Kuruo˘ glu, Nonlinear least lp -norm filters for nonlinear autoregressive α-stable processes, 260

Digit. Signal Process. 12 (1) (2002) 119–142.

[10] J.-N. Juang, C.-H. Lee, Continuous-time bilinear system identification using single experiment

M

with multiple pulses, Nonlinear Dyn. 69 (3) (2012) 1009–1021.

[11] W.-D. Chang, Differential evolution-based nonlinear system modeling using a bilinear series

265

ED

model, Appl. Soft Comput. 12 (11) (2012) 3401–3407. [12] V. J. Mathews, G. L. Sicuranza, Polynomial Signal Processing, Vol. 27, Wiley-Interscience, 2000.

PT

[13] K. S. Narendra, K. Parthasarathy, Identification and control of dynamical systems using neural networks, IEEE Trans. Neural Netw. 1 (1) (1990) 4–27.

wavelet packet and Tsallis entropy, Neurocomputing 142 (2014) 393–407. [15] G. W. Chang, C.-I. Chen, Y.-F. Teng, Radial-basis-function-based neural network for harmonic

AC

270

CE

[14] Z. Liu, Q. Hu, Y. Cui, Q. Zhang, A new detection approach of transient disturbances combining

detection, IEEE Trans. Ind. Electron. 57 (6) (2010) 2171–2179.

[16] S. Khan, I. Naseem, R. Togneri, M. Bennamoun, A novel adaptive kernel for the RBF neural networks, Circuits, Syst., Signal Process. 36 (4) (2017) 1639–1653.

ACCEPTED MANUSCRIPT

275

[17] H.-G. Han, Y.-N. Guo, J.-F. Qiao, Self-organization of a recurrent RBF neural network using an information-oriented algorithm, Neurocomputing 225 (2017) 80–91. [18] S. Lu, T. Basar, Robust nonlinear system identification using neural-network models, IEEE Trans. Neural Netw. 9 (3) (1998) 407–429.

280

CR IP T

[19] M. W. Gardner, S. Dorling, Artificial neural networks (the multilayer perceptron)-a review of applications in the atmospheric sciences, Atmospheric Environment 32 (14) (1998) 2627–2636. [20] Z.-J. Fu, W.-F. Xie, W.-D. Luo, Robust on-line nonlinear systems identification using multilayer dynamic neural networks with two-time scales, Neurocomputing 113 (2013) 16–26.

[21] X. Fan, S. Li, L. Tian, Chaotic characteristic identification for carbon price and an multi-

285

3945–3952.

AN US

layer perceptron network prediction model, Expert Systems with Applications 42 (8) (2015)

[22] H. Fan, Q. Song, A linear recurrent kernel online learning algorithm with sparse updates, Neural Netw. 50 (2014) 142–153.

[23] B. Niu, L. Li, Adaptive neural network tracking control for a class of switched strict-feedback

290

M

nonlinear systems with input delay, Neurocomputing 173 (2016) 2121–2128. [24] Y. Pao, Adaptive pattern recognition and neural networks, Reading, MA (US), Addison-Wesley

ED

Publishing Co., Inc., 1989.

[25] A. Carini, G. L. Sicuranza, A new class of FLANN filters with application to nonlinear active

1954.

[26] D. Comminiello, M. Scarpiniti, S. Scardapane, R. Parisi, A. Uncini, Improving nonlinear mod-

CE

295

PT

noise control, in: 20th European Signal Processing Conference (EUSIPCO), 2012, pp. 1950–

eling capabilities of functional link adaptive filters, Neural Netw. 69 (2015) 51–59.

AC

[27] A. Carini, G. L. Sicuranza, Fourier nonlinear filters, Signal Process. 94 (2014) 183–194. [28] V. Patel, V. Gandhi, S. Heda, N. V. George, Design of adaptive exponential functional link network-based nonlinear filters, IEEE Trans. Circuits Syst. I 63 (9) (2016) 1434–1442.

ACCEPTED MANUSCRIPT

300

[29] K. Yin, H. Zhao, L. Lu, Functional link artificial neural network filter based on the q-gradient for nonlinear active noise control, J. Sound Vib. 435 (2018) 205–217. [30] G. Huang, Y. Luo, Q. Yang, Y. Tian, A semi-analytical formula for estimating peak wind load effects based on Hermite polynomial model, Eng. Struct. 152 (2017) 856–864.

305

CR IP T

[31] E. Carrera, A. De Miguel, A. Pagani, Hierarchical theories of structures based on Legendre polynomial expansions with finite element applications, Int. J. of Mech. Sci. 120 (2017) 286– 300.

[32] A. Carini, G. L. Sicuranza, A study about Chebyshev nonlinear filters, Signal Process. 122 (2016) 24–32.

310

AN US

[33] D. Comminiello, M. Scarpiniti, L. A. Azpicueta-Ruiz, J. Arenas-Garcia, A. Uncini, Functional link adaptive filters for nonlinear acoustic echo cancellation, IEEE Trans. Audio Speech Lang. Process. 21 (7) (2013) 1502–1512.

[34] J. C. Patra, R. N. Pal, A functional link artificial neural network for adaptive channel equalization, Signal Process. 43 (2) (1995) 181–195.

315

M

[35] J. C. Patra, A. C. Kot, Nonlinear dynamic system identification using Chebyshev functional link artificial neural networks, IEEE Trans. Systems, Man, and Cybernetics 32 (4) (2002)

ED

505–511.

[36] S. Purwar, I. N. Kar, A. N. Jha, On-line system identification of complex systems using Cheby-

PT

shev neural networks, Appl. Soft Comput. 7 (1) (2007) 364–372. [37] W. Ma, J. Duan, H. Zhao, B. Chen, Chebyshev functional link artificial neural network based on correntropy induced metric, Neural Process. Lett. 47 (1) (2018) 233–252.

CE

320

[38] D. I. Shuman, P. Vandergheynst, D. Kressner, P. Frossard, Distributed signal processing via Chebyshev polynomial approximation, IEEE Trans. Signal Inf. Process. Netw.doi:10.1109/

AC

TSIPN.2018.2824239.

[39] M. Li, J. Liu, Y. Jiang, W. Feng, Complex-Chebyshev functional link neural network behavioral

325

model for broadband wireless power amplifiers, IEEE Trans. Microwave Theory and Techniques 60 (6) (2012) 1979–1989.

ACCEPTED MANUSCRIPT

[40] W.-D. Weng, C.-S. Yang, R.-C. Lin, A channel equalizer using reduced decision feedback Chebyshev functional link artificial neural networks, Information Sciences 177 (13) (2007) 2642–2654. 330

[41] L. L. Jiang, D. L. Maskell, J. C. Patra, Chebyshev functional link neural network-based mod-

on Neural Networks (IJCNN), 2012, pp. 1–8.

CR IP T

eling and experimental verification for photovoltaic arrays, in: International Joint Conference

[42] J.-P. Richard, Time-delay systems: an overview of some recent advances and open problems, Automatica 39 (10) (2003) 1667–1694. 335

[43] S. Bj¨orklund, L. Ljung, An improved phase method for time-delay estimation, Automatica

AN US

45 (10) (2009) 2467–2470.

[44] J. E. Kolodziej, O. J. Tobias, R. Seara, Stochastic analysis of the modified DNLMS algorithm for Gaussian data, in: International Telecommunications Symposium, 2006, pp. 929–934. [45] M. D. Meyer, D. P. Agrawal, A modular pipelined implementation of a delayed LMS transversal 340

adaptive filter, in: IEEE International Symposium on Circuits and Systems, 1990, pp. 1943–

M

1946.

[46] C.-L. Wang, Bit-serial VLSI implementation of delayed LMS adaptive FIR filters, IEEE Trans.

ED

Signal Process. 42 (8) (1994) 2169–2175.

[47] X. Zhang, J. Xu, Time delay identifiability and estimation for the delayed linear system with incomplete measurement, J. Sound Vib. 361 (2016) 330–340.

PT

345

[48] X. Zhang, J. Xu, Z. Feng, Nonlinear equivalent model and its identification for a delayed

CE

absorber with magnetic action using distorted measurement, Nonlinear Dyn. 88 (2) (2017) 937–954.

[49] G. Long, F. Ling, J. G. Proakis, The LMS algorithm with delayed coefficient adaptation, IEEE Trans. Acoust., Speech, and Signal Process. 37 (9) (1989) 1397–1405.

AC

350

[50] G. Long, F. Ling, J. G. Proakis, Corrections to ‘ The LMS algorithm with delayed coefficient adaptation’, IEEE Trans. Signal Process. 40 (1) (1992) 230–232.

ACCEPTED MANUSCRIPT

[51] T. Wang, C.-L. Wang, Delayed least-mean-square algorithm, Electron. Lett. 31 (7) (1995) 524–526. 355

[52] M. Rupp, R. Frenzel, Analysis of LMS and NLMS algorithms with delayed coefficient update under the presence of spherically invariant processes, IEEE Trans. Signal Process. 42 (3) (1994)

CR IP T

668–672. [53] O. J. Tobias, R. Seara, Leaky delayed LMS algorithm: stochastic analysis for Gaussian data and delay modeling error, IEEE Trans. Signal Process. 52 (6) (2004) 1596–1606. 360

[54] R. D. Poltmann, Conversion of the delayed LMS algorithm into the LMS algorithm, IEEE Signal Process. Lett. 2 (12) (1995) 223.

AN US

[55] A. Yazdizadeh, K. Khorasani, Adaptive time delay neural network structures for nonlinear system identification, Neurocomputing 47 (1) (2002) 207–240. [56] P. S. Diniz, Adaptive filtering, Springer, 1997. 365

[57] W. Yu, Multiple recurrent neural networks for stable adaptive control, Neurocomputing 70 (1)

M

(2006) 430–444.

[58] L. Lu, H. Zhao, B. Chen, Improved-variable-forgetting-factor recursive algorithm based on the

(2016) 588–592. 370

ED

logarithmic cost for Volterra system identification, IEEE Trans. Circuits and Syst. II 63 (6)

[59] M. A. Woodbury, Inverting modified matrices, Memorandum report 42 (106) (1950) 336.

PT

[60] M. D. Meyer, D. P. Agrawal, Vectorization of the DLMS transversal adaptive filter, IEEE Trans. Signal Process. 42 (11) (1994) 3237–3240.

CE

[61] L.-K. Ting, R. Woods, C. F. Cowan, Virtex FPGA implementation of a pipelined adaptive LMS predictor for electronic support measures receivers, IEEE Trans. Very Large Scale Integr. (VLSI) Syst. 13 (1) (2005) 86–95.

AC

375

[62] P. K. Meher, M. Maheshwari, A high-speed FIR adaptive filter architecture using a modified delayed LMS algorithm, in: IEEE International Symposium on Circuits and Systems (ISCAS), IEEE, 2011, pp. 121–124.

ACCEPTED MANUSCRIPT

[63] A. Waibel, T. Hanazawa, G. Hinton, K. Shikano, K. J. Lang, Phoneme recognition using 380

time-delay neural networks, IEEE Trans. Acoust., Speech, and Signal Process. 37 (3) (1989) 328–339. [64] L. M. Dogariu, S. Ciochin˘ a, C. Paleologu, J. Benesty, P. Piantanida, An adaptive solution for

CR IP T

nonlinear system identification, in: IEEE Int. Symposium on Signals, Circuits and Systems, 2017, pp. 1–4. 385

[65] J. Shawash, D. R. Selviah, Real-time nonlinear parameter estimation using the LevenbergMarquardt algorithm on field programmable gate arrays, IEEE Trans. Ind. Electron. 60 (1) (2013) 170–176.

Circuits and Syst. I 39 (1) (1992) 1–10. 390

AN US

[66] G. K. Boray, M. D. Srinath, Conjugate gradient techniques for adaptive filtering, IEEE Trans.

[67] J. Zhao, L. Mili, A robust generalized-maximum likelihood unscented Kalman filter for power system dynamic state estimation, IEEE J. Selected Topics Signal Process. 12 (4) (2018) 578– 592.

Lu Lu was born in Chengdu, China, in 1990. He received the Ph.D. degree

M

in the field of signal and information processing at the School of Electrical Engineering, Southwest Jiaotong University, Chengdu, China, in 2018. From

ED

Jan. 2017 to Jan. 2018, he was a visiting Ph.D. student with the Electrical and Computer Engineering at McGill University, Montreal, QC, Canada. He is currently a postdoctoral fellow with the College of Electronics and Infor-

PT

mation Engineering, Sichuan University, Chengdu, China. His research interests include adaptive

CE

signal processing, kernel methods and distributed estimation.

Yi Yu was born in Sichuan Province, China, in 1989. He received the B.E.

AC

degree at Xihua University, Chengdu, China, in 2011, and the M.S. degree and Ph.D. degree at Southwest Jiaotong University, Chengdu, China, in 2014 and 2018, respectively. From Dec. 2016 to Dec. 2017, he was a visiting Ph.D. student with the Department of Electronic Engineering, University of York, United Kingdom. His research interests include adaptive signal processing,

ACCEPTED MANUSCRIPT

distributed estimation, and compressive sensing.

Xiaomin Yang is currently an Associate Professor in College of Electronics and Information Engineering, Sichuan University. She received her B.S. degree from Sichuan University in 2002, and received her Ph.D. degree in com-

CR IP T

munication and information system from Sichuan University in 2007. From Oct. 2009 to Oct. 2010, she worked at the University of Adelaide as a Postdoctoral for one year. At present, she has authored or co-authored of more than 50 research article in international journals and conferences. Her research interests fall under the umbrella of image processing, particularly image enlargement, super-resolution, image enhance-

AN US

ment as well as computational intelligence.

Wei Wu is currently a Professor in College of Electronics and Information Engineering, Sichuan University. He received his B.S. degree from Tianjin University, in 1998. He received M.S. and Ph.D. degrees in communication and information system from Sichuan University, in 2003 and 2008, respec-

M

tively. From Oct. 2009 to Oct. 2010, he worked in a National Research Council Canada as a post doctorate for one year. He has also edited journals special issues in the area of image enlargement, cloud computing, and embedding. At present, he has

ED

authored or co-authored of more than 70 research article in international journals and conferences. His research interests fall under the umbrella of image processing, particularly image enlargement,

PT

super-resolution, image enhancement as well as computational intelligence. Prof. Wu has served as

AC

CE

an active reviewer for several IEEE Transactions, and other international journals.