Steady-state analysis of the deficient length incremental LMS adaptive networks with noisy links

G Model AEUE-51270; No. of Pages 10

ARTICLE IN PRESS Int. J. Electron. Commun. (AEÜ) xxx (2014) xxx–xxx

Contents lists available at ScienceDirect

International Journal of Electronics and Communications (AEÜ) journal homepage: www.elsevier.com/locate/aeue

Steady-state analysis of the deficient length incremental LMS adaptive networks with noisy links Ghanbar Azarnia ∗ , Mohammad Ali Tinati Faculty of Electrical and Computer Engineering, University of Tabriz, Tabriz, Iran

a r t i c l e

i n f o

Article history: Received 21 December 2013 Accepted 18 August 2014 Keywords: Adaptive networks Deficient length Distributed estimation DILMS algorithm Noisy links

a b s t r a c t In this paper, we analyze the steady-state performance of the distributed incremental least mean square (DILMS) algorithm, considering two realistic conditions; errors that occur due to the noisy links during the transmission of local estimations between nodes, and errors that occur due to the application of deficient length adaptive filter. The length of a deficient filter is less than that of the unknown parameter, in each node. More precisely, we derive a closed-form expression for the mean-square deviation (MSD) to explain the steady-state performance at each individual node. Our simulation results show that there is a good match between simulations and derived theoretical expressions. The results show that, in comparison with the ideal case, the steady-state MSD includes two additional terms: one is related to the induced noise, and the other arises from the deficient length application that includes all the coefficients of unknown parameter that are omitted in the estimation process. © 2014 Elsevier GmbH. All rights reserved.

1. Introduction In many practical problems, we need to estimate an unknown parameter such as temperature, target location, etc., in a sensor network where nodes are distributed over a geographical area. Such a problem can be solved by either a centralized approach or a distributed one [1]. Distributed solution, only exploits local data exchanges when communication is carried out between immediate neighboring nodes, however processing is distributed among all nodes in the network. This scheme provides the network more flexibility in comparison with a centralized solution, and can be more efficient in terms of communication power and networking resources. This is obtained because of reduced processing and communication requirements. Distributed solution, also improves the robustness of performance. Actually distribution of the nodes yields spatial diversity, which can be exploited alongside the temporal dimension in order to enhance the robustness of the processing tasks [2]. In many applications we need to perform the estimation task when the data model is not available or it is changing over time. This motivates the development of distributed adaptive estimation schemes (also known as adaptive

∗ Corresponding author. Tel.: +984133393721. E-mail addresses: [email protected] (G. Azarnia), [email protected] (M.A. Tinati).

networks) [3–12]. Distributed adaptive networks are an extension of adaptive filters and can be implemented without requiring any direct knowledge of data statistics. Using cooperative processing in conjunction with adaptive filtering per node enables the entire network to track not only the variations of the environment but also any changes in the topology of the network [3–12]. Two useful strategies that enable adaption and learning over adaptive networks in real-time are the incremental strategy [3–5], and the diffusion strategy [6–10]. In an incremental strategy, information flows in a sequential manner from one node to the adjacent node. This mode of cooperation requires a cyclic pattern of cooperation among the nodes, and it tends to require the least amount of communication and power [3–5]. When more communication and energy resources are available, a diffusion cooperative scheme can be applied, where each node communicates with all of their neighbors, and no cyclic path is required. The amount of communication in this case is higher than the incremental solution, but nodes have access to more data from their neighbors. This mode of cooperation usually includes two steps: an adaption step where nodes use their individual measurements to adapt their current estimate, and a combination step where nodes combine the estimates from their neighbors to obtain a new estimate [6–10]. Several combination rules, such as the Metropolis and Relative-degree rule [9] have been proposed that are based solely on the network topology. Since, the design

http://dx.doi.org/10.1016/j.aeue.2014.08.007 1434-8411/© 2014 Elsevier GmbH. All rights reserved.

Please cite this article in press as: Azarnia G, Tinati MA. Steady-state analysis of the deficient length incremental LMS adaptive networks with noisy links. Int J Electron Commun (AEÜ) (2014), http://dx.doi.org/10.1016/j.aeue.2014.08.007

G Model AEUE-51270; No. of Pages 10 2

ARTICLE IN PRESS G. Azarnia, M.A. Tinati / Int. J. Electron. Commun. (AEÜ) xxx (2014) xxx–xxx

of combination weights plays a key role in the diffusion mode of cooperation, in [10] a diffusion algorithm with adaptive combiner is proposed in order to improve the performance of the diffusion algorithm. In all of the distributed adaptive estimation algorithms referred above, that is [3–10], the links between nodes are assumed to be ideal. The effect of noisy links on the performance of incremental and diffusion adaptive networks are studied in [11,12]. The importance of such study stems from this fact that the performance of distributed adaptive estimation algorithm can drastically be deteriorated in the presence of noisy links [11,12]. In this paper, in addition to this non-ideality, another nonideality condition that is the deficiency of the filter length is considered. In all of the previous works [3–12], it is assumed that the length of the adaptive filter in each node is equal to that of the unknown parameter. Actually, the length of the unknown parameter similar to its coefficients is unknown. So, in many practical situations, a deficient length adaptive filter whose length is less than that of the unknown parameter is employed in each node. However, for a non-distributive adaptive filter case such an study have been done [13–18], but this analysis in the distributive adaptive network domain is challenging due to the fact that nodes in every neighborhood interact with each other and, therefore a successful analysis must take into account both the temporal and spatial interconnectedness of the data. Different nodes will converge to different MSD levels, reflecting the statistical diversity of the data and the different noise levels. In principle, the minimum mean squared error (MMSE) is a monotonic nonincreasing function of the tap-length. However, it is not suitable to have a too long filter, as it not only unnecessarily increases the complexity but also introduces more adaption noise. On the other hand, the decrease in the MMSE due to the tap-length increase always becomes trivial when the tap-length is long enough. Therefore, there exists an optimum tap-length, Lopt , that best balances the steady-state performance and complexity. Usually, one does not know what is the optimum filter length, therefore each node is equipped with an adaptive filter with M coefficients, in which we assume M < Lopt . In other words, a deficient length adaptive filter in each node is employed. Since the theoretical results in the sufficient length case do not necessarily apply to the realistic deficient length situation, so in this paper we study the performance of the deficient length DILMS algorithm with noisy links. Such an study is challenging due to the fact that, by considering the deficient length condition in context of adaptive networks with noisy links, a length mismatch appears between the unknown parameter and both its local estimation and the noise vector that is being added to the local estimations during the transmission phase. Another problem that is addressed in this analysis is that the nodes with one hop distance from each other interact, and therefore, a successful analysis must take into account both the temporal and spatial interconnectedness of the data. On this basis, in this paper, we analyze the steady-state performance of the DILMS algorithm, considering two non-ideality conditions; noisy links and deficient length. More precisely, we derive a closed-form expression for the MSD to explain the steadystate performance at each individual node. Our simulation results show that there is a good match between simulations and derived theoretical expressions. The results show that, in comparison to the ideal case, the steady-state MSD includes two additional terms: one is related to the induced noise, and the other is arisen from the deficient length application that includes all the coefficients of unknown parameter that are omitted in the estimation process. Notation: For ease of reference, the main symbols used in this paper are listed:

col{·} E{·} ||·||2 |·|2 0M × L IL tr(A) (·)*

column vector; statistical expectation; squared Euclidean norm operation; absolute squared operation; M × L zero matrix; L × L identity matrix; Trace of matrix A; conjugation for scalars and Hermitian transpose for matrices;

2. Deficient length DILMS algorithm Consider a set of N sensors that are randomly distributed over a region. The purpose is to estimate an Lopt × 1 unknown vector w oL opt

from multiple measurements collected at N nodes in the network. We assume that both the coefficients and length Lopt of the vector parameter w oL are unknown. Since the length Lopt is unknown, opt

so a conjectural length M for the unknown parameter is assumed at each node, where M < Lopt . Also, suppose that each node k has access to time realizations {dk (i), uk,i } of zero-mean spatial data {dk , uk }, where each dk is a scalar measurement and each uk is a 1 × M row regression vector. Collecting the regression and measurement data into global matrices results U  col{u1 , u2 , . . .uN } (N × M)

(1)

d  col{d1 , d2 , . . .dN } (N × 1)

(2)

The objective is to estimate the M × 1 vector w that solves argminJ(w)

(3)

w

where J(w) = E{||d − Uw||2 } The optimal solution

(4) wo

satisfies the normal equations [3]

r du = R u w o where

wo

(5)

has length equal to M, and

∗

R u = E{U U},

r du = E{U ∗ d}.

(6)

Note that in order to use (5) to compute w o , each node must have access to the global statistical information {r du , R u } which in turn requires more communications between nodes and computational resources. Moreover, such an approach doesn’t enable the network to respond to changes in statistical properties of data. In [3] a distributed incremental LMS (DILMS) strategy with a cyclic estimator structure with non-noisy-links and a non-optimal pre-determined filter length is proposed as (i) k

=

(i) k−1

+ k u∗k,i (dk (i) − uk,i

(i) ), k−1

k∈N

(7)

(i)

where the M × 1 vector k indicates the local estimate at node k and time i, and k is a suitably chosen positive local step-size parameter. For each time i, each node k utilizes the local data dk (i), (i) (i) uk,i , and k−1 received from the node k − 1 to obtain k . At the (i)

end of this cycle, N is employed as the initial condition for the next time instant at node k = 1. In the presence of noisy links the update equation for DILMS changes to [11] (i) k

=

(i) k−1

+ qk,i + k u∗k,i [dk (i) − uk,i (

(i) k−1

+ qk,i )]

(8)

where the M × 1 vector qk,i is the channel noise term between node k − 1 and k. We assume that qk,i is a time realization of a wide-sense stationary random process qk which is

Please cite this article in press as: Azarnia G, Tinati MA. Steady-state analysis of the deficient length incremental LMS adaptive networks with noisy links. Int J Electron Commun (AEÜ) (2014), http://dx.doi.org/10.1016/j.aeue.2014.08.007

G Model

ARTICLE IN PRESS

AEUE-51270; No. of Pages 10

G. Azarnia, M.A. Tinati / Int. J. Electron. Commun. (AEÜ) xxx (2014) xxx–xxx

assumed to be zero-mean with covariance matrix Q k . We don’t impose any special distribution restriction for the channel noise term.

node. This partitioning makes it easy to work with vectors that have different lengths. By this partitioning, (13) could be written as ek (i) = uLopt k,i w oLopt − uk,i



3. Performance analysis of deficient length DILMS with noisy links

w oL

3



˙ M w

= uLopt k,i

(i) k−1

⎡

(i) k−1

= −uLopt k,i ⎣

opt

We analyze such condition in the context of incremental adaptive network with noisy links.

¨

+ vk (i)

0(Lopt −M)×1

⎤

⎦ + vk (i)



¯ (i)

M,k−1

= −uLopt k,i

To perform the performance analysis, following assumptions that are commonly assumed in the literature for adaptive algorithms, are used as well.

˙ M −w



(i) k−1

−w Lopt −M

 3.1. Data model and assumptions



− uLopt k,i

¨

w Lopt −M

As mentioned, we assume that there is an optimal length for as Lopt , where presumed filter length M is not equal to it.

+ vk (i)

+ vk (i)

¨ Lopt −M −w

(i) = −uLopt k,i ¯ Lopt ,k−1 + vk (i)

(15)

(i)

1) We assume a linear measurement model as dk (i) = uLopt k,i w oLopt + vk (i).

(9)

The vector ¯ Lopt ,k−1 measures the difference between the weight estimate at node k − 1 and the desired optimal solution (i) w o . The vector ¯ is also a measure of difference between M,k−1

Lopt

where vk (i) is some temporal and spatial white noise sequence with zero mean and variance v2,k and is independent of uLopt ,j and d (j) for all , j. In (9) uLopt k,i is a vector whose length equals to Lopt , and uk,i consists of M first coefficients of uLopt k,i as uLopt k,i = [uk (i), uk (i − 1), . . ., uk (i − Lopt + 1)]

(10)

uk,i = [uk (i), uk (i − 1), . . ., uk (i − M + 1)]

(11)

The linear model of (9) is sometimes called stationary model in which the unknown parameter w oL is fixed and the statistics opt

of the input and noise signals are time-invariant. / . 2) uLopt k,i is independent of uLopt ,i for k = 3) uLopt k,i is independent of uLopt k,j for i = / j. 4) The components of uLopt k,i are drawn from a zero mean white 2 , in other words the covariGaussian process with variance u,k 2 I. u,k

ance matrix of uLopt k,i is R u,k = 5) The channel noise qk,i is independent of uLopt ,j , q,j and v (j) for all , j.

weights, but it only uses the coefficients of w oL

opt

by

Substituting (15) into (12) results (i) k

(i) k−1

=

(i) + qk,i − k u∗k,i uLopt k,i ¯ Lopt ,k−1 + k vk (i)u∗k,i

− k u∗k,i uk,i qk,i

(i)

MSDk = E{|| ¯ Lopt ,k−1 ||2 } we must subtract the vector w oL

opt

opt





(i) k

− k

=

+ qk,i + k u∗k,i ek (i) − k u∗k,i uk,i qk,i

(12)

(i) k−1

˙ M w



(i) Lopt ,k−1

u∗k,i

(14)

˙ M is the M first coefficients of w oL where w

opt

˙ M is . In other words w

(i) ¨ Lopt −M is the part of w oL that is modeled by k in each node, and w opt o the part of w L that is excluded in the estimation of w oL in each opt opt

0(Lopt −M)×1



− w oLopt



u∗k,i



+ k vk (i)

+



qk,i 0(Lopt −M)×1

u∗k,i



0(Lopt −M)×1

uk,i qk,i



(i) = ¯ Lopt ,k−1 +

 − k uLopt k,i



opt

¨ Lopt −M w



(i) k−1

0(Lopt −M)×1



(13)

In order to derive an expression for steady state MSD, we partition the unknown parameter w oL as

with length Lopt from both sides

(18)

(i) and according to the definition of ¯ Lopt ,k−1 we have

Lopt ,k

ek (i) = dk (i) − uk,i

=



0(Lopt −M)×1

¯ (i)

where







− w oLopt

0(Lopt −M)×1

3.2. Performance analysis

(i) k−1

(17)

of (16), but problem is that the vectors in (16) have length M. To solve this difficulty we pad the vectors in (16) by Lopt − M zeros. This results vectors with length Lopt that allows the subtracting of w oL from both sides of (16), so we have

− k uLopt k,i ¯

(i) k

(16)

In order to evaluate the MSD defined as

According to these assumptions, we want to evaluate the steady state MSD for each node k.

To proceed, first we rewrite the update Eq. (8) as

that are modeled

(i) . k

− k

0(Lopt −M)×1 u∗k,i

¯ (i)

Lopt ,k−1



u∗k,i 0(Lopt −M)×1

by defining k (i) = ILopt − k





qk,i

0(Lopt −M)×1



 + k vk (i)

uk,i qk,i

u∗k,i 0(Lopt −M)×1

u∗k,i



0(Lopt −M)×1 (19)

 uLopt k,i

(20)

Please cite this article in press as: Azarnia G, Tinati MA. Steady-state analysis of the deficient length incremental LMS adaptive networks with noisy links. Int J Electron Commun (AEÜ) (2014), http://dx.doi.org/10.1016/j.aeue.2014.08.007

G Model

ARTICLE IN PRESS

AEUE-51270; No. of Pages 10 4

G. Azarnia, M.A. Tinati / Int. J. Electron. Commun. (AEÜ) xxx (2014) xxx–xxx

The relation (19) can be written as



¯ (i)

qk,i

(i)

= k (i) ¯ Lopt ,k−1 +

Lopt ,k

 − k uk,i qk,i



 + k vk (i)

0(Lopt −M)×1

u∗k,i 0(Lopt −M)×1



u∗k,i



(21)

0(Lopt −M)×1

In order to derive an expression for MSD, first we express || ¯

(i) 2 Lopt ,k ||

|| ¯



= ¯



¯

(i) Lopt ,k−1

+ ¯



∗(i) + k k (i) ¯ Lopt ,k−1 ∗k (i)

0(Lopt − M) × 1



∗(i) ∗ Lopt ,k−1 k (i)uk,i qk,i



u∗k,i

∗(i) ∗ Lopt ,k−1 k (i)

∗(i)

(i)

= E{ ¯ Lopt ,k−1 E[∗k (i)k (i)] ¯ Lopt ,k−1 }

u∗k,i

+ q∗k,i

0(Lopt −M)×1



+ k k (i) uk,i





01×(Lopt −M)



opt k,i

= ILopt

(22)

To proceed, we should take the expectations from both sides of (22). For this we use the assumptions (1)–(5). From these assumptions several results are deduced as follows: Result (1): Since the matrix k (i) only consists of the regression vectors, so it is independent of measurement noise vk (i) and channel noise qk,i . From

(i) k−1

=

(i) k−2

+ qk−1,i + k−1 u∗k−1,i (dk−1 (i) − (i) k−1

uk−1,i ( + qk−1,i )) it is clear that only depends on the measurements dl (j) of node k − 1 and its previous nodes in current iteration, and also on the measurements of its previous iterations. So, from assumption (1) the noise vk (i) will be independent of (i) (i) and ¯ . Lopt ,k−1

(i)

Result (3): Since k−1 only depends on the regression vectors u,j of node k − 1 and previous nodes in current iteration, and the regressions of previous iterations, so from assumptions (2) and (i) (3) it is clear that uk,i and also k (i) are independent from k−1 (i)

and ¯ Lopt ,k−1 . Result (4): In the similar manner, channel noise qk,i is independent (i) of ¯ Lopt ,k−1 .

Using the assumptions (1)–(5) and their results, and taking expectations of both sides of (22) leads to ∗(i)

uk,i

uLopt k,i



01×(Lopt −M) }

uLopt k,i ||uk,i ||2 }

(25)

⎧⎡ ⎫ ⎤ u∗k (i) ⎪ ⎪ ⎪ ⎪ ⎪ ⎪ ⎥ ⎪⎢ ⎪ ⎪ ⎪ . ⎢ ⎥ ⎪ ⎪ ⎪ ⎪ . ⎥ ⎪⎢ ⎪ . ⎪ ⎪ ⎢ ⎥ ⎪ ⎪ ⎨⎢ u∗ (i − M + 1) ⎥ ⎬ k ⎢ ⎥ − k E ⎢ [u (i). . .u (i − L + 1)] opt k k ⎥ ⎪ ⎪ ⎢ ⎥ ⎪ ⎪ 0 ⎪ ⎪ ⎥ ⎪⎢ ⎪ ⎪ ⎪ ⎢ ⎥ ⎪ ⎪ . ⎪ ⎪ . ⎪ ⎪ ⎣ ⎦ . ⎪ ⎪ ⎪ ⎪ ⎩ ⎭

⎧⎡ ⎪ ⎪ ⎨⎢ − k E ⎢ ⎣ ⎪ ⎪ ⎩

+ 2k E

⎪ ⎪ ⎩

⎢ ⎣

×⎢

⎫ ⎪ ⎪ ⎬

⎤

u∗k (i) .. . u∗k (i − Lopt + 1)

⎥ ⎥ [uk (i). . .uk (i − M + 1), 0. . .0] ⎦ ⎪ ⎪ ⎭ 

2

|uk (i − n)|

n=0

u∗k (i) .. . u∗k (i − Lopt + 1)

⎫ ⎪ ⎪ ⎬

⎤

⎥ ⎥ [uk (i). . .uk (i − Lopt + 1)] ⎦ ⎪ ⎪ ⎭

(26)

From assumption (4) the first expectation in (26) is

⎧⎡ ⎫ ⎤ u∗k (i) ⎪ ⎪ ⎪ ⎪ ⎪ ⎪ ⎢ ⎪ ⎥ ⎪ ⎪ ⎪ . ⎢ ⎪ ⎥ ⎪ ⎪ ⎪ . ⎢ ⎪ ⎥ ⎪ . ⎪ ⎪ ⎪ ⎥ ⎪ ⎨⎢ ⎬ ⎢ u∗ (i − M + 1) ⎥  k ⎥ uk (i) . . . uk (i − Lopt + 1) E ⎢ ⎢ ⎥ ⎪ ⎪ ⎢ ⎪ ⎥ ⎪ 0 ⎪ ⎪ ⎢ ⎪ ⎥ ⎪ ⎪⎢ ⎪ ⎪ ⎥ ⎪ .. ⎪ ⎪ ⎪ ⎪ ⎣ ⎦ . ⎪ ⎪ ⎪ ⎪ ⎩ ⎭ 

(i)

2 = u,k

− 2k E{uk,i qk,i q∗k,i u∗k,i } + 2k E{v2k (i)}E{||uk,i ||2 }

0

⎧ ⎪ M−1 ⎪ ⎨ 

⎡

E{|| ¯ Lopt ,k ||2 } = E{ ¯ Lopt ,k−1 ∗k (i)k (i) ¯ Lopt ,k−1 } + E{||qk,i ||2 }

+ 2k E{||uk,i ||2 q∗k,i u∗k,i uk,i qk,i }

0(Lopt −M)×1



E{∗k (i)k (i)}

(i)

(i) k−2



u∗k,i

By substituting the vector components, we get

(i)

− 2k k (i)||uk,i ||2 q∗k,i u∗k,i + 2k ||uk,i ||2 q∗k,i u∗k,i uk,i qk,i

k−1

opt k,i



01×(Lopt −M) q∗k,i u∗k,i k (i) ¯ Lopt ,k−1

(2):

+ 2k E{u∗L

01×(Lopt −M) k (i) ¯ Lopt ,k−1 + k k (i)uk,i qk,i





− k E{u∗L

+ 2k k2 (i)||uk,i ||2 − 2k k (i)||uk,i ||2 uk,i qk,i − k uk,i qk,i q∗k,i u∗k,i − k uk,i

= ILopt − k E

0(Lopt −M)×1



(24)

So, first we calculate the following expectation

E{∗k (i)k (i)}

(i) ∗k ¯ Lopt ,k−1 + ||qk,i ||2 + k k (i)q∗k,i u∗k,i − k uk,i qk,i q∗k,i u∗k,i

(i)

(i)

 ∗(i) ∗ Lopt ,k−1 k (i)k (i)

qk,i

Result

∗(i)

E{ ¯ Lopt ,k−1 ∗k (i)k (i) ¯ Lopt ,k−1 }

as

(i) 2 Lopt ,k ||

− k ¯

To proceed, we need to evaluate the moments in (23). From result (3) we have

0

IM

0M×(Lopt −M)

0(Lopt −M)×M

0(Lopt −M)×(Lopt −M)

 (27)

(23)

Please cite this article in press as: Azarnia G, Tinati MA. Steady-state analysis of the deficient length incremental LMS adaptive networks with noisy links. Int J Electron Commun (AEÜ) (2014), http://dx.doi.org/10.1016/j.aeue.2014.08.007

G Model

ARTICLE IN PRESS

AEUE-51270; No. of Pages 10

G. Azarnia, M.A. Tinati / Int. J. Electron. Commun. (AEÜ) xxx (2014) xxx–xxx

and the second expectation of (26) could be written as

⎧⎡ ⎪ ⎪ ⎨⎢ E ⎢ ⎣ ⎪ ⎪ ⎩

u∗k (i) .. . u∗k (i − Lopt + 1)



2 = u,k

Substituting (27)–(31) into (26) results



⎫ ⎪ ⎪ ⎬

⎤

E{∗k (i)k (i)}

⎥ ⎥ [uk (i). . .uk (i − M + 1), 0. . .0] ⎦ ⎪ ⎪ ⎭

IM

0M×(Lopt −M)

0(Lopt −M)×M

0(Lopt −M)×(Lopt −M)

 4 + 2k u,k



E

⎪ ⎪ ⎩

 |uk (i − n)|2

n=0

⎡

u∗k (i)

⎢ ⎢ ⎣

.. .

⎤ ⎥ ⎥ ⎦

|uk (i)|2

···

u∗k (i)uk (i − Lopt + 1)

.. .

..

.. .

u∗k (i − Lopt + 1)uk (i)

···

E

0M×(Lopt −M)

0(Lopt −M)×M

MILopt −M

(32)

|uk (i − Lopt + 1)|2



=

ˇk IM

0M×(Lopt −M)

0(Lopt −M)×M

k ILopt −M

 (33)

⎤⎞⎫ ⎪ ⎪ ⎥⎟⎬ ⎥⎟ ⎦⎠⎪ ⎪ ⎭

(34) (35)

(i) On the other hand, from ¯ Lopt ,k−1 = (i) opt ,k−1

(29)



¯ (i)

¨ Lopt −M ||2 ||2 = || ¯ M,k−1 ||2 + ||w

(i)





¯

=E

∗(i) Lopt ,k−1

=E

 ×

¯

∗(i) M,k−1

ˇk IM

0M×(Lopt −M)

0(Lopt −M)×M

k ILopt −M

¨ ∗Lopt −M −w



¯ (i)

M,k−1

¨ ∗Lopt −M −w

!



=E

(i)

n=0

(36)

Using (33) and (36) in (24) we get ∗(i)

E{|uk (i − n)|2 }E{|uk (i − m + 1)|2 } + E{|uk (i − m + 1)|4 }

we have

M,k−1

¨ Lopt −M −w

E{ ¯ Lopt ,k−1 ∗k (i)k (i) ¯ Lopt ,k−1 }

|uk (i − n)|2 |uk (i − m + 1)|2



(i)

|| ¯ L

n=0

=

(M + 2)IM

E{∗k (i)k (i)}



M−1 

0(Lopt −M)×(Lopt −M)



4 k  1 + 2k u,k M

In (29) the non-diagonal components are zero, since for a Gaussian random process with zero-mean, the odd order moments are zero. For the diagonal components, two cases are possible: First: for the (m,m)th component of (29), where 1 ≤ m ≤ M we have

M−1 

0(Lopt −M)×M

2 4 ˇk  1 − 2k u,k + 2k u,k (M + 2)

u∗k (i − Lopt + 1)

.

0M×(Lopt −M)

where we define ˇk and k as

⎧ ⎛ ⎪ ⎪ M−1 ⎨ ⎜ ⎜|uk (i − n)|2 ×[uk (i). . .uk (i − Lopt + 1)] = E ⎝ ⎪ ⎪ ⎪ ⎪ ⎭ ⎩ n=0 ⎢ ×⎢ ⎣

=



IM

We rewrite (32) as

⎫ ⎪ ⎪ ⎬

⎡

2 ILopt − 2k u,k

(28)

and finally for the last expectation of the right hand side of (26) we have

⎧ ⎪ M−1 ⎪ ⎨ 

5

ˇk ¯



 ¯

(i) Lopt ,k−1

ˇk IM

0M×(Lopt −M)

0(Lopt −M)×M

k ILopt −M

∗(i) M,k−1

¨ ∗Lopt −M −k w

!





¯ (i)



M,k−1

¨ Lopt −M −w

(i)

¨ Lopt −M ||2 = ˇk E{|| ¯ Lopt ,k−1 ||2 } = ˇk E{|| ¯ M,k−1 ||2 } + k ||w ¨ Lopt −M ||2 + (k − ˇk )||w

(37)

n= / m−1 =

M−1 

4 4 4 4 4 u,k + 3u,k = (M − 1)u,k + 3u,k = (M + 2)u,k (30)

n=0

Now we need to evaluate the other moments in the right-hand side of (23). We have E{||qk,i ||2 } = tr(E{||qk,i ||2 }) = E(tr{||qk,i ||2 }) = E(tr{q∗k,i qk,i }) = E(tr{qk,i q∗k,i }) = tr(E{qk,i q∗k,i }) = tr(Q k )

n= / m−1 Note that in (30) we used the fact that for a Gaussian random process, kurtosis is zero. Second: for the (m,m)th component of (29), where M + 1 ≤ m ≤ Lopt we have

E{uk,i qk,i q∗k,i u∗k,i } = tr(E{uk,i qk,i q∗k,i u∗k,i }) = E(tr{uk,i qk,i q∗k,i u∗k,i }) = E(tr{u∗k,i uk,i qk,i q∗k,i }) = tr(E{u∗k,i uk,i }E{qk,i q∗k,i }) 2 2 I M Q k } = u,k tr{Q k } = tr{R u,k Q k } = tr{u,k

E

M−1 

=

n=0

(39)

 2

|uk (i − n)| |uk (i − m + 1)|

2

E{v2k (i)}E{||uk,i ||2 } = v2,k tr(E{||uk,i ||2 }) = v2,k E(tr{uk,i u∗k,i })

n=0 M−1 

(38)

= v2,k E(tr{u∗k,i uk,i }) = v2,k tr(E{u∗k,i uk,i }) 4 E{|uk (i − n)|2 }E{|uk (i − m + 1)|2 } = Mu,k

(31)

2 2 I M ) = Mv2,k u,k = v2,k tr(R u,k ) = v2,k tr(u,k

(40)

Please cite this article in press as: Azarnia G, Tinati MA. Steady-state analysis of the deficient length incremental LMS adaptive networks with noisy links. Int J Electron Commun (AEÜ) (2014), http://dx.doi.org/10.1016/j.aeue.2014.08.007

G Model

ARTICLE IN PRESS

AEUE-51270; No. of Pages 10 6

G. Azarnia, M.A. Tinati / Int. J. Electron. Commun. (AEÜ) xxx (2014) xxx–xxx

E{||uk,i ||2 q∗k,i u∗k,i uk,i qk,i } = tr(E{q∗k,i u∗k,i uk,i qk,i ||uk,i ||2 })

Iterating in the same manner, we have E{||P Lopt ,k−1 ||2 } = ˇk−1 ˇk−2 . . .ˇ1 ˇN ˇN−1 . . .ˇk E{||P Lopt ,k−1 ||2 }

= E(tr{q∗k,i u∗k,i uk,i qk,i ||uk,i ||2 })

+ ˇk−1 ˇk−2 . . .ˇ1 ˇN ˇN−1 . . .ˇk+1 fk

= E(tr{qk,i q∗k,i u∗k,i uk,i ||uk,i ||2 })

+ ˇk−1 ˇk−2 . . .ˇ1 ˇN ˇN−1 . . .ˇk+2 fk+1 + . . .

= tr(E{qk,i q∗k,i }E{||uk,i ||2 u∗k,i uk,i })

+ ˇk−1 ˇk−2 . . .ˇ1 ˇN fN−1 + ˇk−1 ˇk−2 . . .ˇ1 fN

4 (M + 2)I M ) = tr(Q k u,k 4 = (M + 2)u,k tr(Q k )

+ ˇk−1 ˇk−2 . . .ˇ2 f1 + ˇk−1 ˇk−2 . . .ˇ4 ˇ3 f2 + . . . (41)

In (41) for the evaluation of E{||uk,i ||2 u∗k,i uk,i }, we used what was obtained in (30). Substituting (37)–(41) into (23) we get (i)

(i)

¨ Lopt −M ||2 + tr(Q k ) E{|| ¯ Lopt ,k ||2 } = ˇk E{|| ¯ Lopt ,k−1 ||2 } + (k − ˇk )||w 2 2 4 − 2k u,k tr{Q k } + 2k Mv2,k u,k + 2k (M + 2)u,k tr(Q k )

+ ˇk tr(Q k ) = ˇk E{|| ¯

¨ Lopt −M ||2 + k − ˇk )||w

(42)

(43)

One must notice that, the effects of noisy links in deficient length DILMS algorithm only appear in k by tr(Q k ). Now we define ¨ Lopt −M ||2 + k fk  (k − ˇk )||w

(44)

By this definition, (42) can be rewritten in a summarized form as

(i)

(i)

E{|| ¯ Lopt ,k ||2 } = ˇk E{|| ¯ Lopt ,k−1 ||2 } + fk

(45)

Since we are interested in steady-state analysis, as (i → ∞) in (∞) (45), with assumption of P = ¯ , (45) can be written as Lopt ,k

Lopt ,k

E{||P Lopt ,k ||2 } = ˇk E{||P Lopt ,k−1 ||2 } + fk

Now we define for each node k, a set of N quantities as

k,  ˇk−1 ˇk−2 . . .ˇ1 ˇN ˇN−1 . . .ˇk+ ˇk+−1 ,

 = 1, . . ., N

sk  k,2 fk + k,3 fk+1 + . . . + k,N−1 fk−3 + k,N fk−2 + fk−1

E{||P Lopt ,k−1 ||2 } = k,1 E{||P Lopt ,k−1 ||2 } + sk

where we define k as 2 + ˇk tr(Q k ) k  M2k v2,k u,k

(49)

(50) (51)

From (49)–(51) we have

(i)

2 ¨ Lopt −M ||2 + 2k Mv2,k u,k = ˇk E{|| ¯ Lopt ,k−1 ||2 } + (k − ˇk )||w (i) 2 Lopt ,k−1 || } + (k

+ ˇk−1 ˇk−2 fk−3 + ˇk−1 fk−2 + fk−1

(52)

From (52) we derive an expression for the desired steady-state MSD as E{||P Lopt ,k−1 ||2 } = (1 − k,1 )

−1

sk

(53)

3.3. Discussion on derived theoretical result The final equation derived for MSD as (53) is a complicated relationship and its implications and how one can gain useful information from it to use in a deficient length adaptive filter in each node is not so obvious. In order to obtain a clear view about the effect of deficient length application and noisy link effects, we simplify the Eq. (53). For this purpose, we assume that k = , ∀k ∈ N, 2 I hold true. Also we assume that  is R u,k = u2 I, and Q k = c,k small enough, so that the 2 term can be ignored in ˇk . With these assumptions ˇk can be approximated as ˇk = 1 − 2u2

(54)

and as a result, ˘k,1 can be approximated as (46)

Observe, however, that (46) is a coupled equation: it involves both E{||P Lopt ,k ||2 } and E{||P Lopt ,k−1 ||2 }, i.e., information from two spatial locations. To resolve this difficulty, we take the advantage of the ring topology that is inherent in the incremental strategy. Thus, by iterating (46) we have

N

k,1  ˇ1 ˇ2 . . .ˇN = (1 − 2u2 ) ≈ (1 − 2Nu2 )

(55)

and also fk can be approximated to 2 ¨ Lopt −M ||2 + M2 u2 v2,k + M(1 − 2u2 )c,k fk ≈ 2u2 (1 − u2 )||w

(56) We have

E{||P Lopt ,1 ||2 } = ˇ1 E{||P Lopt ,N ||2 } + f1 sk =

E{||P Lopt ,2 ||2 } = ˇ2 E{||P Lopt ,1 ||2 } + f2

N 

¨ Lopt −M ||2 + M2 u2 fk = 2Nu2 (1 − u2 )||w

N 

k=1

.. .

v2,k

k=1

 N

E{||P Lopt ,k−2 ||2 } = ˇk−2 E{||P Lopt ,k−3 ||2 } + fk−2

(47)

+ M(1 − 2u2 )

2 c,k

(57)

k=1

E{||P Lopt ,k−1 ||2 } = ˇk−1 E{||P Lopt ,k−2 ||2 } + fk−1

Now, by replacing (55) and (57) in (53), we obtain

.. .

¨ Lopt −M ||2 + E{||P Lopt ,k−1 ||2 } = (1 − u2 )||w

Observe that according to (47), E{||P Lopt ,k−1 ||2 } can be expressed in term of E{||P Lopt ,k−3

||2 }

M(1 − 2u2 )  N

E{||P Lopt ,N ||2 } = ˇN E{||P Lopt ,N−1 ||2 } + fN

as

2Nu2

M  2 v,k 2N

2 c,k

k=1

N

+

(58)

k=1

E{||P Lopt ,k−1 ||2 } = ˇk−1 ˇk−2 E{||P Lopt ,k−3 ||2 } + ˇk−1 fk−2 + fk−1

(48)

Now several results are implied from (58) as

Please cite this article in press as: Azarnia G, Tinati MA. Steady-state analysis of the deficient length incremental LMS adaptive networks with noisy links. Int J Electron Commun (AEÜ) (2014), http://dx.doi.org/10.1016/j.aeue.2014.08.007

G Model

ARTICLE IN PRESS

AEUE-51270; No. of Pages 10

G. Azarnia, M.A. Tinati / Int. J. Electron. Commun. (AEÜ) xxx (2014) xxx–xxx

Fig. 1. The observation noise profile  2v,k , and  2u,k .

1) The first term on the right side of (58), is indeed the additional error due to the deficient length application. This term consist of all the omitted coefficients of unknown parameter, namely ¨ Lopt −M , in the estimation process. Whenever, the considered w length for the adaptive filter in each node, namely M, is less than Lopt , this term will be larger. In other words, as the selected length becomes more deficient, the error will increase. 2) The effect of noisy links on the steady state MSD is observed in the second term on the right side of (58). As the channel becomes noisier, this term would be larger and will have greater contribution in the steady state MSD. 4. Simulation results In this section, we show the simulation results performed to verify the theoretical derivations developed in the previous section. All figures are obtained by averaging the results over 100 Monte Carlo trials of the same experiment. The steady-state curves are obtained by averaging the last 2000 instantaneous samples of 10,000 iterations. We have used a distributed network with incremental topology and noisy links. This network consists of N = 20 sensors, each running as an adaptive filter with M taps to " estimate the Lopt × 1 unknown parameter w oL = col{1, 1, . . ., 1}/ Lopt where opt

Lopt = 16. The measurement data {dk (i)} are generated according to model (9) and the regressors are independent zero-mean Gauss2 I. The observation ians, and have covariance matrix R u,k = u,k 2 are as shown in Fig. 1. We set  = 0.01 noise variance v2,k and u,k k for all nodes. The steady-state MSD curves in every node k for different tap-length M are shown in Figs. 2 and 3 with channel noise covariance matrix Q k = 10−4 I and Q k = 10−3 I, respectively. In these figures we have also shown curves obtained from theoretical derivations as well. As it is clear from Figs. 2 and 3, there is a good match between theory and simulations, and so, the derived theoretical expression for MSD can predict the steady-state performance of deficient length DILMS algorithm. It is obvious from Fig. 2 that, for M = 10, 8, 6 and 4, the steady-state MSD values for node k = 1 are −3.33, −2.332, −1.56, and −0.942 dB respectively. This means that, whenever the considered length for the adaptive filter in each node, namely M, is less than Lopt , the steady-state MSD increases. On the other hand as is obvious from Figs. 2 and 3, a large channel noise covariance matrix, will lead to a large steady state MSD value. In order to compare the steady-state MSD curves in every node k for the cases of no deficiency of the filter length and no noisy links, steady-state MSD curves versus node numbers are shown in Fig. 4.

Fig. 2. The steady-state MSD versus node for different tap-length, Qk = 10−4 I.

Please cite this article in press as: Azarnia G, Tinati MA. Steady-state analysis of the deficient length incremental LMS adaptive networks with noisy links. Int J Electron Commun (AEÜ) (2014), http://dx.doi.org/10.1016/j.aeue.2014.08.007

7

G Model AEUE-51270; No. of Pages 10 8

ARTICLE IN PRESS G. Azarnia, M.A. Tinati / Int. J. Electron. Commun. (AEÜ) xxx (2014) xxx–xxx

Fig. 4. The steady-state MSD versus node for the case of full length and no noise condition.

This is clear from these figures that the two realistic conditions; noisy links and deficient length can drastically deteriorate the performance of distributed adaptive estimation algorithm in term of MSD. These simulations justify the derived theoretical results. In Fig. 5 the steady-state MSD for node k = 1 versus different val2 ∈ (0, 0.5] ues of  is plotted. In this case we used Lopt = 5, M = 3, u,k

and Q k = 10−4 I. The other setup parameters are the same as those in the previous simulation. Again, as it is clear from Fig. 5, simulation results coincide very well to the graph of the theoretical analysis. We note that unlike the ideal link case, here the simulation results show that the steady-state MSD are not monotonically increasing functions of , and there is an optimum value for . This interesting result is due to the presence of the noisy links. Note that, in our analysis we use assumption (4) only in the computation of Eq. (26) in order to get a simple form for the MSD. So the motivation of this assumption is to achieve a simple formula for the MSD that clearly shows the effect of noisy links and deficient length. Nevertheless in order to clarify the matters, here we provide an additional computer simulation in order to compare the theoretical performance with simulation results in the case of correlated regressors. In order to do so, we generate uk,i at each node k according to the recursion

#

uk (i) = ˛k uk (i − 1) + zk (i)

2 (1 − ˛2 ) u,k k

where ˛k is the correlation index per node and zk (i) is timerealization of a white, zero-mean Gaussian random sequence with unit variance. The resulting regressors have Toeplitz covariance

Fig. 3. The steady-state MSD versus node for different tap-length, Qk = 10−3 I.

Fig. 5. The steady-state MSD for node k = 1 versus .

Please cite this article in press as: Azarnia G, Tinati MA. Steady-state analysis of the deficient length incremental LMS adaptive networks with noisy links. Int J Electron Commun (AEÜ) (2014), http://dx.doi.org/10.1016/j.aeue.2014.08.007

G Model

ARTICLE IN PRESS

AEUE-51270; No. of Pages 10

G. Azarnia, M.A. Tinati / Int. J. Electron. Commun. (AEÜ) xxx (2014) xxx–xxx

9

Q k = 10−4 I. As it is clear from Fig. 7, the derived expressions do match with simulation results with an acceptable accuracy. 5. Conclusions

Fig. 6. The regressor power profile  2u,k , and correlation index per node ˛k . 2 (˛ )|i| , matrices R u,k with entries rk (i) = u,k k

i = 0, . . ., M − 1. The

2 u,k

and the correlation indexes {˛k } were regressor power profile chosen at random and are depicted in Fig. 6. The other setup parameters are the same as that in the first simulation. The steady-state MSD curves in every node k for two different tap-lengths M = 10 and 12 are shown in Fig. 7 with channel noise covariance matrix

In this paper, we studied the performance of the distributed DILMS algorithm, considering two realistic conditions: the presence of noisy links and the application of deficient length adaptive filter in each node. Such a study is challenging due to the following fact. Considering the deficient length condition in context of adaptive networks with noisy links, a length mismatch appears. Mismatch is between the unknown parameter and its local estimation and also between the noise vectors that are being added to the local estimations during the transmission phase between nodes. Another problem that is addressed in this analysis is that, nodes with one hop distance from each other interact, and therefore, a successful analysis must take into account both the temporal and spatial interconnectedness of the data. These make our analysis more complex. In our analysis, we derived a closed-form expression for the MSD in order to evaluate the steady-state performance of every individual node. The results show that, in the contrary to the ideal case, the steady-state MSD includes two additional terms: one is the noise-induced, and the other arises from the deficient length application that includes all the coefficients of unknown parameter that are omitted in the estimation process. These theoretical predictions agree well with the simulations. Simulation results show that the two realistic conditions: noisy links and deficient length can drastically deteriorate the performance of the distributed adaptive estimation algorithm in term of MSD. Simulation results also show that unlike the ideal link case, the steady-state MSD are not monotonically increasing functions of , and there is an optimum value for . This interesting result is due to the presence of the noisy links. It must be noted that different learning rules such as RLS could also be applied in the context of a distributed network with incremental topology. References

Fig. 7. The steady-state MSD versus node for different tap-length.

[1] Xiao JJ, Ribeiro A, Luo ZQ, Giannakis GB. Distributed compression-estimation using wireless sensor networks. IEEE Signal Process Mag 2006;23(July (4)):27–41. [2] Bertsekas DP. A new class of incremental gradient methods for least squares problems. SIAM J Optim 1997;7(November (4)):913–26. [3] Lopes CG, Sayed AH. Incremental adaptive strategies over distributed networks. IEEE Trans Signal Process 2007;55(August (8)):4064–77. [4] Sayed AH, Lopes CG. Distributed recursive least-squares strategies over adaptive networks. In: Proc. 40th Asilomar Conference on Signals, Systems and Computers. 2006. p. 233–7. [5] Li L, Chambers JA, Lopes CG, Sayed AH. Distributed estimation over an adaptive incremental network based on the affine projection algorithm. IEEE Trans Signal Process 2010;58(January (1)):151–64. [6] Lopes CG, Sayed AH. Diffusion least-mean squares over adaptive networks: formulation and performance analysis. IEEE Trans Signal Process 2008;56(July (7)):3122–36. [7] Cattivelli FS, Sayed AH. Diffusion LMS strategies for distributed estimation. IEEE Trans Signal Process 2010;58(March (3)):1035–48. [8] Li L, Chambers JA. Distributed adaptive estimation based on the APA algorithm over diffusion networks with changing topology. In: Proc. IEEE Workshop Stat. Signal Process. (SSP). 2009. p. 757–60. [9] Cattivelli FS, Lopes CG, Sayed AH. Diffusion recursive least-squares for distributed estimation over adaptive networks. IEEE Trans Signal Process 2008;56(May (5)):1865–77. [10] Takahashi N, Yamada I, Sayed AH. Diffusion least-mean-squares with adaptive combiners: formulation and performance analysis. IEEE Trans Signal Process 2010;58(September (9)):4795–810. [11] Khalili A, Tinati MA, Rastegarnia A. Steady-state analysis of incremental LMS adaptive networks with noisy links. IEEE Trans Signal Process 2011;59(May (5)):2416–21. [12] Khalili A, Tinati MA, Rastegarnia A, Chambers JA. Steady-state analysis of diffusion LMS adaptive networks with noisy links. IEEE Trans Signal Process 2012;60(February (2)):974–9. [13] Bilcu RC, Kuosmanen P, Egiazarian K. A new variable length LMS algorithm: theoretical analysis and implementations. In: Proc. 9th Int. Conf. Electron., Circuits, Syst., vol. 3. 2002. p. 1031–4.

Please cite this article in press as: Azarnia G, Tinati MA. Steady-state analysis of the deficient length incremental LMS adaptive networks with noisy links. Int J Electron Commun (AEÜ) (2014), http://dx.doi.org/10.1016/j.aeue.2014.08.007

G Model AEUE-51270; No. of Pages 10 10

ARTICLE IN PRESS G. Azarnia, M.A. Tinati / Int. J. Electron. Commun. (AEÜ) xxx (2014) xxx–xxx

[14] Bilcu RC, Kuosmanen P, Egiazarian K. On length adaptation for the least mean square adaptive filters. Signal Process – Fract Calc Appl Signals Syst 2006;86(October):3089–94. [15] Gu Y, Tang K, Cui H, Du W. Convergence analysis of a deficientlength LMS filter and optimal-length sequence to model exponential decay impulse response. IEEE Signal Process Letts 2003;10(January (1)): 4–7.

[16] Mayyas K. Performance analysis of the deficient length LMS adaptive algorithm. IEEE Trans Signal Process 2005;53(August (8)):2727–34. [17] Wu M, Qiu X, Chen G. The statistical behavior of phase error for deficient-order secondary path modeling. IEEE Signal Process Lett 2008;15:313–6. [18] Wu M, Yang J, Xu Y, Qui X. Steady-state solution of the deficient length constrained FBLMS algorithm. IEEE Trans Signal Process 2012;60(December (12)):6681–7.

Please cite this article in press as: Azarnia G, Tinati MA. Steady-state analysis of the deficient length incremental LMS adaptive networks with noisy links. Int J Electron Commun (AEÜ) (2014), http://dx.doi.org/10.1016/j.aeue.2014.08.007