Foundations of the functional approach for signal analysis

ARTICLE IN PRESS

Signal Processing 86 (2006) 3796–3825 www.elsevier.com/locate/sigpro

Foundations of the functional approach for signal analysis$ Jacek Les´ kowa, Antonio Napolitanob, a

Zak!ad Ekonometrii, Wyz˙sza Szko!a Biznesu WSB-NLU, Nowy Sa¸cz, Poland Dipartimento per le Tecnologie, Universita` di Napoli ‘‘Parthenope’’, via Medina 40, Napoli I-80133, Italy

b

Received 5 September 2005; received in revised form 7 March 2006; accepted 22 March 2006 Available online 24 May 2006

Abstract In this paper, the mathematical foundation of the functional (or nonstochastic) approach for signal analysis is established. The considered approach is alternative to the classical one that models signals as realizations of stochastic processes. The work follows the fraction-of-time probability approach introduced by Gardner. By applying the concept of relative measure used by Bochner, Bohr, Haviland, Jessen, Wiener, and Wintner and by Kac and Steinhaus, a probabilistic—but nonstochastic—model is built starting from a single function of time (the signal at hand). Therefore, signals are modeled without resorting to an underlying ensemble of realizations, i.e., the stochastic process model. Several existing results are put in a common, rigorous, measure-theory based setup. It is shown that by using the relative measure concept, a distribution function, the expectation operator, and all the familiar probabilistic parameters can be constructed starting from a single function of time. The new concept of joint relative measurability of two or more functions is introduced in this paper which is shown to be necessary for the joint characterization of signals. Moreover, by using such a concept, the independence of signals is defined. The joint relative measurability property is then used to prove the nonstochastic counterparts of several useful theorems for signal analysis. It is shown that the convergence of parameter estimators requires (analytical) assumptions on the single function of time that are much easier to verify than the classical ergodicity assumptions on stochastic processes. As an example of application, nonrelatively measurable functions are shown to be useful in the design of secure information transmission systems. r 2006 Elsevier B.V. All rights reserved. Keywords: Functional approach; Nonstochastic approach; Fraction-of-time probability; Time series analysis; Relative measurability; Joint relative measurability; Asymptotically almost-periodic functions; Pseudo-random functions; Secure information transmission systems

1. Introduction In the classical stochastic approach for signal or time series analysis, signals are modeled as sample paths or realizations of a stochastic process fX ðt; oÞ; t 2 R; o 2 Og, where O is a sample space $ This work was partially supported by the NATO Grant PST.CLG.978068 and the Matching Grant of WSB-NLU. Corresponding author. E-mail addresses: [email protected] (J. Les´ kow), [email protected] (A. Napolitano).

equipped with a s-field F. Probability P is a measure defined on the elements of F and normalized by PðOÞ ¼ 1. If the stochastic process fulfills asymptotic independence assumptions, then ergodic theorems allow us to identify ensemble averages with time averages [1,2]. For example, for a stationary process we have Z Z 1 T lim X ðt; oÞ dt ¼ X ðt; oÞ dPðoÞ ¼ mX T !1 T 0 O for almost all o,

0165-1684/$ - see front matter r 2006 Elsevier B.V. All rights reserved. doi:10.1016/j.sigpro.2006.03.028

ARTICLE IN PRESS J. Les´kow, A. Napolitano / Signal Processing 86 (2006) 3796–3825

that is, the ensemble average mX 9EfX ðtÞg is equal to the time average of almost all sample paths or realizations. Furthermore, the Wold isomorphism exists between the random variable X ðt0 ; oÞ, o 2 O, t0 fixed, of an ergodic stationary stochastic process and a single realization X ðt; o0 Þ, t 2 R for almost every o0 [3]. More generally, in the usual stochastic approach to statistical inference for signals and time series, ergodicity allows us to estimate a statistical function defined in terms of ensemble average, e.g., the autocorrelation function, by a time average of a function of a single sample path xðtÞ over a finite observation interval, for almost all the sample paths of the stochastic process, obtaining better and better approximation as the data-record length approaches infinity. For example, for the estimation of the autocorrelation function of a stationary ergodic process, the asymptotic independence or mixing assumptions [1] on the stochastic process assure that the limits of the finite-time averages of the lag product xðt þ tÞxðtÞ of almost all realizations xðtÞ exist, they assume the same value, and such a common value is coincident with the autocorrelation function. However, in practice it is very difficult to test or verify whether the process fulfills mixing or asymptotic independence assumptions. In fact, for most of the stochastic process models used in practice, the sample space O and for any fixed t the random variable X ðt; oÞ (that is, the function o ! X ðt; oÞ), cannot be specified. In such a context a fundamental question arises: do we have to construct a highly complex stochastic process model fulfilling the mixing assumption just to show a property like the convergence of the sample autocorrelation function for one realization xðtÞ that we have on hand? Would it not be simpler to check certain analytical properties just for xðtÞ to get the desired convergence? In the approach we investigate in this paper, by following the idea in [4], we exploit the concept of relative measurability [5,6] and we only require the existence of limits of finite-time averages for the available data, as the data-record length approaches infinity, without requiring the existence of limits for almost all the realizations of an underlying stochastic process. Moreover, we also do not require that all such limits are the same and are equal to the theoretical statistical function defined in terms of ensemble average. In the sequel, unless ambiguous, the terms ‘‘function of time’’, ‘‘signal’’, or ‘‘single realization’’ will be used

3797

interchangeably. Moreover, by following [4,7–10], and [11], also the term ‘‘time series’’ will be adopted to denote an individual function of time. The functional (or nonstochastic) approach for signal and time series analysis based on infinite-time averages was first introduced by Wintner in [12] and Wiener in [13] with reference to time-invariant statistics of ordinary functions of time. The concept of relative measurability of sets and functions was investigated in the works of Bochner, Bohr, Haviland, Jessen, and Wintner [12,14–23], in the works of Kac and Steinhaus [5,6,24,25], and was also adopted in [26]. The term fraction-of-time probability was first introduced in [27]. Moreover, in [3] an isometric isomorphism (Wold isomorphism) between a stationary stochastic process and the Hilbert space generated by a single sample path was singled out and a rigorous link between the timeaverage based and the stochastic process frameworks in the stationary case was established. Subsequently, the functional approach was developed in [27–34], and then extended to the case of distributions (generalized functions) in [35]. Gardner [8,9] showed how to extend the Wold’s isomorphism from stationary to cyclostationary time series, and then showed how to extend this to almost-cyclostationary time series. For a mathematically rigorous treatment for the cyclostationary case see [36]. Gardner also introduced for the first time the fraction-of-time probability approach to characterize time-variant statistics of almost-cyclostationary time series. The second-order statistics were widely treated in [7,8,37,38] while the higherorder case was considered in [9–11]. In [8,9], Gardner provides two alternative outlines of proofs that his fraction-of-time distribution for almostcyclostationary time series is indeed a valid probability distribution function, and he explains the meaning of temporally independent almost-cyclostationary time series. He also explains that such time series can be optionally modeled as stationary or cyclostationary by utilizing phase-averaging processes. A further development in the fractionof-time probability theory for nonstationary signals was given in [39]. There, a class of nonstationary time series, namely, the generalized almost-cyclostationary time series, has been introduced and characterized. Second- and higher-order statistics of the impulse–response function of linear timevariant systems are defined and characterized in [40]. Within a nonstochastic approach, the linear filtering problem and properties of time averages are

ARTICLE IN PRESS 3798

J. Les´kow, A. Napolitano / Signal Processing 86 (2006) 3796–3825

also addressed in [41–44], and [66]. The problem of prediction of sequences is treated in [45–47], and the problem of quantile prediction is considered in [48]. Results on the temporal correlation function and spectral analysis are reported in [13,49–51]. In the present paper, the foundations of the functional (or nonstochastic) approach for signal or time series analysis is provided within a measuretheory setup. In Section 2 by following [5,21], the relative measure mR of a set of points of the real line is introduced. It corresponds in some sense to a probability P on ðO; F; PÞ. However, there are many basic differences between mR and P. The relative measure of a set is defined as the Lebesgue measure of the set normalized to that of the whole real axis. It is shown that the class of the relatively measurable (RM) sets does not contain all the Lebesguemeasurable sets, and is not closed under union and intersection. Moreover, it is shown that the relative measure turns out to be additive but not s-additive. In contrast, probability P in the stochastic approach is assumed to be s-additive. Such an assumption makes stochastic processes easier to be mathematically managed than single functions of time. This simplification, however, does not have necessarily a counterpart for sample paths of stochastic processes. In Section 3, RM functions are introduced and characterized and several existing results are put in a common, rigorous, measure-theory based setup. A real-valued Lebesgue-measurable function xðtÞ is said RM if the set of points ft 2 R : xðtÞpxg is RM for any x, except at most a countable set of values of x. It is proved that, with the exception of the right continuity, the obtained function of x exhibits all the properties of a distribution function. Then, it is shown that the expectation operator corresponding to such a distribution function is the infinite-time average operator, from which all the probabilistic parameters for the signal xðtÞ can be defined, without resorting to an ensemble of realizations. Since the relative measure is not closed under union and intersection, the class of the RM functions turns out to be not closed under addition and multiplication. Consequently, the set of the RM functions is not a vector space (see also [28,52–54]). Such a result puts in evidence a strong difference between properties of stochastic processes and properties of single functions of time which are at the hand of the experimenter. Thus, such a result constitutes one of the strongest motivations in adopting the approach presented here as an alternative to the classical stochastic approach. In fact, in the classical stochastic process framework,

in dealing with two stochastic processes, say fX ðt; ox Þ; t 2 R; ox 2 Ox g and fY ðt; oy Þ; t 2 R; oy 2 Oy g, the assumption of joint measurability of the sample spaces Ox and Oy is always made in order to introduce a joint measure and hence, a joint distribution function [2,55]. The aim of introducing (joint) distribution functions is to disregard the knowledge of the sample spaces and of the functions ox ! X ðt; ox Þ and oy ! Y ðt; oy Þ which, in general, can never be specified. Thus, in the stochastic approach, the necessary joint measurability assumption cannot be verified, is made by default and it assures, for example, that the sum of two stochastic processes is in turn a stochastic process with well defined statistical functions. In contrast, within the functional approach, in this paper, in Section 4, the new concept of joint relative measurability for two or more functions is introduced. We show that this concept is necessary to jointly characterize two signals fxðtÞ; t 2 Rg and fyðtÞ; t 2 Rg and is an analytical property of the two signals that can be verified. Moreover, it is shown that not all pairs of functions possess such a property. In particular, it is shown that the lack of relative measurability is not a rare property as the lack of Lebesgue measurability [56, pp. 334–336]. In the paper, several very simple examples of non-RM functions are reported. For example, binary pulse-amplitude-modulated (PAM) signals can be nonrelatively measurable for some bit sequences. As a consequence of the non-s-additivity of the relative measure, the expectation operator, unlike in the classical stochastic approach, turns out to be linear, but not s-linear. By using the concept of joint relative measurability, in this paper, the conditional relative measure of sets and the independence of functions or signals are defined in Section 5. Independence is defined in such a way that it can be easily interpreted as the property of ‘‘signals that have no link each other’’. Furthermore, it is shown that such a definition leads to the familiar property of the factorization of the joint distribution function into the product of the marginal ones. This factorization property, in contrast, is the definition of independence in the classical stochastic framework with no easy link with an intuitive concept of independence [2,55]. Examples of RM functions are provided in Section 6. Specifically, the almost-periodic (AP) functions, the asymptotically almost-periodic (AAP) functions, and the pseudo-random functions are considered. Such functions model several signals of interest in application fields such as communications and econometrics. In Section 7 numerical results of two experiments are

ARTICLE IN PRESS J. Les´kow, A. Napolitano / Signal Processing 86 (2006) 3796–3825

reported in order to show the usefulness of the developed theory. The first is an example of a RM function whose second-order lag product is not RM and hence, does not have an autocorrelation function. The second is aimed at showing how the nonrelative measurability property of functions can be exploited to develop a secure information transmission system. In such a system, an unauthorized third party cannot discover the modulation format of the transmission and hence, cannot demodulate the signal. Proofs of Lemmas and Theorems are provided in the Appendix. Finally, observe that the functional approach allows to build a probabilistic model for signals since all the familiar probability parameters and concept can be defined. Such a model, however, is nonstochastic since no ensemble of realizations or sample paths is considered. Furthermore, observe that some properties assumed for the sample spaces of stochastic processes make them more mathematically tractable than single functions of time and hence, in some sense more attractive. As observed above, the most fundamental assumptions made dealing with stochastic processes are the s-additivity of probability P and the joint measurability of sample spaces corresponding to different stochastic processes. These assumptions can never be verified in practice and the resulting powerful properties of stochastic processes do not necessarily have a counterpart for functions of time (that are possibly modeled as sample paths of stochastic processes). We remark that this fact constitutes the motivation of this paper. In fact, statistical properties of a signal need to be proved starting from analytical properties of the same signal.

2. Relative measurability of sets In this section, the relative measure of sets is introduced and properties of such a measure are derived and discussed. Definition 2.1. Given a set A 2 BR , BR being the sfield of the Borel subsets and m the Lebesgue measure on the real line R, the relative measure of A is defined as mR ðAÞ9 lim

T!1

1 mðA \ ½
T=2; T=2Þ T

provided that the limit exists.

(2.1)

3799

That is, the relative measure of a set is the Lebesgue measure of the set normalized to that of the whole real line. Consequently, only Lebesguemeasurable sets with infinite Lebesgue measure can have nonzero relative measure. Moreover, if the limit in (2.1) exists, then for any t0 2 R 1 mðA \ ½t0
T=2; t0 þ T=2Þ. (2.2) T In other words, the limit in (2.2) is independent of t0 . The class of RM sets is denoted by C: mR ðAÞ ¼ lim

T !1

C9fA 2 BR : mR ðAÞ existsg.

(2.3)

Fact 2.1. The class C is not closed under intersection, that is, there exist A; B 2 C such that A \ BeC. Proof. Define the sets  [
1 A9 n; n þ , 3 n2Z B9

[

B1;k [ B2;k ,

(2.4) (2.5)

k2N0

where
 1 B1;k 9
n;
n þ 3 n2f32k þ1;...;32kþ1 g
 1 [ n; n þ 3 [

ð2:6Þ

and B2;k 9


 1 2
n þ ;
n þ 3 3 n2f32kþ1 þ1;...;32kþ2 g
 1 2 [ n þ ;n þ . 3 3 [

ð2:7Þ

Both A and B are RM in the sense of Definition 2.1 and mR ðAÞ ¼ mR ðBÞ ¼ 13. However, mR ðA \ BÞ does not exists since the function mðTÞ ¼ ð1=TÞmððA \ BÞ \ ½
T=2; T=2Þ oscillates between 13 and 0 as T ! 1. & As an easy corollary of Fact 2.1 we get that not all the Lebesgue-measurable sets are RM sets. For example the set A \ B is not RM. In addition, the simple example of Fact 2.1 shows that non-RM sets are not so rare or sophisticated as non-Lebesguemeasurable sets [56, par. 28]. Fact 2.2. Class C is not closed under union, that is, there exist A; B 2 C such that A [ BeC.

ARTICLE IN PRESS 3800

J. Les´kow, A. Napolitano / Signal Processing 86 (2006) 3796–3825

Proof. Consider the RM sets A and B defined in (2.4) and (2.5), respectively. It results that mR ðA [ BÞ does not exists since the function mðTÞ ¼ 1=TmððA [ BÞ \ ½
T=2; T=2Þ oscillates between 13 and 23 as T ! 1. & Fact 2.3. If A, B, and A \ B 2 C, then A [ B 2 C and mR ðA [ BÞ ¼ mR ðAÞ þ mR ðBÞ
mR ðA \ BÞ. Proof. For any finite T it results 1 mððA [ BÞ \ ½
T=2; T=2Þ T 1 ¼ ½mðA \ ½
T=2; T=2Þ þ mðB \ ½
T=2; T=2Þ T
mððA \ BÞ \ ½
T=2; T=2Þ ð2:8Þ due to the properties of the Lebesgue measure. Consequently, in the limit for T ! 1, we have mR ðA [ BÞ ¼ mR ðAÞ þ mR ðBÞ
mR ðA \ BÞ. & From Fact 2.3 we immediately have the following results. Fact 2.4. The measure mR is additive. That is, if A; B 2 C and A \ B ¼ ;, then A [ B 2 C and mR ðA [ BÞ ¼ mR ðAÞ þ mR ðBÞ. Fact 2.5. The measure mR is not s-additive. Proof. Following Kac example S[6, p. 46], if A Pi ¼ ði; i þ 1Þ, i 2 Z, then mR ð i2Z Ai Þ ¼ 1 but & i2Z mR ðAi Þ ¼ 0. From Fact 2.5 it follows that mR is not an outer measure since the measure of a countable union of sets can be bigger than the sum of the measures of the sets of the union. Moreover, the lack of sadditivity does not allow to prove the continuity of mR . In particular, mR is not continuous in þ1. In fact, with reference to the example inSFact 2.5, n we Pn have that for any finite n it results mR ðSni¼
n Ai Þ ¼ i¼
n PmnR ðAi Þ ¼ 0 and hence, limn mR ðS1i¼
n Ai Þ ¼ limn i¼
n mR ðAi Þ ¼ 0. However, mR ð i¼
1 Ai Þ ¼ mR ðRÞ ¼ 1. The lack of s-additivity for the relative measure mR , in contrast with the s-additivity assumed for the probability P in the classical stochastic approach, constitutes one of the motivations of this paper. In fact, due to such a difference between mR and P, results holding for stochastic processes do not necessarily have a counterpart in terms of functions of time representing sample paths of these stochastic processes.

¯ Fact 2.6. If A is a RM set, then A9R
A is RM and ¯ ¼ 1
mR ðAÞ. mR ðAÞ

(2.9)

¯ \ ½
T=2; Proof. For any finite T it results ð1=TÞmðA T=2Þ ¼ 1
ð1=TÞ mðA \ ½
T=2; T=2Þ. In the limit for T ! 1 we have (2.9). & Fact 2.7. The following relationships between the Lebesgue measure and the relative measure of a set A can occur: (1) There exists T M 40 such that for T4T M mðA \ ½
T=2; T=2Þ ¼ mðAÞ (finite and independent of T). Thus mR ðAÞ ¼ 0. (2) mðA \ ½
T=2; T=2Þ ¼ OðT a Þ as T ! 1 with 0oao1. In this case, even if mðAÞ ¼ 1, mR ðAÞ ¼ 0. (3) mðA \ ½
T=2; T=2Þ ¼ mA T þ oðTÞ as T ! 1. In this case, mðAÞ ¼ 1, the relative measure of A exists and is finite, and mR ðAÞ ¼ mA . (4) mðA \ ½
T=2; T=2Þ has no limit as T ! 1. Thus mðAÞ ¼ 1 but the set A is not RM. Where O and o denote the ‘‘big oh’’ and ‘‘small oh’’ Landau symbols, respectively. From Fact 2.7 it immediately follows the following consideration. Fact 2.8. Any set with finite-Lebesgue measure is RM and its relative measure is zero. Thus, in particular, countable sets and the empty set have zero relative measure. Consequently, the Lebesgue measure is not absolutely continuous with respect to the relative measure (that is, mðAÞ ¼ 0 ) mR ðAÞ ¼ 0, but mR ðAÞ ¼ 0RmðAÞ ¼ 0). Fact 2.9. Let A be a RM set and B  A with finiteLebesgue measure. Then, it results mR ðA
BÞ ¼ mR ðAÞ. Proof. 1 mððA
BÞ \ ½
T=2; T=2Þ T 1 ¼ lim ½mðA \ ½
T=2; T=2Þ T!1 T
mðB \ ½
T=2; T=2Þ

mR ðA
BÞ ¼ lim

T!1

¼ mR ðAÞ,

ð2:10Þ

where the second equality holds since B  A and in the last one the fact that mR ðBÞ ¼ 0 (see Fact 2.8) has been accounted for. &

ARTICLE IN PRESS J. Les´kow, A. Napolitano / Signal Processing 86 (2006) 3796–3825

3801

3. Relative measurability of functions

when t 2 ½t0
T=2; t0 þ T=2 can be expressed as

In this section, the concept of relative measurability of functions is introduced. Moreover, by using such a concept, for a single function of time or signal, a distribution function is constructed and it is shown that the infinite-time average is the corresponding expectation operator, from which all the probabilistic parameters can be constructed.

F T ðxÞ9

Definition 3.1. A Lebesgue-measurable function xðtÞ is said to be RM if and only if the set ft 2 R : xðtÞpxg is RM for every x 2 R
X0 , where X0 is at most a countable set of points. Each RM function xðtÞ generates a function F ðxÞ9mR ðft 2 R : xðtÞpxgÞ 1 ¼ lim mðft 2 ½t0
T=2, T!1 T t0 þ T=2 : xðtÞpxgÞ

ð3:1Þ

in all points x where the limit exists.

1 mðft 2 ½t0
T=2; t0 þ T=2 : xðtÞpxgÞ T Z t0 þT=2 1 1fxðtÞpxg dt, ð3:2Þ ¼ T t0
T=2

where 1fxðtÞpxg is the indicator of the set ft 2 R : xðtÞpxg, that is, 1fxðtÞpxg ¼ 1 8t : xðtÞpx and 1fxðtÞpxg ¼ 0 8t : xðtÞ4x. The function F T ðxÞ has all the properties of a distribution function, including the right continuity (due to the continuity of the Lebesgue measure). In the limit for T ! 1, accounting for Definition 3.1, we have F ðxÞ9mR ðft 2 R : xðtÞpxgÞ Z 1 t0 þT=2 ¼ lim 1fxðtÞpxg dt. T !1 T t
T=2 0

ð3:3Þ

The distribution function of Definition 3.1 allows to define all the familiar probabilistic parameters. For example, moments can be defined as Riemann–Stieltjes integrals Z mk 9 xk dF ðxÞ (3.4) R

From Definition 3.1 it follows that for non-RM functions limit (3.1) does not exist in a more than countable set of values of x. For a RM function xðtÞ, in [15] it is shown that the limit (3.1) does not necessarily exist at the discontinuity points x of F, even if xðtÞ is an AP function. The function F ðxÞ, defined for x 2 R
X0 , has values in the interval ½0; 1 and is nondecreasing. Thus, as for every bounded nondecreasing function, the set of discontinuity points is at most countable [21,22]. The function F has all the properties of a distribution function with the exception of the right continuity (in the discontinuity points). The lack of right continuity can be easily inferred by the lack of s-additivity of the underlying measure mR [6, p. 42, 34] (see Lemma 6.2 for an example of F ðxÞ which is not right continuous). In the sequel, the function F ðxÞ defined in (3.1) will be referred to as a distribution function although it is not right continuous. The function F ðxÞ is also referred to as empirical distribution [57]. An interpretation of the function F ðxÞ has been given by Gardner [8,4,38,9] by using the concept of fraction-of-time probability. Let xðtÞ be a Lebesguemeasurable function. The fraction of time that the values of xðtÞ are below a fixed threshold x

provided that the integrals exist (see Corollary 3.1). Note that the lack of right continuity of F does not influence definition (3.4). In fact, for any
40, from (3.1) we obviously have that F ðx

ÞpF ðxÞpF ðx þ
Þ.

(3.5)

The value of F in the discontinuity points, however, does not influence the values of the moments mk which, in the presence of a discontinuity point x0 , depends only on the size of the jump lim
!0þ F ðx0 þ
Þ
F ðx0

Þ and not from the value F ðx0 Þ. Moreover, even if F ðxÞ is not defined in the countable set X0 , we can define it in each point of X0 with any finite value without influencing the value of integral (3.4). Fact 3.1. Let 1A ðtÞ be the indicator of the set A, that, is 1A ðtÞ ¼ 1 if t 2 A and 1A ðtÞ ¼ 0 if teA. The indicator function xðtÞ ¼ 1A ðtÞ is RM if and only if A is a RM set and it results 8 xo0; > < 0; (3.6) F ðxÞ ¼ 1
mR ðAÞ; 0pxo1; > : 1; xX1: From Fact 3.1 it follows that the lack of relative measurability of a function is not a rare property as the lack of Lebesgue measurability. This is a

ARTICLE IN PRESS J. Les´kow, A. Napolitano / Signal Processing 86 (2006) 3796–3825

3802

consequence of the fact that non-RM sets can be easily constructed unlike non-Lebesgue-measurable sets (see comments following Fact 2.1).

Then, for any t0 2 R it results that Z Z 1 t0 þT=2 lim gðxðtÞÞ dt ¼ gðxÞ dF ðxÞ, T!1 T t0
T=2 R

Definition 3.2. Functions of the type xðtÞ ¼ PK a 1 ðtÞ with ak 2 R, K finite, and all Ak being k¼1 k Ak RM and pairwise disjoint sets will be called simple RM functions.

where the first integral is in the Lebesgue sense and the second is in the Riemann– Stieltjes sense.

Theorem 3.1. Every RM function x is a pointwise limit of a sequence of simple RM functions. Proof. See the Appendix.

&

It is worthwhile to observe that even if every RM function x is a pointwise limit of a sequence of simple RM functions, not all sequences of simple RM functions converge to a RM function. Check, for example, the sequence xn ðtÞ ¼ 1A ðtÞ1fjtjpng where A is not a RM set. Fact 3.2. For any RM function xðtÞ and x1 ; x2 2 R it results that mR ðft 2 R : x1 oxðtÞpx2 gÞ ¼ mR ðft 2 R : xðtÞpx2 gÞ
mR ðft 2 R : xðtÞpx1 gÞ ¼ F ðx2 Þ
F ðx1 Þ,

ð3:7Þ

where F ðxÞ is the distribution function defined in (3.1). Proof. For any finite T we have 1 mðft 2 ½
T=2; T=2 : x1 oxðtÞpx2 gÞ T 1 ¼ mðft 2 ½
T=2; T=2 : xðtÞpx2 gÞ T 1
mðft 2 ½
T=2; T=2 : xðtÞpx1 gÞ T from which (3.7) T ! 1. &

follows

in

the

ð3:8Þ limit

for

Theorem 3.2 (Fundamental theorem of expectation). Let xðtÞ be a RM, nonnecessarily bounded function and let gðÞ satisfy the following assumptions: (1) gðÞ is continuous, bounded, and of bounded variation; (2) for any ‘ 2 R, the equation gðxÞ ¼ ‘ admits at most a finite number of solutions for x belonging to any finite interval.

Proof. See the Appendix.

(3.9)

&

The result of Theorem 3.2 is found under different assumptions in [20,34]. Moreover, similar results are found in [17–19] which, however, are based on the absolute additivity property of underlying measure. Hence, they cannot be applied in general. In the special case where the limit in the left-hand side of (3.9) exists for all functions gðÞ belonging to the space of the continuous and real functions and moreover, equality (3.9) holds, then the function xðtÞ is said to admit an asymptotic measure [58]. Corollary 3.1 (Moments of a bounded RM function). If xðtÞ is bounded and RM and gðxÞ ¼ xp , p 2 N, then Z Z 1 t0 þT=2 p lim x ðtÞ dt ¼ xp dF ðxÞ. (3.10) T!1 T t
T=2 R 0 Proof. The proof immediately follows from Theorem 3.2. & As a special case for p ¼ 1 of Corollary 3.1 we obtain the following result of [5] for bounded functions: Z Z 1 t0 þT=2 lim xðtÞ dt ¼ x dF ðxÞ. (3.11) T!1 T t
T=2 R 0 That is, the expectation of the distribution F ðxÞ of (3.1) and (3.3) is the infinite-time average of the function xðtÞ. A necessary and sufficient condition for the relative measurability of a function is not known. However, the following fact provides a necessary condition and the following theorem a sufficient condition. Fact 3.3 (Necessary condition for relative measurability). If xðtÞ is a bounded function, the existence of the time average Z 1 t0 þT=2 p lim x ðtÞ dt; p 2 N T!1 T t
T=2 0 is a necessary condition for the relative measurability of xðtÞ.

ARTICLE IN PRESS J. Les´kow, A. Napolitano / Signal Processing 86 (2006) 3796–3825

Theorem 3.3 (Sufficient condition for relative measurability [12,21]). If xðtÞ is continuous and bounded and the left-hand side of (3.10) exists for any integer p, then xðtÞ is RM, and equality (3.10) holds. Starting from (3.11), the following remark can be made, which shows the possibility to interchange limit and integral operations. Remark 3.1. By taking in (A.8), with gðxÞ ¼ xp (xðtÞ bounded), the limit as D ! 0, it results Z Z 1 t0 þT=2 p p x dF T;t0 ðxÞ ¼ x ðtÞ dt, (3.12) T t0
T=2 R where 1 F T ;t0 ðxÞ9 mðft 2 ½t0
T=2; t0 þ T=2 : xðtÞpxgÞ T (3.13) is the finite-time distribution function of xðtÞ. Taking the limit as T ! 1 in (3.12) results in Z Z 1 t0 þT=2 p lim xp dF T;t0 ðxÞ ¼ lim x ðtÞ dt T!1 R T!1 T t
T=2 0 Z ¼ xp dF ðxÞ, ð3:14Þ R

where the second limit exists and the second equality holds due to (3.11). Theorem 3.4. Eq. (3.10) holds also for unbounded RM functions xðtÞ, under the following further assumptions: p loc

(1) xðtÞ 2 L ðRÞ. (2) There exists a40 such that, for jxjXa, both the distribution function F ðxÞ and the finite-time distribution function F T;t0 ðxÞ defined in (3.13) are absolutely continuous, that is, dF ðxÞ ¼ f ðxÞ dx and dF T;t0 ðxÞ ¼ f T;t0 ðxÞ dx, with f ðxÞ and f T ;t0 ðxÞ bounded. (3) There exist r40, b4a, and a positive function C b ðTÞ, with C b ðTÞ ! 0 as T ! 1, such that, for jxj4b, sup jf T ;t0 ðxÞ
f ðxÞjp t0

C b ðTÞ . jxjpþ1þr

Proof. See the Appendix.

(3.15)

&

Assumption (3) can be interpreted as follows. The finite-time density function f T;t0 ðxÞ, for any t0 , should not be too far from the (limit) density function f ðxÞ. Moreover, independently on t0 ,

3803

f T;t0 ðxÞ approximates f ðxÞ as T ! 1. A sufficient condition such that assumption (3) is satisfied is that for jxj4b, f ðxÞp

K ; jxjpþ1þr

f T;t0 ðxÞp

K T;t0 jxjpþ1þr

(3.16)

with lim K T;t0 ¼ K.

T!1

(3.17)

Results related to Theorem 3.4 can be found in [16]. Theorem 3.5. The class of the RM functions is not closed under addition and multiplication. Proof. Define xðtÞ ¼ 1A ðtÞ and yðtÞ ¼ 1B ðtÞ, where A and B are defined in (2.4) and (2.5), respectively. Then, although xðtÞ and yðtÞ are both RM, the product xðtÞyðtÞ is not. In fact, xðtÞyðtÞ ¼ 1A\B ðtÞ and A \ B is not RM (see Fact 2.1). Similarly, we can show that a sum of RM functions can be non-RM. In order to do that, define xðtÞ ¼
1A ðtÞ and yðtÞ ¼ 1B ðtÞ, where sets A and B come from (2.4) and (2.5), respectively. Observe, that xðtÞþ yðtÞ ¼ 1C ðtÞ, where [ C9 B2;k , (3.18) k2N0

where the set B2;k is defined in (2.7). It is easy to see that the set C is not RM. This implies that the sum of xðtÞ and yðtÞ is not RM. & 4. Joint relative measurability of functions In Section 3, it is shown that the class of the RM functions is not closed under addition and multiplication operations. Consequently, RM functions do not constitute a vector space. On the contrary, the linear combination of two stochastic processes is still a stochastic process, provided that the two sample spaces are assumed to be jointly measurable. Such a result constitutes one of the strongest motivations of this paper since it enlightens a deep difference between properties of stochastic processes and properties of functions. Therefore, the stochastic process model for a single realization at hand should be used carefully, since properties of the stochastic process could not correspond to analogous properties of the function of time at hand. In this section, the concept of joint relative measurability between functions is introduced. It is shown that the sum and product of jointly RM functions is in turn a RM function. Therefore, for

ARTICLE IN PRESS J. Les´kow, A. Napolitano / Signal Processing 86 (2006) 3796–3825

3804

such a function, a probabilistic model based on time averages can be constructed. The joint relative measurability is an analytical property of functions and hence, easier to be verified than the analogous property in the stochastic process framework, that is, the joint measurability of sample spaces. The latter property, in fact, cannot be easily verified in the applications since generally, the sample spaces are not specified.

Corollary 4.1. If yðtÞ is not RM, then yðtÞ is not jointly RM with every RM xðtÞ. That is, for every t, the finite-time distribution function

Definition 4.1. Two Lebesgue-measurable functions xðtÞ and yðtÞ are said to be jointly RM if the limit

does not converge to any function of ðx1 ; x2 Þ as T ! 1 for every RM xðtÞ.

F ðx1 ; x2 Þ9mR ðft 2 R : xðtÞpx1 g

Theorem 4.2. Let xðtÞ and yðtÞ be jointly RM functions. Then the sum xðtÞ þ yðtÞ and the product xðtÞyðtÞ are RM, provided that at least one of the functions is bounded.

\ ft 2 R : yðtÞpx2 gÞ 1 ¼ lim mðft 2 ½
T=2; T=2: T!1 T xðtÞpx1 ; yðtÞpx2 gÞ

ð4:1Þ

As a consequence of Theorem 4.1 we have the following result.

1 mðft 2 ½t0
T=2; t0 þ T=2: T xðt þ tÞpx; yðtÞpx2 gÞ

F T;t0 ;t ðx1 ; x2 Þ9

Proof. See the Appendix.

ð4:3Þ

&

2

exists for all ðx1 ; x2 Þ 2 R
X0 , where X0 is at most a countable set of lines of R2 . The function F has all the properties of a bivariate joint distribution function with the exception of the right continuity in the discontinuity points. Specifically, we have the following facts. Fact 4.1. (i) F ðx1 ;
1Þ ¼ 0. (ii) F ðx1 ; þ1Þ ¼ F x ðx1 Þ, where F x is the distribution function of x in the sense of (3.1). (iii) For any x0 ; x00 ; Z0 ; Z00 2 R it results mR ðft 2 R : x0 oxðtÞpx00 ; Z0 oyðtÞpZ00 gÞ ¼ F ðx00 ; Z00 Þ
F ðx0 ; Z00 Þ
F ðx00 ; Z0 Þ þ F ðx0 ; Z0 Þ

ð4:2Þ

in those points where F exist. Theorem 4.1 (Property (ii) in Fact 4.1). Let xðtÞ and yðtÞ be jointly RM functions. Then both xðtÞ and yðtÞ are RM. Proof. The function F ðx1 ; x2 Þ defined in (4.1), for each fixed x1 , is nondecreasing in x2 and bounded by 1 uniformly with respect to x1 . Thus lim F ðx1 ; x2 Þ

x2 !1

exists and as a function of x1 , is a distribution function for xðtÞ in the sense of Fact 3.1, that is, xðtÞ is RM. The proof of the relative measurability of yðtÞ is analogous. &

In the special case of independent (see Definition 5.4 and Theorem 5.2) and both bounded functions xðtÞ and yðtÞ, Theorem 4.2 reduces to a result in [25]. Let ( xðtÞ if xðtÞX0; þ x ðtÞ9 0 otherwise; ( xðtÞ if xðtÞo0; ð4:4Þ x
ðtÞ9 0 otherwise be the positive and negative part of xðtÞ, respectively. We have the following results. Theorem 4.3. If xðtÞ is RM, then xþ ðtÞ and x
ðtÞ are jointly RM (and hence both are RM). In addition, mR ðft 2 R : xðtÞpxgÞ ( mR ðft 2 R : xþ ðtÞpxgÞ if xX0; ¼ mR ðft 2 R : x
ðtÞpxgÞ if xo0

ð4:5Þ

and jxðtÞj is RM. Proof. See the Appendix.

&

Theorem 4.4. Let xðtÞ and yðtÞ be jointly RM functions. Then, the functions xþ ðtÞ, x
ðtÞ, yþ ðtÞ, and y
ðtÞ are pairwise jointly RM. Proof. See the Appendix.

&

Fact 4.2. If xðtÞ is RM, then the lag product xðtÞxðt þ tÞ is not necessarily RM 8t 2 R.

ARTICLE IN PRESS J. Les´kow, A. Napolitano / Signal Processing 86 (2006) 3796–3825

Proof. Consider the indicator function xðtÞ ¼ 1A ðtÞ, where A is the set [ A9 Ai (4.6) i2Z

with 8 ði; i þ 12Þ; > < 1 Ai 9 ði; i þ 4Þ > : [ði þ 1; i þ 3Þ; 2 4

(1) gðx1 ; . . . xn Þ is continuous, bounded, and of bounded variation; (2) for any ‘ 2 R, the equation gðx1 ; . . . ; xn Þ ¼ ‘ admits at most a finite number of solutions for ðx1 ; . . . ; xn Þ belonging to any finite parallelepiped of Rn .

a2k pjijoa2kþ1 ; (4.7) a2kþ1 pjijoa2kþ2

and a0 ¼ 0;

3805

ak ¼ ak
1 þ bk ; k 2 N; b41.

(4.8)

Both xðtÞ and xðt þ 14Þ are RM since the set A is RM (and mR ðAÞ ¼ 12), but xðtÞxðt þ 14Þ is not. In fact xðtÞxðt þ 14Þ ¼ 1B ðtÞ where the set B is not RM since ð1=TÞmð½
T=2; T=2 \ BÞ oscillates between 0 and 12 as T ! 1. & It is worthwhile to observe that the lag product xðtÞxðt þ tÞ could be RM for some values of t and not RM for others. However, if xðtÞ and its shifted version xðt þ tÞ are jointly RM, then, as a consequence of Theorem 4.2, the lag product xðtÞxðt þ tÞ is RM. Definition 4.2. A finite collection of Lebesguemeasurable functions x1 ðtÞ; . . . ; xn ðtÞ is jointly RM if the limit F ðx1 ; . . . ; xn Þ9mR ðft 2 R : x1 ðtÞpx1 g \    \ ft 2 R : xn ðtÞpxn gÞ 1 ¼ lim mðft 2 ½
T=2; T=2: T!1 T x1 ðtÞpx1 ; . . . ; xn ðtÞpxn gÞ

Then, for any t0 2 R it results that Z 1 t0 þT=2 lim gðx1 ðt þ t1 Þ; . . . , T!1 T t
T=2 0 xn
1 ðt þ tn
1 Þ; xn ðtÞÞ dt Z ¼ gðx1 ; . . . ; xn Þ Rn

 dF ðx1 ; . . . ; xn ; t1 ; . . . ; tn
1 Þ,

where the first integral is in the Lebesgue sense and the second is in the Riemann–Stieltjes sense and F ðx1 ; . . . ; xn
1 ; xn ; t1 ; . . . ; tn
1 Þ 9mR ðft 2 R : x1 ðt þ t1 Þpx1 ; . . . , xn
1 ðt þ tn
1 Þpxn
1 ; xn ðtÞpxn gÞ. Proof. See the Appendix.

1 T!1 T Z ¼

Z

Y

t0
T=2 n

Y

xi ðt þ ti Þxn ðtÞ dt

i¼1

xi dF ðx1 ; . . . ; xn
1 ; xn ; t1 ; . . . ; tn
1 Þ. ð4:12Þ

ð4:9Þ

Theorem 4.5 (Fundamental theorem of expectation (multivariate case)). Let x1 ðt þ t1 Þ; . . . ; xn
1 ðt þ tn
1 Þ; xn ðtÞ not necessarily bounded functions jointly RM for any t1 ; t2 ; . . . ; tn
1 and let gðx1 ; . . . ; xn Þ satisfy the following assumptions:

&

t0 þT=2 n
1

Rn i¼1

The function F has all the properties of a nthorder joint distribution function, except the right continuity property with respect to each of the x1 ; . . . ; xn variables. In particular, the properties of Fact 4.1 can be easily extended to the nth-order case. Furthermore, we have the following result.

ð4:11Þ

Corollary 4.2 (Cross-moments of bounded jointly RM functions). If x1 ðt þ t1 Þ; . . . ; xn
1 ðt þ tn
1 Þ; xn ðtÞ are bounded jointly RM functions for any t1 ; t2 ; . . . ; tn
1 , then lim

exists for all ðx1 ; . . . ; xn Þ 2 Rn
X0 , where X0 is at most a countable set of ðn
1Þ-dimensional hyperplanes of Rn .

ð4:10Þ

In the special case of n ¼ 2, Corollary 4.2 provides a characterization of the cross-correlation function. Specifically, if xðtÞ and yðtÞ are bounded functions and xðt þ tÞ and yðtÞ are jointly RM for any t, then the cross-correlation function of x and y is given by Z 1 t0 þT=2 Rxy ðtÞ9 lim xðt þ tÞyðtÞ dt T!1 T t
T =2 0 Z ¼ x1 x2 dF xy ðx1 ; x2 ; tÞ, ð4:13Þ R2

where F xy ðx1 ; x2 ; tÞ9mR ðft 2 R : xðt þ tÞpx1 ; yðtÞpx2 gÞ. (4.14)

ARTICLE IN PRESS J. Les´kow, A. Napolitano / Signal Processing 86 (2006) 3796–3825

3806

Fact 4.3 (Necessary condition for joint relative measurability of functions). If xðtÞ and yðtÞ are bounded functions, the existence of the crosscorrelation function Z 1 t0 þT=2 Rxy ðtÞ9 lim xðt þ tÞyðtÞ dt T !1 T t
T=2 0 is a necessary condition for the joint relative measurability of xðtÞ and yðtÞ. From (4.13), the following remark can be made, which shows the possibility to interchange the limit and integral operations. Remark 4.1. By taking in (A.50), with gðx; yÞ ¼ xy (xðtÞ and yðtÞ both bounded), the limit as D ! 0, we have Z x1 x2 dF T;t0 ;t ðx1 ; x2 Þ R2

¼

1 T

Z

t0 þT=2

xðt þ tÞyðtÞ dt,

ð4:15Þ

t0
T=2

where F T ;t0 ;t ðx1 ; x2 Þ is the finite-time joint distribution function of xðt þ tÞ and yðtÞ defined in (4.3). By taking in (4.15) the limit as T ! 1 we have Z lim x1 x2 dF T ;t0 ;t ðx1 ; x2 Þ T!1

R2

Z 1 t0 þT=2 ¼ lim xðt þ tÞyðtÞ dt T!1 T t
T=2 0 Z ¼ x1 x2 dF xy ðx1 ; x2 ; tÞ,

ð4:16Þ

R2

where the second limit exists and the second equality holds due to (4.13). Theorem 4.6. Eq. (4.13) holds also for unbounded jointly RM functions xðtÞ and yðtÞ, under the following further assumptions: (1) xðt þ tÞyðtÞ 2 L1loc ðRÞ 8t (a sufficient condition is that xðtÞ and yðtÞ 2 L2loc ðRÞ). (2) There exists a40 such that, for jx1 j and jx2 jXa, both the joint distribution function F xy ðx1 ; x2 Þ and the finite-time joint distribution function F T;t0 ;t ðx1 ; x2 Þ defined in (4.3) are absolutely continuous, that is, dF xy ðx1 ; x2 Þ ¼ f xy ðx1 ; x2 Þ dx1 dx2 and dF T;t0 ;t ðx1 ; x2 Þ ¼ f T ;t0 ;t ðx1 ; x2 Þ dx1 dx2 , with f xy ðx1 ; x2 Þ and f T;t0 ;t ðx1 ; x2 Þ bounded. (3) There exist r40, b4a, and a positive function C b ðTÞ, with C b ðTÞ ! 0 as T ! 1, such that for

jx1 j and jx2 j4b, sup jf T;t0 ;t ðx1 ; x2 Þ
f xy ðx1 ; x2 Þjp t0

C b ðTÞ . jx1 j2þr jx2 j2þr ð4:17Þ

Proof. See the Appendix.

&

Fact 4.4 (Lack of s-linearity of the expectation operator). As a consequence of the lack of sadditivity of the relative measure mR , the expectation operator is linear, but not s-linear. The infinite-time average of the linear combination of a finite number of jointly RM functions of time (with at least one of them bounded) is equal to the linear combination of the time averages. However, this is not always true if we have a countable infinity of functions of time. For example, the periodic function cos2 ðtÞ has infinite-time average to 12. However, it results Pþ1 equal 2 2 that cos ðtÞ ¼ k¼
1 cos ðtÞ1½k;kþ1Þ ðtÞ and the time average of cos2 ðtÞ1½k;kþ1Þ ðtÞ is zero. This result is different form the corresponding one in the stochastic approach where the expectation operator is s-linear, provided that the underlying infinite series of random variables is absolutely convergent [55]. On this subject see also [41]. 5. Conditional relative measurability and independence In this section, the definition of independence between two functions is introduced. Such a condition leads to the familiar result that the joint distribution function factorizes into the product of the two marginal distributions. Moreover, it is shown how an intuitive concept of independence corresponds to such a mathematical property. Definition 5.1. Let A and B be Lebesgue-measurable sets and fBn g be an arbitrary increasing sequence of Lebesgue-measurable subsets of B with 0omðBn Þo1, such that mon ) Bm  Bn , [ Bn ¼ B, lim Bn 9 n

(5.1) (5.2)

n2N

mðBn Þ o1. (5.3) n The conditional relative measure mR ðjBÞ of the set A given B is defined as 0o lim n

mR ðAjBÞ9 lim n

mðA \ Bn Þ , mðBn Þ

(5.4)

ARTICLE IN PRESS J. Les´kow, A. Napolitano / Signal Processing 86 (2006) 3796–3825

where mðÞ is the Lebesgue measure, provided that the limit exists.

3807

Proof. Take Bn ¼ ½
n=2; n=2 \ B. Then mðA \ B \ ½
n=2; n=2Þ n mðB \ ½
n=2; n=2Þ mðA \ B \ ½
n=2; n=2Þ ¼ lim n mð½
n=2; n=2Þ mð½
n=2; n=2Þ  lim n mðB \ ½
n=2; n=2Þ m ðA \ BÞ ¼ R : & mR ðBÞ

mR ðAjBÞ ¼ lim Eqs. (5.1) and (5.2) express the fact that the sequence of sets fBn g is increasing. Eq. (5.3) means that the Lebesgue measure of the set Bn grows linearly with n as n ! 1. Such a condition, makes Definition 5.1 consistent with the definition of relative measure of sets (2.1). Specifically, if B ¼ R, we obtain a definition of relative measurability of sets equivalent to Definition 2.1. A definition of relative measure similar to (5.4) is given in [14,20] by calling fBn g admissible sequences. No condition ((5.1)–(5.3)), however, is given in [14,20] to characterize the sequences fBn g. In particular, conditions (5.1)–(5.3) allow to prove the following result. Theorem 5.1. If the limit in (5.4) exists, then it is independent of the particular choice made for the sequence fBn g. Proof. Let fB0n g and fB00n g be arbitrary increasing sequences of Lebesgue-measurable subsets of B with 0omðB0n Þo1 and 0omðB00n Þo1, such that (5.1)–(5.3) hold and the limits lim n

mðA \ B0n Þ mðB0n Þ

and

lim n

mðA \ B00n Þ mðB00n Þ

ð5:7Þ

Note that in the stochastic process framework, the stochastic counterpart of nR ðjBÞ, that is, PðAjBÞ9PðA \ BÞ=PðBÞ, is taken as the definition of conditional probability. Therefore, in the stochastic process framework, no conditional probability is allowed with respect to events B not belonging to F. On the contrary, in the functional approach considered here, by using the conditional relative measure mR ðjBÞ of Definition 5.1, also nonRM measurable sets B can be considered. This is enlightened by the following example. Example 5.1. Here we construct two sets A and B such that neither of the two is RM, nor A \ B is RM, but mR ðAjBÞ exists. Let B an arbitrary Lebesgue measurable but nonRM set (see, e.g., Fact 2.1) and set A ¼ B mðA \ Bn Þ mðBn Þ mðB \ Bn Þ ¼ 1, ¼ lim n mðBn Þ

mR ðAjBÞ ¼ lim n

exist. Then limn mðA \ B0n Þ=mðB0n Þ mðB00n Þ mðA \ B0n Þ 00 00 ¼ lim 0 lim 00 ¼ 1, n mðB Þ n mðA \ B Þ limn mðA \ Bn Þ=mðBn Þ n n (5.5) where the last equality is due to the continuity of the Lebesgue measure and (5.2), and limn mðB00n Þ=mðB0n Þ is finite due to (5.3). & Definition 5.2. Let A and B be sets such that B and A \ B are RM. The conditional relative measure nR ðjBÞ of the set A given B is defined as nR ðAjBÞ9

mR ðA \ BÞ , mR ðBÞ

(5.6)

where mR ðÞ is the relative measure defined in (2.1). Fact 5.1. Definitions 5.1 and 5.2 of conditional relative measures mR ðjBÞ and nR ðjBÞ coincide when B and A \ B are RM.

ð5:8Þ

since Bn is a subset of B. Note that, if condition (5.3) is removed in Definition 5.1, then for any non-RM set A that is Lebesgue measurable, a conditional relative measure mR ðAjBÞ can be defined by properly choosing the set B and the sequence fBn g. In fact, from any bounded sequence mðA \ ½
n; nÞ=ð2nÞ, a convergent subsequence mðA \ ½
n; n \ Bnk Þ=mð½
n; n \ Bnk Þ can be extracted (Bolzano–Weierstrass theorem [59, pp. 58–59]). In such a case, however, Theorem 5.1 does not necessarily hold. Definition 5.3. Let the sets A and B be such that mR ðAjBÞ exists and A is RM. The sets A and B are said to be independent if mR ðAjBÞ ¼ mR ðAÞ.

(5.9)

Definition 5.4. Let xðtÞ be a RM function and A9 ft 2 R : xðtÞpx1 g. Let yðtÞ be a Lebesgue-measurable

ARTICLE IN PRESS J. Les´kow, A. Napolitano / Signal Processing 86 (2006) 3796–3825

3808

function and B9ft 2 R : yðtÞpx2 g. Assume also that 8ðx1 ; x2 Þ 2 R2
X0 , where X0 is at most a countable set of lines, mR ðAjBÞ exists (see Definition 5.1). The functions xðtÞ and yðtÞ are called independent if (5.9) holds. Theorem 5.2. Assume that xðtÞ and yðtÞ are jointly RM. The functions xðtÞ and yðtÞ are independent in the sense of Definition 5.4, if and only if, 8ðx1 ; x2 Þ 2 R2 except at most a countable set of lines, it results in F xy ðx1 ; x2 Þ ¼ F x ðx1 ÞF y ðx2 Þ,

(5.10)

where F xy ðx1 ; x2 Þ is the joint distribution function of xðtÞ and yðtÞ in the sense of Definition 4.1 and F x ðx1 Þ and F y ðx2 Þ are the distribution functions of xðtÞ and yðtÞ, respectively, in the sense of Definition 3.1.

defined as the factorization of the joint distribution function into the product of the marginal ones [1,2,55]. Finally, observe that in literature results involving independent functions have been derived in [5,6,24,25], where Eq. (5.10) is assumed as definition of independence and consequently, no link with an intuitive concept of independence is established. Example 5.2. Let us consider the functions xðtÞ ¼ cosðl1 tÞ and yðtÞ ¼ cosðl2 tÞ, with l1 and l2 incommensurate. Then xðtÞ and yðtÞ are independent (see [6, p. 52]). 6. Examples 6.1. AP functions

Proof. From the joint relative measurability of xðtÞ and yðtÞ it follows that the set ft 2 R : xðtÞp x1 ; yðtÞpx2 g  ft 2 R : xðtÞpx1 g \ ft 2 R : yðtÞpx2 g is RM. Moreover, due to Theorem 4.1, also the sets  ft 2 R : xðtÞpx1 g and ft 2 R : yðtÞpx2 g are RM. Therefore, Definitions 5.1 and 5.2 of conditional relative measure are equivalent (see Fact 5.1). Thus,

Definition 6.1 (Besicovitch [60, Chapter 1]). A function xðtÞ is said to be AP if 8
40 9‘
40 such that for any interval ðt0 ; t0 þ ‘
Þ 9t
2 ðt0 ; t0 þ ‘
Þ such that

F xy ðx1 ; x2 Þ9mR ðft 2 R : xðtÞpx1 ; yðtÞpx2 gÞ

The quantity t
is said translation number of xðtÞ corresponding to
.

¼ mR ðft 2 R : xðtÞpx1 gjft 2 R : yðtÞpx2 gÞ mR ðft 2 R : yðtÞpx2 gÞ ¼ mR ðft 2 R : xðtÞpx1 gÞ mR ðft 2 R : yðtÞpx2 gÞ 9F x ðx1 ÞF y ðx2 Þ,

ð5:11Þ

where in the third equality (5.9) is used. & From Definitions 5.3 and 5.4, the following intuitive interpretation of independence between two functions or signals follows. If xðtÞ and yðtÞ are independent, A9ft 2 R : xðtÞpx1 g and B9ft 2 R : yðtÞ px2 g, then mR ðAjBÞ ¼ mR ðAÞ. That is, subsets Bn of the set B constructed from yðtÞ have no influence in the normalization of the measure of the set A constructed from xðtÞ in (5.4) and give rise to the same result as that obtained by considering the normalization Bn ¼ ½
n=2; n=2. In other words, the function yðtÞ from which the normalizing sets Bn are constructed, has no influence on the relative measure mR ðAjBÞ. Therefore, according with the intuitive concept of independence, the two functions or signals xðtÞ and yðtÞ have no link each other. Note that such an intuitive interpretation of independence has no counterpart in the stochastic process approach where independence of processes is

jxðt þ t
Þ
xðtÞjo
8t 2 R.

(6.1)

In [60], it is shown that any AP function is bounded and is the uniform limit of a trigonometric polynomial. As observed in Section 3, limit (3.1) for an AP function xðtÞ does not necessarily exist at the discontinuity points of F ðxÞ. Moreover, the function t ! 1fxðtÞpxg is discontinuous in t for any x such that limit (3.1) exists. In the following lemma it is proved that the indicator function 1fxðtÞpxg is a generalized AP function of t in the sense of Besicovitch, namely, a W p -AP function. Definition 6.2 (Besicovitch [60, Chapter 2]). A function zðtÞ is said to be W p -AP if 8
40 9‘
40 such that for any interval ðt0 ; t0 þ ‘
Þ 9t
2 ðt0 ; t0 þ ‘
Þ such that kzðt þ t
Þ
zðtÞkW p  1=p Z 1 T=2 9 lim jzðt þ t
Þ
zðtÞjp dt o
. T!1 T
T=2

ð6:2Þ

In [60, Chapter 2], it is shown that any W p -AP function the limit of a trigonometric polynomial, where the convergence is defined by the W p -norm k  kW p .

ARTICLE IN PRESS J. Les´kow, A. Napolitano / Signal Processing 86 (2006) 3796–3825

Lemma 6.1. Let xðtÞ be an AP function. Then 1fxðtÞpxg is a W p -AP function for any x 2 R such that limit (3.1) (or (3.3)) exists (see also [20]). Proof. See the Appendix.

&

Distribution functions for AP functions in the sense of Bohr were studied in [22] transforming time-averages into averages over x and then applying the moment method of the calculus of probability. Related results are also in [17,20,23]. For the expression of the distribution function of an AP function with linearly independent frequencies in the trigonometric polynomial expansion, see [14,6]. Theorem 6.1. AP functions are RM. Proof. The distribution function F ðxÞ of any RM function xðtÞ can be expressed as the time average of 1fxðtÞpxg (see (3.3)) for all x 2 R
X0 , where X0 is at most countable. If xðtÞ is AP, then the function of t 1fxðtÞpxg is W p -AP for any x 2 R
X0 (see Lemma 6.1). Thus, since the time average exists for all W p AP functions [60, Chapter 2], the time average in (3.3) exists 8x 2 R
X0 . That is, the AP function xðtÞ is RM. & Example 6.1. Consider the function yðtÞ ¼ pxðtÞ sinð2ptÞ, where xðtÞ ¼ 1A ðtÞ, with the set A defined in (4.6). Since the set A is RM (see Fact 4.2) and sinð2ptÞ is periodic, then both the functions xðtÞ and sinð2ptÞ are RM. Moreover, they are bounded. However, if we introduce the two sequences T 0n 9a2nþ1 and T 00n 9a2n , where an is defined in (4.8), we have Z T 0n 1 lim 0 yðtÞ dt n T n 0 n

21 þ 23 þ 25 þ    þ 22nþ1 2 ¼ , 21 þ 22 þ 23 þ    þ 22nþ1 3

Z

T00n

¼ lim

lim n

1 T 00n

ð6:3Þ

yðtÞ dt 0

¼ lim n

21 þ 23 þ 25 þ    þ 22n
1 1 ¼ . 3 21 þ 22 þ 23 þ    þ 22n

ð6:4Þ

Therefore, the time average of the product yðtÞ ¼ pxðtÞ sinð2ptÞ does not exist. Consequently, according to Fact 3.3, the function yðtÞ is not RM. Therefore, as a consequence of Theorem 4.2 and Fact 4.3, the functions xðtÞ and sinð2ptÞ are not jointly RM.

3809

6.2. AAP functions Definition 6.3. Let xap ðtÞ be an AP function and ZðtÞ 2 L1loc ðRÞ such that lim ZðtÞ ¼ 0.

jtj!1

(6.5)

The function xðtÞ9xap ðtÞ þ ZðtÞ

(6.6)

is said to be an AAP function. AAP functions have been treated in [61]. Any function ZðtÞ 2 Lp ðRÞ that has a zero limit in 1 satisfies (6.5). Moreover, from Definitions 6.2 and 6.3, we see that each AAP function is a W p -AP function in the sense of Besicovitch [60, Chapter 2]. Fact 6.1. The AP functions are obtained as a special case of AAP functions when ZðtÞ  0. Fact 6.2. Let x1 ðtÞ and x2 ðtÞ be AAP functions. We have the following: cx1 ðtÞ is AAP for any c 2 R; x1 ðtÞ þ x2 ðtÞ is AAP; x1 ðtÞx2 ðtÞ is AAP; xþ1 ðtÞ and x
1 ðtÞ defined according to (4.4) are AAP; (v) jx1 ðtÞj is AAP; (vi) maxðx1 ðtÞ; x2 ðtÞÞ and minðx1 ðtÞ; x2 ðtÞÞ are AAP.

(i) (ii) (iii) (iv)

Proof. Let xi ðtÞ9xap;i ðtÞ þ Zi ðtÞ with xap;i ðtÞ AP and limjtj!1 Zi ðtÞ ¼ 0, (i 2 f1; 2g). The proof of (i) and (ii) is trivial. As regards the proof of (iii), observe that xap;i ðtÞxap;k ðtÞ is AP and limjtj!1 xap;i ðtÞZk ðtÞ ¼ 0 (i; k 2 f1; 2g). As regards the proof of (iv), note that if xap ðtÞ is AP, then both xþap ðtÞ and x
ap ðtÞ are AP; moreover if limjtj!1 ZðtÞ ¼ 0 then limjtj!1 Zþ ðtÞ ¼ limjtj!1 Z
ðtÞ ¼ 0. The proof of (v) follows from (ii) and (iv) and observing that jx1 ðtÞj ¼ xþ1 ðtÞ
x
1 ðtÞ. As regards (vi), note that maxðx1 ; x2 Þ ¼ ðx1 þ x2 Þ=2 þ jx1
x2 j=2 and minðx1 ; x2 Þ ¼ ðx1 þ x2 Þ= 2
jx1
x2 j=2. & Lemma 6.2. Let ZðtÞ be such that limjtj!1 ZðtÞ ¼ 0. It results that ( 1; x40; mR ðft 2 R : ZðtÞpxgÞ ¼ (6.7) 0; xo0; whereas for x ¼ 0 any value in ½0; 1 is possible. Note that this is an example of distribution F ðxÞ that is not right continuous in the discontinuity points.

ARTICLE IN PRESS 3810

J. Les´kow, A. Napolitano / Signal Processing 86 (2006) 3796–3825

Proof. See the Appendix.

&

Theorem 6.2. For any AAP function xðtÞ9xap ðtÞ þ ZðtÞ as in Definition 6.3

exists 8ðx1 ; x2 Þ 2 R2
X0 , where X0 is an at most countable set of lines of R2 . & 6.3. Pseudo-random functions

mR ðft 2 R : xap ðtÞ þ ZðtÞpxgÞ ¼ mR ðft 2 R : xap ðtÞpxgÞ

ð6:8Þ

Definition 6.5. Let xðtÞ be the function

in the continuity points of the right-hand side. xðtÞ9 cosð2pPð½tÞÞ, Proof. See the Appendix.

&

From Theorem 6.2, it follows that all the other statistics of an AAP function are the same as those of its AP component. That is, the functions xap ðtÞ þ ZðtÞ and xap ðtÞ are equivalent in distribution (in the nonstochastic sense). Corollary 6.1. AAP functions are RM. Proof. It follows from the relative measurability of AP functions and Theorem 6.2. & Definition 6.4. The class A of sets is defined as A9fft 2 R : xðtÞpxg, x is any AAP function; x 2 Rg.

ð6:9Þ

Lemma 6.3. The class A is closed under intersection. That is, if A; B 2 A, then A \ B 2 A. Proof. By definition of class A we have that A 2 A39 an AAP function xA and x1 2 R: A ¼ ft 2 R : xA ðtÞpx1 g, ð6:10Þ B 2 A39 an AAP function xB and x2 2 R: B ¼ ft 2 R : xB ðtÞpx2 g.

ð6:11Þ

Thus, assuming x1 x2 40, it results in A \ B9ft 2 R : xA ðtÞpx1 ; xB ðtÞpx2 g ¼ ft 2 R : x¯ A ðtÞpx2 ; xB ðtÞpx2 g ¼ ft 2 R : maxðx¯ A ðtÞ; xB ðtÞÞpx2 g 2 A,

ð6:12Þ

where x¯ A ðtÞ9xA ðtÞx2 =x1 and (vi) of Fact 6.2 has been accounted for. The proof for x1 x2 o0 or x1 x2 ¼ 0 is similar. & Theorem 6.3. Two AAP functions are jointly RM. Proof. From the relative measurability of AAP functions (Corollary 6.1) and Lemma 6.3, it follows that for every AAP functions xðtÞ and yðtÞ the distribution function F ðx1 ; x2 Þ9mR ðft 2 R : xðtÞpx1 ; yðtÞpx2 gÞ

(6.13)

(6.14)

where ½t denotes the integer part of t (i.e., the biggest integer less than t) and PðtÞ is the ‘th-order polynomial PðtÞ9p‘ t‘ þ p‘
1 t‘
1 þ    þ p1 t þ p0 .

(6.15)

If ‘X2 and at least one of the coefficients p2 ; . . . ; p‘ is irrational, then xðtÞ is not AP and is said to be ‘‘pseudo random’’ [29, p. 374]. A further interesting example of pseudo-random function is xðtÞ9 cosðp½Pð½tÞÞ.

(6.16)

It is an example of realization of a binary PAM stochastic process. That is, it exhibits the same statistical behavior of a pseudo-noise (PN) sequence [37,8]. In the following, some definitions, lemmas, and theorems are provided, in order to prove the relative measurability of function (6.14) (Theorem 6.6). Such a result turns out to be very useful in communications applications since function (6.14) models almost-all realizations of a PAM stochastic process with full duty-cycle rectangular pulse and bit period equal to 1. Definition 6.6. Let fxðnÞgn2Z be a sequence of real numbers. Its distribution function can be defined analogously to the case of continuous-time functions (see (3.3)): mR ðfn 2 Z : xðnÞpxgÞ 1 # of ðn 2 f
N; . . . ; Ng : xðnÞpxÞ 9 lim N!1 2N þ 1 N X 1 ¼ lim 1fxðnÞpxg ð6:17Þ N!1 2N þ 1 n¼
N provided that the limit exists, where 1fxðnÞpxg is the indicator of the set fn 2 Z : xðnÞpxg. Let ½x denote the integer part of a real number x and x9x
½x its fractional part. ¯

ARTICLE IN PRESS J. Les´kow, A. Napolitano / Signal Processing 86 (2006) 3796–3825

3811

Definition 6.7. The function xðtÞ is said to be uniformly distributed modulo1 if and only if 8 > < 0; xo0; ¼ x; 0pxp1; (6.18) mR ðft 2 R : xðtÞpxgÞ ¯ > : 1; x41:

Thus, accounting for Lemma 6.4, also the stepwise function Pð½tÞ, t 2 R, is uniformly distributed modulo 1. &

An analogous definition can be given for sequences.

Proof. See the Appendix.

Note that any bilateral sequence fxðnÞgn2Z can be put in one-to-one correspondence with a monolateral sequence fx0 ðnÞgn2N . The following results characterize monolateral sequences uniformly distributed modulo 1. Theorem 6.4 (Weyl). The sequence fxðnÞgn2N is uniformly distributed modulo 1 if and only if [29, pp. 378–380] Z 1 N 1X lim f ðxðnÞÞ ¼ f ðxÞ dx (6.19) ¯ N!1 N 0 n¼1 for any function f Riemann integrable in ½0; 1. In addition, by choosing in (6.19) a complex exponential function f, one obtains that the sequence fxðnÞgn2N is uniformly distributed modulo 1 if and only if [29, pp. 380–382] N 1X ej2pkxðnÞ ¼ 0 N!1 N n¼1

lim

(6.20)

for any positive integer k. By using Theorem 6.4 the following result can be proved. Theorem 6.5. Let PðtÞ be a polynomial of degree ‘X2 such that its second-order derivative has at least one irrational coefficient. Then, the sequence fPðnÞgn2Z is uniformly distributed modulo 1. Proof. See [29, p. 383].

&

Lemma 6.4. For any RM function gðtÞ it results in mR ðft 2 R : gð½tÞpxgÞ N X 1 ¼ lim 1fgðnÞpxg . N!1 2N þ 1 n¼
N Proof. See the Appendix.

ð6:21Þ

&

Lemma 6.5. Let PðtÞ be a polynomial of degree ‘X2 such that its second-order derivative has at least one irrational coefficient. Then, the stepwise function Pð½tÞ is uniformly distributed modulo 1. Proof. From Theorem 6.5 we have that the sequence fPðnÞgn2Z is uniformly distributed modulo 1.

Theorem 6.6. The function xðtÞ of Definition 6.5 is RM. &

Theorem 6.7. The pseudo-random functions xðtÞ defined in (6.14) and (6.16) have the autocorrelation Z 1 t0 þT=2 xðt þ tÞxðtÞ dt Rxx ðtÞ9 lim T!1 T t
T=2 0 ( ð1
jtjÞ; jtjp1; ¼ ð6:22Þ 0; jtj41: Proof. See [29, pp. 374–376].

&

A further example of construction of a binary pseudo-random stepwise function can be found in [51, pp. 151–154]. The binary sequence is constructed by considering the binary expansion of a real number l 2 ð0; 1Þ. For all l except those belonging to a set with zero Lebesgue measure, the obtained function exhibits the same properties (distribution and autocorrelation functions) of that defined in (6.16). Finally, note that all the pseudorandom functions considered above are stepwise. A continuous pseudo-random function can be obtained R from a stepwise one xðtÞ by considering yðtÞ ¼ R hðtÞxðt
tÞ dt, where hðtÞ 2 L1 ðRÞ [29, pp. 371, 378].

7. Numerical results and applications In this section, the results of two numerical experiments are reported, in order to show the usefulness and effectiveness of the developed theory. In the first experiment, for increasing values of the data-record length, the behavior of an estimate of the autocorrelation function is evaluated. Two RM functions are considered there: one that has RM lag product and the other that does not. In the second experiment, it is shown that functions that are not RM can be exploited in communications to design secure information transmission systems such that an unauthorized third party cannot demodulate the signal since the modulation format cannot be recognized by measuring statistical functions of the transmitted signal.

ARTICLE IN PRESS J. Les´kow, A. Napolitano / Signal Processing 86 (2006) 3796–3825

3812

7.1. Experiment 1 In this experiment, the sample autocorrelation function, i.e., the correlagram, has been evaluated for two signals (or functions). The former is a pseudo-random function x1 ðtÞ with existing autocorrelation, and the latter is a RM function x2 ðtÞ whose lag product, however, is not RM. Let x1 ðtÞ9 cosðp½Pð½t=T p ÞÞ,

(7.1) p ffiffi ffi where PðtÞ ¼ p2 t2 þ p1 t þ p0 , with p2 ¼ 2, p1 ¼ pffiffiffi 3, and p0 ¼ 0, and T p ¼ 10T s , with T s denoting the sampling period. The function x1 ðtÞ is a binary PAM signal with bit period T p . According to Theorem 6.7, the correlogram Z 1 T=2 Rx1 x1 ðt; TÞ9 x1 ðt þ tÞx1 ðtÞ dt (7.2) T
T=2 approaches the autocorrelation function ( ð1
jtj=T p Þ; jtjpT p ; Rx1 x1 ðtÞ ¼ 0; jtj4T p

(7.3)

as the data-record length T approaches infinity. Such a fact is illustrated by Fig. 1, where the correlogram of the pseudo-random function x1 ðtÞ, as a function of t=T s , is reported for increasing values of N ¼ T=T s . The convergence of correlogram (7.2) to the autocorrelation function (7.3) is well known and evident [37]. Note that the same result can be obtained generating the binary PAM signal x1 ðtÞ by a PN generator [37]. Let us now consider the function qA ðtÞ1A ðtÞ þ qeA¯ ðtÞ1A¯ ðtÞ, x2 ðtÞ9e where

qeB ðtÞ9

0.5

0

0 10

1

20

−0.5 −20

qB ðt
kT p Þ;

¯ B ¼ A; A

(7.6)

N=64

−10

0

10

20 N=512

0.5

0

0

−0.5 −20

−0.5 −20

−10

0

1

10

20

0.5

0

0 −10

0

1

10

20

−0.5 −20

0.5

0.5

0

0 0 τ/Ts

10

0

10

20

−0.5 −20

20 N=4096

−10

0

1

N=11585

−10

−10

1

N=1448

0.5

−0.5 −20

þ1 X

1

N=181

0.5

−0.5 −20

(7.5)

¯ A9R
A, and the sequence fak g is that defined in (4.8) with b ¼ 10. Moreover, in (7.4)

1

N=22

0

ft 2 R : a2k T s pjtjoa2kþ1 T s g,

k2N

k¼
1

1

−10

[

A9

0.5 −0.5 −20

(7.4)

10

20

N=32768

−10

0

10

20

τ/Ts

Fig. 1. Correlogram Rx1 x1 ðt; TÞ of the pseudo-random function x1 ðtÞ defined in (7.1) as a function of t=T s for increasing values of N ¼ T=T s .

ARTICLE IN PRESS J. Les´kow, A. Napolitano / Signal Processing 86 (2006) 3796–3825

with T p ¼ 8T s and X qA ðtÞ9 1½0;T s Þ ðt
nT s Þ n2f0;1;2;3g




X

1½0;T s Þ ðt
nT s Þ,

ð7:7Þ

n2f4;5;6;7g

qA¯ ðtÞ9

X

3813

quently, the limit of the correlogram is not convergent, that is, the autocorrelation function does not exist. Such a lack of convergence is illustrated by Fig. 2 where for increasing values of N ¼ T=T s , the correlogram of x2 ðtÞ is shown as a function of t=T s . 7.2. Experiment 2

1½0;T s Þ ðt
nT s Þ

n2f0;2;4;6g




X

1½0;T s Þ ðt
nT s Þ.

ð7:8Þ

n2f1;3;5;7g

The function x2 ðtÞ is created by alternating two different periodic patterns: the pattern qeA ðtÞ for t 2 A and the pattern qeA¯ ðtÞ for teA. Each periodic pattern is obtained by the periodization of a different (aperiodic) pattern, say qA ðtÞ and qA¯ ðtÞ (see (7.7) and (7.8)). Since qA ðtÞ and qA¯ ðtÞ contain the same number of ‘‘1’’ and ‘‘
1’’, it results that x2 ðtÞ turns out to be RM even if the set A is not RM. However, the same reasoning as in Fact 4.2 and the use of the structure of the set A shows that even if x2 ðtÞ is RM, its lag product is not RM. Conse1

1

N=22

0 −1 −20

−10

0

10

20

−10

0

10

20

N=512

0

−10

0

1

10

20

−1 −20

−10

0

1

N=1448

0

10

20

N=4096

0

−10

0

10

20

N=11585

1

−1 −20

−10

0

1

0 −1 −20

−1 −20 1

N=181

0

−1 −20

N=64

0

1

−1 −20

As an application of the results on the relative measurability of functions, let us consider the problem of transmitting an information signal modulated in such a way that it cannot be demodulated by a third unauthorized party. For this purpose, starting from the information signal xðtÞ, a new non-RM signal yðtÞ is constructed so that its statistical functions cannot be measured. In fact, for an unauthorized party, the first step in demodulating a signal is to identify its modulation format (e.g., binary PAM, quadrature amplitude modulated (QAM), direct-sequence spread-spectrum (DS-SS), etc.). This can be realized by automatic signal classification algorithms that are

10

20

N=32768

0

−10

0 τ/Ts

10

20

−1 −20

−10

0

10

20

τ/Ts

Fig. 2. Correlogram Rx2 x2 ðt; TÞ of the signal x2 ðtÞ defined in (7.4) as a function of t=T s for increasing values of N ¼ T=T s .

ARTICLE IN PRESS J. Les´kow, A. Napolitano / Signal Processing 86 (2006) 3796–3825

3814

based on measurements of the second- and higherorder statistical functions of the received signal (see, e.g., [62–64] and references therein). Consequently, if the second- and higher-order lag products of the transmitted signal are non-RM waveforms, all the statistical function estimators do not converge. In such a case, therefore, it is impossible to perform the modulation format classification. Let us consider the signal yðtÞ ¼ xðtÞcðtÞ,

with PA ðtÞ ¼ p2 t2 þ p1 t þ p0 , p2 ¼ 0, and T p ¼ 8T s , and qeA¯ ðtÞ9

cðtÞ9qA ðtÞ1A ðtÞ þ qeA¯ ðtÞ1A¯ ðtÞ,

(7.10)

where A is the set defined in (7.5), qA ðtÞ9 cosðp½PA ð½t=T p ÞÞ

(7.11)

1

qA¯ ðtÞ9

−10

0

10

20




X

1½0;T s Þ ðt
nT s Þ.

ð7:13Þ

n2f0;2;3;6g

Due to the different patterns qA ðtÞ and qeA¯ ðtÞ, it can be shown that the lag product cðt þ tÞcðtÞ is not RM in t. In a noise-free and ideal channel transmission condition, by observing that c2 ðtÞ ¼ 1 8t, the information signal xðtÞ can be recovered from yðtÞ by simply multiplying yðtÞ and cðtÞ: xðtÞ ¼ yðtÞcðtÞ, provided that the spreading signal cðtÞ is known at the receiver. Note that such a decoding technique is the same as that adopted in code-division

N=2048

−1 −20

−10

0

10

20

N=8192

0

−10

0

1

10

20

−1 −20

−10

0

1

N=16384

0

10

20

N=32768

0

−10

0

1

10

20

−1 −20

−10

0

1

N=65536

0 −1 −20

1½0;T s Þ ðt
nT s Þ

n2f1;4;5;7g

1

N=4096

0

−1 −20

X

0

1

−1 −20

(7.12)

with

1

N=1024

0 −1 −20

qA¯ ðt
kT p Þ

k¼
1

(7.9)

where xðtÞ (the information signal) is a pseudorandom function and cðtÞ (the spreading or coding signal) is independent of xðtÞ. The signal xðtÞ is defined inffiffiffi (7.1), with PðtÞ ¼ p2 t2 þ p1 t þ p0 , pffiffiffias x1 ðtÞ p p2 ¼ 7, p1 ¼ 5, p0 ¼ 0, and T p ¼ 64T s . The signal cðtÞ is defined as

þ1 X

pffiffiffi pffiffiffi 2, p1 ¼ 3, p0 ¼

10

20

N=131072

0

−10

0 τ/Ts

10

20

−1 −20

−10

0

10

20

τ/Ts

Fig. 3. Correlogram Ryy ðt; TÞ of the signal yðtÞ as a function of t=T s , for increasing values of N ¼ T=T s .

ARTICLE IN PRESS J. Les´kow, A. Napolitano / Signal Processing 86 (2006) 3796–3825

multiple-access (CDMA) systems, where cðtÞ is the (long) spreading code. The advantage of using the signal cðtÞ defined in (7.10) as a spreading code appears evident in those applications where a secure (not easy to demodulate) information transmission is necessary. In fact, if a third unauthorized party would like to demodulate the signal yðtÞ to get the information signal xðtÞ, it should at first classify the modulation format of the received signal yðtÞ. However, the function yðtÞ exhibits second- and higher-order lag products that are not RM in t. For example, at the second order we have that the correlogram Rcc ðt; TÞ is not convergent to any function Rcc ðtÞ as the datarecord length approaches infinity. Thus, the correlogram Ryy ðt; TÞ does not converge to Rcc ðtÞRxx ðtÞ. Consequently, no automatic modulation classification of yðtÞ can be made. Such a lack of convergence is illustrated in Fig. 3 where the correlogram Ryy ðt; TÞ of yðtÞ as a function of t=T s , for increasing values of N ¼ T=T s is reported. 8. Conclusions In this paper, the mathematical foundations of the functional (or nonstochastic) approach for signal and time series analysis have been established. This approach is based on observing a single function of time and is motivated by many practical examples when one cannot have multiple realization of the same stochastic process (e.g., econometrics, astronomy, climatology, communications). Moreover, it is motivated by the fact that ergodicity assumptions on stochastic processes that allow to identify time averages with ensemble averages cannot be, in general, easily verified. In contrast, in the functional approach, analytic assumptions made on a single function can be verified on the time series at hand. The central concept of this approach is the relative measure. It has been shown that the relative measure plays, in some sense, the same role played by the probability measure in the classical stochastic approach. However, many differences between relative measure and probability measure have been pointed out. In particular, it has been shown that the relative measure is additive but not s-additive. The concept of relative measurability of functions has been considered. It is shown that the lack of s-additivity of the relative measure should be accounted for when one deals with a single function rather than an ensemble of sample paths (the

3815

stochastic process) as it often happens in the practice. Moreover, due to such a strong difference between relative measure and probability measure, results on stochastic processes should not be always exploited to infer analogous results on single functions of time (modeled as sample paths of the considered stochastic processes). Starting from the concept of relative measure, a distribution function and an expectation operator have been defined and the fundamental theorem of expectation (Theorem 3.2) has been proved. The expectation operator is the infinite-time average and turns out to be linear, but not s-linear. Moreover, all the familiar probabilistic parameters of a signal have been defined. It has been shown that the class of the RM functions is not closed under addition and multiplication (Theorem 3.5). Such a result constitutes one of the strongest motivations of this paper since it enlightens a deep difference between properties of stochastic processes and properties of functions. Specifically, the set of the RM functions, unlike the set of all stochastic processes, does not possess the structure of a vector space. Therefore, the stochastic process model for a single realization at hand should be used carefully, since properties of the stochastic process could not correspond to analogous properties of the single function of time at hand. In the paper, the new concept of jointly RM functions (Definition 4.1) has been introduced. This concept is an analytical property of functions, and allows to jointly characterize two or more signals. It has been shown that the sum and the product of jointly RM functions is a RM function (Theorem 4.2). Thus, the fundamental theorem of expectation for the multivariate case (Theorem 4.5) has been proved. By using the concept of joint relative measurability, the new concept of conditional relative measure has been introduced in this paper (Definition 5.1). Moreover, from such a concept, independence of signals has been defined here in such a way that it corresponds to the intuitive concept of ‘‘signals that are not linked each other in any way’’ (Definition 5.4). Moreover, the familiar result that for independent signals the joint distribution function factorizes into the product of the marginal ones has been proved (Theorem 5.2). The AP functions (Section 6.1), the AAP functions (Section 6.2), and the pseudo-random functions (Section 6.3) have been provided as examples of RM functions. Finally, as an application of the introduced

ARTICLE IN PRESS J. Les´kow, A. Napolitano / Signal Processing 86 (2006) 3796–3825

3816

theory, it has been shown that non-RM functions can be exploited to design modulation techniques that cannot be identified by an unauthorized third party (Section 7). Hence, they are suitable to be exploited in secure information transmission systems. The results presented in this paper provide a measure-theoretic setup for a ‘‘stationary’’ model of signals or time series. In such a model statistical functions do not depend on time. Our future work is directed toward ‘‘nonstationary’’ models. In particular, we will investigate the case of statistical functions AP with respect to time introduced by Gardner [8,9]. Moreover, the case in which a weight with respect to the Lebesgue measure is used will be considered. Furthermore, the problem of spectral analysis [65,8,35,13,51] will be addressed. Acknowledgments This paper is dedicated to Prof. William A. Gardner. The authors would like to thank Prof. Dominique Dehay and Dr. Cyprian Wronka for insightful discussions. Appendix A. Proofs

Define now the sequence of simple functions fsn ðtÞgn2N with n2n
1

X k 1A ðtÞ. 2n k;n k¼0

sn ðtÞ9

(A.4)

For each fixed t there exists N such that jxðtÞjoN. Take now n4N and choose such k that t 2 Ak;n . For such t we have that k=2n oxðtÞpðk þ 1Þ=2n and sn ðtÞ ¼ k=2n . Therefore, in the arbitrary point t we obtain that jxðtÞ
sn ðtÞjo1=2n for each n4N. Let us consider now a possibly unbounded RM function xðtÞ. Define Ak;n as in (A.1) and let C n ¼ ft 2 R : xðtÞ4ng and Dn ¼ ft 2 R : xðtÞo
ng. Define also the function n2n
1

sn ðtÞ9

X

k 1A ðtÞ þ n1C n ðtÞ
n1Dn ðtÞ. 2n k;n k¼
n2n þ1

(A.5)

If t is such that jxðtÞjo þ 1, then apply the previous considerations. If xðtÞ ¼ þ1, then sn ðtÞ ¼ n1C n ðtÞ so that sn ðtÞ ! þ1 ¼ xðtÞ as n ! 1. If xðtÞ ¼
1, then sn ðtÞ ¼
n1Dn ðtÞ so that sn ðtÞ !
1 ¼ xðtÞ as n ! 1. & Proof of Theorem 3.2. Define the sequence f‘k gk2Z , such that [ ð‘k ; ‘kþ1  ¼ R (A.6) k2Z

Proof of Theorem 3.1. Assume first that xðtÞ is nonnegative and bounded. Let b ¼ supt2R xðtÞ and for any n4b define   k kþ1 Ak;n 9 u 2 R : n oxðuÞp n , 2 2 k ¼ 0; 1; . . . ; n2n
1.

ðA:1Þ

The image set of the function xðtÞ, say IðxÞ, has the following property: IðxÞ 

n
1
n2[ k¼0

 k kþ1 ; . 2n 2 n

(A.2)

The set Ak;n is RM. In fact, 1 mðAk;n \ ½
T=2; T=2Þ T 1 ¼ mðfu 2 ½
T=2; T=2 : k=2n oxðuÞpðk þ 1Þ=2n gÞ T 1 ¼ mðfu 2 ½
T=2; T=2 : xðuÞpðk þ 1Þ=2n gÞ T 1 ðA:3Þ
mðfu 2 ½
T=2; T=2 : xðuÞpk=2n gÞ. T

and let D9 sup j‘kþ1
‘k j.

(A.7)

k2Z

Due to the boundedness of gðÞ, there exists K such that, for jkj4K, mðft 2 R : ‘k ogðxðtÞÞp‘kþ1 gÞ ¼ 0. We have the following inequalities: K X

‘k

k¼
K

1 mðft 2 ½t0
T=2; t0 þ T=2: T

‘k ogðxðtÞÞp‘kþ1 gÞ Z 1 t0 þT=2 p gðxðtÞÞ dt T t0
T=2 p

K X k¼
K

‘kþ1

1 mðft 2 ½t0
T=2; t0 þ T=2: T

‘k ogðxðtÞÞp‘kþ1 gÞ.

ðA:8Þ

Since gðÞ is continuous and xðtÞ is Lebesgue measurable, then gðxðtÞÞ is Lebesgue measurable [67]. Thus, due to the boundedness and Lebesgue measurability of gðxðtÞÞ, the Lebesgue integral above exists.

ARTICLE IN PRESS J. Les´kow, A. Napolitano / Signal Processing 86 (2006) 3796–3825

The function gðxÞ is of bounded variation. In addition, accounting for assumption (2), it does not oscillate infinitely rapidly in the neighborhood of any x. Thus, for D sufficiently small we have that the inequality ‘k ogðxÞp‘kþ1 is satisfied in intervals ðx0k;i ; x00k;i Þ i ¼ 1; . . . ; IðkÞ (closed on the left or on the right) in which the function gðxÞ is either nondecreasing or nonincreasing. In those intervals ðx0k;i ; x00k;i Þ where g is nondecreasing we have gðx0k;i Þ ¼ ‘k and gðx00k;i Þ ¼ ‘kþ1 and in those intervals ðx0k;i ; x00k;i Þ where g is nonincreasing we have gðx0k;i Þ ¼ ‘kþ1 and gðx00k;i Þ ¼ ‘k . The sets ðx0k;i ; x00k;i Þ are disjoint (they overlap at most at the end points, that is, in a set of zero Lebesgue measure) and the Lebesgue measure m is additive. Thus, 1 mðft 2 ½t0
T=2; t0 þ T=2 : ‘k ogðxðtÞÞp‘kþ1 gÞ T IðkÞ [ ¼m ft 2 ½t0
T=2; t0 þ T=2 : i¼1

! 00 k;i

IðkÞ K X X k¼
K

¼

x oxðtÞpx00k;i gÞ.

ðA:9Þ

‘k

k¼
K 0 k;i

IðkÞ X 1 i¼1

T

T!1

p

K X

k¼
K

IðkÞ X 1

k¼
K 0 k;i

i¼1

T

x oxðtÞpx00k;i gÞ

ðA:10Þ

or, equivalently, IðkÞ K X X

0

gðx¯ k;i Þ

k¼
K i¼1 0 k;i

1 mðft 2 ½t0
T=2; t0 þ T=2: T

x oxðtÞpx00k;i gÞ Z 1 t0 þT=2 p gðxðtÞÞ dt T t0
T=2 p

IðkÞ K X X k¼
K 0 k;i

i¼1

00

gðx¯ k;i Þ

1 mðft 2 ½t0
T=2; t0 þ T=2: T

x oxðtÞpx00k;i gÞ,

ðA:11Þ

t0 þT=2

gðxðtÞÞ dt t0
T=2

Z

t0 þT=2

gðxðtÞÞ dt t0
T=2 00

gðx¯ k;i Þ

i¼1

ðA:12Þ

which can be written as (see Fact 3.2) 0

gðx¯ k;i Þ½F ðx00k;i Þ
F ðx0k;i Þ

i¼1

p lim sup

1 T

IðkÞ K X X k¼
K

mðft 2 ½t0
T=2; t0 þ T=2:

Z

mR ðft 2 R : x0k;i oxðtÞpx00k;i gÞ,

p

mðft 2 ½t0
T=2; t0 þ T=2:

‘kþ1

1 T

IðkÞ K X X

T!1

x oxðtÞpx00k;i gÞ Z 1 t0 þT=2 gðxðtÞÞ dt p T t0
T=2 p

i¼1

p lim sup

Consequently, (A.8) can be written as K X

00

0

1 p lim inf T!1 T

mðft 2 ½t0
T=2; t0 þ T=2:

i¼1 0 k;i

0

gðx¯ k;i ÞmR ðft 2 R : x0k;i oxðtÞpx00k;i gÞ

1 p lim inf T!1 T

k¼
K IðkÞ X

00

where gðx¯ k;i Þ9‘k and gðx¯ k;i Þ9‘kþ1 (x¯ k;i and x¯ k;i are either x0k;i or x00k;i ). Due the boundedness of g (K finite) and since IðkÞ is finite (assumption (2)) we can take the limits in T in the inequalities in (A.11) to obtain

IðkÞ K X X

x oxðtÞpx g 0 k;i

0

3817

Z

t0 þT=2

gðxðtÞÞ dt t0
T=2

Z

t0 þT=2

gðxðtÞÞ dt t0
T=2 00

gðx¯ k;i Þ½F ðx00k;i Þ
F ðx0k;i Þ,

ðA:13Þ

i¼1

where F is the distribution-type function defined in (3.1). In the limit for D ! 0, both the right and lefthand sides of (A.12) converge to the Riemann– R Stieltjes integral R gðxÞ dF ðxÞ which exists since gðxÞ is bounded and continuous and F ðxÞ is increasing. This proves the theorem, including the existence of the limit in the left-hand side of (3.9) which turns Rout to be independent of t0 . Finally, note that gðxÞ dF ðxÞ cannot be defined as a Lebesgue– R Stieltjes integral since of the lack of s-additivity of mR . & Proof of Theorem 3.4. Let x¯ a ðtÞ be the bounded function ( xðtÞ; t : jxðtÞjpa; x¯ a ðtÞ9 (A.14) 0; t : jxðtÞj4a with a40.

ARTICLE IN PRESS J. Les´kow, A. Napolitano / Signal Processing 86 (2006) 3796–3825

3818

For any finite T it results that Z 1 t0 þT=2 p x¯ ðtÞ dt T t0
T=2 a Z ¼ xp dF¯ T ;t0 ðx; aÞ ZR a Z ¼ xp dF T;t0 ðxÞ þ 0 dF T;t0 ðxÞ,
a

ðA:15Þ

jxj4a

where F¯ T;t0 ðx; aÞ is the finite-time distribution function of x¯ a ðtÞ defined according to (3.13). In (A.15), the first equality is a consequence of Remark 3.1 and the second equality is due to (A.14). Moreover, it results that Z Z 1 t0 þT=2 p 1 t0 þT =2 p lim x¯ a ðtÞ dt ¼ x ðtÞ dt. (A.16) a!1 T t
T =2 T t0
T =2 0 In fact, lim x¯ a ðtÞ ¼ xðtÞ

(A.17)

a!1

pointwise and the interchange of limit and integral operations in (A.16) is allowed by the dominated convergence theorem since jx¯ a ðtÞjpjxðtÞj

8a 2 Rþ

(A.18)

finite due to assumption (1). Thus, accounting for (A.16), in the limit for a ! 1 from (A.15) it follows that Z Z 1 t0 þT=2 p x ðtÞ dt ¼ xp dF T;t0 ðxÞ. (A.19) T t0
T=2 R Then, in the limit for T ! 1 we have Z 1 t0 þT=2 p lim x ðtÞ dt T!1 T t
T=2 0 Z Z ¼ lim xp dF T;t0 ðxÞ ¼ xp dF ðxÞ, R


b

ðA:22Þ

jxj4b

where F¯ ðx; bÞ is the distribution function of x¯ b ðtÞ. Moreover, as regards the second term in the righthand side of (A.21) we have Z  Z   p p  x dF T;t0 ðxÞ
x dF ðxÞ  jxj4b jxj4b  Z   p  ¼ x ðf T;t0 ðxÞ
f ðxÞÞ dx jxj4b Z p jxjp jf T;t0 ðxÞ
f ðxÞj dx jxj4b Z 1 pC b ðTÞ dx, ðA:23Þ 1þr jxj jxj4b where in the first equality assumption (2) is used and in the last inequality, assumption (3) is accounted for. The integral is finite and C b ðTÞ ! 0 as T ! 1. &

with the integral Z 1 t0 þT=2 jxðtÞjp dt T t0
T=2

T!1

xðtÞ bounded as in Remark 3.1. In fact, Z b xp dF T;t0 ðxÞ lim T!1
b Z b xp dF¯ T;t0 ðx; bÞ ¼ lim T!1
b Z b xp dF¯ ðx; bÞ ¼
b Z Z b p x dF ðxÞ þ 0 dF ðxÞ, ¼

Proof of Theorem 4.2. We will prove first that the sum of two jointly RM functions is RM. That is, we will prove that mR ft 2 R : xðtÞ þ yðtÞpxg exists for every x 2 R except at most a countable set of values. Assume that xðtÞ is bounded (and yðtÞ is possibly unbounded). Define the sequence ‘k , k 2 Z, such that [ ð‘k ; ‘kþ1  ¼ R (A.24) k2Z

with ðA:20Þ

R

where proving the second equality proves the theorem. Let b4a as in assumption (3). It results Z Z b xp dF T ;t0 ðxÞ ¼ xp dF T;t0 ðxÞ
b R Z xp dF T;t0 ðxÞ. ðA:21Þ þ

D9 sup j‘kþ1
‘k j.

(A.25)

k2Z

For each k 2 Z we have ft 2 R : ‘k oxðtÞp‘kþ1 ; yðtÞpx
‘kþ1 g  ft 2 R : xðtÞp‘kþ1 ; yðtÞpx
‘kþ1 g  ft 2 R : xðtÞ þ yðtÞpx; yðtÞpx
‘kþ1 g  ft 2 R : xðtÞ þ yðtÞpxg

ðA:26Þ

and

jxj4b

As regards the first term in the right-hand side of (A.21) we have that it is equivalent to the case of

ft 2 R : ‘k oxðtÞp‘kþ1 ; yðtÞpx
xðtÞg  ft 2 R : ‘k oxðtÞp‘kþ1 ; yðtÞpx
‘k g.

ðA:27Þ

ARTICLE IN PRESS J. Les´kow, A. Napolitano / Signal Processing 86 (2006) 3796–3825

Relationships (A.26) and (A.27) imply [ ft 2 R : ‘k oxðtÞp‘kþ1 ; yðtÞpx
‘kþ1 g

where the integrals are in the Riemann–Stieltjes sense. To prove the relative measurability of the product, observe first that

k2Z

 ft 2 R : xðtÞ þ yðtÞpxg [  ft 2 R : ‘k oxðtÞp‘kþ1 ; xðtÞ þ yðtÞpxg

xðtÞyðtÞ ¼ 14 ½xðtÞ þ yðtÞ2
14 ½xðtÞ
yðtÞ2 .

[

Moreover, z ðtÞ is RM when zðtÞ is RM. In fact we have that

ft 2 R : ‘k oxðtÞp‘kþ1 ; yðtÞpx
‘k g.

k2Z

ðA:28Þ Consequently, mR

[

(A.33)

2

k2Z



3819

! ft 2 R : ‘k oxðtÞp‘kþ1 ; yðtÞpx
‘kþ1 g

ft 2 R : z2 ðtÞpxg 8 ;; > > < ¼ ft 2 R : > > :
pffiffixffipzðtÞppffiffixffig;

xo0; ðA:34Þ otherwise:

k2Z

Therefore, due to the joint relative measurability of x and y and accounting for (A.33), the product xðtÞyðtÞ is RM. &

pmR ðft 2 R : xðtÞ þ yðtÞpxgÞ pmR

[

ft 2 R : ‘k oxðtÞ

k2Z

Proof of Theorem 4.3. For xX0 we have

!

p‘kþ1 ; yðtÞpx
‘k g .

ðA:29Þ

Recall that the relative measure mR is not s-additive. Therefore, it is not always true that the relative measure of the union of countably many disjoint sets is the sum of the relative measures of each of the sets (see Fact 2.5). However, since xðtÞ is bounded, then the unions in (A.29) are taken over a finite set of indices K. This yields that X mR ðft 2 R : ‘k oxðtÞp‘kþ1 ; yðtÞpx
‘kþ1 gÞ k2K

pmR ðft 2 R : xðtÞ þ yðtÞpxgÞ X mR ðft 2 R : ‘k oxðtÞp‘kþ1 , p

ft 2 R : x
ðtÞpxg ¼ R

(A.36)

and for xo0 we have ft 2 R : xþ ðtÞpxg ¼ ;,

(A.37)

ft 2 R : x
ðtÞpxg ¼ ft 2 R : xðtÞpxg.

(A.38)

Therefore, for x1 X0 and x2 X0 it results ft 2 R : xþ ðtÞpx1 g \ ft 2 R : x
ðtÞpx2 g ¼ ft 2 R : xðtÞpx1 g \ R ðA:39Þ

for x1 X0 and x2 o0 ðA:30Þ

Applying now (i) and (iii) of Fact 4.1 we get X ½F ð‘kþ1 ; x
‘kþ1 Þ
F ð‘k ; x
‘kþ1 Þ

ft 2 R : xþ ðtÞpx1 g \ ft 2 R : x
ðtÞpx2 g ¼ ft 2 R : xðtÞpx1 g \ ft 2 R : xðtÞpx2 g ¼ ft 2 R : xðtÞpx2 g

ðA:40Þ

and for x1 o0 and any x2

k2K

pmR ðft 2 R : xðtÞ þ yðtÞpxgÞ X p ½F ð‘kþ1 ; x
‘k Þ
F ð‘k ; x
‘k Þ

(A.35)

¼ ft 2 R : xðtÞpx1 g

k2K

yðtÞpx
‘k gÞ.

ft 2 R : xþ ðtÞpxg ¼ ft 2 R : xðtÞpxg,

ðA:31Þ

k2K

for every x 2 R except at most a countable set of values. For D ! 0 we have Z dF ðv; x
vÞ

ft 2 R : xþ ðtÞpx1 g \ ft 2 R : x
ðtÞpx2 g ¼ ;

ðA:41Þ

with the sets in the right-hand sides of (A.39)–(A.41) RM due to the relative measurability of xðtÞ and Fact 2.8. Thus,

R

pmR ðft 2 R : xðtÞ þ yðtÞpxgÞ Z p dF ðv; x
vÞ, R

ðA:32Þ

mR ðft 2 R : xþ ðtÞpx1 ; x
ðtÞpx2 gÞ ¼ mR ðft 2 R : xþ ðtÞpx1 g \ ft 2 R : x
ðtÞpx2 gÞ

ðA:42Þ

ARTICLE IN PRESS J. Les´kow, A. Napolitano / Signal Processing 86 (2006) 3796–3825

3820

exists for all ðx1 ; x2 Þ 2 R except at most a countable set of lines, that is, xþ ðtÞ and x
ðtÞ are jointly RM. Then, xþ ðtÞ and x
ðtÞ are RM due to Theorem 4.1. Finally, note that jxðtÞj ¼ xþ ðtÞ
x
ðtÞ. Thus, the relative measurability of jxðtÞj easily follows from Theorems 4.2 and the joint relative measurability of xþ ðtÞ and x
ðtÞ. & Proof of Theorem 4.4. Reasoning as in Proof of Theorem 4.3, we have the following facts. For x1 X0 and x2 X0 it results þ

We have the following inequalities: K X

‘k

k¼
K

‘k ogðxðtÞ; yðtÞÞp‘kþ1 gÞ Z 1 t0 þT=2 p gðxðtÞ; yðtÞÞ dt T t0
T=2 p

ðA:43Þ

Hence, mR ðft 2 R : xþ ðtÞpx1 ; y
ðtÞpx2 gÞ ðA:44Þ

For x1 X0 and x2 o0 it results

1 mðft 2 ½t0
T=2; t0 þ T=2: T ðA:50Þ

Due to assumption (2) and the boundedness and finite variation of g, there exist intervals ðx0k;i ; x00k;i Þ, i ¼ 1; . . . ; IðkÞ and ðZ0k;j ; Z00k;j Þ, j ¼ 1; . . . ; JðkÞ, with IðkÞ and JðkÞ finite and possibly different, such that ft 2 ½t0
T=2; t0 þ T=2 : ‘k ogðxðtÞ; yðtÞÞp‘kþ1 g

ft 2 R : xþ ðtÞpx1 ; y
ðtÞpx2 g ¼ ft 2 R : xðtÞpx1 ; yðtÞpx2 g.

‘kþ1

‘k ogðxðtÞ; yðtÞÞp‘kþ1 gÞ.

ft 2 R : x ðtÞpx1 ; y ðtÞpx2 g

¼ mR ðft 2 R : xðtÞpx1 gÞ.

K X k¼
K




¼ ft 2 R : xðtÞpx1 g.

1 mðft 2 ½t0
T=2; t0 þ T=2: T

¼

ðA:45Þ

IðkÞ [ JðkÞ [ ft 2 ½t0
T=2; t0 þ T=2: i¼1 j¼1

Hence,

x0k;i oxðtÞpx00k;i ; Z0k;j oyðtÞpZ00k;j g, þ

mR ðft 2 R : x ðtÞpx1 ; y ðtÞpx2 gÞ ¼ mR ðft 2 R : xðtÞpx1 ; yðtÞpx2 gÞ.

ðA:46Þ

Analogously, for all other sign combinations of x1 and x2 , it can be shown that the joint relative measurability of any pair of the signals xþ ðtÞ, x
ðtÞ, yþ ðtÞ, and y
ðtÞ, can be deducted from the joint relative measurability of xðtÞ and yðtÞ. &

where for D sufficiently small, the time intervals in the right-hand side do not overlap (that is, they are disjoint). Thus, due to the finite additivity of the Lebesgue measure m and the continuity of gð; Þ, (A.50) can be written as IðkÞ X JðkÞ K X X k¼
K

[ ð‘k ; ‘kþ1  ¼ R

i¼1

(A.47)

k2Z

p

with

j¼1

IðkÞ X JðkÞ K X X k¼
K

D9 sup j‘kþ1
‘k j.

(A.48)

for jkj4K.

ðA:49Þ

j¼1

ðA:52Þ

0 00 0 where x¯ k;i ; x¯ k;i 2 fx0k;i ; x00k;i g, Z¯ 0k;i ; Z¯ 00k;i 2 fZ0k;i ; Z00k;i g, gðx¯ k;i ; 00 Z¯ 0 Þ ¼ ‘k , and gðx¯ ; Z¯ 00 Þ ¼ ‘kþ1 . k;j

mðft 2 R : ‘k ogðxðtÞ; yðtÞÞp‘kþ1 gÞ

i¼1

00

gðx¯ k;i ; Z¯ 00k;j Þ

1  mðft 2 ½t0
T=2; t0 þ T=2: T x0k;i oxðtÞpx00k;i ; Z0k;j oyðtÞpZ00k;j gÞ,

k2Z

Due to the boundedness of gð; Þ, there exists K such that

0

gðx¯ k;i ; Z¯ 0k;j Þ

1 mðft 2 ½t0
T=2; t0 þ T=2: T x0k;i oxðtÞpx00k;i ; Z0k;j oyðtÞpZ00k;j gÞ Z 1 t0 þT=2 p gðxðtÞ; yðtÞÞ dt T t0
T=2

Proof of Theorem 4.5. Let us consider the case n ¼ 2. The extension to higher values is straightforward. Define the sequence ‘k , with k 2 Z, such that

¼0

ðA:51Þ




k;i

k;j

Due to the boundedness of g (K finite) and since IðkÞ and JðkÞ are finite, we can take the limits in T in

ARTICLE IN PRESS J. Les´kow, A. Napolitano / Signal Processing 86 (2006) 3796–3825

For any finite T it results that Z 1 t0 þT =2 x¯ a ðt þ tÞ¯ya ðtÞ dt T t0
T =2 Z ¼ x1 x2 dF¯ T ;t0 ;t ðx1 ; x2 ; aÞ R2 Z ¼ x1 x2 dF T;t0 ;t ðx1 ; x2 Þ ½
a;a2 Z dF T;t0 ;t ðx1 ; x2 Þ, þ0

the inequalities in (A.52) to obtain IðkÞ X JðkÞ K X X k¼
K

i¼1

0

gðx¯ k;i ; Z¯ 0k;j ÞmR ðft 2 R:

j¼1

x0k;i oxðtÞpx00k;i ; Z0k;j oyðtÞpZ00k;j gÞ Z 1 t0 þT=2 gðxðtÞ; yðtÞÞ dt p lim inf T!1 T t
T =2 0 Z 1 t0 þT=2 gðxðtÞ; yðtÞÞ dt p lim sup T!1 T t
T=2 0 p

IðkÞ X JðkÞ K X X k¼
K

i¼1

00

j¼1

ðA:53Þ

which, accounting for (iii) of Fact 4.1, can be written as IðkÞ X JðkÞ K X X k¼
K

i¼1

0

gðx¯ k;i ; Z¯ 0k;j Þ½F xy ðx00k;i ; Z00k;j Þ
F xy ðx0k;i ; Z00k;j Þ

j¼1


F xy ðx00k;i ; Z0k;j Þ þ F xy ðx0k;i ; Z0k;j Þ Z 1 t0 þT=2 gðxðtÞ; yðtÞÞ dt p lim inf T!1 T t
T =2 0 Z 1 t0 þT=2 gðxðtÞ; yðtÞÞ dt p lim sup T!1 T t
T=2 0 p

IðkÞ X JðkÞ K X X k¼
K

i¼1

lim x¯ a ðtÞ ¼ xðtÞ and

a!1

lim y¯ a ðtÞ ¼ yðtÞ

a!1

(A.58)

pointwise and the interchange of limit and integral operations in (A.57) is allowed by the dominated convergence theorem since

00

jx¯ a ðt þ tÞ¯ya ðtÞjpjxðt þ tÞyðtÞj

j¼1

½F xy ðx00k;i ; Z00k;j Þ
F xy ðx0k;i ; Z00k;j Þ ðA:54Þ

In the limit for D ! 0, both the right- and lefthand sides of (A.54)R converge to the Riemann– Stieltjes integral gðx1 ; x2 Þ dF xy ðx1 ; x2 Þ. This R proves the theorem, including the existence of the limit in (4.10) which turns out to be independent of t0 . & Proof of Theorem 4.6. Let x¯ a ðtÞ and y¯ a ðtÞ be the bounded functions ( xðtÞ; t : jxðtÞjpa; x¯ a ðtÞ9 0; t : jxðtÞj4a; ( yðtÞ; t : jyðtÞjpa; y¯ a ðtÞ9 ðA:55Þ 0; t : jyðtÞj4a with a40.

where F¯ T;t0 ;t ðx1 ; x2 ; aÞ is the finite-time joint distribution function of x¯ a ðtÞ and y¯ a ðtÞ defined according to (4.3). In (A.56), the first equality is a consequence of Remark 4.1 and the second equality is due to (A.55). Moreover, it results that Z 1 t0 þT=2 x¯ a ðt þ tÞ¯ya ðtÞ dt lim a!1 T t
T=2 0 Z 1 t0 þT=2 xðt þ tÞyðtÞ dt. ðA:57Þ ¼ T t0
T=2 In fact,

gðx¯ k;i ; Z¯ 00k;j Þ


F xy ðx00k;i ; Z0k;j Þ þ F xy ðx0k;i ; Z0k;j Þ.

ðA:56Þ

R
½
a;a2

gðx¯ k;i ; Z¯ 00k;j ÞmR ðft 2 R:

x0k;i oxðtÞpx00k;i ; Z0k;j oyðtÞpZ00k;j gÞ,

3821

8a 2 Rþ

(A.59)

with the integral Z 1 t0 þT=2 jxðt þ tÞyðtÞj dt T t0
T=2 finite due to assumption (1). Thus, accounting for (A.57), in the limit for a ! 1 from (A.56) it follows that Z 1 t0 þT =2 xðt þ tÞyðtÞ dt T t0
T =2 Z ¼ x1 x2 dF T ;t0 ;t ðx1 ; x2 Þ. ðA:60Þ R2

Then, in the limit for T ! 1 we have Z 1 t0 þT=2 xðt þ tÞyðtÞ dt lim T!1 T t
T=2 0 Z ¼ lim x1 x2 dF T;t0 ;t ðx1 ; x2 Þ T!1 R2 Z ¼ x1 x2 dF xy ðx1 ; x2 Þ, R2

ðA:61Þ

ARTICLE IN PRESS J. Les´kow, A. Napolitano / Signal Processing 86 (2006) 3796–3825

3822

where proving the second equality proves the theorem. Let b4a as in assumption (3). It results in Z x1 x2 dF T;t0 ;t ðx1 ; x2 Þ R2 Z ¼ x1 x2 dF T;t0 ;t ðx1 ; x2 Þ ½
b;b2 Z þ x1 x2 dF T;t0 ;t ðx1 ; x2 Þ. ðA:62Þ R2
½
b;b2

As regards the first term in the right-hand side of (A.62) we have Z lim x1 x2 dF T;t0 ;t ðx1 ; x2 Þ T!1 ½
b;b2 Z ¼ x1 x2 dF xy ðx1 ; x2 Þ, ðA:63Þ ½
b;b2

since it is equivalent to the case of xðtÞ and yðtÞ bounded as in Remark 4.1. Moreover, as regards the second term we have Z   x1 x2 dF T;t0 ;t ðx1 ; x2 Þ  2 R
½
b;b2  Z 
x1 x2 dF xy ðx1 ; x2 Þ R2
½
b;b2 Z  ¼  x1 x2 ðf T;t0 ;t ðx1 ; x2 Þ R2
½
b;b2 
f xy ðx1 ; x2 ÞÞ dx1 dx2  Z p jx1 x2 jjf T;t0 ;t ðx1 ; x2 Þ R2
½
b;b2


f xy ðx1 ; x2 Þj dx1 dx2 Z 1 1 pC b ðTÞ dx1 dx2 , 1þr jx2 j1þr R2
½
b;b2 jx1 j ðA:64Þ where in the first equality assumption (2) is used and in the last inequality assumption (3) is accounted for. The integral is finite and C b ðTÞ ! 0 as T ! 1. & Proof of Lemma 6.1. Since xðtÞ is AP, then, according to Definition 6.1, 8
40 9‘
40 such that for each interval I with length greater than ‘
there exists a number t
in this interval such that sup jxðt þ t
Þ
xðtÞjo
.

(A.65)

8t 2 I we have k1fxðtþt Þpxg
1fxðtÞpxg kW p 4
. That is 1=p Z 1 T=2 p j1fxðtþt Þpxg
1fxðtÞpxg j dt
o lim T!1 T
T=2 
p 1=p Z 1 T=2 p lim sup j1fxðtþt Þpxg
1fxðtÞpxg j dt T!1 T
T=2 t



¼ sup j1fxðtþt Þpxg
1fxðtÞpxg j.

ðA:67Þ

t

Inequality (A.67) means that for some t0 we have that 8‘ 9I with length greater than ‘ such that 8t 2 I either xðt0 þ t Þpxoxðt0 Þ or xðt0 þ t Þ4xX xðt0 Þ. Let be
0 9jxðt0 þ t Þ
xðt0 Þj40.

(A.68)

Thus, if we chose
o
0 we have that (A.65) is not satisfied. So the function xðtÞ cannot be AP which contradicts the assumption. & Proof of Lemma 6.2. Let ZðtÞ9Zþ ðtÞ þ Z
ðtÞ. It obviously results that limjtj!1 Zþ ðtÞ ¼ 0, that is, 8
40 9T
40 : jtj4T
) Zþ ðtÞo
. Let x40 be such that 0o
ox. It results in mR ðft 2 R : Zþ ðtÞpxgÞ ¼ mR ðft 2 R; jtjpT
: Zþ ðtÞpxgÞ þ mR ðft 2 R; jtj4T
: Zþ ðtÞpxgÞ,

ðA:69Þ

since mR is additive (see Fact 2.4) and the sets ft 2 R; jtjpT
g and ft 2 R; jtj4T
g are disjoint. Moreover, the set ft 2 R; jtjpT
: Zþ ðtÞpxg has finite (less than 2T
) Lebesgue measure and hence, zero relative measure (see Fact 2.8). Thus, from (A.69) it follows that mR ðft 2 R : Zþ ðtÞpxgÞ ¼ mR ðft 2 R; jtj4T
: Zþ ðtÞpxgÞ ¼ mR ðft 2 R; jtj4T
: Zþ ðtÞo
gÞ,

ðA:70Þ

where the second equality holds since
ox and 8t:jtj4T
it results Zþ ðtÞo
. Let IðT; t0 ; T
Þ9 ½t0
T=2; t0 þ T=2
½
T
; T
. It results that

t

We will prove that for each x such that limit (3.1) exists, the function yðtÞ91fxðtÞpxg is W p -AP. Suppose that this is not true and let x 2 R
X0 . In such a case 9
40 such that 8‘ we can find at least one interval I with length greater than ‘ such that

(A.66)

mR ðft 2 R; jtj4T
: Zþ ðtÞo
gÞ 1 ¼ lim mðft 2 IðT; t0 ; T
Þ : Zþ ðtÞo
gÞ T!1 T mðIðT; t0 ; T
ÞÞ ¼ lim T!1 T

ARTICLE IN PRESS J. Les´kow, A. Napolitano / Signal Processing 86 (2006) 3796–3825

1 mðIðT; t0 ; T
ÞÞ mðft 2 IðT; t0 ; T
Þ : Zþ ðtÞo
gÞ

 lim

T!1

¼ 1.

p

1 mðft 2 ½t0
T=2; t0 þ T=2, T jtj4T
: xap ðtÞpx
ZðtÞgÞ

p

1 mðft 2 ½t0
T=2; t0 þ T=2, T

ðA:71Þ

In fact, the first limit is 1 since for T sufficiently large (i.e., such that t0 þ T=24T
) it results mðIðT; t0 ; T
ÞÞ ¼ ðt0 þ T=2
T
Þ þ ð
T

ðt0
T= 2ÞÞ ¼ T
2T
and the second limit is 1 since Zþ ðtÞo
8t 2 IðT; t0 ; T
Þ. Thus, for any x40 mR ðft 2 R : ZðtÞpxgÞ ¼ mR ðft 2 R : Zþ ðtÞpxgÞ ¼ mR ðft 2 R; jtj4T
: Zþ ðtÞo
gÞ ¼ 1,

ðA:72Þ

where the first equality follows from Theorem 4.3 since x40, the second from (A.70), and the third from (A.71). We have limjtj!1 Z
ðtÞ ¼ 0, that is, 8
40 9T
40 : jtj4T
)
Z
ðtÞo
. Let xo0 be such that 0o
ojxj. With arguments similar to those used in the case x40, it results in mR ðft 2 R : ZðtÞpxgÞ

jtj4T
: xap ðtÞpx þ
gÞ.

3823

ðA:75Þ

Since T
is finite, the condition jtj4T
does not influence the relative measure of the sets obtained in the limit as T ! 1 (see Fact 2.9 and Proof of Lemma 6.2). Thus, taking the limit as T ! 1 in (A.75) for any finite
40 we have mR ðft 2 R : xap ðtÞpx

gÞ pmR ðft 2 R : xap ðtÞ þ ZðtÞpxgÞ pmR ðft 2 R : xap ðtÞpx þ
gÞ.

ðA:76Þ

By taking the limit as
! 0þ, we obtain that (6.8) holds in the continuity points of mR ðft 2 R : xap ðtÞpxgÞ. &

Proof of Lemma 6.4. It results in

¼ mR ðft 2 R : Z
ðtÞpxgÞ þ1 X

¼ mR ðft 2 R; jtjpT
: Z
ðtÞpxgÞ þ mR ðft 2 R; jtj4T
: Z
ðtÞpxgÞ

gð½tÞ ¼

¼ mR ðft 2 R; jtj4T
: Z
ðtÞpxgÞ ¼ 1
mR ðft 2 R; jtj4T
: Z
ðtÞ4xgÞ

where 1½0;1Þ ðt
nÞ ¼ 1½n;nþ1Þ ðtÞ. Thus,

gðnÞ1½0;1Þ ðt
nÞ,

(A.77)

n¼
1




¼ 1
mR ðft 2 R; jtj4T
: Z ðtÞ4

gÞ ¼ 0.

ðA:73Þ

Finally, for x ¼ 0 mR ðft 2 R : ZðtÞp0gÞ ¼ mR ðft 2 R : Zþ ðtÞp0gÞ ¼ mR ðft 2 R : Zþ ðtÞ ¼ 0gÞ.

ðA:74Þ

Such a relative measure can assume any value in ½0; 1, depending on how ZðtÞ approaches zero as jtj ! 1 (e.g., from positive values, from negative values, or oscillating). & Proof of Theorem 6.2. It results limjtj!1 ZðtÞ ¼ 0, that is, 8
40 9T
40 : jtj4T
) jZðtÞjo
. Thus, for any finite T 1 mðft 2 ½t0
T=2; t0 þ T=2, T jtj4T
: xap ðtÞpx

gÞ

mR ðft 2 R : gð½tÞpxgÞ Z 1 T=2 ¼ lim 1fgð½tÞpxg dt T!1 T
T=2 ! Z þ1 X 1 T=2 ¼ lim U x
gðnÞ1½0;1Þ ðt
nÞ dt T!1 T
T=2 n¼
1 Z N 1 T=2 X ¼ lim 1fgðnÞpxg 1½0;1Þ ðt
nÞ dt T!1 T
T=2 n¼
N N X 1 1fgðnÞpxg N!1 2N þ 1 n¼
N Z T =2  1½0;1Þ ðt
nÞ dt

¼ lim


T=2 N X 1 1fgðnÞpxg , N!1 2N þ 1 n¼
N

¼ lim

ðA:78Þ

where UðxÞ ¼ 1 for xX0 and UðxÞ ¼ 0 for xo0 and in the third equality, we set N ¼ ½T=2. &

ARTICLE IN PRESS J. Les´kow, A. Napolitano / Signal Processing 86 (2006) 3796–3825

3824

Proof of Theorem 6.6. By reasoning as in Lemma 6.4 we have mR ðft 2 R : cosð2pPð½tÞÞpxgÞ N X 1 1fcosð2pPðnÞÞpxg N!1 2N þ 1 n¼
N

¼ lim

N X 1 1fcosð2pPðnÞÞpxg , ¯ N!1 2N þ 1 n¼
N

¼ lim

ðA:79Þ

where the second equality follows from the periodicity of cosine. The sequence fPðnÞgn2Z is uniformly distributed modulo 1 due to Theorem 6.5. Moreover, the function f ðxÞ ¼ 1fcosð2pxÞpxg is Riemann integrable in ½0; 1. Thus, accounting for Theorem 6.4, the limit in the R 1 right-hand side of (A.79) exists (and is equal to 1fcosð2pxÞpxg dx). That is, the function xðtÞ defined 0 in (6.14) is RM. & References [1] P. Billingsley, Convergence of Probability Measures, Wiley, New York, 1968. [2] J.L. Doob, Stochastic Processes, Wiley, New York, 1953. [3] H.O.A. Wold, On prediction in stationary time series, Ann. Math. Statist. 19 (1948) 558–567. [4] W.A. Gardner, Two alternative philosophies for estimation of the parameters of time series, IEEE Trans. Inform. Theory 37 (January 1991) 216–218. [5] M. Kac, H. Steinhaus, Sur les foncions inde´pendantes IV, Studia Math. 7 (1938) 1–15. [6] M. Kac, Statistical Independence in Probability, Analysis and Number Theory, The Mathematical Association of America, USA, 1959. [7] W.A. Gardner, The spectral correlation theory of cyclostationary time series, Signal Processing 11 (July 1986) 13–36. [8] W.A. Gardner, Statistical Spectral Analysis: A Nonprobabilistic Theory, Prentice-Hall, Englewood Cliffs, NJ, 1987. [9] W.A. Gardner, W.A. Brown, Fraction-of-time probability for time-series that exhibit cyclostationarity, Signal Processing 23 (June 1991) 273–292. [10] W.A. Gardner, C.M. Spooner, The cumulant theory of cyclostationary time-series, part I: foundation, IEEE Trans. Signal Process. 42 (December 1994) 3387–3408. [11] C.M. Spooner, W.A. Gardner, The cumulant theory of cyclostationary time-series, part II: development and applications, IEEE Trans. Signal Process. 42 (December 1994) 3409–3429. [12] A. Wintner, Zur Theorie beschra¨nkten Bilinearformen, Math. Z. 30 (1929) 228–282. [13] N. Wiener, Generalized harmonic analysis, Acta Math. 55 (1930) 117–258. [14] S. Bochner, B. Jessen, Distribution functions and positivedefinite functions, Ann. Math. 35 (2) (April 1934) 252–257. [15] H. Bohr, Kleinere Beitra¨ge zur Theorie der fastperiodischen Funktionen. II, Det Kgl. Danske Videnskabernes Selskab, Mathematisk-Fysiske 10 (6) (1930) 12–17.

[16] H. Bohr, B. Jessen, Mean-value theorems for the Riemann zeta-function, Quart. J. Math. 5 (1934) 43–47. [17] E.K. Haviland, On statistical methods in the theory of almost-periodic functions, Proc. Natl. Acad. Sci. USA 19 (1933) 549–555. [18] E.K. Haviland, A. Wintner, On the Fourier–Stieltjes transform, Amer. J. Math. 56 (1/4) (1934) 1–7. [19] E.K. Haviland, A. Wintner, A note on the Kronecker–Weyl theorem, Amer. J. Math. 56 (1/4) (1934) 17–24. [20] B. Jessen, A. Wintner, Distribution functions and the Riemann zeta function, Trans. Amer. Math. Soc. 38 (1) (July 1935) 48–88. [21] A. Wintner, Remarks on the ergodic theorem of Birkhoff, Proc. Natl. Acad. Sci. USA 18 (1932) 248–251. [22] A. Wintner, On the asymptotic repartition of the values of real almost periodic functions, Amer. J. Math. 54 (2) (April 1932) 339–345. [23] A. Wintner, On the asymptotic differential distribution of almost-periodic and related functions, Amer. J. Math. 56 (1/4) (1934) 401–406. [24] H. Steinhaus, Sur les foncions inde´pendantes VI, Studia Math. 9 (1940) 121–132. [25] H. Steinhaus, Sur les foncions inde´pendantes VII, Studia Math. 10 (1948) 1–20. [26] K. Urbanik, Effective processes in the sense of H. Steinhaus, Studia Math. 17(1958). [27] E.M. Hofstetter, Random processes, in: H. Margenau, G.M. Murphy (Eds.), The Mathematics of Physics and Chemistry, vol. II, Van Nostrand, Princeton, NJ, 1964 (Chapter 3). [28] J. Bass, Suites uniforme´ment denses, moyennes trigonome´triques, fonctions pseudo-ale´atoires, Bull. Soc. Math. France 87 (1959) 1–64. [29] J. Bass, Cours de Mathe´matiques, Tome III, Masson & Cie, Paris, 1971. [30] W.M. Brown, Time statistics of noise, IRE Trans. Inform. Theory (December 1958) 137–144. [31] W.M. Brown, C.J. Palermo, Random Processes, Communications, and Radar, McGraw-Hill, New York, 1969. [32] H. Furstenberg, Stationary Processes and Prediction Theory, Princeton University Press, Princeton, NJ, 1960. [33] Y.W. Lee, Statistical Theory of Communication, Wiley, New York, 1967. [34] P.M. Ma¨kila¨, J.P. Partington, Least-squares LTI approximation of nonlinear systems and quasistationarity analysis, Automatica 40 (July 2004) 1157–1169. [35] E. Pfaffelhuber, Generalized harmonic analysis for distributions, IEEE Trans. Inform. Theory IT-21 (November 1975) 605–611. [36] H.L. Hurd, T. Koski, The Wold isomorphism for cyclostationary sequences, Signal Processing 84 (May 2004) 813–824. [37] W.A. Gardner, Introduction to Random Processes with Applications to Signals and Systems, Macmillan, New York, 1985 (second ed., McGraw-Hill, New York, 1990). [38] W.A. Gardner, An introduction to cyclostationary signals, in: W.A. Gardner (Ed.), Cyclostationarity in Communications and Signal Processing, IEEE Press, New York, 1994, pp. 1–90 (Chapter 1). [39] L. Izzo, A. Napolitano, The higher-order theory of generalized almost-cyclostationary time-series, IEEE Trans. Signal Process. 46 (November 1998) 2975–2989. [40] L. Izzo, A. Napolitano, Linear time-variant transformations of generalized almost-cyclostationary signals, part I: theory

ARTICLE IN PRESS J. Les´kow, A. Napolitano / Signal Processing 86 (2006) 3796–3825

[41] [42]

[43]

[44] [45] [46]

[47]

[48]

[49]

[50] [51]

[52]

[53]

and method, IEEE Trans. Signal Process. 50 (December 2002) 2947–2961. P. Hughett, Linearity and sigma-linearity in discrete-time linear shift-invariant systems, Signal Processing 59 (1997) 329–333. P. Hughett, A representation theorem for local LSI operators on two-sided sequences, Signal Processing 63 (1997) 169–173. P. Hughett, Representation theorems for semilocal and bounded linear shift-invariant operators on sequences, Signal Processing 67 (1998) 199–209. I. Sandberg, A note on the convolution scandal, IEEE Signal Process. Lett. 8 (7) (July 2001) 210–211. P.M. Ma¨kila¨, On chaotic and random sequences, Physica D 198 (2004) 309–318. A.B. Nobel, First order predictive sequences and induced transformations, Technical Report No. 2367, Department of Statistics, University of North Carolina, Chapel Hill, NC, USA, 1999. A.B. Nobel, Some stochastic properties of memoryless individual sequences, IEEE Trans. Inform. Theory 50 (7) (July 2004) 1497–1505. J. Les´ kow, A. Napolitano, Quantile prediction for time series in the fraction-of-time probability framework, Signal Processing 82 (November 2002) 1727–1741. J.J. Benedetto, A multidimensional Wiener–Wintner theorem and spectrum estimation, Trans. Amer. Math. Soc. 327 (2) (October 1991) 833–852. Y. Meyer, Le spectre de Wiener, Studia Math. 27 (1966) 189–201. N. Wiener, The Fourier Integral and Certain of its Applications, Cambridge University Press, London, 1933, Dover Publications, New York, 1958. P.M. Ma¨kila¨, J.P. Partington, T. Norlander, Bounded power signal spaces, in: Proceedings of the 36th IEEE Conference on Decision & Control, San Diego, CA, USA, December 1997. P.M. Ma¨kila¨, J.P. Partington, T. Norlander, Bounded power signal spaces for robust control and modeling, SIAM J. Control Optim. 37 (1) (1998) 92–117.

3825

[54] J. Mari, A counterexample in power signal space, IEEE Trans. Automat. Control 41 (January 1996) 115–116. [55] A.N. Kolmogorov, Foundations of the Theory of Probability, 1933, Chelsea, New York, 1956. [56] A.N. Kolmogorov, S.V. Fomin, Introductory Real Analysis, Prentice-Hall, Englewood Cliffs, NJ, 1970, Dover, Mineola, NY, 1975. [57] A. Papoulis, Probability, Random Variables, and Stochastic Processes, third ed., McGraw-Hill, New York, 1991. [58] P.P. Hien, Sur les mesures asymptotiques, Ph.D. Dissertation, Universite Paris VI, 1972. [59] R. Johnsonbaugh, W.E. Pfaffenberger, Foundations of Mathematical Analysis, Dover, Mineola, NY, 2002. [60] A.S. Besicovitch, Almost-Periodic Functions, Cambridge University Press, London, 1932, Dover, NY, 1954. [61] M. Fre´chet, Les fonctions asymptotiquement presquepe´riodiques, Rev. Sci. 79 (1941) 341–354. [62] B.F. Beidas, C.L. Weber, Higher-order correlation-based approach to modulation classification of digitally frequencymodulated signals, IEEE J. Selected Areas Comm. 13 (1) (January 1995) 89–101. [63] C.M. Spooner, Classification of cochannel communication signals using cyclic cumulants, in: Proceedings of the 29th Annual Asilomar Conference on Signals, Systems, and Computers, Pacific Grove, CA, October 29–November 1, 1995. [64] A. Swami, B.M. Sadler, Hierarchical digital modulation classification using cumulants, IEEE Trans. Comm. 48 (3) (March 2000) 416–429. [65] D.R. Brillinger, Time series, Data Analysis and Theory, Holden Day, San Francisco, CA, 1981, SIAM 2001. [66] W.A. Brown, On the theory of cyclostationary signals, Ph.D. Dissertation, Department of Electrical Engineering and Computer Science, University of California, Davis, CA, 1987. [67] F. Riesz, B.Sz. Nagy, Functional Analysis, Dover, NY, 1990.