A note on Herglotz’s theorem for time series on function spaces

Available online at www.sciencedirect.com

ScienceDirect Stochastic Processes and their Applications xxx (xxxx) xxx www.elsevier.com/locate/spa

A note on Herglotz’s theorem for time series on function spaces Anne van Delfta ,∗, Michael Eichlerb b

a Ruhr-Universität Bochum, Fakultät für Mathematik, 44780 Bochum, Germany Department of Quantitative Economics, Maastricht University, P.O. Box 616, 6200 MD Maastricht, The Netherlands

Received 24 December 2018; received in revised form 27 September 2019; accepted 10 October 2019 Available online xxxx

Abstract In this article, we prove Herglotz’s theorem for Hilbert-valued time series. This requires the notion of an operator-valued measure, which we shall make precise for our setting. Herglotz’s theorem for functional time series allows to generalize existing results that are central to frequency domain analysis on the function space. In particular, we use this result to prove the existence of a functional Cramér representation of a large class of processes, including those with jumps in the spectral distribution and long-memory processes. We furthermore obtain an optimal finite dimensional reduction of the time series under weaker assumptions than available in the literature. The results of this paper therefore enable Fourier analysis for processes of which the spectral density operator does not necessarily exist. c 2019 Elsevier B.V. All rights reserved. ⃝ MSC: primary 60G10; secondary 62M15 Keywords: Functional data analysis; Spectral analysis; Time series

1. Introduction As a result of a surge in data storage techniques, many data sets can be viewed as being sampled continuously on their domain of definition. It is therefore natural to think of the data points as being objects and embed them into an appropriate mathematical space that accounts for the particular properties and structure of the space. The development of meaningful statistical treatment of these objects is known as functional data analysis and views each random element as a point in a function space. Not surprisingly, this has become an active field of ∗ Corresponding author.

E-mail addresses: [email protected] (A. van Delft), [email protected] (M. Eichler). https://doi.org/10.1016/j.spa.2019.10.006 c 2019 Elsevier B.V. All rights reserved. 0304-4149/⃝

Please cite this article as: A. van Delft and M. Eichler, A note on Herglotz’s theorem for time series on function spaces, Stochastic Processes and their Applications (2019), https://doi.org/10.1016/j.spa.2019.10.006.

2

A. van Delft and M. Eichler / Stochastic Processes and their Applications xxx (xxxx) xxx

research in recent years. If the random functions can be considered an ordered collection {X t }t∈Z we call this collection dependent functional data or a functional time series. The function space where each X t takes its values is assumed to be a separable Hilbert space H . While the literature on classical time series finds its origin in harmonic analysis, the literature on its functional counterpart started in the time domain. The frequency domain arises however quite naturally in the analysis of dependent functional data. The second order dependence structure encodes the relevant information on the shape and smoothness properties of the random curves. It provides a way to optimally extract the intrinsically infinite variation carried by the random functions to lower dimension. In case the functional time series is weakly stationary, the second order dependence structure can be specified in the time domain through an infinite sequence of lag h covariance operators [ ] Ch = E (X 0 − m) ⊗ (X h − m) , h ∈ Z, where m is the mean function of X , which is the unique element of H such that ⟨m, g⟩ = E⟨X, g⟩,

g ∈ H.

Unlike independent functional data, where one only needs to consider the within curves dynamics as captured by the operator C0 , functional time series require to take into account also all the between curve dynamics as given by the infinite sequence of lag covariance operators Ch for h ̸= 0. The full second order dynamics are then more straightforwardly captured in the frequency domain, and an initial framework for Fourier analysis of random functions was therefore developed in [24]. Their framework of frequency domain-based inference is however restricted to processes for which the notion of a spectral density operator, defined as the Fourier transform of the sequence of h-lag covariance operators, 1 ∑ Ch e−ihω , ω ∈ [−π, π], Fω = 2π h∈Z exists. In this case, the autocovariance operator at lag h can itself be represented as ∫ π Ch = eihω Fω dω,

(1.1)

−π

where the convergence holds in the appropriate operator norm. For processes of which Fω has absolutely summable eigenvalues, Panaretos and Tavakoli [23] derived a functional Cram´er representation and showed that the eigenfunctions of Fω allow a harmonic principal component analysis, providing an optimal representation of the time series in finite dimension. This was later relaxed in [28] to processes with only weak spectral density operators implicitly defined by (1.1). An optimal finite dimensional representation of a functional time series was p also derived by H¨ormann et al. [12] for L m -approximable sequences under slightly different assumptions. In both works, the spectral density operator can be seen to take the same role as the covariance operator for independent functional data in the classical Karhunen–Lo`eve expansion [16,18]. van Delft and Eichler [9] extended frequency domain-based inference for functional data to locally stationary processes allowing thus to relax the notion of weak stationarity and to consider time-dependent second order dynamics through a time-varying spectral density operator. Since frequency domain-based inference does not require structural modeling assumptions other than weak dependence conditions, it has proved helpful in the construction of stationarity tests (see e.g., [1,7]), but also in a variety of other inference problems (see e.g., [8,13,17,25]). Please cite this article as: A. van Delft and M. Eichler, A note on Herglotz’s theorem for time series on function spaces, Stochastic Processes and their Applications (2019), https://doi.org/10.1016/j.spa.2019.10.006.

3

A. van Delft and M. Eichler / Stochastic Processes and their Applications xxx (xxxx) xxx

The aforementioned literature is restricted to processes for which the spectral density operators exist at all frequencies as elements of the space of trace class operators, S1 (H ). This excludes many interesting processes for which the spectral density operator is not welldefined at all frequencies or that have a spectral measure with discontinuities. For instance, processes with long memory caused by highly persistent cyclical or seasonal components arise quite naturally in a variety of fields such as hydrology or economics (e.g., [19]). A particular example of functional data are the supply and demand curves for electricity prices, which usually show a strong daily as well as weekly pattern (e.g., [29]). Since statistical inference techniques for this type of data must also take into account their within- and between curves dynamics, it is of importance to be able to develop frequency domain analysis under weaker conditions and to investigate under what conditions such an analysis is possible. In this note, we aim to provide the main building blocks for this relaxation and establish functional versions of the two fundamental results that lie at the core of frequency domain analysis for classical stationary time series: Herglotz’s Theorem and the Cram´er Representation Theorem. It is worth remarking that we establish these two results under necessary conditions. These results allow in particular to develop optimal finite dimension reduction techniques for such highly relevant applications, which are currently not available. The structure of this note is as follows. In Section 2, we start by introducing the necessary notation and terminology. In Section 3, we establish the existence of a functional Herglotz’s theorem. For this, we make precise the concept of an operator-valued measure and the notion of operator-valued kernel functions. In Section 4, Herglotz’s theorem is used to prove a generalized functional Cram´er representation for a large class of weakly stationary Hilbert-valued time series, including those with discontinuities in the spectral measure and long-memory processes. Finally, a Karhunen–Lo`eve expansion on the frequency components in the Cram´er representation is applied in order to obtain a harmonic principal component analysis of the series. 2. Notation and preliminaries 2.1. The function space We first introduce some necessary notation. Let (T, B) be a measurable space with σ -finite measure µ. Furthermore, let E be a Banach space with norm ∥·∥ E and equipped with the Borel p σ -algebra. We then define L E (T, µ) as the Banach space of all strongly measurable functions f : T → E with finite norm (∫ )1 p p ∥ f ∥ L p (T,µ) = ∥ f (τ )∥ E dµ(τ ) E

for 1 ⩽ p < ∞ and with finite norm ∥ f ∥L ∞ = inf E (T,µ)

sup ∥ f (τ )∥ E

µ(N )=0 τ ∈T \N

Lp

for p = ∞. We note that two functions f and g are equal in L p , denoted as f = g, if ∥ f − g∥ L p (T,µ) = 0. If E is a Hilbert space with inner product ⟨·, ·⟩ E then L 2E (T, µ) is also a E Hilbert space with inner product ∫ ⟨ f, g⟩ L 2 (T,µ) = ⟨ f (τ ), g(τ )⟩ E dµ(τ ). E

Please cite this article as: A. van Delft and M. Eichler, A note on Herglotz’s theorem for time series on function spaces, Stochastic Processes and their Applications (2019), https://doi.org/10.1016/j.spa.2019.10.006.

4

A. van Delft and M. Eichler / Stochastic Processes and their Applications xxx (xxxx) xxx

For elements f and g of a Hilbert space H , we denote the inner product by ⟨ f, g⟩ and the induced norm by ∥ f ∥. We shall extensively make use of linear operators on a Hilbert space H . A linear operator on a Hilbert space H is a function A : H → H that preserves the operations of scalar multiplication and addition. We shall denote the class of bounded linear operators by L(H ) and its norm by ||| · |||L . Furthermore, the class of trace class operators and Hilbert–Schmidt operators will be denoted by S1 (H ) and S2 (H ), respectively and their norms by ||| · |||1 and ||| · |||2 . Equipped with these norms (L(H ), ||| · |||L ) and (S1 (H ), ||| · |||1 ) form Banach spaces while (S2 (H ), ||| · |||2 ) forms a Hilbert space with inner product ⟨·, ·⟩ S2 . An operator A ∈ L(H ) is called self-adjoint if ⟨A f, g⟩ = ⟨ f, Ag⟩ for all f, g ∈ H , while we say it is non-negative definite if ⟨Ag, g⟩ ⩾ 0 for all g ∈ H . It will be convenient to denote the respective operator subspaces of self-adjoint and non-negative operators with (·)† and (·)+ , respectively. It is straightforward to verify that L(H )+ ⊆ L(H )† and S p (H )+ ⊆ S p (H )† . For f, g, v ∈ H , the tensor product f ⊗ g ∈ S2 (H ) is defined by ( f ⊗ g)v = ⟨v, g⟩ f . Finally, we denote O H for the zero operator on H and denote the topological dual space of a Banach space B by (B)′ and similarly for appropriate subspaces thereof. 2.2. Functional time series We define a functional time series X = {X t : t ∈ Z} as a sequence of random elements on a probability space (Ω , A, P) taking values in a separable Hilbert space H such as, for instance, the space L 2 ([0, 1]) of all square integrable functions on the interval [0, 1]. Throughout this text, we consider functional time series that are weakly stationary in the usual sense, that is, the first and second moments exist and are invariant under translation in time. More precisely, X is weakly stationary if E∥X t ∥2 < ∞ for all t ∈ Z, X has constant mean functions E(X t ) = m for all t ∈ Z, and the second moment tensors of X satisfy E(X t ⊗ X s ) = E(X t−s ⊗ X 0 ) for all t, s ∈ Z. We note that in this case the random elements X t belong to the Hilbert space H = L 2H (Ω , P) of all H -valued random variables X with E∥X ∥2 < ∞. The inner product on this space H is denoted by ⟨·, ·⟩H = E⟨·, ·⟩ and the induced norm is denoted by ∥ · ∥H . Without loss of generality, we assume that the mean function m is zero. In this case, the h-th lag covariance operators Ch can be defined by Ch = E(X h ⊗ X 0 ). The condition E∥X 0 ∥2 < ∞ ensures that the covariance operators Ch for h ∈ Z belong to S1 (H ). We elaborate on this in Section 4. The available literature on frequency domain analysis for weakly stationary functional time series has focused on so-called ‘short-memory’ processes, that is, processes of which the dependence structure decays at a sufficiently fast rate, namely ∑ |||Ch |||1 < ∞. (2.1) h∈Z

Under this condition, it is straightforward to show that the autocovariance operator Ch forms a Fourier pair with the spectral density operator given by 1 ∑ Fω = Ch e−ihω , 2π h∈Z where the convergence holds in ||| · |||1 and the spectral density operator acts on H . Under condition (2.1), a classical Ces`aro-sum argument (see e.g., [3]) was used by Panaretos and Please cite this article as: A. van Delft and M. Eichler, A note on Herglotz’s theorem for time series on function spaces, Stochastic Processes and their Applications (2019), https://doi.org/10.1016/j.spa.2019.10.006.

A. van Delft and M. Eichler / Stochastic Processes and their Applications xxx (xxxx) xxx

5

Tavakoli [24] to derive that Fω is non-negative definite and hence belongs to S1 (H )+ . As already mentioned in the introduction, the autocovariance operator at lag h itself can then be represented as ∫ π Ch = eihω Fω dω, (2.2) −π

where the convergence holds in ||| · |||1 . Panaretos and Tavakoli [23] showed that a zero mean weakly stationary functional time series X satisfying condition (2.1) admits a functional spectral representation of the form ∫ π eiωt d Z ω a.s., (2.3) Xt = −π

where Z ω is a functional orthogonal ∫ ω increment process such that, for fixed ω, Z ω is a random element in H with E∥Z ω ∥22 = −π |||Fλ |||1 dλ. If the summability conditions as in (2.1) do not hold, the spectral density operators might not necessarily exist as elements of S1+ (H ) and, as a consequence, (2.2) and (2.3) might no longer hold true. Tavakoli [28] studied the frequency domain representations for processes that violate the summability condition (2.1) but possess a weak spectral density operator Fω implicitly defined as an element of L p ([−π, π], S1 (H )) with 1 < p ⩽ ∞ satisfying (2.2). In the next section, we provide a frequency domain representation for general non-negative definite autocovariance operator functions {Ch }. In particular, we do not require any assumptions on the rate of decay or the existence of a (weak) spectral density operator. In Section 4, we then use this representation to derive a functional Cram´er representation for general weakly stationary functional time series, which can be seen as a true generalization of the classical Cram´er representation theorem to functional time series. 3. Herglotz’s theorem on a function space In this section we derive a functional generalization of the classical Herglotz’s theorem. Note that it is intuitive from (2.2) that if we do not have a spectral density operator, then the measure itself must be operator-valued for the equation to be balanced. This raises the following questions: firstly, does an operator of the form ∫ π eihω dF (ω), h ∈ Z, (3.1) −π

where F is an operator-valued measure on [−π, π], exist? Secondly, what properties must an operator possess to be represented by such an integral? From the classical Herglotz’s theorem, we know that the non-negative definite complex-valued functions on the integers are precisely those that can be identified to have a frequency domain representation with respect to a finite Radon measure. Theorem 3.1 (Herglotz’s Theorem). A function γ (·) : Z → C is non-negative definite if and only if ∫ π γ (h) = eihω d F(ω), h∈Z π

where F(·) is a right-continuous, non-decreasing bounded function on [−π, π] with F(−π ) = 0. Please cite this article as: A. van Delft and M. Eichler, A note on Herglotz’s theorem for time series on function spaces, Stochastic Processes and their Applications (2019), https://doi.org/10.1016/j.spa.2019.10.006.

6

A. van Delft and M. Eichler / Stochastic Processes and their Applications xxx (xxxx) xxx

Here, the so-called spectral distribution function F(·) with F(−π ) = 0 is uniquely determined by the covariance function γ (h), h ∈ Z. To extend this result to our functional setting, we require the notion of non-negative definiteness of operator-valued functions on Z as well as definitions of operator-valued measures and of integrals with respect to such measures. For the former, we proceed as in the scalar-valued case with the help of non-negative operator-valued kernels. Definition 3.2. (i) A function c : Z × Z → L(H ) is called a non-negative definite L(H )-valued kernel if n ∑

⟨c(i, j) g j , gi ⟩ ⩾ 0

i, j=1

for all g1 , . . . , gn ∈ H , and n ∈ N. (ii) A function C : Z → L(H ) is called non-negative definite if the kernel c : Z×Z → L(H ) defined by c(i, j) = C(i − j) is a non-negative definite kernel. Non-negative definite operator-valued kernels are an extremely powerful tool in functional analysis, especially in operator theory and representation theory. A generalization of this concept to general involutive semigroups other than Z can be found in [21]. For functional time series, Definition 3.2 provides a link between the properties of the covariance kernel and the corresponding covariance operator viewed as a function on the integers. More specifically, we have the following result. Proposition 3.3. Let {X t : t ∈ Z} be a zero-mean weakly stationary functional time series with covariance operators Ch , h ∈ Z. Then C(·) : Z → L(H ) is a non-negative definite function. Proof. The operator-valued kernel c(i, j) = Ci− j , i, j ∈ Z satisfies n ∑ i, j=1

⟨Ci− j g j , gi ⟩ =

n ∑

n ∑ 2  ⟨E(X i ⊗ X j )g j , gi ⟩ = E ⟨X i , gi ⟩ ⩾ 0

i, j=1

and is therefore non-negative definite by Definition 3.2.

i=1

□

3.1. L(H )-valued measures In this subsection, we explain how L(H )-valued measures can be defined. Let us first provide some intuition by making an analogy with positive scalar-valued measures. Recall that a positive measure µ on a measurable space (T, B) is defined as a countably additive map on B taking values in the compactification [0, ∞] of the set R+ := [0, ∞). The compactification is necessary for σ -additivity to hold (although for finite measures the compact subset [0, µ(T )] is sufficient). We note that R+ is an example of a pointed convex cone, that is, it is a convex nonempty subset of R that is closed under non-negative scalar multiplication and contains the zero element. It is moreover dense in R+ ∞ := [0, ∞]. Taking this view, a positive measure more generally can be defined as a countably additive map taking values in the compactification of a pointed convex cone. For the purpose of this paper, we are solely interested in measures taking values in the compactification of L(H )+ , the pointed convex cone of L(H ) consisting of all non-negative elements L(H ). We now explain heuristically how such a measure can be defined. Please cite this article as: A. van Delft and M. Eichler, A note on Herglotz’s theorem for time series on function spaces, Stochastic Processes and their Applications (2019), https://doi.org/10.1016/j.spa.2019.10.006.

A. van Delft and M. Eichler / Stochastic Processes and their Applications xxx (xxxx) xxx

7

The technical argument and more details on properties of cones are relegated to Appendix B. For the general theory on cone-valued measures we refer to Neeb [21] and Glockner [11]. Essential in the construction of the measure is the following duality between the underlying real Banach space of self-adjoint elements L(H )† and the real Banach space of self-adjoint trace class operators, S1 (H )† . More specifically, by the duality pairing ⟨·, ·⟩ : L(H )† × S1 (H )† → R,

⟨B, A⟩ = tr(B A)

(3.2)

and defining φ B : S1 (H )† → R with φ B (A) = ⟨B, A⟩ = tr(B A), we can identify (L(H )† , ||| · |||L ) with the topological dual space of (S1 (H )† , ||| · |||1 ) through the isometric isomorphism φ : L† (H ) → (S1 (H )† )′ ,

B ↦→ φ B ,

that is, we have L(H )† ∼ = (S1 (H )† )′ . This isomorphism provides us with a natural notion of convergence for a sequence of operators in L(H )† , known as the ultraweak topology. Definition 3.4. A sequence of operators {Bn } ∈ L(H )† converges in the ultraweak topology to B0 if φ Bn (A) → φ B0 (A) for all A ∈ S1 (H )† as n → ∞. Thus, the ultraweak topology is the coarsest topology such that the pointwise evaluations φ B (A) for all A ∈ S1 (H )† are continuous as functions in B. The isomorphism φ when restricted to the cone L(H )+ of non-negative definite bounded operators suggests a similar result between L(H )+ and S1 (H )+ , the non-negative definite trace class operators, namely L(H )+ ∼ = (S1 (H )+ )′ . However, a slight problem arises from the fact + + + that L(H ) , S1 (H ), and R are not vector spaces in the strict sense because they are not closed under negative scalar multiplication. Nevertheless, they can be regarded as topological monoids with respect to addition as they are closed under addition, have a zero element, and the addition is continuous (see Appendix A). Similarly, the restricted mappings φ B : S1 (H )+ → R+ for B ∈ L(H )+ now become monoid homomorphisms, that is, they preserve addition and the zero element. Moreover, it can be shown that the mapping φ in this case provides an isomorphism (Appendix B.2) between the monoid L(H )+ and the monoid Hommon (S1 (H )+ , R+ ), which consists of all monoid homomorphisms from S1 (H )+ to R+ . That is, we have φ(L(H )+ ) = Hommon (S1 (H )+ , R+ ).

(3.3)

Since the isometry property of φ is preserved when restricted to the cone L(H ) we can obtain (Theorem B.5) an isometric isomorphism +

φ(L(H )+ ) = Hommon (S1 (H )+ , R+ ) ∼ =m L(H )+ , where the subscript m in ∼ =m emphasizes that the isomorphism is between monoids. Since the set (3.3) is dense in the compact set Hommon (S1 (H )+ , R+ ∞ ) [11], the compactification of L(H )+ naturally can be defined by + + L+ ∞ = Hommon (S1 (H ) , R∞ ).

(3.4)

The set Hommon (S1 (H )+ , R+ ∞ ) inherits the notion of convergence as given in Definition 3.4, ensuring that addition is continuous with respect to this notion. The argument is technical and can be found in Appendix B. With this, L(H )+ -valued measures can now be defined as follows. + Definition 3.5. Let (T, B) be a measurable space. A mapping µ : B → L+ ∞ is an L(H ) -valued measure on (T, B) if it is countably additive and µ(∅) = O H . The measure µ is called finite

Please cite this article as: A. van Delft and M. Eichler, A note on Herglotz’s theorem for time series on function spaces, Stochastic Processes and their Applications (2019), https://doi.org/10.1016/j.spa.2019.10.006.

8

A. van Delft and M. Eichler / Stochastic Processes and their Applications xxx (xxxx) xxx

if µ(T ) ∈ Hommon (S1 (H )+ , R+ ) ∼ =m L(H )+ . It is called ⋃∞σ -finite if T is the countable union of measurable sets with finite measure, that is, if T = i=1 Ti and where µ(Ti ) ∈ L(H )+ for all i ∈ N. Observe that by (3.4), countably additivity in L+ ∞ holds with respect to the ultraweak ∞ topology, that is, for any sequence {E i }i=1 of pairwise disjoint sets in B, we have µ(

∞ ⋃

E i )(A) =

i=1

∞ ∑

µ(E i )(A)

i=1

for all A ∈ S1 (H )+ , where we note that µ(E i )(A) = tr(µ(E i )A). This implies also that a positive L(H )+ -valued measure µ can be represented by a family of positive scalar-valued measures {µ A } A∈S1 (H )+ given by the evaluation functions µ A (E) = µ(E)(A) for all E ∈ B. Conversely, a family of positive scalar-valued measures {µ A } A∈S1 (H )+ defines a positive L(H )+ -valued measure µ if and only if for all E ∈ B the mapping A ↦→ µ A (E) is a monoid homomorphism. We summarize this important relation between the operator-valued measure and the corresponding family of scalar-valued measures in the following theorem, which is a simplification of Theorem I.10 of Neeb [22]. Theorem 3.6. Let {µ A } A∈S1 (H )+ be a family of non-negative measures on the measurable space (T, B) such that for each E ∈ B the assignment A ↦→ µ A (E) is a monoid homomorphism. Then there exists for each E ∈ B a unique element µ(E) ∈ L+ ∞ with µ(E)(A) = µ A (E) for + all A ∈ S1 (H )+ and the function µ : B → L+ is a L(H ) -valued measure. ∞ In particular, it follows that a L+ ∞ -valued Radon measure µ on a locally compact space T is a measure with the property that, for each non-negative trace class operator A ∈ S1 (H )+ , the measure µ A : B → R+ ∞ is a finite non-negative Radon measure on T . Moreover, integrability of a function with respect to the scalar measure implies integrability of the function with respect to the L(H )+ -valued measure. 3.2. Functional Herglotz’s theorem We are now ready to prove the following generalization of Theorem 3.1. Theorem 3.7 (Functional Herglotz’s Theorem). A function Γ : Z → L(H ) is non-negative definite if and only if ∫ π Γ (h) = eihω dF (ω) h ∈ Z, (3.5) −π

where F is a finite L+ ∞ -valued measure on [−π, π] with F (−π ) = O H . The finite measure F is uniquely determined by Γ (h), h ∈ Z. Proof. Suppose first that Γ (h) admits the representation (3.5) with respect to a finite L+ ∞ -valued measure F on [−π, π]. Since F is a finite measure the integral is well-defined. By (ii) of Definition 3.2, it is sufficient to show that the operator-valued kernel γ (h 1 , h 2 ) := Γ (h 1 − h 2 ) Please cite this article as: A. van Delft and M. Eichler, A note on Herglotz’s theorem for time series on function spaces, Stochastic Processes and their Applications (2019), https://doi.org/10.1016/j.spa.2019.10.006.

A. van Delft and M. Eichler / Stochastic Processes and their Applications xxx (xxxx) xxx

9

is non-negative definite. Using (i) of Definition 3.2 and that F takes values in L(H )+ , we have for all h 1 , . . . , h n ∈ Z and g1 , . . . , gn ∈ H . ∫ n n ∑ ∑ ⟨γ (h j , h k )gk , g j ⟩ = ⟨ ei(h j −h k )ω dF (ω)gk , g j ⟩ j,k=1

j,k=1

[−π,π ]

∫ ⟨F (dω)

= [−π,π]

n ∑ k=1

e−ih k ω gk ,

n ∑

e−ih j ω g j ⟩⩾ 0.

j=1

+

Conversely, suppose that Γ (·) is a L(H ) -valued function on Z and let A be an element of S1 (H )+ . Define the function Γ A : Z → C by Γ A (h) = tr(Γ (h)A). Since the square root of both Γ (h) and A are well-defined and the trace satisfies tr(Γ (h)A) = tr(AΓ (h)), it is direct that Γ A is a non-negative definite function on the integers. By the classical Herglotz’s theorem (Theorem 3.1), we therefore have the representation ∫ π eihω dF A (ω) h ∈ Z, (3.6) Γ A (h) = −π

where F A (·) with F A (−π ) = 0 is a uniquely determined Radon measure on [−π, π]. More specifically, Theorem 3.1 implies that F A is a right-continuous non-decreasing bounded function on [−π, π]. The measure satisfies F A ([−π, π]) = Γ A (0) < ∞ and is therefore finite. Let A1 , A2 ∈ S1 (H )+ and observe that A1 + A2 ∈ S1 (H )+ and thus Γ A1 +A2 (h) is again a nonnegative definite function on Z. Hence, another application of the classical Herglotz’s theorem to Γ A1 +A2 (·) yields ∫ π Γ A1 +A2 (h) = eihω dF A1 +A2 (ω) h ∈ Z, −π

∫π ∫π whereas by (3.6) Γ A1 (h)+Γ A2 (h) = −π eihω dF A1 (ω)+ −π eihω dF A2 (ω) and since linearity of the trace yields that Γ A1 (h)+Γ A2 (h) = Γ A1 +A2 (h), we obtain that F A1 (·)+F A2 (·) = F A1 +A2 (·) for any A1 , A2 ∈ S1 (H )+ . This demonstrates that A ↦→ F A (E) is a monoid homomorphism S1 (H )+ → R+ with respect to addition for each E ∈ B. By Theorem 3.6, the family of measures {F A } A∈S1 (H )+ therefore uniquely identifies an element F (E) ∈ L+ ∞ on the measurable space ([−π, π], B) with F (E)(A) = tr(F (E)A) = F A (E) for all A ∈ S1 (H )+ . Since F A ([−π, π]) = tr(F ([−π, π])A) < ∞, ∀A ∈ S1 (H )+ , we obtain that F is a finite operator-valued measure on [−π, π], i.e, F ([−π, π]) ∈ L(H )+ . Finally, note from the above that (∫ π ) tr(Γ (h)A) = tr eihω dF (ω)A ∀A ∈ S1 (H )+ −π

∫π

and thus Γ (h) − −π eihω dF (ω) = O H for all h ∈ Z which proves that F is uniquely determined by Γ (h), h ∈ Z. □ Remark 3.8 (Analogy to the classical Herglotz’s Theorem). Theorem 3.7 tells us that F is a + finite L+ ∞ -valued measure on [−π, π], i.e., F ([−π, π]) ∈ L(H ) . To make the analogy to the classical Herglotz theorem,we note that F ([−π, ω]) can be seen as a non-decreasing, rightcontinuous operator-valued function in ω ∈ [−π, π]. In particular, we identified the measure with a family of finite scalar-valued measures {F A } A∈S1 (H )+ on [−π, π], which satisfied the conditions of the classical Herglotz theorem. Consequently, F is right-continuous in the sense that limω↓ωo F ([−π, ω]) = tr(F (−π, ωo ]) where the convergence holds in the ultraweak Please cite this article as: A. van Delft and M. Eichler, A note on Herglotz’s theorem for time series on function spaces, Stochastic Processes and their Applications (2019), https://doi.org/10.1016/j.spa.2019.10.006.

10

A. van Delft and M. Eichler / Stochastic Processes and their Applications xxx (xxxx) xxx

topology. Moreover, by definition of it being a L(H )+ -valued measure on [−π, π], it is nondecreasing in [−π, π], i.e., for all ω2 ⩾ ω1 , ω1 , ω2 ∈ [−π, π] we have F ([ω1 , ω2 ]) = F (ω2 ) − F (ω1 ) ⩾ O H . 4. A generalized functional Cramér representation The Spectral Representation Theorem [6], often called the Cram´er representation, is as fundamental to frequency domain analysis as Wold’s representation is to the time domain. It asserts that every (finite dimensional) zero-mean weakly stationary process can be represented as a superposition of sinusoids with random amplitudes and phases that are uncorrelated. An important ingredient in establishing this classical theorem is the existence of an isometric isomorphism that allows to identify a weakly stationary time series on the integers with an orthogonal increment process on [−π, π]. As already mentioned, an initial generalization of the Cram´er representation to weakly stationary functional time series was first considered by ∑ Panaretos and Tavakoli [23], but is restricted to processes for which the assumption h∈Z |||Ch |||1 < ∞ holds. In this section, we shall use the established functional Herglotz’s theorem (Theorem 3.7) to derive a functional Cram´er representation that can be seen as a true generalization of the classical theorem to the function space. In addition, we establish a Cram´er–Karhunen–Lo`eve representation –a term first coined by Panaretos and Tavakoli [23]–, and a harmonic principal component analysis for a very general class of processes of which the spectral measure can have finitely many discontinuities. We first show that for a weakly stationary functional time series the full second order structure is given by a sequence of trace class operators. Corollary 4.1. Let X be a weakly stationary functional time series. Then the sequence of lag covariance operators {Ch }h∈Z belongs to S1 (H ). Proof. By Jensen’s inequality, we have ∑ |||E(X h ⊗ X 0 )|||1 ⩽ E|||X h ⊗ X 0 |||1 . X h ⊗ X 0 is therefore a random element of S1 (H ) if E i |⟨(X h ⊗ X 0 )ei , ei ⟩| < ∞. By the Cauchy–Schwarz inequality and Parseval’s identity √∑ √∑ ∑ 2 |⟨ei , X 0 ⟩| |⟨X h , ei ⟩|2 E |⟨(X h ⊗ X 0 )ei , ei ⟩| ⩽ E i

i

i

⩽ ∥X h ∥H ∥X 0 ∥H = ∥X 0 ∥2H < ∞, where the last inequality follows again from the Cauchy–Schwarz inequality and the equality follows from weak stationarity. □ As in the classical case, the proof of the functional Cram´er representation is based on showing that the mapping X t ↦→ eit· forms an Hilbert space isometric isomorphism between L 2H (Ω , P) and L 2 ([−π, π], µF ) where we define µF (E) = |||F (E)|||1

(4.1)

for all Borel sets E ⊆ [−π, π]. Here, F is the operator-valued measure on [−π, π] induced by the sequence of covariance operators {C h }h∈Z of {X t : t ∈ Z}. Before we derive the properties of the mapping, we have to verify that this indeed defines a measure. This is the contents of the following lemma. Please cite this article as: A. van Delft and M. Eichler, A note on Herglotz’s theorem for time series on function spaces, Stochastic Processes and their Applications (2019), https://doi.org/10.1016/j.spa.2019.10.006.

A. van Delft and M. Eichler / Stochastic Processes and their Applications xxx (xxxx) xxx

11

Lemma 4.2. Let X be a weakly stationary functional time series. Then the function µF defined in (4.1) is a finite scalar-valued measure on [−π, π]. Proof of Lemma 4.2. By Proposition 3.3, the covariance function C(·) : Z → S1 (H ) of a weakly stationary H -valued time series is non-negative definite. Using Corollary 4.1, Theorem 3.7 implies this function uniquely determines a S1 (H )+ -valued measure F on [−π, π]. Using the properties of F , it is now straightforward to verify that the function µF : B → [0, ∞] is a non-negative scalar-valued measure on the measurable space ([−π, π], B). Firstly, for each Borel set E ⊆ [−π, π] and every e ∈ H , we have ⟨F (E)e, e⟩ ⩾ 0 and thus µF = tr(F ) ⩾ 0. Secondly, ⟨F (∅)e, e⟩ = ⟨O H e, e⟩ = µF (∅) = 0 which follows by definition of F . Thirdly, for all countable selections of pairwise disjoint set {E i }i∈N in B, countable additivity of F yields µF

∞ (⋃

∞ ∑ ∞ ∞ ∞ ∑ ⋃ ) ∑ F (E i )e j , e j ⟩, ⟨ ⟨F ( E i )e j , e j ⟩ = Ei = j

i=1

j=1

i=1

i

where {e j } j∈N is an orthonormal basis of H . By continuity of the inner product and the fact that F ([−π, π]) < ∞, Fubini’s theorem implies ∞ ∑ ∞ ∑

⟨F (E i )e j , e j ⟩ =

i=1 j=1

∞ ∑

|||F (E i )|||1 =

i=1

∞ ∑

µF (E i )

i=1

and thus µF is countably additive. Finally, since F is S1 (H )+ -valued measure on [−π, π] it is direct that µF ([−π, π]) < ∞. □ Additionally, to be able to properly define the spectral representation we require the notion of an H -valued orthogonal increment process. Definition 4.3. An H -valued random process {Z ω : − π ⩽ ω ⩽ π } defined on a probability space (Ω , A, P) is a functional orthogonal increment process, if for all g1 , g2 ∈ H and −π ⩽ ω ⩽ π (i) (ii) (iii) (iv)

the operator E(Z ω ⊗ Z ω ) is an element of S1 (H )+ ; E⟨Z ( ω , g1 ⟩ = 0; ) ⟨E((Z ω4 − Z ω3 ) ⊗ (Z ω2 − Z ω1 )) g1 , g2 ⟩ = 0 for (ω1 , ω2 ] ∩ (ω3 , ω4 ] = ∅; ⟨E (Z ω+ε − Z ω ) ⊗ Z ω+ε − Z ω g1 , g2 ⟩ → 0 as ε ↓ 0.

To establish the isomorphism, let H ⊂ H denote the space spanned by all finite linear combinations of the random functions X t , i.e., H = sp{X t : t ∈ Z}. We remark that the inner product on H satisfies ⟨X 1 , X 2 ⟩H = tr(E(X 1 ⊗ X 2 )).

(4.2)

Furthermore let the space H denote the space of all square-integrable functions on [−π, π] with respect to the measure µF , i.e., H = L 2 ([−π, π], µF ). This space becomes a Hilbert space once we endow it with the inner product ∫ π ∫ π f, g ∈ H, f (ω)g(ω)dF (ω)) ⟨ f, g⟩H = f (ω)Ę g(ω)dµF (ω) = tr( −π

−π

where the last equality follows by non-negative definiteness of F and linearity of the trace operator. Please cite this article as: A. van Delft and M. Eichler, A note on Herglotz’s theorem for time series on function spaces, Stochastic Processes and their Applications (2019), https://doi.org/10.1016/j.spa.2019.10.006.

12

A. van Delft and M. Eichler / Stochastic Processes and their Applications xxx (xxxx) xxx

Theorem 4.4. Let X be a weakly stationary functional time series, and let F be the S1 (H )+ -valued measure corresponding to the sequence of covariance operators {C h }h∈Z of the process {X t : t ∈ Z}. Then there exists an isometric isomorphism T between sp{X t } and L 2 ([−π, π], µF ) such that T X t = eit· ,

t ∈ Z.

The process defined by ( ) Z ω = T −1 1(−π,ω] (·) is then a functional orthogonal increment process of which the covariance structure is uniquely determined by F and satisfies [ ] E (Z ω − Z λ ) ⊗ (Z ω − Z λ ) = F (ω) − F (λ), −π ⩽ λ ⩽ ω ⩽ π. Proof of Theorem 4.4. Consider first the mapping T : H → H given by T(

n ∑

a j Xt j ) =

j=1

n ∑

a j eit·

t ∈ Z.

j=1

It is straightforward to∑see the mapping is linear and preserves inner products. Let Y = ∑ n n j=1 a j X t j and W = j=1 b j X t j . By Theorem 3.7, ⟨T Y, T W ⟩H = =

= =

n ∑ i, j=1 n ∑ i, j=1 n ∑ i, j=1 n ∑

ai b j ⟨ei·ti , ei·t j ⟩H ∫

π

ai b j

eiλ(ti −t j ) dµF (λ)

−π

∫

π

ai b j tr(

eiλ(ti −t j ) dF (λ))

−π

ai b j tr(Ci− j ) =

i, j=1

n ∑ i, j=1

ai b j ⟨X ti , X t j ⟩H = ⟨Y, W ⟩H .

For the extension of the isomorphism over the closure of H onto the closure of H , note that if Y is an element of H¯ then there must exist a sequence {Yn }n⩾1 ∈ H converging to Y . Denote T (Y ) to be the limit of T (Yn ), i.e., T (Y ) = lim T (Yn ). n→∞

Since {Yn } is a Cauchy sequence and T norm-preserving, {T Yn } is a Cauchy sequence in L 2 ([−π, π], µF ) and thus T (Y ) ∈ H¯ . If there is another sequence {Yn′ } ∈ H converging to Y , then the limit must be unique since lim ∥T (Yn ) − T (Yn′ )∥H = lim ∥Yn − Yn′ ∥H = 0,

n→∞

n→∞

and therefore the extension is well-defined. Preservation of linearity and the isometry property are straightforward from linearity of T on H and continuity of the inner product, respectively. To show that the closure of H is in fact L 2 ([−π, π], µF ), we recall that the Stone–Weierstrass theorem (Fej´er’s theorem) implies that H is dense in the space of 2π -periodic continuous functions on [−π, π]. Moreover, by Lemma 4.2, µF is a finite Radon measure (i.e., finite Please cite this article as: A. van Delft and M. Eichler, A note on Herglotz’s theorem for time series on function spaces, Stochastic Processes and their Applications (2019), https://doi.org/10.1016/j.spa.2019.10.006.

A. van Delft and M. Eichler / Stochastic Processes and their Applications xxx (xxxx) xxx

13

and regular) on [−π, π]. The set of continuous functions with compact support are therefore in turn uniformly dense in L 2 ([−π, π], µF ) (see e.g., [2,26]). Consequently we find H¯ = L 2 ([−π, π], µF ). The inverse mapping T −1 : L 2 ([−π, π], µF ) → H¯ is therefore properly defined. This finishes the proof of the first part of the theorem. Let us then define, for any ω ∈ (−π, π], the process ( ) Z ω = T −1 1(−π,ω] (·) ¯ Therefore with Z −π ≡ 0 ∈ H . By the established isometry, this process is well-defined in H. there must exist a sequence {Yn } in H such that limn→∞ ∥Yn − Z ω ∥H = 0. Since all elements in the sequence have zero mean, continuity of the inner product implies ⟨Z ω , f ⟩H = lim tr(E(Yn ⊗ f )) = lim ⟨E[Yn ], f ⟩ = 0 n→∞

n→∞

∀ f ∈ H,

showing the process {Z ω : −π ⩽ ω ⩽ π } has zero mean. Additionally, ⟨Z ω4 − Z ω3 , Z ω2 − Z ω1 ⟩H = ⟨1(ω3 ,ω4 ] (·), 1(ω1 ,ω2 ] (·)⟩H ∫ π 1(ω3 ,ω4 ] (ω)1(ω1 ,ω2 ] (ω)dµF (ω). =

(4.3)

−π

For all (ω1 , ω2 ] ∩ (ω3 , ω4 ] = ∅, this inner product is zero while for ω3 = ω1 , ω4 = ω2 we have ⟨Z ω2 − Z ω1 , Z ω2 − Z ω1 ⟩H = µF (ω2 ) − µF (ω1 ).

ω1 ⩽ ω2

showing that the {Z ω } is right-continuous. We can also write (4.3) as ∫ ( π ) tr 1(ω3 ,ω4 ] (ω)1(ω1 ,ω2 ] (ω)dF (ω) . −π

For ω3 = ω1 , ω4 = ω2 this implies E(Z ω2 − Z ω1 ) ⊗ (Z ω2 − Z ω1 ) = F (ω2 ) − F (ω1 ),

ω1 ⩽ ω2 .

where the equality holds in ||| · |||1 . The second order structure of Z ω is therefore uniquely defined by the operator-valued measure F of the process X . □ The generalization of the Cram´er representation to processes of which the spectral density operator is not necessarily well-defined is given in the following theorem. Theorem 4.5 (Functional Cramér Representation). Let X be a weakly stationary functional time series. Then there exists a right-continuous functional orthogonal increment process {Z ω , −π ⩽ ω ⩽ π } with Z −π ≡ 0 ∈ H such that ∫ π Xt = eitω d Z ω a.s. −π

Proof of Theorem 4.5. Consider the subspace Hs of H¯ containing the simple functions, i.e., the space Hs contains elements of the form n ∑ g(ω) = ai 1(ωi ,ωi+1 ] i=1

for a partition Pn = {−π = ω0 < ω1 < · · · < ωn+1 = π } of [−π, π] and ai ∈ C. Then define the mapping I : Hs → H¯ given by n ∑ I(g) = ai (Z ωi+1 − Z ωi ). i=0 Please cite this article as: A. van Delft and M. Eichler, A note on Herglotz’s theorem for time series on function spaces, Stochastic Processes and their Applications (2019), https://doi.org/10.1016/j.spa.2019.10.006.

14

A. van Delft and M. Eichler / Stochastic Processes and their Applications xxx (xxxx) xxx

By Theorem 4.4, this is an isomorphism from H¯ onto H¯ and coincides with T −1 . More specifically, I(ei·t ) = T −1 (ei·t ) = T −1 T (X t ) = X t and the statement of the Theorem follows by taking the Riemann–Stieltjes integral limit n ∑  2 Xt − eiωi t (Z ωi+1 − Z ωi )H → 0 as n → ∞, i=0

where mesh(Pn ) → 0 as n → ∞. More generally, for any g ∈ L 2 ([−π, π], µF ), the mapping I(g) corresponds to the Riemann–Stieltjes integral with respect to the orthogonal increment process Z ω . □ Remark 4.6. By means of the isometric isomorphism in Theorem 4.4, we can write Z ω also in terms of the limit of a weighted sum of the functions X t . Firstly, we note that the indicator function 1(−π,ω] (·) can be∑approximated in L 2 ([−π, π], µF ) by the N -th order Fourier itλ ˜ series approximation b N (λ) = where the Fourier coefficients are given by |t|⩽N bω,t e ∫π 1 −itλ b˜ω,t = 2π −π 1(−π,ω] (λ) e dλ. More specifically, we have ∥b N (·) − 1(−π,ω] (·)∥2H → 0 as N → ∞ (see e.g., [4]). Consider therefore the following weighted sum ∫ π ∑ 1 ∑ Z ω(N ) = Xt 1(−π,ω] (λ) eitλ dλ = b˜ω,t X t , 2π |t|⩽N −π |t|⩽N Then by Theorem 4.4 lim ∥Z ωN − Z ω ∥2H = ∥T −1 b N (·) − T −1 1(−π,ω] (·)∥2H = ∥b N (·) − 1(−π,ω] (·)∥2H = 0.

N →∞

In case of discontinuities in the spectral measure F , we can decompose the process into a purely indeterministic component and a purely deterministic component. Proposition 4.7. Assume the spectral measure F of a weakly stationary functional time series {X t } has k points of discontinuity at ω1 , . . . , ωk . Then with probability one ∫ k ∑ (4.4) Xt = eitω d Z ω + (Z ωℓ − Z ω− )eitωℓ , (−π,π]\{ω1 ,...,ωk }

ℓ=1

ℓ

where Z ω− = limω↑ωℓ Z ω in H, that is, limω↑ωℓ ∥Z ω− − Z ω ∥2H = 0. Furthermore, all terms on ℓ ℓ the right hand side of (4.4) are uncorrelated and var(Z ωℓ − Z ω− ) = Fωℓ − Fω− ℓ

ℓ

for each ℓ = 1, . . . , k. The proof is relegated to Appendix. The spectral representation in Proposition 4.7 can be used to define a Cram´er–Karhunen–Lo`eve representation for processes of which the spectral measure has finitely many discontinuities. Definition 4.8 (Cramér–Karhunen–Loève Representation). Suppose that the weakly stationary functional time series X = {X t } is given by ∫ k ∑ Xt = eitω d Z ω + (Z ωℓ − Z ω− )eitωℓ , M

ℓ=1

ℓ

Please cite this article as: A. van Delft and M. Eichler, A note on Herglotz’s theorem for time series on function spaces, Stochastic Processes and their Applications (2019), https://doi.org/10.1016/j.spa.2019.10.006.

A. van Delft and M. Eichler / Stochastic Processes and their Applications xxx (xxxx) xxx

15

where M = (−π, π] \ {ω1 , . . . , ωk } for −π < ω1 < · · · < ωk ⩽ π , and let F be the spectral measure of X .∫ Furthermore, assume there exists an operator-valued function ω ↦→ Fω such that F (M) = M Fω dω with eigendecomposition Fω =

∞ ∑

ν ωj φ ωj ⊗ φ ωj .

j=1

Furthermore, let F (ωℓ ) − F (ωℓ− ) =

∞ ∑

ω

ω

ω

νj ℓφj ℓ ⊗ φj ℓ

j=1

be the eigendecomposition of F (ωℓ ) − F (ωℓ− ) for ℓ = 1, . . . , k. Then we can write ∫ ∞ k (∑ ∞ (∑ ) ∑ ω ω ) eitω Xt = φ ωj ⊗ φ ωj d Z ω + φ j ℓ ⊗ φ j ℓ (Z ωℓ − Z ω− )eitωℓ , (−π,π ]\{ω1 ,...,ωk }

j=1

ℓ

ℓ=1 j=1

which is the Cram´er–Karhunen–Lo`eve representation of the process {X t }. Note that the spectral measure for all measurable sets [−π, π] has positive definite increments and therefore F (ωℓ ) − F (ωℓ− ) has an eigendecomposition with positive eigenvalues. If there are no discontinuities then the Cram´er–Karhunen–Lo`eve representation simply coincides with the indeterministic component of Definition 4.8, i.e., ∫ ∞ (∑ ) eitω Xt = φ ωj ⊗ φ ωj d Z ω . (4.5) (−π,π]

j=1

In order to derive an optimal finite dimensional representation of the indeterministic component of the process, we require in Definition 4.8 that there is a well-defined spectral density operator except on sets of measure zero. We remark that this assumption also covers a harmonic principal component analysis of long-memory processes (see Remark 4.10) and holds under much weaker conditions (see e.g., [12]) than those stated in [23], who originally derived a Cram´er–Karhunen–Lo`eve representation of the form (4.5) for processes with short-memory. The Cram´er–Karhunen–Lo`eve representation in Definition 4.8 can be seen to encapsulate the full second order dynamics of the process and gives insight into an optimal finite dimensional representation. As originally noted in [23], such a representation can be viewed as a ‘double’ spectral representation in the sense that it first decomposes the process into uncorrelated functional frequency components and in turn provides a spectral decomposition in terms of dimension. This is more easily seen by noting that formally we can write it as ∫ k ∑ ∞ ∞ ∑ ∑ ω ω Xt = eitω ⟨d Z ω , φ ωj ⟩φ ωj + ⟨Z ωℓ − Z ω− , φ j ℓ ⟩φ j ℓ eitωℓ . (−π,π]\{ω1 ,...,ωk }

j=1

ℓ=1 j=1

ℓ

Just like the Karhunen–Lo`eve representation for independent functional data, it separates the stochastic part from the functional part and provides information on the smoothness of the random curves. Furthermore, it enables to represent each frequency component into an optimal basis where its dimensionality can be derived from the relative contribution of the component to the total variation of the process. A truncation of the infinite sums at a finite level therefore allows an optimal way to construct a finite dimensional representation of the process. Please cite this article as: A. van Delft and M. Eichler, A note on Herglotz’s theorem for time series on function spaces, Stochastic Processes and their Applications (2019), https://doi.org/10.1016/j.spa.2019.10.006.

16

A. van Delft and M. Eichler / Stochastic Processes and their Applications xxx (xxxx) xxx

Such a truncation for processes that satisfy Definition 4.8 requires that stochastic integrals ∫π of the form −π Uω d Z ω are well-defined where Uω is an element of the Bochner space B∞ = L 2S∞ (H ) ([−π, π], µF ) of all strongly measurable functions U : [−π, π] → S∞ (H ) such that ∫ 2 ∥U ∥B∞ = |||Uω |||2∞ dµ(ω) < ∞ Π

with µ(E) =

∫ dµF (ω), E

for all Borel sets E ⊆ [−π, π] and where µF is the measure in (4.1). This is proved in the Appendix (Proposition C.1) and generalizes the result in Appendix B 2.3 of van Delft and Eichler [9]. With this in place, we obtain a harmonic principal component analysis for processes of which the spectral measure has finitely many jumps. Corollary 4.9 (Harmonic Functional Principal Component Analysis). Suppose {X t } has a Cram´er–Karhunen–Lo`eve representation as in Definition 4.8. Then, for any p : [−π, π] → N c`adl`ag, the random function X t⋆

∫ e

=

iωt

p(ω) (∑

(−π,π]\{ω1 ,...,ωk }

) k ( p(ω ∑ℓ ) ωℓ ∑ ω ) φ ωj ⊗ φ ωj d Z ω + φ j ⊗ φ j ℓ (Z ωℓ − Z ω− )eitωℓ ℓ=1

j=1

j=1

ℓ

minimizes the mean squared error among all linear rank reductions of {X t } to a process {Yt } ∫π with representation Yt = −π eiωt Aω dω, where Aω ∈ B∞ with rank(Aω ) ⩽ p(ω), i.e., ∥X t − X t⋆ ∥2H ⩽ ∥X t − Yt ∥2H subject to the constraint rank(Aω ) ⩽ p(ω). The minimized error is given by ∫ k ( ∑ ( ∑ ω) ∑ ω ) ∥X t − X t⋆ ∥2H = ν j dω + νj ℓ . (−π,π]\{ω1 ,...,ωk } j> p(ω)

ℓ=1 j> p(ωℓ )

Note that the rank of Aω constraints the dimensions of H -valued processes with represen∫π tation −π eiωt Aω dω to a lower-dimensional subspace of H (see also [23]). Proof. Without loss of generality, we prove this for the case of one discontinuity at frequency ωo . By orthogonality of the two components of the representation, we find using Proposition 4.7 and Fubini’s theorem ∫ 2 ( 2 ( ) )     2 ∥X t − Yt ∥H =  I − Aω eiωt d Z ω  +  I − Aωo (Z ωo − Z ωo− )eitωo  (−π,π]\{ωo }

∫

π

=

H

H

(( ) ( )† ) tr I − Aω Fω dω I − Aω

−π

(( )( )( )† ) + tr I − Aωo Fωo − Fωo− I − Aω , Please cite this article as: A. van Delft and M. Eichler, A note on Herglotz’s theorem for time series on function spaces, Stochastic Processes and their Applications (2019), https://doi.org/10.1016/j.spa.2019.10.006.

A. van Delft and M. Eichler / Stochastic Processes and their Applications xxx (xxxx) xxx

17

from which it is straightforward to see that this is minimized by X t⋆ and the error is given by 2 ∫ ) ( ∑ ω   ⋆ 2 eiωt φ j ⊗ φ ωj d Z ω  ∥X t − X t ∥H =  (−π,π ]\{ωo }

H

j> p(ω)

( ∑ 2 )   φ ωj o ⊗ φ ωj o (Z ωo − Z ωo− )eitωo  + H j> p(ωo ) ∫ ( ∑ ) ( ∑ ω) ν ωj o . □ = ν j dω + (−π,π]\{ωo } j> p(ω)

j> p(ωo )

Remark 4.10 (Harmonic Functional Principal Component Analysis of Long-memory Processes). In analogy to classical time series, the covariance structure of a long-memory functional time series does not decay rapidly. Without loss of generality, assume the covariance structure of such a process satisfies Ch ∼ Bh 2d−1

0 < d < 0.5,

where B a strictly positive element of S1+ (H ). It is clear ∑that, for this type of process, the dependence structure does not decay rapidly enough for h∈Z |||C h ||| p < ∞ to hold. In order to understand what can be said about the properties of the spectral density operator, note that we can for simplicity mimic the behavior of such a process by considering the linear process Xt =

∞ ∑ ( ∏ j=0 0
k−1+d k

)

εt− j ,

where εt is H -valued white noise and hence by Theorem 3.7, the second order structure is ∫π given by C0ε = −π dF ε (ω) = 2π F0ε . Using the properties of the Gamma function, a standard argument shows that the filter applied to {ϵt } yields ∫ π ∫ π iωh X X eiωh (1 − e−iω )−2d dF ε (ω) e dF (ω) = Ch = −π

−π

and hence a density of the spectral measure F X at ω = 0 for d > 0 is not defined. Yet, since this has measure 0, we can, under the conditions of Theorem 4.5, define a harmonic principal component analysis as in Corollary 4.9 where the number of discontinuities is k = 0. That is, the optimal approximating process is given by ∫ p(ω) (∑ ) X t⋆ = eiωt φ ωj ⊗ φ ωj d Z ω (−π,π ]

j=1

and the minimized error is given by ∥X t − X t⋆ ∥2H =

∫

(∑ (−π,π]

j> p(ω)

) ν ωj dω.

Declaration of competing interest The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper. Acknowledgments This work has been supported in part by the Collaborative Research Center “Statistical modeling of nonlinear dynamic processes” (SFB 823, Project A1, C1, A7) of the German Research Foundation (DFG). The authors sincerely thank the Editor and two Referees for their constructive comments that helped to produce an improved revision of the original paper. Please cite this article as: A. van Delft and M. Eichler, A note on Herglotz’s theorem for time series on function spaces, Stochastic Processes and their Applications (2019), https://doi.org/10.1016/j.spa.2019.10.006.

18

A. van Delft and M. Eichler / Stochastic Processes and their Applications xxx (xxxx) xxx

Appendix A. Some background material In this section, we collect some definitions and concepts which are needed to formalize the construction of the measure, which was heuristically described in Section 3 and which is described in more detail in Appendix B. For more background on topology (on operator algebras) we refer to, e.g., [14,15,20] and [10] and for functional analysis to Conway [5] and Rudin [27]. Definition A.1 (Monoid). A set M together with a binary operation ◦ is called a monoid, (M, ◦) if the following axioms are satisfied (closure) for any a, b ∈ M, a ◦ b ∈ M; (associativity) for any a, b, c ∈ M, we have (a ◦ b) ◦ c = a ◦ (b ◦ c); (identity) there exists an e ∈ M such that, for any a ∈ M, e ◦ a = a ◦ e = a. Definition A.2 (Monoid Homomorphism). Let (M, ⋆) and (N , ◦) be two monoids with identity elements e M and e N , respectively. Then a function f : (M, ⋆) → (N , ◦) is a monoid homomorphism if it preserves the monoid operation and identity element, i.e., f (m ⋆ n) = f (m) ◦ f (n), for all m, n ∈ M and f (e M ) = e N . Definition A.3 (Topological Monoid). A topological monoid is a monoid (M, ◦) endowed with a topology τ such that the binary operation ◦ : M × M → M is continuous. For two topological monoids (X, ◦) and (Y, ⋆), we denote Hommon (X, Y ) as the set of all monoid homomorphisms X → Y . Definition A.4 (Initial Topology). Given a set X and an indexed family of topological spaces (Yi )i∈I with functions f i : X → Yi , the initial topology on X , τin , induced by the functions ( f i )i∈I is the coarsest topology on X such that each f i : (X, τin ) → Yi is continuous. Examples of initial topologies used in Appendix B: (i) Cartesian product endowed with the product topology τ p : Let (Yx )x∈X be an indexed family of ⋃sets. The cartesian product, denoted by Πx∈X Yx is the set of functions f : X → Yx such that f (x) ∈ Yx for each x ∈ X . The maps πx1 : Πx∈X Yx → Yx1 : f ↦→ f (x1 ), x1 ∈ X , are called the canonical coordinate projections. The product topology on Πx∈X Yx is the initial topology on Πx∈X Yx induced by the projection maps πx , x ∈ X . (ii) Sets of all functions (Y ) X with the product topology τ p : Let X and Y be sets. We denote (Y ) X the set of all functions from X to Y . We remark that by Tychonoff’s theorem, this set is compact if Y is compact. The set (Y ) X is the cartesian product where Yx = Y for every x ∈ X . In this case, the coordinate projections become evaluation maps, i.e., for each x1 ∈ X , the map πx1 : (Y ) X → Y takes each point f ∈ (Y ) X to its value, i.e., πx1 ( f ) = f (x1 ). The product topology on Y is known as the topology of pointwise convergence since, when endowed with this topology, a net of functions ( f α ) in (Y X , τ p ) converges to a function f ∈ Y X if and only if f α (x) → f (x) in Y for every x ∈ X . Please cite this article as: A. van Delft and M. Eichler, A note on Herglotz’s theorem for time series on function spaces, Stochastic Processes and their Applications (2019), https://doi.org/10.1016/j.spa.2019.10.006.

A. van Delft and M. Eichler / Stochastic Processes and their Applications xxx (xxxx) xxx

19

(iii) Subsets endowed with the subspace topology: Let (Y, τ ) be a topological space and let Yo ⊂ Y be a subset. The subspace topology, τYo , on Yo is the initial topology with respect to the inclusion map i : Yo → Y , i.e., the map i(y) = y for all y ∈ Yo . The topological space (Yo , τYo ) is called a topological subspace of (Y, τ ). Yo is a closed subspace of Y if Y \ Yo ∈ τ (i.e., the complement is open). A closed subspace of a compact topological space is compact (see e.g., Munk). Appendix B. On the construction of the measure In this section we provide some more detail of the construction of the measure, which was described heuristically in Section 3. The construction is achieved by means of an embedding of the cone into its bidual cone φ : L(H )+ → (L(H )+ )′′ , where (L(H )+ )′′ consists of all positive continuous linear functionals on the dual cone (L(H )+ )′ . To make this more precise, we mention some properties of cones. B.1. Dual pair of cones Definition B.1 (Cones and Some Properties). A subset C of a real vector space V is called a cone if it is closed under positive scalar multiplication, i.e., if λ > 0, c ∈ C, then λc ∈ C. It is called a convex cone if it is moreover closed under linear combinations, i.e., λ, β > 0, c1 , c2 ∈ C, then λc1 + βc2 ∈ C. A cone is called pointed if it contains the zero element, i.e., if it satisfies C ∩ −C = {0}, while it is called generating if C − C = V . We moreover need the notion of a topological dual cone, which we shall simply refer to as the dual cone in the subsequent sections. Definition B.2 (Topological Dual Cone). The topological dual cone (C)′ to a cone C of a real vector space V can be defined as the set {v ∈ V ′ : ⟨v, c⟩ ⩾ 0

∀c ∈ C}

where ⟨·, ·⟩ denotes the duality pairing between V and its topological dual V ′ . B.2. The embedding Given the duality pairing in (3.2) of the underlying spaces L(H )† and S1 (H )† , we can identify the cone S1 (H )+ as the dual cone of L(H )+ . Proposition B.3.

The dual cone of L(H )+ , (L(H )+ )′ , is given by †

S1+ (H ) = {A ∈ S1 (H ) : tr(AB) ⩾ 0

∀B ∈ L(H )+ }.

Proof of Proposition B.3. We start by remarking that positive elements on a C ⋆ - algebra form a closed convex cone {a ⋆ a : a ∈ C ⋆ } (see e.g., [5]). Since elements of L(H )+ have a positive square root, L(H )+ forms a pointed convex cone in the Banach algebra of bounded linear operators. By the spectral theorem it is moreover generating the space L(H )† . With a similar argument, it is straightforward to verify that C V = S1 (H )+ is a pointed convex cone in the Banach space V = (S1 (H )† , ||| · |||1 ). Its topological dual is the Banach space V ′ = (L(H )† , ||| · |||L ) with cone C V ′ := (L(H )+ , ||| · |||L ) in V ′ . We can naturally identify Please cite this article as: A. van Delft and M. Eichler, A note on Herglotz’s theorem for time series on function spaces, Stochastic Processes and their Applications (2019), https://doi.org/10.1016/j.spa.2019.10.006.

20

A. van Delft and M. Eichler / Stochastic Processes and their Applications xxx (xxxx) xxx

an element of a Banach space with an element from its bidual by means of a canonical embedding (see e.g., [14]). More specifically, we have that V canonically embeds into (V ′ )′ i.e., V ⊆ (V ′ )′ → C V ⊆ (C V ′ )′ . Since the cone C V ′ is closed, the Hahn–Banach separation theorem implies that C V = (C V ′ )′ , which identifies the cone of non-negative trace class operators as the dual cone of the non-negative bounded linear operators. □ To identify L(H )+ with its image into its bidual cone (L(H )+ )′′ , we consider an injection B ↦→ (A ↦→ tr(AB)), A ∈ S1 (H )+ , B ∈ L(H )+ . The image of the mapping φ is therefore simply given by +

φ(L(H )+ ) = {φ B ∈ (R+ ) S1 (H ) : B ∈ L(H )+ },

(B.1)

where φ B : S1 (H ) → R is given by φ B (A) = tr(B A) ⩾ 0. The following result is essential in order to uniquely identify L(H )+ with a family of R+ -valued measures and to use φ(L(H )+ ) in order to construct the compactification of L(H )+ . +

+

Proposition B.4. The set φ(L(H )+ ) coincides with the set of all positive continuous linear functionals S1+ (H ) → R+ , that is, φ(L(H )+ ) = (S1+ (H ))′ . Proof of Proposition B.4. Recall that L(H )† is the topological dual space of S1 (H )† where the pairing is given by (3.2). A functional on a Banach space V is said to be positive if f (v) ⩾ 0 for all non-negative elements v ∈ V . It can be shown that all positive functionals on a Banach space are continuous [22, Prop I.7]. Since (S1 (H )† , ||| · |||1 ) is a Banach space this implies that all f : S1 (H )+ → R+ must be continuous. But since L(H )† consists of all continuous linear functionals on S1 (H )† , all continuous positive functionals on S1 (H )+ must be of the form (B.1). □ Given this is in place, we can now obtain that φ provides an isomorphism between L(H )+ and Hommon (S1 (H )+ , R+ ), where the isometry property of the spaces of self-adjoint operators continues to hold when restricted to the cones. Theorem B.5. Let Hommon (S1 (H )+ , R+ ) denote the set of all monoid homomorphisms + S1 (H ) → R+ . Then φ in (B.1) is an isometric isomorphism between L(H )+ and Hommon (S1 (H )+ , R+ ), i.e., φ(L(H )+ ) = Hommon (S1 (H )+ , R+ ) ∼ =m L(H )+ .

(B.2)

Proof. While it is immediate the mapping is injective, note that Proposition B.4 implies that for any linear functional ϕ : S1 (H )+ → R+ there exists an element B ∈ L(H )+ , such that φ(B) = ϕ. We therefore have established that φ is a bijective map between L(H )+ and (S1 (H )+ )′ . It is easily verified using the definition (see Appendix A) that the cones L(H )+ , S1 (H )+ , R+ , which are not vector spaces, have the algebraic structure of topological monoids with respect to addition. It is moreover straightforward to verify that all functionals in (S1 (H )+ )′ preserve the monoid structure between S1 (H )+ and R+ . Indeed, since any element in (S1 (H )+ )′ is given by a functional of the form φ B : S1 (H )+ → R+ , A ↦→ tr(B A), B ∈ L(H )+ , we obtain from linearity of the trace that φ B (A1 + A2 ) = φ B (A1 ) + φ B (A2 ), while φ B (O H ) = 0, for any B ∈ L(H )+ and A1 , A2 ∈ S1 (H )+ . By Proposition B.4, φ(L(H )+ ) consists of all such monoid homomorphisms, i.e., φ(L(H )+ ) = Hommon (S1 (H )+ , R+ ), where φ(L(H )+ ) has itself a monoid structure, since the axioms of closure, associativity and identity are easily checked (See Definition A.1). Hence, we find that φ is an isomorphism between the monoids L(H )+ and Please cite this article as: A. van Delft and M. Eichler, A note on Herglotz’s theorem for time series on function spaces, Stochastic Processes and their Applications (2019), https://doi.org/10.1016/j.spa.2019.10.006.

A. van Delft and M. Eichler / Stochastic Processes and their Applications xxx (xxxx) xxx

21

Hommon (S1 (H )+ , R+ ). It remains to verify the isometry property ∥B∥∞ = ∥φ(B)∥ for elements restricted to the cone L(H )+ . For completeness, we provide the argument. Note that since L(H )+ generates L(H )† , the norm ∥B∥∞ is trivially the same in L(H )+ and L(H )† . Moreover, ∥φ(B)∥ = sup{|tr(B A)| : |||A||| = 1, A ∈ S1 (H )† } = sup{|tr(B A)| : |||A||| = 1, A ∈ S1 (H )+ }. Now, it is immediate that ∥φ(B)∥ ⩽ ∥B∥∞ |||A|||1 ⩽ ∥B∥∞ . In order to show the reverse inequality, note that for nonzero elements in B ∈ L(H )† we can write the operator norm as ∥B∥∞ = sup{|⟨Bx, y⟩| : ∥y∥ = ∥x∥ = 1}, where we set y = Bx/∥Bx∥. From this we obtain ∥B∥∞ =

sup

|⟨Bx, y⟩| ⩽

∥y∥=∥x∥=1

sup

|tr(B A)| = ∥φ(B)∥

A∈S1 (H )+ ,|||A|||=1

for all B ∈ L(H )+ . The result now follows.

□

B.3. Compactification of L(H )+ -valued measures Proposition B.6. The set Hommon (S1 (H )+ , R+ ∞ ) is a compact topological monoid. Furthermore, addition is continuous with respect to the topology of pointwise convergence inherited S1 (H )+ from (R+ . This coincides with the ultraweak operator topology on L(H )+ . ∞) Proof. We shall make use of some concepts which are collected in Appendix A. We start by showing that the set Hommon (S1 (H )+ , R+ ∞ ) is a compact topological monoid of monoid homomorphisms from S1 (H )+ into R+ . More specifically, it consists of all positive continuous ∞ linear functionals S1 (H )+ → R+ by Proposition B.4. To see that this is a compact topological ∞ monoid, recall that S1 (H )+ , R+ are additive topological monoids, i.e., addition is continuous and the sets admit a zero element O H , and 0, respectively. Similarly, the compact set R+ ∞ can be viewed as a compact additive topological monoid, where ∞ is also treated as a zero element, i.e., for all x ∈ R+ ∞ , x + ∞ = ∞ + x = ∞. The set Hommon (S1 (H )+ , R+ ∞)

(B.3) + + S1 (H )

is closed when viewed as a subset of the set (R∞ ) , the set of all functions from S1 (H )+ + S (H ) + 1 is compact since R+ into R+ ∞ is compact. Being a closed ∞ . By Tychonoff’s theorem, (R∞ ) subset of a compact topological space, it is itself compact and the result now follows. S (H )+ For the second part, note that the initial topology on (R+ )1 , implies that the evaluation ∞ S1 (H )+ + are continuous. Being a subset mappings π A ( f ) = f (A), f ∈ (R+ ) for all A ∈ S (H ) 1 ∞ + + S1 (H ) this implies for all the functionals φ B in (B.3) that π A (φ B ) = φ B (A) = tr(B A), of (R∞ ) are continuous for all A ∈ S1 (H )+ . Hence, we inherit the topology of pointwise convergence. Addition is therefore continuous with respect to this topology. Note that due to the form of the functionals φ B , this notion of convergence coincides exactly with convergence in the ultraweak topology as given in Definition 3.4. □ Appendix C. Proofs of Section 4 For the proof of Proposition 4.7, we require the following two auxiliary results. Proposition C.1. Define the Bochner space B∞ = L 2S∞ (H ) ([−π, π], µF ) of all strongly measurable functions U : [−π, π] → S∞ (H ) such that ∫ 2 ∥U ∥B∞ = |||Uω |||2∞ dµ(ω) < ∞. (C.1) Π

Please cite this article as: A. van Delft and M. Eichler, A note on Herglotz’s theorem for time series on function spaces, Stochastic Processes and their Applications (2019), https://doi.org/10.1016/j.spa.2019.10.006.

22

A. van Delft and M. Eichler / Stochastic Processes and their Applications xxx (xxxx) xxx

with µF (E) =

∫ dµF (ω)

(C.2)

E

for all Borel sets E ⊆ [−π, π]. Then for U ∈ B∞ , the integral ∫ π Uω d Z ω −π

exists and belongs to H . Proof. The Proposition follows directly from section B 2.3 of van Delft and Eichler [9] by replacing Lemma B 2.5 of the corresponding paper with the following auxiliary lemma. □ ∫π Lemma C.2. Let X t be a functional process with spectral representation X t = −π∫ eiωt d Z ω for ω some functional orthogonal increment process Z ω that satisfies Etr(Z ω ⊗ Z ω ) = −π dµF (α). Then for U1 , U2 ∈ S∞ (HC ) and α, β ∈ [−π, π] ( [∫ α∧β ] ) † ⟨U1 Z α , U2 Z β ⟩H = tr U1 dF (ω) U2 (i) −π

and ∥U1 Z α ∥2H ⩽ |||U1 |||2∞

∫

α

dµF (ω).

(ii)

−π

Consequently, for ω1 > ω2 ⩾ ω3 > ω4 ⟨U1 (Z ω1 − Z ω2 ), U2 (Z ω3 − Z ω4 )⟩H = 0 and ∥U1 (Z ω1 −

Z ω2 )∥2H

⩽

|||U1 |||2∞ tr(F (ω2 )

− F (ω1 )) =

|||U1 |||2∞

∫

ω2

ω1

µF (ω).

Proof. Using (4.2) and the invariance of the trace under cyclical permutations †

⟨U1 Z α , U2 Z β ⟩H = E⟨U2 U1 Z α , Z β ⟩ ( † ) = Etr U2 U1 (Z α ⊗ Z β ) ( †) = Etr U1 (Z α ⊗ Z β )U2 ∫ ] ) ( [ α∧β † = tr U1 dF (ω) U2 . −π

Secondly, we note that by Cauchy–Schwarz inequality and (4.2) ⟨U1 Z α , U2 Z β ⟩H ⩽ ⩽ |||U1 |||∞ |||U2 |||∞ E∥Z α ∥2 ∥Z β ∥2 ⩽ |||U1 |||∞ |||U2 |||∞ Etr(Z α∧β ⊗ Z α∧β ) ∫ α∧β ⩽ |||U1 |||∞ |||U2 |||∞ dµF (ω) < ∞.

□

−π

Proof of Proposition 4.7. We prove the case for one discontinuity at ωo as the argument for finitely many discontinuities is similar. First we remark that the left limit Z ωo− is well-defined in H for any non-decreasing sequence {ωn } ↑ ωo . The limit exists because Please cite this article as: A. van Delft and M. Eichler, A note on Herglotz’s theorem for time series on function spaces, Stochastic Processes and their Applications (2019), https://doi.org/10.1016/j.spa.2019.10.006.

23

A. van Delft and M. Eichler / Stochastic Processes and their Applications xxx (xxxx) xxx

∥Z ωm − Z ωn ∥2H = |µ F (ωm ) − µ F (ωn )| → 0 as m, n → ∞ and the limit is unique for all {νn } ↑ ωo since ∥Z νn − Z ωn ∥2H = |µ F (νn ) − µ F (ωn )| → 0 as n → ∞. Under the conditions of Theorem 4.5, {X t } has a well-defined spectral representation, which can alternatively be written as ∫ ∫ Xt = eitω d Z ω + eitω d Z ω , (C.3) (−π,π ]\(ωo −δ,ωo +δ]

(ωo −δ,ωo δ]

for 0 < δ < π − |ωo |. It can be directly observed that, by orthogonality of these two integrals and continuity of the inner product, the mean square limit of these two terms must be orthogonal. We can therefore treat their respective limits separately. It is straightforward to see that ∥eit· 1(−π,π]\(ωo −δ,ωo +δ] (·) − eit· 1(−π,π ]\{ωo } (·)∥H¯ → 0 as δ → 0, and thus for the first term of (C.3), we find ∫ ∫  eitω d Z ω − 

(−π,π]\{ωo }

(−π,π]\(ωo −δ,ωo +δ]

2  eitω d Z ω  → 0 as δ → 0. H

For the second term in (C.3), Minkowski’s inequality implies ∫ 2   eitω d Z ω − (Z ωo − Z ωo− )eitωo   H (ωo −δ,ωo +δ] ∫ 2   ⩽ eit(ω−ωo ) 1(ωo −δ,ωo +δ] (ω)d Z ω  H

(−π,π ]

 2   + (Z ωo +δ − Z ωo −δ )eitω1 − (Z ωo − Z ωo− )eitωo  .

(C.4)

H

For the first term in (C.4), the isometric mapping established in Theorem 4.4 together with continuity of the function eit· on R imply ∫ 2   eit(ω−ωo ) 1(ωo −δ,ωo +δ] (ω)d Z ω   H

(−π,π]

⩽

(

sup

ωo −δ⩽ω⩽ωo +δ

[ ])1/2 → 0 as δ → 0. |eit(ω−ωo ) | µF (ωo + δ) − µF (ωo − δ)

By right-continuity of the functional-valued increment process {Z ω }, the second term in (C.4) converges to 0 as δ → 0. Hence, with probability one, ∫ (C.5) Xt = eitω d Z ω + (Z ωo − Z ωo− )eitωo . (−π,π ]\{ωo }

Finally, since the left limit Z ωo− is well-defined, Theorem 4.4 implies [ ] var(Z ωo − Z ωo− ) = lim E (Z ωo − Z ωn ) ⊗ (Z ωo − Z ωn ) = F (ωo ) − F (ωo− ). ωn ↑ωo

□

References [1] A. Aue, A. van Delft, Testing for stationarity of functional time series in the frequency domain, Ann. Statist. (2019) doi:10.1214/19-AOS1895, forthcoming. [2] V.I. Bogachev, Measure Theory, Vol. I, Springer, New York, 2006. [3] D. Brillinger, Time Series: Data Analysis and Theory, McGraw Hill, New York, 1981. [4] P. Brockwell, R. Davis, Time Series: Theory and Methods, Springer, New York, 1991. [5] J.B. Conway, A Course in Functional Analysis, Springer-Verlag, New York, 1990. [6] H. Cramér, On harmonic analysis in certain functional spaces, Ark. Mat., Astron. Fys. 28B (1942) 1–7. Please cite this article as: A. van Delft and M. Eichler, A note on Herglotz’s theorem for time series on function spaces, Stochastic Processes and their Applications (2019), https://doi.org/10.1016/j.spa.2019.10.006.

24

A. van Delft and M. Eichler / Stochastic Processes and their Applications xxx (xxxx) xxx

[7] A. van Delft, V. Characiejus, H. Dette, A nonparametric test for stationarity in functional time series, Statist. Sinica (2019) doi:10.5705/ss.202018.0320, forthcoming. [8] A. van Delft, H. Dette, A similarity measure for second order properties of non-stationary functional time series with applications to clustering and testing, 2018, arXiv:1810.08292. [9] A. van Delft, M. Eichler, Locally stationary functional time series, Electron. J. Stat. 12 (1) (2018) 107–170. [10] J.M. Erman, Functional Analysis and Operator Algebras: An Introduction, Portland State University, 2015. [11] H. Glockner, Positive definite functions on infinite-dimensional convex cones, Mem. Amer. Math. Soc. 166 (789) (2003). [12] S. Hörmann, L. Kidzi´nski, M. Hallin, Dynamic functional principal components, R. Stat. Soc. Ser. B 77 (2015) 319–348. [13] S. Hörmann, P. Kokoszka, G. Nisol, Testing for periodicity in functional time series, Ann. Statist. 46 (6A) (2018) 2960–2984. [14] R.V. Kadison, J.R. Ringrose, Fundamentals of the Theory of Operator Algebras. Vol I, in: Graduate Studies in Mathematics, vol. 15, American Mathematical Society, Providence, RI, 1997. [15] R.V. Kadison, J.R. Ringrose, Fundamentals of the Theory of Operator Algebras. Vol II, in: Graduate Studies in Mathematics, vol. 16, American Mathematical Society, Providence, RI, 1997. [16] K. Karhunen, Über lineare Methoden in der Wahrscheinlichkeitsrechnung, Ann. Acad. Sci. Fenn. AI 37 (1947) 1–79. [17] A. Leucht, E. Paporoditis, T. Sapatinas, Testing equality of spectral density operators for functional linear processes, 2018, arXiv:1804.03366. [18] M. Loève, Fonctions aléatoires du second ordre. Supplement to P. lévy, in: Processus Stochastiques et Mouvement Brownian, Gauthier-Villars, Paris, 1948. [19] T.S. McElroy, D.N. Politis, Spectral density and spectral distribution inference for long memory time series via fixed-b asymptotics, J. Econometrics 128 (2014) 211–225. [20] J.R. Munkres, Topology, second ed., Prentice Hall, Upper Saddle River, NJ, 2000. [21] K.-H. Neeb, Holomorphy and Convexity in Lie theory, in: De Gruyter Expositions in Mathematics, 28, Berlin, 2000. [22] K.-H. Neeb, Operator-valued positive definite kernels on tubes, Monatsh. Math. 126 (1998) 125–160. [23] V. Panaretos, S. Tavakoli, Cramér–Karhunen–Loève representation and harmonic principal component analysis of functional time series, Stochastic Process. Appl. 123 (2013) 2779–2807. [24] V. Panaretos, S. Tavakoli, Fourier analysis of stationary time series in function space, Ann. Statist. 41 (2) (2013) 568–603. [25] T. Pham, V. Panaretos, Methodology and convergence rates for functional time series regression, Statist. Sinica 28 (2018) 2521–2539. [26] W. Rudin, Real and Complex Analysis, third ed., McGraw Hill, New York, 1987. [27] W. Rudin, Functiona Analysis, second ed., McGraw Hill, New York, 1991. [28] S. Tavakoli, Fourier Analysis of Functional Time Series with Applications to DNA Dynamics, EPFL Ph.D. thesis, 2014. [29] F. Ziel, R. Steinert, Electricity price forecasting using sale and purchase curves: The X-model, Energery Econ. 59 (2016) 435–455.

Please cite this article as: A. van Delft and M. Eichler, A note on Herglotz’s theorem for time series on function spaces, Stochastic Processes and their Applications (2019), https://doi.org/10.1016/j.spa.2019.10.006.