Noise in unspecified, non-linear time series

J O U R N A L OF

Econometrics ELSEVIER

Journal of Econometrics 7811997) 229-255

Noise in unspecified, non-linear time series George G. Szpiro The Israeli Centerfor Academic ,Studies, Kiriat Ono, Israel

(Received November 1993; final version received March 1996)

Abstract In this paper a method is developed which allows the determination of the noise level (the size of external shocks) which is present in a linear or non-linear autoregressive time series. Nothing need to be known about the functional form or about the; lag structure of the underlying model. The technique regards the time series as a hypersurface, lying in a higher dimensional embedding space, and makes use of the correlation integral, as introduced in the physical sciences. Uniformly distributed noise is considered at first, then the technique is generalized to normal and other distributions. And even if the distribution of the noise is unknown, one can still make comparisons between the intensities of noise in different time series. The method is put to a test with empirically assessed stock market data. Key words: Time series; Noise, Correlation integral; Chaos J E L classification: C22

I. Introduction

Noise, as a "diversified array of unrelated causal elements" plays an important role in such diverse fields a.~ finance, econometrics, and macroeconomics (Black, 1986). It is what makes tra~ling in financial markets possible, it may be responsible for business cycles, it[may determine the ch:~kacterof inflation. Noise also

I would like to thank B~qke IgeBaron and seminar participants at the Hebrew University, the University of Pennsylvania, the University of Geneva, and the University of Ziirich for helpful discussions, and A. Ronald Gallant, the editor of the Journal, for very helpful suggestions. Any remaining errors remain my own. Author's address: P.O.B. 6278, Jerusalem 91060, Israel. Email: [email protected]. 0304-4076/97/$17.00 ~) 1997 Elsevier Science S.A. All rights reserved PII S 0 3 0 4 - 4 0 7 6 ( 9 6 ) 0 0 0 1 0 - 3

230

G.G. Szpiro /Journal of Econometrics 78 (1997) 229-255

plays an important role in the sludy of an economy's reaction to new pieces of information, to market interventions, or to other external shocks (Engle et al., 1990), and is at the core of the impulse-response (or error-shock) methodology, which studies how perturbations are propagated in an economy (Gallant et al., 1993). These are only some of the issues raised by noise. Obviously, it would be important to get a quantitative grasp of this important factor. Usually, noise is quantified as the residual variance of linear or non-linear models, whose functional form has been specified a priori. In this paper we develop a method which allows the determination of the noise level (the size of external shocks) in a completely unspecified model. It may be noted that in the physical sciences a distinction is usually made between measurement noise and dynamical noise. In economics the latter is far more important, even though assessment problems, like roundoff errors, for example, may play a role in aggregation or under conditions of inflation (for an effect of measurement noise on time series see, for example, Szpiro, 1993a). Let us consider a non-linear autoregressive time series of the kind y, =1~ + ~ty~-t +)'Y~-2 + "'" + p y t ~ i y ~ - ~ + "" + r y e - p + et,

(I)

where Yk is the observed variable in period k, ~t, fl, 7, 6.... are the model's parameters, and ~:, is independent, identically distributed noise. We assume in this paper n~t only that the parameters are unknown, but. that we also ignore the functional form and the lag structure of the underlying model. It will be shown that in spite of this dearth of information, the level of noise can be determined to within a few percentage points, if only its distribution is knowr:. And even if the distribution is m~known, the method to be presented in this paper still allows precise comparisons of the intensity of noise in different time series. Apart from interest in the size of noise per se, the method proposed in this paper also has some implications for the estimation of the underlying model: it allows determination of the 'unexplained' variance, without actually knowing the model itself. The proposed method uses the correlation integral, which was developed for the physical sciences by Grassberger and Procaccia (I983a) in order to determine the dimension of so-called strange attractors (limit cycles to differential or difference equations, which are stable, but non-periodic). The correlation integral was used by Brock (1986), Brock and Sayers (1988),' Frank and Stengos (1988), Scheinkman and LeBaron (1989), Hsieh (1989), Ramscy et ai. (1990), and others, to determine, for example, whether a low-dimensional strange attractor underlies the dynamics of stock prices (see Ramsey et al., 1990, Hsieh, 1991 for more references). Since an autoregressive time series, as in Eq. (1), can be considered a difference equation in one variable, with noise, the ~:eehnique is also applicable to such cases. We will first assume that noise is uniformly distributed in the interval [ - M, + M] and derive ~ method to estimate M. Then we adapt the method to estimate the standard deviation of non-uniform noise, again without any knowledge whatsoever about the lag structure and the

G.G. Szpiro /Jo~trnal of Econometrics 78 (1997) 229-255

231

non-linearities involved. Finally, we show that the technique allows qualitative and quantitative comparisons about the intensity of noise, even if we ignore the characteristics of the noise's distribution. The method has been applied in a different context in Szpiro (1993b). An earlier paper (Szpiro, 1991) presented a less sophisticated model, and dealt only ~vith linear time series. In the following section we introduce the notions of "strange attractor" and 'phase space', and present the Grassberger-Procaccia method to estimate fractal (non-integer) dimensions of dynamical systems. Section 3 shows how this method caa be used to measure noise that is uniforml~distributed, and Section 4 applies the technique to non-linear time series. Section 5 extends the technique to normaUy distributed shocks, and fat-tailed noise is examined in Section 6. While Sections 3-6 assume that the noise's distribtltion is known, Section 7 dispenses with this requirement. Sectiol~ 8 applies the technique to an empirically assessed economic time series, namely the daily observations of the Standard and Poors 500 index for the years 1968-1991. Section 9 summarizes the paper. Appendix A describes the computational steps for the implementation of the method, and Appendix B lists the source code.

2. The Grassberger-Procaccia dimension estimate A strange attractor is a stable, but non-periodic limit cycle of a differential equation. The trajectories of strange attractors cannot be analyzed with traditional tools and even the definition of dimension had to be broadened to include non-integer dimensions. Such so-called 'fractar dimensions are by now considered quite common for physical systems or objects and, in order to determine their value, Grassberger and Procaccia (1983a) proposed th~ use of the correlation integral, which will be described now. First, we introduce the notion of phase space. In order to investigate the nature of a dynamical syst~rn, e.g., a pendulum, it is useful to observe the development over time of the system,s characteristics (the variables themselves, and their time derivatives). This development can be plotted in the so-called phase space, whose axes are the variable x, and x', x", etc. For difference equations the axes are replaced by xt, x,_ t, x,_ z, etc. The Grassberger-Procaccia method works as follows: the observations deriving from the dynamical system with the unknown fractal dimension d are embedded in a phase space of sufficiently high dimension m. A reference point from among the observations is chosen, and the number of points which lie within a sphere of radius r around this reference poi~,(the nearest neighbours) are counted. Each observation is then used, in turn, as a reference point, to compute the so-called correlation int,~gral, Cm(r),

(2)

232

G.G. Szpiro /Journal of Econometrics 78 (1997) 229--255

wher$ N is the total number of observations, # (.) stands for the number of obsel vatic?as j which satisfy the condition within the parantheses (j ~ i), and I. It"° denotes the Euclidean norm in m-dimensional space. Then the radius is increased, and Cm(r) is recomputed. Fc,.r a system with dimension d, Cm(r) grows as r d with increasing r, on condition that the d.'-'men.sion of the embedding space is greater than d: (3)

Cm(r) "" :o"~.

Hence, the attractor's dimension is the slope of the regression equation of Ln C ( r ) versus Ln r,

Ln C(r) = fl + ,4 Ln r,

(4)

If the dime.nsion of the embedding spat.; m is chosen too small (i.e., smaller than d), then Cm(r) grows only as r" .with inclcasing r, " C,,(r) .,. :~r",

']

(5)

L

since the observations deriving from the dynamical system ill! ~.ilc,whole space. To sum up, the dimension of the system is determined by computing the correlation coefficient and running the regression (4) in successively higher embedding dimensions: the regression coefficient of Eq. (4) will converge towards 6 as m increases.

3. Dynamical systems with noise The Grassberger-Procaccia technique was originally developed fo? noise-free, completely deterministic systems, but Ben-Mizrac~.i et al. (1984) showed that the correlation integral allows the determination of the dynamical system's dimension, even when noise is present( when the embedding dimension is sufficiently large, C,,(r) will scale as r n' for :~mail values o f t (since the observa6ons are space filling at such lengths), and as r ~ thereafter, t It is well-known, however, that the break in the slope of Eq. (4) gives only a rough indication, at best, of the order of magnitude of the noise level, and we now propose a method to estimate the level of noise in a system to a much more precise degree. In this section and the next, we consider noise that is uniformly distributed between - M and M. Since each of the system's variables contains noise which is independent of the other variables' noise, observations are distributed within m-dimensional boxes around the true (noise-free) values. The edge-lengths of such boxes correspond to the noise levels of the various variables. When analyzing a one-dimensional systen,, embedded in m-dimensional phase space, one deals with m-dirhensional 'For large values of r. how.,z:'er,the slope levelsout becauseof the finite size of the attractor.

G.G. Szpiro /Journal of Econometrics 78 (1997) 229-255

233

boxes, where m - 1 edges are of equal length, since there is only one kind of noise. As long as the distances r stay within the noise level, observations are space, filling in all embeddieg dimensions. Hence, in m-dimensional space, the |raction 6f all observations j that satisfy

ixi -- xj] ('') ~< r (j =~ i),

-,....

(6)

for reference point ~!,can be measured by the proportion of Vm(r), the volume of the m-dimensional'sphere around i with radius r, to Sin, the volume of the attractor, together with its noise:

!

Vm(r)

N - 1 ['# Ixi -- xjl (") ~< r = S---~--

(7)

Since C,.(r) is the mean of the above expression for all i, and since Vm(r) does not depend on the reference point if r is small, Cm(r) =

Vm(r)

(8)

Sm

Now, F~(r) can also be seen as the projection of a cylinder in (m + l)-dimensional embedding space, with base Vm(r), into m-dimensional embedding space. The height of the cylinder is 2M, since the additional variable's noise is ~niformly distributed around the attractor, within the interval [ - M, + M]. H~:,nce, the cylinder's volume is 2M V,,(r). The number of observations in Vm(r) and~'~n the cylinder are the sa:ne (as long as the embedding dimension is sufficiently large), since the former are simply projections onto m-dimensional space of the latter. Hence,

Vm(r)

2 M Vm(r) Sm+ I "

Sm

(9)

where Sin+ 1 is the volume of the attractor, together with its noise, in On + 1)dimensional space. More generally,

Vm(r) sm

=

(2M)m-nVm(r) s.

(lO)

Hence, the proportion of observations that lie within a sphere in m-dimensional space, to those that lie within an n-dimensional sphere (m > n), is given by

C,,,~rt C.(r)

V.,(r) (2M)m-"V#(r)"

(I I)

234

G.G. S-piro

.hmrna101' E¢ommletric.~

78 (1997) 229 255

Since t he vol u me of an m-dimensional spherc is given by ( Smi mow. 1968. V ol. 2) f

I'~ 1.,2

/

~

{_,err-_

m(m

-

r"

if m even.

2 ) -.- 2

V , , ( r ) = ] 9~,,- ,I -'rto,,-- i~,

(12)

17--- . . . . I.... ~. m l m - 2 ) . . . I

if m odd.

the proportions of observations, which we call ih,.,(r), are

q,,,.,,(r)=-~-:.=t¢ ......

.

113)

where //zr,~,,,- .... J)2 n( n _ _ 2 ) . . . , ~

m(m-2)..(

if m odd. n even. I

i

n)

I, ...... =

-

m(m-2)..-ln+2)

/re \''

.... "-'

,,(,,-

i f m . n e,en. o r m , n odd. (14)

2)..-I

m(m - 2) ... 2

if m even, n odd.

(For example, k_,.~ = 0.7854, /:3.2 = 0.6667. etc.) Using Eq. (13), homogenous regressions can be run for st, itably small values of r. (I 5)

~h..,,( r) = ";,..,,r'" - "

and the estimated noise level, M,,,. can now be computed as (" h ...... y

Mlr) = \ ~ /

'"' - "'

.

116)

It may be noted in passing, that the expression i I ....... _ t(r) is closely related to the "Kolmogorov entropy', K , . , , ( r ) , as approximated by Grassberger and Procaccia (1983b). which is generally used to describ." how fast nearby trajectories of a dynamical system diverge. K,.,,,_ ,(,') = Ln ( ~ ) .

(17)

The derivation so far holds for reference points in the center of the 'noise box' and for small values of r. However. since reference points are distributed across the whole width of the box, edge effects must be taken into account (Brandsffiter and Swinney, 1987), and the equations must be adapted accordingly. Let us

(;.G. S-piro .hmrmd 0I Etommzetric.~ 7,'¢ (1997; 229 255

235

assume for simplicity t h a t , = m - !. (We drop the subscript n in what follows.) In this case such a correction inw)lves a term containing (r'M)-':-" )l,l(r)

Kll I

/-ii I

= ,~-," + ~,~_,1 .

(18)

and homogenous regressions can be run with dependent variable ii,,(rl, and independent variables r and r-': -= ,',,ir + ~,,ir2.

Ill.(r)

(19)

Two estimates of the noise level M can then be computed for each embedding dimension separately as h"m

,'fl, . . . ;',,i . .

i---I f /- , = X/ / --'~.l, ,'-,,,

(20)

(We will ignore '~!_,. since the term containing r-" also includes influences other than edge effects (see below), and since ;',,i provides :dl the necessary information.) By alternatively adding and subtracting 2.57 standard errors of the ,'-estimates in regression 120). an approximate 99% conlidcnce interval can be determined for the estimated noise level. [ !

=

Km h',ll ] , ;'l,, + 2.57a o. ,'l, - -K-.57a~..

.

121)

At the outset it is not clear which range oft-values (r,,,~. to r,,~,) should be used for the regression: r . , . should be large enot,gh so that a sulticient number of observations hdls within the sphere with that radius, and rr,..,, should be smaller than M , while M is not known beforehand• The appropriate r.,~. can be determined by inspection, and the problem concerning r,.... is also easily resolved: as long as r.,,, is not substantially larger than the noise-level, the term containing 1--' in regression (18) picks up the effects that arise when part of the sphere lies ot, tside the noise box. A decision must also be made concerning the embedding dimensions whi,.n arc to be used. if the system crosses itself in space, superimposed cylinders project onto the same (m - I )-dimensional sphere, thus invalidating the method. However, as Whitney (1944) has shown, embedding a d-dimensional system into an m-dimensional space, where m/> 2d + I. is sufficient to guarantee that no self-intersections of the system occur (lower-dimensional embedding spaces may also suffice). Hence, our method will give the correct result in a sufficiently high-dimensional space. On a practical level, since the dimension of the dynamical system is usually unknown, the method should be used for si .... : F o r cml'~dding dimensions 2 and 3, ~.,, can be compnted' ;.2 = - ~, 2.~ = - ~. We o m i t the details.

because they arc of no relevancyin what follows.

G.G. Srpiro /Journal ~!]"Econometrics 78 (1997) 229-255

236

m = 2, 3, 4 .... In the beginning, the noise estimates will be too high because the superimposed m-dimensional cylinders project onto the same ( m - l)-dimensional sphere. With increasing embedding dimension the estimates decrease towards the true noise level, and a correct estimate has been r~,a,"hed when the results no longer change. Fig. I illustrates the method. Let us consider a one-dimensional system with noise, embedded in two-dimensional phase space. The situation can be visualized as a line cutting through the plane, with a 'noise-band' of width M on either side. Embedded in three-dimensional phase space, the system can be seen as a box of edge size 2M, winding through space. The line together with its noise band is simply the downward projection of the three-dimensional box into two-dimensional phase space. Hence, the observations in two-dimensional space found in a circle of radius r around a reference point, C2(r), are the downward projection of the observations in three-dimensional space contained in the cylinder above C2(r). On the other hand, C3(r), the number of nearestneighbors in three-dimensional space, is represented by the observations which fall within a ball of radius r around the reference point (not shown in the figure). Hence, for uniformly distributed noise, the proportion of observations lying within the ball, to the observations lying within the cylinder, permits computation of the cylinder's height, 2M, by way of Eq. (16), respectively (19).

Xt'2

xt

/

~

4

r

onto sphm e

Fig. !. Schematicrepresentationof the projectionofdata points from an m-dimensionalcylinder to an (m - I)-dimensionalsphere.

G.G. S:piro Journal of Econometrics 78 (1997) 229 255

237

4. Time-series For m > p, a structure as in Eq. (i) but without the error term. can be regarded as a p-dimensional hypersurface in m-dimensional embedding space. Adding noise, i.e., external shocks ~:,, gives "thickness" to the hypersurface. 3 We shall call the 'thickened" hyperplane the "hyperbody'. Increasing the dimension of the embedding space, adds the same noise to the hyperbody in the new dimension, i.e. the same thickness, orthogonal to the lower-dimensional hyperbody. Hence, we have the same situation as described in the last section, and can use the ratio-of-observations method to determine M. F o r simplicity, let n again equal m - I. Using Eq. (19), we can run homogenous regressions, with the radius r as the independent variable, and the proportion of observations, q,,(r), as the dependent variable. The estimates for the noise level arc then given as in Eq. (20}.4 in Szpiro (1991) a simpler version of the technique was presented, and an example of a linear time series was investigated, s Here we present, in Table l, an example of a non-linear time series, .l't = It + ~.l'{t- I + '}')'~-4 + [~)'~--2.1"t--t5 + t;t,

(22)

where the parameters were chosen as random numbers between zero and one, and ~:t is noise, uniformly distributed in [ - 0.200, + 0.200]. A series of length 500 was used, corresponding to, say, two years of daily observations of a variable. The top panel of the table sets out, first of all, the observations which lie within m-dimensional spheres of radius r (m = i. 2 . . . . . 10), and next, the ratios of observations lying in (m + l)-dimensional spheres, to those lying in mdimensional spheres, f h e bottom panel gives the results of regression (19) for each embedding dimension, using rm,, = 0.30 (which guarantees at least 1000 nearest neighbors in the spheres for all d~mensions), and rm~ = 0.45, ~' as well as the estimated noise levels and the limits of the confidence intervals. As can be seen, the estimates are initially too high, because the embedding dimension is apparently lower than the system's dimension, but quickly converge towards

-~Sincc wc only dcal with uniformly distributed noise for the the time being, the thickness is M. "~The prcceding argument presumes that tile system's curvatures in cmbedding space arc not so narrow, that the noises of different sheets of the system overlap. If the attractor is that sharply fi~lded ererywhere and the noise is large relative to the system's curvature, the ploposet~ method may give inexact results. Sln the version presented in Szpiro ( 1991 ). regression (19) uses only r as an independent variable (not r-'), and all r-values must be smaller than the noise level M. Since M is not known a priori, and since

for small r there are only few "nearestneighbors',the method that is presented there is limited in its applicability. 6Note, that neither r,,~. nor rma, have any direct relation to the size of M. Thi,~lack of restriction on the values of r makes the use of the method especiallysimple.

EDI

4834 9600 14414 19164 24017 28716 33394 37880 42383 46778 511"70 55269 59253 63191 67030 70745 74332 77768 81026 84144 87217 90179 92913 95510 97961 100313 102612 104696 106573 108378 110067

Rad

0.01 O.P2 0.03 0.04 0.05 0.06 0.07 008 0.09 O.I O.I I 0.12 0.13 O.14 O.15 0.16 O.17 O.18 O.19 0.2 O.21 0.22 0.23 0.24 0.25 0.26 0.27 0.28 0.29 0.3 0.31

147 582 1324 2352 3679 5258 7050 9128 11467 13982 16765 19762 22935 26164 29589 33133 36750 40400 44126 47838 51702 55480 59297 62999 66699 70375 74005 77435 80805 83965 87029

ED2

3 32 96 228 487 804 1295 1915 2728 3667 4802 6172 7733 9509 11543 13738 16132 18669 21456 24406 27489 3~677 34077 37513 41042 44753 48488 52200 55972 59703 63327

ED3

(a) Cumulatit'e obserrations

0 I 7 21 44 IO1 181 326 533 830 1176 1684 2310 3069 3958 5054 6309 7724 9279 11031 13027 15109 17504 20077 22857 25769 28734 31975 35308 38867 42355

ED4 0 0 1 3 5 14 32 62 ill 192 293 470 714 1018 1395 1903 2519 3245 4161 5087 6197 7501 9028 10675 12596 14642 16900 19309 21915 24759 27752

ED5 0 0 0 I I 2 5 I1 25 40 74 122 194 304 464 647 907 1230 1671 2183 2767 3474 4350 5280 6461 7743 9268 10929 12722 14836 17109

ED6 0 0 0 0 0 O O 2 5 8 17 38 52 85 150 231 313 440 641 8'77 !181 1532 2027 2525 3181 3955 4839 5857 7020 8422 10029

ED7 1) 0 0 0 0 0 (1 0 3 3 5 8 14 24 40 70 109 158 236 353 497 649 864 !155 1479 1909 2408 3038 3766 4629 5620

ED8 0 0 0 0 0 0 0 0 0 2 2 3 5 6 8 19 39 58 89 149 198 272 365 509 688 900 1174 1493 1946 2487 3086

0 0 0 0 0 0 0 0 0 0 1 I 3 3 3 6 16 20 35 59 82 109 145 21)3 294 396 535 706 947 1279 1629

ED9 EDIO ~1~

0.020 0.055 0.092 0.073 0.123 0.097 0.153 0.132 O.183 0.153 O.211 0.184 O.241 0.210 0.271 0.238 0.299 0.262 0.328 0.286 0.358 0.312 0.387 0.337 O.414 0.363 0.441 0.390 0.468 0.415 0.494 0.439 O.519 0.462 0.545 0.486 0.569 0.510 0.593 0.532 O.615 0.553 0.638 0.575 0.660 0.595 0.681 0.615 0.702 0.636 O.721 0.655 0.740 0.674 0.758 0.693 0.775 0.711 O.791 0.728

0.030 0.061

~1"-

Ib} Ratios

tt = 0.139325. • = 0.1111434. [~ = 0.129773. ;' = 0.557048. (i = 0.875350. p = 0.119174. tr = 0.336320. r = 0.29346

y, = tt + ~ty~_t + ;'Y~-.* + PY;'--'Y-~ + ~:,

Table 1 Non-linear autoregress ve time-series uniform noise: ,~,; = 0.2000

q.~

0 0.031 0.073 0.092 0.090 0.126 0.140 O.170 0.195 0.226 0.245 0.273 0.299 0.323 0.343 0.368 0.391 0.414 0.432 0.452 0.474 0.493 0.514 0.535 0.557 0.576 0.593 O.613 O.631 O.651 0.669

I1~

* 0 0.143 0.143 0.114 O.139 0.177 O.190 0.208 O.231 0.249 0.279 0.309 0.332 0.352 0.377 0.399 0.420 0.448 0.461 0.476 0.496 O.516 0.532 0.551 0.568 0.588 0.604 0.621 0.637 0.655

~1"~

* * * 0 0 O 0 O.182 0.200 0.200 0.230 O.311 0.272 0.268 0.299 0.280 0.333 0.323 0.340 0.357 0.360 0.345 0.379 0.358 0.402 0.384 0.429 0.402 0.447 0.427 0.463 0.441 0.482 0.466 0.495 0.478 0.513 0.492 0.529 O.511 0.548 0.522 0.566 0.536 0.581 0.552 0.599 0.568 0.616 0.586

q(,

* * 0 0.333 0.200 0.143 0.156 O.177 0.225 0.208 0.253 0.260

Sl~

* * * * * * * 0 0.600 0.375 0.294 0.211 0.269 0.282 0.267 0.303 0.348 0.359 0.368 0.403 0.421 0.424 0.426 0.457 0.465 0.483 0.498 0.519 0.536 0.550 0.560

~h,,

* * * * * * * * * O 0.500 0.333 0.600 0.500 0.375 O.316 O.410 0.367 0.345 0.377 0.393 0.422 0.396 0.398 O.414 0.419 O.401 0.422 0.397 0.441 0.399 0.465 0.427 0.471 0.440 0.488 0.456 0.491 0.473 0.517 0.487 0.537 O.514 0.549 0.528

*1,~

* * * * * * * * 0 0.667 0.400 0.375 0.357 0.250 0.200 0.271 0.358

!'~

t~

~.

'~~.

"~" =.~

,~ --

.~

111688 113105 114430 115578 116695 117679 118553 119334 120064 120721 121312 121807 122213 122595 122928 123204 123475 123692 123888 124750

90014 92988 95815 98413 100906 103191 105299 107208 109062 110813 112375 113786 115111 116312 117383 118320 119166 119927 120606

Estimated noise 1M ) (Eq. 120)i Lower limit (Eq. (21)1 Upper limit

StO err of ~ h-,~ (Eq. (14)1

Constant Std err of constant R-" No. of observations 7 Std err of ;'

I]2

45865 49499 53263 57059 60914 64674 68241 71967 75526 78915 82218 85404 88540 91590 94392 97067 99613 102016 104255 ....

I]3

19512 22091 24749 27546 30610 33835 37335 40769 44244 47722 51277 54841 58406 62053 65518 68884 72401 75758 79043 124750

0.2155

0.2177

0.2166

0.2206 0 .". .~. .t .)

0 0.001 0.999 16 3.0779 0.0059 - 2.3462 0.0151 0.6667

30811 33959 37309 40803 44454 48020 51751 55428 59121 62719 66267 69853 73298 76637 79909 83149 86264 89252 92084

0.2217

0 0.001 0.999 16 3.54 ! 8 0.0074 - 3.1799 0.0190 0.7854

66978 70659 74245 77765 81164 84486 87573 90636 93483 96197 98769 101169 103453 105686 107592 109445 1110:~0 112577 113974

(cl Rcg,'ession output ( Eq. ( 1911

0.32 0.33 0.34 0.35 0.36 0.37 0.38 0.39 0.4 0.41 0.42 0.43 0.44 0.45 0.46 0.47 0.48 0.49 0.5 Total

0.2187 0.2232

0,2209

0 0.002 0.999 16 2.6659 0.0104 - !.6296 0.0268 0.5g90

1114

11897 13769 15739 17984 20262 22915 25702 28717 31790 34938 38194 41586 4508b 48558 52036 55597 59083 62547 65977

6858 8226 9629 11283 12987 14970 17115 19476 22033 24690 27492 30420 33477 36658 39926 43357 46722 50159 53501

0.1985 0.2014

0.2000

0 0.001 0.999 16 2.6673 0.0075 - 1.7824 0.0192 0.5333

l/5

3831 4729 5748 6850 8138 9558 11153 12990 14958 16990 19314 21779 24295 27041 29869 32989 36029 39258 42358

tie

0,806 0.822 0,837 0,851 0,865 0,877 0.888 0.898 0.908 0.918 0.926 0.934 0.942 0.949 0.955 0.960 0.965 0.970 0.974 1.01~0

0.2017 0.21)60

0,2038

0 0.002 0.999 16 2.4082 0.0098 - 1.3514 0.0251 11.4909

2~18 2628 3276 4027 4904 5903 7071 8341 9740 11367 13122 15091 17151 19403 21698 24394 27113 29923 32886 ........

I]~.

0.685 0.701 0.717 0.734 0.751 0.765 0.779 0.794 0.808 0,820 0.832 0.844 0.856 0.867 0.877 0.887 0.897 0.906 0.915 ....

l]8

0.633 0.651 0.663 0.675 0.689 0.705 0.721 0.736 0.748 0,761 0.774 0.785 0.797 0.810 0,820 0.828 0.839 0.849 0.858

0.2007

0 0.002 0.999 16 2.1399 0.0135 - !.0198 11.0348 0.4295

3672 0,686 0,700 0,715 0.730 0.742 0.758 0.770 0.783 0.795 0.806 0.818 0.828 0.837 0,847 0.857 0.866 0.875 0.883 1.000

0,610 0,623 0,636 0,653 0.662 0.677 0.688 0.704 0.719 0.732 0.745 0.758 0.772 0.783 0,794 0.807 0.816 0.826 0.835

0.576 0,597 0.612 0.627 0.641 0.653 0.666 0.678 0.093 0.707 0.720 0.731 0.743 0.755 0.767 0.780 0.791 0.802 0.811

0.1926 11.21115

O.1969

0 0.0113 0.998 16 2.0632 0.0180 0.9293 0.0064 0.4063

II~

* division by zero

0.1990: 0.1975 0.2(~,6 0.2000 Mean ( M s - ,~lso): 0.2002 Error 0.12%

0.2027

0 0.0113 0.998 16 2.2549 0.0166 - 1.1442 0.0427 0.4571

0.744 0.760 0.775 0.790 0.804 0,819 0,832 0,845 0.857 0,868 0.879 0.889 0.899 0.909 0.917 0.925 0.932 0.939 0.945

0,559 0,575 0,597 0.607 0.627 0.63g 0.652 0.667 0.679 0.688 0.703 0.716 0.726 0.738 0.748 0.761 0.771 0.783 0.795

0.1934 11.2012

O.1972

0 0.003 0.998 16 1.9598 0.015 I 0.8073 0.0389 0.3866

I}1o

0.548 0.556 I).570 11.587 0,603 0.618 0.634 0.642 0.651 0.669 0.679 0.693 0.706 0.718 0.726 0.739 0.753 0.762 0.773 ......

~"

~;

.~.

'n g,.. ~-

.~ .~

G.G. Szpiro /Journal of Econometrics 78 (1997) 229-255

240

a value of 0.20. For sufficently large embedding dimension the estimates, and especially the average estimates, lie within one or two percentage points of the true noise level.

5. Normally distributed noise In the previous sections we illustrated the technique with uniformly distributed noise. We now show how to apply it, when noise is normally distributed. Instead of computing volumes of spheres, as above, we must now compute the mass of m-dimensional spheres, where density is distributed as the time-series in one direction, and normally distributed, with standard deviation ~, in the m -- 1 other directions. A problem arises because - when densities are not u~liformly distributed - the masses depend on the hyperbody's orientation in embedding space: since, for non-uniform distributions, the points are more densely distributed close to the attractor, a larger part of them lie outside the sphere if the attractor is sloped with respect to the axes. Hence, when the hyperbody lies orthogonal to the axes, the mass of such a sphere of radius r, with the center in the middle of the hyperbody, is given by Win(s, r), +r

+ x / ' r - " - - .x-"

W,,(a, r) = ~ f ( x ) - r

~ -

~ 'r

~

+ x,"r-" - - x-" - - )'-" -

g(a, y) ... x

...

~ x ' r-" - x :

- )'-" -

g(a, z) dz ... dy dx, ...

(23) wheref(x) is the time-series' actual distribution along the x-axis, and a(s," ) is the density function of the normal distribution with standard deviation a. When the attractor is sloped with respect to the axes, Eq. (23) must be adapted accordingly. Furthermore, since the sphere's center need not be in the middle of the attractor, edge effects must again be taken into consideration; y ... z in Eq. (23) must be varied from - r to + r, and the results added up. The integrals cannot be solved in closed form, and therefore Monte Carlo techniques are used to simulate the proportions, q,,(s, r),

W..((r, r) qm(a, r) W,,_ I (a, r)" Since, for the density of normal distributions,

(24)

we obtain

w.(#) qm(fl)

W,,,_, (fl)'

(26)

G.G. S:piro /Jounto/ oj Ecormwtrics

211

78 (1997) 229-255

where /J’is the radius, measured in fractions or multiples of rr, and the standard deviation is unity (which is henceforth dropped from the equations). We now take the hyperbody’s orientation at various regions in embedding space into account. For simplicity, we illustrate the technique for embedding dimension 2. (The method can be extended to higher dimensions, but becomes rather complicated, because, for embedding dimenston tn, there are tn - 1 slopes to be estimated, and the integrals must be simulated for all combinations of such slopes.) The distribution of the hyperbody’s slopes in embedding space can be determined by (a) partitioning the embedding space into a number of boxes, (b) computing the slopes between pairs of observations lying in the same box,’ and (c) calculating the mean of the slopes which were found in each of the boxes. The Monte-Carlo simulated integrals for the various slopes are then weighted according to the frcqucncy with which 2 cpccific Fartition is visited. and the result compared to the r12 of the observed time series. Let us demonstratL the method with a non-linear, threshold autoregressive model of the form ( - (x - _v,)“~ A h)e- )‘r+ c y,+1=

((y,-n)

I

113

+he-'8)+c

d( 10J,&’ -uo’3l!;l’“) + c

for yr < 0.125 for 0.125
+E,+,,

(27)

i

with parameters z = 0.125, /I = 0.50607357, ;’ = 0.0232885279, 6= 0.121ZO5692, and where s, is normally distributed noise with standard deviation Q = 0.008.* This time series is known to have a fractal dimension of less than one.’ Hence, the proposed method can be employed in embedding dimensions one and two. A problem which arises with non-uniformly distributed noise is that much longer time series are needed to perform the estimations. 16.0 observations were generated, which correspond to about three days of tick-bytick data on the foreign exchange market. lo By comparing qz of the simulated integrals (I$‘) with qz of the observed time series ($‘) at several radii, we obtain a number of estimates of the noise’s standard deviation (rr,J. Following ‘bootstrapping’ techniques, we can, as a by-product, assess the approximate standard error of rrA, by computing the standard deviation of these estimates. The results, presented in Table 2, show that the method works very well: panel (a) of the table

‘To reduce the biases introduced by unduly large tangents. on the one hand, and by the attractor’s curvature, on the other hand. only those pairs are considered whose interpair distances satisfy some restrictions. “Eq (27) simulates the so-called Belousov-Zhabotinskii

reaction in chemical dynamics.

%ee Matsumoto and Tsuda (1983).If the dimension of the time series under inspction beforehand, then the method described in Section 2 can be used to verify it. l”Se~r for example, the HFDF-tapes, as distributed by Olsen & Associates. Ziirich.

is not known

0.132 0.174 0.218 0.260

0.300 0.339 0.377 0.413 0.448

0.482 0.514 0.545

0.575 0.602 0.627

0.652 0.674 0.695

0.302 0.341 0.379 0.416 0.452

0.486 0.520 0.552

0.582 0.610 0.637

0.662 0.686 0.708

0.3 0.4

1.3 1.4 1.5 1.6 1.7

1.8 1.9 2

1.1 i.2

0.5 0.6 0.7 0.8 0.9 !

0.044 0.088

0.132 0.176 0.218 0.260

0.2 10.18%

0.044 0.088

0 15.08%

0.1 0.2

Radius for MC

Slope Weight

.-. ....-

-.. ... ...

... ... .-.

.-..... --.-.

... ... ... ...

... --.

... ...

0.493 0.500 0.506

0.464 0.476 0.485

0.414 0.434 0.451

0.285 0.318 0.346 0.372 0.395

0.132 0.174 0.213 0.250

0.045 0.089

0.462 0.467 0.472

0.437 0.446 0.454

0.397 0.412 0.426

0.279 0.308 0.335 0.358 0.378

0.133 0.174 0.212 0.247

0.045 0.089

1.8 14.78%

time series normal

1.6 17.77%

o f ~1-"

autoregressive

simulations

threshold

(a) M o n t e - C a r l o

Table 2 Non-linear

..... ...

-.. -.. -..

.-. .....

... .-... -.. ...

-.. .-... ...

..-..

.-. -..

0.397 0.401 0.404

0.384 0.389 0.394

0.361 0.372 0.379

0.266 0.292 0.314 0.333 0.348

0.127 0.167 0.204 0.238

0.042 0.086

2.2 8.49%

.-... ...

... .-...

.......

... ... ......-

... .-. --...

... ...

... ...

0.361 0.363 0.366

0.352 0.356 0.359

0.333 0.340 0.347

0.260 0.281 0.298 0.312 0.323

0.127 0.166 0.202 0.234

0.044 0.086

2.6 6.90%

.' n o i s e : tr ~ = 0 . 0 0 8 0 0

0.224 0.225

0.226

... .-.

0.222 0.223 0.224

0.219 0.220 0.221

0.198 0.206 0.211 0.215 0.217

0.119 0.147 0.169 0.186

0.043 0.084

4.2 4.44%

...

... .....

... .-...

... ... ... ... ..-

... ... --...

... ...

... .-.

--,

.--,

--,

---, ~

~ ---, ~

~ --, ~ ~ ~

~ --, ~ --*

--* -,

.--

0.471

0.462

0.453

0.418 0.431 0.442

0.370 0.388 0.404

0.256 0.283 0.307 0,330 0.351

0.124 0.160 0.194 0.226

0.044 0.085

Weighted ~1stc_,

0.00819 0.00012 2.38

0.00802 0.00801 0.00806

0.00820 0.00810 0.00806

0.00832 0.00822 0.00806

0.00822 0.00827 0.00833 0,00836 0.00836

0.00847 0.00798 0.00808 0.00831

0.01122 0.00947

Error:

0.454 0.463 0.475

0.423 0.433 0.444

0.379 0.394 0.405

0.261 0.290 0.316 0.339 0.360

0.131 0.160 0.196 0.233

0.061 0.100

al~ s

(c) E s t i m a t e d

(d) A v e r a g e e s t i m a t e : Standard error:

0.0144 0.0152 0.016

0.012 0.0128 0.0136

0.0096 0.0104 0.0112

0.0056 0.0064 0.0072 0,008 0.0088

00024 0.0032 0.004 0.0048

0.0008 0.0016

Radius for TS

(b) A c t u a l T S

o

G.G. Szpiro /Journal of Econometrics 78 (1997) 229-255

243

sets out the Monte-Carlo simulations of equation (23) for normally distributed noise, and the weighted r/2Mc. Panel (b) presents the actual r/~s, and in panel (c) we estimate the noise's standard deviation - for radii between 0.1 and 2.0 - by comparing r/Ts with t12 uc, and by linear interpolation. Panel (d) shows that on average (for radii between 0.5 and 2.0), the estimates lie within about 2% of the actual standard deviations.

6. Extension to other distributions

We started by examining uniform noise, and then extended the method to shocks which are normally distributed. However, financial time series are known to be non-normal with fat tails (Gallant, et al., 1992), and therefore an example with fat-tailed noise is presented in this section. We again simulate a time series of the non-linear, threshold autoregressive model (Eq. (27)), this time adding noise of the form ~, = .~,.~,,

where .~eN(~,a)

and

~cU[a,b].

(28)

(N(0t, a) is the normal distribution, with mean ~t and standard deviation a, U [ a , b ] is a uniform distribution between a and b.) We stipulate 0t = 0, tr = 0.0055, a = 1, and b = 2; the parameter to be estimated is the standard deviation of ~,.11 As in the previous section, we must compute the mass of m-dimensional spheres, with densities appropriately distributed, in addition, the slopes of the time series in embedding space must again be taken into account. Since we know that the time series of equation (27) is of dimension less than one, it suffices to examine w/a.The results for fat-tailed noise are set out in Table 3, panels (a)-(d). Again, the estimates are quite good: as panel (d) shows, the mean of the estimated a - for radii between 0.5 and 2.0 - is 0.00761, which corresponds to an error of 5.6%. The same procedure can, in principle, be utilized for any other noise distribution. It is simply a matter of creating tables of tli for the simulation of Eq. (23), and comparing the appropriately weighted ~/~c with the ~/[s of the time series. For time series which have a dimension higher than one, the combinations of slopes for all dimensions must be considered. Of course, in order to apply the proposed method in this manner, the noise's distribution must be

i ~Thcfat-tailednoise thus simulatedhad standard error 0.00804,skewness = - 0.04, and kurtosis = 0.38.

... ...

0.412 0.424

0.401 0.410

0.356 0.369 0.381 0.391 0.378 0.386

0.339 0.351 0.361 0.369 ... ..-

... ..--...

... ... ...

0.309 0.312

0.292 0.298 0.303 0.306

0.268 0.279 0.287

....-

... .-. .....

... .-. ...

... --. --. ---

0.242 0.242

0.239 0.240 0.241 0.242

0.228 0.234 0.237

0.188 0.201 0.212 0.220

... ...

-.... ... ...

-.-.. .-.

-.-.... ...

---.. ... ...

0.182 0.182

0.184 0.184 0.183 0.183

0.183 0.185 0.185

0.162 0.169 0.175 0.179

0.105 0.123 0.139 0.152

-4 --,

-, ---, --} ---,

~ -, ---.

~ --* -4 --,

--* ~ ~ -4

0.366 0.375

0.325 0.337 0.348 0.357

0.285 0.301 0.314

0.207 0.228 0.249 0.267

0.111 0.137 0.161 0.185

0.489 0.510

0.359 0.374 0.388 0.400

0.298 0.315 0.328

0.205 0.223 0.239 0.254

0.110 0.133 0.154 0.173

-4 -4 -4

1.9 2

... .-. ... ...

0.310 0.328 0.344

... ... -.. ...

... ... .-. ...

0.031 0.058 0.082

0.402 0.424 0.446 0.468

0.309 0.327 0.344

0.215 0.238 0.259 0.279

0.114 0.139 0.164 0.186

-.... ...

1.5 1.6 1.7 1.8

... .-. ...

0.222 0.246 0.270 0.290

... ... ... ...

0.030 0.059 0.086

0.332 0.358 0.381

0.218 0.242 0.266 0.288

0.112 0.140 0.166 0.192

... .-. ...

5 ... 4.75%

1.2 1.3 1.4

... ... .-. ...

0.116 0.144 0.172 0.197

0.029 0.059 0.087

--. ...

0.225 0.253 0.280 0.306

0.114 0.140 0.166 0.193

... ... ...

3.6 3.23%

0.8 0.9 1 1.1

... -.. .....

0.032 0.058 0.085

... .-.

0.112 0.142 0.170 0.197

0.032 0.060 0.088

2.6 9.08%

0.4 0.5 0.6 0.7

0.028 0.058 0.086

....-

0.030 0.058 0.085

..... ...

1.8 15.35%

0.029 0.057 0.085

1.6 15.75%

0.1 0.2 0.3

1.4 39.29% Weighted 9 ''it 2

.. ...

time series fat-tailed noise: a a = 0.00800

Radius for MC

0 3.44%

o f ~lz

autoregressive

simulations

threshold

Monte-Carlo

Slope Weight

{a}

Table 3 Non-linear

0.384 0.394

0.337 0.350 0.362 0.374

0.290 0.307 0.323

0.212 0.233 0.254 0.273

0.114 0.140 0.165 0.189

0.029 0.058 0.086

II TM 2

Error:

(d) A v e r a g e e s t i m a t e : Standard error:

0.0152 0.016

0.012 0.0128 0.0136 0.0144

0.0096 0.0104 0.0112

0.0064 0, 0 0 7 2 0.008 0.0088

0.0032 0.004 0.0048 0.0056

0.0008 0.0016 0.0024

Radius for TS

(b} A c t u a l T S

5.65%

0.00761 0.00024

0.00720 0.00714

0.00754 0.00746 0.00739 0.00728

0.00779 0.00775 0.00766

0.0077~ 0.00777 0.00779 0.00778

0.00782 0.00778 0.00778 0.00780

0.00806 0.00794 0.00786

(c) E s t i m a t e d

a

G.G. S:piro /Journal of Econometrk's 78 (1997) 229-255

245

known beforehand. 12 In the following section we investigate the method's capabilities when even this requirement is eliminated.

7. Noise with unknown distribution So far we have been able to determine the noise's parameter, given its distribution. Let us now consider the situation where - in addition to ignorance about the autoregressive process - we are also unaw,:re of the noise's distribution. Even in this case, in which virtually nothing is known about the datagenerating process, we can make quite strong statements about the noise. F o r example, given two time series, or different parts of the same time series, that are generated by the same unknown process, but whose noise strengths differ, we can compare the intensities, ta We show, in this section, that it is possible to make qualitative statements of the form 'there is more noise in part one of the series, than there is in part two', or even stronger quantitative statements like 'there is twice as much noise in series A, as there is in series B'. In order to compare iatensities we must seek spheres for both series, such that the proportions ~lm are equal - realizing that the series with less noise requires a larger sphere. The ratio of the larger to the smaller radius is equal to the ra'6o of the intensities of the noises. Hence, if the noise in series A is I' times as intense as the noise in series B, then, for sufficiently large m, we would expect to see r/~(ri)

=

~l~(Tri).

(29)

T o illustrate, let us create the following series: ~'* xt = 1 - 1.4x2_1 + 0.3xt-z + (~.t-/~t)fl,

(30)

where fl is the noise's intensity, and the noise terms are of the form g',e N( - 0.25, 1), {1 /~t =

if et < 0.1, U [2, 2.5-1 otherwise.

(31)

The distribution of the noise is asymmetric, and in our simulation of 8000 values we had a mean of 0.00058, standard deviation of 0.00788, skewness 0.99886, and kurtosis 1.05316. Two series of length 4000 were created (corresponding to about 8 hours worth of foreign exchange trading), with noise terms randomly drawn ~'However, it may even be possible to determine the distribution of noise (without positing it a priori), by verifyingwhich of several Monte-Carlo simulations gives the best fit to the observed proportions tl.. SThat the noise of the two series be generated by the same process is a minimal requirement.If it were not met, there would be nothing to compare. 1"This series is known as a H6non attractor {Bcn-Mizrachiet al., 1984).

0.0008 0.0012 0.0016 0.002 0.0024 0.0028 0.0032 0.0036 0.004 0.0044 0.0048 0.0052 0.0056 0.006 0.0064 0.0068 0.0072 0.0076 0.008

0.025 0.035 0.047 0.059 0.067 0.076 0.084 0.093 0.100 0.107 0.!13 0.119 0.124 0.127 0.132 0.136 0.138 0.142 0.145

0.071 0.127 0.154 0.211 0.243 0.261 0.287 0.310 0.337 0.351 0.366 0.386 0.397 0.411 0.418 0.432 0.441 0.448 0.459

0.091 0.171 0.189 0.201 0.208 0.233 0.259 0.261 0.296 0.325 0.343 0.359 0.364 0.381 0.393 0.405 0.417 0.427 0.438

0.000 0.000 0.118 0.205 0.190 0.198 0.258 0.265 0.284 0.310 0.330 0.352 0.370 0.381 0.387 0.403 0.417 0.426 0.431

0.029 0.0~3 0.056 0.068 0.079 0.088 0.099 0.107 0.115 0.121 0.126 0.132 0.137 0.142 0.146 0.151 0.155 0.159 0.162

0.085 0.128 0.177 0.214 0.251 0.267 0.291 0.320 0.336 0.352 0.368 0.384 0.401 0.409 0.416 0.423 0.433 0.442 0.448

q3

q4

116

0.069 0.101 0.131 0.137 0.201 0.229 0.255 0.283 0.312 0.331 0.345 0.367 0.376 0.397 0.409 0.419 0.427 0.436 0.449

0.000 0.000 0.032 0.067 0.190 0.230 0.260 0.289 0.286 0.330 0.355 0.360 0.368 0.390 0.397 0.417 0.429 0.437 0.444

• * Min. 200 observations • ** Emb. dimension 3 to 5 and rain. 200 observations

• N.A.

0.0016 0.0024 0.0032 0.004 0.0048 0.0056 0.0064 0.0072 0.008 0.0088 0.0096 0.0104 0.0112 0.012 0.0128 0.0136 0.0144 0.0152 0.016

s12

0.0014 0.0019 0.0026 0.0034 0.0039 0.0046 0.0052 0.0060 0.0066 0.0072 0.0078 0.0085 0.0092 0.0097 0.0104 0.0110 0.0114 0.0120 0.0124

0.0021 0.0044 0.0046 0.0048 0.0050 0.0057 0.0065 0.0066 0.0076 0.0086 0.0095 0.0101 0.0103 0.0114 0.0118 0.0125 0.0135 0.0144 0.0153

Average estimate***: Standard error***: Error***:

(e) Mean**:

0.0015 0.0024 0.0028 0.0039 0.0046 0.0053 0.0063 0.0069 0.0080 0.0088 0.0088 0.0105 0.0110 0.0123 0.0130 0.0143 0.0151 0.0160 0.0181

4 * * 0.0043 0.0051 0.0048 0.0050 0.0063 0.0065 0.0071 0.0084 0.0088 0.0095 0.0113 0.0117 0.0119 0.0130 0.0136 0.0142 0.0146

1.62

1.63 1.64 1.62 !.63 i.62 1.59 1.58 1.56

1.64 1.63

1.64

1.66

1.69 1.59 !.63 1.69 1.62 1.65 1.64

2.01

1.82 1.99 1.77 1.97 i.93 1.89 1.96 1.93 2.00 1.99 1.83 2.02 1.97 2.05 2.03 2.11 2.10 2.11 2.27

1.95 0.10 2.4%

1.92

2.68 3.68 2.90 2.40 2.08 2.04 2.03 1.83 1.89 1.95 1.98 1.94 i.84 !.90 1.85 1.84 1.87 1.89 1.92

4

1.88

1.86 1.92 ! .89 ! .86 i.82

1.94

1.83 2.01

1.92 1.83

1.82 i .76

1.77 1.98

2.71 2.55 2.00

5

Emb. Dim. 2 3

rio

Emb. Dim. 2 3

)14

Radius

)13

Radius

)12

(d) Relative noise intensity (,')

(c) Interpolated radius

(b} Series P,

(a) Series A

Table 4 Noise with unknown distribution relative intensities: 7,~ = 2.0

G.G. Szpiro /Journal of Econome:rics 78 (1997) 229-255

247

from the pool of simulated data. For series A the value of/~ was set to 0.0025, in series B the noise's intensity was doubled to p = 0.0050. Panels (a) and (b) of Table 4 present the proportions t/,,, for m = 2 to 5, and for series A and B, respectively. We now seek ~l,,'s in panels (a) and (b) which have similar values. For example, ~/A(0.004) ~ 0.337 ~ ~/3B(0.008).

(32)

(The two panels in Table 4 are already arranged in such a manner that the appropriate values come to lie on the same line. Example (32) is highlighted in the table.) In panel (c) the interpolated radii are computed, and in panel (d) we estimate the relative intensity or noise by comparing the radii for the spheres. We note, first of all, that when we use v/2, the estimates are mcch higher than when we use r/,,, m > 2. This means that embedding dimension 2 is too small for the time s~ries at hand, and one has to move to higher dimensions, ts Inspecting the ~/,, for higher embedding dimensions we see that, in fact, Eq. (30) holds for 7 ~. 2 (if r~ is not too small). Panel (e) of Table 4 gives the mean estimates of ~ for those radii, whose spheres contained at least 200 points. The estimates are consistent with y = 2, and the implication is that noise is twice as intense in series B as it is in series A. The overall average of the estimated 7's is 1.95, which is only 2.4% removed from the true value. Let us recall that this surprisingly precise inference could be made without knowledge of any of the characteristics of t'~e noise's distribution.

8. Application to an economic time-series We now put the method to an empirical test, using economic data from the stock market. We do not know how noise is distributed in the stock market, and in order to test the applicability of the method, we first of all assume uniform distribution, and check to see whether the estimates of the noise level M s converge. If they do, the method has been shown to work. The data consist of 5831 daily observations of a price index of the New York Stock Exchange, the Standard and Poors 500, for the period January 1968-March 1991. We compute log-differences, and split the series into an early part (January 1968-December 1979, 3002 observations), and a late part (January 1980-March 1991, 2827 observations). The two series are analyzed for possible differences in their noise intensities. According to Scheinkman and LeBaron (1989) the dimension ~ of the stock market is about six. 16 Following Whitney (1944), one may expect the estimates to converge towards the true level for embedding dimensions of about

t SSincethe Hdnon attractor has a fractal dimension af about 1.14this is not surprising. t~This claim has been criticized, however,by Eckmann and Ruelle(1992).

G.G. S'.piro /Journal of Econometrics 78 (1997) 229-255

248

0.02 0.018 0.016 0.014 0.012 0.01 0.008

1/68-12/79

0.006 0.004 0.002 0

;

',

I

I

;

;

I

l

I

;

l

I

I

~ l

I

I

I

'.

I

'. ',

,

Embedding dimension Fig. 2. Estimated noise in the S&P 500 time serics.

1 = 13 (i.e., for v/t,0 and higher. We choose rmi,,=0.03, and rm~x = 0.05 for embedding dimension m = I to m = 25, which guarantees at least 1000 pairs in all spheres, in Fig. 2 the estimated noise levels are plotted against m. As expected, we obtain large values at first, but for sufficiently high embedding dimenqion the estimates converge. For the late series the noise estimates start to flatten at about m = 12. For the early period the evidence is not quite as clear-cut, but nevertheless, we may conclude that the test of the method has been successful. It is obvious from Fig. 2, that the late part of the series is much noisier than the early part.17 However, Fig. 2 was created under the assumption of uniformly distributed noise, and this assumption is not justified for the S&P time series. In order to compare numerically the noise intensities of the two series, without making the uniformity assumption, we utilize the method of Section 7. Now the only assumption is that the noise has the same distribution in both parts of the series. To apply the method, we use embedding dimension 20, and seek the radii, such that the proportions, r/2o, are identical in both series. Table 5 sets out the results. For example, ~l.,o equals 0.816 at radius 0.320 in the early series, and at radius = 0.4.7"3 in the late series. This means that the relative intensity of the noise ~, is 0.433/0.320 = 1.35. Looking at additional values of ~12o, we can m =2d+

J TWe make no attempt introduce the method.

here to explain or interpret

t h i s r e s u l t , s i n c e t h e s o l e a i m o f t h i s p a p e r is t o

249

G.G. Szpiro /Journal of Econometrics 78 (1997) 229-255

Table 5 S&P; relative intensity of noise

(a) Early part of S&P {I/68-12/79) Radius

02o

0.0300

0.789

0.03 I0

0.803

0.0320

0.816

(b} Late part of S&P {1/80-3/91

(c) Relative imensity

Radius

Pl.,o

Radius Interpolated

3'

0.0400 0.0405 0.0410 0.0415 0.0420 0.0~,25 0.0430

0.772 0.778 0.785 0.792 0.798 0.805 0.811 0.818 0.824 0.830 0.835 0.841 0.847 0.852 0.856 0.861 0.866 0.87 !

0.0412

i.37

0.0423

1.36

0.0433

1.35

0.0444

1.35

0.0456

1.34

0.0466

1.33

0.0477

1.33

0.0487

1.32

0.0498

1.31

0.0435

0.0330

0.829

0.0340

0.842

0.0350

0.853

0.0360

0.863

0.0370

0.873

0.0380

0.882

0.0440 0.0445 0.0450 0.0455 0.0460 0.0465 0.0470 0.0475 0.0480 0.0485 0.0490 0.0495

0.875 0.879

0.0500

0.883

summarize, that the late part o f the S & P series contains a b o u t one-third m o r e noise than the early l~art.

9. Conclusions and extensions In this paper, a m e t h o d was devised to determine the size o f external s h o c k s in unspecified, linear o r non-linear autoregressive time series; neither the lag structure nor, indeed, the functional form o f the process is k n o w n . T h e m e t h o d considers the time series as a h y p e r b o d y in an e m b e d d i n g space, a n d estimates the 'thickness' o f this h y p e r b o d y by geometric means. W e consider uniform, normal, and fat-tailed noise; extensions to o t h e r distributions possible. It is s h o w n that even if the noise's distribution is u n k n o w n , the m e t h o d still allows the c o m p a r i s o n o f intensities.

250

G.G. S:piro /Journal ~#" Econometrics 78 (1997) 229-255

Apart from interest in the size of noise per se, the method proposed in this paper also has some implications for the estimation of the underlying model. First of all, and perhaps most surprisingly, the method allows determination of the 'unexplained' variance, without actually knowing the model itself. Second, it allows a check for misspecification of the model: for models like Eq. (1), for example, the lag structure and the parameters are generally estimated by regression techniques in which the sum of squared residuals is minimized. The et, however, need not be "errors" which should be minimized, but could be economic realities, e.g., external shocks, with a certain, finite size. Minimizing the residuals, by implicitly assuming that they should vanish, may give incorrect results. Hence, one could check for misspecification by comparing 'errors', due to regression, with the size of the 'shocks', as determined by the proposed method.

Appendix A In this appendix the computation of the correlation integral, and its usage for the estimation of noise is described step-by-step. This will allow researchers to implement the method of tkis paper (a) to estimate the level of uniformly distributed noise, and (b) to compare noise intensities, when the noise's distribution is unknown. Steps (I)-(3), as described below, are common to both purposes. Then, to estimate the level of uniformly distributed noise, we use Steps (4a) and (5a), to compare intensities of noise with unknown distribution, Steps (4b) and {5b) are employed. (If one wishes to estimate the level of non-uniformly distributed noise numerically, the slopes of the 'attractor' in embedding space must also be estimated, and the appropriately weighted ~i's must be compared to benchmark values, as described in Sections 5 and 6 -ff the paper.) We consider a time series of length n, and demonstrate the method for embedding dimension m. From the time series, m-dimensional vectors of the form Xt = ( x t . x2 . . . . . x,,), X 2 = ( x 2 , x 3 . . . . . X m + l > . . . . . X n _ m + 1 = < X n _ m + i, x,,-m+2. . . . . . x,,> can be constructed. Step (l): As the first step, one needs to compute the Euclidean distance between each (or a sample) of the 01 - m)(n - m - I)/2 pairs of vectors: a"'m = ]Xp--X~] =

(xv+t -- xq+t)\

andq=p+

where p = I, . . . , n - m,

i=O

1. . . . . n - m + l .

This procedure is performed for m = 1 to a sufficiently high embedding dimension. In the example of Table 1, we used m = 1 to 10. For the S&Panalysis in Section 8, we went as high as m = 25 to check convergence of the M-values.

G.G. S:piro /Journal of Econometrk:s 78 H997) 229-255

251

Step (2): Next, the distances, r ~ are distributed, according to their sizes, into bins. For example, Table 1, shows that in embedding dimension 1, there are 4834 vector pairs whose interpair distances fall into the bin r < 0.01, and in embedding dimension 5. there are 192 pairs with interpair distances < 0.10. The sizes for the bins are not predetermined, they are specific to each time series that is being analyzed, and can be determined by trial and error. The largest interpair distance in one dimension - the absolute value of the largest observation of the time series minus the smallest observation- gives an idea of the bin sizes that are needed. One-half of that value may be a good initial value for the size of the largest bin. (Note however, that the Euclidean distance grows with the square root of the dimension.) When deciding oa the number of bins, it is convenient to err on the side of diversity, since bins can always be combined at a later stage, if it turns out that the vector pairs lhat are contained in each bin, are too spar~. In the example of "Fable 1, we used 50 bins, for the interpair distances 0.01, 0.02, 0.03 . . . . . 0.50. Step (3): The number of interpair distances in the bins are used to compate the ratios r/n,. This is done by dividing the number of pairs in a bin of a certain size, by the number of pairs in the corresponding bin in the next lower dimension. Table 1 shows, for example, that in embedding dimension 3, there are 13,982 vector pairs whose interpair distances are less than 0.10, and there are 3667 vector pairs in the corresponding bin in embedding dimension 2. Hence, ~13 equals 3667/13982 = 0.262 for bin size 0.10. Referring to Fig. 1, this means that there are 13,982 points in a circle (i.e., in the two-dimensional sphere) of radius 0.10, and the same number of points in the cylinder above the circle, whose height is 2M. A ball of radius 0.10, inscribed in the cylinder, contains 3667 points. The ratio r/3 denotes the ratio of the points in the ball, to the points in the cylinder. In other words, 26.2% of the cylinder's points are containe in the inscribed ball (see Fig. 3). Under the assumption of uniformly distributed noise, the following two steps complete the procedure. Step (4a): A regression is run with the dependent variable II,,, and the independent variables r and r 2 (cf. Eq. (19)). The problem here is to choose values for rmi, and rm~ with which to run the regression. In order to avoid erratic behavior due to the paucity of pairs with very small interpair distances, the smallest bin size used for the regression must be chosen in such a way, that a sufficient number of pairs are contained in the bin. In Table 1, for example, a radius greater than 0.30 guarantees that there are at least 1000 pairs in each bin, in all embedding dimensions. In the S&P-analysis, we also used rml, such, that each bin contained at least 1000 pairs. The value of rm~xis less crucial, as explained in Section 3, but should not be too large. Step (5a): Finally, the estimates for the noise level M are computed, according to Eq. (20).

252

G.G. Szpiro /Journal of Econometrics 78 (1997) 229-255

Example: 13982obs.in the cylinder 3667 obs. in the ball y

Projection

13982obs. in the circle rl0=3667/13982=0.262 Fig. 3. Computation of q,,. If no assumption is to be made about the noise's distribution, and the aim is only to compare the intensities in different time serie,:., the following two steps must be performed: S t e p (4b): In sufficiently high dimension - and for r-values between rmi. and rm~x - the r-values are sought, where the proportions tlm of the two series are identical. F o r example, in Table 5, series A has an ~/2o of 0.816 at r = 0.0320. Series B has a similar ~/2o at about r = 0.0430. If there is no r-value in Series B, whose ~/,, corresponds exactly to an q,, of Series A, then one simply interpolates between the two r-values that are closest. In Table 5 panel (b), t/2o = 0.816 lies between 0.0430 and 0.0435, and we get an interpolated value of 0.0433. Step (Sb): The relative intensity 7 is given by the ratio of the two r-values. In Table 5, for example, we get 7 = 0.0433/0.0320 = 1.35 for 112o. I.e., the noise in the second time series is 35% higher than in the first series.

Appendix B. Listing of Estnoise.C Appendix B lists the source code, in the C-language, of the program that was used for steps (1) and (2) described in Appendix A. //Estnoise.C // //Program to accompanythe paper"Noise in unspecilied,non-lineartime-series" //by GeorgeG. Szpiro

G.G. S=piro/Journal of Econometrics 78 (1997j 229-255 #include < process.h > #include < string.h > #include < stdio.h > #include < math.h > //*********************************************************** #define #define #define #define #define #define #define

EndSeries MinDim MaxDim Last Step Input Output

5000 I 10 50 0.01 "c:\\timeser.txt'" "c:\\noiseout.txt ~

/ / N u m b e r of observations in the time-series //Smallest embedding dimension (usually = I) //Largest embedding dimension / / N u m b e r of bins //Increase in the size of the bins / / I n p u t file (the time-series, length = End.Series) / / O u t p u t file

typedef long MatrixType[MaxDim-MinDim + 2][Last + 2]; typedef float DistanceType[MaxDim + I]; typedef float VectorType[EndSeries + 1]; /7 Variables Distance'] ype Euclid; int Dim; long P.Q; VectorType Vector; MatrixType Matrix: long K,Position: FILE *F: /I/*********************************************************** //Prototypes void INITIALIZE(void); float READDATA(float Vector[EndSeries'I,): void COM PDIST(Iong P, long Q, DistanccType Euclid): void CUM MATRIX(MatrixType M~,',rix); void WRITEFILE(MatrixType Matrb;): ************************************************************* void INITIALIZE(void) //This procedure sets the Array "Vector[_]", long J; / / a n d the output-matrix "Matrix[. ]" to zero for(Dim = I; Dim < = MaxDim-MinDim + I; Dim+ +) for(J = !: J < = Last + 1; J + + ) Matrix[Dim][J] = 0;

I forlJ = I; J < = EndSeries;J+ +1 Vector[J] = (float)~0); ************************************************************* float READDATA(float Vector[EndSeries]) int i; FILE *fptr; fptr = fopen(Input, "r"): f o r ( i = 0 ; i < EndSeries;i+ + ) fseanf(fptr. "'%f", & Vector[i]);

//This procedure reads the datapoints //(i.e., until EndSeries) of the Infile //into the Array "Vector[_]"

fclose(fpt r); ret urn(Vector[EndSeries]); //*********************************************************** void COMPDIST(Iong P, long Q. DistanceType Euclid) int J;

253

G.G. Szpiro /Journal of Econometrics 78 (1997) 229-255

254

long Position: E u c l i d [ 0 ] = (floatX0): for (J = 1; J < = M a x D i m ; J + + ) float pq; pq = V e c t o r [ P + J - 1] - V e c t o r [ Q + J Euclid[J] = Euelid[J - 1] + (pq * pq):

/ / C o m p u t e s Euclidean distances //between 2 vectors, P a n d Q, / / f o r dimensions i to M a x D i m . 1];

for (J = 1: J < = ( M a x D i m - M i n D i m + 1): J + + ) Position = (Iong)(sqrt(Euelid[J + MinDim-I]l/Step): i f ( P o s i t i o n < Last) I Matrix[)][Position + I]+ + ; / / I n M a t r i x [ d i m e n s i o n , R] the entry / / w h o s e R c o r r e s p o n d s to the else M a t r i x [ ) ] [ L a s t + ! ] + + : / / E u c l i d e a n distance is increased by one.

//********************************************************* void C U M M A T R I X I M a t r i x T y p e Matrix) int K,J; for (K = 2; K < = Last + I; K + + ) for (J = M i n D i m : J < = M a x D i m : J + + ) Matrix[J-MinDim + i][K] + = Matrix[J-MinDim + I][K-I]:

/ / T h i s p r o c e d u r e c u m u l a t e s the entries in the / ' / c o l u m n s of the M a t r i x [ - , - ] . Hence, for each column, / / t h e entry in the n-th r o w of the new matrix / / i s the sum of the entries in the / / f i r s t n rows of the old matrix.

//****************************************¢******************* void W R I T E F I L E l M a t r i x T y p e Matrix) F I L E *fptr: fptr = fopen(Output. "'w"): fprint flfptr, "\ n"): for (K = I; K < = Last: K + + ) ~t fprintf(fptr, "'%8.4f "°, K ' S t e p ) : for IDim = M i n D i m ; D i m < = M a x D i m ; Dim + + ) g~ fprintf{fptr, " % 1 0 1 d " , M a t r i x [ D i m - ( M i n D i m - I)][K]}:

/ / H e r e the M a t r i x [ , . , - ] is written as / / O u t p u t te a disk file.. T h e first c o l u m n / / g i v e s the distance, the next c o l u m n s //are the c u m u l a t e d counter.i

I fprintflfptr. " \ n"); fprintflfptr, "%8d". 9999): for (Dim = M i n D i m ; Dim < = M a x D i m ; D i m + + I fprintf(fptr, "'%101d". M a t r i x [ D i m q M i n D i m - l l ] [ L a s t

+ I]):

fprintflfptr. "', n"); fcloselfptr):

//********************************************************* main( ) // M a i n b o d y of the p~ogram { I N I T I A L I Z E ( ): READDATA(Vector); for {P = !; P < = E n d S e r i e s - M a x D i m - I: P + + ) { i f ( l P % 100) = = 0) //Prints progress to screen print f l " % 8 u " . P); Q = P + I: while{Q <

/ / F i r s t Q - v e c t o r for given P-vector =

EndSeries-MaxDim)

G.G. Szpiro /Journal of Econometrics 78 (1997) 229-255

255

{ COMPDIST(P,Q,Euclid); Q + = l: CUMMATRIX(Matrix); WRITEFILE(Matrix); exitl0); return 0;

//Cumulatesthe matrix, //and writes the output to a file. //and is done.

References Ben-Mizrachi, A., P. Grassberger, and I. Procaccia, 1984,Characterization of experimental (Noisy) strange attractors, Physical Review A 29, 975-986. Black, F. 1986, Noise, Journal of Finance 41,529-543. Brandst~iter, A. and H. L. Swinney, 1987, Strange attractors in weakly turbulent Couette-Taylor flow, Physical Review A 35, 2207-2220. Brock, W. A., 1986, Distinguishing random and deterministic systems: Abridged version, Jonrnal of Economic Theory 40, 168-195. Brock, W. A. and C. Sayers, 1988, Is the business cycle characterized by deterministic chaos.'?, Journal of Monetary Economics 22, 71-90. Eckmann, J.-P. and D. Ruelle, 1992, Fundamental limitations for estimating dimensions and Lyapunov exponents in dynamical systems, Physiea D 56, 185-187. Engle, R. F., T. lto. and Wen-Ling Liu, 1990, Meteor showers or heat waves? Heteroskedastic intra-daily volatility in the foreign exchange market, Econometrica 58, 525-542. Frank, M. and T. Stengos, 1988, Some evidence concerning macroeconomic chaos', Journal of Monetary Economics 22, 423-438. Gallant, A. R., P. E. Rossi and G. Tauchen, 1992, Stock prices and volume, Review of Financial Studies 5, 199-242. Gallant, A. R.. P. E. Rossi and G. Tauchen, 1993, Nonlineardynamical structures, Econometrica 61, 871-907. Grassberger, P. and I. Procaccia, 1983a, Characterization of strange attractors, Physical Review Letters 50, 346-349. Grassberger, P. and I. Procaccia, 1983b, Estimating the Kolmogorov entropy from a chaotic signal, Physical Review A 28, 2591-2593. Hsieh, D. A., 1989, Testing for nonlinearity in daily foreign exchange rate changes, Journal of Business 62, 339-368. Hsieh, D. A., 1991, Chaos and nonlinear dynamics: Application to financial markets, Journal of Finance 46, 1839-1877. Matsumoto, K. and I. Tsuda, 1983, Noise-induced order, Journal of Statistical Physics 31, 87-106. Ramsey, J. B., C. Sayers and P. Rothman, 1990, Bias in dimension calculations, International Economic Review 31,991-1020. Seheinkman, J. A. and B. Le Baron, 1989, Nonlinear dynamics and stock returns, Journal of Business 62, 311-337. Smirnow, W. I., 1968, Lehrgang der htiheren mathematik, 8th ed. (Springer, Berlin). Szpiro, G. G., 1991, Noise in autoregrcssive time-series, Economics Letters 37. 45-50. Szpiro, G. G., 1993a, Cycles and circles in roundofferrors, Physical Review E 47, 4560~563. Szpiro, G. G. 1993b, Measuring dynamical noise in dynamical systems, Physica D 65, 289-2'99. Whitney, H., 1944,The self-intersectionsof a smooth n-manifold in 2n-space,Annalsof Mathematics 45, 220-246.