Improved cost functions for modelling of noisy chaotic time series

PHYSICA ELSEVIER

Physica D 109 (1997) 59--69

Improved cost functions for modelling of noisy chaotic time series Holger Kantz *, Lars Jaeger Max Planck [nstitute.]br Physics of Complex ,S~'slems, NiJthnilz.erStrasse 38, D Ol 187 Dresden, Germany

Abstract Standard least squares cost functions yield unbiased results only if the independent variables are noise free, which is not the case in time series analysis. New minimization problems for the determination of the dynamics underlying noisy chaotic data are formulated, which can overcome the problem of noise in the independent variables. For a given model, these improved cost functions give the chance to estimate its parameters with a strongly reduced bias in the case of large amplitude measurement noise. The method is illustrated by the help of numerical and experimental examples. Results for dymmaical noise are discussed. Keywords: Modelling; "lime series analysis; Noise

1. Introduction Low-dimensi0nal deterministic but chaotic systems create signals with an extremely complicated time dependence. Vice versa, signals from nature or laboratory experiments which are aperiodic and difficult to interpret might stem from a low-dimensional deterministic but chaotic system. Showing that this is the case, one can learn a lot about the underlying system, much more than if one just assumes some stochastic process being responsible for the irregulttrities. Methods to test for the presence of deterministic chaos and to extract the knowledge about the inherent determinism are known as nonlinear time series analysis. A pioneering work in this direction was Packard et al. [1 ], for a detailed presentation of the current state see [2]. The most useful and moreover most convincing result of such an analysis is the determination of the * Corresponding author.

deterministic part of the dynamics. When we succeed in constructing dynamic',tl equations which allow not only to make reasonable short time predictions but which moreover can be regarded as a good model with a long time behaviour which is compatible with the observed data, we did not only confirm the hypothesis of low-dimensional chaos, but moreover we cons~ructed a starting point for a deeper understanding of the underlying system. In contrast to the computation of the attractor dimension, which is commonly used to give evidence that data are confined to a low-dimensional manifold and therefore have to obey some deterministic rule, the extraction of the dynamics has the advantage to be very robust against all kinds o f defects of the data set. A meaningful dimension can only be attributed to data which represent the invariant measure on an invariant set. This requires absolute stationarity o f all system parameters, a sufficiently large data base to cover the attractor in a reasonable way, and the absence of transients or strong intermittency. Moreover, a noise level

0167-2789/97/$17.00 Copyright © 1997 Elsevier Science B.V. All rights reserved PI1 S0167-2789(97)00159-0

60

H. Kantz, L. Jaeger/Physica D 109 (1997) 59--69

of more than a few percent destroys all nontrivial self-similarity properties and makes it impossible to compute any finite dimension [3]. Similar problems arise when Lyapunov exponents are to be computed. The underlying deterministic dynamical equations, however, remain unaffected (at least approximately in the case of weak nonstationarity) and can be reconstructed in all these cases. Unfortunately, equations of motion have one severe drawback which one must keep in mind: they do not possess reasonable invariance properties. Whereas the fractal dimension or the Lyapunov exponents are invariant under smooth transformation of the state space and therefore can be used to characterize the underlying attractor and not only the given data set in the particular reconstructed state space we have chosen, the dynamical equations have to be transformed when transforming the state space. Another severe problem is the lack of structural stability of almost all real-world systems. We said before that we require a good model to reproduce the long term behaviour of the observed data, i.e. we require that the attractor of the model equations is as close as possible to the attractor of the observed system. In structurally unstable systems the slightest perturbation of any system parameter might change the attractor dramatically, and the standard situation is that stable periodic orbits are located everywhere in parameter space, in arbitrary close vicinity of chaotic attractors (Newhouse phenomenon) (for a detailed treatment, see [4] and references to Newhouse therein). In experiments, this instability is usually hidden through the fact that all system parameters fluctuate with small amplitudes and thus wash out such effects, but for the model systems this might be crucial. Stochastic stability is the mathematical counterpart to the robustness of experimental results. If one reinterprets small perturbations of the system state as perturbations of the dynamical equations, one speaks of a random dy= namical system. Averages with the invariant measure of such a random dynamical system converge towards the average on the invariant measure on the deterministic chaotic attractor in the limit of vanishing randomness. This property is proven to hold e.g. for the Hrnon map with particular parameter values [5], and

it is reasonable to assume its validity for a large class of systems. Therefore, despite the unsolvable problem of structural instability, it is obvious that the optimal model consists of the best achievable approximation to the unknown dynamics. A problem which is well discussed in the literature is the choice of a suitable ansatz, a suitable class of dynamical systems from which the model will be taken (e.g. [6-9]). Along with this goes the problem of overfitting and of model complexity, i.e. the consideration that the more free parameters a model has the better it should be able to describe the input data. This will not be the concern of this paper. We are interested in the question how for a given model one can determine its parameters in an optimal way. It will turn out that the answer is nontrivial if the data either contain a large amount of noise or the system is perturbed by interactive noise. In Section 2 we shall briefly show that the standard way for the construction of the dynamics leads to biased results when the observations are contaminated by a large amount of noise. We Shall present the solution in the form of a modified minimization problem. It will turn out that the determination of the optimal model is directly connected with the construction of the most probable noise free trajectory. Applications to experimental data will illustrate the problem and the solution. In Section 4 we briefly discuss the problem of noise interacting with the dynamics, which leads to nontrivial modifications of the shape of the attractor.

2. Biased results for noisy data: Errors in the independent variables Since equations of motion require a state space, the first step in their construction from observed data is the reconstruction of such a space. This is conveniently done by a time delay embedding [10] (but all what we will say in this article can be easily applied to derivative coordinates or vector-valued observations). Starting from a time series {Sn}, n = l, . . . . N, of scalar observations equidistant in time one forms delay vectors sn = (sn, s n - v . . . . , sn-(m-1)v), where the integer

H. Kantz, L. Jaeger/Physica D 109 (1997) 59~59

lag v in the simplest case is unity and for flow data has to be chosen appropriately. According to the theorems [10], an embedding dimension m > 2Dr is sufficient, if the data stem from a deterministic dynamical system and are confined to an attractor with fractal dimension Df. The simplest way to write an equation of motion in the reconstructed state space is

61

6.05

6 5.95 5.9 -~ ~

\

5.65

\

\

\

\

\

\

\

~.8

Sn+l = F(sn).

(1)

5.7

If our state space is a delay embedding space, all cornponents of Sn+l apart from the first one are trivially known from sn, so that all information is given by the map

5.65 6.8

Sn+l = f(Sn).

(2)

The standard way to determine the scalar field f from the data is to minimize the one-step-prediction error e 2 = E ( S n + l --

f(Sn)) 2

(3)

with respect to f , or, more precisely, with respect to parameters in a suitable ansatz for f (e.g. [6-9]). This cost function of least squares type can easily be derived by a maximum likelihood approach under the assumption of Gaussian distributed errors in the dependent variables sn+l. If additionally the unknown function can be accurately reproduced by the ansatz f , the parameters thus determined are unbiased and consistent. In time series analysis, the most important assumption obviously is violated: if the dependent variables Sn+l are noisy, then necessarily also the components of the independent ones {sn }j are noisy with the same noise amplitude. This leads to systematically wrong results for the estimated parameters even in the limit of infinitely many data. One can very easily check this for a linear relation [12], and more drastically wrong results are obtained when f is nonlinear. For the estimate of tangent maps this problem is treated in [11]. As an example, we present in Fig. 1 the analytic result of the estimation of the parameter k of the standard map,

Xn+l = 2Xn + k sin Xn

Xn-1

(4)

\

\

5.75

\

\

\

\x

\ 0.05

i 0.1 noise level (rrns)

0.15,

0.2

Fig. 1. Analytic results (continuous line J for the parameter k of the standard map, Eq. (4), obtained by minimization of the one-step-prediction error from infinitely long trajectories with the indicated amount of white Gaussian measurement noise, and numerical estimate on a finite sample, using the Euclidean distance Eq. (5).

as a function of the noise level. Using Eq. (4) to model data generated by the same equation with given but unknown k0 and contaminated by additive measurement noise with variance ~r2, we find systematic and significant deviations of the estimated k from ko. The analytic result of the straightforward solution of this minimization problem is plotted in Fig. 1. Here, as in the remainder of the paper, the noise level is measured in the root mean square sense, i.e. it is the ratio of the standard deviation of the noise and the standard deviation of the unperturbed signal. If noise is contained in all variables, the appropriate minimization problem is given by the total least

squares e 2 - - ~ _ d 2, dn = min[(sn+l

f ( y ) ) 2 + (Sn --y)2],

(5)

Y

which represents a minimization of the orthogonal Euclidean distances between every noisy point (Sn+l, Sn) and the hyper-surface in ~m+l defined by f . Minimizing this cost function for the standard map with observational noise leads to the parameter values indicated by diamonds in Fig. 1. Since these results stem from numerical minimizations (the new cost function is nonlinear in the parameters of the

62

H. Kantz, L Jaeger/Physica D 109 (1997) 59~59

map and cannot be solved analytically any more), the results fluctuate due to statistical errors. It is obvious that on the one hand the results are by far closer to the true value k0 = 6, but that on the other hand for large noise levels a tendency towards too small values remains. This latter bias becomes more relevant, if the model equation contains more freedom than necessary to approximate the data, i.e. when there is an excess of free parameters. At first sight this looks like a problem of overfitting. But if the precise function to be modelled is unknown (as it is always in realistic situations), one can hardly avoid more parameters than strictly necessary. Thus one has to construct an even more suitable minimization problem for less biased results.

3. More-step errors The minimization of one-step errors, regardless of whether the total least squares or the standard cost function are used, reduces the problem to a geometrical one. The delay vectors in the extended (m + D-dimensional delay embedding space are treated as a cloud of independent points, which are to be approximated by the hypersurface yn+l = f(Y). The underlying assumption for the maximum likelihood principle that leads to the cost functions in Eqs. (3) and (5) is that the points s~ in the time series are uncorrelated: The probability of a certain model f to be the origin of the given data set {sn } is given by

P({si}, f ) o~ H e(Si+1-f(si))2/2ff2

(6)

i

This approach obviously ignores the fact that all data stem from the same single time series, such that points in this space with subsequent time indices are correlated by exactly the same dynamical relation we hope to find by surface approximation. Let us consider a function f which is slightly different from the true dynamics. Thus the one-step error is small. But under iteration of f , the difference between the images of a point and the subsequent delay vectors constructed from the measurements will increase sys-

tematically and faster than under the true dynamics, Thus, a sub-optimal f will result in large distances between sn+k and the kth image of the vectory which defines the distance between sn and the hyper-surface, y:((Sn - y ) 2 + (sn+x - fly))2) = min. In chaotic systems errors in initial conditions grow exponentially fast. Thus the latter statement is true also for the original unknown dynamics fo, but on the average over many nearby points Sn these effects cancel, whereas a slightly wrong f leads to systematic deviations which should be minimized. Signatures of this mutual inconsistency are discussed in more detail in [12]. Following these considerations, we introduce an accumulated more-step Euclidean cost function: N k e 2 =-- ~ rainy ~ I(Sn+i, Sn+i-1) n=l i=1 - - ( f (F (i-l) (y)), F (i-1) (y) )12,

(7)

where F(y) = ( f (y), Yl, Y2. . . . . Ym-1) is the action of f on a vector in the delay embedding space. Thus for every noisy point sn ~ a vector y is sought, which under iteration of the dynamics f leads to a segment of a trajectory which during k time steps is optimally close to the observed sequence of v e c t o r s {$n+i, i = 1. . . . . k}. The minimization of e 2 with respect to the parameters of f can be successfully done numerically for k of the order of 15.

3.1. Numerical examples Let us introduce a specific L2-distance between the (in numerical simulations) known function f0 and the optimal f obtained by a fit. Two aspects are relevant for its definitionl First, the dynamics f are determined by the minimization problem only in those regions of the delay embedding space, where the observed points are located. In other regions the behaviour of f cannot be expected to be close to f0. Second, f is obtained in the delay embedding space, whereas most model systems are defined in some state space which cannot be easily transformed into a delay embedding space analytically. Therefore we define the distance

H. Kantz, L. Jaeger/Physica D 109 (1997) 59-69

63

0.07

f d 2 -= ] (fo(X) -- f ( x ) ) 2 d/z(x)

0.06 N

L ~(fo(X.n ) _

f ( X n ) )

0.05

2

N

N = L ~(Xn+l N n=l

g

-- f ( X n ) ) 2,

(8)

i.e. the distance is defined to be the average one-step prediction error of the fit f on an unperturbed trajectory of the model, such that we do not need to know the unperturbed dynamics in the delay embedding space. We also compare global properties o f the dynamics, such as the attractor geometry or the Lyapunov exponents which we find when iterating f . Details o f the shape of the attractor depend quite sensitively on f , such that its similarity to the attractor formed by fo or even its noisy version is also a good error measure for f , but it cannot be easily turned into a quantitative concept. We create data of the Hdnon map, Xn+l = 1 1.4x 2 +0.3Xn-1, contaminate them with uniform white noise, and perform the fits using the standard onestep prediction error and the Euclidean cost functions Eq. (7) for k = 1 . . . . . 10. We do not assume that we know the structure o f the H t n o n map, but instead use a general ansatz, which we chose to be a neural net with four neurons in one hidden layer for f , i.e.

f (x)

0.03

0.02

0.01 ......,

_.

-

. . . . . . ~ •- . - L - ~ : - . _ = . ~ . . ~ . . - ~ - - ~ . ~ = ~

. . . . .

0 0.05

0

0,1

0.15 0.2 noise level (rms)

0.25

0.3

0.35

Fig. 2. The L2-distance between the dynamics obtained by a fit and the Htnon dynamics underlying noisy data. The fitted function is a neural net with four neurons, obtained by minimization of the standard one-step prediction error and k-step accumulated Euclidean errors, k = 1, 2, 4, 6, 8, 10, from upper left to lower right. 0,5

..

~..

' =

~

'

~-'_

...

' _-

=..'

' I

..~:=,--.~--.~

0

-0.5 o

~o ==

-1

5"

-1.5

/

/

/

-2

4

al Z.~ 1 + exp(btx + Cl)

0.04

(9)

l=1

in the two-dimensional delay embedding space. We employed a conjugate gradient minimization scheme in the space of vectors Yi and parameters of the function f to obtain the minimum o f (7). The results for the L2-distances defined b y Eq. (8) are shown in Fig. 2, from w h i c h it becomes obvious that the minimization of the accumulated more-step errors indeed can cope with large amplitude measurement noise. In accordance with this, the properties of the attractors found under iteration of f are much closer to the original H6non dynamics, as illustrated by the Lyapunov exponents in Fig..3. To recall, the deviations between f and f0 found by the minimization of the standard onestep error (most left line in Fig. 2) are of systematical

-2.5

f

i

0.05

0.1

i

~

0.15 0.2 noise level (rms)

i

i

0.25

0.3

"; ~

r 0.35

Fig. 3. Lyapmlov exponents of the dynamics obtained from fits of noisy H6non data as in Fig. 2. The ordering is again from left to right, i.e. the curve first deviating from the accurate value )~+ ,~ 0.42 corresponds to the standard one-step error, and the curve last deviating to the 10-step accumulated Euclidean error. All curves are truncated at noise levels where the attractor becomes a periodic orbit.

nature, and the general tendency o f these results does not depend on the particular class of functions chosen for f . In fact, very equivalent dependencies are even found, if f is chosen to have exactly the structure o f the H6non map with unknown parameters. For this case, the bias in the estimate o f the H t n o n parameters

64

14. Kantz, L. Jaeger/Physica D 109 (1997) 59-69 I

• .. ,~;:.:~(~

.

0.5

:.-,.%c~4~.~

~-;-

~

,

i -0.5

,~ ,:7~

-1

-I

: " ::: ~:-"~i~,'~"-'~ .... •

,

..:..~... ,~.: ,,z,:~,-/.,

-1.5

-15 -.

.2

,,&

;-:-.. +.~-~%,~.~ ;2~s--~ ~

i ,,0.4

i -02

,i 0

i G2

i 0.4

i. 06

-.:

...

i O.6

~ 1

i 1.2

, ,

i 1,4

,,0.4

, "02

O

. 0 2

,

.

04

0.6

.

. 0.9

. I

t2

14

i

i

i

(b) ...i., •

i"

.:~ 4 T ~ : : . ~ ] . ' . . , ~

..~':*Y

'.

. . . J ~ %~, . . . = ~ " ....... ~.,",,,,

.~:,~..,!>

0 0

• ~7. f,:7" ~ ">.-.-" s ""~l" 1 1 ; 7 , ~ / - N "~ I ,.:~ ! ~ .,,; , ~. j ;

J -G5

:~ ',.%

"O.S

~:,.. "~':;,~- ~;,..,

-1

,,~:,.. "-;':,.~.

...........

.1.S

.,s -0.I....

"02

0

(c)

,

i

02

•

04

=

'

6.6

0

i,

'

1

'

12

i

14

0.3304

"

. -0.4

. -02 .

(d)

.0

02

,~ =

i

04

0

is

~.

.

1.

. 12.

1.4

0.4342

Fig. 4. Attractors of the reconstructed dynamics for the noisy Ikeda-auractor (a) with the parameters p = 0.92, B =- 0.9, x = 0.4 and c~ = 6.0 (10% white, uniform noise). The clean attractor is given in (b). (c) Usage of the standard cost function Eq. (3). (d) Usage of the multi-step prediction cost function Eq. (7) with k = 6. As an ansatz we choose a neural net with 20 neurons.

obtained from the minimization of the standard onestep error was computed analytically in [12]. As another, more complicated example we chose the Ikeda map

Zn+l = P q- BZn ei~-ia/(l+lz'[2)

(10)

(Z 6 77) with parameters p = 0.92, B = 0.9, tc -0.4 and ~ = 6.0, and contaminate them with uniform white noise. The treatment of a multivariate time

series can be done analogously to the above discussion and is further e l a b o r a t e d in [13]. Again, we do not a s s u m e that we know the structure of the Ikeda map, but instead we use as an ansatz f o r the function f = ( f l . . . . . f N ) a neural net with 20 neurons. The result is given in Fig. 4. W e also indicated the positive L y a p u n o v exponents of the iterated d y n a m ics. The L y a p u n o v exponent of the original noise free trajectory is )~ = 0.436. A clear i m p r o v e m e n t between

H. Kantz, L. Jaeger/Physica D 109 (1997) 5949 ~00 i

~

L

,

,

0

leO0

65

-4000

.4000 -3000 -2QOO -1000

2000 ,~000

(b)

i - . . ~ ¸.......

0 ..............................................................................................

-1(110

-1000

"x

.31~

!

\ i -30OO ~

-1000

0

1000 2O0O 3OOO

(c)

(d)

Fig. 5. Series of attractors of the NMR-laser data, resulting from an iteration of the dynamics from the minimization of Eq. (3) (Panel c) and the orthogonal three-step errors (Panel d), together with the noisy data used for the fit (Panel b) and the original data before contamination with additional noise (Panel a).

the standard one-step-prediction cost function and the minimization of the multi-step-prediction cost function can be observed. 3.2. Experimental data W h e n dealing with experimental data the model verification is less straightforward. To demonstrate the relevance o f our results, we can follow two ways. We can use relatively clean experimental data, where the standard fit yields reasonable results, contaminate them numerically with nOise and repeat the fitprocedure. This we did using the experimental data from the Ziirich N M R - l a s e r experiment o f the group of Prof. Brun in Ziirich. After noise reduction of the al-

ready very clean data 1 we can fit the data very nicely both with R B F and a multivariate polynomial of degree 7 in a three-dimensional delay embedding space. The attractor created b y this model dynamics is almost indistinguishable from the attractor formed by the experimental data shown in Fig. 5, Panel a. After numerical contamination with 7.5% Gaussian measurement noise (Fig. 5, Panel b) and u s i n g again an RBF-ansatz in three dimensions (centers positioned on all nonempty boxes of a 53 grid), the minimization of Eq, (3) does not yield any reasonable dynamics (for most numbers o f centres/widths of the basis functions

1 For details about the data, see [14] and references therein concerning the experiment.

66

H. Kantz, L. Jaeger/Physica D 109 (1997) 59-69

" ~

..

~o

~:~10

~oo

esoo

2~2r.~

~zgO

~

26~0

2629

2~30

2~40

~SO

Fig. 6. The INO laser data in a suitable Poincar6 surface of section (sufficient signal-to-noise ratio), and the attractor found by iteration of the dynamics obtained by minimization of the standard one-step prediction error. )

ZS3S

:ii:t~'-

2~0 ~S

'i:-:, v, . ,..~

2SlS

. ...:.

2SIS 2S10

~0

".:~.~.

,~,.

. --. ,: '.

zsoo 24~

24 24eo

~OS

24~

~oo

~

2s~o

2sls

2s2o

2sas

as~

~

2S.~S

2s4o

i 24~

i ~

, a~os

i 2slo

, ~s~

I z~o

i ~

, ~

, ~s

2s, o

\

~SlO ~SCS ~'SO0

Fig. 7. The INO-laser data in an unreasonable Poincar6 surface of section (insufficient signal-to-noise ratio), and attractors of dynamics obtained by the minimization of the standard one-step prediction error and of the accumulated 4- and 6-step Euclidean error (from left to right).

it yields periodic orbits or c o m p l e t e l y d e f o r m e d ob-

on the t w o sections cannot but differ, but their c o m -

jects, see Fig. 5, Panel c). The Euclidean errors in-

plexities are identical.

stead, and in particular the three-step error, y i e l d again

The data w e want to use for this purpose stem

very reasonable attractors, w h i c h w e s h o w in Fig. 5, Panel d.

from a laser experiment with feedback, run at the

Alternatively, w e can l o o k at a single experimental data set in different Poincar6 surfaces o f section without adding artificial noise. In this case the d y n a m i c s

National Institute o f Optics (INO) at Florence, Italy, by Arecchi et al. [15]. With a s a m p l i n g rate o f about 25 m e a s u r e m e n t s per internal period the data represent a flow on a (2 ÷ E)-dimensional chaotic attractor.

67

H. Kantz, L. Jaeger/Physica D 109 (1997) 59-69 2

2

%5

1.5

1

1

0.5

0.5

0

0

i

-O.5

/ i , / ; " ~ \

-0.5

.',.,. -1

-1

-1.5

-1.5

-2

-2

r -1.5

a -1

i ~.5

i 0

i 0.5

f 1

i 1.5

-2 -2

i

I

i

-1.5

-1

-0.5

i

0 "

t

i

i

0.5

1

1.5

Fig. 8. First panel: The noisy Htnon attractor Xn+ 1 = 1 , 1 . 4 X n + 0.3Xn_l q-~n (~n white noise, uniform distribution) (cloudy points)

together with the attractor of the unperturbed map (sharp lines). Second panel: Attractor of a neural net whose parameters are fitted by minimization of the 12-step accumulated Euclidean error. On a suitable surface of section, we find a relatively neat, almost one-dimensional map. Since we know that the flow lives in an at least three-dimensional space and, moreover, is invertible, we do not model the data by such a map. It is more natural to work in a twodimensional embedding, which aiso gives room for fractal structures. Using a neural net with six neurons for f : ( s n , sn-]) ~ Sn+l we receive reasonable results by minimizing the standard one:step prediction error. The attractor of this map is shown in Fig. 6. If instead we rotate and shift the surface of section such that the attractor in the section is very thin, more precisely, such that the variance of the data in the section is small, the relative noise level is high. Although the data represent the s a m e dynamical system with identical entropy and dimension, now the minimization of the standard one-step prediction error does not yield a reasonable model any more. Instead, the dynamics obtained from a minimization of accumulated k-step errors yields good results, if we choose k between 5 and 10 (Fig. 7).

4. Outlook: Interactive noise Up to now we discussed noise which just contaminates the measurements and thus reduces our knowledge about the states of the system, but does

not interfere with the deterministic dynamics itself. If the latter is the case, noise is called interactive or dynamical. For axiom A systems and Anosov flows the shadowing theorem [16] states that it can basically be interpreted as a particular kind of measurement noise. For nonhyperbolic systems, however, this is no longer true. In this case, even far from parameter values at which the attractor undergoes obvious bifurcations, dynamical noise can drastically change the shape of attractors. This is d u e t o homoclinic tangencies, where stable and unstable manifolds are tangential to each other, and where therefore perturbations transverse to the attractor cannot be attracted back to the attractor in the vicinity of the images of the unperturbed trajectory. As a typical example'we show in Fig. 8 (first panel) the H t n o n attractor with 3% interactive noise, together with the attractor of the unperturbed map. The prolongations ('noise tails') appear at images of the primary homoclinic tangencies and their occurrence, their structure and their quantitative properties are thoroughly examined in [18]. In the presence of dynamical noise, the goal of modelling is less well described. The optimal model would yield the deterministic part of the dynamical equations, the details of the noise process (distribution, temporal correlations, higher moments), and the interaction between them. It is obvious that this is too ambitious in many situations, and from a more practical point of

68

H. Kantz, L. Jaeger/Physica D 109 (1997) 59-69

view one might ask for a deterministic dynamical system which can reproduce the observed attractor up to m e a s u r e m e n t noise. This asks for a generalized version of shadowing, shadowing of trajectories of the system subject to dynamical noise by unperturbed trajectories of a 'nearby' deterministic system. For maps on the interval, this possibility was explored theoretically in [17]. The minimization of accumulated k-step errors Eq. (7) by construction yields deterministic dynamical equations which allow exactly for this shadowing property during short time intervals, namely during k time-steps. In Fig. 8 we show the attractor obtained under iteration of the dynamics found by minimization of a 12-step error. More details including quantitative comparisons will be published elsewhere [19]. A n experimental verification of the noise induced attractor deformation is currently in progress.

5. Conclusions Modelling based on time series data always has to cope with the problem of errors in the independent variables of the dynamical model. A minimization of the standard one-step prediction error therefore yields biased results, which become dramatic if the noise level exceeds a few percent of the signal's standard deviation, independent of the chosen functional form of the model. This problem can be overcome by a combination of total least squares (which symmetrizes the role of the different variables) and more step errors, which allow to exploit further dynamical correlations without using a more complex model class. We have demonstrated the success of this modified minimization problems and their practical applicability. In the case of dynamical noise, they allow for the construction of effective models, which take into account attractor deformations due to nonhyperbolicity.

Acknowledgements We gratefully acknowledge the cooperation of T. Arecchi, M. Ciofini, and R. Meucci from the INO,

who generously supplied us their excellent experimental data.

References [1] N.H. Packard, J.R Crutchfield, J.D. Farmer and R.S. Shaw, Geometry from a time series, Phys. Rev. Lett. 45 (1980) 712-716. [2] H. Kantz and T. Schreiber, Nonlinear Time Series Analysis, Nonlinear Science Series No. 7 (Cambridge University Press, Cambridge, 1997). [3] H. Kantz and T. Schreiber, Dimension estimates and physiological data, Chaos 5 (1995) 143. ]4] J. Palis and F. Takens, Hyperbolicity and sensitive chaotic dynamics at homoclinic bifurcations, Cambridge Studies in Advanced Mathematics (Cambridge University Press, Cambridge, 1993). [5] M. Viana, personal communication. [6] J.-P. Eckmann, S. Oliffson Kamphorst, D. Ruelle and S. Ciliberto, Liapunov exponents from time series, Phys. Rev. A 34 (1986) 4971-4979. [7] D.S. Broomhead and D. Lowe, Multivariate functional interpolation and adaptive networks, Complex systems 2 (1988) 321-355. [8] M. Casdagli, Chaos and deterministic versus stochastic nonlinear modelling, J. Roy. Star. Soc. B 54 (1991) 303. [9] R. Gencay and W.D. Dechert, An algorithm for the n Lyapunov exponents of an n-dimensional unknown dynamical system, Physica D 59 (1992) 142-157. [10] E Takens, Detecting Strange Attractors in Turbulence, Lecture Notes in Mathematics, Vol. 898 (Springer, New York, 1981); T. Saner, J. Yorke and M. Casdagli, Embedology, J. Stat. Phys. 65 (1991) 579. [11] A.G. Darbyshire and D.S. Broomhead, Robust estimation of tangent maps and Lyapunov spectra, Physica D 89 (1996) 287. [12] L. Jaeger and H. Kantz, Unbiased reconstruction of the dynamics underlying noisy chaotic data, Chaos 6 (1996) 440. [13] R. Hegger, L. Jaeger and H. Kantz, Efficient reconstruction of the dynamics of multivariate time series, preprint MPIPKS. [14] H. Kantz, T. Schreiber, I. Hoffmann, T. Buzug, G. Pfister, L.G. Flepp, J. Simonet, R. Badii and E. Brun, Nonlinear noise reduction: A case study on experimental data, Phys. Rev. E 48 (1993) 1529. [15] ET. Arecchi, W. Gadomski and R. Meucci, Generation of chaotic dynamics by feedback on a laser, Phys. Rev. A 34 (1985) 1617; M. Ciofini, R. Meucci and F.T. Arecchi, Experimental control of chaos in a laser, Phys. Rev. E 52 (1995) 94. [16] B.V. Anosov, Proc. Steklov Inst. Math. 90 (1) (1967); R. Bowen, J. Differential Equations 18 (1975) 333. [17] H.E. Nusse and J.A. Yorke, Comm. Math. Phys. 114 (1988) 363-379.

H. Kantz, L. Jaeger/Physica D 109 (1997) 59~59 [18] L. Jaeger and H. Kantz, Homoclinic tangencies and nonnormal Jacobians - effects of noise in non-hyperbolic chaotic systems, Physica D 105 (1997) 79-96.

69

[19] L. Jaeger and H. Kantz, Effective deterministic models for chaotic motion perturbed by interactive noise, Phys. Rev. E 55 (1997) 5234.