Nonlinear pediction of the hurly FOF2 tme sries in cnjunction with the inerpolation of mssing dta pints

Phys. Chem .Earth (c), Vol. 25,No.4, pp. 261-265,200O CrownCopyright 0 2000 Published by Elsevier Science Ltd

Pergamon

All rights reserved 1464-1895/00/$ - see front matter

PII: Sl464-1917(00)00014-3

Nonlinear Prediction of the Hourly FOF2 Time Series in Conjunction with the Interpolation of Missing Data Points N. M. Francis’, A. G. Brown’, P. S. Cannon’, D. S. Broomhead3 ‘Radio Science and Propagation Group, 2Signal and Information Processing Group, Defence Evaluation and Research Agency, Malvem, Worcestershire, WR14 3PS, UK. 3Department of Mathematics, UMIST, P.O. Box 88, Manchester M60 lQD, UK Received 29 April 1999; accepted 20 September 1999

ABSTRACT. This paper proposes a novel technique for the prediction of solar-terrestrial data sets that contain a significant proportion of missing data points. A nonlinear interpolation technique is employed to assign values to gaps in a time series. Each missing point is interpolated such that the error introduced into any specific predictive function is minimised. Radial Basis Function’ Neural Networks are adopted for the purpose of prediction, and their advantages over their Multi-Layer Perceptron counterparts are outlined. This technique has general application in any instance where the effects of interpolation upon a given analysis process need to be minimised or a complete time series needs to be constructed from incomplete data.

timc series as u whole contains 6.6% missing data points. The first 23,ooO points have been used to train the model, while the remaining 3304 points were used to test the ability of the model to generalise to unseen data. In each instance, training and testing data were processed entirely separately; i.e. the sliding window used to create the input vectors wab not allowed to overlap both of the data segments at the same time. The effectiveness of the model has been assessed relative to the performance of an uninterpolated (completc input vector only) model, as wel1 as the relerence interpolated models, as shown in Figure 1.

Crown Coovrkht 8 2000 Published by Elsevier Science Ltd.

2.0 ANALYSIS TECHNIQUE

Key words: Ionosphere, network, data gaps.

A Radial Basis Function (RBF) Neural Network (NN) has been employed to provide the core modelling capability tor the interpolation process. This process is detailed comprehensively by Fruncis et al. [1998]. I’he most important advantage of this network over the nmre commonly used Multi-Layer Perceptron (MLP) architecture [Hertz et al., 19911 is the ability of the RBF [Btwrrriwtd and Lowe, 19881 to find a globally optimum solution, to ;I time series prediction problem, in a single pass process thal determines the appropriate model weights. MLPs can only produce locally optimum solutions through an itcrative process. Figure 2 details this hasic, uninterpolatcd (complete input vector only) modelling process, which acts as a bootstrap for the subsequent nonlinear interpolalio!] ot missing data points. Figure 3 details the nonlinear interpolation process. The first iteration of this process takes the uninterpolated RBF model and uses it to create and solve a constraincd optimisation problem using an iterative minimisation technique. The essence of the problem is that each missing point in the input time series needs to be interpolated such that the error introduced into the given RBF prediclive function is minimised. The interpolation process can then he iterated using the resultant interpolated input time series. This allows the interpolation process to use lhe information stored in the initially discarded input vectors to improve its

forecasting,

nonlinear, neural

1.0 INTRODUCTION Incomplete data sets are a recurrent problem associated with geophysical time series. Discontinuities pose a significant obstacle for prospective nonlinear prediction schemes, which generally require continuous data as a condition of their use. Relaxing this condition necessarily leads to adapting tbe predictive techniques to cape with this particular problem. Simple linear interpolation is undesirable because it can disrupt the delicate nonlinear dynamics within the data, thereby degrading the performance of the modelling process. A nonlinear interpolative scheme is presented that minimises the effect of interpolation upon any given modelling process. Specifically, this refers to the underlying modelling technique adopted for this study. However, it should be possible to adapt the interpolative technique to work with any predictive model that utilises an input time series. A nonlinearly interpolated model has been implemented using this scheme, for the prediction of hourly foF2 vaIues from 1971 to 1973 for Slough station, UK. The

261

0.6

0.8

Figure

1. Relrtive

Perfomnce

Hours ahead of Interpolation

Techniques

Completevectorsonly - - - - - - Nonlinearintefpolation -Zero interpolation ~~--. - Persistente interpolatior Recurrenceinterpolation

- - -

Prediction Accuracy for 1973 foF2 hourly values

Predictive RBF model constructed from time series and functíons.

wediction accuracv.

Choose fíxed no. of equally spaced functíons requíred to optímíse

Slough FoF2 time series de-meaned and reduced to unit varíance.

1

1 Figure

1

2. Flowchart Representing

Uninterpolated

Modelling

Models constructed to predict from 1 to 30 steps ahead.

Remove SVD components that only describe features that are training specífíc.

Data vectors constructed usíng optímal length sliding wíndow.

Process

error.

Results compared with persistente / recurrence model, usíng RMS

Data vectors projected on to princípal axes usíng SVD.

Data vectors that contaín míssíng data points are removed.

1

Create 3rd RBF model using second filled time series.

Fill gaps in original input time series using second gap filling model.

Create RBF with same parameters as basic uninterpolated model.

I

I

I

A

Interpolation

Modelling

Results compared with Reference interpolations, using RMS error.

Predict from 1 to 30 steps ahead.

Figure 3. Fiowchart Repmenting

[

Create 2nd RBF model using interpolated time series as new input.

Use nonlinear method to minimise RBF error due to interpolation of a gap.

Output time series retains gaps, to prevent feedback loop.

Perform one step ahead prediction.

N.

M. Franciset 01.:NonlinearRektion

ofHourlyFOF.2 Time Series in Conjunction with InterpolationOfMissing Data Points

and the accuracy of the model used in the process. Consequently, the iterative process wil1 continue to reduce the deleterious effects of interpolation until no more information can be extracted from t.be input time series. This point is reached when the RBF model error on the test set is minimised. Two iterations of the interpolation process were sufficient to achieve the optimal results for this study. Simpler nonlinear techniques, such as interpolating missing points using the prediction output from the RBF NN, were not found to give stable solutions. Once the simple model moves far outside of it’s normal operational regime (i.e. complete input vectors only), solutions often become asymptotic. Three simple reference interpolation schemes were adopted for tbe purpose of comparison with the nonlinear gap-fdling method. The first technique, the zero interpolation scheme, sets al1 missing points in the demeaned and normalised time series to a value of zero. The second uses persistente to fill in missing data points. The third and final technique uses the 24-hour recurrent structure of the hourly fol72 time series to replace missing points. In addition, the complete vector only (uninterpolated) RBF model is also included. However, the complete vector only test set is much smaller than the interpolated vector test set, due to the rejection of incomplete input vectors. Consequently, the results are not directly comparable with the other techniques.

interpolations minimisation

3.0 RESULTS Figure 1 shows a graph of root mean square (RMS) error, on the interpolated test vector set, plotted against number of hours ahead for tbe prediction. A curve is displayed for each of the three reference interpolation schemes, in addition to the complete vector only and the nonlinearly interpolated model. Using any of the proposed interpolation schemes, the input vector rejection rate falls from 60% to 6.6%. Residual rejection occurs because training vectors are stil1 rejected if there is a gap at the comesponding point in the required output time series (tbe output time series is identical to the input time series for a self-prediction problem). The graph clearly shows that the nonlinear interpolation technique consistently produces the best results in terms of RMS error. It shows a 2.3% improvement over the recurrence model, which characterises tbe dominant diumal variation of foF2, for each of the one to thirty hour ahead predictions. In turn, tbe recurrence model shows a similar improvement relative to the persistente model. The zero interpolation model falls signifícantly behind these reference models, indicating the necessity of adopting an approach to deal witb tbe problem of missing data points in geophysical time series. It is interesting to note that the nonlinearly interpolated model produced lower prediction errors, relative to the complete vector only model, when tested on the complete vector only test set instead of the interpolated vector test set.

265

~his shows the positive benefits that can be accrued, in terms of improved network training, through interpolation of the input time series. This point is further strengthened by the fact, on the same test set, the zero interpolation model performed less accurately than the complete vector only model. These results suggest that rejecting incomplete data vectors might be preferable to including them in a model, presumably because incomplete with poor training material.

vectors provide the network

4.0 CONCLUSIONS In summary, this paper describes a novel and general scheme to deal with the problem of non-contiguity in geophysical time series. This scheme can be used to

improve the viability, accuracy and generality of the underlying uninterpolated predictive model, which can then be used to provide real-time predictions. This baseline nonlinear interpolation scheme provides a 2.3% improvement upon simple schemes that rely upon persistente or recurrence properties of the observed ionospheric parameter. Currently, the optimal parameters for the nonlinearly interpolated model are the same as those derived for the complete vector only model, due to processing constraints. By re-optimising the model parameters at every step, it is likely that larger gains in predictive and interpolative accuracy can be made. Finally, in instances where simple climatological models cannot be used to provide sensible interpolations of missing points, this technique wil1 find the best interpolation solution for that particular problem. In such a case, the resultant improvement may be greater than for the prediction of the hourly foF2 time series, where the dominant diurnal variation of this ionospheric parameter is wel1 modelled by simple recurrence. 0 British Crown Copyright 1999/DERA Published with the permission of the controller of Her Britannic Majesty’s Stationary Office. REFERENCES Broomhead, D. S.. D. Lowe ‘Multivariable functional interpolation and adaptive networks’,Complex_srems 2,321-355. 1988. Brown, A. G., N. M. Francis, D. S. Broomhead,P. S. Cannon, A. AhaIn. ‘IS there Evidente for the Existente of Nonlinear Behaviour within the Interplanetary Sok Sector Structure?‘, J. Geophvs. ROS., 104, ,46. 12537-12547, June 1999.

Francis, N. M., A. G. Brown, P. S. Cannon, D. S. Broomhead, ‘Prediction of the Ionospheric Parameter, fol% on hourly, daily and monthly timescales’, Submitted to J. Geophys. Res., 1999. Hm, J., Gogh, A., Palmer,R. G., Introduction to the Theory of Neum/ Computation, Santa Fe Institute SNdies, Addison Wesley, 1991.