- Email: [email protected]
Neural network approach to forecasting of quasiperiodic financial time series
European Journal of Operational Research 175 (2006) 1357–1366 www.elsevier.com/locate/ejor
Neural network approach to forecasting of quasiperiodic financial time series Yevgeniy Bodyanskiy, Sergiy Popov
*
Control Systems Research Laboratory, Kharkiv National University of Radioelectronics, 14 Lenin av., Kharkiv 61166, Ukraine Available online 16 March 2005
Abstract A novel neural network approach to forecasting of financial time series based on the presentation of the series as a combination of quasiperiodic components is presented. Separate components may have aliquant, and possibly non-stationary frequencies. All their parameters are estimated in real time in an ensemble of predictors, whose outputs are then optimally combined to obtain the final forecast. Special architecture of artificial neural network and learning algorithms implementing this approach are developed. 2005 Elsevier B.V. All rights reserved. Keywords: Time series; Forecasting; Finance; Neural networks; Combining of forecasts
1. Introduction Forecasting is a key element of financial and managerial decision making. This is not surprising, since the ultimate effectiveness of any decision depends upon a sequence of events following the decision. The ability to predict the uncontrollable events prior to making the decision should enable an improved choice over the one that would otherwise be made. The main purpose of forecasting is to reduce the risk in decision-making that is especially important for organizations, firms, and private investors, whose activity is connected with financial markets. Here the prediction is made for financial time series (stock prices, currency exchange rates, price indices, and so on), known to be very complex, difficult for econometric modeling, non-stationary, very noisy, badly fitted by linear models [2]. Moreover, there exist many interdependencies between different financial markets, various market sectors, financial assets, etc. that make the problem even more complicated. *
Corresponding author. Tel.: +380 57 7021890. E-mail addresses: [email protected] (Y. Bodyanskiy), [email protected] (S. Popov).
0377-2217/$ - see front matter 2005 Elsevier B.V. All rights reserved. doi:10.1016/j.ejor.2005.02.012
1358
Y. Bodyanskiy, S. Popov / European Journal of Operational Research 175 (2006) 1357–1366
The problem of economic and financial forecasting attracts rapt attention of researchers. During last decades, several approaches evolved that differ in goals of forecasting, nature of the information used, and mathematical apparatus. Historically first, approach to social-economic and technological forecasting based on tendencies analysis was formed [1,18,32]. This approach relies to a considerable extent on the apparatus of expert judgments and mathematical programming. As follows from its name, it estimates long-term outlooks of complex systemsÕ evolution. Naturally, it does not provide short-term predictions of some specific indexes in real time. First successful results in financial and economic indexes forecasting given in the form of time series were obtained using classic statistical approach and first of all based on regression, correlation and spectral analysis [11,17,21]. Within the framework of this approach, studies upon development of forecasting econometric models were carried out in parallel [16,19,25]. For the problems of short-term forecasting, regression and econometric models proved inefficient since they allow only averaged trends assessment (usually polynomial) on a rather long time intervals. Correlation and spectral models turned out to be more successful, but their use is limited by statistical premises they rest upon: hypothesis of a rather strong autocorrelation within a series, stationarity, linear structure, Gaussian nature of disturbances distribution. As a rule, in reallife economic and financial series, these conditions are not satisfied that leads to decrease of prediction quality. The Box–Jenkins forecasting approach [10] became a quintessence of the above approaches. It is based on autoregression-integrated moving-average (ARIMA) models including polynomial and seasonal trends. And though this method is more computationally and functionally effective, it is based mostly on correlation analysis it has all its drawbacks. Development of adaptive approaches [3,15,24] became a breakthrough in time series forecasting. Here, two classes can be emphasized: procedures based on exponential smoothing and forecasting based on learning models. These methods proved effective in the conditions of lack of a priori information about the seriesÕ possible structure, non-stationarity, small amount of data, sudden changes. They are rather computationally effective and have high convergence rate since they are based on optimal recursive procedures of adaptive identification. But in the same time, the problems of modelÕs (still linear) structure substantiation are not solved yet. Intellectual forecasting methods are the radical development of the above approaches. Here, the apparatus of artificial neural networks (ANNs) [9,14,22,26,33,35] must be emphasized that offers a natural alternative to traditional forecasting techniques. ANNs have three great advantages over traditional methods: • they have universal approximation capabilities; • they can recognize ‘‘on their own’’ implicit dependencies and relationships in data; • they can ‘‘learn’’ to adapt their behavior (their prediction) to changed conditions quickly and without complication. These important capabilities of neural networks can solve, at least in principle, some of the above-mentioned problems. ANNs offer many attractive features when attempting to model poorly understood problem domains. And therefore, over the last years, they have established themselves as credible financial forecasting techniques. At present, neural networks are one of the most successfully applied technique in the financial domain. The most difficult part of using ANNs in financial forecasting problems is to choose ‘‘the best’’ network providing the ‘‘optimal’’ prediction for every particular index. At present, in financial forecasting problems, quite a few types of traditional ANNs are used, i.e. multilayer perceptrons (MLP), Kohonen maps (SOM), dynamic recurrent nets (DRNN), radial basis function nets (RBFN). Hereby, the sequence being forecasted is assumed to be quite precisely describable by non-linear difference autoregression moving average equation (NARMA) with non-stationary parameters. There are no recommendations on how to choose ‘‘the
Y. Bodyanskiy, S. Popov / European Journal of Operational Research 175 (2006) 1357–1366
1359
best’’ from the listed types of ANNs, and such a choice can only be made for a particular series. It is important to note that for any other data the chosen network may not provide the required prediction quality. Moreover, known structures like MLP, RBFN, SOM, DRNN absolutely do not take into account specific issues of financial forecasting that leads to loss of very important information. One of such important features of financial time series that cannot be revealed by conventional approaches is the presence of oscillatory or chaotic components [12,33], which cannot be reliably detected by Fourier analysis because of the frequencies deviation. The use of so-called Fourier-nets for prediction of such series is also impossible because these ANNs imply the knowledge of the number of harmonics and their frequencies a priory. It is also believed that financial time series often obey not NARMA but non-linear fractal equations of Mandelbrot structure [23], which also cannot be revealed using traditional ANNs. As it is impossible to find the best network in advance and, moreover, on different time frames these could be different structures, it is reasonable to use different networks including non-traditional ones in parallel with the aim to increase the precision and reliability of the prediction. During the last years the approach has evolved, related to the use of an ensemble of different networks and finding the best prediction among them [6,27,31]. There also exist methods that change network structure (e.g. add or delete neurons) during the process of training with the ultimate goal of finding an optimal network for a particular data set. These methods cannot be used in the considered problem for at least two reasons: (1) they all work in off-line (batch) mode; (2) if the time series properties vary in different regions of the data set, there is no single ‘‘best’’ structure for the whole series. In this paper, we present a novel approach to financial time series forecasting that addresses most of the aforementioned difficulties. We apply a special artificial neural network specifically designed for prediction of time series containing oscillatory components. This network contains an ensemble of predictors of different orders that are capable to efficiently deal with such signals using their decomposition into quasiperiodic components, i.e. with different, aliquant, possibly non-stationary frequencies. Since the structure of the predicted series is not known in advance and it may change during observation, the multimodel adaptive generalization of predictions [6] is used, which provides optimal selection of the best prediction in real time. The architecture is easily scalable so that any other predictors can be included in the ensemble, e.g. other ANNs or even expert judgments.
2. Description of the approach The time series under analysis is considered as a signal containing quasiperiodic components [20,34] m X sðkÞ ¼ a0 þ ðaj cos xj k þ bj sin xj kÞ; k ¼ 1; 2; . . . ; t; j¼1
where m—the number of components, which is not known in advance, 1 6 m 6 M; M—the maximum possible value of m; a0—absolute term; aj, bj—unknown, possibly non-stationary parameters of separate components; 0 < xj = 2pfjT0 < p—unknown aliquant frequencies to be estimated; T0—sampling period, observed against the noise background yðkÞ ¼ sðkÞ þ nðkÞ; where n(k)—stochastic component: white noise with zero mean and bounded variance, and given as a set of t equidistant points y(1), y(2), . . ., y(t). In many situations a0 may be subject to some kind of trend, then appropriate detrending method should be applied to eliminate it. As the key point of this paper is prediction of quasiperiodic components, we restrict our consideration to the case with constant a0.
1360
Y. Bodyanskiy, S. Popov / European Journal of Operational Research 175 (2006) 1357–1366
The problem of modelÕs parameters identification can be solved with the multistage procedure introduced in [20]. This includes parameters estimation of an intermediate model with eliminated absolute term a0; unknown frequencies xj estimation; coefficients aj, bj evaluation; and, finally, prediction itself. The problem is complicated by the fact that the modelÕs order m must be known exactly, otherwise the problem of selection from M possible structures arises, e.g. using the group method of data handling, that is quite cumbersome, especially when M is big and m may vary during the observation (structural changes). Moreover, if the data sample is growing in real time k = 1, 2, . . ., t, t + 1, . . . and parameters aj, bj, xj are non-stationary, the traditional approach may not work at all. A good alternative is to take advantage of adaptive identification and prediction methods with neural network parallel processing technologies [8]. This approach is rather capable and universal, but certain applications may require computationally simpler procedures, especially when real-time processing is of concern. For this purpose, a feedforward neural network for multimodel prediction of quasiperiodic sequences is proposed, which is shown in Figs. 1 and 2 for the case of M = 4. Note that the network in Fig. 2 has a cascade structure, i.e. increasing the order M does not change the previously formed architecture but only adds new neurons. The input sequence y(k) is processed by the input neuron implementing the exponential smoothing algorithm to detect the mean of the signal y ðkÞ ¼ a0 y ðk 1Þ þ ð1 a0 ÞyðkÞ
ð1Þ
with large values of a 0 (0 6 a 0 6 1; for constant mean, a 0 is defined by (k 1)k1). This operation performs linear detrending, which is sufficient to eliminate absolute term a0 even if it is subject to small drift. In case of the presence of more complex trends in the signal, other detrending techniques may be used. For example, for polynomial trends, higher order exponential smoothing may be appropriate. Then elementary operation is performed y 0 ðkÞ ¼ yðkÞ y ðkÞ;
ð2Þ
0
after that the signal y (k) with eliminated absolute term is again processed by the similar neuron y 00 ðkÞ ¼ a00 y 00 ðk 1Þ þ ð1 a00 Þy 0 ðkÞ;
a00 a0
ð3Þ
with the purpose of random disturbances n(k) smoothing. Then the smoothed signal y00 (k) is fed to the neural network ANNQP, designed for quasiperiodic analysis of the signal.
1
1 1
Fig. 1. Artificial neural network for quasiperiodic sequences prediction.
1
Y. Bodyanskiy, S. Popov / European Journal of Operational Research 175 (2006) 1357–1366
1361
1 1 1
2 1
3
1
1 4 1
1
1
8
Fig. 2. Artificial neural network for quasiperiodic analysis ANNQP for the case of M = 4.
In the design of ANNQP, it is assumed that the signal y00 (k) can be presented in the form m X ðaj cos xj k þ bj sin xj kÞ; m ¼ 1; 2; . . . ; M; k ¼ 1; 2; . . . ; t y 00 ðkÞ ¼
ð4Þ
j¼1
or (which is the same) m Y ð1 2 cos xj z1 þ z2 Þy 00 ðkÞ ¼ 0;
ð5Þ
j¼1
where z1—back-shift operator defined so that z1y00 (k) = y00 (k 1). Description (5) corresponds to the difference equation y 00 ðkÞ ¼
m1 X
bm;jþ1 ðy 00 ðk m þ jÞ þ y 00 ðk m jÞÞ y 00 ðk 2mÞ ¼ bTm yðk; mÞ y 00 ðk 2mÞ;
ð6Þ
j¼0
where bm = (bm,1, bm,2, . . ., bm,m)T, y(k, m) = (2y00 (k m), (y00 (k m + 1) + y00 (k m 1)), (y00 (k m + 2) + y00 (k m 2)), . . ., (y00 (k 1) + y00 (k 2m + 1)))T. Introducing the estimation criterion t 2 X ^T yðk; mÞ ; J LS y 00 ðkÞ þ y 00 ðk 2mÞ b ð7Þ m m ¼ k¼2mþ1
^ ðtÞ. it is easy to obtain the (m · 1)-vector of least squares estimates b m When the neural network is intended to operate in real time, it is expedient to use a recurrent procedure of the type [4] for its training, in this case taking the following form: 8
1362
Y. Bodyanskiy, S. Popov / European Journal of Operational Research 175 (2006) 1357–1366
which coincides with Goodwin–Ramadge–Caines procedure [15] when a = 1 and, when a = 0, with Widrow–Hoff learning algorithm [13] that is widely used in the artificial neural networks theory. Using expression (1), one-step predictions can be obtained (
^T ðtÞyðt þ 1; mÞ y 00 ðt 2m þ 1Þ; ^y 00 ðt þ 1jtÞ ¼ b m ^y ðt þ 1Þ ¼ ^y 00 ðt þ 1jtÞ þ y ðtÞ:
ð9Þ
As long as the exact value of m is unknown, using the adaptive multimodel approach [5,7,9], the ensemble of the predictors is introduced 8 00 ^ ðk 1Þy 00 ðk 1Þ y 00 ðk 2Þ; > ^y 1 ðkÞ ¼ 2b 1;1 > > > > > 00 ^ ðk 1Þy 00 ðk 2Þ þ b ^ ðk 1Þðy 00 ðk 1Þ y 00 ðk 3ÞÞ y 00 ðk 4Þ; > ^y 2 ðkÞ ¼ 2b > 2;1 2;2 > > > > 00 > 00 ^ ðk 1Þy ðk 3Þ þ b ^ ðk 1Þðy 00 ðk 2Þ y 00 ðk 4ÞÞ > ^y 3 ðkÞ ¼ 2b > 3;1 3;2 > > > > 00 ^ < þb3;3 ðk 1Þðy ðk 1Þ y 00 ðk 5ÞÞ y 00 ðk 6Þ; > .. > > > . > > > > ^ ðk 1Þy 00 ðk MÞ þ b ^ ðk 1Þðy 00 ðk M þ 1Þ y 00 ðk M 1ÞÞ > ^y 00M ðkÞ ¼ 2b > M;1 M;2 > > > > > 00 ^ ðk 1Þðy ðk M þ 2Þ y 00 ðk M 2ÞÞ þ þ b ^ ðk 1Þ > þb > M3 M;M > > : ðy 00 ðk 1Þ y 00 ðk 2M þ 1ÞÞ y 00 ðk 2MÞ;
ð10Þ
each of them corresponds to different m = 1, 2, . . ., M and is trained by the algorithm of the type (8). Signals from their outputs are fed to the output neuron of the network ANNQP, which implements the model 00 ^ ^y ðtÞ ¼ cT ðtÞ^y 00 ðtÞ ¼
M X
cm ðtÞ^y 00m ðtÞ;
ð11Þ
m¼1
where ^y 00 ðtÞ ¼ ð^y 001 ðtÞ; ^y 002 ðtÞ; . . . ; ^y 00M ðtÞÞT ; c = (c1, c2, . . ., cM)T—weights vector satisfying an additional condi00
^y ðtÞ unbiasedness, which is commonly adopted in the problems of stochastic filtering tion of the prediction ^ [29], M X
cm ¼ I T c ¼ 1;
ð12Þ
m¼1
where I = (1, 1, . . ., 1)T. For the output neuron tuning, introduce into consideration the ensembleÕs error 00
^y ðtÞ ¼ y 00 ðtÞ cT ^y 00 ðtÞ ¼ cT Iy 00 ðtÞ cT ^y 00 ðtÞ ¼ cT ðIy 00 ðtÞ ^y 00 Þ ¼ cT EðtÞ eðtÞ ¼ y 00 ðtÞ ^
ð13Þ
and Lagrange function Lðc; k; tÞ ¼
t X
cT EðkÞET ðkÞc þ kðcT I 1Þ ¼ cT RðtÞc þ kðcT I 1Þ;
ð14Þ
k¼1
where k—indeterminate Lagrange multiplier. The saddle point corresponding to the optimal weights set can be found either with the following procedure:
Y. Bodyanskiy, S. Popov / European Journal of Operational Research 175 (2006) 1357–1366
8 > P ðtÞ^y 00 ðt þ 1Þ^y 00T ðt þ 1ÞP ðtÞ > > P ðt þ 1Þ ¼ P ðtÞ ; > > > 1 þ ^y 00T ðt þ 1ÞP ðtÞ^y 00 ðt þ 1Þ > > < c ðt þ 1Þ ¼ c ðtÞ þ P ðt þ 1Þ yðt þ 1Þ ^y 00T ðt þ 1Þc ðtÞ ^y 00 ðt þ 1Þ; > > 1 T > > I c ðt þ 1Þ 1 I; cðt þ 1Þ ¼ c ðt þ 1Þ P ðt þ 1Þ I T P ðt þ 1ÞI > > > > : cm ð0Þ ¼ M 1 ;
1363
ð15Þ
where c*(t) is a standard least squares estimate, or with the convergence-rate-optimized procedure based on Arrow–Hurwicz–Uzawa algorithm [28]: 8 rc Lðc; k; t þ 1Þ ¼ ð2^y 00 ðt þ 1Þeðt þ 1Þ kðtÞI Þ; > > > < eðt þ 1Þrc Lðc; k; t þ 1Þ cðt þ 1Þ ¼ cðtÞ þ 00T ; ð16Þ > ^y ðt þ 1Þrc Lðc; k; t þ 1Þ > > : kðt þ 1Þ ¼ kðtÞ þ ck ðt þ 1ÞðcT ðt þ 1ÞI 1Þ; where ck(t + 1)—learning rate parameter. To achieve appropriate tradeoff between smoothing and following properties of algorithm (16), parameter ck(t) must be adjusted during training in the same way as parameter qm(t) in (8). It is readily seen that when the condition (12) is satisfied, procedure (16) automatically becomes Widrow–Hoff algorithm. It is proven [7] that r^^2y 00 6 minfr^2y 00 ; r^2y 00 ; . . . ; r^2y 00 g (here r2 is the variance of the corresponding prediction error), 1
2
M
i.e. precision of the model (11) is not lower than the best ensembleÕs model ^y 00m ðtÞ provides. At the neural networkÕs output, one-step prediction is obtained 8 M P 00 > <^ ^y ðt þ 1jtÞ ¼ cm ðtÞ^y 00m ðt þ 1jtÞ; m¼1 > 00 : ^ ^y ðt þ 1Þ ¼ ^ ^y ðt þ 1jtÞ þ y ðtÞ:
ð17Þ
Thus, the proposed artificial neural network consists of M + 1 Adalines tuned by similar learning algorithms. M Adalines in the hidden layer detect sets of quasiperiodic components (from 1 to M components), and the (M + 1)th Adaline in the output layer performs multimodel adaptive generalization of predictions according to (11). The performance of such a network is mostly determined by how well non-periodic (trend) component was detected and predicted, and how well disturbances in the original signal were smoothed. It is a design question to choose adequate values for a-parameters in (1), (3), (8), and (16) that provide acceptable tradeoff between smoothing and following properties of the whole system. The network is designed to perform one-step prediction of the time series. However multi-step prediction can easily be obtained by feeding back previous predictions to the networkÕs input—iterated prediction.
3. Experiment ^2 ðtÞ of the Dow–Jones To check the capabilities of the proposed approach, we forecasted volatility r industrial average returns. Our experimental setup and comparison methods are based on [30]. In particular, the returns series r(t) was obtained from the daily closing values s(t) by applying transformation rðtÞ ¼ 100 log
sðtÞ : sðt 1Þ
ð18Þ
1364
Y. Bodyanskiy, S. Popov / European Journal of Operational Research 175 (2006) 1357–1366
The following error measures were adopted: ^2 ðtÞ to the mean • the normalized mean squared error (NMSE) relates the MSE of the modeled volatility r 2 2 ^ ðtÞ ¼ r ðt 1Þ: squared error of the Naı¨ve model r sffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffi PN 2 ð^ r2 ðtÞ r2 ðtÞÞ ; ð19Þ NMSE ¼ PN t¼1 2 2 2 t¼1 ðr ðt 1Þ r ðtÞÞ • the normalized mean absolute error (NMAE) also compares the actual model to the Naı¨ve model, however, it is more robust against outliers: PN 2 ^ ðtÞ r2 ðtÞ t¼1 r NMAE ¼ PN ; ð20Þ 2 2 t¼1 jr ðt 1Þ r ðtÞj • the hit rate (HR) is the relative frequency of correctly indicated increases and decreases of volatility: it measures how often the model gives the correct direction of change of volatility. The HR lies between 0 and 1, a value of 0.5 indicates that the model is not better than a random predictor generating a random sequence of ups and downs (provided that ups and downs are equally likely): HR ¼ hðtÞ ¼
N 1 X hðtÞ; N t¼1 ( 2 ^ ðtÞ r2 ðt 1Þ ðr2 ðtÞ r2 ðt 1ÞÞ P 0; 1; if r
0;
ð21Þ
else;
• the weighted hit rate (WHR) additionally takes the real changes r2(t) r2(t 1) into account meaning that large changes are considered more important than small changes. The WHR lies between 1 (worst case) and 1 (best case): PN WHR ¼
t¼1 sgn
^2 ðtÞ r2 ðt 1Þ ðr2 ðtÞ r2 ðt 1ÞÞ jr2 ðtÞ r2 ðt 1Þj r : PN 2 2 t¼1 jr ðtÞ r ðt 1Þj
ð22Þ
Note that since our network works in adaptive mode, i.e. learning is performed online, we do not have to split the data into training and test sets. The data are presented to the network only once that is why each data point is previously unseen. When the forecasting proceeds to the next point, the previous one is used for training and so on. We compare our quasiperiodic (QP) network with predictions obtained on test sets by GARCH (Generalized Autoregressive Conditional Heteroskedasticity) and RMDN (Recurrent Mixture Density Network) models [30]. QP network with M = 5 is used, neurons of the hidden layer are trained with algorithm (8), parameter a is set to 0.9, the output neuron is trained with algorithm (16). The averaged values of the error measures are presented in Table 1.
Table 1 Comparison of methods
NMSE NMAE HR WHR
GARCH(1, 1)
RMDN(1)
GARCH(1, 1)-t
RMDN(1)-t
LRMDN(2)
RMDN(2)
QP
0.780 0.808 0.685 0.681
0.784 0.833 0.681 0.668
0.789 0.882 0.679 0.675
0.779 0.842 0.679 0.661
0.786 0.842 0.676 0.648
0.772 0.787 0.688 0.674
0.785 0.852 0.723 0.715
Y. Bodyanskiy, S. Popov / European Journal of Operational Research 175 (2006) 1357–1366
1365
The comparison shows that QP model provides the same prediction quality in terms of NMSE and NMAE, and has higher values for HR and WHR. This is an expected result since QP model is inherently better in predicting directions than in predicting levels.
4. Conclusions The problem of financial time sequences analysis and prediction have been under attention of researchers—both mathematicians and economists—already for a long time. Initially, there were attempts to solve it using methods of regression, correlation and spectral analysis, exponential smoothing, adaptive identification, Box–Jenkins approach, and others. All this attempts have failed, because the inherent complexity of financial time series (non-stationary, non-linear, non-Gaussian, oscillatory nature) exceeds the capabilities of these approaches. Attempts to create prediction expert systems have also had no significant success because of the difficulty to formalize expertsÕ knowledge, who often make decision by insight but not by exact computations. In the last decade, significant success in this domain has been achieved due to use of artificial neural networks. From a number of possible ANN structures, in prediction problems, four architectures are used: multilayer feedforward ANN with back-propagation learning, radial basis function nets, self-organizing maps, and recurrent networks. Unfortunately, it is impossible to say in advance, which is the best net for the particular sequence. Therefore an approach using an ensemble of nets with different architectures has been developed recently. Analysis of financial time series reveals that they often contain quasiperiodic or chaotic components. However, they cannot be detected by Fourier analysis because of the frequencies deviation. We have proposed a novel ANN architecture for prediction of time series containing an arbitrary number of quasiperiodic components with aliquant frequencies, which may vary in time. Comparing to conventional approaches to financial time series forecasting, our new ANN takes into account specific issues of the application domain, and minimizes subjectivity in the decision making process by introducing the ensemble of predictors and providing optimal generalization of their outputs (10)–(16).
References [1] A Guide to Practical Technological Forecasting, Prentice Hall Inc., Englewood Cliffs, NJ, 1973. [2] E. de Bodt, J. Rynkiewicz, M. Cottrell, Some known facts about financial data, in: Proc. European Symp. Artificial Neural Networks, Bruges, 2001, pp. 223–236. [3] Ye. Bodyanskiy, I. Pliss, N. Madzharov, Adaptive forecasting of non-stationary processes, Avtomatika i Izchislitelna Tekhnika 6 (1983) 5–12 (in Bulgarian). [4] Ye.V. Bodyanskiy, I.P. Pliss, T.V. Solovyeva, Multistep optimal predictors of multidimensional non-stationary stochastic processes, Doklady AN USSR, series A 12 (1986) 47–49 (in Ukrainian). [5] Ye.V. Bodyanskiy, I.P. Pliss, T.V. Solovyeva, Adaptive generalized forecasting of multidimensional random sequences, Doklady AN USSR, series A 9 (1989) 73–75 (in Ukrainian). [6] Ye.V. Bodyanskiy, I.P. Pliss, Adaptive generalized forecasting of multivariate stochastic signals, in: Proc. Latvian Signal Processing Int. Conf, Riga, vol. 2, 1990, pp. 80–83. [7] Ye.V. Bodyanskiy, I.A. Rudneva, On one adaptive algorithm of fault detection in random sequences, Avtomatika i Telemekhanika 10 (1995) 101–106 (in Russian). [8] Ye. Bodyanskiy, S. Popov, A. Stephan, Fault diagnosis of machines based on use of artificial neural networks, in: Proc. 44th International Scientific Colloquium, Technical University of Ilmenau, 1999, pp. 495–500. [9] Ye.V. Bodyanskiy, S.V. Popov, Forecasting neural network and its learning algorithms, Radioelektronika, Informatika, Upravlinnya 1 (2000) 60–64 (in Russian). [10] G.E.P. Box, G.M. Jenkins, Time Series Analysis. Forecasting and Control, Holden Day, San Francisco, 1970. [11] D.R. Brillinger, Time Series. Data Analysis and Theory, Holt, Rinehart and Winston Inc., NY, 1975.
1366
Y. Bodyanskiy, S. Popov / European Journal of Operational Research 175 (2006) 1357–1366
[12] D.N. Chorafas, et al., Chaos Theory in the Financial Markets, Applying Fractals, Fuzzy Logic, Genetic Algorithms, Swarn Simulation and the Monte Carlo Method to Manage Markets, Probus Publishing Company, Chicago, 1994. [13] A. Cichocki, R. Unbehauen, Neural Networks for Optimization and Signal Processing, Teubner, Stuttgart, 1993. [14] K. Fueser, Neuronale Netze in Der Finanzwirtschaft, Gabler, Wiesbaden, 1995. [15] G.C. Goodwin, P.J. Ramadge, P.E. Caines, A globally convergent adaptive predictor, Automatica 17 (1981) 135–140. [16] Z. Griliches, M.D. Intriligator (Eds.), Handbook of Econometrics, Elsevier Science, 1994. [17] E.J. Hannan, Multiple Time Series, John Wiley and Sons, Inc., NY, 1979. [18] H. Haustein, Prognoseverfahren in Der Sozialistischen Wirtschaft, Vertrag die Wirtschaft, Berlin, 1971. [19] M.D. Intriligator, Econometric Models, Techniques and Applications, Prentice Hall, Inc., Englewood Cliffs, NJ, 1978. [20] A.G. Ivakhnenko, J.A. Mueller, Self-organization of Forecasting Models, Tehnika, Kiev, 1985 (in Russian). [21] M.G. Kendall, A. Stuart, The Advanced Theory of Statistics. Design and Analysis, and Time Series, Charles Griffin and Company, Ltd., London, 1968. [22] J. Kingdon, Intelligent Systems and Financial Forecasting, Springer-Verlag, Berlin, 1997. [23] B.B. Mandelbrot, Die Fraktale Geometrie Der Natur, Birkha¨user Verlag, Basel, 1991. [24] D.C. Montgomery, L.A. Johnson, J.S. Gardiner, Forecasting and Time Series Analysis, McGraw-Hill, Inc., NY, 1990. [25] T.H. Naylor, Computer Simulation Experiments with Models of Economic Systems, John Wiley and Sons, Inc., NY, 1971. [26] Neural networks in financial engineering, in: A.-P.N. Refenes, Y. Abu-Mostafa, J. Moody, A. Weigend (Eds.), Proc. of the Fourth Int. Conf. Neural Networks in the Capital Markets (NNCM-95), World Scientific, Singapore, 1995. [27] D.W. Opitz, J.W. Shavlik, Actively searching for an effective neural network ensemble, Connectionist Science 8 (1996) 337–353. [28] B.T. Polyak, Introduction to optimization, Moscow, 1983, (in Russian). [29] L.R. Rabiner, B. Gold, Theory and Application of Digital Signal Processing, Prentice Hall Inc., Englewood Cliffs, NJ, 1975. [30] C. Schittenkopf, G. Dorffner, E.J. Dockner, Non-linear versus non-Gaussian volatility models, Report series SFB adaptive information systems and modelling in economics and management science, No. 39, Wirtschaftsuniv., Wien, Oktober 1999. [31] A.J.C. Sharkey, On combining artificial neural nets, Connectionist Science 8 (1996) 299–313. [32] H. Theil, Applied Economic Forecasting, North-Holland Publishing Company, Amsterdam, 1966. [33] G.J. Deboeck (Ed.), Trading on the Edge, Neural, Genetic, and Fuzzy Systems for Chaotic Financial Markets, Wiley Finance Edition., John Wiley and Sons Inc., NY, 1994. [34] Yu.P. Yurachkovskiy, Modeling of multidimensional polyharmonic processes, Avtomatica 5 (1985) 38–41 (in Russian). [35] J.S. Zirilli, Financial Prediction Using Neural Networks, International Thompson Computer Press, London, 1997.
Neural network approach to forecasting of quasiperiodic financial time series Yevgeniy Bodyanskiy, Sergiy Popov
*
Control Systems Research Laboratory, Kharkiv National University of Radioelectronics, 14 Lenin av., Kharkiv 61166, Ukraine Available online 16 March 2005
Abstract A novel neural network approach to forecasting of financial time series based on the presentation of the series as a combination of quasiperiodic components is presented. Separate components may have aliquant, and possibly non-stationary frequencies. All their parameters are estimated in real time in an ensemble of predictors, whose outputs are then optimally combined to obtain the final forecast. Special architecture of artificial neural network and learning algorithms implementing this approach are developed. 2005 Elsevier B.V. All rights reserved. Keywords: Time series; Forecasting; Finance; Neural networks; Combining of forecasts
1. Introduction Forecasting is a key element of financial and managerial decision making. This is not surprising, since the ultimate effectiveness of any decision depends upon a sequence of events following the decision. The ability to predict the uncontrollable events prior to making the decision should enable an improved choice over the one that would otherwise be made. The main purpose of forecasting is to reduce the risk in decision-making that is especially important for organizations, firms, and private investors, whose activity is connected with financial markets. Here the prediction is made for financial time series (stock prices, currency exchange rates, price indices, and so on), known to be very complex, difficult for econometric modeling, non-stationary, very noisy, badly fitted by linear models [2]. Moreover, there exist many interdependencies between different financial markets, various market sectors, financial assets, etc. that make the problem even more complicated. *
Corresponding author. Tel.: +380 57 7021890. E-mail addresses: [email protected] (Y. Bodyanskiy), [email protected] (S. Popov).
0377-2217/$ - see front matter 2005 Elsevier B.V. All rights reserved. doi:10.1016/j.ejor.2005.02.012
1358
Y. Bodyanskiy, S. Popov / European Journal of Operational Research 175 (2006) 1357–1366
The problem of economic and financial forecasting attracts rapt attention of researchers. During last decades, several approaches evolved that differ in goals of forecasting, nature of the information used, and mathematical apparatus. Historically first, approach to social-economic and technological forecasting based on tendencies analysis was formed [1,18,32]. This approach relies to a considerable extent on the apparatus of expert judgments and mathematical programming. As follows from its name, it estimates long-term outlooks of complex systemsÕ evolution. Naturally, it does not provide short-term predictions of some specific indexes in real time. First successful results in financial and economic indexes forecasting given in the form of time series were obtained using classic statistical approach and first of all based on regression, correlation and spectral analysis [11,17,21]. Within the framework of this approach, studies upon development of forecasting econometric models were carried out in parallel [16,19,25]. For the problems of short-term forecasting, regression and econometric models proved inefficient since they allow only averaged trends assessment (usually polynomial) on a rather long time intervals. Correlation and spectral models turned out to be more successful, but their use is limited by statistical premises they rest upon: hypothesis of a rather strong autocorrelation within a series, stationarity, linear structure, Gaussian nature of disturbances distribution. As a rule, in reallife economic and financial series, these conditions are not satisfied that leads to decrease of prediction quality. The Box–Jenkins forecasting approach [10] became a quintessence of the above approaches. It is based on autoregression-integrated moving-average (ARIMA) models including polynomial and seasonal trends. And though this method is more computationally and functionally effective, it is based mostly on correlation analysis it has all its drawbacks. Development of adaptive approaches [3,15,24] became a breakthrough in time series forecasting. Here, two classes can be emphasized: procedures based on exponential smoothing and forecasting based on learning models. These methods proved effective in the conditions of lack of a priori information about the seriesÕ possible structure, non-stationarity, small amount of data, sudden changes. They are rather computationally effective and have high convergence rate since they are based on optimal recursive procedures of adaptive identification. But in the same time, the problems of modelÕs (still linear) structure substantiation are not solved yet. Intellectual forecasting methods are the radical development of the above approaches. Here, the apparatus of artificial neural networks (ANNs) [9,14,22,26,33,35] must be emphasized that offers a natural alternative to traditional forecasting techniques. ANNs have three great advantages over traditional methods: • they have universal approximation capabilities; • they can recognize ‘‘on their own’’ implicit dependencies and relationships in data; • they can ‘‘learn’’ to adapt their behavior (their prediction) to changed conditions quickly and without complication. These important capabilities of neural networks can solve, at least in principle, some of the above-mentioned problems. ANNs offer many attractive features when attempting to model poorly understood problem domains. And therefore, over the last years, they have established themselves as credible financial forecasting techniques. At present, neural networks are one of the most successfully applied technique in the financial domain. The most difficult part of using ANNs in financial forecasting problems is to choose ‘‘the best’’ network providing the ‘‘optimal’’ prediction for every particular index. At present, in financial forecasting problems, quite a few types of traditional ANNs are used, i.e. multilayer perceptrons (MLP), Kohonen maps (SOM), dynamic recurrent nets (DRNN), radial basis function nets (RBFN). Hereby, the sequence being forecasted is assumed to be quite precisely describable by non-linear difference autoregression moving average equation (NARMA) with non-stationary parameters. There are no recommendations on how to choose ‘‘the
Y. Bodyanskiy, S. Popov / European Journal of Operational Research 175 (2006) 1357–1366
1359
best’’ from the listed types of ANNs, and such a choice can only be made for a particular series. It is important to note that for any other data the chosen network may not provide the required prediction quality. Moreover, known structures like MLP, RBFN, SOM, DRNN absolutely do not take into account specific issues of financial forecasting that leads to loss of very important information. One of such important features of financial time series that cannot be revealed by conventional approaches is the presence of oscillatory or chaotic components [12,33], which cannot be reliably detected by Fourier analysis because of the frequencies deviation. The use of so-called Fourier-nets for prediction of such series is also impossible because these ANNs imply the knowledge of the number of harmonics and their frequencies a priory. It is also believed that financial time series often obey not NARMA but non-linear fractal equations of Mandelbrot structure [23], which also cannot be revealed using traditional ANNs. As it is impossible to find the best network in advance and, moreover, on different time frames these could be different structures, it is reasonable to use different networks including non-traditional ones in parallel with the aim to increase the precision and reliability of the prediction. During the last years the approach has evolved, related to the use of an ensemble of different networks and finding the best prediction among them [6,27,31]. There also exist methods that change network structure (e.g. add or delete neurons) during the process of training with the ultimate goal of finding an optimal network for a particular data set. These methods cannot be used in the considered problem for at least two reasons: (1) they all work in off-line (batch) mode; (2) if the time series properties vary in different regions of the data set, there is no single ‘‘best’’ structure for the whole series. In this paper, we present a novel approach to financial time series forecasting that addresses most of the aforementioned difficulties. We apply a special artificial neural network specifically designed for prediction of time series containing oscillatory components. This network contains an ensemble of predictors of different orders that are capable to efficiently deal with such signals using their decomposition into quasiperiodic components, i.e. with different, aliquant, possibly non-stationary frequencies. Since the structure of the predicted series is not known in advance and it may change during observation, the multimodel adaptive generalization of predictions [6] is used, which provides optimal selection of the best prediction in real time. The architecture is easily scalable so that any other predictors can be included in the ensemble, e.g. other ANNs or even expert judgments.
2. Description of the approach The time series under analysis is considered as a signal containing quasiperiodic components [20,34] m X sðkÞ ¼ a0 þ ðaj cos xj k þ bj sin xj kÞ; k ¼ 1; 2; . . . ; t; j¼1
where m—the number of components, which is not known in advance, 1 6 m 6 M; M—the maximum possible value of m; a0—absolute term; aj, bj—unknown, possibly non-stationary parameters of separate components; 0 < xj = 2pfjT0 < p—unknown aliquant frequencies to be estimated; T0—sampling period, observed against the noise background yðkÞ ¼ sðkÞ þ nðkÞ; where n(k)—stochastic component: white noise with zero mean and bounded variance, and given as a set of t equidistant points y(1), y(2), . . ., y(t). In many situations a0 may be subject to some kind of trend, then appropriate detrending method should be applied to eliminate it. As the key point of this paper is prediction of quasiperiodic components, we restrict our consideration to the case with constant a0.
1360
Y. Bodyanskiy, S. Popov / European Journal of Operational Research 175 (2006) 1357–1366
The problem of modelÕs parameters identification can be solved with the multistage procedure introduced in [20]. This includes parameters estimation of an intermediate model with eliminated absolute term a0; unknown frequencies xj estimation; coefficients aj, bj evaluation; and, finally, prediction itself. The problem is complicated by the fact that the modelÕs order m must be known exactly, otherwise the problem of selection from M possible structures arises, e.g. using the group method of data handling, that is quite cumbersome, especially when M is big and m may vary during the observation (structural changes). Moreover, if the data sample is growing in real time k = 1, 2, . . ., t, t + 1, . . . and parameters aj, bj, xj are non-stationary, the traditional approach may not work at all. A good alternative is to take advantage of adaptive identification and prediction methods with neural network parallel processing technologies [8]. This approach is rather capable and universal, but certain applications may require computationally simpler procedures, especially when real-time processing is of concern. For this purpose, a feedforward neural network for multimodel prediction of quasiperiodic sequences is proposed, which is shown in Figs. 1 and 2 for the case of M = 4. Note that the network in Fig. 2 has a cascade structure, i.e. increasing the order M does not change the previously formed architecture but only adds new neurons. The input sequence y(k) is processed by the input neuron implementing the exponential smoothing algorithm to detect the mean of the signal y ðkÞ ¼ a0 y ðk 1Þ þ ð1 a0 ÞyðkÞ
ð1Þ
with large values of a 0 (0 6 a 0 6 1; for constant mean, a 0 is defined by (k 1)k1). This operation performs linear detrending, which is sufficient to eliminate absolute term a0 even if it is subject to small drift. In case of the presence of more complex trends in the signal, other detrending techniques may be used. For example, for polynomial trends, higher order exponential smoothing may be appropriate. Then elementary operation is performed y 0 ðkÞ ¼ yðkÞ y ðkÞ;
ð2Þ
0
after that the signal y (k) with eliminated absolute term is again processed by the similar neuron y 00 ðkÞ ¼ a00 y 00 ðk 1Þ þ ð1 a00 Þy 0 ðkÞ;
a00 a0
ð3Þ
with the purpose of random disturbances n(k) smoothing. Then the smoothed signal y00 (k) is fed to the neural network ANNQP, designed for quasiperiodic analysis of the signal.
1
1 1
Fig. 1. Artificial neural network for quasiperiodic sequences prediction.
1
Y. Bodyanskiy, S. Popov / European Journal of Operational Research 175 (2006) 1357–1366
1361
1 1 1
2 1
3
1
1 4 1
1
1
8
Fig. 2. Artificial neural network for quasiperiodic analysis ANNQP for the case of M = 4.
In the design of ANNQP, it is assumed that the signal y00 (k) can be presented in the form m X ðaj cos xj k þ bj sin xj kÞ; m ¼ 1; 2; . . . ; M; k ¼ 1; 2; . . . ; t y 00 ðkÞ ¼
ð4Þ
j¼1
or (which is the same) m Y ð1 2 cos xj z1 þ z2 Þy 00 ðkÞ ¼ 0;
ð5Þ
j¼1
where z1—back-shift operator defined so that z1y00 (k) = y00 (k 1). Description (5) corresponds to the difference equation y 00 ðkÞ ¼
m1 X
bm;jþ1 ðy 00 ðk m þ jÞ þ y 00 ðk m jÞÞ y 00 ðk 2mÞ ¼ bTm yðk; mÞ y 00 ðk 2mÞ;
ð6Þ
j¼0
where bm = (bm,1, bm,2, . . ., bm,m)T, y(k, m) = (2y00 (k m), (y00 (k m + 1) + y00 (k m 1)), (y00 (k m + 2) + y00 (k m 2)), . . ., (y00 (k 1) + y00 (k 2m + 1)))T. Introducing the estimation criterion t 2 X ^T yðk; mÞ ; J LS y 00 ðkÞ þ y 00 ðk 2mÞ b ð7Þ m m ¼ k¼2mþ1
^ ðtÞ. it is easy to obtain the (m · 1)-vector of least squares estimates b m When the neural network is intended to operate in real time, it is expedient to use a recurrent procedure of the type [4] for its training, in this case taking the following form: 8
1362
Y. Bodyanskiy, S. Popov / European Journal of Operational Research 175 (2006) 1357–1366
which coincides with Goodwin–Ramadge–Caines procedure [15] when a = 1 and, when a = 0, with Widrow–Hoff learning algorithm [13] that is widely used in the artificial neural networks theory. Using expression (1), one-step predictions can be obtained (
^T ðtÞyðt þ 1; mÞ y 00 ðt 2m þ 1Þ; ^y 00 ðt þ 1jtÞ ¼ b m ^y ðt þ 1Þ ¼ ^y 00 ðt þ 1jtÞ þ y ðtÞ:
ð9Þ
As long as the exact value of m is unknown, using the adaptive multimodel approach [5,7,9], the ensemble of the predictors is introduced 8 00 ^ ðk 1Þy 00 ðk 1Þ y 00 ðk 2Þ; > ^y 1 ðkÞ ¼ 2b 1;1 > > > > > 00 ^ ðk 1Þy 00 ðk 2Þ þ b ^ ðk 1Þðy 00 ðk 1Þ y 00 ðk 3ÞÞ y 00 ðk 4Þ; > ^y 2 ðkÞ ¼ 2b > 2;1 2;2 > > > > 00 > 00 ^ ðk 1Þy ðk 3Þ þ b ^ ðk 1Þðy 00 ðk 2Þ y 00 ðk 4ÞÞ > ^y 3 ðkÞ ¼ 2b > 3;1 3;2 > > > > 00 ^ < þb3;3 ðk 1Þðy ðk 1Þ y 00 ðk 5ÞÞ y 00 ðk 6Þ; > .. > > > . > > > > ^ ðk 1Þy 00 ðk MÞ þ b ^ ðk 1Þðy 00 ðk M þ 1Þ y 00 ðk M 1ÞÞ > ^y 00M ðkÞ ¼ 2b > M;1 M;2 > > > > > 00 ^ ðk 1Þðy ðk M þ 2Þ y 00 ðk M 2ÞÞ þ þ b ^ ðk 1Þ > þb > M3 M;M > > : ðy 00 ðk 1Þ y 00 ðk 2M þ 1ÞÞ y 00 ðk 2MÞ;
ð10Þ
each of them corresponds to different m = 1, 2, . . ., M and is trained by the algorithm of the type (8). Signals from their outputs are fed to the output neuron of the network ANNQP, which implements the model 00 ^ ^y ðtÞ ¼ cT ðtÞ^y 00 ðtÞ ¼
M X
cm ðtÞ^y 00m ðtÞ;
ð11Þ
m¼1
where ^y 00 ðtÞ ¼ ð^y 001 ðtÞ; ^y 002 ðtÞ; . . . ; ^y 00M ðtÞÞT ; c = (c1, c2, . . ., cM)T—weights vector satisfying an additional condi00
^y ðtÞ unbiasedness, which is commonly adopted in the problems of stochastic filtering tion of the prediction ^ [29], M X
cm ¼ I T c ¼ 1;
ð12Þ
m¼1
where I = (1, 1, . . ., 1)T. For the output neuron tuning, introduce into consideration the ensembleÕs error 00
^y ðtÞ ¼ y 00 ðtÞ cT ^y 00 ðtÞ ¼ cT Iy 00 ðtÞ cT ^y 00 ðtÞ ¼ cT ðIy 00 ðtÞ ^y 00 Þ ¼ cT EðtÞ eðtÞ ¼ y 00 ðtÞ ^
ð13Þ
and Lagrange function Lðc; k; tÞ ¼
t X
cT EðkÞET ðkÞc þ kðcT I 1Þ ¼ cT RðtÞc þ kðcT I 1Þ;
ð14Þ
k¼1
where k—indeterminate Lagrange multiplier. The saddle point corresponding to the optimal weights set can be found either with the following procedure:
Y. Bodyanskiy, S. Popov / European Journal of Operational Research 175 (2006) 1357–1366
8 > P ðtÞ^y 00 ðt þ 1Þ^y 00T ðt þ 1ÞP ðtÞ > > P ðt þ 1Þ ¼ P ðtÞ ; > > > 1 þ ^y 00T ðt þ 1ÞP ðtÞ^y 00 ðt þ 1Þ > > < c ðt þ 1Þ ¼ c ðtÞ þ P ðt þ 1Þ yðt þ 1Þ ^y 00T ðt þ 1Þc ðtÞ ^y 00 ðt þ 1Þ; > > 1 T > > I c ðt þ 1Þ 1 I; cðt þ 1Þ ¼ c ðt þ 1Þ P ðt þ 1Þ I T P ðt þ 1ÞI > > > > : cm ð0Þ ¼ M 1 ;
1363
ð15Þ
where c*(t) is a standard least squares estimate, or with the convergence-rate-optimized procedure based on Arrow–Hurwicz–Uzawa algorithm [28]: 8 rc Lðc; k; t þ 1Þ ¼ ð2^y 00 ðt þ 1Þeðt þ 1Þ kðtÞI Þ; > > > < eðt þ 1Þrc Lðc; k; t þ 1Þ cðt þ 1Þ ¼ cðtÞ þ 00T ; ð16Þ > ^y ðt þ 1Þrc Lðc; k; t þ 1Þ > > : kðt þ 1Þ ¼ kðtÞ þ ck ðt þ 1ÞðcT ðt þ 1ÞI 1Þ; where ck(t + 1)—learning rate parameter. To achieve appropriate tradeoff between smoothing and following properties of algorithm (16), parameter ck(t) must be adjusted during training in the same way as parameter qm(t) in (8). It is readily seen that when the condition (12) is satisfied, procedure (16) automatically becomes Widrow–Hoff algorithm. It is proven [7] that r^^2y 00 6 minfr^2y 00 ; r^2y 00 ; . . . ; r^2y 00 g (here r2 is the variance of the corresponding prediction error), 1
2
M
i.e. precision of the model (11) is not lower than the best ensembleÕs model ^y 00m ðtÞ provides. At the neural networkÕs output, one-step prediction is obtained 8 M P 00 > <^ ^y ðt þ 1jtÞ ¼ cm ðtÞ^y 00m ðt þ 1jtÞ; m¼1 > 00 : ^ ^y ðt þ 1Þ ¼ ^ ^y ðt þ 1jtÞ þ y ðtÞ:
ð17Þ
Thus, the proposed artificial neural network consists of M + 1 Adalines tuned by similar learning algorithms. M Adalines in the hidden layer detect sets of quasiperiodic components (from 1 to M components), and the (M + 1)th Adaline in the output layer performs multimodel adaptive generalization of predictions according to (11). The performance of such a network is mostly determined by how well non-periodic (trend) component was detected and predicted, and how well disturbances in the original signal were smoothed. It is a design question to choose adequate values for a-parameters in (1), (3), (8), and (16) that provide acceptable tradeoff between smoothing and following properties of the whole system. The network is designed to perform one-step prediction of the time series. However multi-step prediction can easily be obtained by feeding back previous predictions to the networkÕs input—iterated prediction.
3. Experiment ^2 ðtÞ of the Dow–Jones To check the capabilities of the proposed approach, we forecasted volatility r industrial average returns. Our experimental setup and comparison methods are based on [30]. In particular, the returns series r(t) was obtained from the daily closing values s(t) by applying transformation rðtÞ ¼ 100 log
sðtÞ : sðt 1Þ
ð18Þ
1364
Y. Bodyanskiy, S. Popov / European Journal of Operational Research 175 (2006) 1357–1366
The following error measures were adopted: ^2 ðtÞ to the mean • the normalized mean squared error (NMSE) relates the MSE of the modeled volatility r 2 2 ^ ðtÞ ¼ r ðt 1Þ: squared error of the Naı¨ve model r sffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffi PN 2 ð^ r2 ðtÞ r2 ðtÞÞ ; ð19Þ NMSE ¼ PN t¼1 2 2 2 t¼1 ðr ðt 1Þ r ðtÞÞ • the normalized mean absolute error (NMAE) also compares the actual model to the Naı¨ve model, however, it is more robust against outliers: PN 2 ^ ðtÞ r2 ðtÞ t¼1 r NMAE ¼ PN ; ð20Þ 2 2 t¼1 jr ðt 1Þ r ðtÞj • the hit rate (HR) is the relative frequency of correctly indicated increases and decreases of volatility: it measures how often the model gives the correct direction of change of volatility. The HR lies between 0 and 1, a value of 0.5 indicates that the model is not better than a random predictor generating a random sequence of ups and downs (provided that ups and downs are equally likely): HR ¼ hðtÞ ¼
N 1 X hðtÞ; N t¼1 ( 2 ^ ðtÞ r2 ðt 1Þ ðr2 ðtÞ r2 ðt 1ÞÞ P 0; 1; if r
0;
ð21Þ
else;
• the weighted hit rate (WHR) additionally takes the real changes r2(t) r2(t 1) into account meaning that large changes are considered more important than small changes. The WHR lies between 1 (worst case) and 1 (best case): PN WHR ¼
t¼1 sgn
^2 ðtÞ r2 ðt 1Þ ðr2 ðtÞ r2 ðt 1ÞÞ jr2 ðtÞ r2 ðt 1Þj r : PN 2 2 t¼1 jr ðtÞ r ðt 1Þj
ð22Þ
Note that since our network works in adaptive mode, i.e. learning is performed online, we do not have to split the data into training and test sets. The data are presented to the network only once that is why each data point is previously unseen. When the forecasting proceeds to the next point, the previous one is used for training and so on. We compare our quasiperiodic (QP) network with predictions obtained on test sets by GARCH (Generalized Autoregressive Conditional Heteroskedasticity) and RMDN (Recurrent Mixture Density Network) models [30]. QP network with M = 5 is used, neurons of the hidden layer are trained with algorithm (8), parameter a is set to 0.9, the output neuron is trained with algorithm (16). The averaged values of the error measures are presented in Table 1.
Table 1 Comparison of methods
NMSE NMAE HR WHR
GARCH(1, 1)
RMDN(1)
GARCH(1, 1)-t
RMDN(1)-t
LRMDN(2)
RMDN(2)
QP
0.780 0.808 0.685 0.681
0.784 0.833 0.681 0.668
0.789 0.882 0.679 0.675
0.779 0.842 0.679 0.661
0.786 0.842 0.676 0.648
0.772 0.787 0.688 0.674
0.785 0.852 0.723 0.715
Y. Bodyanskiy, S. Popov / European Journal of Operational Research 175 (2006) 1357–1366
1365
The comparison shows that QP model provides the same prediction quality in terms of NMSE and NMAE, and has higher values for HR and WHR. This is an expected result since QP model is inherently better in predicting directions than in predicting levels.
4. Conclusions The problem of financial time sequences analysis and prediction have been under attention of researchers—both mathematicians and economists—already for a long time. Initially, there were attempts to solve it using methods of regression, correlation and spectral analysis, exponential smoothing, adaptive identification, Box–Jenkins approach, and others. All this attempts have failed, because the inherent complexity of financial time series (non-stationary, non-linear, non-Gaussian, oscillatory nature) exceeds the capabilities of these approaches. Attempts to create prediction expert systems have also had no significant success because of the difficulty to formalize expertsÕ knowledge, who often make decision by insight but not by exact computations. In the last decade, significant success in this domain has been achieved due to use of artificial neural networks. From a number of possible ANN structures, in prediction problems, four architectures are used: multilayer feedforward ANN with back-propagation learning, radial basis function nets, self-organizing maps, and recurrent networks. Unfortunately, it is impossible to say in advance, which is the best net for the particular sequence. Therefore an approach using an ensemble of nets with different architectures has been developed recently. Analysis of financial time series reveals that they often contain quasiperiodic or chaotic components. However, they cannot be detected by Fourier analysis because of the frequencies deviation. We have proposed a novel ANN architecture for prediction of time series containing an arbitrary number of quasiperiodic components with aliquant frequencies, which may vary in time. Comparing to conventional approaches to financial time series forecasting, our new ANN takes into account specific issues of the application domain, and minimizes subjectivity in the decision making process by introducing the ensemble of predictors and providing optimal generalization of their outputs (10)–(16).
References [1] A Guide to Practical Technological Forecasting, Prentice Hall Inc., Englewood Cliffs, NJ, 1973. [2] E. de Bodt, J. Rynkiewicz, M. Cottrell, Some known facts about financial data, in: Proc. European Symp. Artificial Neural Networks, Bruges, 2001, pp. 223–236. [3] Ye. Bodyanskiy, I. Pliss, N. Madzharov, Adaptive forecasting of non-stationary processes, Avtomatika i Izchislitelna Tekhnika 6 (1983) 5–12 (in Bulgarian). [4] Ye.V. Bodyanskiy, I.P. Pliss, T.V. Solovyeva, Multistep optimal predictors of multidimensional non-stationary stochastic processes, Doklady AN USSR, series A 12 (1986) 47–49 (in Ukrainian). [5] Ye.V. Bodyanskiy, I.P. Pliss, T.V. Solovyeva, Adaptive generalized forecasting of multidimensional random sequences, Doklady AN USSR, series A 9 (1989) 73–75 (in Ukrainian). [6] Ye.V. Bodyanskiy, I.P. Pliss, Adaptive generalized forecasting of multivariate stochastic signals, in: Proc. Latvian Signal Processing Int. Conf, Riga, vol. 2, 1990, pp. 80–83. [7] Ye.V. Bodyanskiy, I.A. Rudneva, On one adaptive algorithm of fault detection in random sequences, Avtomatika i Telemekhanika 10 (1995) 101–106 (in Russian). [8] Ye. Bodyanskiy, S. Popov, A. Stephan, Fault diagnosis of machines based on use of artificial neural networks, in: Proc. 44th International Scientific Colloquium, Technical University of Ilmenau, 1999, pp. 495–500. [9] Ye.V. Bodyanskiy, S.V. Popov, Forecasting neural network and its learning algorithms, Radioelektronika, Informatika, Upravlinnya 1 (2000) 60–64 (in Russian). [10] G.E.P. Box, G.M. Jenkins, Time Series Analysis. Forecasting and Control, Holden Day, San Francisco, 1970. [11] D.R. Brillinger, Time Series. Data Analysis and Theory, Holt, Rinehart and Winston Inc., NY, 1975.
1366
Y. Bodyanskiy, S. Popov / European Journal of Operational Research 175 (2006) 1357–1366
[12] D.N. Chorafas, et al., Chaos Theory in the Financial Markets, Applying Fractals, Fuzzy Logic, Genetic Algorithms, Swarn Simulation and the Monte Carlo Method to Manage Markets, Probus Publishing Company, Chicago, 1994. [13] A. Cichocki, R. Unbehauen, Neural Networks for Optimization and Signal Processing, Teubner, Stuttgart, 1993. [14] K. Fueser, Neuronale Netze in Der Finanzwirtschaft, Gabler, Wiesbaden, 1995. [15] G.C. Goodwin, P.J. Ramadge, P.E. Caines, A globally convergent adaptive predictor, Automatica 17 (1981) 135–140. [16] Z. Griliches, M.D. Intriligator (Eds.), Handbook of Econometrics, Elsevier Science, 1994. [17] E.J. Hannan, Multiple Time Series, John Wiley and Sons, Inc., NY, 1979. [18] H. Haustein, Prognoseverfahren in Der Sozialistischen Wirtschaft, Vertrag die Wirtschaft, Berlin, 1971. [19] M.D. Intriligator, Econometric Models, Techniques and Applications, Prentice Hall, Inc., Englewood Cliffs, NJ, 1978. [20] A.G. Ivakhnenko, J.A. Mueller, Self-organization of Forecasting Models, Tehnika, Kiev, 1985 (in Russian). [21] M.G. Kendall, A. Stuart, The Advanced Theory of Statistics. Design and Analysis, and Time Series, Charles Griffin and Company, Ltd., London, 1968. [22] J. Kingdon, Intelligent Systems and Financial Forecasting, Springer-Verlag, Berlin, 1997. [23] B.B. Mandelbrot, Die Fraktale Geometrie Der Natur, Birkha¨user Verlag, Basel, 1991. [24] D.C. Montgomery, L.A. Johnson, J.S. Gardiner, Forecasting and Time Series Analysis, McGraw-Hill, Inc., NY, 1990. [25] T.H. Naylor, Computer Simulation Experiments with Models of Economic Systems, John Wiley and Sons, Inc., NY, 1971. [26] Neural networks in financial engineering, in: A.-P.N. Refenes, Y. Abu-Mostafa, J. Moody, A. Weigend (Eds.), Proc. of the Fourth Int. Conf. Neural Networks in the Capital Markets (NNCM-95), World Scientific, Singapore, 1995. [27] D.W. Opitz, J.W. Shavlik, Actively searching for an effective neural network ensemble, Connectionist Science 8 (1996) 337–353. [28] B.T. Polyak, Introduction to optimization, Moscow, 1983, (in Russian). [29] L.R. Rabiner, B. Gold, Theory and Application of Digital Signal Processing, Prentice Hall Inc., Englewood Cliffs, NJ, 1975. [30] C. Schittenkopf, G. Dorffner, E.J. Dockner, Non-linear versus non-Gaussian volatility models, Report series SFB adaptive information systems and modelling in economics and management science, No. 39, Wirtschaftsuniv., Wien, Oktober 1999. [31] A.J.C. Sharkey, On combining artificial neural nets, Connectionist Science 8 (1996) 299–313. [32] H. Theil, Applied Economic Forecasting, North-Holland Publishing Company, Amsterdam, 1966. [33] G.J. Deboeck (Ed.), Trading on the Edge, Neural, Genetic, and Fuzzy Systems for Chaotic Financial Markets, Wiley Finance Edition., John Wiley and Sons Inc., NY, 1994. [34] Yu.P. Yurachkovskiy, Modeling of multidimensional polyharmonic processes, Avtomatica 5 (1985) 38–41 (in Russian). [35] J.S. Zirilli, Financial Prediction Using Neural Networks, International Thompson Computer Press, London, 1997.