Regularized dynamic self-organized neural network inspired by the immune algorithm for financial time series prediction

Author’s Accepted Manuscript Regularized dynamic self-organized neural network inspired by the immune algorithm for financial time series prediction Abir Jaafar Hussain, Dhiya Al-Jumeily, Haya AlAskar, Naeem Radi www.elsevier.com/locate/neucom

PII: DOI: Reference:

S0925-2312(15)01824-X http://dx.doi.org/10.1016/j.neucom.2015.01.109 NEUCOM16419

To appear in: Neurocomputing Received date: 13 September 2014 Revised date: 18 December 2014 Accepted date: 29 January 2015 Cite this article as: Abir Jaafar Hussain, Dhiya Al-Jumeily, Haya Al-Askar and Naeem Radi, Regularized dynamic self-organized neural network inspired by the immune algorithm for financial time series prediction, Neurocomputing, http://dx.doi.org/10.1016/j.neucom.2015.01.109 This is a PDF file of an unedited manuscript that has been accepted for publication. As a service to our customers we are providing this early version of the manuscript. The manuscript will undergo copyediting, typesetting, and review of the resulting galley proof before it is published in its final citable form. Please note that during the production process errors may be discovered which could affect the content, and all legal disclaimers that apply to the journal pertain.

Regularized dynamic self-organized neural network inspired by the immune algorithm for financial time series prediction Abir Jaafar Hussaina,*, Dhiya Al-Jumeilya, Haya Al-Askara, and Naeem Radib a

School of Computer Science

Faculty of Engineering and Technology Liverpool John Moores University Liverpool L3 3AF, UK b

Al Khawarizmi International College, Abu Dhabi, UAE

Abstract- This paper presents a novel type of recurrent neural network, the regularized dynamic self-organized neural network inspired by the immune algorithm. The regularization technique is used with the dynamic self-organized multilayer perceptrons network that is inspired by the immune algorithm. The regularization has been addressed to improve the generalization and to solve the over-fitting problem. In this work, the average values of 30 simulations generated from 10 financial time series are examined. The results of the proposed network were compared with the standard dynamic self-organized multilayer perceptrons network inspired by the immune algorithm, the regularized multilayer neural networks and the regularized self-organized neural network inspired by the immune algorithm. The simulation results indicated that the proposed network showed average improvement using the annualized return for all signals of 0.491, 8.1899 and 1.0072 in comparison to the benchmarked networks, respectively.

Keywords—Dynamic neural network, exchange rate time series, and financial time series prediction *Corresponding author Tel.: +44(0)1512312458, Email: [email protected]

1.

Introduction:

Financial time series analysis is a fundamental subject that has been addressed widely in economic fields. A time series is a collection of observations of well-defined data items obtained through repeated measurements during a period of time. The analyses of financial time series have economic importance. It is a promising and crucial task for any future investment used to make decisions in different areas, such as businesses and financial institutions. Financial time series involve different time scales such as intraday (high frequency), hourly, daily, weekly, monthly, or tick-bytick stock prices of exchange rates [1]. The distance between variables in financial time series is influenced by real economic activities [2]. The effect of these activities have been represented by a mixture of hills and bumps in financial time series charts [3]. Thus, the prediction aims to forecast these activities. Financial data analysis usually provides the fundamental basis for decision models to achieve good returns [4], which is the first and the most important factor for any investor. This can help to improve companies’ strategies and decrease the risk of potentially high losses [5]. Furthermore, it can help investors to cover the potential market risk to establish some techniques to progress the quality of financial decisions. Financial data are naturally dynamic, nonlinear, nonparametric, complex, and chaotic [6]. This type of time series is non-stationary, has a high level of uncertainty, is highly noisy and has an unstructured nature which includes regular structural breaks. In addition, the financial time series holds several types of data, which are incomplete, unclear and unlimited [7]. Financial time series such as the stock market are facing dramatic changes, as well as rapid information exchange all the time. Hence, the prediction of its economic activity in the future is extremely challenging [7, 8,10]. Furthermore, Yao et al. [11] identified that there are a number of interrelated factors influencing the direction of the price movement on the stock market, besides the economic factors, which are political factors and psychological factors affecting both powerful decision makers and individuals or consumers. Even traditional Economics studies indicated that even microeconomic drivers can affect demand consumption patterns and vice versa, thus forming a very complicated, highly interrelated system to model [11,12]. Therefore, researchers can face many challenges when handling time series forecasting [13]. The selection of an appropriate model for solving time series prediction problem has been considered by many scholars and public investors [14].

Neural

networks

have

been widely

applied

to

the domain of financial time

series prediction. During the last decades, a number of neural network architectures have been proposed and investigated in this regard. According to Zekic [15], neural networks have different applications to financial time series analysis, which include classification of stock, recommendation of trading, predicting price changes of stock indexes, stock price forecasting and modelling the time series of stock markets. A number of studies, including Bansal et al. [16], Yao et al. [11], Vojinovic et al. [17] Dunis and William [18, 19], Pissarenko [20], Aryal and Yao-wu [21] , Kamruzzaman [1, 4], Bagherifard et al. [22] have confirmed that neural networks perform better than traditional prediction models such as the ARIMA model in financial time series prediction in terms of statistical and financial matrices. Researches indicated that artificial neural networks (ANNs) can forecast the future currency with up to 60% accuracy [24, 25], and consequently are being widely used to model the behaviour of financial data and to forecast future values for time series [26, 27]. There are several features that make neural network the ideal technique over other traditional models of prediction. For example, since the majority of traditional models of prediction are linear, they fail to understand data patterns when the underlying relationship in time series is nonlinear. ANN has emerged as a powerful learning technique to perform complex tasks in highly nonlinear dynamical environments. The advantage of using a neural network as a prediction tool is that it includes automatic learning of dependencies and it can be adapted to any type of data [22]. Utilizing a neural network to forecast financial time series such as a stock market has a number of advantages including: saving investors’ time when making financial decisions, and aiding them to decrease investment loss and risk caused by stock market fluctuation [28]. Artificial neural networks (ANNs) have been widely used in both modern industries

and scientific

research to perform diverse

and

sophisticated tasks, such as data processing, pattern recognition, forecasting the

exchange rates between currencies or forecasting the sign of price increments, capture the underlying information and rules of the movement in currency exchange rates [11, 31, 33, 34]. In 1991, some banks started utilizing neural network models to help them make conclusions about loan application of financial prediction [20]. Several large investment banks (including Goldman Sachs and Morgan Stanley)

currently have departments dealing exclusively with neural network for their business investment models [29]. MLP networks have powerful problem solving capability, however these networks suffer from various problems when dealing with temporal patterns. MLP represents static mapping between inputs and outputs. Therefore, MLP network does not consider the dependence of an individual input on others previous inputs [35, 56, 57]. In theoretical analysis, Herrera [36] assumed that time series are generated by dynamic systems. This means that their state changes over time. Prediction is considered a temporal signal processing problem. Therefore, using a temporal model such as recurrent neural network can achieve better prediction accuracy. Recurrent neural network can address the temporal relationship of the inputs by maintaining an internal state [30]. It is considered an alternative method of time series forecasting through the use of dynamic memory [37]. The memory of the previous values generated by the neural network is kept in the context layer; the values are then fed back to the neural network to predict the next values [38]. As result, the predicted values will rely on the current values as well as the previous neural network outputs. This can help the neural network to learn the dynamic information involved in time series. Several models and techniques have already been developed to enhance the forecasting ability of neural networks [59], such as the regularization methods. This method is based on using weight decay in order to improve the training of the neural network. Mahdi et al [39] has used regularization technique in self-organized multilayer network inspired by the immune algorithm (SONIA) network in order to forecast physical time series data as well as financial time series data [40]. Their result demonstrated an improvement of the predication performance of SONIA network using the weight decay. In this paper the regularization technique is applied with dynamic self-organized multilayer neural network which is inspired by immune algorithm (R-DSMIA). The aim is to improve the generalization capability of the DSMIA network for time series forecasting. To evaluate the forecasting performance of the proposed neural

network, a number of simulation experiments have been preformed. The proposed R-DSMIA is used to predict ten financial time series. The remainder of this paper is organized as follows. Section 2 describes selforganized multilayer network inspired by the immune algorithm (SONIA). In section 3, the proposed dynamic self-organized multilayer neural inspired by the immune algorithm is presented. Section 4 shows the application of the regularization technique in dynamic self-organized multilayer neural network inspired by the immune algorithm. Section 5 describes the set of data that has been used in this research. Section 6 includes the learning parameters, and performance measures. Section 7 presents the simulation results for the prediction of ten signals using various neural network models. Finally, the conclusions of this paper are discussed in Section 8. 2. Self-organized multilayer network inspired by immune algorithm (SONIA): Self-organized network inspired by the immune algorithm (SONIA) [41] is a single hidden layer neural network, which uses a self-organization hidden layer inspired by the immune and back-propagation algorithms for the training of the output layer. The immune algorithm is simulated as the nature immune system, which is based on the relationship between its components and involve antigens and cells (Recognition Ball). Thus, the immune system can allow its components to change and learn patterns by changing the strength of connections between individual components. The inspiration of the immune system in the self-organized neural network will provide hidden unit creation in backpropagation neural networks (BP-NN). The SONIA network was proposed to improve the generalization and recognition capability of the back-propagation neural network [41]. The input units are called antigens, while hidden units are called recognition balls (RBs). RBs in the immune system are used to create hidden units. The relation between the antigen and the RB is based on the definition of local pattern relationships between input vectors and hidden nodes. These relationships help SONIA to easily recognize and define the input data’s local characteristics, which increases the networks ability to recognize patterns. The mutation process, which is biologically, a B cell, can be created and mutated to produce a diverse set of antibodies in order to remove and fight viruses

that attack the body [42]. In SONIA, the mutated hidden nodes are designed to deal with unknown data, which is the test data, to develop the generalization ability of the network. 3. Dynamic self-organized multilayer network inspired by immune algorithm (DSMIA): The dynamic self-organised multilayer network inspired by the immune algorithm (DSMIA) is used to predict financial time series. The structure of the DSMIA network is shown in Figure 1. The DSMIA network has three layers: the input, the selforganised hidden layer, and the output layers with feedback connections from the output layer to the input layer. The input layer holds copies of the current inputs as well as the previous output produced by the network. This provides the network with memory. As such, the previous behaviour of the network is used as an input affecting current behaviour. Similar to the Jordan recurrent network [43], the output of the network is fed back to the input through the context units.

Figure 1. The structure of the proposed DSMIA network.

Suppose that N is the number of external inputs x(n) to the network, and y(n-1) is the output of the network from the previous time step while O refers to the number of outputs. In the proposed DSMIA, the total input to the network will be the component of x(n) and the previous output where

( )

{

( ) (

(1)

)

The output of the hidden layer is computed as: () ( )

√∑ ( √∑

( ))

(

( )

( )

( )

(

(

(2) ))

(3)

( )

(4)

( ))

(5)

The output of the network is computed as: ( )

(∑

( )

)

(6)

Where fht,, fot are nonlinear activation functions, N is the number of external inputs, O is the number of output units, NH is the number of hidden units. wojk is the weight corresponds to the external input while wzhjk is the weight corresponding to the hidden layer output, and n is the current time step, while α,β are selected parameters with 0< α and 0< β. The term Bok is the bias value. The first layer of the DSMIA is a self-organised hidden layer trained similar to the recursive self-organized map RecSOM [44]. In this case, the training rule for updating the weights is based on the same technique for updating the weights of the selforganized network inspired by the immune algorithm (SONIA) network [41]. The change in the proposed network is that the weights of the context nodes are also updated in the same way as the weights of the external inputs. This is done by first finding D, which is the distance between the input units and the centroid of the jth hidden units: ( )

√∑

( ( )

( ))

√∑

( (

)

( ))

(7)

The position of the closest match will be determined as: ( )

( )

(8)

If the shortest distance is less than the stimulation level value, s1(0, 1), this indicates that the input has found its corresponding hidden unit. In this case, the weight from the external input vector and the context vector are updated as follows: (

) (

( ) )

( )

( ) ( )

(9) (10)

Where wzhjk is the weight of the previous output, whji is the weight for the external inputs, Dc is the position of the closest match while γ is the learning rate determined experimentally and selected between 0 and 1. It should be noted that the simulation level value s1(0, 1) is experimentally determined. 4.

Regularized DSMIA Network (R-DSMIA):

The regularization technique has been used with the DSMIA network. The regularization has been addressed to improve the generalization and to solve the over-fitting problem; also, the elimination of the weights leads to a reduction in the complexity of the training network and makes the learning process easier [45]. This is performed by using weight decay which is the simplest technique for the regularization method and it is applied in order to make the prediction of the network more consistent [46]. Regularization attempts to smoothing the cost function and reduces variance by pruning some weights besides keeping the functional capability necessary to solve the problem [47]. The main aim of the regularization is to decrease the generalization error. Regularization is the technique of adding a penalty term Ω to the error function, which can help obtain a smoother network mapping. It is given by Ereg = Estd + Ω

(11)

Where Estd represents one of the standard cost functions such as the Sum-ofsquares error and the parameter  controls the range of the penalty term Ω in which it can influence the form of the solution. The network training should be implemented by minimizing the total error function Ereg [46] . The weight decay is performed by adding a bias term to the original objective function Estd, thus the weight decay cost function is determined as follows [24]:

Ereg = Estd + (/2) B

(12)

Where  is the weight decay rate, B represents the penalty term. The simplest form of calculating the penalty term B is: ∑

(13)

Where wij is the weight connections between the ith units and jth nodes in the next layer. In the R-DSMIA network, the weight decay was used to adjust the weights between the hidden and the output units. The change of weights using weight decay method could be calculated as follows: (

(∑

̀

∑

)



∑

)

(14)

(15)

Where ∆wojk is the updated weight that connects the hidden and the output units while ̀ is the derivative of the output transfer function

. The significant role of

weight decay is to manage the complexity of the cost function. This will improve the neural network performance. Since the number of weights to be computed in the DSMIA network can be quite large in some problems, there is a need to eliminate some weights from the network without losing the functional ability required to solve the problem. The R-DSMIA network is used to find the effect of the regularization technique and to enhance the performance of the DSMIA network in time series prediction. 5.

Financial Time Series Data:

Three different types of financial time series are applied in this research work; the exchange rate time series, stock opening and closing prices, and the oil price. The exchange rate time series and the stock process are daily time series for the period between 1st July 2002 to 11th November 2008, giving 1,605 trading days, as shown in Table 1. The oil price data is monthly data and covers the period between1 st January 1985 to 1st November 2008, with a total of 389 trading months. The source of the data can be found at http://www.economagic.com/ecb.htm.

Since most of the published work about financial time series prediction have focused on exchange rate prediction [18, 34, 48, 49], this research has used six exchange rate signals. The foreign exchange market is considered to be the largest market, with more than $1 trillion traded everyday [31, 50]. The US dollar is the most significant currency in the market and it has been used as a reference currency. Another time series used in this research work is the West Texas Intermediate (WTI) crude oil spot prices. Crude oil is well known as a central source of energy. The future oil price has a great impact on governments and industries and companies’ activities [51]. TABLE 1: Financial Time Series Data Used No

1

Time Series Data US dollar to EURO exchange rate (USD/EUR)

Total

1607

01/07/2002 - 13/11/2008

2

US dollar to UK pound exchange rate (USD/UKP)

1607

01/07/2002 - 13/11/2008

3

Japanese yen to US dollar exchange rate (JPY/USD)

1607

01/07/2002 - 13/11/2008

4

Dow Jones Industrial Average stock opening price (DJIAO)

1605

01/07/2000 - 11/11/2008

5

Dow Jones Industrial Average stock closing price (DJIAC) 01/07/2000 11/11/2008

1605

6

Dow Jones Utility Average stock opening price (DJUAO) 01/07/2000 11/11/2008

1605

7

Dow Jones Utility Average stock closing price (DJUAC) 01/07/2000 11/11/2008

1605

8 9

NASDAQ composite stock opening price (NASDAQO) 01/07/2000 -12/11/2008 NASDAQ composite stock closing

1606 1606

price (NASDAQC) 01/07/2000 - 12/11/2008 10

6.

Oil price of West Texas Intermediate crude (OIL) 01/01/1985 -01/11/2008

389

Modelling DSMIA for Financial Time Series Prediction:

In this section the structure of the neural network models is explained. The selection of a suitable forecasting horizon is the first step that must be taken before financial forecasting can begin. From a trading principle, Cao and Tay [54] asserted that a long forecasting horizon could avoid over-trading resulting in extreme transaction rates. However, if the forecasting horizon is very long, that might increase the complexity of the forecasting procedure. On the other hand, predictors claimed that the forecast horizon must be sufficiently short as the persistence of financial time series is of limited period [12]. As Thomason recommended in his work [52], the optimal choice of forecasting horizon for the daily data is five days. From the trading and prediction view, this research applies five days ahead prediction. Financial time series are a non-stationary, high noise type of data. The noise variables in data are identified as harmful variables in neural network learning [39]. Therefore, it is crucial to have a pre-processing method to deal with data before passing them to the neural network. The original raw non-stationary signals are transformed into stationary signals before sending them to the neural network, by a transformation technique known as Relative Difference in Percentage of price (RDP) [52]. It is used by many researchers in this field for example [34, 48, 53, 54]. It creates a five-day measure of the relative difference in price data. In this transformation, the signal is expected to be more symmetrical and closely follow the normal distribution [39]; consequently it can improve the prediction process [54]. Ghazali et al. [34] stated that RDP helps to reduce the influence of trends on financial time series, smoothing the data and helping to reduce noise. The input variables are computed from four lagged RDP values based on five-day periods (RDP-5, RDP-10, RDP-15, and RDP-20) and one transformed signal (EMA15) which is computed by subtracting a 15-day exponential moving average from the raw signals [54]. The computation of the inputs is presented in Table 2, where α is the weight factor which is experimentally

determined in these experiments as 0.85, P(i) is the values of signal for the ith day, and h is a horizon of five steps ahead prediction. TABLE 2: The data pre-processed into a stationary series Indicator

Input variables

EMA15

RDP-5

Output variables

Calculations

()

̅̅̅̅̅̅̅̅̅̅̅̅̅ ()

(

))

(

() ( ()

)

RDP-10

( ()

(

))

(

)

RDP-15

( ()

(

))

(

)

RDP-20

( ()

(

))

(

)

̅̅̅̅̅̅ ()

̅̅̅̅̅̅̅̅̅̅̅̅ ()

RDP+k

There are various evaluation functions that have been applied to estimate the network performance; some of them are related to financial measurement and some of them are statistical methods. The performance of the proposed network is measured with four financial metrics [19] and five statistical metrics [55] which measure the accuracy of the prediction signal. The calculations of these criteria are illustrated in Table 3. TABLE 3: The quality measures Name of criteria

The calculations of criteria

Annualized Return (%AR)

∑

{

| |

( )( ̂ )

| | ∑

Maximum Drawdown (MD)

(∑(

∑

(

)))

{

( )( ̂ )

| | | |

Annualized Volatility (AV) √

√

̅)

∑(

Sharpe ratio (SR) Mean absolute error (MAE)

̂

∑

Normalized Mean Squared Error (NMSE)

̂)

∑(

∑( Mean squared error (MSE)

̅) ̂)

∑( (

Signal to Noise Ratio (SNR) ∑(

)

̂) ( )

Correct Directional Change (CDC)

∑

{

7.

(

)( ̂

̂

)

The Simulation Results:

In this section, the simulation results of the regularized dynamic self-organized multilayer network inspired by immune algorithm are presented. In this research work, the networks were tested on five steps ahead predictions of financial time series. The simulation result for five steps ahead forecasting using stationary signals are presented in Table 4 for the DSMIA network and Table 5 for the R-DSMIA network. In term of evaluating the performance of the network based on annualized return measure (AR). AR measure estimates the ability of the networks as traders. It measures the total profitability in a year by using the “buy“ and “sell” signals. The network which can predict higher profits than other networks is considered as the best financial prediction model.

The R-DSMIA has obtained the highest percentage compared to DSMIA network except NASDAQC time series. The comparison between the performance of DSMIA and R-DSMIA networks showed that using regularization techniques on R-DSMIA network has significantly improved the performance of DSMIA. From Tables 4 to 7, the simulation results indicated that the proposed R-DSMIA showed the highest mean values for all signals in terms of the AR with improvements of 0.491, 8.1899 and 1.0072 in comparison to the standard DSMIA, R-MLP and R-SONIA, respectively. This is an indication that the proposed network generates more profits than the benchmarked networks.

In term of the maximum drawdown (MDD) measure, it can be detected that RDSMIA has achieved the highest value of maximum drawdown when predicting the USDUK, JPYUSD NASDAQC, NASDAQO, DJIAO, DJUAO, DJUAC OIL time series. However, DSMIA network achieved good result on maximum drawdown when predicting the USDEUR signal. In this case, it should be noted that the aim of using the MDD measure is to evaluate the risk of diverse financial prediction models. It is defined as the minimum of the accumulated losses. It determines the downside risk (the maximum loss of the model during the sample period). Small value of MDD indicates less risk or less losing, which is desirable in financial prediction. For the Signal Noise Ratio (SNR) criterion, Tables 4 to 7 show that the proposed RDSMIA obtained the highest value of SNR for all signals except DJIAC time series. It should be noted that the signal to noise ratio compares the amount of significant information provides by the signal, with the amount of background noise of the signal (distraction from the signal). A higher ratio refers to a clearer reading of the signal. The mean results for all 10 financial time series also showed that the proposed RDSMIA achieved improvements of 0.3510 dB, 0.6290 dB and 0.1720 dB in comparison to the DSMIA, R-MLP and R-SONIA, respectively. TABLE 4. The result of five step ahead prediction using stationary signals on DSMIA network Time Series

AR

MDD

AV

SR

SNR

CDC

NMSE

MSE

MAE

USDUK

89.047661

-2.963665

15.757916

5.661860

24.85

65.93

0.429909

0.002104

0.028496

JPYUSD

87.201969

-2.722772

16.760712

5.202832

22.30

63.48

0.388892

0.002968

0.039623

USDEUR

92.028790

-1.432039

15.607786

5.896499

23.38

65.23

0.377785

0.002941

0.033627

NASDAQO

86.323692

-6.109268

35.637459

2.422628

24.93

61.85

0.411139

0.001331

0.023618

NASDAQC

85.105993

-7.658861

36.041823

2.361925

24.00

59.39

0.449935

0.001747

0.026903

DJIAO

86.629872

-9.305728

31.347378

2.766082

24.72

63.27

0.520153

0.001560

0.023078

DJIAC

86.106254

-9.268669

31.426138

2.754751

25.62

63.54

0.432033

0.001298

0.022549

DJUAO

87.135184

-3.357676

35.864341

2.429673

27.10

62.35

0.292279

0.000780

0.019636

DJUAC

87.517079

-3.410785

36.089621

2.425167

27.36

62.41

0.274922

0.000738

0.019372

OIL

92.039333

-11.644154

156.745293

0.588284

25.02

61.21

0.387805

0.001012

0.026859

Mean

87.9136

-5.7874

41.1278

3.2510

24.928

62.8660

0.3965

0.0016

0.0264

STD

2.3999

3.5007

41.5731

1.7329

1.539

1.8945

0.0722

8.06e-04

0.0063

TABLE 5. The result of five step ahead prediction using stationary signals on R-DSMIA network Time Series

AR

MDD

AV

SR

SNR

CDC

NMSE

MSETE

MAE

USDUK

88.191276

-2.707524

15.881707

5.554011

25.29

65.05

0.388020

0.001899

0.027956

JPYUSD

86.521042

-2.72277

16.857634

5.132709

23.20

62.95

0.316219

0.002414

0.036264

USDEUR

92.777338

-1.45280

15.492416

5.988683

23.87

65.51

0.337786

0.002630

0.032606

NASDAQO

87.033172

-5.90720

35.453549

2.454971

24.88

60.41

0.415961

0.001346

0.023746

NASDAQC

84.746644

-7.05441

36.136873

2.345472

24.51

60.10

0.400550

0.001555

0.026497

DJIAO

87.711564

-5.91551

31.118235

2.819477

25.47

62.94

0.436948

0.001311

0.022956

DJIAC

88.801796

-6.29735

30.920888

2.872796

25.26

64.49

0.458317

0.001377

0.022437

DJUAO

87.573340

-3.08812

35.770111

2.448449

27.36

63.52

0.275195

0.000734

0.019251

DJUAC

87.251821

-3.38523

36.146601

2.414027

27.44

62.76

0.270321

0.000726

0.019125

OIL

93.439713

-10.42434

152.728230

0.612524

25.51

59.85

0.345309

0.000901

0.025388

Mean

88.4048

-4.8955

40.6506

3.2643

25.279

62.7580

0.3645

0.0015

0.0256

STD

2.7077

2.7114

40.3470

1.7136

1.3395

2.0416

0.0657

6.5e-004

0.0055

TABLE 6: The result of five step ahead prediction using stationary signals on R-MLP network Time Series

AR

MDD

AV

SR

SNR

CDC

NMSE

MSE

MAE

USD/UKP

87.1223

-2.45425

16.01374

5.441671

26.59

66.02

0.28828

0.001411

0.02807

JPYUSD

83.74337

-2.72277

17.240726

4.857615

23.46

63.58

0.297567

0.002271

0.03626

USDEUR

90.66374

-2.555499

15.812136

5.735123

25.87

64.57

0.214152

0.001667

0.03039

NASDAQO

77.94086

-9.173755

37.578259

2.079828

25.27

61.52

0.392295

0.00127

0.02618

NASDAQC

84.49206

-7.024514

36.195788

2.335423

25.5

60.69

0.322956

0.001254

0.02656

DJIAO

74.35885

-8.166544

33.752804

2.20523

24.04

62.12

0.620153

0.00186

0.03118

DJIAC

69.99318

-10.711946

34.536964

2.032267

23.77

62.3

0.65989

0.001982

0.03222

DJUAO

76.14721

-12.706323

37.978064

2.005844

25.07

62.05

0.471418

0.001258

0.02555

DJUAC

75.9389

-13.174437

38.347615

1.98104

24.94

61.1

0.48512

0.001302

0.02587

OIL

81.74823

-57.254185

180.79057

0.46131

22.02

54.7

0.851406

0.00222

0.03846

Mean

80.2149

-12.5944

44.8247

2.9135

24.65

61.8650

0.4603

0.0016

0.0301

STD

6.4011

16.1994

48.7210

1.7685

1.3374

3.0043

0.1998

4.1e-004

0.0045

TABLE 7: The result of five step ahead prediction using stationary signals on R-SONIA network Time Series

AR

MDD

AV

SR

USD/UKP

90.07944

-1.77722

15.64255

5.758673

JPYUSD

86.87133

-2.72277

16.80812

USDEUR

91.62076

-1.35824

NASDAQO

85.4554

NASDAQC

SNR

CDC

NMSE

MSE

MAE

25.68

66.16

0.354363

0.001734

0.027244

5.168465

23.5

64.07

0.29506

0.002252

0.035276

15.67015

5.846852

23.61

65.02

0.358039

0.002787

0.033194

-6.3989

35.86336

2.382872

23.82

61.59

0.37723

0.001221

0.023468

86.29109

-7.08089

35.73558

2.414784

24.67

59.75

0.386041

0.001499

0.026094

DJIAO

83.8411

-7.00081

31.95809

2.623515

24.99

62.31

0.487654

0.001463

0.023946

DJIAC

85.13061

-6.72123

31.74464

2.681791

24.99

62.86

0.486933

0.001463

0.023834

DJUAO

86.84489

-3.4824

35.92529

2.417534

27.42

62.59

0.270406

0.000721

0.019334

DJUAC

86.55567

-4.75658

36.2965

2.384731

27.03

61.98

0.296947

0.000797

0.020296

OIL

91.28559

-13.3041

158.8773

0.575535

25.36

60.03

0.358022

0.0000934

0.025879

Mean

87.3976

-5.4603

41.4522

3.2255

25.1070

62.636

0.3671

0.0014

0.0259

STD

2.6712

3.5165

42.2062

1.7471

1.3391

2.0265

0.0739

7.78e-004

0.0051

For the other statistical quality measures including the NMSE, MSE and the MAE, it can be shown from Tables 4 to 7 that similar low values are obtained for all benchmarked networks. The NMSE, MSE and the MAE show the overall deviations between predicted and measured values. They are useful measures since low values indicating that the neural networks are performing well both in space and time. However, it can be argued that high values of NMSE, MSE and the MAE do not necessarily mean that the predicted model is completely wrong, as that case could be partly due to time and/or space shifting.

In terms of the correct directional change (CDC) which determines the ability of a prediction model to accurately forecast the subsequent actual change of a forecast variable, the best value was achieved by the R-SONIA network for predicting USD/UKP time series. The DSMIA and R-DSMIA achieved the highest value of CDC when predicting USD/UKP, USD/EU, DJIAO, DJIAC, DJUAO and OIL time series, while SONIA obtained the best value on NASDAQO and DJUAC signals compared to the other networks. 100 80 60

DSMIA

40

R-DSMIA

20

R-MLP

0

R-SONIA

Figure 2. The Annualised Return result of the benchmarks networks and the R-DSMIA in Stationary signals.

The comparison between R-DSMIA and other networks is illustrated in Figure 2. The proposed network was compared with the Regularized MLP (R-MLP) and the Regularized SONIA (R-SONIA) neural networks. The R-DSMIA has obtained the highest percentage compared to the benchmarks network except USDUKP, JPYUSD, NASDAQC and DJUAC time series. The R-SONIA has achieved the best values of AR on USDUKP and NASDAQC time series. The R-MLP networks achieved the lowest profits on average for all ten time series. The R-DSMIA successfully obtained the best profits in comparison to other Regularized neural networks. As can be seen in Table 8, the network structures for the MLP achieved the best result with networks of five to eight hidden units. Meanwhile, the proposed DSMIA network needs more hidden units between 12 to 22. For the SONIA network, the best average profit is achieved using 12 to 25 hidden units. This indicates that due to the utilization of immune algorithm in the hidden layer, more number of hidden units is required to achieve good results.

A paired t-test [58] is conducted to determine if there is significant difference among the proposed R-DSIMA and R-SONIA based on the best results for the AR on the 10 financial time series. The calculated t-value showed that the proposed technique outperform R-SONIA with α = 5% significance level for a one-tailed test. TABLE 8: Number of hidden nodes in MLP, SONIA and DSMIA for five step ahead stationary signals.

Time Series

Five step for stationary signal MLP

SONIA

DSMIA

USD/UKP

5

12

12

JPYUSD

6

24

21

USD/EUR

8

19

17

NASDAQO

8

18

16

NASDAQC

8

21

19

DJIAO

8

19

16

DJIAC

8

19

16

DJUAO

7

25

22

DJUAC

8

23

20

OIL

8

13

13

To further analyze the ability of the proposed R-DSMIA, we have benchmarked our simulation results with the work shown by Cao and Tay [55] for the prediction of the Standard & Poor 500 stock index futures (CME-SP) financial time series in the period between 4/1/1988 to 11/7/1995. In their simulations, the authors transformed the financial data into relative difference in price similar to our work with an average of 30 simulations using the Support Vector Machine (SVM) for regression, the Radial Basis Function (RBF) and the MLP networks using the Backpropagation (BP) learning algorithm. Their results are shown in Table 9 (a), while the simulation results using the proposed D-SMIA are shown in Table 9 (b). TABLE 9: The simulation results for the prediction of the stationary data using (a) the SVM, the RBF and the MLP trained using the backpropagation algorithm (BP) [55] (b) the proposed R-DSIMA network (a) Futures CME-SP

SVM NMSE 1.0635

MAE 0.5403

DS 45.72

R-DSMIA NMSE MAE 0.279228 0.033937

CDC 62.49

(b) Futures CME-SP

RBF NMSE 1.0644

MAE 0.5422

DS 48.74

BP NMSE 1.0708

MAE 0.5454

CDC 43.31

As it can be shown from Table 9, the proposed R-DSMIA network achieved higher value for the correct directional change which indicates that the proposed network is a better financial predictor while the values for the NMSE and the MAE are also lower than the SVM, the RBF and the BP networks. 8.

Conclusion:

In this paper a novel neural network architecture based on the regularization is proposed which is called the regularized dynamic self-organized neural network inspired by the immune algorithm. The proposed network was utilized for the prediction of financial time series. The financial data was transformed into stationary signal and the results for 5 step ahead prediction were shown. The simulation results indicated that using recurrent links with regularization techniques can significantly improve the results due to the temporal aspect of the financial time series. Future research will involve the use of one step and multi-step ahead prediction for stationary and nonstationary financial time series prediction. For nonstationary prediction, the financial data will be presented to the neural network directly to test the performance of the proposed network to extract the information and model the financial signals data. Another possible direction of this research will involve the use of the proposed neural network for the prediction of physical time series such as the earthquake and the sunspot time series which exhibit extreme nonlinearity and nonstationary behaviours. References 1. Kamruzzaman, J.: ANN-Based Forecasting of Foreign Currency Exchange Rates, Neural Information Processing - Letters and Reviews, Vol. 3, No .2, pp.49–58 (2004). 2. Espinoza, R., Lombardi, M.J., Fornari, F.: The role of financial variables in Predicting economic activity , ECB Working Paper Series, No. 1108, Germany (2009). 3. Leondes, C.T.: Intelligent Knowledge-Based Systems: Business Technology in the New Millennium illustrate, Springer, (2010).

and

4. Kamruzzaman, J. & Sarker, R.: Forecasting of currency exchange rates using ANN: a case study, Proceedings of the 2003 Conference on Neural Networks and Signal Processing, Nanjing, China, pp. 793–797 Vol. 1, (2003).

5. Krollner, B.: Risk Management in the Australian Stockmarket using Artificial Neural networks, PhD Thesis, (2011). 6. Tan, T.Z., Quek, C. & Ng, G.S.: Brain-inspired Genetic Complementary Learning for Stock Market Prediction, IEEE congress on evolutionary computation, Vol.3, pp.2653–2660, (2005). 7. Ahmadifard, M., Sadenejad, F., Mohammadi, I., Aramesh, K.: Forecasting stock market return using ANFIS : the case of Tehran Stock Exchange, International Journal of Advanced Studies in Humanities and Social Science, Vol.1, No. 5, pp.452–459 (2013). 8. Ahangar, R.G., Yahyazadehfar, M. & Pournaghshband, H.: The Comparison of Methods Artificial Neural Network with Linear Regression Using Specific Variables for Prediction Stock Price in Tehran Stock Exchange, (IJCSIS) International Journal of Computer Science and Information Security, Vol.7, No.2, pp.38–46, (2010). 9. Kurihara, Y.: The Relationship between Exchange Rate and Stock Prices during the Quantitative Easing Policy in Japan. International Journal of Business, Vol.11, No.4, (2006). 10. Agrawal, G., Srivastav, A.K. & Srivastava, A.: A Study of Exchange Rates Movement and Stock Market Volatility. International Journal of Business and Management, Vol.5, No.12, pp.62-73, (2010). 11. Yao, J., Poh, H. & Jasic, T.: Foreign Exchange Rates Forecasting with Neural Networks, International Conference on Neural Information Processing, Hong Kong, pp. 754-759, (1996). 12. Ghazali, R. : Higher Order Neural Networks For Financial Time Series Prediction. Liverpool John Moores University, PhD Thesis (2007). 13. Schwaerzel, R., Bylander, T.: Predicting currency exchange rates by genetic programming with trigonometric functions and high-order statistics, Proceedings of the 8th annual conference on Genetic and evolutionary computation, Vol.1, pp.955-956 (2006). 14. Zhang, G.P., Patuwo, B.E. & Hu, M.Y.: A simulation study of artificial neural networks for nonlinear time-series forecasting, Computers & Operations Research, Vol.28, No. 4, pp. 381–396, (2001). 15. Zekic., M.:Neural network applications in stock market predictions: A methodology analysis, Proceedings of the 9th International Conference on Information and Intelligent Systems, pp.255–263 (1998). 16. Bansal, A., Kauffman, R.J. & Weitz, R.R.: Comparing the Performance of regression and neural networks a data quality varies: A business value approach, Journal of Management Information Systems, Vol. 10, No. 10, pp.11–32, (1993).

17. Vojinovic, Z., Kecman, V. & Seidel, R.: A data mining approach to financial time series modelling and forecasting, Intelligent Systems in Accounting, Finance and Management, Vol. 239, No. 4, pp.225–239, (2001). 18. Dunis, C.L. & Williams, M.: Modelling and Trading the EUR / USD Exchange Rate: Do Neural Network Models Perform Better?, Trading & Regulation, Vol. 8, No. 3, pp.1–24 (2002). 19. Dunis, C.L. & Williams, M.: Applications of Advanced Regression Analysis for Trading and Investment, John Wiley & Sons, (2003). 20. Pissarenko, D.: Neural networks for financial time series prediction: Overview over recent research, BSc thesis, University of Derby, Austria, (2002). 21. Aryal, D.R., Yao-wu, W.: Neural Network Forecasting of the Production Level of Chinese Construction Industry, Journal of Comparative International Management, Vol.6, No.2, pp.45–64, (2003.) 22. Bagherifard, K., Nilashi, M., Ibrahim, O., Janahmadi, N., & Ebrahim, L.: Comparative study of artificial neural network and ARIMA models in predicting exchange rate, Research Journal of Applied Sciences, Engineering and Technology, Vol.4, No.21, pp. 4397-4403, (2012). 23. Faraway, J. & Chatfield, C.: Time series forecasting with neural networks: a comparative study using the airline data, Journal of the Royal Statistical Society: Series C (Applied Statistics), Vol.47, No.2, pp.231–250, (1998). 24. Steurer, E.: Nonlinear modelling of the DEM/USD exchange rate. In Neural Networks in the Capital Markets, New York: Wiley, pp.199–211, (1993). 25. Walczak, S.: An empirical analysis of data requirements for financial forecasting with neural networks, Journal of Management Information Systems, Vol.17, No.4, pp.203-222, (2001). 26. Widrow, B., Rumelhart, D.E., Lehr, M.A.: Neural networks: Applications in industry, business and science, Communications of the ACM, Vol. 37, No. 3, pp. 93–105 (1994). 27. Hammerstrom,D.:Neural networks at work, IEEE Spectrum, Vol. 30, No. 6, pp. 26-32, (1993). 28. Lin, T. W., Yu, C. C.: Forecasting stock market with neural networks, SSRN Electronic Journal, http://dx.doi.org/10.2139/ssrn.1327544 , (2009). 29. Guresen, E., Kayakutlu, G., Daim, T.U.: Using artificial neural network models in stock market index prediction, Expert Systems with Applications, Vol. 38, No. 8, pp. 10389–10397, (2011) 30. Giles, C.L., Lawrence, S., Tsoi, A.C.: Noisy time series prediction using a recurrent neural network and grammatical inference. Machine Learning, Vol. 44, No. 1/2, pp. 161–183, (2001).

31. Yao, J. & Tan, C.L.: A case study on using neural networks to perform technical forecasting of forex, Neurocomputing, Vol.34, No.1-4, pp.79–98, (2000). 32. Castiglione, F.: Forecasting price increments using an artificial neural network. Advances in Complex Systems, Vol. 4, No. 1, pp. 45–56, (2001). 33. Chen, A.-S. & Leung, M.T.: Performance evaluation of neural network architectures: the case of predicting foreign exchange correlations, Journal of Forecasting, Vol. 24, No. 6, pp. 403–420, (2005). 34. Ghazali, R., Hussain,A., Nawi,N., Mohamad,B.:Non-stationary and stationary prediction of financial time series using dynamic ridge polynomial neural network. Neurocomputing, Vol. 72, No. 10–12, pp. 2359–2367, (2009). 35. Pacifici, F.: Novel neural network-based algorithms for urban classification and change detection from satellite imagery. PhD thesis, Tor Vergata University, Rome, Italy (2010). 36. Herrera, J.: Time series prediction using inductive reasoning techniques, PhD thesis, Instituto de Organizacion y Control de Sistemas Industriales, Universitat Politècnica de Catalunya, BarcelonaTech, Spain, (1999). 37. Zhang, X., Liu, Y., Yang, M., Zhang,T., Young, A., Li, X.:Comparative study of four time series methods in forecasting typhoid fever incidence in China, PLoS One, Vol. 8, No. 5, p. e63116, (2013). 38. Pedersen, M.: Optimization of recurrent neural networks for time series modelling, PhD thesis, Technical University of Denmark, Denmark (1997). 39. Mahdi, A.:The Application of Neural Network inFinancial Time Series Analysis and Prediction Using Immune System. MPhil thesis, Liverpool John Moores University, UK (2010). 40. Mahdi, A., Hussain, A., Al-Jumeily, D.:The Prediction of Non-Stationary Physical Time Series Using the Application of Regularization Technique in Self-organised Multilayer Perceptrons Inspired by the Immune Algorithm, 2010 Development of E-systems Engineering Conference, pp. 213–218, (2010). 41. Widyanto, M. R., Nobuhara, H., Kawamoto, K., Hirota, K., Kusumoputro, B.:Improving recognition and generalization capability of back-propagation NN using self-organized network inspired by immune algorithm, Applied Soft Computing, Vol.6, pp.72–84, (2005). 42. Shen, X., Gao, X. Z., Bie, R.: Artificial immune networks: Model and application, International Journal of Computational Intelligence Systems, Vol.1, No. 2, pp.168–176, (2008). 43. Jordan, M.I.: Attractor dynamics and parallelism in a connectionist sequential machine,” in in Artificial neural networks, NJ, USA: IEEE Press Piscataway, pp.112–127 (1990).

44. Voegtlin, T.:Recursive self-organizing maps, Neural Networks, Vol.15, No.8– 9, pp. 979–91 (2002). 45. Siwek, K., Osowski, S.: Regularization of neural networks for improved load forecasting in power system. ICECS 2001. 8th IEEE International Conference on Electronics, Circuits and Systems, pp.1255–1258 (2001). 46. Bishop, C.M.:Neural Networks for Pattern Recognition, Cambridge, UK (1995). 47. Larsen, J., Svarer, C., Andersen, L. N., & Hansen, L. K.: Adaptive regularization in neural network modelling, Neural Networks: Tricks of the Trade, Springer Berlin Heidelberg. pp.113–132, (1998). 48. Mahdi, A., Hussain, A., Lisbo, P., Al-Jumeily, D.: The Application of the Neural Network Model Inspired by the Immune System in Financial Time Series Prediction, 2009 Development of E-Systems Engineering conference, pp. 370–376, (2009). 49. Hussain, A. J., Al-jumeily, D.: How good is the backpropogation neural network using a self-organised network inspired by immune algorithm (SONIA) when used for multi-step financial time series prediction?, 4th International Symposium on Neural Networks, ISNN, Vol.4492, No.2, pp.921– 930, (2007). 50. Huang, Y., Lai, W., and Nakamori , K.K.: Forecasting foreign exchange rates with artificial neural networks: A review, International Journal of Information Technology and Decision Making, Vol.3, No.1, pp.145–165, (2004). 51. Alexandridis, A., Livanis, E.: Forecasting Crude Oil Prices Using Wavelet Neural Networks. In 5th FSDET, Athens, Greece, pp.1–8 (2008). 52. Thomason, M.: The practitioner method and tools, Journal of Computational Intelligence in Finance, Vol.7, No.3, pp.36–45, (1999). 53. Sundareshan, M., Wong,Y.C., Condarcure, T.: Training Algorithems For Recurrent Neural Nets Thet Eliminate The Need For Computation of Error Gradients With Application to Trajectory Production Problem. In Recurrent Neural Networks: Design and Applications, L. C. J. Larry Medsker, Ed. (1999). 54. Tay, F. E. H., Cao, L. J.: Modified support vector machines in Financial Time Series Forecasting. Neurocomputing, Vol.48, pp. 847–861, (2002). 55. Cao, L. J., Tay, F. E. H.: Support Vector Machine with Adaptive Parameters in Financial time Series Forecasting. IEEE Transactions on Neural Networks, Vol. 14, No. 6, pp. 1506–1518, (2003). 56. Huang, D. S.: The local minima free condition of feedforward neural networks for outer-supervised learning, IEEE Transactions on Systems, Man and Cybernetics, Vol.28B, No.3, pp.477-480, (1998).

57. Huang, D. S.: A constructive approach for finding arbitrary roots of polynomials by neural networks,” IEEE Transactions on Neural Networks, Vol.15, No.2, pp.477-491, (2004). 58. Montgomery, D. C. and Runger, G. C.: Applied Statistics and Probability for Engineers. New York: Wiley (1999). 59. Huang, D.S: “Radial basis probabilistic neural networks: Model and application,” International Journal of Pattern Recognition and Artificial Intelligence, Vol.13, No.7, pp.1083-1101, (1999).

Dr. Abir Jaafar Hussain is a senior lecturer at the School of Computing and Mathematical Sciences at Liverpool John Moores University, UK. She completed her PhD study at The University of Manchester, UK in 2000 with a thesis title Polynomial Neural Networks for Image and Signal Processing. She has published numerous referred research papers in conferences and Journal in the research areas of Neural Networks, Signal Prediction, Telecommunication Fraud Detection and Image Compression. She is a PhD supervisor and an external examiner for research degrees including PhD and MPhill.

Dr. Haya Al-Askar is a full time research student working at the School of Computing and Mathematical Sciences at Liverpool John Moores University. During her PhD study, she has published in a number of refereed journal and conference papers for the prediction of financial time series and the classification of medical data.

Dr. Dhiya Al-Jumeily is a principle lecturer in eSystems Engineering and leads the Applied Computing Research Group at the faculty of Engineering and Technology. He has already developed fully online MSC and BSc programmes and currently heads the enterprise activities at the School of Computing and Mathematical Sciences. Dr. Al-Jumeily has published numerous referred research papers in multidisciplinary research areas including: Technology Enhanced Learning, Applied Artificial Intelligence, Neural Networks, Signal Prediction, Telecommunication Fraud Detection and Image Compression. He is a PhD supervisor and an external examiner for the degree of PhD. He has been actively involved as a member of editorial board and review committee for a number peer reviewed international journals, and is on program committee or as a general chair for a number of international conferences.

Dr. Naeem Radi is the CEO of the Al Khawarizmi University College in the UAE. His main area of research involves the design of image processing algorithms for image compression. Dr Radi has published many journal and conference papers and reports on many aspects of this research and has also acted as Session Chair and on program committees for many international conferences. He serves as professional fellowship for BCS and as a Professional Member for IEEE.

Abir Jaafar Hussain

Dhiya Al-Jumeily

Naeem Radi