Particle swarm optimization of ensemble neural networks with fuzzy aggregation for time series prediction of the Mexican Stock Exchange

INS 10870

No. of Pages 17, Model 3G

19 May 2014 Information Sciences xxx (2014) xxx–xxx 1

Contents lists available at ScienceDirect

Information Sciences journal homepage: www.elsevier.com/locate/ins 6 7

Particle swarm optimization of ensemble neural networks with fuzzy aggregation for time series prediction of the Mexican Stock Exchange

3 4 5 8

Q1

9

Martha Pulido, Patricia Melin ⇑, Oscar Castillo Tijuana Institute of Technology, Division of Graduate Stusies and Research, Calzada Tecnologico s/n, Fracc. Tomas Aquino, 22379 Tijuana, BC, Mexico

10

a r t i c l e

1 3 2 2 13 14 15 16 17

i n f o

Article history: Received 23 October 2013 Received in revised form 27 March 2014 Accepted 3 May 2014 Available online xxxx

18 19 20 21 22

Keywords: Ensemble neural network Particle swarm optimization Time series prediction

a b s t r a c t This paper describes a hybrid method based on particle swarm optimization for designing ensemble neural networks with fuzzy aggregation of responses to forecast complex time series. The time series that was considered in this paper, to compare the hybrid approach with traditional methods, is the Mexican Stock Exchange, and the results shown are for the optimization of the structure of the ensemble neural network with type-1 and type-2 fuzzy logic integration. Simulation results show that the optimized ensemble neural network approach produces good prediction of the Mexican Stock Exchange. Ó 2014 Published by Elsevier Inc.

24 25 26 27 28 29 30 31 32 33

34 35

1. Introduction

36

A time series is a sequence of data points, measured typically at successive points in time and spaced at uniform time intervals [2,11]. Examples of time series are the daily closing value of the Mexican Stock Exchange. Time series are very frequently plotted via line charts. Time series are used in statistics, signal processing, pattern recognition, econometrics, mathematical finance, weather forecasting, earthquake prediction, electroencephalography, control engineering, astronomy and communications engineering. Stock fund managers and financial analysts attempt to predict price activity in the stock market on the basis of either their professional knowledge or stock analyzing tools, in order to obtain gains in prices in the stock market. For more than one decade, time series models have also been applied to solve various domain problems, such as financial forecasting. In this paper the design of a neural network ensemble with the Particle Swarm Optimization algorithm (PSO) and its integration with type-1 and type-2 fuzzy systems, along with simulation results for the prediction of the Mexican Stock Exchange are presented. In the literature there has been a lot of recent work in time series prediction [3,4,7–10,20,22,26–28,37,42,46], which indicate the recent importance of the topic. Time series prediction with hybrid models has also been a hot topic in recent years [40,41,43]. This paper shows the results of an optimized ensemble neural network and its fuzzy response aggregation for predicting the time series of the Mexican Stock Exchange [1]. The PSO technique is used to optimize the neural network ensemble, as this can help in finding a good solution to a complex problem, which in this case is to find the best ensemble neural network architecture to obtain

37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52

Q2

⇑ Corresponding author. Tel.: +52 6646236318. E-mail address: [email protected] (P. Melin). http://dx.doi.org/10.1016/j.ins.2014.05.006 0020-0255/Ó 2014 Published by Elsevier Inc.

Please cite this article in press as: M. Pulido et al., Particle swarm optimization of ensemble neural networks with fuzzy aggregation for time series prediction of the Mexican Stock Exchange, Inform. Sci. (2014), http://dx.doi.org/10.1016/j.ins.2014.05.006

INS 10870

No. of Pages 17, Model 3G

19 May 2014 2

M. Pulido et al. / Information Sciences xxx (2014) xxx–xxx

81

the minimum forecast error for the time series mentioned above and to integrate the results with type-1 and type-2 fuzzy systems. The main contribution of the paper is the proposed model of a neural network ensemble with type-2 fuzzy logic for response integration. In addition the particle swarm optimization method determines the number of modules of the neural network ensemble, number of layers and number of neurons per layer, and thus obtains the best architecture of the ensemble neural network. After obtaining this architecture the results from the modules are aggregated with type-1 and type-2 fuzzy systems, the inputs to the fuzzy system are the responses according to the number of network modules and this are the number of inputs of the fuzzy system. In this case, the maximum number of inputs is of 5 inputs and one output with two Gaussian membership functions and these are be granulated in two linguistic variables that are low and high forecast, and the forecast output will also be low or high and thereby obtain the forecast error for this series of the Mexican Stock Exchange. The proposed hybrid ensemble of neural networks with fuzzy response aggregation and its optimization with PSO is main contribution of the paper, as this hybrid approach has not been proposed previously in the literature for this kind of time series prediction problems. This paper considers the times series of the Mexican Stock Exchange because the problem is quite complex and the time series is chaotic in some periods of time and for this reason an ensemble model is justified. To consider different possible behaviors we have also previously worked with other time series, such as the Mackey–Glass s ¼ 17; 34 and 68 and this series has chaotic behavior and as the value of s increases it will be more chaotic. Also the US/Dollar/MX Peso and the Dow Jones time series, which are quite complex, have been considered with ensembles and have produced good results in these cases [30]. We can find that there are similar ensemble approaches in the literature, but the main difference is that in the proposed approach in this paper we use type-2 fuzzy aggregation of responses in the ensemble, which is not found into other similar works. The idea of using type-2 fuzzy logic is the better manage the uncertainty in prediction. In addition, we also use PSO to optimize ensemble neural network, which is not so commonly used in ensembles. We consider this as the new contribution, since the results obtained are better than other methods proposed in the literature. The rest of the paper is organized as follows: Section 2 describes the architecture of the ensemble neural network, basic concepts of the proposed method, Section 3 describes the concepts particle swarm optimization, In Section 4 Optimization Ensemble Neural Network with Particle Swarm Optimization. Section 5 describes the concepts of fuzzy systems as methods of integration. In Section 6 the methodology for solving it are represented, Section 7 describes the simulation results of the proposed method, in Section 8 the comparison and the t student test are presented, and Section 9 shows the conclusions.

82

2. Time series and prediction

83

103

Time series are analyzed to understand the past and to predict the future, enabling managers or policy makers to make properly informed decisions. A time series analysis quantifies the main features in data and the random variation. These facts, combined with improved computing power, have made time series methods widely applicable in government, industry, and commerce. Time series models often form the basis of computer simulations. Some examples are assessing different strategies for control of inventory using a simulated time series of demand; comparing designs of wave power devices using a simulated series of sea states; and simulating daily rainfall to investigate the long-term environmental effects of proposed water management policies A time-series is defined as a sequence of observations on a set of values that takes a variable (quantitative) at different points in time. Time series are widely used today because organizations need to know the future behavior of certain relevant phenomena in order to plan, prevent, and so on, their actions. That is, to predict what will happen with a variable in the future from the behavior of that variable in the past [1]. The data can behave in different ways over time, this may be a trend, which is the component that represents a long-term growth or decline over a period of time. A cycle is also possible, which refers to the wave motion that occurs around the trend, or may not have a defined or random manner; there are seasonal variations (annual, biannual, etc.), which are a behavior patterns that are repeated year after year at a particular time [2–4]. The word ‘‘prediction’’ comes from the Latin prognosticum, which means I know in advance. Prediction is to issue a statement about what is likely to happen in the future, based on analysis and considerations of experiments. Making a forecast is to obtain knowledge about uncertain events that are important in decision-making [5]. Time series prediction tries to predict the future based on past data, it take a series of real data xt  n, . . . xt  2.0 xt  1, xt and then obtains the prediction of the data xt + 1, xt + 2 . . . xn + n. The goal of time series prediction or its model is to observe the series of real data, so that future data may be accurately predicted [29].

104

3. Ensemble neural network

105

Artificial neural networks are inspired by the biological nervous system architecture, which consists of a large number of relatively simple neurons that work in parallel to facilitate rapid decision making [32]. A neural network is a system of parallel processors interconnected as a directed graph. Schematically each processing element (neurons) in the network is represented as a node. Its most important advantage is to solve problems that are

53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80

84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102

106 107 108

Please cite this article in press as: M. Pulido et al., Particle swarm optimization of ensemble neural networks with fuzzy aggregation for time series prediction of the Mexican Stock Exchange, Inform. Sci. (2014), http://dx.doi.org/10.1016/j.ins.2014.05.006

INS 10870

No. of Pages 17, Model 3G

19 May 2014 M. Pulido et al. / Information Sciences xxx (2014) xxx–xxx

3

115

too complex for conventional technologies, problems that have no solution algorithm and its solution algorithm is very difficult finding. Artificial neural networks are formed by a large number of neurons; these are not called artificial neurons but output nodes or units. Ensemble models are formed by a collection of neural networks with the idea of combining multiple decisions and achieving a better result. These ensemble models behave remarkably well, recently it has become a very hot topic in both the neural networks and machine learning communities [21], and has already been successfully applied to diverse areas such as face recognition [35,19], optical character recognition [13,18,28], scientific image analysis [40], medical diagnosis [13,46], seismic signals classification [36], etc.

116

3.1. Training of the neural networks

117

In the literature there are many methods for training neural networks like, the gradient descendent method, Levenberg– Marquardt and others. The Levenberg–Marquardt method can handle well ill conditioned matrices JTJ by using the following equation:

109 110 111 112 113 114

118 119

120

122 123 124 125

126 128

 1 hnext ¼ hnow  J T J þ kI gh

ð1Þ

where k is some nonnegative value and g h  12 g for simplicity.For instance, we consider a trivial nonlinear model f(xk  h) with two adjustable parameters p and q; i.e., h = (p, q)T. The objective is to find the optimal h⁄ = (1, 10) that minimizes the following sum of squared errors.

EðhÞ ¼

10 X 2 ðt k f ðxk ; hÞÞ

ð2Þ

k¼1

131

where tk is the desired output E is the objective function, and the Levenberg–Marquardt direction that is determined by using Eq. (1) when k ¼ 0:07 or any other particular value. Clearly, the Levenberg–Marquardt direction (in highlighted arrow) is an intermediate between the Gauss Newton direction ðk ! 0Þ and the steepest descendent direction ðk ! 1Þ.

132

4. Particle swarm optimization

133

Particle Swarm Optimization (PSO) is a bio-inspired optimization method proposed by Eberhart and Kennedy [14,25] in 1995. PSO is a search algorithm based on the behavior of biological communities that exhibits individual and social behavior [15,16], and examples of these communities are groups of birds, schools of fish and swarms of bees [23,24,44]. A PSO algorithm maintains a swarm of particles, where each particle represents a potential solution. In analogy with the paradigms of evolutionary computation, a swarm is similar to a population, while a particle is similar to an individual. In simple terms, the particles are ‘‘flown’’ through a multidimensional search space, where the position of each particle is adjusted according to its own experience and that of its neighbors. Let xi denote the position i in the search space at time step t; unless otherwise stated, t denotes discrete time steps. The position of the particle is changed by adding a velocity, mt(t), to the current position, i.e.

129 130

134 135 136 137 138 139 140

141

142 144 145 146 147 148

149 151

152 153 154 155

156 158

159 160 161

162 164

165 166 167

168 170

xi ðt þ 1Þ ¼ xi ðtÞ þ v i ðt þ 1Þ

ð3Þ

with xi(0)  U(Xmin, Xmax). It is the velocity vector the one that drives the optimization process, and reflects both the experimental knowledge of the particles and the information exchanged in the vicinity of particles. For gbest PSO, the particle velocity is calculated as:

v ij ðt þ 1Þ ¼ v ij ðtÞ þ c1 r1 ½yij ðtÞ  xij ðtÞ; þc2 r2 ðtÞ½y^j ðtÞ  xij ðtÞ

ð4Þ

where vij(t) is the velocity of the particle i in dimension j at time step t; c1 and c2 are the positive acceleration constants used to scale the contribution of cognitive and social skills, respectively, yr1j(t), yr2j(t)  U(0, 1) are random values in the range [0, 1]. The best personal position in the next time step t + 1 is calculated as:

 yi ðt þ 1Þ ¼

yi ðtÞ

if f ðxi ðxi ðt þ 1ÞÞ P f yi ðtÞÞ

xi ðt þ 1Þ if f ðxi ðxi ðt þ 1ÞÞ > f yi ðtÞÞ

ð5Þ

where f : Rnx ! R is the fitness function, as with EAs, the goal is measuring the fitness function closely corresponding to the optimal solution, for example the objective function quantifies the performance, or the quality of a particle (or solution). The overall best position, y_ (i) (t) at time step t, is defined as:

^ðtÞfyo ðtÞ; . . . ; ys ðtÞgf ðyðtÞÞ ¼ min ff ðyo ðtÞÞ; . . . f ðys ðtÞÞ;g y

ð6Þ

^ the where ns is the total number of particles in the swarm, More importantly, the above equation defining and establishing y best position is uncovered by either of the particles so far as this is usually calculated as the best personal best position. The global best position can also be selected from the particles of the current swarm, in which case it expressed as:

^ðtÞ ¼ min ff ðxo ðtÞÞ; . . . f ðxns ðtÞÞ;g y

ð7Þ

Please cite this article in press as: M. Pulido et al., Particle swarm optimization of ensemble neural networks with fuzzy aggregation for time series prediction of the Mexican Stock Exchange, Inform. Sci. (2014), http://dx.doi.org/10.1016/j.ins.2014.05.006

INS 10870

No. of Pages 17, Model 3G

19 May 2014 4

172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192

193 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210

5. Fuzzy systems According to [23,24,44], fuzzy logic was conceived by Lotfi A. Zadeh in 1965, based on a fuzzy set that differs from the traditional sets considering now that the membership degree is define by a function that represents a membership, which evaluates the input, and according to some predefined rules, assigns the membership degree to a fuzzy set. Fuzzy Set: Let X be a space of objects and x be a generic element of X. A classical set A, A # X, is defined as a collection of elements or objects x e X, such that each x can either belong or not belong to the set A. By defining a characteristic function for each element x in X, we can represent a classical set A by a set of ordered pairs (x, 0) or (x, 1) which indicates x R A or xA, respectively. Unlike the aforementioned conventional set, a fuzzy set expresses the degree to which an element belongs to a set. Hence the characteristic function of a fuzzy set is allowed to have values between 0 and 1, which denotes the degree of membership of an element in a given set. The basic structure of a fuzzy inference system consists of three conceptual components: a rule base, which contains a selection of fuzzy rules; a dictionary, which defines the membership functions used in the fuzzy rules; and a reasoning mechanism, which performs the inference produce (usually the fuzzy reasoning) [9]. Q3 According to [36], there is another class of fuzzy systems called type-2 fuzzy systems, which were also proposed by Zadeh. The reason to augment the original fuzzy systems is to consider higher levels of uncertainty, broadening their scope. In type-2 fuzzy systems the membership functions can now return a range of values that varies depending on the uncertainty involved, not only in the input but also on the same membership function. The structure of the type-2 fuzzy rules is the same as for the type-1 case because the distinction between type-2 and type1 is associated with the nature of the membership functions. Hence, the only difference is that now some or all the sets involved in the rules are of type-2 [46]. As an example: Consider the case of a fuzzy set characterized by a Gaussian membership function with mean m and a standard deviation that can take values in [1r1, r2 ], i.e.,



lðxÞ ¼ exp 1=2

hx  mi2  ;

r½r1 ; r2 

r

ð8Þ

Corresponding to each value of r we will get a different membership curve (as shown in Fig. 1). So, the membership grade of any particular x(except x = m) can take any of a number of possible values depending upon the value of r i.e., the membership grade is not a crisp number, it is a fuzzy set. Fig. 3 shows the domain of the fuzzy set associated with x = 0.7; however, the membership function associated with this fuzzy set is not shown in the Fig. 2. Gaussian type-2 fuzzy set. A Gaussian type-2 fuzzy set is one in which the membership degree of every domain point is a Gaussian type-1 set contained in [0, 1]. Interval type-2 fuzzy set. An interval type-2 fuzzy set is one in which the membership degree of every domain point is a crisp set whose domain is some interval contained in [0, 1]. Footprint of uncertainty: The uncertainty in the primary memberships of a type-2 fuzzy set, Ã, consists of a bounded region that we call the ‘‘footprint of uncertainty’’ (FOU). Mathematically, it is the union of all primary membership functions [22]. Upper and lower membership functions. An ‘‘upper membership function’’ and a ‘‘lower membership functions’’ are two type-1 membership functions that are the bounds for the FOU of a type-2 fuzzy set Ã. The upper membership function is associated with the upper bound of the FOU(Ã). The lower membership function is associated with the lower bound of the FOU(Ã).

Pron1Low

1

Degree of membership

171

M. Pulido et al. / Information Sciences xxx (2014) xxx–xxx

Pron1High

0.8

0.6

0.4

0.2

0 2.5

2.6

2.7

2.8

2.9

3

3.1

3.2

3.3

3.4

3.5

Prediction1 Fig. 1. Type-1 membership functions.

Please cite this article in press as: M. Pulido et al., Particle swarm optimization of ensemble neural networks with fuzzy aggregation for time series prediction of the Mexican Stock Exchange, Inform. Sci. (2014), http://dx.doi.org/10.1016/j.ins.2014.05.006

INS 10870

No. of Pages 17, Model 3G

19 May 2014 5

M. Pulido et al. / Information Sciences xxx (2014) xxx–xxx

1

Pron1Low

Pron2High

0.9 0.8 0.7 0.6 0.5 0.4 0.3 0.2 0.1 0 2.5

2.6

2.7

2.8

2.9

3

3.1

3.2

3.3

3.4

3.5

Fig. 2. Type-2 membership functions.

211 212 213

214

Operations of type-2 fuzzy sets f1 and A f2 is another type-2 fuzzy set, just as the union of type-1 fuzzy sets A1 and Union of type-2 fuzzy sets. The union of A A2 is another type-1 fuzzy set. More formally, we have the following expression:

f1 [ A f2 ¼ A

216 217 218

219

x2X

223

224

f1 \ A f2 ¼ A

228 229

230

1

ð9Þ

2

l Ae \ Ae ðxÞ=x 1

ð10Þ

2

Complement of a type-2 fuzzy set. The complement of a set is another type-2 fuzzy set, just as the complement of type-1 fuzzy set A is another type-1 fuzzy set. More formally we have:

f1 ¼ A

Z x

leA 0 1 ðxÞ=x

ð11Þ

where the prime denotes complement in the above equation. In this equation le0 is the secondary membership function. A1 Type-2 fuzzy rules. Consider a type-2 FLS having r inputs x1 2 X 1 ; . . . ; xr 2 X r and one output y e Y. As in the type-1 case, we can assume that there are M rules; but, in the type-2 case the ‘th rule has the form:

~ 1 and . . . xp is A e 1 ; THEN y is Y b1 R1: IF x1 is A 1 P

232

233

Z x2X

226 227

l Ae [ Ae ðxÞ=x

f1 and A f2 is another type-2 fuzzy set, just as the intersection of Intersection of type-2 fuzzy sets. The intersection of A f type-1 fuzzy sets A1 and A2 is another type-1 fuzzy set. More formally, we have the following expression:

221 222

Z

1 ¼ 1; . . . M:

6. Optimization of the ensemble neural network with particle swarm optimization

234

The operation of the optimization algorithm by the swarm of particles is given by a sequence of steps, which are:

235

1. 2. 3. 4.

236 237 238

ð12Þ

Generate the initial swarm. Evaluate the particles of the swarm. Updating the particle velocity. Calculating new positions of the particles.

239 240 241 242 243 244 245 246 247 248

From steps 2–4, begins an iterative process until a stopping criterion is satisfied. In Fig. 3, we can see the outline of the algorithm of particle swarm optimization. To optimize the Ensemble Neural Network following the outline of the algorithm of particle swarm optimization, the first step is to create a swarm of particles, and to create the swarm we have to choose the representation of candidate solutions (particles), for which we choose a real representation. Once we have the representation of candidate solutions (particles), set the number of variables to indicate the position of the particle or possible solution to our problem, number of the particle. Set the number of variables that will form the particle, the initial population is generated randomly within a set range for each variable according to upper and lower limits of the search space for each variable. Please cite this article in press as: M. Pulido et al., Particle swarm optimization of ensemble neural networks with fuzzy aggregation for time series prediction of the Mexican Stock Exchange, Inform. Sci. (2014), http://dx.doi.org/10.1016/j.ins.2014.05.006

INS 10870

No. of Pages 17, Model 3G

19 May 2014 6

M. Pulido et al. / Information Sciences xxx (2014) xxx–xxx

Fig. 3. Ensemble neural network model. 249

7. Model description

250

This section describes the proposed model for the neural network ensembles, the objective is to optimize the architecture of a neural network with the particle swarm optimization method. Fig. 3 represents the Ensemble Neural Network Model, where we have first the historical data, then the optimization algorithm based on swarms means of particles determines the number of modules of the ensemble neural network, which can be from 1 to 5 modules, as well as the number of layers that this module, that could of from 1 to 3 and the number of neurons by layer that could be of 1 to 3. Then after the responses of neural network ensemble are obtained the integration is performed with type-1 and type-2 fuzzy systems. The input and output linguistic variables of the fuzzy system have Gaussian membership functions. The performance of the type-2 fuzzy integrators is analyzed under different levels of uncertainty to find out the best design of the membership functions for the rules of the fuzzy system. For the type-2 fuzzy integrator using 2 membership functions, which are called ‘‘low prediction’’ and ‘‘high prediction’’ for each of the inputs and the output of the fuzzy system. Membership functions are of Gaussian type and we deal with 3 footprint uncertainty sizes of 0.3, 0.4 and 0.5 to obtain a better prediction of the time series. If we consider 5 modules in the ensemble then the fuzzy system will have 5 input linguistic variables and the rules used are determined according to the possible combinations according to the number of inputs and membership functions of the fuzzy system. Considering that have 5 inputs and two membership functions for each of the inputs then there are 32 possible combinations. Each of the modules in the ensemble is a neural network and the calculations are performed as follows: x of a node is defined by the weighted sum of the incoming signals plus a bias term. For instance, the net The net input  input of the output of the node j is given as follows.

251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269

xj ¼

X

xij þ xj ;

i

271 272 273 274 275

276 278

x ¼ f ðxj Þ ¼

1 ; 1 þ expðxj Þ

ð13Þ

where xi is the output of node i located in any of the previous layers, xij is the weight associated with the link connecting nodes i and j, and xj is the bias of node j. Since the weights xij are actually internal parameters associated with each node j changing the weights of the node will alter behavior of the node and in turn alter behavior of the whole backpropagation MLP. A squared error measure for the pth input–output pair is defined as:

EP ¼

X

ðdk  xk Þ

2

ð14Þ

Please cite this article in press as: M. Pulido et al., Particle swarm optimization of ensemble neural networks with fuzzy aggregation for time series prediction of the Mexican Stock Exchange, Inform. Sci. (2014), http://dx.doi.org/10.1016/j.ins.2014.05.006

INS 10870

No. of Pages 17, Model 3G

19 May 2014 M. Pulido et al. / Information Sciences xxx (2014) xxx–xxx 279 280 281 282 283 284

285 287

7

where dk is the desired output for the node xk is the actual output for the node when the input part of the pth data pair is presented. In addition a fuzzy system is used as a response integrator for the ensemble. This means that the outputs of the modules (neural network) are the inputs to the fuzzy system (integrator of the responses). The global or total results are obtained by using the inference process in the fuzzy system. The total result of the fuzzy integrator system is calculated with the following equation:

Pn i¼1 li ðxÞdx y¼ P ; n i¼1 li ðxÞ

ð15Þ

288

where x are the input data and l are the corresponding membership functions.

289

8. Methodology

290

This section describes the general architecture of the proposed ensemble neural network model. The aim of this work was to implement a PSO algorithm to optimize the neural network architectures, for each of the modules, number of neurons and number of layers and thus find an architecture for the neural network that produces the best results in each particular time series that is considered. Fig. 4 shows the overall architecture, where historical data for each time series prediction, in this case the Mexican Stock Exchange is used as input, and then the time series analyzed and in the next step the data provided to the modules, which are optimized with PSO in which the particle contains information of the ensemble neural network for the optimization. The particle gives us the number of modules, number of layers and number of neurons are then applied to the neural network ensemble and then these results of these modules are integrated with an integration method based on type-1 and type-2 fuzzy logic and finally obtain the prediction error.

291 292 293 294 295 296 297 298

Fig. 4. General architecture of the proposed ensemble neural network design.

Please cite this article in press as: M. Pulido et al., Particle swarm optimization of ensemble neural networks with fuzzy aggregation for time series prediction of the Mexican Stock Exchange, Inform. Sci. (2014), http://dx.doi.org/10.1016/j.ins.2014.05.006

INS 10870

No. of Pages 17, Model 3G

19 May 2014 8 299 300 301 302 303 304 305 306 307 308 309 310 311 312 313

M. Pulido et al. / Information Sciences xxx (2014) xxx–xxx

The Mexican Stock Exchange (MSE) is a financial institution that operates by a grant from the Department of Finance and Public Credit, with the goal of following closely the Securities Market of Values in Mexico. Derived from the monitoring of global trends and changes that have occurred in legislation, the MSE concluded with the conversion process, becoming a company whose shares are likely to be traded on the market stock exchange, with the initial public offering taking place on June 13 of 2008 with its shares representing its capital [31]. Data of the Mexican Stock Exchange time series: In this case 800 points are used, that correspond to a period from 11/09/ 05 to 01/15/09 where we plot in the x axis the historical data in this case the day and in the y axis the closing of the Mexican Stock Exchange (as shown in Fig. 5). In the experiment 70% of the data were used for the ensemble neural network training and 30% to test the network [31]. After some tests were performed, another period of data from 04/08/08 to 05/09/11 for this series was also selected, where we plot in the x axis the historical data in this case day, and the y axis is the closing of the Mexican Stock Exchange (as shown in Fig. 6). For the optimization of the structure of the ensemble neural network a particle swarm optimization algorithm was used. The structure of the swarm particle represents the structure of the ensemble. The objective function is defined to minimize the prediction error as follows:

314

EM ¼

318

D

ð16Þ

i¼1

where a, corresponds to the predicted data depending on the output of the network modules, X represents real data, D the Number of Data points and EM is the total average prediction error.

3.4

Mexican Stock Exchange

x 104

3.2 3 2.8 2.6

Closing

317

!,

2.4 2.2 2 1.8 1.6 1.4 0

100

200

300

400

500

600

700

800

700

800

Days Fig. 5. Mexican Stock Exchange of time series 1.

4

Mexican Stock Exchange

x 104

3.5

Closing

316

D X jai  xi j

3

2.5

2

1.5 0

100

200

300

400

500

600

Days Fig. 6. Mexican Stock Exchange of time series 2.

Please cite this article in press as: M. Pulido et al., Particle swarm optimization of ensemble neural networks with fuzzy aggregation for time series prediction of the Mexican Stock Exchange, Inform. Sci. (2014), http://dx.doi.org/10.1016/j.ins.2014.05.006

INS 10870

No. of Pages 17, Model 3G

19 May 2014 9

M. Pulido et al. / Information Sciences xxx (2014) xxx–xxx 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340 341 342 343 344 345 346

The corresponding particle structure is shown in Fig. 7. Fig. 7 represents the particle structure to optimize the ensemble neural network, where the parameters that are optimized are the number of modules, number of layers, and number of neurons of the ensemble neural network. PSO determines the number of modules, number of layers and number of neurons per layer, that the neural network ensemble should have and that meets the objective to achieve a better Prediction error. The parameters for the particle swarm optimization algorithm are: 100 Particles, 100 iterations, Cognitive Component (C1) = 2, Social Component (C2) = 2, Constriction coefficient of linear increase (C) = (0–0.9) and Inertia weight with linear decrease (W) = (0.9–0). We consider a number of 1–5 modules, number of layers of 1–3, number neurons of 1–30. The network parameters used to train the neural network ensemble for each of the ensembles are 100 epochs, learning rate of = 0.01, the error goal 0.01, training method of Levenberg–Marquardt (trainlm), these parameters were used in this work because previous tests were performed and we managed to obtain a good prediction error with these parameters. The parameters for the particle swarm optimization algorithm are: 100 particles, 100 iterations and these parameters were used because previously an optimization with a genetic algorithm was considered and to compare it with this method we used similar parameters, Cognitive Component (C1) = 2, Social Component (C2) = 2, Constriction coefficient of linear increase (C) = (0–0.9) and Inertia weight with linear decrease (W) = (0.9–0), these parameters were used since we did perform a manual testing and by changing the values and with these parameters we obtain a better prediction error also. We consider a number of 1–5 modules, number of layers of 1–3, neurons number of 1–30, as maximum number of modules, layers and neurons that had already previously been tested with 3 modules and increasing error gives us better prediction. The aggregation of the responses of the optimized ensemble neural network is performed with type-1 and type-2 fuzzy systems. In this work a fuzzy system consisting of 5 inputs depending on the number of modules of the neural network ensemble and one output are used. Each input and output linguistic variable of the fuzzy system uses 2 Gaussian membership functions because the performance of the type-2 fuzzy aggregators is analyzed under different levels of uncertainty to find out the best design of the membership functions for the 32 rules of the fuzzy system with good results. Previous tests have been performed only with a three input fuzzy system, however the fuzzy system changes according to the responses of the neural network to the fuzzy system to give us better prediction error. In the type-2 fuzzy system we also change the levels of uncertainty to obtain the best prediction error. Fig. 8 shows a fuzzy system consisting of 5 inputs depending on the number of modules of the neural network ensemble and one output. Each input and output linguistic variable of the fuzzy system uses 2 Gaussian membership functions. The

Number Of Module

Number Of Layers

Neurons 1

...

Neurons n

Fig. 7. Particle structure to optimize the ensemble neural network.

Prediction1 (2)

MexicanStock Prediction2 (2)

(mamdani) Prediction3 (2) 32 rules

Prediction (2)

Prediction4 (2)

Prediction5 (2) System MexicanStock: 5 inputs, 1 outputs, 32 rules

Fig. 8. Type-2 fuzzy system for the Mexican Stock Exchange.

Please cite this article in press as: M. Pulido et al., Particle swarm optimization of ensemble neural networks with fuzzy aggregation for time series prediction of the Mexican Stock Exchange, Inform. Sci. (2014), http://dx.doi.org/10.1016/j.ins.2014.05.006

INS 10870

No. of Pages 17, Model 3G

19 May 2014 10

M. Pulido et al. / Information Sciences xxx (2014) xxx–xxx

Fig. 9. Type-2 fuzzy rules for predicting the Mexican Stock Exchange.

347 348

performance of the type-2 fuzzy aggregators is analyzed under different levels of uncertainty to find out the best design of the membership functions for the 32 rules of the fuzzy system. Please cite this article in press as: M. Pulido et al., Particle swarm optimization of ensemble neural networks with fuzzy aggregation for time series prediction of the Mexican Stock Exchange, Inform. Sci. (2014), http://dx.doi.org/10.1016/j.ins.2014.05.006

INS 10870

No. of Pages 17, Model 3G

19 May 2014 M. Pulido et al. / Information Sciences xxx (2014) xxx–xxx

11

350

Fig. 9 represents the possible 32 rules of the fuzzy system; we have 5 inputs in the fuzzy system with 2 membership functions, and outputs with 2 membership functions. These fuzzy rules are used for the type-1 and type-2 fuzzy systems.

351

9. Simulation results

352

In this section we present the simulation and test results obtained with the Particle swarm optimization algorithm applied in the design of the ensemble neural network for the Mexican Stock Exchange. A particle swarm algorithm was used to optimize the structure of an ensemble neural network and the best obtained architecture is represented in Fig. 10. Fig 10 shows the architecture of the neural network obtained by particle swarm optimization. We used two layers in each module. In module 1, in the first layer we used 21 neurons and 16 neurons in the second layer and in module 2 we used 21 neurons in the first layer and 28 neurons in the second. The optimization method using swarms of particles determines the number of neurons in the neural network ensemble. The Levenberg–Marquardt (LM) training method was used; we also applied 3 delays to the network.

349

353 354 355 356 357 358 359

Fig. 10. Ensemble neural network architecture for the MSE series 1.

Table 1 Particle swarm results for the ensemble neural network.

Please cite this article in press as: M. Pulido et al., Particle swarm optimization of ensemble neural networks with fuzzy aggregation for time series prediction of the Mexican Stock Exchange, Inform. Sci. (2014), http://dx.doi.org/10.1016/j.ins.2014.05.006

INS 10870

No. of Pages 17, Model 3G

19 May 2014 12

M. Pulido et al. / Information Sciences xxx (2014) xxx–xxx

Mexican Stock Exchange

x 10 4 3.4 3.2 3

Close

2.8 2.6 2.4 2.2 2 1.8 1.6 500

550

600

650

700

750

800

Days Fig. 11. Prediction with the optimized ensemble neural network of the MSE for time series 1.

Table 2 Results type-1 fuzzy aggregation of MSE for time series 1.

360 361 362 363 364 365 366 367 368 369 370 371 372 373 374 375 376 377 378 379 380 381

Table 1 shows the results of the optimization algorithm using particles for which 30 experiments were performed, but this table only shows the best 10 results, where the best obtained result was in the 6 wherein the prediction error was: 0.0089454. Fig. 11 shows the plot of real data against the predicted data generated by the ensemble neural network for time series Q4 number 1. Pink color1 represents the real data and the green color represents prediction data. Fuzzy integration is performed by implementing a type-1 fuzzy system in which the best result was in the experiment of row number 7 of Table 2 with an error of: 0.1458. Fuzzy aggregation is also performed by implementing a type-2 fuzzy system in which the results are as follows: for the best experiment with a degree of uncertainty of 0.3 a forecast error of 0.3685 was obtained, and with a degree of uncertainty of 0.4 a forecast error of 0.3960 was obtained and with a degree of uncertainty of 0.5 a forecast error of 0.3761 was obtained, as shown in Table 3. Fig. 12 shows the architecture of the neural network obtained by the particle swarm optimization method. We used two layers in each module. In module 1, in the first layer we used 12 neurons and 11 neurons, in the second layer; in module 2 we used 6 neurons in the first layer and 18 neurons in the second. The optimization method using swarms of particles determines the number of neurons in the neural network ensemble. The Levenberg–Marquardt (LM) training method was used; we also applied 3 delays to the network. Table 4 shows the particle swarm results (as shown in Fig. 12), where the best prediction error is of 0.0079875, which is shown in row number two. Fig. 13 shows the plot of real data against the predicted data generated by the ensemble neural network for time series number 2. The Pink color represents the real data and the green color represents the prediction data. Fuzzy aggregation is performed by implementing a type-1 fuzzy system in which the best result was in the experiment of row number 2 of Table 5 with an error of: 0.0952.

1

For interpretation of color in Figs. 11 and 13, the reader is referred to the web version of this article.

Please cite this article in press as: M. Pulido et al., Particle swarm optimization of ensemble neural networks with fuzzy aggregation for time series prediction of the Mexican Stock Exchange, Inform. Sci. (2014), http://dx.doi.org/10.1016/j.ins.2014.05.006

INS 10870

No. of Pages 17, Model 3G

19 May 2014 M. Pulido et al. / Information Sciences xxx (2014) xxx–xxx

13

Table 3 Results of type-2 fuzzy aggregation of MSE for time series 1.

384

Fuzzy aggregation is also performed by implementing a type-2 fuzzy system in which the results for the best experiment were follows: with a degree of uncertainty of 0.3 yielded a forecast error of 0.1780, with a degree of uncertainty of 0.4 an error of 0.1750 and with a degree of uncertainty of 0.5 an error of 0.1829, as shown in Table 6.

385

10. Comparison of results

386

A comparison of results for this time series of the Mexican Stock Exchange for the period of 11/09/05 to 01/15/09 with the paper entitled: ‘‘An Ensemble Neural Network Architecture with Fuzzy Response Integration for Complex Time Series Prediction’’ [34], shows that the best result was using an ensemble neural network architecture with 1 layer using 3 delays. The error obtained by the average integration was 0.0479, by the average weighted integration was 0.0519 and by fuzzy integration was 0.1094, respectively. In the same work the authors decided to implement an evolutionary approach to optimize the membership functions and rules of this system and the obtained error was of 0.045848. The genetic algorithm was applied to optimize the monolithic neural network for the Mexican Stock Exchange time series: 0.1092, (as shown in Fig. 9 and Table 1). In this paper the best result when applying the genetic algorithm to optimize the ensemble neural network was: 0.00112, (as shown in Fig. 11 and Table 4). This shows that our hybrid ensemble neural approach produces better results for this time series. After a comparison of results for this time series of the Mexican Stock Exchange for the period of 11/09/05 to 01/15/09 with the paper entitled: ‘‘A new Approach for Time Series Prediction Using Ensembles of ANFIS Models’’ [30], where the best result was achieved when integration by weighted average was used (an error of 0.0123 was obtained). Also, in this paper with integration by average an error of 0.0127 was obtained, which shows a very similar result. In this case, no significant difference can be found. In the literature there are only two references to forecast the time series of the Mexican Stock Exchange, and this is why the comparison was made with only these two studies. After a comparison of results for this time series of the Mexican Stock Exchange for the period of 11/09/05 to 01/15/09 and period of data from 04/08/08 to 05/09/11 with the paper entitled: ‘‘ Genetic Optimization of Ensemble Neural Networks

382 383

387 388 389 390 391 392 393 394 395 396 397 398 399 400 401 402 403

Fig. 12. Ensemble neural network architecture for the MSE series 2.

Please cite this article in press as: M. Pulido et al., Particle swarm optimization of ensemble neural networks with fuzzy aggregation for time series prediction of the Mexican Stock Exchange, Inform. Sci. (2014), http://dx.doi.org/10.1016/j.ins.2014.05.006

INS 10870

No. of Pages 17, Model 3G

19 May 2014 14

M. Pulido et al. / Information Sciences xxx (2014) xxx–xxx

Table 4 Particle swarm results for the ensemble neural network of MSE for time series 2.

Mexican Stock Exchange

3.6 3.4 3.2 3

Close

2.8 2.6 2.4 2.2 2 1.8 1.6 500

550

600

650

700

750

800

Days Fig. 13. Prediction with the optimized ensemble neural network of the MSE for time series 2.

409

for Complex Time Series Prediction of the Mexican Exchange Prediction’’ [33], where the best result was obtained for the network optimization for series 1 was: 0.0112 (as shown in Fig. 9 and Table 1) and for the series 2 where the best result was obtained for the network optimization for series 1 was: 0.0132 (as shown in Fig. 11 and Table 4). When we apply the optimization by the particle swarm method to optimize the architecture the for the neural network ensemble, it improves the prediction error for this time series of the Mexican Stock Exchange in 95%, as can be seen when comparing with the Refs. [30,33].

410

10.1. Statistical t students test

404 405 406 407 408

411 412 413

Q5

In statistics, the student t-test or t-test is a test that uses the statistical Student’s t distribution to find out if a proposed null hypothesis is true. The test is applied when the population is normally distributed, but the sample size is too small for making the inference based on a normally distributed statistic, using an estimate of the standard deviation instead of the Please cite this article in press as: M. Pulido et al., Particle swarm optimization of ensemble neural networks with fuzzy aggregation for time series prediction of the Mexican Stock Exchange, Inform. Sci. (2014), http://dx.doi.org/10.1016/j.ins.2014.05.006

INS 10870

No. of Pages 17, Model 3G

19 May 2014 15

M. Pulido et al. / Information Sciences xxx (2014) xxx–xxx Table 5 Results type-1 fuzzy aggregation of MSE for time series 2.

Table 6 Results type-2 fuzzy aggregation of MSE for time series 2.

Table 7 Results of t student test for the time series. Time series

N(type-1)

N(type-2)

t value

P value

Mexican Stock Exchange series 1 Mexican Stock Exchange series 2

30 30

90 90

5.96 4.56

0.00 0.00

Table 8 Results of MSE for time series 1. Experiment Experiment Experiment Experiment Experiment Experiment Experiment Experiment Experiment Experiment Experiment

414 415 416 417 418 419 420 421 422

1 2 3 4 5 6 7 8 9 10

Prediction error for ensemble neural network

Prediction error with type-1 fuzzy aggregation

Prediction error with type-2 fuzzy aggregation

0.405 0.5763 0.3333 1.8024 1.7938 1.7880 0.3337 0.2724 1.121 0.2916

0.1726 0.0952 0.1565 0.1554 0.3523 0.2608 0.1070 0.2497 0.1126 0.1400

0.1700 0.1750 0.2176 0.2152 0.1829 0.2129 0.2182 0.2258 0.2316 0.1319

actual value. In this section the results with the t student test are shown, and it can be noticed that there is statistical evidence of significant difference in the results between the type-1 and type-2 fuzzy systems. In other words, it can be said that for this time series prediction results are better with type-2 fuzzy aggregation than with type-1 fuzzy integration. The number of samples used for the type-1 were 10 and 30 samples for type-2, the t value obtained for type-1 in series number 1 is: 5.96 and 4.56 for series number 2, so that the results show that in our tests more than 99% confidence was obtained of significant improvement with type-2 fuzzy logic according to the achieved value of P, as shown in Table 7. Table 8 shows results of the prediction error for the Ensemble Neural Network (without fuzzy aggregation), the prediction error with type-1 fuzzy aggregation and the prediction error with type-2 fuzzy and a 0.3 level of uncertainty for the time series 1.

Please cite this article in press as: M. Pulido et al., Particle swarm optimization of ensemble neural networks with fuzzy aggregation for time series prediction of the Mexican Stock Exchange, Inform. Sci. (2014), http://dx.doi.org/10.1016/j.ins.2014.05.006

INS 10870

No. of Pages 17, Model 3G

19 May 2014 16

M. Pulido et al. / Information Sciences xxx (2014) xxx–xxx

425

In Table 8 it is clearly noted that using the response aggregation (type-1 or type-2) produces results that are significantly improved with respect to only using the results of the modules in the ensemble (first column of results). Both type-2 and type-1 fuzzy aggregation in the ensemble clearly help outperform the ensemble without fuzzy logic.

426

11. Conclusions

427

433

In this paper we considered the PSO algorithm to optimize the architecture of the ensemble neural network to predict the time series of the Mexican Stock Exchange, where good results were obtained, and we can say that this algorithm is good in speed when compared with other optimization techniques. PSO is an effective and efficient meta-heuristic to find the solution of this kind of problems. In conclusion the use of ensemble neural networks with type-2 fuzzy integration could be a good choice in predicting complex time series. Future works would be to optimize with the particle swarm algorithm the type-1 and type-2 fuzzy system aggregators to obtain an even better prediction error. type-1 and type-2 fuzzy systems could be optimized in terms of the parameters of the membership functions, membership type and number of rules.

434

12. Uncited references

423 424

428 429 430 431 432

435

Q6

[6,12,17,38,39,45].

436

Acknowledgment

437 438

We would like to express our gratitude to the CONACYT, Tijuana Institute of Technology for the facilities and resources granted for the development of this research

439

References

440 441 442 443 444 445 446 447 448 449 450 451 452 453 454 455 456 457 458 459 460 461 462 463 464 465 466 467 468 469 470 471 472 473 474 475 476 477 478 479 480 481 482 483 484 485

[1] A. Andreas, W. Sheng, Introduction optimization, Practical Optimization Algorithms and Engineering Applications, Springer, 2007. pp. 1–4. [2] P.D. Brockwell, R.A. Davis, Introduction to Time Series and Forecasting, Springer-Verlag, New York, 2002, pp. 1–219. [3] O. Castillo, P. Melin, Hybrid intelligent systems for time series prediction using neural networks, fuzzy logic, and fractal theory, IEEE Trans. Neural Netw. 13 (6) (2002) 1395–1408. [4] O. Castillo, P. Melin, Simulation and forecasting complex economic time series using neural networks and fuzzy logic, Proc. Int. Neural Netw. Conf. 3 (2001) 1805–1810. [5] O. Castillo, P. Melin, Simulation and forecasting complex financial time series using neural networks and fuzzy logic, Proc. IEEE Int. Conf. Syst. Man Cybernet. 4 (2001) 2664–2669. [6] O. Castillo, P. Melin, Type-2 Fuzzy Logic: Theory and Applications, Springer-Verlag, New York, 2008, pp. 29–40. [7] J.R. Chang, L.Y. Wei, C.H. Cheng, A hybrid ANFIS model based on AR and volatility for TAIEX forecasting 11(1) (2011) 1388–1395. [8] J.R. Chang, L.Y. Wei, C.H. Cheng, OWA-based ANFIS model for TAIEX forecasting, Econ. Model. 30 (2013) 442–448. [9] C.H. Cheng, T.L. Chen, H.J. Theo, C.H. Chiang, Fuzzy time-series based on adaptive expectation model for TAIEX forecasting, Expert Syst. Appl. 34 (2) (2008) 1126–1132. [10] C.H. Cheng, G.W. Cheng, J.W. Wang, Multi-attribute fuzzy time series method based on fuzzy clustering 34(2) (2008) 1235–124. [11] P. Cowpertwait, A. Metcalfe, Time series, Introductory Time Series R, Springer, Dordrecht, Heidelberg, London, New York, 2009. pp. 2–5. [12] P. Cunningham, J. Carney, S. Jacob, Stability problems with artificial neural networks and the ensemble solution, Artif. Intell. Med. 20 (3) (2000) 217–225. [13] H. Drucker, R. Schapire, P. Simard, Improving performance in neural networks using a boosting algorithm, in: S.J. Hanson, J.D. Cowan, C.L. Giles (Eds.), Advances in Neural Information Processing Systems 5, Denver, CO, Morgan Kaufmann, San Mateo, CA, 1993, pp. 42–49. [14] R. Eberhart, J. Kennedy, A new optimizer using particle swarm theory, in: Proc. 6th Int. Symp. Micro Machine and Human Science (MHS), October 1995, pp. 39–43. [15] R. Eberhart, Y. Shi, J. Kennedy, Swam Intelligence, Morgan Kaufmann, San Mateo, California, 2001. [16] A.P. Engelbrecht, Fundamentals of Computational Swarm Intelligence, John Wiley & Sons, 2006. [17] H.J. Escalante, M. Montes, L.E. Sucar, Particle swarm model selection, J. Mach. Learn. Res. 10 (2009) 405–440. [18] J. Hampshire, A. Waibel, A novel objective function for improved phoneme recognition using time-delay neural networks, IEEE Trans. Neural Netw. 1 (2) (1990) 216–228. [19] L.K. Hansen, P. Salomon, Neural network ensembles, IEEE Trans. Patt. Anal. Mach. Intell. 12 (10) (1990) 993–1001. [20] K. Huarng, Effective lengths of intervals to improve forecasting in fuzzy time series 123(3) (2001) 387–394. [21] J.S.R. Jang, C.T. Sun, E. Mizutani, Neuro-Fuzzy and Soft Computing, Prentice Hall, 1996. [22] N. Karnik, J.M. Mendel, Applications of type-2 fuzzy logic systems to forecasting of time-series, Inform. Sci. 120 (1–4) (1999) 89–111. [23] N. Karnik, J.M. Mendel, Introduction to type-2 fuzzy logic systems, IEEE Trans. Signal Process. 2 (1998) 915–920. [24] N. Karnik, J.M. Mendel, Operations on type-2 fuzzy sets, Fuzzy Sets Syst. 122 (2001) 327–348. [25] J. Kennedy, R. Eberhart, Particle swam optimization, Proc. IEEE Int. Conf. Neural Network (ICNN) 4 (1995) 1942–1948. [26] R. Maclin, J.W. Shavlik, Combining the predictions of multiple classifiers: using competitive learning to initialize neural networks, in: Proc. IJCAI-95, Montreal, Canada, Morgan Kaufmann, San Mateo, CA, 1995, pp. 524–530. [27] L.P. Maguire, B. Roche, T.M. McGinnity, L.J. McDaid, Predicting a chaotic time series using a fuzzy neural network, Inform. Sci. 112 (1–4) (1998) 125–136. [28] J. Mao, A case study on bagging, boosting and basic ensembles of neural networks for O CR, Proc. IJCNN-98, vol. 3, IEEE Computer Society Press, Anchorage, AK, Los Alamitos, CA, 1998, pp. 1828–1833. [29] P. Melin, O. Castillo, S. Gonzalez, J. Cota, W. Trujillo, P. Osuna, Design of Modular Neural Networks with Fuzzy Integration Applied to Time Series Prediction, vol. 41/2007, Springer, Berlin/Heidelberg, 2007. [30] P. Melin, J. Soto, O. Castillo, J. Soria, A new approach for time series prediction using ensembles of ANFIS models, Expert System with Applications, 2011. [31] Mexico Bank Database. (30.08.11). [32] I. Multaba, A. Hussain, Application of Neural Networks and Other Learning Technologies in Process Engineering, Imperial Collage Press, 2001 (03.12.08).

Q7

Please cite this article in press as: M. Pulido et al., Particle swarm optimization of ensemble neural networks with fuzzy aggregation for time series prediction of the Mexican Stock Exchange, Inform. Sci. (2014), http://dx.doi.org/10.1016/j.ins.2014.05.006

INS 10870

No. of Pages 17, Model 3G

19 May 2014 M. Pulido et al. / Information Sciences xxx (2014) xxx–xxx 486 487 488 489 490 491 492 493 494 495 496 497 498 499 500 501 502 503 504 505 506 507 508

Q8

17

[33] M. Pulido, O. Castillo, P. Melin, Genetic optimization of ensemble neural networks for complex time series prediction of the mexican exchange prediction, Int. J. Innov. Comput. Inform. Control 9(10), 4151–4166. [34] M. Pulido, A. Mancilla, P. Melin, An ensemble neural network architecture with fuzzy response integration for complex time series prediction, Evolutionary Design of Intelligent Systems in Modeling, Simulation and Control. [35] A. Sharkey, Combining Artificial Neural Nets: Ensemble and Modular Multi-Net Systems, Springer-Verlag, London, 1999. [36] Y. Shimshoni, Intrator classification of seismic signal by integrating ensemble of neural networks, IEEE Trans. Signal Process. 461 (5) (1998) 1194–1201. [37] S.R. Singh, A simple method of forecasting based on fuzzy time series, Appl. Math. Comput. 186 (1) (2007) 330–339. [38] T. Takagi, M. Sugeno, Fuzzy systems theory and its applications to modeling and control, IEEE Trans. Syst. Man Cybernet. SMC-15 (1) (1991) 116–132. [39] Y. Valle, G.K. Venayagamoorthy, S. Mohagheghi, J.-C. Hernandez, R.G. Harley, Particle swarm optimization: basic concepts, variants and applications in power systems, IEEE Trans. Evol. Comput. (2008) 171–195. [40] L.-Y. Wei, C.-H. Cheng, A hybrid recurrent neural networks model based on synthesis features to forecast the taiwan stock market, Int. J. Innov. Comput. Inform. Control 8(8), 5559–5571. [41] J. Xue, Z. Xu, J. Watada, Building an integrated hybrid model for short-term and mid-term load forecasting with genetic optimization, Int. J. Innov. Comput. Inform. Control 8 (10) (2012) 7381–7391. [42] R.N. Yadav, P.K. Kalra, J. John, Time series prediction with single multiplicative neuron model, Soft Comput. Time Ser. Predict. Appl. Soft Comput. 7 (4) (2007) 1157–1163. [43] R. Yusof, U. Khairuddin, M. Khalid, A new mutation operation for faster convergence in genetic algorithm feature selection, Int. J. Innov. Comput. Inform. Control 8 (10) (2012) 7363–7379. [44] L.A. Zadeh, in: R.R. Yager et al. (Eds.), Fuzzy Sets and Applications: Selected Papers, John Wiley, New York, 1987. [45] L. Zhao, Y. Yang, PSO-based single multiplicative neuron model for time series prediction, Expert Syst. Appl. 36 (2) (2009) 2805–2812. Part 2. [46] Z.-H. Zhou, Y. Jiang, Y.-B. Yang, S.-F. Chen, Lung cancer cell identification based on artificial neural network ensembles, Artif. Intell. Med. 24 (1) (2002) 25–36.

509

Please cite this article in press as: M. Pulido et al., Particle swarm optimization of ensemble neural networks with fuzzy aggregation for time series prediction of the Mexican Stock Exchange, Inform. Sci. (2014), http://dx.doi.org/10.1016/j.ins.2014.05.006