- Email: [email protected]
High-order fuzzy-neuro expert system for time series forecasting
Knowledge-Based Systems 46 (2013) 12–21
Contents lists available at SciVerse ScienceDirect
Knowledge-Based Systems journal homepage: www.elsevier.com/locate/knosys
High-order fuzzy-neuro expert system for time series forecasting Pritpal Singh ⇑, Bhogeswar Borah Department of Computer Science and Engineering, Tezpur University, Tezpur 784 028, Assam, India
a r t i c l e
i n f o
Article history: Received 18 July 2012 Received in revised form 6 December 2012 Accepted 25 January 2013 Available online 21 March 2013 Keywords: Fuzzy time series High-order Temperature Stock exchange Interval Fuzzy logical relation Artificial neural network
a b s t r a c t In this article, we present a new model based on hybridization of fuzzy time series theory with artificial neural network (ANN). In fuzzy time series models, lengths of intervals always affect the results of forecasting. So, for creating the effective lengths of intervals of the historical time series data set, a new ‘‘RePartitioning Discretization (RPD)’’ approach is introduced in the proposed model. Many researchers suggest that high-order fuzzy relationships improve the forecasting accuracy of the models. Therefore, in this study, we use the high-order fuzzy relationships in order to obtain more accurate forecasting results. Most of the fuzzy time series models use the current state’s fuzzified values to obtain the forecasting results. The utilization of current state’s fuzzified values (right hand side fuzzy relations) for prediction degrades the predictive skill of the fuzzy time series models, because predicted values lie within the sample. Therefore, for advance forecasting of time series, previous state’s fuzzified values (left hand side of fuzzy relations) are employed in the proposed model. To defuzzify these fuzzified time series values, an ANN based architecture is developed, and incorporated in the proposed model. The daily temperature data set of Taipei, China is used to evaluate the performance of the model. The proposed model is also validated by forecasting the stock exchange price in advance. The performance of the model is evaluated with various statistical parameters, which signify the efficiency of the model. Ó 2013 Elsevier B.V. All rights reserved.
1. Introduction Advance prediction of events like temperature, rainfall, stock price, population growth, and economy growth, are major scientific issues in the area of forecasting. Forecasting of all these events are tedious tasks because of their dynamic nature. Forecasting of all these events with 100% accuracy may not be possible, but the forecasting accuracy and the speed of forecasting process can be improved. So, in this article, we present a novel forecasting model, which is developed by hybridizing fuzzy time series theory with artificial neural network (ANN). The main aim of designing such a hybridized model is explained next. For fuzzification of time series data set, determination of lengths of intervals of the historical time series data set is very important. In most of the fuzzy time series models [1–5], the lengths of the intervals were kept the same. No specific reason is mentioned for using the fix lengths of intervals. Huarng [6] shows that the lengths of intervals always affect the results of forecasting. So, for creating the effective lengths of intervals, a new ‘‘Re-Partitioning Discretization (RPD)’’ approach is incorporated in the proposed model. After generating the intervals, time series data set is fuzzified based on the fuzzy time series theory. Most of the previous fuzzy ⇑ Corresponding author. E-mail addresses: [email protected] (P. Singh), [email protected] (B. Borah). 0950-7051/$ - see front matter Ó 2013 Elsevier B.V. All rights reserved. http://dx.doi.org/10.1016/j.knosys.2013.01.030
time series models [1,7,2–4,8] use first-order fuzzy relationships to get the forecasting results. Many researchers show that high-order fuzzy relationships improve the forecasting accuracy of the models [9–14]. Therefore, in this work, we employ the high-order fuzzy relationships for obtaining the forecasting results. Song and Chissom [1] adopted the following method to forecast enrollments of the University of Alabama:
YðtÞ ¼ Yðt 1Þ R;
ð1Þ
where Y(t 1) is the fuzzified enrollment of year (t 1), Y(t) is the forecasted enrollment of year ‘‘t’’ represented by fuzzy set, ‘‘’’ is the max–min composition operator, and ‘‘R’’ is the union of fuzzy relations. This method takes much time to compute the union of fuzzy relations R, especially when the number of fuzzy relations is more in (1) [15,16]. In 1996, Chen [3] used simplified arithmetic operations for defuzzification operation by avoiding this complicated max–min operations and their method produced better results than Song and Chissom models [1,7,2]. Most of the existing fuzzy time series models use Chen’s defuzzification method [3] in order to obtain the forecasting results. However, forecasting accuracy of these models are not good enough. Also, previous fuzzy time series models use the current state’s fuzzified values for forecasting. This approach, no doubt, improves the forecasting accuracy, but it degrades the predictive skill of the fuzzy time series models, because predicted values lie within the sample. So, for obtaining the forecasting results out of sample (i.e., in advance), we use the
P. Singh, B. Borah / Knowledge-Based Systems 46 (2013) 12–21
previous state’s fuzzified values (left hand side of fuzzy relations) in this model. To defuzzify these fuzzified values, an ANN based architecture is developed, and incorporated in this model. So, we have entitled this model as ‘‘High-order fuzzy-neuro time series forecasting model’’. The proposed model has the advantage that it can produce good forecasting results. We demonstrate the application of the proposed model using the following two real-world data set: 1. Daily average temperature data set of Taipei, China. 2. Daily stock exchange price data set of Bombay Stock Exchange (BSE), India. The rest of this paper is organized as follows. In Section 2, we present related works for fuzzy time series models. In Section 3, we review the theory of fuzzy set with an overview of fuzzy time series. In Section 4, we give an overview of ANN along with its application in the proposed model. In Section 5, description of data set is provided. Section 6 shows the application of a new approach to find the length of intervals in the universe of discourse. The architecture of the proposed model and its training phases are presented in Sections 7 and 8 respectively. The performance of the model is assessed with various statistical parameters, which are discussed in Section 9. Empirical analysis for forecasting the daily temperature is presented in Section 10. Section 11 shows the application of the proposed model for forecasting the stock exchange price. The conclusions are discussed in Section 12.
2. Related works Forecasting using fuzzy time series is applied in several areas including forecasting university enrollments, sales, road accidents and financial forecasting. In a conventional time series models, the recorded values of a special dynamic process are represented by crisp numerical values. But, in a fuzzy time series model, the recorded values of a special dynamic process are represented by linguistic values. Based on fuzzy time series theory, first forecasting model was introduced by Song and Chissom [1,7,2]. They presented the fuzzy time series model by fuzzy relational equations involving max–min composition operation and applied the model to forecast the enrollments in the University of Alabama. In 1996, Chen [3] used simplified arithmetic operations avoiding the complicated max–min operations and their method produced better results. Later, many studies provided some improvements to the fuzzy time series methods in determining the lengths of intervals, fuzzification process and defuzzification techniques. Hwang et al. [4] used the differences of the available historical data as fuzzy time series rather than direct usage of raw numeric values. Sah and Degtiarev also used a similar approach in [17]. Huarng tried to improve the forecasting accuracy based on determination of the length of intervals [6] and heuristic approaches [5]. Lee and Chou [18] forecasted the university enrollments with the average error rate less than Chen’s method [3] by defining the supports of the fuzzy numbers that represent the linguistic values of the linguistic variables more appropriately. Yu [19] proposed weighted fuzzy time series model to resolve issues of recurrence and weighting in fuzzy time series forecasting. Cheng et al. [8] used entropy minimization to create the intervals. They also used trapezoidal membership functions in the fuzzification process. Chang et al. [20] presented cardinality-based fuzzy time series forecasting model, which builds weighted fuzzy rules by calculating the cardinality of fuzzy relations. To enhance the performance of fuzzy time series models, Chen et al. [21] incorporates the concept of the Fibonacci sequence in the existing models as proposed by Song and Chissom [1,2] and Yu [19]. To obtain less
13
number of intervals, Cheng et al. [22] proposed a model using fuzzy clustering technique to partition the data effectively. The K-means clustering algorithm has been applied to partition the universe of discourse in [23]. Chou et al. [24] forecasted the tourism demand based on hybridization of rough set with fuzzy time series. Singh and Borah [25] forecasted the university enrollments with the help of new proposed algorithm by dividing the universe of discourse of the historical time series data into different length of intervals. Recent advancement in fuzzy time series forecasting models can be found in [26–29]. Recently, many researchers have proposed various hybridization based models to solve complex problems in forecasting. For example, Hadavandi et al. [30] presented a new approach based on genetic fuzzy systems and ANNs for building a stock price forecasting expert system to improve the forecasting accuracy. Cheng et al. [31] proposed a new stock price forecasting model based on hybridization of genetic algorithm with rough set theory. Kuo et al. [32] hybridized the particle swarm optimization with fuzzy time series to adjust the lengths of intervals in the universe of discourse. Aladag et al. [33] introduced a new approach to define fuzzy relation in high order fuzzy time series using feed forward neural networks. Teoh et al. [34] proposed a fuzzy-rough hybrid forecasting model, where rules (fuzzy logical relationships) are generated by rough set algorithm. Pal and Mitra [35] proposed a rough-fuzzy hybridization scheme for case generation. They used the fuzzy set theory for linguistic representation of patterns and then obtained the dependency rules by using the rough set theory. For advance prediction of dwelling fire occurrence in Derbyshire (United Kingdom), Yang et al. [36] employed three approaches: logistic regression, ANN and Genetic Algorithm. Keles et al. [37] proposed a model for forecasting the domestic dept by Adaptive Neuro-Fuzzy Inference System. Chang et al. [38] developed a hybrid model by integrating K-mean cluster and fuzzy neural network to forecast the future sales of a printed circuit board factory. Huarng and Yu [39], and Yu and Huarng [40] presented a new hybrid model based on neural network and fuzzy time series to forecast TAIEX. Kuo et al. [41] and Huang et al. [42] introduced a new enrollments forecasting model based on hybridization of fuzzy time series and particle swarm optimization. 3. Fuzzy sets and fuzzy time series In 1965, Zadeh [43] introduced fuzzy sets theory involving continuous set membership for processing data in presence of uncertainty. He also presented fuzzy arithmetic theory and its application [44–46]. In this section, we will briefly review fuzzy sets theory from [43] and fuzzy time series concepts from [1,7,2]. Definition 3.1 (Fuzzy Set [43]). A fuzzy set is a class with varying degrees of membership in the set. Let U be the universe of discourse, which is discrete and finite, then fuzzy set A can be defined as follows:
A ¼ flA ðx1 Þ=x1 þ lA ðx2 Þ=x2 þ g ¼ Ri lA ðxi Þ=xi ;
ð2Þ
where lA is the membership function of A, lA: U ? [0, 1], and lA(xi) is the degree of membership of the element xi in the fuzzy set A. Here, the symbol ‘‘+’’ indicates the operation of union and the symbol ‘‘/’’ indicates the separator rather than the commonly used summation and division in algebra respectively. When U is continuous and infinite, then the fuzzy set A of U can be defined as:
A¼
Z
lA ðxi Þ=xi ; 8xi 2 U;
ð3Þ
14
P. Singh, B. Borah / Knowledge-Based Systems 46 (2013) 12–21
R
where the integral sign ‘‘ ’’ stands for the union of the fuzzy singletons, lA(xi)/xi. Fuzzy time series concept was proposed in [1,7,2] and the main difference between the traditional time series and the fuzzy time series is that the values of the former are crisp numerical values, while the values of the latter are fuzzy sets. The crisp numerical values can be represented by real numbers whereas in fuzzy sets, the values of observations are represented by linguistic values. The definition of fuzzy time series is briefly reviewed as follows: Definition 3.2 (Fuzzy Time Series [1,7,2]). Let Y(t) (t = 0, 1, 2, . . .) be a subset of R and the universe of discourse on which fuzzy sets li(t) (i = 1, 2, . . .) are defined and let F(t) be a collection of li(t) (i = 1, 2, . . .). Then, F(t) is called a fuzzy time series on Y(t) (t = 0, 1, 2, . . .). From Definition 3.2, we can see that F(t) is a function of time t, and li(t) are linguistic values of F(t), where li(t) (i = 1, 2, . . .) are represented by fuzzy sets, and the values of F(t) can be different at different times because the universe of discourse can be different at different times. Fuzzy time series can be divided into two categories which are the time-invariant fuzzy time series and the time-variant fuzzy time series. If F(t) is caused by F(t 1), i.e., F(t 1) ? F(t), then this relationship can be represented as follows:
FðtÞ ¼ Fðt 1Þ Rðt; t 1Þ;
ð4Þ
where R(t, t 1) is fuzzy relationship between F(t) and F(t 1). Here, R is the union of fuzzy relations and ‘‘’’ is max–min composition operator. It is also called the first-order model of F(t). Definition 3.3 (Fuzzy time-variant and time-invariant series [15]). Let F(t) be a fuzzy time series, and R(t, t 1) be a first-order model of F(t). If R(t, t 1) = R(t 1,t 2) for any time t, and F(t) only has finite elements, then F(t) is referred as a time-invariant fuzzy time series. Otherwise, it is referred as a time-variant fuzzy time series. Definition 3.4 (Fuzzy logical relationship [1–3]). Assume that F(t 1) = Ai and F(t) = Aj. The relationship between F(t) and F(t 1) is referred as a fuzzy logical relationship (FLR), which can be represented as:
Ai ! Aj ;
ð5Þ
where Ai and Aj refer to the previous state and current state of the FLR respectively. Definition 3.5 (Fuzzy logical relationship group [1–3]). Assume the following FLRs:
Ai ! Ak1 ; Ai ! Ak2 ; Ai ! Akm : Chen [3] suggested that the FLRs having the same previous state are grouped into a same fuzzy logical relationship group (FLRG). So, based on Chen’s model [3], these FLRs can be grouped into the same FLRG as:
4. ANN and its application ANNs are massively parallel adaptive networks of simple nonlinear computing elements called neurons which are intended to abstract and model some of the functionality of the human nervous system in an attempt to partially capture some of its computational strengths [47]. The neurons in an ANN are organized into different layers. Inputs to the network are existed in the input layer; whereas outputs are produced as signals in the output layer. These signals may pass through one or more intermediate or hidden layers which transform the signals depending upon the neuron signal functions. A simple neural network architecture as proposed by Lippmann [48] is shown in Fig. 1. In Fig. 1, Z1, Z2, . . . , Zn are the set of input neurons, which transmitting information or signals to another output neuron, say Y. Each input neuron Z1, Z2, . . . , Zn has an interconnection links with another neuron. Each interconnection link of input neurons are associated with some weights as W1, W2, . . . , Wn. For this neural network architecture, the net input can be calculated as:
Y input ¼ z1 w1 þ z2 w2 þ þ zn wn ¼
n X zi wi ;
where z1, z2, . . . , zn are the activations or output of input neurons Z1, Z2, . . . , Zn, and w1, w2, . . . , wn are the weights associated with z1, z2, . . . , zn. For output neuron Y, output y can be determined by applying activation function on the net input Yinput as:
y ¼ f ðY input Þ:
ð8Þ
The back-propagation neural network (BPNN) is one of the significant developments in the area of ANN [49,50]. The BPNN can consist of multi-layer feed-forward neural network with one input layer, limited number of hidden layers and one output layer. In Fig. 2, an architecture of the BPNN is shown, which consists of only one hidden layer. The main objective of using the BPNN with multi-layer feed-forward neural network is to minimize the output error obtained from the difference between the calculated output (o1 ,o2, . . . , on) and target output (n1, n2, . . . , nn) of the neural network by adjusting the weights. So, in the BPNN, each information is sent back again in the reverse direction until the output error is very small or zero. The BPNN is trained under the process of three phases as: 1. Using the feed-forward neural network for training process of input information. Adjustment of weights and nodes are made in this phase. 2. To calculate the error. 3. Update the weights. Due to a large number of additional parameters [51], e.g., initial weight, learning rate, momentum, epoch, activation function, etc., the ANN model has great capability to learn by making proper adjustment of these parameters, to produce the desired output. In this study, we use the BPNN algorithm to defuzzify the fuzzified
Ai ! Ak1 ; Ak2 ; . . . ; Akm : Definition 3.6 (High-order FLR [14]). Assume that F(t) is caused by F(t 1), F(t 2), . . . , F(t n) (n > 0), then high-order FLR can be expressed as:
Fðt nÞ; . . . ; Fðt 2Þ;
Fðt 1Þ ! FðtÞ:
ð7Þ
i¼1
ð6Þ Fig. 1. A simple neural network architecture.
15
P. Singh, B. Borah / Knowledge-Based Systems 46 (2013) 12–21
Table 1 Historical data of the daily average temperature from June 1996 to September 1996 in Taipei (Unit: °C). Day
Fig. 2. A multi-layer BPNN architecture.
time series data set. Paradigms adopted for building the basic architecture for the proposed neural network is explained next. Designing the right neural network architecture is a heuristic based approach and also a very time consuming process. The performance of the neural network architecture depends on number of layers, number of nodes in each layer and number of interconnection links with the nodes [52]. Since, a neural network with more than three layers generate arbitrarily complex decision regions. Therefore, a single hidden layer with one input layer and one output layer is considered here in designing the architecture. The number of nodes in input layer will depend on order of FLRs. For example, for third-order FLR, there would be three nodes in input layer; for fourth-order FLR, there would be four nodes in input layer, and so on. The minimum number of nodes in hidden layer is determined by the following equation:
Hiddennodes ¼ Inputnodes 1;
ð9Þ
where Hiddennodes and Inputnodes represent the number of nodes in hidden and input layers respectively. A neural network architecture for the fifth-order FLRs is shown in Fig. 3. The neural network as shown in Fig. 3 have five nodes (Ii, i = 1, 2, . . . , 5) in input layer. The arrangement of nodes in input layer is done in the following sequence:
Yðt 5Þ; Yðt 4Þ; Yðt 3Þ; Yðt 2Þ; YðtÞ ! Yðt þ 1Þ:
ð10Þ
Here, each input node take the previous days (t 5, t 4, . . . , t) fuzzified time series values (e.g., A5, A4, . . . , A1) to predict 1 day (t + 1) advance daily temperature value ‘‘At+1’’. In Eq. (10), each ‘‘t’’ represent the day for considered fuzzified time series values.
Month
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31
June
July
August
September
26.1 27.6 29.0 30.5 30.0 29.5 29.7 29.4 28.8 29.4 29.3 28.5 28.7 27.5 29.5 28.8 29.0 30.3 30.2 30.9 30.8 28.7 27.8 27.4 27.7 27.1 28.4 27.8 29.0 30.2
29.9 28.4 29.2 29.4 29.9 29.6 30.1 29.3 28.1 28.9 28.4 29.6 27.8 29.1 27.7 28.1 28.7 29.9 30.8 31.6 31.4 31.3 31.3 31.3 28.9 28.0 28.6 28.0 29.3 27.9 26.9
27.1 28.9 28.9 29.3 28.8 28.7 29.0 28.2 27.0 28.3 28.9 28.1 29.9 27.6 26.8 27.6 27.9 29.0 29.2 29.8 29.6 29.3 28.0 28.3 28.6 28.7 29.0 27.7 26.2 26.0 27.7
27.5 26.8 26.4 27.5 26.6 28.2 29.2 29.0 30.3 29.9 29.9 30.5 30.2 30.3 29.5 28.3 28.6 28.1 28.4 28.3 26.4 25.7 25.0 27.0 25.8 26.4 25.6 24.2 23.3 23.5
6. Re-Partitioning Discretization (RPD) approach In this section, we propose a new discretization approach referred to as ’’RPD’’ for determining the universe of discourse of the historical time series data set and partitioning it into different lengths of intervals. To explain this approach, the daily average temperature data set from June 1, 1996 to June 30, 1996, shown in Table 1, is employed. Each step of the approach is explained next. Step 6.1. Compute range (R) of a sample, S = {x1, x2, . . . , xn} as:
5. Description of data set For verifying the model, the daily average temperature data set [15] from June 1996 to September 1996 in Taipei is employed. This data set is shown in Table 1. Taipei, which is the capital of the Republic of China, is situated at the northern tip of the island of China. It is the political, economic, and cultural center of China. So, advance prediction of daily temperature of Taipei is very advantageous for the inhabitant of Taipei.
R ¼ Maxv alue Minv alue ;
ð11Þ
where Maxvalue and Minvalue are the maximum and minimum values of S respectively. From Table 1, Maxvalue and Minvalue for the June temperature data set (S) are 30.9 and 26.1 respectively. Therefore, the range R for this data set is computed as:
R ¼ 30:9 26:1 ¼ 4:8: Step 6.2. Split the data range R into M equally spaced classes, where M can be defined as [53]:
n M ¼ 1 þ log2 ;
ð12Þ
where n is the size of the sample S. Based on Eq. (12), we can compute M as:
M ¼1þ
1:477 ¼ 5:907; where sample size n ¼ 30: 0:3010
Step 6.3. Obtain width of an interval (H) as:
H¼ Fig. 3. A BPNN architecture for the fifth-order FLRs.
R : M
Based on Eq. (13), we can calculate the width as:
ð13Þ
16
P. Singh, B. Borah / Knowledge-Based Systems 46 (2013) 12–21
H¼
v i ¼ ½MðiÞ; VðiÞ; i ¼ 1; 2; 3; . . . ; < Bmax ; v i 2 U B ;
4:8 ¼ 0:8126: 5:907
1 6 VðiÞ ð27Þ
Step 6.4. Define the universe of discourse U of the sample S as:
U ¼ ½Lb ; U b ;
ð14Þ
where Lb = Minvalue H, and Ub = Maxvalue + H. Based on Table 1, we have the universe of discourse of the sample S as:
where M(i) = Bmin + (i 1) DFB, and V(i) = Bmin + i DFB. Based on Eq. (26), intervals for the sub-boundary UA are:
u1 ¼ ½26:1; 26:34; u2 ¼ ½27:06; 27:30; . . . ; u6 ¼ ½28:26; 28:50:
U ¼ ½26:1 0:8126; 30:9 þ 0:8126 ¼ ½25:287; 31:713: Step 6.5. Compute mid-point (Umid) of the universe of discourse U as:
U mid ¼
Lb þ U b : 2
ð15Þ
The Umid of the sample S is obtained as:
U mid ¼
25:287 þ 31:713 ¼ 28:5: 2
Step 6.6. Find sub-sets of the sample S such that:
A ¼ fx 2 Sjx 6 U mid g; B ¼ fx 2 Sjx P U mid g:
ð16Þ ð17Þ
From Table 1, we have obtained the elements of A and B as:
Similarly, based on Eq. (27), intervals for the sub-boundary UB are:
v 1 ¼ ½28:70; 28:81; v 2 ¼ ½28:92; 29:03; v 11 ¼ ½30:79; 30:90: Step 6.10. Allocate the elements to their corresponding intervals. Assign the elements of A and B to their corresponding intervals obtained after partitioning the boundaries UA and UB respectively. All these intervals along with their corresponding elements are shown in Table 2. Last column of Table 2 represents mid-points of the intervals. Intervals which do not cover historical data are discarded from the list. Intervals for the remaining three months July, August and September as shown in Table 1 are obtained in a similar way.
A ¼ f26:1; 27:1; 27:4; 27:5; 27:6; 27:7; 27:8; 27:8; 28:4; 28:5g; 7. Architecture of the model
B ¼ f28:7; 28:7; 28:8; 28:8; 29; 29; 29; 29:3; 29:4; 29:4; 29:5; 29:5; 29:7; 30; 30:2; 30:2; 30:3; 30:5; 30:8; 30:9g: Step 6.7. Define sub-boundaries for A and B as:
U A ¼ ½Amin ; Amax ;
ð18Þ
U B ¼ ½Bmin ; Bmax ;
ð19Þ
where UA and UB are the sub-boundaries for A and B respectively. Here, Amin and Amax represent the minimum and maximum values of the sub-set A respectively. Similarly, Bmin and Bmax represent the minimum and maximum values of the sub-set B respectively. From Eqs. (18) and (19), we can define the sub-boundaries for A and B as:
U A ¼ ½26:1; 28:5;
ð20Þ
U B ¼ ½28:7; 30:9:
ð21Þ
Step 6.8. Determine deciding factors for A and B as:
Amax Amin ; NA Bmax Bmin ; DF B ¼ NB
DF A ¼
ð22Þ
28:5 26:1 ¼ 0:24; 10 30:9 28:7 ¼ 0:11: DF B ¼ 20
DF A ¼
ð24Þ ð25Þ
Step 6.9. Partition the sub-boundaries UA and UB into different length of intervals as:
i ¼ 1; 2; 3; . . . ;
Step Step Step Step Step Step
7.1. 7.2. 7.3. 7.4. 7.5. 7.6.
Partition the universe of discourse into intervals. Define linguistic terms for each of the interval. Fuzzify the time series data set. Establish the FLRs based on Definition 3.4. Construct the FLRGs based on Definition 3.5. Defuzzify and compute the forecasted values.
In this article, an improved fuzzy time series forecasting model is proposed, which is based on the hybridization of fuzzy time series theory with ANN. This model also employs the high-order FLRs to obtain the forecasting results. Therefore, above steps are modified, which is represented by the data-flow diagram (see Fig. 4).
Table 2 Intervals and their corresponding elements for the June daily temperature data set.
ð23Þ
where DFA and DFB are the deciding factors for A and B respectively. Here, NA and NB represent the total number of elements of A and B respectively. From Eqs. (22) and (23), the deciding factors for A and B are:
ui ¼ ½LðiÞ; UðiÞ;
Most of the existing fuzzy time series models as discussed earlier use the following six common steps to deal with the forecasting problems:
1 6 UðiÞ < Amax ; ui
2 UA ; where L(i) = Amin + (i 1) DFA, and U(i) = Amin + i DFA.
ð26Þ
Corresponding element
Mid-point
Interval for UA [26.1, 26.34] [27.06, 27.30] [27.30, 27.54] [27.54, 27.78] [27.78, 28.02] [28.26, 28.50]
(26.1) (27.1) (27.4, 27.5) (27.6, 27.7) (27.8, 27.8) (28.4, 28.5)
26.22 27.18 27.42 27.66 27.9 28.38
Interval for UB [28.70, 28.81] [28.92, 29.03] [29.25, 29.36] [29.36, 29.47] [29.47, 29.58] [29.69, 29.80] [29.91, 30.02] [30.13, 30.24] [30.24, 30.35] [30.46, 30.57] [30.79, 30.90]
(28.7, 28.7, 28.8, 28.8) (29, 29,2 9) (29.3) (29.4, 29.4) (29.5, 29.5) (29.7) (30) (30.2, 30.2) (30.3) (30.5) (30.8, 30.9)
28.755 28.975 29.305 29.415 29.525 29.745 29.965 30.185 30.295 30.515 30.845
17
P. Singh, B. Borah / Knowledge-Based Systems 46 (2013) 12–21
8. High-order fuzzy-neuro time series forecasting model We apply the proposed model to forecast the daily temperature of Taipei from June, 1996 to September, 1996. This model is trained with the June daily temperature data set. Each phase of the training process is explained next. Phase 8.1. Divide the universe of discourse into different lengths of intervals. Define the universe of discourse U for the June temperature data set. Based on Eq. (14), we have defined the universe of discourse U as U = [25.287, 31.713]. Then, based on the RPD approach, the universe of discourse U is partitioned into n different lengths of intervals: a1, a2, a3, . . . , an. The experimental results are presented in Table 2. Phase 8.2. Define linguistic terms for each of the interval. Assume that the historical time series data set is distributed among n intervals (i.e., a1, a2, . . . , an). Therefore, define n linguistic variables A1, A2, . . . , An, which can be represented by fuzzy sets, as shown below: A1 ¼ 1=a1 þ 0:5=a2 þ 0=a3 þ 0=a4 þ 0=a5 þ þ 0=an2 þ 0=an1 þ 0=an ; A2 ¼ 0:5=a1 þ 1=a2 þ 0:5=a3 þ 0=a4 þ 0=a5 þ þ 0=an2 þ 0=an1 þ 0=an ; A3 ¼ 0=a1 þ 0:5=a2 þ 1=a3 þ 0:5=a4 þ 0=a5 þ þ 0=an2 þ 0=an1 þ 0=an ; .. . Aj ¼ 0=a1 þ 0=a2 þ 0=a3 þ 0=a4 þ 0=a5 þ þ 0=an2 þ 0:5=an1 þ 1=an :
Obtain the degree of membership of each day’s temperature value belonging to each Ai. Here, maximum degree of membership of fuzzy set Ai occurs at interval ai, and 1 6 i 6 n. Phase 8.3. Fuzzify the historical time series data. If one day’s datum belongs to the interval ai, then it is fuzzified into Ai, where 1 6 i 6 n. If one day’s temperature value belongs to the interval ai, then the fuzzified temperature value for that day is considered as Ai. For example, the temperature value of June 1, 1996 belongs to the interval a1, so it is fuzzified to A1. In this way, we have fuzzified historical time series data set. The fuzzified temperature values are shown in Table 3. Phase 8.4. Establish the high-order FLRs between the fuzzified daily temperature values. Based on Definition 3.6, we have established the fifth-order FLRs between the fuzzified daily temperature values. For example, in Table 3, the fuzzified daily temperature values for days 1, 2, 3, 4, 5 and 6 are A1, A4, A8, A16, A13 and A11, respectively. Here, to establish the fifth-order FLR among these fuzzified values, it is considered that A11 is caused by the previous five fuzzified values A1, A4, A8, A16 and A13. Hence, the fifth-order FLR is represented in the following form:
A1 ; A4 ; A8 ; A16 ; A13 ! A11 :
ð28Þ
Here, left hand side of the FLR is called the previous state, whereas right hand side of the FLR is called the current state. Previously, most of the fuzzy time series models [9–13] use the current state’s fuzzified value for defuzzification. The main downside of using such fuzzified value for defuzzification is that prediction scope of these models [9–13] lie within the sample. For most of the real and complex problems, out of sample prediction (i.e., advance prediction) is very much essential. Therefore, in this model, the previous state’s fuzzified values are used to obtain the forecasting results. Table 3 Fuzzified historical data set for the June daily temperature.
Fig. 4. High-order fuzzy-neuro time series forecasting model.
Day
June
Fuzzified temperature
Mid-point
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30
26.1 27.6 29 30.5 30 29.5 29.7 29.4 28.8 29.4 29.3 28.5 28.7 27.5 29.5 28.8 29 30.3 30.2 30.9 30.8 28.7 27.8 27.4 27.7 27.1 28.4 27.8 29 30.2
A1 A4 A8 A16 A13 A11 A12 A10 A7 A10 A9 A6 A7 A3 A11 A7 A8 A15 A14 A17 A17 A7 A5 A3 A4 A2 A6 A5 A8 A14
26.22 27.66 28.975 30.515 29.965 29.525 29.745 29.415 28.755 29.415 29.305 28.38 28.755 27.42 29.525 28.755 28.975 30.295 30.185 30.845 30.845 28.755 27.9 27.42 27.66 27.18 28.38 27.9 28.975 30.185
18
P. Singh, B. Borah / Knowledge-Based Systems 46 (2013) 12–21
Table 4 Fifth-order FLRs for the June fuzzified daily temperature data set. Fifth-order FLR A1, A4, A8, A16, A13 ? ?h6i A4, A8, A16, A13, A11 ? ?h7i A8, A16, A13, A11, A12 ? ?h8i A16, A13, A11, A12, A10 ? ?h9i A13, A11, A12, A10, A7 ? ?h10i A11, A12, A10, A7, A10 ? ?h11i A12, A10, A7, A10, A9 ? ?h12i A10, A7, A10, A9, A6 ? ?h13i A7, A10, A9, A6, A7 ? ?h14i A10, A9, A6, A7, A3 ? ?h15i A9, A6, A7, A3, A11 ? ?h16i A6, A7, A3, A11, A7 ? ?h17i A7, A3, A11, A7, A8 ? ?h18i A3, A11, A7, A8, A15 ? ?h19i A11, A7, A8, A15, A14 ? ?h20i A7, A8, A15, A14, A17 ? ?h21i A8, A15, A14, A17, A17 ? ?h22i A15, A14, A17, A17, A7 ? ?h23i A14, A17, A17, A7, A5 ? ?h24i A17, A17, A7, A5, A3 ? ?h25i A17, A7, A5, A3, A4 ? ?h26i A7, A5, A3, A4, A2 ? ?h27i A5, A3, A4, A2, A6 ? ?h28i A3, A4, A2, A6, A5 ? ?h29i A4, A2, A6, A5, A8 ? ?h30i
FLRs, so to explain the defuzzification operation, we use the nth-order FLRs, where n P 5. The steps involve in the defuzzification operation are explained next. Step 8.5.1. For forecasting day Y(t), obtain the nth-order FLR, which can be represented in the following form:
Atn ; Atðn1Þ ; . . . ; At1 ! ?hti;
ð29Þ
where ‘‘t’’ represent a day which we want to forecast, and ‘‘n’’ is the order of FLR (n P 5). Here, Atn, At(n1), . . . , At1 are the previous state’s fuzzified values from days, Y(t n), . . . , Y(t 2) to Y(t 1). Step 8.5.2. Find the intervals where the maximum membership values of the fuzzy sets Atn, At(n1), . . . , At1 occur, and let these intervals be an, an1, . . . , a1, respectively. All these intervals have the corresponding mid-points Cn, Cn1, . . . , C1. Step 8.5.3. Replace each previous state’s fuzzified value of (29) with their corresponding mid-point as:
C n ; C n1 ; . . . ; C 1 ! ?hti; n P 5: Step 8.5.4.
The fifth-order FLRs obtained for the fuzzified daily temperature data are presented in Table 4. In this table, each symbol ‘‘?’’ represents the desired output for corresponding day ‘‘t’’ in the symbol ‘‘hi’’, which would be determined by the proposed model. Phase 8.5. Defuzzify the fuzzified time series data set. In this model, we use the BPNN algorithm to defuzzify the fuzzified time series data set. The neural network architecture which is used here for defuzzification operation is presented in Section 4. The proposed model is based on the high-order
Table 5 Advance prediction of the daily temperature for June. Day
Actual temperature
Predicted temperature
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30
26.1 27.6 29 30.5 30 29.5 29.7 29.4 28.8 29.4 29.3 28.5 28.7 27.5 29.5 28.8 29 30.3 30.2 30.9 30.8 28.7 27.8 27.4 27.7 27.1 28.4 27.8 29 30.2
– – – – – 28.6 29.14 29.54 29.7 29.42 29.31 29.27 29.02 28.92 28.55 28.53 28.46 28.53 28.75 29.33 29.75 30.14 30.05 29.39 29.1 28.47 27.77 27.61 27.55 27.81
ð30Þ
Use the mid-points of (30) as inputs in the proposed BPNN architecture to compute the desired output ‘‘?’’ for the corresponding day ‘‘t’’.
The scaling of mid-points are done before beginning the defuzzification operation using min–max normalization [54]. For example, array of mid-points ‘‘Xi’’ are normalized based on the minimum and maximum values of ‘‘Xi’’. A mid-point ‘‘v’’ of ‘‘Xi’’ is ’’ by computing: normalized to ’’v
v ¼
v minA ðnewmaxA newminA Þ þ newminA ; maxA minA
ð31Þ
where minA and maxA are the minimum and maximum values of array ‘‘Xi’’ respectively. Min–max normalization maps a value ‘‘v’’ to ’’ in the range ½newmaxA ; newminA , where newmaxA represents ‘‘1’’ ‘‘v and newminA represents ‘‘0’’. A sample of the results obtained for the June temperature data set are presented in Table 5. For rest of the months, similar approach is adopted for obtaining the results.
9. Performance analysis parameters The performance of the proposed model is evaluated with the help of means and standard deviations (SDs) of the observed and predicted values, root mean square error (RMSE) and Theil’s U Statistic. All these parameters are defined as follows: 1. The mean can be defined as:
Pn A¼
i¼1 Ai
n
:
ð32Þ
2. The SD can be defined as:
rffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffi 1 Xn SD ¼ ðAi AÞ2 : i¼1 n
ð33Þ
3. The RMSE can be defined as:
sffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffi Pn 2 i¼1 ðF i Ai Þ RMSE ¼ : n
ð34Þ
4. The formula used to calculate Theil’s U statistic [55] is:
qffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffi Pn 2 i¼1 ðAi F i Þ ffi ffiffiffiffiffiffiffiffiffiffiffiffiffiffiffi q U ¼ qffiffiffiffiffiffiffiffiffiffiffiffiffiffiffi Pn 2 Pn 2ffi : i¼1 Ai þ i¼1 F i
ð35Þ
19
P. Singh, B. Borah / Knowledge-Based Systems 46 (2013) 12–21 Table 6 Advance prediction of the daily temperature from June (1996) to September (1996) in Taipei (Unit: °C). Day
Actual June
Predicted June
Actual July
Predicted July
Actual August
Predicted August
Actual September
Predicted September
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31
26.1 27.6 29 30.5 30 29.5 29.7 29.4 28.8 29.4 29.3 28.5 28.7 27.5 29.5 28.8 29 30.3 30.2 30.9 30.8 28.7 27.8 27.4 27.7 27.1 28.4 27.8 29 30.2 –
– – – – – 28.6 29.14 29.54 29.7 29.42 29.31 29.27 29.02 28.92 28.55 28.53 28.46 28.53 28.75 29.33 29.75 30.14 30.05 29.39 29.1 28.47 27.77 27.61 27.55 27.81 –
29.9 28.4 29.2 29.4 29.9 29.6 30.1 29.3 28.1 28.9 28.4 29.6 27.8 29.1 27.7 28.1 28.7 29.9 30.8 31.6 31.4 31.3 31.3 31.3 28.9 28 28.6 28 29.3 27.9 26.9
– – – – – 29.25 29.19 29.59 29.45 29.26 28.95 28.77 28.76 28.45 28.67 28.36 28.36 28.18 28.54 28.96 29.72 30.37 30.99 31.29 31.3 30.84 29.92 29.46 28.86 28.49 28.37
27.1 28.9 28.9 29.3 28.8 28.7 29 28.2 27 28.3 28.9 28.1 29.9 27.6 26.8 27.6 27.9 29 29.2 29.8 29.6 29.3 28 28.3 28.6 28.7 29 27.7 26.2 26 27.7
– – – – – 28.37 28.82 28.69 28.43 28.19 28.14 28.18 27.97 28.17 28.31 27.97 27.85 27.82 27.77 27.8 28.42 28.99 28.91 29.01 28.63 28.74 28.39 28.33 28.23 27.99 27.32
27.5 26.8 26.4 27.5 26.6 28.2 29.2 29 30.3 29.9 29.9 30.5 30.2 30.3 29.5 28.3 28.6 28.1 28.4 28.3 26.4 25.7 25 27 25.8 26.4 25.6 24.2 23.3 23.5 –
– – – – – 26.81 26.97 27.5 27.69 27.86 29.04 29.59 29.93 29.94 29.86 29.8 29.72 29.33 28.64 28.43 28 27.78 26.87 26.66 26 25.96 25.96 25.81 25 25.05 –
Table 7 Performance analysis of the model for the fifth-order FLRs.
Table 10 Performance analysis of the model for the eighth-order FLRs.
Statistics
June
July
August
September
Statistics
June
July
August
September
Mean observed (°C) Mean predicted (°C) SD observed (°C) SD predicted (°C) RMSE (°C) U
28.98 28.91 1.05 0.72 1.23 0.0213
29.25 29.32 1.35 0.94 1.33 0.0225
28.27 28.29 1.04 0.38 1.05 0.0189
27.66 27.77 2.23 1.61 1.35 0.0189
Mean observed (°C) Mean predicted (°C) SD observed (°C) SD predicted (°C) RMSE (°C) U
28.90 29.04 1.10 0.42 1.25 0.0216
29.20 29.27 1.43 0.72 1.47 0.0257
28.23 28.44 1.07 0.33 1.03 0.0185
27.51 28.01 2.33 1.44 1.57 0.0282
Table 8 Performance analysis of the model for the sixth-order FLRs. Statistics
June
July
August
September
Mean observed (°C) Mean predicted (°C) SD observed (°C) SD predicted (°C) RMSE (°C) U
28.95 29.04 1.07 0.65 1.27 0.0219
29.24 29.38 1.38 0.83 1.36 0.0235
28.26 28.48 1.04 0.39 1.03 0.0185
27.64 27.94 2.27 1.61 1.43 0.0256
Table 9 Performance analysis of the model for the seventh-order FLRs. Statistics
June
July
August
September
Mean observed (°C) Mean predicted (°C) SD observed (°C) SD predicted (°C) RMSE (°C) U
28.92 29.05 1.08 0.53 1.22 0.0210
29.20 29.34 1.40 0.74 1.37 0.0239
28.23 28.43 1.05 0.35 1.02 0.0180
27.57 27.97 2.30 1.57 1.39 0.0249
Table 11 Additional parameters and their values during the training and testing processes of neural network. S. no.
Additional parameter
Input value
1 2 3 4 5
Initial weight Learning rate Epochs Learning radius Activation function
0.3 0.5 10,000 3 Sigmoid
Here, each Fi and Ai is the forecasted and actual value of day i respectively, n is the total number of days to be forecasted. In Eqs. (32) and (33), {A1, A2, . . . , An} are the observed values of the actual time series data set and A is the mean value of these observations. Similarly, mean and SD for predicted time series data set are computed. For a good forecasting, the observed means and SDs should be close to predicted means and SDs. In Eq. (34), a small RMSE value indicates good forecasting. In Eq. (35), U is bound between 0 and 1, with values closer to 0 indicating good forecasting accuracy.
20
P. Singh, B. Borah / Knowledge-Based Systems 46 (2013) 12–21
Table 12 Advance prediction of the BSE price from 7/30/2012 to 9/11/2012 (In Rupee). Date (mm-dd-yy)
Actual price
7/30/2012 7/31/2012 8/1/2012 8/2/2012 8/3/2012 8/6/2012 8/7/2012 8/8/2012 8/9/2012 8/10/2012 8/13/2012 8/14/2012 8/16/2012 8/17/2012 8/21/2012 8/22/2012 8/23/2012 8/24/2012 8/27/2012 8/28/2012 8/29/2012 8/30/2012 8/31/2012 9/3/2012 9/4/2012 9/5/2012 9/6/2012 9/7/2012 9/10/2012 9/11/2012
17143.68 17236.18 17257.38 17224.36 17197.93 17412.96 17601.78 17600.56 17560.87 17557.74 17633.45 17728.2 17657.21 17691.08 17885.26 17846.86 17850.22 17783.21 17678.81 17631.71 17490.81 17541.64 17380.75 17384.4 17440.87 17313.34 17346.27 17683.73 17766.78 17852.95
– – – – – 17177.27 17259.28 17299.01 17387.01 17469.96 17547.98 17589.91 17537.01 17592.65 17617.07 17686.52 17709.69 17760.77 17754.85 17774.83 17725.83 17669.82 17561.74 17488.92 17423.17 17431.07 17412.51 17374.28 17369.4 17469.24
Table 13 Performance analysis of advance prediction of the BSE price for different orders of FLRs. Statistics
Fifthorder
Sixthorder
Seventhorder
Eightorder
Mean observed (Rupee) Mean predicted (Rupee) SD observed (Rupee) SD predicted (Rupee) RMSE (Rupee) U
17612.86
17621.19
17622.03
17623.01
17523.59
17538.02
17550.14
17561.56
169.51 165.76 203.10 0.0058
167.84 152.44 201.63 0.0057
171.56 143.56 193.19 0.0055
175.54 135.84 186.77 0.0053
10. Empirical analysis – daily temperature prediction Advance predicted values of temperature from June (1996) to September (1996) in Taipei for the fifth-order FLRs are presented in Table 6. The proposed model is also tested with different orders of FLRs. The performance of the model is evaluated with various statistical parameters, which are presented in Tables 7–10. From Tables 7–10, it is clear that mean of observed values are close to mean of predicted values. The comparison of SD values between observed and predicted values show that predictive skill of our proposed model is good for June, July and September. But, SD for predicted values for August shows slight deflection from SD of observed values. Forecasted results in terms of RMSE indicate very small error rate. In Tables 7–10, U values are closer to 0, which indicate the effectiveness of the proposed model. During the training and testing processes of neural network, a number of experiments were carried out to set additional parameters, viz., initial weight, learning rate, epochs, learning radius and activation function to obtain the optimal results, and we have chosen the ones that exhibit the best behavior in terms of accuracy.
The determined optimal values of all these parameters are given in Table 11. 11. About BSE and its advance prediction BSE Limited formerly known as Bombay Stock Exchange (BSE) is a stock exchange located in Mumbai (India) and is the oldest stock exchange in Asia. The equity market capitalization of the companies listed on the BSE was US$1 trillion as of December 2011, making it the 6th largest stock exchange in Asia and the 14th largest in the world (www.World-exchanges.org). The BSE has the largest number of listed companies in the world. To further demonstrate the applicability of the proposed model, daily stock exchange price of the BSE is tried to be predicted. The BSE data set for the period 7/30/2012–9/11/2012 is collected from [56]. The predicted values of the BSE based on the fifth-order FLRs are presented in Table 12. To check the efficiency of the model, results are also obtained with different orders of FLRs. The performance of the model is evaluated with various statistical parameters, which are presented in Table 13. All these statistical analyzes signify the robustness of the proposed model for advance prediction of the BSE price. 12. Conclusions This article presents a novel approach combining ANN with fuzzy time series for building a time series forecasting expert system. For training process, the daily average temperature data of Taipei from June 1, 1996 to June 30, 1996 are used; while for testing process, the daily average temperature data of Taipei from July, 1996 to September, 1996 are considered. The proposed model is also validated by predicting the BSE price from the period 7/30/2012 to 9/ 11/2012. In this work, we have incorporated ‘‘RPD’’ approach for determining the lengths of the intervals effectively, which is an improvement over the original works presented by [1,7,2]. Also, many existing fuzzy time series models as discussed earlier, use the current state’s fuzzified values for defuzzification, and limit their predictive skill within the sample. So, to make the prediction out of sample, we have used the previous state’s fuzzified values for defuzzification. In this study, for defuzzification operation, an ANN based architecture is developed, which is based on the BPNN algorithm. The proposed neural network architecture takes the previous state’s fuzzified values as inputs and outputs are computed in advance. In this study, we try to obtain the forecasting results with optimal number of intervals. To obtain the results for the months–June, July, August and September, only 17, 17, 20 and 21 intervals are used respectively. On the other hand, for the BSE price prediction, only 21 intervals are employed. The performance of the model is evaluated with different orders of fuzzy logical relations, which signify the efficiency of the proposed model in case of temperature as well as stock exchange price prediction. There is a limitation of the proposed model is that it can applicable only in one-factor time series data set. Hence, we have tried to make our model generalize enough so that it can deal with different kinds of one-factor time series data set and can be used in various domains efficiently. However, there is a scope to test the proposed model on other domains in the following way: 1. Apply the proposed model on different regions of temperature data set (one-factor), and check its accuracy and performance with different size of intervals and orders. 2. To test the performance of the model for different types of financial, stocks and marketing data set (one-factor).
P. Singh, B. Borah / Knowledge-Based Systems 46 (2013) 12–21
Hence, this study implies that the proposed model can be applied to improve the accuracy and performance of fuzzy time series forecasting model. References [1] Q. Song, B.S. Chissom, Forecasting enrollments with fuzzy time series – part I, Fuzzy Sets and Systems 54 (1993) 1–9. [2] Q. Song, B.S. Chissom, Forecasting enrollments with fuzzy time series – part II, Fuzzy Sets and Systems 62 (1994) 1–8. [3] S.M. Chen, Forecasting enrollments based on fuzzy time series, Fuzzy Sets and Systems 81 (1996) 311–319. [4] J.R. Hwang, S.M. Chen, C.H. Lee, Handling forecasting problems using fuzzy time series, Fuzzy Sets and Systems 100 (1998) 217–228. [5] K. Huarng, Heuristic models of fuzzy time series for forecasting, Fuzzy Sets and Systems 123 (2001) 369–386. [6] K. Huarng, Effective lengths of intervals to improve forecasting in fuzzy time series, Fuzzy Sets and Systems 123 (2001) 387–394. [7] Q. Song, B.S. Chissom, Fuzzy time series and its models, Fuzzy Sets and Systems 54 (1993) 1–9. [8] C. Cheng, J. Chang, C. Yeh, Entropy-based and trapezoid fuzzification-based fuzzy time series approaches for forecasting IT project cost, Technological Forecasting and Social Change 73 (2006) 524–542. [9] C.-C. Tsai, S.-J. Wu, Forecasting enrolments with high-order fuzzy time series, in: 19th International Conference of the North American, Fuzzy Information Processing Society, Atlanta, GA, 2000, pp. 196–200. [10] S.-M. Chen, Forecasting enrollments based on high-order fuzzy time series, Cybernetics and Systems 33 (2002) 1–16. [11] S.-M. Chen, N.-Y. Chung, Forecasting enrollments using high-order fuzzy time series and genetic algorithms, International Journal of Intelligent Systems 21 (2006) 485–501. [12] T.-L. Chen, C.-H. Cheng, H.-J. Teoh, High-order fuzzy time-series based on multi-period adaptation model for forecasting stock markets, Physica A: Statistical Mechanics and its Applications 387 (2008) 876–888. [13] S.-M. Chen, C.-D. Chen, Handling forecasting problems based on high-order fuzzy logical relationships, Expert Systems with Applications 38 (2011) 3857– 3864. [14] S.-M. Chen, K. Tanuwijaya, Fuzzy forecasting based on high-order fuzzy logical relationships and automatic clustering techniques, Expert Systems with Applications 38 (2011) 15425–15437. [15] S.M. Chen, J.R. Hwang, Temperature prediction using fuzzy time series, IEEE Transactions on Systems, Man and Cybernetics 30 (2000) 263–275. [16] K.-H. Huarng, T.H.-K. Yu, Y.W. Hsu, A multivariate heuristic model for fuzzy time-series forecasting, IEEE Transactions on Systems, Man, and Cybernetics, Part B: Cybernetics 37 (2007) 836–846. [17] M. Sah, K. Degtiarev, Forecasting enrollment model based on first-order fuzzy time series, in: Proceedings of World Academy of Sciences, Engineering and Technology, 2005, vol. 1, pp. 375–378. [18] H.S. Lee, M.T. Chou, Fuzzy forecasting based on fuzzy time series, International Journal of Computer Mathematics 81 (2004) 781–789. [19] H.-K. Yu, Weighted fuzzy time series models for TAIEX forecasting, Physica A: Statistical Mechanics and its Applications 349 (2005) 609–624. [20] J. Chang, Y. Lee, S. Liao, C. Cheng, Cardinality-based fuzzy time series for forecasting enrollments, in: New Trends in Applied Artificial Intelligence, Japan, vol. 4570, 2007, pp. 735–744. [21] T.-L. Chen, C.-H. Cheng, H.J. Teoh, Fuzzy time-series based on Fibonacci sequence for stock price forecasting, Physica A: Statistical Mechanics and its Applications 380 (2007) 377–390. [22] C.H. Cheng, G.W. Cheng, J.W. Wang, Multi-attribute fuzzy time series method based on fuzzy clustering, Expert Systems with Applications 34 (2008) 1235– 1242. [23] C. Kai, F.F. Ping, C.W. Gang, A novel forecasting model of fuzzy time series based on k-means clustering, in: 2010 Second International Workshop on Education Technology and Computer Science, China, pp. 223–225. [24] H.-L. Chou, J.-S. Chen, C.-H. Cheng, H. Teoh, Forecasting tourism demand based on improved fuzzy time series model, in: N. Nguyen, M. Le, J. Swiatek (Eds.), Intelligent Information and Database Systems, vol. 5990, Springer, Berlin/ Heidelberg, 2010, pp. 399–407. [25] P. Singh, B. Borah, An efficient method for forecasting using fuzzy time series, in: U. Sharma, B. Nath, D.K. Bhattacharya (Eds.), Machine Intelligence, Tezpur University, Assam, Narosa, India, 2011, pp. 67–75. [26] S.R. Singh, A computational method of forecasting based on high-order fuzzy time series, Expert Systems with Applications 36 (2009) 10551–10559.
21
[27] H.-T. Liu, M.-L. Wei, An improved fuzzy forecasting method for seasonal time series, Expert Systems with Applications 37 (2010) 6310–6318. [28] Q. Wangren, L. Xiaodong, L. Hailin, A generalized method for forecasting based on fuzzy time series, Expert Systems with Applications 38 (2011) 10446– 10453. [29] M. Shah, Fuzzy based trend mapping and forecasting for time series data, Expert Systems with Applications 39 (2012) 6351–6358. [30] E. Hadavandi, H. Shavandi, A. Ghanbari, Integration of genetic fuzzy systems and artificial neural networks for stock price forecasting, Knowledge-Based Systems 23 (2010) 800–808. [31] C.-H. Cheng, T.-L. Chen, L.-Y. Wei, A hybrid model based on rough sets theory and genetic algorithms for stock price forecasting, Information Sciences 180 (2010) 1610–1629. [32] I.-H. Kuo, S.-J. Horng, T.-W. Kao, T.-L. Lin, C.-L. Lee, Y. Pan, An improved method for forecasting enrollments based on fuzzy time series and particle swarm optimization, Expert Systems with Applications 36 (2010) 6108–6117. [33] C. Aladag, M. Basaran, E. Egrioglu, U. Yolcu, V. Uslu, Forecasting in high order fuzzy times series by using neural networks to define fuzzy relations, Expert Systems with Applications 36 (2009) 4228–4231. [34] H. Teoh, C. Cheng, H. Chu, J. Chen, Fuzzy time series model based on probabilistic approach and rough set rule induction for empirical research in stock markets, Data and Knowledge Engineering 67 (2008) 103–117. [35] S.K. Pal, P. Mitra, Case generation using rough sets with fuzzy representation, IEEE Transactions on Knowledge and Data Engineering 16 (2004) 292–300. [36] L. Yang, C.W. Dawson, M.R. Brown, M. Gell, Neural network and ga approaches for dwelling fire occurrence prediction, Knowledge-Based Systems 19 (2006) 213–219. [37] A. Keles, M. Kolcak, A. Keles, The adaptive neuro-fuzzy model for forecasting the domestic debt, Knowledge-Based Systems 21 (2008) 951–957. [38] P.-C. Chang, C.-H. Liu, C.-Y. Fan, Data clustering and fuzzy neural network for sales forecasting: a case study in printed circuit board industry, KnowledgeBased Systems 22 (2009) 344–355. [39] K. Huarng, T.H.-K. Yu, The application of neural networks to forecast fuzzy time series, Physica A: Statistical Mechanics and its Applications 363 (2006) 481–491. [40] T.H.-K. Yu, K.-H. Huarng, A bivariate fuzzy time series model to forecast the TAIEX, Expert Systems with Applications 34 (2008) 2945–2952. [41] I.-H. Kuo, S.-J. Horng, T.-W. Kao, T.-L. Lin, C.-L. Lee, Y. Pan, An improved method for forecasting enrollments based on fuzzy time series and particle swarm optimization, Expert Systems with Applications 36 (2009) 6108–6117. [42] Y.-L. Huang, S.-J. Horng, M. He, P. Fan, T.-W. Kao, M.K. Khan, J.-L. Lai, I.-H. Kuo, A hybrid forecasting model for enrollments based on aggregated fuzzy time series and particle swarm optimization, Expert Systems with Applications 38 (2011) 8014–8023. [43] L.A. Zadeh, Fuzzy sets, Information and Control 8 (1965) 338–353. [44] L.A. Zadeh, The concept of a linguistic variable and its application to approximate reasoning – 1, Information Sciences 8 (1975) 199–249. [45] L.A. Zadeh, Outline of a new approach to the analysis of complex system and decision process, IEEE Transactions on System, Man and Cybernetics 3 (1973) 28–44. [46] L.A. Zadeh, Similarity relations and fuzzy orderings, Information Sciences 3 (1971) 177–200. [47] S. Kumar, Neural Networks: A Classroom Approach, Tata McGraw-Hill Education Pvt. Ltd., New Delhi, 2004. [48] P.R. Lippmann, Pattern classification using neural networks, IEEE Communications Magazine 11 (1989) 47–54. [49] A.E. Bryson, Y.C. Ho, Applied Optimal Control: Optimization, Estimation, and Control, Blaisdell Publishing Company or Xerox College Publishing, 1969. [50] E.D. Rumelhart, E.G. Hinton, J.R. Williams, Learning representations by backpropagating errors, Nature 323 (1986) 533–536. [51] S.N. Sivanandam, S.N. Deepa, Principles of Soft Computing, Wiley India (P) Ltd., New Delhi, 2007. [52] I. Wilson, S. Paris, J. Ware, D. Jenkins, Residential property price time series forecasting with neural networks, Knowledge-Based Systems 15 (2002) 335– 341. [53] H. Sturges, The choice of a class-interval, Journal of the American Statistical Association 21 (1926) 65–66. [54] J. Han, M. Kamber, Data Mining: Concepts and Techniques, first ed., Morgan Kaufmann Publishers, USA, 2001. [55] H. Theil, Applied Economic Forecasting, Rand McNally, New York, 1966. [56] Y. Finance, Daily Stock Exchange Price List of BSE, 2012.
Contents lists available at SciVerse ScienceDirect
Knowledge-Based Systems journal homepage: www.elsevier.com/locate/knosys
High-order fuzzy-neuro expert system for time series forecasting Pritpal Singh ⇑, Bhogeswar Borah Department of Computer Science and Engineering, Tezpur University, Tezpur 784 028, Assam, India
a r t i c l e
i n f o
Article history: Received 18 July 2012 Received in revised form 6 December 2012 Accepted 25 January 2013 Available online 21 March 2013 Keywords: Fuzzy time series High-order Temperature Stock exchange Interval Fuzzy logical relation Artificial neural network
a b s t r a c t In this article, we present a new model based on hybridization of fuzzy time series theory with artificial neural network (ANN). In fuzzy time series models, lengths of intervals always affect the results of forecasting. So, for creating the effective lengths of intervals of the historical time series data set, a new ‘‘RePartitioning Discretization (RPD)’’ approach is introduced in the proposed model. Many researchers suggest that high-order fuzzy relationships improve the forecasting accuracy of the models. Therefore, in this study, we use the high-order fuzzy relationships in order to obtain more accurate forecasting results. Most of the fuzzy time series models use the current state’s fuzzified values to obtain the forecasting results. The utilization of current state’s fuzzified values (right hand side fuzzy relations) for prediction degrades the predictive skill of the fuzzy time series models, because predicted values lie within the sample. Therefore, for advance forecasting of time series, previous state’s fuzzified values (left hand side of fuzzy relations) are employed in the proposed model. To defuzzify these fuzzified time series values, an ANN based architecture is developed, and incorporated in the proposed model. The daily temperature data set of Taipei, China is used to evaluate the performance of the model. The proposed model is also validated by forecasting the stock exchange price in advance. The performance of the model is evaluated with various statistical parameters, which signify the efficiency of the model. Ó 2013 Elsevier B.V. All rights reserved.
1. Introduction Advance prediction of events like temperature, rainfall, stock price, population growth, and economy growth, are major scientific issues in the area of forecasting. Forecasting of all these events are tedious tasks because of their dynamic nature. Forecasting of all these events with 100% accuracy may not be possible, but the forecasting accuracy and the speed of forecasting process can be improved. So, in this article, we present a novel forecasting model, which is developed by hybridizing fuzzy time series theory with artificial neural network (ANN). The main aim of designing such a hybridized model is explained next. For fuzzification of time series data set, determination of lengths of intervals of the historical time series data set is very important. In most of the fuzzy time series models [1–5], the lengths of the intervals were kept the same. No specific reason is mentioned for using the fix lengths of intervals. Huarng [6] shows that the lengths of intervals always affect the results of forecasting. So, for creating the effective lengths of intervals, a new ‘‘Re-Partitioning Discretization (RPD)’’ approach is incorporated in the proposed model. After generating the intervals, time series data set is fuzzified based on the fuzzy time series theory. Most of the previous fuzzy ⇑ Corresponding author. E-mail addresses: [email protected] (P. Singh), [email protected] (B. Borah). 0950-7051/$ - see front matter Ó 2013 Elsevier B.V. All rights reserved. http://dx.doi.org/10.1016/j.knosys.2013.01.030
time series models [1,7,2–4,8] use first-order fuzzy relationships to get the forecasting results. Many researchers show that high-order fuzzy relationships improve the forecasting accuracy of the models [9–14]. Therefore, in this work, we employ the high-order fuzzy relationships for obtaining the forecasting results. Song and Chissom [1] adopted the following method to forecast enrollments of the University of Alabama:
YðtÞ ¼ Yðt 1Þ R;
ð1Þ
where Y(t 1) is the fuzzified enrollment of year (t 1), Y(t) is the forecasted enrollment of year ‘‘t’’ represented by fuzzy set, ‘‘’’ is the max–min composition operator, and ‘‘R’’ is the union of fuzzy relations. This method takes much time to compute the union of fuzzy relations R, especially when the number of fuzzy relations is more in (1) [15,16]. In 1996, Chen [3] used simplified arithmetic operations for defuzzification operation by avoiding this complicated max–min operations and their method produced better results than Song and Chissom models [1,7,2]. Most of the existing fuzzy time series models use Chen’s defuzzification method [3] in order to obtain the forecasting results. However, forecasting accuracy of these models are not good enough. Also, previous fuzzy time series models use the current state’s fuzzified values for forecasting. This approach, no doubt, improves the forecasting accuracy, but it degrades the predictive skill of the fuzzy time series models, because predicted values lie within the sample. So, for obtaining the forecasting results out of sample (i.e., in advance), we use the
P. Singh, B. Borah / Knowledge-Based Systems 46 (2013) 12–21
previous state’s fuzzified values (left hand side of fuzzy relations) in this model. To defuzzify these fuzzified values, an ANN based architecture is developed, and incorporated in this model. So, we have entitled this model as ‘‘High-order fuzzy-neuro time series forecasting model’’. The proposed model has the advantage that it can produce good forecasting results. We demonstrate the application of the proposed model using the following two real-world data set: 1. Daily average temperature data set of Taipei, China. 2. Daily stock exchange price data set of Bombay Stock Exchange (BSE), India. The rest of this paper is organized as follows. In Section 2, we present related works for fuzzy time series models. In Section 3, we review the theory of fuzzy set with an overview of fuzzy time series. In Section 4, we give an overview of ANN along with its application in the proposed model. In Section 5, description of data set is provided. Section 6 shows the application of a new approach to find the length of intervals in the universe of discourse. The architecture of the proposed model and its training phases are presented in Sections 7 and 8 respectively. The performance of the model is assessed with various statistical parameters, which are discussed in Section 9. Empirical analysis for forecasting the daily temperature is presented in Section 10. Section 11 shows the application of the proposed model for forecasting the stock exchange price. The conclusions are discussed in Section 12.
2. Related works Forecasting using fuzzy time series is applied in several areas including forecasting university enrollments, sales, road accidents and financial forecasting. In a conventional time series models, the recorded values of a special dynamic process are represented by crisp numerical values. But, in a fuzzy time series model, the recorded values of a special dynamic process are represented by linguistic values. Based on fuzzy time series theory, first forecasting model was introduced by Song and Chissom [1,7,2]. They presented the fuzzy time series model by fuzzy relational equations involving max–min composition operation and applied the model to forecast the enrollments in the University of Alabama. In 1996, Chen [3] used simplified arithmetic operations avoiding the complicated max–min operations and their method produced better results. Later, many studies provided some improvements to the fuzzy time series methods in determining the lengths of intervals, fuzzification process and defuzzification techniques. Hwang et al. [4] used the differences of the available historical data as fuzzy time series rather than direct usage of raw numeric values. Sah and Degtiarev also used a similar approach in [17]. Huarng tried to improve the forecasting accuracy based on determination of the length of intervals [6] and heuristic approaches [5]. Lee and Chou [18] forecasted the university enrollments with the average error rate less than Chen’s method [3] by defining the supports of the fuzzy numbers that represent the linguistic values of the linguistic variables more appropriately. Yu [19] proposed weighted fuzzy time series model to resolve issues of recurrence and weighting in fuzzy time series forecasting. Cheng et al. [8] used entropy minimization to create the intervals. They also used trapezoidal membership functions in the fuzzification process. Chang et al. [20] presented cardinality-based fuzzy time series forecasting model, which builds weighted fuzzy rules by calculating the cardinality of fuzzy relations. To enhance the performance of fuzzy time series models, Chen et al. [21] incorporates the concept of the Fibonacci sequence in the existing models as proposed by Song and Chissom [1,2] and Yu [19]. To obtain less
13
number of intervals, Cheng et al. [22] proposed a model using fuzzy clustering technique to partition the data effectively. The K-means clustering algorithm has been applied to partition the universe of discourse in [23]. Chou et al. [24] forecasted the tourism demand based on hybridization of rough set with fuzzy time series. Singh and Borah [25] forecasted the university enrollments with the help of new proposed algorithm by dividing the universe of discourse of the historical time series data into different length of intervals. Recent advancement in fuzzy time series forecasting models can be found in [26–29]. Recently, many researchers have proposed various hybridization based models to solve complex problems in forecasting. For example, Hadavandi et al. [30] presented a new approach based on genetic fuzzy systems and ANNs for building a stock price forecasting expert system to improve the forecasting accuracy. Cheng et al. [31] proposed a new stock price forecasting model based on hybridization of genetic algorithm with rough set theory. Kuo et al. [32] hybridized the particle swarm optimization with fuzzy time series to adjust the lengths of intervals in the universe of discourse. Aladag et al. [33] introduced a new approach to define fuzzy relation in high order fuzzy time series using feed forward neural networks. Teoh et al. [34] proposed a fuzzy-rough hybrid forecasting model, where rules (fuzzy logical relationships) are generated by rough set algorithm. Pal and Mitra [35] proposed a rough-fuzzy hybridization scheme for case generation. They used the fuzzy set theory for linguistic representation of patterns and then obtained the dependency rules by using the rough set theory. For advance prediction of dwelling fire occurrence in Derbyshire (United Kingdom), Yang et al. [36] employed three approaches: logistic regression, ANN and Genetic Algorithm. Keles et al. [37] proposed a model for forecasting the domestic dept by Adaptive Neuro-Fuzzy Inference System. Chang et al. [38] developed a hybrid model by integrating K-mean cluster and fuzzy neural network to forecast the future sales of a printed circuit board factory. Huarng and Yu [39], and Yu and Huarng [40] presented a new hybrid model based on neural network and fuzzy time series to forecast TAIEX. Kuo et al. [41] and Huang et al. [42] introduced a new enrollments forecasting model based on hybridization of fuzzy time series and particle swarm optimization. 3. Fuzzy sets and fuzzy time series In 1965, Zadeh [43] introduced fuzzy sets theory involving continuous set membership for processing data in presence of uncertainty. He also presented fuzzy arithmetic theory and its application [44–46]. In this section, we will briefly review fuzzy sets theory from [43] and fuzzy time series concepts from [1,7,2]. Definition 3.1 (Fuzzy Set [43]). A fuzzy set is a class with varying degrees of membership in the set. Let U be the universe of discourse, which is discrete and finite, then fuzzy set A can be defined as follows:
A ¼ flA ðx1 Þ=x1 þ lA ðx2 Þ=x2 þ g ¼ Ri lA ðxi Þ=xi ;
ð2Þ
where lA is the membership function of A, lA: U ? [0, 1], and lA(xi) is the degree of membership of the element xi in the fuzzy set A. Here, the symbol ‘‘+’’ indicates the operation of union and the symbol ‘‘/’’ indicates the separator rather than the commonly used summation and division in algebra respectively. When U is continuous and infinite, then the fuzzy set A of U can be defined as:
A¼
Z
lA ðxi Þ=xi ; 8xi 2 U;
ð3Þ
14
P. Singh, B. Borah / Knowledge-Based Systems 46 (2013) 12–21
R
where the integral sign ‘‘ ’’ stands for the union of the fuzzy singletons, lA(xi)/xi. Fuzzy time series concept was proposed in [1,7,2] and the main difference between the traditional time series and the fuzzy time series is that the values of the former are crisp numerical values, while the values of the latter are fuzzy sets. The crisp numerical values can be represented by real numbers whereas in fuzzy sets, the values of observations are represented by linguistic values. The definition of fuzzy time series is briefly reviewed as follows: Definition 3.2 (Fuzzy Time Series [1,7,2]). Let Y(t) (t = 0, 1, 2, . . .) be a subset of R and the universe of discourse on which fuzzy sets li(t) (i = 1, 2, . . .) are defined and let F(t) be a collection of li(t) (i = 1, 2, . . .). Then, F(t) is called a fuzzy time series on Y(t) (t = 0, 1, 2, . . .). From Definition 3.2, we can see that F(t) is a function of time t, and li(t) are linguistic values of F(t), where li(t) (i = 1, 2, . . .) are represented by fuzzy sets, and the values of F(t) can be different at different times because the universe of discourse can be different at different times. Fuzzy time series can be divided into two categories which are the time-invariant fuzzy time series and the time-variant fuzzy time series. If F(t) is caused by F(t 1), i.e., F(t 1) ? F(t), then this relationship can be represented as follows:
FðtÞ ¼ Fðt 1Þ Rðt; t 1Þ;
ð4Þ
where R(t, t 1) is fuzzy relationship between F(t) and F(t 1). Here, R is the union of fuzzy relations and ‘‘’’ is max–min composition operator. It is also called the first-order model of F(t). Definition 3.3 (Fuzzy time-variant and time-invariant series [15]). Let F(t) be a fuzzy time series, and R(t, t 1) be a first-order model of F(t). If R(t, t 1) = R(t 1,t 2) for any time t, and F(t) only has finite elements, then F(t) is referred as a time-invariant fuzzy time series. Otherwise, it is referred as a time-variant fuzzy time series. Definition 3.4 (Fuzzy logical relationship [1–3]). Assume that F(t 1) = Ai and F(t) = Aj. The relationship between F(t) and F(t 1) is referred as a fuzzy logical relationship (FLR), which can be represented as:
Ai ! Aj ;
ð5Þ
where Ai and Aj refer to the previous state and current state of the FLR respectively. Definition 3.5 (Fuzzy logical relationship group [1–3]). Assume the following FLRs:
Ai ! Ak1 ; Ai ! Ak2 ; Ai ! Akm : Chen [3] suggested that the FLRs having the same previous state are grouped into a same fuzzy logical relationship group (FLRG). So, based on Chen’s model [3], these FLRs can be grouped into the same FLRG as:
4. ANN and its application ANNs are massively parallel adaptive networks of simple nonlinear computing elements called neurons which are intended to abstract and model some of the functionality of the human nervous system in an attempt to partially capture some of its computational strengths [47]. The neurons in an ANN are organized into different layers. Inputs to the network are existed in the input layer; whereas outputs are produced as signals in the output layer. These signals may pass through one or more intermediate or hidden layers which transform the signals depending upon the neuron signal functions. A simple neural network architecture as proposed by Lippmann [48] is shown in Fig. 1. In Fig. 1, Z1, Z2, . . . , Zn are the set of input neurons, which transmitting information or signals to another output neuron, say Y. Each input neuron Z1, Z2, . . . , Zn has an interconnection links with another neuron. Each interconnection link of input neurons are associated with some weights as W1, W2, . . . , Wn. For this neural network architecture, the net input can be calculated as:
Y input ¼ z1 w1 þ z2 w2 þ þ zn wn ¼
n X zi wi ;
where z1, z2, . . . , zn are the activations or output of input neurons Z1, Z2, . . . , Zn, and w1, w2, . . . , wn are the weights associated with z1, z2, . . . , zn. For output neuron Y, output y can be determined by applying activation function on the net input Yinput as:
y ¼ f ðY input Þ:
ð8Þ
The back-propagation neural network (BPNN) is one of the significant developments in the area of ANN [49,50]. The BPNN can consist of multi-layer feed-forward neural network with one input layer, limited number of hidden layers and one output layer. In Fig. 2, an architecture of the BPNN is shown, which consists of only one hidden layer. The main objective of using the BPNN with multi-layer feed-forward neural network is to minimize the output error obtained from the difference between the calculated output (o1 ,o2, . . . , on) and target output (n1, n2, . . . , nn) of the neural network by adjusting the weights. So, in the BPNN, each information is sent back again in the reverse direction until the output error is very small or zero. The BPNN is trained under the process of three phases as: 1. Using the feed-forward neural network for training process of input information. Adjustment of weights and nodes are made in this phase. 2. To calculate the error. 3. Update the weights. Due to a large number of additional parameters [51], e.g., initial weight, learning rate, momentum, epoch, activation function, etc., the ANN model has great capability to learn by making proper adjustment of these parameters, to produce the desired output. In this study, we use the BPNN algorithm to defuzzify the fuzzified
Ai ! Ak1 ; Ak2 ; . . . ; Akm : Definition 3.6 (High-order FLR [14]). Assume that F(t) is caused by F(t 1), F(t 2), . . . , F(t n) (n > 0), then high-order FLR can be expressed as:
Fðt nÞ; . . . ; Fðt 2Þ;
Fðt 1Þ ! FðtÞ:
ð7Þ
i¼1
ð6Þ Fig. 1. A simple neural network architecture.
15
P. Singh, B. Borah / Knowledge-Based Systems 46 (2013) 12–21
Table 1 Historical data of the daily average temperature from June 1996 to September 1996 in Taipei (Unit: °C). Day
Fig. 2. A multi-layer BPNN architecture.
time series data set. Paradigms adopted for building the basic architecture for the proposed neural network is explained next. Designing the right neural network architecture is a heuristic based approach and also a very time consuming process. The performance of the neural network architecture depends on number of layers, number of nodes in each layer and number of interconnection links with the nodes [52]. Since, a neural network with more than three layers generate arbitrarily complex decision regions. Therefore, a single hidden layer with one input layer and one output layer is considered here in designing the architecture. The number of nodes in input layer will depend on order of FLRs. For example, for third-order FLR, there would be three nodes in input layer; for fourth-order FLR, there would be four nodes in input layer, and so on. The minimum number of nodes in hidden layer is determined by the following equation:
Hiddennodes ¼ Inputnodes 1;
ð9Þ
where Hiddennodes and Inputnodes represent the number of nodes in hidden and input layers respectively. A neural network architecture for the fifth-order FLRs is shown in Fig. 3. The neural network as shown in Fig. 3 have five nodes (Ii, i = 1, 2, . . . , 5) in input layer. The arrangement of nodes in input layer is done in the following sequence:
Yðt 5Þ; Yðt 4Þ; Yðt 3Þ; Yðt 2Þ; YðtÞ ! Yðt þ 1Þ:
ð10Þ
Here, each input node take the previous days (t 5, t 4, . . . , t) fuzzified time series values (e.g., A5, A4, . . . , A1) to predict 1 day (t + 1) advance daily temperature value ‘‘At+1’’. In Eq. (10), each ‘‘t’’ represent the day for considered fuzzified time series values.
Month
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31
June
July
August
September
26.1 27.6 29.0 30.5 30.0 29.5 29.7 29.4 28.8 29.4 29.3 28.5 28.7 27.5 29.5 28.8 29.0 30.3 30.2 30.9 30.8 28.7 27.8 27.4 27.7 27.1 28.4 27.8 29.0 30.2
29.9 28.4 29.2 29.4 29.9 29.6 30.1 29.3 28.1 28.9 28.4 29.6 27.8 29.1 27.7 28.1 28.7 29.9 30.8 31.6 31.4 31.3 31.3 31.3 28.9 28.0 28.6 28.0 29.3 27.9 26.9
27.1 28.9 28.9 29.3 28.8 28.7 29.0 28.2 27.0 28.3 28.9 28.1 29.9 27.6 26.8 27.6 27.9 29.0 29.2 29.8 29.6 29.3 28.0 28.3 28.6 28.7 29.0 27.7 26.2 26.0 27.7
27.5 26.8 26.4 27.5 26.6 28.2 29.2 29.0 30.3 29.9 29.9 30.5 30.2 30.3 29.5 28.3 28.6 28.1 28.4 28.3 26.4 25.7 25.0 27.0 25.8 26.4 25.6 24.2 23.3 23.5
6. Re-Partitioning Discretization (RPD) approach In this section, we propose a new discretization approach referred to as ’’RPD’’ for determining the universe of discourse of the historical time series data set and partitioning it into different lengths of intervals. To explain this approach, the daily average temperature data set from June 1, 1996 to June 30, 1996, shown in Table 1, is employed. Each step of the approach is explained next. Step 6.1. Compute range (R) of a sample, S = {x1, x2, . . . , xn} as:
5. Description of data set For verifying the model, the daily average temperature data set [15] from June 1996 to September 1996 in Taipei is employed. This data set is shown in Table 1. Taipei, which is the capital of the Republic of China, is situated at the northern tip of the island of China. It is the political, economic, and cultural center of China. So, advance prediction of daily temperature of Taipei is very advantageous for the inhabitant of Taipei.
R ¼ Maxv alue Minv alue ;
ð11Þ
where Maxvalue and Minvalue are the maximum and minimum values of S respectively. From Table 1, Maxvalue and Minvalue for the June temperature data set (S) are 30.9 and 26.1 respectively. Therefore, the range R for this data set is computed as:
R ¼ 30:9 26:1 ¼ 4:8: Step 6.2. Split the data range R into M equally spaced classes, where M can be defined as [53]:
n M ¼ 1 þ log2 ;
ð12Þ
where n is the size of the sample S. Based on Eq. (12), we can compute M as:
M ¼1þ
1:477 ¼ 5:907; where sample size n ¼ 30: 0:3010
Step 6.3. Obtain width of an interval (H) as:
H¼ Fig. 3. A BPNN architecture for the fifth-order FLRs.
R : M
Based on Eq. (13), we can calculate the width as:
ð13Þ
16
P. Singh, B. Borah / Knowledge-Based Systems 46 (2013) 12–21
H¼
v i ¼ ½MðiÞ; VðiÞ; i ¼ 1; 2; 3; . . . ; < Bmax ; v i 2 U B ;
4:8 ¼ 0:8126: 5:907
1 6 VðiÞ ð27Þ
Step 6.4. Define the universe of discourse U of the sample S as:
U ¼ ½Lb ; U b ;
ð14Þ
where Lb = Minvalue H, and Ub = Maxvalue + H. Based on Table 1, we have the universe of discourse of the sample S as:
where M(i) = Bmin + (i 1) DFB, and V(i) = Bmin + i DFB. Based on Eq. (26), intervals for the sub-boundary UA are:
u1 ¼ ½26:1; 26:34; u2 ¼ ½27:06; 27:30; . . . ; u6 ¼ ½28:26; 28:50:
U ¼ ½26:1 0:8126; 30:9 þ 0:8126 ¼ ½25:287; 31:713: Step 6.5. Compute mid-point (Umid) of the universe of discourse U as:
U mid ¼
Lb þ U b : 2
ð15Þ
The Umid of the sample S is obtained as:
U mid ¼
25:287 þ 31:713 ¼ 28:5: 2
Step 6.6. Find sub-sets of the sample S such that:
A ¼ fx 2 Sjx 6 U mid g; B ¼ fx 2 Sjx P U mid g:
ð16Þ ð17Þ
From Table 1, we have obtained the elements of A and B as:
Similarly, based on Eq. (27), intervals for the sub-boundary UB are:
v 1 ¼ ½28:70; 28:81; v 2 ¼ ½28:92; 29:03; v 11 ¼ ½30:79; 30:90: Step 6.10. Allocate the elements to their corresponding intervals. Assign the elements of A and B to their corresponding intervals obtained after partitioning the boundaries UA and UB respectively. All these intervals along with their corresponding elements are shown in Table 2. Last column of Table 2 represents mid-points of the intervals. Intervals which do not cover historical data are discarded from the list. Intervals for the remaining three months July, August and September as shown in Table 1 are obtained in a similar way.
A ¼ f26:1; 27:1; 27:4; 27:5; 27:6; 27:7; 27:8; 27:8; 28:4; 28:5g; 7. Architecture of the model
B ¼ f28:7; 28:7; 28:8; 28:8; 29; 29; 29; 29:3; 29:4; 29:4; 29:5; 29:5; 29:7; 30; 30:2; 30:2; 30:3; 30:5; 30:8; 30:9g: Step 6.7. Define sub-boundaries for A and B as:
U A ¼ ½Amin ; Amax ;
ð18Þ
U B ¼ ½Bmin ; Bmax ;
ð19Þ
where UA and UB are the sub-boundaries for A and B respectively. Here, Amin and Amax represent the minimum and maximum values of the sub-set A respectively. Similarly, Bmin and Bmax represent the minimum and maximum values of the sub-set B respectively. From Eqs. (18) and (19), we can define the sub-boundaries for A and B as:
U A ¼ ½26:1; 28:5;
ð20Þ
U B ¼ ½28:7; 30:9:
ð21Þ
Step 6.8. Determine deciding factors for A and B as:
Amax Amin ; NA Bmax Bmin ; DF B ¼ NB
DF A ¼
ð22Þ
28:5 26:1 ¼ 0:24; 10 30:9 28:7 ¼ 0:11: DF B ¼ 20
DF A ¼
ð24Þ ð25Þ
Step 6.9. Partition the sub-boundaries UA and UB into different length of intervals as:
i ¼ 1; 2; 3; . . . ;
Step Step Step Step Step Step
7.1. 7.2. 7.3. 7.4. 7.5. 7.6.
Partition the universe of discourse into intervals. Define linguistic terms for each of the interval. Fuzzify the time series data set. Establish the FLRs based on Definition 3.4. Construct the FLRGs based on Definition 3.5. Defuzzify and compute the forecasted values.
In this article, an improved fuzzy time series forecasting model is proposed, which is based on the hybridization of fuzzy time series theory with ANN. This model also employs the high-order FLRs to obtain the forecasting results. Therefore, above steps are modified, which is represented by the data-flow diagram (see Fig. 4).
Table 2 Intervals and their corresponding elements for the June daily temperature data set.
ð23Þ
where DFA and DFB are the deciding factors for A and B respectively. Here, NA and NB represent the total number of elements of A and B respectively. From Eqs. (22) and (23), the deciding factors for A and B are:
ui ¼ ½LðiÞ; UðiÞ;
Most of the existing fuzzy time series models as discussed earlier use the following six common steps to deal with the forecasting problems:
1 6 UðiÞ < Amax ; ui
2 UA ; where L(i) = Amin + (i 1) DFA, and U(i) = Amin + i DFA.
ð26Þ
Corresponding element
Mid-point
Interval for UA [26.1, 26.34] [27.06, 27.30] [27.30, 27.54] [27.54, 27.78] [27.78, 28.02] [28.26, 28.50]
(26.1) (27.1) (27.4, 27.5) (27.6, 27.7) (27.8, 27.8) (28.4, 28.5)
26.22 27.18 27.42 27.66 27.9 28.38
Interval for UB [28.70, 28.81] [28.92, 29.03] [29.25, 29.36] [29.36, 29.47] [29.47, 29.58] [29.69, 29.80] [29.91, 30.02] [30.13, 30.24] [30.24, 30.35] [30.46, 30.57] [30.79, 30.90]
(28.7, 28.7, 28.8, 28.8) (29, 29,2 9) (29.3) (29.4, 29.4) (29.5, 29.5) (29.7) (30) (30.2, 30.2) (30.3) (30.5) (30.8, 30.9)
28.755 28.975 29.305 29.415 29.525 29.745 29.965 30.185 30.295 30.515 30.845
17
P. Singh, B. Borah / Knowledge-Based Systems 46 (2013) 12–21
8. High-order fuzzy-neuro time series forecasting model We apply the proposed model to forecast the daily temperature of Taipei from June, 1996 to September, 1996. This model is trained with the June daily temperature data set. Each phase of the training process is explained next. Phase 8.1. Divide the universe of discourse into different lengths of intervals. Define the universe of discourse U for the June temperature data set. Based on Eq. (14), we have defined the universe of discourse U as U = [25.287, 31.713]. Then, based on the RPD approach, the universe of discourse U is partitioned into n different lengths of intervals: a1, a2, a3, . . . , an. The experimental results are presented in Table 2. Phase 8.2. Define linguistic terms for each of the interval. Assume that the historical time series data set is distributed among n intervals (i.e., a1, a2, . . . , an). Therefore, define n linguistic variables A1, A2, . . . , An, which can be represented by fuzzy sets, as shown below: A1 ¼ 1=a1 þ 0:5=a2 þ 0=a3 þ 0=a4 þ 0=a5 þ þ 0=an2 þ 0=an1 þ 0=an ; A2 ¼ 0:5=a1 þ 1=a2 þ 0:5=a3 þ 0=a4 þ 0=a5 þ þ 0=an2 þ 0=an1 þ 0=an ; A3 ¼ 0=a1 þ 0:5=a2 þ 1=a3 þ 0:5=a4 þ 0=a5 þ þ 0=an2 þ 0=an1 þ 0=an ; .. . Aj ¼ 0=a1 þ 0=a2 þ 0=a3 þ 0=a4 þ 0=a5 þ þ 0=an2 þ 0:5=an1 þ 1=an :
Obtain the degree of membership of each day’s temperature value belonging to each Ai. Here, maximum degree of membership of fuzzy set Ai occurs at interval ai, and 1 6 i 6 n. Phase 8.3. Fuzzify the historical time series data. If one day’s datum belongs to the interval ai, then it is fuzzified into Ai, where 1 6 i 6 n. If one day’s temperature value belongs to the interval ai, then the fuzzified temperature value for that day is considered as Ai. For example, the temperature value of June 1, 1996 belongs to the interval a1, so it is fuzzified to A1. In this way, we have fuzzified historical time series data set. The fuzzified temperature values are shown in Table 3. Phase 8.4. Establish the high-order FLRs between the fuzzified daily temperature values. Based on Definition 3.6, we have established the fifth-order FLRs between the fuzzified daily temperature values. For example, in Table 3, the fuzzified daily temperature values for days 1, 2, 3, 4, 5 and 6 are A1, A4, A8, A16, A13 and A11, respectively. Here, to establish the fifth-order FLR among these fuzzified values, it is considered that A11 is caused by the previous five fuzzified values A1, A4, A8, A16 and A13. Hence, the fifth-order FLR is represented in the following form:
A1 ; A4 ; A8 ; A16 ; A13 ! A11 :
ð28Þ
Here, left hand side of the FLR is called the previous state, whereas right hand side of the FLR is called the current state. Previously, most of the fuzzy time series models [9–13] use the current state’s fuzzified value for defuzzification. The main downside of using such fuzzified value for defuzzification is that prediction scope of these models [9–13] lie within the sample. For most of the real and complex problems, out of sample prediction (i.e., advance prediction) is very much essential. Therefore, in this model, the previous state’s fuzzified values are used to obtain the forecasting results. Table 3 Fuzzified historical data set for the June daily temperature.
Fig. 4. High-order fuzzy-neuro time series forecasting model.
Day
June
Fuzzified temperature
Mid-point
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30
26.1 27.6 29 30.5 30 29.5 29.7 29.4 28.8 29.4 29.3 28.5 28.7 27.5 29.5 28.8 29 30.3 30.2 30.9 30.8 28.7 27.8 27.4 27.7 27.1 28.4 27.8 29 30.2
A1 A4 A8 A16 A13 A11 A12 A10 A7 A10 A9 A6 A7 A3 A11 A7 A8 A15 A14 A17 A17 A7 A5 A3 A4 A2 A6 A5 A8 A14
26.22 27.66 28.975 30.515 29.965 29.525 29.745 29.415 28.755 29.415 29.305 28.38 28.755 27.42 29.525 28.755 28.975 30.295 30.185 30.845 30.845 28.755 27.9 27.42 27.66 27.18 28.38 27.9 28.975 30.185
18
P. Singh, B. Borah / Knowledge-Based Systems 46 (2013) 12–21
Table 4 Fifth-order FLRs for the June fuzzified daily temperature data set. Fifth-order FLR A1, A4, A8, A16, A13 ? ?h6i A4, A8, A16, A13, A11 ? ?h7i A8, A16, A13, A11, A12 ? ?h8i A16, A13, A11, A12, A10 ? ?h9i A13, A11, A12, A10, A7 ? ?h10i A11, A12, A10, A7, A10 ? ?h11i A12, A10, A7, A10, A9 ? ?h12i A10, A7, A10, A9, A6 ? ?h13i A7, A10, A9, A6, A7 ? ?h14i A10, A9, A6, A7, A3 ? ?h15i A9, A6, A7, A3, A11 ? ?h16i A6, A7, A3, A11, A7 ? ?h17i A7, A3, A11, A7, A8 ? ?h18i A3, A11, A7, A8, A15 ? ?h19i A11, A7, A8, A15, A14 ? ?h20i A7, A8, A15, A14, A17 ? ?h21i A8, A15, A14, A17, A17 ? ?h22i A15, A14, A17, A17, A7 ? ?h23i A14, A17, A17, A7, A5 ? ?h24i A17, A17, A7, A5, A3 ? ?h25i A17, A7, A5, A3, A4 ? ?h26i A7, A5, A3, A4, A2 ? ?h27i A5, A3, A4, A2, A6 ? ?h28i A3, A4, A2, A6, A5 ? ?h29i A4, A2, A6, A5, A8 ? ?h30i
FLRs, so to explain the defuzzification operation, we use the nth-order FLRs, where n P 5. The steps involve in the defuzzification operation are explained next. Step 8.5.1. For forecasting day Y(t), obtain the nth-order FLR, which can be represented in the following form:
Atn ; Atðn1Þ ; . . . ; At1 ! ?hti;
ð29Þ
where ‘‘t’’ represent a day which we want to forecast, and ‘‘n’’ is the order of FLR (n P 5). Here, Atn, At(n1), . . . , At1 are the previous state’s fuzzified values from days, Y(t n), . . . , Y(t 2) to Y(t 1). Step 8.5.2. Find the intervals where the maximum membership values of the fuzzy sets Atn, At(n1), . . . , At1 occur, and let these intervals be an, an1, . . . , a1, respectively. All these intervals have the corresponding mid-points Cn, Cn1, . . . , C1. Step 8.5.3. Replace each previous state’s fuzzified value of (29) with their corresponding mid-point as:
C n ; C n1 ; . . . ; C 1 ! ?hti; n P 5: Step 8.5.4.
The fifth-order FLRs obtained for the fuzzified daily temperature data are presented in Table 4. In this table, each symbol ‘‘?’’ represents the desired output for corresponding day ‘‘t’’ in the symbol ‘‘hi’’, which would be determined by the proposed model. Phase 8.5. Defuzzify the fuzzified time series data set. In this model, we use the BPNN algorithm to defuzzify the fuzzified time series data set. The neural network architecture which is used here for defuzzification operation is presented in Section 4. The proposed model is based on the high-order
Table 5 Advance prediction of the daily temperature for June. Day
Actual temperature
Predicted temperature
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30
26.1 27.6 29 30.5 30 29.5 29.7 29.4 28.8 29.4 29.3 28.5 28.7 27.5 29.5 28.8 29 30.3 30.2 30.9 30.8 28.7 27.8 27.4 27.7 27.1 28.4 27.8 29 30.2
– – – – – 28.6 29.14 29.54 29.7 29.42 29.31 29.27 29.02 28.92 28.55 28.53 28.46 28.53 28.75 29.33 29.75 30.14 30.05 29.39 29.1 28.47 27.77 27.61 27.55 27.81
ð30Þ
Use the mid-points of (30) as inputs in the proposed BPNN architecture to compute the desired output ‘‘?’’ for the corresponding day ‘‘t’’.
The scaling of mid-points are done before beginning the defuzzification operation using min–max normalization [54]. For example, array of mid-points ‘‘Xi’’ are normalized based on the minimum and maximum values of ‘‘Xi’’. A mid-point ‘‘v’’ of ‘‘Xi’’ is ’’ by computing: normalized to ’’v
v ¼
v minA ðnewmaxA newminA Þ þ newminA ; maxA minA
ð31Þ
where minA and maxA are the minimum and maximum values of array ‘‘Xi’’ respectively. Min–max normalization maps a value ‘‘v’’ to ’’ in the range ½newmaxA ; newminA , where newmaxA represents ‘‘1’’ ‘‘v and newminA represents ‘‘0’’. A sample of the results obtained for the June temperature data set are presented in Table 5. For rest of the months, similar approach is adopted for obtaining the results.
9. Performance analysis parameters The performance of the proposed model is evaluated with the help of means and standard deviations (SDs) of the observed and predicted values, root mean square error (RMSE) and Theil’s U Statistic. All these parameters are defined as follows: 1. The mean can be defined as:
Pn A¼
i¼1 Ai
n
:
ð32Þ
2. The SD can be defined as:
rffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffi 1 Xn SD ¼ ðAi AÞ2 : i¼1 n
ð33Þ
3. The RMSE can be defined as:
sffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffi Pn 2 i¼1 ðF i Ai Þ RMSE ¼ : n
ð34Þ
4. The formula used to calculate Theil’s U statistic [55] is:
qffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffi Pn 2 i¼1 ðAi F i Þ ffi ffiffiffiffiffiffiffiffiffiffiffiffiffiffiffi q U ¼ qffiffiffiffiffiffiffiffiffiffiffiffiffiffiffi Pn 2 Pn 2ffi : i¼1 Ai þ i¼1 F i
ð35Þ
19
P. Singh, B. Borah / Knowledge-Based Systems 46 (2013) 12–21 Table 6 Advance prediction of the daily temperature from June (1996) to September (1996) in Taipei (Unit: °C). Day
Actual June
Predicted June
Actual July
Predicted July
Actual August
Predicted August
Actual September
Predicted September
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31
26.1 27.6 29 30.5 30 29.5 29.7 29.4 28.8 29.4 29.3 28.5 28.7 27.5 29.5 28.8 29 30.3 30.2 30.9 30.8 28.7 27.8 27.4 27.7 27.1 28.4 27.8 29 30.2 –
– – – – – 28.6 29.14 29.54 29.7 29.42 29.31 29.27 29.02 28.92 28.55 28.53 28.46 28.53 28.75 29.33 29.75 30.14 30.05 29.39 29.1 28.47 27.77 27.61 27.55 27.81 –
29.9 28.4 29.2 29.4 29.9 29.6 30.1 29.3 28.1 28.9 28.4 29.6 27.8 29.1 27.7 28.1 28.7 29.9 30.8 31.6 31.4 31.3 31.3 31.3 28.9 28 28.6 28 29.3 27.9 26.9
– – – – – 29.25 29.19 29.59 29.45 29.26 28.95 28.77 28.76 28.45 28.67 28.36 28.36 28.18 28.54 28.96 29.72 30.37 30.99 31.29 31.3 30.84 29.92 29.46 28.86 28.49 28.37
27.1 28.9 28.9 29.3 28.8 28.7 29 28.2 27 28.3 28.9 28.1 29.9 27.6 26.8 27.6 27.9 29 29.2 29.8 29.6 29.3 28 28.3 28.6 28.7 29 27.7 26.2 26 27.7
– – – – – 28.37 28.82 28.69 28.43 28.19 28.14 28.18 27.97 28.17 28.31 27.97 27.85 27.82 27.77 27.8 28.42 28.99 28.91 29.01 28.63 28.74 28.39 28.33 28.23 27.99 27.32
27.5 26.8 26.4 27.5 26.6 28.2 29.2 29 30.3 29.9 29.9 30.5 30.2 30.3 29.5 28.3 28.6 28.1 28.4 28.3 26.4 25.7 25 27 25.8 26.4 25.6 24.2 23.3 23.5 –
– – – – – 26.81 26.97 27.5 27.69 27.86 29.04 29.59 29.93 29.94 29.86 29.8 29.72 29.33 28.64 28.43 28 27.78 26.87 26.66 26 25.96 25.96 25.81 25 25.05 –
Table 7 Performance analysis of the model for the fifth-order FLRs.
Table 10 Performance analysis of the model for the eighth-order FLRs.
Statistics
June
July
August
September
Statistics
June
July
August
September
Mean observed (°C) Mean predicted (°C) SD observed (°C) SD predicted (°C) RMSE (°C) U
28.98 28.91 1.05 0.72 1.23 0.0213
29.25 29.32 1.35 0.94 1.33 0.0225
28.27 28.29 1.04 0.38 1.05 0.0189
27.66 27.77 2.23 1.61 1.35 0.0189
Mean observed (°C) Mean predicted (°C) SD observed (°C) SD predicted (°C) RMSE (°C) U
28.90 29.04 1.10 0.42 1.25 0.0216
29.20 29.27 1.43 0.72 1.47 0.0257
28.23 28.44 1.07 0.33 1.03 0.0185
27.51 28.01 2.33 1.44 1.57 0.0282
Table 8 Performance analysis of the model for the sixth-order FLRs. Statistics
June
July
August
September
Mean observed (°C) Mean predicted (°C) SD observed (°C) SD predicted (°C) RMSE (°C) U
28.95 29.04 1.07 0.65 1.27 0.0219
29.24 29.38 1.38 0.83 1.36 0.0235
28.26 28.48 1.04 0.39 1.03 0.0185
27.64 27.94 2.27 1.61 1.43 0.0256
Table 9 Performance analysis of the model for the seventh-order FLRs. Statistics
June
July
August
September
Mean observed (°C) Mean predicted (°C) SD observed (°C) SD predicted (°C) RMSE (°C) U
28.92 29.05 1.08 0.53 1.22 0.0210
29.20 29.34 1.40 0.74 1.37 0.0239
28.23 28.43 1.05 0.35 1.02 0.0180
27.57 27.97 2.30 1.57 1.39 0.0249
Table 11 Additional parameters and their values during the training and testing processes of neural network. S. no.
Additional parameter
Input value
1 2 3 4 5
Initial weight Learning rate Epochs Learning radius Activation function
0.3 0.5 10,000 3 Sigmoid
Here, each Fi and Ai is the forecasted and actual value of day i respectively, n is the total number of days to be forecasted. In Eqs. (32) and (33), {A1, A2, . . . , An} are the observed values of the actual time series data set and A is the mean value of these observations. Similarly, mean and SD for predicted time series data set are computed. For a good forecasting, the observed means and SDs should be close to predicted means and SDs. In Eq. (34), a small RMSE value indicates good forecasting. In Eq. (35), U is bound between 0 and 1, with values closer to 0 indicating good forecasting accuracy.
20
P. Singh, B. Borah / Knowledge-Based Systems 46 (2013) 12–21
Table 12 Advance prediction of the BSE price from 7/30/2012 to 9/11/2012 (In Rupee). Date (mm-dd-yy)
Actual price
7/30/2012 7/31/2012 8/1/2012 8/2/2012 8/3/2012 8/6/2012 8/7/2012 8/8/2012 8/9/2012 8/10/2012 8/13/2012 8/14/2012 8/16/2012 8/17/2012 8/21/2012 8/22/2012 8/23/2012 8/24/2012 8/27/2012 8/28/2012 8/29/2012 8/30/2012 8/31/2012 9/3/2012 9/4/2012 9/5/2012 9/6/2012 9/7/2012 9/10/2012 9/11/2012
17143.68 17236.18 17257.38 17224.36 17197.93 17412.96 17601.78 17600.56 17560.87 17557.74 17633.45 17728.2 17657.21 17691.08 17885.26 17846.86 17850.22 17783.21 17678.81 17631.71 17490.81 17541.64 17380.75 17384.4 17440.87 17313.34 17346.27 17683.73 17766.78 17852.95
– – – – – 17177.27 17259.28 17299.01 17387.01 17469.96 17547.98 17589.91 17537.01 17592.65 17617.07 17686.52 17709.69 17760.77 17754.85 17774.83 17725.83 17669.82 17561.74 17488.92 17423.17 17431.07 17412.51 17374.28 17369.4 17469.24
Table 13 Performance analysis of advance prediction of the BSE price for different orders of FLRs. Statistics
Fifthorder
Sixthorder
Seventhorder
Eightorder
Mean observed (Rupee) Mean predicted (Rupee) SD observed (Rupee) SD predicted (Rupee) RMSE (Rupee) U
17612.86
17621.19
17622.03
17623.01
17523.59
17538.02
17550.14
17561.56
169.51 165.76 203.10 0.0058
167.84 152.44 201.63 0.0057
171.56 143.56 193.19 0.0055
175.54 135.84 186.77 0.0053
10. Empirical analysis – daily temperature prediction Advance predicted values of temperature from June (1996) to September (1996) in Taipei for the fifth-order FLRs are presented in Table 6. The proposed model is also tested with different orders of FLRs. The performance of the model is evaluated with various statistical parameters, which are presented in Tables 7–10. From Tables 7–10, it is clear that mean of observed values are close to mean of predicted values. The comparison of SD values between observed and predicted values show that predictive skill of our proposed model is good for June, July and September. But, SD for predicted values for August shows slight deflection from SD of observed values. Forecasted results in terms of RMSE indicate very small error rate. In Tables 7–10, U values are closer to 0, which indicate the effectiveness of the proposed model. During the training and testing processes of neural network, a number of experiments were carried out to set additional parameters, viz., initial weight, learning rate, epochs, learning radius and activation function to obtain the optimal results, and we have chosen the ones that exhibit the best behavior in terms of accuracy.
The determined optimal values of all these parameters are given in Table 11. 11. About BSE and its advance prediction BSE Limited formerly known as Bombay Stock Exchange (BSE) is a stock exchange located in Mumbai (India) and is the oldest stock exchange in Asia. The equity market capitalization of the companies listed on the BSE was US$1 trillion as of December 2011, making it the 6th largest stock exchange in Asia and the 14th largest in the world (www.World-exchanges.org). The BSE has the largest number of listed companies in the world. To further demonstrate the applicability of the proposed model, daily stock exchange price of the BSE is tried to be predicted. The BSE data set for the period 7/30/2012–9/11/2012 is collected from [56]. The predicted values of the BSE based on the fifth-order FLRs are presented in Table 12. To check the efficiency of the model, results are also obtained with different orders of FLRs. The performance of the model is evaluated with various statistical parameters, which are presented in Table 13. All these statistical analyzes signify the robustness of the proposed model for advance prediction of the BSE price. 12. Conclusions This article presents a novel approach combining ANN with fuzzy time series for building a time series forecasting expert system. For training process, the daily average temperature data of Taipei from June 1, 1996 to June 30, 1996 are used; while for testing process, the daily average temperature data of Taipei from July, 1996 to September, 1996 are considered. The proposed model is also validated by predicting the BSE price from the period 7/30/2012 to 9/ 11/2012. In this work, we have incorporated ‘‘RPD’’ approach for determining the lengths of the intervals effectively, which is an improvement over the original works presented by [1,7,2]. Also, many existing fuzzy time series models as discussed earlier, use the current state’s fuzzified values for defuzzification, and limit their predictive skill within the sample. So, to make the prediction out of sample, we have used the previous state’s fuzzified values for defuzzification. In this study, for defuzzification operation, an ANN based architecture is developed, which is based on the BPNN algorithm. The proposed neural network architecture takes the previous state’s fuzzified values as inputs and outputs are computed in advance. In this study, we try to obtain the forecasting results with optimal number of intervals. To obtain the results for the months–June, July, August and September, only 17, 17, 20 and 21 intervals are used respectively. On the other hand, for the BSE price prediction, only 21 intervals are employed. The performance of the model is evaluated with different orders of fuzzy logical relations, which signify the efficiency of the proposed model in case of temperature as well as stock exchange price prediction. There is a limitation of the proposed model is that it can applicable only in one-factor time series data set. Hence, we have tried to make our model generalize enough so that it can deal with different kinds of one-factor time series data set and can be used in various domains efficiently. However, there is a scope to test the proposed model on other domains in the following way: 1. Apply the proposed model on different regions of temperature data set (one-factor), and check its accuracy and performance with different size of intervals and orders. 2. To test the performance of the model for different types of financial, stocks and marketing data set (one-factor).
P. Singh, B. Borah / Knowledge-Based Systems 46 (2013) 12–21
Hence, this study implies that the proposed model can be applied to improve the accuracy and performance of fuzzy time series forecasting model. References [1] Q. Song, B.S. Chissom, Forecasting enrollments with fuzzy time series – part I, Fuzzy Sets and Systems 54 (1993) 1–9. [2] Q. Song, B.S. Chissom, Forecasting enrollments with fuzzy time series – part II, Fuzzy Sets and Systems 62 (1994) 1–8. [3] S.M. Chen, Forecasting enrollments based on fuzzy time series, Fuzzy Sets and Systems 81 (1996) 311–319. [4] J.R. Hwang, S.M. Chen, C.H. Lee, Handling forecasting problems using fuzzy time series, Fuzzy Sets and Systems 100 (1998) 217–228. [5] K. Huarng, Heuristic models of fuzzy time series for forecasting, Fuzzy Sets and Systems 123 (2001) 369–386. [6] K. Huarng, Effective lengths of intervals to improve forecasting in fuzzy time series, Fuzzy Sets and Systems 123 (2001) 387–394. [7] Q. Song, B.S. Chissom, Fuzzy time series and its models, Fuzzy Sets and Systems 54 (1993) 1–9. [8] C. Cheng, J. Chang, C. Yeh, Entropy-based and trapezoid fuzzification-based fuzzy time series approaches for forecasting IT project cost, Technological Forecasting and Social Change 73 (2006) 524–542. [9] C.-C. Tsai, S.-J. Wu, Forecasting enrolments with high-order fuzzy time series, in: 19th International Conference of the North American, Fuzzy Information Processing Society, Atlanta, GA, 2000, pp. 196–200. [10] S.-M. Chen, Forecasting enrollments based on high-order fuzzy time series, Cybernetics and Systems 33 (2002) 1–16. [11] S.-M. Chen, N.-Y. Chung, Forecasting enrollments using high-order fuzzy time series and genetic algorithms, International Journal of Intelligent Systems 21 (2006) 485–501. [12] T.-L. Chen, C.-H. Cheng, H.-J. Teoh, High-order fuzzy time-series based on multi-period adaptation model for forecasting stock markets, Physica A: Statistical Mechanics and its Applications 387 (2008) 876–888. [13] S.-M. Chen, C.-D. Chen, Handling forecasting problems based on high-order fuzzy logical relationships, Expert Systems with Applications 38 (2011) 3857– 3864. [14] S.-M. Chen, K. Tanuwijaya, Fuzzy forecasting based on high-order fuzzy logical relationships and automatic clustering techniques, Expert Systems with Applications 38 (2011) 15425–15437. [15] S.M. Chen, J.R. Hwang, Temperature prediction using fuzzy time series, IEEE Transactions on Systems, Man and Cybernetics 30 (2000) 263–275. [16] K.-H. Huarng, T.H.-K. Yu, Y.W. Hsu, A multivariate heuristic model for fuzzy time-series forecasting, IEEE Transactions on Systems, Man, and Cybernetics, Part B: Cybernetics 37 (2007) 836–846. [17] M. Sah, K. Degtiarev, Forecasting enrollment model based on first-order fuzzy time series, in: Proceedings of World Academy of Sciences, Engineering and Technology, 2005, vol. 1, pp. 375–378. [18] H.S. Lee, M.T. Chou, Fuzzy forecasting based on fuzzy time series, International Journal of Computer Mathematics 81 (2004) 781–789. [19] H.-K. Yu, Weighted fuzzy time series models for TAIEX forecasting, Physica A: Statistical Mechanics and its Applications 349 (2005) 609–624. [20] J. Chang, Y. Lee, S. Liao, C. Cheng, Cardinality-based fuzzy time series for forecasting enrollments, in: New Trends in Applied Artificial Intelligence, Japan, vol. 4570, 2007, pp. 735–744. [21] T.-L. Chen, C.-H. Cheng, H.J. Teoh, Fuzzy time-series based on Fibonacci sequence for stock price forecasting, Physica A: Statistical Mechanics and its Applications 380 (2007) 377–390. [22] C.H. Cheng, G.W. Cheng, J.W. Wang, Multi-attribute fuzzy time series method based on fuzzy clustering, Expert Systems with Applications 34 (2008) 1235– 1242. [23] C. Kai, F.F. Ping, C.W. Gang, A novel forecasting model of fuzzy time series based on k-means clustering, in: 2010 Second International Workshop on Education Technology and Computer Science, China, pp. 223–225. [24] H.-L. Chou, J.-S. Chen, C.-H. Cheng, H. Teoh, Forecasting tourism demand based on improved fuzzy time series model, in: N. Nguyen, M. Le, J. Swiatek (Eds.), Intelligent Information and Database Systems, vol. 5990, Springer, Berlin/ Heidelberg, 2010, pp. 399–407. [25] P. Singh, B. Borah, An efficient method for forecasting using fuzzy time series, in: U. Sharma, B. Nath, D.K. Bhattacharya (Eds.), Machine Intelligence, Tezpur University, Assam, Narosa, India, 2011, pp. 67–75. [26] S.R. Singh, A computational method of forecasting based on high-order fuzzy time series, Expert Systems with Applications 36 (2009) 10551–10559.
21
[27] H.-T. Liu, M.-L. Wei, An improved fuzzy forecasting method for seasonal time series, Expert Systems with Applications 37 (2010) 6310–6318. [28] Q. Wangren, L. Xiaodong, L. Hailin, A generalized method for forecasting based on fuzzy time series, Expert Systems with Applications 38 (2011) 10446– 10453. [29] M. Shah, Fuzzy based trend mapping and forecasting for time series data, Expert Systems with Applications 39 (2012) 6351–6358. [30] E. Hadavandi, H. Shavandi, A. Ghanbari, Integration of genetic fuzzy systems and artificial neural networks for stock price forecasting, Knowledge-Based Systems 23 (2010) 800–808. [31] C.-H. Cheng, T.-L. Chen, L.-Y. Wei, A hybrid model based on rough sets theory and genetic algorithms for stock price forecasting, Information Sciences 180 (2010) 1610–1629. [32] I.-H. Kuo, S.-J. Horng, T.-W. Kao, T.-L. Lin, C.-L. Lee, Y. Pan, An improved method for forecasting enrollments based on fuzzy time series and particle swarm optimization, Expert Systems with Applications 36 (2010) 6108–6117. [33] C. Aladag, M. Basaran, E. Egrioglu, U. Yolcu, V. Uslu, Forecasting in high order fuzzy times series by using neural networks to define fuzzy relations, Expert Systems with Applications 36 (2009) 4228–4231. [34] H. Teoh, C. Cheng, H. Chu, J. Chen, Fuzzy time series model based on probabilistic approach and rough set rule induction for empirical research in stock markets, Data and Knowledge Engineering 67 (2008) 103–117. [35] S.K. Pal, P. Mitra, Case generation using rough sets with fuzzy representation, IEEE Transactions on Knowledge and Data Engineering 16 (2004) 292–300. [36] L. Yang, C.W. Dawson, M.R. Brown, M. Gell, Neural network and ga approaches for dwelling fire occurrence prediction, Knowledge-Based Systems 19 (2006) 213–219. [37] A. Keles, M. Kolcak, A. Keles, The adaptive neuro-fuzzy model for forecasting the domestic debt, Knowledge-Based Systems 21 (2008) 951–957. [38] P.-C. Chang, C.-H. Liu, C.-Y. Fan, Data clustering and fuzzy neural network for sales forecasting: a case study in printed circuit board industry, KnowledgeBased Systems 22 (2009) 344–355. [39] K. Huarng, T.H.-K. Yu, The application of neural networks to forecast fuzzy time series, Physica A: Statistical Mechanics and its Applications 363 (2006) 481–491. [40] T.H.-K. Yu, K.-H. Huarng, A bivariate fuzzy time series model to forecast the TAIEX, Expert Systems with Applications 34 (2008) 2945–2952. [41] I.-H. Kuo, S.-J. Horng, T.-W. Kao, T.-L. Lin, C.-L. Lee, Y. Pan, An improved method for forecasting enrollments based on fuzzy time series and particle swarm optimization, Expert Systems with Applications 36 (2009) 6108–6117. [42] Y.-L. Huang, S.-J. Horng, M. He, P. Fan, T.-W. Kao, M.K. Khan, J.-L. Lai, I.-H. Kuo, A hybrid forecasting model for enrollments based on aggregated fuzzy time series and particle swarm optimization, Expert Systems with Applications 38 (2011) 8014–8023. [43] L.A. Zadeh, Fuzzy sets, Information and Control 8 (1965) 338–353. [44] L.A. Zadeh, The concept of a linguistic variable and its application to approximate reasoning – 1, Information Sciences 8 (1975) 199–249. [45] L.A. Zadeh, Outline of a new approach to the analysis of complex system and decision process, IEEE Transactions on System, Man and Cybernetics 3 (1973) 28–44. [46] L.A. Zadeh, Similarity relations and fuzzy orderings, Information Sciences 3 (1971) 177–200. [47] S. Kumar, Neural Networks: A Classroom Approach, Tata McGraw-Hill Education Pvt. Ltd., New Delhi, 2004. [48] P.R. Lippmann, Pattern classification using neural networks, IEEE Communications Magazine 11 (1989) 47–54. [49] A.E. Bryson, Y.C. Ho, Applied Optimal Control: Optimization, Estimation, and Control, Blaisdell Publishing Company or Xerox College Publishing, 1969. [50] E.D. Rumelhart, E.G. Hinton, J.R. Williams, Learning representations by backpropagating errors, Nature 323 (1986) 533–536. [51] S.N. Sivanandam, S.N. Deepa, Principles of Soft Computing, Wiley India (P) Ltd., New Delhi, 2007. [52] I. Wilson, S. Paris, J. Ware, D. Jenkins, Residential property price time series forecasting with neural networks, Knowledge-Based Systems 15 (2002) 335– 341. [53] H. Sturges, The choice of a class-interval, Journal of the American Statistical Association 21 (1926) 65–66. [54] J. Han, M. Kamber, Data Mining: Concepts and Techniques, first ed., Morgan Kaufmann Publishers, USA, 2001. [55] H. Theil, Applied Economic Forecasting, Rand McNally, New York, 1966. [56] Y. Finance, Daily Stock Exchange Price List of BSE, 2012.