- Email: [email protected]
Introducing adaptive neurofuzzy modeling with online learning method for prediction of time-varying solar and geomagnetic activity indices
Expert Systems with Applications 37 (2010) 8267–8277
Contents lists available at ScienceDirect
Expert Systems with Applications journal homepage: www.elsevier.com/locate/eswa
Introducing adaptive neurofuzzy modeling with online learning method for prediction of time-varying solar and geomagnetic activity indices Masoud Mirmomeni a,*, Caro Lucas a,b, Behzad Moshiri a, Babak Nadjar Araabi a,b a b
Control and Intelligent Processing Center of Excellence, School of Electrical and Computer Engineering, University College of Engineering, University of Tehran, Tehran, Iran School of Cognitive Sciences, Institute for Studies in Theoretical Physics and Mathematics, Tehran, Iran
a r t i c l e
i n f o
Keywords: Space weather Solar activity Geomagnetic disturbance Online learning Recursive method Neurofuzzy modeling
a b s t r a c t Research in space weather has in recent years become an active field of research requiring international cooperation because of its importance in hazard warning especially for satellite technology and power utility systems. The time-varying sun as the main source of space weather impacts the Earth’s magnetosphere by emitting hot magnetized plasma called solar wind into interplanetary space. The emission of Solar Energetic Particles (SEPs) and consequently the magnitude of Interplanetary Magnetic Field (IMF) vary almost periodically with an approximate life cycle of 11 years. It is shown that the solar and geomagnetic activity indices have complex behavior often characterizable as quasi-periodic or even chaotic, which causes the long-term prediction to be a conundrum. Moreover, solar and geomagnetic activity indices and their chaotic characteristics vary abruptly during solar and geomagnetic storms. This variation depicts the difficulties in modeling and long-term prediction of solar and geomagnetic storms. On the other hand, neural networks and related neurofuzzy tools as general function approximators have been the subjects of interest due to their many practical applications in modeling and predicting complex phenomena. However, most of these systems are trained by algorithms that need to be carried out by an off-line data set which influence their performance in prediction of time-varying solar and geomagnetic activity indices. This paper proposes an adaptive neurofuzzy approach with a recursive learning algorithm for modeling and prediction of space weather indices which fulfill requirements of prediction of time-varying solar and geomagnetic activities for long time spans. The obtained results depict the power of the proposed method in online prediction of time-varying solar and geomagnetic activity indices. Ó 2010 Elsevier Ltd. All rights reserved.
1. Introduction In recent years, the term ‘‘Space Weather” has achieved a great international scientific and public importance due to dependency of modern human life on satellites and other distributed facilities. During the last 60 years, human race has extended the borders of human’s civilization to outer space and become a spacefaring civilization. With robotic and manned spaceships, modern man has started to survey the Solar System and by this investigation, he has learned that he lives in the atmosphere of a time-varying dynamic (Mirmomeni & Lucas, 2008b), violent Sun that provides energy for life on the Earth, but also can cause catastrophe by making faults in satellites communication, transportation, and distributed power grids (Moldwin, 2008). Space weather can be understood as
* Corresponding author. Address: N. Kargar Avenue, after Al-e Ahmad Avenue, Campus II Fanni, School of Electrical and Computer Engineering, Tehran 1439957131, Iran. Tel.: +98 21 61114313; fax: +98 21 88778690. E-mail addresses: [email protected], [email protected] (M. Mirmomeni), [email protected], [email protected] (C. Lucas), [email protected] (B. Moshiri), [email protected] (B.N. Araabi). 0957-4174/$ - see front matter Ó 2010 Elsevier Ltd. All rights reserved. doi:10.1016/j.eswa.2010.05.059
a branch of science that will give new insights into the complex influences and effects of our Sun and other cosmic sources on interplanetary space, the Earth’s magnetosphere, ionosphere, and thermosphere that can influence the performance and reliability of space-borne and ground-based technological systems, and beyond that, on their endangering affects to life and health (Bothmer & Daglis, 2007; Moldwin, 2008). The Sun–Earth system is a very complicated system with a large variety of physical processes, ranging from magnetic field reconnection and accelerated solar wind as hot plasma to impact of charged particles on manmade electronic devices and biological systems. Fig. 1 schematically shows the complicated the Sun–Earth system. Effects of space weather disturbances on our space environment ranging from producing faults in spacecraft operations to disruptions of distributed electrical power systems to the manufacturing of precision equipment have been well documented for more than 35 years (Donnelly, 1979; Heckman, Marubashi, Shea, Smart, & Thompson, 1997; Hruska, Shea, Smart, & Heckman, 1993; Kane, 2006; Simon, Heckman, & Shea, 1986; Thompson et al., 1990). It has estimated that the space weather storms cause annual losses of the order of more than $100 million (Horwitz & Moore, 2000;
8268
M. Mirmomeni et al. / Expert Systems with Applications 37 (2010) 8267–8277
Fig. 1. Schematic view of the complex Sun–Earth system (Bothmer & Daglis, 2007).
Maynard, 1995). Fig. 2 depicts the known space weather effects on man-made technologies. Accurate predictions of space weather seems to be an urge for modern society, and modern society depends on understanding of space weather and climate (Vassiliadis, 2000; Weigel, Baker, Rigler, & Vassiliadis, 2004) for communication, agriculture, transportation, and natural disaster mitigation. It has to be said that predicting space weather phenomena is the ultimate challenge to a space physics model. It is one of the most challenging areas since, in addition to formulating and a test of model or theory, it is burdened with the complications of issuing a predictive statement with limited information and in advance of a phenomenon (Bothmer & Daglis, 2007). Space weather includes a large number of complicated electrodynamic and plasma physical phenomena and their effects man-made technologies. On the one hand, today the matured science of plasma physics makes it possible to describe many of complicated phenomena in space weather (Clauer et al., 2000). By taking advantage of modern super computers
and sophisticated solution-adaptive approaches, and fundamental advances in numerical methods, it is possible to build a predictive, physic based space weather model (Clauer et al., 2000; Daglis, Delcourt, Metallinou, & Kamide, 2004; De Zeeuw, Gombosi, Groth, Powell, & Stout, 2000; Mavromichalaki et al., 2005; Vassiliadis, 2000; Volberg et al., 2005). On the other hand, the solar wind– magnetosphere–ionosphere system exhibits coherence on the global scale and such behavior can arise from nonlinearity in the dynamics. In addition, it is shown that the solar and geomagnetic activity has chaotic characteristics (Mirmomeni & Lucas, 2008a) which vary during solar and geomagnetic storms (Mirmomeni & Lucas, 2008b). The variation of chaotic characteristics of solar and geomagnetic activities, especially during storms and substorms shows their time-varying dynamics which depicts the difficulties in long-term prediction of solar and geomagnetic activity indices (Gholipour, Lucas, Araabi, Mirmomeni, & Shafiee, 2007; Mirmomeni, Lucas, Araabi, & Shafiee, 2007; Stefanski, 2000). Furthermore, ionosphere (and magnetosphere) system as a part of satellites environment is a time-varying environment. This variation occurs both naturally and artificially (Canon, Angling, & Lundborg, 2002; Daglis, 2004). Natural variability occurs due to bulk (average) effects and that due to small-scale irregularities with sizes less than a Fresnel radius (Daglis, 2004). The ionosphere as a naturally occurring plasma environment, can be artificially change by several techniques such as releasing of large volumes of chemically reactive gases, uses high-power ground-based transmitters at LF or higher frequencies, or using charged particle accelerators (Daglis, 2004). Therefore, it is meaningful to use adaptive methods to model and predict space weather considering its time-varying characteristics and time-varying environments of satellites as main targets of space weather hazards. On the other hand, neural networks and neurofuzzy models which are general nonlinear function approximators have demonstrated good performance in the prediction of solar and geomag-
Fig. 2. Summary of the known space weather effects (Lanzerotti, Thomson, & Maclennan, 1997).
M. Mirmomeni et al. / Expert Systems with Applications 37 (2010) 8267–8277
netic activity indices (Gholipour, Araabi, & Lucas, 2006; Gholipour, Lucas, Araabi, & Shafiee, 2005; Mirmomeni, Lucas, & Moshiri, 2007; Mirmomeni, Shafiee, Lucas, & Araabi, 2006; Vassiliadis, 2000). The best way for system identification of such time-varying phenomena, is an online and recursive method. Many methods can be found in the literature for recursive tuning of the parameters of linear systems (Liu, 2001; Ljung, 1999; Nelles, 2001). Compared to other artificial neural networks, Locally Linear Neurofuzzy (LLNF) models are particularly well suited for online learning since they are capable of solving the so-called stability/plasticity dilemma (Deng & Kasabov, 2000). One of the most popular learning algorithms for LLNF models is LoLiMoT (Locally Linear Model Tree) which is an incremental learning algorithm to adjust the premise and consequence parameters of such models and is used in several applications (Gholipour et al., 2005; Gholipour et al., 2007; JaliliKharaajoo, Rahmati, & Rashidi, 2003; Mirmomeni, Lucas, & Moshiri, 2007). However, this algorithm needs to be carried out by an off-line data set. Moreover, LoLiMoT algorithm has some restrictions in its structure. For example, LoLiMoT is a growing and a one way algorithm and it is not possible to return to some former state during learning phase which is not suitable for a time-varying problem. In addition, using LoLiMoT algorithm for online applications is difficult to utilize directly, since its computational demand grows linearly with the number of training data samples (Gholipour et al., 2007; Nelles, 1996). In this paper, by using a recursive learning method for LLNF models based on the regular LoLiMoT algorithm, it is tried to predict time-varying solar and geomagnetic activity indices. This method is called Recursive Locally Linear Model Tree (RLoLiMoT). In addition, by using this recursive algorithm with its online learning nature, the performance of the system identification for such time-varying dynamic systems will be improved in comparison with other well-known methods in prediction of space weather indices. To show the advantage of this novel learning algorithm in predicting time-varying solar and geomagnetic activity indices, prediction of three solar and geomagnetic activity indices (Dst., Kp, and Plasma Speed) are considered as case studies. Simulation results depict the great performance of proposed method in describing and forecasting such complex and time-varying dynamics. The remaining sections of this paper are structured as follows. Section 2 briefly illustrates the main aspects of LLNF models and LoLiMoT algorithm. Section 3 describes the online learning methodology which is used for LLNF models to model nonlinear timevarying systems. Section 4 is devoted to show the performance of proposed method for tuning the parameters of the LLNF models in prediction of solar and geomagnetic activity indices. The last section contains the concluding remarks.
neurons, and xij denotes the LLM parameters of the ith neuron. The validity functions are chosen as normalized Gaussians; normalization is necessary for a proper interpretation of validity functions:
l ðuÞ j¼1 lj ðuÞ
/i ðuÞ ¼ PM i
ð3Þ
!! 1 ðu1 ci1 Þ2 ðup cip Þ2 li ðuÞ ¼ exp þ þ 2 r2i1 r2ip ! ! 1 ðu1 ci1 Þ2 1 ðup cip Þ2 ¼ exp exp 2 2 r2i1 r2ip
xi0 – 0; xi1 ¼ xi2 ¼ ¼ xip ¼ 0
^¼ y
^i /i ðuÞ y
ð1Þ ð2Þ
i¼1 T This structure is depicted in Fig. 3, where u ¼ ½ u1 u2 up is the model input, M is the number of Locally Linear Models (LLMs)
ð5Þ
LoLiMoT algorithm as an incremental tree based learning algorithm which is introduced in Nelles (2001) is appropriate for tuning the rule premise parameters, i.e., determining the validation hypercube for each locally linear model; however, this algorithm has some restrictions for system identification of time-varying processes in real time. LoLiMoT learning algorithm belongs to the class of growing strategies because it incorporates an additional rule in each iteration of the algorithm. During the training procedure which is done off-line, some of the formerly made divisions may become suboptimal or even superfluous. By extending LoLiMoT with a pruning strategy in this paper, which is able to merge formerly divided local linear models, this drawback can be remedied. In addition, for sys-
Neurofuzzy models which are originally proposed by Martin Brown and Chris Harris stands for a fuzzy system implemented using a neural network structure (Brown & Harris, 1995). The fundamental approach with LLNF models is dividing the input space into small linear subspaces with fuzzy validity functions. The total model is a neurofuzzy network and a linear neuron in the output layer which simply calculates the weighted sum of the outputs of locally linear neurons:
M X
ð4Þ
The M P parameters of the nonlinear hidden layer are the parameters of Gaussian validity functions: center (cij) and standard deviation (rij). Optimization or learning methods are used to adjust the two sets of parameters, the rule consequent parameters of the locally linear models (xij entries) and the rule premise parameters of validity functions (cij entries and rij entries). Global optimization of linear consequent parameters is simply obtained by least squares technique. It has to be said that LLMs are equivalent to Takagi–Sugeno neurofuzzy systems (Gholipour et al., 2006; Takagi & Sugeno, 1985) with linear output equations. They do not include all the possible rules, but have the capability of being trained by intuitive constructive learning algorithms. LLMs are equal to trainable Takagi– Sugeno fuzzy systems under some restrictions, and they include a large class of networks with basis functions. For example, normalized RBF can be interpreted as a special case of LLNF model when
2. Neurofuzzy modeling with LoLiMoT learning algorithm
^i ¼ xi0 þ xi1 u1 þ xi2 u2 þ þ xip up y
8269
Fig. 3. Structure of LLNF model.
8270
M. Mirmomeni et al. / Expert Systems with Applications 37 (2010) 8267–8277
tem identification of time-varying processes such as solar and geomagnetic activity indices in real time, the LoLiMoT algorithm is difficult to utilize directly since its computational demand grows linearly with the number of training data samples. Thus a recursive algorithm is required that needs constant computation time in order to guarantee execution within one (or a fixed number of) sampling interval(s) to be suitable for modeling and prediction of solar and geomagnetic activity indices which is the aim of this paper. 3. Recursive algorithm for modeling time-varying solar and geomagnetic dynamics This section is devoted to describe the novel online recursive learning method for LLMs based on ordinary LoLiMoT learning algorithm and k-means clustering algorithm modeling and prediction of time-varying solar and geomagnetic activity indices. First of all, a tree based learning algorithm for tuning the premise parameters of LLNF is given. After that, a Recursive Least Square (RLS) method is used to set the parameters of the consequent parts. In this paper, the k-means clustering algorithm is used in the proposed algorithm. The k-means clustering minimizes the following loss function:
I¼
C X N X j¼1
i¼1
2
lji kuðiÞ cj k ! min c
ð6Þ
j
where the index i runs over all elements of the sets Sj, C is the number of clusters, cj are the cluster centers (prototypes) and lji = 1 if the sample uðiÞ is associated (belongs) to cluster j and lji = 0. The sets Sj contain all indices of those data samples (out of all N) that belong to the cluster j, i.e., which are nearest to the cluster center cj . The cluster centers cj are the parameters that the clustering technique varies in order to minimize (6). The k-means algorithm used for tuning premise parameters (validity functions) of LLNF models in the proposed method is modified to be useful for online applications in the proposed method which is called Recursive Locally Linear Model Tree (RLoLiMoT) learning algorithm. The RLoLiMoT algorithm with k-means clustering technique for online nonlinear system identification applications works as follows: Step 1. Assigning current data sample to a proper cluster (locally linear model): because the number of clusters or locally linear models is not known a priori, the minimum distance between the current data sample and the clusters’ prototype are calculated. If this value is more than a threshold which is a designing parameter, a new cluster with a prototype as current data sample is created and if it is not, current data sample is assigned to the cluster with nearest cluster prototype to the data sample. It has to be emphasized that choosing a proper value for this tuning parameter is very important. Actually this parameter is similar to the vigilance factor of ART-networks in FLEXFIS learning algorithm which has been proposed in Lughofer and Klement (2005). A small value for this parameter is chosen before the algorithm goes to operation. The choice of small value for this parameter may create too many clusters or locally linear models. Then, the run time, this value in-
creases gradually to prevent the production of extra clusters. This approach is useful for real time applications. Step 2. Tuning the center of validity functions: If current data sample is assigned to a cluster, the centroid (mean) as the prototype of that cluster should be modified. The cluster prototype is modified as
cþj ¼
Nj cj þ uðiÞ Nj
;
Nþj ¼ Nj þ 1
ð7Þ
where uðiÞ is the current input data sample, and Nj is the number of those data samples that belong to cluster j, and cj is the cluster’s prototype. The modification rule has a recursive form to be useful for online applications. Step 3. Tuning the covariance matrix of the validity functions: after tuning the clusters’ prototype the covariance matrix of each validity functions is set as follows:
1 Rj ¼ diag minðcj ; ci 3 i–j
! ð8Þ
where Rj is the covariance matrix of validity function j (cluster j), and cj is the cluster prototype of the cluster j, and ci is the cluster prototype for other clusters. Step 4. Check for merging clusters: after tuning the parameters of the cluster’s prototype (mean of Gaussians validity functions and covariance matrixes), it is checked that if some clusters are close enough to merge to one cluster. If the distances between some clusters are smaller than a threshold which is a designing parameter, these clusters is merged to a single cluster and its prototype could be calculated as follows:
Pp N i ci cj ¼ Pi¼1 ; p i¼1 N i
Nj ¼
p X
Ni
ð9Þ
i¼1
where Ni is the number of data samples belong to those near classes i = 1, . . ., p which have to be merged to a class j; ci is the center of those near classes i = 1, . . ., p which have to be merged to a class j. A small value (but greater than the value used for former threshold parameter) for this parameter at initial stage is chosen. Choosing small value for this parameter restricts the merging mechanism. During the run time, this value increases gradually to increase the rate of merging. The covariance matrix of this validity function (cluster) could be tuned according to 3. Step 5. Using a local recursive weighted least square algorithm for tuning the consequence parts’ parameters: For an online adaptation of the rule consequent parameters, the local estimation approach is chosen. Assuming that the rule premises and thus the validity functions Ui are known, the following local recursive weighted least squares algorithm with exponential forgetting factor can be applied separately for each rule consequent i = 1, . . ., M:
Table 1 The RLoLiMoT algorithm for a given data sample uðiÞ. Step 1. Check if this data sample is near enough to a cluster according to a distance measure such as Euclidean distance or not. Step 2. If there is not any near cluster, create a new cluster whose its center of validity function is this data sample. The covariance matrix and the consequence parts’ parameters are chosen arbitrary. Wait for new data sample and go to step 1. Step 3. If there is a cluster near enough to this data sample, update the center of validity functions of this cluster. Step 4. Update the covariance matrix of validity functions of assigned cluster. Step 5. Check for merging present clusters by checking the distance between clusters. If some clusters are close enough, they have to be merged to a one cluster. The center and covariance matrix of new cluster have to be calculated according to steps 3 and 4. Step 6. Calculate the consequence parts’ parameters of clusters by a local recursive weighted least square algorithm.
M. Mirmomeni et al. / Expert Systems with Applications 37 (2010) 8267–8277
Fig. 4. The RLoLiMoT algorithm process.
8271
8272
M. Mirmomeni et al. / Expert Systems with Applications 37 (2010) 8267–8277
^ i ðkÞ ¼ w ^ i ðk 1Þ þ ci ðkÞei ðkÞ w T
~ ðkÞw ^ i ðk 1Þ ei ðkÞ ¼ yðkÞ u 1 ~ ðkÞ ci ðkÞ ¼ T P i ðk 1Þu ~ ðkÞPi ðk 1Þ þ ki u /i ðzi ðkÞÞ 1 ~ T ðkÞ Pi ðk 1Þ Pi ðkÞ ¼ I ci ðkÞu ki
4. Simulation results
ð10aÞ ð10bÞ
ð10cÞ
In comparison with u, the augmented consequent input vector ~ ¼ ½ 1 u1 up T additionally contains the regressor ‘‘1” for u adaptation of the offsets wi0. If prior knowledge is available, different forgetting factors ki and initial covariance matrixes Pi ð0Þ can be implemented for each LLM i. Table 1 provides a clear presentation of the proposed algorithm. Fig. 4 shows the proposed online algorithm ‘‘Recursive Locally Linear Model Tree algorithm (RLoLiMoT)”. One of the most important issues which should be considered for neurofuzzy models is the curse of dimensionality. This problem has been addressed several times (Harris, Hong, & Gon, 2002; Nelles, 1996). For LLNF models with RLoLiMoT learning algorithm, it has to be pointed out that the computational complexity problem has been the very motivating factor for adoption of suboptimal algorithms like Model Tree (MoT) in LLNF systems. The algorithm, despite its failure to guarantee Polynomial bounds, has the advantage that it incrementally increases the number of LLMs till the required quality is achieved. Our modification has the additional advantage that unnecessary complexities can also be removed through mergers. It thus achieves further computational economies while at the same time paying decreased price in terms of sub-optimality.
This section is devoted to show the performance of proposed adaptive LLNF model with proposed recursive LoLiMoT learning algorithm to model predict time-varying solar and geomagnetic activity dynamics. Three solar and geomagnetic activity indices are used for adaptive modeling and prediction in this paper: Dst. (Disturbance Storm Time) index as an important measure for predicting magnetic storms, Kp (‘‘Kennziffer planetarisch”) index, and Plasma Speed index as a Solar Wind Plasma parameter measured by various spacecrafts near the Earth’s orbit. All of these indices reflect severity of the geomagnetic storm and knowing them in advance will help satellite operators and some surface technologies avoid huge disasters. These solar activity indices are used from ‘‘OMNI 2” data set contains the hourly mean values of the interplanetary magnetic field (IMF) and solar wind plasma parameters measured by various
Table 2 Performance of the proposed adaptive LLNF model with RLoLiMoT learning algorithm in comparison with some well-known off-line methods in one step ahead prediction of daily Dst. index between 2000 and 2006. Method
NMSE for test sets of off-line methods
LLNF with LoLiMoT learning algorithm MLP with error back propagation learning algorithm RBF with error back propagation learning algorithm Adaptive LLNF with RLoLiMoT learning algorithm
0.5348 0.5724 0.5520 0.0968
Fig. 5. Performance of the proposed adaptive LLNF model with RLoLiMoT learning algorithm in one step ahead prediction of Dst. index. Upper left: predicted and real values of Dst. index between 2000 and 2006; upper right: predicted and real values of Dst. index during 2004; lower left: prediction error of proposed adaptive LLNF with RLoLiMoT algorithm; lower right: prediction error of proposed adaptive LLNF model in comparison with other off-line methods during 2004.
M. Mirmomeni et al. / Expert Systems with Applications 37 (2010) 8267–8277
spacecraft near the Earth’s orbit, as well as geomagnetic and solar activity indices, and energetic proton fluxes. The OMNI data sets characterize the state of the solar wind near the magnetosphere. All versions have magnetic field and plasma data from each of mul-
8273
tiple spacecraft, where the plasma densities and temperatures have been cross-normalized. OMNI 2 also has energetic particle fluxes and geomagnetic and solar activity indices. This data set was created at NSSDC in 2003 as a successor to the OMNI data
Fig. 6. Predicted Dst. values vs. observed values between 2000 and 2006.
Fig. 7. Performance of the proposed adaptive LLNF model with RLoLiMoT learning algorithm in one step ahead prediction of Kp index. Upper left: predicted and real values of Kp index between 2000 and October 2008; upper right: predicted and real values of Dst. index during 2004; lower left: prediction error of proposed adaptive LLNF with RLoLiMoT algorithm; lower right: prediction error of proposed adaptive LLNF model in comparison with other off-line methods during 2004.
8274
M. Mirmomeni et al. / Expert Systems with Applications 37 (2010) 8267–8277
set first created in the mid-1970s. A detailed discussion of the creation OMNI 2 is at http://www.nssdc.gsfc.nasa.gov/omniweb/. 4.1. Case study 1: adaptive prediction of Dst. index The geomagnetic response associated with the symmetric ring current has an average amplitude represented by the geomagnetic index Dst. (Burton, McPherron, & Russell, 1975; Vassiliadis, 2000). The Dst. index has been designed to measure the geomagnetic effect of the symmetric part of the ring current (Kivelson & Russell, 1995; Vassiliadis, 2000) and is a rough measure of mid-latitude, and even global, magnetospheric activity level. Therefore, the Dst. index is an important measure for geomagnetic activity and is used for prediction by many researchers (Gleisner & Lundstedt, 1997; O’Brien and McPherron, 2000; Vassiliadis, Klimas, Valdivia, & Baker, 2000). In this paper, it is tried to predict the daily average of the Dst. index during 2000–2006. The threshold parameter for splitting in RLoLiMoT algorithm is set to 0.2 in this case study and the threshold parameter for merging is set to 0.15. To evaluate the performance of adaptive LLNF model with RLoLiMoT learning algorithm in modeling and prediction of Dst. index, the results of one step ahead prediction of Dst. index is compared with three well-known off-line methods: LLNF with LoLiMoT learning algorithm, MLP (Multi Layered Perceptron), and RBF (Radial Basis Function) neural networks with error back
propagation learning algorithms. Fig. 5 shows the performance of the RLoLiMoT algorithm in one step ahead prediction of time-varying Dst. index. It can be seen that in the beginning of the RLoLiMoT algorithm’s operation, the performance of the adaptive LLNF model is poor, but after a short time, the performance of the model increases rapidly and remains almost constant even during major magnetic storms in 2004. It can be seen that compared to proposed algorithm, the performance of other off-line methods in one step ahead prediction of Dst. index varies during time. In fact, it is obvious that these methods have poor performance during major magnetic storms in 2004. Table 2 contains the results of several methods for one step ahead prediction of Dst. index. It is obvious that the performance of the adaptive LLNF model is superior in comparison with other well-known methods. The error index used in table one is Normalized Mean Square Error (NMSE) which is defined as
Pn NMSE ¼
i¼1 ðy Pn i¼1 ðy
^ Þ2 y Þ2 y
! ð11Þ
^; and y are observed data, predicted data, and average of where y; y observed data, respectively. Note that just an average estimation of data gives a NMSE of 1 (Gholipour et al., 2006). Fig. 6 shows predicted Dst. values vs. measured values between 2000 and 2006. 4.2. Case study 2: adaptive prediction of Kp index
Table 3 Performance of the proposed adaptive LLNF model with RLoLiMoT learning algorithm in comparison with some well-known off-line methods in one step ahead prediction of daily Kp index between 2000 and October 2008. Method
NMSE for test sets of off-line methods
LLNF with LoLiMoT learning algorithm MLP with error back propagation learning algorithm RBF with error back propagation learning algorithm Adaptive LLNF with RLoLiMoT learning algorithm
0.5918 0.7811 0.8371 0.0888
The planetary geomagnetic index, Kp, expresses the average amplitude of the horizontal components obtained from 13 selected observatories situated at mid-latitudes between 45° and 60° (Mayaud, 1980). Comparable with the ‘‘Richter–Skala” characterizing earthquake magnitudes, the overall geomagnetic activity index Kp is a quasi-logarithmic scale in 3 h intervals from 0 to 9 in thirds of a unit (totally 27 discrete values):
00 ; 0þ ; 1 ; 10 ; 1þ ; . . . ; 8 ; 80 ; 8þ ; 9 ; 90 where 0 means field amplitude less than 5 nT and 9 shows a geomagnetic activity more than 500 nT (Bothmer & Daglis, 2007). Kp
Fig. 8. Predicted Kp values vs. observed values between 2000 and October 2008.
8275
M. Mirmomeni et al. / Expert Systems with Applications 37 (2010) 8267–8277
index is a good measure for the satellites’ warning and alert systems. This geomagnetic index has been used as a surrogate for short-term atmosphere heating caused by geomagnetic disturbances. The spacecraft drag, latitude losing in low earth orbit satellites, is an important problem as a result of atmospheric heating, which occurs normally at Kp values more than 6. Some spacecrafts use the Earth’s magnetic field as an aid in orientation, or as a force to dump momentum. The geomagnetic disturbances with Kp values exceeding 5 normally cause miss-orientations. Finally, surface charging effects at Kp values between 4 and 5 have long-term effects on satellites. Considering all of these, predicting the Kp index is very important for geomagnetic K-index warning and alert systems. In this case study, it is tried to predict the daily average of the Kp index during 2000 to October 2008. As it said before, Kp index is used from ‘‘OMNI 2” data set. In this data set, Kp was treated specially. After determining daily or 27-averages (The 27-day averages are for discrete Bartels rotation numbers) from basic values such as 10 (1), 13 (1+), 17 (2), 20 (2), the average was rounded to the nearest ‘‘standard value” of Kp (i.e., 10, 13, 17, 20, . . .). In this case study, the threshold parameter for splitting in RLoLiMoT algorithm is set to 0.15 and the threshold parameter for merging is set to 0.05. Like Dst. index, the performance of adaptive LLNF model with RLoLiMoT learning algorithm in one step ahead prediction of daily Kp index is compared with LLNF with LoLiMoT learning algorithm, MLP, and RBF neural networks. Fig. 7 shows the performance of the RLoLiMoT algorithm in one step ahead prediction of time-varying Kp index. Like former case study, in the beginning of the RLoLiMoT algorithm’s operation, the performance of the adaptive LLNF model is poor, but after a short time, the performance of the model increases rapidly and remains almost constant even during major magnetic storms in 2004.
Table 3 contains the results of several methods for one step ahead prediction of Kp Index. It is obvious that the performance of the adaptive LLNF model is superb in comparison with other well-known methods. Fig. 8 shows predicted Kp values vs. measured values between 2000 and October 2008. 4.3. Case study 3: adaptive prediction of plasma speed The supersonic solar wind plasma flow continuously has harmful effects on the Earth’s magnetic field. Superimposed on this quasi-steady flow of the solar wind and energetic particles are transient solar wind and particle flows in which the solar wind can blow against the Earth’s magnetosphere with speed more than 2000 km/s and particle energies up to GeV range as well as large solar flares, sporadic emissions of EM radiation, that can lead to short-term disturbances on the Earth’s atmospheric conditions
Table 4 Performance of the proposed adaptive LLNF model with RLoLiMoT learning algorithm in comparison with some well-known off-line methods in one step ahead prediction of daily solar wind plasma speed between 2000 and October 2008. Method
NMSE for test sets of off-line methods
LLNF with LoLiMoT learning algorithm MLP with error back propagation learning algorithm RBF with error back propagation learning algorithm Adaptive LLNF with RLoLiMoT learning algorithm
0.3208 0.3665 0.3981 0.0735
Fig. 9. Performance of the proposed adaptive LLNF model with RLoLiMoT learning algorithm in one step ahead prediction of solar wind plasma speed. Upper left: predicted and real values of solar wind plasma speed index between 2000 and October 2008; upper right: predicted and real values of solar wind plasma speed during 2004; lower left: prediction error of proposed adaptive LLNF with RLoLiMoT algorithm; lower right: prediction error of proposed adaptive LLNF model in comparison with other off-line methods during 2004.
8276
M. Mirmomeni et al. / Expert Systems with Applications 37 (2010) 8267–8277
Fig. 10. Predicted solar wind plasma speed values vs. observed values between 2000 and October 2008.
(Bothmer & Daglis, 2007). In this case study, modeling and prediction of the solar wind plasma speed as one of the drivers of geomagnetic activity is considered. The threshold parameter for splitting in RLoLiMoT algorithm is set to 0.3 and the threshold parameter for merging is set to 0.2. Like Dst. and Kp indices, the performance of adaptive LLNF model with RLoLiMoT learning algorithm in one step ahead prediction of daily plasma speed is compared with LLNF, MLP, and RBF neural networks. Fig. 9 shows the performance of the RLoLiMoT algorithm in one step ahead prediction of time-varying solar wind plasma speed. Like former case studies, in the beginning of the RLoLiMoT algorithm’s operation, the performance of the adaptive LLNF model is poor, but after a short time, the performance of the model increases rapidly and remains almost constant even during major magnetic storms in 2004. Table 4 contains the results of several methods for one step ahead prediction of solar wind plasma. It is obvious that the performance of the adaptive LLNF model is excellent in comparison with other off-line methods. Fig. 10 shows predicted plasma speed values vs. measured values between 2000 and October 2008.
input space. By using proposed adaptive neurofuzzy model, an accurate prediction is provided which can describe time-varying solar and geomagnetic activity indices even during geomagnetic storms and sub-storms. The performance of the proposed adaptive LLNF models with RLoLiMoT learning algorithm is evaluated for modeling and prediction of three solar and geomagnetic activity indices: Dst., Kp, and solar wind plasma speed. Obtained results justify the potentials of the proposed adaptive neurofuzzy models for describing time-varying solar and geomagnetic activity indices in comparison with other well-known methods. The improved accuracy resulting from the retuning of the predictor, which can be important even in the case of stationary time series because new observations allow reduction of estimation errors, has been achieved with minimal additional computational burden due to the recursive nature of the proposed method. Acknowledgments The authors thank the ‘‘National Space Science Data Center” for using this data set. References
5. Discussion and conclusions This paper proposed an adaptive neurofuzzy model with a recursive learning algorithm for modeling and prediction of timevarying solar and geomagnetic activity indices. Although neural network and adaptive system modeling had begun as a single discipline, they later developed their separate subcultures. Learning and recall modes are strictly separated in the case of neural networks. Adaptive algorithms, on the other hand, carry out learning and decision making tasks at the same time. This, in adaptive control literature, has been designated as dual control. Relearning after every observation in online applications is very important especially in time varying or nonstationary, even slowly varying cases. However, it will be very time consuming if a recursive learning algorithm is not used. The algorithm proposed in this paper is basically an embedded version of LoLiMoT with recursive least squares adaptation of parameters, as well as online k-means clustering of
Bothmer, V., & Daglis, I. (2007). Space weather: Physics and effects. Berlin: Springer/ Praxies. Brown, M., & Harris, C. (1995). Neurofuzzy adaptive modeling and control. Englewood Cliffs, NJ: Prentice-Hall. Burton, R. K., McPherron, R. L., & Russell, C. T. (1975). An empirical relationship between interplanetary conditions and Dst. Journal of Geophysical Research, 80(31), 4204–4214. Canon, P. S., Angling, M. J., & Lundborg, B. (2002). Characterisation and modelling of the HF communications channel. In W. R. Stone (Ed.), Review of Radio Science 1999–2002 (pp. 597–623). IEEE-Wiley. Clauer, C. R., Gombosi, T. I., De Zeeuw, D. L., Ridley, A. J., Powell, K. G., Leer, B. V., et al. (2000). High performance computer methods applied to predictive space weather simulations. IEEE Transactions on Plasma Science, 28(6), 1931–1937. Daglis, I. A. (2004). Effects of space weather on technology infrastructure. The Netherland: Kluwer Academic Publishers. Daglis, I. A., Delcourt, D., Metallinou, F. A., & Kamide, Y. (2004). Particle acceleration in the frame of the storm–substorm relation. IEEE Transaction on Plasma Science, 32(4-1), 1449–1454. De Zeeuw, D. L., Gombosi, T. I., Groth, C. P. T., Powell, K. G., & Stout, Q. F. (2000). An adaptive MHD method for global space weather simulations. IEEE Transactions on Plasma Science, 28(6), 1956–1965.
M. Mirmomeni et al. / Expert Systems with Applications 37 (2010) 8267–8277 Deng, D., & Kasabov, N. (2000). Evolving self-organizing maps for online learning, data analysis and modeling. In Proceedings of the IJCNN’2000 neural networks, neural computing, new challenges perspectives new millennium, New York, NY (pp. 3–8). Donnelly, R. F. (Ed.). (1979). Solar-terrestrial predictions proceedings (4 volumes). Environment Research Laboratories, National Oceanic and Atmospheric Administration, US Department of Commerce, Boulder, Colorado. Gholipour, A., Araabi, B. N., & Lucas, C. (2006). Predicting chaotic time series using neural and neurofuzzy models: a comparative study. Neural Processing Letters, 24, 217–239. Gholipour, A., Lucas, C., Araabi, B. N., Mirmomeni, M., & Shafiee, M. (2007). Extracting the main patterns of natural time series for long-term neurofuzzy prediction. Neural Computing and Applications, 16(4–5), 383–393. Gholipour, A., Lucas, C., Araabi, B. N., & Shafiee, M. (2005). Solar activity forecast: spectral analysis and neurofuzzy prediction. Journal of Atmospheric and Solar Terrestrial Physics, 67, 595–603. Gleisner, H., & Lundstedt, H. (1997). Response of the auroral electrojets to the solar wind modeled with neural networks. Journal of Geophysical Research-Space Physics, 102(A7), 14269–14278. Harris, C., Hong & Gon, Q. (2002). Adaptive modelling, estimation and fusion from data: A neurofuzzy approach. London: Springer. Heckman, G. K., Marubashi, M. A., Shea, D. F., Smart, R., & Thompson (Eds.). (1997). Solar-terrestrial predictions – V. Proceedings of a Workshop at Hitachi, Japan, January 23–27, 1996, RWC Tokyo, Hiraiso Solar Terrestrial Research Center, Communications Research Laboratory, Hitachinaka, Ibaraki, Japan. Horwitz, J. L., & Moore, T. E. (2000). Core plasmas in space weather regions. IEEE Transactions on Plasma Science, 28(6), 18840–21853. Hruska, J, Shea, M. A., Smart, D. F., & Heckman, G. (Eds.). (1993), Solar-terrestrial predictions – IV (3 volumes). US Department of Commerce, NOAA, ERL, Boulder, Colorado. Jalili-Kharaajoo, M., Rahmati, A., & Rashidi, F. (2003). Internal motor control based on Local Linear Model Tree (LoLiMoT) model with application to a PH neutralization process. In Proceedings of IEEE (pp. 3051–3055). Kane, R. P. (2006). The idea of space weather – A historical perspective. Advances in Space Research, 37, 1261–1264. Kivelson, M., & Russell, C. T. (1995). Introduction to space physics. Cambridge, UK. Lanzerotti, L. J., Thomson, D. J., & Maclennan, C. G. (1997). Wireless at high altitudes: Environmental effects on space-based assets. Bell Labs Technical Journals. Liu, G. P. (2001). Nonlinear identification and control: a neural network approach. London: Springer. Ljung, L. (1999). System identification: Theory for the user. UK: Prentice-Hall. Lughofer, E., & Klement, E. P. (2005). FLEXFIS: A variant for incremental learning of Takagi–Sugeno fuzzy systems. In IEEE international conference on fuzzy systems (pp. 915–920). Mavromichalaki, H., Gerontidou, M., Mariatos, G., Plainaki, C., Papaioannou, A., Sarlanis, C., et al. (2005). Space weather forecasting at the new Athens center: The recent extreme events of January 2005. IEEE Transactions on Nuclear Science, 52(6), 2307–2312. Dec.. Mayaud, P. N. (1980). Derivation, meaning, and use of geomagnetic indices. American Geophysical Union. Maynard, N. C. (1995). Space weather prediction. Rev. Geophys., US Nat. Rep. IUGG, 547.
8277
Mirmomeni, M., Lucas, & Moshiri, B. (2007). Long-term prediction of chaotic time series with the aid of neurofuzzy models, spectral analysis and correlation analysis. In Proceedings of international joint conference on neural networks, Orlando, Florida, USA, August 12–17. Mirmomeni, M., & Lucas, C. (2008a). Model based method for determination the minimum embedding dimension from solar activity chaotic time series. International Journal of Engineering, Transactions A: Basics, 21(1), 31–44. Mirmomeni, M., & Lucas, C. (2008b). Analyzing the variation of embedding dimension of solar and geomagnetic activity indices during geomagnetic storm time. Earth Planets Space, 60, 1–11. Mirmomeni, M., Lucas, C., Araabi, B. N., & Shafiee, M. (2007). Forecasting sunspot numbers with the aid of fuzzy descriptor models. Space Weather, International Journal of Research and Applications. doi:10.1029/2006SW000289. Mirmomeni, M., Shafiee, M., Lucas, C., & Araabi, B. N. (2006). Introducing a new learning method for fuzzy descriptor systems with the aid of spectral analysis to forecast solar activity. Journal of Atmospheric and Solar-Terrestrial Physics, 68, 2061–2074. Moldwin, M. B. (2008). An introduction to space weather. Cambridge: Cambridge University Press. Nelles, O. (1996). Local linear model tree for on-line identification of time-variant nonlinear dynamic systems. In Proceedings of the international conference on artificial neural networks (ICANN), Bochum, Germany (pp. 115–120). Nelles, O. (2001). Nonlinear system identification. Berlin: Springer. O’Brien, T. P., & McPherron, R. L. (2000). Forecasting the ring current index Dst in real time. Journal of Atmospheric and Solar-Terrestrial Physics, 62(14), 1295–1299. Simon, P. A., Heckman, G., & Shea, M. A. (Eds.). (1986). Solar-terrestrial predictions. Proceedings of a workshop at Meudon, France, June l8–22, 1984, Environment Research Laboratories, National Oceanic and Atmospheric Administration, US Department of Commerce, Boulder, Colorado. Stefanski, A. (2000). Estimation of the largest Lyapunov exponent in systems with impacts. Chaos, Solitons and Fractals 11, 2443–2451. Takagi, T., & Sugeno, M. (1985). Fuzzy identification of systems and its applications to modeling and control. IEEE Transactions on System, Man, and Cybernetics, 15, 116–132. Thompson, R. J., Cole, D. G., Wilkinson, P. J., Shea, M. A., Smart, D., & Heckman, G. (Eds.). (1990). Solar-terrestrial predictions. Proceedings of a Workshop at Leura, Australia (2 volumes), US Department of Commerce, NOAA, ERL, Boulder, Colorado. Vassiliadis, D. (2000). System identification, modeling, and prediction for space weather environments. IEEE Transactions on Plasma Science, 28(6), 1944–1955. Vassiliadis, D., Klimas, A. J., Valdivia, J. A., & Baker, D. N. (2000). The nonlinear 12 dynamics of space weather. Space Weather: Physics and Applications, 26, 197–207. Volberg, O., Toth, G., Gombosi, T. I., Stout, Q. F., Powell, K. G., De Zeeuw, D., et al. (2005). A high-performance framework for Sun-to-Earth space weather modeling. In 19th IEEE international conference on parallel and distributed processing symposium, 4–8 April (p. 8). Weigel, R. S., Baker, D. N., Rigler, E. J., & Vassiliadis, D. (2004). Predictability of large geomagnetic disturbances based on solar wind conditions. IEEE Transactions on Plasma Science, 28(6), 1506–1510.
Contents lists available at ScienceDirect
Expert Systems with Applications journal homepage: www.elsevier.com/locate/eswa
Introducing adaptive neurofuzzy modeling with online learning method for prediction of time-varying solar and geomagnetic activity indices Masoud Mirmomeni a,*, Caro Lucas a,b, Behzad Moshiri a, Babak Nadjar Araabi a,b a b
Control and Intelligent Processing Center of Excellence, School of Electrical and Computer Engineering, University College of Engineering, University of Tehran, Tehran, Iran School of Cognitive Sciences, Institute for Studies in Theoretical Physics and Mathematics, Tehran, Iran
a r t i c l e
i n f o
Keywords: Space weather Solar activity Geomagnetic disturbance Online learning Recursive method Neurofuzzy modeling
a b s t r a c t Research in space weather has in recent years become an active field of research requiring international cooperation because of its importance in hazard warning especially for satellite technology and power utility systems. The time-varying sun as the main source of space weather impacts the Earth’s magnetosphere by emitting hot magnetized plasma called solar wind into interplanetary space. The emission of Solar Energetic Particles (SEPs) and consequently the magnitude of Interplanetary Magnetic Field (IMF) vary almost periodically with an approximate life cycle of 11 years. It is shown that the solar and geomagnetic activity indices have complex behavior often characterizable as quasi-periodic or even chaotic, which causes the long-term prediction to be a conundrum. Moreover, solar and geomagnetic activity indices and their chaotic characteristics vary abruptly during solar and geomagnetic storms. This variation depicts the difficulties in modeling and long-term prediction of solar and geomagnetic storms. On the other hand, neural networks and related neurofuzzy tools as general function approximators have been the subjects of interest due to their many practical applications in modeling and predicting complex phenomena. However, most of these systems are trained by algorithms that need to be carried out by an off-line data set which influence their performance in prediction of time-varying solar and geomagnetic activity indices. This paper proposes an adaptive neurofuzzy approach with a recursive learning algorithm for modeling and prediction of space weather indices which fulfill requirements of prediction of time-varying solar and geomagnetic activities for long time spans. The obtained results depict the power of the proposed method in online prediction of time-varying solar and geomagnetic activity indices. Ó 2010 Elsevier Ltd. All rights reserved.
1. Introduction In recent years, the term ‘‘Space Weather” has achieved a great international scientific and public importance due to dependency of modern human life on satellites and other distributed facilities. During the last 60 years, human race has extended the borders of human’s civilization to outer space and become a spacefaring civilization. With robotic and manned spaceships, modern man has started to survey the Solar System and by this investigation, he has learned that he lives in the atmosphere of a time-varying dynamic (Mirmomeni & Lucas, 2008b), violent Sun that provides energy for life on the Earth, but also can cause catastrophe by making faults in satellites communication, transportation, and distributed power grids (Moldwin, 2008). Space weather can be understood as
* Corresponding author. Address: N. Kargar Avenue, after Al-e Ahmad Avenue, Campus II Fanni, School of Electrical and Computer Engineering, Tehran 1439957131, Iran. Tel.: +98 21 61114313; fax: +98 21 88778690. E-mail addresses: [email protected], [email protected] (M. Mirmomeni), [email protected], [email protected] (C. Lucas), [email protected] (B. Moshiri), [email protected] (B.N. Araabi). 0957-4174/$ - see front matter Ó 2010 Elsevier Ltd. All rights reserved. doi:10.1016/j.eswa.2010.05.059
a branch of science that will give new insights into the complex influences and effects of our Sun and other cosmic sources on interplanetary space, the Earth’s magnetosphere, ionosphere, and thermosphere that can influence the performance and reliability of space-borne and ground-based technological systems, and beyond that, on their endangering affects to life and health (Bothmer & Daglis, 2007; Moldwin, 2008). The Sun–Earth system is a very complicated system with a large variety of physical processes, ranging from magnetic field reconnection and accelerated solar wind as hot plasma to impact of charged particles on manmade electronic devices and biological systems. Fig. 1 schematically shows the complicated the Sun–Earth system. Effects of space weather disturbances on our space environment ranging from producing faults in spacecraft operations to disruptions of distributed electrical power systems to the manufacturing of precision equipment have been well documented for more than 35 years (Donnelly, 1979; Heckman, Marubashi, Shea, Smart, & Thompson, 1997; Hruska, Shea, Smart, & Heckman, 1993; Kane, 2006; Simon, Heckman, & Shea, 1986; Thompson et al., 1990). It has estimated that the space weather storms cause annual losses of the order of more than $100 million (Horwitz & Moore, 2000;
8268
M. Mirmomeni et al. / Expert Systems with Applications 37 (2010) 8267–8277
Fig. 1. Schematic view of the complex Sun–Earth system (Bothmer & Daglis, 2007).
Maynard, 1995). Fig. 2 depicts the known space weather effects on man-made technologies. Accurate predictions of space weather seems to be an urge for modern society, and modern society depends on understanding of space weather and climate (Vassiliadis, 2000; Weigel, Baker, Rigler, & Vassiliadis, 2004) for communication, agriculture, transportation, and natural disaster mitigation. It has to be said that predicting space weather phenomena is the ultimate challenge to a space physics model. It is one of the most challenging areas since, in addition to formulating and a test of model or theory, it is burdened with the complications of issuing a predictive statement with limited information and in advance of a phenomenon (Bothmer & Daglis, 2007). Space weather includes a large number of complicated electrodynamic and plasma physical phenomena and their effects man-made technologies. On the one hand, today the matured science of plasma physics makes it possible to describe many of complicated phenomena in space weather (Clauer et al., 2000). By taking advantage of modern super computers
and sophisticated solution-adaptive approaches, and fundamental advances in numerical methods, it is possible to build a predictive, physic based space weather model (Clauer et al., 2000; Daglis, Delcourt, Metallinou, & Kamide, 2004; De Zeeuw, Gombosi, Groth, Powell, & Stout, 2000; Mavromichalaki et al., 2005; Vassiliadis, 2000; Volberg et al., 2005). On the other hand, the solar wind– magnetosphere–ionosphere system exhibits coherence on the global scale and such behavior can arise from nonlinearity in the dynamics. In addition, it is shown that the solar and geomagnetic activity has chaotic characteristics (Mirmomeni & Lucas, 2008a) which vary during solar and geomagnetic storms (Mirmomeni & Lucas, 2008b). The variation of chaotic characteristics of solar and geomagnetic activities, especially during storms and substorms shows their time-varying dynamics which depicts the difficulties in long-term prediction of solar and geomagnetic activity indices (Gholipour, Lucas, Araabi, Mirmomeni, & Shafiee, 2007; Mirmomeni, Lucas, Araabi, & Shafiee, 2007; Stefanski, 2000). Furthermore, ionosphere (and magnetosphere) system as a part of satellites environment is a time-varying environment. This variation occurs both naturally and artificially (Canon, Angling, & Lundborg, 2002; Daglis, 2004). Natural variability occurs due to bulk (average) effects and that due to small-scale irregularities with sizes less than a Fresnel radius (Daglis, 2004). The ionosphere as a naturally occurring plasma environment, can be artificially change by several techniques such as releasing of large volumes of chemically reactive gases, uses high-power ground-based transmitters at LF or higher frequencies, or using charged particle accelerators (Daglis, 2004). Therefore, it is meaningful to use adaptive methods to model and predict space weather considering its time-varying characteristics and time-varying environments of satellites as main targets of space weather hazards. On the other hand, neural networks and neurofuzzy models which are general nonlinear function approximators have demonstrated good performance in the prediction of solar and geomag-
Fig. 2. Summary of the known space weather effects (Lanzerotti, Thomson, & Maclennan, 1997).
M. Mirmomeni et al. / Expert Systems with Applications 37 (2010) 8267–8277
netic activity indices (Gholipour, Araabi, & Lucas, 2006; Gholipour, Lucas, Araabi, & Shafiee, 2005; Mirmomeni, Lucas, & Moshiri, 2007; Mirmomeni, Shafiee, Lucas, & Araabi, 2006; Vassiliadis, 2000). The best way for system identification of such time-varying phenomena, is an online and recursive method. Many methods can be found in the literature for recursive tuning of the parameters of linear systems (Liu, 2001; Ljung, 1999; Nelles, 2001). Compared to other artificial neural networks, Locally Linear Neurofuzzy (LLNF) models are particularly well suited for online learning since they are capable of solving the so-called stability/plasticity dilemma (Deng & Kasabov, 2000). One of the most popular learning algorithms for LLNF models is LoLiMoT (Locally Linear Model Tree) which is an incremental learning algorithm to adjust the premise and consequence parameters of such models and is used in several applications (Gholipour et al., 2005; Gholipour et al., 2007; JaliliKharaajoo, Rahmati, & Rashidi, 2003; Mirmomeni, Lucas, & Moshiri, 2007). However, this algorithm needs to be carried out by an off-line data set. Moreover, LoLiMoT algorithm has some restrictions in its structure. For example, LoLiMoT is a growing and a one way algorithm and it is not possible to return to some former state during learning phase which is not suitable for a time-varying problem. In addition, using LoLiMoT algorithm for online applications is difficult to utilize directly, since its computational demand grows linearly with the number of training data samples (Gholipour et al., 2007; Nelles, 1996). In this paper, by using a recursive learning method for LLNF models based on the regular LoLiMoT algorithm, it is tried to predict time-varying solar and geomagnetic activity indices. This method is called Recursive Locally Linear Model Tree (RLoLiMoT). In addition, by using this recursive algorithm with its online learning nature, the performance of the system identification for such time-varying dynamic systems will be improved in comparison with other well-known methods in prediction of space weather indices. To show the advantage of this novel learning algorithm in predicting time-varying solar and geomagnetic activity indices, prediction of three solar and geomagnetic activity indices (Dst., Kp, and Plasma Speed) are considered as case studies. Simulation results depict the great performance of proposed method in describing and forecasting such complex and time-varying dynamics. The remaining sections of this paper are structured as follows. Section 2 briefly illustrates the main aspects of LLNF models and LoLiMoT algorithm. Section 3 describes the online learning methodology which is used for LLNF models to model nonlinear timevarying systems. Section 4 is devoted to show the performance of proposed method for tuning the parameters of the LLNF models in prediction of solar and geomagnetic activity indices. The last section contains the concluding remarks.
neurons, and xij denotes the LLM parameters of the ith neuron. The validity functions are chosen as normalized Gaussians; normalization is necessary for a proper interpretation of validity functions:
l ðuÞ j¼1 lj ðuÞ
/i ðuÞ ¼ PM i
ð3Þ
!! 1 ðu1 ci1 Þ2 ðup cip Þ2 li ðuÞ ¼ exp þ þ 2 r2i1 r2ip ! ! 1 ðu1 ci1 Þ2 1 ðup cip Þ2 ¼ exp exp 2 2 r2i1 r2ip
xi0 – 0; xi1 ¼ xi2 ¼ ¼ xip ¼ 0
^¼ y
^i /i ðuÞ y
ð1Þ ð2Þ
i¼1 T This structure is depicted in Fig. 3, where u ¼ ½ u1 u2 up is the model input, M is the number of Locally Linear Models (LLMs)
ð5Þ
LoLiMoT algorithm as an incremental tree based learning algorithm which is introduced in Nelles (2001) is appropriate for tuning the rule premise parameters, i.e., determining the validation hypercube for each locally linear model; however, this algorithm has some restrictions for system identification of time-varying processes in real time. LoLiMoT learning algorithm belongs to the class of growing strategies because it incorporates an additional rule in each iteration of the algorithm. During the training procedure which is done off-line, some of the formerly made divisions may become suboptimal or even superfluous. By extending LoLiMoT with a pruning strategy in this paper, which is able to merge formerly divided local linear models, this drawback can be remedied. In addition, for sys-
Neurofuzzy models which are originally proposed by Martin Brown and Chris Harris stands for a fuzzy system implemented using a neural network structure (Brown & Harris, 1995). The fundamental approach with LLNF models is dividing the input space into small linear subspaces with fuzzy validity functions. The total model is a neurofuzzy network and a linear neuron in the output layer which simply calculates the weighted sum of the outputs of locally linear neurons:
M X
ð4Þ
The M P parameters of the nonlinear hidden layer are the parameters of Gaussian validity functions: center (cij) and standard deviation (rij). Optimization or learning methods are used to adjust the two sets of parameters, the rule consequent parameters of the locally linear models (xij entries) and the rule premise parameters of validity functions (cij entries and rij entries). Global optimization of linear consequent parameters is simply obtained by least squares technique. It has to be said that LLMs are equivalent to Takagi–Sugeno neurofuzzy systems (Gholipour et al., 2006; Takagi & Sugeno, 1985) with linear output equations. They do not include all the possible rules, but have the capability of being trained by intuitive constructive learning algorithms. LLMs are equal to trainable Takagi– Sugeno fuzzy systems under some restrictions, and they include a large class of networks with basis functions. For example, normalized RBF can be interpreted as a special case of LLNF model when
2. Neurofuzzy modeling with LoLiMoT learning algorithm
^i ¼ xi0 þ xi1 u1 þ xi2 u2 þ þ xip up y
8269
Fig. 3. Structure of LLNF model.
8270
M. Mirmomeni et al. / Expert Systems with Applications 37 (2010) 8267–8277
tem identification of time-varying processes such as solar and geomagnetic activity indices in real time, the LoLiMoT algorithm is difficult to utilize directly since its computational demand grows linearly with the number of training data samples. Thus a recursive algorithm is required that needs constant computation time in order to guarantee execution within one (or a fixed number of) sampling interval(s) to be suitable for modeling and prediction of solar and geomagnetic activity indices which is the aim of this paper. 3. Recursive algorithm for modeling time-varying solar and geomagnetic dynamics This section is devoted to describe the novel online recursive learning method for LLMs based on ordinary LoLiMoT learning algorithm and k-means clustering algorithm modeling and prediction of time-varying solar and geomagnetic activity indices. First of all, a tree based learning algorithm for tuning the premise parameters of LLNF is given. After that, a Recursive Least Square (RLS) method is used to set the parameters of the consequent parts. In this paper, the k-means clustering algorithm is used in the proposed algorithm. The k-means clustering minimizes the following loss function:
I¼
C X N X j¼1
i¼1
2
lji kuðiÞ cj k ! min c
ð6Þ
j
where the index i runs over all elements of the sets Sj, C is the number of clusters, cj are the cluster centers (prototypes) and lji = 1 if the sample uðiÞ is associated (belongs) to cluster j and lji = 0. The sets Sj contain all indices of those data samples (out of all N) that belong to the cluster j, i.e., which are nearest to the cluster center cj . The cluster centers cj are the parameters that the clustering technique varies in order to minimize (6). The k-means algorithm used for tuning premise parameters (validity functions) of LLNF models in the proposed method is modified to be useful for online applications in the proposed method which is called Recursive Locally Linear Model Tree (RLoLiMoT) learning algorithm. The RLoLiMoT algorithm with k-means clustering technique for online nonlinear system identification applications works as follows: Step 1. Assigning current data sample to a proper cluster (locally linear model): because the number of clusters or locally linear models is not known a priori, the minimum distance between the current data sample and the clusters’ prototype are calculated. If this value is more than a threshold which is a designing parameter, a new cluster with a prototype as current data sample is created and if it is not, current data sample is assigned to the cluster with nearest cluster prototype to the data sample. It has to be emphasized that choosing a proper value for this tuning parameter is very important. Actually this parameter is similar to the vigilance factor of ART-networks in FLEXFIS learning algorithm which has been proposed in Lughofer and Klement (2005). A small value for this parameter is chosen before the algorithm goes to operation. The choice of small value for this parameter may create too many clusters or locally linear models. Then, the run time, this value in-
creases gradually to prevent the production of extra clusters. This approach is useful for real time applications. Step 2. Tuning the center of validity functions: If current data sample is assigned to a cluster, the centroid (mean) as the prototype of that cluster should be modified. The cluster prototype is modified as
cþj ¼
Nj cj þ uðiÞ Nj
;
Nþj ¼ Nj þ 1
ð7Þ
where uðiÞ is the current input data sample, and Nj is the number of those data samples that belong to cluster j, and cj is the cluster’s prototype. The modification rule has a recursive form to be useful for online applications. Step 3. Tuning the covariance matrix of the validity functions: after tuning the clusters’ prototype the covariance matrix of each validity functions is set as follows:
1 Rj ¼ diag minðcj ; ci 3 i–j
! ð8Þ
where Rj is the covariance matrix of validity function j (cluster j), and cj is the cluster prototype of the cluster j, and ci is the cluster prototype for other clusters. Step 4. Check for merging clusters: after tuning the parameters of the cluster’s prototype (mean of Gaussians validity functions and covariance matrixes), it is checked that if some clusters are close enough to merge to one cluster. If the distances between some clusters are smaller than a threshold which is a designing parameter, these clusters is merged to a single cluster and its prototype could be calculated as follows:
Pp N i ci cj ¼ Pi¼1 ; p i¼1 N i
Nj ¼
p X
Ni
ð9Þ
i¼1
where Ni is the number of data samples belong to those near classes i = 1, . . ., p which have to be merged to a class j; ci is the center of those near classes i = 1, . . ., p which have to be merged to a class j. A small value (but greater than the value used for former threshold parameter) for this parameter at initial stage is chosen. Choosing small value for this parameter restricts the merging mechanism. During the run time, this value increases gradually to increase the rate of merging. The covariance matrix of this validity function (cluster) could be tuned according to 3. Step 5. Using a local recursive weighted least square algorithm for tuning the consequence parts’ parameters: For an online adaptation of the rule consequent parameters, the local estimation approach is chosen. Assuming that the rule premises and thus the validity functions Ui are known, the following local recursive weighted least squares algorithm with exponential forgetting factor can be applied separately for each rule consequent i = 1, . . ., M:
Table 1 The RLoLiMoT algorithm for a given data sample uðiÞ. Step 1. Check if this data sample is near enough to a cluster according to a distance measure such as Euclidean distance or not. Step 2. If there is not any near cluster, create a new cluster whose its center of validity function is this data sample. The covariance matrix and the consequence parts’ parameters are chosen arbitrary. Wait for new data sample and go to step 1. Step 3. If there is a cluster near enough to this data sample, update the center of validity functions of this cluster. Step 4. Update the covariance matrix of validity functions of assigned cluster. Step 5. Check for merging present clusters by checking the distance between clusters. If some clusters are close enough, they have to be merged to a one cluster. The center and covariance matrix of new cluster have to be calculated according to steps 3 and 4. Step 6. Calculate the consequence parts’ parameters of clusters by a local recursive weighted least square algorithm.
M. Mirmomeni et al. / Expert Systems with Applications 37 (2010) 8267–8277
Fig. 4. The RLoLiMoT algorithm process.
8271
8272
M. Mirmomeni et al. / Expert Systems with Applications 37 (2010) 8267–8277
^ i ðkÞ ¼ w ^ i ðk 1Þ þ ci ðkÞei ðkÞ w T
~ ðkÞw ^ i ðk 1Þ ei ðkÞ ¼ yðkÞ u 1 ~ ðkÞ ci ðkÞ ¼ T P i ðk 1Þu ~ ðkÞPi ðk 1Þ þ ki u /i ðzi ðkÞÞ 1 ~ T ðkÞ Pi ðk 1Þ Pi ðkÞ ¼ I ci ðkÞu ki
4. Simulation results
ð10aÞ ð10bÞ
ð10cÞ
In comparison with u, the augmented consequent input vector ~ ¼ ½ 1 u1 up T additionally contains the regressor ‘‘1” for u adaptation of the offsets wi0. If prior knowledge is available, different forgetting factors ki and initial covariance matrixes Pi ð0Þ can be implemented for each LLM i. Table 1 provides a clear presentation of the proposed algorithm. Fig. 4 shows the proposed online algorithm ‘‘Recursive Locally Linear Model Tree algorithm (RLoLiMoT)”. One of the most important issues which should be considered for neurofuzzy models is the curse of dimensionality. This problem has been addressed several times (Harris, Hong, & Gon, 2002; Nelles, 1996). For LLNF models with RLoLiMoT learning algorithm, it has to be pointed out that the computational complexity problem has been the very motivating factor for adoption of suboptimal algorithms like Model Tree (MoT) in LLNF systems. The algorithm, despite its failure to guarantee Polynomial bounds, has the advantage that it incrementally increases the number of LLMs till the required quality is achieved. Our modification has the additional advantage that unnecessary complexities can also be removed through mergers. It thus achieves further computational economies while at the same time paying decreased price in terms of sub-optimality.
This section is devoted to show the performance of proposed adaptive LLNF model with proposed recursive LoLiMoT learning algorithm to model predict time-varying solar and geomagnetic activity dynamics. Three solar and geomagnetic activity indices are used for adaptive modeling and prediction in this paper: Dst. (Disturbance Storm Time) index as an important measure for predicting magnetic storms, Kp (‘‘Kennziffer planetarisch”) index, and Plasma Speed index as a Solar Wind Plasma parameter measured by various spacecrafts near the Earth’s orbit. All of these indices reflect severity of the geomagnetic storm and knowing them in advance will help satellite operators and some surface technologies avoid huge disasters. These solar activity indices are used from ‘‘OMNI 2” data set contains the hourly mean values of the interplanetary magnetic field (IMF) and solar wind plasma parameters measured by various
Table 2 Performance of the proposed adaptive LLNF model with RLoLiMoT learning algorithm in comparison with some well-known off-line methods in one step ahead prediction of daily Dst. index between 2000 and 2006. Method
NMSE for test sets of off-line methods
LLNF with LoLiMoT learning algorithm MLP with error back propagation learning algorithm RBF with error back propagation learning algorithm Adaptive LLNF with RLoLiMoT learning algorithm
0.5348 0.5724 0.5520 0.0968
Fig. 5. Performance of the proposed adaptive LLNF model with RLoLiMoT learning algorithm in one step ahead prediction of Dst. index. Upper left: predicted and real values of Dst. index between 2000 and 2006; upper right: predicted and real values of Dst. index during 2004; lower left: prediction error of proposed adaptive LLNF with RLoLiMoT algorithm; lower right: prediction error of proposed adaptive LLNF model in comparison with other off-line methods during 2004.
M. Mirmomeni et al. / Expert Systems with Applications 37 (2010) 8267–8277
spacecraft near the Earth’s orbit, as well as geomagnetic and solar activity indices, and energetic proton fluxes. The OMNI data sets characterize the state of the solar wind near the magnetosphere. All versions have magnetic field and plasma data from each of mul-
8273
tiple spacecraft, where the plasma densities and temperatures have been cross-normalized. OMNI 2 also has energetic particle fluxes and geomagnetic and solar activity indices. This data set was created at NSSDC in 2003 as a successor to the OMNI data
Fig. 6. Predicted Dst. values vs. observed values between 2000 and 2006.
Fig. 7. Performance of the proposed adaptive LLNF model with RLoLiMoT learning algorithm in one step ahead prediction of Kp index. Upper left: predicted and real values of Kp index between 2000 and October 2008; upper right: predicted and real values of Dst. index during 2004; lower left: prediction error of proposed adaptive LLNF with RLoLiMoT algorithm; lower right: prediction error of proposed adaptive LLNF model in comparison with other off-line methods during 2004.
8274
M. Mirmomeni et al. / Expert Systems with Applications 37 (2010) 8267–8277
set first created in the mid-1970s. A detailed discussion of the creation OMNI 2 is at http://www.nssdc.gsfc.nasa.gov/omniweb/. 4.1. Case study 1: adaptive prediction of Dst. index The geomagnetic response associated with the symmetric ring current has an average amplitude represented by the geomagnetic index Dst. (Burton, McPherron, & Russell, 1975; Vassiliadis, 2000). The Dst. index has been designed to measure the geomagnetic effect of the symmetric part of the ring current (Kivelson & Russell, 1995; Vassiliadis, 2000) and is a rough measure of mid-latitude, and even global, magnetospheric activity level. Therefore, the Dst. index is an important measure for geomagnetic activity and is used for prediction by many researchers (Gleisner & Lundstedt, 1997; O’Brien and McPherron, 2000; Vassiliadis, Klimas, Valdivia, & Baker, 2000). In this paper, it is tried to predict the daily average of the Dst. index during 2000–2006. The threshold parameter for splitting in RLoLiMoT algorithm is set to 0.2 in this case study and the threshold parameter for merging is set to 0.15. To evaluate the performance of adaptive LLNF model with RLoLiMoT learning algorithm in modeling and prediction of Dst. index, the results of one step ahead prediction of Dst. index is compared with three well-known off-line methods: LLNF with LoLiMoT learning algorithm, MLP (Multi Layered Perceptron), and RBF (Radial Basis Function) neural networks with error back
propagation learning algorithms. Fig. 5 shows the performance of the RLoLiMoT algorithm in one step ahead prediction of time-varying Dst. index. It can be seen that in the beginning of the RLoLiMoT algorithm’s operation, the performance of the adaptive LLNF model is poor, but after a short time, the performance of the model increases rapidly and remains almost constant even during major magnetic storms in 2004. It can be seen that compared to proposed algorithm, the performance of other off-line methods in one step ahead prediction of Dst. index varies during time. In fact, it is obvious that these methods have poor performance during major magnetic storms in 2004. Table 2 contains the results of several methods for one step ahead prediction of Dst. index. It is obvious that the performance of the adaptive LLNF model is superior in comparison with other well-known methods. The error index used in table one is Normalized Mean Square Error (NMSE) which is defined as
Pn NMSE ¼
i¼1 ðy Pn i¼1 ðy
^ Þ2 y Þ2 y
! ð11Þ
^; and y are observed data, predicted data, and average of where y; y observed data, respectively. Note that just an average estimation of data gives a NMSE of 1 (Gholipour et al., 2006). Fig. 6 shows predicted Dst. values vs. measured values between 2000 and 2006. 4.2. Case study 2: adaptive prediction of Kp index
Table 3 Performance of the proposed adaptive LLNF model with RLoLiMoT learning algorithm in comparison with some well-known off-line methods in one step ahead prediction of daily Kp index between 2000 and October 2008. Method
NMSE for test sets of off-line methods
LLNF with LoLiMoT learning algorithm MLP with error back propagation learning algorithm RBF with error back propagation learning algorithm Adaptive LLNF with RLoLiMoT learning algorithm
0.5918 0.7811 0.8371 0.0888
The planetary geomagnetic index, Kp, expresses the average amplitude of the horizontal components obtained from 13 selected observatories situated at mid-latitudes between 45° and 60° (Mayaud, 1980). Comparable with the ‘‘Richter–Skala” characterizing earthquake magnitudes, the overall geomagnetic activity index Kp is a quasi-logarithmic scale in 3 h intervals from 0 to 9 in thirds of a unit (totally 27 discrete values):
00 ; 0þ ; 1 ; 10 ; 1þ ; . . . ; 8 ; 80 ; 8þ ; 9 ; 90 where 0 means field amplitude less than 5 nT and 9 shows a geomagnetic activity more than 500 nT (Bothmer & Daglis, 2007). Kp
Fig. 8. Predicted Kp values vs. observed values between 2000 and October 2008.
8275
M. Mirmomeni et al. / Expert Systems with Applications 37 (2010) 8267–8277
index is a good measure for the satellites’ warning and alert systems. This geomagnetic index has been used as a surrogate for short-term atmosphere heating caused by geomagnetic disturbances. The spacecraft drag, latitude losing in low earth orbit satellites, is an important problem as a result of atmospheric heating, which occurs normally at Kp values more than 6. Some spacecrafts use the Earth’s magnetic field as an aid in orientation, or as a force to dump momentum. The geomagnetic disturbances with Kp values exceeding 5 normally cause miss-orientations. Finally, surface charging effects at Kp values between 4 and 5 have long-term effects on satellites. Considering all of these, predicting the Kp index is very important for geomagnetic K-index warning and alert systems. In this case study, it is tried to predict the daily average of the Kp index during 2000 to October 2008. As it said before, Kp index is used from ‘‘OMNI 2” data set. In this data set, Kp was treated specially. After determining daily or 27-averages (The 27-day averages are for discrete Bartels rotation numbers) from basic values such as 10 (1), 13 (1+), 17 (2), 20 (2), the average was rounded to the nearest ‘‘standard value” of Kp (i.e., 10, 13, 17, 20, . . .). In this case study, the threshold parameter for splitting in RLoLiMoT algorithm is set to 0.15 and the threshold parameter for merging is set to 0.05. Like Dst. index, the performance of adaptive LLNF model with RLoLiMoT learning algorithm in one step ahead prediction of daily Kp index is compared with LLNF with LoLiMoT learning algorithm, MLP, and RBF neural networks. Fig. 7 shows the performance of the RLoLiMoT algorithm in one step ahead prediction of time-varying Kp index. Like former case study, in the beginning of the RLoLiMoT algorithm’s operation, the performance of the adaptive LLNF model is poor, but after a short time, the performance of the model increases rapidly and remains almost constant even during major magnetic storms in 2004.
Table 3 contains the results of several methods for one step ahead prediction of Kp Index. It is obvious that the performance of the adaptive LLNF model is superb in comparison with other well-known methods. Fig. 8 shows predicted Kp values vs. measured values between 2000 and October 2008. 4.3. Case study 3: adaptive prediction of plasma speed The supersonic solar wind plasma flow continuously has harmful effects on the Earth’s magnetic field. Superimposed on this quasi-steady flow of the solar wind and energetic particles are transient solar wind and particle flows in which the solar wind can blow against the Earth’s magnetosphere with speed more than 2000 km/s and particle energies up to GeV range as well as large solar flares, sporadic emissions of EM radiation, that can lead to short-term disturbances on the Earth’s atmospheric conditions
Table 4 Performance of the proposed adaptive LLNF model with RLoLiMoT learning algorithm in comparison with some well-known off-line methods in one step ahead prediction of daily solar wind plasma speed between 2000 and October 2008. Method
NMSE for test sets of off-line methods
LLNF with LoLiMoT learning algorithm MLP with error back propagation learning algorithm RBF with error back propagation learning algorithm Adaptive LLNF with RLoLiMoT learning algorithm
0.3208 0.3665 0.3981 0.0735
Fig. 9. Performance of the proposed adaptive LLNF model with RLoLiMoT learning algorithm in one step ahead prediction of solar wind plasma speed. Upper left: predicted and real values of solar wind plasma speed index between 2000 and October 2008; upper right: predicted and real values of solar wind plasma speed during 2004; lower left: prediction error of proposed adaptive LLNF with RLoLiMoT algorithm; lower right: prediction error of proposed adaptive LLNF model in comparison with other off-line methods during 2004.
8276
M. Mirmomeni et al. / Expert Systems with Applications 37 (2010) 8267–8277
Fig. 10. Predicted solar wind plasma speed values vs. observed values between 2000 and October 2008.
(Bothmer & Daglis, 2007). In this case study, modeling and prediction of the solar wind plasma speed as one of the drivers of geomagnetic activity is considered. The threshold parameter for splitting in RLoLiMoT algorithm is set to 0.3 and the threshold parameter for merging is set to 0.2. Like Dst. and Kp indices, the performance of adaptive LLNF model with RLoLiMoT learning algorithm in one step ahead prediction of daily plasma speed is compared with LLNF, MLP, and RBF neural networks. Fig. 9 shows the performance of the RLoLiMoT algorithm in one step ahead prediction of time-varying solar wind plasma speed. Like former case studies, in the beginning of the RLoLiMoT algorithm’s operation, the performance of the adaptive LLNF model is poor, but after a short time, the performance of the model increases rapidly and remains almost constant even during major magnetic storms in 2004. Table 4 contains the results of several methods for one step ahead prediction of solar wind plasma. It is obvious that the performance of the adaptive LLNF model is excellent in comparison with other off-line methods. Fig. 10 shows predicted plasma speed values vs. measured values between 2000 and October 2008.
input space. By using proposed adaptive neurofuzzy model, an accurate prediction is provided which can describe time-varying solar and geomagnetic activity indices even during geomagnetic storms and sub-storms. The performance of the proposed adaptive LLNF models with RLoLiMoT learning algorithm is evaluated for modeling and prediction of three solar and geomagnetic activity indices: Dst., Kp, and solar wind plasma speed. Obtained results justify the potentials of the proposed adaptive neurofuzzy models for describing time-varying solar and geomagnetic activity indices in comparison with other well-known methods. The improved accuracy resulting from the retuning of the predictor, which can be important even in the case of stationary time series because new observations allow reduction of estimation errors, has been achieved with minimal additional computational burden due to the recursive nature of the proposed method. Acknowledgments The authors thank the ‘‘National Space Science Data Center” for using this data set. References
5. Discussion and conclusions This paper proposed an adaptive neurofuzzy model with a recursive learning algorithm for modeling and prediction of timevarying solar and geomagnetic activity indices. Although neural network and adaptive system modeling had begun as a single discipline, they later developed their separate subcultures. Learning and recall modes are strictly separated in the case of neural networks. Adaptive algorithms, on the other hand, carry out learning and decision making tasks at the same time. This, in adaptive control literature, has been designated as dual control. Relearning after every observation in online applications is very important especially in time varying or nonstationary, even slowly varying cases. However, it will be very time consuming if a recursive learning algorithm is not used. The algorithm proposed in this paper is basically an embedded version of LoLiMoT with recursive least squares adaptation of parameters, as well as online k-means clustering of
Bothmer, V., & Daglis, I. (2007). Space weather: Physics and effects. Berlin: Springer/ Praxies. Brown, M., & Harris, C. (1995). Neurofuzzy adaptive modeling and control. Englewood Cliffs, NJ: Prentice-Hall. Burton, R. K., McPherron, R. L., & Russell, C. T. (1975). An empirical relationship between interplanetary conditions and Dst. Journal of Geophysical Research, 80(31), 4204–4214. Canon, P. S., Angling, M. J., & Lundborg, B. (2002). Characterisation and modelling of the HF communications channel. In W. R. Stone (Ed.), Review of Radio Science 1999–2002 (pp. 597–623). IEEE-Wiley. Clauer, C. R., Gombosi, T. I., De Zeeuw, D. L., Ridley, A. J., Powell, K. G., Leer, B. V., et al. (2000). High performance computer methods applied to predictive space weather simulations. IEEE Transactions on Plasma Science, 28(6), 1931–1937. Daglis, I. A. (2004). Effects of space weather on technology infrastructure. The Netherland: Kluwer Academic Publishers. Daglis, I. A., Delcourt, D., Metallinou, F. A., & Kamide, Y. (2004). Particle acceleration in the frame of the storm–substorm relation. IEEE Transaction on Plasma Science, 32(4-1), 1449–1454. De Zeeuw, D. L., Gombosi, T. I., Groth, C. P. T., Powell, K. G., & Stout, Q. F. (2000). An adaptive MHD method for global space weather simulations. IEEE Transactions on Plasma Science, 28(6), 1956–1965.
M. Mirmomeni et al. / Expert Systems with Applications 37 (2010) 8267–8277 Deng, D., & Kasabov, N. (2000). Evolving self-organizing maps for online learning, data analysis and modeling. In Proceedings of the IJCNN’2000 neural networks, neural computing, new challenges perspectives new millennium, New York, NY (pp. 3–8). Donnelly, R. F. (Ed.). (1979). Solar-terrestrial predictions proceedings (4 volumes). Environment Research Laboratories, National Oceanic and Atmospheric Administration, US Department of Commerce, Boulder, Colorado. Gholipour, A., Araabi, B. N., & Lucas, C. (2006). Predicting chaotic time series using neural and neurofuzzy models: a comparative study. Neural Processing Letters, 24, 217–239. Gholipour, A., Lucas, C., Araabi, B. N., Mirmomeni, M., & Shafiee, M. (2007). Extracting the main patterns of natural time series for long-term neurofuzzy prediction. Neural Computing and Applications, 16(4–5), 383–393. Gholipour, A., Lucas, C., Araabi, B. N., & Shafiee, M. (2005). Solar activity forecast: spectral analysis and neurofuzzy prediction. Journal of Atmospheric and Solar Terrestrial Physics, 67, 595–603. Gleisner, H., & Lundstedt, H. (1997). Response of the auroral electrojets to the solar wind modeled with neural networks. Journal of Geophysical Research-Space Physics, 102(A7), 14269–14278. Harris, C., Hong & Gon, Q. (2002). Adaptive modelling, estimation and fusion from data: A neurofuzzy approach. London: Springer. Heckman, G. K., Marubashi, M. A., Shea, D. F., Smart, R., & Thompson (Eds.). (1997). Solar-terrestrial predictions – V. Proceedings of a Workshop at Hitachi, Japan, January 23–27, 1996, RWC Tokyo, Hiraiso Solar Terrestrial Research Center, Communications Research Laboratory, Hitachinaka, Ibaraki, Japan. Horwitz, J. L., & Moore, T. E. (2000). Core plasmas in space weather regions. IEEE Transactions on Plasma Science, 28(6), 18840–21853. Hruska, J, Shea, M. A., Smart, D. F., & Heckman, G. (Eds.). (1993), Solar-terrestrial predictions – IV (3 volumes). US Department of Commerce, NOAA, ERL, Boulder, Colorado. Jalili-Kharaajoo, M., Rahmati, A., & Rashidi, F. (2003). Internal motor control based on Local Linear Model Tree (LoLiMoT) model with application to a PH neutralization process. In Proceedings of IEEE (pp. 3051–3055). Kane, R. P. (2006). The idea of space weather – A historical perspective. Advances in Space Research, 37, 1261–1264. Kivelson, M., & Russell, C. T. (1995). Introduction to space physics. Cambridge, UK. Lanzerotti, L. J., Thomson, D. J., & Maclennan, C. G. (1997). Wireless at high altitudes: Environmental effects on space-based assets. Bell Labs Technical Journals. Liu, G. P. (2001). Nonlinear identification and control: a neural network approach. London: Springer. Ljung, L. (1999). System identification: Theory for the user. UK: Prentice-Hall. Lughofer, E., & Klement, E. P. (2005). FLEXFIS: A variant for incremental learning of Takagi–Sugeno fuzzy systems. In IEEE international conference on fuzzy systems (pp. 915–920). Mavromichalaki, H., Gerontidou, M., Mariatos, G., Plainaki, C., Papaioannou, A., Sarlanis, C., et al. (2005). Space weather forecasting at the new Athens center: The recent extreme events of January 2005. IEEE Transactions on Nuclear Science, 52(6), 2307–2312. Dec.. Mayaud, P. N. (1980). Derivation, meaning, and use of geomagnetic indices. American Geophysical Union. Maynard, N. C. (1995). Space weather prediction. Rev. Geophys., US Nat. Rep. IUGG, 547.
8277
Mirmomeni, M., Lucas, & Moshiri, B. (2007). Long-term prediction of chaotic time series with the aid of neurofuzzy models, spectral analysis and correlation analysis. In Proceedings of international joint conference on neural networks, Orlando, Florida, USA, August 12–17. Mirmomeni, M., & Lucas, C. (2008a). Model based method for determination the minimum embedding dimension from solar activity chaotic time series. International Journal of Engineering, Transactions A: Basics, 21(1), 31–44. Mirmomeni, M., & Lucas, C. (2008b). Analyzing the variation of embedding dimension of solar and geomagnetic activity indices during geomagnetic storm time. Earth Planets Space, 60, 1–11. Mirmomeni, M., Lucas, C., Araabi, B. N., & Shafiee, M. (2007). Forecasting sunspot numbers with the aid of fuzzy descriptor models. Space Weather, International Journal of Research and Applications. doi:10.1029/2006SW000289. Mirmomeni, M., Shafiee, M., Lucas, C., & Araabi, B. N. (2006). Introducing a new learning method for fuzzy descriptor systems with the aid of spectral analysis to forecast solar activity. Journal of Atmospheric and Solar-Terrestrial Physics, 68, 2061–2074. Moldwin, M. B. (2008). An introduction to space weather. Cambridge: Cambridge University Press. Nelles, O. (1996). Local linear model tree for on-line identification of time-variant nonlinear dynamic systems. In Proceedings of the international conference on artificial neural networks (ICANN), Bochum, Germany (pp. 115–120). Nelles, O. (2001). Nonlinear system identification. Berlin: Springer. O’Brien, T. P., & McPherron, R. L. (2000). Forecasting the ring current index Dst in real time. Journal of Atmospheric and Solar-Terrestrial Physics, 62(14), 1295–1299. Simon, P. A., Heckman, G., & Shea, M. A. (Eds.). (1986). Solar-terrestrial predictions. Proceedings of a workshop at Meudon, France, June l8–22, 1984, Environment Research Laboratories, National Oceanic and Atmospheric Administration, US Department of Commerce, Boulder, Colorado. Stefanski, A. (2000). Estimation of the largest Lyapunov exponent in systems with impacts. Chaos, Solitons and Fractals 11, 2443–2451. Takagi, T., & Sugeno, M. (1985). Fuzzy identification of systems and its applications to modeling and control. IEEE Transactions on System, Man, and Cybernetics, 15, 116–132. Thompson, R. J., Cole, D. G., Wilkinson, P. J., Shea, M. A., Smart, D., & Heckman, G. (Eds.). (1990). Solar-terrestrial predictions. Proceedings of a Workshop at Leura, Australia (2 volumes), US Department of Commerce, NOAA, ERL, Boulder, Colorado. Vassiliadis, D. (2000). System identification, modeling, and prediction for space weather environments. IEEE Transactions on Plasma Science, 28(6), 1944–1955. Vassiliadis, D., Klimas, A. J., Valdivia, J. A., & Baker, D. N. (2000). The nonlinear 12 dynamics of space weather. Space Weather: Physics and Applications, 26, 197–207. Volberg, O., Toth, G., Gombosi, T. I., Stout, Q. F., Powell, K. G., De Zeeuw, D., et al. (2005). A high-performance framework for Sun-to-Earth space weather modeling. In 19th IEEE international conference on parallel and distributed processing symposium, 4–8 April (p. 8). Weigel, R. S., Baker, D. N., Rigler, E. J., & Vassiliadis, D. (2004). Predictability of large geomagnetic disturbances based on solar wind conditions. IEEE Transactions on Plasma Science, 28(6), 1506–1510.