- Email: [email protected]
Optimization by Canonical Analysis in a Radial Basis Function
Expert Systems with Applications xxx (2015) xxx–xxx
Contents lists available at ScienceDirect
Expert Systems with Applications journal homepage: www.elsevier.com/locate/eswa
Review
Optimization by Canonical Analysis in a Radial Basis Function Rolando J. Praga-Alejo a,⇑, Mario Cantú-Sifuentes a,1, David S. González-González b a b
Corporación Mexicana de Investigación en Materiales (COMIMSA), Calle Ciencia y Tecnología, # 790, Frac. Saltillo 400, Saltillo, Coahuila, Mexico Facultad de Sistemas, Universidad Autónoma de Coahuila, Ciudad Universitaria, Carretera a México Km 13, Arteaga, Coahuila, Mexico
a r t i c l e
i n f o
Article history: Available online xxxx Keywords: Radial Basis Function Canonical Analysis Optimization
a b s t r a c t Generally, statistical methods and mathematical models are useful for process optimization. Nonetheless, other methods might be used for modeling and optimizing the manufacturing process. Among these, we can mention the neural networks and the Radial Basis Function technique. Hence, a suitable alternative is complementing statistical methods and neural networks as a Hybrid Learning Process. This work applies the Radial Basis Function Canonical Analysis in order to achieve the welding process optimization. One of the most important results is that the Radial Basis Function neural networks along with the Canonical Analysis are really useful methods. These methods are applied for predicting the optimal point, which establishes a reliable method for the process modeling and optimizing. The Canonical Analysis can determine stationary and saddle points, as it was in this case of study, which Canonical Analysis with RBF represented it adequately and can plot a surface and contour lines. Since in this case of study there is a surface that contains a ridge saddle system, also often called minimax. Then the results show that the Canonical Analysis can explore the region with oblique stationary and rising ridge systems. In this way, the RBF neural network with Canonical Analysis could be an alternative method for analyzing data, whenever the Hybrid Learning Process is adequate or satisfies the test assumption and fulfills the evaluation criteria. In this case of study, validation is represented by the Hybrid Learning Process (Radial Basis Function with Canonical Analysis) presenting an excellent effectiveness. As a conclusion we can say that the resulting Radial Basis Function has improved the model accuracy after using the Canonical Analysis. Ó 2015 Elsevier Ltd. All rights reserved.
1. Introduction Generally, modeling and optimization related with manufacturing processes is made by means of linear models (Romero-Villafranca, Zúnica, & Romero-Zúnica, 2007). Denote as y the random variable describing the statistical behavior of the interest characteristic of a particular product; as x ¼ ðx1 ; x2 ; . . . ; xk Þ the vector that collects the k input variables or operation parameters determining y, where x1 2 Xj . for j ¼ 1; 2; . . . k and Xj are specific variation ranks. It is assumed that y ¼ f ðx1 ; x2 ; . . . ; xk Þ þ e, where f ðÞ is an unknown function from Rk to R, and e is a random variable who describes the effect that non considered factors have on y. It is assumed that the specific distribution of e depends at most of a set of estimable parameters. Then the statistical variation of y and the ⇑ Corresponding author. Tel.: +52 01 844 4 11 32 00x1217. E-mail addresses: [email protected] (R.J. Praga-Alejo), [email protected] (M. Cantú-Sifuentes), [email protected] (D.S. González-González). 1 Tel.: +52 01 844 4 11 32 00x1217.
functional form of f are estimated, via an experimental design, for a polynomial function. The purpose is to get and estimate the optimal value, x ¼ x1 ; x1 ; . . . ; x1 , for the vector of input variables, x; it is:
x ¼ optimizex2X ff ðx1 ; x2 ; . . . ; xk Þ þ eg;
ð1Þ
where, optimize is to be understood as minimize or maximize depending of the underlying context (Conn, Scheinberg, & Vicente, 2009); X ¼ X1 X2 . . . Xk # Rk is the search space. Polynomials of first and second order are technically desirables, and usually sufficient to approximate the function f ðx1 ; x2 ; . . . ; xk Þ., and the usual fitting technique is minimum squares. Residuals are used to describe the behavior of y, and to estimate the parameters of e. Among the experimental designs, the central composite one is frequently used to fit full second order models, whose general form is:
^ ¼ b0 þ y
k X XX bj xj þ bij xi xj þ e; j¼1
ð2Þ
iPj
http://dx.doi.org/10.1016/j.eswa.2015.04.013 0957-4174/Ó 2015 Elsevier Ltd. All rights reserved.
Please cite this article in press as: Praga-Alejo, R. J., et al. Optimization by Canonical Analysis in a Radial Basis Function. Expert Systems with Applications (2015), http://dx.doi.org/10.1016/j.eswa.2015.04.013
2
R.J. Praga-Alejo et al. / Expert Systems with Applications xxx (2015) xxx–xxx
where e is assumed, and tested, to have a normal distribution with mean 0 and unknown variance r2 . The fixed interaction points of the operation parameters must, of course, fall in X. Associated with models Eq. (2) is the response surface methodology, an experimental technique developed to find the optimal response within specified ranges of the parameters. The model in Eq. (2) is used to estimate the assumed curvature in the true, and unknown, response function. If a maximum or minimum exists inside the search space, response surface method can be used to estimate it. Through the fitted polynomial:
^0 þ ^¼b y
k X XX ^ij xi xj : ^ j xj þ b b j¼1
ð3Þ
iPj
The response surface methodology can determine the sub-region
X0 X where y presents an optimum. However, the fitted second order polynomial model in Eq. (3) can provide not an optimal point but a stationary (or a saddle point) in X0 . If this is the case, Canonical Analysis can be used to determine the nature of the stationary point. It can be distinguish two canonical forms: the A form who can be applied if the stationary point is in the frontier of X0 ; and the form B otherwise. Moreover, Canonical Analysis can be used to define the point, line, or subspace on which a maximum, minimum or saddle point is obtained. In the modeling and optimization of manufacturing processes, some other alternative techniques can be used. Among these: Fuzzy logic, Artificial Neural Networks, Evolutionary Algorithms and Hybrid systems (Paliwal & Kumar, 2009; Benyounis & Olabi, 2008). In particular, the use of Radial Basis Functions as activation functions in artificial neural networks can be used to accurately predicting the processes. Such a neural network is generically referred as Radial Basis Function Neural Network (RBFNN). The RBF is a second order or hyper-spherical type function in which the network value represents the distance with respect to a given reference pattern (For more details about RBFNN see Gregorcˇicˇ & Lightbody, 2008; Martín del Brío & Sanz, 2007; Arbib, 2003; Nelles, 2001; Haykin, 1999). The RBF modeling and optimization applications might be founded in Seyed Javan, Mashhadi, and Rouhani (2013), Yua and Duan (2013), Xiong, Shi, Chen, Zhu, and Duan (2013), Sermpinis, Theofilatos, Karathanasopoulos, Georgopoulos, and Dunis (2013), Giordanoa, Beccariab, Goicoecheaa, and Olivieri (2013), Mateo and Rieta (2013), Ko (2013), Ko and Lee (2013), Ting, Hongjun, and Lei (2012), Oh, Kim, Pedrycz, and Joo (2012) and Wang, Jia, Huang, and Chen (2010), Park, Pedrycz, Chung, and Oh (2012). Liu, Xu, Wang, Wu, and Sond (2014) although the three proposed algorithms obtained a good performance, DIRECT scheme was better with respect to compute coming faster to the target. Wu, Luo, Zhang, and Zhang (2014) propose a new sampling method based on the zeros of the Chebyshev polynomial to capture information efficiently sampling. Jie, Wu, and Ding (2014) present an adaptive global optimization (AMGO) the results indicate that the proposed method was satisfactory accuracy and efficiency of the convergence of the iterative process with low computational cost. In the work published by Shahlaei, Bahrami, Abdolmaleki, Sadrjavadi, and Bagher (2015) applied a principal component analysis combined with a RBFNN; the results indicate that the proposed method is a good alternative for modeling. Zhu, Zhang, Fatikow, and Wang (2015) and Zhu, Zhang, and Chen (2015) developed a bidirectional method for solving optimization problems; the method was designed using a RBF as implicit level set function. Cartis, Fowkes, and Gould (2015) have improved techniques for global optimization functions, using built-Lipschitz continuity to develop procedures delimitation and parallelization strategies. Haji, Pirmoradi, Cheng, and Wang (2015) in their scheme propose
for solving optimization problems indicated that problems with strongly correlated variables cannot be decomposed. Couckuyt, Deschrijver, and Dhaene (2014) propose a multi objective optimization algorithm based on Kriging models and probability criteria applicable to standard benchmark problems improved. Liu, Xu, Ma, and Wang (2015) develop a global optimization algorithm using Lipschitz functions and response surfaces for the computations in black box functions; both methods are able to approximate the actual function efficiently, so it is possible to estimate lower bounds efficiently. Rajendra and Shankar (2015) introduced a new type of network for complex values; the accuracy of the proposed network was significantly better compared to the RBF which does not use complex values. Mateo and Rieta (2013) propose a system based on RBF for their ability to select adjustable parameters to minimize the variance at the output of the method. In the work of Tyan, Van Nguyen, and Lee (2014) a global modeling is presented, using low-fidelity and a scaling function to approximate efficiently in optimization problems. Huda et al. (2014) developed a hybrid method for detection of signals outside multivariate control manufacturing processes. Müller and Shoemaker (2014) examined the influence on the quality of the substitute solution computationally expensive algorithm for global optimization in the black box. The data and the performance of the proposed method show that combinations containing a cubic model on RBF and work best regardless of the sampling strategy. Jiang et al. (2014) use a neural network model to investigate the relationship between the process parameters and polymer curing viscosity and curing time; developed an artificial neural network with a genetic algorithm for modeling and optimization in a polymer curing process. Ugrasen, Ravindra, Naveen Prakash, and Keshavamurthy (2014) optimized the parameters in EDM machining process by ANOVA. Akhtar and Shoemaker (2015) proposed a model based on response surface for a multi target approach using a RBF iteratively to calculate approximations of the objective function. Zhu, Zhang, Fatikow et al. (2015) and Zhu, Zhang, and Chen (2015) developed a strategy of sequential sampling for robust design with no restrictions applied to problems with restrictions. Li, Liu, Long, and Chen (2015) applied a RBF for multimodal functions calculation. Gomes and Canedo (2015) propose a Gaussian RBF for diagnostic a system for identifying problems in nuclear power plants with pressurized water reactor. Müller and Shoemaker (2014) present an algorithm for solving optimization problems with linear objective functions. These papers applied in different ways the RBFs, to approximate some objective function optimization problems. The RBFs for these problems reduce computing costs required by other methodologies. Other applications, with RBFs, are used for prediction for obtaining models that represent a good accuracy. But what happens when the stationary point is a saddle point or point of maximum or minimum response that resides well outside the experimental region, or where several response variables must be simultaneously considered?. Thus, Praga-Alejo, Torres-Treviño, González- González, Acevedo-Dávila, and Cepeda-Rodríguez (2012) proposed the redesigned RBF neural network, which can be properly fitted to models because its hybrid learning method is constructed by means of a genetic algorithm that calculates the matrix of centers and the Mahalanobis distance, maximizing the coefficient of determination, R2 . This statistic becomes the evaluation function in the genetic algorithm, improving the accuracy of the RBF. In this paper time-consuming is not the main objective. The advantage is that redesigned RBF neural network has been fitted in a near stationary region in which there appears to be some kind of maximum minimum, or saddle point, so that, it is now important to determine the nature of the local surface or search region which we
Please cite this article in press as: Praga-Alejo, R. J., et al. Optimization by Canonical Analysis in a Radial Basis Function. Expert Systems with Applications (2015), http://dx.doi.org/10.1016/j.eswa.2015.04.013
R.J. Praga-Alejo et al. / Expert Systems with Applications xxx (2015) xxx–xxx
tentatively believe contains a maximum, minimum, or saddle point. And the questions are: Is it, in fact, a maximum, minimum, or saddle point? How can we define the point, line, or space on which a maximum, minimum, or saddle point response is obtained? Then in this paper we propose the use of Canonical Analysis with RBFNN to answer these questions. As illustration (or case study), the proposed approach is applied to a Gas Arc Metal welding process.
2.1. Radial Basis Function (RBF) Neural Network The RBFNN values represent the distance to a given reference pattern (Haykin, 1999). The structure in a RBF, in its basic form, includes three totally different layers: 1. The input layer, which consists of source nodes xi (sensory units). 2. The hidden layer, which is a layer of high dimension and the units (neurons) that conform it are the basic functions for the input data ui ðxi Þ. 3. The output layer dj , which is responsible for the activation of patterns considering the input layer network. Here the notation for outputs is taken as d instead y for custom reasons. The output layer neurons are linear. The hidden layer neurons calculate the difference between the vector of inputs and the centroids. This difference applies a radial function with a Gaussian shape mainly, but it has the advantage of being able to use other radial functions (Gregorcˇicˇ & Lightbody, 2008). The function of transfer radial in Gaussian type adopts the following form:
1 2 : x t k k i 2r2i
ð4Þ
It is simplified by (5):
ui ðxÞ ¼ GðÞ;
ð5Þ
where x are the inputs and t are the centers or centroids formed by the Euclidean distance or Euclidean norm of kx t k2 . When an input vector is introduced, each neuron in the hidden layer has a Radial Basis Function. Each neuron in the hidden layer is the distance between the vector input and its vector of weights. The relationship between inputs and outputs from neural network is given by Eq. (6) where y xj ¼ dj :
yðxÞ ¼
m X wGðkx t i kÞ þ b þ e:
where e is the residual or error in the model and b is the vector of thresholds or bias, which is a column of ones and is added in the matrix G. The matrix G is given by Eq. (7):
2
G¼6 6 .. 4 .
.. .
..
.
7 .. 7: . 5
The R2 coefficient of determination is considered by the Function Evaluation; which represents (as in regression models) the proportion of variance explained by the model or regressor x. A value for R2 close to 1 implies that most of the variability of the prediction y is explained by the model (Montgomery, 2012). Therefore, the objective is to determine the centroids ti such
R2 ¼ 1
y0 y w0 G0 y Pn 2 . : y0 y n i¼1 yi
ð9Þ
2.2. Canonical Analysis Canonical analysis is a method of rewriting a fitted second degree equation (as Eq. (3)) in a form in which it can be more readily understood. The detection of the nature of the system and the location of the stationary point are an important part of the second-order analysis. The nature of the response surface system (maximum, minimum, or saddle point) depends on the signs and magnitudes of the coefficients in the model of Eq. (3). The second-order coefficients (interaction and pure quadratic terms) play a vital role. One must keep in mind that the coefficients used are estimates of the b’s of Eq. (3). These estimated b’s are obtained by Ordinary Least Squares (OLS) so when this technique is transferred to the RBF, the coefficients become the weights of the neural network; these weights are also calculated with OLS. This is when these two techniques are brought together. Thus, even the system itself (saddle, maximum, or minimum points) is part of the estimation process. The stationary point and the general nature of the system arise as a result of a fitted model. This fitted model can be calculated with the RBF neural network. Thus the Nature of the Stationary Point is a Canonical Analysis and it is determined from the signs of the eigenvalues of the matrix B. This entire translation and rotation of axes described here is called a Canonical Analysis of the response system. This is achieved by a rotation of axes which removes all crossproduct terms. We call this simplification the A canonical form. If desired, this may be accompanied by a change of origin to remove first-order terms as well. We call the result the B canonical form (Myers, Montgomery, & Anderson-Cook, 2009). An alternate canonical form:
ð6Þ
i¼1
u11 u12 u1m 3 7 6u 6 21 u22 u2m 7
In this way, it is possible to say that there are different Radial Basis Functions. This work employs the Gaussian function.
that R2 , given by Eq. (9), be maximum. R2 is a global metric evaluation, where yi represents the experimental response, G is given by (7) and w is given by (8):
2. Methods
ui ðxÞ ¼ exp
3
2.2.1. The A canonical form In the first place the variables should be coded as follows (Montgomery, 2012):
X iD ¼
ni ni ; Si
ð10Þ
where ni ¼ Gi ; see Eq. (7):
ð7Þ
uN1 uN2 uNm
– ni is the input variable vector or independent variable in the original form; – ni is the input variable mean and, – Si is the standard deviation of each input variable vector.
The equation is written in a matrix form Gw ¼ d, where the variable 0 and dj is the output or response, d ¼ ½d1 ; d2 ; . . . ; dN 0 w ¼ ½w1 ; w2 ; . . . ; wm which represents the weights determined by Eq. (8), calculated by Ordinary Least Squares:
Thus, we should consider a fitted second degree model in Eq. (3), the equation in matrix form:
1 w ¼ G0 G G0 d:
^ ¼ b0 þ X0D b þ X0D BXD ; y
ð8Þ
ð11Þ
Please cite this article in press as: Praga-Alejo, R. J., et al. Optimization by Canonical Analysis in a Radial Basis Function. Expert Systems with Applications (2015), http://dx.doi.org/10.1016/j.eswa.2015.04.013
4
R.J. Praga-Alejo et al. / Expert Systems with Applications xxx (2015) xxx–xxx
where
3
2
x1 6x 7 6 27 7 XD ¼ 6 6 .. 7; 4 . 5
3
2
b1 6b 7 6 27 7 b¼6 6 .. 7; 4 . 5
xk
2
b11 6 1=2b 12 6 B¼6 .. 6 4 .
1=2b12 b22 .. .
.. .
1=2b1k
1=2b2k
bk
1=2b12
3
1=2b2k 7 7 7 .. 7 5 . bkk
If M0 M ¼ I, rewriting:
^ ¼ b0 þ X0D M ðM0 bÞ þ X0D M ðM0 BMÞ M0 XD : y
ð12Þ
If all signs of ki are negative, this will mean that there is a maximum located in the surface curvature. If all signs of ki are positive, this will mean that there is a minimum. In case there are ki different signs, this will mean there is a saddle point. When the fitted surface is a rising ridge, it can be useful to use a slightly different canonical form; it is usually called the A canonical form. It basically assumes that the axes have been rotated exactly as in the B canonical form, but not first translated to the stationary point. These Eqs. (10)–(18) can be calculated by the RBFNN.
If we now write:
h ¼ M 0 b:
ð13Þ
h given by multiplying the bj vector (coefficient of first order terms of Eq. (3)) by the eigenvectors of B matrix defined in (11). Where mi ¼ ðmi1 ; mi2 ; . . . ; mik Þ0 . The elements of the vector mi ¼ ðmi1 ; mi2 ; . . . ; mik Þ0 are, in fact, the direction cosines of the X i axis in the x space. Furthermore, if we construct a matrix M whose ith columns is mi , the transformation from old ðxÞ to new ðXÞ coordinate is given by Eq. (14). Equivalently, for the respectively parallel coordinate systems x and X, measured from the natural origin, we have the transformation. Because the transformation is an orthogonal one and m0i mj ¼ 0; m0i mi ¼ 1; i; j ¼ 1; 2; . . . ; k, it follows 0
0
that M M ¼ MM ¼ I, that is, M mations are:
1
0
¼ M . Thus the inverse transfor-
X ¼ M0 X D ;
ð14Þ
X is a rotation matrix obtained by multiplying the eigenvectors of B matrix ðMÞ by the original matrix X D given in (11). From these forms, we see that the jth row of M, with elements m1j m2j ; . . . ; mkj , provides the direction cosines of the xj axis in the X space (or of the xj axis in the X space). This can be computed with spectral decomposition. Similar apply to the B canonical form. Therefore:
K ¼ M 0 BM;
ð15Þ
ð16Þ
Eq. (16) is the A canonical form, achieved by the axes rotation which removes all cross-product terms. Thus, A canonical form can be expressed in an expanding form as (17):
^ ¼ b0 þ h1 X 1 þ þ hk X k þ k1 X 21 þ þ kk X 2k : y
ð17Þ
Differentiating Eq. (17) with respect to X. In this way, we find that the stationary point coordinates X 1s ; . . . ; X ks , are:
X is ¼
hi : 2ki
1 xis ¼ B1 b: 2
ð19Þ
Or of 2KX s ¼ h equal than Eq. (18). At this point, the fitted response is equal to:
1 ^s ¼ b0 þ x0is b: y 2
ð20Þ
Or, equivalently:
1 ^s ¼ b0 þ X 0is h: y 2
ð21Þ
e i and ~xi where: The new vectors: X e can be rewritten in coded variable as: –X
e ¼ M 0 ~x; X
ð22Þ
~x ¼ xi xis ; referring to the coordinates that were measured from stationary point S. The B canonical form is achieved by changing the modeling system origin at the stationary point (xis ) given by:
e 0 K X; e ^¼y ^s þ X y
K is a diagonal matrix of eigenvalues of B matrix. In this way, Eq. (12) can be rewritten as:
^ ¼ b0 þ X0 h þ X0 KX: y
2.2.2. The B canonical form This is achieved by differentiating and equating to zero Eq. (11) with respect to X D , and derived in relation to X the Eq. (16). In this way, it is possible finding the stationary point S coordinates:
ð18Þ
The signs of ki eigenvalues defining the kind of surface. where: – ki is the i-th diagonal element of K. – hi measures the surface slope in X ¼ 0 in the rotated axes. – X is indicates how far the canonical axes are in relation to the stationary point ðSÞ. – ki are the characteristic values or symmetric matrix B roots. – mi are the characteristic vectors, where: Bmi ¼ mi ki . M is an orthonormal matrix with standardized k columns characteristic vectors.
ð23Þ
B Canonical form, which might be expressed through an expanding form:
e 2 þ þ kk X e 2: ^¼y ^s þ k1 X y 1 k
ð24Þ
The canonical form of the second-order model given by Eq. (24) is usually called the B Canonical form; these Eqs. (19)–(24) can be calculated by the RBFNN. It is very useful for determining the nature of the fitted response surface, particularly in identifying saddle systems and ridges (Myers et al., 2009). 2.2.3. A or B canonical forms? When should the A or B canonical form must be applied? In design units, the stationary point ðSÞ distance (represented by the system center) from the design center O should be given by:
" #12 " #12 k k X X 2 D ¼ OS ¼ xis ¼ xjs : i¼1
ð25Þ
j¼1
That is:
1 1 D ¼ X 0is X is 2 ¼ x0is xis 2 :
ð26Þ
This means: – If D is greater than 1, the A canonical form must be used to explore the stationary point neighborhood. – If D is closer to zero or one, the B canonical form must be used.
Please cite this article in press as: Praga-Alejo, R. J., et al. Optimization by Canonical Analysis in a Radial Basis Function. Expert Systems with Applications (2015), http://dx.doi.org/10.1016/j.eswa.2015.04.013
5
R.J. Praga-Alejo et al. / Expert Systems with Applications xxx (2015) xxx–xxx
– The A canonical form should be used in presence of a saddle point (Myers et al., 2009). 2.3. Hybrid Learning Process The current paper employs the hybrid intelligent system, genetic algorithm (GA) and other statistical methods (Praga-Alejo et al., 2012). The Hybrid Learning Process involved in the RBFNN applies the following steps: – Step 1: Generate centroids with genetic algorithm. Applies a GA to calculate the matrix of centers t so that it maximizes R2 . Eliminate the Euclidean distance and to calculate the Mahalanobis distance. – Step 2: Apply the Radial Basis Function, e.g. Eq. (4), to obtain the matrix G given by Eq. (7). – Step 3: Generate the weights w with Eq. (8). – Step 4: Evaluating the fitness function in GA. where the coefficient of determination R2 becomes the statistical evaluation function of GA which helps in the accuracy of the prediction and optimization of the model. Thus fit and train the RBFNN model until to maximize R2 . – Step 5: Codified or standardize the variables with Eq. (10). – Step 6: Fitted second degree model as Eq. (11). – Step 7: Apply the Canonical Analysis with the second degree model obtained of the RBF.
In this kind of welding process, two metallic pieces are joined by heat, pressure or a combination of both. In some cases, an external contribution metal might be required. For this process, two metal pieces are joined by heat, through an arc which is generated by an electrode, fine wire and metals. This welding process is improved and protected by inert gases like argon and helium. There are usually three parameters numbers involved in the GMAW process: Speed feed, Voltage and Torch travel speed or speed welding. These parameters are in charge of regulating the welded pieces performance; for instance: penetration, legs and throat (Fig. 1). It is important to mention that this case of study is centered in the throat. The tests were made with the help of a KUKA KR-16 robot, with PW455x Lincoln analog interface. Fig. 2 shows a test sample considered in the experimental design (first test of Table 1). Therewith, the objective is finding the best parameters levels that the process must perform, in order to reach the best response. In this specific case, we must refer to the ‘‘throat’’ (Output variable). 3.2. GMAW process results Mathematical formulation: Given:
y f ðx1 ; x2 ; . . . ; xk Þ þ e where:
3. Application The method was applied in a welding process; nonetheless the proposed RBF model might be applied in different engineering problems.
f ðx1 ; x2 ; . . . ; xk Þ Is the RBF model given by Eq. (6): yðxÞ ¼ b þ e. Is:
3.1. Gas Metal Arc welding application
x : f ðx Þ P f ðxÞ8x 2 Rk
In Table 1, DOE results are shown. These represent a CCD; this means, an analysis of the second order response surface model with RBFNN. The second order model is composed by linear, quadratic and interaction terms or variables. The application was performed in a Gas Metal Arc Welding (GMAW) process.
xi 2 X;
Pm
i¼1 wGðkx
ti kÞþ
x ¼ optimizex2X ff ðx1 ; x2 ; . . . ; xk Þ þ eg where:
X ¼ X1 X2 . . . Xk is the search space. Thus:
Table 1 CCD in the GMAW process.
f ðx1 ; x2 ; x3 Þ;
Input Variables
Output variable
Speed welding
Voltage
Speed feed
Throat
28 25 25 25 25 22 25 19.95 25 28 30.04 22 28 25 25 22 28 22 25
24 28 28 28 28 24 28 28 28 32 28 32 32 28 21.27 24 24 32 34.72
280 305 305 262.95 305 330 305 305 305 280 305 330 330 347.04 305 280 330 280 305
2.28 3.08 2.74 2.27 2.77 2.66 2.39 2.32 2.92 1.28 3.45 4.93 2.19 3.19 2.89 2.94 2.89 3.33 2.54
Fig. 1. The welded pieces.
Please cite this article in press as: Praga-Alejo, R. J., et al. Optimization by Canonical Analysis in a Radial Basis Function. Expert Systems with Applications (2015), http://dx.doi.org/10.1016/j.eswa.2015.04.013
6
R.J. Praga-Alejo et al. / Expert Systems with Applications xxx (2015) xxx–xxx
2
0:82113127
3
6 7 b ¼ 4 0:41637855 5; 0:63385247 2 6 B¼4
1:84458463 0:63299856
0:48938275
3
7 0:74212297 1:06527651 5:
symmetric
1:02056899
– The A canonical form is: Applying Eqs. (13)–(18). Then:
2
0:73017983
3
6 7 h ¼ 4 0:41292658 5
Fig. 2. Test sample in the CCD.
where the bound constraints are:
19:95 P x1 6 30:04; 21:27 P x2 6 34:72; 262:95 P x3 6 347:04: and:
x1 : Speed Welding; x2 : Voltage; x3 : Speed Feed: The best quality of quality and favorable joints in this kind of welding process are: Throat between 2.8 and 3.9 mm. So, with the Table 1 were presented the following results, through the Hybrid Learning Process application. This is achieved through Steps 1 to 7, presenting the respective equations: – First, the coded variables presented as Eq. (10). – Weights wj are computed in the RBF Redesigned Model determined by the RSM, with the complete second order polynomial model applied to Eqs. (4)–(11). Where: ni ¼ Gi rewritten as (8), yield:
1 w ¼ X 0D X D X D y; 3 0:82113127 6 0:41637855 7 7 6 7 6 6 0:63385247 7 7 6 6 1:84458463 7 7 6 7 6 6 0:74212297 7 7 w¼6 6 1:02056899 7: 7 6 7 6 6 1:26599711 7 7 6 6 0:9787655 7 7 6 7 6 4 2:13055301 5 2
3:20666468 Then, obtaining b0 ; b and B: Where: b0 ¼ w0 then w0 ¼ 3:20666468 and:
b0 ¼ 3:20666468;
0:73873248 2 2:16806156 0 6 K¼4 0 0:12977941 0
0
0 0
3 7 5:
1:95638947
The eigenvalues ki signs are different. Thus, the surface presents a saddle point. – hi is in charge of measuring the surface slope in X ¼ 0 in the rotated axes. – X is a rotation matrix obtained by multiplying the matrix B eigenvectors by the original matrix X D ; X ¼ M 0 X D Where: – M are the eigenvectors; thereby the A canonical form ^ ¼ b0 þ X0 h þ X0 KX polynomial is obtained y Where: b0 ¼ w0 , in an expanding form is:
^ ¼ 3:206 þ 0:730X 1 þ 0:412X 2 þ 0:738X 3 2:168X 21 y þ 0:129X 22 þ 1:956X 23 : In this way it is possible finding out the stationary point coordinates X is , applying Eq. (18):
2
0:16839463
3
6 7 X is ¼ 4 1:5908786 5 0:18879995 X is are considered as the best parameters (in coded variables) fitted by the A canonical form. – The B canonical form is: The stationary point S coordinates, Eq. (19), are:
2
3 0:43733706 6 7 xis ¼ 4 1:02107707 5; 1:16663396 X is are considered as the best parameters (in coded variables) fitted by the B canonical form. The fitted response, Eq. (20), at this stationary point S is:
^s ¼ 2:8699495 y ^s is very close to test sample (Fig. 2), and this y ^s is e.g. this data y one of the best region, where we can find the best quality of welding. e i and ~xi . The new vectors are represented by X Where: e ¼ M0 ~x and ~x ¼ xi xis . e can be rewritten in a coded variable as X X Hence, B canonical form can be expressed in an expanding form Eq. (24) as:
e 2 þ 0:129 X e 2 þ 1:956 X e2 ^ ¼ 2:8699495 2:168 X y 1 2 3
Please cite this article in press as: Praga-Alejo, R. J., et al. Optimization by Canonical Analysis in a Radial Basis Function. Expert Systems with Applications (2015), http://dx.doi.org/10.1016/j.eswa.2015.04.013
7
R.J. Praga-Alejo et al. / Expert Systems with Applications xxx (2015) xxx–xxx Table 2 Results in optimization.
Canonical Prediction 5 Target Canonical A Canonical B
4.5
Test
4
1 2
Input Variables Speed Welding
Voltage
Speed Feed
Output variable Throat
17.71 17.71
24.63 24.63
335.98 335.98
3.83 (Fig. 5) 3.89 (Fig. 6)
RBF with Canonical Analysis 3.84 3.84
Throat (mm)
3.5 3 2.5 2 1.5 1
0
2
4
6
8
10 Data
12
14
16
18
20
Fig. 3. Observed (real process) vs. fitted.
– A or B canonical form? The stationary point S distance (center of the system) from the design center O Eq. (26) is:
D ¼ 1:61: The B canonical form must be used whenever D is close to 0 or near to 1. – The fitted model (A and B canonical form). Welding process predictions Fig. 3. Fig. 4 presents the welding process experiment predictions with RBFNN and canonical polynomial.
Fig. 5. Test 1 throat in the real process 3.83 mm.
4. Validation The fitted models present good predictions about the welding process. Still, we must ask: Are those predictions accurate? For this reason, it is necessary validating the model in the real process. Table 2 shows the employed parameters. Table 2 shows the values predicted by RBFNN with Canonical Analysis, as well as two field tests. Fig. 5 shows the first test. This test was performed in order to validate the model optimization proposal in a real GMAW process. The test was performed with these parameters: – Speed Welding = 17.71. – Voltage = 24.63. – Speed Feed = 335.98.
Fig. 6. Test 2 throat in the real process 3.89 mm.
Fig. 5 shows 3.83 mm (throat) versus the predicted 3.84 mm. (Table 2). The second test was performed with the purpose of validating the model optimization proposal in a real GMAW process. The test was performed with these parameters:
8 6 4 2
– Speed Welding = 17.71. – Voltage = 24.63. – Speed Feed = 335.98.
0 -2 -4 2 1
2 1
0 0
-1
-1 -2
-2
Fig. 4. RBFNN in the Canonical polynomial.
The Fig. 6 shows 3.89 mm (throat) versus the predicted 3.84 mm (Table 2). These results are extremely important, due to the fact that the proposed model is a good method for prediction and optimization. As RBF with Canonical Analysis allows the model validation, this can be applied in real cases. For this reason, it is possible performing highly accurate predictions all through the welding process. The
Please cite this article in press as: Praga-Alejo, R. J., et al. Optimization by Canonical Analysis in a Radial Basis Function. Expert Systems with Applications (2015), http://dx.doi.org/10.1016/j.eswa.2015.04.013
8
R.J. Praga-Alejo et al. / Expert Systems with Applications xxx (2015) xxx–xxx
welding pieces were obtained with appropriate acceptance criteria, good quality and favorable joints, achieving the expected success and presenting the parameters required by the GMAW process.
5. Conclusions The Canonical Analysis with RBF method presented in this work successfully applies a statistical evaluation, improving the model prediction and optimization. In this case of study, it is possible to conclude that the proposed model represents a suitable prediction and optimization technique, whenever the experiments are controlled. The results show that statistical and mathematical method, along with neural networks can be considered as good alternative techniques, in order to predict and optimize complex processes. Specifically, this case presents some predicted values (by Canonical Analysis with RBFNN). The results showed 3.83 mm (throat) versus 3.84 mm (predicted). As we can notice, these are extremely close. For this reason, it is possible to have an adequate acceptance criteria, good quality and favorable joints. The Canonical Analysis with RBFNN is a good representation of the response surface. This can be translated into an advantage to find a better optimum point in relation to the welding process. As a conclusion we can say that the Canonical Analysis method with RBFNN is a good optimization technique. The main idea is to improve the model prediction with RBFNN that can employ a second degree polynomial to represent the response in the neighborhood of interest, theorical systems that approximate ridges. Then redesigned RBF neural network has been fitted in a near stationary region in which there appears to be some kind of maximum minimum, or saddle point, so that, it is now important to determine the nature of the local surface or search region which we tentatively believe contains a maximum, minimum, or saddle point. Where the Canonical Analysis can be of great help in characterizing such systems and can locate the optimum too and we could select, for our optimum process that point near the crest of the ridge for the response that yielded the most satisfactory value of the response. In this paper is not the goal the time-consuming. We do not need model-based derivative-free trust-region methods because in our processes need to identify stationary points or rising ridges. When this work was completed, great practical interest was associated with the fact that, for this system, there was not a single point maximum but, locally at least, a plane of near-maxima. This allowed considerable choice of operating conditions; it allowed conditions to be chosen so that other criteria attained their best levels. The Canonical Analysis with RBF as model can determine stationary and saddle points, as it was in this case of study, which Canonical Analysis with RBF represented it adequately and can plot a surface and contour lines. Since in this case of study there is a surface that contains a ridge saddle system, also often called minimax. Then the results show that the Canonical Analysis can explore the region with oblique stationary and rising ridge systems. The advantage is that redesigned RBF neural network has been fitted in a near stationary region in which there appears to be some kind of maximum minimum, or saddle point, so that, it is now important to determine the nature of the local surface or search region which we tentatively believe contains a maximum, minimum, or saddle point. Theoretical complications which we consider can occur when some of the eigenvalues (given by Eq. (15)) are equal to zero or when certain eigenvalues are equal to one another. The system can be imagined as a limiting case; it is allowed to become
vanishingly small at the same time as the center of the system is moved to infinity. When we say that the model is a like a limiting case, we mean that it is best thought of as approximated by that case. Approximation by a limiting form of some kind is suggested whenever one of the canonical coefficients is close to zero and small in absolute magnitude compared with the other. When this happens, it may also happen that the stationary point for the fitted surface will be remote from the center of the design. The reason for this is that, when the system contains a ridge, steepest ascent will bring the experimental region close to the ridge, but the system that represents that ridge may have a remote center. In this paper time-consuming is not the main objective and future work could be: ‘‘how does our method compare with an RBFNN without using Canonical Analysis? We could compare how many function evaluations it takes these methods (including function evaluations used to train your RBFNN) to find the global optimum. Ideally, our RBFNN method provides some savings in computational cost compared to an ordinary genetic algorithm or an RBFNN without Canonical Analysis’’. And future work too, it will be included the methods performance for a multivariate model with multi-objective optimization.
References Akhtar, T., & Shoemaker, C. (2015). Multi objective optimization of computationally expensive multi-modal functions with RBF surrogates and multi-rule selection. Global Optimization. http://dx.doi.org/10.1007/s10898-015-0270-y. Arbib, M. A. (2003). The handbook of brain theory and neural networks. London, England: The MIT Press. Benyounis, K. Y., & Olabi, A. G. (2008). Optimization of different welding process using statistical and numerical approaches – a reference guide. Advances in Engineering Software, 39, 483–496. Cartis, C., Fowkes, J., & Gould, N. (2015). Branching and bounding improvements for global optimization algorithms with Lipschitz continuity properties. Global Optimization, 61, 429–457. Conn, A. R., Scheinberg, K., & Vicente, L. N. (2009). Introduction to derivative-free optimization. Mathematical Programming Society and the Society for Industrial and Applied Mathematics (MPS-SIAM). Couckuyt, I., Deschrijver, D., & Dhaene, T. (2014). Fast calculation of multiobjective probability of improvement and expected improvement criteria for Pareto optimization. Global Optimization, 60, 575–594. Giordanoa, P. C., Beccariab, A. J., Goicoecheaa, H. C., & Olivieri, A. C. (2013). Optimization of the hydrolysis of lignocellulosic residues by using radial basis functions modeling and particle swarm optimization. Biochemical Engineering Journal, 80, 1–9. Gomes, C., & Canedo, J. (2015). Neural network of Gaussian radial basis function applied to the problem of identification of nuclear accidents in a PWR nuclear power plant. Annals of Nuclear Energy, 77, 285–293. Gregorcˇicˇ, G., & Lightbody, G. (2008). Nonlinear system identification: From multiple-model networks to Gaussian processes. Engineering Applications of Artificial Intelligence, 21(7), 1035–1055. Haji, K., Pirmoradi, Z., Cheng, G., & Wang, G. (2015). Decomposition for large-scale global optimization based on quantified variable correlations uncovered by metamodelling. Engineering Optimization, 429–452. Haykin, S. (1999). Neural networks: A comprehensive foundation. Prentice Hall. Huda, S., Abdollahian, M., Mammadov, M., Yearwood, J., Ahmed, S., & Sultan, I. (2014). A hybrid wrapper-filter approach to detect the source(s) of out-ofcontrol signals in multivariate manufacturing process. European Journal of Operational Research, 857–870. Jiang, B., Zhang, F., Sun, Y., Zhou, X., Jiaxin, D., & Luhong, Z. (2014). Modeling and optimization for curing of polymer flooding using an artificial neural network and a genetic algorithm. Journal of the Taiwan Institute of Chemical Engineers, 45, 2217–2224. Jie, H., Wu, Y., & Ding, J. (2014). An adaptative metamodel-based global optimization algorithm for black-box type problems. Engineering Optimization, 1–22. http://dx.doi.org/10.1080/0305215X.2014.979814. Ko, C.-N. (2013). Reinforcement radial basis function neural networks with an adaptive annealing learning algorithm. Applied Mathematics and Computation, 221, 503–513. Ko, C.-N., & Lee, C.-M. (2013). Short-term load forecasting using SVR (support vector regression)-based radial basis function neural network with dual extended Kalman filter. Energy, 49, 413–422. Li, Y., Liu, L., Long, T., & Chen, X. (2015). Multiple-optima search method based on a metamodel and mathematical morphology. Engineering Optimization, 1–17. http://dx.doi.org/10.1080/0305215X.2015.1016507. Liu, H., Xu, S., Wang, X., Wu, J., & Sond, Y. (2014). A global optimization algorithm for simulation-based problems via the extended DIRECT scheme. Engineering Optimization, 1–18. http://dx.doi.org/10.1080/0305215X.2014.971777.
Please cite this article in press as: Praga-Alejo, R. J., et al. Optimization by Canonical Analysis in a Radial Basis Function. Expert Systems with Applications (2015), http://dx.doi.org/10.1016/j.eswa.2015.04.013
R.J. Praga-Alejo et al. / Expert Systems with Applications xxx (2015) xxx–xxx Liu, H., Xu, S., Ma, Y., & Wang, X. (2015). Global optimization of expensive black box functions using potential Lipschitz constants and response surfaces. Global Optimization. Martín del Brío, B., & Sanz, A. (2007). Redes Neuronales y Sistemas Difusos. España: Alfaomega. Mateo, J., & Rieta, J. J. (2013). Radial basis function neural networks applied to efficient QRST cancellation in atrial fibrillation. Computers in Biology and Medicine, 43, 154–163. Montgomery, D. C. (2012). Design and analysis of experiments. Wiley. Müller, J., & Shoemaker, C. (2014). Influence of ensemble surrogate models and sampling strategy on the solution quality of algorithms for computationally expensive black-box global optimization problems. Global Optimization, 60, 123–144. Myers, R. H., Montgomery, D. C., & Anderson-Cook, C. M. (2009). Response surface methodology: Process and product optimization using designed experiments. Wiley series in probability and statistics: John Wiley & Sons, Inc. Nelles, O. (2001). Nonlinear system identification: From classical approaches to neural networks and fuzzy models. Berlin, Heidelberg, Germany: SpringerVerlag. Oh, S.-K., Kim, W.-D., Pedrycz, W., & Joo, S.-C. (2012). Desing of K-means clusteringbased polynomial radial basis function neural networks (pRBF NNs) realized with the aid of particle swarm optimization and differential evolution. Neurocomputing, 78, 21–132. Paliwal, M., & Kumar, A. U. (2009). Neural networks and statistical techniques: A review of applications. Expert Systems with Applications, 36, 2–17. Park, H.-S., Pedrycz, W., Chung, Y.-D., & Oh, S.-K. (2012). Modeling of the charging characteristic of linear-type superconducting power supply. Expert Systems with Applications, 39, 1021–1039. Praga-Alejo, R. J., Torres-Treviño, L. M., González- González, D. S., Acevedo-Dávila, J., & Cepeda-Rodríguez, F. (2012). Analysis and evaluation in a welding process applying a redesigned radial basis function. Expert Systems with Applications, 39, 9669–9675. Rajendra, M., & Shankar, K. (2015). Improved complex-valued radial basis function (ICRBF) neural networks on multiple crack identification. Applied Soft Computing, 28, 285–300. Romero-Villafranca, R., Zúnica, L., & Romero-Zúnica, R. (2007). Ds-optimal experimental plans for robust parameter design. Journal of Statistical Planning and Inference, 137(4), 1488–1495.
9
Sermpinis, G., Theofilatos, K., Karathanasopoulos, A., Georgopoulos, E. F., & Dunis, C. (2013). Forecasting foreign exchange rates with adaptive neural networks using radial-basis functions and Particle Swarm Optimization. European Journal of Operational Research, 225, 528–540. Seyed Javan, D., Mashhadi, H. R., & Rouhani, M. (2013). A fast static security assessment method based on radial basis function neural networks using enhanced clustering. Electrical Power and Energy Systems, 44, 988–996. Shahlaei, M., Bahrami, G., Abdolmaleki, S., Sadrjavadi, K., & Bagher, M. (2015). Application of unfolded principal component analysis-radial basis function neural network for determination of celecoxib in human serum by threedimensional excitation-emission matrix fluorescence. Spectrochimica Acta Part A: Molecular and Biomolecular Spectroscopy, 138, 675–683. Ting, D., Hongjun, W., & Lei, Shi. (2012). Study of camshaft grinders faults prediction based on RBF neural network. Applied Mechanics and Materials, 141, 519–523. Tyan, M., Van Nguyen, N., & Lee, J. (2014). Improving variable-fidelity modelling by exploring global design space and radial basis function networks for aerofoil design. Engineering Optimization, 1–24. http://dx.doi.org/10.1080/ 0305215X.2014.941290. Ugrasen, G., Ravindra, H. V., Naveen Prakash, G. V., & Keshavamurthy, R. (2014). Process optimization and estimation of machining performances using artificial neural network in wire EDM. Procedia Materials Science, 6, 1752–1760. Wang, H., Jia, M., Huang, P., & Chen, Z. (2010). A study on a new algorithm to optimize ball mill system based on modeling and GA. Energy Conversion and Management, 51, 846–850. Wu, J., Luo, Z., Zhang, N., & Zhang, Y. (2014). A new sampling scheme for developing metamodels with the zeros of Chebyshev polynomials. Engineering Optimization, 1–25. http://dx.doi.org/10.1080/0305215X.2014.963071. Xiong, G., Shi, D., Chen, J., Zhu, L., & Duan, X. (2013). Divisional fault diagnosis of large-scale power systems based on radial basis function neural network and fuzzy integral. Electric Power Systems Research, 105, 9–19. Yua, J., & Duan, H. (2013). Artificial bee colony approach to information granulationbased fuzzy radial basis function neural networks for image fusion. Optik, 124, 3103–3111. Zhu, B., Zhang, X., Fatikow, S., & Wang, N. (2015). Bi-directional evolutionary level set method for topology optimization. Engineering Optimization, 47-3, 390–406. Zhu, Ping., Zhang, S., & Chen, W. (2015). Multi-point objective-oriented sequential sampling strategy or constrained robust design. Engineering Optimization, 47-3, 287–307.
Please cite this article in press as: Praga-Alejo, R. J., et al. Optimization by Canonical Analysis in a Radial Basis Function. Expert Systems with Applications (2015), http://dx.doi.org/10.1016/j.eswa.2015.04.013
Contents lists available at ScienceDirect
Expert Systems with Applications journal homepage: www.elsevier.com/locate/eswa
Review
Optimization by Canonical Analysis in a Radial Basis Function Rolando J. Praga-Alejo a,⇑, Mario Cantú-Sifuentes a,1, David S. González-González b a b
Corporación Mexicana de Investigación en Materiales (COMIMSA), Calle Ciencia y Tecnología, # 790, Frac. Saltillo 400, Saltillo, Coahuila, Mexico Facultad de Sistemas, Universidad Autónoma de Coahuila, Ciudad Universitaria, Carretera a México Km 13, Arteaga, Coahuila, Mexico
a r t i c l e
i n f o
Article history: Available online xxxx Keywords: Radial Basis Function Canonical Analysis Optimization
a b s t r a c t Generally, statistical methods and mathematical models are useful for process optimization. Nonetheless, other methods might be used for modeling and optimizing the manufacturing process. Among these, we can mention the neural networks and the Radial Basis Function technique. Hence, a suitable alternative is complementing statistical methods and neural networks as a Hybrid Learning Process. This work applies the Radial Basis Function Canonical Analysis in order to achieve the welding process optimization. One of the most important results is that the Radial Basis Function neural networks along with the Canonical Analysis are really useful methods. These methods are applied for predicting the optimal point, which establishes a reliable method for the process modeling and optimizing. The Canonical Analysis can determine stationary and saddle points, as it was in this case of study, which Canonical Analysis with RBF represented it adequately and can plot a surface and contour lines. Since in this case of study there is a surface that contains a ridge saddle system, also often called minimax. Then the results show that the Canonical Analysis can explore the region with oblique stationary and rising ridge systems. In this way, the RBF neural network with Canonical Analysis could be an alternative method for analyzing data, whenever the Hybrid Learning Process is adequate or satisfies the test assumption and fulfills the evaluation criteria. In this case of study, validation is represented by the Hybrid Learning Process (Radial Basis Function with Canonical Analysis) presenting an excellent effectiveness. As a conclusion we can say that the resulting Radial Basis Function has improved the model accuracy after using the Canonical Analysis. Ó 2015 Elsevier Ltd. All rights reserved.
1. Introduction Generally, modeling and optimization related with manufacturing processes is made by means of linear models (Romero-Villafranca, Zúnica, & Romero-Zúnica, 2007). Denote as y the random variable describing the statistical behavior of the interest characteristic of a particular product; as x ¼ ðx1 ; x2 ; . . . ; xk Þ the vector that collects the k input variables or operation parameters determining y, where x1 2 Xj . for j ¼ 1; 2; . . . k and Xj are specific variation ranks. It is assumed that y ¼ f ðx1 ; x2 ; . . . ; xk Þ þ e, where f ðÞ is an unknown function from Rk to R, and e is a random variable who describes the effect that non considered factors have on y. It is assumed that the specific distribution of e depends at most of a set of estimable parameters. Then the statistical variation of y and the ⇑ Corresponding author. Tel.: +52 01 844 4 11 32 00x1217. E-mail addresses: [email protected] (R.J. Praga-Alejo), [email protected] (M. Cantú-Sifuentes), [email protected] (D.S. González-González). 1 Tel.: +52 01 844 4 11 32 00x1217.
functional form of f are estimated, via an experimental design, for a polynomial function. The purpose is to get and estimate the optimal value, x ¼ x1 ; x1 ; . . . ; x1 , for the vector of input variables, x; it is:
x ¼ optimizex2X ff ðx1 ; x2 ; . . . ; xk Þ þ eg;
ð1Þ
where, optimize is to be understood as minimize or maximize depending of the underlying context (Conn, Scheinberg, & Vicente, 2009); X ¼ X1 X2 . . . Xk # Rk is the search space. Polynomials of first and second order are technically desirables, and usually sufficient to approximate the function f ðx1 ; x2 ; . . . ; xk Þ., and the usual fitting technique is minimum squares. Residuals are used to describe the behavior of y, and to estimate the parameters of e. Among the experimental designs, the central composite one is frequently used to fit full second order models, whose general form is:
^ ¼ b0 þ y
k X XX bj xj þ bij xi xj þ e; j¼1
ð2Þ
iPj
http://dx.doi.org/10.1016/j.eswa.2015.04.013 0957-4174/Ó 2015 Elsevier Ltd. All rights reserved.
Please cite this article in press as: Praga-Alejo, R. J., et al. Optimization by Canonical Analysis in a Radial Basis Function. Expert Systems with Applications (2015), http://dx.doi.org/10.1016/j.eswa.2015.04.013
2
R.J. Praga-Alejo et al. / Expert Systems with Applications xxx (2015) xxx–xxx
where e is assumed, and tested, to have a normal distribution with mean 0 and unknown variance r2 . The fixed interaction points of the operation parameters must, of course, fall in X. Associated with models Eq. (2) is the response surface methodology, an experimental technique developed to find the optimal response within specified ranges of the parameters. The model in Eq. (2) is used to estimate the assumed curvature in the true, and unknown, response function. If a maximum or minimum exists inside the search space, response surface method can be used to estimate it. Through the fitted polynomial:
^0 þ ^¼b y
k X XX ^ij xi xj : ^ j xj þ b b j¼1
ð3Þ
iPj
The response surface methodology can determine the sub-region
X0 X where y presents an optimum. However, the fitted second order polynomial model in Eq. (3) can provide not an optimal point but a stationary (or a saddle point) in X0 . If this is the case, Canonical Analysis can be used to determine the nature of the stationary point. It can be distinguish two canonical forms: the A form who can be applied if the stationary point is in the frontier of X0 ; and the form B otherwise. Moreover, Canonical Analysis can be used to define the point, line, or subspace on which a maximum, minimum or saddle point is obtained. In the modeling and optimization of manufacturing processes, some other alternative techniques can be used. Among these: Fuzzy logic, Artificial Neural Networks, Evolutionary Algorithms and Hybrid systems (Paliwal & Kumar, 2009; Benyounis & Olabi, 2008). In particular, the use of Radial Basis Functions as activation functions in artificial neural networks can be used to accurately predicting the processes. Such a neural network is generically referred as Radial Basis Function Neural Network (RBFNN). The RBF is a second order or hyper-spherical type function in which the network value represents the distance with respect to a given reference pattern (For more details about RBFNN see Gregorcˇicˇ & Lightbody, 2008; Martín del Brío & Sanz, 2007; Arbib, 2003; Nelles, 2001; Haykin, 1999). The RBF modeling and optimization applications might be founded in Seyed Javan, Mashhadi, and Rouhani (2013), Yua and Duan (2013), Xiong, Shi, Chen, Zhu, and Duan (2013), Sermpinis, Theofilatos, Karathanasopoulos, Georgopoulos, and Dunis (2013), Giordanoa, Beccariab, Goicoecheaa, and Olivieri (2013), Mateo and Rieta (2013), Ko (2013), Ko and Lee (2013), Ting, Hongjun, and Lei (2012), Oh, Kim, Pedrycz, and Joo (2012) and Wang, Jia, Huang, and Chen (2010), Park, Pedrycz, Chung, and Oh (2012). Liu, Xu, Wang, Wu, and Sond (2014) although the three proposed algorithms obtained a good performance, DIRECT scheme was better with respect to compute coming faster to the target. Wu, Luo, Zhang, and Zhang (2014) propose a new sampling method based on the zeros of the Chebyshev polynomial to capture information efficiently sampling. Jie, Wu, and Ding (2014) present an adaptive global optimization (AMGO) the results indicate that the proposed method was satisfactory accuracy and efficiency of the convergence of the iterative process with low computational cost. In the work published by Shahlaei, Bahrami, Abdolmaleki, Sadrjavadi, and Bagher (2015) applied a principal component analysis combined with a RBFNN; the results indicate that the proposed method is a good alternative for modeling. Zhu, Zhang, Fatikow, and Wang (2015) and Zhu, Zhang, and Chen (2015) developed a bidirectional method for solving optimization problems; the method was designed using a RBF as implicit level set function. Cartis, Fowkes, and Gould (2015) have improved techniques for global optimization functions, using built-Lipschitz continuity to develop procedures delimitation and parallelization strategies. Haji, Pirmoradi, Cheng, and Wang (2015) in their scheme propose
for solving optimization problems indicated that problems with strongly correlated variables cannot be decomposed. Couckuyt, Deschrijver, and Dhaene (2014) propose a multi objective optimization algorithm based on Kriging models and probability criteria applicable to standard benchmark problems improved. Liu, Xu, Ma, and Wang (2015) develop a global optimization algorithm using Lipschitz functions and response surfaces for the computations in black box functions; both methods are able to approximate the actual function efficiently, so it is possible to estimate lower bounds efficiently. Rajendra and Shankar (2015) introduced a new type of network for complex values; the accuracy of the proposed network was significantly better compared to the RBF which does not use complex values. Mateo and Rieta (2013) propose a system based on RBF for their ability to select adjustable parameters to minimize the variance at the output of the method. In the work of Tyan, Van Nguyen, and Lee (2014) a global modeling is presented, using low-fidelity and a scaling function to approximate efficiently in optimization problems. Huda et al. (2014) developed a hybrid method for detection of signals outside multivariate control manufacturing processes. Müller and Shoemaker (2014) examined the influence on the quality of the substitute solution computationally expensive algorithm for global optimization in the black box. The data and the performance of the proposed method show that combinations containing a cubic model on RBF and work best regardless of the sampling strategy. Jiang et al. (2014) use a neural network model to investigate the relationship between the process parameters and polymer curing viscosity and curing time; developed an artificial neural network with a genetic algorithm for modeling and optimization in a polymer curing process. Ugrasen, Ravindra, Naveen Prakash, and Keshavamurthy (2014) optimized the parameters in EDM machining process by ANOVA. Akhtar and Shoemaker (2015) proposed a model based on response surface for a multi target approach using a RBF iteratively to calculate approximations of the objective function. Zhu, Zhang, Fatikow et al. (2015) and Zhu, Zhang, and Chen (2015) developed a strategy of sequential sampling for robust design with no restrictions applied to problems with restrictions. Li, Liu, Long, and Chen (2015) applied a RBF for multimodal functions calculation. Gomes and Canedo (2015) propose a Gaussian RBF for diagnostic a system for identifying problems in nuclear power plants with pressurized water reactor. Müller and Shoemaker (2014) present an algorithm for solving optimization problems with linear objective functions. These papers applied in different ways the RBFs, to approximate some objective function optimization problems. The RBFs for these problems reduce computing costs required by other methodologies. Other applications, with RBFs, are used for prediction for obtaining models that represent a good accuracy. But what happens when the stationary point is a saddle point or point of maximum or minimum response that resides well outside the experimental region, or where several response variables must be simultaneously considered?. Thus, Praga-Alejo, Torres-Treviño, González- González, Acevedo-Dávila, and Cepeda-Rodríguez (2012) proposed the redesigned RBF neural network, which can be properly fitted to models because its hybrid learning method is constructed by means of a genetic algorithm that calculates the matrix of centers and the Mahalanobis distance, maximizing the coefficient of determination, R2 . This statistic becomes the evaluation function in the genetic algorithm, improving the accuracy of the RBF. In this paper time-consuming is not the main objective. The advantage is that redesigned RBF neural network has been fitted in a near stationary region in which there appears to be some kind of maximum minimum, or saddle point, so that, it is now important to determine the nature of the local surface or search region which we
Please cite this article in press as: Praga-Alejo, R. J., et al. Optimization by Canonical Analysis in a Radial Basis Function. Expert Systems with Applications (2015), http://dx.doi.org/10.1016/j.eswa.2015.04.013
R.J. Praga-Alejo et al. / Expert Systems with Applications xxx (2015) xxx–xxx
tentatively believe contains a maximum, minimum, or saddle point. And the questions are: Is it, in fact, a maximum, minimum, or saddle point? How can we define the point, line, or space on which a maximum, minimum, or saddle point response is obtained? Then in this paper we propose the use of Canonical Analysis with RBFNN to answer these questions. As illustration (or case study), the proposed approach is applied to a Gas Arc Metal welding process.
2.1. Radial Basis Function (RBF) Neural Network The RBFNN values represent the distance to a given reference pattern (Haykin, 1999). The structure in a RBF, in its basic form, includes three totally different layers: 1. The input layer, which consists of source nodes xi (sensory units). 2. The hidden layer, which is a layer of high dimension and the units (neurons) that conform it are the basic functions for the input data ui ðxi Þ. 3. The output layer dj , which is responsible for the activation of patterns considering the input layer network. Here the notation for outputs is taken as d instead y for custom reasons. The output layer neurons are linear. The hidden layer neurons calculate the difference between the vector of inputs and the centroids. This difference applies a radial function with a Gaussian shape mainly, but it has the advantage of being able to use other radial functions (Gregorcˇicˇ & Lightbody, 2008). The function of transfer radial in Gaussian type adopts the following form:
1 2 : x t k k i 2r2i
ð4Þ
It is simplified by (5):
ui ðxÞ ¼ GðÞ;
ð5Þ
where x are the inputs and t are the centers or centroids formed by the Euclidean distance or Euclidean norm of kx t k2 . When an input vector is introduced, each neuron in the hidden layer has a Radial Basis Function. Each neuron in the hidden layer is the distance between the vector input and its vector of weights. The relationship between inputs and outputs from neural network is given by Eq. (6) where y xj ¼ dj :
yðxÞ ¼
m X wGðkx t i kÞ þ b þ e:
where e is the residual or error in the model and b is the vector of thresholds or bias, which is a column of ones and is added in the matrix G. The matrix G is given by Eq. (7):
2
G¼6 6 .. 4 .
.. .
..
.
7 .. 7: . 5
The R2 coefficient of determination is considered by the Function Evaluation; which represents (as in regression models) the proportion of variance explained by the model or regressor x. A value for R2 close to 1 implies that most of the variability of the prediction y is explained by the model (Montgomery, 2012). Therefore, the objective is to determine the centroids ti such
R2 ¼ 1
y0 y w0 G0 y Pn 2 . : y0 y n i¼1 yi
ð9Þ
2.2. Canonical Analysis Canonical analysis is a method of rewriting a fitted second degree equation (as Eq. (3)) in a form in which it can be more readily understood. The detection of the nature of the system and the location of the stationary point are an important part of the second-order analysis. The nature of the response surface system (maximum, minimum, or saddle point) depends on the signs and magnitudes of the coefficients in the model of Eq. (3). The second-order coefficients (interaction and pure quadratic terms) play a vital role. One must keep in mind that the coefficients used are estimates of the b’s of Eq. (3). These estimated b’s are obtained by Ordinary Least Squares (OLS) so when this technique is transferred to the RBF, the coefficients become the weights of the neural network; these weights are also calculated with OLS. This is when these two techniques are brought together. Thus, even the system itself (saddle, maximum, or minimum points) is part of the estimation process. The stationary point and the general nature of the system arise as a result of a fitted model. This fitted model can be calculated with the RBF neural network. Thus the Nature of the Stationary Point is a Canonical Analysis and it is determined from the signs of the eigenvalues of the matrix B. This entire translation and rotation of axes described here is called a Canonical Analysis of the response system. This is achieved by a rotation of axes which removes all crossproduct terms. We call this simplification the A canonical form. If desired, this may be accompanied by a change of origin to remove first-order terms as well. We call the result the B canonical form (Myers, Montgomery, & Anderson-Cook, 2009). An alternate canonical form:
ð6Þ
i¼1
u11 u12 u1m 3 7 6u 6 21 u22 u2m 7
In this way, it is possible to say that there are different Radial Basis Functions. This work employs the Gaussian function.
that R2 , given by Eq. (9), be maximum. R2 is a global metric evaluation, where yi represents the experimental response, G is given by (7) and w is given by (8):
2. Methods
ui ðxÞ ¼ exp
3
2.2.1. The A canonical form In the first place the variables should be coded as follows (Montgomery, 2012):
X iD ¼
ni ni ; Si
ð10Þ
where ni ¼ Gi ; see Eq. (7):
ð7Þ
uN1 uN2 uNm
– ni is the input variable vector or independent variable in the original form; – ni is the input variable mean and, – Si is the standard deviation of each input variable vector.
The equation is written in a matrix form Gw ¼ d, where the variable 0 and dj is the output or response, d ¼ ½d1 ; d2 ; . . . ; dN 0 w ¼ ½w1 ; w2 ; . . . ; wm which represents the weights determined by Eq. (8), calculated by Ordinary Least Squares:
Thus, we should consider a fitted second degree model in Eq. (3), the equation in matrix form:
1 w ¼ G0 G G0 d:
^ ¼ b0 þ X0D b þ X0D BXD ; y
ð8Þ
ð11Þ
Please cite this article in press as: Praga-Alejo, R. J., et al. Optimization by Canonical Analysis in a Radial Basis Function. Expert Systems with Applications (2015), http://dx.doi.org/10.1016/j.eswa.2015.04.013
4
R.J. Praga-Alejo et al. / Expert Systems with Applications xxx (2015) xxx–xxx
where
3
2
x1 6x 7 6 27 7 XD ¼ 6 6 .. 7; 4 . 5
3
2
b1 6b 7 6 27 7 b¼6 6 .. 7; 4 . 5
xk
2
b11 6 1=2b 12 6 B¼6 .. 6 4 .
1=2b12 b22 .. .
.. .
1=2b1k
1=2b2k
bk
1=2b12
3
1=2b2k 7 7 7 .. 7 5 . bkk
If M0 M ¼ I, rewriting:
^ ¼ b0 þ X0D M ðM0 bÞ þ X0D M ðM0 BMÞ M0 XD : y
ð12Þ
If all signs of ki are negative, this will mean that there is a maximum located in the surface curvature. If all signs of ki are positive, this will mean that there is a minimum. In case there are ki different signs, this will mean there is a saddle point. When the fitted surface is a rising ridge, it can be useful to use a slightly different canonical form; it is usually called the A canonical form. It basically assumes that the axes have been rotated exactly as in the B canonical form, but not first translated to the stationary point. These Eqs. (10)–(18) can be calculated by the RBFNN.
If we now write:
h ¼ M 0 b:
ð13Þ
h given by multiplying the bj vector (coefficient of first order terms of Eq. (3)) by the eigenvectors of B matrix defined in (11). Where mi ¼ ðmi1 ; mi2 ; . . . ; mik Þ0 . The elements of the vector mi ¼ ðmi1 ; mi2 ; . . . ; mik Þ0 are, in fact, the direction cosines of the X i axis in the x space. Furthermore, if we construct a matrix M whose ith columns is mi , the transformation from old ðxÞ to new ðXÞ coordinate is given by Eq. (14). Equivalently, for the respectively parallel coordinate systems x and X, measured from the natural origin, we have the transformation. Because the transformation is an orthogonal one and m0i mj ¼ 0; m0i mi ¼ 1; i; j ¼ 1; 2; . . . ; k, it follows 0
0
that M M ¼ MM ¼ I, that is, M mations are:
1
0
¼ M . Thus the inverse transfor-
X ¼ M0 X D ;
ð14Þ
X is a rotation matrix obtained by multiplying the eigenvectors of B matrix ðMÞ by the original matrix X D given in (11). From these forms, we see that the jth row of M, with elements m1j m2j ; . . . ; mkj , provides the direction cosines of the xj axis in the X space (or of the xj axis in the X space). This can be computed with spectral decomposition. Similar apply to the B canonical form. Therefore:
K ¼ M 0 BM;
ð15Þ
ð16Þ
Eq. (16) is the A canonical form, achieved by the axes rotation which removes all cross-product terms. Thus, A canonical form can be expressed in an expanding form as (17):
^ ¼ b0 þ h1 X 1 þ þ hk X k þ k1 X 21 þ þ kk X 2k : y
ð17Þ
Differentiating Eq. (17) with respect to X. In this way, we find that the stationary point coordinates X 1s ; . . . ; X ks , are:
X is ¼
hi : 2ki
1 xis ¼ B1 b: 2
ð19Þ
Or of 2KX s ¼ h equal than Eq. (18). At this point, the fitted response is equal to:
1 ^s ¼ b0 þ x0is b: y 2
ð20Þ
Or, equivalently:
1 ^s ¼ b0 þ X 0is h: y 2
ð21Þ
e i and ~xi where: The new vectors: X e can be rewritten in coded variable as: –X
e ¼ M 0 ~x; X
ð22Þ
~x ¼ xi xis ; referring to the coordinates that were measured from stationary point S. The B canonical form is achieved by changing the modeling system origin at the stationary point (xis ) given by:
e 0 K X; e ^¼y ^s þ X y
K is a diagonal matrix of eigenvalues of B matrix. In this way, Eq. (12) can be rewritten as:
^ ¼ b0 þ X0 h þ X0 KX: y
2.2.2. The B canonical form This is achieved by differentiating and equating to zero Eq. (11) with respect to X D , and derived in relation to X the Eq. (16). In this way, it is possible finding the stationary point S coordinates:
ð18Þ
The signs of ki eigenvalues defining the kind of surface. where: – ki is the i-th diagonal element of K. – hi measures the surface slope in X ¼ 0 in the rotated axes. – X is indicates how far the canonical axes are in relation to the stationary point ðSÞ. – ki are the characteristic values or symmetric matrix B roots. – mi are the characteristic vectors, where: Bmi ¼ mi ki . M is an orthonormal matrix with standardized k columns characteristic vectors.
ð23Þ
B Canonical form, which might be expressed through an expanding form:
e 2 þ þ kk X e 2: ^¼y ^s þ k1 X y 1 k
ð24Þ
The canonical form of the second-order model given by Eq. (24) is usually called the B Canonical form; these Eqs. (19)–(24) can be calculated by the RBFNN. It is very useful for determining the nature of the fitted response surface, particularly in identifying saddle systems and ridges (Myers et al., 2009). 2.2.3. A or B canonical forms? When should the A or B canonical form must be applied? In design units, the stationary point ðSÞ distance (represented by the system center) from the design center O should be given by:
" #12 " #12 k k X X 2 D ¼ OS ¼ xis ¼ xjs : i¼1
ð25Þ
j¼1
That is:
1 1 D ¼ X 0is X is 2 ¼ x0is xis 2 :
ð26Þ
This means: – If D is greater than 1, the A canonical form must be used to explore the stationary point neighborhood. – If D is closer to zero or one, the B canonical form must be used.
Please cite this article in press as: Praga-Alejo, R. J., et al. Optimization by Canonical Analysis in a Radial Basis Function. Expert Systems with Applications (2015), http://dx.doi.org/10.1016/j.eswa.2015.04.013
5
R.J. Praga-Alejo et al. / Expert Systems with Applications xxx (2015) xxx–xxx
– The A canonical form should be used in presence of a saddle point (Myers et al., 2009). 2.3. Hybrid Learning Process The current paper employs the hybrid intelligent system, genetic algorithm (GA) and other statistical methods (Praga-Alejo et al., 2012). The Hybrid Learning Process involved in the RBFNN applies the following steps: – Step 1: Generate centroids with genetic algorithm. Applies a GA to calculate the matrix of centers t so that it maximizes R2 . Eliminate the Euclidean distance and to calculate the Mahalanobis distance. – Step 2: Apply the Radial Basis Function, e.g. Eq. (4), to obtain the matrix G given by Eq. (7). – Step 3: Generate the weights w with Eq. (8). – Step 4: Evaluating the fitness function in GA. where the coefficient of determination R2 becomes the statistical evaluation function of GA which helps in the accuracy of the prediction and optimization of the model. Thus fit and train the RBFNN model until to maximize R2 . – Step 5: Codified or standardize the variables with Eq. (10). – Step 6: Fitted second degree model as Eq. (11). – Step 7: Apply the Canonical Analysis with the second degree model obtained of the RBF.
In this kind of welding process, two metallic pieces are joined by heat, pressure or a combination of both. In some cases, an external contribution metal might be required. For this process, two metal pieces are joined by heat, through an arc which is generated by an electrode, fine wire and metals. This welding process is improved and protected by inert gases like argon and helium. There are usually three parameters numbers involved in the GMAW process: Speed feed, Voltage and Torch travel speed or speed welding. These parameters are in charge of regulating the welded pieces performance; for instance: penetration, legs and throat (Fig. 1). It is important to mention that this case of study is centered in the throat. The tests were made with the help of a KUKA KR-16 robot, with PW455x Lincoln analog interface. Fig. 2 shows a test sample considered in the experimental design (first test of Table 1). Therewith, the objective is finding the best parameters levels that the process must perform, in order to reach the best response. In this specific case, we must refer to the ‘‘throat’’ (Output variable). 3.2. GMAW process results Mathematical formulation: Given:
y f ðx1 ; x2 ; . . . ; xk Þ þ e where:
3. Application The method was applied in a welding process; nonetheless the proposed RBF model might be applied in different engineering problems.
f ðx1 ; x2 ; . . . ; xk Þ Is the RBF model given by Eq. (6): yðxÞ ¼ b þ e. Is:
3.1. Gas Metal Arc welding application
x : f ðx Þ P f ðxÞ8x 2 Rk
In Table 1, DOE results are shown. These represent a CCD; this means, an analysis of the second order response surface model with RBFNN. The second order model is composed by linear, quadratic and interaction terms or variables. The application was performed in a Gas Metal Arc Welding (GMAW) process.
xi 2 X;
Pm
i¼1 wGðkx
ti kÞþ
x ¼ optimizex2X ff ðx1 ; x2 ; . . . ; xk Þ þ eg where:
X ¼ X1 X2 . . . Xk is the search space. Thus:
Table 1 CCD in the GMAW process.
f ðx1 ; x2 ; x3 Þ;
Input Variables
Output variable
Speed welding
Voltage
Speed feed
Throat
28 25 25 25 25 22 25 19.95 25 28 30.04 22 28 25 25 22 28 22 25
24 28 28 28 28 24 28 28 28 32 28 32 32 28 21.27 24 24 32 34.72
280 305 305 262.95 305 330 305 305 305 280 305 330 330 347.04 305 280 330 280 305
2.28 3.08 2.74 2.27 2.77 2.66 2.39 2.32 2.92 1.28 3.45 4.93 2.19 3.19 2.89 2.94 2.89 3.33 2.54
Fig. 1. The welded pieces.
Please cite this article in press as: Praga-Alejo, R. J., et al. Optimization by Canonical Analysis in a Radial Basis Function. Expert Systems with Applications (2015), http://dx.doi.org/10.1016/j.eswa.2015.04.013
6
R.J. Praga-Alejo et al. / Expert Systems with Applications xxx (2015) xxx–xxx
2
0:82113127
3
6 7 b ¼ 4 0:41637855 5; 0:63385247 2 6 B¼4
1:84458463 0:63299856
0:48938275
3
7 0:74212297 1:06527651 5:
symmetric
1:02056899
– The A canonical form is: Applying Eqs. (13)–(18). Then:
2
0:73017983
3
6 7 h ¼ 4 0:41292658 5
Fig. 2. Test sample in the CCD.
where the bound constraints are:
19:95 P x1 6 30:04; 21:27 P x2 6 34:72; 262:95 P x3 6 347:04: and:
x1 : Speed Welding; x2 : Voltage; x3 : Speed Feed: The best quality of quality and favorable joints in this kind of welding process are: Throat between 2.8 and 3.9 mm. So, with the Table 1 were presented the following results, through the Hybrid Learning Process application. This is achieved through Steps 1 to 7, presenting the respective equations: – First, the coded variables presented as Eq. (10). – Weights wj are computed in the RBF Redesigned Model determined by the RSM, with the complete second order polynomial model applied to Eqs. (4)–(11). Where: ni ¼ Gi rewritten as (8), yield:
1 w ¼ X 0D X D X D y; 3 0:82113127 6 0:41637855 7 7 6 7 6 6 0:63385247 7 7 6 6 1:84458463 7 7 6 7 6 6 0:74212297 7 7 w¼6 6 1:02056899 7: 7 6 7 6 6 1:26599711 7 7 6 6 0:9787655 7 7 6 7 6 4 2:13055301 5 2
3:20666468 Then, obtaining b0 ; b and B: Where: b0 ¼ w0 then w0 ¼ 3:20666468 and:
b0 ¼ 3:20666468;
0:73873248 2 2:16806156 0 6 K¼4 0 0:12977941 0
0
0 0
3 7 5:
1:95638947
The eigenvalues ki signs are different. Thus, the surface presents a saddle point. – hi is in charge of measuring the surface slope in X ¼ 0 in the rotated axes. – X is a rotation matrix obtained by multiplying the matrix B eigenvectors by the original matrix X D ; X ¼ M 0 X D Where: – M are the eigenvectors; thereby the A canonical form ^ ¼ b0 þ X0 h þ X0 KX polynomial is obtained y Where: b0 ¼ w0 , in an expanding form is:
^ ¼ 3:206 þ 0:730X 1 þ 0:412X 2 þ 0:738X 3 2:168X 21 y þ 0:129X 22 þ 1:956X 23 : In this way it is possible finding out the stationary point coordinates X is , applying Eq. (18):
2
0:16839463
3
6 7 X is ¼ 4 1:5908786 5 0:18879995 X is are considered as the best parameters (in coded variables) fitted by the A canonical form. – The B canonical form is: The stationary point S coordinates, Eq. (19), are:
2
3 0:43733706 6 7 xis ¼ 4 1:02107707 5; 1:16663396 X is are considered as the best parameters (in coded variables) fitted by the B canonical form. The fitted response, Eq. (20), at this stationary point S is:
^s ¼ 2:8699495 y ^s is very close to test sample (Fig. 2), and this y ^s is e.g. this data y one of the best region, where we can find the best quality of welding. e i and ~xi . The new vectors are represented by X Where: e ¼ M0 ~x and ~x ¼ xi xis . e can be rewritten in a coded variable as X X Hence, B canonical form can be expressed in an expanding form Eq. (24) as:
e 2 þ 0:129 X e 2 þ 1:956 X e2 ^ ¼ 2:8699495 2:168 X y 1 2 3
Please cite this article in press as: Praga-Alejo, R. J., et al. Optimization by Canonical Analysis in a Radial Basis Function. Expert Systems with Applications (2015), http://dx.doi.org/10.1016/j.eswa.2015.04.013
7
R.J. Praga-Alejo et al. / Expert Systems with Applications xxx (2015) xxx–xxx Table 2 Results in optimization.
Canonical Prediction 5 Target Canonical A Canonical B
4.5
Test
4
1 2
Input Variables Speed Welding
Voltage
Speed Feed
Output variable Throat
17.71 17.71
24.63 24.63
335.98 335.98
3.83 (Fig. 5) 3.89 (Fig. 6)
RBF with Canonical Analysis 3.84 3.84
Throat (mm)
3.5 3 2.5 2 1.5 1
0
2
4
6
8
10 Data
12
14
16
18
20
Fig. 3. Observed (real process) vs. fitted.
– A or B canonical form? The stationary point S distance (center of the system) from the design center O Eq. (26) is:
D ¼ 1:61: The B canonical form must be used whenever D is close to 0 or near to 1. – The fitted model (A and B canonical form). Welding process predictions Fig. 3. Fig. 4 presents the welding process experiment predictions with RBFNN and canonical polynomial.
Fig. 5. Test 1 throat in the real process 3.83 mm.
4. Validation The fitted models present good predictions about the welding process. Still, we must ask: Are those predictions accurate? For this reason, it is necessary validating the model in the real process. Table 2 shows the employed parameters. Table 2 shows the values predicted by RBFNN with Canonical Analysis, as well as two field tests. Fig. 5 shows the first test. This test was performed in order to validate the model optimization proposal in a real GMAW process. The test was performed with these parameters: – Speed Welding = 17.71. – Voltage = 24.63. – Speed Feed = 335.98.
Fig. 6. Test 2 throat in the real process 3.89 mm.
Fig. 5 shows 3.83 mm (throat) versus the predicted 3.84 mm. (Table 2). The second test was performed with the purpose of validating the model optimization proposal in a real GMAW process. The test was performed with these parameters:
8 6 4 2
– Speed Welding = 17.71. – Voltage = 24.63. – Speed Feed = 335.98.
0 -2 -4 2 1
2 1
0 0
-1
-1 -2
-2
Fig. 4. RBFNN in the Canonical polynomial.
The Fig. 6 shows 3.89 mm (throat) versus the predicted 3.84 mm (Table 2). These results are extremely important, due to the fact that the proposed model is a good method for prediction and optimization. As RBF with Canonical Analysis allows the model validation, this can be applied in real cases. For this reason, it is possible performing highly accurate predictions all through the welding process. The
Please cite this article in press as: Praga-Alejo, R. J., et al. Optimization by Canonical Analysis in a Radial Basis Function. Expert Systems with Applications (2015), http://dx.doi.org/10.1016/j.eswa.2015.04.013
8
R.J. Praga-Alejo et al. / Expert Systems with Applications xxx (2015) xxx–xxx
welding pieces were obtained with appropriate acceptance criteria, good quality and favorable joints, achieving the expected success and presenting the parameters required by the GMAW process.
5. Conclusions The Canonical Analysis with RBF method presented in this work successfully applies a statistical evaluation, improving the model prediction and optimization. In this case of study, it is possible to conclude that the proposed model represents a suitable prediction and optimization technique, whenever the experiments are controlled. The results show that statistical and mathematical method, along with neural networks can be considered as good alternative techniques, in order to predict and optimize complex processes. Specifically, this case presents some predicted values (by Canonical Analysis with RBFNN). The results showed 3.83 mm (throat) versus 3.84 mm (predicted). As we can notice, these are extremely close. For this reason, it is possible to have an adequate acceptance criteria, good quality and favorable joints. The Canonical Analysis with RBFNN is a good representation of the response surface. This can be translated into an advantage to find a better optimum point in relation to the welding process. As a conclusion we can say that the Canonical Analysis method with RBFNN is a good optimization technique. The main idea is to improve the model prediction with RBFNN that can employ a second degree polynomial to represent the response in the neighborhood of interest, theorical systems that approximate ridges. Then redesigned RBF neural network has been fitted in a near stationary region in which there appears to be some kind of maximum minimum, or saddle point, so that, it is now important to determine the nature of the local surface or search region which we tentatively believe contains a maximum, minimum, or saddle point. Where the Canonical Analysis can be of great help in characterizing such systems and can locate the optimum too and we could select, for our optimum process that point near the crest of the ridge for the response that yielded the most satisfactory value of the response. In this paper is not the goal the time-consuming. We do not need model-based derivative-free trust-region methods because in our processes need to identify stationary points or rising ridges. When this work was completed, great practical interest was associated with the fact that, for this system, there was not a single point maximum but, locally at least, a plane of near-maxima. This allowed considerable choice of operating conditions; it allowed conditions to be chosen so that other criteria attained their best levels. The Canonical Analysis with RBF as model can determine stationary and saddle points, as it was in this case of study, which Canonical Analysis with RBF represented it adequately and can plot a surface and contour lines. Since in this case of study there is a surface that contains a ridge saddle system, also often called minimax. Then the results show that the Canonical Analysis can explore the region with oblique stationary and rising ridge systems. The advantage is that redesigned RBF neural network has been fitted in a near stationary region in which there appears to be some kind of maximum minimum, or saddle point, so that, it is now important to determine the nature of the local surface or search region which we tentatively believe contains a maximum, minimum, or saddle point. Theoretical complications which we consider can occur when some of the eigenvalues (given by Eq. (15)) are equal to zero or when certain eigenvalues are equal to one another. The system can be imagined as a limiting case; it is allowed to become
vanishingly small at the same time as the center of the system is moved to infinity. When we say that the model is a like a limiting case, we mean that it is best thought of as approximated by that case. Approximation by a limiting form of some kind is suggested whenever one of the canonical coefficients is close to zero and small in absolute magnitude compared with the other. When this happens, it may also happen that the stationary point for the fitted surface will be remote from the center of the design. The reason for this is that, when the system contains a ridge, steepest ascent will bring the experimental region close to the ridge, but the system that represents that ridge may have a remote center. In this paper time-consuming is not the main objective and future work could be: ‘‘how does our method compare with an RBFNN without using Canonical Analysis? We could compare how many function evaluations it takes these methods (including function evaluations used to train your RBFNN) to find the global optimum. Ideally, our RBFNN method provides some savings in computational cost compared to an ordinary genetic algorithm or an RBFNN without Canonical Analysis’’. And future work too, it will be included the methods performance for a multivariate model with multi-objective optimization.
References Akhtar, T., & Shoemaker, C. (2015). Multi objective optimization of computationally expensive multi-modal functions with RBF surrogates and multi-rule selection. Global Optimization. http://dx.doi.org/10.1007/s10898-015-0270-y. Arbib, M. A. (2003). The handbook of brain theory and neural networks. London, England: The MIT Press. Benyounis, K. Y., & Olabi, A. G. (2008). Optimization of different welding process using statistical and numerical approaches – a reference guide. Advances in Engineering Software, 39, 483–496. Cartis, C., Fowkes, J., & Gould, N. (2015). Branching and bounding improvements for global optimization algorithms with Lipschitz continuity properties. Global Optimization, 61, 429–457. Conn, A. R., Scheinberg, K., & Vicente, L. N. (2009). Introduction to derivative-free optimization. Mathematical Programming Society and the Society for Industrial and Applied Mathematics (MPS-SIAM). Couckuyt, I., Deschrijver, D., & Dhaene, T. (2014). Fast calculation of multiobjective probability of improvement and expected improvement criteria for Pareto optimization. Global Optimization, 60, 575–594. Giordanoa, P. C., Beccariab, A. J., Goicoecheaa, H. C., & Olivieri, A. C. (2013). Optimization of the hydrolysis of lignocellulosic residues by using radial basis functions modeling and particle swarm optimization. Biochemical Engineering Journal, 80, 1–9. Gomes, C., & Canedo, J. (2015). Neural network of Gaussian radial basis function applied to the problem of identification of nuclear accidents in a PWR nuclear power plant. Annals of Nuclear Energy, 77, 285–293. Gregorcˇicˇ, G., & Lightbody, G. (2008). Nonlinear system identification: From multiple-model networks to Gaussian processes. Engineering Applications of Artificial Intelligence, 21(7), 1035–1055. Haji, K., Pirmoradi, Z., Cheng, G., & Wang, G. (2015). Decomposition for large-scale global optimization based on quantified variable correlations uncovered by metamodelling. Engineering Optimization, 429–452. Haykin, S. (1999). Neural networks: A comprehensive foundation. Prentice Hall. Huda, S., Abdollahian, M., Mammadov, M., Yearwood, J., Ahmed, S., & Sultan, I. (2014). A hybrid wrapper-filter approach to detect the source(s) of out-ofcontrol signals in multivariate manufacturing process. European Journal of Operational Research, 857–870. Jiang, B., Zhang, F., Sun, Y., Zhou, X., Jiaxin, D., & Luhong, Z. (2014). Modeling and optimization for curing of polymer flooding using an artificial neural network and a genetic algorithm. Journal of the Taiwan Institute of Chemical Engineers, 45, 2217–2224. Jie, H., Wu, Y., & Ding, J. (2014). An adaptative metamodel-based global optimization algorithm for black-box type problems. Engineering Optimization, 1–22. http://dx.doi.org/10.1080/0305215X.2014.979814. Ko, C.-N. (2013). Reinforcement radial basis function neural networks with an adaptive annealing learning algorithm. Applied Mathematics and Computation, 221, 503–513. Ko, C.-N., & Lee, C.-M. (2013). Short-term load forecasting using SVR (support vector regression)-based radial basis function neural network with dual extended Kalman filter. Energy, 49, 413–422. Li, Y., Liu, L., Long, T., & Chen, X. (2015). Multiple-optima search method based on a metamodel and mathematical morphology. Engineering Optimization, 1–17. http://dx.doi.org/10.1080/0305215X.2015.1016507. Liu, H., Xu, S., Wang, X., Wu, J., & Sond, Y. (2014). A global optimization algorithm for simulation-based problems via the extended DIRECT scheme. Engineering Optimization, 1–18. http://dx.doi.org/10.1080/0305215X.2014.971777.
Please cite this article in press as: Praga-Alejo, R. J., et al. Optimization by Canonical Analysis in a Radial Basis Function. Expert Systems with Applications (2015), http://dx.doi.org/10.1016/j.eswa.2015.04.013
R.J. Praga-Alejo et al. / Expert Systems with Applications xxx (2015) xxx–xxx Liu, H., Xu, S., Ma, Y., & Wang, X. (2015). Global optimization of expensive black box functions using potential Lipschitz constants and response surfaces. Global Optimization. Martín del Brío, B., & Sanz, A. (2007). Redes Neuronales y Sistemas Difusos. España: Alfaomega. Mateo, J., & Rieta, J. J. (2013). Radial basis function neural networks applied to efficient QRST cancellation in atrial fibrillation. Computers in Biology and Medicine, 43, 154–163. Montgomery, D. C. (2012). Design and analysis of experiments. Wiley. Müller, J., & Shoemaker, C. (2014). Influence of ensemble surrogate models and sampling strategy on the solution quality of algorithms for computationally expensive black-box global optimization problems. Global Optimization, 60, 123–144. Myers, R. H., Montgomery, D. C., & Anderson-Cook, C. M. (2009). Response surface methodology: Process and product optimization using designed experiments. Wiley series in probability and statistics: John Wiley & Sons, Inc. Nelles, O. (2001). Nonlinear system identification: From classical approaches to neural networks and fuzzy models. Berlin, Heidelberg, Germany: SpringerVerlag. Oh, S.-K., Kim, W.-D., Pedrycz, W., & Joo, S.-C. (2012). Desing of K-means clusteringbased polynomial radial basis function neural networks (pRBF NNs) realized with the aid of particle swarm optimization and differential evolution. Neurocomputing, 78, 21–132. Paliwal, M., & Kumar, A. U. (2009). Neural networks and statistical techniques: A review of applications. Expert Systems with Applications, 36, 2–17. Park, H.-S., Pedrycz, W., Chung, Y.-D., & Oh, S.-K. (2012). Modeling of the charging characteristic of linear-type superconducting power supply. Expert Systems with Applications, 39, 1021–1039. Praga-Alejo, R. J., Torres-Treviño, L. M., González- González, D. S., Acevedo-Dávila, J., & Cepeda-Rodríguez, F. (2012). Analysis and evaluation in a welding process applying a redesigned radial basis function. Expert Systems with Applications, 39, 9669–9675. Rajendra, M., & Shankar, K. (2015). Improved complex-valued radial basis function (ICRBF) neural networks on multiple crack identification. Applied Soft Computing, 28, 285–300. Romero-Villafranca, R., Zúnica, L., & Romero-Zúnica, R. (2007). Ds-optimal experimental plans for robust parameter design. Journal of Statistical Planning and Inference, 137(4), 1488–1495.
9
Sermpinis, G., Theofilatos, K., Karathanasopoulos, A., Georgopoulos, E. F., & Dunis, C. (2013). Forecasting foreign exchange rates with adaptive neural networks using radial-basis functions and Particle Swarm Optimization. European Journal of Operational Research, 225, 528–540. Seyed Javan, D., Mashhadi, H. R., & Rouhani, M. (2013). A fast static security assessment method based on radial basis function neural networks using enhanced clustering. Electrical Power and Energy Systems, 44, 988–996. Shahlaei, M., Bahrami, G., Abdolmaleki, S., Sadrjavadi, K., & Bagher, M. (2015). Application of unfolded principal component analysis-radial basis function neural network for determination of celecoxib in human serum by threedimensional excitation-emission matrix fluorescence. Spectrochimica Acta Part A: Molecular and Biomolecular Spectroscopy, 138, 675–683. Ting, D., Hongjun, W., & Lei, Shi. (2012). Study of camshaft grinders faults prediction based on RBF neural network. Applied Mechanics and Materials, 141, 519–523. Tyan, M., Van Nguyen, N., & Lee, J. (2014). Improving variable-fidelity modelling by exploring global design space and radial basis function networks for aerofoil design. Engineering Optimization, 1–24. http://dx.doi.org/10.1080/ 0305215X.2014.941290. Ugrasen, G., Ravindra, H. V., Naveen Prakash, G. V., & Keshavamurthy, R. (2014). Process optimization and estimation of machining performances using artificial neural network in wire EDM. Procedia Materials Science, 6, 1752–1760. Wang, H., Jia, M., Huang, P., & Chen, Z. (2010). A study on a new algorithm to optimize ball mill system based on modeling and GA. Energy Conversion and Management, 51, 846–850. Wu, J., Luo, Z., Zhang, N., & Zhang, Y. (2014). A new sampling scheme for developing metamodels with the zeros of Chebyshev polynomials. Engineering Optimization, 1–25. http://dx.doi.org/10.1080/0305215X.2014.963071. Xiong, G., Shi, D., Chen, J., Zhu, L., & Duan, X. (2013). Divisional fault diagnosis of large-scale power systems based on radial basis function neural network and fuzzy integral. Electric Power Systems Research, 105, 9–19. Yua, J., & Duan, H. (2013). Artificial bee colony approach to information granulationbased fuzzy radial basis function neural networks for image fusion. Optik, 124, 3103–3111. Zhu, B., Zhang, X., Fatikow, S., & Wang, N. (2015). Bi-directional evolutionary level set method for topology optimization. Engineering Optimization, 47-3, 390–406. Zhu, Ping., Zhang, S., & Chen, W. (2015). Multi-point objective-oriented sequential sampling strategy or constrained robust design. Engineering Optimization, 47-3, 287–307.
Please cite this article in press as: Praga-Alejo, R. J., et al. Optimization by Canonical Analysis in a Radial Basis Function. Expert Systems with Applications (2015), http://dx.doi.org/10.1016/j.eswa.2015.04.013