- Email: [email protected]
Prediction of Mechanical Properties of Welded Joints Based on Support Vector Regression
Available online at www.sciencedirect.com Available online at www.sciencedirect.com
Procedia Engineering
ProcediaProcedia Engineering 00 (2011) 000–000 Engineering 29 (2012) 1471 – 1475 www.elsevier.com/locate/procedia
2012 International Workshop on Information and Electronics Engineering (IWIEE 2012)
Prediction of Mechanical Properties of Welded Joints Based on Support Vector Regression Gao Shuangshenga*, Tang Xingweia, Ji Shudea, Yang Zhitaob, b
a Shenyang Airspace University, Shen Yang, 110136, China Harbin University of Science and Technology, Harbin, 150080, China
Abstract Support vector regression (SVR) networks were developed based on kernel functions of linear kernel, polynomial kernel, radial basis function (RBF) and Sigmoid in this paper. The input parameters of TC4 alloy plates include weld current, weld speed and argon flow while the output parameters include tensile strength, flexural strength and elongation. The SVR networks were used to build the mechanical properties model of welded joints and make predictions. A comparison was made between the predictions based on SVR and that based on adaptive-network based fuzzy inference system (ANFIS). The results indicated that the predicted precision based on SVR with radial basis kernel function was higher than that with the other three kernel functions and that based on ANFIS.
© 2011 Published by Elsevier Ltd. Keywords: Support vector regression, mechanical properties, modeling
1. Introduction As an advanced connecting technique, welding has been widely used in many fields of industrial production for its high efficiency, energy conservation, high quality, automation and intelligentization. To better predict the mechanical properties of welded joints, it is very important to develop a model between welding parameters and the mechanical properties. In recent years, lots of studies have been done by researchers both in home and abroad, and some prediction methods have been developed which are mainly about neural network or modified neural network[1-3]. Although these methods can get better predictions, the prediction accuracy and training speed are still not enough. The contradiction between * Corresponding author. Tel.: +86-024-89723472; fax: +86-024-89723472 E-mail address: [email protected].
1877-7058 © 2011 Published by Elsevier Ltd. doi:10.1016/j.proeng.2012.01.157
21472
Gao Shuangsheng et al. / Engineering Procedia Engineering (2012) 1471 – 1475 Author name / Procedia 00 (2011) 29 000–000
over-fitting and generalization can not be reconciled easily with these methods. Also, the neural network may converge to a local optimum rather than a global one. Therefore, it is quite necessary to find a faster, more accurate and more efficient prediction method. Support vector machine (SVM) is a new machine learning method developed by Vapnik based on statistical learning with succinct mathematical terms and good generalized applications[4]. Compared with other prediction methods, the rationale of SVM is more complete, and the parameters needed is relatively less, and it can better avoid getting stuck in local optimum. The author used experimental data of mechanical properties of TIG welded joints of TC4 alloys reported in reference[3], and developed the mechanical properties models of welded joints under different technological parameters based on SVR and made predictions. Compared with the prediction results with adaptive-network based fuzzy inference system (ANFIS) used in reference [3], the results showed that SVR was superior than neural network. 2. Support vector regression network As a machine learning method, support vector machine (SVM) is based on statistical learning and can be divided into support vector classifier (SVC) and support vector regression (SVR). The aim of SVR is to ∧ obtain the mapping approximate f (⋅) , i.e. SVR network f (⋅) , according to the given training data . It can be formulated in parameterized formation as: ∧
(1)
f (⋅) =Ck k ( x, xk )
where k (⋅,⋅) is kernel function. Radial basis function (RBF) is chosen as kernel function in this paper. According to the principle of Structural Risk Minimization, formula (1) is the solution of the regularization problem mentioned below: N
= min J 0 ( f ) C ∑ L ( yk , f ( xk ) ) + f ∈F
k =1
1 f, f 2
(2)
F
in which, F is the reproducing Hilbert space defined by k (⋅,⋅) ; which can be defined as: | y − f ( x ) |< ε ⎧⎪0 , L ( y, f ( x ) ) = ⎨ ⎪⎩| y − f ( x ) | , the others
⋅, ⋅
F
the inner product; the loss function, (3)
where L ( y, f ( x ) ) is the error of y obtained by measuring function f (⋅) at x. The parameter C is used to balance the accuracy of approximation and complexity of mapping. The above mentioned regularization problem can be equally described as optimization one with certain restraints. And the optimization problem can be transformed into quadratic programming though solving the coefficients c = [c1 , c2 L, cN ]T by using Lagrange multiplier. N N 1 N ck cl k ( xk , xl ) − ∑ ck yk + ε ∑ | ck | ∑ 2 k ,l 1 = = k 1= k 1
min = J 2 (c) c
(4)
To meet the restraints, let ck < C ,
N
∑c k =1
k
= 0
(5)
The training error is estimated by the formula (2) and the result shows the solution of the above mentioned quadratic programming question is sparse, i.e. only a few ck are not zero. The amount of has nothing to do with the training data N. The corresponding non-zero ck is the so-called SV. The formula (1) can be written as SV network:
1473 3
Gao Shuangsheng et /al. / Procedia Engineering 29 (2012) 1471 – 1475 Author name Procedia Engineering 00 (2011) 000–000 ∧
M
f ( x) = ∑α jφj ( x)
(6)
j =1
where {αj} is non-zero sub-aggregate in {ck}; {φ j ( x )} the corresponding sub-aggregate in k (x,xk) . The performance of SVR depends on the selection of a proper kernel function and its parameters. Common kernel functions include linear kernel, polynomial kernel, radial basis function and Sigmoid, etc. Since different kernel functions may lead to different predictions and generalized applications, how to choose a proper kernel function is a problem in the application field of SVR. At present, the research of kernel functions is not deep enough to provide the guidance of choosing a proper kernel function. Therefore, this paper constructed SVR networks with four kernel functions mentioned above respectively in order to get the optimal kernel function. 3. Prediction model of mechanical properties of welded joints 3.1. Build SVR model The data in this study derived from reference [3], in which TIG welding was used on TC4 alloy plates with 2mm thick by Zhang Yanfei. Twenty-seven groups of data were measured including tensile strength, bending strength and elongation according to the national standard GB2651-1989, GB2653-1989. With Binary linear interpolation method, nine groups of data were got as SVR training samples to make predictions. To compare with the predictions in reference [3] with ANFIS, this study built training models with twenty-seven groups of measured samples in data set based on SVR, and made predictions with nine groups of interpolation estimate samples. That is, making weld current, weld speed and argon flow as input variables (x1,x2,x3) while making tensile strength, bending strength, and elongation as output variables (y) to build SVR model and make predictions. In general, to build the SVR prediction model of mechanical properties of welded joints, input space shall be transformed into high-dimensional space using the nonlinear transformation defined by inner product functions, and get (generalized) optimal hyper plane in this space. The output layer is the linear combination of intermediate nodes, and each intermediate node corresponds to a support vector, as shown in Fig.1. Convex quadratic programming was used in getting the solution to avoid getting stuck in local minima. Another important feature is the sparsity of the solutions, that is, αi corresponding to most samples is zero while αi corresponding to few samples is not zero, thus only a few samples (support vectors) are needed to make up the optimal classifier. y
Output layer y ∑
a1y1
a2y2
K(x,x1)
x1
K(x,x2)
…
x2
weight
alyl
…
Fig.1. Prediction SVR model of mechanical properties of welded joints.
K(x,xl)
xl
inner product
Input layer x
1474 4
Gao Shuangsheng et al. / Procedia Engineering (2012) 1471 – 1475 Author name / Procedia Engineering 00 (2011)29000–000
3.2. The optimization of SVR grid parameters The generalization ability of SVR regression model depends on error ε, error penalty factor C and kernel function parameter γ, so it is significant to optimize parameters ε，С and γ. There are usually three methods to get the optimal parameters: genetic algorithms, chaos optimize algorithms and gridsearch method. The first two methods are suitable for predictions of large sample data, but optimal parameters can’t be ensured. Grid-search method is a method of exhaustion, partitioning each dimension of parameter space into several grids, and inputting all of the grid intersections in the space to get the optimal solution. With this method, optimal parameters can be ensured despite its lower prediction speed, thus it is suitable for small sample prediction models. The sample of prediction model in this paper was small, so the third method was applied. 3.3. Results and discussion This paper took LIBSVM[5] software package as a tool to build SVR network, and optimized parameters with a built-in parameter optimization program based on grid-search method. The optimal parameters were used to do SVR grid training, and predictions of nine groups of samples were made, and the results were shown in Fig.2. 1600
1200
expected value linear polynomial RBF sigmoid
Tensile Strength σb /Mpa
1400 1300 1200 1100 1000 900 800 700 600
1
2
3
4
5
6
expected value linear polynomial RBF sigmoid
1100
flexural strength σfM/Mpa
1500
7
8
1000 900 800 700
9
1
2
3
Test sample
4
5
6
7
Test sample (b)
(a)
expected value linear polynomial RBF sigmoid
30
Elo / (%)
20 10 0 -10
1
2
3
4
5
6
7
8
9
Test smple
-20
(c)
Fig.2. Comparisons of predicted values and expected values (a) tensile strength (b) flexural strength (c) elongation
8
9
Gao Shuangsheng et /al. / Procedia Engineering 29 (2012) 1471 – 1475 Author name Procedia Engineering 00 (2011) 000–000
3.4. Result discussion It can be seen from Fig.2 that predictions of all three kinds of mechanical properties can well reflect trends of expected values when linear kernel, polynomial kernel, radial basis functions and sigmoid were used as kernel functions to make predictions, and the radial basis kernel function had the highest predicted precision. However, predictions of tensile strength and elongation with Sigmoid kernel function had a great disparity with expected values, and almost half of elongation prediction samples got negative values, so this kernel function is unfit for this study. In order to compare prediction abilities of four kernel functions, error analyses were applied, and average errors of mechanical properties predictions with different kernel functions are shown in Table.1. Compared with average errors of the other three kernel functions, average errors of radial basis kernel functions were the lowest, and lower than 1.65%， 3.86%，1.91% in reference [3], and far lower than handling error 7%, so it can meet the practical need. Table 1. Average errors of SVR prediction values and expected values with different kernel functions Mechanical properties
Linear
Polynomial
Radial basis functions
Sigmoid
Tensile strength
2.90%
1.25%
1.41%
44.16%
flexural strength
0.78%
1.16%
0.66%
0.91%
Elongation
7.58%
6.14%
1.82%
183.75%
4. Conclusions SVR is a kind of effective method to predict mechanical properties of welded joints. Different kernel functions have a great impact on the predictions based on SVR network. The predicted precisions based on SVR with radial basis kernel function was higher than that with the other three kernel functions and that based on ANFIS. SVR needs only a few measured data to build the model, and has such advantages as fast modeling, simple, high prediction precision and good generalization. This method provides a new effective way to better predict mechanical properties of welded joints with different parameters, and can provide theoretical optimal designs of parameters in the welding procedure test. References [1] Yousif Y K, Daws K M, Kazem B I. Prediction of Friction Stir Welding Characteristic Using Neural Network. Jordan Journal of Mechanical and Industrial Engineering 2008, 2(3):151-55. [2] Prasanta Kanjilal, Tapan Kumar Pal, Sujit Kumar Majumdar. Prediction of mechanical properties in submerged arc weld metal of C-Mn steel. Materials and Manufacturing Processes 2007, 22(1):114-127. [3] Zhang Yanfei, Dong Junhui, Zhang Yongzhi. Prediction mechanical properties of welded joints based on ANFIS . Hanjie Xuebao/Transactions of the China Welding Institution 2007. 28(9):5-8. [4] Cortes C, Vapnik V. Support-Vector Networks. Machine Learning 1995, 20(3):273-97. [5] Chang C C, Lin C J. Libsvm: Introduction and Benchmarks. http:// www. csie. ntu. edu. Tw~cjlin/papers.html 2004, 3.
1475 5
Procedia Engineering
ProcediaProcedia Engineering 00 (2011) 000–000 Engineering 29 (2012) 1471 – 1475 www.elsevier.com/locate/procedia
2012 International Workshop on Information and Electronics Engineering (IWIEE 2012)
Prediction of Mechanical Properties of Welded Joints Based on Support Vector Regression Gao Shuangshenga*, Tang Xingweia, Ji Shudea, Yang Zhitaob, b
a Shenyang Airspace University, Shen Yang, 110136, China Harbin University of Science and Technology, Harbin, 150080, China
Abstract Support vector regression (SVR) networks were developed based on kernel functions of linear kernel, polynomial kernel, radial basis function (RBF) and Sigmoid in this paper. The input parameters of TC4 alloy plates include weld current, weld speed and argon flow while the output parameters include tensile strength, flexural strength and elongation. The SVR networks were used to build the mechanical properties model of welded joints and make predictions. A comparison was made between the predictions based on SVR and that based on adaptive-network based fuzzy inference system (ANFIS). The results indicated that the predicted precision based on SVR with radial basis kernel function was higher than that with the other three kernel functions and that based on ANFIS.
© 2011 Published by Elsevier Ltd. Keywords: Support vector regression, mechanical properties, modeling
1. Introduction As an advanced connecting technique, welding has been widely used in many fields of industrial production for its high efficiency, energy conservation, high quality, automation and intelligentization. To better predict the mechanical properties of welded joints, it is very important to develop a model between welding parameters and the mechanical properties. In recent years, lots of studies have been done by researchers both in home and abroad, and some prediction methods have been developed which are mainly about neural network or modified neural network[1-3]. Although these methods can get better predictions, the prediction accuracy and training speed are still not enough. The contradiction between * Corresponding author. Tel.: +86-024-89723472; fax: +86-024-89723472 E-mail address: [email protected].
1877-7058 © 2011 Published by Elsevier Ltd. doi:10.1016/j.proeng.2012.01.157
21472
Gao Shuangsheng et al. / Engineering Procedia Engineering (2012) 1471 – 1475 Author name / Procedia 00 (2011) 29 000–000
over-fitting and generalization can not be reconciled easily with these methods. Also, the neural network may converge to a local optimum rather than a global one. Therefore, it is quite necessary to find a faster, more accurate and more efficient prediction method. Support vector machine (SVM) is a new machine learning method developed by Vapnik based on statistical learning with succinct mathematical terms and good generalized applications[4]. Compared with other prediction methods, the rationale of SVM is more complete, and the parameters needed is relatively less, and it can better avoid getting stuck in local optimum. The author used experimental data of mechanical properties of TIG welded joints of TC4 alloys reported in reference[3], and developed the mechanical properties models of welded joints under different technological parameters based on SVR and made predictions. Compared with the prediction results with adaptive-network based fuzzy inference system (ANFIS) used in reference [3], the results showed that SVR was superior than neural network. 2. Support vector regression network As a machine learning method, support vector machine (SVM) is based on statistical learning and can be divided into support vector classifier (SVC) and support vector regression (SVR). The aim of SVR is to ∧ obtain the mapping approximate f (⋅) , i.e. SVR network f (⋅) , according to the given training data . It can be formulated in parameterized formation as: ∧
(1)
f (⋅) =Ck k ( x, xk )
where k (⋅,⋅) is kernel function. Radial basis function (RBF) is chosen as kernel function in this paper. According to the principle of Structural Risk Minimization, formula (1) is the solution of the regularization problem mentioned below: N
= min J 0 ( f ) C ∑ L ( yk , f ( xk ) ) + f ∈F
k =1
1 f, f 2
(2)
F
in which, F is the reproducing Hilbert space defined by k (⋅,⋅) ; which can be defined as: | y − f ( x ) |< ε ⎧⎪0 , L ( y, f ( x ) ) = ⎨ ⎪⎩| y − f ( x ) | , the others
⋅, ⋅
F
the inner product; the loss function, (3)
where L ( y, f ( x ) ) is the error of y obtained by measuring function f (⋅) at x. The parameter C is used to balance the accuracy of approximation and complexity of mapping. The above mentioned regularization problem can be equally described as optimization one with certain restraints. And the optimization problem can be transformed into quadratic programming though solving the coefficients c = [c1 , c2 L, cN ]T by using Lagrange multiplier. N N 1 N ck cl k ( xk , xl ) − ∑ ck yk + ε ∑ | ck | ∑ 2 k ,l 1 = = k 1= k 1
min = J 2 (c) c
(4)
To meet the restraints, let ck < C ,
N
∑c k =1
k
= 0
(5)
The training error is estimated by the formula (2) and the result shows the solution of the above mentioned quadratic programming question is sparse, i.e. only a few ck are not zero. The amount of has nothing to do with the training data N. The corresponding non-zero ck is the so-called SV. The formula (1) can be written as SV network:
1473 3
Gao Shuangsheng et /al. / Procedia Engineering 29 (2012) 1471 – 1475 Author name Procedia Engineering 00 (2011) 000–000 ∧
M
f ( x) = ∑α jφj ( x)
(6)
j =1
where {αj} is non-zero sub-aggregate in {ck}; {φ j ( x )} the corresponding sub-aggregate in k (x,xk) . The performance of SVR depends on the selection of a proper kernel function and its parameters. Common kernel functions include linear kernel, polynomial kernel, radial basis function and Sigmoid, etc. Since different kernel functions may lead to different predictions and generalized applications, how to choose a proper kernel function is a problem in the application field of SVR. At present, the research of kernel functions is not deep enough to provide the guidance of choosing a proper kernel function. Therefore, this paper constructed SVR networks with four kernel functions mentioned above respectively in order to get the optimal kernel function. 3. Prediction model of mechanical properties of welded joints 3.1. Build SVR model The data in this study derived from reference [3], in which TIG welding was used on TC4 alloy plates with 2mm thick by Zhang Yanfei. Twenty-seven groups of data were measured including tensile strength, bending strength and elongation according to the national standard GB2651-1989, GB2653-1989. With Binary linear interpolation method, nine groups of data were got as SVR training samples to make predictions. To compare with the predictions in reference [3] with ANFIS, this study built training models with twenty-seven groups of measured samples in data set based on SVR, and made predictions with nine groups of interpolation estimate samples. That is, making weld current, weld speed and argon flow as input variables (x1,x2,x3) while making tensile strength, bending strength, and elongation as output variables (y) to build SVR model and make predictions. In general, to build the SVR prediction model of mechanical properties of welded joints, input space shall be transformed into high-dimensional space using the nonlinear transformation defined by inner product functions, and get (generalized) optimal hyper plane in this space. The output layer is the linear combination of intermediate nodes, and each intermediate node corresponds to a support vector, as shown in Fig.1. Convex quadratic programming was used in getting the solution to avoid getting stuck in local minima. Another important feature is the sparsity of the solutions, that is, αi corresponding to most samples is zero while αi corresponding to few samples is not zero, thus only a few samples (support vectors) are needed to make up the optimal classifier. y
Output layer y ∑
a1y1
a2y2
K(x,x1)
x1
K(x,x2)
…
x2
weight
alyl
…
Fig.1. Prediction SVR model of mechanical properties of welded joints.
K(x,xl)
xl
inner product
Input layer x
1474 4
Gao Shuangsheng et al. / Procedia Engineering (2012) 1471 – 1475 Author name / Procedia Engineering 00 (2011)29000–000
3.2. The optimization of SVR grid parameters The generalization ability of SVR regression model depends on error ε, error penalty factor C and kernel function parameter γ, so it is significant to optimize parameters ε，С and γ. There are usually three methods to get the optimal parameters: genetic algorithms, chaos optimize algorithms and gridsearch method. The first two methods are suitable for predictions of large sample data, but optimal parameters can’t be ensured. Grid-search method is a method of exhaustion, partitioning each dimension of parameter space into several grids, and inputting all of the grid intersections in the space to get the optimal solution. With this method, optimal parameters can be ensured despite its lower prediction speed, thus it is suitable for small sample prediction models. The sample of prediction model in this paper was small, so the third method was applied. 3.3. Results and discussion This paper took LIBSVM[5] software package as a tool to build SVR network, and optimized parameters with a built-in parameter optimization program based on grid-search method. The optimal parameters were used to do SVR grid training, and predictions of nine groups of samples were made, and the results were shown in Fig.2. 1600
1200
expected value linear polynomial RBF sigmoid
Tensile Strength σb /Mpa
1400 1300 1200 1100 1000 900 800 700 600
1
2
3
4
5
6
expected value linear polynomial RBF sigmoid
1100
flexural strength σfM/Mpa
1500
7
8
1000 900 800 700
9
1
2
3
Test sample
4
5
6
7
Test sample (b)
(a)
expected value linear polynomial RBF sigmoid
30
Elo / (%)
20 10 0 -10
1
2
3
4
5
6
7
8
9
Test smple
-20
(c)
Fig.2. Comparisons of predicted values and expected values (a) tensile strength (b) flexural strength (c) elongation
8
9
Gao Shuangsheng et /al. / Procedia Engineering 29 (2012) 1471 – 1475 Author name Procedia Engineering 00 (2011) 000–000
3.4. Result discussion It can be seen from Fig.2 that predictions of all three kinds of mechanical properties can well reflect trends of expected values when linear kernel, polynomial kernel, radial basis functions and sigmoid were used as kernel functions to make predictions, and the radial basis kernel function had the highest predicted precision. However, predictions of tensile strength and elongation with Sigmoid kernel function had a great disparity with expected values, and almost half of elongation prediction samples got negative values, so this kernel function is unfit for this study. In order to compare prediction abilities of four kernel functions, error analyses were applied, and average errors of mechanical properties predictions with different kernel functions are shown in Table.1. Compared with average errors of the other three kernel functions, average errors of radial basis kernel functions were the lowest, and lower than 1.65%， 3.86%，1.91% in reference [3], and far lower than handling error 7%, so it can meet the practical need. Table 1. Average errors of SVR prediction values and expected values with different kernel functions Mechanical properties
Linear
Polynomial
Radial basis functions
Sigmoid
Tensile strength
2.90%
1.25%
1.41%
44.16%
flexural strength
0.78%
1.16%
0.66%
0.91%
Elongation
7.58%
6.14%
1.82%
183.75%
4. Conclusions SVR is a kind of effective method to predict mechanical properties of welded joints. Different kernel functions have a great impact on the predictions based on SVR network. The predicted precisions based on SVR with radial basis kernel function was higher than that with the other three kernel functions and that based on ANFIS. SVR needs only a few measured data to build the model, and has such advantages as fast modeling, simple, high prediction precision and good generalization. This method provides a new effective way to better predict mechanical properties of welded joints with different parameters, and can provide theoretical optimal designs of parameters in the welding procedure test. References [1] Yousif Y K, Daws K M, Kazem B I. Prediction of Friction Stir Welding Characteristic Using Neural Network. Jordan Journal of Mechanical and Industrial Engineering 2008, 2(3):151-55. [2] Prasanta Kanjilal, Tapan Kumar Pal, Sujit Kumar Majumdar. Prediction of mechanical properties in submerged arc weld metal of C-Mn steel. Materials and Manufacturing Processes 2007, 22(1):114-127. [3] Zhang Yanfei, Dong Junhui, Zhang Yongzhi. Prediction mechanical properties of welded joints based on ANFIS . Hanjie Xuebao/Transactions of the China Welding Institution 2007. 28(9):5-8. [4] Cortes C, Vapnik V. Support-Vector Networks. Machine Learning 1995, 20(3):273-97. [5] Chang C C, Lin C J. Libsvm: Introduction and Benchmarks. http:// www. csie. ntu. edu. Tw~cjlin/papers.html 2004, 3.
1475 5