Support vector machines applied to uniaxial compressive strength prediction of jet grouting columns

Computers and Geotechnics 55 (2014) 132–140

Contents lists available at ScienceDirect

Computers and Geotechnics journal homepage: www.elsevier.com/locate/compgeo

Support vector machines applied to uniaxial compressive strength prediction of jet grouting columns Joaquim Tinoco a,⇑, A. Gomes Correia a, Paulo Cortez b a b

Centre for Territory, Environment and Construction, School of Engineering, University of Minho, Guimarães, Portugal Centro Algoritmi/Dep. de Sistemas de Informação, School of Engineering, University of Minho, Guimarães, Portugal

a r t i c l e

i n f o

Article history: Received 18 April 2013 Received in revised form 5 July 2013 Accepted 28 August 2013 Available online 19 September 2013 Keywords: Data mining Support vector machines Sensitivity analysis Soft-soil Soil cement mixtures Soil improvement Jet grouting Uniaxial compressive strength

a b s t r a c t Learning from data is a very attractive alternative to ‘‘manually’’ learning. Therefore, in the last decade the use of machine learning has spread rapidly throughout computer science and beyond. This approach, supported on advanced statistics analysis, is usually known as Data Mining (DM) and has been applied successfully in different knowledge domains. In the present study, we show that DM can make a great contribution in solving complex problems in civil engineering, namely in the field of geotechnical engineering. Particularly, the high learning capabilities of Support Vector Machines (SVMs) algorithm, characterized by it flexibility and non-linear capabilities, were applied in the prediction of the Uniaxial Compressive Strength (UCS) of Jet Grouting (JG) samples directly extracted from JG columns, usually known as soilcrete. JG technology is a soft-soil improvement method worldwide applied, extremely versatile and economically attractive when compared with other methods. However, even after many years of experience still lacks of accurate methods for JG columns design. Accordingly, in the present paper a novel approach (based on SVM algorithm) for UCS prediction of soilcrete mixtures is proposed supported on 472 results collected from different geotechnical works. Furthermore, a global sensitivity analysis is applied in order to explain and extract understandable knowledge from the proposed model. Such analysis allows one to identify the key variables in UCS prediction and to measure its effect. Finally, a tentative step toward a development of UCS prediction based on laboratory studies is presented and discussed. Ó 2013 Elsevier Ltd. All rights reserved.

1. Introduction and background We are always trying to improve our knowledge of the world and to find solutions to complex problems. To this end, new and more powerful tools are constantly being developed and applied in order to satisfy the most ambitious goals and solve more efficiently complex problems. A good example is the development and the use of advanced statistical analysis to solve real problems characterized by high dimensionality. These tools are usually known as Data Mining (DM) techniques, and they present a very attractive alternative to the traditional statistical analysis. Indeed, in the last decade these tools have spread rapidly throughout computer science and beyond. The goal of DM is to use computational tools to extract useful knowledge from raw data [1]. Therefore, as more data becomes available, more ambitious problems can be tackled. As mentioned above, DM techniques are being applied to solve real complex problems, particularly where conventional ⇑ Corresponding author. E-mail addresses: [email protected] (J. Tinoco), [email protected] (A. Gomes Correia), [email protected] (P. Cortez). URL: http://www3.dsi.uminho.pt/pcortez/Home.html (P. Cortez). 0266-352X/$ - see front matter Ó 2013 Elsevier Ltd. All rights reserved. http://dx.doi.org/10.1016/j.compgeo.2013.08.010

approaches work poorly. Indeed, the best way to assess the real potential and usefulness of such tools is measuring performance when applied to real-world problems. By searching in the bibliography, a wide range of successful applications of DM techniques in different knowledge domains can be found. A few examples are its application in web search, spam filters, recommender systems, and fraud detection [2]. Also, in the field of civil engineering some successful examples can be found. For example, Support Vector Machines (SVMs) and Artificial Neural Networks (ANNs), two of the interesting supervised learning DM algorithms characterized by their flexibility and non-linear learning capabilities, have been applied to solving both classification and regression DM tasks in the geotechnics field [3–5]. Lai and Serra [6] applied ANNs to predict the compressive strength of cement conglomerates. Prasad et al. [7] proposed an ANN to predict the 28-day compressive strength of a normal- and a high-strength self-compacting concrete and high performance concrete with high volume fly ash. Miranda et al. [8] proposed a new alternative regression model using ANNs for the analytical calculation of strength and deformability parameters of rock masses. Goh and Goh [9] used SVMs to assess seismic liquefaction. Ezrin [10] studied the relationship between the swell pressure and soil suction behavior in specimens of Bentonite– Kaolinite clay mixtures with varying soil properties using ANNs.

J. Tinoco et al. / Computers and Geotechnics 55 (2014) 132–140

Tinoco et al. [11] applied SVMs and ANNs to predict deformability properties of Jet Grouting (JG) laboratory formulations over time. Additionally to this SVM application there are many other examples that can be found in the literature. SVMs have the potential to produce high quality predictions in several classification or regression tasks. Particularly, in the civil engineering field, SVMs have been applied in seismic reliability assessment [12], in reliability analysis [13] or in soil type classification [14]. In the present work, we applied a SVM to predict the Uniaxial Compressive Strength (UCS) of JG samples directly extracted from real JG columns. JG technology is a soft-soil improvement method widely applied all over the world [15,16], aiming to yield the best results when we are forced to built under soft-soils characterized by high porosity, plasticity, compressibility and low strength [17]. The Yamakoda brothers introduced this technology in 1960’s decade and since then, it has aroused interest within the geotechnical community due to it great versatility [18]. With JG technology it is possible to improve the mechanical and physical properties of different soil types (ranging from fine- to coarse-grained soils) and to obtain different geometries shapes (columns, panels, etc.). Additionally, to carried out the treatment there are minimal equipment needs and, when compared with similar methods JG is economically attractive. Conceptually, JG technology intends to mix the natural soil with cement slurry, which is directly injected into the subsoil. The new material obtained, also known as soilcrete will be characterized by enhanced resistance, compressibility, and permeability. To produce this improved mass of soil, a rod is driven into the subsoil to the intended depth. After that, grout is injected, with or without other fluids (air and/or water), at high pressure and velocity through small nozzles placed at the end of the rod, which is continually rotated and slowly withdrawn. The high cinematic energy of the injected fluids cuts the soil and mixes them with the injected grout. According to the number of fluids injected, three systems are currently in use [18]. Using the single fluid system, grout only is injected at high pressure and velocity. The double fluid system is very similar to the single system but with the addition of an air shrouding to the nozzle. Finally, the triple fluid system combines conceptually the single and double fluid systems: that is, the erosion of the ground is carried out by a high-pressure water jet shrouded with air and an additional low-pressure grout line is responsible to mix the eroded soil with the cement slurry injected. Depending on the system applied, several parameters need to be controlled. Generally, the most important variables that affect the design of JG columns are soil type, mixture influx between soil and grout, exiting jet energy from the nozzle, grout flow rate, and rotating and lifting speed [19]. JG technology has been applied for different purposes. Padura et al. [20] describe a case study where JG was adopted to consolidate the ground in the restoration of the emblematic building La Normal in Granada. This technology also has been used to improve foundation soils, slope stabilization, and underpinning [21]. Furthermore, JG was also used as a method of performing block stabilization of contaminated soils, as well as for the formation of barriers for the control of contaminant migration in the environmental field [22]. Although the technology has been widely applied over many years, there are still important limitations to overcome. One of the most relevant involves the absence of reliable approaches for JG physical and mechanical properties design. Indeed, even in large-scale works, JG design is essentially based on empirical methods [23,24] and supported by JG companies’ know-how. Fig. 1 shows schematically the quality control process currently used in JG design. The soil heterogeneity and the high number of variables involved in the JG process, sometimes with complex

133

relationships between them, are the main factors for such scenarios. Since the empirical approaches currently applied are often too conservative and have limited applicability, the economy and the quality of the treatment can be affected. Hence, and bearing in mind the high versatility of JG technology and its role in important geotechnical works, the need to develop new and more reliable methods to accurately estimate the effects of the different variables involved in the JG process on physical and mechanical properties of JG material is evident. Supported by case studies, some mathematical expressions have been proposed to predict JG mechanical properties [26,27,19]. However, since such expressions have been based on traditional statistics approaches and use only data from particular conditions, they are unable to give an adequate answer when applied to unseen data, leading to poor results. In order to help to overcome JG columns design complexity, the use of advanced statistical analysis can be seen as a promising alternative. Hence, the application of DM techniques to data from past projects, mainly from large geotechnical works, can provide a strong framework to help to support decisions in future projects, namely in small-scale works where information is scarce. In previous work [28], the performance and learning capabilities of DM tools were successful tested using data from JG laboratory formulations. This was the first time that such tools were applied in the study of JG mechanical properties, particularly to predict UCS of JG laboratory formulations. In the present paper, we analyse and discuss DM techniques’ performance when applied to the prediction of UCS of JG samples collected directly from JG columns. Particularly, an SVM is trained and tested with JG data collected from different JG columns (built under different soil types and with different JG parameters) in order to estimate its UCS over time based on soil and mixtures properties. The main results are here described and interpreted. Furthermore, in order to overcome the main criticism of DM algorithms, that is, their lack of understandable results [9], a novel visualization approach [29] is applied, based on a sensitivity analysis method. Such approach enables the identification of the most important input parameters and their average influence in UCS prediction, allowing a better understanding about what was learned by the model. In addition, based on a two-dimensional (2-D) Global Sensitivity Analysis (GSA), the iteration between input variables is also measured and interpreted. With this paper, an update of preliminary results depicted at [30], where JG data from just one case study were analysed, is made. Furthermore, the sensitivity analysis performed and described at [31] is here improved.

2. Support vector machines algorithm The present study focuses on adapting the SVM algorithm [32] to predicting the UCS of JG samples over time. Following is presented a brief description about the SVM algorithm. SVM performs a supervised learning, where the model is adjusted to a dataset of examples that map I inputs (independent variables) into a given target (the dependent variable). SVM are a very specific class of algorithms, which is characterized by use of kernels, absence of local minima during the learning phase, sparseness of the solution and capacity control obtained by acting on the margin or on number of support vectors. When compared with other types of base learners, such as the widely used multilayer perceptron (also known as back-propagation ANN), SVM represents a significant enhancement in functionality, since it always achieves the optimal learning convergence, while ANNs might get stuck in local minima. By using a non-linear kernel, the SVM implicitly maps the input space into high-dimensional feature space (see Fig. 2). In this feature space, the algorithm finds the best

134

J. Tinoco et al. / Computers and Geotechnics 55 (2014) 132–140

No

Laboratory Formulations

Verified project requirement?

Tests

Yes

No

Test Columns

Verified project requirement?

Tests

Yes

Verified project requirement?

Yes

Project Columns

No

Tests

Fig. 1. Quality control process currently used in JG design [25].

  P linear separating hyperplane yi ¼ xo þ m i¼1 xi /ðxÞ , related to a set of support vector points (most representative points). Hence, SVMs are capable of learning both simple (linear) and complex (non-linear) mapping functions. Besides the theoretical advantages of SVMs over other algorithms, such as ANNs, the model also presents some drawbacks. An important practical question is the selection of the kernel function and its parameters. Although there are some approaches to guide this issue, as proposed in [34] (followed in this study) or [35], more research is needed in order to establish a more robust solution. Moreover, from a practical point of view the most serious problem with SVMs is related with its high computational effort and memory requirements, in particular when it is fitted to very large datasets. SVM was initially proposed for classification problems by Vladimir Vapnik and his co-workers [36]. Later, after the introduction of an alternative loss function proposed by Vapnik [37], called -insensitive loss function (see Fig. 3), it became possible to apply SVM to regression problems [38]. When working with SVM, it is well known that its generalization performance (estimation accuracy) depends on a good setting of hyperparameters: C – penalty parameter;  – width of a -insensitive zone; and c – the kernel parameters. The problem of choosing a good parameter setting in a learning task is the so-called model selection. This task is further complicated by the fact that SVM model complexity (and hence its generalization performance) depends on all three parameters. Parameter C controls the trade-off between the model complexity and the amount up to which deviations larger than  are tolerated and may be viewed as a ‘‘regularization’’ parameter [9]. For example, if C is too large (infinity), then the objective is to minimize the empirical risk only, without regard to model complexity part in the optimization formulation. This parameter is usually determined experimentally (trial-and-error) via the use of a training and test (validation) set. Parameter  controls the width of the -insensitive zone, used to fit the training data. The value of  can affect the number of support vectors used to construct the regression function. The bigger the (value of) , the fewer support vectors are se-

Transformation Ø

Real Space

Feature Space

+ε 0

−ε

−ε

0

+ε

Fig. 3. Example of a linear SVM regression and the -insensitive loss function [40].

lected. On the other hand, bigger  values result in more ‘‘flat’’ estimates. Hence, both C and -values affect model complexity (but in different ways). Selecting a particular kernel type and kernel function parameters is usually based on application-domain knowledge and also should reflect the distribution of input (x) values of the training data. The kernel functions most commonly used are the Gaussian and the Polynomial kernel [39]. The present work adopted the popular Gaussian kernel, since it presents fewer parameters than other kernels:

  kðx; x0 Þ ¼ exp c  kx  x0 k2 ;

c>0

ð1Þ

For model selection, several approaches have been proposed [35,41,42] in order to find the best set of parameters with the least effort (time and computing consuming). In the present work, the values of C and  were defined according to the heuristics proposed by Cherkassky and Ma [34]: C = 3 (for a standardized output) and pffiffiffiffi P  ¼ r^ = N, where r^ ¼ 1:5=N  Ni¼1 ðyi  y^i Þ2 ; yi is the measured value, y^i is the value predicted by a 3-nearest neighbor algorithm, and N the number of examples. To optimize the Kernel parameter c, a grid search of {215, 213, . . . , 23} was adopted under an internal (i.e., applied over training data) 3-fold cross validation [43]. All experiments were implemented in R tool [44], using rminer library [33], which is particularly suitable for SVM training through the use of advanced functions (particularly mining and fit). Before fitting the SVM model, the data attributes were standardized to a zero mean and one standard deviation, and before analysing the predictions, the outputs post-processed with the inverse transformation [43]. 3. Model assessment and interpretation 3.1. Evaluation measures

Support Vectors

Fig. 2. Example of a SVM transformation (adapted from [33]).

One of the best ways to assess the performance of a regression model is to measure the difference between experimental and predicted values for all N examples. In a model for which such difference is close to zero high accuracy is expected. To assess our model, three common metrics were calculated [28]: Mean Absolute Deviation (MAD), Root Mean Squared Error (RMSE), and

135

J. Tinoco et al. / Computers and Geotechnics 55 (2014) 132–140

Coefficient of Correlation (R2). Low values of MAD and RMSE, and an R2 close to the unit value should be interpreted as high predictive capacity of the model. The main difference between MAD and RMSE is that the former is less sensitive to extreme values. The Regression Error Characteristic (REC) curve [45], which plots the error tolerance on the x-axis versus the percentage of points predicted within the tolerance on the y-axis was also adopted during model performance analysis.

3.2. Generalization capacity The overall generalization performance of the trained model was assessed by using 20 runs under a 20-fold cross-validation approach [43]. Under this scheme, the data are divided into 20 different subsets. At each iteration, one is used to test the model and the remainder to fit it. At the end, all data are used for training and testing. Yet, this method requires approximately 20 times more computation, because 20 models must be fitted. The final generalization estimate is evaluated by computing the MAD, RMSE, and R2 metrics for all N test samples.

3.3. Sensitivity analysis Frequently, the absence of explanatory power of DM algorithms is pointed out as their main drawback. In order to overcome this problem, Cortez and Embrechts [29] proposed a novel visualization approach based on a Sensitivity Analysis (SA) method. SA is a simple method that is applied after the training phase and that measures the model responses when a given input is changed, allowing the quantification of the relative importance of each attribute, as well as its average effect on the target variable. In the present work, we applied the GSA method [29], which is able to detect interactions among input attributes. This is achieved by performing a simultaneous variation of F inputs (that can range from 1, one-dimensional SA, denoted as 1-D, to I, I-D SA). Each input is varied through its range with L levels and the remaining inputs fixed to a given baseline value. In this work, we adopted the average input variable value as a baseline (ga) and set L = 12, which allows an interesting detail level under a reasonable amount of computational effort. With the sensitivity response of the GSA, two important visualization techniques can be computed. The input importance barplot shows the relative influence (Ra) of each attribute in the model. To measure this effect, first we calculate the gradient metric for all input:

ga ¼

L  X  y ^a;j  y ^a;j1 =ðL  1Þ

ð2Þ

j¼2

^a;j is the where a denotes the input variable under analysis and y sensitivity response for xa,j. After that, the relative importance is calculated using:

, Ra ¼ g a

I X g i  100ð%Þ

ð3Þ

i¼1

To analyse the average impact of a given input in the fitted model, the Variable Effect Characteristic (VEC) curve can be used. For a given input variable, the VEC curve plots the attribute L level values (x-axis) versus the sensitivity analysis responses (y-axis). Similarly, when a pair of inputs is simultaneously varied (F > 2), the VEC surface can be plotted, showing the average responses to changes in the pair, allowing a more realistic interpretation of the model.

4. Jet grouting data The SVM model proposed in the present paper was fitted and tested with a dataset of 472 results. All samples were collected from different columns, that is, constructed on different places with different soil properties and stored in a box until being tested. This procedure aims to protect the sample’s integrity and maintain its water content. The UCS of each sample was obtained in an unconfined compression test with on sample strain instrumentation [46]. The selection of input variables was based on previous author experience, particularly those achieved with JG laboratory formulations [28], as well as on expert knowledge related to soil–cement mixtures [47]. Thus, the following set of nine input variables was chosen:  relation between the mixture porosity and the volumetric content of cement – n/(Civ)d;  age of the mixture (days) – t;  percentage of clay – %Clay;  JG method – JGM;  inverse of dry density of the soil–cement mixture – 1/qd;  void ratio – e;  cement content (mass of cement/(mass of soil + mass of cement)) – %C;  water content – x;  water/cement ratio – W/C. The main statistics of the input variables are shown in Table 1. For a physical characterization of the natural soil, few soil samples were collected from the different geotechnical works and submitted to some laboratory tests. Although all soils were of clayed nature, they have different percentages of sand, silt, clay, and organic matter. A detailed description of the distinct soil types can be found in Table 2. In the table, the first column denotes the construction site, while the third column shows the number of records that contain such soil. All columns where the samples were extracted were build with cement type CEM I 42.5R and CEM II 42.5R. 5. Results and discussion In this section, the achieved results by the SVM model trained with UCS data collected from JG columns (further termed as SVM-UCS.Field) are presented, analysed, and discussed as well as predictions made of these data through the UCS data collected from laboratory formulations. Although this paper focus only in the results achieved by SVM algorithm, other approaches were also applied under the doctoral works of the first author paper [48]. Thus, Table 3 compares multiple regression, ANNs, functional networks, as well as the analytical expression proposed by EC2 [49] for strength prediction of concrete with SVM-UCS.Field model, using MAD, RMSE and R2 as

Table 1 Summary of the input and output variables in UCS prediction. Variable

Minimum

Maximum

Mean

Standard deviation

n/(Civ)d t (days) %Clay JGM 1/qd (m3/kg) e C W/C

37.88 9.00 22.50 1.00 5.63E4 0.56 0.14 2.50 0.83

79.17 181.00 45.00 3.00 1.68E3 2.99 0.28 96.80 1.05

59.49 46.12 30.84 2.03 8.43E4 1.32 0.22 38.89 0.93

6.88 32.80 6.87 0.38 1.23E4 0.34 0.03 12.12 0.07

UCS (MPa)

0.32

20.27

4.05

2.83

x

136

J. Tinoco et al. / Computers and Geotechnics 55 (2014) 132–140

Table 2 Soil types present in the collected data.

Absolute Deviation

33.0 57.0 52.5 55.0 48.5

27.0 37.0 22.5 45.0 41.0

8.3 1.8 0.4 3.9 1.0

Table 3 Performance comparison of SVM-UCS.Field model with multiple regression analysis (MR-UCS.Field), ANNs (ANN-UCS.Field), functional networks (FN-UCS.Field) and EC2 (EC2-UCS.Field) models, using MAD, RMSE and R2 as performance measure (test set values, best values in bold). Model

MAD

RMSE

R2

EC2-UCS.Field FN-UCS.Field MR-UCS.Field ANN-UCS.Field SVM-UCS.Field

1.75 ± 0.00 1.40 ± 0.00 1.53 ± 0.00 1.41 ± 0.02 1.38 ± 0.01

2.65 ± 0.00 1.95 ± 0.00 2.13 ± 0.01 2.01 ± 0.06 1.99 ± 0.01

0.13 ± 0.00 0.19 ± 0.00 0.43 ± 0.00 0.49 ± 0.03 0.51 ± 0.01

75

39.0 6.0 25.0 0.0 10.5

50

288 69 37 49 29

10.12

MAD = 1.38 ± 0.01 RMSE = 1.99 ± 0.01 R2 = 0.51 ± 0.01

Accuracy (%)

Lean clay (CL) Organic lean clay (OL) Silty clay (CL-ML) Lean clay (CL) Lean clay (CL)

7.59

25

A B D E G

5.06

REC curve Diagonal Line - 45º

0

%MO

20

%Clay

15

%Silt

10

%Sand

5

Frequency

Predicted values (UCS, MPa)

Soil type

0

Site

2.53

100

0

0

5

10

15

20

Experimental values (UCS, MPa) predictive performance measures. This table shows that SVM-UCS.Field model achieved a better performance than the other ones, which is the main reason why we only consider the SVM model for a detailed analysis in the present paper. Fig. 4 compares UCS experimental values of JG samples with those predicted by SVM-UCS.Field model for all 20 runs performed (this relationship is also presented in Fig. 5 through the respective scatter plot). The average SVM-UCS.Field model hyperparameters (see Section 2) and fitted time values (and respective 95% level confidence intervals according to a t-student distribution) are: c = 0.07 ± 0.01,  = 0.1 ± 0.1, and t = 40.94 ± 0.13 s. In Fig. 4 are also identified the areas (shaded) corresponding to an absolute deviation of 20%, 40%, and 60%. As observed in Fig. 4, the SVM algorithm found some difficulties to learn the complex relationships between UCS of JG samples and its contributing factors. The metrics values of SVM-UCS.Field model, with an R2 = 0.51 ± 0.01 (see Fig. 5) cannot be considered as ideal indicators of high accuracy. However, the

20

Target Predictions

15

Absolute Deviation of 60 %

10

Absolute Deviation of 20 %

0

5

UCS (MPa)

Absolute Deviation of 40 %

0

100

200

300

400

Records Fig. 4. Deviation of UCS experimental values against those predicted by the SVMUCS.Field model.

Fig. 5. Relationship between UCS experimental values versus predicted by the SVMUCS.Field model and respective REC curve (adapted from [31]).

REC curve of SVM-UCS.Field model plotted in same figure, evidences some predictive capacity of the proposed model. Indeed, 80% of the records can be predicted by the SVM-UCS.Field model with an absolute deviation lower than 2 MPa, which represents an error of 45% away from the average. Moreover, it should be stressed that the different approaches found in the literature to predict UCS of JG mixtures for different jet systems, at different ages and under different soil conditions are very limited (in terms of applicability) when compared with our approach. Accordingly, although a value of R2 = 0.51 can be viewed as a low value, in the present context and considering the actual state of art, the proposed approach represents an important step toward to UCS prediction of soilcrete mixtures based on analytical expressions. Despite the low accuracy of the model, it can make a valuable contribution toward understanding JG mixtures behavior. Hence, performing a GSA over the SVM-UCS.Field models, some useful information can be extracted. This sensitivity analysis (GSA) approach is an interesting ways of opening ‘‘black box’’ models, since it allows to visualize what are the most relevant inputs and how these inputs do affect the output response. By improving the interpretability of the SVM model, problem domain users can more easily check for its validity and usefulness, thus should be more willing to accept this predictive model. Based on a 1-D GSA, where one variable is changed at each time, keeping the remaining variables at their mean values, the average relative importance of each input variable and the correspondent tstudent 95% confidence interval for all 20 runs performed was quantified. Fig. 6 shows that n/(Civ)d, JGM, t, and %Clay are the four key variables in UCS prediction of JG samples over time. Among them we can identify one that is related to the soil type (%Clay), another related with the JG process (JGM), and two others related to the JG mixture, namely its age and the relation n/(Civ)d that combines the porosity and cement content effect. Furthermore, it is interesting to observe that this variable ranking has a physical explanation and is empirically understandable. Experimental studies related with soil–cement mixtures have shown that both soil properties and age of the mixture should be taken into account in soil–cement mixtures behavior. Moreover, if we are talking about soil improvement using JG technology, it make sense that JG system used should be considered, since it will determine the energy applied, as well as the impact of the fluids against the soil.

137

J. Tinoco et al. / Computers and Geotechnics 55 (2014) 132–140

0

5 n

10

15

20

(C i v )d

19.16

JGM

15.99

t

12.85

%Clay

11.64

C

8.88

1 ρd

8.8

e

8.74

w

8.61

W C

0

5.33

5

10

15

20

Importance (%) Fig. 6. Relative importance of each variable according to the SVM-UCS.Field model, quantified by one-dimensional sensitivity analysis.

For a better understanding of what was learned from the SVMUCS.Field model, Fig. 7 shows the effect of each key variable on UCS prediction. These VEC curves, also quantified by a 1-D GSA, show a predominant non-linear effect on UCS behavior. As expected, according to soil–cement mixtures studies and previous authors’ works related with JG laboratory formulations [25], VEC curve of t has a positive impact on UCS, following an exponential law. This convex shape indicates that the first days of cure are responsible by the main gain of strength of the mixture. VEC curves for both n/(Civ)d and %Clay have a very similar effect, contributing to the not increase of the JG mixtures strength, particularly for n/(Civ)d lower than 60 and %Clay up to 30%. According to the JGM VEC curve, the JG strength and JG system have an almost linear

4.0 3.5 2.0

2.5

3.0

UCS (MPa)

4.5

5.0

n (C iv )d JGM t %Clay

0.0

0.2

0.4

0.6

0.8

1.0

relationship, being the highest strength values achieved when single fluid system is applied. All observations previously exposed (Figs. 6 and 7) were supported on a 1-D GSA, that is, ranging only one attribute at each time and holding the remains at its means values. However, in the real world these conditions rarely if ever exist. Hence, and keeping in mind a more realistic and detailed interpretation of the model, a 2-D GSA was carried out. Here, two variables are changed simultaneously, enabling the measurement of the iteration level between variables, as well as the effect on UCS prediction. In this paper are shown and discussed the results of a 2-D GSA involving the first two key variables, that is, n/(Civ)d and JGM. Table 4 summarizes the iteration level of all variables with the first two key variables previously identified (see Fig. 6). As we can observe, JGM has the highest iteration with n/(Civ)d in UCS prediction of JG mixtures, presenting a relative importance of 14%. Plotting the VEC surface when this pair of variables is changed simultaneously, holding the remaining variables at their means values (see Fig. 8), it is shown that the highest strength is achieved for single fluid JG columns with low values of n/(Civ)d. It is also indicated that UCS prediction is more sensitive to changes in n/(Civ)d than in JGM. Performing a 2-D GSA with the second key variable (JGM), Table 4 shows that t has the strongest iteration with JGM presenting a relative importance of 16%. VEC surface plotted on Fig. 9 depicts the iteration effect on UCS prediction when these two variables are changed simultaneously. An interesting observation taken from this representation is that the t effect on UCS is more pronounced when soil treatment is performed with the single fluid system. For the other systems, mainly for triple system, UCS of JG mixtures just slightly increase over time. Moreover, it is also observed that the influence of the JG system applied will be particularly noteworthy for advanced ages. A tentative step toward a development of UCS prediction based in laboratory studies is now presented and discussed. On previous author’s works [25] a new approach was developed to accurately predict UCS of JG laboratory mixtures (further termed as SVM-UCS.Lab) and within this study the same novelty was applied for field samples from JG columns of geotechnical works. From these studies, a plausible, although tentative framework as a step toward the prediction of UCS of JG columns from the laboratory database was attempted. In Fig. 10, the mean values at 28 days of UCS experimental field samples by geotechnical works are compared with those predicted by the SVM-UCS.Lab model (also by mean values at 28 days of geotechnical works). The analysis of these results shows an acceptable relationship between UCS of laboratory formulations and field samples except for one particular geotechnical work (G). Indeed, if this case is excluded, a relationship between laboratory formulations and field samples with an R2 = 0.64 is achieved. Moreover, it is observed that UCS of field samples is around 11% higher than the equivalent laboratory formulation, following reference values found in the literature [50]. For geotechnical work G, the mean value predicted by the SVM-UCS.Lab model (see [25] for more details) was considerably overestimated. Since the applicability of this model is satisfied for all field records used in this experiment, an attempt to find a plausible justification for such situation was performed.

Table 4 Iteration level (%) of all variables with n/(Civ)d and JGM, according to the SVM-UCS.Field model.

(scaled) Fig. 7. VEC curves for the more relevant variables in UCS prediction, according to SVM-UCS.Field model measured by one-dimensional sensitivity analysis (adapted from [31]).

Key variables

n/(Civ)d

t

%Clay

JGM

1/qd

e

C

x

W/C

n/(Civ)d JGM

– 13

12 16

13 12

14 –

11 11

11 10

13 14

12 11

14 13

J. Tinoco et al. / Computers and Geotechnics 55 (2014) 132–140

7

138

7

5

UCS

4

3

2

6 5

y = 0.89x 2 R = 0.05

2 Fig. 8. VEC surface for n/(Civ)d and JGM in UCS prediction, according to the SVM-UCS.Field model, quantified by a 2-D GSA.

y = 1.11x 2 R = 0.64

4

6

Geotechnical works - A Geotechnical works - B Geotechnical works - E Geotechnical works - G Geotechnical works - D

3

JGM

UCS Experimental (MPa) - mean at 28 days

n (C i v)

Regression excluding marked point Regression with all points Diagonal line - 45º

2

d

3

4

5

6

7

UCS Predicted by SVM-UCS.Lab(MPa) - mean at 28 days Fig. 10. Deviation of the mean by geotechnical works of UCS experimental field samples against those predicted by the SVM-UCS.Lab model (mean values by geotechnical works).

5.5

5.0

4.5

4.0

UCS 3.5

3.0 2.5

JGM t

2.0

1.5

Fig. 9. VEC surface for JGM and t in UCS prediction, according to the SVM-UCS.Field model, quantified by a 2-D GSA.

Considering that the study of field samples mixtures is based on laboratory formulations, we agree that such deviation should be related with a given variable not contemplated by the SVM-UCS.Lab model. However, since this deviation is observed just for one particular geotechnical work, this behavior is probably related with a particular situation of this geotechnical work. Therefore, we performed a deep analysis of all available information related with each of the geotechnical works, such as the amount of cement applied during the soil improvement, the water content of the mixture, the depth where the samples were collected (influence of environment effects), water table level, and so on. Some experiments were also performed, using UCS of each sample normalized by the 28 days strength of the respective formulation. However, no significant differences were observed. The only relevant difference is related to the water table level of geotechnical work G. In this case, there is information that the columns were built below water table level, leading us to conclude that this is probably the reason

for the low values of UCS observed for field samples when compared to what was expected based on laboratory formulations behavior. However, this issue requires a more detailed analysis in order to establish stronger evidence-based conclusions.

6. Final remarks and conclusions The high learning capabilities of Data Mining (DM) tools were here applied to solving a complex geotechnical problem, allowing an assessment of its potential to solve related problems in this knowledge domain. Particularly, these tools were applied to overcoming one of the main drawbacks of Jet Grouting (JG) technology, that is, the absence of accurate design approaches to predicting the mechanical properties of JG mixtures, namely its Uniaxial Compressive Strength (UCS). Indeed, particularly in the preliminary stages of the design of smaller geotechnical works, where information is scarce, the decision regarding the parameter values is based on limited information. In order to overcome this problem, the use of data from past projects can be seen as a viable solution. On the other hand, the application of DM tools to well-organized data gathered from large geotechnical works can provide a strong framework for developing novel and more accurate models, which can be very useful in future projects. Therefore, in the present work, a Support Vector Machines (SVMs) algorithm was fitted to JG data collected directly from field JG columns in order to predict its UCS over time. The obtained results reveal some difficulties of the SVM algorithm to learn the complex relationships between UCS field values and their contributing factors (R2 = 0.51). However, although a value of R2 = 0.51 is far from the ideal result, it should be stressed that the proposed model is able to predict UCS of soilcrete mixtures for different jet systems, at different ages and under different soil conditions, and that in JG process are involved several parameters. Despite some lack of model accuracy, a few interesting observations were pointed out based on a novel Global Sensitivity Analysis (GSA). Performing a one-dimensional (1-D) GSA, the relation between the mixture porosity and the volumetric content of cement

J. Tinoco et al. / Computers and Geotechnics 55 (2014) 132–140

(n/(Civ)d), JG method (JGM), age of the mixture (t), and percentage of clay (%Clay) were identified as key variables on UCS prediction of JG mixtures. Moreover, it is appealing to observe a predominantly non-linear effect of these key variables in soilcrete strength prediction, except for JGM. Particularly, t effect follows an exponential law, corroborating the empirical knowledge related to soil cement mixtures as well as previous authors’ conclusions related to JG laboratory formulations study. In addition, and based on a 2-D GSA, a strong iteration between the first two key variables (n/(Civ)d and JGM) with JGM and t, respectively, was observed. Furthermore, this analysis evidenced that the t effect is more pronounced on JG columns built with the single fluid system. Additionally, a first attempt toward the prediction of UCS columns from the laboratory database was made, showing that UCS of field samples at 28 days of curing is approximately 11% higher than the equivalent laboratory formulation. However, a more detailed study needs to be performed in order to establish stronger evidence-based conclusions. Based on the achieved results, it is clear the contributions of the high learning capabilities of DM techniques can make to improve the knowledge about JG technology. Moreover, the proposed model can be seen as a starting point to describe statistically the actual empirical knowledge related to JG mixtures’ behavior, which are not intended to substitute for the actual approaches, but to complement it. This can be viewed as an important contribution to improving future JG columns design. Furthermore, combining DM tools with detailed sensitivity analysis, an important contribution can be made to JG mixtures’ behavior understanding, optimizing both the quality and the costs of the JG soil improvement. However, it should be strongly stressed that all results presented here, as well as all conclusions taken, are based on the database used. The data were collected from just one private company and there are other potentially relevant variables (e.g., nozzle geometry) that were not considered because they are not measured by the construction company. Moreover, the proposed models should only be applied in the same conditions for which they were developed. In future work, an attempt will be made to improve the predictive capability of the SVM model based on field data. This will be made by combining other variables, subdividing the database into clusters, or even optimizing SVM hyperparameters. Moreover, the SVM algorithm and other DM techniques will be applied in the study of JG columns diameter as well as soilcrete stiffness. Moreover, keeping in mind a practical implementation of the SVM model in field problems and the mathematical complexity of SVM algorithm, the development of a decision support system would be very useful. Such system could be implemented in a computational device (e.g. laptop) through a user friendly graphical interface. Acknowledgements The authors wish to thank to ‘‘Fundação para a Ciência e a Tecnologia’’ (FCT) for the financial support under the strategic project PEst-OE/ECI/UI4047/2011 and the doctoral Grant SFRH/BD/ 45781/2008. Also, the authors would like to thank the interest and financial support by Tecnasol-FGE as well as to Tiago Valente for the dataset gathered from the laboratory formulations. References [1] Fayyad U, Piatetsky-Shapiro G, Smyth P. The KDD process for extracting useful knowledge from volumes of data. Commun ACM 1996;39(11):27–34. [2] Domingos P. A few useful things to know about machine learning. Commun ACM 2012;55(10):78–87. [3] Liao K, Fanb J, Huangb C. An artificial neural network for groutability prediction of permeation grouting with microfine cement grouts. Comput Geotech 2011;38(8):978–86.

139

[4] Samui P. Support vector machine applied to settlement of shallow foundations on cohesionless soils. Comput Geotech 2008;35(3):419–27. [5] Zhao H. Slope reliability analysis using a support vector machine. Comput Geotech 2008;35(3):459–67. [6] Lai S, Serra M. Concrete strength prediction by means of neural network. Construct Build Mater 1997;11(2):93–8. [7] Prasad B, Eskandari H, Reddy B. Prediction of compressive strength of SCC and HPC with high volume fly ash using ANN. Construct Build Mater 2009;23(1):117–28. [8] Miranda T, Correia A, Santos M, Sousa L, Cortez P. New models for strength and deformability parameter calculation in rock masses using data-mining techniques. Int J Geomech 2011;11:44–58. [9] Goh A, Goh S. Support vector machines: their use in geotechnical engineering as illustrated using seismic liquefaction data. Comput Geotech 2007;34(5):410–21. [10] Erzin Y. Artificial neural networks approach for swell pressure versus soil suction behaviour. Can Geotech J 2007;44(10):1215–23. [11] Tinoco J, Gomes Correia A, Cortez P. Using data mining techniques to predict deformability properties of jet grouting laboratory formulations over time. Progr Artif Intell 2011;7026:491–505. [12] Khatibinia M, Fadaee M, Salajegheh J, Salajegheh E. Seismic reliability assessment of rc structures including soil-structure interaction using wavelet weighted least squares support vector machine. Reliability Engineering & System Safety 2013;110:22–33. [13] Tan Xh, Bi Wh, Hou Xl, Wang W. Reliability analysis using radial basis function networks and support vector machines. Comput Geotech 2011;38(2):178–86. [14] Kovacˇevic´ M, Bajat B, Gajic´ B. Soil type classification and estimation of soil properties using support vector machines. Geoderma 2010;154(3):340–7. [15] Falcão J, Pinto A, Pinto F. Case histories of ground improvement solutions using jet-grouting. Geotechnical News; Tecnasol FGE: Fundações e Geotecnia S.A; 2000. [16] Gazzarrini P, Kokan M, Jungaro S. Case history of jet grouting in british columbia. underpinning of CN rail tunnel in North Vancouver. Geotechnical News; The Grout Line; 2005. [17] Liu SY, Zhang DW, Liu ZB, Deng YF. Assessment of unconfined compressive strength of cement stabilized marine clay. Marine Georesour Geotechnol 2008;26(1):19–35. [18] Xanthakos P, Abramson L, Bruce D. Ground control and improvement. WileyInterscience; 1994. ISBN 0471552313. [19] Nikbakhtan B, Ahangari K, Rahmani N. Estimation of jet grouting parameters in Shahriar Dam, Iran. Min Sci Technol 2010;20(3):472–7. [20] Padura A, Sevilla J, Navarro J, Bustamante E, Crego E. Study of the soil consolidation using reinforced jet grouting by geophysical and geotechnical techniques. Construct Build Mater 2009;23(3):1389–400. [21] Welsh J, Burke G. Jet grouting: uses for soil improvement. Geotech Eng Cong (GSP N27) 1991:334–45. [22] Gazaway H, Jasperse B. Jet grouting in contaminated soils. In: Grouting, soil improvement and geosynthetics (GSP 30). New Orleans (LA, USA): ASCE; 1992. p. 206–14. [23] Croce P, Flora A. Analysis of single-fluid jet grouting. Geotechnique 2001;50(6):739–48. [24] Kirsch F, Sondermann W. Ground improvement and its numerical analysis. In: Proceedings of the international conference on soil mechanics and geotechnical engineering, vol. 3. AA Balkema Publishers; 2001. p. 1775–8. [25] Tinoco J, Gomes Correia A, Cortez P. Application of a sensitivity analysis procedure to interpret uniaxial compressive strength prediction of jet grouting laboratory formulations performed by svm model. In: Huybrechts NDN, editor. International symposium & short courses. Brussels, Belgium; 2012. p. 317–26. [26] Lee F, Lee Y, Chew S, Yong K. Strength and modulus of marine clay–cement mixes. J Geotech Geoenviron Eng 2005;131(2):178–86. [27] Narendra B, Sivapullaiah P, Suresh S, Omkar S. Prediction of unconfined compressive strength of soft grounds using computational intelligence techniques: a comparative study. Comput Geotech 2006;33(3):196–208. [28] Tinoco J, Gomes Correia A, Cortez P. Application of data mining techniques in the estimation of the uniaxial compressive strength of jet grouting columns over time. Construct Build Mater 2011;25(3):1257–62. [29] Cortez P, Embrechts M. Using sensitivity analysis and visualization techniques to open black box data mining models. Inform Sci 2013;225:1–17. [30] Tinoco J, Gomes Correia A, Cortez P. Uniaxial compressive strength prediction of jet grouting columns using support vector machines. In: ESM – The Euro Simul Model Conf. Portugal: Guimarães, EUROSIS; 2011. p. 326–30. [31] Tinoco J, Gomes Correia A, Cortez P. Jet grouting mechanicals properties prediction using data mining techniques. In: 4th International conference on grouting and deep mixing. New Orleans. LA, USA: ASCE; 2012. p. 2082–91. [32] Chen Y, Councill I. An introduction to support vector machines: a review. AI Magaz 2003;24(2):105–7. [33] Cortez P. Data mining with neural networks and support vector machines using the R/rminer tool. In: Perner P. 10th Industrial conference on data mining, advances in data mining: applications and theoretical aspects. Berlin, Germany: LNAI 617: Springer; 2010. p. 572–83. [34] Cherkassky V, Ma Y. Practical selection of SVM parameters and noise estimation for SVM regression. Neural Netw 2004;17(1):113–26. [35] Huang C, Lee Y, Lin D, Huang S. Model selection for support vector machines via uniform design. Comput Stat Data Anal 2007;52(1):335–46. [36] Cortes C, Vapnik V. Support vector networks. Machine Learning 1995;20(3):273–97.

140

J. Tinoco et al. / Computers and Geotechnics 55 (2014) 132–140

[37] Smola A. Regression estimation with support vector learning machines. Master’s thesis. Technische Universit at Munchen; Munchen, Germany; 1996. [38] Smola A, Schölkopf B. A Tutorial on Support Vector Regression. Statistics and Computing 2004;14(3):199–222. [39] Hamel L. Knowledge Discovery with Support Sector Machines. WileyInterscience; 2009. ISBN 0470371927. [40] Cortez P, Portelinha M, Rodrigues S, Cadavez V, Teixeira A. Lamb Meat Quality Assessment by Support Vector Machines. Neural Processing Letters 2006;24(1):41–51. [41] Momma M, Bennett K. A pattern search method for model selection of support vector regression. In: Proceedings of the SIAM international conference on data mining. Arizona, USA: Philadelphia: SIAM; 2002. p. 261–74. [42] Frohlich H, Zell A. Efficient parameter selection for support vector machines in classification and regression via model-based global optimization. Proceedings of the International Joint Conference on Neural Networks (IJCNN’05), vol. 3. Montréal, Québec, Canada: IEEE; 2005. p. 1431–6. [43] Hastie T, Tibshirani R, Friedman J. The Elements of Statistical Learning: Data Mining, Inference, and Prediction. second edition ed. New York: SpringerVerlag; 2009. [44] R. Team. R: a language and environment for statistical computing. Viena Austria: R Foundation for Statistical Computing; 2008.

[45] Bi J, Bennett K. Regression error characteristic curves. In: Proceedings of the twentieth international conference on machine learning. Washington, DC, USA: AAAI Press; 2003. p. 43–50. [46] Gomes Correia A, Valente T, Tinoco J, Falção J, Barata J, Cebola D, et al. Evaluation of mechanical properties of jet grouting columns using different test methods. In: 17th International conference on soil mechanics and geotechnical engineering (17th ICSMGE). Alexandria, Egypt: IOS Press; 2009. p. 2169–71. [47] Shibazaki M. State of practice of jet grouting. In: Grouting and ground treatment: proceedings of 3rd international specialty conference on grouting and ground treatment. New Orleans, Louisiana, USA: ASCE; 2004. p. 198–217. [48] Tinoco J. Application of data mining techniques to jet grouting columns design. Ph.D. thesis. Guimarães, Portugal: School of Engineering, University of Minho; 2012. [49] CEN,. Eurocode 2: Design of concrete structures – part 1-1: general rules and rules for buildings. Brussels: European Committee for Standardization; 2004. [50] Van Impe W, Verástegui Flores R, Mengé P, Van den Broeck M. Considerations on laboratory test results of cement stabilised sludge. In: Deep mixing ’05: 1st international conference on deep mixing – best practice and recent advances; 2005. p. 163–8.