Accepted Manuscript Title: A novel ANN approach for modeling of alternating pulse current electrocoagulation-flotation (APC-ECF) process: Humic acid removal from aqueous media Authors: Gona Hasani, Hiua Daraei, Behzad Shahmoradi, Fardin Gharibi, Afshin Maleki, Kaan Yetilmezsoy, Gordon McKay PII: DOI: Reference:
S0957-5820(18)30131-9 https://doi.org/10.1016/j.psep.2018.04.017 PSEP 1360
To appear in:
Process Safety and Environment Protection
Received date: Revised date: Accepted date:
9-1-2018 26-4-2018 28-4-2018
Please cite this article as: Hasani, Gona, Daraei, Hiua, Shahmoradi, Behzad, Gharibi, Fardin, Maleki, Afshin, Yetilmezsoy, Kaan, McKay, Gordon, A novel ANN approach for modeling of alternating pulse current electrocoagulation-flotation (APC-ECF) process: Humic acid removal from aqueous media.Process Safety and Environment Protection https://doi.org/10.1016/j.psep.2018.04.017 This is a PDF file of an unedited manuscript that has been accepted for publication. As a service to our customers we are providing this early version of the manuscript. The manuscript will undergo copyediting, typesetting, and review of the resulting proof before it is published in its final form. Please note that during the production process errors may be discovered which could affect the content, and all legal disclaimers that apply to the journal pertain.
A novel ANN approach for modeling of alternating pulse current electrocoagulation-flotation (APC-ECF) process: Humic acid
SC RI PT
removal from aqueous media
Gona Hasani1, Hiua Daraei2, Behzad Shahmoradi2, Fardin Gharibi2, Afshin Maleki2, Kaan Yetilmezsoy3, Gordon McKay4
2,*
U
Student Research Committee, Kurdistan University of Medical Sciences, Sanandaj, Iran.
Environmental Health Research Center, Research Institute for Health Development, Kurdistan
Department of Environmental Engineering, Faculty of Civil Engineering, Yildiz Technical
M
3,*
A
University of Medical Sciences, Sanandaj, Iran.
N
1
University, Davutpasa Campus, 34220, Esenler, Istanbul, Turkey. Division of Sustainability, College of Science and Engineering, Hamad Bin Khalifa
D
4,*
EP
TE
University, Education City, Qatar Foundation, Doha, Qatar.
Afshin Maleki (
[email protected])
CC
Gordon McKay (
[email protected])
A
Kaan Yetilmezsoy (
[email protected])
1
CC
Highlights
EP
TE
D
M
A
N
U
SC RI PT
REVISED GRAPHICAL ABSTRACT (PSEP-D-18-00027.R2)
A
A new alternating pulse current electrocoagulation-flotation process is introduced. APC-ECF system is first modeled for removal of humic acid from water using ANN/DOE.
About 100% of HA is removed from water with an energy consumption of 1.08 kWh/kg HA. 2
ANN/DOE-based approach can describe behavior of a complex electrochemical process.
Abstract
SC RI PT
A novel application of artificial neural networks (ANN) combined with Taguchi orthogonal experimental design methodology (27 runs, 3 levels, 6 factors) was introduced for modeling and optimization of a new alternating pulse current electrocoagulation-flotation (APC-ECF) process for the removal of humic acid (HA) from aqueous media. Two different ANN architectures, such as multilayer perceptron (MLP NN) and generalized feed forward (GFF NN), were proposed and
U
trained to describe the nonlinear behavior of a laboratory-scale batch APC-ECF reactor. Various
N
operating parameters, such as initial HA concentration (C0), initial pH (pH0), electrical
A
conductivity (EC0), current density (CD), and number of pulses (Npls), were used as inputs for the
M
proposed networks, and the HA removal was selected as the output. According to the goodnessof-fit criteria, the computational results showed that the single hidden-layered GFF NN (5:6:1),
D
where a sigmoid axon transfer function was used at its hidden layer and its output layer was
TE
trained by the Levenberg–Marquardt algorithm, showed the best performance (R2 = 0.999, MSE
EP
= 0.00006). For the optimal conditions of C0 = 42 mg/L, pH0 = 6.63, CD = 24.3 A/m2, EC0 = 856 μS/cm, and Npls = 3, the maximum HA removal was obtained based on the predicted outputs of
CC
the best ANN model (GFF NN). The results of the computational analysis clearly corroborated that ANN integrated design of experiments (DOE)-based modeling was rapidly and effectively
A
used for predicting the optimum performance of a complex electrochemical process in removal of HA from water using aluminum electrodes in monopolar arrangement.
Keywords: Artificial neural network; Design of experiments; Electrocoagulation-flotation; Humic acid 3
1 Introduction Humic substances are one of the main issues in drinking water treatment. They are the most abundant natural organic materials derived from microbial activity and decomposition of plant
SC RI PT
and animal residues [1]. They manipulate the re-growth of the microorganisms in water distribution systems, causing color, taste, and odor problems [2,3] in the presence of micro-
pollutants and heavy metals associated with humic substances [3,4]. More significantly, these substances contribute to the formation of disinfection by-products (DBPs) such as
trihalomethanes (THMs) and haloacetic acids (HAAs) [5,6]. Due to the harmful effects noted
U
above, humic substances should be removed from water. Some methods such as coagulation [7],
N
activated carbon adsorption [8], Fenton treatment [9], nano-TiO2 photo-catalysis [10], membrane
A
filtration [11], biological treatment [12], and ozonation [9] have been employed to remove humic
M
substances.
In developed countries, the electrochemical techniques have successfully assisted in
D
improving environmental impact assessments [13–15]. The electrocoagulation/flotation process
TE
includes the in-situ generation of coagulants via the electro-dissolution of a sacrificial anode,
EP
which usually consists of iron or aluminum [16]. The demerits of electrocoagulation are higher cost of electricity and lower efficiency because of the passivation of electrodes, and it is even
CC
worse when aluminum is used. Typically, the cost of the electric power electrocoagulation unit is more than 50% of the capital investment [17,18]. Therefore, it is urgent to introduce
A
technological solutions for a cost-effective use of the electrolysis process. To overcome such drawbacks, it is proposed that the application of a novel current feed style to the electrocoagulation system using an alternating pulse current (APC) will not only prevent the passivation of Al electrodes due to the off-time between each pulse, but also reduce the energy
4
consumption [19–21]. Nevertheless, complexity of the proposed process makes difficult its mathematical description by conventional mechanistic methods [22]. Recently, modeling has gained a great deal of attention for predicting optimum
SC RI PT
performances and process-related conditions in wastewater treatment plants [23–25]. In this regard, artificial intelligence-based methods are frequently used for predicting optimum
performance in several disciplines including water resources and environmental science [26–28]. Among them, the use of artificial neural networks (ANN) for predicting optimum conditions in specific water treatment processes has been received more interest, since many inputs
U
(independent variables) can be handled within the framework of these modeling tools in
N
determination of one or more outputs (dependent variables) [29]. Comparing with traditional
A
mathematical models, ANN-based methods have many advantages such as lack of necessity to
M
involve a mathematical description of the phenomena in the process, and prediction ability using a limited number of experiments [30–32].
D
As seen from the relevant literature, a one-layered back propagation neural network was
TE
conducted for the modeling of the removal of humic substances with ozonation [33]. The best
EP
result was achieved with the use of the Levenberg–Marquardt algorithm [33]. In the study, the hyperbolic tangent function and the linear activation function were selected as the activation and
CC
transfer functions at the hidden layer and at the output layer, respectively [33]. It was concluded that the optimal network structure was consisted of eight inputs, one hidden layer with ten
A
neurons, and one output layer [33]. In another study [34], three and four layered back propagation feed forward networks trained with the Levenberg–Marquardt algorithm have been successfully applied for modeling the performance of a batch-scale electrochemical reactor in chemical oxygen demand (COD) removal. In a more recent study [35], a single hidden-layered feedforward back propagation ANN (9:3:1) has been reported to be adequate for modeling fluoride 5
removal in an electrocoagulation/flotation process. In the study, the “tansig” (tangent sigmoid) activation function was selected for the hidden layer, and “purelin” (linear) transfer function was used for the output layer [35]. The optimization results proved that the ANN model is more
SC RI PT
compatible with the experimental results compared to the response surface methodology (RSM), and it can predict the optimum fluoride removal with a good accuracy [35].
Although several other artificial intelligence-based modeling studies in the recent
literature have been introduced for solving various other real-life environmental problems [36–
42], to the best of the authors’ knowledge, there are no systematic papers specifically devoted to
U
a study of the implementation of an ANN-based approach for modeling of a new alternating pulse
N
current electrocoagulation-flotation (APC-ECF) process in removal of humic acid (HA) from
A
water. Therefore, to fulfill this gap, the present research first utilizes a novel application of ANN
M
combined with Taguchi orthogonal experimental design methodology (27 runs, 3 levels, 6 factors) for modeling and optimization of a laboratory-scale batch APC-ECF reactor for humic
D
acid removal from aqueous media within the experimental domain of the various operating
TE
parameters.
EP
In this study, design of experiments (DOE) and two different ANN network architectures, multilayer perceptron (MLP) and generalized feed forward (GFF), were first used together for
CC
modeling and optimization of a new alternating pulse current electrocoagulation-flotation (APCECF) for the removal of humic acid (HA) from aqueous media. In this study, after a brief
A
explanation of the experimental procedure and DOE methodology, the ANN-based studies (including optimization of network parameters, selection of number of hidden layers, selection of number of processing elements at the hidden layer, selection of learning algorithm, selection of transfer function, and selection of optimal learning parameters) on the proposed APC-ECF process for the removal of HA is described in detail. The accuracy of the predictions obtained is 6
reviewed via several statistical performance indicators. Finally, a sensitivity analysis is performed to explore the influence of input parameters on the dependent variable for the present system. 2 Materials and methods
SC RI PT
2.1 Preparation of feed solution In this study, humic substance obtained from Sigma-Aldrich Co. was used the model pollutant.
The chemical structure and some physicochemical properties of the humic substance molecules for use after their immobilization are shown in Table 1 [43,44]. A stock solution (1 g/L) of humic acid (HA) was prepared by dissolving 1 g of HA in 62.5 mL of NaOH (2 N) solution, as HA
U
dissolves well under alkaline conditions. This solution was made up to 1 L using distilled water
N
(with 10 μS/cm conductivity at 25 °C). This solution was subjected to magnetic agitation for 48 h
A
and then stored at 4 °C in the absence of light [45]. Feed solutions were prepared daily by
M
dilution of the stock solution in deionized water. Potassium nitrate (KNO3) was used as the
D
background electrolyte, and hydrochloric acid (HCl) and sodium hydroxide (NaOH), which were
TE
used to adjust the solution pH, were of analytical grade from Merck (Germany). Aluminum (Al) electrodes were purchased from a local supplier and were prepared by cutting them to the desired
EP
size.
CC
[Table 1, here]
2.2 Experimental set-up and procedure
A
An electrochemical cell of 600 mL with two L-shaped aluminum electrodes in monopolar arrangement were connected to a DC analogue digital power source (RXN-303D, China) was used to carry out the experimental measurements. Fig. 1 illustrates a photograph of the experimental system. The spacing between Al electrodes was 1 cm, and the surface area of the
7
electrodes was about 99.55 cm2. Each electrode was perforated with 20 holes, each 2 mm in diameter [46]. A digital calibrated pH-meter (EC30 pH meter) and a conductivity-meter (Model 3210) were used to measure the pH value and the electrical conductivity of the solution,
SC RI PT
respectively. Two digital multimeters (Brymen BM 201) ammeter and voltmeter were used to measure the current passing through the circuit and the applied potential, respectively. A
magnetic stirring bar was placed on the floor of the chamber and rotated at 100 rpm (≈ 10.47 rad/s) during the experiment.
U
[Fig. 1, here]
N
Before introducing HA solution (500 mL) in to the electrocoagulation reactor, the pH was
A
adjusted to desired initial values (3.0, 7.0, and 9.0) using HCl and NaOH solutions (0.1 N) and
M
also the electrical conductivity was adjusted to selected initial values (500, 1000, and 2000 μS/cm) using KNO3. To follow the progress of the treatment, samples of 10 mL were taken at 10,
D
20, 30, 50, and 70 min intervals. The HA concentration was quantified using UV absorbance at
TE
wavelength of 254 nm with a UV/Vis spectrophotometer. A standard calibration curve of UV254
EP
absorbance against HA concentration (0.1–30 mg/L) was produced, from which the concentration of an unknown sample was obtained. Finally, the amount of HA removed (mg) was calculated for
CC
samples using Eq. (1).
HA removal [1 (C / C0 )] V C0
(1)
A
where V is the solution volume (L); and C0 and C are the HA concentration (mg/L) before and after the EC process, respectively.
8
2.3 Design of experiments Design of experiments (DOE) methodology was implemented as a cost-effective and strategic approach to reduce the number of experiments [47–50]. In this study, 27 experimental runs were
SC RI PT
designed based on the Taguchi method (Taguchi orthogonal array design, L27). It was applied for the investigation of initial HA concentration (C0), initial pH (pH0), electrical conductivity (EC0), time pulse (Tpls), number of pulses (Npls), and voltage (V) in three levels for each parameter. The factors and their respective levels considered in the present experimental design are summarized in Table 2.
A
2.4 Development of ANN-based architecture
N
U
[Table 2, here]
M
The collected experimental data (n = 128) were normalized, randomized, and then randomly divided into three subsets including training set (TR) (60%), cross-validation set (CV) (20%), and
D
testing set (TEST) (20%). A CV set was used to prevent over-training by testing network outputs
TE
during the training procedure. It is noted that trial and error methods are the most common approaches to optimize training parameters and procedures that influence goodness of fit and
EP
predictability including dataset selection and the number of runs. Therefore, in this kind of study,
CC
a lot of randomly selected subsets were tested, and the model was run several times for each of them. The best run determined the best subset, and this selected subset was kept same for the next
A
steps like algorithm selection. Variations of the normalized responses of all data points are illustrated in Fig. 2. In this study, All ANNs were built within the framework of NeuroSolutions software (Version 5, NeuroDimension, Inc., Gainesville, FL, US) [51], running on a AMD AthlonTM II X3 460 Processor 3.40 GHz, 4 GB of RAM) PC, for modeling and simulation purposes. 9
[Fig. 2, here] It is noted that, in electrocoagulation/electrolysis studies, only one parameter (voltage or electrical current) can be independently adjustable as a variable, and the other one can be
SC RI PT
determined according to Ohm’s Law (R=V/I), when the resistance (or conductivity) of the solution is determined. Therefore, in the experimental design (DOE methodology), only voltage and EC0 were considered and current density (CD) was determined by ammeter. The parameters Tpls and V were neglected in the ANN study. The ANN is based on a black-box modeling
approach that the model is not extracted from the knowledge of the system. It means that the
U
significant parameters that should be included in the model and insignificant parameters that
N
should be neglected would be identified by statistical approaches or trials and errors.
A
Additionally, the economics of model force the user to study with the minimum the number of
M
included parameters to prevent the over-training and other deficiencies that would be risen from a
D
complex model. For these reasons, the inputs and outputs of the network were introduced based
TE
on the variables definition. To simplify the modeling process and over-training prevention, more important factors were selected by the sensitivity analysis. These factors including, C0, pH0, EC0,
EP
CD, and Npls were used as inputs for the proposed networks, and the HA removal (mg) was defined as the network output.
CC
The mean squared error (MSE), determination coefficient (R2), normalized mean squared
error (NMSE), and mean absolute error (MAE) were selected to appraise the goodness of the
A
estimate. The MSE is defined as errors squared to penalize the larger errors and to cancel the effect of the positive and negative values of the differences [52]. Consequently, the best ANN models are those that can successfully predict the output (herein HA removal) in the testing set with the highest R2 and the lowest MSE.
10
There are several parameters (e.g., type of architecture, algorithm of optimizing network parameters, initial adjusted parameters values, etc.) that should be optimized by a trial and/or error approach for determination of the ANN network performance. Among various ANN
SC RI PT
architectures, the multilayer perceptron (MLP) is the most widely used ANN type for classification or regression-based problems [38]. The generalized feed forward (GFF), which has the additional layer-to-layer forward connections showing additional computing power over standard MLP, are presented in this study. In the following, the methodology of the studied architectures is described in detail.
U
The number of hidden layers and the number of processing elements (PEs) at the hidden
N
layers determine the performance, as well as the complexity of the network. Initially, the number
A
of hidden layers were studied from 1 to 3, and the other architecture parameters were chosen
M
based on the software default settings. The number of PEs in the input layer was 5, the number of PEs in all hidden layers was 4, and the number of PEs at the output layer was 1. The error
D
threshold was set to 0.01, the transfer function in all layers was tanh, the learning rule was
TE
momentum, step size was equal to 1, and finally, the momentum coefficient was equal to 0.7.
EP
2.5 Learning algorithm selection
Several types of training algorithms, such as LM, CG, MOM, QP, STP, LA, and DBD
CC
(Levenberg–Marquardt, Conjugate-Gradient, Momentum, Quick-Propagation, Step, and DeltaBar-Delta, respectively), were implemented for learning convergence. It is noted that the most
A
appropriate learning algorithm should be chosen ensuring that the convergence should be sufficiently quick on both TR and CV data. Additionally, the prediction error (i.e., MSE, MIN AVE MSE, NMSE, and MAE) should be the lowest possible, so that the determination coefficient (R2) should be the highest.
11
2.6 Transfer function selection After choosing the best number of hidden layers, the number of PEs (axons) at the hidden layer and the learning algorithm, the activation function was then optimized. The different types of
SC RI PT
activation functions like axon, bias axon, linear axon, tanh axon, linear tanh axon, sigmoid axon, and linear sigmoid axon were verified for the best performance and convergence. 2.7 Optimal learning parameters selection
Several computer-based simulations were employed to determine the optimal learning
parameters. The learning constants and momentum coefficients of the processing elements at the
U
hidden and output layers of the proposed ANN models were studied. The MOM learning
N
algorithm was conducted for training, and the linear axon transfer function was implemented to
A
the PEs at the hidden and output layers of the neural network. Initially, the momentum coefficient
M
of the processing elements at the hidden layer was set to a random default value of 0.7. Likewise,
D
the learning constant (LC) and momentum coefficients (MC) of the PEs at the output layer were
TE
also assigned to random default values of 0.1 and 0.7, respectively. In fact, these initial random values were effectively determined by the NeuroSolutions 5.0 neural network design tool. Based
EP
on the above default settings, the learning constant of the PEs (i.e., PE = 9) belonging to the
CC
hidden layer was ranged from 0.1 to 1.0 with an increment rate of 0.1 units. 3 Results
A
3.1 Optimization of network parameters At run 23, the highest removal efficiency (about 100%) with the lowest electrical energy consumption (1.08 kWh/kg HA) was achieved under the following operating conditions: initial humic acid concentration = 50 mg/L, pH0 = 7.0, electrical conductivity = 500 μS/cm, time pulse = 5, number of pulses = 10, voltage = 5 V, and time of reaction = 10 min. On the other hand, at run 12
11, the lowest removal efficiency (14%) with an electrical energy consumption of 160 kWh/kg HA was obtained under the following operating conditions: initial humic acid concentration = 20 mg/L, pH0 = 3.0, electrical conductivity = 2000 μS/cm, time pulse = 5, number of pulses = 10,
SC RI PT
voltage = 10 V, and time of reaction = 10 min. Table 3 summarizes the optimized parameters of the two studied ANN architectures (MLP NN model and GFF NN model). [Table 3, here]
In each of the studied ANN architectures, the network parameters (including the number
U
of hidden layers, the number of processing elements (axons) in each hidden layer, type of transfer
N
axon (tanh axon, sigmoid axon, etc.) and mathematical functions (momentum/Quick prob, etc.))
A
were optimized by the trial and the error method. The training iterations were set at 3 runs with
M
1000 epochs for each run, enabling the termination option by the CV performance. The values of the parameters, where the best performance was obtained, were chosen as the set of optimal
D
values. Thus, each network parameter was selected by investigating every single effect imposed
TE
by each of them on the performance of the network.
EP
3.2 Selection of number of hidden layers The goodness of fit of model parameters presented in Table 4 reveals that the MLP NN
CC
configuration and GFF NN configuration with single hidden layer show a reasonable performance. There is famous statement in ANN that one hidden layer with sufficient nodes can
A
solve any of problem that can be presented in one mathematical equation without attention to its complexity. As indicated in Table 4, more hidden layer (i.e., from 1 to 3) may cause to more error especially in predictability of the network. However, naturally, it should cause to more precision in solving the problem. This disagreement could be interpreted by the concept of over-training. It means that more nodes and layers cause to more parameters in models that make ANN more 13
powerful to model the training data and decrease in error of fitting. However, it cannot improve the predictability of model. Overall, the simpler models with less parameters are always preferred. These results are accordance with famous statement in the ANN field.
SC RI PT
[Table 4, here] 3.3 Selection of number of PEs at the hidden layer
When the hidden layer architectures were selected, the PEs were studied from 1 to 10 for the
hidden layers. According to Fig. 3, the lowest “MIN AVE MSE” on both TR and CV data sets
U
was obtained for 9 and 6 PEs for the MLP NN configuration and the GFF NN configuration,
N
respectively. Thus, the general optimal design configuration of MLP NN consists of input layers
A
with 5 PEs, single hidden-layer with 9 PEs, and an output layer with one number of PEs (MSE =
M
0.0042 for TR data set, MSE = 0.0097 for CV data set, and R2 = 0.966). On the other hand, the general optimal design architecture of GFF NN consists of input layers with 5 PEs, single hidden-
D
layer with 6 PEs, and an output layer with one number of PEs (MSE = 0.0046 for TR data set,
EP
[Fig. 3, here]
TE
MSE = 0.0094 for CV data set, and R2 = 0.978).
CC
3.4 Selection of learning algorithm The nature of convergence on the TR data for various learning algorithms in MLP NN is shown
A
in Fig. 4a. On this basis, it was observed that the convergence of the TR data is fast with STP, MOM, QP, and DBD as compared to the rest of the other learning algorithms. In the same way, the nature of convergence of the TR data for various learning algorithms in GFF NN is shown in Fig. 4b. In cases of MOM, DBD, and LM learning algorithms, the convergence of the TR data is fast compared to the rest of the other learning algorithms. 14
[Figs. 4, here] Figs. 5a and 5b illustrate the variation of “MIN AVE MSE” on TR and CV data for different learning algorithms in the MLP NN and GFF NN, respectively. With the use of the
SC RI PT
MOM learning algorithm, the “MIN AVE MSE” was the lowest of on the CV (0.0090) data set for the MLP NN, while the “MIN AVE MSE” was the lowest on both TR (0.0007) and CV
(0.0036) data sets with the use of the LM learning algorithm for the GFF NN. Table 5 shows the variations of R2 and MSE for the CV data and TEST data with different learning rules for both
U
architectures.
N
[Figs. 5, here] and [Table 5, here]
A
For the MLP NN, the highest R2 and the minimum MSE for the CV data were in the cases
M
of LM and CG, and MOM. For the GFF NN, the highest R2 and minimum MSE for both CV data
D
and TEST data were in the case of the LM learning algorithm. In view of the overall
TE
performance, as shown in Figs. 4 and 5 and Table 5, it seems that the MOM learning algorithm for the MLP NN and LM learning algorithm for the GFF NN give the best optimal results. As
EP
seen from Table 5, in most cases, errors are obtained to be smaller for the testing set than the one used for cross-validation set. It is noted that the error on the cross-validation set was continuously
CC
monitored during the training process, and the validation error normally decreased during the
A
initial phase of training, as did the training set error. However, when the network began to overfit the data, the error on the validation set typically began to rise [41]. On the other hand, it is also possible to have a higher training or validation error (which is not caused by overfitting) when the training set is large, but a testing set is small. This may also be attributed to the combinatorial
15
nature and non-linear structure of the considered problem, as well as to the MSE performance index and the characteristics of the input vector used in this study. 3.5 Selection of transfer function
SC RI PT
The nature of convergence on the TR data for different transfer functions in MLP NN is shown in Fig. 6a. Based on the nature of convergence, it is observed that the convergence on TR data set is faster with tanh axon (MSE = 0.0041), bias axon (MSE = 0.0030), and linear axon (MSE =
0.0030) transfer functions compared to the rest of the other transfer functions. In the same way,
the nature of convergence on the TR data for various transfer functions in GFF NN is depicted in
U
Fig. 6b. In the cases of linear sigmoid axon (MSE = 0.0311) and axon (MSE = 0.0060) transfer
N
functions, the convergence on the TR data is slower compared to the rest of the other learning
M
A
algorithms. [Fig. 6, here]
TE
D
Fig. 7a and 7b illustrate the variation of “MIN AVE MSE” on TR and CV data for different transfer functions in the MLP NN and the GFF NN, respectively. With the use of the
EP
linear axon transfer function, the “MIN AVE MSE” was the lowest on both TR (0.0030) and CV (0.0079) data sets for the MLP NN, while the “MIN AVE MSE” was the lowest on both TR
CC
(0.0003) and CV (0.0015) data sets with sigmoid axon transfer function for the GFF NN.
A
[Fig. 7, here]
Table 6 shows the variations of R2 and MSE on the CV data and TEST data with different
transfer functions. For the MLP NN, the highest R2 and the minimum MSE was related to bias axon, linear axon, and tanh axon transfer functions. For the GFF the highest R2 and the minimum MSE was related to tanh axon transfer function. In view of the overall performance, as shown in 16
Figs. 6 and 7 and Table 6, it is observed that the linear axon transfer function for the MLP NN, and tanh axon transfer function for the GFF NN yield the best possible convincing optimal results.
SC RI PT
[Table 6, here] 3.6 Selection of optimal learning parameters
The performance measure of “MIN AVE MSE” was recorded for the variation of the learning
constant (LC) on TR and CV data sets. The value of the LC for the PEs at the hidden layer was
U
finally chosen as the optimum value, at which the “MIN AVE MSE” converges to the minimum
N
value. According to the general optimal design based on MLP type of ANN architecture, the
A
simulation result illustrated in the Fig. 8a indicates that the performance measure of “MIN AVE
M
MSE” converges to the minimum value for the LC of 0.5 within the variable range on TR and CV data sets. Similarly, several computer-based simulations were conducted to determine the
D
optimum value of the momentum coefficient (MC) of the PEs at the hidden layer with the
TE
optimum value of the LC. Based on these simulation results, the performance measure of “MIN AVE MSE” converges to a minimum at a value of the MC equal to 0.7 within the variable range
EP
(i.e., from 0.1 to 1.0) of TR and CV data sets (Fig. 8b).
CC
[Fig. 8, here]
A
Several simulations were also performed to explore the optimum values of LC and MC of
the PEs at the output layer. The optimum values of LC and MC of the PEs at the hidden layer were selected as 0.7 and 0.7, respectively. Based on the simulation results, it can be observed in Fig. 8c and 8d that the performance measure of “MIN AVE MSE” yields to the minimum value at LC = 0.1 and MC = 0.7, respectively, within the variable range (i.e., from 0.1 to 1.0) of TR and 17
CV data sets. Thus, the optimum values of LC and MC of the PEs at the output layer were determined as 0.1 and 0.7, respectively. The single hidden-layered MLP NN (5:9:1), with the linear axon transfer function for the
SC RI PT
hidden layer and output layer trained with MOM algorithm, was optimized, so that the network can effectively estimate the output. Figs. 9a and 9b show the regression capability of MLP NN and GFF NN on the TEST and CV data sets, respectively, indicating the agreement between
desired and actual outputs of NN. For comparative purposes, the overall simulation results for the best MLP NN and GFF NN architecture are summarized in Table 7. GFF NN was determined as
U
the best network architecture with the highest R2 on the TEST set which was equal to about 0.99
N
with the minimum MSE of 0.00006 for the processed data.
M
A
[Fig. 9, here] and [Table 7, here] 3.7 Sensitivity analysis
D
Sensitivity analysis is a useful method that provides a measure of the relative importance among
TE
the inputs of the ANN model and demonstrates how the model output changes in response to variation of an input. The testing process explores the cause and effect relationship between input
EP
and output variables of the network. The network learning is disabled during this operation, so
CC
that the network weights are not influenced by this process. The basic idea of sensitivity analysis is that the inputs to the network are modified slightly, and the respective change in the output is
A
presented either as a percentage or as a raw difference [53,54]. The NeuroSolutions 5.0 neural network design tool provides a useful tool to detect sensitive input variables called ‘‘Sensitivity about the Mean’’. In the present study, the sensitivity analysis was implemented by batch testing on the single hidden-layered GFF NN (5:6:1) (where a sigmoid axon transfer function was used at its 18
hidden layer and its output layer was trained by the LM algorithm) after fixing the best weights. Then the testing process was started by varying the first input (by default) between its mean ± one standard deviation, while all other inputs are fixed at their respective means. The network
SC RI PT
output was computed for 50 steps above and below the mean. This process was then repeated for each input. Finally, a report summarizing the variation of output with respect to the variation of each input was created by the program. To explore which one of the input parameters (C0, pH0,
EC0, CD, and Npls) has more impact on the dependent variable (HA removal, mg), the sensitivity values of each input in the HA removal forecasting are presented in Fig. 10. As seen from Fig.
U
10, the sensitivity analysis revealed that the initial HA concentration (C0) is the most effective
N
input with a sensitivity ratio of about 55%. Number of pulses (Npls), electrical conductivity (EC0),
A
initial pH (pH0), and current density (CD) follow it with 21%, 17%, 10%, and 7% sensitivity
M
ratios, respectively. Current density (CD) is the least effective input displaying 7% sensitivity in the neural network model as illustrated in Fig. 10.
D
Sensitivity analysis was applied as the final verification for the prediction performance of
TE
the GFF NN model. The process revealed the most significant inputs that could be utilized to
EP
improve the performance of the model by re-arrangements of input parameters. For the present study, each input parameter was found to be important for the prediction performance of the
CC
model. For instance, the current density (CD) is the least effective input compared to the other inputs with a sensitivity ratio of 7%. However, excluding this input from the model structure
A
caused a decrease in the prediction performance of the model as any missing situation of other inputs. With the exclusion of any input component from the model, MSE, NMSE, and MAE values started to increase above 0.0006, 0.0007, and 0.0056, respectively, for the testing set, and the of determination coefficient (R2) decreased below its best value (0.999). Overall, the results indicated that the best neural network model (the single hidden-layered GFF NN (5:6:1) where a 19
sigmoid axon transfer function was used at its hidden layer and its output layer was trained by the LM algorithm) with the present input parameters (C0, pH0, EC0, CD, and Npls) could be successfully used for predicting HA removal. It should be noted that details of
SC RI PT
electrocoagulation/flotation studies and parameters' effects on removal of various contaminants were fully elaborated in the literature [1,20,21,55,56]. Therefore, without going into process-
related details too much, the present computational analysis only focuses on a specific numerical strategy to implement an ANN-based modeling and optimization of a proposed APC-ECF reactor system within the experimental limits of the present process-related variables.
U
Furthermore, the network outputs from separated sensitivity analyses performed for each
N
input variable are shown in Fig. 11. It is noted that there is not a specific reason for some inputs
A
(i.e., C0 and CD) going all the way up to 1. According to the separated sensitivity analysis, an
M
increase in the initial HA concentration (C0) will cause an increase in the amount of HA removed (mg) (Fig. 11a), and this trend was also observed in the experimental study. The maximum HA
D
removal, which was based on the predicted outputs of the best GFF NN model, was achieved
TE
when the value of C0 increased from 10 to 42 mg/L. The separated sensitivity analysis of the
EP
number of pulses (Npls) depicted in Fig. 11b showed that HA removal seems to diminish with an increase in this input component. The experimental results indicated that the maximum HA
CC
removal could be obtained when the number of pulses was studied below 5 (Npls = 3). Fig. 11c shows that an increase in the initial pH (pH0) up to certain level will cause an increase in HA
A
removal, and then illustrates a decreasing trend after this point. As also observed from the experimental studies that the maximum HA removal was obtained when the value of the initial pH increased from 3.0 to 6.63. As seen in Fig. 11d and 11e, HA removal follows a declining trend with the increases of both electrical conductivity (EC0) and current density (CD). According to the experimental findings, the maximum HA removal was attained when the values 20
of EC0 and CD were below 50 A/m2 and 1000 μS/cm, respectively (CD = 24.3 A/m2, EC0 = 856 μS/cm).
SC RI PT
[Figs. 10 and 11, here] 4 Discussion
Artificial intelligence-based methods have become popular problem solving techniques for realtime forecasting in a wide range of scientific and engineering fields. Additionally, the DOE methodology has gained a lot of interest as a systematic tool for discovering relationships
U
between factors and responses. Nevertheless, in the literature, ANN and DOE have been used
N
together in extremely few studies. More specifically, the combination of these methods has not
A
been proposed before for modeling and optimization of an electrochemical process for the
M
removal of humic substances from aqueous media. However, the majority of the literature on electrochemical techniques focuses not on the numerical simulation and modeling of such
D
processes, but rather on the effect of different process-related variables and changing operating
TE
conditions. Considering the relevant literature gap, the present research was conducted as the first
EP
study to introduce a novel application of ANN combined with Taguchi orthogonal experimental design methodology for modeling of a new alternating pulse current electrocoagulation-flotation
CC
(APC-ECF) process in removal of HA from water. For the comparative purpose, a performance data regarding the implementation of ANN-based methodology in various electrochemical
A
processes are summarized in Table 8. [Table 8, here] Daneshvar et al. (2006) conducted an electrocoagulation process for the removal of color from solution containing C. I. Basic Yellow 28. They developed an ANN model (single hidden 21
layer feed forward back-propagation neural network) to forecast the performance of decolorization efficiency based on experimental data obtained in a laboratory batch reactor for seven inputs including CD (A/m2), electrolysis time (min), pH, initial dye concentration (mg/L),
SC RI PT
conductivity (mS/cm), retention time of sludge (min), distance between electrodes (mm). The authors concluded that the proposed ANN topology could describe the color removal percent under different conditions, and almost complete removal of color from dye solution could be achieved by electrocoagulation. In another study undertaken by Ahmed Basha et al. [34],
electrochemical degradation of wastewater from a medium-scale, specialty chemical industry was
U
explored by using Ti/RuOx–TiOx anode in various types of reactor configurations (e.g., batch,
N
batch recirculation, and continuous recycle systems). The authors proposed an ANN model
A
(single hidden layer feed forward back-propagation neural network) for prediction of the
M
performance (in terms of percentage of COD removal) of the batch electrochemical treatment for three input variables such as CD (A/dm2), electrolysis duration (h), and supporting electrolyte
D
concentration (g/L). The study concluded that the proposed ANN architecture was found
TE
adequate to estimate the performance of the process, and the recycle reactor was reported to be
EP
better configuration for commercial application because of its flexibility of operation. Curteanu et al. [58] used a pilot-scale indirect electrolysis system, which comprised of
CC
two basic compartments for coagulation and sedimentation–flotation, for removal of chlorophyll a (as indicator of algae) from the final effluent of aerated lagoons. In the study, predictions of the
A
main system outputs (initial values of chlorophyll a, total suspended solids (TSS), COD) were performed for operation conditions (electrical power, temperature, time, electrode distance, electrode type, initial amount of TSS, initial chlorophyll a, initial COD) using various stacked ANNs. The authors emphasized that the proposed ANN-based methodologies were quite general and could be readily adapted for other treatment processes. Furthermore, in a more recent study, 22
Khataee et al. [59] investigated the removal of phenol as a model pollutant from water by applying a photocatalytic process with the use of immobilized TiO 2 nanoparticles combined with photoelectro-Fenton-like process with Mn2+ cations as catalyst and carbon nanotube
SC RI PT
polytetrafluoroethylene (CNT-PTFE) electrode as cathode. They also conducted a comparison of ultraviolet (UV)/TiO2, electro-Fenton (EF), photoassisted-electro-Fenton (PEF), and PEF/TiO2 in terms of oxidizing efficiency. In the study, the authors developed an ANN model coupled with
genetic algorithm based on five operational parameters, such as oxidizing time (min), pH, applied current (mA), initial phenol concentration (mg/L), and initial Mn2+ concentration (mmol/L) to
U
predict and find the best optimum operating conditions for maximum phenol removal. The study
N
concluded that the ANN model provided reasonable predictive performance (R2 = 0.949), and
A
phenol could be destroyed at 180 min by using PEF/TiO2 process, yielding an oxidizing
M
efficiency percentage of 78%.
Consequently, it is clear from the previous studies (including the present study) that the
D
ANN-based methodology has been successfully implemented in various electrochemical
TE
processes due to its salient characteristics in capturing the nonlinear interactions between
EP
input/output components in complex treatment systems. More importantly, as other soft computing methods, the ANN-based approach provides several potential advantages for
CC
modeling and optimization at a reasonable cost without needing a complex mathematical formulation of the studied process [60,61]. Finally, it is suggested that depending on the
A
characteristics of the input vector associated with the studied experimental conditions, preoptimization of several network parameters (e.g., number of hidden layers, number of processing elements at the hidden layer, type of learning algorithm, type of activation and transfer functions, and determination of optimal learning parameters, etc.) should be properly applied prior to transferring the proposed computational methodology to real-time scale. 23
5 Conclusions In this study, two different ANN network architectures (MLP NN and GFF NN) were developed and trained using a total of 128 data sets divided into training, cross-validation, and testing
SC RI PT
subsets. Computational results indicated that although the predictions of the single hiddenlayered MLP NN (5:9:1), where a tanh axon transfer function was used at its hidden layer and its output layer was trained by the MOM algorithm, were satisfactory (R2 = 0.971, MSE = 0.0031), the single hidden-layered GFF NN (5:6:1), where a sigmoid axon transfer function was used at its hidden layer and its output layer was trained by the LM algorithm showed the best performance
U
(R2 = 0.999, MSE = 0.00006), and the predicted results were found to be very close to the
N
experimental data. Based on the predictions of the best ANN model (GFF NN), the maximum
A
HA removal was achieved for the optimal conditions of C0 = 42 mg/L, pH0 = 6.63, CD = 24.3
M
A/m2, EC0 = 856 μS/cm, and Npls = 3. The results of the present computational analysis clearly confirmed that ANN-based modeling effectively reproduced the experimental data and predict
D
the optimum performance of electrocoagulation/flotation processes for the removal of HA from
TE
aqueous solution. Findings of this computational study clearly concluded that the proposed
EP
ANN/DOE-based methodology described the behavior of a complex reaction system very well
CC
within the working ranges of the implemented experimental conditions.
A
Compliance with ethical standards Conflict of interest The authors declare that they have no conflict of interest.
24
Acknowledgments This study was based on the Research Dissertation work of the first author. The authors acknowledge the support of this work by Kurdistan University of Medical Sciences, Sanandaj,
A
CC
EP
TE
D
M
A
N
U
SC RI PT
Iran.
25
References [1] Mahvi AH, Maleki A, Rezaee R, Safari M (2010) Reduction of humic substances in water by application of ultrasound waves and ultraviolet irradiation. Journal of Environmental
SC RI PT
Health Science & Engineering 6(4):233–240 [2] Eish MYA, Wells MJ (2006) Assessing the trihalomethane formation potential of aquatic fulvic and humic acids fractionated using thin-layer chromatography. Journal of Chromatography A. 1116(1):272–276, doi: 10.1016/j.chroma.2006.03.064
[3] Heiderscheidt E, Leiviskä T, Kløve B (2016) Coagulation of humic waters for diffused
U
pollution control and the influence of coagulant type on DOC fractions removed. Journal
N
of Environmental Management 181:883–893, doi: 10.1016/j.jenvman.2016.06.043
A
[4] Ratnaweera H, Gjessing E, Oug E (1999) Influence of physical-chemical characteristics of
M
natural organic matter (NOM) on coagulation properties: an analysis of eight Norwegian
1223(99)00644-7
D
water sources. Water Science and Technology 40(9):89–95, doi: 10.1016/S0273-
TE
[5] Krasner SW, Mcguire MJ, Jacangelo JG, Patania NL, Reagan KM, Aieta EM (1989) The
EP
occurrence of disinfection by-products in US drinking water. Journal of the American Water Works Association 81(8):41–53
CC
[6] Nikolaou AD, Lekkas TD (2001) The role of natural organic matter during formation of
A
chlorination by-products: a review. Acta Hydrochimica et Hydrobiologica 29(2–3):63–77, doi: 10.1002/1521-401X(200109)29:2/3<63::AID-AHEH63>3.3.CO;2-3
[7] Wang Y, Wang Q, Gao B-Y, Yue Q, Zhao Y (2012) The disinfection by-products precursors removal efficiency and the subsequent effects on chlorine decay for humic acid synthetic water treated by coagulation process and coagulation–ultrafiltration process. Chemical Engineering Journal 193:59–67, doi: 10.1016/j.cej.2012.04.003 26
[8] Zouboulis AI, Chai X-L, Katsoyiannis IA (2004) The application of bioflocculant for the removal of humic acids from stabilized landfill leachates. Journal of Environmental Management 70(1):35–41, doi: 10.1016/j.jenvman.2003.10.003
SC RI PT
[9] Wei M-C, Wang K-S, Hsiao T-E, Lin I-C, Wu H-J, Wu Y-L (2011) Effects of UV irradiation on humic acid removal by ozonation, Fenton and Fe0/air treatment: THMFP and biotoxicity evaluation. Journal of Hazardous Materials 195:324–331, doi: 10.1016/j.jhazmat.2011.08.044
[10] Liu S, Lim M, Fabris R, Chow C, Chiang K, Drikas M et al. (2008) Removal of humic acid
U
using TiO2 photocatalytic process–fractionation and molecular weight characterisation
N
studies. Chemosphere 72(2):263–271, doi: 10.1016/j.chemosphere.2008.01.061
A
[11] Peeva PD, Palupi AE, Ulbricht M (2011) Ultrafiltration of humic acid solutions through
M
unmodified and surface functionalized low-fouling polyethersulfone membranes–effects of feed properties, molecular weight cut-off and membrane chemistry on fouling behavior
D
and cleanability. Separation and Purification Technology 81(2):124–133, doi:
TE
10.1016/j.seppur.2011.07.005
EP
[12] Moura NN, Martín MJ, Burguillo FJ (2007) A comparative study of the adsorption of humic acid, fulvic acid and phenol onto Bacillus subtilis and activated sludge. Journal of
CC
Hazardous Materials 149(1):42–48, doi: 10.1016/j.jhazmat.2007.02.074
A
[13] Holt PK, Barton GW, Wark M, Mitchell CA (2011) A quantitative comparison between chemical dosing and electrocoagulation. Colloids and Surfaces A: Physicochemical and Engineering Aspects 211(2):233–248, doi: 10.1016/S0927-7757(02)00285-6
[14] Emamjomeh MM, Sivakumar M (2009) Review of pollutants removed by electrocoagulation and electrocoagulation/flotation processes. Journal of Environmental Management 90(5):1663–1679, doi: 10.1016/j.jenvman.2008.12.011 27
[15] Kyzas GZ, Matis KA (2016) Electroflotation process: A review. Journal of Molecular Liquids 220:657–664, doi: 10.1016/j.molliq.2016.04.128 [16] Gomes JA, Daida P, Kesmez M, Weir M, Moreno H, Parga JR et al. (2007) Arsenic removal
SC RI PT
by electrocoagulation using combined Al–Fe electrode system and characterization of products. Journal of Hazardous Materials 139(2):220–231, doi: 10.1016/j.jhazmat.2005.11.108
[17] Mollah MYA, Schennach R, Parga JR, Cocke DL (2001) Electrocoagulation (EC)—science and applications. Journal of Hazardous Materials 84(1):29–41, doi: 10.1016/S0304-
U
3894(01)00176-5
N
[18] Rezaee R, Maleki A, Jafari A, Mazloomi S, Zandsalimi Y, Mahvi AH (2014) Application of
A
response surface methodology for optimization of natural organic matter degradation by
M
UV/H2O2 advanced oxidation process. Journal of Environmental Health Science and Engineering 12(1):67–74, doi: 10.1186/2052-336X-12-67
D
[19] Mao X, Hong S, Zhu H, Lin H, Wei L, Gan F (2008) Alternating pulse current in
TE
electrocoagulation for wastewater treatment to prevent the passivation of al electrode.
EP
Journal of Wuhan University of Technology-Mater. Sci. Ed. 23(2):239–241, doi: 10.1007/s11595-006-2239-7
CC
[20] Keshmirizadeh E, Yousefi S, Rofouei MK (2011) An investigation on the new operational
A
parameter effective in Cr(VI) removal efficiency: A study on electrocoagulation by alternating pulse current. Journal of Hazardous Materials 190(1–3):119–124, doi: 10.1016/j.jhazmat.2011.03.010
[21] Cerqueira AA, Souza PSA, Marques MRC (2014) Effects of direct and alternating current on the treatment of oily water in an electroflocculation process. Brazilian Journal of Chemical Engineering 31(3):693–701, doi: 10.1590/0104-6632.20140313s00002363 28
[22] Prakash N, Manikandan S, Govindarajan L, Vijayagopal V (2008) Prediction of biosorption efficiency for the removal of copper (II) using artificial neural networks, Journal of Hazardous Materials 152(3):1268–1275, doi: 10.1016/j.jhazmat.2007.08.015
SC RI PT
[23] Al-Abri M, Hilal N (2008) Neural network simulation of combined humic substance coagulation and membrane filtration. Chemical Engineering Journal 141(1–3):27–34, doi: 10.1016/j.cej.2007.10.005
[24] Akratos CS, Papaspyros JN, Tsihrintzis VA (2008) An artificial neural network model and design equations for BOD and COD removal prediction in horizontal subsurface flow
U
constructed wetlands. Chemical Engineering Journal 143(1–3):96–110, doi:
N
10.1016/j.cej.2007.12.029
A
[25] Soleymani AR, Saien J, Bayat H (2011) Artificial neural networks developed for prediction
M
of dye decolorization efficiency with UV/K2S2O8 process. Chemical Engineering Journal 170(1):29–35, doi: 10.1016/j.cej.2011.03.021
D
[26] Elmolla ES, Chaudhuri M, Eltoukhy MM (2010) The use of artificial neural network (ANN)
TE
for modeling of COD removal from antibiotic aqueous solution by the Fenton process.
EP
Journal of Hazardous Materials 179(1):127–134, doi: 10.1016/j.jhazmat.2010.02.068 [27] Maleki A, Daraei H, Khodaei F, Bayazid-Aghdam K, Rezaee R, Naghizadeh A (2013)
CC
Investigation of potato peel-based bio-sorbent efficiency in reactive dye removal:
A
Artificial neural network modeling and genetic algorithms optimization. Journal of Advances in Environmental Health Research 1(1):21–28
[28] Maleki A, Daraei H, Shahmoradi B, Razee S, Ghobadi N (2014) Electrocoagulation efficiency and energy consumption probing by artificial intelligent approaches. Desalination and Water Treatment 52(13–15):2400–2411, doi: 10.1080/19443994.2013.797545 29
[29] Moghadam M, Asgharzadeh S (2016) On the application of artificial neural network for modeling liquid-liquid equilibrium. Journal of Molecular Liquids 220:339–34, doi: 10.1016/j.molliq.2016.04.098
SC RI PT
[30] Pareek V, Brungs V, Adesina A, Sharma R (2002) Artificial neural network modeling of a multiphase photodegradation system. Journal of Photochemistry and Photobiology A: Chemistry 149(1):139–146, doi: 10.1016/S1010-6030(01)00640-2
[31] Sibanda W, Pretorius P (2011) Novel application of Multi-Layer Perceptrons (MLP) neural networks to model HIV in South Africa using Seroprevalence data from antenatal clinics.
U
International Journal of Computer Applications 35(5):81–87, doi: 10.5120/4398-6106
N
[32] Hatami T, Rahimi M, Daraei H, Heidaryan E, Alsairafi AA (2012) PRSV equation of state
A
parameter modeling through artificial neural network and adaptive network-based fuzzy
10.1007/s11814-011-0235-x
M
inference system. Korean Journal of Chemical Engineering 29(5):657–671, doi:
D
[33] Oguz E, Tortum A, Keskinler B (2008) Determination of the apparent rate constants of the
TE
degradation of humic substances by ozonation and modeling of the removal of humic
EP
substances from the aqueous solutions with neural network. Journal of Hazardous Materials 157(2):455–463, doi: 10.1016/j.jhazmat.2008.01.018
CC
[34] Ahmed Basha C, Soloman P, Velan M, Miranda LR, Balasubramanian N, Siva R (2010)
A
Electrochemical degradation of specialty chemical industry effluent. Journal of Hazardous Materials 176(1):154–164, doi: 10.1016/j.jhazmat.2009.10.131
[35] Maleki A, Mahvi AV, Daraei H, Rezaee R, Meihami N, Mohammadi K, Zandi S (2015) Influence of selected anions on fluoride removal in electrocoagulation/electroflotation. Fluoride 48(1):45–55
30
[36] Yetilmezsoy K, Demirel S (2008) Artificial neural network (ANN) approach for modeling of Pb(II) adsorption from aqueous solution by Antep pistachio (Pistacia Vera L.). Journal of Hazardous Materials 153(3):1288–1300, doi: 10.1016/j.jhazmat.2007.09.092
SC RI PT
[37] Yetilmezsoy K (2010) Modeling studies for the determination of completely mixed activated sludge reactor volume: Steady-state, empirical and ANN applications. Neural Network World 20(5):559–589
[38] Yetilmezsoy K, Ozkaya B, Cakmakci M (2011) Artificial intelligence-based prediction models for environmental engineering. Neural Network World 21(3):193–218
U
[39] Turkdogan-Aydinol FI, Yetilmezsoy K (2010) A fuzzy logic-based model to predict biogas
N
and methane production rates in a pilot-scale mesophilic UASB reactor treating molasses
A
wastewater. Journal of Hazardous Materials 182(1):460–471, doi:
M
10.1016/j.jhazmat.2010.06.054
[40] Ozkan C, Ozturk C, Sunar F, Karaboga D (2011) The artificial bee colony algorithm in
D
training artificial neural network for oil spill detection. Neural Network World 21(6):473–
TE
492, doi: 10.14311/NNW.2011.21.028
EP
[41] Yetilmezsoy K, Turkdogan FI, Temizel I, Gunay A (2013) Development of ANN-based models to predict biogas and methane productions in anaerobic treatment of molasses
CC
wastewater. International Journal of Green Energy 10(9):885–907, doi: 10.1080/15435075.2012.727116
A
[42] Kisi O, Aytek A (2013) Explicit neural network in suspended sediment load estimation. Neural Network World 23(6):587–607, doi: 10.14311/NNW.2013.23.035
[43] Klavins M, Eglite L (2002) Immobilisation of humic substances. Colloids and Surfaces A: Physicochemical and Engineering Aspects 203(1–3):47–54, doi: 10.1016/S09277757(01)01066-4 31
[44] World of Chemicals. Humic Acid. Copyright © (2017) Kimberlite Softwares Pvt. Ltd., India. Available at: http://www.worldofchemicals.com/chemicals/chemical-properties/humicacid.html (Accessed 17 February 2017)
SC RI PT
[45] Bazrafshan E, Biglari H, Mahvi AH (2012) Humic acid removal from aqueous environments by electrocoagulation process using iron electrodes. E-Journal of Chemistry 9(4):2453– 2461, doi: 10.1155/2012/876739
[46] Khandegar V, Saroha AK (2016) Effect of electrode geometry on the performance of electrocoagulation, In: 4th International Conference on Innovations in Science
U
Engineering and Management, New Delhi, India.
N
[47] Aggarwal VK, Staubitz AC, Owen M (2006) Optimization of the Mizoroki−Heck Reaction
A
Using Design of Experiment (DoE). Organic Process Research & Development
M
10(1):64−69.
[48] Keane AJ (2003) Wing optimization using design of experiment, response surface, and data
D
fusion methods. Journal of Aircraft 40(4):741−750.
TE
[49] Kim NH, Choi MH, Kim SY, Chang EG (2006) Design of experiment (DOE) method
EP
considering interaction effect of process parameters for optimization of copper chemical mechanical polishing (CMP) process. Microelectronic Engineering 83(3):506−512.
CC
[50] Condra LW (2001). Reliability Improvement with Design of Experiments, Second Edition, Revised and Expanded, Marcel Dekker Inc., NY, USA.
A
[51] NeuroSolution. Version 5.0, Copyright © (2015) Neuro Dimension, Inc. 3701 NW 40th Terrace, Suite 1, Gainesville, FL 32606 Available at: http://www.neurosolutions.com/ (Accessed 17 February 2017)
32
[52] Rizkalla N, Hildgen P (2005) Artificial neural networks: Comparison of two programs for modeling a process of nanoparticle preparation. Drug Development and Industrial Pharmacy 31(10):1019–1013, doi: 10.1080/03639040500306294
SC RI PT
[53] Hernandez S, Nešić S, Weckman G, Ghai V (2006) Use of artificial neural networks for predicting crude oil effect on carbon dioxide corrosion of carbon steels. Corrosion, 62(6): 467−482.
[54] Mady M (2013) Prediction model of construction labor production rates in Gaza strip using artificial neural network. MSc Thesis in Civil Engineering, The Islamic university of
U
Gaza (IUG), Gaza, Palestine.
N
[55] Eyvaz M, Kirlaroglu M, Aktas TS, Yuksel E (2009) The effects of alternating current
A
electrocoagulation on dye removal from aqueous solutions. Chemical Engineering Journal
M
153(1–3):16–22, doi: 10.1016/j.cej.2009.05.028
[56] Yoosefian M, Ahmadzadeh S, Aghasi M, Dolatabadi M (2017) Optimization of
D
electrocoagulation process for efficient removal of ciprofloxacin antibiotic using iron
TE
electrode; kinetic and isotherm studies of adsorption. Journal of Molecular Liquids
EP
225:544–553, doi: 10.1016/j.molliq.2016.11.093 [57] Daneshvar N, Khataee AR, Djafarzadeh N (2006) The use of artificial neural networks
CC
(ANN) for modeling of decolorization of textile dye solution containing CI Basic Yellow
A
28 by electrocoagulation process. Journal of Hazardous Materials 137(3):1788–1795, doi: 10.1016/j.jhazmat.2006.05.042
[58] Curteanu S, Piuleac, CG, Godini K, Azaryan G (2011) Modeling of electrolysis process in wastewater treatment using different types of neural networks. Chemical Engineering Journal 172(1):267–276, doi: 10.1016/j.cej.2011.05.104
33
[59] Khataee AR, Fathinia M, Zarei M, Izadkhah B, Joo SW (2014) Modeling and optimization of photocatalytic/photoassisted-electro-Fenton like degradation of phenol using a neural network coupled with genetic algorithm. Journal of Industrial and Engineering Chemistry
SC RI PT
20(4):1852–1860, doi: 10.1016/j.jiec.2013.08.042 [60] Yetilmezsoy K, Turkdogan FI, Temizel I, Gunay A (2013) Development of ANN-based
models to predict biogas and methane productions in anaerobic treatment of molasses wastewater. International Journal of Green Energy 10(9):885–907, doi:10.1080/15435075.2012.727116
U
[61] Yetilmezsoy K (2018) Applications of soft computing methods in environmental
N
engineering, In: Handbook of Environmental Materials Management, Section IX:
A
Environmental Modeling (Mathematical Modeling and Environmental Problems), Dr.
M
Chaudhery Mustansar Hussain (Eds.). Springer International Publishing AG,
A
CC
EP
TE
D
Gewerbestrasse 11, 6330 Cham, Switzerland, pp. 1–47.
34
REVISED FIGURE CAPTIONS Fig. 1 A photograph of the laboratory-scale batch APC-ECF reactor Fig. 2 Variations of the normalized responses of all data points
SC RI PT
Fig. 3 Performance measure of “MIN AVE MSE” for the optimal selection of number of PEs in the hidden layer: (a) MLP NN and (b) GFF NN
Fig. 4 Convergence of AVE MSE on the TR data set for different learning algorithms: (a) MLP NN and (b) GFF NN
Fig. 5 Variation of “MIN AVE MSE” on TR and CV data sets for different learning algorithms:
U
(a) MLP NN and (b) GFF NN
N
Fig. 6 Convergence of AVE MSE on the TR data set for different transfer functions: (a) MLP NN
A
and (b) GFF NN
M
Fig. 7 Variation of “MIN AVE MSE” on TR and CV data sets for different transfer functions: (a)
D
MLP NN and (b) GFF NN
TE
Fig. 8 Performance measure of “MIN AVE MSE” for variation in learning constant (LC) and momentum coefficient (MC) of processing elements for “hidden layer” (a and b) and “output
EP
layer” (c and d)
Fig. 9 Regression capability on TEST and CV data sets for MLP NN (a and b) and GFF NN (c
CC
and d)
Fig. 10 Sensitivity of GFF NN input parameters in HA removal forecasting
A
Fig. 11 Network outputs from separated sensitivity analyses performed for each input variable
35
D
TE
EP
CC
A
SC RI PT
U
N
A
M
Fig. 1
36
D
TE
EP
CC
A
SC RI PT
U
N
A
M
Fig. 2
37
D
TE
EP
CC
A Fig. 3
38
SC RI PT
U
N
A
M
D
TE
EP
CC
A Fig. 4
39
SC RI PT
U
N
A
M
D
TE
EP
CC
A Fig. 5
40
SC RI PT
U
N
A
M
D
TE
EP
CC
A Fig. 6
41
SC RI PT
U
N
A
M
D
TE
EP
CC
A Fig. 7
42
SC RI PT
U
N
A
M
D
TE
EP
CC
A
SC RI PT
U
N
A
M
Fig. 8
43
D
TE
EP
CC
A Fig. 9
44
SC RI PT
U
N
A
M
D
TE
EP
CC
A
SC RI PT
U
N
A
M
Fig. 10
45
D
TE
EP
CC
A Fig. 11
46
SC RI PT
U
N
A
M
Table 1 Physicochemical properties of humic acid
Molecular Formula
C9H9NO6
SC RI PT
Molecular Structure
227.17
Product Number
53680
CAS (Chemical Abstracts Service)
1415-93-6
A
N
U
Molecular Weight (g/mol)
M
Number 300 °C
EINECS (European Inventory of
215-809-6
D
Melting Point
TE
Existing Commercial Substances) Number
EP
RTECS (Registry of Toxic Effects
MT6544000
CC
of Chemical Substances) Number Black granules
Stability
Stable and incompatible with strong oxidizing agents.
A
Appearance
Solubility
Insoluble
pH Value
5.0–9.0
47
Table 2 Details of the present experimental design based on Taguchi method
C0
pH0
EC0
(mg/L)
Tpls (min)
Npls
(μS/cm)
10
3.0
500
Level 2
20
7.0
1000
Level 3
50
9.0
2000
(volts) 1
1
5
5
5
10
10
10
15
A
CC
EP
TE
D
M
A
N
U
Level 1
48
V
SC RI PT
Factor
Table 3 Optimized parameters of the studied ANN architectures
Typical range
1
Hidden layer
1 to 3
2
Processing
1 to 10
Elements (PEs) 3
Learning rule
Momentum (MOM),
Optimal
parameter
parameter
for MLP
for GFF NN
NN model
model
1
1
9
6
MOM
LM
U
Conjugate gradient (CG),
Optimal
SC RI PT
Parameter ID Parameter
N
Levenberg–Marquardt (LM), Quick Propagation
A
(QP), Step (STP), Delta-Bar-
Transfer
Hyperbolic tangent function
Linear:
tanh x:
function
(tanh x), Sigmoid, Linear tanh,
y=x
1 e2 x 1 e2 x
0.1 to 1
0.5
-
0.1 to 1
0.7
-
0.1 to 1
0.1
-
0.1 to 1
0.7
-
D
4
M
Delta (DBD)
TE
Linear Sigmoid, Bias, Linear,
5
Hidden layer:
Axon
EP
Learning constant
6
Hidden layer:
CC
Momentum coefficient
A
7
8
Output layer: Learning constant Output layer: Momentum coefficient
49
Table 4 Variation of statistical performance indicators for the optimal selection of the number of hidden layers Number of TR set
CV set
#
MSE
MSE
MSE
NMSE
MAE
R2
1
0.0042
0.0097
0.0037
0.0425
0.0503
0.966
2
0.0092
0.0096
0.0066
0.0679
0.0710
0.946
3
0.0106
0.0099
0.0064
0.0658
0.0625
0.956
1
0.0046
0.0094
U
ANN hidden
Descriptive statistics on testing set
0.0024
0.0276
0.0379
0.978
2
0.0047
0.0099
0.0034
0.0384
0.0468
0.968
3
0.0033
0.0099
0.0018
0.0210
0.0316
0.984
SC RI PT
architecture layers
A
M
D
GFF
N
MLP
TE
MLP = Multilayer perception, GFF = Generalized feed forward, TR = Training set, CV = Cross-validation, MSE = Mean squared error, NMSE = Normalized mean squared error, MAE =
A
CC
EP
Mean absolute error, R2 = Determination coefficient
50
Table 5 Variations of MSE and R2 for different learning algorithms
A
Learning algorithm Data
Performance
set
measure
STP
MOM
CG
MSE
0.0030
0.0047
0.0052
R2
0.9712
0.9541
0.9449
MSE
0.0051
0.0045
0.0044
R2
0.9647
0.9664
0.9674
MSE
0.0027
0.0026
0.0043
R2
0.9727
0.9741
MSE
0.0049
R2
0.9651
QP
DBD
0.00007
0.0038
0.0025
0.9990
0.9682
0.9791
0.0020
0.0051
0.0046
0.9852
0.9609
0.9653
0.0004
0.0035
0.0029
0.9600
0.9962
0.9643
0.9714
0.0038
0.0043
0.0018
0.0051
0.0048
0.9716
0.9649
0.9852
0.9633
0.9651
type
TEST M LP
FF
N
M
TEST G
D
CV
U
CV
SC RI PT
LM
A
NN
TE
MLP = Multilayer perception, GFF = Generalized feed forward, TEST = Testing set, CV
EP
= Cross-validation, MSE = Mean squared error, R2 = Determination coefficient, STP = Step, MOM = Momentum, CG = Conjugate-Gradient, LM = Levenberg–Marquardt, QP = Quick-
A
CC
Propagation, DBD = Delta-Bar-Delta
51
Table 6 Variations of MSE and R2 for different transfer functions
Transfer function Data
Performance
type
set
measure
Linear tanh
Bias
MSE
0.0033
0.0204
0.0034
R2
0.9662
0.9745
0.9704
0.0179
0.0021
0.0033
0.0029
0.9137
0.9804
0.9763
0.9700
MSE
0.0039
0.0204
0.0042
0.0196
0.0047
0.0040
0.0066
R2
0.9721
0.9395
0.9656
0.8843
0.9633
0.9664
0.9578
MSE
0.00006
0.0006
0.0005
0.0236
0.0023
0.0019
0.0022
R2
0.9992
0.9942
0.9942
0.9030
0.9806
0.9820
0.9751
MSE
0.0019
0.0031
0.0024
0.0356
0.0045
0.0050
0.0070
R2
0.9848
0.9773
0.9802
0.9269
0.9647
0.9637
0.9609
U
CV
TEST
A
M
CV
Axon
sigmoid
TEST
GFF
Linear
N
tanh
MLP
Linear
Sigmoid
SC RI PT
ANN
D
MLP = Multilayer perception, GFF = Generalized feed forward, TEST = Testing set, CV
A
CC
EP
TE
= Cross-validation, MSE = Mean squared error, R2 = Determination coefficient,
52
Table 7 Performance measures for the best ANN architectures
Best
TR
CV
TEST
CV (testing)
MSE
MSE
MSE
NMSE
MAE
R2
MLP (5:9:1)
0.0030
0.0075
0.00310
0.0354
0.0442
0.9710
0.0039
0.0335
0.0464
0.9668
GFF (5:6:1)
0.0008
0.0043
0.00006
0.0007
0.0056
0.9992
0.0019
0.0162
0.0246
0.9848
ANN MSE
NMSE
MAE
R2
SC RI PT
architectures
MLP = Multilayer perception, GFF = Generalized feed forward, TR = Training set, CV = Cross-validation, TEST = Testing set, MSE = Mean squared error, NMSE = Normalized mean squared error, MAE = Mean absolute error, R2 =
A
CC
EP
TE
D
M
A
N
U
Determination coefficient
53
Table 8 Performance data regarding the implementation of different ANN topologies in various electrochemical processes Size of workpiece and materials
A NN topology
Electroc oagulation for decolorization of textile dye solution containing C. I. Basic Yellow 28
Electroly tic cell with iron (ST 372) plates, EEA = 25 cm2
EC degradation of wastewater from a medium-scale, specialty chemical industry
Indirect electrolysis for the removal of chlorophyll a (as indicator of algae) from the final effluent of aerated lagoons
Model variables Input
Ou tput(s)
SH L FF BP ANN (7:1:1) with a sigmoidal transfer function (tansig)
CD, electrolysis time, pH, initial dye concentration, conductivity, retention time of sludge, ED
Co lor removal (%)
500 mL batch reactor with SSFP and a RFEMoT electrodes, EEA = 25.52 cm2
SH L FF BP ANN (3:1:1) with LMA
CD (A/dm2), electrolysis duration (h), SEC (g/L)
C OD removal (%)
Electroly sis system of two 8 and 12.15 dm3 basic compartment with aluminum electrodes in monopolar arrangement, EEA = 3379 cm2
Sta cked ANNs 30 % MLP (8:16:3), 60% MLP (8:15:5:3), 10% MLP (8:20:3)
U
N A
M
TE
EP
Elect
rical
pow er, temperature, time, ED, electrode type, initia l amount of TSS, initial chlorophyll a, initial COD
Performa nce assessment (for the best model) (i) Almost complete removal of color from dye solution; (ii) ANN model can describe the color removal percent under different conditions
Re ference and region
(i) COD removal (65.3– 82.7%); (ii) ANN is adequate enough to predict the performance of the process
A hmed Basha et al. [34], India
D aneshvar et al. [57], Iran
SC RI PT
s
D
Type of process
Fi nal values of TSS (mg/L), chlorophyll a (mg/m3), COD (mg/L)
(i) Purification by electrolysis is suitable for separating algae from the effluents of the aerated lagoons; (ii) ANN results represent accurate predictions, useful for experimental practice
C urteanu et al. [58], Romania and Iran
2500 mL cubic tank with CNT-PTFE electrode as cathode, D = 25 mm and t = 0.6 mm
SH L FF BP ANN (5:12:1) with a sigmoidal transfer function (tansig)
Oxid izing time, pH, applied current, initial phenol concentration, initial Mn2+ concentration
Ox idizing efficiency (%)
(i) 78% of phenol was destroyed at 180 min by using PEF/TiO2 process; (ii) ANN model provided reasonable predictive performance (R2 = 0.949)
K hataee et al. [59], Iran and Korea
APCECF system for HA removal from aqueous media
600 mL EC cell with two L-shaped aluminum electrodes in
SH L MLP NN (5:9:1) with MOM and GFF NN
C0 (mg/L), pH0, EC0 (μS/cm), CD (A/m2), Npls
A mount of HA removed (mg)
(i) HA removal (100%); (ii) ANN model can describe the behavior of APC-
Pr esent study, Iran and Turkey
A
CC
Immobil ized TiO2 nanoparticles combined with photoelectroFenton-like process for phenol removal from in aqueous media
54
monopolar arrangement, EEA = 99.55 cm2
ECF very well (R2 = 0.999, MSE = 0.00006)
(5:6:1) with LMA
APC-ECF = Alternating Pulse Current Electrocoagulation-Flotation; BP = Back Propagation; CD = Current density; COD = Chemical Oxygen Demand; EC = Electrochemical; ED = Electrode distance; EEA = Effective Electrode Area; FF = Feed Forward; LMA = Levenberg–Marquardt Algorithm; RFEMoT = Rectangular Flat
SC RI PT
Expanded Mesh of Titanium; SEC = Supporting Electrolyte Concentration; SHL = Single Hidden Layered; SSFP =
A
CC
EP
TE
D
M
A
N
U
Stainless Steel Flat Plate; TSS = Total Suspended Solids.
55