- Email: [email protected]
Mining and fusion of petroleum data with fuzzy logic and neural network agents
Journal of Petroleum Science and Engineering 29 Ž2001. 221–238 www.elsevier.nlrlocaterjpetscieng
Mining and fusion of petroleum data with fuzzy logic and neural network agents Masoud Nikravesh a,b,) , Fred Aminzadeh b,c,1 a
Berkeley InitiatiÕe in Soft Computing (BISC), Computer Science DiÕision, Department of EECS, UniÕersity of California, Berkeley, CA 94720, USA b Zadeh Institute for Information Technology (ZIFIT), USA c dGB-USA, Sugar Land, TX, 77478, USA
Abstract Analyzing data from well logs and seismic is often a complex and laborious process because a physical relationship cannot be established to show how the data are correlated. In this study, we will develop the next generation of AintelligentB software that will identify the nonlinear relationship and mapping between well logsrrock properties and seismic information and extract rock properties, relevant reservoir information and rules Žknowledge. from these databases. The software will use fuzzy logic techniques because the data and our requirements are imperfect. In addition, it will use neural network techniques, since the functional structure of the data is unknown. In particular, the software will be used to group data into important data sets; extract and classify dominant and interesting patterns that exist between these data sets; discover secondary, tertiary and higher-order data patterns; and discover expected and unexpected structural relationships between data sets. q 2001 Published by Elsevier Science B.V. Keywords: Well log analysis; Seismic; Knowledge extraction; Intelligent; Reservoir characterization
1. Introduction In reservoir engineering, it is of great importance to characterize how seismic information Žattributes., the lithology and geology of the rocks are related to the well logs, such as porosity, density and gamma ray. However, data from well logs and seismic attributes are often difficult to analyze because of their complexity. A physical relationship cannot usually be established to show how the data are correlated,
)
Corresponding author. Tel.: q1-510-643-4522; fax: q1-510642-5775. E-mail address: [email protected] ŽM. Nikravesh.. 1 Fax: q1-281-265-2512
except by a laborious process, and the human ability to understand and use the information content of these data is limited. Neural networks provide the potential to establish a model from nonlinear, complex and multidimensional data and find wide application in analyzing experimental, industrial and field data ŽAminzadeh and Chatterjee, 1984r1985; Baldwin et al., 1989, 1990; Rogers et al., 1992; Wong et al., 1995a,b; Nikravesh et al., 1996; Pezeshk et al., 1996.. In recent years, the neural network literature has stimulated growing interest among reservoir engineers, geologists and geophysicist ŽKlimentos and McCann, 1990; Aminzadeh et al., 1994; Boadu, 1997; Nikravesh, 1998; Nikravesh and Aminzadeh, 1998.. Boadu Ž1997. applied artificial neural network suc-
0920-4105r01r$ - see front matter q 2001 Published by Elsevier Science B.V. PII: S 0 9 2 0 - 4 1 0 5 Ž 0 1 . 0 0 0 9 2 - 4
222
M. NikraÕesh, F. Aminzadehr Journal of Petroleum Science and Engineering 29 (2001) 221–238
cessfully to find the relationship between seismic and rock properties for Sandstones Rocks. In our recent studies ŽNikravesh and Aminzadeh, 1998., we further analyzed data published by Klimentos and McCann Ž1990. for Sandstones and recently analyzed by Boadu Ž1997. using an artificial neural network. It was concluded that neural network model had a better performance than a multiple linear regression model. Neural network, neuro-fuzzy and knowledge-based models have been successfully used to model rock properties based on well log databases ŽNikravesh, 1998.. However, using neural networks for identification purposes is more useful when a large number of data are available. In addition, conventional neural network models cannot deal with uncertainty in data due to fuzziness. In this study, our AintelligentB software will be built around two technologies: artificial neural network and fuzzy logic ŽAppendix A.. Each of them is expected to remedy one or more shortcomings of conventional data mining and knowledge management systems. The software will take advantage of the tolerance for imprecision that fuzzy logic can bring to bear on the process of knowledge acquisition from massive data sets. The software will use fuzzy logic techniques because the data and our requirements are imperfect. Fuzzy logic is considered to be appropriate to deal with the nature of uncertainty on system and human error which are not included in current reliability theories. We will integrate our recently developed algorithms and heuristics to induce fuzzy logic rules from data sets. An artificial neural network will be used to extract and fine-tune the rule base including granulation of variables and the characteristics of the membership functions. In this study, we begin by mining wireline logs Žsuch as density, gamma ray, travel time, SP and resistivity.. Then, the results will be used to train the neural network models. In addition, we will analyze the data to recognize the most important patterns, structures, relationships and characteristics based on neural network and neuro-fuzzy models. 2. Neural network and nonlinear mapping In this section, a series of neural network models ŽAppendix B. will be developed for nonlinear map-
ping between wireline logs. A series of neural network models will also be developed to analyze actual well log data and seismic information and the nonlinear mapping between wireline logs and seismic attributes will be recognized. In this study, wireline logs, such as travel time ŽDT., gamma ray ŽGR. and density ŽRHOB., will be predicted based on SP and resistivity ŽRILD. logs. In addition, we will predict travel time ŽDT. based on induction resistivity and vice versa. In this study, all logs are scaled uniformly between y1 and 1 and results are given in scaled domain. Fig. 1a through e shows typical behavior of SP, RILD, DT, GR and RHOB logs in scaled domain. The design of a neural network to predict DT, GR and RHOB based on RILD and SP logs starts with filtering, smoothing and interpolating values Žin a small horizon. for missing information in the data set. A first-order filter and a simple linear recursive parameter estimator for interpolating were used to filter and reconstruct the noisy data. The available data were divided into three data sets: training, testing and validation. The network was trained based on the training data set and continuously tested using a test data set during the training phase. The network was trained using a backpropagation algorithm and modified Levenberge–Marquardt optimization technique ŽAppendix C.. Training was stopped when prediction deteriorated with step. 2.1. TraÕel time (DT) prediction based on SP and resistiÕity (RILD) logs The neural network model to predict the DT has 14 input nodes Žtwo windows of data each with seven data points. representing SP Žseven points or input nodes. and RILD Žseven points or input nodes. logs. The hidden layer has five nodes. The output layer has three nodes representing the prediction of the DT Ža window of three data point.. Typical performance of the neural network for training, testing and validation data sets is shown in Fig. 2a, b, c and d. The network shows good performance for prediction of DT for training, testing and validation data sets. However, there is not a perfect match between actual and predicted values for DT is the testing and validation data sets. This is due to changes of the lithology from point to point. In other words,
M. NikraÕesh, F. Aminzadehr Journal of Petroleum Science and Engineering 29 (2001) 221–238
Fig. 1. Typical behavior of SP, RILD, DT, GR, and RHOB logs.
223
224
M. NikraÕesh, F. Aminzadehr Journal of Petroleum Science and Engineering 29 (2001) 221–238
Fig. 2. Typical neural network performance for prediction of DT based on SP and RILD.
some of the data points in the testing and validation data sets are in a lithological layer which was not presented in the training phase. Therefore, to have perfect mapping, it would be necessary to use the layering information Žusing other types of logs or linguistic information. as input into the network or use a larger data set for the training data set which represent all the possible behaviors in the data. 2.2. Gamma ray (GR) prediction based on SP and resistiÕity (RILD) logs In this study, a neural network model is developed to predict GR based on SP and RILD log. The network has 14 input nodes Žtwo windows of data, each with seven data points. representing SP Žseven
points or input nodes. and RILD Žseven points or input nodes. logs. The hidden layer has five nodes. The output layer has three nodes representing the prediction of the GR. Fig. 3a through d shows the performance of the neural network for training, testing and validation data. The neural network model shows a good performance for prediction of GR for training, testing and validation data sets. In comparison with previous studies ŽDT prediction., this study shows that the GR is not as sensitive as DT to noise in the data. In addition, a better global relationship exits between SP-resistivity–GR rather than SP-resistivity–DT. However, the local relationship is in the same order of complexity. Therefore, two models have the same performance for training Žexcellent performance.. However, the model for prediction of
M. NikraÕesh, F. Aminzadehr Journal of Petroleum Science and Engineering 29 (2001) 221–238
225
Fig. 3. Typical neural network performance for prediction of GR based on SP and RILD.
GR has a better generalization property. Since the two models have been trained based on the same criteria, it is unlikely that this lack of mapping for generalization is due to over fitting during the training phase. 2.3. Density (RHOB) prediction based on SP and resistiÕity (RILD) logs To predict density based on SP and RILD logs, a neural network model with 14 input nodes representing SP Žseven points or input nodes. and RILD Žseven points or input nodes. logs, five nodes in the hidden layer and three nodes in the output layer representing the prediction of the RHOB is developed. Fig. 4a through d shows a typical performance
of the neural network for the training, testing and validation data sets. The network shows excellent performance for the training data set as shown in Fig. 4a and b. The model shows a good performance for the testing data set as shown in Fig. 4c. Fig. 4d shows the performance of the neural network for the validation data set. The model has relatively good performance for the validation data set. Therefore, there is not a perfect match between the actual and predicted values for RHOB for the testing and validation data set. Since RHOB is directly related to lithology and layering, and to have perfect mapping, it would be necessary to use the layering information Žusing other types of logs or linguistic information. as an input into the network or use a larger data set
226
M. NikraÕesh, F. Aminzadehr Journal of Petroleum Science and Engineering 29 (2001) 221–238
Fig. 4. Typical neural network performance for prediction of RHOB based on SP and RILD.
for the training data set which represent all the possible behaviors in the data. In these cases, one can use a knowledge-based approach using knowledge of an expert and select more diverse information which represent all different possible layering as a training data set. Alternatively, one can use an automated clustering technique to recognize the important clusters existing in the data and use this information for selecting the training data set. 2.4. TraÕel time (DT) prediction based on resistiÕity (RILD) The neural network model to predict the DT has 11 input nodes representing a RILD log. The hidden
layer has seven nodes. The output layer has three nodes representing the prediction of the DT. Using engineering knowledge, a training data set is carefully selected so as to represent all the possible layering existing in the data. The typical performance of neural network for the training, testing and validation data sets is shown in Fig. 5a through d. As expected, the network has excellent performance for prediction of DT. Even though only RILD logs are used for prediction of DT, the network model has better performance than when SP and RILD logs used for prediction of DT Žcomparing Fig. 2a through d with 5a through D.. However, in this study, knowledge of an expert was used as extra information.
M. NikraÕesh, F. Aminzadehr Journal of Petroleum Science and Engineering 29 (2001) 221–238
227
Fig. 5. Typical neural network performance for prediction of DT based on RILD.
This knowledge not only reduced the complexity of the model, but also better prediction was achieved. 2.5. ResistiÕity (RILD) prediction based on traÕel time (DT) In this section, to show that the technique presented in the previous section is effective, the performance of the inverse model is tested. The network model has 11 input nodes representing DT, seven nodes in the hidden layer and three nodes in the output layer representing the prediction of the RILD. Fig. 6a through d shows the performance of the neural network model for the training, testing and
validation data sets. Fig. 6a and b shows that the neural network has excellent performance for the training data set. Fig. 6c and d shows the performance of the network for the testing and validation data set. The network shows relatively excellent performance for testing and validation purposes. As was mentioned in the previous section, using engineering knowledge, the complexity of the model was reduced and better performance was achieved. In addition, since the network model Žprediction of DT from resisitivity. and its inverse Žprediction of resisitivity based on DT. have relatively excellent performance and generalization properties, a one-toone mapping was achieved. Therefore, this implies
228
M. NikraÕesh, F. Aminzadehr Journal of Petroleum Science and Engineering 29 (2001) 221–238
Fig. 6. Typical neural network performance for prediction of RILD based on DT.
that a good representation of layering was selected based on knowledge of an expert.
3. Neural-fuzzy model for rule extraction In this section, a neuro-fuzzy model ŽAppendix . D will be developed for model identification and knowledge extraction Žrule extraction. purposes. The model is characterized by a set of rules which can be further used for representation of data in the form of linguistic variables. Therefore, in this situation the fuzzy variables become linguistic variables. The neuro-fuzzy technique is used to implicitly cluster the data while finding the nonlinear mapping. The neuro-fuzzy model developed in this study is an
approximate fuzzy model with triangular and Gaussian membership functions originally presented by Sugeno and Yasukawa Ž1993.. k-Mean technique ŽAppendix E. is used for clustering and the network is trained using a backpropagation algorithm and modified Levenberge–Marquardt optimization technique ŽAppendix C.. In this study, the effect of rock parameters and seismic attenuation on permeability will be analyzed based on soft computing techniques and experimental data. The software will use fuzzy logic techniques because the data and our requirements are imperfect. In addition, it will use neural network techniques, since the functional structure of the data is unknown. In particular, the software will be used to group data into important data sets; extract and classify domi-
M. NikraÕesh, F. Aminzadehr Journal of Petroleum Science and Engineering 29 (2001) 221–238
Fig. 7. Actual data ŽBoadu, 1997..
229
230
M. NikraÕesh, F. Aminzadehr Journal of Petroleum Science and Engineering 29 (2001) 221–238
Fig. 8. Typical rules extracted from data, 7 Rules. Ža. Typical rules extracted from data, 7 Rules ŽPorosity.. Žb. Typical rules extracted from data, 7 Rules ŽGrain Size.. Žc. Typical rules extracted from data, 7 Rules ŽClay Content.. Žd. Typical rules extracted from data, 7 Rules ŽP-Wave Velocity.. Že. Typical rules extracted from data, 7 Rules ŽP-Wave Attenuation..
M. NikraÕesh, F. Aminzadehr Journal of Petroleum Science and Engineering 29 (2001) 221–238
231
Table 1 Boundary of rules extracted from data Porosity
Grain size
Clay content
P-wave velocity
P-wave attenuation
wy0.4585, y0.3170x w0.4208, 0.5415x wy0.3610, y0.1599x wy0.2793, y0.0850x wy0.3472, y0.1856x w0.2700, 0.4811x wy0.2657, y0.1061x
wy0.6501, y0.3604x wy0.9351, y0.6673x wy0.7866, y0.4923x wy0.5670, y0.2908x wy0.1558, 0.1629x wy0.8077, y0.5538x w0.0274, 0.3488x
wy0.6198, y0.3605x w0.2101, 0.3068x wy0.3965, y0.1535x wy0.4005, y0.1613x wy0.8093, y0.5850x wy0.0001, 0.2087x wy0.4389, y0.1468x
w0.0893, 0.2830x wy0.7981, y0.7094x wy0.0850, 0.1302x wy0.1801, 0.0290x w0.1447, 0.3037x wy0.6217, y0.3860x wy0.1138, 0.1105x
wy0.6460–0.3480x w0.0572 0.2008x wy0.4406–0.1571x wy0.5113–0.2439x wy0.8610–0.6173x wy0.1003 0.1316x wy0.5570–0.1945x
nant and interesting patterns that exist between these data sets; and discover secondary, tertiary and
higher-order data patterns. The objective of this section is to predict the permeability based on grain
Fig. 9. Performance of Neural-Fuzzy model for prediction of permeability. Ža. Performance of Neural-Fuzzy model for prediction of permeability w K s f ŽP, G, C, PWV, PWA.x. Žb. Performance of Neural-Fuzzy model for prediction of permeability w K s f ŽP, C, PWV, PWA.x. Žc. Performance of Neural-Fuzzy model for prediction of permeability w K s f ŽP, PWV, PWA. x. Žd. Performance of Neural-Fuzzy model for prediction of permeability w K s f ŽG, C, PWV, PWA.x.
232
M. NikraÕesh, F. Aminzadehr Journal of Petroleum Science and Engineering 29 (2001) 221–238
Fig. 10. Relationship between P-wave velocity and porosity.
size, clay content, porosity, P-wave velocity and P-wave attenuation. 3.1. Prediction of permeability based on porosity, grain size, clay content, P-waÕe Õelocity and P-waÕe attenuation In this section, a neural-fuzzy model will be developed for nonlinear mapping and rule extraction Žknowledge extraction. between porosity, grain size, clay content, P-wave velocity, P-wave attenuation and permeability. Fig. 7 shows typical data, which has been used in this study. In this study, permeability will be predicted based on the following rules and Eqs. ŽD-1. through ŽD-4. see Appendix D: IF Rock Type s Sandstones AND Porositys w p1, p2x AND Grain Size s w g1, g 2x AND Clay Contents w c1,c2x AND P-Wave Vel.s wpwv1,pwv2x AND P-Wave Att.s wPwa1,pwa2x THEN Y ) s a0 q a1) P q a2)G q a3)C q a4) PWV q a5) PWA
where P is %porosity, G is grain size, C is clay content, PWV is P-wave velocity and P-wave attenuation, Y ) , is equivalent to f in Eq. ŽD-1.. Data are scaled uniformly between y1 and 1 and the result is given in the scaled domain. The available data were divided into three data sets: training, testing, and validation. The neuro-fuzzy model was trained based on a training data set and continuously tested using a test data set during the training phase. Training was stopped when it was found that the model’s prediction suffered upon continued training. Next, the number of rules was increased by one and training was repeated. Using this technique, an optimal number of rules were selected. Fig. 8a through e and Table 1 show typical rules extracted from the data. In Table 1, Columns 1 through 5 show the membership functions for porosity, grain size, clay content, P-wave velocity, and P-wave attenuation, respectively. Using the model defined by Eq. ŽD-1. through ŽD-4. and membership functions defined in Fig. 8a through e and Table 1, permeability was predicted as shown in Fig. 9a. In this study, seven rules were identified for prediction of permeability based on porosity, grain size, clay content, P-wave velocity, and P-wave attenuation ŽFig. 8a through e
M. NikraÕesh, F. Aminzadehr Journal of Petroleum Science and Engineering 29 (2001) 221–238
233
Fig. 11. Relationship between P-wave attenuation and clay content.
and Fig. 9a.. In addition, eight rules were identified for prediction of permeability based on porosity, clay content, P-wave velocity, and P-wave attenuation ŽFig. 9b.. Ten rules were identified for prediction of permeability based on porosity, P-wave velocity, and P-wave attenuation ŽFig. 9c.. Finally, six rules were identified for prediction of permeability based on grain size, clay content, P-wave velocity, and P-wave attenuation ŽFig. 9d.. The neural network model shows very good performance for prediction of permeability. In this situation, not only a nonlinear mapping and relationship was identified between porosity, grain size, clay content, P-wave velocity, and P-wave attenuation, and permeability; however, the rules existing between data were also identified. For this case study, our software clustered the parameters as grain size, P-wave velocityrporosity Žas confirmed by Fig. 10 since a clear linear relationship exists between these two variables., and P-wave attenuationrclay content Žas it is confirmed by Fig. 11 since an approximate linear relationship exists between these two variables.. In addition, using the rules extracted, it was shown that P-wave velocity is closely related to porosity and P-wave attenuation is
closely related to clay content. Boadu Ž1997. also indicated that the most influential rock parameter on the attenuation is the clay content. In addition, our software ranked the variables in the order grain size, p-wave velocity, p-wave attenuation and clay contentrporosity Žsince clay content and porosity can be predicted from p-wave velocity and p-wave attenuation..
4. Conclusion In this paper, we developed the next generation of AintelligentB software that will identify the nonlinear relationship and mapping between well logs and seismic attributes. We developed a series of neural network models to analyze actual well logs. Wireline logs, such as travel time ŽDT., gamma ray ŽGR. and density ŽRHOB., were predicted based on SP, and resistivity logs. We also predicted travel time ŽDT. based on induction resistivity and vice versa. In addition, we developed the next generation of AintelligentB software that will identify the nonlinear relationship and mapping between rock proprieties
234
M. NikraÕesh, F. Aminzadehr Journal of Petroleum Science and Engineering 29 (2001) 221–238
and seismic attenuation and extracts rules Žknowledge. from these databases. The neuro-fuzzy model was used to implicitly cluster the data while finding the nonlinear mapping. For the example studied here, our software clustered the parameters as grain size, P-wave velocityrporosity, and P-wave attenuationr clay content. In addition our software ranked the variables as grain size, p-wave velocity, p-wave attenuation and clay contentrporosity.
as Abackpropagation.B The technique known as a Abackpropagation learning algorithmB is most often used to train a neural network towards a desired outcome by running a Atraining setB of data with known patterns through the network. Feedback from the training data is used to adjust weights until the correct patterns appear. Hecht-Nielsen Ž1990. and Medsker Ž1994. provide additional information. A.4. Perceptron
Acknowledgements The authors express their thanks to Dr. Roy Adams from EGI-University of Utah for his useful comments and suggestions. Appendix A, in part, is provided by Dr. Roy Adams.
Appendix A. A basic primer on neural network and fuzzy logic terminology A.1. Neural networks Neural networks are systems that A . . . use a number of simple computational units called ‘neurons’ . . . B and each neuron A . . . processes the incoming inputs to an output. The output is then linked to other neurons.B Žvon Altrock, 1995.. Neurons are also called Aprocessing elements.B A.2. Weight When used in reference to neural networks, AweightB defines the robustness or importance of the connection Žalso known as a link or synapse. between any two neurons. Medsker Ž1994. notes that weights A . . . express the relative strengths Žor mathematical value. of the various connections that transfer data from layer to layer.B A.3. Backpropagation learning algorithm In the simplest neural networks, information Žinputs and outputs. flows only one way. In more complex neural networks, information can flow in two directions, a AfeedforwardB direction and a AfeedbackB direction. The feedback process is known
There are two definitions of this term ŽHechtNielsen, 1990.. The AperceptronB is a classical neural network architecture. In addition, processing elements Žneurons. have been called Aperceptrons.B A.5. C FuzzinessD and C fuzzyD It is perhaps best to introduce the concept of AfuzzinessB using Zadeh’s original definition of fuzzy sets ŽZadeh, 1965.: AA fuzzy set is a class of objects with a continuum of grades of membership. Such a set is characterized by a membership Žcharacteristic. function which assigns to each object a grade of membership ranging between zero and one.B Zadeh Ž1973. further elaborates that fuzzy sets are A . . . classes of objects in which the transition from membership to nonmembership is gradual rather than abrupt.B Fuzzy logic is then defined as the A . . . use of fuzzy sets defined by membership functions in logical expressionsB Žvon Altrock, 1995.. Fuzziness and fuzzy can then be defined as having the characteristics of a fuzzy set. A.6. Neuro-fuzzy This is a noun that looks like an adjective. Unfortunately, Aneuro-fuzzyB is also used as an adjective, e.g. Aneuro-fuzzy logicB or Aneuro-fuzzy systems.B Given this confusing situation, a useful definition to keep in mind is: AThe combination of fuzzy logic and neural net technology is called ANeuroFuzzyB and combines the advantages of the two technologies.B Žvon Altrock 1995.. In addition, a neuro-fuzzy system is a neural network system that is self-training, but uses fuzzy logic for knowledge representation, the rules for behavior of the system, and for training the system.
M. NikraÕesh, F. Aminzadehr Journal of Petroleum Science and Engineering 29 (2001) 221–238
235
Fig. B.1. Typical neural network model.
A.7. CCrisp setsD and C Fuzzy setsD AConventional Žor crisp. sets contain objects that satisfy precise properties required for membership.B ŽBezdek and Pal, 1992.. Compare this to their definition that Afuzzy setsB A . . . contain objects that satisfy imprecise properties to varying degrees . . . B. Each member of a crisp set is either AtrueB or is Afalse,B whereas each member of a fuzzy set may have a certain degree of truth or a certain degree of falseness or may have of some degree of each!
Appendix B. Neural networks Details of neural networks are available in the literature ŽKohonen, 1987, 1997; Cybenko, 1989; Hecht-Nielsen, 1989; Widrow and Lehr, 1990; Lin and Lee, 1996. and, therefore, only the most important characteristics of neural networks will be mentioned. The typical neural network ŽFig. B.1. has an input layer, an output layer, and at least one hidden layer. Each layer is in communication with the succeeding layer via a set of connections of various weights, i.e. strengths. In a neural network, nonlinear elements are called various names, including nodes, neurons, or processing elements ŽFig. B.2.. A biological neuron is a nerve cell that receives, processes, and passes on information. Artificial neurons are
simple first-order approximations of biological neurons. Consider a single artificial neuron ŽFig. B.2. with a transfer function Ž y1Ž i. s f Ž z Ž i. .., connection weights, wj , and a node threshold, u . For each pattern i: z Ž i. s x 1Ž i. w 1 q x 2Ž i. w 2 q PPP qx NŽ i. wN q u for i s 1, . . . , P .
Ž B.1 .
All patterns may be represented in matrix notation as: z
Ž1.
zŽ 2 . . s . . zŽ P .
x 1Ž 1 .
x 2Ž 1 .
x 1Ž 2 .
x 2Ž 2 .
P P P
P P P
x 1Ž P .
x 2Ž P .
PPP
x NŽ 1 .
1
PPP PPP PPP PPP PPP
x NŽ 2 .
1 P P P 1
P P P x NŽ P .
w1 w2 . . . wN u
Ž B.2 . and: y1 s f Ž z . .
Ž B.3 .
The transfer function, f, is typically defined by a sigmoid function, such as the hyperbolic tangent function: f Ž z. s
e z y eyz e z q eyz
.
M. NikraÕesh, F. Aminzadehr Journal of Petroleum Science and Engineering 29 (2001) 221–238
236
jective of the learning process is to minimize the global error in the output nodes by adjusting the weights. This minimization is usually set up as an optimization problem. Here, we use the Levenberg– Marquardt algorithm, which is faster and more robust than conventional algorithms; however, it requires more memory. Using nonlinear statistical techniques, the conventional Levenberge–Marquardt algorithm Žoptimization algorithm for training the neural network. is modified. In this situation, the final global error in the output at each sampling time is related to the network parameters and a modified version of learning coefficient is defined. The following equations briefly show the difference between the conventional and the modified technique as used in this study. For the conventional technique:
Fig. B.2. Typical neuron.
In more compact notation: z s X 1wu s Xw q u ,
Ž B.4 . DW s Ž J T J q m2 I .
where: T
wu s w w T N u x ,
Ž B.5 . Ž B.6 .
X 1 s wX N 1x ,
and 1 s column vector of ones with P rows; X s P = N matrix with N input and P pattern; u s bias vector, vector with P rows of u ; and w s weights, vector with N rows. During learning, the information is propagated back through the network and used to update the connection weights Žbackpropagation algorithm.. The objective function for the training algorithm is usually set up as a squared error sum: Es
1 2
P
ÝŽ
2 Ži. i. y ŽŽ observed . y y Ž prediction . .
.
y1 T
J e,
Ž C-1.
whereas in the modified technique: DW s Ž J TL TL J q G TG .
y1 T T
J L L e,
Ž C-2.
where:
ˆ y1 , L TL s V Vˆi j s
1 2mq1
Ž C-3. m
Ý eˆiqk eˆjqk ,
Ž C-4.
ksym
ˆ s sˆ 2 I, V
Ž C-5.
Wˆ s W " k sˆ .
Ž C-6.
Ž B.7 .
is1
This objective function defines the error for the observed value at the output layer, which is propagated back through the network. During training, the weights are adjusted to minimize this sum of squared errors.
Appendix C. Modified Levenberge–Marquardt technique Several techniques have been proposed for training the neural network models. The most common technique is the backpropagation approach. The ob-
Appendix D. Neuro-fuzzy models In recent years, considerable attention has been devoted to the use of hybrid neural network-fuzzy logic approaches ŽJang, 1991, 1992. as an alternative for pattern recognition, clustering, and statistical and mathematical modeling. It has been shown that neural network models can be used to construct internal models that capture the presence of fuzzy rules. Neuro-fuzzy modeling is a technique of describing the behavior of a system using fuzzy inference rules using a neural network structure. The model
M. NikraÕesh, F. Aminzadehr Journal of Petroleum Science and Engineering 29 (2001) 221–238
has a unique feature in which it can express linguistically the characteristics of the complex nonlinear system. In this study, we will use the neuro-fuzzy model originally presented by Sugeno and Yasukawa Ž1993.. The neuro-fuzzy model is characterized by a set of rules. The rules are expressed as follows: R i : if x 1 is A1i and x 2 is Ai2 . . . and x n is Ain
Ž Antecedent. then y ) s f i Ž x 1 , x 2 , . . . , x n .
Ž D-1. Ž Consequent. ,
where f i Ž x 1 , x 2 , . . . , x n . can be constant, linear or fuzzy set. For the linear case: fi Ž x1 , x 2 , . . . , x n . s a i0 q a i1 x 1 q a i2 x 2 q . . . qa i n x n .
Ž D-2.
Therefore, the predicted value for output y is given by: y s Ý m i f i Ž x 1 , x 2 , . . . , x n . rÝ m i ,
237
iteratively adjusted so that each point is assigned to one cluster, and the centroid of each cluster is the mean of its assigned points. In general, the k-means technique will produce exactly kdifferent clusters of the greatest possible distinction. The algorithm is summarized in the following: 1. Consider each cluster consisting of a set of Msamples that are similar to each other: x 1 , x2 , x3, . . . , x m; 2. Choose a set of clusters y 1 , y 2 , y 3 , . . . , y k 4 ; 3. Assign the M samples to the clusters using the minimum Euclidean distance rule; 4. Compute a new cluster so as to minimize the cost function; 5. If any cluster changes, return to step 3; otherwise stop; 6. End.
Ž D-3.
i
with:
References
m i s Ł Aij Ž x j . ,
Ž D-4.
j
where R i is the ith rule, x j are input variables, yis output, A j Ž i are fuzzy membership functions Žfuzzy variables., and a i j are constant values. In this study, we will use the Adaptive NeuroFuzzy Inference System ŽANFIS. technique ŽJang and Gulley, 1995; The Math Workse, 1995.. The model uses neuro-adaptive learning techniques. This learning method is similar to that of neural networks. Given an inputroutput data set, the ANFIS can construct a fuzzy inference system ŽFIS. whose membership function parameters are adjusted using the backpropagation algorithm or similar optimization techniques. This allows fuzzy systems to learn from the data they are modeling.
Appendix E. k-Means clustering An early paper on k-means clustering was written by MacQueen Ž1967.. k-Means is an algorithm to assign a specific number of centers, k, to represent the clustering of N points Ž k - N .. These points are
Aminzadeh, F., Chatterjee, S., 1984r1985. Applications of clustering in exploration seismology. Geoexploration 23, 147–159. Aminzadeh, F., Katz, S., Aki, K., 1994. Adaptive neural network for generation of artificial earthquake precursors. IEEE Trans. Geosci. Remote Sens. 32 Ž6.. Baldwin, J.L., Otte, D.N., Wheatley, C.L., 1989. Computer emulation of human mental process: Application of neural network simulations to problems in well Log interpretation, Society of Petroleum Engineers, SPE Paper a19619, 481. Baldwin, J.L., Bateman, A.R.M., Wheatley, C.L., 1990. Application of neural network to the problem of mineral identification from well logs. Log Anal. 3, 279. Boadu, F.K., 1997. Rock properties and seismic attenuation: neural network analysis. Pure Appl. Geophys. 149, 507–524. Cybenko, G., 1989. Approximation by superposition of a sigmoidal function. Math. Control Sig. Syst. 2, 303. Hecht-Nielsen, R., 1989. Theory of backpropagation neural networks. IEEE Proc., Int. Conf. On Neural Networks, Washington DC, I-593. Hecht-Nielsen, R., 1990. Neurocomputing. Addison-Wesley Publishing, p. 433. Jang, J.S.R., 1991. Fuzzy modeling using generalized neural networks and Kalman filter algorithm. Proc. Ninth Natl. Conf. Artif. Intell. pp. 762–767. Jang, J.S.R., 1992. Self-learning fuzzy controllers based on temporal backpropagation. IEEE Trans. Neural Networks 3 Ž5.. Jang, J.S.R., Gulley, N., 1995. Fuzzy Logic Toolbox. The Math Works, Natick, MA. Klimentos, T., McCann, C., 1990. Relationship among compres-
238
M. NikraÕesh, F. Aminzadehr Journal of Petroleum Science and Engineering 29 (2001) 221–238
sional wave attenuation, porosity, clay content and permeability in sandstones. Geophysics 55, 991014. Kohonen, T., 1987. Self-Organization and Associate Memory. 2nd edn. Springer-Verlag, Berlin. Kohonen, T., 1997. Self-Organizing Maps. 2nd edn. Springer, Berlin. Lin, C.T., Lee, C.S.G., 1996. Neural Fuzzy Systems. Prentice-Hall, Englewood Cliffs, NJ. MacQueen, J., 1967. Some methods for classification and analysis of multivariate observation. In: LeCun, L.M., Neyman, J. ŽEds.., Proc. 5th Berkeley Symp. Math. Stat. Probab., vol. 1, Univ. California Press, pp. 281–297. Medsker, L.R., 1994. Hybrid Neural Network and Expert Systems. Kluwer Academic Publishing, p. 240. Nikravesh, M., 1998, ANeural network knowledge-based modeling of rock properties based on well log databases.B SPE 46206, 1998 SPE Western Regional Meeting, Bakersfield, California, 10–13 May. Nikravesh, M., Aminzadeh, F., 1998. Data mining and fusion with integrated neuro-fuzzy agents: rock properties and seismic attenuation. JCIS 1998, The Fourth Joint Conference on Information Sciences, Research Triangle Park, NC, USA, October 23–28. Nikravesh, M., Farell, A.E., Stanford, T.G., 1996. Model identifi-
cation of nonlinear time-variant processes via artificial neural network. Comput. Chem. Eng. 20 Ž11., 1277. Pezeshk, S., Camp, C.C., Karprapu, S., 1996. Geophysical log interpretation using neural network. J. Comput. Civil Eng. 10, 136 pp. Rogers, S.J., Fang, J.H., Karr, C.L., Stanley, D.A., 1992. Determination of lithology, from well logs using a neural network. AAPG Bull. 76, 731. Sugeno, M., Yasukawa, T., 1993. A fuzzy-logic-based approach to qualitative modeling. IEEE Trans. Fuzzy Syst. 1 Ž1.. The Math Workse, 1995. Natick. von Altrock, C., 1995. Fuzzy Logic and NeuroFuzzy Applications Explained. Prentice Hall PTR, 350 pp. Wong, P.M., Jiang, F.X., Taggart, I.J., 1995a. A critical comparison of neural networks and discrimination analysis in lithofacies, porosity and permeability prediction. J. Pet. Geol. 18, 191. Wong, P.M., Gedeon, T.D., Taggart, I.J., 1995b. An improved technique in prediction: a neural network approach. IEEE Trans. Geosci. Remote Sens. 33, 971. Zadeh, L.A., 1965. Fuzzy Sets: Information Control 8, 33353. Zadeh, L.A., 1973. Outline of a new approach to the analysis of complex systems and decision processes. IEEE Trans. Syst., Man, Cybernet., SMC-3, 244.
Mining and fusion of petroleum data with fuzzy logic and neural network agents Masoud Nikravesh a,b,) , Fred Aminzadeh b,c,1 a
Berkeley InitiatiÕe in Soft Computing (BISC), Computer Science DiÕision, Department of EECS, UniÕersity of California, Berkeley, CA 94720, USA b Zadeh Institute for Information Technology (ZIFIT), USA c dGB-USA, Sugar Land, TX, 77478, USA
Abstract Analyzing data from well logs and seismic is often a complex and laborious process because a physical relationship cannot be established to show how the data are correlated. In this study, we will develop the next generation of AintelligentB software that will identify the nonlinear relationship and mapping between well logsrrock properties and seismic information and extract rock properties, relevant reservoir information and rules Žknowledge. from these databases. The software will use fuzzy logic techniques because the data and our requirements are imperfect. In addition, it will use neural network techniques, since the functional structure of the data is unknown. In particular, the software will be used to group data into important data sets; extract and classify dominant and interesting patterns that exist between these data sets; discover secondary, tertiary and higher-order data patterns; and discover expected and unexpected structural relationships between data sets. q 2001 Published by Elsevier Science B.V. Keywords: Well log analysis; Seismic; Knowledge extraction; Intelligent; Reservoir characterization
1. Introduction In reservoir engineering, it is of great importance to characterize how seismic information Žattributes., the lithology and geology of the rocks are related to the well logs, such as porosity, density and gamma ray. However, data from well logs and seismic attributes are often difficult to analyze because of their complexity. A physical relationship cannot usually be established to show how the data are correlated,
)
Corresponding author. Tel.: q1-510-643-4522; fax: q1-510642-5775. E-mail address: [email protected] ŽM. Nikravesh.. 1 Fax: q1-281-265-2512
except by a laborious process, and the human ability to understand and use the information content of these data is limited. Neural networks provide the potential to establish a model from nonlinear, complex and multidimensional data and find wide application in analyzing experimental, industrial and field data ŽAminzadeh and Chatterjee, 1984r1985; Baldwin et al., 1989, 1990; Rogers et al., 1992; Wong et al., 1995a,b; Nikravesh et al., 1996; Pezeshk et al., 1996.. In recent years, the neural network literature has stimulated growing interest among reservoir engineers, geologists and geophysicist ŽKlimentos and McCann, 1990; Aminzadeh et al., 1994; Boadu, 1997; Nikravesh, 1998; Nikravesh and Aminzadeh, 1998.. Boadu Ž1997. applied artificial neural network suc-
0920-4105r01r$ - see front matter q 2001 Published by Elsevier Science B.V. PII: S 0 9 2 0 - 4 1 0 5 Ž 0 1 . 0 0 0 9 2 - 4
222
M. NikraÕesh, F. Aminzadehr Journal of Petroleum Science and Engineering 29 (2001) 221–238
cessfully to find the relationship between seismic and rock properties for Sandstones Rocks. In our recent studies ŽNikravesh and Aminzadeh, 1998., we further analyzed data published by Klimentos and McCann Ž1990. for Sandstones and recently analyzed by Boadu Ž1997. using an artificial neural network. It was concluded that neural network model had a better performance than a multiple linear regression model. Neural network, neuro-fuzzy and knowledge-based models have been successfully used to model rock properties based on well log databases ŽNikravesh, 1998.. However, using neural networks for identification purposes is more useful when a large number of data are available. In addition, conventional neural network models cannot deal with uncertainty in data due to fuzziness. In this study, our AintelligentB software will be built around two technologies: artificial neural network and fuzzy logic ŽAppendix A.. Each of them is expected to remedy one or more shortcomings of conventional data mining and knowledge management systems. The software will take advantage of the tolerance for imprecision that fuzzy logic can bring to bear on the process of knowledge acquisition from massive data sets. The software will use fuzzy logic techniques because the data and our requirements are imperfect. Fuzzy logic is considered to be appropriate to deal with the nature of uncertainty on system and human error which are not included in current reliability theories. We will integrate our recently developed algorithms and heuristics to induce fuzzy logic rules from data sets. An artificial neural network will be used to extract and fine-tune the rule base including granulation of variables and the characteristics of the membership functions. In this study, we begin by mining wireline logs Žsuch as density, gamma ray, travel time, SP and resistivity.. Then, the results will be used to train the neural network models. In addition, we will analyze the data to recognize the most important patterns, structures, relationships and characteristics based on neural network and neuro-fuzzy models. 2. Neural network and nonlinear mapping In this section, a series of neural network models ŽAppendix B. will be developed for nonlinear map-
ping between wireline logs. A series of neural network models will also be developed to analyze actual well log data and seismic information and the nonlinear mapping between wireline logs and seismic attributes will be recognized. In this study, wireline logs, such as travel time ŽDT., gamma ray ŽGR. and density ŽRHOB., will be predicted based on SP and resistivity ŽRILD. logs. In addition, we will predict travel time ŽDT. based on induction resistivity and vice versa. In this study, all logs are scaled uniformly between y1 and 1 and results are given in scaled domain. Fig. 1a through e shows typical behavior of SP, RILD, DT, GR and RHOB logs in scaled domain. The design of a neural network to predict DT, GR and RHOB based on RILD and SP logs starts with filtering, smoothing and interpolating values Žin a small horizon. for missing information in the data set. A first-order filter and a simple linear recursive parameter estimator for interpolating were used to filter and reconstruct the noisy data. The available data were divided into three data sets: training, testing and validation. The network was trained based on the training data set and continuously tested using a test data set during the training phase. The network was trained using a backpropagation algorithm and modified Levenberge–Marquardt optimization technique ŽAppendix C.. Training was stopped when prediction deteriorated with step. 2.1. TraÕel time (DT) prediction based on SP and resistiÕity (RILD) logs The neural network model to predict the DT has 14 input nodes Žtwo windows of data each with seven data points. representing SP Žseven points or input nodes. and RILD Žseven points or input nodes. logs. The hidden layer has five nodes. The output layer has three nodes representing the prediction of the DT Ža window of three data point.. Typical performance of the neural network for training, testing and validation data sets is shown in Fig. 2a, b, c and d. The network shows good performance for prediction of DT for training, testing and validation data sets. However, there is not a perfect match between actual and predicted values for DT is the testing and validation data sets. This is due to changes of the lithology from point to point. In other words,
M. NikraÕesh, F. Aminzadehr Journal of Petroleum Science and Engineering 29 (2001) 221–238
Fig. 1. Typical behavior of SP, RILD, DT, GR, and RHOB logs.
223
224
M. NikraÕesh, F. Aminzadehr Journal of Petroleum Science and Engineering 29 (2001) 221–238
Fig. 2. Typical neural network performance for prediction of DT based on SP and RILD.
some of the data points in the testing and validation data sets are in a lithological layer which was not presented in the training phase. Therefore, to have perfect mapping, it would be necessary to use the layering information Žusing other types of logs or linguistic information. as input into the network or use a larger data set for the training data set which represent all the possible behaviors in the data. 2.2. Gamma ray (GR) prediction based on SP and resistiÕity (RILD) logs In this study, a neural network model is developed to predict GR based on SP and RILD log. The network has 14 input nodes Žtwo windows of data, each with seven data points. representing SP Žseven
points or input nodes. and RILD Žseven points or input nodes. logs. The hidden layer has five nodes. The output layer has three nodes representing the prediction of the GR. Fig. 3a through d shows the performance of the neural network for training, testing and validation data. The neural network model shows a good performance for prediction of GR for training, testing and validation data sets. In comparison with previous studies ŽDT prediction., this study shows that the GR is not as sensitive as DT to noise in the data. In addition, a better global relationship exits between SP-resistivity–GR rather than SP-resistivity–DT. However, the local relationship is in the same order of complexity. Therefore, two models have the same performance for training Žexcellent performance.. However, the model for prediction of
M. NikraÕesh, F. Aminzadehr Journal of Petroleum Science and Engineering 29 (2001) 221–238
225
Fig. 3. Typical neural network performance for prediction of GR based on SP and RILD.
GR has a better generalization property. Since the two models have been trained based on the same criteria, it is unlikely that this lack of mapping for generalization is due to over fitting during the training phase. 2.3. Density (RHOB) prediction based on SP and resistiÕity (RILD) logs To predict density based on SP and RILD logs, a neural network model with 14 input nodes representing SP Žseven points or input nodes. and RILD Žseven points or input nodes. logs, five nodes in the hidden layer and three nodes in the output layer representing the prediction of the RHOB is developed. Fig. 4a through d shows a typical performance
of the neural network for the training, testing and validation data sets. The network shows excellent performance for the training data set as shown in Fig. 4a and b. The model shows a good performance for the testing data set as shown in Fig. 4c. Fig. 4d shows the performance of the neural network for the validation data set. The model has relatively good performance for the validation data set. Therefore, there is not a perfect match between the actual and predicted values for RHOB for the testing and validation data set. Since RHOB is directly related to lithology and layering, and to have perfect mapping, it would be necessary to use the layering information Žusing other types of logs or linguistic information. as an input into the network or use a larger data set
226
M. NikraÕesh, F. Aminzadehr Journal of Petroleum Science and Engineering 29 (2001) 221–238
Fig. 4. Typical neural network performance for prediction of RHOB based on SP and RILD.
for the training data set which represent all the possible behaviors in the data. In these cases, one can use a knowledge-based approach using knowledge of an expert and select more diverse information which represent all different possible layering as a training data set. Alternatively, one can use an automated clustering technique to recognize the important clusters existing in the data and use this information for selecting the training data set. 2.4. TraÕel time (DT) prediction based on resistiÕity (RILD) The neural network model to predict the DT has 11 input nodes representing a RILD log. The hidden
layer has seven nodes. The output layer has three nodes representing the prediction of the DT. Using engineering knowledge, a training data set is carefully selected so as to represent all the possible layering existing in the data. The typical performance of neural network for the training, testing and validation data sets is shown in Fig. 5a through d. As expected, the network has excellent performance for prediction of DT. Even though only RILD logs are used for prediction of DT, the network model has better performance than when SP and RILD logs used for prediction of DT Žcomparing Fig. 2a through d with 5a through D.. However, in this study, knowledge of an expert was used as extra information.
M. NikraÕesh, F. Aminzadehr Journal of Petroleum Science and Engineering 29 (2001) 221–238
227
Fig. 5. Typical neural network performance for prediction of DT based on RILD.
This knowledge not only reduced the complexity of the model, but also better prediction was achieved. 2.5. ResistiÕity (RILD) prediction based on traÕel time (DT) In this section, to show that the technique presented in the previous section is effective, the performance of the inverse model is tested. The network model has 11 input nodes representing DT, seven nodes in the hidden layer and three nodes in the output layer representing the prediction of the RILD. Fig. 6a through d shows the performance of the neural network model for the training, testing and
validation data sets. Fig. 6a and b shows that the neural network has excellent performance for the training data set. Fig. 6c and d shows the performance of the network for the testing and validation data set. The network shows relatively excellent performance for testing and validation purposes. As was mentioned in the previous section, using engineering knowledge, the complexity of the model was reduced and better performance was achieved. In addition, since the network model Žprediction of DT from resisitivity. and its inverse Žprediction of resisitivity based on DT. have relatively excellent performance and generalization properties, a one-toone mapping was achieved. Therefore, this implies
228
M. NikraÕesh, F. Aminzadehr Journal of Petroleum Science and Engineering 29 (2001) 221–238
Fig. 6. Typical neural network performance for prediction of RILD based on DT.
that a good representation of layering was selected based on knowledge of an expert.
3. Neural-fuzzy model for rule extraction In this section, a neuro-fuzzy model ŽAppendix . D will be developed for model identification and knowledge extraction Žrule extraction. purposes. The model is characterized by a set of rules which can be further used for representation of data in the form of linguistic variables. Therefore, in this situation the fuzzy variables become linguistic variables. The neuro-fuzzy technique is used to implicitly cluster the data while finding the nonlinear mapping. The neuro-fuzzy model developed in this study is an
approximate fuzzy model with triangular and Gaussian membership functions originally presented by Sugeno and Yasukawa Ž1993.. k-Mean technique ŽAppendix E. is used for clustering and the network is trained using a backpropagation algorithm and modified Levenberge–Marquardt optimization technique ŽAppendix C.. In this study, the effect of rock parameters and seismic attenuation on permeability will be analyzed based on soft computing techniques and experimental data. The software will use fuzzy logic techniques because the data and our requirements are imperfect. In addition, it will use neural network techniques, since the functional structure of the data is unknown. In particular, the software will be used to group data into important data sets; extract and classify domi-
M. NikraÕesh, F. Aminzadehr Journal of Petroleum Science and Engineering 29 (2001) 221–238
Fig. 7. Actual data ŽBoadu, 1997..
229
230
M. NikraÕesh, F. Aminzadehr Journal of Petroleum Science and Engineering 29 (2001) 221–238
Fig. 8. Typical rules extracted from data, 7 Rules. Ža. Typical rules extracted from data, 7 Rules ŽPorosity.. Žb. Typical rules extracted from data, 7 Rules ŽGrain Size.. Žc. Typical rules extracted from data, 7 Rules ŽClay Content.. Žd. Typical rules extracted from data, 7 Rules ŽP-Wave Velocity.. Že. Typical rules extracted from data, 7 Rules ŽP-Wave Attenuation..
M. NikraÕesh, F. Aminzadehr Journal of Petroleum Science and Engineering 29 (2001) 221–238
231
Table 1 Boundary of rules extracted from data Porosity
Grain size
Clay content
P-wave velocity
P-wave attenuation
wy0.4585, y0.3170x w0.4208, 0.5415x wy0.3610, y0.1599x wy0.2793, y0.0850x wy0.3472, y0.1856x w0.2700, 0.4811x wy0.2657, y0.1061x
wy0.6501, y0.3604x wy0.9351, y0.6673x wy0.7866, y0.4923x wy0.5670, y0.2908x wy0.1558, 0.1629x wy0.8077, y0.5538x w0.0274, 0.3488x
wy0.6198, y0.3605x w0.2101, 0.3068x wy0.3965, y0.1535x wy0.4005, y0.1613x wy0.8093, y0.5850x wy0.0001, 0.2087x wy0.4389, y0.1468x
w0.0893, 0.2830x wy0.7981, y0.7094x wy0.0850, 0.1302x wy0.1801, 0.0290x w0.1447, 0.3037x wy0.6217, y0.3860x wy0.1138, 0.1105x
wy0.6460–0.3480x w0.0572 0.2008x wy0.4406–0.1571x wy0.5113–0.2439x wy0.8610–0.6173x wy0.1003 0.1316x wy0.5570–0.1945x
nant and interesting patterns that exist between these data sets; and discover secondary, tertiary and
higher-order data patterns. The objective of this section is to predict the permeability based on grain
Fig. 9. Performance of Neural-Fuzzy model for prediction of permeability. Ža. Performance of Neural-Fuzzy model for prediction of permeability w K s f ŽP, G, C, PWV, PWA.x. Žb. Performance of Neural-Fuzzy model for prediction of permeability w K s f ŽP, C, PWV, PWA.x. Žc. Performance of Neural-Fuzzy model for prediction of permeability w K s f ŽP, PWV, PWA. x. Žd. Performance of Neural-Fuzzy model for prediction of permeability w K s f ŽG, C, PWV, PWA.x.
232
M. NikraÕesh, F. Aminzadehr Journal of Petroleum Science and Engineering 29 (2001) 221–238
Fig. 10. Relationship between P-wave velocity and porosity.
size, clay content, porosity, P-wave velocity and P-wave attenuation. 3.1. Prediction of permeability based on porosity, grain size, clay content, P-waÕe Õelocity and P-waÕe attenuation In this section, a neural-fuzzy model will be developed for nonlinear mapping and rule extraction Žknowledge extraction. between porosity, grain size, clay content, P-wave velocity, P-wave attenuation and permeability. Fig. 7 shows typical data, which has been used in this study. In this study, permeability will be predicted based on the following rules and Eqs. ŽD-1. through ŽD-4. see Appendix D: IF Rock Type s Sandstones AND Porositys w p1, p2x AND Grain Size s w g1, g 2x AND Clay Contents w c1,c2x AND P-Wave Vel.s wpwv1,pwv2x AND P-Wave Att.s wPwa1,pwa2x THEN Y ) s a0 q a1) P q a2)G q a3)C q a4) PWV q a5) PWA
where P is %porosity, G is grain size, C is clay content, PWV is P-wave velocity and P-wave attenuation, Y ) , is equivalent to f in Eq. ŽD-1.. Data are scaled uniformly between y1 and 1 and the result is given in the scaled domain. The available data were divided into three data sets: training, testing, and validation. The neuro-fuzzy model was trained based on a training data set and continuously tested using a test data set during the training phase. Training was stopped when it was found that the model’s prediction suffered upon continued training. Next, the number of rules was increased by one and training was repeated. Using this technique, an optimal number of rules were selected. Fig. 8a through e and Table 1 show typical rules extracted from the data. In Table 1, Columns 1 through 5 show the membership functions for porosity, grain size, clay content, P-wave velocity, and P-wave attenuation, respectively. Using the model defined by Eq. ŽD-1. through ŽD-4. and membership functions defined in Fig. 8a through e and Table 1, permeability was predicted as shown in Fig. 9a. In this study, seven rules were identified for prediction of permeability based on porosity, grain size, clay content, P-wave velocity, and P-wave attenuation ŽFig. 8a through e
M. NikraÕesh, F. Aminzadehr Journal of Petroleum Science and Engineering 29 (2001) 221–238
233
Fig. 11. Relationship between P-wave attenuation and clay content.
and Fig. 9a.. In addition, eight rules were identified for prediction of permeability based on porosity, clay content, P-wave velocity, and P-wave attenuation ŽFig. 9b.. Ten rules were identified for prediction of permeability based on porosity, P-wave velocity, and P-wave attenuation ŽFig. 9c.. Finally, six rules were identified for prediction of permeability based on grain size, clay content, P-wave velocity, and P-wave attenuation ŽFig. 9d.. The neural network model shows very good performance for prediction of permeability. In this situation, not only a nonlinear mapping and relationship was identified between porosity, grain size, clay content, P-wave velocity, and P-wave attenuation, and permeability; however, the rules existing between data were also identified. For this case study, our software clustered the parameters as grain size, P-wave velocityrporosity Žas confirmed by Fig. 10 since a clear linear relationship exists between these two variables., and P-wave attenuationrclay content Žas it is confirmed by Fig. 11 since an approximate linear relationship exists between these two variables.. In addition, using the rules extracted, it was shown that P-wave velocity is closely related to porosity and P-wave attenuation is
closely related to clay content. Boadu Ž1997. also indicated that the most influential rock parameter on the attenuation is the clay content. In addition, our software ranked the variables in the order grain size, p-wave velocity, p-wave attenuation and clay contentrporosity Žsince clay content and porosity can be predicted from p-wave velocity and p-wave attenuation..
4. Conclusion In this paper, we developed the next generation of AintelligentB software that will identify the nonlinear relationship and mapping between well logs and seismic attributes. We developed a series of neural network models to analyze actual well logs. Wireline logs, such as travel time ŽDT., gamma ray ŽGR. and density ŽRHOB., were predicted based on SP, and resistivity logs. We also predicted travel time ŽDT. based on induction resistivity and vice versa. In addition, we developed the next generation of AintelligentB software that will identify the nonlinear relationship and mapping between rock proprieties
234
M. NikraÕesh, F. Aminzadehr Journal of Petroleum Science and Engineering 29 (2001) 221–238
and seismic attenuation and extracts rules Žknowledge. from these databases. The neuro-fuzzy model was used to implicitly cluster the data while finding the nonlinear mapping. For the example studied here, our software clustered the parameters as grain size, P-wave velocityrporosity, and P-wave attenuationr clay content. In addition our software ranked the variables as grain size, p-wave velocity, p-wave attenuation and clay contentrporosity.
as Abackpropagation.B The technique known as a Abackpropagation learning algorithmB is most often used to train a neural network towards a desired outcome by running a Atraining setB of data with known patterns through the network. Feedback from the training data is used to adjust weights until the correct patterns appear. Hecht-Nielsen Ž1990. and Medsker Ž1994. provide additional information. A.4. Perceptron
Acknowledgements The authors express their thanks to Dr. Roy Adams from EGI-University of Utah for his useful comments and suggestions. Appendix A, in part, is provided by Dr. Roy Adams.
Appendix A. A basic primer on neural network and fuzzy logic terminology A.1. Neural networks Neural networks are systems that A . . . use a number of simple computational units called ‘neurons’ . . . B and each neuron A . . . processes the incoming inputs to an output. The output is then linked to other neurons.B Žvon Altrock, 1995.. Neurons are also called Aprocessing elements.B A.2. Weight When used in reference to neural networks, AweightB defines the robustness or importance of the connection Žalso known as a link or synapse. between any two neurons. Medsker Ž1994. notes that weights A . . . express the relative strengths Žor mathematical value. of the various connections that transfer data from layer to layer.B A.3. Backpropagation learning algorithm In the simplest neural networks, information Žinputs and outputs. flows only one way. In more complex neural networks, information can flow in two directions, a AfeedforwardB direction and a AfeedbackB direction. The feedback process is known
There are two definitions of this term ŽHechtNielsen, 1990.. The AperceptronB is a classical neural network architecture. In addition, processing elements Žneurons. have been called Aperceptrons.B A.5. C FuzzinessD and C fuzzyD It is perhaps best to introduce the concept of AfuzzinessB using Zadeh’s original definition of fuzzy sets ŽZadeh, 1965.: AA fuzzy set is a class of objects with a continuum of grades of membership. Such a set is characterized by a membership Žcharacteristic. function which assigns to each object a grade of membership ranging between zero and one.B Zadeh Ž1973. further elaborates that fuzzy sets are A . . . classes of objects in which the transition from membership to nonmembership is gradual rather than abrupt.B Fuzzy logic is then defined as the A . . . use of fuzzy sets defined by membership functions in logical expressionsB Žvon Altrock, 1995.. Fuzziness and fuzzy can then be defined as having the characteristics of a fuzzy set. A.6. Neuro-fuzzy This is a noun that looks like an adjective. Unfortunately, Aneuro-fuzzyB is also used as an adjective, e.g. Aneuro-fuzzy logicB or Aneuro-fuzzy systems.B Given this confusing situation, a useful definition to keep in mind is: AThe combination of fuzzy logic and neural net technology is called ANeuroFuzzyB and combines the advantages of the two technologies.B Žvon Altrock 1995.. In addition, a neuro-fuzzy system is a neural network system that is self-training, but uses fuzzy logic for knowledge representation, the rules for behavior of the system, and for training the system.
M. NikraÕesh, F. Aminzadehr Journal of Petroleum Science and Engineering 29 (2001) 221–238
235
Fig. B.1. Typical neural network model.
A.7. CCrisp setsD and C Fuzzy setsD AConventional Žor crisp. sets contain objects that satisfy precise properties required for membership.B ŽBezdek and Pal, 1992.. Compare this to their definition that Afuzzy setsB A . . . contain objects that satisfy imprecise properties to varying degrees . . . B. Each member of a crisp set is either AtrueB or is Afalse,B whereas each member of a fuzzy set may have a certain degree of truth or a certain degree of falseness or may have of some degree of each!
Appendix B. Neural networks Details of neural networks are available in the literature ŽKohonen, 1987, 1997; Cybenko, 1989; Hecht-Nielsen, 1989; Widrow and Lehr, 1990; Lin and Lee, 1996. and, therefore, only the most important characteristics of neural networks will be mentioned. The typical neural network ŽFig. B.1. has an input layer, an output layer, and at least one hidden layer. Each layer is in communication with the succeeding layer via a set of connections of various weights, i.e. strengths. In a neural network, nonlinear elements are called various names, including nodes, neurons, or processing elements ŽFig. B.2.. A biological neuron is a nerve cell that receives, processes, and passes on information. Artificial neurons are
simple first-order approximations of biological neurons. Consider a single artificial neuron ŽFig. B.2. with a transfer function Ž y1Ž i. s f Ž z Ž i. .., connection weights, wj , and a node threshold, u . For each pattern i: z Ž i. s x 1Ž i. w 1 q x 2Ž i. w 2 q PPP qx NŽ i. wN q u for i s 1, . . . , P .
Ž B.1 .
All patterns may be represented in matrix notation as: z
Ž1.
zŽ 2 . . s . . zŽ P .
x 1Ž 1 .
x 2Ž 1 .
x 1Ž 2 .
x 2Ž 2 .
P P P
P P P
x 1Ž P .
x 2Ž P .
PPP
x NŽ 1 .
1
PPP PPP PPP PPP PPP
x NŽ 2 .
1 P P P 1
P P P x NŽ P .
w1 w2 . . . wN u
Ž B.2 . and: y1 s f Ž z . .
Ž B.3 .
The transfer function, f, is typically defined by a sigmoid function, such as the hyperbolic tangent function: f Ž z. s
e z y eyz e z q eyz
.
M. NikraÕesh, F. Aminzadehr Journal of Petroleum Science and Engineering 29 (2001) 221–238
236
jective of the learning process is to minimize the global error in the output nodes by adjusting the weights. This minimization is usually set up as an optimization problem. Here, we use the Levenberg– Marquardt algorithm, which is faster and more robust than conventional algorithms; however, it requires more memory. Using nonlinear statistical techniques, the conventional Levenberge–Marquardt algorithm Žoptimization algorithm for training the neural network. is modified. In this situation, the final global error in the output at each sampling time is related to the network parameters and a modified version of learning coefficient is defined. The following equations briefly show the difference between the conventional and the modified technique as used in this study. For the conventional technique:
Fig. B.2. Typical neuron.
In more compact notation: z s X 1wu s Xw q u ,
Ž B.4 . DW s Ž J T J q m2 I .
where: T
wu s w w T N u x ,
Ž B.5 . Ž B.6 .
X 1 s wX N 1x ,
and 1 s column vector of ones with P rows; X s P = N matrix with N input and P pattern; u s bias vector, vector with P rows of u ; and w s weights, vector with N rows. During learning, the information is propagated back through the network and used to update the connection weights Žbackpropagation algorithm.. The objective function for the training algorithm is usually set up as a squared error sum: Es
1 2
P
ÝŽ
2 Ži. i. y ŽŽ observed . y y Ž prediction . .
.
y1 T
J e,
Ž C-1.
whereas in the modified technique: DW s Ž J TL TL J q G TG .
y1 T T
J L L e,
Ž C-2.
where:
ˆ y1 , L TL s V Vˆi j s
1 2mq1
Ž C-3. m
Ý eˆiqk eˆjqk ,
Ž C-4.
ksym
ˆ s sˆ 2 I, V
Ž C-5.
Wˆ s W " k sˆ .
Ž C-6.
Ž B.7 .
is1
This objective function defines the error for the observed value at the output layer, which is propagated back through the network. During training, the weights are adjusted to minimize this sum of squared errors.
Appendix C. Modified Levenberge–Marquardt technique Several techniques have been proposed for training the neural network models. The most common technique is the backpropagation approach. The ob-
Appendix D. Neuro-fuzzy models In recent years, considerable attention has been devoted to the use of hybrid neural network-fuzzy logic approaches ŽJang, 1991, 1992. as an alternative for pattern recognition, clustering, and statistical and mathematical modeling. It has been shown that neural network models can be used to construct internal models that capture the presence of fuzzy rules. Neuro-fuzzy modeling is a technique of describing the behavior of a system using fuzzy inference rules using a neural network structure. The model
M. NikraÕesh, F. Aminzadehr Journal of Petroleum Science and Engineering 29 (2001) 221–238
has a unique feature in which it can express linguistically the characteristics of the complex nonlinear system. In this study, we will use the neuro-fuzzy model originally presented by Sugeno and Yasukawa Ž1993.. The neuro-fuzzy model is characterized by a set of rules. The rules are expressed as follows: R i : if x 1 is A1i and x 2 is Ai2 . . . and x n is Ain
Ž Antecedent. then y ) s f i Ž x 1 , x 2 , . . . , x n .
Ž D-1. Ž Consequent. ,
where f i Ž x 1 , x 2 , . . . , x n . can be constant, linear or fuzzy set. For the linear case: fi Ž x1 , x 2 , . . . , x n . s a i0 q a i1 x 1 q a i2 x 2 q . . . qa i n x n .
Ž D-2.
Therefore, the predicted value for output y is given by: y s Ý m i f i Ž x 1 , x 2 , . . . , x n . rÝ m i ,
237
iteratively adjusted so that each point is assigned to one cluster, and the centroid of each cluster is the mean of its assigned points. In general, the k-means technique will produce exactly kdifferent clusters of the greatest possible distinction. The algorithm is summarized in the following: 1. Consider each cluster consisting of a set of Msamples that are similar to each other: x 1 , x2 , x3, . . . , x m; 2. Choose a set of clusters y 1 , y 2 , y 3 , . . . , y k 4 ; 3. Assign the M samples to the clusters using the minimum Euclidean distance rule; 4. Compute a new cluster so as to minimize the cost function; 5. If any cluster changes, return to step 3; otherwise stop; 6. End.
Ž D-3.
i
with:
References
m i s Ł Aij Ž x j . ,
Ž D-4.
j
where R i is the ith rule, x j are input variables, yis output, A j Ž i are fuzzy membership functions Žfuzzy variables., and a i j are constant values. In this study, we will use the Adaptive NeuroFuzzy Inference System ŽANFIS. technique ŽJang and Gulley, 1995; The Math Workse, 1995.. The model uses neuro-adaptive learning techniques. This learning method is similar to that of neural networks. Given an inputroutput data set, the ANFIS can construct a fuzzy inference system ŽFIS. whose membership function parameters are adjusted using the backpropagation algorithm or similar optimization techniques. This allows fuzzy systems to learn from the data they are modeling.
Appendix E. k-Means clustering An early paper on k-means clustering was written by MacQueen Ž1967.. k-Means is an algorithm to assign a specific number of centers, k, to represent the clustering of N points Ž k - N .. These points are
Aminzadeh, F., Chatterjee, S., 1984r1985. Applications of clustering in exploration seismology. Geoexploration 23, 147–159. Aminzadeh, F., Katz, S., Aki, K., 1994. Adaptive neural network for generation of artificial earthquake precursors. IEEE Trans. Geosci. Remote Sens. 32 Ž6.. Baldwin, J.L., Otte, D.N., Wheatley, C.L., 1989. Computer emulation of human mental process: Application of neural network simulations to problems in well Log interpretation, Society of Petroleum Engineers, SPE Paper a19619, 481. Baldwin, J.L., Bateman, A.R.M., Wheatley, C.L., 1990. Application of neural network to the problem of mineral identification from well logs. Log Anal. 3, 279. Boadu, F.K., 1997. Rock properties and seismic attenuation: neural network analysis. Pure Appl. Geophys. 149, 507–524. Cybenko, G., 1989. Approximation by superposition of a sigmoidal function. Math. Control Sig. Syst. 2, 303. Hecht-Nielsen, R., 1989. Theory of backpropagation neural networks. IEEE Proc., Int. Conf. On Neural Networks, Washington DC, I-593. Hecht-Nielsen, R., 1990. Neurocomputing. Addison-Wesley Publishing, p. 433. Jang, J.S.R., 1991. Fuzzy modeling using generalized neural networks and Kalman filter algorithm. Proc. Ninth Natl. Conf. Artif. Intell. pp. 762–767. Jang, J.S.R., 1992. Self-learning fuzzy controllers based on temporal backpropagation. IEEE Trans. Neural Networks 3 Ž5.. Jang, J.S.R., Gulley, N., 1995. Fuzzy Logic Toolbox. The Math Works, Natick, MA. Klimentos, T., McCann, C., 1990. Relationship among compres-
238
M. NikraÕesh, F. Aminzadehr Journal of Petroleum Science and Engineering 29 (2001) 221–238
sional wave attenuation, porosity, clay content and permeability in sandstones. Geophysics 55, 991014. Kohonen, T., 1987. Self-Organization and Associate Memory. 2nd edn. Springer-Verlag, Berlin. Kohonen, T., 1997. Self-Organizing Maps. 2nd edn. Springer, Berlin. Lin, C.T., Lee, C.S.G., 1996. Neural Fuzzy Systems. Prentice-Hall, Englewood Cliffs, NJ. MacQueen, J., 1967. Some methods for classification and analysis of multivariate observation. In: LeCun, L.M., Neyman, J. ŽEds.., Proc. 5th Berkeley Symp. Math. Stat. Probab., vol. 1, Univ. California Press, pp. 281–297. Medsker, L.R., 1994. Hybrid Neural Network and Expert Systems. Kluwer Academic Publishing, p. 240. Nikravesh, M., 1998, ANeural network knowledge-based modeling of rock properties based on well log databases.B SPE 46206, 1998 SPE Western Regional Meeting, Bakersfield, California, 10–13 May. Nikravesh, M., Aminzadeh, F., 1998. Data mining and fusion with integrated neuro-fuzzy agents: rock properties and seismic attenuation. JCIS 1998, The Fourth Joint Conference on Information Sciences, Research Triangle Park, NC, USA, October 23–28. Nikravesh, M., Farell, A.E., Stanford, T.G., 1996. Model identifi-
cation of nonlinear time-variant processes via artificial neural network. Comput. Chem. Eng. 20 Ž11., 1277. Pezeshk, S., Camp, C.C., Karprapu, S., 1996. Geophysical log interpretation using neural network. J. Comput. Civil Eng. 10, 136 pp. Rogers, S.J., Fang, J.H., Karr, C.L., Stanley, D.A., 1992. Determination of lithology, from well logs using a neural network. AAPG Bull. 76, 731. Sugeno, M., Yasukawa, T., 1993. A fuzzy-logic-based approach to qualitative modeling. IEEE Trans. Fuzzy Syst. 1 Ž1.. The Math Workse, 1995. Natick. von Altrock, C., 1995. Fuzzy Logic and NeuroFuzzy Applications Explained. Prentice Hall PTR, 350 pp. Wong, P.M., Jiang, F.X., Taggart, I.J., 1995a. A critical comparison of neural networks and discrimination analysis in lithofacies, porosity and permeability prediction. J. Pet. Geol. 18, 191. Wong, P.M., Gedeon, T.D., Taggart, I.J., 1995b. An improved technique in prediction: a neural network approach. IEEE Trans. Geosci. Remote Sens. 33, 971. Zadeh, L.A., 1965. Fuzzy Sets: Information Control 8, 33353. Zadeh, L.A., 1973. Outline of a new approach to the analysis of complex systems and decision processes. IEEE Trans. Syst., Man, Cybernet., SMC-3, 244.