- Email: [email protected]
A case study on homogeneous and heterogeneous reservoir porous media reconstruction by using generative adversarial networks
Available online at www.sciencedirect.com
ScienceDirect ScienceDirect Energy Procedia 00 (2018) 000–000
Availableonline onlineatatwww.sciencedirect.com www.sciencedirect.com Available Energy Procedia 00 (2018) 000–000
ScienceDirect ScienceDirect
www.elsevier.com/locate/procedia www.elsevier.com/locate/procedia
Energy Procedia 158 Energy Procedia 00(2019) (2017)6164–6169 000–000 www.elsevier.com/locate/procedia
10th International Conference on Applied Energy (ICAE2018), 22-25 August 2018, Hong Kong, China(ICAE2018), 22-25 August 2018, Hong Kong, 10th International Conference on Applied Energy China
A case study on homogeneous and heterogeneous reservoir porous A media case study and heterogeneous reservoir porous Theon 15thhomogeneous International Symposium on District Heating and Cooling reconstruction by using generative adversarial networks media reconstructiona, by using generative adversarial networks b c Liu *, Zhi Zhong , Ali Takbiri-Borujeni Assessing Siyan the feasibility of using the heat demand-outdoor d b , Qinwen Fua, Yuhao Yangca Mohammad Siyan Liua,for *,Kazemi Zhi Zhong , Ali Takbiri-Borujeni temperature function a long-term district heat demand forecast d a a Mohammad Kazemi , Qinwen Fu , Yuhao Yang *, A. Pina , P. Ferrão , J. Fournier ., B. Lacarrière , O. Le Corre
The University of Kansas, Department of Chemical & Petroleum Engineering, Lawrence, Kansas, USA, 66045 b a,b,c a Geology, The University a c c Bureau of Economic of Texas atb Austin, Texas, USA, 78713 a The UniversitycWest of Kansas, Department Chemical &ofPetroleum Kansas, USA, 66045 Virginia University,ofDepartment PetroleumEngineering, and Natural Lawrence, Gas Engineering bd Bureau of Rock Economic Geology, The University of Texasand at Austin, USA, 78713 Slippery University, Department of Petroleum NaturalTexas, Gas Engineering a IN+ Center for Innovation, c Technology and Policy Research - Instituto Superior Técnico, Av. Rovisco Pais 1, 1049-001 Lisbon, Portugal West Virginia University, Department of Petroleum and Natural Gas Engineering b d Veolia Recherche & Innovation, 291 Avenue Dreyfous Daniel, 78520 Limay, France Slippery Rock University, Department of Petroleum and Natural Gas Engineering c Département Systèmes Énergétiques et Environnement - IMT Atlantique, 4 rue Alfred Kastler, 44300 Nantes, France a
I. Andrić
Abstract
Abstract The advancing of modern X-ray computer tomography technology provides a powerful tool for us to illustrate the details inside the reservoir rock in three-dimensional space. Pore-scale rock characterization, modeling, and related fluid flow simulation can Abstract The advancing due of modern X-ray computer of tomography technology a powerful tool structure for us to modeling illustrate the detailssuch inside be challenging to the high complexity various rock samples. provides Conventional pore scale methods as the reservoir rock in three-dimensional space.for Pore-scale rock characterization, modeling, and related fluidinflow simulation can various stochastic methods were developed reservoir rock 3D microscopic structure reconstruction order to generate District heating networks are commonly addressed in the literature as one of the most effective solutions for decreasing the be challenging realizations due to the high complexity of various rock Conventional pore scaleIn structure modeling methods such as representative numerical simulations and samples. property approaches. this work, generative adversarial greenhouse gas emissions for from the building sector. These systems upscaling require high investments which are returned through the heat various stochastic methods were developed for reservoir rock 3D microscopic structure reconstruction in order to generate networks (GANs) used for climate generating the synthetic micro representations of porous by acquiring non-linear sales. Due to theis changed conditions and building renovation policies, heatrock demand in the future couldstatistical decrease, representative realizations for rock numerical and property upscaling approaches. thisimage work,pre-processing, generative adversarial information the real 3D imagessimulations in an unsupervised learning scheme. The relatedIn3D network prolonging from the investment return period. networks (GANs) is used for generating the synthetic micro representations of porousThe rocknetwork by acquiring non-linear statistical training and adjusting as well as data post-processing procedures are addressed. prediction a The main scope of this paper is to assess the feasibility of using the heat demand – outdoor temperature function results for heatfrom demand information from the sandstone real 3D rock in an unsupervised learning scheme. The related the 3D capability image pre-processing,for network homogeneous andimages a heterogeneous highforecast. The Berea district of Alvalade, located in LisbonEstaillades (Portugal),carbonate was used demonstrated as a case study. The districtofisGANs consisted of 665 training and adjusting as well as data post-processing procedures are network prediction results from a resolution rockinimage representations reconstruction, generated and addressed. real scenarios imagesThe are(low, compared via high) various visualizations buildingsporous that vary both construction period and typology. Three weather medium, and three district homogeneous Berea sandstone and a heterogeneous Estaillades carbonate demonstrated the capability of GANs for and inspections. The study also illustrated the importance the training imagethepreprocessing, which the highdata renovation scenarios were developed (shallow, intermediate,ofdeep). To estimate error, obtained heat indicating demand values were resolution porous rock image representations reconstruction, generated and real images arethe compared via various visualizations augmentation techniques can be one of the promising improvements in terms of capturing sparsely distributed features from compared with results from a dynamic heat demand model, previously developed and validated by the authors. and inspections. study also illustrated the synthetic importance of the training imagethepreprocessing, indicating the data heterogenous 3D The images realizations, meanwhile, robustness of which the model during training The results showed that and whenreconstructing only weather the change is considered, the margin of error could be acceptable for some applications augmentation techniques can be one of the promising improvements in terms of capturing the sparsely distributed features from process is enhanced limited datathan is available. (the error in annualwhen demand wasreal lower 20% for all weather scenarios considered). However, after introducing renovation heterogenous 3D images and reconstructing the synthetic realizations, meanwhile, the robustness of the model during training scenarios, the error value increased up to 59.5% (depending on the weather and renovation scenarios combination considered). process is enhanced when limited real data is available. The value of slope coefficient increased on average within the range of 3.8% up to 8% per decade, that corresponds to the Copyright © 2018 Elsevier All rights inAuthors. the number ofLtd. heating hoursreserved. of 22-139h during the heating season (depending on the combination of weather and ©decrease 2019 The Published by Elsevier Ltd. International Conference on Applied Selection peer-review under the license scientific committee of the for 10th7.8-12.7% renovation scenarios considered). On CC the BY-NC-ND other of hand, function intercept increased per decade (depending on the This is an and open access article underresponsibility the (http://creativecommons.org/licenses/by-nc-nd/4.0/) Copyright © 2018 Elsevier Ltd. All rights reserved. Energy (ICAE2018). Peer-review under responsibility the scientific committee The 10th parameters International Conference on Applied Energy. coupled scenarios). The valuesofsuggested could be used of to ICAE2018 modify the– function for the scenarios considered, and Selection and peer-review under responsibility of the scientific committee of the 10th International Conference on Applied improve the accuracy of heat demand estimations. Energy (ICAE2018).
© 2017 The Authors. Published by Elsevier Ltd. Peer-review under responsibility of the Scientific Committee of The 15th International Symposium on District Heating and * Corresponding author. Tel.: +1-(785)330-3163 Cooling.
E-mail address: [email protected] * Corresponding author. Tel.: +1-(785)330-3163 Keywords: Heat demand; Forecast; Climate change E-mail address: [email protected] 1876-6102 Copyright © 2018 Elsevier Ltd. All rights reserved. Selection and peer-review under responsibility of the scientific committee of the 10th International Conference on Applied Energy (ICAE2018). 1876-6102 Copyright © 2018 Elsevier Ltd. All rights reserved. Selection and peer-review under responsibility of the scientific committee of the 10th International Conference on Applied Energy (ICAE2018). 1876-6102 © 2017 The Authors. Published by Elsevier Ltd. Peer-review under responsibility of the Scientific Committee of The 15th International Symposium on District Heating and Cooling. 1876-6102 © 2019 The Authors. Published by Elsevier Ltd. This is an open access article under the CC BY-NC-ND license (http://creativecommons.org/licenses/by-nc-nd/4.0/) Peer-review under responsibility of the scientific committee of ICAE2018 – The 10th International Conference on Applied Energy. 10.1016/j.egypro.2019.01.493
2
Siyan Liu et al. / Energy Procedia 158 (2019) 6164–6169 Author name / Energy Procedia 00 (2018) 000–000
6165
Keywords: porous media, image reconstruction, generative adversarial networks, neural networks
1. Introduction Modeling and simulating porous media materials or rocks from pore-scale perspectives are substantial topics in various science and engineering field such as hydrology, material science, environmental engineering and petroleum engineering[1]. One of the powerful tools for revealing the micro-scale properties of the porous media is Micro-CT imaging technique which generates high-resolution three-dimensional images for microscopic structure reconstruction, thereafter the analytical and numerical studies can be conducted within the 3D realizations in terms of geometrical, morphological statistical analysis. Conventional grid-based computational fluid dynamics methods need huge amount of grid blocks to discretize the computation domain, also the local grid block refinements are necessary in order to capture the properties in smaller scale due to the heterogeneity which requires even more grids for reducing the calculation uncertainties Even equipped with modern massive paralleled computer system, the number of degree of freedoms are several orders of magnitude greater than the actual number of unknowns we can handle effectively, also the highly heterogenous geometrical domain coupled with Multiphysics phenomenon can causes convergence failure during numerical simulations. Pore network based upscaling techniques aimed at reduce the oversimplification and enhance the representations of pore network by applying coarse-scale discretization for void space[2]. In addition to pore network extraction and simplified fluid flow simulations, stochastic reconstruction of the pore scale to larger scale structure models are developed in order to generate synthetic realizations which respect the geometrical statistical distribution from the real samples[3].Unlike the various conventional stochastic 3D structure reconstruction techniques such as two-points, MPS methods[4]. Generative adversarial networks (GANs) is a type of delicate artificial neural network developed by Goodfellow [5] which contains a pair of neural networks working together during the training and predicting process, one of them called ‘generator’ G(z) which will evolve itself during training process to generate data from input noise variables pz(z), and another one called ‘discriminator’ D(x) which represents probability that the generated data x belongs to real data, they will be trained simultaneously to enhance the discriminability of D and maximize the probability of G to generate realizations. The network D and G itself can be either Multi-layer perceptron (MLP) network or convolutional neural networks (CNNs)[5]. The training process can be analog to a ‘min-max game’ between two players based on the value function V(G, D):[5]
min max V ( D , G ) E x ~ pdata ( x ) [log D ( x )] E z ~ pz ( z ) [log(1 D (G ( z )))] G
D
(1)
The nature of the unstable running mechanisms leads to the highly unstable training process, therefore, improved GANs are proposed to enhance the training stabilities in order to suit for multiple purposes[6]–[8]. Instead of applying GANs for 2D image reconstructions[5], there are some works proposed to reconstruct 3D images and objects[9]. Lukas et al., 2017 applied GANs to reconstruct the 3D realizations from Micro-CT images and evaluated the capabilities of learning pore-scale representations and microscopic properties from real images[10]. We constructed our GANs and demonstrated the training and realizations reconstructions from different rock Micro-CT images, also illustrated the impact of improved pre-processing and post-processing especially when dealing with the heterogeneous porous medium. 2. Methodology and experimental 2.1. Experimental data preprocessing The experimental data used in this study are two segmented 3D Micro-CT scanner images, which means the 3D images are three-dimensional binary data that can be easily converted to arrays with Cartesian coordinates and segmented labels. The label within each one of the space coordinates is either zero or one, which represents void space or rock matrix respectively. In this study, ‘Berea sandstone’[11] and ‘Estaillades carbonate’[12] are
6166
Siyan Liu et al. / Energy Procedia 158 (2019) 6164–6169 Author name / Energy Procedia 00 (2018) 000–000
3
downloaded from the website of Petroleum Engineering & Rock Mechanics Group, Department of Earth Science and Engineering, Imperial College.
Figure 1 Workflow of applying GANs in 3D image reconstruction (modified from Deep Learning for Computer Vision: Generative models and adversarial training (UPC 2016))
The former sample is very homogeneous fluvial sandstone in terms of grain size and shape, and the latter sample shows high heterogeneities in grain, pore size and shape distributions. Image from Berea sandstone contains 400 3 voxels and the image resolution is 5.345μm, Estaillades carbonate image has a larger size (1000 3 voxels) with 3.0035μm resolution. Due to the limitation of the image size as well as the computation resources, we use the method proposed by Lukas et al.[10] to extract sub-volumes of the original 3D image through a smaller size ‘moving cube sampler’, in order to extract sufficient representative training images, the moving strides are smaller than the size of sub-volume which means the sub-samples can overlap each other for generating more sub-samples. In the case of Berea sandstone image, the sub sample with 643 is big enough for capturing the pore, grain geometrical properties and the interactions among grains (Figure 2.a), but 643 sampling size is far less than favorable representative sub-volume due to the highly heterogeneous microscopic structure within Estaillades carbonate image, even after the optimization of our training strategy that eventually we are able to fit the subvolume of size 1283 into memory, it is still not properly reflect the geometrical properties (Figure 2.b). To tackle this issue, we tried several down sampling approaches to reduce the size of the original 3D image while preserving the geometrical properties as much as possible, as the results, the original 10003 images are reduced to 5123 and we are using 1283 as the sub-sampling size for Estaillades carbonate image (Figure 2.c).
4
Siyan Liu et al. / Energy Procedia 158 (2019) 6164–6169 Author name / Energy Procedia 00 (2018) 000–000
6167
Figure 2 Shows the sub-sampling strategies for the two rock samples. 2.a shows the 643 sub-volume size is sufficient to include the pore, grain geometrical properties, and interactions; 2.b shows the initially tested 1283 volume size is not able to capture enough information for training; 2.c depicts the solution for sub-sampling with 1283 volume size in a down-sampled 5123 for Estaillades carbonate sample
Ideally, the deep neural network can approximate any non-linear functions or non-linear problems as far as the network is well trained with representative training data set. Through visual inspections from the sub-volumes of Estaillades images, we found that part of the images contains sufficient information of the properties, but a lot of them contain much less representative structures due to the sparsely distributed pore-grain sizes and interactions. To solve this problem, we extended the 2D data augmentation techniques[13] to our 3D sub-sampling process by regularly or arbitrarily rotating the image (Figure 3a, 3b), therefore, a significant number of enhanced images are acquired for improving the robustness of the system.
Figure 3 Illustrate the data augmentation pre-processes. 3.a shows a rotated 643 sub-sample in the original 4003 Berea sandstone image; 3.b shows a rotated 1283 sub-sample in the resized 5123 Estaillades carbonate image.
2.2. Neural Network configurations and training Our customized generative adversarial networks (GANs) are modified from the prevailing network structure Deep Convolutional Generative Adversarial Networks (DCGAN)[6]. During the training processes, the discriminator is trained from the real images and serve as a ‘judge’ to determine whether the generated realizations from the generator are correct or not with loss indications. Various hyper-parameters such as input size, filter size, layers, activation functions, optimizers, learning rates are fixed or dynamically tuned based on the performance of the networks, training stabilities as well as the manual inspection of the final outputs. The training was performed on a desktop computer with a single NVIDIA GTX 1080TI GPU, so it is important to find the optimum training strategy for different case scenarios due to the limited computation power and memory available. 2.3. Neural Network output and post-processing The trained GAN models are capable of generating different size of realizations based on the image size input to the generator. Since the network learned the stochastic representations of the microscopic structure properties, then re-generating much larger size of 3D images are applicable, also the reconstruction of realizations is extremely fast compared to training processes, the only limitation again is the GPU memory limitation since the memory and storing space for 3D images are dramatically increasing with the linearly increasing image size or length. The
Siyan Liu et al. / Energy Procedia 158 (2019) 6164–6169 Author name / Energy Procedia 00 (2018) 000–000
6168
5
maximum size the image we tested on our desktop computer is 5283. Appropriate post-processing is necessary for image reconstructions. The networks can hardly be trained perfectly for certain case scenario, also due to the various level of complexity within different cases, the networks can be less trained or overtrained, as the results, de-noising processes and image enhancement techniques such as median filters are used, especially the results from Estaillades carbonate. The networks generated images are examined during the training process, and the final results are evaluated through 2D and 3D visualizations and comparisons. 3. Results After various testing and optimization of the workflow including preprocessing, training and post-processing strategies, we obtained the results showing the 3D image reconstruction capabilities of GANs. Our input data is the sub-volume images sampled from the original 3D image through rotating augmentation approach. Size of subsamples for Berea sandstone and Estaillades carbonate are 643 and 1283 respectively. During the training, the network will periodically store checkpoints with necessary training status information and the instance for current status generator and discriminator. A semi-auto script is assisting us to do the manual visual inspection for the generated realizations from the generator. Figure 4 shows the generated images from the generator in different training stages, more training iterations is not necessarily producing better results, but it is good to try a longer training test in most cases.
Figure 4. The real images and the images generated by the generator during training process for Berea sandstone sample and Estaillades carbonate sample
After a long period of training time when the loss of generator, discriminator is stabilized, also when the quality of the synthetic images tends to be stable. The generator status from last several iterations or the ones showing best performance through manual inspection is used for image reconstruction in different sizes. Figure 5 shows the 3D visualizations of image comparison between real rock scanned image and GANs synthetic image. We also tested the training process from the input data with or without data augmentation preprocess, the training for Berea sandstone became stable after around 3,000 generator iterations instead of more than 10,000 iterations in our previous experiments without data augmentations. In the Estaillades carbonate case, before augmentation we can hardly train the GANs to produce the meaningless images after proper post-processing, with data augmentation, our generator can produce some images that are somewhat showing heterogeneities in small scale. But as we can see the 3D view and 3D slicing view for real Estaillades carbonate and synthetic images, the generated 3D image is not very representative in the whole original image scale, it seems that the network learned the features from the smaller scale which comes from the 1283 size of sub-volume samples. Due to the computation memory limitation, it is hard to increase the sub-volume size for obtaining more representative training data, but by using properly designed preprocessing workflow tends to improve the feature learning capability of the networks.
6
Author name Energy Procedia 00 (2018) Siyan Liu/ et al. / Energy Procedia 158000–000 (2019) 6164–6169
6169
Figure 5. The selected synthetic 3D image and the real images comparison in 3D view and 3D slicing view
4. Conclusion In this study, the GAN models are created based on the improved the sub-sampling approach to reconstruct the 3D realizations of relative homogeneous Berea sandstone and relative heterogeneous Estaillades carbonate, they show promising capabilities to represent rock samples’ physical properties in 3-dimensional space. The network shows good capabilities of reconstructing homogeneous rock sample and still need to be improved for obtaining good representations of heterogeneous porous media. Data augmentation pre-processing speeds up the convergence of the training process and also shows advantages of capturing some level of heterogeneities in 3D images. Future work will include applying GANs for multi-properties characterizations in porous media, and improvement of the network configurations and training stabilities in order to enhance the generalization capability to deal with more complex porous media problems. References [1] [2] [3] [4] [5] [6] [7] [8] [9] [10] [11] [12] [13]
Martin J. Blunt, Multiphase Flow in Permeable Media A Pore-Scale Perspective. London: Cambridge University Press, 2017. A. Q. Raeini, B. Bijeljic, and M. J. Blunt, “Generalized network modeling : Network extraction as a coarse-scale discretization of the void space of porous media,” vol. 013312, pp. 1–17, 2017. G. Mariethoz and J. Caers, Multiple-point Geostatistics: Stochastic Modeling with Training Images. 2014. S. Strebelle, “Conditional Simulation of Complex Geological Structures Using Multiple-Point Statistics 1,” vol. 34. 1, pp. 1–21, 2002. I. Goodfellow et al., “Generative Adversarial Nets,” Adv. Neural Inf. Process. Syst. 27, pp. 2672–2680, 2014. A. Radford, L. Metz, and S. Chintala, “Unsupervised Representation Learning with Deep Convolutional Generative Adversarial Networks,” pp. 1–16, 2015. M. Arjovsky, S. Chintala, and L. Bottou, “Wasserstein GAN,” 2017. I. Gulrajani, “Improved Training of Wasserstein GANs.” J. Wu, C. Zhang, T. Xue, W. T. Freeman, and J. B. Tenenbaum, “Learning a Probabilistic Latent Space of Object Shapes via 3D Generative-Adversarial Modeling,” no. Nips, 2016. L. Mosser, O. Dubrule, and M. J. Blunt, “Reconstruction of three-dimensional porous media using generative adversarial neural networks,” vol. 043309, no. April, 2017. H. Dong and M. J. Blunt, “Pore-network extraction from micro-computerized-tomography images,” no. September, pp. 1–11, 2009. “Micro-CT images of Estaillades carbonate rock,” 2015. [Online]. Available: http://www.imperial.ac.uk/earthscience/research/research-groups/perm/research/pore-scale-modelling/micro-ct-images-and-networks/. J. Wang and L. Perez, “The Effectiveness of Data Augmentation in Image Classification using Deep Learning.”
ScienceDirect ScienceDirect Energy Procedia 00 (2018) 000–000
Availableonline onlineatatwww.sciencedirect.com www.sciencedirect.com Available Energy Procedia 00 (2018) 000–000
ScienceDirect ScienceDirect
www.elsevier.com/locate/procedia www.elsevier.com/locate/procedia
Energy Procedia 158 Energy Procedia 00(2019) (2017)6164–6169 000–000 www.elsevier.com/locate/procedia
10th International Conference on Applied Energy (ICAE2018), 22-25 August 2018, Hong Kong, China(ICAE2018), 22-25 August 2018, Hong Kong, 10th International Conference on Applied Energy China
A case study on homogeneous and heterogeneous reservoir porous A media case study and heterogeneous reservoir porous Theon 15thhomogeneous International Symposium on District Heating and Cooling reconstruction by using generative adversarial networks media reconstructiona, by using generative adversarial networks b c Liu *, Zhi Zhong , Ali Takbiri-Borujeni Assessing Siyan the feasibility of using the heat demand-outdoor d b , Qinwen Fua, Yuhao Yangca Mohammad Siyan Liua,for *,Kazemi Zhi Zhong , Ali Takbiri-Borujeni temperature function a long-term district heat demand forecast d a a Mohammad Kazemi , Qinwen Fu , Yuhao Yang *, A. Pina , P. Ferrão , J. Fournier ., B. Lacarrière , O. Le Corre
The University of Kansas, Department of Chemical & Petroleum Engineering, Lawrence, Kansas, USA, 66045 b a,b,c a Geology, The University a c c Bureau of Economic of Texas atb Austin, Texas, USA, 78713 a The UniversitycWest of Kansas, Department Chemical &ofPetroleum Kansas, USA, 66045 Virginia University,ofDepartment PetroleumEngineering, and Natural Lawrence, Gas Engineering bd Bureau of Rock Economic Geology, The University of Texasand at Austin, USA, 78713 Slippery University, Department of Petroleum NaturalTexas, Gas Engineering a IN+ Center for Innovation, c Technology and Policy Research - Instituto Superior Técnico, Av. Rovisco Pais 1, 1049-001 Lisbon, Portugal West Virginia University, Department of Petroleum and Natural Gas Engineering b d Veolia Recherche & Innovation, 291 Avenue Dreyfous Daniel, 78520 Limay, France Slippery Rock University, Department of Petroleum and Natural Gas Engineering c Département Systèmes Énergétiques et Environnement - IMT Atlantique, 4 rue Alfred Kastler, 44300 Nantes, France a
I. Andrić
Abstract
Abstract The advancing of modern X-ray computer tomography technology provides a powerful tool for us to illustrate the details inside the reservoir rock in three-dimensional space. Pore-scale rock characterization, modeling, and related fluid flow simulation can Abstract The advancing due of modern X-ray computer of tomography technology a powerful tool structure for us to modeling illustrate the detailssuch inside be challenging to the high complexity various rock samples. provides Conventional pore scale methods as the reservoir rock in three-dimensional space.for Pore-scale rock characterization, modeling, and related fluidinflow simulation can various stochastic methods were developed reservoir rock 3D microscopic structure reconstruction order to generate District heating networks are commonly addressed in the literature as one of the most effective solutions for decreasing the be challenging realizations due to the high complexity of various rock Conventional pore scaleIn structure modeling methods such as representative numerical simulations and samples. property approaches. this work, generative adversarial greenhouse gas emissions for from the building sector. These systems upscaling require high investments which are returned through the heat various stochastic methods were developed for reservoir rock 3D microscopic structure reconstruction in order to generate networks (GANs) used for climate generating the synthetic micro representations of porous by acquiring non-linear sales. Due to theis changed conditions and building renovation policies, heatrock demand in the future couldstatistical decrease, representative realizations for rock numerical and property upscaling approaches. thisimage work,pre-processing, generative adversarial information the real 3D imagessimulations in an unsupervised learning scheme. The relatedIn3D network prolonging from the investment return period. networks (GANs) is used for generating the synthetic micro representations of porousThe rocknetwork by acquiring non-linear statistical training and adjusting as well as data post-processing procedures are addressed. prediction a The main scope of this paper is to assess the feasibility of using the heat demand – outdoor temperature function results for heatfrom demand information from the sandstone real 3D rock in an unsupervised learning scheme. The related the 3D capability image pre-processing,for network homogeneous andimages a heterogeneous highforecast. The Berea district of Alvalade, located in LisbonEstaillades (Portugal),carbonate was used demonstrated as a case study. The districtofisGANs consisted of 665 training and adjusting as well as data post-processing procedures are network prediction results from a resolution rockinimage representations reconstruction, generated and addressed. real scenarios imagesThe are(low, compared via high) various visualizations buildingsporous that vary both construction period and typology. Three weather medium, and three district homogeneous Berea sandstone and a heterogeneous Estaillades carbonate demonstrated the capability of GANs for and inspections. The study also illustrated the importance the training imagethepreprocessing, which the highdata renovation scenarios were developed (shallow, intermediate,ofdeep). To estimate error, obtained heat indicating demand values were resolution porous rock image representations reconstruction, generated and real images arethe compared via various visualizations augmentation techniques can be one of the promising improvements in terms of capturing sparsely distributed features from compared with results from a dynamic heat demand model, previously developed and validated by the authors. and inspections. study also illustrated the synthetic importance of the training imagethepreprocessing, indicating the data heterogenous 3D The images realizations, meanwhile, robustness of which the model during training The results showed that and whenreconstructing only weather the change is considered, the margin of error could be acceptable for some applications augmentation techniques can be one of the promising improvements in terms of capturing the sparsely distributed features from process is enhanced limited datathan is available. (the error in annualwhen demand wasreal lower 20% for all weather scenarios considered). However, after introducing renovation heterogenous 3D images and reconstructing the synthetic realizations, meanwhile, the robustness of the model during training scenarios, the error value increased up to 59.5% (depending on the weather and renovation scenarios combination considered). process is enhanced when limited real data is available. The value of slope coefficient increased on average within the range of 3.8% up to 8% per decade, that corresponds to the Copyright © 2018 Elsevier All rights inAuthors. the number ofLtd. heating hoursreserved. of 22-139h during the heating season (depending on the combination of weather and ©decrease 2019 The Published by Elsevier Ltd. International Conference on Applied Selection peer-review under the license scientific committee of the for 10th7.8-12.7% renovation scenarios considered). On CC the BY-NC-ND other of hand, function intercept increased per decade (depending on the This is an and open access article underresponsibility the (http://creativecommons.org/licenses/by-nc-nd/4.0/) Copyright © 2018 Elsevier Ltd. All rights reserved. Energy (ICAE2018). Peer-review under responsibility the scientific committee The 10th parameters International Conference on Applied Energy. coupled scenarios). The valuesofsuggested could be used of to ICAE2018 modify the– function for the scenarios considered, and Selection and peer-review under responsibility of the scientific committee of the 10th International Conference on Applied improve the accuracy of heat demand estimations. Energy (ICAE2018).
© 2017 The Authors. Published by Elsevier Ltd. Peer-review under responsibility of the Scientific Committee of The 15th International Symposium on District Heating and * Corresponding author. Tel.: +1-(785)330-3163 Cooling.
E-mail address: [email protected] * Corresponding author. Tel.: +1-(785)330-3163 Keywords: Heat demand; Forecast; Climate change E-mail address: [email protected] 1876-6102 Copyright © 2018 Elsevier Ltd. All rights reserved. Selection and peer-review under responsibility of the scientific committee of the 10th International Conference on Applied Energy (ICAE2018). 1876-6102 Copyright © 2018 Elsevier Ltd. All rights reserved. Selection and peer-review under responsibility of the scientific committee of the 10th International Conference on Applied Energy (ICAE2018). 1876-6102 © 2017 The Authors. Published by Elsevier Ltd. Peer-review under responsibility of the Scientific Committee of The 15th International Symposium on District Heating and Cooling. 1876-6102 © 2019 The Authors. Published by Elsevier Ltd. This is an open access article under the CC BY-NC-ND license (http://creativecommons.org/licenses/by-nc-nd/4.0/) Peer-review under responsibility of the scientific committee of ICAE2018 – The 10th International Conference on Applied Energy. 10.1016/j.egypro.2019.01.493
2
Siyan Liu et al. / Energy Procedia 158 (2019) 6164–6169 Author name / Energy Procedia 00 (2018) 000–000
6165
Keywords: porous media, image reconstruction, generative adversarial networks, neural networks
1. Introduction Modeling and simulating porous media materials or rocks from pore-scale perspectives are substantial topics in various science and engineering field such as hydrology, material science, environmental engineering and petroleum engineering[1]. One of the powerful tools for revealing the micro-scale properties of the porous media is Micro-CT imaging technique which generates high-resolution three-dimensional images for microscopic structure reconstruction, thereafter the analytical and numerical studies can be conducted within the 3D realizations in terms of geometrical, morphological statistical analysis. Conventional grid-based computational fluid dynamics methods need huge amount of grid blocks to discretize the computation domain, also the local grid block refinements are necessary in order to capture the properties in smaller scale due to the heterogeneity which requires even more grids for reducing the calculation uncertainties Even equipped with modern massive paralleled computer system, the number of degree of freedoms are several orders of magnitude greater than the actual number of unknowns we can handle effectively, also the highly heterogenous geometrical domain coupled with Multiphysics phenomenon can causes convergence failure during numerical simulations. Pore network based upscaling techniques aimed at reduce the oversimplification and enhance the representations of pore network by applying coarse-scale discretization for void space[2]. In addition to pore network extraction and simplified fluid flow simulations, stochastic reconstruction of the pore scale to larger scale structure models are developed in order to generate synthetic realizations which respect the geometrical statistical distribution from the real samples[3].Unlike the various conventional stochastic 3D structure reconstruction techniques such as two-points, MPS methods[4]. Generative adversarial networks (GANs) is a type of delicate artificial neural network developed by Goodfellow [5] which contains a pair of neural networks working together during the training and predicting process, one of them called ‘generator’ G(z) which will evolve itself during training process to generate data from input noise variables pz(z), and another one called ‘discriminator’ D(x) which represents probability that the generated data x belongs to real data, they will be trained simultaneously to enhance the discriminability of D and maximize the probability of G to generate realizations. The network D and G itself can be either Multi-layer perceptron (MLP) network or convolutional neural networks (CNNs)[5]. The training process can be analog to a ‘min-max game’ between two players based on the value function V(G, D):[5]
min max V ( D , G ) E x ~ pdata ( x ) [log D ( x )] E z ~ pz ( z ) [log(1 D (G ( z )))] G
D
(1)
The nature of the unstable running mechanisms leads to the highly unstable training process, therefore, improved GANs are proposed to enhance the training stabilities in order to suit for multiple purposes[6]–[8]. Instead of applying GANs for 2D image reconstructions[5], there are some works proposed to reconstruct 3D images and objects[9]. Lukas et al., 2017 applied GANs to reconstruct the 3D realizations from Micro-CT images and evaluated the capabilities of learning pore-scale representations and microscopic properties from real images[10]. We constructed our GANs and demonstrated the training and realizations reconstructions from different rock Micro-CT images, also illustrated the impact of improved pre-processing and post-processing especially when dealing with the heterogeneous porous medium. 2. Methodology and experimental 2.1. Experimental data preprocessing The experimental data used in this study are two segmented 3D Micro-CT scanner images, which means the 3D images are three-dimensional binary data that can be easily converted to arrays with Cartesian coordinates and segmented labels. The label within each one of the space coordinates is either zero or one, which represents void space or rock matrix respectively. In this study, ‘Berea sandstone’[11] and ‘Estaillades carbonate’[12] are
6166
Siyan Liu et al. / Energy Procedia 158 (2019) 6164–6169 Author name / Energy Procedia 00 (2018) 000–000
3
downloaded from the website of Petroleum Engineering & Rock Mechanics Group, Department of Earth Science and Engineering, Imperial College.
Figure 1 Workflow of applying GANs in 3D image reconstruction (modified from Deep Learning for Computer Vision: Generative models and adversarial training (UPC 2016))
The former sample is very homogeneous fluvial sandstone in terms of grain size and shape, and the latter sample shows high heterogeneities in grain, pore size and shape distributions. Image from Berea sandstone contains 400 3 voxels and the image resolution is 5.345μm, Estaillades carbonate image has a larger size (1000 3 voxels) with 3.0035μm resolution. Due to the limitation of the image size as well as the computation resources, we use the method proposed by Lukas et al.[10] to extract sub-volumes of the original 3D image through a smaller size ‘moving cube sampler’, in order to extract sufficient representative training images, the moving strides are smaller than the size of sub-volume which means the sub-samples can overlap each other for generating more sub-samples. In the case of Berea sandstone image, the sub sample with 643 is big enough for capturing the pore, grain geometrical properties and the interactions among grains (Figure 2.a), but 643 sampling size is far less than favorable representative sub-volume due to the highly heterogeneous microscopic structure within Estaillades carbonate image, even after the optimization of our training strategy that eventually we are able to fit the subvolume of size 1283 into memory, it is still not properly reflect the geometrical properties (Figure 2.b). To tackle this issue, we tried several down sampling approaches to reduce the size of the original 3D image while preserving the geometrical properties as much as possible, as the results, the original 10003 images are reduced to 5123 and we are using 1283 as the sub-sampling size for Estaillades carbonate image (Figure 2.c).
4
Siyan Liu et al. / Energy Procedia 158 (2019) 6164–6169 Author name / Energy Procedia 00 (2018) 000–000
6167
Figure 2 Shows the sub-sampling strategies for the two rock samples. 2.a shows the 643 sub-volume size is sufficient to include the pore, grain geometrical properties, and interactions; 2.b shows the initially tested 1283 volume size is not able to capture enough information for training; 2.c depicts the solution for sub-sampling with 1283 volume size in a down-sampled 5123 for Estaillades carbonate sample
Ideally, the deep neural network can approximate any non-linear functions or non-linear problems as far as the network is well trained with representative training data set. Through visual inspections from the sub-volumes of Estaillades images, we found that part of the images contains sufficient information of the properties, but a lot of them contain much less representative structures due to the sparsely distributed pore-grain sizes and interactions. To solve this problem, we extended the 2D data augmentation techniques[13] to our 3D sub-sampling process by regularly or arbitrarily rotating the image (Figure 3a, 3b), therefore, a significant number of enhanced images are acquired for improving the robustness of the system.
Figure 3 Illustrate the data augmentation pre-processes. 3.a shows a rotated 643 sub-sample in the original 4003 Berea sandstone image; 3.b shows a rotated 1283 sub-sample in the resized 5123 Estaillades carbonate image.
2.2. Neural Network configurations and training Our customized generative adversarial networks (GANs) are modified from the prevailing network structure Deep Convolutional Generative Adversarial Networks (DCGAN)[6]. During the training processes, the discriminator is trained from the real images and serve as a ‘judge’ to determine whether the generated realizations from the generator are correct or not with loss indications. Various hyper-parameters such as input size, filter size, layers, activation functions, optimizers, learning rates are fixed or dynamically tuned based on the performance of the networks, training stabilities as well as the manual inspection of the final outputs. The training was performed on a desktop computer with a single NVIDIA GTX 1080TI GPU, so it is important to find the optimum training strategy for different case scenarios due to the limited computation power and memory available. 2.3. Neural Network output and post-processing The trained GAN models are capable of generating different size of realizations based on the image size input to the generator. Since the network learned the stochastic representations of the microscopic structure properties, then re-generating much larger size of 3D images are applicable, also the reconstruction of realizations is extremely fast compared to training processes, the only limitation again is the GPU memory limitation since the memory and storing space for 3D images are dramatically increasing with the linearly increasing image size or length. The
Siyan Liu et al. / Energy Procedia 158 (2019) 6164–6169 Author name / Energy Procedia 00 (2018) 000–000
6168
5
maximum size the image we tested on our desktop computer is 5283. Appropriate post-processing is necessary for image reconstructions. The networks can hardly be trained perfectly for certain case scenario, also due to the various level of complexity within different cases, the networks can be less trained or overtrained, as the results, de-noising processes and image enhancement techniques such as median filters are used, especially the results from Estaillades carbonate. The networks generated images are examined during the training process, and the final results are evaluated through 2D and 3D visualizations and comparisons. 3. Results After various testing and optimization of the workflow including preprocessing, training and post-processing strategies, we obtained the results showing the 3D image reconstruction capabilities of GANs. Our input data is the sub-volume images sampled from the original 3D image through rotating augmentation approach. Size of subsamples for Berea sandstone and Estaillades carbonate are 643 and 1283 respectively. During the training, the network will periodically store checkpoints with necessary training status information and the instance for current status generator and discriminator. A semi-auto script is assisting us to do the manual visual inspection for the generated realizations from the generator. Figure 4 shows the generated images from the generator in different training stages, more training iterations is not necessarily producing better results, but it is good to try a longer training test in most cases.
Figure 4. The real images and the images generated by the generator during training process for Berea sandstone sample and Estaillades carbonate sample
After a long period of training time when the loss of generator, discriminator is stabilized, also when the quality of the synthetic images tends to be stable. The generator status from last several iterations or the ones showing best performance through manual inspection is used for image reconstruction in different sizes. Figure 5 shows the 3D visualizations of image comparison between real rock scanned image and GANs synthetic image. We also tested the training process from the input data with or without data augmentation preprocess, the training for Berea sandstone became stable after around 3,000 generator iterations instead of more than 10,000 iterations in our previous experiments without data augmentations. In the Estaillades carbonate case, before augmentation we can hardly train the GANs to produce the meaningless images after proper post-processing, with data augmentation, our generator can produce some images that are somewhat showing heterogeneities in small scale. But as we can see the 3D view and 3D slicing view for real Estaillades carbonate and synthetic images, the generated 3D image is not very representative in the whole original image scale, it seems that the network learned the features from the smaller scale which comes from the 1283 size of sub-volume samples. Due to the computation memory limitation, it is hard to increase the sub-volume size for obtaining more representative training data, but by using properly designed preprocessing workflow tends to improve the feature learning capability of the networks.
6
Author name Energy Procedia 00 (2018) Siyan Liu/ et al. / Energy Procedia 158000–000 (2019) 6164–6169
6169
Figure 5. The selected synthetic 3D image and the real images comparison in 3D view and 3D slicing view
4. Conclusion In this study, the GAN models are created based on the improved the sub-sampling approach to reconstruct the 3D realizations of relative homogeneous Berea sandstone and relative heterogeneous Estaillades carbonate, they show promising capabilities to represent rock samples’ physical properties in 3-dimensional space. The network shows good capabilities of reconstructing homogeneous rock sample and still need to be improved for obtaining good representations of heterogeneous porous media. Data augmentation pre-processing speeds up the convergence of the training process and also shows advantages of capturing some level of heterogeneities in 3D images. Future work will include applying GANs for multi-properties characterizations in porous media, and improvement of the network configurations and training stabilities in order to enhance the generalization capability to deal with more complex porous media problems. References [1] [2] [3] [4] [5] [6] [7] [8] [9] [10] [11] [12] [13]
Martin J. Blunt, Multiphase Flow in Permeable Media A Pore-Scale Perspective. London: Cambridge University Press, 2017. A. Q. Raeini, B. Bijeljic, and M. J. Blunt, “Generalized network modeling : Network extraction as a coarse-scale discretization of the void space of porous media,” vol. 013312, pp. 1–17, 2017. G. Mariethoz and J. Caers, Multiple-point Geostatistics: Stochastic Modeling with Training Images. 2014. S. Strebelle, “Conditional Simulation of Complex Geological Structures Using Multiple-Point Statistics 1,” vol. 34. 1, pp. 1–21, 2002. I. Goodfellow et al., “Generative Adversarial Nets,” Adv. Neural Inf. Process. Syst. 27, pp. 2672–2680, 2014. A. Radford, L. Metz, and S. Chintala, “Unsupervised Representation Learning with Deep Convolutional Generative Adversarial Networks,” pp. 1–16, 2015. M. Arjovsky, S. Chintala, and L. Bottou, “Wasserstein GAN,” 2017. I. Gulrajani, “Improved Training of Wasserstein GANs.” J. Wu, C. Zhang, T. Xue, W. T. Freeman, and J. B. Tenenbaum, “Learning a Probabilistic Latent Space of Object Shapes via 3D Generative-Adversarial Modeling,” no. Nips, 2016. L. Mosser, O. Dubrule, and M. J. Blunt, “Reconstruction of three-dimensional porous media using generative adversarial neural networks,” vol. 043309, no. April, 2017. H. Dong and M. J. Blunt, “Pore-network extraction from micro-computerized-tomography images,” no. September, pp. 1–11, 2009. “Micro-CT images of Estaillades carbonate rock,” 2015. [Online]. Available: http://www.imperial.ac.uk/earthscience/research/research-groups/perm/research/pore-scale-modelling/micro-ct-images-and-networks/. J. Wang and L. Perez, “The Effectiveness of Data Augmentation in Image Classification using Deep Learning.”