- Email: [email protected]
Efficient numerical modeling of 3D-printed lattice-cell structures using neural networks
Accepted Manuscript Letters Efficient numerical modeling of 3D-printed lattice-cell structures using neural networks Arnd Koeppe, Carlos Alberto Hernandez Padilla, Maximilian Voshage, Johannes Henrich Schleifenbaum, Bernd Markert PII: DOI: Reference:
S2213-8463(18)30005-1 https://doi.org/10.1016/j.mfglet.2018.01.002 MFGLET 125
To appear in:
Manufacturing Letters
Received Date: Revised Date: Accepted Date:
22 September 2017 9 January 2018 10 January 2018
Please cite this article as: A. Koeppe, C.A.H. Padilla, M. Voshage, J.H. Schleifenbaum, B. Markert, Efficient numerical modeling of 3D-printed lattice-cell structures using neural networks, Manufacturing Letters (2018), doi: https://doi.org/10.1016/j.mfglet.2018.01.002
This is a PDF file of an unedited manuscript that has been accepted for publication. As a service to our customers we are providing this early version of the manuscript. The manuscript will undergo copyediting, typesetting, and review of the resulting proof before it is published in its final form. Please note that during the production process errors may be discovered which could affect the content, and all legal disclaimers that apply to the journal pertain.
Efficient numerical modeling of 3Dprinted lattice-cell structures using neural networks Arnd Koeppe1,*, Carlos Alberto Hernandez 2,3 Johannes Henrich Schleifenbaum , Bernd Markert1 1 2 3
Padilla1,
Maximilian
Voshage2,
Institute of General Mechanics (IAM), RWTH Aachen University, Templergraben 64, 52064 Aachen, Germany Digital Additive Production (DAP), RWTH Aachen University, Steinbachstraße 15, 52074 Aachen, Germany Fraunhofer Institute for Laser Technology (ILT), Steinbachstraße 15, 52074 Aachen, Germany
Additively manufactured structures can be tailor-made to optimally distribute mechanical loads while remaining light-weight. To efficiently analyze the locally unique mechanical behavior of structures made from a large number of small lattice cells, a strategy which employs neural networks and deep learning to predict the maximum stresses in the realm of linear elasto-plasticity of a detail-level finite-element model is presented. The strategy is demonstrated on a single lattice cell specimen. Good agreements between experimental, finite element and neural network results are found at a significant reduction in computation time. Keywords:
structural mechanics; neural networks; additive manufacturing; multiscale modeling
Introduction Additive manufacturing enables the production of light-weight components from micro- and nanoscale structures, such as lattice cells, tailor-made to optimally support the structures loading conditions. To find the optimal geometry at every point of the component, finite element simulations at detail level imply a large number of degrees of freedom to accurately resolve each lattice cell, which severely increases computation time. For components made from materials with a regular micro- and nanostructure, multi-scale approaches using homogenization can reduce the degrees of freedom and computation time using a reference volume element over a smeared domain. Due to the unique lattice cell at every point of the component, a large number of unique reference volume elements would be necessary, which counteracts the performance gains. The long-term objective of this research is a novel multi-scale strategy able to parameterize each unique lattice cell, independent of its geometrical sizes, by using (artificial) neural networks (NN). This strategy will enable efficient simulations and topology optimization of large 3D-printed structures with locally unique microstructure. In the scope of this work, we propose a strategy employing a neural network to learn a parameterized mechanical model of a reference lattice-cell structure with a linear elasto-plastic
material behavior from validated simulation data. In future works, these neural networks will be used in the simulation of unique micro- and nanoscale structures as part of larger components.
Theoretical foundation and previous work The additive manufacturing (AM) technology enables the production of nearly unlimited complex geometries in various materials at highest precision without the need of part-specific tooling or preproduction costs [1]. One approach to exploiting the fundamental advantages of AM is using periodic lattice structures, characterized by an excellent ratio of stiffness to weight and by high mechanical energy absorption [2]. Therefore, lattice structures have a huge impact on lightweight construction and increasing the component functionality. The mechanical properties can be obtained by the selection of the lattice type, variation of the cell width and the strut diameter [3]. Artificial neural networks are directed computation graphs that map inputs to outputs. They consist of units (often called neurons or nodes), which compute the weighted sum of their inputs and apply a usually non-linear activation function (e.g. a sigmoid, hyperbolic tangent or rectified linear function). These units are arranged in layers or cells and connected by mathematical operations. Assembled as neural networks, they are universal function approximators that can be used for regression or classification [4]. In supervised learning of neural networks, the gradients of a loss function (e.g. the mean squared error) are used to iteratively train the strengths of these connections for the output to predict specific targets. In [6] Koeppe et al. use a fully-connected (feed-forward) neural network (FCNN, Figure 1a) to predict the deformation of a linear-elastic beam. For time-variant problems such as plasticity or settling processes, neural network architectures with recursions are better suited to memorize the deformation history of the structure. In [7] a recurrent neural network with fuzzy logic is used as surrogate model for the real-time prediction of settlement in soil during mechanized tunneling. The present work uses Long Short-Term Memories (LSTMs, Figure 1b), which are faster to train and can memorize data over longer time sequences [8, 9], to model 3D-printed lattice-cell structures.
The proposed strategy Our proposed strategy, illustrated in Figure 2, combines experiments, finite element (FE) simulations and neural-network based deep learning. Firstly, lattice-cell specimens are manufactured and tested under controlled loading conditions. The experimental results validate a parameterized FE model with a potentially large number of elements. Secondly, this simplified linear elastic plastic FE model calculates the stresses in the structure during deformation with different design parameters. Finally, these deformations and design parameters are used to train a neural network to efficiently predict the stresses, significantly faster than a full finite element simulation. The selected reference geometry is a single 5-by-5mm cubic area-centered lattice cell with a strut diameter of 700µm. To ensure manufacturability and to minimize the influence of clamping during testing, additional cells with gradually thickening struts lead to a 6-by-6mm full-material clamping area (see Figure 2a). All six lattice-cell specimens are manufactured from polylactic acid (PLA) using an Ultimaker3 fused deposition modeling 3D printer at 200°C printing temperature and 70mm/s printing speed with two 0.4 AA nozzles and water-soluble support material. The mechanical tensile tests are
performed on a hydraulic tensile test bench with an experimental loading speed of 0.01mm/s. The material properties are evaluated based on DIN EN ISO 527-2 with five specimens of shape 1B, using the same printing configuration and testing as for the lattice specimen. The reference continuum model shown in Figure 2b is created using the commercial software ABAQUS 2017. By employing symmetric boundary conditions, only one eighth of the reference geometry is evaluated during the analysis. The model consists of about 75000 hexahedral and tetrahedral elements with quadratic formulation (C3D20R, C3D10). It employs linear elastic-plastic material behavior with Mises yield surface, isotropic hardening, and material properties evaluated from the tensile test specimens. Loading and boundary conditions are chosen in accordance with the experiments. In the scope of the parameter study, the strut radius is varied between 0.2 and 0.5mm and the total cell scale from 50% to 150% of the reference geometry. Each combination of design parameters is generated randomly uniform from the aforementioned parametric domains, resulting in 85 individual samples. The Simulations are run on 8 to 16 cores of Intel Xeon E5-2640 CPU at 2.5GHz, generating about 400GB of ABAQUS ODB data which is condensed into 108MB of neural network training data. This dataset for the neural network of 85 simulation samples split 70-15-15% into training, validation and testing datasets. The training dataset is used to optimize the weights during training, while the validation dataset is used to check the performance on new data during training and thus adjust the training hyperparameters. The test set is not used during training and only evaluated after the neural network model showed satisfactory performance on both training and validation set. The global loads and displacements as well as the design parameters are selected as inputs. The neural network is trained to predict the maximum Von Mises and equivalent principal stresses in all the struts and joints of the lattice cell. The neural network model is implemented in tensorflow [10] and consists of a fully-connected layer with 1024 rectified linear units, 2 LSTM-cells with 1024 units respectively and a fully-connected linear output layer. It is trained on 4 NVIDIA Tesla K80 GPUs for 30000 iterations with increasing batch size and on chunks of increased sequence length. In Figure 3, the training and validation loss per training iteration and used hyper parameters are given.
Results and discussion The results are presented in Figure 4. We observe an acceptable agreement between the loaddisplacement curves (Figure 4a) of the experiments and the FEM simulation models. Figure 4b compares the maximum Von Mises stresses in the struts of the FEM simulation and the neural network for a randomly chosen test sample. The stresses of the neural network (dashed curves) are in satisfactory agreement with the finite element results (solid curves). The final mean absolute error of the neural network on the entire test dataset is 0.0832, which demonstrates that the neural network learned to generalize the full-scale FEM solution beyond the training dataset. The computation time for each of the 85 FEM simulations is in the order of about 5 to 10h (wall clock time). The complete training schedule of the neural network requires about 22h. After training the neural network, inference of the Von Mises and principal stresses from given load-conditions takes about 0.47s. This significant increase in performance is limited by the availability of a reasonably-sized dataset during training. When such data is available, for example harvested from existing databases of institutions or open source projects, the full performance gain can be exploited.
Conclusion and outlook In this work, we presented a proof of concept study, outlining a multi-scale strategy which employs neural networks and deep learning to quickly and efficiently simulate parameterized mechanical models of arbitrary micro- and nanoscale structures. For the example of a single lattice-cell, a significant gain in evaluation speed from several hours down to milliseconds and good agreement between both results were achieved. This high performance gain comes at the cost of an offline training schedule, which can be pushed below the computation time for a single FE model in a parameter study in the future. In addition, refined FE models will increase the pool of available data by parameter sensitivity studies of the individual cells with varying design parameters (e.g. cell type, individual strut diameter and strut orientation) under different loading conditions and taking different material effects (e.g. fracture) into account. Furthermore, neural-network models trained to simulate the mechanical behavior of these lattice cells will be assembled into larger structures and integrated into existing FEM frameworks. Finally, based on these highly-efficient, parameterized models, a topology optimization strategy for larger components made from small lattice cell structures will be developed.
References W. Meiners and R. Poprawe, “Direktes selektives Laser-Sintern einkomponentiger metallischer Werkstoffe” Zugl.: Aachen, Techn. Hochsch., Diss., 1999, 1999. [2] O. Rehme, “Cellular design for laser freeform fabrication,” Cuvillier Göttingen, 2010. [3] T. T. Wohlers and T. Caffrey, “Wohlers report 2014: 3D printing and additive manufacturing state of the industry annual worldwide progress report,” Fort Collins, Col.: Wohlers Associates, 2014. [4] K. Hornik, M. Stinchcombe, and H. White, “Multilayer feedforward networks are universal approximators,” Neural Netw., vol. 2, no. 5, pp. 359–366, 1989. [5] Y. LeCun, Y. Bengio, and G. Hinton, “Deep learning,” Nature, vol. 521, no. 7553, pp. 436–444, Mai 2015. [6] A. Koeppe, F. Bamer, and B. Markert, “Model reduction and submodelling using neural networks,” PAMM, vol. 16, no. 1, pp. 537–538, Oktober 2016. [7] B.-T. Cao, S. Freitag, and G. Meschke, “A hybrid RNN-GPOD surrogate model for real-time settlement predictions in mechanised tunnelling,” Adv. Model. Simul. Eng. Sci., vol. 3, p. 5, Mar. 2016. [8] S. Hochreiter and J. Schmidhuber, “Long Short-Term Memory,” Neural Comput., vol. 9, no. 8, pp. 1735–1780, Nov. 1997. [9] K. Greff, R. K. Srivastava, J. Koutník, B. R. Steunebrink, and J. Schmidhuber, “LSTM: A Search Space Odyssey,” ArXiv150304069 Cs, Mar. 2015. [10] M. Abadi et al., TensorFlow: “Large-Scale Machine Learning on Heterogeneous Systems,” 2015. [1]
a) Fully-Connected layers (FC)
b) Long Short-Term Memory cell (LSTM)
Figure 1: Two neural network architectures.
a) Specimen
b) FEM
c) NN
Figure 2: From specimen to FEM model to NN model.
Hyperparameter number of iterations
Values per training phases 10000
10000
10000
batch size
200
500
1000
base learning rate
3-4
1e-4
3e-5
learning-rate decay
exponentially by factor of 0.9 every 1000 steps
gradient clipping
3.0
dropout (keep probability)
0.7
Figure 3: Training and validation loss (mean square error) per iteration and the used hyperparameters.
a) Load-displacement curves (solid: FEM, dotted: experiment, filled: experimental standard deviation/min-max interval)
b) Maximum stresses over displacement (solid: FEM, dashed: NN)
Figure 4: Comparison of the experimental, simulation and neural network results.
S2213-8463(18)30005-1 https://doi.org/10.1016/j.mfglet.2018.01.002 MFGLET 125
To appear in:
Manufacturing Letters
Received Date: Revised Date: Accepted Date:
22 September 2017 9 January 2018 10 January 2018
Please cite this article as: A. Koeppe, C.A.H. Padilla, M. Voshage, J.H. Schleifenbaum, B. Markert, Efficient numerical modeling of 3D-printed lattice-cell structures using neural networks, Manufacturing Letters (2018), doi: https://doi.org/10.1016/j.mfglet.2018.01.002
This is a PDF file of an unedited manuscript that has been accepted for publication. As a service to our customers we are providing this early version of the manuscript. The manuscript will undergo copyediting, typesetting, and review of the resulting proof before it is published in its final form. Please note that during the production process errors may be discovered which could affect the content, and all legal disclaimers that apply to the journal pertain.
Efficient numerical modeling of 3Dprinted lattice-cell structures using neural networks Arnd Koeppe1,*, Carlos Alberto Hernandez 2,3 Johannes Henrich Schleifenbaum , Bernd Markert1 1 2 3
Padilla1,
Maximilian
Voshage2,
Institute of General Mechanics (IAM), RWTH Aachen University, Templergraben 64, 52064 Aachen, Germany Digital Additive Production (DAP), RWTH Aachen University, Steinbachstraße 15, 52074 Aachen, Germany Fraunhofer Institute for Laser Technology (ILT), Steinbachstraße 15, 52074 Aachen, Germany
Additively manufactured structures can be tailor-made to optimally distribute mechanical loads while remaining light-weight. To efficiently analyze the locally unique mechanical behavior of structures made from a large number of small lattice cells, a strategy which employs neural networks and deep learning to predict the maximum stresses in the realm of linear elasto-plasticity of a detail-level finite-element model is presented. The strategy is demonstrated on a single lattice cell specimen. Good agreements between experimental, finite element and neural network results are found at a significant reduction in computation time. Keywords:
structural mechanics; neural networks; additive manufacturing; multiscale modeling
Introduction Additive manufacturing enables the production of light-weight components from micro- and nanoscale structures, such as lattice cells, tailor-made to optimally support the structures loading conditions. To find the optimal geometry at every point of the component, finite element simulations at detail level imply a large number of degrees of freedom to accurately resolve each lattice cell, which severely increases computation time. For components made from materials with a regular micro- and nanostructure, multi-scale approaches using homogenization can reduce the degrees of freedom and computation time using a reference volume element over a smeared domain. Due to the unique lattice cell at every point of the component, a large number of unique reference volume elements would be necessary, which counteracts the performance gains. The long-term objective of this research is a novel multi-scale strategy able to parameterize each unique lattice cell, independent of its geometrical sizes, by using (artificial) neural networks (NN). This strategy will enable efficient simulations and topology optimization of large 3D-printed structures with locally unique microstructure. In the scope of this work, we propose a strategy employing a neural network to learn a parameterized mechanical model of a reference lattice-cell structure with a linear elasto-plastic
material behavior from validated simulation data. In future works, these neural networks will be used in the simulation of unique micro- and nanoscale structures as part of larger components.
Theoretical foundation and previous work The additive manufacturing (AM) technology enables the production of nearly unlimited complex geometries in various materials at highest precision without the need of part-specific tooling or preproduction costs [1]. One approach to exploiting the fundamental advantages of AM is using periodic lattice structures, characterized by an excellent ratio of stiffness to weight and by high mechanical energy absorption [2]. Therefore, lattice structures have a huge impact on lightweight construction and increasing the component functionality. The mechanical properties can be obtained by the selection of the lattice type, variation of the cell width and the strut diameter [3]. Artificial neural networks are directed computation graphs that map inputs to outputs. They consist of units (often called neurons or nodes), which compute the weighted sum of their inputs and apply a usually non-linear activation function (e.g. a sigmoid, hyperbolic tangent or rectified linear function). These units are arranged in layers or cells and connected by mathematical operations. Assembled as neural networks, they are universal function approximators that can be used for regression or classification [4]. In supervised learning of neural networks, the gradients of a loss function (e.g. the mean squared error) are used to iteratively train the strengths of these connections for the output to predict specific targets. In [6] Koeppe et al. use a fully-connected (feed-forward) neural network (FCNN, Figure 1a) to predict the deformation of a linear-elastic beam. For time-variant problems such as plasticity or settling processes, neural network architectures with recursions are better suited to memorize the deformation history of the structure. In [7] a recurrent neural network with fuzzy logic is used as surrogate model for the real-time prediction of settlement in soil during mechanized tunneling. The present work uses Long Short-Term Memories (LSTMs, Figure 1b), which are faster to train and can memorize data over longer time sequences [8, 9], to model 3D-printed lattice-cell structures.
The proposed strategy Our proposed strategy, illustrated in Figure 2, combines experiments, finite element (FE) simulations and neural-network based deep learning. Firstly, lattice-cell specimens are manufactured and tested under controlled loading conditions. The experimental results validate a parameterized FE model with a potentially large number of elements. Secondly, this simplified linear elastic plastic FE model calculates the stresses in the structure during deformation with different design parameters. Finally, these deformations and design parameters are used to train a neural network to efficiently predict the stresses, significantly faster than a full finite element simulation. The selected reference geometry is a single 5-by-5mm cubic area-centered lattice cell with a strut diameter of 700µm. To ensure manufacturability and to minimize the influence of clamping during testing, additional cells with gradually thickening struts lead to a 6-by-6mm full-material clamping area (see Figure 2a). All six lattice-cell specimens are manufactured from polylactic acid (PLA) using an Ultimaker3 fused deposition modeling 3D printer at 200°C printing temperature and 70mm/s printing speed with two 0.4 AA nozzles and water-soluble support material. The mechanical tensile tests are
performed on a hydraulic tensile test bench with an experimental loading speed of 0.01mm/s. The material properties are evaluated based on DIN EN ISO 527-2 with five specimens of shape 1B, using the same printing configuration and testing as for the lattice specimen. The reference continuum model shown in Figure 2b is created using the commercial software ABAQUS 2017. By employing symmetric boundary conditions, only one eighth of the reference geometry is evaluated during the analysis. The model consists of about 75000 hexahedral and tetrahedral elements with quadratic formulation (C3D20R, C3D10). It employs linear elastic-plastic material behavior with Mises yield surface, isotropic hardening, and material properties evaluated from the tensile test specimens. Loading and boundary conditions are chosen in accordance with the experiments. In the scope of the parameter study, the strut radius is varied between 0.2 and 0.5mm and the total cell scale from 50% to 150% of the reference geometry. Each combination of design parameters is generated randomly uniform from the aforementioned parametric domains, resulting in 85 individual samples. The Simulations are run on 8 to 16 cores of Intel Xeon E5-2640 CPU at 2.5GHz, generating about 400GB of ABAQUS ODB data which is condensed into 108MB of neural network training data. This dataset for the neural network of 85 simulation samples split 70-15-15% into training, validation and testing datasets. The training dataset is used to optimize the weights during training, while the validation dataset is used to check the performance on new data during training and thus adjust the training hyperparameters. The test set is not used during training and only evaluated after the neural network model showed satisfactory performance on both training and validation set. The global loads and displacements as well as the design parameters are selected as inputs. The neural network is trained to predict the maximum Von Mises and equivalent principal stresses in all the struts and joints of the lattice cell. The neural network model is implemented in tensorflow [10] and consists of a fully-connected layer with 1024 rectified linear units, 2 LSTM-cells with 1024 units respectively and a fully-connected linear output layer. It is trained on 4 NVIDIA Tesla K80 GPUs for 30000 iterations with increasing batch size and on chunks of increased sequence length. In Figure 3, the training and validation loss per training iteration and used hyper parameters are given.
Results and discussion The results are presented in Figure 4. We observe an acceptable agreement between the loaddisplacement curves (Figure 4a) of the experiments and the FEM simulation models. Figure 4b compares the maximum Von Mises stresses in the struts of the FEM simulation and the neural network for a randomly chosen test sample. The stresses of the neural network (dashed curves) are in satisfactory agreement with the finite element results (solid curves). The final mean absolute error of the neural network on the entire test dataset is 0.0832, which demonstrates that the neural network learned to generalize the full-scale FEM solution beyond the training dataset. The computation time for each of the 85 FEM simulations is in the order of about 5 to 10h (wall clock time). The complete training schedule of the neural network requires about 22h. After training the neural network, inference of the Von Mises and principal stresses from given load-conditions takes about 0.47s. This significant increase in performance is limited by the availability of a reasonably-sized dataset during training. When such data is available, for example harvested from existing databases of institutions or open source projects, the full performance gain can be exploited.
Conclusion and outlook In this work, we presented a proof of concept study, outlining a multi-scale strategy which employs neural networks and deep learning to quickly and efficiently simulate parameterized mechanical models of arbitrary micro- and nanoscale structures. For the example of a single lattice-cell, a significant gain in evaluation speed from several hours down to milliseconds and good agreement between both results were achieved. This high performance gain comes at the cost of an offline training schedule, which can be pushed below the computation time for a single FE model in a parameter study in the future. In addition, refined FE models will increase the pool of available data by parameter sensitivity studies of the individual cells with varying design parameters (e.g. cell type, individual strut diameter and strut orientation) under different loading conditions and taking different material effects (e.g. fracture) into account. Furthermore, neural-network models trained to simulate the mechanical behavior of these lattice cells will be assembled into larger structures and integrated into existing FEM frameworks. Finally, based on these highly-efficient, parameterized models, a topology optimization strategy for larger components made from small lattice cell structures will be developed.
References W. Meiners and R. Poprawe, “Direktes selektives Laser-Sintern einkomponentiger metallischer Werkstoffe” Zugl.: Aachen, Techn. Hochsch., Diss., 1999, 1999. [2] O. Rehme, “Cellular design for laser freeform fabrication,” Cuvillier Göttingen, 2010. [3] T. T. Wohlers and T. Caffrey, “Wohlers report 2014: 3D printing and additive manufacturing state of the industry annual worldwide progress report,” Fort Collins, Col.: Wohlers Associates, 2014. [4] K. Hornik, M. Stinchcombe, and H. White, “Multilayer feedforward networks are universal approximators,” Neural Netw., vol. 2, no. 5, pp. 359–366, 1989. [5] Y. LeCun, Y. Bengio, and G. Hinton, “Deep learning,” Nature, vol. 521, no. 7553, pp. 436–444, Mai 2015. [6] A. Koeppe, F. Bamer, and B. Markert, “Model reduction and submodelling using neural networks,” PAMM, vol. 16, no. 1, pp. 537–538, Oktober 2016. [7] B.-T. Cao, S. Freitag, and G. Meschke, “A hybrid RNN-GPOD surrogate model for real-time settlement predictions in mechanised tunnelling,” Adv. Model. Simul. Eng. Sci., vol. 3, p. 5, Mar. 2016. [8] S. Hochreiter and J. Schmidhuber, “Long Short-Term Memory,” Neural Comput., vol. 9, no. 8, pp. 1735–1780, Nov. 1997. [9] K. Greff, R. K. Srivastava, J. Koutník, B. R. Steunebrink, and J. Schmidhuber, “LSTM: A Search Space Odyssey,” ArXiv150304069 Cs, Mar. 2015. [10] M. Abadi et al., TensorFlow: “Large-Scale Machine Learning on Heterogeneous Systems,” 2015. [1]
a) Fully-Connected layers (FC)
b) Long Short-Term Memory cell (LSTM)
Figure 1: Two neural network architectures.
a) Specimen
b) FEM
c) NN
Figure 2: From specimen to FEM model to NN model.
Hyperparameter number of iterations
Values per training phases 10000
10000
10000
batch size
200
500
1000
base learning rate
3-4
1e-4
3e-5
learning-rate decay
exponentially by factor of 0.9 every 1000 steps
gradient clipping
3.0
dropout (keep probability)
0.7
Figure 3: Training and validation loss (mean square error) per iteration and the used hyperparameters.
a) Load-displacement curves (solid: FEM, dotted: experiment, filled: experimental standard deviation/min-max interval)
b) Maximum stresses over displacement (solid: FEM, dashed: NN)
Figure 4: Comparison of the experimental, simulation and neural network results.