An ontology for numerical design of experiments processes

Computers in Industry 94 (2018) 26–40

Contents lists available at ScienceDirect

Computers in Industry journal homepage: www.elsevier.com/locate/compind

An ontology for numerical design of experiments processes Gaëtan Blondet, Julien Le Duigou* , Nassim Boudaoud, Benoît Eynard Sorbonne Universités, Université de Technologie de Compiègne, Mechanical laboratory Roberval UMR UTC/CNRS 7337, France

A R T I C L E I N F O

Article history: Received 22 February 2017 Received in revised form 10 September 2017 Accepted 19 September 2017 Available online xxx Keywords: Ontology Design of experiments Simulation data management

A B S T R A C T

Numerical Designs of Experiments (NDoE) are used in a product development process to optimize the product. A NDoE may combine a costly numerical model and numerous experiments. The NDoE process consequently becomes very expensive. However, some methods and algorithms were developed to shorten the NDoE process, as sensitivity analysis, surrogate modelling and adaptive DoE. Because of their complexity, advanced expert knowledge or a long preparation step is required to optimally choose and configure all of these methods, in order to run the most efficient NDoE process. To answer this issue, a knowledge management approach is proposed in this paper. It capitalizes and reuses knowledge about NDoE process. This solution is proposed because of the lack in term of models and standardized processes for this specific NDoE application. An ontology was developed to manage, share and reuse knowledge and enable queries for information retrieval in a database. The database lists every NDoE processes executed. Then, the knowledge is analysed by a decision-support system to help designers to choose the best configuration. © 2017 Elsevier B.V. All rights reserved.

1. Introduction Numerical simulations are used in a Product Development Process (PDP) to continuously optimize products to improve quality to reduce the cost and the time to market. Simulation activities lead to fulfil customer's requirements without any physical prototype and with a minimal amount of re-design loops. Numerical simulations are more and more numerous and complex, as decision processes are more and more based on numerical models. A simulation data management strategy must be set to control the simulation process and to make it as profitable as possible [1,2]. These simulations may need and produce a large amount of data. Numerical models may cover multiple physics on multiple parts of the product. Simulation data must be shared across extended enterprises to enhance collaborative engineering activities between different teams. A Numerical Design of Experiments (NDoE) process is defined by an ordered sequence of simulations, based on a parameterized numerical model. Each simulation/experiment is defined by a specific set of values of model’s parameters. NDoE can be used to model uncertainties or to optimize the product. NDoE multiply the simulation process cost and the amount of data by the number of

* Corresponding author. E-mail addresses: [email protected] (G. Blondet), [email protected] (J.L. Duigou), [email protected] (N. Boudaoud), [email protected] (B. Eynard). http://dx.doi.org/10.1016/j.compind.2017.09.005 0166-3615/© 2017 Elsevier B.V. All rights reserved.

experiments. NDoE processes may increase drastically the need to manage data and to shorten the simulation process. The aim of NDoE process is to consider the properties, the environment and requirements of the product as variables. NDoE process fulfils specific objectives, such as design optimization [3], surrogate modelling [4] or sensitivity analysis [5]. For each objective, specific methods post-process outputs of each experiment. Some of these methods can shorten the NDoE process, by minimizing the number of required experiments, as:  sensitivity analysis methods identify most influential parameters of the numerical model, according to the studied output. This dimensionality reduction decreases the number of required experiments.  surrogate modelling methods replace the initial numerical model by a cheaper function (e.g. polynomial functions). Once a first NDoE is executed with the initial numerical model, a surrogate model is computed and can be used for further studies. This method reduces the computational cost of future experiments.  Adaptive sampling methods [6] are applied to obtain an optimal NDoE. An optimization algorithm, as gradient-based algorithms and metaheuristics, can be used so that each experiment gives more relevant results. Thus, the number of required experiments can be minimized.

G. Blondet et al. / Computers in Industry 94 (2018) 26–40

While these methods shorten the execution step, they require advanced knowledge to be configured, especially if they are combined and applied to complex products. For instance, an adaptive NDoE may be used for surrogate modelling with a multiphysic simulation model with 50 parameters, which takes one day to be executed. Designers must choose wisely the type of initial NDoE, the number of initial experiments, the metaheuristics to search new useful experiments, the type of surrogate model, the criterion to select the best new experiments, etc. A wrong configuration may lead to a significant waste of time and computational resources. The configuration of the NDoE process is as critical and difficult as the product is complex and simulations are expensive. The time and the knowledge required to define the NDoE process is increased. As discussed in [7], a solution consists in efficiently managing data used in each NDoE process. Regarding a SDM strategy, data are capitalised and reused to shorten the configuration step of the simulation process. This work was done in the French FUI 16 project SDM4DOE (“Simulation Data Management for Design of Experiments”). This project aims to propose an open-source framework to define, run and manage NDoE processes during the PDP. A SDM tool supports NDoE processes to capitalize, trace and reuse data, in order to help designers for further studies. First applications of this framework are being done in automotive industries (Valeo1 ) and for civil engineering applications (NECS2 ). In this paper, an ontological approach is proposed to structure, share and reuse knowledge required to set complex and efficient NDoE processes fast. The paper is structured as follow: Section 2 presents a literature review of product development ontologies. Section 3 describes the proposal: a specific ontology for the NDoE process. This proposal is illustrated by a use-case, in Section 4, with a specific application in mechanical design, extracted from the project SDM4DOE. A discussion on the use of this proposed ontology in enterprises, to manage and reuse data and knowledge linked to NDOE processes, concludes this paper. 2. Ontologies in product development process An ontology can be defined as an “explicit specification of a conceptualization” [8,9]. In other words, ontologies describe, by a shared vocabulary:  semantic representations of concepts of a specific domain.  relationships between these concepts. An ontology must be easily understandable and editable [10]. It should support links with other ontologies to enrich and extend the semantic domain. Many ontologies support a PDP subjected to collaborative constraints, heterogeneity, interoperability, temporal changes and knowledge management issues in every step of the product lifecycle. Since NDoE process may involve specific concepts used by different teams and departments, a specific ontology is appropriate to give a clear and sharable semantic description of this domain. The literature review shown in this section covers (1) ontologies proposed to support the PDP and (2) ontologies which support more specific concepts involved in the NDoE process. The aim of this literature review is to identify already described concepts and missing concepts.

1 2

http://www.valeo.com/en/who-we-are/ http://necs.fr/

27

2.1. Literature review on product ontologies Many applications of ontologies have been developed to support the PDP. Their common goal is to improve system interoperability and collaborative processes. They cover different stages of the product lifecycle, with different level of details. All of these developments are analysed to identify concepts, involved in the semantic domain of NDoE processes, which are already described or not. This study will provide elements to define a specific ontology for NDoE processes, enriched by existing developments. The semantic domain of NDoE process, covers, for instance, methods for surrogate modelling [11], methods for sensitivity analysis [12], types of NDoE [13], the concept of experiment, optimization algorithms. This semantic domain is also connected to other concepts such as numerical model, CAD model and product data. The ontology PRONOIA2 manage product data from the design and manufacturing process, focusing on the Beginning of Life (BOL) of the product lifecycle [14]. This ontology is composed of three main levels of description, from the most general description (Meta-ontology) to a specific description (application ontology). PRONOIA2 adds a description of product evolutions through the BOL. Evolutions are represented by spatiotemporal concepts, as transformations (ex: welding, riveting, clinching, etc.), changes (ex: addition, deletion, permutation, etc.) or movements. These concepts are linked to the physical description of the product, which describes geometry and assemblies. It seems unable to manage data from the numerical simulation process, according to the current description. But the meta-ontology is general so that new specialized sub-classes could be added. For instance, a new subclass of the class “spatial_region” could be added, describing the numerical description of the product, in the same way as the class “Physical-product”. An ontology was proposed to enrich exchanges of product data, commonly covered by standards, such as STEP [15]. This development focused on the management of geometric data, as variational geometric constraint data. The variational aspect is described as a variation of spatial parameters, linking a referenced feature and a constrained feature in the CAD model. Such a semantic description can be linked to the semantic domain of NDoE process by concepts as parameter and geometric model. PROMISE Semantic Object Model [16] was translated in OWL (Web Ontology Language) and improved to create an ontology. This ontological model supports the concepts used for product design, and for various processes and industrial sectors. The translation was made to enrich the data model to be used for closed-loop Product Lifecycle Management. The classes “physical_product” and “As_designed_product” embed product identification data and definition data, as CAD model, tests and specifications. These concepts are used in the semantic domain of the NDoE process. ONTO-PDM is a product ontology managing heterogeneous data, as material used and related properties, relationships between components and products, versions, manufacturing equipment, etc. [17]. Two standards, ISO 10303 and IEC 62264, were used to develop this ontology. ISO 10303 gives a sharable description of product information (geometry, identification data, etc.). IEC 62264 links the ontology with the Manufacturing Execution System (MES) and the Enterprise Resource Planning (ERP). It seems that Onto-PDM does not cover any analysis activity, such as numerical simulation, but it is still based on ISO 10303, which specify data models for numerical simulation model. Some ontologies have been developed for the manufacturing context. For instance, OntoSTEP-NC [18] was proposed to improve interoperability between CAD, CAM and MOCN tools. OntoSTEP-NC is based on the STEP-NC standard, which is built on ISO10303AP238 and on ISO10649. These two standards can manage

28

G. Blondet et al. / Computers in Industry 94 (2018) 26–40

manufacturing data and strategies. InVor is an ontology supporting manufacturing process, by searching quickly for an alternative machine in case of machine failure [19]. PARO ontology is based on a generic Product/Activities/ Resource/Organization data model used for mechanical product design and production engineering [20]. PARO embeds classes such as “Resources” (human, material and methodological), “Activity”, “Product” and “Organization”. PARO supports complex product definitions. It also supports extended enterprises and each product lifecycle step, including the design process, distribution and maintenance processes, and end-of-life processes. These concepts can manage generic data needed to ensure traceability during the PDP, and also for each kind of process, including the NDoE process. OntoStep is based on two data models developed by NIST: the Core Product Model (CPM) and the Open Assembly Model (OAM) [21,22]. These two models were combined and enriched with semantic and reasoning capabilities. CPM is a generic product data model designed to support PLM. Each artefact of the product is composed of features. They are defined by a function, a form and a behaviour. OAM describes more specifically assembly data. OntoSTEP covers geometric data and data related to function, design rationale or behaviour. This ontology does not cover explicitly data related to any kind of analysis, more specific than design rationales. Concerning numerical simulation, processed during the PDP, [23] have proposed an ontology which gives a generic knowledge representation about analysis models and simulations. This ontology supports all kinds of physics, hardware/software requirements, justifications of each idealization made. It was detailed for finite element models and discrete models. This ontology matches with the description needed to define initial numerical models used in the NDoE process. An ontology was proposed by [24] to support the design process. It links simulation data, requirements data, and geometric data. It is also based on the same product description than Core Product Model (entity, artefacts, behaviour, etc.). It focuses also on the description of stakeholders, such as customers, project partners or design team members. Many of these ontologies are based on standards to extend their use. The main standard used for PDP is ISO 10303 “STEP” [25]. This standard provides a complete, neutral and clear description of a product throughout its lifecycle. It specifies a neutral language (EXPRESS), implementation methods, and Application Protocols (AP). AP describes the representation of product data for one or more domains, such as STEP-NC (AP238) [26] for manufacturing applications, or AP242 for product design and digital representation of assemblies and parts [27]. An international standard like ISO 10303, used as a basis for ontology definition, provides a great advantage to ensure a common understanding of concepts and facilitate the integration of ontologies in real applications. The AP 209 of the ISO 10303 standard, named “Multidisciplinary analysis and design”, defines a data model to improve the exchange of product definitions [28]. It covers geometric models, numerical analysis models (finite-element models and computational-fluid dynamics models) and associated idealized geometries and meshes, simulation results, and product properties. AP209 is dedicated to product with composite and metallic materials. Assembly data are also managed. AP209 provides required data to control the product definition for each step of the design process. For instance, the module “Analysis” of the AP209 describes in details finite-element models, by taking into account:  the numerical model definition, defined by an ID, a name, a description, softwares used to create and to run the model and the analysis type.  temporal and spatial discretization.

 each run of the numerical model, defined by an ID, a name and a description.  results of each simulation.  the definition of each part and its associated behaviour. These elements, supported by AP209, are required to manage and trace NDoE process data and knowledge, since the NDoE process is based on the simulation process. Others ontologies were specified to provide decision support, by automating definition of optimal solutions from gathered data and knowledge. A first example is an application which combines text-mining methods to extract data from crowd-sourced database, and an ontology to re-structure these extracted data [29]. This leads to an automated selection of best product concepts. A second example is an ontology-based knowledge framework for automatic material selection [30]. It is composed of an ontology supporting knowledge about material properties, a reasoning layer using the Semantic Query-enhanced Web Rule Language (SQWRL) to support complex “if . . . then” rules, and a user interface. This decision support system is based on pre-defined logical rules. These examples illustrate some ways to reuse capitalized data to shorten the PDP. This first part of the literature review was done to identify general ontologies, in order to ensure a compatibility with a more specific development for NDoE processes. Several ontologies were developed to capitalize and structure the knowledge required for product development. Some of them can be reused for a specific application for NDoE processes. These ontologies do not manage specific data and knowledge required to set a NDoE process. Concepts related to statistics, sampling, probability, surrogate modelling and optimization algorithms are not covered. The next step of this literature review consists in researching and analysing more specific ontologies, which describes some elements of the NDoE process itself.

2.2. Literature review on specialized ontologies This second step covers ontologies describing very specific concepts used inside the NDoE process. The ontology EXPO [31] covers the semantic domain of scientific experiments. It describes concepts such as physical and numerical experiments, results, statistical tests, factors and target variables. Concepts to ensure traceability of experiments are also embedded, as author’s name, titles, etc. All of these concepts are required in the NDoE process. EXPO is based on a more general meta-ontology, SUMO (Suggested Upper Merged Ontology) [32], including concepts such as time, unit, process, and object. Exposé [33] describes the domain of data mining. This ontology is based on EXPO. Many concepts are defined, such as data sets (for learning and validation), parameters (both for model and algorithms), the concept of predictivity, computational resources, regression models (e.g. neural networks, gaussian processes, decision trees). Many of these concepts are necessary to describe surrogate modelling methods. It defines also workflows and processes, as experiment workflow (execution of simulations) or data processing workflow (to define the dataset). Surrogate model types as neural networks and kriging are covered by Exposé. STATO (http://stato-ontology.org/) is dedicated to biology, genomics and neurology fields. Some of classes (“generically dependent continuant” and “planned process”) contain detailed descriptions of statistical tests, measurement, probability distributions, regression models, physical DoE and variables. These descriptions are useful for sensitivity analysis and surrogate modelling.

G. Blondet et al. / Computers in Industry 94 (2018) 26–40

2.3. Overview Ontologies are applied to the PDP at different levels and for different domains. This literature review reveals a lack in term of semantic description of the NDoE domain applied for product design. No specific ontology, describing the NDoE process, was found (Fig. 1). Ontologies related to the PDP does not support NDoE. Ontologies which describes DoE are not connected to any concept of the PDP. A gap between these two types of ontology was noticed. Some of ontologies integrate concepts which can be reused to define a specific ontology for the NDoE process. For instance, some of ontologies presented in this section describe the simulation process or several methods used in the NDoE process. As a result, a specific ontology, ODE (Ontology for numerical Design of Experiments) is proposed in connection with existing product ontologies. The aim of this proposal is to enhance knowledge capitalization and reuse in extended enterprises to define an efficient NDoE process as fast as possible for new and complex products. The aim of ODE is to fill the gap between ontologies specialized in statistics and machine learning, and ontologies designed to manage data from PDP. ODE is detailed in the next section. Its aim is to support the huge amount of data generated by NDoE processes and to share and reuse the knowledge acquired during these complex processes. This ontology is designed to cover links with product analysis (simulation processes) and product design processes. 3. Presentation of the ontology for numerical design of experiments In the previous section, a lack of ontological description of the NDoE process domain was observed. The needs have been clarified

29

to manage, share and reuse available knowledge, in order to configure the whole NDoE process according to designers’ requirements. The Ontology for numerical Design of Experiments (ODE) introduced here, was developed to provide a decision-support system for designers. The rise of NDoE processes to design and optimize robust and reliable products leads to a need of data and knowledge management. NDoE processes can be powerful tools, but they are complex. NDoE processes may demand advanced knowledge to be used in a PDP. NDoE processes required also a structure to manage used and produced data. The aim of ODE is to manage and share data and knowledge, from simulation and NDoE processes, across extended enterprises. ODE was developed according to current needs of companies to manage data from simulation processes [1,2]. The NDoE process can be defined by different types of data:  Context data cover product data, available resources, numerical model data, users’ data, etc.;  Strategic data concern the objective and constraints of a NDoE process;  Technical data contain methods used to fulfil the objective, such as the type of NDoE, the type surrogate model, etc.;  Parametric data cover parameters of every method used. For instance, if a surrogate model is defined as artificial neural networks, the number of neurons must be defined. The aim of ODE is to cover all of these types to fully define the NDoE process and to ensure data traceability in a PDP. Four criteria [8] must be fulfilled to design an ontology and to ensure knowledge sharing:

Fig. 1. Overview of the literature review.

30

G. Blondet et al. / Computers in Industry 94 (2018) 26–40

Fig. 2. Global view of the proposed ontology (arrows: sub-class relationships; dotted arrows: object properties).

 Clarity of concept definitions, consistent logic between these concepts;  Extendibility, to enrich the ontology with new concepts and properties;  Minimal encoding bias to avoid bias from chosen representations of concepts;  Minimal ontological commitment, to ensure its genericity. First, this section describes the structure and main concepts of ODE. Then, aspects set to fulfil these criteria are listed. This ontology was created with the Web Ontology Language (OWL2) with the ontology editor “Protégé”,3 and with the Description Logic (OWL-DL) knowledge modelling [16]. It produces object-oriented knowledge model, extendable and able to support reasoning algorithms. 3.1. Main concepts The core of the proposed ontology is composed by 14 classes, covering concepts as NDoE process, NDoE, Surrogate models, and also Objective, Resources, Constraints and Results linked to the NDoE process (Fig. 2). The first and main concept of this ontology is “NDoE_Process”. The NDoE process is a link between product design processes and every specific method applied for NDoE. As illustrated by Fig. 2, the NDoE process concept is defined by:  exactly one objective: among five different objectives which were considered (Table 1). This concept will determine types of methods used during the NDoE process;

3

http://protege.stanford.edu/

 constraints: the process may be subjected to a time limit or a specific computational budget, and must reach results quality thresholds;  at least one NDoE: This concept is described below;  results: this concept concerns every result computed during the process (from experiments, analysis methods, etc.);  project data: the NDoE is linked to a specific project, with identified authors and stakeholders. It concerns a specific product or one of its parts. Project data covers also traceability data (date, versions, etc.). It also defines the initial numerical model used to execute numerical experiments (e.g. a finite element model);  resources: the NDoE process is supported by specific software and hardware. These data trace the origin of experimental results (as execution time), and define the available computational budget. The execution time depends on this budget. In case of multi-disciplinary and optimization computations, complex simulation chains may be used, combining different software. The second major class of ODE is NDoE. It covers technical data required to execute a NDoE process and analyse results. A NDoE is defined by:  exactly one type: The type specifies the sampling methods use to fill the design space with experiments. A lot of NDoE type exists. They are classified according to a specific hierarchy, from [13];  a number of experiment: this attribute is a positive integer greater than 0;  exactly one initial numerical model: required to compute results for each experiment, it is assumed to be valid and accurate;  at least one factor: selected from parameters of the initial numerical model, it is defined by a statistical distribution;

G. Blondet et al. / Computers in Industry 94 (2018) 26–40

31

Table 1 Method classes used according to NDoE process’s objective.

Methods

Surrogate

Sensivity

Opmizaon

models

methods

algorithms

NDoE Objecves Diagnosc



Exploraon



Surrogate modelling





Sensivity analysis







Opmizaon









 at least one output: selected from outputs of the initial numerical model;  a surrogate model: a mathematical model is used to be linked to the NDoE to analyse results from each experiment. Under some quality conditions, this model can replace the initial numerical model;  some analysis methods: as statistical tests, sensitivity analysis [12] or robustness analysis methods. A specific class has been defined to manage Adaptive NDoE processes. Adaptive NDoE is mainly used to improve the accuracy and the predictivity of the surrogate model with few experiments. In addition to the definition of a NDoE, Adaptive NDoE requires:  exactly one optimization algorithm: it searches for the experiment which could provide the best improvement for the accuracy of the surrogate model. A lot of algorithms exist, as gradient-based algorithms or metaheuristics. Each of these has their own data properties (e.g. convergence threshold);  exactly one infill criterion: it quantifies the improvement brought by added experiment to the surrogate model (ex: Expected improvement [34], maximal variance);  some quality objectives: the quality of the surrogate model is defined by statistical measurements to assess its accuracy (error made on known experiments) and its predictivity (error made on unknown experiments). These objectives set minimal quality levels demanded by designers;

Fig. 3. Specialisation of the Surrogate model type concept.



 a maximal number of added experiments: this data property is used to stop the adaptive process and to limit the computational cost. This ontology can be specialized according to methods available in a specific company, department or organization. Sub-classes are defined to detail concepts such as NDoE types, Surrogate model types or optimization algorithm types. For instance, the full definition of function generating a kriging surrogate model is modelled, according to its specific implementation (Figs. 3 and 4). 3.2. Ontology extendibility A main property of ontologies is the ability to be linked to others ontologies. Links with other ontologies are created to:  avoid useless redundancies, by describing a concept which already described in others developments;  extend the covered semantic domain;  enrich and specialized an ontology to a specific application. Based on the state of the art detailed in Section 2, ODE is connected to several ontologies, as illustrated by Figs. 5 and 6, with object properties such as “EquivalentTo” and subclasses.First, the

Fig. 4. Data properties for Kriging.

32

G. Blondet et al. / Computers in Industry 94 (2018) 26–40

Fig. 5. ODE links with existing ontologies.

Fig. 6. Main links considered between ODE and other ontologies.

class “Finite_elements_Model”, which exists in ODE, is linked to the class “ISO_10303_AP209” by the object property “IsDescribedBy”, which contains all its description. Many other concepts may be enriched by this standard, as project data, to link the NDoE process to the simulation process and the general design process and ensure traceability.

Second, the PARO ontology [20] is re-used to complete the definition of more general concepts. While some concepts defined in this ontology are covered by ISO 10303 AP 209, PARO provides some specific concept, as “Resources”, integrating human, technical and methodological resources. It also defines companies’ organization. The class of ODE “Resources” is integrated in the class

G. Blondet et al. / Computers in Industry 94 (2018) 26–40

33

Fig. 7. Results of the reasoner Pellet.

“Resources” of PARO. Other concepts enrich the “Project data” class in addition to ISO 10303 AP209. Third, the ontology EXPO enriches ODE with concepts of computational experiments, multi-factor experiments, models and statistical hypothesis tests [31]. For instance, the class of ODE “NDoE” is expanded with the subclass “Computational-Experiment” and new subclasses for “NDoE_Type”. Fourth, STATO describes various concepts for statistical studies. The class “data_transformation” contains more than 60 subclasses of statistical tests and methods, as ANalysis Of VAriance (ANOVA) methods. This class may be very useful for NDoE to assess its sampling, to analyse results and for sensitivity analysis. These subclasses are integrated into ODE, represented by the class “Statistical_methods”. NDoE types are also described in STATO, by the class “study design”. 3.3. Clarity and logic The hierarchy of classes, subclasses and properties provided by OWL-DL enables the ontology to represent clearly the semantic structure of the NDoE process domain. Object properties were created to semantically link every concept of this ontology. The property’s name indicates its domain, its function and its range. For instance, the property linking the NDoE process and the NDoE is simply named “NDoE_Process_hasNDoE”. The cardinality is specified to restrict the number of instance for a specific instance of NDoE process. In the same example, there is at least one NDoE. Then, property’s characteristics are added. The property “NDoE_hasNDoEprocess” is considered as the inverse property of “NDoE_Process_hasNDoE”. These rules ensure the consistency of the database, preventing a NDoE process from having more than one objectives, no NDoE or a NDoE with no factors. Ontologies can be checked by appropriate reasoning algorithms. ODE was checked with the algorithm Pellet (Fig. 7). Pellet is an open-source OWL-DL reasoner [35]. First, it checks the OWL syntax, and then, the consistency of the ontology. The consistency checking ensures ODE does not contain any contradiction between the structure ontology (classes, object and data properties, etc.) and individuals which populate it. It also computes missing relations from existing properties and computes the type of each individual. Pellet algorithm fully supports OWL-DL, as every object properties characteristics (inverse, transitive properties, etc.) and cardinality restrictions.

3.4. Minimal encoding bias and ontological commitment Most of concepts defined in ODE were named according to literature reviews on NDoE, statistical methods and optimization algorithms, or from data models, the ISO 10303-209 standard and ontologies [6,11,13,36]. However, some concepts’ name may be specific to a research field. For instance, “Surrogate_model” class is also called “Metamodel”, which can be ambiguous or misleading in some scientific communities. Thus, this second name is specified as a synonym. Annotations and comments are added to describe each concept. Properties definitions are defined in a more subjective way, and may require an external validation. 3.5. Synthesis of the proposed ontology Extended enterprises, developing complex products, need to manage simulation data, as numerical tools are more and more used to fulfil quality, cost and time objectives. Complex processes, such as NDoE processes, require advanced knowledge and generate a large amount of heterogeneous data. The literature review detailed in Section 2 revealed a lack in term of semantic description of NDoE processes. It revealed also many ontologies which can be linked to ODE. As a result, ODE is focused on the NDOE process and also connected to other knowledge and data models, to ensure knowledge and data traceability. These connections were also made to propose a consistent ontology with other, and already used, ontologies. The proposed ontology, ODE (Fig. 8), was design to enhance data and knowledge management involved in the NDoE process. Data and knowledge can be captured, capitalized and shared across extended enterprises to improve and shorten decision processes. As a result, ODE provides a coherent and unified semantic description. ODE is based on existing ontologies. This base is completed by new classes, such as “NDoE_Process”, " Adaptive_DoE” and “Optimization_Algorithm”. These general classes are specialized according to the set of tools, methods, programs and softwares available for designers. 4. Use-Case The aim of the use-case is to illustrate functionalities of the proposed ontology for data reusing. This use-case takes place in a PDP, during the design phase of an air-blower sub-system of

34

G. Blondet et al. / Computers in Industry 94 (2018) 26–40

Fig. 8. global view of ODE in Protégé..

Fig. 9. HVAC system (Left) and 3D-model of the air-blower (Right) (Valeo©).

Heating, Ventilation and Air-Conditioning (HVAC) systems (Fig. 9). This system is developed by Valeo, a company in automotive industry, member of the consortium of the SDM4DOE project.

This design process focused on the optimization of elastomer parts in charge of the dynamic insulation between the rotor and the stator. Customers’ requirements were analysed and a first design was produced. The 3D model was parameterized and

G. Blondet et al. / Computers in Industry 94 (2018) 26–40

35

Fig. 10. Use-case definition.

numerical models were created to simulate the behaviour of this product, and more specifically its dynamic behaviour. NDoE processes are applied to improve designers ‘understanding of this product and to optimize its design as fast as possible. For instance, NDoE processes are used to identify relevant parameters according to the dynamic behaviour, and also to estimate a surrogate model to shorten decision processes with fast simulation. This use-case focuses on one NDoE process used for this product. The goal of this process is to product a surrogate model, to replace the FEM model by a faster model. Other NDoE processes may be done before and/or after this NDoE process to, for instance, optimize the product, improve its robustness and its reliability. ODE supports the knowledge and structures data required to configure the NDoE process. This would lead to choose the right number of needed experiments, to obtain an accurate surrogate model, and the time spent by designers to search for a good configuration. (Fig. 10) describes the use-case in several steps. Designers submit a new and incompletely configured NDoE process to search for previously executed NDoE process, applied for similar objectives, constraints and numerical models. These configurations are extracted from instances of ODE, which can be reused by designers to execute their new NDoE process. 4.1. Tools This use case is based on a database of 600 different NDoE processes. Seven finite-element models were used to execute these NDoE. These models represent 2 different structures (Fig. 11, Table 2). The T-model is an idealized system to check developments made during the SDM4DOE project. The R-model is a finiteelement model of the air-blower which belongs to air-conditioning systems used in cars. This model is used to analyse the dynamic behaviour of this rotating system, to isolate neighbour systems from vibrations, with elastomer dampers (in green in Fig. 11). Rmodels were used to validate results computed from tools used in the SDM4DOE framework. The FEA software Code_Aster (http:// www.code-aster.org) was used to compute outputs from finite-

element models, URANIE [37] and ROOT [38] were used to plan NDoE and analyse results. The ontology was instantiated by using OpenLink© Virtuoso server. Queries were done in SPARQL.The R1 model embeds 8 parameters, which were selected to be the factors of the NDoE (Table 3). It covers geometrical parameter of elastomer dampers, its elastic modulus, but also the length of the stiffener used for dampers, the vertical position of the centre of mass of the stator and the rotor, and finally, the phase difference between rotating unbalance forces. The first three eigenmodes were studied through eigenfrequencies and their associated participation factors along the vertical axis. NDoE were applied to identify the influence of each of these parameters (and interactions of these parameters) on the dynamic behaviour of the air-blower, to model this behaviour by a surrogate model and to optimize this system. 4.2. Instantiation Each NDoE process is described by numerous types of data. These data are shown in Table 4, with two examples of instantiation. In this example, the first instance involves many choices which must be done manually by designers (e.g. the DoE type, the surrogate model type and related parameters, etc.). The aim of ODE is to make this configuration step easier and shorter. After a consistency check operation with the Pellet algorithm, the database is ready to be queried.

Fig. 11. numerical models used in the use case (Left: T-model; Right: R-model).

36

G. Blondet et al. / Computers in Industry 94 (2018) 26–40

Table 2 FEA model description. Model Name

Physics

Factors

Outputs

T1 T2 T3 T4 T5 R1

Linear static Linear static with contacts Modal analysis Linear static Thermo-mechanics Modal analysis

3: 3: 3: 3: 3: 8:

2: 2: 3: 3: 3: 5:

Young modulus, Force components Young modulus, Force components Young modulus of each part Young modulus of each part 1 thermal expansion coefficients, 2 thermal conductivities geometric and material parameters

reaction force, maximal strain reaction force, maximal strain eigenfrequencies stress, strain, displacement stress, strain, displacement eigenfrequencies

Table 3 Parameters and outputs of the R1 model (air-blower).

Parameters

Output

Name

Unit

Min Value

Max Value

Distribution

Damper_Thickness Damper_Length Damper_Height Stiffener_Length Damper_Young Z_Stator Z_Rotor Rotating unbalance phase First three eigenfrequencies First three participation factor/z axis

m m m m Pa m m Degree ( ) Hz /

0.0015 0.01 0.01 0.002 1000000 0.022 0.068 0 – –

0.0045 0.04 0.02 0.007 5000000 0.008 0.028 180 – –

Uniform

– –

Table 4 Example of NDoE process available in the database. Type of data

Instance 1

Instance 2

Author Initial Model Type of physic Model_execution_time Non_linearity_level Factor number Output Objective DoE_Type Number of initial experiments NDoE_Mode Maximal number of added experiments Accuracy constraint Predictivity constraint Optimisation algorithm GA population GA mutation rate GA survival rate GA convergence threshold Type of surrogate model Polynomial chaos_degree Kriging_Correlation_function Kriging_Screening size Kriging_trend Kriging_optimisation_criterion Kriging_optimisation_algorithm Kriging_iteration_number Infill criterion Results

Author1 Name: T1 Mechanics-Static linear 2s Low 3 Strain Surrogate modelling Latin Hypercube 17 Adaptive 10 Normalized Root-Mean-Squared Error NRMSE <0.01 Q2 > 0.98 Genetic algorithm (GA) 10 0.3 0.8 1E-10 Kriging – matern7/2 300 Constant Maximum likelihood BOBYQA 800 Expected Improvement Time=2600s Accuracy NRMSE = 0.0095 Predictivity Q2 = 0.985 Added experiments: 5

Author3 Name: R1 Modal analysis 125s Low 8 1st mode Sensitivity Sobol’ 40 Non Adaptive – – – – – – – – Polynomial Chaos 2 – – – – – – – Time=9000s Accuracy NRMSE=0.013 Predictivity Q2=0.976

4.3. Queries Multiple queries are executed to find efficient NDoE processes previously executed with a similar context to produce a surrogate model. General data are specified about the current NDoE process, such as authors’ name, the objective and the initial numerical

model. The initial numerical model is a finite-element model, for a static linear analysis of a simple structure. A computational budget is set, so that a maximum of 40 experiments can be executed. Results of each of these queries are shown in Figs. 12–15. The aim of the first query (Table 5) is to retrieve every process instance with a surrogate modelling objective, or, at least, a surrogate model (as surrogate models can be used for different

G. Blondet et al. / Computers in Industry 94 (2018) 26–40

37

Fig. 12. Excerpt from results of Query 1.

Fig. 13. Excerpt from results of Query 2.

Fig. 14. Excerpt from results of Query 3.

Fig. 15. Excerpt from results of Query 4.

objectives).No NDoE process associated to an objective of surrogate modelling was found in the knowledge base. But, every NDoE processes used for a sensitivity analysis were listed, with the surrogate model used. Surrogate models produced for sensitivity analysis may be less accurate than those which were produced with a surrogate modelling objective, but even these NDoE processes can be interesting (Fig. 12). The second query extends the first by retrieving initial models which are similar according to their name, output, inputs, execution time and non-linearity level (defined by authors). Initial models, used during NDoE processes which were found by the first query, are listed with their main properties. Designers can compare their own model and previously used models. In this use-case, finite elements models were used. From these results,

designers can select processes with a similar initial model, regarding its nature and its execution time. According to these results, the initial model “FEM_Model_1” is the closest to the current model. For sake of clarity, resources data were not added to this query, but the field “Exec_Time” is directly linked to available resources, such as computers used and their performance (Fig. 13). The third query removes all configurations which did not produce an accurate and predictive surrogate model. Reference levels of the normalized RMSE and of the predictivity index Q2 are respectively set to 0.01 (lower is better) and 0.98 (1 is perfect). The instruction used to filter results (FILTER (?Accuracy_Value <0.01 &&?Pred_Value>0.98 &&?Pred_Value < 909909)) integrates a third condition to avoid NDoE_Process which had failed (909909 means an error during surrogate modelling). This query reveals 11

38

G. Blondet et al. / Computers in Industry 94 (2018) 26–40

Table 5 First query written with SPARQL.

cases which can be now studied in details. The NDOE_Process_100 is the best configuration according to these results (Fig. 14). The fourth query gives the whole configuration for each of these NDoE process. Designers are able to choose a configuration which minimise the number of experiments to obtain a good surrogate model. The NDoE_Process_89 seems to be the best configuration, with only 27 experiments. They are also free to adapt these configurations (Fig. 15). By such queries, designers can retrieve previously used NDoE processes. Advanced methods, as adaptive NDoE, are easier to be used to estimate surrogate model, although the expert knowledge does not exist in the company. The new NDoE process is then executed. Results from the NDoE are analysed to produce a surrogate model. For further studies, the costly numerical model can be replaced by the surrogate model. It embeds also uncertainties about design parameters earlier during the PDP. It can be executed almost instantaneously, so that designers can use it during meetings to shorten decision processes. New NDoE processes can be applied on this surrogate model to analyse more accurately the dynamic behaviour of the air-blower system, since thousands simulations can be now considered with such a model. The knowledge base provided by ODE helps designers to find the most efficient configuration. The surrogate model can be produced with the most appropriate number of experiments. The configuration step is also faster. After the execution of the NDoE process, its configuration, results and performance indicators (e.g. execution time, accuracy of the surrogate model) are stored and structured by ODE. These new results are available for other teams and departments in the company. The use of NDoE process, useful to optimize the product, can be spread in the company in an easier way.

5. Discussion Thanks to the proposed ontology, designers can query the database according to a shared understanding of the NDoE process. ODE fills the gap identified in the literature review by providing a semantic description of NDoE applied to develop better products. ODE provides a knowledge structure to capitalize, manage, share, trace and reuse data and knowledge about the NDoE process. The core of this ontology can be used as a basis for a specialization to specific industrial contexts. The aim of ODE is to be applied in various organizations, and linked to others ontologies to be adapted. Core concepts, as NDoE type, should be adapted to particular context, as a particular company would not have access to the same tools as those used in these researches. The use case showed how to query the database and to filter useful information to configure the NDoE process faster. Several concepts must be defined by users, as project data, the objective of the process, the nature of the initial numerical model, etc. These concepts are essential to specify, identify and assess a NDoE process in a particular context. Context and strategic data must be defined by the user to initiate the decision-support process. This solution can be profitable for extended enterprises involving different teams or departments which use NDoE. NDoE processes are configured faster and complex methods, which can shorten the execution step, are more easily accessible to every designer. The configuration step and the execution can be both shortened. Consequently, decisions are faster and the PDP is more profitable. Data and knowledge security is ensured since explicit product data are not required to configure the NDoE process. Results from simulation, giving information about product behaviour and performances, are useless and can be hidden. Knowledge about

G. Blondet et al. / Computers in Industry 94 (2018) 26–40

NDoE process can be shared between several companies without any security issue. The knowledge base is as useful as instances are numerous. The use of ODE is more profitable for extended enterprises with large teams dedicated to numerical simulation, as they must work together and share large amount of data. Moreover, due to the complexity of NDoE process configuration (very high number of possible configurations), the database must contain numerous instances to make reliable proposals of new configurations. Then, this solution is more appropriate to comparable products and behaviours which require accurate analysis with multiple simulations (e.g. a same family of products with different configurations). Since a lot of instances may be stored, a decision-support system is also required to analyse results and automatically suggest new configuration to designers. Furthermore, the configuration step does not follow only logical rules. An inference engine will be connected to this ontology, based on bayesian networks. Such an inference engine is able to combine expert knowledge and knowledge deduced from statistical data analysis. This would reflect the uncertainty of knowledge and predict new efficient configurations, which may not exist in the database. The ultimate goal of this strategy is to discover innovative NDoE process configuration. ODE is potentially applicable to a large variety of product, since NDoE can be used for every type of simulation. The methodology of DoE is used in various domains, and the core concepts of ODE are independent of the studied behaviour: it may be applied to mechanical systems, but also, for instance, to chemical, electronical and biological products, and more generally to multi-physical behaviour. To assess the benefits of ODE in a context of extended enterprise, the ontology should be set across different departments or stakeholders of a specific project. The main feature which must be assessed is the ability to enhance the share of knowledge required to configure and to execute an efficient NDoE process. The application of ODE may shorten both the configuration step and the execution step. Less simulations are required to obtain more useful results. Thus, ODE may lead to an increase of productivity. The designer will spend less time to search for knowledge manually and to try different configurations. The designer would be able to be more focused on other tasks. If the number of simulations is decreased, computational resources will be available for other tasks, for other designers. ODE may affect gradually the productivity of the company with the continuous knowledge capture. This improvement should be measured to test the efficiency of ODE. The computational load should be measured for a period covering multiple projects. The number of simulations for a specific project may be reduced. Projects may be finished faster. A real test should last enough to capture the effect of ODE, according to the company (e.g. a month, a year). The time spent to configure the process should be measured also. If ODE is effective, the number of trial and error loops for configuration may be reduced, and each loop may be shortened. Such a test should be applied to heterogeneous teams, with junior and senior designers, with different levels of expertise. The final goal of ODE is to help designers to learn thanks to the ontology itself. This could be measured by stopping temporarily the use of ODE and to continue to measure the time and resources spent on projects. 6. Conclusion While product development process is more and more based on complex and expensive numerical simulations, methods required to control the product design are more and more complex, as the

39

NDoE process. Knowledge required and data generated by NDoE processes reduce its efficiency. NDoE process may be complex to be managed. It can generate a lot of data, it may require specific expertise, which can be unknown, uncertain or implicit, to be configured and to analyse results, and it may be used in a context of extended enterprise for complex products. Therefore, knowledge management is needed to capture, classify, trace, share and reuse data and knowledge involved in this process. An ontology, ODE, is proposed to solve these issues. ODE provides a unified semantic description to manage data and knowledge of NDoE in a product development process. ODE is based on existing ontologies. Several classes of these ontologies were connected to ODE. ODE was developed regarding few criteria, as consistency, extendibility, and clarity of its concepts and properties. This ontology was populated and queried to show its ability for data retrieving. With ODE, designers are able to share and reuse the capitalized knowledge to shorten the configuration of NDoE process. The use of ODE leads to use complex and efficient NDoE processes more frequently. Several complex methods involved in NDoE processes can be now used more easily by designers to improve the quality of complex products. The product can be (1) optimized (e.g. mass reduction, energy consumption reduction, etc.), (2) more reliable and (3) more robust with a fast set up of efficient NDoE processes. Because of the potential large size of database, a specific inference engine is also used in combination of ODE to analyse and to automatically propose adapted configuration according to designers’ current problem. This inference engine is based on bayesian networks, which provide a probabilistic analysis of expert knowledge and data capitalized with ODE. A bayesian network is built on extracted data and existing expert knowledge corresponding to the current NDoE process configuration task. Thus, this approach combines best practices about NDoE and capitalized data from previous NDoE processes, to describe causal links between each element of the configuration, such as a NDoE type, the number of experiment, or the type of surrogate model. The bayesian inference engine is able to predict the most probable efficient configuration for a given context, and also to diagnose the efficiency of a user-defined configuration. The combination of ODE and the bayesian inference engine lead to a more efficient design process and a continuous improvement of knowledge about NDoE processes in companies. Designers can submit their problem and the system proposes an optimal configuration according to available knowledge stored in ODE. The NDoE process gives more accurate results faster, by minimizing required computational resources. Currently, this solution is being implemented in the company Valeo to improve their products faster with a minimal cost, but also to spread the use of NDoE inside the company. Acknowledgments This work is done in the French FUI project SDM4DOE. We also thank all consortium partners for their contribution during the development of ideas and concepts proposed in the paper. References [1] M. Norris, Business Value from Simulation Data Management-a Decade of Production Experience, (2012) . [2] CIMDATA Simulation & Analysis Governance, (2014) . [3] W. Hu, L.G. Yao, Z.Z. Hua, Optimization of sheet metal forming processes by adaptive response surface based on intelligent sampling method, J. Mater. Process. Technol. 197 (2008) 77–88, doi:http://dx.doi.org/10.1016/j. jmatprotec.2007.06.018. [4] S. Castric, Z. Cherfi, G.J. Blanchard, N. Boudaoud, Modeling pollutant emissions of diesel engine based on kriging models: a comparison between geostatistic and gaussian process approach, 4th IFAC Symposium on Information Control

40

[5]

[6]

[7]

[8] [9]

[10] [11]

[12]

[13]

[14]

[15]

[16]

[17]

[18]

[19]

[20]

G. Blondet et al. / Computers in Industry 94 (2018) 26–40 Problems in Manufacturing, INCOM’12 14 (2015) 1708–1715, doi:http://dx.doi. org/10.3182/20120523-3-RO-2023.00038. E. Patelli, H. Murat Panayirci, M. Broggi, B. Goller, P. Beaurepaire, H.J. Pradlwarter, et al., General purpose software for efficient uncertainty management of large finite element models, Finite Elem. Anal. Des. 51 (2012) 31–48, doi:http://dx.doi.org/10.1016/j.finel.2011.11.003. V. Picheny, D. Ginsbourger, O. Roustant, R.T. Haftka, N.-H. Kim, Adaptive designs of experiments for accurate approximation of a target region, J. Mech. Des. 132 (2010) 71008, doi:http://dx.doi.org/10.1115/1.4001873. G. Blondet, J. Le Duigou, N. Boudaoud, B. Eynard, Simulation data management for adaptive design of experiments: a litterature review, Mech. Ind. 16 (2015) 611, doi:http://dx.doi.org/10.1051/meca/2015041. T.R. Gruber, A translation approach to portable ontology specifications, Knowl. Acquisition 5 (1993) 199–220, doi:http://dx.doi.org/10.1006/knac.1993.1008. C. Feilmayr, W. Wöß, An analysis of ontologies and their success factors for application to business, Data Knowl. Eng. 101 (2016) 1–23, doi:http://dx.doi. org/10.1016/j.datak.2015.11.003. N.F. Noy, D.L. Mcguinness, Ontology Development 101: A Guide to Creating Your First Ontology, (2001) , pp. 1–25. A.I.J. Forrester, A.J. Keane, Recent advances in surrogate-based optimization, Prog. Aerosp. Sci. 45 (2009) 50–79, doi:http://dx.doi.org/10.1016/j. paerosci.2008.11.001. B. Iooss, P. Lemaître, A review on global sensitivity analysis methods, in: G. Dellino, C. Meloni (Eds.), Uncertainty Management in SimulationOptimization of Complex Systems-Algorithms and Applications-Part 2, Springer, US, 2014, pp. 101–122, doi:http://dx.doi.org/10.1007/978-1-48997547-8_5. J. Franco, Planification d’expériences Numériques En Phase Exploratoire Pour La Simulation Des Phénomènes Complexes, Ecole Nationale Supérieure des Mines de Saint-Etienne (ENSM-SE), 2008. E. Gruhier, F. Demoly, O. Dutartre, S. Abboudi, S. Gomes, A formal ontologybased spatiotemporal mereotopology for integrated product design and assembly sequence planning, Adv. Eng. Inf. 29 (2015) 495–512, doi:http://dx. doi.org/10.1016/j.aei.2015.04.004. W. Lu, Y. Qin, X. Liu, M. Huang, L. Zhou, X. Jiang, Enriching the semantics of variational geometric constraint data with ontology, CAD Comput. Aided Des. 63 (2015) 72–85, doi:http://dx.doi.org/10.1016/j.cad.2014.12.008. A. Matsokis, D. Kiritsis, An ontology-based approach for product lifecycle management, Comput. Ind. 61 (2010) 787–797, doi:http://dx.doi.org/10.1016/j. compind.2010.05.007. H. Panetto, M. Dassisti, A. Tursi, ONTO-PDM: product-driven ontology for product data management interoperability within manufacturing process environment, Adv. Eng. Inf. 26 (2012) 334–348, doi:http://dx.doi.org/10.1016/j. aei.2011.12.002. C. Danjou, J. Le Duigou, B. Eynard, OntoSTEP-NC for information feedbacks from CNC to CAD/CAM systems, Adv. Prod. Manage. Syst. 440 (2014) 256–263, doi:http://dx.doi.org/10.1007/978-3-662-44733-8_32. G. Rehage, J. Gausemeier, Ontology-based determination of alternative CNC machines for a flexible resource allocation, 15th CIRP Conference on Modelling of Machining Operations, Elsevier B.V., 2015, pp. 47–52, doi:http://dx.doi.org/ 10.1016/j.procir.2015.03.054. J. Le Duigou, A. Bernard, Product lifecycle management model for design information management in mechanical field, 21st CIRP Design Conference, Korea, 2011, pp. 207–213.

[21] X. Fiorentini, I. Gambino, V.-C. Liang, R. Sudarsan, M. Mani, C. Bock, An ontology for assembly representation, NISTIR 7436 (2007) 76. [22] R. Barbau, S. Krima, R. Sudarsan, A. Narayanan, X. Fiorentini, S. Foufou, et al., OntoSTEP: Enriching product model data using ontologies, Comput.-Aided Des. 44 (2012) 575–590, doi:http://dx.doi.org/10.1016/j.cad.2012.01.008. [23] I.R. Grosse, J.M. Milton–Benoit, J.C. Wileden, Ontologies for supporting engineering analysis models, AI EDAM. 19 (2005) 1–18, doi:http://dx.doi. org/10.1017/S0890060405050018. [24] I. Assouroko, G. Ducellier, P. Boutinaud, B. Eynard, Knowledge management and reuse in collaborative product development a semantic relationship management-based approach, Int. J. Prod. Lifecycle Manage. 7 (2014) 54–74. [25] M.J. Pratt, ISO 10303, the STEP standard for product data exchange, and its PLM capabilities, Int. J. Prod. Lifecycle Manage. 1 (2005) 86, doi:http://dx.doi.org/ 10.1504/IJPLM.2005.007347. [26] X.W. Xu, H. Wang, J. Mao, S.T. Newman, T.R. Kramer, F.M. Proctor, et al., STEPcompliant NC research: the search for intelligent CAD/CAPP/CAM/CNC integration, Int. J. Prod. Res. 43 (2005) 3703–3743. [27] ISO, ISO 10303-242 – Application Protocol: Managed Model-based 3D Engineering, (2014) . [28] K. a. Hunten, A. Barnard Feeney, V. Srinivasan, Recent advances in sharing standardized STEP composite structure design and manufacturing information, CAD Comput. Aided Des. 45 (2013) 1215–1221, doi:http://dx.doi. org/10.1016/j.cad.2013.05.001. [29] D. Chang, C.-H. Chen, Product concept evaluation and selection using data mining and domain ontology in a crowdsourcing environment, Adv. Eng. Inf. 29 (2015) 759–774, doi:http://dx.doi.org/10.1016/j.aei.2015.06.003. [30] Y. Zhang, X. Luo, Y. Zhao, H. Zhang, An ontology-based knowledge framework for engineering material selection, Adv. Eng. Inf. 29 (2015) 985–1000, doi: http://dx.doi.org/10.1016/j.aei.2015.09.002. [31] L.N. Soldatova, R.D. King, An ontology of scientific experiments, J. R. Soc. Interface 3 (2006) 795–803, doi:http://dx.doi.org/10.1098/rsif.2006.0134. [32] A. Pease, I. Niles, J. Li, The Suggested Upper Merged Ontology: A Large Ontology for the Semantic Web and Its Applications, (2002) . http://home.earthlink.net/ ?adampease/professional/Pease.ps. [33] J. Vanschoren, L.N. Soldatova, Exposé: An ontology for data mining experiments, Proc. of the 3rd Int. Workshop on Third Generation Data Mining: Towards Service-Oriented Knowledge Discovery (SoKD), Barcelona, Spain, 2010, pp. 31–46. [34] J.P.C. Kleijnen, W. Van Beers, I. Van Nieuwenhuyse, Expected improvement in efficient global optimization through bootstrapped kriging, J. Global Optim. 54 (2012) 59–73, doi:http://dx.doi.org/10.1007/s10898-011-9741-y. [35] E. Sirin, B. Parsia, B.C. Grau, A. Kalyanpur, Y. Katz, Pellet: a practical OWL-DL reasoner, Web Semant. 5 (2007) 51–53, doi:http://dx.doi.org/10.1016/j. websem.2007.03.004. [36] H. Liu, S. Maghsoodloo, Simulation optimization based on Taylor Kriging and evolutionary algorithm, Appl. Soft Comput. J. 11 (2011) 3451–3462, doi:http:// dx.doi.org/10.1016/j.asoc.2011.01.017. [37] F. Gaudier, URANIE: the CEA/DEN uncertainty and sensitivity platform, Sixth International Conference on Sensitivity Analysis of Model Output, Elsevier Masson SAS, 2010, pp. 7660–7661, doi:http://dx.doi.org/10.1016/j. sbspro.2010.05.166. [38] I. Antcheva, M. Ballintijn, B. Bellenot, M. Biskup, R. Brun, N. Buncic, et al., ROOT — A C + + framework for petabyte data storage, statistical analysis and visualization, Comput. Phys. Commun. 180 (2009) 2499–2512, doi:http://dx. doi.org/10.1016/j.cpc.2009.08.005.