Experimental Design in a Multicriteria Optimization Context: An Adaptive Scheme

Proceedings ofModelling the 9th Vienna International Conference on Mathematical Vienna, Austria, February 21-23, 2018 Proceedings ofModelling the 9th Vienna International Conference on Mathematical Vienna, Austria, February 21-23, 2018 Proceedings ofModelling the 9th Vienna International Conference on at www.sciencedirect.com Mathematical online Vienna, Austria, February 21-23, 2018 Available Mathematical Modelling Vienna, Austria, February 21-23, 2018 Vienna, Austria, February 21-23, 2018

ScienceDirect

Experimental Design in a Multicriteria Optimization Context: Experimental Design in a Multicriteria IFAC PapersOnLine 51-2 (2018) 747–752 Optimization Context: Experimental Design in Adaptive a Multicriteria Optimization Context: An Scheme Experimental DesignAn in Adaptive a Multicriteria Optimization Context: Scheme Experimental DesignAn in Adaptive a Multicriteria Optimization Context: Scheme Adaptive Scheme M.An Bortz*. J. Höller*, J. Schwientek*, Bortz*. J. Höller*, J. Schwientek*, An Adaptive Scheme R.M.Böttcher**, O. Hirth**, N. Asprion**

Bortz*. J. Höller*, J. Schwientek*, R.M.Böttcher**, O. Hirth**, N. Asprion** Bortz*. J. Höller*, J. Schwientek*, R.M.Böttcher**, O. Hirth**, N. Asprion** Bortz*. J. Höller*, J. Schwientek*, R.M.Böttcher**, O. Hirth**, N. Asprion**  R. for Böttcher**, Hirth**, N.67663 Asprion** *Fraunhofer Institute Industrial O. Mathematics, Kaiserslautern, Germany  *Fraunhofer Institute Mathematics, Kaiserslautern, Germany **BASFfor SE,Industrial 67056 Ludwigshafen am67663 Rhein, Germany  *Fraunhofer Institute Mathematics, Kaiserslautern, Germany **BASFfor SE,Industrial 67056 Ludwigshafen am67663 Rhein, Germany  *Fraunhofer Institute Mathematics, Kaiserslautern, Germany **BASFfor SE,Industrial 67056 Ludwigshafen am67663 Rhein, Germany *Fraunhofer Institute Mathematics, Kaiserslautern, Germany **BASFfor SE,Industrial 67056 Ludwigshafen am67663 Rhein, Germany The identification of a promising in design space where to obtain an optimal experiment al **BASFregion SE, 67056 Ludwigshafen amstrategies Rhein, Germany The identification of aispromising in design space In where to obtain anfrom optimal experiment al design can be applied crucial in region practical applications. this strategies contribution, starting a model adjusted The identification of aispromising region in design space In where strategies to obtain anfrom optimal experiment al design can be applied crucial in practical applications. this contribution, starting a model adjusted to previously experiments, a computationally efficient multicriteria optimization scheme is used The identification of aispromising in design space In where to obtain anfrom optimal experiment al design can beconducted applied crucial in region practical applications. this strategies contribution, starting a model adjusted to previously conducted experiments, a computationally efficient multicriteria optimization scheme is used The identification of aisboundary, promising region in design space where strategies to obtain anfrom optimal experiment al to identify theapplied Pareto minimization of In thethis prediction errors of the objective functions is design can be crucial in where practical applications. contribution, starting a model adjusted to previously conducted experiments, a computationally efficient multicriteria optimization scheme is used to identify the Pareto isboundary, minimization thethis prediction errors of the from objective functions is design can applied crucial in where practical applications. In contribution, a model adjusted included asbe additional objective. Thisa guarantees thatofbest compromises arestarting found between the processto previously conducted experiments, computationally efficient multicriteria optimization scheme is used to identifyasthe Pareto boundary, where minimization ofbest the compromises prediction errors of the objective functions is included additional objective. This guarantees that are found between the processto previously conducted experiments, a computationally efficient multicriteria optimization scheme is used relevant objectives, cost andwhere quality criteria, while simultaneously quantifying the trade -off between to identify the Paretolike boundary, minimization ofbest the compromises prediction errors of the objective functions is included as additional objective. This guarantees that are found between the processrelevant objectives, cost andwhere quality criteria, while simultaneously quantifying the trade -off between to identify the Pareto boundary, minimization ofbest the prediction errors of theallows objective functions is those objectives andlike their prediction errors. In a real-time navigation procedure, this to narrow down included as additional objective. This guarantees that compromises are found between the processrelevant objectives, like cost and This quality criteria, while simultaneously quantifying the trade -off between those objectives and their prediction errors. In a real-time navigation procedure, this allows to narrow down included as additional objective. guarantees that best compromises are found between the processthe most objectives, promising region in and design space, wherewhile then strategies of model-based experimental design are relevant like quality criteria, simultaneously quantifying the trade -off between those objectives andregion theircost prediction errors. In a real-time navigation thisexperimental allows to narrow down the most promising in and design space, where then strategies of procedure, model-based design are relevant objectives, like cost quality criteria, while simultaneously quantifying the trade -off between applied. The entire workflow is illustrated with an intuitive example which shows that an unacceptably those objectives and their prediction errors. In a real-time navigation procedure, this allows to narrow down the most The promising region in design space, where strategies of model-based experimental design are applied. entire ispoints illustrated with anthen intuitive example which that experiments an unacceptably those objectives andworkflow their prediction errors. a real-time navigation procedure, thisexperimental allows to narrow down highmost prediction error of Pareto can beIn efficiently reduced byofonly a fewshows additional . are the promising region in design space, where then strategies model-based design applied. The entire workflow is illustrated with an intuitive example which shows that an unacceptably high prediction error of Pareto points can be efficiently reduced by only a few additional experiments . are the most promising region in design space, where then strategies of model-based experimental design applied. The entire workflow is illustrated with an intuitive example which shows that an unacceptably high prediction errorworkflow of Pareto points can be efficiently reduced by only aParameter fewshows additional experiments . © 2018, IFAC (International Federation of Automatic Hosting by Elsevier Ltd.estimation; All reserved. Keywords: Experiment Design; Multiobjective optimizations; Prediction applied. TheOptimal entire illustrated with an Control) intuitive example thatrights an unacceptably high prediction error Experiment of Pareto ispoints can be efficiently reduced by onlywhich aParameter few additional experiments . Keywords: Optimal Design; Multiobjective optimizations; estimation; Prediction error methods; Statistical design high prediction error Experiment of Pareto points can be efficiently reduced by only aParameter few additional experiments . Keywords: Optimal Design; Multiobjective optimizations; estimation; Prediction error methods; Statistical design Keywords: Optimal Experiment Design; Multiobjective optimizations; Parameter estimation; Prediction error methods; Statistical design  Keywords: Optimal Experiment optimizations; Parameter estimation; Prediction error methods; Statistical design Design; Multiobjective  commonly is used to perform model-based optimizatio n error methods; Statistical design  1. INTRODUCTION commonly is used perform model-based optimizatio n studies which predictto favourable designs to meet certain  1. INTRODUCTION commonly is used to favourable perform model-based optimizatio n studies which predict designs to criteria. meet certain  1. INTRODUCTION process-related objectives, like cost or quality It has is used to favourable perform model-based optimizatio n Different strategies to obtain optimal model-based commonly studies which predict designs to criteria. meet certain 1. INTRODUCTION process-related objectives, like cost or quality It has isin used to favourable perform model-based optimizatio n Different strategies obtain model-based been shown the context of chemical process engineering, studies which predict to criteria. meet certain experimental designs to (OEDs) are optimal well-known, cf. e.g. commonly 1. INTRODUCTION process-related objectives, like costdesigns or quality It has Different strategies to obtain optimal model-based been shown in predict the context of that chemical process engineering, studies which favourable designs to criteria. meet certain experimental designs (OEDs) are well-known, cf. e.g. see e.g. Bortz et al (2014), multicriteria optimizatio n process-related objectives, like cost or quality It has Franceschini etdesigns al (2008),to andobtain have successfully applied Different strategies optimal model-based beene.g. shown in the context of that chemical process optimizatio engineering, experimental (OEDs) arebeen well-known, cf. e.g. see Bortz et al (2014), multicriteria n process-related objectives, like or quality criteria. It has Franceschini etdesigns alin(2008), andobtain have been successfully applied (MCO) offers possibility ofcost balancing such conflicting Different strategies to optimal model-based been shown in the the context of that chemical process engineering, to experiments different contexts; for application in experimental (OEDs) are well-known, cf. e.g. see e.g. Bortz et al (2014), multicriteria optimizatio n Franceschini et al (2008), and have been successfully applied (MCO) offers the possibility of balancing such conflicting been shown in the context of chemical process engineering, to experiments in different contexts; for application in objectives. Thus for(2014), the that practitioner, the optimizatio impact of experimental designs arebeen well-known, cf. see e.g. offers Bortz et al multicriteria n chemical engineering, cf.(OEDs) for example Arrellano-Garcia ete.g. al (MCO) Franceschini et al (2008), and have successfully applied the possibility of balancing such conflicting to experiments in different contexts;Arrellano-Garcia for applicationet in objectives. Thus for(2014), the that practitioner, theboundary impact is of of e.g. offers Bortz et al multicriteria optimizatio n chemical engineering, cf. for al see uncertainties in the model parameters the Pareto Franceschini et al andexample have beenissuccessfully (MCO) possibility of on balancing such conflicting (2007). Atengineering, the core of these strategies the application Fisher applied matrix, to experiments in(2008), different contexts; for Thus forparameters the practitioner, theboundary impact of chemical cf. for example Arrellano-Garcia et in al objectives. uncertainties in model on the Pareto is of (MCO) offers the possibility of balancing such conflicting (2007). At the core of these strategies is the Fisher matrix, special interest. Generally, these can is be to experiments in adifferent contexts; for application in objectives. Thus forparameters the practitioner, theboundary impact of which, in engineering, case of linear can be viewed asetthe chemical cf. for model, example Arrellano-Garcia al uncertainties in model on theuncertainites Pareto of (2007). Atengineering, the core these strategies is be the viewed Fisher matrix, special interest. Generally, these uncertainites can is be objectives. forparameters the practitioner, the of which, in case ofbetween aoflinear can asetthe quantified byThus sensitivity analysis as inthe Asprion et alimpact (2016). In chemical cf. for example Arrellano-Garcia al special uncertainties in model on Pareto boundary of covariance matrix themodel, experimental points in matrix, design (2007). At the core of these strategies is the Fisher interest. Generally, these uncertainites can is be which, in case ofbetween a linearthemodel, can be points viewed as the quantified by sensitivity analysis as in Asprion et al (2016). In uncertainties in model parameters on the Pareto boundary of covariance matrix experimental in design the present situation, where we assume that the model is (2007). At nonlinear the core oflinear theseitmodel, strategies the Fisher matrix, special interest. Generally, these uncertainites can be space. For constitutes covariance mat rix quantified which, in case ofbetween amodels, canis abe viewed as the bysituation, sensitivity analysis asassume in Asprion etthe al (2016). In covariance matrix the experimental points in design the present where we that model is special interest. Generally, these uncertainites can be space. For nonlinear models, it constitutes a covariance mat rix obtained from a regression, the prediction error can be used which, in case of a linear model, can be viewed as the quantified bysituation, sensitivitywhere analysis asassume in Asprion etthe al (2016). In between derivatives of the output function with respect to covariance matrix between the experimental points in mat design the present weprediction that model is space. For nonlinear models, it constitutes a covariance rix obtained from a regression, the error can be used sensitivity analysis asassume indesign Asprion et al (2016). In between derivatives of the output function with respect to the quantified which is anby analytical function of the variables, yielding covariance matrix evaluated between the experimental points in points. design the present situation, where weprediction that is model parameters, at experimental design space. Forderivatives nonlinear models, it the constitutes awith covariance mat rix obtained from a regression, the errorthe canmodel be used between of the output function respectpoints. to the which is bar an analytical function of the design that variables, yielding the present situation, where weprediction assume the model is model parameters, evaluated at the experimental design an error on the model output function. space. For nonlinear models, it constitutes a covariance mat rix obtained from a regression, the error can be used The ultimate goal experimental design between derivatives ofofthemodel-based output function with design respectpoints. to the which is bar an analytical function of function. the design variables, yielding model parameters, evaluated at the experimental an error on the model output obtained from a regression, the prediction error can be used The ultimate goal model-based experimental design between derivatives ofofthe output function respectmatrix, to the which is bar an analytical function of function. the design variables, yielding consists in maximizing a scalar measure ofwith the Fisher model parameters, evaluated at the experimental design points. error onisthe output The ultimate goal ofa scalar model-based experimental design an Our key idea to model include theofminimization of the maxima which is bar an analytical function the design variables, yieldingl consists in maximizing measure oftrace the Fisher matrix, model parameters, evaluated at the experimental design points. an error on the model output function. like the determinant (D-criterion) or the (A-criterion ). The ultimate goal ofa scalar model-based experimental design Our key bar idea isthe toof include the minimization of the maxima consists in maximizing measure oftrace the Fisher matrix, prediction error the process -related objectives as anl an error on model output function. like the determinant (D-criterion) or the (A-criterion ). The goal non-linear ofa scalar model-based experimental design Our key idea is toofinclude the minimization of the maxima This isultimate a determinant optimization problem (NLP). consists inchallenging maximizing measure oftrace the Fisher matrix, prediction error the process -related objectives as anl like the (D-criterion) or the (A-criterion ). additional objective in the MCO. This generalizes themaxima concept key idea is toofinclude the minimization of the This is a in challenging optimization problem (NLP). consists maximizing a scalar of the Fisher matrix, prediction error the process -related objectives as anl Different strategies tonon-linear solve this measure problems have been reported , Our like the determinant (D-criterion) or the trace (A-criterion ). additional objective inwhich the MCO. Thisof generalizes themaxima concept Our idea is toofinclude the minimization of the This is a determinant challenging non-linear optimization problem (NLP). of a key utility function, consists a Bayes measure for prediction error the process -related objectives as anl Different strategies to solve this problems have been reported , like the (D-criterion) or the trace (A-criterion ). additional inwhich the MCO. Thisof generalizes the concept cf. e.g. et al (2011). As willoptimization always be theproblem case for(NLP). NLPs , of This is Goos a challenging non-linear a utilityobjective function, consists a Bayes measure for prediction error of the process -related objectives as an Different strategies to solve this problems have been reported , information content and the maximization of the model additional inwhich the MCO. Thisof generalizes the concept cf. e.g. et al (2011). As will always be theproblem case for(NLP). NLPs This is Goos a challenging optimization of a utilityobjective function, consists a Bayes measure for the chance of obtaining thethis global minimum is higher the,,, information Different strategies tonon-linear solve problems have been reported contentin and the maximization ofetmeasure the model additional objective the MCO. This generalizes the concept cf. e.g. Goos et al (2011). As will always be the case for NLPs response functions, as introduced in Verdinelli al (1992). of a utility function, which consists of a Bayes for the chance of obtaining thethis global minimum is higher the,, information content and the maximization of the model Different strategies to solve problems been reported better the starting point is.As cf. Goos et al (2011). always behave the case for NLPs functions, aswhich introduced inpermits Verdinelli al (1992). of function, consists of a Bayes measure for thee.g. chance of obtaining the will global minimum is higher the, response Thea utility resulting Pareto boundary toofet identify the information content the maximization the model better the starting point is.As cf. e.g. Goos et al (2011). will always be the case for NLPs response functions, asand introduced inpermits Verdinelli et al (1992). the chance of obtaining the global minimum is higher the The resulting Pareto boundary to identify the information content and the maximization of the model better the starting point is. corresponding trade-offs. Only if the compromise on the response functions, as introduced in Verdinelli et al (1992). While setting the NLP, one has tominimum identify an the chance ofup obtaining global is admissible higher the corresponding The resulting trade-offs. Pareto boundary permits to identify the better starting point is.the the compromise onsmall the functions, as introduced inpermits Verdinelli al (1992). While the setting up space. the NLP, one has toto the identify an strategy admissible process-related objectives inOnly orderifto reach a sufficiently The resulting Pareto boundary to et identify the region in starting design According general of response better the point is. corresponding trade-offs. Only if the compromise on the While setting up space. the NLP, one has toto the identify an strategy admissible process-related objectives in orderhigh, to reach a sufficiently small resulting Pareto boundary to identify the region in design According general of The prediction error trade-offs. is unacceptably additional experiment s corresponding iftopermits the compromise onsmall the OED, be such that additional While setting up space. theshould NLP, oneconducted has toto the identify an strategy admissible process-related objectives inOnly orderhigh, reach a sufficiently region experiments in design According general of corresponding prediction error is unacceptably additional experiment s trade-offs. Only if the compromise on the OED, experiments should be conducted such that additional are needed. In that case, box constraints can be set in objective While setting up space. the NLP, one has totofrom identify admissible process-related objectives in order to reach a sufficiently small model knowledge can be extracted themanwhere it of is region in design According the general strategy prediction error is case, unacceptably high, additional experiment s OED, experiments should be conducted such that additional are needed. In that box constraints can be set in objective process-related objectives in order to reach a sufficiently small model knowledge can be extracted from them where it is space to define most interesting range of experiment the Paretos region inmost. design According thesuch general strategy of prediction error isthe unacceptably high, additional neededexperiments Letspace. us assume thattoexperiments have been OED, should be conducted thatwhere additional are needed. In that case, box constraints can be set in objective model knowledge can be extracted from them it is space to define the most interesting range of the Pareto error is case, unacceptably high, experiment s needed most. usmodel assume that experiments have been prediction boundary, and a linear navigation basedadditional on the convex hull of OED, experiments should conducted such thatwhere additional are needed. In that box interesting constraints can be set objective conducted, andLet thatcan parameters have been estimated model be be extracted from them it is space to define the most range of inthe Pareto needed knowledge most. Let usmodel assume that experiments have been boundary, and a linear navigation based can on the convex hull of are needed. In that case, box constraints be set in objective conducted, and that parameters have been estimated the Pareto points is used to obtain an estimate for the model knowledge can be extracted from them where it is to define the most interesting of the hull Pareto according to a regression procedure. a practical point of space needed most. Let assume that From experiments been boundary, and a linear navigation based range on the convex of conducted, and thatus model parameters have beenhave estimated Pareto points used to space. obtain an the estimate the space to define theis in most interesting range of defines the for Pareto according toneed a regression procedure. aispractical point of the corresponding design This region the needed most. Let assume that From experiments have been boundary, and aregion linear navigation based on convex hull of view, the forusmodel further experiments not necessarily conducted, and that parameters have been estimated the Pareto points is used to obtain an estimate for the according toneed a regression procedure. From aispractical point of boundary, corresponding in design space. This region defines the aregion linear based on convex hullthe of view, the for model further experiments not necessarily admissible space which is to then used forthe OED strategies, conducted, and that parameters have been estimated Paretoand points is innavigation used obtain an estimate for defined bytothose regions inprocedure. the admissible design space where according a regression From aispractical point of the corresponding region design space. This region defines the view, the need for further experiments not necessarily admissible space which is then used for OED strategies, Pareto points is inused to space. obtainthese an region estimate forhave the defined byintothose regions inprocedure. thefunctions admissible space where leading to anspace experimental according a regression From aispractical of the corresponding region design This the the error the model output is design largest: Thepoint model view, the need for further notspace necessarily admissible which isplan. thenOnce used for experiments OED defines strategies, defined byinthose regions in theexperiments admissibleis design where corresponding leading to anspace experimental plan. Once these experiments have region in design space. This region defines the the error the model output functions largest: The model view, the need for further experiments is not necessarily admissible which is then used for OED strategies, defined regions in thefunctions admissibleis design where leading to an experimental plan. Once these experiments have the errorbyinthose the model output largest:space The model admissible space which is then used for OED strategies, defined regions in thefunctions admissibleis design where leading to an experimental plan. Once these experiments have the errorbyinthose the model output largest:space The model Copyright IFAC output functions is largest: The model 1 leading to an experimental plan. Once these experiments have the error ©in2018 the model Copyright © 2018 IFAC 1 Copyright © 2018, 2018 IFAC 1 Hosting by Elsevier Ltd. All rights reserved. 2405-8963 © IFAC (International Federation of Automatic Control) Copyright 2018 responsibility IFAC 1 Control. Peer review©under of International Federation of Automatic Copyright © 2018 IFAC 1 10.1016/j.ifacol.2018.04.003

Proceedings of the 9th MATHMOD 748 Vienna, Austria, February 21-23, 2018

M. Bortz et al. / IFAC PapersOnLine 51-2 (2018) 747–752

these, the prediction error ∆𝑦𝑦 (𝑥𝑥 1 , … , 𝑥𝑥 𝑁𝑁𝑑𝑑𝑑𝑑𝑑𝑑 ) is obtained, which can be viewed as the confidence interval of a model prediction 𝑦𝑦 = 𝑦𝑦(𝛼𝛼1∗ , … , 𝛼𝛼𝑁𝑁∗ 𝑝𝑝𝑝𝑝𝑝𝑝 ; 𝑥𝑥1 , … , 𝑥𝑥 𝑁𝑁𝑑𝑑𝑑𝑑𝑑𝑑 ) at an arbitrary point in design space:

been realized, the model is updated using again a regression procedure. Using this workflow, the experimentalist is supported to identify those regions in design space where experiments are really needed in order to reduce the prediction error on those predictions which are of particular importance for the optimization of the process.

∆𝑦𝑦 (𝑥𝑥1 , … , 𝑥𝑥 𝑁𝑁𝑑𝑑𝑑𝑑𝑑𝑑 )

= √𝑁𝑁𝑝𝑝𝑝𝑝𝑝𝑝 𝑞𝑞𝑁𝑁𝑒𝑒𝑒𝑒𝑒𝑒,𝑁𝑁𝑒𝑒𝑒𝑒𝑒𝑒 −𝑁𝑁𝑝𝑝𝑝𝑝𝑝𝑝 (∇𝜶𝜶 𝑦𝑦) 𝐹𝐹 −1 (∇𝜶𝜶 𝑦𝑦) 𝑇𝑇 ,

where the Fischer matrix 𝐹𝐹 has elements

This paper is organized as follows. In the next section, a short account on the methods used in this work is given, which include how to obtain the prediction error from a model regression, an adaptive scheme to sample the Pareto boundary, the interpolation strategy to obtain restrictions in design space from set restrictions in objective space and the approach for a model-based optimal design. Section 3 contains the illustration of the entire procedure with a simple example. Finally , conclusions are drawn and an outlook is given on the application to a real process .

𝐹𝐹𝑣𝑣 ,𝜇𝜇

𝑁𝑁𝑒𝑒𝑒𝑒𝑒𝑒 𝑗𝑗=1

(4)

∆𝑦𝑦 (𝑥𝑥1 , … , 𝑥𝑥 𝑁𝑁𝑑𝑑𝑑𝑑𝑑𝑑 )

Let 𝑋𝑋 be the design matrix resulting from the already conducted experiments. The matrix 𝑋𝑋 has 𝑁𝑁𝑒𝑒𝑒𝑒𝑒𝑒 rows corresponding to the number of measurements and 𝑁𝑁𝑑𝑑𝑑𝑑𝑑𝑑 columns corresponding to the dimensionality of design space. The design points of the experiments are denoted by 𝑥𝑥̃ 𝑗𝑗,𝑘𝑘 , with 𝑗𝑗 = 1, … , 𝑁𝑁𝑒𝑒𝑒𝑒𝑒𝑒 and 𝑘𝑘 = 1, … , 𝑁𝑁𝑑𝑑𝑑𝑑𝑑𝑑 , and the corresponding measured values of the response function as 𝑦𝑦̃𝑗𝑗 . The model output 𝑦𝑦 is a function of the design variables 𝑥𝑥 1 , … , 𝑥𝑥 𝑁𝑁𝑑𝑑𝑑𝑑𝑑𝑑, and parametrized by the model parameters 𝛼𝛼1 , … , 𝛼𝛼𝑁𝑁𝑝𝑝𝑝𝑝𝑝𝑝 , thus

𝑝𝑝𝑝𝑝𝑝𝑝

𝑁𝑁𝑝𝑝𝑝𝑝𝑝𝑝

= √𝑁𝑁𝑝𝑝𝑝𝑝𝑝𝑝 𝑞𝑞𝑁𝑁𝑒𝑒𝑒𝑒𝑒𝑒,𝑁𝑁𝑒𝑒𝑒𝑒𝑒𝑒 −𝑁𝑁𝑝𝑝𝑝𝑝𝑝𝑝 ∑ 𝑥𝑥 𝑣𝑣 [ 𝐹𝐹−1 ] 𝑣𝑣,𝜇𝜇 𝑥𝑥𝜇𝜇 . 𝑣𝑣 ,𝜇𝜇

Thus asymptotically, the prediction error grows linearly with the distance from the experimental design points. Conventional experimental design aims at maximizing a scalar measure of the Fisher matrix as a function of the experimen t al design points. In this work, we pursue a two-fold strategy: In a first step, the prediction error as a function of the design variables is taken as an additional objective function in a MCO setting. This allows to restrict the range of the design variables such that in a second step, an experimental design can be obtained within this region by maximizing a scalar measure like the determinant of the Fisher matrix derived from a set of new experimental design points.

(1)

We assume that the model parameters are fixed to values 𝛼𝛼𝜇𝜇∗ , 𝜇𝜇 = 1, … , 𝑁𝑁𝑝𝑝𝑝𝑝𝑝𝑝 that have been found by a least-square regression, i.e. = arg min𝛼𝛼1 ,…,𝛼𝛼𝑁𝑁

,

𝑣𝑣 =1

∆𝑦𝑦𝑙𝑙𝑙𝑙𝑙𝑙 reads:

2.1 Prediction Error

𝑁𝑁𝑒𝑒𝑒𝑒𝑒𝑒

𝛼𝛼∗1,…,𝛼𝛼∗𝑁𝑁 𝑝𝑝𝑝𝑝𝑝𝑝

𝑞𝑞𝑛𝑛,𝑚𝑚 is the 𝛼𝛼-quantile of the Fisher F distribution with 𝑛𝑛 and 𝑚𝑚 degrees of freedom and 𝛼𝛼 = 0.05, and the gradients ∇𝜶𝜶 𝑦𝑦 denote the derivative vector of 𝑦𝑦 with respect to the parameters 𝛼𝛼𝜇𝜇 , evaluated at 𝛼𝛼𝜇𝜇∗ . Thus the prediction error is composed of two contributions: On the one hand, the dependence on 𝐹𝐹−1 , which is a measure of the covariance of the 𝛼𝛼𝜇𝜇∗ . On the other hand, the dependence on the design point 𝑥𝑥1 , … , 𝑥𝑥 𝑁𝑁𝑑𝑑𝑑𝑑𝑑𝑑 induced by the gradient ∇𝜶𝜶 𝑦𝑦. To get an intuitive understanding of ∆𝑦𝑦, let us consider a linear model 𝑦𝑦𝑙𝑙𝑙𝑙𝑙𝑙 with 𝑁𝑁𝑑𝑑𝑑𝑑𝑑𝑑 = 𝑁𝑁𝑝𝑝𝑝𝑝𝑝𝑝 , such that 𝑁𝑁𝑝𝑝𝑝𝑝𝑝𝑝 𝑦𝑦𝑙𝑙𝑙𝑙𝑙𝑙 = ∑ 𝛼𝛼𝑣𝑣 𝑥𝑥 𝑣𝑣 . Then the corresponding prediction error

In this section the methods are described which will be applied afterwards to an example. This method section starts with a short account on the quantification of the prediction error obtaines from a least-square regression, before the frameworks of multicriteria optimization and online decision support are sketched. This section ends with some remarks on model-based experimental design.

( 𝛼𝛼1∗ , … , 𝛼𝛼𝑁𝑁∗ 𝑝𝑝𝑝𝑝𝑝𝑝 )

𝜕𝜕𝜕𝜕(𝑥𝑥̃𝑗𝑗,1 , … , 𝑥𝑥̃𝑗𝑗,𝑁𝑁𝑑𝑑𝑑𝑑𝑑𝑑 ) 𝜕𝜕𝜕𝜕(𝑥𝑥̃𝑗𝑗,1 , … , 𝑥𝑥̃𝑗𝑗,𝑁𝑁𝑑𝑑𝑑𝑑𝑑𝑑 ) ] 𝜕𝜕𝛼𝛼𝑣𝑣 𝜕𝜕𝛼𝛼𝜇𝜇

= ∑[

2. METHODS

𝑦𝑦 = 𝑦𝑦(𝛼𝛼1 , … , 𝛼𝛼𝑁𝑁𝑝𝑝𝑝𝑝𝑝𝑝 ; 𝑥𝑥1 , … , 𝑥𝑥𝑁𝑁𝑑𝑑𝑑𝑑𝑑𝑑 ).

(3)

(2)

∑ ( 𝑦𝑦̃𝑗𝑗 − 𝑦𝑦(𝑥𝑥̃𝑗𝑗,1 , … , 𝑥𝑥̃𝑗𝑗,𝑁𝑁𝑑𝑑𝑑𝑑𝑑𝑑 ))2 .

2.2 Multicriteria Optimization

𝑗𝑗 =1

Experiments serve as a starting point to build a model, or as a benchmark for model validations. From the model, predictions can be made, with an error bar which can be calculated as sketched in the previous section.

Here we skipped for brevity the parametric dependence of 𝑦𝑦. Thus the error of the design coordinates is neglected. If the error of the measured responses is assumed to be normally distributed with mean zero and standard deviation 𝜎𝜎𝑦𝑦 , then confidence regions of the 𝛼𝛼1∗ , … , 𝛼𝛼𝑁𝑁∗ 𝑝𝑝𝑝𝑝𝑝𝑝 are obtained by standard statistical arguments, cf. Bates et al (2007). Fro m

In practical applications, models are used to improve processes by mathematical optimization algorithms, which avoids costly and time-consuming trial-and-error procedures on the process itself. In many practical situations, these are multicriteria 2

Proceedings of the 9th MATHMOD Vienna, Austria, February 21-23, 2018

M. Bortz et al. / IFAC PapersOnLine 51-2 (2018) 747–752

optimization problems, because not only one, but a set of objective functions is needed to distinguish “bad” designs from “good” ones. Generally, these objective functions encode cost and quality measures: One aims at reducing expenditures while at the same time maintaining high quality stand ards. These process-related objectives are competing. Therefore, not a single optimal solution exists, but rather a set of optimal compromises, the Pareto frontier. The Pareto frontier is defined by all points in objective space where no further improvement in one objective can be realized without worsening the value of at least one other objective.

constraints can be included. Convex and non-convex regions are identified by convexity checks , where the convexity conditions are tested for some convex combinations of design variables, cf. Bortz et al (2014) . 2.3 Navigation Due to the adaptive scalarization scheme, the Pareto points calculated by the methods described in the previous section can be readily used as interpolation points in order to calculate restrictions in design space which are obtained due to a restriction set in objective space. Therefore, denote the coordinates of the 𝑗𝑗-th Pareto point by 𝑧𝑧𝑗𝑗,𝑘𝑘 , where 𝑧𝑧𝑗𝑗,𝑘𝑘 = 𝑦𝑦𝑗𝑗,𝑘𝑘 for 𝑘𝑘 = 1, … , 𝑁𝑁𝑜𝑜𝑜𝑜𝑜𝑜 and 𝑧𝑧𝑗𝑗,𝑘𝑘+𝑁𝑁𝑜𝑜𝑜𝑜𝑜𝑜 = 𝑥𝑥𝑗𝑗,𝑘𝑘 for 𝑘𝑘 = 1, … , 𝑁𝑁𝑑𝑑𝑑𝑑𝑑𝑑 ,

Thus knowledge of the Pareto frontier is of high relevance to the designers of a process. The value of the Pareto frontier is higher the more reliable the model predictions are. We now make the assumption that each of the objectives 𝑦𝑦𝑘𝑘 (𝑥𝑥1 , … , 𝑥𝑥 𝑁𝑁𝑑𝑑𝑑𝑑𝑑𝑑 ), 𝑘𝑘 = 1, … , 𝑁𝑁𝑜𝑜𝑜𝑜𝑜𝑜 is obtained as an output of a regression model as in Equation (2). That is, each objective is evaluated for the same point in design space, but has different parameters. Then it is possible to evaluate the corresponding prediction error ∆𝑦𝑦𝑘𝑘 (𝑥𝑥 1 , … , 𝑥𝑥 𝑁𝑁𝑑𝑑𝑑𝑑𝑑𝑑 ) as in Equation (3). Evaluating the prediction errors for a Pareto frontier obtained from process-related objectives then gives a reliable measure on the confidence intervals in the different objectives. Additionally, it is possible to include the minimization of the prediction errors themselves into the multicriteria optimization. This allows to discover designs that may reduce the prediction error while at the same leading to small tradeoffs with respect to the purely process -related Pareto frontier.

and 𝑗𝑗 = 1, … , 𝑁𝑁𝑃𝑃 runs over all Pareto points calculated before by the adaptive scalarization scheme. Assume that an active upper restriction 𝑢𝑢 𝑙𝑙 in the 𝑙𝑙-th objective is set. To update the corresponding restrictions in the other objectives and design coordinates, the following linear optimization problems are solved: 𝑁𝑁𝑃𝑃 𝑗𝑗 𝑧𝑧𝑗𝑗,𝑘𝑘 ∀𝑘𝑘 = 1, … , 𝑁𝑁𝑜𝑜𝑜𝑜𝑜𝑜 + 𝑁𝑁𝑑𝑑𝑑𝑑𝑑𝑑 , min 𝑘𝑘 s.t. 𝑘𝑘 = ∑𝑗𝑗=1 𝑙𝑙 < 𝑢𝑢 𝑙𝑙 ,

as well as the analogous maximization problems, 𝑁𝑁𝑃𝑃

max 𝑘𝑘 s.t. 𝑘𝑘 = ∑ 𝑗𝑗 𝑧𝑧𝑗𝑗,𝑘𝑘 ∀𝑘𝑘 = 1, … , 𝑁𝑁𝑜𝑜𝑜𝑜𝑜𝑜 + 𝑁𝑁𝑑𝑑𝑑𝑑𝑑𝑑

𝑙𝑙 < 𝑢𝑢 𝑙𝑙

𝑑𝑑𝑑𝑑𝑑𝑑

𝑘𝑘=1

for the weighted sum scalarization with weights 𝑤𝑤𝑘𝑘 and

max 𝜑𝜑 ,𝑥𝑥1,…,𝑥𝑥𝑁𝑁

𝑧𝑧𝑘𝑘 ∈ [𝑘𝑘 , 𝑘𝑘 ], 𝑘𝑘 = 1, … , 𝑁𝑁𝑜𝑜𝑜𝑜𝑜𝑜 + 𝑁𝑁𝑑𝑑𝑑𝑑𝑑𝑑 .

𝑑𝑑𝑑𝑑𝑑𝑑

(6)

The error made by this convex combination in objective space is restricted by the known and predefined error made by approximating the true Pareto boundary between the Pareto points by linear interpolation. In case of a linear dependence of the model outputs on the design variables the restrictions obtained by the above scheme in design space are exact , otherwise, the error depends on the type of nonlinearity. In any case, these errors due to the interpolation can always be made smaller than the prediction error by choosing a sufficiently high accuracy in the adaptive scalarization schemes, which therefore can be regarded as the dominant error source when calculating the Pareto points.

𝑁𝑁𝑜𝑜𝑜𝑜𝑜𝑜

∑ 𝑤𝑤𝑘𝑘 𝑦𝑦𝑘𝑘 (𝑥𝑥 1 , … , 𝑥𝑥 𝑁𝑁𝑑𝑑𝑑𝑑𝑑𝑑 )

𝑗𝑗=1

𝑁𝑁𝑃𝑃 𝑗𝑗 = 1 with all 𝑗𝑗 > 0. Then the where always and ∑𝑗𝑗=1 values 𝑧𝑧𝑘𝑘 , 𝑘𝑘 = 1, … , 𝑁𝑁𝑜𝑜𝑜𝑜𝑜𝑜 + 𝑁𝑁𝑑𝑑𝑑𝑑𝑑𝑑 for each objective and each optimization variable which are obtained by convex combination of the Pareto points fulfill:

Obtaining the Pareto frontier is a non-trivial task, especially in higher dimensions. Here, we use an adaptive scalarizatio n scheme according to Bortz et al (2014). This adaptive scheme consists in an efficient determination of scalarizatio n parameters. These are determined such that only a min imal number of Pareto points is calculated so that a linear interpolation between these points can be performed with a known, predefined error with respect to the unknown Pareto frontier located between these points. The above mentioned scalarization parameters include both weight vectors for weighted sums, sampling the convex part of the Pareto frontier, and direction vectors for the Pascoletti-Serafin i scalarization, which allow to calculated Pareto points in the non-convex regions of the Pareto boundary. The corresponding optimization problems read min𝑥𝑥1 ,…,𝑥𝑥𝑁𝑁

749

(5)

𝜑𝜑

𝑠𝑠. 𝑡𝑡. 𝑟𝑟𝑘𝑘 + 𝜑𝜑𝑑𝑑 𝑘𝑘 > 𝑦𝑦𝑘𝑘 (𝑥𝑥1 , … , 𝑥𝑥 𝑁𝑁𝑑𝑑𝑑𝑑𝑑𝑑 ) ∀𝑘𝑘 = 1, … , 𝑁𝑁𝑜𝑜𝑜𝑜𝑜𝑜

2.4 Model-based experimental design Once restrictions in design space have been obtained according to Equation (5) from the above sketched interpolation scheme, an optimal experimental plan in this hyperbox is obtained by maximizing the determinant of the

for the Pascoletti Serafini scalarization with a reference point having coordinates 𝑟𝑟𝑘𝑘 on the dominated side of the Pareto boundary and a direction vector with components 𝑑𝑑 𝑘𝑘 pointing from the reference point towards the ideal point. For both scalarization schemes, additional equality and inequality 3

Proceedings of the 9th MATHMOD 750 Vienna, Austria, February 21-23, 2018

M. Bortz et al. / IFAC PapersOnLine 51-2 (2018) 747–752

Fischer matrix, i.e. the following optimization problem is adressed: max 𝑥𝑥1 ,…,𝑥𝑥𝑁𝑁

𝑑𝑑𝑑𝑑𝑑𝑑

Table 1. Optimal model parameters

det 𝐹𝐹,

Parameter

Half width of confidence interval 𝛼𝛼1,1 1.2 ∙ 10−2 4 ∙ 10 −2 𝛼𝛼1,2 1. 0.1 𝛼𝛼1,3 −6.8 ∙ 10−3 2 ∙ 10 −2 𝛼𝛼1,4 −5.2 ∙ 10−1 3 ∙ 10 −2 −3 𝛼𝛼2,1 −1.∙ 10 5 ∙ 10 −2 −2 𝛼𝛼2,2 1.1 ∙ 10 0.3 𝛼𝛼2,3 7.3 ∙ 10−2 1.2 ∙ 10 −2 𝛼𝛼2,4 1.1 0.5 The results show that only 𝛼𝛼1,2 , 𝛼𝛼1,4 , 𝛼𝛼2,4 can be estimated with some reliability. Model reduction techniques are known in literature, see for example Schittkowski (2007). However, this is not the focus of this work, so that we will continue workin g with the above model in the following.

where the elements of 𝐹𝐹 are given in Equation (4) with 𝑥𝑥̃ replaced by 𝑥𝑥 and the box constraints form Equation (5) are respected in design space. This completes the summary of methods used in this work. The next section is dedicated to illustrate the whole workflow based on a toy example. 3. ILLUSTRATIVE EXAMPLE To illustrate the above procedure, we set an examp le consisting of 40 data points in a two-dimensional design space with coordinates 𝑥𝑥1 , 𝑥𝑥 2 , and 40 pseudo-measured data in a twodimensional response space with coordinates 𝑦𝑦1 , 𝑦𝑦2 . Half of the data points is chosen such that 𝑥𝑥 1 ~𝑁𝑁 (0.5,0.25) , 𝑥𝑥 2 ~𝑁𝑁(0,0.1) and the other half according to 𝑥𝑥 1 ~𝑁𝑁 (0.5,0.025) , 𝑥𝑥 2 ~𝑁𝑁(0,1). The values for the responses 𝑥𝑥2 are obtained from 𝑦𝑦1 = 𝑥𝑥 1 (1 − 2 ) (1 + 𝜀𝜀𝑥𝑥 ), 𝑦𝑦2 = 𝑥𝑥1 𝑥𝑥2 (1 +

A multicriteria optimization (MCO) is now set up with the objectives to maximize the model outputs 𝑦𝑦1 , 𝑦𝑦2 and to minimize the prediction error of 𝑦𝑦2 , denoted by ∆𝑦𝑦2 as in Equation (3), as a function of the design parameters 𝑥𝑥 1 , 𝑥𝑥 2 with 𝑥𝑥 1 ∈ [0,1] and 𝑥𝑥 2 ∈ [0,1.5]. For the sake of a better representation of the results in a two-dimensional chart, the MCO is done using an adaptive sandwiching scheme in the two-dimensional 𝑦𝑦1 , 𝑦𝑦2 -space. The third objective is taken account of by an 𝜀𝜀-constrained method. Following Equation (5), the following optimization problems are then solved:

2

𝜀𝜀𝑦𝑦 ), where 𝜀𝜀𝑥𝑥 ~𝑁𝑁 (0.05, 0.1) and 𝜀𝜀𝑦𝑦 ~𝑁𝑁(0.08, 0.15). Figure 1 shows plots of the data. 2

1

x2

0

max 𝑥𝑥1 ,𝑥𝑥2 𝑤𝑤1 𝑦𝑦1 ( 𝑥𝑥1 , 𝑥𝑥 2 ) + 𝑤𝑤2 𝑦𝑦2 (𝑥𝑥 1 , 𝑥𝑥 2 ) 𝑠𝑠. 𝑡𝑡. ∆𝑦𝑦2 < 𝑏𝑏

-1 -2 -0,05

0,15

0,35

0,55

Value

0,75

(7)

with 𝑏𝑏 = 0.01, 0.02, 0.05, 0.1, ∞. As explained in the previous section, the weights are chosen according to an adaptive scheme. The results of the optimization are shown in Figure 2,

x1 1,5 1

2

y2

0,5 0

1,5

-0,5

1

-1

-0,5

0

0,5

1

y2

-1

y1

0,5 0

Figure 1: Plot of pseudo-data in design space (above) and in response space (below).

-0,5

Motivated by these data, the following model ansatz is used: 𝑦𝑦1 = 𝛼𝛼1,1 + 𝛼𝛼1,2 𝑥𝑥 1 + 𝛼𝛼1,3 𝑥𝑥 2 + 𝛼𝛼1,4 𝑥𝑥 1 𝑥𝑥22 𝑦𝑦2 = 𝛼𝛼2,1 + 𝛼𝛼2,2 𝑥𝑥1 + 𝛼𝛼2,3 𝑥𝑥2 + 𝛼𝛼2,4 𝑥𝑥1 𝑥𝑥2 . Thus there are eight model parameters which are obtained according to Equation (2). The resulting values are given in Table 1. Optimal model parameters together with their 95% confidence intervals.

-1 -0,5

4

0

y1

0,5

1

Figure 2: Pareto points in objective space. The values of the third objective, ∆𝑦𝑦2 , as well as those of ∆𝑦𝑦1 , are given by the error bars, unless they are hidden by the symbols . Filled symbols are Pareto points (: 𝑏𝑏 = 0.01, : 𝑏𝑏 = 0.02, : 𝑏𝑏 = 0.1, : 𝑏𝑏 = ∞) with different values of the upper boundary 𝑏𝑏

Proceedings of the 9th MATHMOD Vienna, Austria, February 21-23, 2018

M. Bortz et al. / IFAC PapersOnLine 51-2 (2018) 747–752

according to Equation (7). Open triangles denote the pseudodata.

751

case, according to Equation (6), this leads to box constraints in design space. The box has a finite volume since the Pareto points for 𝒃𝒃 = 𝟎𝟎. 𝟏𝟏 are included into the interpolation on the convex hull leading to (6). This box now constitutes the definition range for a new experimental design.

Additionally, the prediction error ∆𝑦𝑦1 is calculated, but not used as an objective. As can be observed from the error bars in Figure 2, it turns out that ∆𝑦𝑦1 is strongly correlated with ∆𝑦𝑦2 and approximately one order of magnitude smaller. Therefore, ∆𝑦𝑦1 is not considered further.

2 1

x2

As can be seen from Figure 2, maximizing 𝑦𝑦1 , 𝑦𝑦2 drives the Pareto points away from the data, towards the upper right corner in objective space. Minimizing the prediction error means placing Pareto points close to the data, which can also be observed in design space in Figure 3 below. Thus all three objectives are competing.

0

-1 -2 -0,05

0,45

0,95

x1

Figure 4: Previous (circles) and new (triangles) pseudo-data in design space. Since the model we consider is linear in the model parameters, an appropriate design is the full factorial one with four pseudoexperimental points. The design coordinates of these additional experiments are shown in Figure 4, their response values in Figure 5.

2 1,5 1

1,5

0,5

0,5 y2

x2

1 0 -0,5

0

-1

-0,5

-1,5

-1 -1

-2 -0,05

0,45

x1

-0,5

0 y1

0,5

1

Figure 5: Pseudo-data of responses due to the previous (circles) and new (triangles) experiments.

0,95

Figure 3: Pareto points in design space. The values of the third objective, ∆𝒚𝒚𝟐𝟐 , are given by the areas of the circles, scaled to the largest circle denoting a value ∆𝑦𝑦2 = 0.2. Different gray scales stand for different values of the upper bound 𝑏𝑏 = 0.01, 0.02, ∞ according to Equation (7). Small black circles are the pseudo-data.

With these additional data, the linear model is retrained, resulting in the model parameters given in Table 2. Table 2: Optimal model parameters after retraining Parameter 𝛼𝛼1,1 𝛼𝛼1,2 𝛼𝛼1,3 𝛼𝛼1,4 𝛼𝛼2,1 𝛼𝛼2,2 𝛼𝛼2,3 𝛼𝛼2,4

The above results show that the most favourable solutions in terms of large 𝑦𝑦1 , 𝑦𝑦2 also come with a high prediction error ∆𝑦𝑦2 . Instead of aiming at reducing the prediction error over the entire Pareto boundary, we rather aim at performing additional experiments such that the reliability is increased exactly there were it is needed. Without loss of generality, assume, for example, that one favors the Pareto frontier around the point 𝑃𝑃 = (0.72, 0.84) in objective space, corresponding to (1, 0.75) in design space. The values for the prediction errors in objective space are ( ∆𝑦𝑦1 , ∆𝑦𝑦2 ) = (0.02, 0.1). Now bo x constraints are defined around 𝑃𝑃 taking account of the prediction errors. This choice is left somewhat arbitrary and can incorporate the individual needs of the planner. In any

Value

7.4 ∙ 10−4 1. −4.7 ∙ 10−3 −5.2 ∙ 10−1 3.9 ∙ 10−3 −4.5 ∙ 10−2 4.2 ∙ 10−3 1.2

Half width of confidence interval 3 ∙ 10 −2 0.06 2 ∙ 10 −2 3 ∙ 10 −2 5 ∙ 10 −2 0.1 9 ∙ 10 −2 0.2

The confidence intervals for the most influential parameters 𝛼𝛼1,2 , 𝛼𝛼1,4 , 𝛼𝛼2,4 have been reduced substantially. Based on this model, again a MCO calculation is done with two objectives 5

Proceedings of the 9th MATHMOD 752 Vienna, Austria, February 21-23, 2018

M. Bortz et al. / IFAC PapersOnLine 51-2 (2018) 747–752

(maximizing 𝑦𝑦1 and 𝑦𝑦2 ), while recording the prediction errors ∆𝑦𝑦1 , ∆𝑦𝑦2 as well. The resulting Pareto frontier is compared to the original one in Figure 6. It can be seen that due to only few experiments placed around a chosen design point, it is possible to reduce significantly the prediction errors, thus arriving at a much more reliable prediction of the Pareto boundary.

planned according to a model-based DoE strategy within this range. In the case of a model linear in the parameters, this is a factorial design. ACKNOWLEDGM ENTS The authors gratefully acknowledge helpful discussions with Raoul Heese, Dimitri Nowak and Phil Süß.

It would now be possible to repeat the loop of setting up a new experimental plan at the design(s) one is most interested in, retraining the model and recalculating the Pareto b oundary with the retrained model, thus arriving at even more accurate estimates if necessary. Such a sequential design of experiments has been considered within a stochastic approach in the past, Robbins (1952). In the present context, it is worthwhile noting that even for models with a non-linear dependence on the parameters, the factorial design scheme can be a good strategy to set up a plan for additional experiments: it is fast, does not need any model derivatives, can be reduced in higher dimensions such that the number of experimental points scales only linearly with the number of design parameters. It is generally not optimal in a model-based DoE-sense, but the deviation from optimality can be expected to be small the better a linear model approximation is. A quantification of these remarks is a topic for further research.

REFERENCES Arellano-Garcia, H., Schöneberger, J. and Körkel. S. (2007) Optimale Versuchsplanung in der chemischen Verfahrenstechnik, Chem. Ing. Techn. 79 (10), 1625-163 8 Asprion, N. Blagov, S., Böttcher, R., Schwientek, J., Burger, J., von Harbou, E., and Bortz, M.; Simulation and Multicriteria Optimization under Uncertain Model Parameters of a Cumene Process, Chem. Ing. Techn. 89 (4), 1-11 Bates, D.M. and Watts, D.G., (2007) Nonlinear Regression Analysis and Its Applications, Wiley, United Kingdom Bortz, M., Burger, J., Asprion, N., Blagov, S., Böttcher, R., Nowak,U., Scheithauer, A., Welke, R., Küfer, K.-H., Hasse, H., Multi-criteria optimization in chemical process design and decision support by navigation on Pareto sets, Comp. Chem. Eng. 60, 354-363 Goos, P. and Bradley, J. (2011). Optimal Design of Experiments: A Case Study Approach, Wiley, United Kingdom Franceschini, G., and Macchietto, S. (2008) Model-based design of experiments for parameter precision: State of the art, Chem. Eng. Science 63, 4846-4872 Robbins, H. (1952), Some Aspects of the Sequential Design of Experiments, Bull. Amer. Math. Soc. 58 (5), 527-535 Schittkowski, K. (2007) Experimental Design Tools for Ordinary and Algebraic Differential Equations, Ind. Eng. Chem. Res. 46 (26) 9137-9147 Verdinelli, I. and Kadane, J.B. (1992) Bayesian Designs for Maximizing Information and Outcome, Journal of the American Statistical Association 87 (418) 510-515

y2

2 1,8 1,6 1,4 1,2 1 0,8 0,6 0,4 0,2 0 -0,2

0

0,2

0,4

0,6

0,8

1

y1 Figure 6: Pareto points based on the original () and retrained () models in objective space. The bars denote the prediction errors. The retrained model shows a significantly reduced prediction error. 4. CONCLUSION The response functions of a model that has been obtained by a regression procedure are used within a multicriteria context to predict a Pareto boundary. Inclusion of the prediction error as an additional objective yields valuable information concerning the need for additional experiments . Individually defined box constraints in objective space enclose the region of the Pareto boundary one is most interested in. These are used to obtain the corresponding box constraints in design space by solving an LP on the convex hull of the Pareto points. Thus a defining range for additional experiments is obtained, which can be 6