Quantification of model-form and predictive uncertainty for multi-physics simulation

Computers and Structures 89 (2011) 1206–1213

Contents lists available at ScienceDirect

Computers and Structures journal homepage: www.elsevier.com/locate/compstruc

Quantification of model-form and predictive uncertainty for multi-physics simulation Matthew E. Riley ⇑, Ramana V. Grandhi Department of Mechanical and Materials Engineering, Wright State University, Dayton, OH 45435, United States

a r t i c l e

i n f o

Article history: Received 1 June 2010 Accepted 1 October 2010 Available online 10 November 2010 Keywords: Uncertainty quantification Model-form uncertainty Predictive uncertainty Multi-physics simulation

a b s t r a c t Traditional uncertainty quantification in multi-physics design problems involves the propagation of parametric uncertainties in input variables such as structural or aerodynamic properties through a single, or series of models constructed to represent the given physical scenario. These models are inherently imprecise, and thus introduce additional sources of error to the design problem. In addition, there often exists multiple models to represent the given situation, and complete confidence in selecting the most accurate model among the model set considered is beyond the capability of the user. Thus, quantification of the errors introduced by this modeling process is a necessary step in the complete quantification of the uncertainties in multi-physics design problems. In this work, a modeling uncertainty quantification framework was developed to quantify to quantify both the model-form and predictive uncertainty in a design problem through the use of existing methods as well as newly developed modifications to existing methods in the literature. The applicability of this framework to a problem involving full-scale simulation was then demonstrated using the AGARD 445.6 Weakened Wing and three different aeroelastic simulation packages to quantify the flutter conditions of the wing. Ó 2010 Elsevier Ltd. All rights reserved.

1. Introduction In many multi-physics design problems, such as aeroelasticity, multiple models often exist to represent the given physical scenario. Examples of this are numerous in the variety of aeroelastic simulation packages that are available to users ranging from panel methods, such as those available in Nastran [1] or ASTROS [2], to full CFD simulations coupled with dynamic structural responses such as ZEUS [3] or Navier–Stokes solvers [4]. Often, these models produce different results for the same set of design parameters. This variation in results is due to the different assumptions that are made in the mathematical formulations of the individual models. Traditional uncertainty quantification methods in aeroelastic design involve selecting the best model among the model set being considered and then propagating the uncertain input variables through the model to calculate a non-deterministic response of the system. However, one primary flaw with this methodology in that it is beyond the capability of the designer to select with complete certainty the model which is most accurate in all areas of the design space. Although a particular model might be shown to be most accurate, or ever fully correct, at a particular point in the design space, this result cannot be inherently translated to all regions ⇑ Corresponding author. Tel.: +1 419 704 3143. E-mail addresses: [email protected] (M.E. Riley), [email protected] (R.V. Grandhi). 0045-7949/$ - see front matter Ó 2010 Elsevier Ltd. All rights reserved. doi:10.1016/j.compstruc.2010.10.004

of the design space. A simple example of this could be that although a model might be shown to be accurate in the subsonic Mach regime, there is no guarantee that it will maintain its accuracy in the transonic or supersonic Mach regimes. It is thus proposed that instead of utilizing only a single model in the uncertainty quantification of the design process, that multiple models be considered, in a systematic fashion, so that the additional uncertainty introduced to the design problem through the modeling process can be quantified and mitigated, and the total uncertainty in the problem can be completely quantified. This work introduces an uncertainty quantification framework that quantifies the uncertainties introduced through the modeling process by utilizing existing methods in the literature – with relevant modifications in particular methods – to construct a complete and accurate representation of the modeling uncertainty that is present in an analysis. 2. Background and methods 2.1. Uncertainty definition In the process of discretizing a physical scenario for modeling, assumptions are made to achieve a simplified representation of the problem. As it is beyond the capability of the designer to completely understand any true engineering problem in its full complexity, these assumptions can produce a discrepancy between

M.E. Riley, R.V. Grandhi / Computers and Structures 89 (2011) 1206–1213

1207

Nomenclature E½j Cov ½j AGARD CFD

expected value of a variable covariance of a variable Advisory Group for Aerospace Research and Development computational fluid dynamics

the physical scenario and the results produced by the model, meaning that the resulting model is merely a partial representation of reality. This discrepancy between the result of the model and the true physical scenario is referred to as predictive uncertainty [5], and the degree of this uncertainty is often a function of the ability of the model to capture the phenomena in the physical scenario of interest. It is also very common in engineering problems for multiple models to be constructed to represent the same given scenario. Guedes Soares states that in situations such as this, there exists only one correct model [6]. However, it is beyond the capability of the designer to select the model which is correct in every given situation. Thus, there exists uncertainty in the selection of the model which best represents the physical scenario of interest. This uncertainty in the identification of the best model among a model set that is being considered is referred to as model-form uncertainty [7]. To analyze the discretization of the physical scenario, parameters are defined within the models to represent aspects of the physics, such as dimensions, material properties, environmental conditions, or modeling constants. Although these parameters are often represented as deterministic values within the model, they rarely can be considered deterministic in the true physical scenario. As a result, there exists a third type of uncertainty in the modeling process – parametric uncertainty – which refers to the uncertainty inherent to the parameters that are input into a model [5]. These parametric uncertainties are commonly split into two distinct categories: aleatory and epistemic uncertainty [8]. Aleatory uncertainty is defined as the uncertainty that arises as a result of natural, unpredictable variation in the performance of the system [9]. This type of uncertainty is commonly thought of as the type of uncertainty of which enough information is known to assign probability density functions to represent the random nature of the variable. Epistemic uncertainty, on the other hand, is defined as the type of uncertainty that is due to the lack of knowledge regarding the behavior of a system that could, in theory, be resolved through the introduction of additional information [10]. Epistemic uncertainty is commonly referred to as incomplete uncertainty; or more simply put, inherent variability of which not enough is known to accurately approximate the uncertainty. Model-form, predictive, and parametric uncertainties are all present in modeling problems. Eq. (1) shows the general formulation of a modeling problem as the function of three variables, ~f i ; ~ x, and ^e:

FEM PDF BMA AFA MAFA

finite element method probability density function Bayesian model averaging adjustment factors approach modified adjustment factors approach

Although the above three types of uncertainty are defined uniquely, they are not necessarily independent of each other. Methods that exist in the literature to quantify uncertainty of a particular form – such as parametric uncertainty – are not necessary applicable, or even valid, to quantify model-form or predictive uncertainty. In aeroelastic design, extensive work has been done working on the quantification of parametric uncertainty. Kurdi et al. explored the effects of uncertainty on structural finite element parameters, such as rib and skin sizes, in the transonic aeroelastic regime using sampling based uncertainty quantification methods for aleatory uncertainties in the parameters of an aeroelastic model [11]. Ueda also explored the effects of aleatory uncertainties on calculating the sensitivities of a structure to the uncertain variables [12]. Tonon et al. further explored the uncertainties in the structural parameters in aeroelastic design, but explored the effects of considering epistemic uncertainties in the problem outside of simply uncertainties in the individual parameters using random set theory [13]. Pettit and Grandhi further quantified the epistemic uncertainty in aerodynamic parameters, such as gust loads, to perform a reliability based optimization [14]. Although much work has been done in the aeroelastic community regarding the quantification of parametric uncertainties, represented as both aleatory and epistemic, little work has been done on the quantification of the model-form and predictive uncertainty in these problems. Due to the increasing complexity of aeroelastic models being constructed, as well as the advancement of aeroelastic modeling into complex regions of the design space, it is important to quantify these additional uncertainties in order to maintain a robust design that accurately considers potential uncertainties in the design problem from all possible sources, including the modeling process itself. 2.2. Model-form and predictive uncertainty quantification methods

ð1Þ

To quantify the uncertainties introduced in a modeling process, multiple approaches have been developed that consider the results of multiple models – and the presence of any available experimental data – to develop a non-deterministic representation of a composite model. The first proposal of model combination as a method to quantify the uncertainty between models was made by Barnard [15], a rudimentary combination of multiple airline passenger models. Roberts later suggested an aggregate distribution that combines the opinions of two models using weighting factors [16]. Leamer expanded on this idea, developing the basic paradigm for what is now known as Bayesian model averaging by accounting for the uncertainty in the selection of the model itself [17].

~f i ðxÞ represents the result of a particular model, model i, to a set of input parameters, x. ^ei represents the discrepancy between the result of model i, and the true physical scenario, y. In this regard, the possibility of disagreement between multiple ~f i ðxÞ can be said to represent model-form uncertainty, the variation in ~f i ðxÞ due to uncertainties in the set of input parameters, x can be shown to represent parametric uncertainty, and ^e represents the predictive uncertainty inherent to model i.

2.2.1. Bayesian model averaging As opposed to basing a prediction of a physical scenario of interest upon the results of a singular model, Bayesian model averaging constructs a distribution for the adjusted model, Pr(y|D), as an average of the posterior distributions of each of the N models considered, weighted by its posterior model probability. If y is considered as the adjusted model of interest, then its posterior distribution given experimental data, D, is shown in Eq. (2):

y ¼ ~f i ðxÞ þ ^ei

1208

M.E. Riley, R.V. Grandhi / Computers and Structures 89 (2011) 1206–1213

PrðyjDÞ ¼

N X

PrðM i jDÞPrðyjM i ; DÞ

ð2Þ

i¼1

where Pr(Mi|D) represents the posterior model probability of model i given experimental data, D, and Pr(y|Mi, D) is the posterior distribution of model i. To calculate the posterior model probability of a particular model of interest, Bayes theory is employed, as shown in Eq. (3):

PrðMi Þ  PrðDjMi Þ PrðM i jDÞ ¼ PN j¼1 PrðM j Þ  PrðDjM j Þ

ð3Þ

where PrðMi Þ represents the probability of model Mi prior to observation of data, D, and PrðDjMi Þ represents the marginal likelihood of model Mi . In Bayesian model averaging, the individual model probabilities are assumed to be equal in the absence of additional data: i.e. PrðMi Þ ¼ 1=N 8i. The marginal likelihood of model Mi can then be calculated by the integral shown in Eq. (4):

PrðDjM i Þ ¼

Z

PrðDjx; M i ÞPrðxjMi Þdx

ð4Þ

where PrðDjx; Mi Þ is the likelihood of model i and Prð xjM i Þ is the prior density of model i. For individual models with non-deterministic outputs, the integral shown in Eq. (4) can be difficult to compute explicitly. In the case of non-deterministic models, a numerical integration technique, such as Markov Chain Monte Carlo is commonly used [18] to approximate the integral shown in Eq. (4). To calculate the posterior distribution of model i, shown in Eq. (2), the prior density of the original model must be updated to include the information introduced to the problem by the data. In the case of a deterministic model with output fi ð xÞ, the posterior distribution of model i is shown in Eq. (5):

  PrðyjM i ; DÞ ¼ Normal fi ðxÞ; r2i

Pm

2 k¼1 ik

e

ð6Þ

m

e2ik ¼ dk  fi ðxk Þ

ð7Þ

This allows for the posterior mean and variance of the adjusted model, y, to be calculated as shown in Eqs. (8) and (9). N X

PrðM i jDÞEðPrðyjM i ; DÞÞ

ð8Þ

The additive adjustment factor, is assumed to be a normal random variable with first and second moments that can be calculated as shown in Eqs. (11) and (12): N   X E Ea ¼ PðMi Þðyi  y Þ

VarðPrðyjDÞÞ ¼

ð11Þ

i¼1 N   X Var Ea ¼ PðMi Þðyi  EðyÞÞ2

PrðM i jDÞVarðPrðyjM i ; DÞÞ2 þ

i¼1

 ðEðPrðyjMi ; DÞÞ  EðPrðyjDÞÞÞ2

N X

ð12Þ

i¼1

In Eqs. (11) and (12), P(Mi) represents the model probability of model i; by definition, the probability that model i is the best model among the model set considered. This model probability is defined by expert opinions regarding the models of interest. As the model probabilities remain bounded by the laws of probability theory, constraints exist upon the values of the model probabilities (Eq. (13)).

PðM i Þ ¼ 1 such that 0 6 PðM i Þ 6 1

ð13Þ

i¼1

By definition, the models considered in the adjustment factors approach are deterministic. Thus, the expected value of the adjusted model y is equal to the sum of the deterministic result of the best model, y*, and the expected value of the additive adjustment factor (Eq. (14)).

  EðyÞ ¼ y þ E Ea

i¼1 N X

ð10Þ Ea ,

N X

where

EðPrðyjDÞÞ ¼

y ¼ y þ Ea

ð5Þ

where r2i is the variance of the random error involved in the prediction, fi(x), by model i. This variance is calculated by determining the variance of the model predictions to the m experimental data points, dk, as shown in Eqs. (6) and (7):

r2i ¼

2.2.2. Adjustment factors approach The adjustment factors approach was first demonstrated by Mosleh and Apostolakis as a method to utilize expert opinions in the absence of empirical data to quantify model-form uncertainty using Bayes’ Theorem [19]. The adjustment factors approach modifies the result of the best model – the model with the highest model probability among the model set being considered – by an adjustment factor to account for the uncertainty that exists in selecting the best model. This approach has been demonstrated on multiple engineering problems in the literature. Zio and Apostolakis used an adjustment factors approach to quantify the uncertainty in the selection of radioactive waste repository models [20] and Reinert and Apostolakis included an adjustment factors approach in the assessment of risk for decision-making processes [21]. Multiple derivatives of the adjustment factors approach exist in the literature. The difference among the approaches is with the factor used to adjust the ‘‘best” model – specifically the distribution assigned to the factor. In the additive adjustment factors approach, the adjusted model, y, is formed by adding an additive adjustment factor, Ea , to the best model of the N models considered, denoted y*, as shown in Eq. (10).

ð14Þ *

PrðM i jDÞ

i¼1

ð9Þ

The Bayesian model averaging methodology produces an adjusted model, y, which accounts for both the predictive and model-form uncertainty in the problem. However, this methodology relies upon the presence of experimental data to quantify the predictive uncertainty in the problem, and reduce the model-form uncertainty. In most engineering design cases, experimental data regarding the design of interest is often unavailable or restrictively expensive to obtain. As such, a methodology that would allow for the quantification of the modeling uncertainty in the problem without the necessity for experimental data points would be beneficial in the initial stages of design.

Similarly, as the best model, y , is deterministic, the variance of the adjustment model is shown to simply be equal to the variance of the adjustment factor (Eq. (15)).

  VarðyÞ ¼ Var Ea

ð15Þ

As the additive adjustment factor was assumed to be normally distributed, and the best model, y*, is deterministic, the adjusted model, y, is shown to be normally distributed with first and second moments given by Eqs. (14) and (15). Other derivations of the adjustment factors approach exist in the literature [20] with different assumptions made regarding the distribution of the adjustment factor, such as, for example, a lognormal assumption regarding the adjustment factor. A caveat to the adjustment factors approach is that while it does not require experimental data to quantify the model-form

M.E. Riley, R.V. Grandhi / Computers and Structures 89 (2011) 1206–1213

1209

uncertainty within the problem, it is dependent upon the quality of the expert opinions utilized to construct the model probabilities. Determining the sensitivity of the adjusted model, y, to these model probabilities would allow for the user to see determine the degree of effect that the model probabilities have upon the adjusted model. The modified adjustment factors approach was developed by Riley et al. to approximate these sensitivities for problems with deterministic models [22]. 2.2.3. Modified adjustment factors approach While the traditional adjustment factors approach treats the model probabilities, P(Mi), as deterministic values, the modified adjustment factors approach defines the individual model probabilities as normally distributed stochastic values, as defined in Eq. (16).

PðMi Þ ¼ NðPðM i Þexp ; ri Þ where :

ri ¼ min½0:05; 0:25  PðMi Þ ð16Þ

where P(Mi)exp represents the original model probability as defined based upon the expert opinions and ri is a variance applied to the model probabilities in the approach. As a result, there are now N distinct distributions of model probabilities for each of the models that are then independently sampled using Monte Carlo Sampling to obtain a set model probability values. Before these sampled values are used, though, they must be renormalized to maintain the constraints set forth in Eq. (13). After normalization, the sampled values are then used in a traditional adjustment factors approach (Eqs. (11)–(15)) to obtain an adjusted model, now referred to as yjadj . This process of sampling the model probability distributions is then repeated n times, resulting in a set of adjusted models: fy1adj ; y2adj ; . . . ; ynadj g, with each adjusted model representing the result of a different set of model probabilities. These individual adjusted models are then sampled using Markov Chain Monte Carlo sampling using a Metropolis Chain [23], with m samples. Using the m samples of the n adjusted models, a new aggregate adjusted model, ymaf, can then be constructed. After completing the modified adjustment factors approach, there now exists two adjusted models that represent the modelform uncertainty in the problem of interest: y, which uses the deterministic model probability values obtained from expert opinions and ymaf, which represents the potential variance in the prediction of the adjusted model as a result of perturbations of P(Mi). A metric must now be implemented that measures the similarity of the two models. The Bhattacharyya distance is a metric developed to measure the geometric similarity between two distinct distributions (Eq. (17)) [24].

BCðfx1 ; fx2 Þ ¼

Z

1

fx1 ðxÞ0:5 fx2 ðxÞ0:5 dx

ð17Þ

1

fx1(x) and fx2(x) represent the distributions of the two models of interest, y and ymaf. The Bhattacharyya distance is a bounded value between 0 and 1 where a value of 1 implies that the two models of interest are identically distributed. 3. Modeling uncertainty framework While approaches such as Bayesian model averaging are very powerful and capable of quantifying model-form and predictive uncertainty, they also have a necessity for experimental data points – which are not always readily available. The limited availability of these data points is even more present in the preliminary design phase, where numerous design configurations are often considered concurrently. Thus, it is infeasible to assume that experimental data points will always be readily available for quantification of model-form and predictive uncertainty. Similarly, it would be inefficient and cost restrictive to obtain experimental

Fig. 3.1. Modeling uncertainty framework.

data points at each stage of the preliminary design phase to quantify this uncertainty. To efficiently solve this problem, a framework has been proposed in this work that uses the model-form uncertainty present in the problem at the time to determine the necessity of additional experimental data points (Fig. 3.1). In this framework, the modelform uncertainty in the problem is initially quantified using the adjustment factors approach. In this approach, the model probabilities are assigned using expert opinions regarding the relative accuracy and prior validity of the models in consideration. Next, the modified adjustment factors approach is utilized to calculate the sensitivity of the adjusted model’s response to the individual model probabilities. As a result of these two approaches, two different adjusted models will be created, y and ymaf. The Bhattacharyya distance will then be calculated between these two distributions to determine the similarity of the adjusted models. If the Bhattacharyya distance is shown to be less than a critical value – denoting a significant variance between the two adjusted models – then the adjusted model y can be considered sensitive to the model probabilities assigned through expert opinions. As such, the model-form uncertainty in the problem could then be further reduced through the introduction of experimental data points. In addition, the introduction of these data points will allow for the quantification of the predictive uncertainty in the problem as well. This further quantification will then be performed using Bayesian model averaging, with experimental data points now obtained for the problem. By utilizing a framework such as the one proposed in this work, the modeling uncertainty in the problem can still be quantified and reduced to an acceptable amount at minimal experimental cost. Instead of performing a blanket number of experiments at various data points and configurations, or even using traditional design of experiments methodology, this framework utilizes the modeling uncertainty itself to drive the necessity of further experimental data points. 4. Demonstration and results 4.1. AGARD Wing background To demonstrate the applicability of the modeling uncertainty framework on a full scale multi-physics problem, the flutter analysis of an AGARD Standard 445.6 Weakened Wing will be investigated. The AGARD Standard 445.6 Weakened Wing is a tapered wing with a 45° back-sweep, an aspect ratio of 6, and a NASA

1210

M.E. Riley, R.V. Grandhi / Computers and Structures 89 (2011) 1206–1213

Fig. 4.1. Planform view of AGARD 445.6 Wing.

Table 4.1 Experimental natural frequencies of AGARD 445.6 Wing.

Mode Mode Mode Mode

1 2 3 4

Experimental frequency (Hz) [25]

Mode shape

9.60 38.10 50.70 98.50

1st Bending 1st Torsion 2nd Bending 2nd Torsion

64A004 airfoil [25]. The wing has a span of 2.50 ft with a planform shape seen in Fig. 4.1, and measured natural frequencies shown in Table 4.1. The aeroelastic flutter behavior of the AGARD 445.6 Wing has been explored and addressed in great detail in the literature in both the experimental and simulation field. Yates explored the transonic flutter of the airfoil through experimental measures by developing wing tunnel data for a model with both standard atmospheric and Freon-flow conditions [26]. Initial studies in CFD simulation by Rausch et al. [27] and Lee-Rausch and Batina [28] using Euler methods experienced errors in the prediction of flutter conditions when compared to experimental data. Lee-Rausch et al. then studied the wing using an unsteady Navier–Stokes code (CFL3D) to further investigate the initial discrepancies that were found between the Euler investigations and the experimental data [29]. It was found that both the Euler and Navier–Stokes codes initially experienced much greater success at predicting the flutter boundary of the wing in the upper transonic regime (M = 0.96) than in the transonic/supersonic border region (M = 1.141). Further analysis by Liu et al. demonstrated the capture of the transonic flutter dip phenomenon through the use of a coupled CFD-CSD method [30].

sonic aeroelastic problem, this model is to be considered in this example to demonstrate the possibility of having an erroneous model being included in the model set. ZTAIC is a nonlinear method that utilizes a transonic equivalent strip method with the steady pressure input of an external CFD program–in the case of this work, CAPTSD, a small-disturbance transonic code was used. The third software package of interest, ZEUS is a program developed by Zona Technology as well that utilizes a Euler unsteady aerodynamic solver on a Cartesian grid using cell-centered finite volume [3]. Similar to ZONA6 and ZTAIC, ZEUS requires the input of externally computed structural free vibration solutions. Each of the three models was constructed with identical structural and aerodynamic parameters to replicate the experimental data available in the literature [26]. An overview of these parameters can be seen below in Table 4.2. As mentioned, the aeroelastic modeling packages of interest require a representation of the structural components of the wing using FEM. For this, a Nastran structural model of the AGARD Wing was first constructed that could first be validated against the experimental data and then be used with the modeling packages to perform the aeroelastic analysis. Table 4.3 shows the comparison of the frequencies of the Nastran structural model to the experimental data points published by Yates [26]. It can be seen in Table 4.3 that the structural model’s dynamic response demonstrates an acceptable level of agreement with the published experimental results. Although the higher frequencies, such as the third and fourth frequencies, show approximately 2– 4% disagreement, the contribution of these frequencies to the aerodynamic flutter phenomenon is less than the lower frequencies. In addition, as will be shown in the analysis, the first and second mode shapes are the primary modes contributing to the aerodynamic flutter, so agreement between the model and the physical scenario for these two frequencies is most critical. In the wind tunnel tests conducted by Yates at NASA Langley [26], the results were presented in terms of the flutter velocity coefficient (Eq. (18)):

Vf ¼

U pffiffiffiffi bs xa l

ð18Þ

where in Eq. (18), U refers to the flutter velocity of the wing, bs is the streamwise semi-chord at the root of the wing, xa is the circular natural frequency of the wing, and l is a term representing the relative density of the wing, calculated as shown below in Eq. (19):

l¼

m

ð19Þ

qV

m is defined as the mass of the wing, q is the density of the air at the flutter velocity, U, and V is the volume of a conical frustrum with the streamwise root chord as a lower base diameter, the streamwise tip chord as the upper base diameter, and the span of the wing panel as the height.

4.2. Model definitions and parameters For analysis in the scope of the problem, models for the AGARD 445.6 Wing were constructed in three aeroelastic software packages: ZONA6, ZTAIC (CAPTSD), and ZEUS. ZAERO is a Zona Technology program that imports externally computed structural free vibration solutions from an external structural finite element code [31]. Contained within this package are two solution methods of interest, ZONA6 Linear Method and ZTAIC. The ZONA6 Linear Method applies a linear subsonic solution method based upon the K and PK-Methods. Although the problem of interest is a tran-

Table 4.2 Structural and aerodynamic parameters of three aeroelastic models considered. Panel span Aspect ratio Streamwise semichord Wing mass Sweep angle Mach number

2.50 [ft] 1.6525 0.9165 [ft] 0.12764 [slugs] 45° 0.954

1211

M.E. Riley, R.V. Grandhi / Computers and Structures 89 (2011) 1206–1213 Table 4.3 Frequency comparison of structural model of AGARD 445.6 Weakened Wing.

Mode Mode Mode Mode

1 2 3 4

(Hz) (Hz) (Hz) (Hz)

Table 4.6 Distribution parameters for probabilistic adjustment factors approach.

Nastran model

Experimental [26]

Mode shape

9.57 38.17 48.35 91.55

9.60 38.10 50.70 98.50

1st Bending 1st Torsion 2nd Bending 2nd Torsion

Table 4.4 Results of three simulation models. Solver

Vf

ZEUS g-Method ZONA6 Linear Method ZTAIC (CAPTSD)

0.3086 0.3460 0.3170

Additive AFA Modified AFA Model disagreement Bhattacharyya measure

Mean [ft/s]

Standard dev.

0.32089 0.32037 0.162%

0.01494 0.01261 18.477% 0.99183

Continuing in the framework, the modified adjustment factors approach was applied to the problem, constructing a second model shown in Eq. (21).

ymfa ¼ normð0:32037; 0:01361Þ

ð21Þ

For this analysis, the uncertainty in the prediction of the flutter coefficient as a result of model-form and predictive uncertainty was explored. The three models discussed in the prior section were run with identical parameters – as listed in Table 4.2. This resulted in three deterministic solutions for the flutter velocity coefficient of the wing shown in Table 4.4. It can be seen in Table 4.4 that there is approximately 10% disagreement between the models’ prediction for the flutter velocity coefficient, Vf. As a result, uncertainty analyses done with each of the packages would result in dramatically different predictions for the reliability of the system. In addition, while prior design methods would select the model considered to be most accurate, and use that model for the analysis, it is beyond the capability of the user to pick the model which will be most accurate unless all data is known – in which case there would not be a need for simulation [17]. Thus, the three individual results of the models are implemented in the proposed modeling uncertainty framework to completely quantify the model-form and predictive uncertainty in the problem.

An overview of these two adjusted models can be seen in Table 4.6, and the distribution of the flutter coefficient from the two models can be seen in Fig. 4.2. As seen in Table 4.6, the Bhattacharyya measure for this problem is 0.99183, which is less than the critical value of 0.995 that was assigned for this problem. As a result of this, the adjusted model (Eq. (19)) is shown to be sensitive to the model probabilities shown in Table 4.5. The next step in the modeling uncertainty framework is then to obtain experimental data points for use in Bayesian model averaging. For the parameters listed in Table 4.2, wind tunnel tests were run at NASA to calculate the flutter velocity coefficient for the AGARD 445.6 Weakened Wing [26]. Yates showed that the experimental value for this flutter velocity coefficient was 0.3059. As this experiment was not repeated, there is only one experimental data point to be incorporated into the BMA methodology. However, if additional repetitions of the test had been done, all experimental values could have been utilized. Bayesian model averaging was first used to update the model probabilities from their uniformly distributed values – as defined by BMA – to the values shown in the final column of Table 4.7. Using these updated model probabilities in conjunction with the Bayesian model averaging approach yields an adjusted model shown in Eq. (22).

4.4. Implementation in modeling uncertainty framework

ybma ¼ normð0:31210; 0:01360Þ

4.3. Simulation results

As seen in Fig. 3.1, the first step in the modeling uncertainty framework is to input the individual model results, as well as model probabilities formed based on expert opinions, into the adjustment factors approach. Table 4.5 shows the results of the three simulation packages, as well as the model probabilities that were assigned. The model probabilities assigned in this problem are based upon the relative fidelity of the modeling package of interest. The linear panel method, ZONA6, is assigned the lowest model probability due to the transonic nature of the problem while the Euler CFD method, ZEUS, is assigned the highest model probability due to its application for transonic flutter calculations. Using the values in Table 4.5 for the adjustment factors approach, an adjusted model was found for the flutter velocity coefficient, as shown in Eq. (20).

y ¼ normð0:32089; 0:01494Þ

ð20Þ

ð22Þ

Plotting the distribution of this new adjusted model against the original adjusted model calculated by the adjustment factors approach (Eq. (20)) demonstrates two points of interest (Fig. 4.3): First, it can be noticed that the mean value of the flutter velocity coefficient for the adjusted model has shifted down from 0.32089 to 0.31210. This shift in the mean is attributed to the quantification of the predictive uncertainty in the problem. As the experimental value obtained in this problem of 0.3059 is less than the values of each of three models considered, it is logical for this mean to shift downward when this experimental value is introduced. The second note that can be made is that the standard deviation of the adjusted model was reduced from 0.01494 to 0.01360 with the introduction of the data. This reduction in variance is attributed to reduction of the model-form uncertainty in the problem by updating the model probabilities based upon the new data made available. 5. Conclusions

Table 4.5 Model probabilities. Solver

Vf

P(Mi)prior

ZEUS g-Method ZONA6 Linear Method ZTAIC (CAPTSD)

0.3086 0.3460 0.3170

0.40 0.25 0.35

This work addresses two less-considered sources of uncertainty in multi-physics design problems: model-form and predictive uncertainty. A framework was constructed using existing methods in the literature, such as the adjustment factors approach and Bayesian model averaging, as well as modifications to these methods that were made specific to this work, such as the modified

1212

M.E. Riley, R.V. Grandhi / Computers and Structures 89 (2011) 1206–1213

Fig. 4.2. Flutter coefficient prediction of two adjustment factors approaches.

Table 4.7 Bayesian updated model probabilities. Solver

vf

P(Mi)prior

P(Mi)post

ZEUS g-Method ZONA6 Linear Method ZTAIC (CAPTSD)

0.3086 0.3460 0.3170

0.33 0.33 0.33

0.7630 0.0514 0.1856

adjustment factors approach, that allows for the complete quantification of modeling uncertainty in these design problems. Due to the expense of experimental data, the framework utilizes the model-form uncertainty to drive the necessity of any additional data points to be introduced into the design process. This yields a methodology that instead of spanning regions of the design space with

an assortment of data points, concentrates the data points on the locations that are most beneficial for the overall reduction of the uncertainty in the problem. It was shown in the application problem that the framework can be applied to full-scale problems involving complex simulations. The variation in the prediction of the flutter velocity coefficient of the AGARD 445.6 Weakened Wing due to model-form and predictive uncertainty was quantified using the results of three commercially available aeroelastic packages, ZONA6, ZTAIC, and ZEUS. The modified adjustment factors approach showed that the introduction of additional experimental data to the framework could result in a reduction of the modeling uncertainty, which was seen in the results. As a result of this framework, an adjusted model was constructed to represent the variation in the flutter

Fig. 4.3. Flutter coefficient prediction adjustment factors approach and Bayesian model averaging.

M.E. Riley, R.V. Grandhi / Computers and Structures 89 (2011) 1206–1213

velocity coefficient of the wing that considers the variation between the models that were considered in the analysis, and the experimental data points that were made available during the analysis, resulting in a complete quantification of the uncertainty that was introduced during the modeling process itself. Acknowledgements The acknowledge the support of the United States Air Force through Contract FA8650-09-2-3938, Collaborative Center for Multidisciplinary Sciences. References [1] NASTRAN user’s manual, r3. Santa Ana, CA: MSC Software Corporation; 2005. [2] ASTROS user’s reference manual, Version 20. Torrance, CA: Universal Analytics; 1997. [3] ZEUS user’s manual. Scottsdale, AZ: ZONA Technology; 2009. [4] Schuster DM, Beran PS, Huttsell LJ. Application of the ENS3DAE Euler/Navier– Stokes aeroelastic method. AGARD 1998. [5] Droguett EL, Mosleh A. Bayesian methodology for model uncertainty using model performance data. Risk Anal 2008;28(5):1457–76. [6] Guedes Soares C. Quantification of model uncertainty in structural reliability. In: Guedes Soares C, editor. Probabilistic methods for structural design. Netherlands: Kluwer Academic Publishers; 1997. [7] Zhang R, Mahadevan S. Model uncertainty and Bayesian updating in reliabilitybased inspection. Struct Saf 2000;22:145–60. [8] Pate-Cornell ME. Uncertainties in risk analysis: six levels of treatment. Reliab Syst Saf 1996;54:95–111. [9] Hacking J. The emergence of probability. Cambridge, UK: University Press; 1975. [10] Chernoff H, Moses LE. Elementary decision theory. New York: Wiley; 1959. [11] Kurdi M, Lindsley N, Beran P. Uncertainty quantification of the Goland Wing’s flutter boundary. In: AIAA atmospheric flight mechanics conference and exhibit, Hilton Head, SC. AIAA; 2007. [12] Ueda T. Aeroelastic Analysis considering structural uncertainty. In: Proceedings of the sixth international seminar on ‘‘recent research and design progress in aeronautical engineering and its influence on education”, Riga, Latvia; 2005. p. 3–7.

1213

[13] Tonon F, Bae H, Grandhi RV, Pettit C. Using random set theory to calculate reliability bounds for a wing structure. Struct Infrastruct Eng 2006;2:191–200. [14] Petit CL, Grandhi RV. Optimization of a wing structure for gust response and aileron effectiveness reliability. AIAA J Aircraft 2004;40:1185–91. [15] Barnard GA. New methods of quality control. J Royal Stat Soc 1963(Series A):255–67. [16] Roberts HV. Probabilistic prediction. J Am Stat Assoc 1965;60:50–62. [17] Leamer EE. Specification searches: ad hoc inference with no experimental data. New York: Wiley & Sons; 1978. [18] Gilks WR, Richardson S, Spiegelhalter DJ. Markov Chain Monte Carlo in practice. Bungay: Chapman & Hall; 1996. [19] Mosleh A, Apostolakis G. The assessment of probability distributions from expert opinions with an application to seismic fragility curves. Risk Anal 1986;6(4):447–61. [20] Zio E, Apostolakis G. Two methods for the structural assessment of model uncertainty by experts in performance assessments of radioactive waste repositories. Reliab Eng Syst Saf 1996;54(2-3):225–41. [21] Reinert JM, Apostolakis GE. Including model uncertainty in risk-informed decision making. Ann Nucl Energy 2006;33(4):354–69. [22] Riley ME, Grandhi RV, Kolonay R. Quantification of modeling uncertainty in aeroelastic design. In: 51st AIAA/ASME/ASCE/AHS/ASC structures, structural dynamics, and materials conference, Orlando, FL; 2010. AIAA. [23] Petit CL. Uncertainty quantification in aeroelasticity: recent results and research challenges. J Aircraft 2004;41(5):1217–29. [24] Pradlwarter HJ, Pellissetti MF, Schenk CA, Schuller GI, Kreis A, Fransen S, et al. Realistic and efficient reliability estimation for aerospace structures. Comput Methods Appl Mech Eng 2005;194(12–16):597–617. [25] Draper D. Model uncertainty in ‘‘stochastic” and ‘‘deterministic” systems. In: Proceedings of the 12th international workshop on statistical modeling, Biel/ Bienne, Switzerland; 1997. p. 43–59. [26] Kreinovich V, Hajago JG, Tucker T, Beck J, Ginzburg LR, Ferson S. Propagating uncertainty through a quadratic response surface model. Sandia National Laboratory, Albuquerque, NM; 2008. SAND2008-5983. [27] Rausch RD, Batina JT, Yang HTY. Three dimensional time-marching aeroelastic analyses using an unstructured Grid-Euler method. AIAA; 1992. [28] Lee-Rausch EM, Batina JT. Wing flutter boundary predictions using unsteady Euler method. AIAA; 1993. [29] Lee-Rausch EM, Batina JT. Calculation of agard wing 445.6 flutter using NavierStokes aerodynamics. In: 11th AIAA applied aerodynamics conference (Monetery, California 1993). [30] Liu F, Cai J, Zhu Y, Tsai HM, Wong ASF. Calculation of wing flutter by a coupled Fluid-Structure method. AIAA J Aircraft 2001;38(2):334–42. [31] ZAERO applications manual, Vol.1. Zona Tech, 2008, Version 8.3.