Assessing uncertainty in extreme events: Applications to risk-based decision making in interdependent infrastructure sectors

ARTICLE IN PRESS Reliability Engineering and System Safety 94 (2009) 819–829

Contents lists available at ScienceDirect

Reliability Engineering and System Safety journal homepage: www.elsevier.com/locate/ress

Assessing uncertainty in extreme events: Applications to risk-based decision making in interdependent infrastructure sectors Kash Barker a,, Yacov Y. Haimes b,1 a b

School of Industrial Engineering, University of Oklahoma, 202 West Boyd Street, Room 124, Norman, OK 73019, USA Center for Risk Management of Engineering Systems, University of Virginia, 112 Olsson Hall, Charlottesville, VA 22903, USA

a r t i c l e in fo

abstract

Article history: Received 21 March 2008 Received in revised form 19 August 2008 Accepted 1 September 2008 Available online 17 October 2008

Risk-based decision making often relies upon expert probability assessments, particularly in the consequences of disruptive events and when such events are extreme or catastrophic in nature. Naturally, such expert-elicited probability distributions can be fraught with errors, as they describe events which occur very infrequently and for which only sparse data exist. This paper presents a quantitative framework, the extreme event uncertainty sensitivity impact method (EE-USIM), for measuring the sensitivity of extreme event consequences to uncertainties in the parameters of the underlying probability distribution. The EE-USIM is demonstrated with the Inoperability input–output model (IIM), a model with which to evaluate the propagation of inoperability throughout an interdependent set of economic and infrastructure sectors. The EE-USIM also makes use of a two-sided power distribution function generated by expert elicitation of extreme event consequences. & 2008 Elsevier Ltd. All rights reserved.

Keywords: Uncertainty analysis Extreme events Risk management Inoperability input–output model Multiobjective decision making

1. Introduction Recent natural disasters and malevolent man-made events have increased the interest in understanding extreme events and planning for their occurrence. Recent focus (e.g., by the US Department of Homeland Security (DHS)), has been given to preparedness activities that improve response to and recovery from such extreme events. Indeed, the utility of risk-based decision making is not necessarily to articulate the ‘‘best’’ policy option, but rather to avoid the extreme, the worst, and the most disastrous options. To do so, a decision maker must be able to measure the outcomes of such extreme events and measure how risk management can control them. Asbeck and Haimes [1] and Haimes [2] introduce the partitioned multiobjective risk method (PMRM) and discuss the fallacy behind using the expected value of adverse outcomes when analyzing and managing risks as the sole measure for risk, because such a measure does not accurately capture outcomes that are due to catastrophic, not-unlikely events. Frowhein et al. [3] provide a number of measures to quantifying extreme event consequences, including the PMRM. However, Taleb [4] warns that parametric uncertainties and estimation errors in probability

 Corresponding author. Tel.: +1 405 325 3721; fax: +1 405 325 7555.

E-mail addresses: [email protected] (K. Barker), [email protected] (Y.Y. Haimes). 1 Tel.: +1 434 924 0960; fax: +1 434 924 0865. 0951-8320/$ - see front matter & 2008 Elsevier Ltd. All rights reserved. doi:10.1016/j.ress.2008.09.008

distributions can result in adverse effects in the understanding of extreme events. Data describing extreme events, due to their low probability of occurrence, are understandably sparse. The assessment of likelihood can suffer from the subjectivity of the data source, particularly due to the sparseness of data, as expert-elicited likelihoods are frequently used. Probability distributions derived from expert elicitation can suffer from a number of errors [5–8]. Oberkampf et al. [9] highlight the difficulties that arise in interpreting model outputs when uncertainties exist in model parameters. Further discussions of the propagation of uncertainties as applied to risk assessments include Refs. [10–13]. Expected and conditional expected values of consequences due to disruptive events can vary widely depending on the choice of parameters of the underlying probability distribution from which these risk measures are calculated. This paper provides a framework, referred to here as the extreme event uncertainty sensitivity index method (EE-USIM) [14], for calculating and analyzing the sensitivity of extreme event consequences with respect to uncertainty in the parameters of underlying probability distributions of disruption consequences. The EE-USIM, graphically in Fig. 1, is derived from three main methodological components: sensitivity analysis of probability distribution parameter uncertainties using the uncertainty sensitivity index method (USIM) [2,15,16], extreme event analysis using the partitioned multiobjective risk method (PMRM) [1,2], and the comparison of risk management strategies via the tradeoffs among multiple, noncommensurate, and competing objectives.

ARTICLE IN PRESS 820

K. Barker, Y.Y. Haimes / Reliability Engineering and System Safety 94 (2009) 819–829

Distribution Parameter Uncertainty Analysis

input

out put

Multiobjective Decision making

Extreme Event Analysis

Extreme Event Uncertainty Sensitivity Index Method

IIIustration: Risk Management of Interdependent Systems

Fig. 1. The extreme event uncertainty sensitivity index method framework for analyzing the uncertainty in extreme event measures.

The EE-USIM is applied with the fourth component, the study of interdependent economic and infrastructure sectors with the inoperability input–output model (IIM) [2,17–20]. The IIM is a model that can measure the efficacy of risk management strategies to improve preparedness within and among the affected interdependent sectors. While the EE-USIM approach can be applied to any model containing stochastic elements whose distributional parameters may be uncertain, the IIM provides an appropriate model to demonstrate the EE-USIM for interdependent sectors because they tend to be readily affected by extreme events. The IIM illustration demonstrates the usefulness of the EE-USIM as a means to compare risk management strategies that seek to minimize a number of objectives, including the total economic loss experienced to the economy following an extreme disruptive event, the sensitivity of that economic loss to uncertainties in underlying expert-elicited probability distributions, and the cost of implementation for each strategy. The expert-elicited distribution used in this paper is the twosided power distribution, described in Section 2.1. The component methodologies integrated in Fig. 1, the PMRM, the USIM, and the IIM, are described in Sections 2.2–2.4. Replacing point-estimate IIM parameters with two-sided power distributions is described in Section 3. The two PMRM metrics, the average (mean) and extreme economic losses of resulting IIM output distribution, are calculated in Section 4. In Section 5, the sensitivities of these two PMRM metrics are incorporated within the USIM. To guide the reader in the subsequent sections of this paper, the development of the EE-USIM logically parallels these sections as follows: probabilistic parameters serve as input into a model thus resulting in a probability distribution of model output; expected and conditional expected values are calculated from the resulting output probability distribution; indices are calculated to measure the sensitivity of these average and extreme losses to uncertainties in the probabilistic input parameters; and a multiobjective formulation is devised with which to minimize the average and extreme economic losses, the sensitivities of these losses, and other objectives of interest.

2. Methodological background The several methodologies shown in Fig. 1 are combined to comprise the EE-USIM framework; each component of which is described as follows: 2.1. Probability elicitation in risk analysis Vose [21] provides two information sources for deriving probability distributions for uncertain model parameters: histor-

ical data and expert elicitation. In the first source, an appropriate probability function is fit from a set of collected data. Rychlik and Ryden [22] discuss approaches to fitting probability distributions to data, though such historic data may not exist in the cases of extreme events where risk analysis techniques are frequently required. The second approach to derive probability distributions is with the use of expert evidence. In cases where historical data do not exist, experienced individuals may be able to provide sufficient perspectives. Approaches to expert elicitation in assessing probability distributions are found in Refs. [23–27], among others. To focus on the uncertainty in expert-elicited distribution parameters, this paper makes use of a particular bounded, nonlinear distribution called the two-sided power distribution, discussed in Section 2.1.1.

2.1.1. Two-sided power distribution A common technique for assessing a probability distribution from expert knowledge is the triangular distribution [2,28–30]. Its common use is due to its ease in assessment, as the triangular distribution requires only the elicitation of minimum, maximum, and most likely values of outcomes with which to fit a distribution. A potentially more flexible alternative is the two-sided power distribution [31–34], of which the triangular distribution is a special case. van Dorp and Kotz [31] provide a four-parameter two-sided power (TSP) distribution that is meant to strengthen the threeparameter triangular distribution by allowing the distribution to follow a nonlinear form instead of the rigid, linear, triangular form. The four-parameter TSP distribution is described by parameters denoting minimum value, a, maximum value, b, and mode or most likely value, c, and adds a fourth parameter, Z, describing the curvature of the distribution. The probability density function is found in Eq. (1) for TSP(a,b,c,Z). Note that when Z ¼ 1, the TSP and uniform distribution are equivalent, and when Z ¼ 2, the TSP and triangular distribution are equivalent (see, for example [35]). 8 Z x  aZ1 > > > > > b  a ca <   Z b  x Z1 pðxÞ ¼ > > >b a b c > > : 0

if aoxpc if cpxob

(1)

otherwise

The expected value of a TSP distribution is provided in Eq. (2) [34]: E½X ¼

a þ ðZ  1Þc þ b Zþ1

(2)

ARTICLE IN PRESS K. Barker, Y.Y. Haimes / Reliability Engineering and System Safety 94 (2009) 819–829

p (x)

p (x)

p (x)

x

x

x η=1

η = 0.75

η = 0.05 p (x)

p (x)

821

p (x)

x

x η = 1.25

x

η=2

η=3

Fig. 2. Examples of four-parameter two-sided power probability density functions with different values of shape parameter Z.

Fig. 2 provides examples of symmetric TSP probability density function curves for several values of the shape parameter, Z. The indirect elicitation of parameters a–c, and Z through expert evidence is similar to that of the triangular distribution or betaPERT distribution [21], where the minimum, maximum, and most likely values are elicited. The parameter Z requires expert evidence for the relative importance of the most likely value relative to distribution bounds a and b. That is, if Z ¼ 2, c is equally important as a and b; if Z42, c has more weight than the bounds; if Zo2, c is given less emphasis relative to the bounds. van Dorp and Kotz [31] provide a maximum likelihood estimate for Z when historic data are available for direct elicitation of distribution parameters.

p (x)

 

0

x

Fig. 3. The probability density function plot of damage variable X highlighting the upper-tail damage range defined by b.

R1

b xpðxÞ dx f 4 ðÞ ¼ E½XjX4b ¼ R 1 b pðxÞ dx

2.2. Partitioned multiobjective risk method Traditionally, the quantification of risk has been performed with respect to the expected value of a probability distribution of some adverse consequence, where the shape of the consequence distribution is ignored, and the over-simplified expected value provides no indication of the low-likelihood but catastrophic damages that may occur. Asbeck and Haimes [1] developed the PMRM to address this fallacy by comparing risks associated with damages from both average events and extreme events in a multiobjective framework. Consequences of average events are measured with f5, the metric associated with the traditional unconditional expected value of a disruptive consequence, provided in Eq. (3) (and in Eq. (2) for the TSP distribution, in particular). Extreme event consequences are described with f4, the conditional expected value associated with low probability and high severity (the upper tail of Fig. 3), in Eq. (4). The upper-tail range of the probability density function for event consequence (random variable X) in Fig. 3 is defined to be X4b, chosen such that p(X4b) ¼ a. The value of a represents some extreme event likelihood, e.g., a ¼ 0.10 or 1 in 10. An example of a consequence random variable of interest could be economic loss resulting from a disruptive event, as discussed later in this paper; the upper-tail area of such a distribution would correspond to the economic loss experienced after a particularly extreme disruptive event. R1 Z 1 xpðxÞ dx f 5 ðÞ ¼ E½X ¼ R1 ¼ xpðxÞ dx 1 1 1 pðxÞ dx

(3)

(4)

The PMRM provides a multiobjective formulation which compares the cost of risk management implementation versus the consequences, both mean risk (f5) and extreme risk (f4) resulting from each in a set of risk management strategies. The PMRM has been applied to the study of extreme events in a number of risk analysis contexts, including infrastructure investments [36,37], portfolio optimization [38], and sequential decision making [39–41]. Analogous to the idea of f4 is the tail-conditional expectation (see, e.g., [42]), introduced much later than PMRM and frequently applied to value-at-risk, a common financial risk metric [43]. Extreme event analysis applied to the TSP distribution is introduced here. The TSP distribution provides an analytical solution to conditional expected value calculations, a notable downside of the beta distribution [44]. The upper-tail partition b can be solved in terms of distribution parameters a–c, and Z, and the upper-tail probability, a, where P(x4b) ¼ a. Eq. (5), derived from the TSP cumulative distribution function, provides the calculation for b. 1

 Z bc bb ðb  aÞZþ1 ¼1a)b¼b Z ba bc a ðb  cÞZ

(5)

2.3. Uncertainty sensitivity index method The USIM was developed by Haimes and Hall [15] and extended by Li and Haimes [16] to address the sensitivity of

ARTICLE IN PRESS 822

K. Barker, Y.Y. Haimes / Reliability Engineering and System Safety 94 (2009) 819–829

optimal model response to potential variation in model parameters. That is, the USIM is designed to allow a decision maker to compare options that seek to optimize an objective function while minimizing sensitivity to unforeseen errors. Consider a single-objective constrained optimization problem where f(u,h) is an objective function to be minimized subject to a vector function g(u,h) of constraints. Vector u represents a vector of decision variables, and h represents a vector of m model parameters. The USIM measures sensitivity of f(u,h) with respect to changes in model parameters, a, with the calculation of sensitivity index c(u,h). Li and Haimes [16] show that a change in an objective function of multiple uncertain parameters, e.g., the vector h ¼ (y1, y, yi, y, ym), can be minimized when the sensitivity index in Eq. (6) is minimized. That is, for m uncertain parameters, the sensitivity index is defined as the sum of m partial derivatives of f(u,h) with respect to each parameter. For simplicity in this presentation, each partial derivative is equally weighted as follows:

cðu; hÞ ¼

m  X q i¼1

qyi

f ðu; hÞ

2

(6)

Adding sensitivity to the single-objective problem results in the multiobjective optimization formulation in Eq. (7). The objective function and sensitivity index are both minimized with respect to decision u and are evaluated at a nominal value h^ , e.g., the original estimate of h. It is assumed that h, though its true values may be uncertain, varies in the neighborhood of h^ . min u

f ðu; h^ Þ

cðu; h^ Þ s:t:

(7)

gðu; h^ Þp0

The above optimization problem simultaneously minimizes the objective function of interest to the decision maker while also minimizing the sensitivity of the system output to the unknown parameters y1, y, yi, y, ym as they vary around their nominal values y^ 1 ; . . . ; y^ i ; . . . ; y^ m . See Ref. [2] for a more detailed development of the USIM and other extensions to the general formulation provided here. A recent application of the USIM for analyzing uncertainty in the strength of sector interdependencies is found in Ref. [45]. 2.4. Inoperability input–output model and its foundations The EE-USIM is applied in this paper to the extreme event analysis of interdependencies among infrastructure and economic sectors. Several different modeling approaches have been developed to describe such interdependencies, including those based on system dynamics [46], agent-based simulation [47,48], and network models [49,50], among others. Another approach, and the one used as the model illustration in this paper, is based on the input–output (I–O) model. 2.4.1. Inoperability input–output model The I–O model [51–54], which was awarded a Nobel Prize in 1973, was published as a means to provide a ‘‘numerical description of the American economic structure’’ [55]. The linear I–O model enables the analysis of the generation, flow, and consumption of various commodities among infrastructure sectors. An accepted model worldwide, over 60 countries maintain current I–O accounts of their economies [56], including a significant data collection effort undertaken by the US Census Bureau, the Bureau of Economic Analysis (BEA), and other US federal agencies.

Taking advantage of the rich databases collected by these agencies, the I–O model was extended to describe how inoperability propagates through a set of interconnected economic sectors with the IIM [2,17,19,20,57,58]. Inoperability is interpreted as the percentage to which a particular sector does not satisfy the as-planned level of production output. The IIM, a linear transformation of the Leontief I–O model, is presented in Eq. (8) for an economy of n sectors, resulting in matrices of size n  n and vectors of length n. q ¼ An q þ cn ) q ¼ ½I  An 1 cn

(8)

The vector q represents the inoperability vector, the elements of which measure the proportion of ‘‘unrealized’’ production per asplanned production. Inoperability is analogous to the concept of unreliability from the field of reliability engineering (see, e.g., [59]). The demand perturbation is expressed by vector c*, whose elements quantify reduced final demand as a proportion of total as-planned output. The economy could experience a reduction in demand following a disruption for a number of reasons, including the result of a forced demand reduction from diminished supply and due to lingering consumer fear or doubt. The matrix A* is the normalized interdependency matrix describing the extent of economic interdependence between sectors of the economy. The row elements of A* indicate the proportions of additional inoperability that are contributed by a column sector to the row sector. For further details about the derivation of IIM parameters, the reader is referred to Ref. [19]. Total economic loss experienced across all sectors of the economy can be found by multiplying the inoperability vector, q, by the transpose of the total output vector, denoted by xT. The calculation for total economic loss, defined as Q Q ¼ xT q

(9)

The IIM is used here to measure the efficacy of risk management strategies, as suggested by Crowther and Haimes [60]. That is, risk management strategies can alter the value of demand perturbation (e.g., through inventory and storage capabilities), and such strategies can be compared according to their Q values, the result of inputting demand perturbation in the IIM; each strategy leads to a different total economic loss experienced after a disruptive event. As such, vector c* can be viewed here as a pseudo-decision variable; risk management strategies manifest themselves in some way in c*. The IIM has been applied to study a number of sources of risk to interconnected systems, including terrorist attacks [20,57,58], the efficacy of cyber security [61,62], the Northeast US blackout [63], and Hurricane Katrina [64]. 2.4.2. Probabilistic IIM A common trait among the IIM applications listed previously is the use of point estimates, whether hypothesized or based on historical data, in the calculation of demand reduction, c*. Hattis and Burmaster [65] motivate analyses that do not involve singlepoint treatments in various parameters in risk assessments, because such point estimates could have significant effects on risk assessment outcomes if they are incorrect. Rose [66] stated that point estimates in I–O model parameters ‘‘exaggerate the certainty of the analysis and will almost assuredly be incorrect.’’ Previous attempts at probabilistic IIM modeling have involved the insertion of a probability distribution into a single element of the c* vector [67], as illustrated in Fig. 4. If only one cni follows a probability distribution, and the remaining elements of c* are point estimates, then the resulting total economic loss metric, Qi, follows the same probability distribution as cni due to the linearity of the IIM. Note that the superscript in the notation of Qi denotes

ARTICLE IN PRESS K. Barker, Y.Y. Haimes / Reliability Engineering and System Safety 94 (2009) 819–829

p (ci∗)

823

p(Qi) Inoperability Input-Output Model

a

c

b

ci∗

Qimin

i Qmode

i Qmax

Qi

(c∗1,....,c∗i−1 , c∗i+1 ,....,c∗n) Fig. 4. Relationship between probabilistic demand perturbations and resulting distribution total economic loss in IIM (adapted from [41]).

that the random variable for economic loss was derived from a probability distribution in the ith element of cni . When multiple elements of c* follow (independent) probability distributions, the resulting distribution of Qi is a convolution of several cni . Analytical solutions of the convolutions of several random variables can be difficult, or impossible, to derive depending on their distribution. While Ref. [33] discusses the convolution of two TSP-distributed random variables, such complexity is outside the scope of this paper. A nonanalytical extreme event analysis approach using Monte Carlo simulation can allow for probability distributions in multiple elements of c* [41,44], however, an analytical form of Qi is sought in this paper so the PMRM can be applied. As such, only one element of c* is perturbed.

Defining the matrix D* as the Leontief inverse of the normalized technical coefficient matrix, [I–A*]1, Eq. (12) computes the minimum value of the total economic loss distribution.

3. Two-sided power-distributed demand perturbation

Q imin

The approach to probabilistic IIM illustrated in Fig. 4, involving a probability distribution in only one element of c*, is taken here. Eq. (10) presents a vector of demand perturbations with a probabilistic ith element following a two-sided power distribution. That is, the distributional parameters for a probabilistic demand perturbation for the Sector i was elicited by some means (e.g., via an expert) while the remaining elements were provided as point estimates. 2 3 cn1 7 6 .. 7 6 6 7 . 6 7 6 7 cn ¼ 6 cni ¼ TSPða; b; c; ZÞ 7 (10) 6 7 7 6 .. 6 7 . 4 5 cnn

2

 Q imin ¼ xT Dn cnmin ¼ x1

cnn

Inserting Eq. (11) into Eqs. (8) and (9) provides the minimum value of the resulting total economic loss distribution, Q imin .



 .. .  .. . 

n

d1i ... n dii .. . n dni

 .. .  .. . 

32 3 n d1n cn 76 1 7 . 6 ... 7 76 .. 7 7 7 7 n 76 6 din 76 a 7 76 . 7 .. 76 . 7 . 7 . 7 54 5 n cnn dnn

(12) Rewriting Eq. (12) in scalar summation form results in Eq. (13). Note that index i is associated with the ith element of the vector in Eq. (11), while indices j and k are used for the summation operations. n

n

n

x1 ðd11 cn1 þ    þ d1i a þ    þ d1n cnn Þ .. .

¼

n

n

n

þxi ðdi1 cn1 þ    þ dii a þ    þ din cnn Þ .. . n n n þxn ðdn1 cn1 þ    þ dni a þ    þ dnn cnn Þ n n P P P n n n xj djk ck þ a xj dji

¼

j¼1

(13)

j¼1

kai

Similarly, Q imax and Q imode are calculated as in Eqs. (14) and (15). Note the distinction in Eq. (15) between cnk , the demand perturbation of a given sector k, and c, the most likely TSP distribution parameter found in the ith element of c*. Q imax ¼

n X

xj

j¼1

The insertion of Eq. (10) into the IIM results in a total economic loss metric that follows a TSP distribution. Fig. 4 is a general graphical depiction of this, where Q imin , Q imax , and Q imode are the minimum, maximum, and most likely parameters, respectively, of the resulting TSP distribution for total economic loss. In Fig. 4, the random variable for the leftmost distribution is cni , and the random variable for the rightmost distribution is Qi. Eq. (11) provides the vector of demand perturbation when the random variable cni takes on its minimum value, a. 2 3 cn1 7 6 .. 6 7 6 7 . 6 7 7 6 n cnmin ¼ 6 ci ¼ a 7 (11) 6 7 6 7 .. 6 7 . 4 5

xi



n

d11 6 6 .. 6 . 6 n xn 6 6 di1 6 6 .. 6 . 4 n dn1

Q imode ¼

n X j¼1

X

n

djk cnk þ b

X kai

n

xj dji

(14)

j¼1

kai

xj

n X

n

djk cnk þ c

n X

n

xj dji

(15)

j¼1

Since cni follows a TSP distribution, the resulting total economic loss IIM output follows a TSP distribution. The probability density function for Qi, whose parameters are found in Eqs. (13)–(15), is found in Eq. (16). 8  Z1 > Q i Q imin Z > > for Q imin oQ i pQ imode > Q i Q i i i > Q Q > min mode < max min  Z1 (16) pðQ i Þ ¼ Q imax Q i Z > for Q imode oQ i oQ imax > i i i i > Q Q Q Q > max min max mode > > : 0 otherwise Substituting Eqs. (13)–(15) into Eq. (16) results in the piecewise calculation for p(Qi) presented in Eq. (17). Note the distinction between n, referring to the number of sectors in the economy of interest, and Z, the shape parameter of the

ARTICLE IN PRESS 824

K. Barker, Y.Y. Haimes / Reliability Engineering and System Safety 94 (2009) 819–829

TSP distribution. 8 0 1Z1 n n P P n n P > n > Qi xj djk ck þa xj dji > > B C > Z > B j¼1 kai  j¼1 C  > > @ A n n > P P > > xj dnji ðbaÞ xj dnji ðcaÞ > > > j¼1 j¼1 < 0 n 1Z1 n pðQ i Þ ¼ P P n n P n > xj djk ck þb xj dji Q i > > B C > Z kai > B j¼1  C j¼1 > > @ A n n P P > n n > > xj dji ðbaÞ xj dji ðbcÞ > > > j¼1 j¼1 > : 0

for Q imin oQ i pQ imode

for Q imode oQ i oQ imax

(21) otherwise

(17) Eq. (17) provides the piecewise probability density function for the TSP distribution of total economic losses as a function of cni TSP distribution parameters a–c, of upper-tail probability a, and of BEA-driven values from D* and x. Extreme event metrics of this resulting distribution can be calculated in an analytical fashion, as discussed in Section 4.

4. Metrics of two-sided power-distributed total economic loss An analytical approach is desired to quantify economic losses following ‘‘typical’’ disruptive events and following a disruptive event that has more extreme consequences on Sector i than only the expected value can evaluate. Perturbing a single sector allows for such an analytical approach to calculate average and extreme values, as discussed in Eqs. (3) and (4). The expected value of risk, f5, of total economic loss in Eq. (18) is a straightforward result from the definition of the expected value of a four-parameter TSP distribution in Eq. (2) [31]. Eq. (18) shows that f5 is the sum of sector economic losses from all sectors Pn P n n other than Sector i, calculated from j¼1 xj kai djk ck , and the expected value of TSP-distributed demand perturbation multiplied by the direct and indirect contributions to Sector i’s economic loss. f 5 ðÞ ¼

n X n Q imin þ ðZ  1ÞQ imode þ Q imax X ¼ xj djk cnk Zþ1 j¼1 kai

þ

n a þ ðZ  1Þc þ b X n xj dji Zþ1 j¼1

(18)

The calculation of f4, whose definition is found in Eq. (4), is initiated for the TSP distribution in Eq. (19). The calculation of f4 makes use of the upper piecewise section of the probability density function between c and b, as total economic losses experienced after extreme events would tend to fall in the upper tail of the distribution in Eq. (17). Note that for Z ¼ 2, a triangular distribution, Eq. (19) simplifies to the horizontal location of the centroid of a triangle (see, e.g., [68]). R Q imax b

Q i pðQ i Þ dQ i

f 4 ðÞ ¼ R i Q max b

i

pðQ Þ dQ

i

¼

Q imax

Zb þ Zþ1

(19)

Extending the calculation of b from Eq. (5) to account for total economic loss, Eq. (20) reflects the upper-tail partition point for the TSP-distributed economic loss axis. We want to express b, and subsequently f4, as a function of a, thereby holding the upper-tail probability, and not the partition, constant.

b ¼ Q imax  ¼

n X j¼1

xj

ðQ imax  Q imode ÞZþ1

aZ ðQ imax  Q imin ÞZ

X kai

n

djk cnk þ b

n X j¼1

IIM parameters, cni distribution parameters, and upper-tail probability, a. Eq. (21) shows that f4 is calculated as Q imax less some fraction of the direct and indirect contributions to Sector i’s P n economic loss contained within nj¼1 xj dji . 2 3   n n n X n X X Z ðb  cÞZþ1 X n n n f 4 ðÞ ¼ 4 xj djk ck þ b xj dji 5  xj dji Z ðb  aÞZ þ 1 Z a j¼1 j¼1 j¼1 kai

n ðb  cÞZþ1 X n n xj dji  Z xj dji a ðb  aÞZ

(20)

j¼1

Inserting the values of b from Eq. (20) and Q imax from Eq. (14) into Eq. (19) results in the following calculation of f4, in terms of

We now have a means to study the potential risk of economic losses stemming from both average and extreme disruptive events using the TSP distribution. This concept is illustrated with the following example dealing with a disruption in oil and gas production.

4.1. Illustrative example The calculation and interpretation of average and extreme total economic loss are demonstrated with a 15-sector example from the 2005 Bureau of Economic Analysis annual I–O tables [69]. The 15 sectors, representing all industry types with a low granularity aggregation, are enumerated in Table 1. The A* matrix and the x vector of total annual output for the 15 sectors are provided in Tables 2 and 3, respectively. Note that the tables provide national I–O account data, and a model with regional data may be more appropriate for many disruptive events [70]. For this example, assume that oil and gas operations, part of the Mining sector (Sector 2) in Table 1, is perturbed following a disruptive event. This perturbation, cn2 , follows a symmetric TSP distribution whose parameters are found in Eq. (22), interpreted to mean that demand, a surrogate measurement here for supply reduction, in the Mining sector is reduced by an amount between 20% and 50% with a most likely value of 35%. All other sector demand perturbations are zero, meaning that no independent effects are experienced in those sectors. It is assumed that this inoperability lasts for 2 weeks. Therefore, the calculation of total economic loss requires that the elements of x (total annual output) be divided by 26, assuming that total output is linear with time. 2 6 cn

6 6 cn ¼ 6 6 6 4

2

3 0 ¼ TSPð0:20; 0:50; 0:35; ZÞ 7 7 7 7 0 7 .. 7 5 .

(22)

0 Table 1 15 sectors in the 2005 BEA annual input–output accounts [69] Sector

Title

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15

Agriculture, forestry, fishing, and hunting Mining Utilities Construction Manufacturing Wholesale trade Retail trade Transportation and warehousing Information Finance, insurance, real estate, rental, and leasing Professional and business services Educational services, health care, and social assistance Arts, entertainment, recreation, accommodation, and food services Other services, except government Government

ARTICLE IN PRESS K. Barker, Y.Y. Haimes / Reliability Engineering and System Safety 94 (2009) 819–829

825

Table 2 A* matrix for the 15-sector example, values rounded to four decimal places, derived from Ref. [69] Sec.

1

2

3

4

5

6

7

8

9

10

11

12

13

14

15

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15

0.2258 0.0021 0.0117 0.0012 0.0136 0.0117 0.0001 0.0113 0.0012 0.0043 0.0020 1.4E-5 0.0007 0.0048 0.0011

7.7E-6 0.1432 0.0057 0.0001 0.0078 0.0049 0.0003 0.0123 0.0018 0.0063 0.0099 0.0001 0.0008 0.0007 0.0006

4.1E-6 0.2896 0.0008 0.0025 0.0012 0.0010 4.8E-5 0.0392 0.0003 0.0008 0.0012 0.0002 0.0005 0.0002 0.0005

0.0062 0.0245 0.0082 0.0010 0.0754 0.0377 0.0594 0.0295 0.0148 0.0102 0.0381 0.0001 0.0030 0.0190 0.0019

0.5512 0.7635 0.0989 0.0061 0.3230 0.2497 0.0107 0.1859 0.0461 0.0277 0.1301 0.0020 0.0211 0.0665 0.0103

0.0009 0.0006 0.0127 0.0018 0.0092 0.0307 0.0023 0.0221 0.0180 0.0128 0.0397 0.0005 0.0065 0.0116 0.0028

0.0002 0.0006 0.0288 0.0039 0.0090 0.0063 0.0045 0.0286 0.0235 0.0258 0.0627 0.0004 0.0081 0.0118 0.0044

0.0001 0.0197 0.0072 0.0014 0.0219 0.0186 0.0019 0.1150 0.0120 0.0097 0.0237 0.0004 0.0071 0.0129 0.0021

0.0001 0.0009 0.0111 0.0023 0.0167 0.0132 0.0006 0.0133 0.2061 0.0188 0.0527 0.0018 0.0183 0.0252 0.0039

0.0049 0.0028 0.1251 0.0292 0.0135 0.0074 0.0091 0.0462 0.0482 0.1879 0.1178 0.0010 0.0302 0.0473 0.0126

0.0196 0.0022 0.0369 0.0085 0.0230 0.0154 0.0080 0.0402 0.0938 0.0418 0.1392 0.0018 0.0405 0.0424 0.0104

0.0014 0.0017 0.0251 0.0076 0.0289 0.0176 0.0021 0.0238 0.0395 0.0390 0.0562 0.0080 0.0260 0.0171 0.0092

0.0307 0.0017 0.0402 0.0063 0.0267 0.0182 0.0031 0.0152 0.0196 0.0219 0.0224 0.0004 0.0250 0.0142 0.0042

0.0011 0.0015 0.0171 0.0033 0.0243 0.0145 0.0072 0.0112 0.0178 0.0145 0.0222 0.0005 0.0067 0.0132 0.0030

0.0054 0.0447 0.1037 0.0417 0.0573 0.0282 2.9E-5 0.0635 0.0756 0.0227 0.1038 0.0220 0.0294 0.0559 0.0122

Table 3 Total output for the 15-sector example, from Ref. [69] and in 106 dollars Sector

Total output

Sector

Total output

Sector

Total output

1 2 3 4 5

312,754 396,563 399,409 1,302,388 4,481,063

6 7 8 9 10

1,055,806 1,187,408 713,320 1,121,310 3,925,989

11 12 13 14 15

2,378,257 1,543,848 843,819 660,645 2,574,707

cn

i

The 2 and Q distributions are depicted graphically in Fig. 5 for several values of Z. Fig. 5 is a realization of the general probabilistic IIM approach in Fig. 4. Applying the average and extreme total economic loss calculations from Eqs. (18) and (21) to the 15-sector study results in Table 4. Several values of Z are provided in Fig. 5 and Table 4 are provided for illustrative purposes; it is assumed that expert elicitation would also involve evidence on a representative value of Z for the analysis, and the illustration with multiple values of Z is for illustration purposes. Extreme total economic loss, f4, assumes that the upper-tail probability is a ¼ 0.10, or that an extreme event is defined as the upper 10% of adverse consequences. Since the distribution of cn2 is symmetric about the most likely value, as shown in Fig. 5, f5 is constant for all values of Z. Recall that the partition, b, is not constant, but is a function of upper-tail probability, a; as a remains constant, b decreases with increasing Z. Similarly, f4 decreases with increasing Z. Mathematically, this is detailed in Eqs. (20) and (21). Logically, as Z increases, more emphasis is placed on the most likely distribution parameter, c. Holding a constant, the value of b is drawn closer to c with increasing Z. Such a pattern would not hold if c ¼ b.

5. Applying USIM to two-sided power-distributed total economic loss Discussed previously, the elicitation of expert evidence, particularly evidence regarding the outcomes of low-probability disruptive events, is fraught with uncertainty. Subsequently, the results of models that use such expert-elicited parameters can suffer. For example, an a priori analysis of a disruptive event and its effect on supply and demand in a sector, e.g., a disruption in oil and gas production, can lead to misestimates of economic loss when uncertainty is presented. And poor decision making can follow, as risk management strategies are frequently built on such

a priori analyses. In this section, the sensitivity of average and extreme economic losses stemming from uncertainty in TSPdistributed IIM demand perturbations is addressed with the USIM, completing the EE-USIM framework. Specifically, the sensitivity of f5 and f4 calculations of total economic loss is measured with respect to the uncertainty in cni distribution parameters a–c. The USIM is applied to the TSP distribution calculations of f5 and f4 by means of Eq. (6). The partial derivative with respect to minimum parameter, a, is presented as follows: n qf 5 1 X n ¼ xd qa Z þ 1 j¼1 j ji

(23)

The partial derivative with respect to the distribution’s maximum parameter, b, is equal to that of Eq. (23). The partial derivative of f5 with respect to most likely parameter, c, is provided as follows: n qf 5 Z  1 X n ¼ xd qc Z þ 1 j¼1 j ji

(24)

The resulting sensitivity index for the f5 risk measure could be the sum of the squared partial derivatives with respect to each of the uncertain parameters, as described in Eq. (6). However, since the focus of this paper is on extreme events, emphasis is placed on the uncertainty in the maximum parameter of the distribution, b. The sensitivity index, provided in Eq. (25), is the square of the partial derivative of f5 with respect to b. Note that cf 5 is constant with respect to a–c. 

qf 5 cf 5 ða; b; cÞ ¼ qb

2 ¼

2

1 ðZ þ 1Þ2

4

n X

32 n5

xj dji

(25)

j¼1

The application of the sensitivity index calculation to the conditional expected value, f4, appears in Eqs. (26)–(29). The partial derivatives are provided individually. Again, due to the focus on the study of extreme events, the sensitivity index, cf 4 , is calculated only from the partial derivative of f4 with respect to b; this calculation appears in Eq. (29). 

Zþ1 X n

qf 4 Z2 bc ¼ Z qa a ðZ þ 1Þ b  a "



Z

qf 4 Z bc ¼ 1 Z qb a ba

n

xj dji

(26)

j¼1



Zþ1 # X n

Z2 bc þ Z a ðZ þ 1Þ b  a

j¼1

n

xj dji

(27)

ARTICLE IN PRESS 826

K. Barker, Y.Y. Haimes / Reliability Engineering and System Safety 94 (2009) 819–829

Fig. 5. Examples of four-parameter two-sided power distribution with different values of shape parameter Z. Q measured in 106 dollars.

Table 4 b, expected values, and conditional expected values of total economic loss for the two-sided power distribution examples in Fig. 5, in 106 dollars

Z

f5

b

f4

0.5 1 1.5 2 2.5 3

9355 9355 9355 9355 9355 9355

13,200 12,560 11,990 11,570 11,260 11,020

13,310 12,960 12,540 12,170 11,860 11,610



qf 4 Z bc ¼ qc aZ b  a 

qf 4 cf 4 ða; b; cÞ ¼ qb

Z X n

n

xj dji

(28)

j¼1

2

2 ¼4

n X j¼1

32 " n5

xj dji

   Zþ1 #2 Z bc Z Z2 bc 1 Z þ Z a ðZ þ 1Þ b  a a ba

(29) Eq. (25) measures the sensitivity of the anticipated economic loss to uncertainty in the expert-elicited upper bound of the distribution for consequences. However, since cf 5 is constant with respect to b, the total economic loss from an average disruptive event is insensitive to potential uncertainties in the expert assessment of demand perturbation in Sector i. Eq. (29) measures the sensitivity of the predicted ‘‘extreme event’’ to the choice of those parameters and is heavily dependent on the value of b. In fact, were the squared partial derivatives in Eqs. (26) and (28) considered in the calculation of cf 4 , the contribution of uncertainty in b would be far greater than that of a and c. Eqs. (25) and (29) provide insight into those risk management strategies that can best ‘‘absorb’’ both expected and extreme

situations when their associated elicited demand perturbation distribution parameters may be incorrect. The general multiobjective optimization problem of interest is found in Eq. (30). An expert is asked to determine the demand perturbation resulting from the implementation of each of strategy in a set of discrete risk management strategies, given that a disruptive event occurs. One of the sectors, in the ith element of c*, is given as a probability distribution. The EE-USIM seeks to minimize, for a set of discrete risk management strategies, total economic losses and sensitivity to parameter uncertainty in both average (f5 and cf 5 , respectively) and extreme (f4 and cf 4 , respectively) disruptive events. Eq. (30) also seeks to minimize the cost associated with implementing each strategy, h(c*), noting again that c* is a pseudo-decision variable representing how risk management controls each sector. The minimization occurs with respect to the c* vector, which includes the distribution parameter b in the ith element of c*, and acts as the surrogate for each of the discrete strategies. 9 8 f5 > > > > > > > > > f > > > > = < 4 > c (30) min f5 > cn ;...;cn ;> > 1 i1 > > > > cf 4 > a;b;c; > > > cn ;...;cn > n> iþ1 ; : hðcn Þ > Note that the sensitivity index does not directly involve (i.e., no partial derivatives taken with respect to) two parameters that have an impact on the values of f5 and f4: a and Z. While a essentially shapes our definition of extreme event (e.g., a ¼ 0.10, or 1 in 10), sensitivity analyses are not performed. Similarly, it is assumed that while Z is elicited from the expert, the value of Z does not vary from risk management option to option, thereby being held fixed in the optimization problem. Should it be concluded that any sensitivity indices should account for these two parameters, they can easily be included to Eqs. (25) and (29).

ARTICLE IN PRESS K. Barker, Y.Y. Haimes / Reliability Engineering and System Safety 94 (2009) 819–829

One would expect that a costly investment in sound risk management strategies would reduce the economic impact experienced as a result of a disruptive event. Thus, the larger the investment, the larger is the reduction in economic impact. Such are competing objectives. Further, the sensitivity indices may also compete with the economic loss and cost objectives. Fundamental to multiobjective analysis is the notion of the Pareto optimum, or noninferior solution. A noninferior solution is defined as a solution to a multiobjective problem where any improvement in one objective comes only at the expense of another objective [71]. As cf 5 is not a function of any of the distribution parameters, it is not particularly interesting to study in a multiobjective analysis. Therefore the multiobjective optimization problem in Eq. (30) is reduced to only f4, cf 4 , and h(c*), thereby focusing on how the risk management strategies perform under extreme disruptive events. Incorporating the analytical results of Eqs. (21) and (29) into Eq. (30) results in the more descriptive multiobjective problem below, specifically for the insertion of a TSP distribution into the ith entry of c*. The objectives are evaluated at the nominal values ^ of b, or the expert-elicited value of the maximum value, denoted b. Note that a cost function is not explicit here, and the approach is designed for discrete risk management strategies whose inherent design is assumed to include an analysis of cost. # 9 8"   n n n > P P P n n n n> > Z ðbcÞZþ1 P > > xj djk ck þ b xj dji  Zþ1 aZ ðbaÞZ xj dji > > > > > > > > > j¼1 j¼1 j¼1 kai > > = < " #2 h i n min 2   P 2 n Z bc Z Z bc Zþ1 > cn ;...;cn ; > > xj dji 1  aZ ba þ aZ ðZþ1Þ ba 1 i1 > > > > > a;b;c; > > j¼1 > cn ;...;cn > > n > iþ1 > > ; : n hðc Þ ^ b¼b

(31) The use of the USIM is different here than that described in Refs. [15,16]. The USIM is used to minimize, with respect to a set of decision variables, a number of objectives and indices measuring the sensitivity of the objectives with respect to changes in uncertain parameters in the objectives. Here, however, the decision variables include the parameter that is uncertain: b. Associated with each proposed risk management strategy is a different vector of demand perturbations, as shown in Eq. (10). 5.1. Illustrative example Consider the same 15-industry example whose A* and x values are provided in Tables 2 and 3. Assume that a disruptive event could have a sustained two-week inoperative effect on the Mining sector, upon which the other 14 sectors rely. Five risk management strategies are developed to reduce the effects of the disruptive event on oil and gas operations, manifested in the Mining sector (Sector 2). Such strategies, which could include preparedness investments addressing system hardening, prepositioned recovery material, or inventory and storage capabilities, attempt to lessen the demand perturbation in the Mining sector as modeled in the IIM in the form of cn2 . There is some relationship between each strategy and cn2 whose quantification, while assumed to be known, is outside of the scope of this paper. The five risk management strategies, A–E, each result in a different set of TSP distribution parameters for cn2 , enumerated in Table 5. To simplify the example, it is assumed that the expertelicited cn2 follows a triangular distribution, where TSP shape parameter Z ¼ 2. Like the previous example, the remaining elements of c* are zero. The costs required to implement each strategy are also provided in Table 5. For example, the disruptive event is expected to have a demand perturbation on the Mining sector between 19% and 32% (following a TSP distribution with

827

Table 5 Anticipated Mining sector demand perturbation parameters and implementation costs (in 106 dollars) associated with the five risk management strategies Strategy

c2

Implementation cost

A B C D E

TSP(0.19, 0.32, 0.28, 2) TSP(0.15, 0.36, 0.31, 2) TSP(0.05, 0.25, 0.18, 2) TSP(0.20, 0.40, 0.25, 2) TSP(0.20, 0.50, 0.35, 2)

331 382 515 195 83

Table 6 Multiobjective metrics for each risk management strategy Strategy

f5 (106 dollars)

f4 (106 dollars)

cf 5 (106 dollars2)

cf 4 (106 dollars2)

A B C D E

7040 7310 4280 7570 9360

8150 9050 6020 9720 12,200

79 79 79 79 79

403 383 412 443 431

most likely value of 28%) should Strategy A be implemented at a cost of $331 million. Note that the example assumes that a disruptive event will occur and focuses on various preparedness strategies that lessen the adverse effects of the disruption rather than on preventative strategies which would theoretically result in 0% demand perturbations. The numerical results of the five strategies are provided in Table 6. Average and extreme economic losses, and the sensitivities of these values, are calculated for each strategy. As expected, the sensitivity index of the expected total economic loss is constant for Z ¼ 2. The sensitivity metric, cf 4 , represents the square of the slope of economic loss in relation to a change in demand perturbation. Essentially, these individual sensitivity values represent a measure of steepness, where a steeper slope results in greater sensitivity to uncertainty in the upper bound of the distribution of cn2 . As the numerical value of the sensitivity index may have little intuitive meaning to a decision maker (i.e., units are in dollars2), cf 4 could be scaled to range on [0, 1] over the set of strategies for ease in comparing strategies. Given our interest in the response of interdependent sectors to extreme disruptive events, we focus on the f4 and cf 4 columns of Table 6. Due to the multiobjective nature of the problem, quantitative tradeoffs exist among f4, cf 4 , and h(c*) for the five strategies. These tradeoffs are depicted graphically in Fig. 6. As the options are discrete, one approach to calculate tradeoffs is illustrated in Eq. (32) between Options A and B, derived from the continuous multiobjective tradeoff calculations found in, e.g., Ref. [71].



f 4 OptionA  f 4 Option B Df 4



lf 4 ;cjA;B ¼  ¼ (32)



Dcf 4 cf 4

 cf 4

Option A

Option B

Fig. 6 demonstrates the tradeoffs made between total economic loss from an extreme disruptive event, on the ordinate axis, and the sensitivity of that loss to the uncertainty in Mining sector demand perturbation, on the abscissa axis. As a causal relationship between implementation cost and the other two objectives is not known, nor is it assumed, h(c*) is depicted in Fig. 6 with bubble size. That is, implementation cost is an important consideration in the decision-making process, but quantitative tradeoffs between it and the other two objectives are not calculated.

ARTICLE IN PRESS 828

K. Barker, Y.Y. Haimes / Reliability Engineering and System Safety 94 (2009) 819–829

16000 14000 12000 E

10000 8000

D B

6000

A

4000

C

2000 0 370

380

390

400

410

420

430

440

450

460

Fig. 6. Graphical comparison of the five risk management strategies depicting tradeoffs among conditional expected total economic loss (ordinate axis), the sensitivity of that conditional expected value (abscissa axis), and the strategies’ implementation costs (size of the point). Losses are in 106 dollars, sensitivity indices are in 106.

Table 7 Quantitative tradeoffs between economic loss and sensitivity metrics for the five strategies (in dollars1) Strategies

A

B

C

D

E

A B C D E

– 45 237 39 144

45 – 104 11 65

237 104 – 119 324

39 11 119 – 204

144 65 324 204 –

In Fig. 6, we see that Strategies A–C are Pareto-optimal with respect to f4 economic loss and cf 4 and that Strategies D and E are inferior options. However, given the consideration of implementation cost, D and E are still considered. Quantitative tradeoffs between f4 and cf 4 are found in Table 7 for the five strategies. Tradeoffs are negative for A–C when compared with noninferior strategies D and E. For example, a decision maker, skeptical that the results of an interdependency model will correctly anticipate the behavior of interdependent sectors following an extreme disruptive event, may be less interested in potential economic losses and more interested in how sensitive the model is to uncertainty in expert evidence. Such a decision maker may desire a strategy which holds up well during extremes even when the expert is incorrect, opting for Strategy B, as Strategy A is far more sensitive with little reduction in total economic loss. Another decision maker may disregard such extreme events altogether and opt for the strategy that costs least to implement, Strategy E. The EE-USIM, and its associated quantitative and graphical tools, can accommodate a host of decision maker preferences.

6. Concluding remarks Assessments of extreme disruptive events tend to suffer from a lack of data that explain such events, as, naturally, such extreme events occur very infrequently. In place of empirically driven probability distributions describing such events, expert-elicited assessments of consequences are frequently used. And such elicitations may suffer from errors which may affect risk-based decision making. This paper develops a quantitative framework, the extreme event uncertainty sensitivity index method (EE-USIM), to measure the sensitivity of extreme event metrics to changes in underlying probability distribution parameters. The usefulness of the

EE-USIM, whose framework is depicted graphically in Fig. 1, is demonstrated for risk-based decision making; that is, sensitivity of model outputs to the distribution parameters of probabilistic inputs (from the original parameter assessments) is treated as part of a multiobjective approach with which to compare risk management strategies. The EE-USIM is demonstrated with the inoperability input–output model (IIM) [2,17,19,20,57,58], which quantifies the propagation of sector dysfunctionality throughout interdependent economic and infrastructure sectors. The IIM serves as an appropriate case study, as interdependent systems are increasingly apparent in our society, suffer from uncertainty in their response to a disruptive event, and are often the subject of risk-based decision making. Though the EE-USIM is illustrated with the IIM, thereby strengthening the IIM to potential errors in expert-elicited probability distribution parameters, the framework could be generalized to other models also containing probabilistic variables which may have uncertain distribution parameters. Further, this paper introduces the calculation of extreme event calculations for the two-sided power (TSP) distribution [31,32]. The TSP distribution was used here due to its usefulness in providing analytical results, unlike many other popular nonlinear probability distributions.

Acknowledgments The authors acknowledge the thoughtful discussions provided by Professors Joost R. Santos and James H. Lambert and the administrative support of Erika Evans, all of the Center for Risk Management of Engineering Systems at the University of Virginia. Further, the thoughtful comments made during the review process are much appreciated. References [1] Asbeck E, Haimes YY. The partitioned multiobjective risk method. Large Scale Syst 1984;6(1):13–38. [2] Haimes YY. Risk modeling, assessment, and management. 3rd ed. Hoboken, NJ: Wiley; 2008. [3] Frohwein HI, Lambert JH, Haimes YY. Alternative measures of risk of extreme events in decision trees. Reliab Eng Syst Saf 1999;66(1):69–84. [4] Taleb NN. Black swans and the domains of statistics. Am Stat 2007;61(3):198–200. [5] Lichtenstein S, Fischhoff B. Do those who know more also know more about how much they know? Organ Behav Hum Perform 1977;20(2):159–83. [6] Shlyakhter AI. An improved framework for uncertainty analysis: accounting for unsuspected errors. Risk Anal 1994;14(4):441–7. [7] Bier VM. Implications of the research on expert overconfidence and dependence. Reliab Eng Syst Saf 2004;85(1–3):321–9. [8] Lin S-W, Bier VM. A study of expert overconfidence. Reliab Eng Syst Saf 2008;93(5):711–21. [9] Oberkampf WL, Helton JC, Joslyn CA, Wojtkiewicz SF, Ferson S. Challenge problems: uncertainty in system response given uncertain parameters. Reliab Eng Syst Saf 2004;85:11–9. [10] Hoffman FO, Hammonds JS. Propagation of uncertainty in risk assessments: the need to distinguish between uncertainty due to lack of knowledge and uncertainty due to variability. Risk Anal 1994;14(5):707–12. [11] Helton JC. Treatment of uncertainty in performance assessments for complex systems. Risk Anal 1994;14(4):483–511. [12] Pate-Cornell ME. Uncertainties in risk analysis: six levels of treatment. Reliab Eng Syst Saf 1996;54(2):95–111. [13] Winkler RL. Uncertainty in probabilistic risk assessment. Reliab Eng Syst Saf 1996;54:127–32. [14] Barker K. Extensions of inoperability input–output modeling for preparedness decisionmaking: uncertainty and inventory. PhD. dissertation, University of Virginia, Charlottesville, VA, 2008. [15] Haimes YY, Hall WA. Sensitivity, responsivity, stability, and irreversibility as multiobjectives in civil systems. Adv Water Resour 1977;1:71–81. [16] Li D, Haimes YY. The uncertainty sensitivity index method (USIM) and its extension. Naval Res Logist 1988;35(6):655–72. [17] Haimes YY, Jiang P. Leontief-based model of risk in complex interconnected infrastructures. J Infrastruct Syst 2001;7(1):1–12. [18] Jiang P, Haimes YY. Risk management for Leontief-based interdependent systems. Risk Anal 2004;24(5):1215–29.

ARTICLE IN PRESS K. Barker, Y.Y. Haimes / Reliability Engineering and System Safety 94 (2009) 819–829

[19] Santos JR, Haimes YY. Modeling the demand reduction input–output (I–O) inoperability due to terrorism of interconnected infrastructures. Risk Anal 2004;24(6):1437–51. [20] Santos JR. Inoperability input–output modeling of disruptions to interdependent economic systems. Syst Eng 2006;9(1):20–34. [21] Vose D. Monte Carlo risk analysis modeling. In: Molak V, editor. Fundamentals of risk analysis and risk management. Boca Raton, FL: CRC Lewis Publishers; 1997. p. 45–66. [22] Rychlik I, Ryden J. Probability and risk analysis: an introduction for engineers. New York: Springer; 2006. [23] Mosleh A, Apostolakis G. The assessment of probability distributions from expert opinions with an application to seismic fragility curves. Risk Anal 1986;6(4):447–61. [24] Cooke RM. Experts in uncertainty: opinion and subjective probability in science. Oxford, UK: Oxford University Press; 1991. [25] Ayyub BM. Elicitation of expert opinions for uncertainty and risks. Boca Raton, FL: CRC Press; 2002. [26] Cooke RM, Goossens LHJ. Expert judgement elicitation for risk assessments of critical infrastructures. J Risk Res 2004;7(6):643–56. [27] Garthwaite PH, Kadane JB, O’Hagan A. Statistical methods for eliciting probability distributions. J Am Stat Assoc 2005;100(470):680–701. [28] Eschenbach TG, Gimpel RJ. Stochastic sensitivity analysis. Eng Econ 1990; 35(4):305–21. [29] Williams TM. Practical use of distributions in network analysis. J Oper Res Soc 1992;43(3):265–70. [30] Johnson D. The triangular distribution as a proxy for the beta distribution in risk analysis. J R Stat Soc Ser D 1997;46(3):387–98. [31] van Dorp JR, Kotz S. A novel extension of the triangular distribution and its parameter estimation. Statistician 2002;51:63–79. [32] van Dorp JR, Kotz S. The standard two-sided power distribution and its properties: with applications in financial engineering. Am Stat 2002;56(2):90–9. [33] van Dorp JR, Kotz S. Generalizations of two-sided power distributions and their convolution. Commun Stat 2003;32(9):1703–23. [34] Kotz S, van Dorp JR. Beyond beta: other continuous families of distributions with bounded support and applications. Hackensack, NJ: World Scientific Publishing; 2004. [35] Evans M, Hastings N, Peacock B. Statistical distributions. 2nd ed. New York: Wiley; 1993. [36] Olsen JR, Lambert JH, Haimes YY. Risk of extreme events under nonstationary conditions. Risk Anal 1998;18(4):497–510. [37] Leung MF, Lambert JH, Mosenthal A. A risk-based approach to setting priorities in protecting bridges against terrorist attacks. Risk Anal 2004;24(4): 963–84. [38] Santos JR, Haimes YY. Applying the partitioned multiobjective risk method (PMRM) to portfolio selection. Risk Anal 2004;24(3):697–713. [39] Frohwein HI, Lambert JH. Risk of extreme events in multiobjective decision trees, Part 1: Severe events. Risk Anal 2000;20(1):113–23. [40] Frohwein HI, Haimes YY, Lambert JH. Risk of extreme events in multiobjective decision trees, Part 2: Rare events. Risk Anal 2000;20(1):125–34. [41] Santos JR, Barker K, Zelinke IV PJ. Sequential decision-making in interdependent sectors with multiobjective inoperability decision trees: application to biofuel subsidy analysis. Econ Syst Res 2008;20(1):29–56. [42] Artzner P. Application of coherent risk measures to capital requirements in insurance. N Am Actuar J 1999;2(2):11–25. [43] Jorion P. Value at risk: the new benchmark for managing financial risk. 3rd ed. New York: McGraw-Hill; 2006. [44] Santos JR. Interdependency analysis with multiple probabilistic sector inputs. J Ind Manage Optim 2008;4(3):489–510. [45] Barker K, Haimes YY. Analysis of interdependency uncertainties in the dynamic inoperability input–output model. 2008, submitted for publication. [46] Brown T, Beyeler W, Barton D. Assessing infrastructure interdependencies: the challenge of risk analysis for complex adaptive systems. Int J Crit Infrastruct 2004;1(1):108–17.

829

[47] Bagheri E, Ghorbani AA. Conceptualizing critical infrastructures as service oriented complex interdependent systems. In: Proceedings of the international conference on information technology and management, 2007. [48] Tolone WJ, Wilson D, Raja A, Xiang W, Hao H, Phelps S, et al. Critical infrastructure integration modeling and simulation. In: Proceedings of second symposium on intelligence and security informatics, 2004. [49] Buzna L, Peters K, Helbing D. Modelling the dynamics of disaster spreading in networks. Phys A Stat Mech Appl 2006;363(1):132–40. [50] Buzna L, Peters K, Ammoser H, Kuhnert C, Helbing D. Efficient response to cascading disaster spreading. Phys Rev E 2007;75:056107. [51] Leontief WW. Quantitative input and output relations in the economic system of the United States. Rev Econ Stat 1936;18(3):105–25. [52] Leontief WW. Input–output economics. Sci Am 1951;October:15–21. [53] Leontief WW. The structure of the American economy, 1919–1939: an empirical application of equilibrium analysis. 2nd ed. New York: International Arts and Sciences Press; 1951. [54] Leontief WW. Input–output economics. New York: Oxford University Press; 1966. [55] Dorfman R. Wassily Leontief’s contribution to economics. Swed J Econ 1973; 75(4):430–49. [56] Dietzenbacher E, Lahr ML. Introduction. In: Lahr ML, Dietzenbacher E, editors. Input–output analysis: frontiers and extensions. New York: Palgrave; 2001. [57] Haimes YY, Horowitz BM, Lambert JH, Santos JR, Crowther K, Lian C. Inoperability input–output model for interdependent infrastructure sectors. I: Theory and methodology. J Infrastruct Syst 2005;11(2):67–79. [58] Haimes YY, Horowitz BM, Lambert JH, Santos JR, Crowther K, Lian C. Inoperability input–output model for interdependent infrastructure sectors. II: Case studies. J Infrastruct Syst 2005;11(2):80–92. [59] McCormick NJ. Reliability and risk analysis. New York: Academic Press; 1981. [60] Crowther KG, Haimes YY. Application of the inoperability input–output model (IIM) for systemic risk assessment and management of interdependent infrastructures. Syst Eng 2005;8(4):323–41. [61] Haimes YY, Chittister CG. A roadmap for quantifying the efficacy of risk management of information security and interdependent SCADA systems. J Homeland Security Emergency Manage 2005;2(2) [Article 12]. [62] Andrijcic E, Horowitz B. A macro-economic framework for evaluation of cyber security risks related to protection of intellectual property. Risk Anal 2006;26(4):907–23. [63] Anderson CW, Santos JR, Haimes YY. A risk-based input–output methodology for measuring the effects of the August 2003 northeast blackout. Econ Syst Res 2007;19(2):183–204. [64] Crowther KG, Haimes YY, Taub G. Systemic valuation of strategic preparedness with illustrations from Hurricane Katrina. Risk Anal 2007;27(5): 1345–64. [65] Hattis D, Burmaster DE. Assessment of variability and uncertainty distributions for practical risk analyses. Risk Anal 1994;14(5):713–30. [66] Rose A. Economic principles, issues, and research priorities in hazard loss estimation. In: Okuyama Y, Chang S, editors. Modeling the spatial economic impacts of natural hazards. Heidelberg: Springer; 2004. p. 13–36. [67] Lian C, Santos JR, Haimes YY. Extreme risk analysis of interdependent economic and infrastructure sectors. Risk Anal 2007;7(5):1283–97. [68] Hibbeler RC. Engineering mechanics: statics and dynamics. 8th ed. Upper Saddle River, NJ: Prentice-Hall; 1998. [69] Bureau of Economic Analysis. Interactive access to input–output accounts data, 2007 /http://www.bea.gov/industry/iotables/prod/table_list.cfm?anon= 54883S. Last accessed on 30 September 2007. [70] Crowther KG, Haimes YY, Development and deployment of the multiregional inoperability input–output model for strategic preparedness, Syst Eng 2009;13(1). [71] Chankong V, Haimes YY. Multiobjective decision making: theory and methodology. Mineola, NY: Dover Publications Inc.; 2008.