An application of Bayesian risk theory to electrochemical processes

Electrochimica Acta 49 (2004) 1397–1401

An application of Bayesian risk theory to electrochemical processes Thomas Z. Fahidy∗ Department of Chemical Engineering, University of Waterloo, Waterloo, Ont., Canada N2L 3G1 Received 22 August 2003; received in revised form 23 October 2003; accepted 28 October 2003

Abstract The utility of Bayesian risk theory, in minimizing risks encountered in action assignment, to electrochemical processes is illustrated by means of a cathodic deposition process accompanied by parasitic co-deposition. The effect of risk parameters and observation probabilities on the weighted-average Bayesian risk (WABR) is also discussed. © 2003 Elsevier Ltd. All rights reserved. Keywords: Bayesian risk; Electrodeposition; Prior and posterior probability

1. Introduction Bayes’ theorem, expressed in words as posterior probability equals likelihood times prior probability, has been a subject of controversy essentially from its inception. The major objection against it stems from the difficulty of expressing the prior quantitatively when it does not represent direct measurements of frequency. In contrast with Jeffrey’s assertion [1] that a rational degree of belief is the only valid concept of probability, Bulmer offers the supposition [2] that likelihoods are statistical probabilities, but the prior and posterior probabilities are inductive ones. In a somewhat different manner, Papoulis [3] defines an “objective” view of probability as a measure of averages, and a “subjective” view of probability as a measure of belief. Ironically, if the size of the experimental observation set (sample) is small, neither the Bayesian nor the non-Bayesian method is particularly reliable. If the size is large, the role of the prior becomes small and has no effect on the outcome [4]. This controversial situation is discussed in more detail by Earman [5], and in historical accounts [6,7]. Heavily increasing interest in Bayesian methods is manifested by numerous recent publications [8–10], and a continuing series of workshops and seminars [11]. In two earlier companion papers [12,13], the usefulness of the minimax strategy of decision theory [14–17] in electrochemical process technology was described. The approach ∗

Tel.: +1-519-888-4567x2092; fax: +1-519-746-4979. E-mail address: [email protected] (T.Z. Fahidy).

0013-4686/$ – see front matter © 2003 Elsevier Ltd. All rights reserved. doi:10.1016/j.electacta.2003.10.027

selects a mixture of admissible pure strategies to minimize maximum losses resulting from the employment of actions which carry certain penalties with choice. An alternative to the minimax approach, based on the concept of Bayesian risk, is mathematically less involved and carries the additional advantage of updating the results of analysis when (further) empirical evidence is available for the functioning of a particular process (a brief numerical illustration of the Bayesian risk principle is given in (Appendix A). In-depth treatments may be found in the seminal texts by Bunn [18] and LaValle [19]. The purpose of this paper is to demonstrate by relatively simple examples the application of the Bayesian risk method to electrochemical processes by considering two levels of operation (two states), and three separate action choices. The low level of dimensions chosen limits the extent of sophistication and scope as well as mathematical encumbrance, but it is sufficient for the understanding of the subject matter, and specifically, for illustrating the required computation paths.

2. Theory Let θ 1 and θ 2 denote two separate levels of operation, also called states, with prior probabilities w1 and w2 , respectively. Let a1 , a2 , and a3 denote actions contemplated in view of the states; each action carries its own risk r(θ i , aj ); i = 1, 2; j = 1, 2, 3 representing penalty ascribed to each action, if the action is not the right one to choose. The smallest of risk functions: B(w1 , w2 , a1 ) =

1398

T.Z. Fahidy / Electrochimica Acta 49 (2004) 1397–1401

r(θ1 ; a1 )w1 + r(θ2 ; a1 )w2 ; B(w1 , w2 , a2 ) = r(θ1 ; a2 )w1 + r(θ2 ; a2 )w2 ; B(w1 , w2 , a3 ) = r(θ1 ; a3 )w1 + r(θ2 ; a3 )w2 is called the Bayesian risk, and it defines the appropriate minimizing action (if, e.g. B(w1 , w2 , a2 ) is the smallest, then a2 is the appropriate choice of action). If, in addition, empirical observations of relative frequencies of actions having occurred in the past, or occurring at present (e.g. in certain reactors, plants, etc.) are known, a posterior state probability array can be established, with array elements fij wi ; i = 1, 2 and j = 1, 2, 3 where fij are the relative frequencies pertaining to state θ i and action aj . The posterior probabilities are computed as W1j =

f1j w1 ; f1j w1 + f2j w2

j = 1, 2, 3

(1)

W2j =

f2j w2 ; f1j w1 + f2j w2

j = 1, 2, 3

(2)

The minimized Bayesian risk Bmin = {min(B1 ); min(B2 ); min(B3 )}

(3)

is the set of individually minimized components Min(Bj ) = min{(r11 W1j + r21 W2j ); (r12 W1j + r22 W2j ); (r13 W1j + r23 W2j )};

j = 1, 2, 3

(4)

and it defines minimum-risk actions for observation sets (f11 ; f21 ); (f12 ; f22 ), and (f13 ; f23 ), given risks (r1j ; r2j ) assigned to action aj ; j = 1, 2, 3. Finally, a weighted-average of risks corresponding to the Bayes strategy may be obtained as WABR = F1 min(B1 ) + F2 min(B2 ) + F3 min(B3 ) j = 1, 2, 3

w1

w2

B(w; a1 )

B(w; a2 )

B(w; a3 )

min B(w; a)

Actiona

0.3 0.4 0.5 0.6 0.7

0.7 0.6 0.5 0.4 0.3

3.50 3.00 2.50 2.00 1.50

1.60 1.80 2.00 2.20 2.40

2.10 2.80 3.50 4.20 4.90

1.60 1.80 2.00 2.00 1.50

a2 a2 a2 a1 a1

a

In the Bayesian sense.

decisions of: (i) changing cells when operating conditions are admissible, and conversely, (ii) operating at conditions which favor inadmissible cell performance. The assignment of zero to correct state/action combinations, i.e. (θ 1 ; a1 ); (θ 2 ; a3 ) is conventional. The prior state probabilities are considered to be variable, not only for the sake of numerical illustration, but also due to their somewhat subjective nature emanating from plant experience, managerial factors, etc. In a three-state variation of the theme, the current efficiency domains might be envisaged, e.g. as CE ≥ 90%, CE < 80% and 80% < CE < 90%. The assignment of risk factors may require more circumspection, and the risk factor array elements may be widely different in a numerical sense. There is a point of diminishing returns in increasing the number of allowable actions. 3.1. Bayesian risk based on solely prior state probabilities

(5)

where Fj ≡ f1j w1 + f2j w2 ;

Table 1 Bayesian risks computed on the basis of prior probabilities of cell operating states, as a function of prior state probabilities

(6)

are the denominator in Eqs. (1) and (2).

3. Illustration for an electrochemical process: cathodic deposition with a parasitic side reaction

Table 1 summarizes the computations involved and the related Bayesian actions, with details shown in the Appendix B. Not unexpectedly, the choice of maintaining the current operational state is justified only if operation with acceptable energy losses is more than 50% probable. The w1 < 0.3; w1 > 0.7 range is assumed to be impractical, hence omitted from the tabulation. 3.2. Bayesian risk based on posterior state probabilities

A cathodic deposition process is envisaged to operate at a current efficiency CE = 90% with acceptable energy loss (state θ 1 ), or at CE < 90% with unacceptable energy loss (state θ 2 ). The energy loss is due to a parallel parasitic reaction (e.g. proton discharge). The proposed actions are a1 : operation with existing cells within a certain current density range; a2 : modification of certain parameters and/or operating conditions (e.g. current density/potential range; electrolyte temperature and/or flow rate; changing to intermittent or pulsed electrode potential, etc.) in existing cells; a3 : design and installation of conceptually new cells. Risk factors, arising from an economic analysis, are set as r11 = 0; r12 = 3; r13 = 7; r21 = 5; r22 = 1; r23 = 0. The numerical values portray a heavy penalty assigned to the highly questionable

While analysis in Section 3.1 excludes actual observations of (θ i :aj ) combinations, it is assumed in this section that such data are at the disposal of the process analyst from various sources, e.g. technical publications, plant records, the Internet, and personal communications. Accordingly, the array elements f11 = 0.85; f12 = 0.08; f13 = 0.07, and f21 = 0.10; f22 = 0.15; f23 = 0.75 are postulated (f11 = 0.85 indicates that 85% of cells reported by sources are conventional and operate under existing conditions; f12 = 0.08 indicates that 8% of reported cells operate under existing conditions, but with some modifications of cell parameters, etc.). As shown in Table 2, the weighted-average Bayesian risk (WABR) is a mild function of prior state probabilities, when other parameters are fixed.

T.Z. Fahidy / Electrochimica Acta 49 (2004) 1397–1401

1399

Table 2 Bayesian risks computed on the basis of posterior probabilities of cell operating states, as a function of prior state probabilities. Severe penalty case

Table 5 The effect of the observation probability structure. Severe penalty case f-structure

Y1

Y2

Y3

Actionsa

WABR

w1

w2

Y1

Y2

0.3 0.4 0.5 0.6 0.7

0.7 0.6 0.5 0.4 0.3

1.075 0.750 0.525 0.365 0.240

1.302 1.524 1.696 1.888 2.108

A B C D

0.4 0.525 1.667 1.5

1.625 1.696 2 1.736

0.250 0.595 1.667 2.090

a1 ; a1 ; a1 ; a1 ;

0.435 0.688 1.750 1.774

f11

f12

f13

f21

f22

f23

0.92 0.85 0.50 0.35

0.05 0.08 0.25 0.35

0.03 0.07 0.25 0.30

0.08 0.10 0.25 0.15

0.10 0.15 0.25 0.60

0.82 0.75 0.50 0.25

Y3

Actionsa

WABR

0.266 0.413 0.595 0.861 1.253

a1 ; a1 ; a1 : a1 ; a1 ;

0.662 0.683 0.688 0.699 0.706

a3 ; a2 ; a2 ; a2 ; a2 ;

a3 a3 a3 a3 a3

Yj ≡min Bj (w; a); j = 1, 2, 3. a In the Bayesian sense. Table 3 Bayesian risks computed on the basis of posterior probabilities of cell operating states, as a function of prior state probabilities. Mild penalty case w1

w2

Y1

Y2

Y3

Actionsa

WABR

0.3 0.4 0.5 0.6 0.7

0.7 0.6 0.5 0.4 0.3

0.645 0.295 0.315 0.219 0.144

0.930 1 1 1 1

1 0.295 0.425 0.615 0.895

a1 ; a3 ; a1 ; a1 ; a1 ;

0.433 0.381 0.439 0.439 0.436

a

a3 ; a2 ; a2 ; a2 ; a2 ;

a3 a3 a3 a3 a3

In the Bayesian sense; Yj = min Bj (w; a); j = 1, 2, 3.

A B C D

a2 ; a2 ; a2 ; a2 ;

a3 a3 a2 a2

w1 = w2 = 1/2. a In the Bayesian sense; Y = min B (w, a); j = 1, 2, 3. j j

fluenced by observation probabilities, as depicted in Table 5. The assignment of very low risk, e.g. to the (θ 1 ; a1 ) and the (θ 2 ; a3 ) state/action pair creates relatively large WABR values if the f11 and f23 frequencies are low. If they are below a certain critical value, the option of a conceptually new cell design/installation (action a3 ) is not advised. Conversely, if f11 and f23 are above this critical value, the action set (a1 ; a2 ; a3 ) yields invariably the smallest available WABR.

4. Discussion

4.3. Analysis of Bayesian risks

4.1. The effect of risk (regret) parameters

In the absence of empirical information from operating electrolyzers, Bayesian risks can be computed only on the basis of prior state probabilities and risk parameters. The minimal value of the Bayesian risk identifies a singular action whose nature depends on the prior state probabilities. If nothing is known about the prior probability distribution, each state is (conventionally) assumed to possess the same probability. When empirical information is available, a set of Bayesian actions can be obtained whose dimension is determined by experimental fractional frequencies of electrolyzers working under various state/action configurations. The results shown in Tables 2–5 stem from a full set of such frequencies, where all three actions are considered to be valid in the presence of both states. This “full-set” scenario is postulated for the sake of illustration of the numerical procedure, with the understanding that in a given technological situation only partial sets may be attainable. If, for instance, observations are limited to f11 = 0.85; f21 = 0.10 in the severe penalty case in Table 2, and if w1 = w2 = 1/2, the minimum value 0.525 indicates that a1 is the right action, since there is no average-weighting possible in the absence of other observations. With a full contingent, the WABR value computed in the Appendix A may be interpreted as the least Bayesian risk encountered if retention of existing cells (a1 ) carries a 47.5%, cell modifications (a2 ) carries an 11.5%, and installation of a conceptually new cells (a3 ) carries a 41.0% probability. The choice is essentially between actions a1 and a3 , in this case.

The assignment of risk parameters is of major importance, as demonstrated by comparison of Tables 2 and 3. The latter is based on a less stringent risk array with elements r11 = 0; r12 = 1; r13 = 5, and r21 = 3; r22 = 1; r23 = 0. Table 4 shows that a symmetric structure (demonstrated for the specific case of r11 = r23 = 0; r12 = r22 = 1; r13 = r21 = 5) does not necessarily yield a minimal WABR set, although it might be an inviting choice, at least as a first attempt, dictated by ignorance, or superficial knowledge of individual risk factors. 4.2. The effect of observation probabilities Quantitative information obtained from posterior state probability-based estimation of Bayesian risks is strongly inTable 4 Bayesian risks computed on the basis of posterior probabilities of operating states, as a function of prior state probabilities w1

w2

Y1

Y2

Y3

Actiona

0.3 0.4 0.5 0.6 0.7

0.7 0.6 0.5 0.4 0.3

1.075 0.750 0.525 0.365 0.240

1 1 1 1 1

0.190 0.295 0.425 0.615 0.895

a2 ; a1 ; a1 ; a1 ; a1 ;

a3 ; a2 ; a2 ; a2 ; a2 ;

WABR a3 a3 a3 a3 a3

Symmetric penalty structure case. a In the Bayesian sense; Y = min B (w, a); j = 1, 2, 3. j j

0.549 0.563 0.539 0.519 0.496

1400

T.Z. Fahidy / Electrochimica Acta 49 (2004) 1397–1401

4.4. Higher dimensions

Appendix B

Extension of the approach to higher dimensions is rather straightforward, although beyond the scope of the current paper. In general, the Bayesian risk would carry a prior probability vector w = (w1 , w2 , . . . , wK ) related to a total of K states of operation, and an action vector a = (a1, a2 , . . . , aN ) related to a total of N actions, etc. with a concomitant increase in computational complexity.

Detailed computation of the weighted-average of Bayesian risks (WABRs)

5. Concluding remarks The weighted-average of risks corresponding to the Bayesian strategy is an attractive alternative to analysis based on the more involved minimax optimization of expected loss utility. Not only are the computations simpler, but the WABR principle yields the same decisions as the weighted average of utility losses, inasmuch as it is also minimized by the minimization of WABR. The statement: “. . . it is customary for statisticians to compute automatically regrets instead of losses and not even bother with expected losses . . . ” [20] may be taken as an indirect endorsement of the Bayesian approach.

Acknowledgements

Actions a1 : maintain operation with existing cells within a set current density range a2 : modify certain parameters and/or operating conditions in existing cells a3 : design, install and operate conceptually new cells Risk (regret) factors (severe penalty case) r11 = 0; r12 = 3; r13 = 7; r21 = 5 r22 = 1; r23 = 0 Prior state probabilities w1 = w2 = 1/2 Observed fractional operation occurrences/probabilities f11 = 0.85; f12 = 0.08; f13 = 0.07 f21 = 0.10; f22 = 0.15; f13 = 0.75

(a) Posterior state probabilities

Author’s work in this area has been supported by the Natural Sciences and Engineering Research Council of Canada (NSERC), and by facilities provided by the University of Waterloo.

Eq. (1) W11 = (0.85)(0.5)/([(0.85)(0.5) + (0.1)(0.5)] = 0.895 W12 = (0.08)(0.5)/[(0.08)(0.5) + (0.15)(0.5)] = 0.348 W13 = (0.07)(0.5)/[(0.07)(0.5) + (0.75)(0.5)] = 0.085 Eq. (2) W21 = 1 − 0.895 = 0.105 W22 = 1 − 0.348 = 0.652 W23 = 1 − 0.085 = 0.915

Appendix A A numerical illustration of the Bayesian risk principle. Consider two states θ 1 and θ 2 with probabilities w1 = 0.7 and w2 = 0.3, respectively. Assume that three actions are dictated by various considerations, carrying risk factors shown in the array below. θ1 θ2

States θ 1 : cells operate at CE = 90% θ 2 : cells operate at CE < 80%

a1

a2

a3

w

0 6

2 1

7 1

0.7 0.3

The Bayesian risks are B(w, a1 ) = (0.7)(0) + (0.3)(6) = 1.8; B(w, a2 ) = (0.7)(2) + (0.3)(1) = 1.7; B(w, a3 ) = (0.7)(7)+(0.3)(1) = 5.2. Their smallest element, 1.7, is the minimum Bayesian risk, hence a2 is the Bayesian action. Extension to a larger number of states is straightforward; if, e.g. a fourth action is also envisaged with risk factor set (8, 0), the Bayesian risk set is augmented by B(w, a4 ) = (0.7)(8) + (0.3)(0) = 5.6, with the same minimum element.

(2) Bayesian risks (rij = r(θi ; aj ); i = 1, 2; j = 1, 2, 3) r11 W11 r21 W21 r12 W11 r22 W21 r13 W11 r23 W21 r11 W12 r21 W22 r12 W12 r22 W22 r13 W12 r23 W22 r11 W13 r21 W23 r12 W13 r22 W23 r13 W13 r23 W23

(0)(0.895) (5)(0.105) (3)(0.895) (1)(0.105) (7)(0.895) (0)(0.105) (0)(0.348) (5)(0.652) (3)(0.348) (1)(0.652) (7)(0.343) (0)(0.652) (0)(0.085) (5)(0.915) (3)(0.085) (1)(0.915) (7)(0.085) (0)(0.915)

0 0.525 2.685 0.105 6.265 0 0 3.260 1.044 0.652 2.436 0 0 4.575 0.255 0.915 0.595 0

T.Z. Fahidy / Electrochimica Acta 49 (2004) 1397–1401

Eq. (4): min(B1 ) = min{0.525; 2.685 + 0.105 = 2.790; 6.265} = 0.525 → a1 min(B2 ) = min{3.260; 1.044 + 0.652 = 1.696; 2.436} = 1.696 → a2 min(B3 ) = min{4, 575; 0.255 + 0.915 = 1.170; 0.595} = 0.595 → a3 Eq. (6): F1 = f11 w1 + f21 w2 = (0.85)(0.5) + (0.10)(0.5) = 0.475 F2 = f12 w1 + f22 w2 = (0.08)(0.5) + (0.15)(0.5) = 0.115 F3 = f13 w1 + f23 w2 = (0.07)(0.5) + (0.75)(0.5) = 0.410 (c) Weighted-average Bayesian risk Eq. (5): WABR = (0.475)(0.525) + (0.115)(1.696) + (0.410)(0.595) = 0.688 implying action a1 at 47.5%, action a2 at 11.5%, action a3 at 41.0% probability List of symbols aj jth action; j = 1, 2, 3 (in general, j = 1, . . . , N); a: vector representation B(w1 , w2 , aj ) Bayesian risk due to action aj , given prior state probabilities w1 , and w2 fij fractional observation frequency of the jth action in the presence of the ith state R(␪i , aj ) risk (regret) factor assigned to the ith state, and the jth action; denoted also as rij WABR weighted-average Bayesian risk, defined by Eq. (5) wi prior probability of state θ I ; w: vector representation Wi posterior probability of state θ i in the presence of action aj

1401

Greek symbols θi ith state; i = 1,2 (in general, i = 1, . . . K)

References [1] H. Jeffreys, Theory of Probability, third ed., Oxford University Press, Oxford, 1973. [2] M.G. Bulmer, Principles of Statistics, Dover, NY, 1979, p. 174. [3] A. Papoulis, Probability and Statistics, Prentice Hall, Englewood Cliffs, NJ, 1990, pp. 247–248. [4] A. Papoulis, Loc. Cit., Example 8.4, pp. 248–249. [5] J. Earman, Bayes or Bust? A Critical Examination of Bayesian Confirmation Theory, MIT Press, Cambridge, MA, 1992. [6] A. Houle, Bibliographie: Statistique Bayésienne, Faculté des Sciences de l’Administration, Université Laval, Québec, 1973. [7] J.D. Weber, Historical Aspects of the Bayesian Controversy With Comprehensive Bibliography, Division of Economic and Business Research, Tucson, AZ, 1973. [8] A. Pole, Applied Bayesian Forecasting Time Series Analysis, Chapman and Hall, New York, 1994. [9] B.P. Carlin, Bayes and Empirical Bayes Methods for Data Analysis, second ed., Chapman and Hall/CRC Press, Boca Raton, FL, 2000. [10] P. Congdon, Applied Bayesian Modelling, Wiley, Hoboken, NJ, 2003. [11] http://www.prlab.ee.memphis.edu. [12] T.Z. Fahidy, Electrochim. Acta 44 (1999) 3559. [13] T.Z. Fahidy, Electrochim. Acta 47 (2002) 4441. [14] J.W. Piatt, H. Raiffa, R. Schlaifer, Introduction to Statistical Decision Theory, McGraw-Hill, New York, 1965. [15] H. Raiffa, Decision Analysis: Introductory Lectures on Choices Under Uncertainty, Addison-Wesley, Reading, MA, 1968. [16] R. Schlaifer, Analysis of Decision Under Uncertainty, McGraw-Hill, New York, 1969. [17] H. Chernoff, L.E. Moses, Elementary Decision Theory, Dover, New York, 1969. [18] D.W. Bunn, Applied Decision Analysis, McGraw-Hill, New York, 1984. [19] I.H. LaValle, An Introduction to Probability Decision and Inference, Holt Rinehart and Winston, New York, 1970. [20] H. Chernoff, L.E. Moses, Loc. Cit. Chapter 5, Section 6, pp. 154– 155.