- Email: [email protected]
Uncertainty in incident rates for trucks carrying dangerous goods
Accident Analysis and Prevention 32 (2000) 797 – 804 www.elsevier.com/locate/aap
Uncertainty in incident rates for trucks carrying dangerous goods Nancy P. Button a,*, Park M. Reilly b b
a Department of Ci6il Engineering, Uni6ersity of Waterloo, Waterloo, Ont., Canada N2L 3G1 Department of Chemical Engineering, Uni6ersity of Waterloo, Waterloo, Ont., Canada N2L 3G1
Received 10 August 1999; received in revised form 2 December 1999; accepted 7 December 1999
Abstract This paper addresses the uncertainty associated with release and fire incident rates for trucks in transit carrying dangerous goods. The research extends the treatment of uncertainty beyond sensitivity analysis, low – best – high estimates and confidence intervals, and represents the uncertainty through probability density functions. The analysis uses Monte Carlo simulations to propagate the uncertainty in the input variables through to the resulting release and fire incident rates. The paper illustrates how we can combine information on accident and non-accident releases and fires to generate probability density functions for the total expected releases and fires per billion vehicle kilometres for trucks carrying dangerous goods. © 2000 Elsevier Science Ltd. All rights reserved. Keywords: Trucks; Dangerous goods; Risk; Uncertainty
1. Introduction
2. Research problem
This paper illustrates how we can generate probability density functions (PDFs) for the expected release and fire incident rates for trucks in transit carrying dangerous goods (DG). The incident rates are given in terms of the number of incidents per billion vehicle kilometres (Bvkm). The paper combines information on accident and non-accident releases and fires to provide total expected incident rates. The model incorporates characteristics of roads and trucks that significantly affect the incident rates. The analysis represents the uncertainty in the input variables through PDFs, and uses Monte Carlo simulations to propagate the uncertainty in the input variables through to the resulting release and fire incident rates.
We do not understand and cannot predict the world precisely, yet analysts have often developed models based on a single value or ‘point estimate’ for each input variable, ignoring uncertainty. Often the point estimate is the mean value of the sample data. If the analyst gives only one value, then all further calculations use that value, even though a range of values may exist in the mind of the analyst. The range of values represents uncertainty, arising because the value is changeable (for example, because it fluctuates from year to year) or is not known precisely. Uncertainty can cause significant discrepancies in predicted risk and can influence decisions regarding modes and routes for the transport of DG. For example, using point estimates of the input variables we can calculate a point estimate of the incident rate on a route. On the other hand, if we incorporate the uncertainty in the input variables into the analysis, we can estimate a PDF for the incident rate on the same route. The point estimate is not necessarily the mean of the PDF. The PDF may lead us to different conclusions than the point estimate. We prefer to use PDFs rather
* Corresponding author. Present address: 16 Karen Avenue, Guelph, Ontario, Canada N1G 2N9. Tel.: +1-519-8248406; fax: +1-519-8861697. E-mail addresses: oh – [email protected] (N.P. Button)., [email protected] (P.M. Reilly).
0001-4575/00/$ - see front matter © 2000 Elsevier Science Ltd. All rights reserved. PII: S 0 0 0 1 - 4 5 7 5 ( 0 0 ) 0 0 0 0 3 - 8
798
N.P. Button, P.M. Reilly / Accident Analysis and Pre6ention 32 (2000) 797–804
than point estimates because the PDFs communicate more of the available information, help to put the incident rates in context, and allow for appropriate use of the rates in future risk assessment. Several studies have addressed the uncertainty associated with the risk of transporting DG by providing sensitivity analysis, low – best – high estimates, or confidence intervals (Leeming and Saccomanno, 1993; Saccomanno et al., 1993; Shortreed et al., 1994). However, a literature review did not reveal any examples of PDFs for release and fire incident rates. The analysis of uncertainty in this paper assigns PDFs to the input variables. The analysis then uses Monte Carlo simulations to propagate the uncertainty in the input variables through the model and generate PDFs for the resulting incident rates.
3. Research scope We use four outcome characteristics to classify DG incidents: 1. Release or no release. Every truck carries DG, in its fuel tank. However, this paper specifically addresses trucks carrying DG loads in addition to their fuel tank. The analysis defines a ‘release’ as a spill or leak of the DG load. 2. Fire or no fire. Due to the scarcity of fire and explosion data, we combine fire and explosions under a category called simply ‘fires’. 3. Release type (spill or leak). A spill is an immediate or continuous release of the DG from its containment, usually of short duration. A leak is a sporadic release, usually of a long duration. 4. Release size. Both spills and leaks can be large ( ] 1000 l) or small (B1000 l). The possible types of DG incidents include the ten possible combinations of the above characteristics, such as a large spill with fire, a small leak with no fire, a fire with no release, etc. The paper addresses both accident and non-accident releases and fires from trucks carrying DG. An accident is defined as an event involving one or a combination of a collision, an overturn or the truck running off the road. A non-accident incident could include a release that occurs, for example, if a hatch or valve is not properly closed, if a package falls off the truck, or if a fire starts from a brake or tire overheating during transport. We consider four types of DG including: 1. DG1: liquefied gases that are non-combustible but toxic and/or corrosive, such as liquefied chlorine gas. 2. DG2: liquefied gases that are flammable, such as propane. 3. DG3: flammable liquids, such as gasoline. 4. DG4: liquids that are non-combustible but toxic and/or corrosive, such as pesticides.
We did not include an analysis of accident rates in the scope of this research. The required accident rates for a study of this sort are often available from observed data or from estimates specific to the section of road being considered.
4. Data sources The research uses three databases containing records of DG incidents involving trucks in transit: 1. The Dangerous Goods Accident Information System (DGAIS) from Transport Canada (1988–1995). 2. The Occurrence Report Information System (ORIS) from the Ministry of Environment of Ontario (1988– 1997). 3. The Mission Transport des Matie`res Dangereuses (MTMD) from France (1987–1992). and two databases containing records of general truck accidents: 1. The Accident Data System (ADS) from the Ministry of Transportation of Ontario (1988–1995). 2. The Master Accident Record System (MARS) from the Washington State Department of Transportation (1990–1996). Our model contains statistical relationships rather than cause–effect relationships, because none of our databases contain information about the sequence of events in an incident. For example, there may be an incident with a release and fire. We do not know whether the fire started with say the vehicle fuel tank and then propagated to the DG load, or whether the fire started with the DG load and then spread to the rest of the vehicle. We only know that both a release and fire occurred. In determining the proportion of DG incidents with certain characteristics, we sometimes have two data sources. We assume that estimates provided by the data sources come from the same population, and differences in the estimates can be explained by differences in reporting. Therefore, where there are two sources, the best estimate is a combination of the sources.
5. The model We have insufficient DG incident data to cross-tabulate by every outcome characteristic without empty cells. For example, there are no records in our data of a truck carrying DG and having a large non-accident leak with a fire. However, we believe that the probability is greater than 0 that a large leak with a fire will occur given a truck carrying DG. We combine the conditional probabilities of each of the outcome characteristics to provide the probability of each type of DG incident.
N.P. Button, P.M. Reilly / Accident Analysis and Pre6ention 32 (2000) 797–804
The model includes 32 accident scenarios, based on the type of accident (collision and/or overturn), load size (large or small) and DG. It also includes 16 non-accident scenarios, based on the type of truck (tanker or non-tanker), road (urban or rural), and DG. There are ten possible outcomes for a DG incident, such as a large spill with fire, a small leak with no fire, a fire with no release, etc. For each of the scenario and outcome combinations, a different model equation combines the conditional probabilities of the outcome characteristics. This equation includes only statistically significant factors and, if the terms of the equation are dependent, the relationships between terms. For example, for a truck carrying a large load of DG1 and involved in an overturn and a collision, the equation for the probability of a large spill with fire reduces to: Pr(large spill with fire truck carrying a large load of DG1 with an overturn and collision) = Pr(release truck carrying DG1 involved in an accident with an overturn) × Pr(fire accident with a collision and a release)
assume that the input variables in our model that are rates of releases have lognormal PDFs, and we fit lognormal distributions to the available data.
6.2. Probability density functions for input 6ariables bounded by 0 and 1 Most of our input variables are probabilities, p, such as Pr(fire collision). These random variables are continuous and are bounded by 0 and 1. For reasons that will be explained, the beta distribution is the natural expression of the uncertainty in these probabilities. It takes the form: PDF(p; a, b)=
G(a + b) a − 1 p (1−p)b − 1 G(a)G(b)
6. Uncertainty in input values We assign a PDF to each input variable that incorporates the range of possible values over time and from each data source. Where there are larger differences over time or between data sources, the PDFs are wider, i.e. they have larger standard deviations. There are not enough sample data to fit PDFs to the input variables using goodness-of-fit measures. Therefore, we select an appropriate form for each PDF as discussed below.
6.1. Probability density functions for input 6ariables bounded by 0 The first input variable for non-accident scenarios is the rate of releases per Bvkm. The rate of releases is a continuous random variable and must be greater than 0, but has no practical upper limit. We know that the expected rate of release cannot be 0 because we have observations of releases occurring. Usually, we expect that the distribution would have one mode and be positively skewed, which corresponds with the observed values for the rate of releases per Bvkm. Henrion (1995) notes that if there is a sharp lower bound of 0 for a quantity, but no sharp upper bound, a single mode, and right skew, then the lognormal or gamma distributions are good candidates for empirical PDFs, with the lognormal being used more widely. We
(2)
where a and b are parameters of the beta distribution, such that: Mean or expected value of p= E(p)=
× Pr(spill release from a large load) × Pr(large release release that is a spill from a large load) (1) Our model includes over 400 such equations.
799
Variance of p= Var(p)=
a a+ b
ab (a+ b)2 (a+b+ 1)
The beta distribution is commonly used to represent the uncertainty in the probability of occurrence of an event, because its range is limited between 0 and 1. As noted by Lindley (1965), the family of beta distributions has the important property that if p has a beta distribution, then (1− p) also has a beta distribution, but with the parameters a and b interchanged. This makes it particularly useful if p is the probability of success, for (1 − p) is then the probability of failure and has a distribution of the same family. The beta distribution is also very flexible in terms of the wide variety of shapes it can assume, including positively or negatively skewed, depending on the values of its parameters. We assume that the input variables in our model that are probabilities have beta PDFs. We can estimate the parameters of a beta PDF for Pr(p) using Bayes’ theorem and an observed number of DG incidents, or trials, with or without a given characteristic. In statistical terms, a relevant DG incident with the characteristic is called a success and without the characteristic is called a failure. For a set of trials, the probability function for x successes has a binomial distribution as follows: P(x; p, n)=
n x p (1−p)y x
(3)
where x is the number of successes, y represents the number of failures, n, the number of trials equals x+y and p the probability of success on each trial.
800
N.P. Button, P.M. Reilly / Accident Analysis and Pre6ention 32 (2000) 797–804 k
Therefore the likelihood is also binomial: l(p; x, y)8 p x(1− p)y
% (mi /s 2i ) (4)
Lindley (1965) suggests that the family of beta distributions is the natural one to consider as prior distributions for the probability p of success, with the form p a (1− p)b. We want to chose values for a and b that provide a non-informative prior. Lindley (1965) notes that, since any observation always increases either a or b, it corresponds to the greatest possible ignorance to take a and b as small as possible. For the prior density to have a total integral 1 it is necessary and sufficient that both a and b exceed −1. Lindley (1965) therefore recommends that a=b = −1. Lindley’s prior has the PDF: PDF(p)8p − 1(1−p) − 1
(5)
If we use Bayes’ theorem to combine Lindley’s prior with our likelihood function, then the posterior PDF is: PDF(p; x, y)8 p x − 1(1 − p)y − 1
(6)
Therefore for a set of trials with x successes and y failures, we can assign to p a posterior beta distribution, PDF(p; x, y). When there is a large number of trials, the PDF is narrow. When there is a small number of trials, the PDF is wider. For each input variable, we have several sets of trials with successes and failures, from different data sources and from different time periods within each data source. Each set of trials provides a different estimate of the beta distribution for the input variable. If we simply sum the successes and failures and use the totals to estimate a beta distribution, the beta distribution becomes too narrow and does not reflect the uncertainty between data sources and between time periods. Therefore, we use an overall beta distribution that has the same mean and variance as the following mixture of beta distributions from the different sets of trials: Overall beta distribution=PDF(p; p1, p2, p3,...,pk ) k
= % li PDF(pi ; xi, yi )
(7)
i=1
where p is the overall probability of success, pi is the probability of success on each trial in trials set i, k is the number of sets of trials, xi is the number of successes in trials set i, yi is the number of failures in trials set i and PDF(pi; xi, yi ) is the beta PDF for the ith set of trials. li is the weighting factor for the ith beta distribution, where ki= 1 li = 1and the value of each li is proportional to the reciprocal of its variance, i.e. the beta distribution from each set of trials is weighted in proportion to its precision. We calculate the mean and variance of the overall beta PDF as follows:
E(p)=
i=1 k
(8)
% (1/s ) 2 i
i=1 k
k+ % (m 2i /s 2i ) Var(p)=
i=1 k
− [E(p)]2
(9)
% (1/s ) 2 i
i=1
where mi =
xi xi + yi
s 2i =
xi yi (xi + yi ) (xi + yi + 1) 2
We then calculate the values of the parameters, a and b, for the overall beta distribution by solving the standard expressions for the mean and variance of the beta distribution, as given in Eq. (2). Fig. 1 shows the beta distributions for an example input variable, Pr(fire release) for a truck carrying DG and involved in a collision. The figure shows the posterior beta distributions for each set of trials for the example input variable as well as the overall beta distribution for PDF(p; p1, p2, p3, …, pk ).
7. Output distributions Monte Carlo simulations propagate the uncertainty from the input variables and provide PDFs for each of the output variables. Most of the output PDFs turned out to be lognormal in shape, as expected. The output variables are products of several random input variables. Hence, the logarithms of the output variables are the sums of logarithms of several random input variables. The Central Limit theorem provides that these sums are approximately normally distributed, and therefore the logarithms of the sums are lognormally distributed. The model output can be used to estimate release and fire incident rates for specific truck routes (rural or urban), types of trucks (tanker or non-tanker), and types of loads (type of DG, large or small load). To illustrate, Table 1 shows the means and standard deviations for the model output variables for a tanker truck carrying a large load of DG3: flammable liquid. For the accident scenarios, the table shows the probability of an outcome, depending on the type of accident. For the non-accident scenarios, the table shows the rates of non-accident incidents per Bvkm, depending on the type of truck route. Table 1 provides sufficient decimal places for each value to allow a visual comparison between the largest and smallest values.
N.P. Button, P.M. Reilly / Accident Analysis and Pre6ention 32 (2000) 797–804
For a tanker truck carrying a large load of flammable liquid, the probability of an accident outcome with a release is much higher if the accident involves an overturn. For non-accident incidents, the incident rates for releases with fires are higher in rural compared with urban areas. Large non-accident spills or leaks without fires are also more frequent in rural areas.
8. Sample application of model The application of the model produces probability distributions of the expected release and fire incident rates, which is useful information for comparing different routes. This sample application of the model illustrates the high level of uncertainty that is associated with DG incident rates. If we compare PDFs rather than just point estimates of the incident rates, we may come to different conclusions regarding preferred routes and modes.
8.1. Sample 6ehicle and roads To use our model to predict release and fire incident rates, we need to know the following information about the vehicle: 1. The type of DG load.
801
2. Whether the truck is a tanker or non-tanker. 3. The load size. For this sample application of the model, we assume that the vehicle is a tanker truck carrying a large load of flammable liquid. To show a range of results, we assume two sample highways, A and B. Each sample highway is homogeneous throughout its length in terms of: 1. The proportion of truck accidents with overturns and/or collisions. 2. Whether the road is urban or rural. 3. The truck accident rate. We assume that highway A is an urban road, that it has a mean truck accident rate 1330 accidents per Bvkm, and that, on average, the truck accidents on highway A include 1.3% overturn and collision, 1.9% overturn with no collision, 90.6% collision with no overturn, and 6.2% no overturn and no collision. We assume that highway B is a rural road, that it has a mean truck accident rate of 1200 accidents per Bvkm, and that, on average, the truck accidents on highway B include 3.8% overturn and collision, 5.4% overturn with no collision, 78.3% collision with no overturn, and 12.4% no overturn and no collision. For comparison, from ADS for all Ontario highways combined, approximately 2% of truck accidents involve overturns and collisions, 5% involve overturns with no collision, 82%
Fig. 1. Beta probability distribution for input variable.
N.P. Button, P.M. Reilly / Accident Analysis and Pre6ention 32 (2000) 797–804
802
Table 1 Model output for a tanker truck carrying a large load of flammable liquid Probability of outcome given an accident
Incident outcome
Non-accident incidents per Bvkm
Overturn and collision
Overturn, no collision
Collision, no overturn
No overturn, no collision
Rural road
Urban road
Large spill with fire
Mean SD
0.06868 0.11031
0.02618 0.04196
0.00039 0.00039
0.00015 0.00016
0.05856 0.05302
0.00654 0.00713
Small spill with fire
Mean SD
0.00911 0.01678
0.00347 0.00630
0.00005 0.00006
0.00002 0.00002
0.07395 0.06220
0.03522 0.02709
Large leak with fire
Mean SD
0.00446 0.00797
0.00170 0.00312
0.00003 0.00003
0.00001 0.00001
0.02199 0.02361
0.00379 0.00423
Small leak with fire
Mean SD
0.00393 0.00768
0.00150 0.00286
0.00002 0.00003
0.00001 0.00001
0.10478 0.07971
0.03607 0.02776
Large spill no fire
Mean SD
0.38616 0.52824
0.42866 0.58814
0.00217 0.00176
0.00241 0.00195
0.97485 0.47530
0.36015 0.24741
Small spill no fire
Mean SD
0.05124 0.08083
0.05687 0.09022
0.00029 0.00026
0.00032 0.00029
1.23147 0.51131
1.93692 0.52782
Large leak no fire
Mean SD
0.02522 0.04057
0.02798 0.04483
0.00014 0.00015
0.00016 0.00017
0.36460 0.24987
0.20866 0.14997
Small leak no fire
Mean SD
0.02219 0.03800
0.02462 0.04233
0.00012 0.00013
0.00014 0.00015
1.74204 0.50666
1.98490 0.52588
Fire no release
Mean SD
0.00505 0.01126
0.00392 0.00877
0.01163 0.00814
0.00904 0.00521
0.22000 0.22000
0.22000 0.22000
No fire no release
Mean SD
0.42397 0.77713
0.42510 0.77765
0.98516 0.00851
0.98775 0.00580
involve collisions with no overturn, and 11% involve no overturn and no collision. To summarise our sample roads, highway A has a higher mean accident rate but a lower mean proportion of accidents with overturns. Highway B has a lower mean accident rate but a higher mean proportion of accidents with overturns.
8.2. Output distributions for sample application We combine the model output for the relevant accident and non-accident scenarios to determine the total incident rates, for accident and non-accident incidents combined. For example, we calculate the total number of large spills with fires per Bvkm as follows: Total number of large spills with fire per Bvkm = number of non-accident large spills with fire per Bvkm + number of accident-induced large spills with fire per Bvkm = number of non-accident large spills with fire per Bvkm + {accident rate × [Pr(large spill with fire overturn & collision) × Pr(overturn & collision accident)
+ Pr(large spill with fire overturn no collision) × Pr(overturn no collision accident) + Pr(large spill with fire collision no overturn) × Pr(collision no overturn accident) + Pr(large spill with fire no overturn no collision) × Pr(no overturn no collision accident)]} (10) We combine these uncertain inputs using Monte Carlo simulations. These simulations provide PDFs for the incidents per Bvkm for each type of incident outcome, and for each sample road. Table 2 summarises the statistics for the PDFs for the incident rates for the sample model application. It is interesting that although highway A has the higher mean accident rate, highway B is expected to have more releases per Bvkm, with and without fires. The expected number of releases is 71.4 per Bvkm for highway B and 34.3 per Bvkm for highway A. The higher release rate is related to the higher proportion of overturn accidents on highway B. For the reported incidents for both highways, spills are expected to be more common than leaks, and large releases more common than small, with and without fires. Each of the incident rates for the two highways has a lognormal PDF. As an example, Fig. 2 shows the PDF for the incident rate for large spills with fire for the two
N.P. Button, P.M. Reilly / Accident Analysis and Pre6ention 32 (2000) 797–804 Table 2 Incidents per Bvkm for sample roads Highway A
Highway B
Incident outcome
Mean
SD
Mean
SD
Large spill with fire Small spill with fire Large leak with fire Small leak with fire Large spill no fire Small spill no fire Large leak no fire Small leak no fire Fire no release No release no fire
2.50 0.36 0.17 0.17 21.58 4.73 1.60 3.19 15.11 1279.60
4.59 0.69 0.37 0.29 36.58 4.85 4.71 2.48 16.88 995.51
5.29 0.77 0.37 0.40 49.08 7.54 3.49 4.50 13.05 1123.58
7.75 1.17 0.56 0.54 65.24 9.04 5.03 4.50 12.74 761.01
Total Total releases Total releases with fire
1329.02 34.31 3.21
1208.07 71.44 6.82
highways. The mean incident rate is 5.3 incidents per Bvkm for highway B and 2.5 incidents per Bvkm for highway A. In comparing the means, it seems clear that there is more risk of large spills with fires on highway B compared to highway A. However, the PDFs overlap. Although highway B has a higher expected incident rate, there is a still a chance that highway A could have a higher actual incident rate. A comparison of the two distributions using Monte Carlo simulations indicates a probability of approximately 28% that there will be a higher incident rate on highway A compared to highway B.
803
For the two sample highways, a comparison of the means indicates the same conclusion as a comparison of the distributions: highway B is more likely to have higher release and fire incident rates. However, if we were comparing one of the highways to an alternate mode of transport, it would be important to compare the PDFs of the incident rates for the two modes. To illustrate, we compare highway B to a hypothetical mode C of transporting DG. For highway B, the incident rate for large spills with fire has a lognormal distribution with a mean of 5.3 and a standard deviation of 7.8 incidents per Bvkm. We assume that, for mode C, the incident rate for large spills with fire also has a lognormal distribution with a mean of say 4.5 and a standard deviation of say 3 incidents per Bvkm. Fig. 3 shows the PDFs and the means for the incident rates of large spills with fire for highway B and mode C. If we simply compare the means of the distri butions, there appears to be less risk of large spills with fire with the mode C compared to highway B. However, if we compare the distributions using Monte Carlo simulations, there is probability of approximately 57% that there will be a higher incident rate for large spills with fire on mode C compared to highway B.
9. Summary The paper illustrates how we can combine accident and non-accident information to generate PDFs for release and fire incident rates for trucks carrying DG. The sample model application for highways A and B shows the high level of uncertainty associated with the
Fig. 2. Comparison of incident rates for large spills with fire for sample highways.
804
N.P. Button, P.M. Reilly / Accident Analysis and Pre6ention 32 (2000) 797–804
Fig. 3. Comparison of incident rates for large spills with fire for highway B and mode C.
estimates of release and fire incident rates. In our sample application, the mean values of the predicted release incident rates are higher for highway B. However, there is a large overlap of the PDFs for the incident rates for the two highways. There is a substantial probability that highway A could have higher release incident rates. Analysts should not ignore this uncertainty when estimating incident rates from the model and applying the incident rates to risk analysis. There is further uncertainty in the impacts of the incidents to the environment and to the public. The combined uncertainty in the incident rates and impacts results in a significant amount of uncertainty in the predicted risks of transporting DG by truck.
Acknowledgements This research is part of a Ph.D. thesis that has benefited from financial support from the Organisation for Economic Co-Operation and Development, the Natural Sciences and Engineering Research Council of Canada, the Transportation Association of Canada, the
.
Canadian Institute of Transportation Engineers, and the University of Waterloo.
References Henrion, M., 1995. Assessment of probability distributions. In: An Introductory Guide to Uncertainty Analysis in Dose Reconstruction, presented to Centers for Disease Control and Prevention, Atlanta, Georgia. Senes Oak Ridge Center for Risk Analysis. Leeming, D., Saccomanno, F.F., 1993. The Major Hazard Aspects of the Transport of Chlorine to Albright and Wilson (Oldbury): Quantified Risk Assessment of Site and Delivery System. Health and Safety Executive, Research and Laboratory Services Division, Sheffield. Lindley, D.V., 1965. Introduction to Probability and Statistics from a Bayesian Viewpoint, vol. 2. Cambridge University Press, Cambridge. Saccomanno, F.F., Yu, M., Shortreed, J.H., 1993. Risk Uncertainty in the Transport of Hazardous Materials. In Transportation Research Record 1383: Issues in Marine and Intermodal Transportation. Transportation Research Board, National Research Council, Washington, DC. Shortreed, J., Del Bel Belluz, D., Saccomanno, F., Nassar, S., Craig, L., Paoli, G., 1994. Transportation Risk Assessment for the Alberta Special Waste Management System Final Report. The Institute for Risk Research, Waterloo.
Uncertainty in incident rates for trucks carrying dangerous goods Nancy P. Button a,*, Park M. Reilly b b
a Department of Ci6il Engineering, Uni6ersity of Waterloo, Waterloo, Ont., Canada N2L 3G1 Department of Chemical Engineering, Uni6ersity of Waterloo, Waterloo, Ont., Canada N2L 3G1
Received 10 August 1999; received in revised form 2 December 1999; accepted 7 December 1999
Abstract This paper addresses the uncertainty associated with release and fire incident rates for trucks in transit carrying dangerous goods. The research extends the treatment of uncertainty beyond sensitivity analysis, low – best – high estimates and confidence intervals, and represents the uncertainty through probability density functions. The analysis uses Monte Carlo simulations to propagate the uncertainty in the input variables through to the resulting release and fire incident rates. The paper illustrates how we can combine information on accident and non-accident releases and fires to generate probability density functions for the total expected releases and fires per billion vehicle kilometres for trucks carrying dangerous goods. © 2000 Elsevier Science Ltd. All rights reserved. Keywords: Trucks; Dangerous goods; Risk; Uncertainty
1. Introduction
2. Research problem
This paper illustrates how we can generate probability density functions (PDFs) for the expected release and fire incident rates for trucks in transit carrying dangerous goods (DG). The incident rates are given in terms of the number of incidents per billion vehicle kilometres (Bvkm). The paper combines information on accident and non-accident releases and fires to provide total expected incident rates. The model incorporates characteristics of roads and trucks that significantly affect the incident rates. The analysis represents the uncertainty in the input variables through PDFs, and uses Monte Carlo simulations to propagate the uncertainty in the input variables through to the resulting release and fire incident rates.
We do not understand and cannot predict the world precisely, yet analysts have often developed models based on a single value or ‘point estimate’ for each input variable, ignoring uncertainty. Often the point estimate is the mean value of the sample data. If the analyst gives only one value, then all further calculations use that value, even though a range of values may exist in the mind of the analyst. The range of values represents uncertainty, arising because the value is changeable (for example, because it fluctuates from year to year) or is not known precisely. Uncertainty can cause significant discrepancies in predicted risk and can influence decisions regarding modes and routes for the transport of DG. For example, using point estimates of the input variables we can calculate a point estimate of the incident rate on a route. On the other hand, if we incorporate the uncertainty in the input variables into the analysis, we can estimate a PDF for the incident rate on the same route. The point estimate is not necessarily the mean of the PDF. The PDF may lead us to different conclusions than the point estimate. We prefer to use PDFs rather
* Corresponding author. Present address: 16 Karen Avenue, Guelph, Ontario, Canada N1G 2N9. Tel.: +1-519-8248406; fax: +1-519-8861697. E-mail addresses: oh – [email protected] (N.P. Button)., [email protected] (P.M. Reilly).
0001-4575/00/$ - see front matter © 2000 Elsevier Science Ltd. All rights reserved. PII: S 0 0 0 1 - 4 5 7 5 ( 0 0 ) 0 0 0 0 3 - 8
798
N.P. Button, P.M. Reilly / Accident Analysis and Pre6ention 32 (2000) 797–804
than point estimates because the PDFs communicate more of the available information, help to put the incident rates in context, and allow for appropriate use of the rates in future risk assessment. Several studies have addressed the uncertainty associated with the risk of transporting DG by providing sensitivity analysis, low – best – high estimates, or confidence intervals (Leeming and Saccomanno, 1993; Saccomanno et al., 1993; Shortreed et al., 1994). However, a literature review did not reveal any examples of PDFs for release and fire incident rates. The analysis of uncertainty in this paper assigns PDFs to the input variables. The analysis then uses Monte Carlo simulations to propagate the uncertainty in the input variables through the model and generate PDFs for the resulting incident rates.
3. Research scope We use four outcome characteristics to classify DG incidents: 1. Release or no release. Every truck carries DG, in its fuel tank. However, this paper specifically addresses trucks carrying DG loads in addition to their fuel tank. The analysis defines a ‘release’ as a spill or leak of the DG load. 2. Fire or no fire. Due to the scarcity of fire and explosion data, we combine fire and explosions under a category called simply ‘fires’. 3. Release type (spill or leak). A spill is an immediate or continuous release of the DG from its containment, usually of short duration. A leak is a sporadic release, usually of a long duration. 4. Release size. Both spills and leaks can be large ( ] 1000 l) or small (B1000 l). The possible types of DG incidents include the ten possible combinations of the above characteristics, such as a large spill with fire, a small leak with no fire, a fire with no release, etc. The paper addresses both accident and non-accident releases and fires from trucks carrying DG. An accident is defined as an event involving one or a combination of a collision, an overturn or the truck running off the road. A non-accident incident could include a release that occurs, for example, if a hatch or valve is not properly closed, if a package falls off the truck, or if a fire starts from a brake or tire overheating during transport. We consider four types of DG including: 1. DG1: liquefied gases that are non-combustible but toxic and/or corrosive, such as liquefied chlorine gas. 2. DG2: liquefied gases that are flammable, such as propane. 3. DG3: flammable liquids, such as gasoline. 4. DG4: liquids that are non-combustible but toxic and/or corrosive, such as pesticides.
We did not include an analysis of accident rates in the scope of this research. The required accident rates for a study of this sort are often available from observed data or from estimates specific to the section of road being considered.
4. Data sources The research uses three databases containing records of DG incidents involving trucks in transit: 1. The Dangerous Goods Accident Information System (DGAIS) from Transport Canada (1988–1995). 2. The Occurrence Report Information System (ORIS) from the Ministry of Environment of Ontario (1988– 1997). 3. The Mission Transport des Matie`res Dangereuses (MTMD) from France (1987–1992). and two databases containing records of general truck accidents: 1. The Accident Data System (ADS) from the Ministry of Transportation of Ontario (1988–1995). 2. The Master Accident Record System (MARS) from the Washington State Department of Transportation (1990–1996). Our model contains statistical relationships rather than cause–effect relationships, because none of our databases contain information about the sequence of events in an incident. For example, there may be an incident with a release and fire. We do not know whether the fire started with say the vehicle fuel tank and then propagated to the DG load, or whether the fire started with the DG load and then spread to the rest of the vehicle. We only know that both a release and fire occurred. In determining the proportion of DG incidents with certain characteristics, we sometimes have two data sources. We assume that estimates provided by the data sources come from the same population, and differences in the estimates can be explained by differences in reporting. Therefore, where there are two sources, the best estimate is a combination of the sources.
5. The model We have insufficient DG incident data to cross-tabulate by every outcome characteristic without empty cells. For example, there are no records in our data of a truck carrying DG and having a large non-accident leak with a fire. However, we believe that the probability is greater than 0 that a large leak with a fire will occur given a truck carrying DG. We combine the conditional probabilities of each of the outcome characteristics to provide the probability of each type of DG incident.
N.P. Button, P.M. Reilly / Accident Analysis and Pre6ention 32 (2000) 797–804
The model includes 32 accident scenarios, based on the type of accident (collision and/or overturn), load size (large or small) and DG. It also includes 16 non-accident scenarios, based on the type of truck (tanker or non-tanker), road (urban or rural), and DG. There are ten possible outcomes for a DG incident, such as a large spill with fire, a small leak with no fire, a fire with no release, etc. For each of the scenario and outcome combinations, a different model equation combines the conditional probabilities of the outcome characteristics. This equation includes only statistically significant factors and, if the terms of the equation are dependent, the relationships between terms. For example, for a truck carrying a large load of DG1 and involved in an overturn and a collision, the equation for the probability of a large spill with fire reduces to: Pr(large spill with fire truck carrying a large load of DG1 with an overturn and collision) = Pr(release truck carrying DG1 involved in an accident with an overturn) × Pr(fire accident with a collision and a release)
assume that the input variables in our model that are rates of releases have lognormal PDFs, and we fit lognormal distributions to the available data.
6.2. Probability density functions for input 6ariables bounded by 0 and 1 Most of our input variables are probabilities, p, such as Pr(fire collision). These random variables are continuous and are bounded by 0 and 1. For reasons that will be explained, the beta distribution is the natural expression of the uncertainty in these probabilities. It takes the form: PDF(p; a, b)=
G(a + b) a − 1 p (1−p)b − 1 G(a)G(b)
6. Uncertainty in input values We assign a PDF to each input variable that incorporates the range of possible values over time and from each data source. Where there are larger differences over time or between data sources, the PDFs are wider, i.e. they have larger standard deviations. There are not enough sample data to fit PDFs to the input variables using goodness-of-fit measures. Therefore, we select an appropriate form for each PDF as discussed below.
6.1. Probability density functions for input 6ariables bounded by 0 The first input variable for non-accident scenarios is the rate of releases per Bvkm. The rate of releases is a continuous random variable and must be greater than 0, but has no practical upper limit. We know that the expected rate of release cannot be 0 because we have observations of releases occurring. Usually, we expect that the distribution would have one mode and be positively skewed, which corresponds with the observed values for the rate of releases per Bvkm. Henrion (1995) notes that if there is a sharp lower bound of 0 for a quantity, but no sharp upper bound, a single mode, and right skew, then the lognormal or gamma distributions are good candidates for empirical PDFs, with the lognormal being used more widely. We
(2)
where a and b are parameters of the beta distribution, such that: Mean or expected value of p= E(p)=
× Pr(spill release from a large load) × Pr(large release release that is a spill from a large load) (1) Our model includes over 400 such equations.
799
Variance of p= Var(p)=
a a+ b
ab (a+ b)2 (a+b+ 1)
The beta distribution is commonly used to represent the uncertainty in the probability of occurrence of an event, because its range is limited between 0 and 1. As noted by Lindley (1965), the family of beta distributions has the important property that if p has a beta distribution, then (1− p) also has a beta distribution, but with the parameters a and b interchanged. This makes it particularly useful if p is the probability of success, for (1 − p) is then the probability of failure and has a distribution of the same family. The beta distribution is also very flexible in terms of the wide variety of shapes it can assume, including positively or negatively skewed, depending on the values of its parameters. We assume that the input variables in our model that are probabilities have beta PDFs. We can estimate the parameters of a beta PDF for Pr(p) using Bayes’ theorem and an observed number of DG incidents, or trials, with or without a given characteristic. In statistical terms, a relevant DG incident with the characteristic is called a success and without the characteristic is called a failure. For a set of trials, the probability function for x successes has a binomial distribution as follows: P(x; p, n)=
n x p (1−p)y x
(3)
where x is the number of successes, y represents the number of failures, n, the number of trials equals x+y and p the probability of success on each trial.
800
N.P. Button, P.M. Reilly / Accident Analysis and Pre6ention 32 (2000) 797–804 k
Therefore the likelihood is also binomial: l(p; x, y)8 p x(1− p)y
% (mi /s 2i ) (4)
Lindley (1965) suggests that the family of beta distributions is the natural one to consider as prior distributions for the probability p of success, with the form p a (1− p)b. We want to chose values for a and b that provide a non-informative prior. Lindley (1965) notes that, since any observation always increases either a or b, it corresponds to the greatest possible ignorance to take a and b as small as possible. For the prior density to have a total integral 1 it is necessary and sufficient that both a and b exceed −1. Lindley (1965) therefore recommends that a=b = −1. Lindley’s prior has the PDF: PDF(p)8p − 1(1−p) − 1
(5)
If we use Bayes’ theorem to combine Lindley’s prior with our likelihood function, then the posterior PDF is: PDF(p; x, y)8 p x − 1(1 − p)y − 1
(6)
Therefore for a set of trials with x successes and y failures, we can assign to p a posterior beta distribution, PDF(p; x, y). When there is a large number of trials, the PDF is narrow. When there is a small number of trials, the PDF is wider. For each input variable, we have several sets of trials with successes and failures, from different data sources and from different time periods within each data source. Each set of trials provides a different estimate of the beta distribution for the input variable. If we simply sum the successes and failures and use the totals to estimate a beta distribution, the beta distribution becomes too narrow and does not reflect the uncertainty between data sources and between time periods. Therefore, we use an overall beta distribution that has the same mean and variance as the following mixture of beta distributions from the different sets of trials: Overall beta distribution=PDF(p; p1, p2, p3,...,pk ) k
= % li PDF(pi ; xi, yi )
(7)
i=1
where p is the overall probability of success, pi is the probability of success on each trial in trials set i, k is the number of sets of trials, xi is the number of successes in trials set i, yi is the number of failures in trials set i and PDF(pi; xi, yi ) is the beta PDF for the ith set of trials. li is the weighting factor for the ith beta distribution, where ki= 1 li = 1and the value of each li is proportional to the reciprocal of its variance, i.e. the beta distribution from each set of trials is weighted in proportion to its precision. We calculate the mean and variance of the overall beta PDF as follows:
E(p)=
i=1 k
(8)
% (1/s ) 2 i
i=1 k
k+ % (m 2i /s 2i ) Var(p)=
i=1 k
− [E(p)]2
(9)
% (1/s ) 2 i
i=1
where mi =
xi xi + yi
s 2i =
xi yi (xi + yi ) (xi + yi + 1) 2
We then calculate the values of the parameters, a and b, for the overall beta distribution by solving the standard expressions for the mean and variance of the beta distribution, as given in Eq. (2). Fig. 1 shows the beta distributions for an example input variable, Pr(fire release) for a truck carrying DG and involved in a collision. The figure shows the posterior beta distributions for each set of trials for the example input variable as well as the overall beta distribution for PDF(p; p1, p2, p3, …, pk ).
7. Output distributions Monte Carlo simulations propagate the uncertainty from the input variables and provide PDFs for each of the output variables. Most of the output PDFs turned out to be lognormal in shape, as expected. The output variables are products of several random input variables. Hence, the logarithms of the output variables are the sums of logarithms of several random input variables. The Central Limit theorem provides that these sums are approximately normally distributed, and therefore the logarithms of the sums are lognormally distributed. The model output can be used to estimate release and fire incident rates for specific truck routes (rural or urban), types of trucks (tanker or non-tanker), and types of loads (type of DG, large or small load). To illustrate, Table 1 shows the means and standard deviations for the model output variables for a tanker truck carrying a large load of DG3: flammable liquid. For the accident scenarios, the table shows the probability of an outcome, depending on the type of accident. For the non-accident scenarios, the table shows the rates of non-accident incidents per Bvkm, depending on the type of truck route. Table 1 provides sufficient decimal places for each value to allow a visual comparison between the largest and smallest values.
N.P. Button, P.M. Reilly / Accident Analysis and Pre6ention 32 (2000) 797–804
For a tanker truck carrying a large load of flammable liquid, the probability of an accident outcome with a release is much higher if the accident involves an overturn. For non-accident incidents, the incident rates for releases with fires are higher in rural compared with urban areas. Large non-accident spills or leaks without fires are also more frequent in rural areas.
8. Sample application of model The application of the model produces probability distributions of the expected release and fire incident rates, which is useful information for comparing different routes. This sample application of the model illustrates the high level of uncertainty that is associated with DG incident rates. If we compare PDFs rather than just point estimates of the incident rates, we may come to different conclusions regarding preferred routes and modes.
8.1. Sample 6ehicle and roads To use our model to predict release and fire incident rates, we need to know the following information about the vehicle: 1. The type of DG load.
801
2. Whether the truck is a tanker or non-tanker. 3. The load size. For this sample application of the model, we assume that the vehicle is a tanker truck carrying a large load of flammable liquid. To show a range of results, we assume two sample highways, A and B. Each sample highway is homogeneous throughout its length in terms of: 1. The proportion of truck accidents with overturns and/or collisions. 2. Whether the road is urban or rural. 3. The truck accident rate. We assume that highway A is an urban road, that it has a mean truck accident rate 1330 accidents per Bvkm, and that, on average, the truck accidents on highway A include 1.3% overturn and collision, 1.9% overturn with no collision, 90.6% collision with no overturn, and 6.2% no overturn and no collision. We assume that highway B is a rural road, that it has a mean truck accident rate of 1200 accidents per Bvkm, and that, on average, the truck accidents on highway B include 3.8% overturn and collision, 5.4% overturn with no collision, 78.3% collision with no overturn, and 12.4% no overturn and no collision. For comparison, from ADS for all Ontario highways combined, approximately 2% of truck accidents involve overturns and collisions, 5% involve overturns with no collision, 82%
Fig. 1. Beta probability distribution for input variable.
N.P. Button, P.M. Reilly / Accident Analysis and Pre6ention 32 (2000) 797–804
802
Table 1 Model output for a tanker truck carrying a large load of flammable liquid Probability of outcome given an accident
Incident outcome
Non-accident incidents per Bvkm
Overturn and collision
Overturn, no collision
Collision, no overturn
No overturn, no collision
Rural road
Urban road
Large spill with fire
Mean SD
0.06868 0.11031
0.02618 0.04196
0.00039 0.00039
0.00015 0.00016
0.05856 0.05302
0.00654 0.00713
Small spill with fire
Mean SD
0.00911 0.01678
0.00347 0.00630
0.00005 0.00006
0.00002 0.00002
0.07395 0.06220
0.03522 0.02709
Large leak with fire
Mean SD
0.00446 0.00797
0.00170 0.00312
0.00003 0.00003
0.00001 0.00001
0.02199 0.02361
0.00379 0.00423
Small leak with fire
Mean SD
0.00393 0.00768
0.00150 0.00286
0.00002 0.00003
0.00001 0.00001
0.10478 0.07971
0.03607 0.02776
Large spill no fire
Mean SD
0.38616 0.52824
0.42866 0.58814
0.00217 0.00176
0.00241 0.00195
0.97485 0.47530
0.36015 0.24741
Small spill no fire
Mean SD
0.05124 0.08083
0.05687 0.09022
0.00029 0.00026
0.00032 0.00029
1.23147 0.51131
1.93692 0.52782
Large leak no fire
Mean SD
0.02522 0.04057
0.02798 0.04483
0.00014 0.00015
0.00016 0.00017
0.36460 0.24987
0.20866 0.14997
Small leak no fire
Mean SD
0.02219 0.03800
0.02462 0.04233
0.00012 0.00013
0.00014 0.00015
1.74204 0.50666
1.98490 0.52588
Fire no release
Mean SD
0.00505 0.01126
0.00392 0.00877
0.01163 0.00814
0.00904 0.00521
0.22000 0.22000
0.22000 0.22000
No fire no release
Mean SD
0.42397 0.77713
0.42510 0.77765
0.98516 0.00851
0.98775 0.00580
involve collisions with no overturn, and 11% involve no overturn and no collision. To summarise our sample roads, highway A has a higher mean accident rate but a lower mean proportion of accidents with overturns. Highway B has a lower mean accident rate but a higher mean proportion of accidents with overturns.
8.2. Output distributions for sample application We combine the model output for the relevant accident and non-accident scenarios to determine the total incident rates, for accident and non-accident incidents combined. For example, we calculate the total number of large spills with fires per Bvkm as follows: Total number of large spills with fire per Bvkm = number of non-accident large spills with fire per Bvkm + number of accident-induced large spills with fire per Bvkm = number of non-accident large spills with fire per Bvkm + {accident rate × [Pr(large spill with fire overturn & collision) × Pr(overturn & collision accident)
+ Pr(large spill with fire overturn no collision) × Pr(overturn no collision accident) + Pr(large spill with fire collision no overturn) × Pr(collision no overturn accident) + Pr(large spill with fire no overturn no collision) × Pr(no overturn no collision accident)]} (10) We combine these uncertain inputs using Monte Carlo simulations. These simulations provide PDFs for the incidents per Bvkm for each type of incident outcome, and for each sample road. Table 2 summarises the statistics for the PDFs for the incident rates for the sample model application. It is interesting that although highway A has the higher mean accident rate, highway B is expected to have more releases per Bvkm, with and without fires. The expected number of releases is 71.4 per Bvkm for highway B and 34.3 per Bvkm for highway A. The higher release rate is related to the higher proportion of overturn accidents on highway B. For the reported incidents for both highways, spills are expected to be more common than leaks, and large releases more common than small, with and without fires. Each of the incident rates for the two highways has a lognormal PDF. As an example, Fig. 2 shows the PDF for the incident rate for large spills with fire for the two
N.P. Button, P.M. Reilly / Accident Analysis and Pre6ention 32 (2000) 797–804 Table 2 Incidents per Bvkm for sample roads Highway A
Highway B
Incident outcome
Mean
SD
Mean
SD
Large spill with fire Small spill with fire Large leak with fire Small leak with fire Large spill no fire Small spill no fire Large leak no fire Small leak no fire Fire no release No release no fire
2.50 0.36 0.17 0.17 21.58 4.73 1.60 3.19 15.11 1279.60
4.59 0.69 0.37 0.29 36.58 4.85 4.71 2.48 16.88 995.51
5.29 0.77 0.37 0.40 49.08 7.54 3.49 4.50 13.05 1123.58
7.75 1.17 0.56 0.54 65.24 9.04 5.03 4.50 12.74 761.01
Total Total releases Total releases with fire
1329.02 34.31 3.21
1208.07 71.44 6.82
highways. The mean incident rate is 5.3 incidents per Bvkm for highway B and 2.5 incidents per Bvkm for highway A. In comparing the means, it seems clear that there is more risk of large spills with fires on highway B compared to highway A. However, the PDFs overlap. Although highway B has a higher expected incident rate, there is a still a chance that highway A could have a higher actual incident rate. A comparison of the two distributions using Monte Carlo simulations indicates a probability of approximately 28% that there will be a higher incident rate on highway A compared to highway B.
803
For the two sample highways, a comparison of the means indicates the same conclusion as a comparison of the distributions: highway B is more likely to have higher release and fire incident rates. However, if we were comparing one of the highways to an alternate mode of transport, it would be important to compare the PDFs of the incident rates for the two modes. To illustrate, we compare highway B to a hypothetical mode C of transporting DG. For highway B, the incident rate for large spills with fire has a lognormal distribution with a mean of 5.3 and a standard deviation of 7.8 incidents per Bvkm. We assume that, for mode C, the incident rate for large spills with fire also has a lognormal distribution with a mean of say 4.5 and a standard deviation of say 3 incidents per Bvkm. Fig. 3 shows the PDFs and the means for the incident rates of large spills with fire for highway B and mode C. If we simply compare the means of the distri butions, there appears to be less risk of large spills with fire with the mode C compared to highway B. However, if we compare the distributions using Monte Carlo simulations, there is probability of approximately 57% that there will be a higher incident rate for large spills with fire on mode C compared to highway B.
9. Summary The paper illustrates how we can combine accident and non-accident information to generate PDFs for release and fire incident rates for trucks carrying DG. The sample model application for highways A and B shows the high level of uncertainty associated with the
Fig. 2. Comparison of incident rates for large spills with fire for sample highways.
804
N.P. Button, P.M. Reilly / Accident Analysis and Pre6ention 32 (2000) 797–804
Fig. 3. Comparison of incident rates for large spills with fire for highway B and mode C.
estimates of release and fire incident rates. In our sample application, the mean values of the predicted release incident rates are higher for highway B. However, there is a large overlap of the PDFs for the incident rates for the two highways. There is a substantial probability that highway A could have higher release incident rates. Analysts should not ignore this uncertainty when estimating incident rates from the model and applying the incident rates to risk analysis. There is further uncertainty in the impacts of the incidents to the environment and to the public. The combined uncertainty in the incident rates and impacts results in a significant amount of uncertainty in the predicted risks of transporting DG by truck.
Acknowledgements This research is part of a Ph.D. thesis that has benefited from financial support from the Organisation for Economic Co-Operation and Development, the Natural Sciences and Engineering Research Council of Canada, the Transportation Association of Canada, the
.
Canadian Institute of Transportation Engineers, and the University of Waterloo.
References Henrion, M., 1995. Assessment of probability distributions. In: An Introductory Guide to Uncertainty Analysis in Dose Reconstruction, presented to Centers for Disease Control and Prevention, Atlanta, Georgia. Senes Oak Ridge Center for Risk Analysis. Leeming, D., Saccomanno, F.F., 1993. The Major Hazard Aspects of the Transport of Chlorine to Albright and Wilson (Oldbury): Quantified Risk Assessment of Site and Delivery System. Health and Safety Executive, Research and Laboratory Services Division, Sheffield. Lindley, D.V., 1965. Introduction to Probability and Statistics from a Bayesian Viewpoint, vol. 2. Cambridge University Press, Cambridge. Saccomanno, F.F., Yu, M., Shortreed, J.H., 1993. Risk Uncertainty in the Transport of Hazardous Materials. In Transportation Research Record 1383: Issues in Marine and Intermodal Transportation. Transportation Research Board, National Research Council, Washington, DC. Shortreed, J., Del Bel Belluz, D., Saccomanno, F., Nassar, S., Craig, L., Paoli, G., 1994. Transportation Risk Assessment for the Alberta Special Waste Management System Final Report. The Institute for Risk Research, Waterloo.